How to use DSpark with OpenVINO
DSpark is not currently supported by OpenVINO GenAI, so it cannot be used through the GenAI API.
DSpark is not supported by OpenVINO GenAI
You cannot use DSpark with OpenVINO GenAI.
OpenVINO GenAI supports speculative decoding, but DSpark is not currently exposed as a supported speculative decoding method in OpenVINO GenAI. A DSpark GGUF therefore cannot be loaded as a DSpark draft model alongside a main GGUF through the GenAI API.
If you are trying to run a main GGUF with a DSpark GGUF as its draft model, OpenVINO GenAI does not currently support that setup.
Comments
One comment per thread every 30 minutes · edits are unlimited.