How to Convert and Run DFlash Models with OpenVINO

DFlash models are specialized draft models designed for speculative decoding. This guide covers converting supported DFlash checkpoints to OpenVINO IR and using the resulting models with the OpenVINO inference stack.

DFlash is a newer type of draft model, and therefore you cannot use it with the latest OpenVINO release. For the newest DFlash export functionality, use Optimum Intel from source together with an OpenVINO nightly build. The development version contains the newer exporter paths and DFlash-specific handling that may not be available in released packages, till at least 2026.5.0. Please report this article when it 2026.5.0 is out. Currently, vision models are not supported, but they will be enabled with https://github.com/huggingface/optimum-intel/pull/1926.

This process requires A LOT of RAM. For a 9B model, you may require 80 GB of RAM. 35B models need 200 GB just for text and 35b A3B pushes 300 TO 400 GB when the vision component is left in. This is because you will be working with unquantized safetensors.

Install the conversion environment

Create a clean virtual environment first:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip

Then install Optimum Intel directly from its Git repository with the OpenVINO dependencies:

python -m pip install "optimum-intel[openvino] @ git+https://github.com/huggingface/optimum-intel.git"

This installs the development version rather than the version currently published on PyPI.

Install OpenVINO nightly

Use the OpenVINO nightly build with the source version of Optimum Intel.

Install the nightly wheels with:

python -m pip install --pre -U openvino --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly

If a stable OpenVINO release is already installed in the environment, the command above upgrades it to the nightly package.

Verify the installed versions:

python -c "import openvino, optimum.intel; print('OpenVINO:', openvino.__version__); print('Optimum Intel:', getattr(optimum.intel, '__version__', 'unknown'))"

Keeping the packages aligned matters because the development exporter can depend on OpenVINO functionality that is not present in an older stable release.

Check the exporter

Make sure the OpenVINO exporter is available:

optimum-cli export openvino --help

You can also verify that Python can import OpenVINO and Optimum Intel:

python -c "import openvino; from optimum.intel import OVModelForCausalLM; print(openvino.__version__)"

Convert a normal model to OpenVINO

The same exporter can convert ordinary supported Hugging Face models to OpenVINO IR. For example:

optimum-cli export openvino \
  --model Qwen/Qwen3-8B \
  --task text-generation-with-past \
  qwen3-8b-openvino

For a local model, replace the model ID with its local directory:

optimum-cli export openvino \
  --model ./my-model \
  --task text-generation-with-past \
  ./my-model-openvino

If the model requires custom Transformers code, add --trust-remote-code when appropriate. As it stands, you will need to use task text-generation-with-past. If that spits an error, extract the text part of the model via the following code segment, then follow the above instructions to convert. Once https://github.com/huggingface/optimum-intel/pull/1926 is merged, you won’t need to put that type for VLMs.

import transformers

name = "Qwen/Qwen3.6-35B-A3B"
save = "qwen3.6-text-35b-a3b"

m = transformers.AutoModelForCausalLM.from_pretrained(name)
t = transformers.AutoTokenizer.from_pretrained(name)

m.save_pretrained(save)
t.save_pretrained(save)

Convert a DFlash model

DFlash is different from an ordinary LLM checkpoint. A DFlash checkpoint is a draft model intended to generate candidate tokens for a speculative decoding workflow.

The current Optimum Intel source contains a dedicated DFlash export path. Its model loader recognizes the DFlashDraftModel architecture and maps it to the appropriate Qwen DFlash model classes.

Supported DFlash families currently include:

• Qwen3-*-DFlash
• Qwen3.5-*-DFlash
• Qwen3.6-*-DFlash
• Qwen3-Coder-30B-A3B-DFlash

For a local DFlash checkpoint, the basic export pattern is:

optimum-cli export openvino \
  --model ./qwen3-dflash \
  --task text-generation \
  ./qwen3-dflash-openvino

For a DFlash model hosted on Hugging Face, use the same pattern with the model ID:

optimum-cli export openvino \
  --model <DFlash-model-id> \
  --task text-generation \
  ./dflash-openvino

Use the task and model format required by the particular checkpoint. If automatic task detection does not select the correct task, specify it explicitly.

The checkpoint must actually be a supported DFlash draft model. A normal Qwen checkpoint does not become a DFlash draft model simply by adding a command-line option.

DFlash export is not ordinary quantization

Do not confuse DFlash export with OpenVINO weight compression such as INT4.

For example:

optimum-cli export openvino \
  --model <model> \
  --weight-format int4 \
  ./model-openvino

--weight-format int4 tells Optimum Intel to perform OpenVINO weight compression during export. DFlash support is separate and preserves the information needed to use the exported draft model in a DFlash speculative-decoding workflow.

If you already have a quantized DFlash checkpoint and want to preserve its existing precision rather than quantize it again, do not blindly add --weight-format int4. Export the appropriate DFlash checkpoint and let the exporter handle the architecture.

Why OpenVINO nightly matters

The DFlash export path is part of the newer Optimum Intel development code. The exporter adds DFlash-specific information to the resulting OpenVINO model and the source tree contains dedicated DFlash export handling.

The workflow is therefore:

DFlash checkpoint
       ↓
Optimum Intel from source
       +
OpenVINO nightly
       ↓
OpenVINO exporter
       ↓
OpenVINO IR
       ↓
OpenVINO inference / DFlash workflow

Using an older stable OpenVINO runtime with a newer Optimum Intel checkout can produce compatibility errors or missing functionality. If you are using a newly added exporter feature, update both sides rather than mixing old OpenVINO packages with the newest Optimum Intel source.

Loading the resulting OpenVINO model

An ordinary exported causal language model can be loaded through Optimum Intel:

from optimum.intel import OVModelForCausalLM
from transformers import AutoTokenizer

model_dir = "qwen3-8b-openvino"

model = OVModelForCausalLM.from_pretrained(model_dir)
tokenizer = AutoTokenizer.from_pretrained(model_dir)

The exported directory contains the OpenVINO model representation, including the XML graph and binary weights.

For DFlash, the exported draft model is intended to be used by an inference workflow that understands the DFlash-specific OpenVINO representation. OpenVINO GenAI is the relevant runtime stack for generative inference, with DFlash support depending on the versions of OpenVINO and OpenVINO GenAI installed.

Troubleshooting conversion

If the exporter fails, first check the installed versions:

python -c "import openvino; print(openvino.__version__)"
python -m pip show optimum-intel

Make sure the environment is actually using the source Optimum Intel installation and the OpenVINO nightly build. Mixing an old stable OpenVINO package with the development exporter is a common source of missing functionality.

For reproducibility, record the Optimum Intel Git revision and OpenVINO version used for a successful export. Development exporters can change rapidly, so keeping the exact environment makes it much easier to reproduce a working conversion.