Best OpenVINO Models with DFlash Speculative Decoding for Intel Arc and Core Ultra

Running local LLMs on Intel Arc discrete GPUs and Intel Core Ultra integrated GPUs with OpenVINO is fastest when you pair a pre-converted OpenVINO IR target model with a matching DFlash speculative-decoding draft model. Most generic model recommendations still point to unvalidated community conversions or older checkpoints that lack the OpenVINO runtime metadata required for speculative decoding. Every pre-converted target and DFlash draft in the blaj OpenVINO IR model family collection is exported with verified hidden_states_decoder_layers locators and stateful beam_idx KV-cache inputs, delivering 2.26x to 4.35x measured throughput gains on Intel Arc GPUs with byte-identical greedy output.

Why random OpenVINO conversions fail at DFlash speculative decoding

A DFlash draft model is not a standalone language model. Instead of predicting tokens from its own token embeddings alone, it reads intermediate activations from a fixed subset of the target model’s decoder layers (declared in dflash_config.target_layer_ids), concatenates them into a hidden_states input tensor, and proposes a block of candidate tokens that the target model verifies in a single forward pass.

For that pairing to work in OpenVINO Model Server (OVMS) or OpenVINO GenAI, three export contracts must hold simultaneously:

1. The target IR must carry hidden_states_decoder_layers runtime metadata. Optimum Intel writes producer and output-index locators for every decoder layer into the target XML’s rt_info. Because this annotation does not add public inputs or outputs to the graph, a target exported with an older toolchain loads without error and silently ignores the attached draft—running at 1.0x baseline speed.
2. The draft IR must be exported through Qwen3DFlashForCausalLM and remain stateful. Exporting through the base Qwen3DFlashDraftModel class omits the last_hidden_state = outputs.last_hidden_state[:, 1:, :] slice, shifting every emitted hidden state by one position so every draft proposal is rejected. Exporting with use_past_in_inputs=False causes patch_stateful_decoder to bail out early, leaving the draft without a beam_idx input so it recomputes context every step and runs slower than no draft at all (8.5 tok/s vs 9.0 tok/s baseline).
3. The draft’s target_layer_ids must sample a uniform attention layer type. On hybrid architectures such as 32-layer Qwen3.5 (three linear_attention layers followed by one full_attention layer per four-layer cycle), a draft that mixes linear and full attention layers cannot align with the target.

Every target and draft in the blaj OpenVINO IR collection is verified against all three checks after NNCF weight compression.

Best pre-converted OpenVINO IR + DFlash model pairs by hardware tier

Use this reference table to pick the target and DFlash draft that match your Intel Arc VRAM or Core Ultra shared system memory:

Hardware / RAM Tier	Target Model (blaj/*)	Matching DFlash Draft (blaj/*)	Target Layers Read (hidden_states)	Measured Speedup (Arc 130V iGPU)
16 GB RAM / 12–16 GB VRAM (Best 9B VLM & Roleplay)	blaj/Ornith-1.5-9B-int4-ov (int8)	blaj/Ornith-1.5-9B-DFlash-int4-ov	8 layers: 1, 5, 9, 13, 17, 21, 25, 29 ([?, ?, 32768])	51.0 tok/s (2.93x) on int4; 37.0 tok/s (4.35x) on int8
16 GB RAM / 12–16 GB VRAM (Best Uncensored 9B Text)	blaj/Qwen3.5-9B-abliterated-int4-ov (int8)	blaj/Qwen3.5-9B-abliterated-DFlash-int4-ov (int8, bf16)	5 layers: 1, 8, 15, 22, 29 ([?, ?, 20480])	43.7 tok/s (2.26x) on int4; 25.6 tok/s (2.71x) on int8
12–16 GB RAM / 8–12 GB VRAM (Standard 8B)	blaj/Qwen3-8B-Base-int4-ov	blaj/Qwen3-8B-DFlash-int4-ov (int8)	5 layers × 4096 ([?, ?, 20480])	36-layer target with full RT locators
8–12 GB RAM / 6–8 GB VRAM (Compact 4B)	blaj/Qwen3-4B-instruct-int4-ov (Base int4, Base int8)	blaj/Qwen3-4B-DFlash-int4-ov (int8)	5 layers: 1, 9, 17, 25, 33 ([?, ?, 12800])	36-layer compact target + stateful draft
32–64 GB RAM (Flagship 27B Uncensored VLM)	blaj/Qwen3.8-27B-heretic-ara-int4-ov (int8)	blaj/Qwen3.8-27B-heretic-ara-DFlash-int4-ov (int8, fp16)	5 layers: 1, 16, 31, 46, 61 ([?, ?, 25600])	64-layer VLM with both DFlash + openvino_mtp_model

16 GB sweet spot: Ornith-1.5-9B and Qwen3.5-9B-abliterated with DFlash

For 16 GB Intel Arc cards (Arc A770, Arc B580) and 16–32 GB Core Ultra laptops (Arc 130V/140V iGPUs), the two strongest DFlash-enabled model families are Ornith-1.5-9B and Qwen3.5-9B-abliterated.

Ornith-1.5-9B + DFlash (51.0 tok/s at INT4, 4.35x speedup at INT8)

blaj/Ornith-1.5-9B-int4-ov (8.1 GB) and blaj/Ornith-1.5-9B-int8-ov are decomposed VLM exports (openvino_language_model.xml, vision tower, merger, and MTP head) with hidden_states_decoder_layers annotated across all 32 decoder layers (0–31).

Its companion draft, blaj/Ornith-1.5-9B-DFlash-int4-ov (866 MB), was converted from the upstream Q4_K_M GGUF by dequantizing the 16-element sub-blocks and re-quantizing with NNCF at group size 16—preserving the source quantization grid with 0.000% second-rounding error against the held values.

Measured on an Intel Core Ultra 7 258V (Arc 130V iGPU, greedy decoding, 128 tokens):

Configuration	Throughput	Speedup
blaj/Ornith-1.5-9B-int4-ov (target only)	17.4 tok/s	1.00x
+ blaj/Ornith-1.5-9B-DFlash-int4-ov	51.0 tok/s	2.93x
blaj/Ornith-1.5-9B-int8-ov (target only)	8.5 tok/s	1.00x
+ blaj/Ornith-1.5-9B-DFlash-int4-ov	37.0 tok/s	4.35x

Because LLM decode on an integrated GPU is memory-bandwidth bound, the heavier INT8 target gets an even larger relative speedup (4.35x) from the same INT4 draft—recovering most of the throughput penalty of running INT8 weights while keeping full INT8 accuracy.

Qwen3.5-9B-abliterated + DFlash (43.7 tok/s at INT4)

blaj/Qwen3.5-9B-abliterated-int4-ov (5.7 GB) and blaj/Qwen3.5-9B-abliterated-int8-ov are two-stage exports of lukey03/Qwen3.5-9B-abliterated (orthogonal-projection abliteration across all 32 layers plus QLoRA fine-tuning). Because abliteration modifies weight values without altering layer geometry, the target pairs cleanly with the abliterated DFlash drafts:

• blaj/Qwen3.5-9B-abliterated-DFlash-int4-ov
• blaj/Qwen3.5-9B-abliterated-DFlash-int8-ov (1002 MB)
• blaj/Qwen3.5-9B-abliterated-DFlash-bf16-ov

Measured on the same Arc 130V iGPU with OVMS 2026.4.0:

Target	Draft	Throughput	Speedup
Qwen3.5-9B-abliterated-int4-ov	none (baseline)	19.3 tok/s	1.00x
Qwen3.5-9B-abliterated-int4-ov	Qwen3.5-9B-abliterated-DFlash-int4-ov	43.7 tok/s	2.26x
Qwen3.5-9B-abliterated-int8-ov	none (baseline)	9.4 tok/s	1.00x
Qwen3.5-9B-abliterated-int8-ov	Qwen3.5-9B-abliterated-DFlash-int4-ov	25.6 tok/s	2.71x
Qwen3.5-9B-abliterated-int8-ov	Qwen3.5-9B-abliterated-DFlash-int8-ov	24.2 tok/s	2.57x

32 GB+ tier: 64-layer Qwen3.8-27B VLMs with DFlash and MTP

On 32 GB and 64 GB Core Ultra systems (see What Can You Run on an Intel Core Ultra iGPU with OpenVINO?), 27B dense vision-language models fit comfortably in shared memory when compressed to INT4 or INT8:

• Uncensored 27B VLM + DFlash: blaj/Qwen3.8-27B-heretic-ara-int4-ov and blaj/Qwen3.8-27B-heretic-ara-int8-ov expose hidden_states_decoder_layers across all 64 decoder layers (0–63, 48 linear-attention + 16 full-attention, hidden size 5120) and also include the exported Multi-Token Prediction head (openvino_mtp_model.*). Pair either target with blaj/Qwen3.8-27B-heretic-ara-DFlash-int4-ov, int8-ov, or fp16-ov, which read target layers 1, 16, 31, 46, 61 (hidden_states shape [?, ?, 25600]).
• JetBrains 3.6/3.8 27B Blend: blaj/Qwen3.8-3.6-27B-blend-int4-ov, int8-ov, and fp16-ov ship with all 64 decoder layer locators exposed plus openvino_mtp_model.* for built-in MTP speculative decoding in ov_genai.VLMPipeline (see How to Use Qwen3.5 MTP Speculative Decoding with OpenVINO).

Compact and specialized OpenVINO IR builds (Qwen3-4B/8B, LFM2.5 Heretic, Qwen-Image-2.1)

When you need smaller memory footprints or specialized modalities from the same blaj collection:

• Qwen3-4B & Qwen3-8B with DFlash: blaj/Qwen3-4B-instruct-int4-ov (plus Base-int4-ov and Base-int8-ov) pairs with blaj/Qwen3-4B-DFlash-int4-ov / int8-ov. blaj/Qwen3-8B-Base-int4-ov pairs with blaj/Qwen3-8B-DFlash-int4-ov / int8-ov.
• LFM2.5 Heretic (2.6B Dense & 8B-A1B MoE): blaj/LFM2.5-2.6B-heretic-int8-ov (int4) and the mixture-of-experts blaj/LFM2.5-8B-A1B-heretic-int8-ov (int4). For the two-stage export and OpenMP thread-cap recipe used to build these without OOM kills, see How to Convert an Abliterated (Uncensored) Model to OpenVINO int8 IR.
• Uncensored Vision & Diffusion: blaj/Qwen2.5-VL-7B-Instruct-abliterated-int4-ov (int8) for 7B vision-language inference, and blaj/Qwen-Image-2.1-Uncensored-OpenVINO-INT4 for text-to-image generation with an abliterated Qwen3-VL text encoder (detailed in How to Convert Qwen-Image-2.1 to OpenVINO IR (and Build an Uncensored Version)).

How to download, verify, and serve a DFlash pair with OVMS

Download both the target and its matching DFlash draft from Hugging Face:

huggingface-cli download blaj/Ornith-1.5-9B-int4-ov \
  --local-dir ./Ornith-1.5-9B-int4-ov

huggingface-cli download blaj/Ornith-1.5-9B-DFlash-int4-ov \
  --local-dir ./Ornith-1.5-9B-DFlash-int4-ov

Verify in two seconds that the target carries hidden_states_decoder_layers and the draft exposes beam_idx:

import json, openvino as ov

core = ov.Core()
target = core.read_model("Ornith-1.5-9B-int4-ov/openvino_language_model.xml")
rt = json.loads(target.get_rt_info()["hidden_states_decoder_layers"].value)
assert len(rt["layers"]) == 32, "target missing hidden-state locators"

draft = core.read_model("Ornith-1.5-9B-DFlash-int4-ov/openvino_model.xml")
draft_inputs = [i.get_any_name() for i in draft.inputs]
assert "beam_idx" in draft_inputs, "draft is stateless"
print("Verified:", len(rt["layers"]), "target layers;", draft_inputs)

Wire the pair together in the target directory’s graph.pbtxt by setting draft_models_path:

node_options: {
    [type.googleapis.com / mediapipe.LLMCalculatorOptions]: {
        models_path: "."
        draft_models_path: "/absolute/path/to/Ornith-1.5-9B-DFlash-int4-ov"
    }
}

When OVMS loads the graph on your Intel GPU, confirm that the startup log prints Draft model strategy: DFlash. For deep-dive troubleshooting on custom DFlash exports and GGUF draft conversions, see How to Convert and Run DFlash Models with OpenVINO.