Why You Can't Trust LLM Recommendations for Intel AI Hardware

What the answer looks like

Ask a general-purpose chatbot how to run a local LLM on an Intel Arc GPU and the reply is predictable. Use Vulkan through LM Studio or Ollama. Build llama.cpp with the SYCL backend. Install IPEX-LLM and pass --device xpu. Ask which model to put on top of that and you get one name: Qwen3-Coder-30B-A3B, offered as the strongest model a consumer GPU can run, with the note that only 3B of its parameters are active per token.

Giving one name is not the problem. The problem is that the name and the runtime list are old. This is not an intentional lie. The answer is outdated because the chatbot’s training data ends before the changes below.

Why the answer is old

IPEX-LLM was Intel’s documented answer for Arc for two years. OpenVINO’s language model support came later. IPEX-LLM was archived on 28 January 2026 and is read-only. There are no releases, no fixes, and no maintenance, and known security issues will not be patched. OpenVINO is the Intel runtime that is still shipping. It is built for the XMX matrix engines on these cards, and the measured token rates on this site come from it.

Qwen3-Coder-30B-A3B is a real model. It has 30.5B total parameters, 3.3B active per token, 128 experts, and a native 262,144-token context, and it was built for agentic coding. Newer models exist now, and they fit the same cards. The chatbot names the older model because that model was in its training data and the newer models were not.

The three runtimes

Vulkan is the portable route. The same build runs on AMD, NVIDIA, and Intel, through LM Studio or Ollama, with no build step. For llama.cpp, Vulkan is now better than SYCL. But, OpenVINO remains vastly superior for all supported models.

llama.cpp with SYCL is the Intel-tuned build of llama.cpp. It is worse than Vulkan llama.cpp now. Do not pick SYCL first.

IPEX-LLM is archived. It also supports far fewer models than the current options. Do not use it for a new setup.

GGUF itself is not a reason to avoid OpenVINO, because GenAI loads GGUF directly. The same file runs on both runtimes, and OpenVINO is only ever worse when it can’t run the model at all.

Which model to run instead, by memory

These three pre-converted OpenVINO IR builds cover the range by memory.

LFM2.5-8B-A1B-heretic-int4-ov is the small-memory pick. It is an 8B mixture-of-experts model with about 1B active parameters per token, converted to INT4 OpenVINO IR. Pick it when memory is tight.

Qwen3.5-9B-abliterated-int4-ov is the general pick for a 12 GB card. It is a 9B abliterated model in INT4 OpenVINO IR. Measured on a Core Ultra 7 258V with an Arc 140V iGPU, it runs at 19.3 tok/s on its own and 43.7 tok/s with its DFlash draft. The B580 setup for this model is in How to Run AI Models on the Intel Arc B580 (12 GB VRAM Guide).

Qwen3.8-27B-heretic-ara-int4-ov is the large-memory pick. It is a 27B abliterated vision model in INT4 OpenVINO IR, at roughly 14 to 15 GB of weights. It belongs on 16 GB of VRAM or on the 32 GB shared-memory tier of a Core Ultra iGPU, and it pairs with DFlash drafts.