How to Run llama.cpp on Intel Arc with SYCL

What SYCL is doing here

SYCL is the Intel-oriented GPU backend you will run into most often when using llama.cpp with Intel GPUs. The llama.cpp project has a dedicated SYCL backend for Intel GPUs, including Arc discrete GPUs and integrated Arc graphics.

It sits on top of Intel’s oneAPI ecosystem, including the DPC++ compiler/runtime and Level Zero. You can think of it as the Intel-specific route into llama.cpp rather than a completely different LLM runtime.

I would recommend using OpenVINO instead, whether it be through llama.cpp (which sucks) or OVMS or OpenArc. They’re faster and more consistent. If you need llama.cpp, use Vulkan. SYCL is effectively worthless. You can follow my tutorial to run OpenVINO here and Vulkan here. SYCL is only maintained by one person now, it has effectively been superseded by OpenVINO, and even Vulkan has better performance in my testing.

Install the Intel side first

On Linux, the basic stack is:

Intel GPU driver
↓
oneAPI
↓
SYCL
↓
llama.cpp

The current llama.cpp documentation lists Linux and Windows support for the SYCL backend and specifically lists Intel Arc hardware among the supported devices. For Linux, install the appropriate Intel GPU driver and the Intel oneAPI components required by llama.cpp.

After installing the oneAPI environment, the usual environment setup is:

source /opt/intel/oneapi/setvars.sh

You can then inspect the available SYCL devices with llama.cpp’s device utility:

./build/bin/llama-ls-sycl-device

The tool reports the available devices and their memory and compute information.

Build llama.cpp

You have to build llama.cpp yourself. Do not use the prebuilt binaries for this setup.

While running the prebuilt version, I ran into a library issue where libggml.so.8 was not found. This happened even after sourcing Intel’s setvars.sh:

error while loading shared libraries: libggml.so.8: cannot open shared object file: No such file or directory

Because of this, the prebuilts will not work correctly in this setup. Building llama.cpp locally ensures the binaries and libraries are produced together in your environment.

Source the Intel environment before building:

source /opt/intel/oneapi/setvars.sh

Then configure and build llama.cpp with the SYCL backend enabled:

cmake -B build -DGGML_SYCL=ON
cmake --build build --config Release

The current llama.cpp documentation also supports specifying the Intel compilers explicitly when needed, including icx and icpx. There are additional build options for things such as the target architecture, FP16, oneDNN, and SYCL graphs. You generally do not need to change those just to get your first build working.

Pick the GPU

If you have more than one Intel GPU, llama.cpp can detect multiple SYCL devices. Current versions use --main-gpu for selecting the main GPU, while older documentation used the GGML_SYCL_DEVICE environment variable.

For example, you can inspect the devices first and then choose the appropriate device when launching llama.cpp. The exact command-line options change as llama.cpp develops, so check the current SYCL documentation if an option from an older guide does not work with your build.

Running a GGUF model

Once the SYCL build is working, llama.cpp can run the same general GGUF model ecosystem used by its other GPU backends. The point of this setup is that the GPU work is handled through the SYCL backend instead of CUDA, Vulkan, or another backend.

For a basic test, use a GGUF model and offload its layers to the GPU with llama.cpp’s normal GPU-offload options. The exact amount of offloading depends on the model and available memory.

Memory matters

The model still has to fit into the memory available to the GPU or into whatever CPU/GPU split you configure. A larger Arc card can therefore handle models that would not fit comfortably on a smaller card.

On integrated Arc graphics, system memory is shared with the GPU, so available system memory becomes part of the equation as well. llama.cpp’s SYCL documentation specifically calls out device memory as a limitation for large models.

Useful environment variables

llama.cpp exposes several SYCL-specific environment variables. Some of the useful ones include:

GGML_SYCL_DEVICE_ARCH
GGML_SYCL_F16
GGML_SYCL_DEBUG
GGML_SYCL_DISABLE_DNN
ZES_ENABLE_SYSMAN

You do not need to set these for a normal installation. They become useful when tuning a particular Intel GPU or troubleshooting a build. The current project documentation lists their behavior and defaults.

Windows

Windows is supported as well. The general setup is the same idea: Intel GPU drivers, oneAPI components, a SYCL-enabled llama.cpp build, and the appropriate Intel compiler environment. For Windows-specific builds, the llama.cpp documentation provides CMake and Visual Studio instructions.

The short version

If you have an Intel Arc GPU and specifically want llama.cpp + SYCL, the stack is basically:

Intel Arc
↓
Intel driver
↓
oneAPI / Level Zero
↓
SYCL
↓
llama.cpp
↓
GGUF model

SYCL is one of the more Intel-specific ways to run llama.cpp. If you would rather use a more general GPU API, see How to Run Local LLMs on Intel Arc with Vulkan. If you want Intel’s inference stack instead, see How to Run Local LLMs with OpenVINO on Intel Arc. New to Arc? Start with the overview at How to Run Local LLMs on Intel Arc.