How to Convert Models to OpenVINO IR

What is OpenVINO IR?

OpenVINO Intermediate Representation (IR) is OpenVINO’s model format. A saved IR normally consists of two files:

• .xml contains the model topology and graph structure.
• .bin contains the model weights and other binary data.

Converting a model to IR ahead of time avoids repeating model conversion when the application starts. OpenVINO recommends explicit conversion when you want a production-ready model, lower first-inference latency, model optimization options, and potentially smaller storage requirements.

Install OpenVINO

Install the OpenVINO Python package in your environment:

pip install openvino

You can check the installed version with:

ovc --version

The current OpenVINO documentation describes two main conversion paths: the Python API and the ovc command-line converter. The important exception is that PyTorch and JAX/Flax models must be converted through the Python API rather than ovc.

The basic Python workflow

For a model that can be passed to openvino.convert_model, the basic workflow is:

import openvino as ov

ov_model = ov.convert_model("path/to/model")
ov.save_model(ov_model, "model.xml")

convert_model() creates an OpenVINO Model in memory. save_model() serializes it as IR. If model.xml is used as the output name, OpenVINO creates the accompanying model.bin file automatically.

You can then load the saved IR without loading the original framework model:

import openvino as ov

core = ov.Core()
model = core.read_model("model.xml")
compiled_model = core.compile_model(model, "AUTO")

Using the ovc command-line converter

For supported file-based model formats, ovc provides a convenient command-line interface:

ovc path/to/model.onnx

To choose the output filename explicitly:

ovc path/to/model.onnx --output_model model.xml

The output is an OpenVINO IR pair:

model.xml
model.bin

ovc combines the conversion and serialization steps. It is not a separate conversion engine from the Python API. The official documentation describes it as providing the same results when the same conversion and saving parameters are used.

ONNX to IR

ONNX is one of the simplest formats to convert because OpenVINO can accept the .onnx file directly.

CLI

ovc model.onnx --output_model model.xml

Python

import openvino as ov

ov_model = ov.convert_model("model.onnx")
ov.save_model(ov_model, "model.xml")

The resulting model.xml and model.bin can then be loaded by OpenVINO Runtime.

PyTorch to IR

PyTorch models are handled differently. The official OpenVINO documentation states that openvino.convert_model is the applicable conversion method for PyTorch models. ovc should not be used for PyTorch model objects.

For example:

import openvino as ov
import torch
from torchvision.models import resnet50

model = resnet50(weights="DEFAULT")
model.eval()

example_input = torch.rand(1, 3, 224, 224)
ov_model = ov.convert_model(model, example_input=example_input)
ov.save_model(ov_model, "resnet50.xml")

The example_input is useful when the model needs an example tensor to trace or resolve its inputs during conversion.

If you already have a PyTorch model object, pass that object to convert_model() rather than trying to point ovc at a PyTorch checkpoint and expecting it to perform framework-specific loading.

TensorFlow and Keras to IR

TensorFlow and compatible TensorFlow/Keras models can be converted with the Python API. For example, a SavedModel can be passed directly to convert_model():

import openvino as ov

ov_model = ov.convert_model("saved_model.pb")
ov.save_model(ov_model, "model.xml")

For a SavedModel directory, pass the directory path instead:

import openvino as ov

ov_model = ov.convert_model("saved_model")
ov.save_model(ov_model, "model.xml")

The command-line equivalent is:

ovc saved_model --output_model model.xml

Keras 3 has additional export options. A Keras model can first be exported as a TensorFlow SavedModel and then converted with OpenVINO, or Keras 3 can export directly to OpenVINO format with its OpenVINO export support.

TensorFlow Lite to IR

A TensorFlow Lite model can be converted directly:

ovc model.tflite --output_model model.xml

Or with Python:

import openvino as ov

ov_model = ov.convert_model("model.tflite")
ov.save_model(ov_model, "model.xml")

PaddlePaddle to IR

PaddlePaddle inference models can also be converted directly. For a .pdmodel file:

ovc model.pdmodel --output_model model.xml

Or:

import openvino as ov

ov_model = ov.convert_model("model.pdmodel")
ov.save_model(ov_model, "model.xml")

The associated PaddlePaddle parameter files must be available as required by the model.

JAX/Flax to IR

JAX/Flax conversion is another case where the Python conversion API is required. OpenVINO documents support for Python model representations such as JAX ClosedJaxpr and Flax modules.

A simplified pattern is:

import openvino as ov

# Create or obtain the supported JAX/Flax model representation here.
model = ...

ov_model = ov.convert_model(model)
ov.save_model(ov_model, "model.xml")

The exact conversion code depends on how the JAX/Flax model is represented, so the framework-specific OpenVINO conversion guide should be used for that model.

Choosing the output precision

OpenVINO can save an IR with FP16-compressed weights:

import openvino as ov

ov_model = ov.convert_model("model.onnx")
ov.save_model(ov_model, "model.xml", compress_to_fp16=True)

FP16 compression can substantially reduce model storage. The OpenVINO documentation notes that it can cut the size by about half for many models.

If you specifically need to preserve the original precision, disable the compression:

ov.save_model(ov_model, "model.xml", compress_to_fp16=False)

The precision of the resulting IR should not be confused with arbitrary integer quantization. Converting a BF16, FP16, or FP32 source model to IR does not automatically mean the model becomes INT4. Weight compression and quantization are separate model-optimization steps.

Conversion parameters

Both convert_model() and ovc support parameters for cases where the default conversion is not sufficient. Common parameters include input and output specifications, example inputs, extensions, verbosity, and weight sharing.

For example, verbose conversion can help diagnose a failed conversion:

import openvino as ov

ov_model = ov.convert_model(
    "model.onnx",
    verbose=True,
)
ov.save_model(ov_model, "model.xml")

The CLI also provides its own help output:

ovc --help

What if the model is already quantized?

Conversion and quantization are separate operations. If you have an ONNX model containing quantization operators such as Quantize/Dequantize nodes, OpenVINO can convert that model to IR while preserving the graph representation supported by OpenVINO.

For additional optimization, OpenVINO’s NNCF tooling provides post-training quantization and weight-compression workflows. These should be treated as optimization steps rather than as a requirement for producing an IR file.

For example, the basic conversion remains:

import openvino as ov

ov_model = ov.convert_model("model.onnx")
ov.save_model(ov_model, "model.xml")

Optimization can then be applied to the OpenVINO model before serialization when the particular workflow requires it.

Verify the generated IR

After conversion, verify that both files exist:

ls -lh model.xml model.bin

Then load and compile the IR:

import openvino as ov

core = ov.Core()
model = core.read_model("model.xml")
compiled_model = core.compile_model(model, "AUTO")

print(model)

If OpenVINO can read and compile the model, the conversion and serialization stages have produced a usable IR representation.

Common mistakes

Trying to convert every model with ovc

ovc does not replace the framework-specific Python conversion path. In particular, OpenVINO’s current documentation says PyTorch and JAX/Flax models must use openvino.convert_model().

Expecting one file

An ordinary OpenVINO IR consists of an XML graph file and a BIN data file. Keep both files together when moving or deploying the model.

Assuming conversion is quantization

Converting a model to IR changes its representation for OpenVINO. It does not mean that an FP32 or BF16 model automatically becomes INT4. Quantization and weight compression are separate optimization workflows.

Converting every time the application starts

For repeated deployment, save the converted model once and load the IR at runtime. This avoids repeating the conversion process and can reduce first-inference latency.

Quick reference

Source model	Recommended conversion method
ONNX	ovc model.onnx or ov.convert_model()
PyTorch	ov.convert_model()
TensorFlow SavedModel	ovc or ov.convert_model()
TensorFlow Lite	ovc or ov.convert_model()
PaddlePaddle	ovc or ov.convert_model()
JAX/Flax	ov.convert_model()

The general pattern is:

Source model
     |
     v
OpenVINO conversion
     |
     v
ov.Model
     |
     v
ov.save_model()
     |
     v
model.xml + model.bin
     |
     v
OpenVINO Runtime

Official documentation

• Convert to OpenVINO IR
• Conventional Model Preparation
• Conversion Parameters
• OpenVINO IR Format