Run ONNX Runtime tests. Use this skill when asked to run tests, debug test failures, or find and execute specific test cases in ONNX Runtime.
ONNX Runtime uses Google Test for C++ and unittest (preferred) / pytest for Python.
| Executable | What it tests |
|---|---|
onnxruntime_test_all | Core framework, graph, optimizer, session tests |
onnxruntime_provider_test | Operator/kernel tests (Conv, MatMul, etc.) across execution providers |
Use --gtest_filter to select specific tests:
./onnxruntime_provider_test --gtest_filter="*Conv3D*"
— tests may fail to find dependencies otherwise.
# Linux
cd build/Linux/Release
./onnxruntime_provider_test --gtest_filter="*TestName*"
# macOS
cd build/MacOS/Release
./onnxruntime_provider_test --gtest_filter="*TestName*"
# Windows
cd build\Windows\Release
.\onnxruntime_provider_test.exe --gtest_filter="*TestName*"
You can also run all tests via the build script (assumes a prior successful build):
./build.sh --config Release --test
.\build.bat --config Release --test # Windows
The default path follows the pattern build/<Platform>/<Config>/ where Platform is Linux, MacOS, or Windows. With Visual Studio multi-config generators on Windows, the config may appear twice (e.g., build/Windows/Release/Release/). The path can also be customized via --build_dir.
If you can't find a test binary, search for it:
# Windows
Get-ChildItem -Path build -Recurse -Filter "onnxruntime_provider_test.exe" | Select-Object -ExpandProperty FullName
# Linux/macOS
find build -name "onnxruntime_provider_test" -type f
Use pytest as the test runner:
pytest onnxruntime/test/python/test_specific.py # entire file
pytest onnxruntime/test/python/test_specific.py::TestClass::test_method # specific test
pytest -k "test_keyword" onnxruntime/test/python/ # by keyword
Python test naming convention: test_<method>_<expected_behavior>_[when_<condition>]
AGENTS.md.> test_output.txt 2>&1) — output can be large.--gtest_filter to run a targeted subset when the full suite takes too long.