What's Changed

NOTE: This release is a unified release of the NVIDIA Container Toolkit that consists of the following packages:

• libnvidia-container 1.20.0
• nvidia-container-toolkit 1.20.0

The packages for this release are published to the libnvidia-container package repositories.

Fixes and Features

• CDI specifications can now include an application-profile hook that limits EGL and Vulkan visibility to the GPUs assigned to the container. Graphics applications no longer see unassigned host GPUs through these APIs. For more information, refer to issue #1899 and PR #1939.
• CUDA compatibility handling now uses libcuda.so ELF metadata whenever it is available. This improves CUDA minor-version compatibility by selecting the container’s compatibility libraries only when they are appropriate for the installed driver.
• Driver file discovery now supports libraries spread across multiple directories, as occurs on distributions such as Debian, and matches graphics libraries against the exact installed driver version. This fix avoids both missing required libraries and injecting libraries from another installed driver version. For information about multiple-directory discovery, refer to issue #1559 and PR #1820. For information about exact-version matching, refer to PR #1948.
• CDI specifications now include the NVIDIA OpenCL ICD file and the legacy libnvidia-nvvm70.so library when present. OpenCL loaders can locate the NVIDIA implementation, and workloads that depend on the legacy NVVM library receive it automatically. For information about OpenCL support, refer to issue #682 and PR #1893. For information about legacy NVVM support, refer to issue #1875 and PR #1876.
• On WSL2, CDI discovery now includes additional .so, .bin, and .dll files from the NVIDIA driver store instead of relying only on a fixed file list. This fix enables containers to receive driver components introduced by newer Windows driver releases without waiting for a toolkit-specific allowlist update. For more information, refer to issue #1864 and PR #1890.
• IMEX channel requests in CDI and JIT-CDI mode are now validated for both the supported ID range and the presence of the corresponding host device. Invalid requests fail with a clear error. For more information, refer to issue #1309 and PR #1913.
• JIT-CDI mode now honors the nvidia-container-runtime.modes.jit-cdi.nvcdi-disable-hooks configuration option. You can disable individual CDI hooks for environments where a generated hook is unnecessary or incompatible.
• Updating config.toml no longer reverts previously modified options to their defaults.
• CDI generation no longer adds an update-ldcache hook when it discovers no driver libraries. This fix prevents containers from running an unnecessary hook and avoids failures on systems or modes that do not inject libraries. For more information, refer to issue #373 and PR #1894.
• Fixed an issue where NVIDIA runtime handlers in a generated containerd drop-in configuration could omit runtime_type when the base configuration did not define it. The affected containers failed to start with a container.Runtime.Name must be set error. The toolkit now sets the default runtime type when the field is missing or empty. For more information, refer to issue #1956 and PR #1969.

Packaging Changes

• RPMs rebuilt by the toolkit packaging image now use XZ payload compression instead of zstd. The resulting packages can be installed on older Linux distributions that do not support zstd compression, such as Amazon Linux 2.
• Source package builds can use Podman by setting DOCKER=podman. The build handles Podman’s local image naming, SELinux volume labeling, and artifact-directory creation automatically.

Enhancements to container-toolkit Container Image

• The container-toolkit image now uses the non-development distroless base and includes a static BusyBox shell. Init-container wrappers and lifecycle hooks retain the shell commands they need without depending on the development image.
• The NRI plugin can inject ordinary workload CDI devices outside the toolkit namespace. For management devices, you can authorize additional namespaces with the --nri-management-cdi-device-namespaces option or the NRI_MANAGEMENT_CDI_DEVICE_NAMESPACES environment variable, enabling centralized management workloads without granting access cluster-wide.
• The toolkit installer now installs nvidia-cdi-hook instead of wrapping it with a shell script. NRI-based deployments can invoke the hook on hosts that do not provide a shell.

Full Changelog: v1.19.1...v1.20.0

What's Changed

• Validate imex channels for CDI/JIT-CDI mode
• [wsl2] discover all .so, .bin, and .dll files at nvidia driver store path
• [nri plugin] move cdi inject info log to debug level
• [CDI Hooks] add ability to specify OCI hook type
• [cudacompat] use Masterminds/semver for parsing/comparing versions
• Expand cudacompat hook to always inspect libcuda.so ELF header if available
• Add libnvidia-nvvm70.so library to CDI spec
• fix: prevent config option from reverting modified configurations
• [nri-plugin] support injection of management CDI devices in multiple namespaces
• docker.mk: Add Podman support and fix volume/artifact handling
• Only generate update-ldcache hook if libraries are discovered
• Mount nvidia.icd file needed by OpenCL
• Update SECURITY.md
• Allow multiple driver library paths
• import text/template instead of html/template
• feat: drop nvidia-cdi-hook shell shim
• Fix nvidia-cdi-refresh systemd packaging
• Add ability to disable CDI hooks in jit-cdi mode
• [nri] only restrict management CDI devices to the toolkit namespace

Changes in the Toolkit Container

• Ship static busybox shell in container-toolkit image

Changes in libnvidia-container

• refactor: Use version script instead of linker script for library symbols

NOTE: This release is a unified release of the NVIDIA Container Toolkit that consists of the following packages:

• libnvidia-container 1.19.1
• nvidia-container-toolkit 1.19.1

The packages for this release are published to the libnvidia-container package repositories.

What's Changed

• Fix the nvidia-cdi-refresh systemd unit conditions to work on WSL2.
• Remove the dependency on multi-user.target in the nvidia-cdi-refresh systemd service. For more information, see issue #1735.
• The egl-wayland2 library and configuration file are now discovered and added to CDI specifications.
• Fix handling of the CUDA compat header on Orin systems.
• Fix default path used by nvidia-ctk and nvidia-ctk-installer for cri-o’s drop-in configuration directory.
• Add support for injecting /dev/dri* device nodes for MIG devices.
• Add disable-ipc-discoverer feature flag to the nvcdi library for disabling the discovery of IPC sockets.

What's Changed

NOTE: This release is a unified release of the NVIDIA Container Toolkit that consists of the following packages:

• libnvidia-container 1.19.0
• nvidia-container-toolkit 1.19.0

The packages for this release are published to the libnvidia-container package repositories.

This is a feature release and includes the following high-level changes:

• Report errors when unknown OCI runtime spec fields are detected.
• Added support for IGX 2.0 Thor-based systems including those with dGPUs installed.
• Added support for CUDA Forward Compatibility on Tegra-based systems. On Orin-based systems, this requires specific compat libraries in the container.
• Added support for running containers as a user that may not have explicit access to a device node without requiring that additional groups be explicitly specified.
• Improve triggering of systemd service that ensures that CDI specifications are up-to-date.
• Add support for read-only root filesystems such as those on an initramfs.

Changes since v1.19.0-rc.7

• Bump version for v1.19.0 release by @elezar in #1722

v1.19.0-rc.7

• build(deps): Bump actions/download-artifact from 7 to 8 by @dependabot[bot] in #1683
• build(deps): Bump actions/upload-artifact from 6 to 7 by @dependabot[bot] in #1684
• stop marking PRs as stale by @rahulait in #1675
• fix: Ensure that CUDA Compat Container path is set by default by @elezar in #1690
• fix: Use correct version when updating versions.mk by @elezar in #1681
• chore: Run goimports on mocks by @elezar in #1692
• chore: Rename isIntegratedGPUID function to isOrinGPUID by @elezar in #1674
• Use automatic CDI spec generation to generate CDI specs for other modifiers by @elezar in #1667
• fix: Don't use driver version in ELF header for compat check by @elezar in #1697
• fix: Reuse instantiated editsFactory in CDI by @elezar in #1666
• Bump version for v1.19.0-rc.7 release by @elezar in #1698

v1.19.0-rc.6

• Use host CUDA version for all CSV compat checks by @elezar in #1679
• Bump version for v1.19.0-rc.6 release by @elezar in #1680

v1.19.0-rc.5

• Separate nvcdi construction options from runtime options by @elezar in #1659
• build(deps): Bump nvidia/distroless/go from v4.0.1-dev to v4.0.2-dev in /deployments/container by @dependabot[bot] in #1672
• chore: Fix release tooling to support annotated tags by @elezar in #1669
• allow stale cache to be deleted by @rahulait in #1670
• Remove dlopen locator by @elezar in #1676
• Bump version for v1.19.0-rc.5 release by @elezar in #1678

v1.19.0-rc.4

• build(deps): Bump golang from 1.25.6 to 1.25.7 in /deployments/devel by @dependabot[bot] in #1640
• build(deps): Bump golang.org/x/mod from 0.32.0 to 0.33.0 by @dependabot[bot] in #1642
• build(deps): Bump github.com/onsi/ginkgo/v2 from 2.27.5 to 2.28.1 in /tests by @dependabot[bot] in #1619
• build(deps): Bump golang from 1.25.7 to 1.26.0 in /deployments/devel by @dependabot[bot] in #1647
• build(deps): Bump golang.org/x/crypto from 0.47.0 to 0.48.0 in /tests by @dependabot[bot] in #1644
• build(deps): Bump golang.org/x/sys from 0.40.0 to 0.41.0 by @dependabot[bot] in #1641
• Refactor device node tests by @elezar in #1637
• Add debug-level log option for ctk-installer by @jactor-sue in #1551
• build(deps): Bump golang.org/x/mod from 0.32.0 to 0.33.0 in /tests by @dependabot[bot] in #1643
• fix: Correct container compat path for orin by @elezar in #1649
• Refactor ldcache locator by @elezar in #1648
• test: Add to package with Ptr function by @elezar in #1653
• chore: Rename management receiver from m to l by @elezar in #1654
• test: Don't use modifier to add test runtime hook by @elezar in #1656
• refactor: Move update-ldcache argument processing into hook creator by @elezar in #1651
• Add a modifier factory by @elezar in #1655
• refactor: Prefer cdilib methods over functions by @elezar in #1658
• Fix golangci-lint errors by @elezar in #1662
• fix: Fix support for nvswitch mode by @elezar in #1661
• refactor: Move NormalizeSearchPaths into lookup package by @elezar in #1660
• Allow Orin CUDA forward compat root to be specified by @elezar in #1614
• fix: Fix typo due to merge conflict by @elezar in #1664
• refactor: Add AsOptional wrapper for lookup.Locator by @elezar in #1645
• Generate CDI specification including additional GIDs by @elezar in #630
• Add dev root to driver by @elezar in #1652
• Add dlopen library locator by @elezar in #1646
• Bump version for v1.19.0-rc.4 release by @elezar in #1668

v1.19.0-rc.3

• [CI] : add github issue templates by @rahulait in #1613
• test: Adjust e2e test for Docker 29.2.0 by @elezar in #1615
• Use rpmrebuild instead of fpm to rebuild rpm packages by @elezar in #1612
• build(deps): Bump third_party/libnvidia-container from a83ddc0 to fe0d8e5 by @dependabot[bot] in #1622
• build(deps): Bump nvidia/distroless/go from v4.0.0-dev to v4.0.1-dev in /deployments/container by @dependabot[bot] in #1618
• build(deps): Bump third_party/libnvidia-container from fe0d8e5 to 7585946 by @dependabot[bot] in #1623
• build(deps): Bump github.com/onsi/gomega from 1.39.0 to 1.39.1 in /tests by @dependabot[bot] in #1620
• fix: Allow config options to be set to default values by @elezar in #1629
• Expose internal packages for external consumption by @elezar in #1577
• fix: Log actual CDI spec version by @elezar in #1633
• fix: Set device node GID in CDI specs by @elezar in #1631
• Bump version for v1.19.0-rc.3 release by @elezar in #1636
• Remove restart logic from nvidia-cdi-refresh.service by @elezar in #1638

v1.19.0-rc.2

• [no-relnote] Fix typo in release script by @elezar in #1529
• Bump github.com/onsi/gomega from 1.38.2 to 1.38.3 in /tests by @dependabot[bot] in #1533
• trim extraneous whitespaces when parsing comma-separated config sources by @tariq1890 in #1535
• Bump nvidia/distroless/go from v3.2.1-dev to v3.2.2-dev in /deployments/container by @dependabot[bot] in #1547
• fix: Return error on JIT CDI spec generation failure by @elezar in #1538
• Bump tags.cncf.io/container-device-interface from 1.0.2-0.20251114135136-1b24d969689f to 1.1.0 by @dependabot[bot] in https://gith...

What's Changed

• build(deps): Bump actions/download-artifact from 7 to 8 by @dependabot[bot] in #1683
• build(deps): Bump actions/upload-artifact from 6 to 7 by @dependabot[bot] in #1684
• stop marking PRs as stale by @rahulait in #1675
• fix: Ensure that CUDA Compat Container path is set by default by @elezar in #1690
• fix: Use correct version when updating versions.mk by @elezar in #1681
• chore: Run goimports on mocks by @elezar in #1692
• chore: Rename isIntegratedGPUID function to isOrinGPUID by @elezar in #1674
• Use automatic CDI spec generation to generate CDI specs for other modifiers by @elezar in #1667
• fix: Don't use driver version in ELF header for compat check by @elezar in #1697
• fix: Reuse instantiated editsFactory in CDI by @elezar in #1666
• Bump version for v1.19.0-rc.7 release by @elezar in #1698

Full Changelog: v1.19.0-rc.6...v1.19.0-rc.7

What's Changed

• Use host CUDA version for all CSV compat checks by @elezar in #1679
• Bump version for v1.19.0-rc.6 release by @elezar in #1680

Full Changelog: v1.19.0-rc.5...v1.19.0-rc.6

Fixes a critical error in CDI spec generation in v1.19.0-rc.4.

What's Changed

• Separate nvcdi construction options from runtime options by @elezar in #1659
• build(deps): Bump nvidia/distroless/go from v4.0.1-dev to v4.0.2-dev in /deployments/container by @dependabot[bot] in #1672
• chore: Fix release tooling to support annotated tags by @elezar in #1669
• allow stale cache to be deleted by @rahulait in #1670
• Remove dlopen locator by @elezar in #1676
• Bump version for v1.19.0-rc.5 release by @elezar in #1678

Full Changelog: v1.19.0-rc.4...v1.19.0-rc.5

What's Changed

• build(deps): Bump golang from 1.25.6 to 1.25.7 in /deployments/devel by @dependabot[bot] in #1640
• build(deps): Bump golang.org/x/mod from 0.32.0 to 0.33.0 by @dependabot[bot] in #1642
• build(deps): Bump github.com/onsi/ginkgo/v2 from 2.27.5 to 2.28.1 in /tests by @dependabot[bot] in #1619
• build(deps): Bump golang from 1.25.7 to 1.26.0 in /deployments/devel by @dependabot[bot] in #1647
• build(deps): Bump golang.org/x/crypto from 0.47.0 to 0.48.0 in /tests by @dependabot[bot] in #1644
• build(deps): Bump golang.org/x/sys from 0.40.0 to 0.41.0 by @dependabot[bot] in #1641
• Refactor device node tests by @elezar in #1637
• Add debug-level log option for ctk-installer by @jactor-sue in #1551
• build(deps): Bump golang.org/x/mod from 0.32.0 to 0.33.0 in /tests by @dependabot[bot] in #1643
• fix: Correct container compat path for orin by @elezar in #1649
• Refactor ldcache locator by @elezar in #1648
• test: Add to package with Ptr function by @elezar in #1653
• chore: Rename management receiver from m to l by @elezar in #1654
• test: Don't use modifier to add test runtime hook by @elezar in #1656
• refactor: Move update-ldcache argument processing into hook creator by @elezar in #1651
• Add a modifier factory by @elezar in #1655
• refactor: Prefer cdilib methods over functions by @elezar in #1658
• Fix golangci-lint errors by @elezar in #1662
• fix: Fix support for nvswitch mode by @elezar in #1661
• refactor: Move NormalizeSearchPaths into lookup package by @elezar in #1660
• Allow Orin CUDA forward compat root to be specified by @elezar in #1614
• fix: Fix typo due to merge conflict by @elezar in #1664
• refactor: Add AsOptional wrapper for lookup.Locator by @elezar in #1645
• Generate CDI specification including additional GIDs by @elezar in #630
• Add dev root to driver by @elezar in #1652
• Add dlopen library locator by @elezar in #1646
• Bump version for v1.19.0-rc.4 release by @elezar in #1668

New Contributors

• @jactor-sue made their first contribution in #1551

Full Changelog: v1.19.0-rc.3...v1.19.0-rc.4

What's Changed

• [CI] : add github issue templates by @rahulait in #1613
• test: Adjust e2e test for Docker 29.2.0 by @elezar in #1615
• Use rpmrebuild instead of fpm to rebuild rpm packages by @elezar in #1612
• build(deps): Bump third_party/libnvidia-container from a83ddc0 to fe0d8e5 by @dependabot[bot] in #1622
• build(deps): Bump nvidia/distroless/go from v4.0.0-dev to v4.0.1-dev in /deployments/container by @dependabot[bot] in #1618
• build(deps): Bump third_party/libnvidia-container from fe0d8e5 to 7585946 by @dependabot[bot] in #1623
• build(deps): Bump github.com/onsi/gomega from 1.39.0 to 1.39.1 in /tests by @dependabot[bot] in #1620
• fix: Allow config options to be set to default values by @elezar in #1629
• Expose internal packages for external consumption by @elezar in #1577
• fix: Log actual CDI spec version by @elezar in #1633
• fix: Set device node GID in CDI specs by @elezar in #1631
• Bump version for v1.19.0-rc.3 release by @elezar in #1636
• Remove restart logic from nvidia-cdi-refresh.service by @elezar in #1638

New Contributors

• @rahulait made their first contribution in #1613

Full Changelog: v1.19.0-rc.2...v1.19.0-rc.3

What's Changed

• [no-relnote] Fix typo in release script by @elezar in #1529
• Bump github.com/onsi/gomega from 1.38.2 to 1.38.3 in /tests by @dependabot[bot] in #1533
• trim extraneous whitespaces when parsing comma-separated config sources by @tariq1890 in #1535
• Bump nvidia/distroless/go from v3.2.1-dev to v3.2.2-dev in /deployments/container by @dependabot[bot] in #1547
• fix: Return error on JIT CDI spec generation failure by @elezar in #1538
• Bump tags.cncf.io/container-device-interface from 1.0.2-0.20251114135136-1b24d969689f to 1.1.0 by @dependabot[bot] in #1536
• Bump github.com/opencontainers/runc from 1.3.3 to 1.4.0 by @dependabot[bot] in #1504
• Bump actions/download-artifact from 6 to 7 by @dependabot[bot] in #1549
• Bump actions/upload-artifact from 5 to 6 by @dependabot[bot] in #1548
• Fix library driver library resolution in musl-based containers by @elezar in #1530
• Bump github.com/onsi/gomega from 1.38.3 to 1.39.0 in /tests by @dependabot[bot] in #1566
• Bump golang.org/x/sys from 0.39.0 to 0.40.0 by @dependabot[bot] in #1564
• fix: Fix arguments when creating DRM device symlinks by @elezar in #1563
• Bump github.com/onsi/ginkgo/v2 from 2.27.3 to 2.27.4 in /tests by @dependabot[bot] in #1565
• fix: Don't inject device nodes when NVIDIA_VISIBLE_DEVICES=none by @elezar in #1562
• Bump golang.org/x/mod from 0.31.0 to 0.32.0 by @dependabot[bot] in #1572
• Bump golang.org/x/crypto from 0.46.0 to 0.47.0 in /tests by @dependabot[bot] in #1575
• Bump github.com/onsi/ginkgo/v2 from 2.27.4 to 2.27.5 in /tests by @dependabot[bot] in #1574
• Bump nvidia/distroless/go from v3.2.2-dev to v4.0.0-dev in /deployments/container by @dependabot[bot] in #1576
• implement NRI plugin server to inject management CDI devices by @tariq1890 in #1498
• Bump github.com/urfave/cli/v3 from 3.6.1 to 3.6.2 by @dependabot[bot] in #1590
• Ensure that IPC sockets are not mounted read-only by @faganihajizada in #1593
• Simplify logger.Interface by @elezar in #1591
• fix: Allow nvidia-cdi-refresh.service to be restarted by @elezar in #1586
• fix: Ensure consistent device permissions for CDI devices by @oOraph in #1539
• Bump golang from 1.25.5 to 1.25.6 in /deployments/devel by @dependabot[bot] in #1592
• Bump github.com/sirupsen/logrus from 1.9.3 to 1.9.4 by @dependabot[bot] in #1584
• Add no-runtime-configure option to the toolkit installer by @elezar in #1521
• Add basic test for CSV CDI spec generation by @elezar in #1604
• [no-relnote] Set golang version when building libnvidia-container by @elezar in #1597
• restrict NRI injection of CDI devices to pods in the toolkit's namespace by @tariq1890 in #1601
• Ensure that the CSV files define the required devices when only a iGPU is present by @elezar in #1556
• Query CUDA forward compatibility elf note if available by @elezar in #1598
• fix: Add docker swarm support to jit-cdi mode by @elezar in #1600
• fix: Include /dev/nvidia2 in container requesting iGPU if required by @elezar in #1561
• Bump version for v1.19.0-rc.2 release by @elezar in #1609
• [no-relnote] Fix version bump by @elezar in #1610

New Contributors

• @faganihajizada made their first contribution in #1593
• @oOraph made their first contribution in #1539

Full Changelog: v1.19.0-rc.1...v1.19.0-rc.2