Use when setting up NVIDIA GPUs, CUDA, container toolkit, or Ollama on Ubuntu 24.04 LTS — driver installation (proprietary vs open kernel module), CUDA toolkit, cuDNN, NVIDIA Container Toolkit for Docker, Ollama installation and model management, GPU monitoring (nvidia-smi), vGPU basics, multi-GPU configuration, and troubleshooting. Part of the ubuntu-* skill family.
Companion skill to ubuntu-server-admin covering GPU compute and local LLM workloads on Ubuntu Server 24.04.4 LTS (Noble Numbat). See also: ubuntu-docker-host, ubuntu-monitoring.
# Recommended: auto-detect and install
sudo apt update && ubuntu-drivers list
sudo ubuntu-drivers install # recommended driver
sudo ubuntu-drivers install nvidia:560 # specific version
sudo reboot
# Alternative: PPA for latest branches
sudo add-apt-repository ppa:graphics-drivers/ppa -y && sudo apt update
sudo apt install nvidia-driver-560 -y && sudo reboot
# Open kernel module (Turing+ / RTX 20xx and newer)
sudo apt install nvidia-driver-560-open -y
cat /proc/driver/nvidia/version # confirms "Open Module" if active
| Architecture | Proprietary | nvidia-open |
|---|---|---|
| Maxwell / Pascal (GTX 10xx) | Supported | NOT supported |
| Turing (RTX 20xx / T4) | Supported | Supported |
| Ampere (RTX 30xx / A100) | Supported | Recommended |
| Ada Lovelace (RTX 40xx / L40) | Supported | Recommended |
| Hopper / Blackwell (H100/B200) | Supported | Required |
# Blacklist nouveau (ubuntu-drivers does this automatically; manual installs need it)
sudo tee /etc/modprobe.d/blacklist-nouveau.conf > /dev/null <<'EOF'
blacklist nouveau
options nouveau modeset=0
EOF
sudo update-initramfs -u && sudo reboot
# Verify installation
nvidia-smi # GPU info, driver + CUDA version
cat /proc/driver/nvidia/version # kernel module details
dkms status # DKMS module state
lsmod | grep nvidia # loaded modules
# DKMS rebuild after kernel update
sudo dkms autoinstall && sudo reboot
# Secure Boot check
mokutil --sb-state
# If driver fails after install: reboot -> blue MOK Manager -> Enroll MOK -> enter password
# Add NVIDIA CUDA repo (Ubuntu 24.04)
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb && sudo apt update
# Install (also installs driver if missing)
sudo apt install cuda-toolkit -y # latest
sudo apt install cuda-toolkit-12-6 -y # specific version
# Environment variables -- add system-wide
sudo tee /etc/profile.d/cuda.sh > /dev/null <<'EOF'
export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
EOF
source /etc/profile.d/cuda.sh
# Verify
nvcc --version
ls -la /usr/local/cuda # symlink to active version
# Multiple versions side-by-side
sudo apt install cuda-toolkit-12-4 cuda-toolkit-12-6 -y
# Switch: update the symlink
sudo rm /usr/local/cuda && sudo ln -s /usr/local/cuda-12.4 /usr/local/cuda
# Or per-session: export PATH=/usr/local/cuda-12.6/bin:$PATH
| CUDA Version | Minimum Driver | Notes |
|---|---|---|
| 12.6.x | 560.28+ | Current recommended |
| 12.4.x | 550.54+ | Widely deployed |
| 12.2.x | 535.86+ | LTS-friendly |
| 11.8.x | 520.61+ | Legacy workloads |
# Install from NVIDIA CUDA repo (added in section 2)
sudo apt install libcudnn9-cuda-12 libcudnn9-dev-cuda-12 -y
dpkg -l | grep cudnn # verify version
# cuDNN 9.x pairs with CUDA 12.x; cuDNN 8.x with CUDA 11.x/12.x (legacy)
Requires Docker Engine (see ubuntu-docker-host).
# Install
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt update && sudo apt install nvidia-container-toolkit -y
# Configure Docker runtime
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
# Test
docker run --rm --gpus all nvidia/cuda:12.6.3-base-ubuntu24.04 nvidia-smi
# Specific GPUs
docker run --rm --gpus '"device=0,1"' nvidia/cuda:12.6.3-base-ubuntu24.04 nvidia-smi
Docker Compose GPU reservation: