In this article, you'll build a reusable GPU-optimized machine image using Packer, pre-loaded with NVIDIA drivers, CUDA Toolkit, NVIDIA Container Toolkit, DCGM, and system-level GPU tuning like persistence mode.

Table of Contents

• Prerequisites

• Project Setup

• Step 1: Install Packer

• Step 2: Set Up Project Directory

• Step 3: Install Packer's Plugins

• Step 4: Define Your Source

• Step 5: Writing the Build Template

• Step 6: Writing the GPU Provisioning Script

  • section 1: Pre-Installation (Kernel Headers)

  • Section 2: Installing NVIDIA's Apt Repository

  • Section 3: Pinning NVIDIA Drivers Version

  • Section 4: Installing the Driver

  • Section 5: CUDA Toolkit Installation

  • Section 6: Nvidia Container Toolkit

  • Section 7: Installing DCGM — Data Center GPU Manager

  • Section 8: Enabling Persistence Mode

  • Section 9: System Tuning for GPU Compute Workloads

• Step 7:Assembling and Running the Build

• Step 8: Test the Image and Verify the GPU Stack

• Conclusion

• References

Prerequisites

• HashiCorp Packer >= 1.9

• Google Compute Packer plugin (installed via packer init)

• Optionally, the AWS Packer plugin can be used for EC2 builds by adding an amazon-ebs source to node.pkr.hcl

• GCP project with Compute Engine API enabled (or AWS account with EC2 access)

• GCP authentication (gcloud auth application-default login) or AWS credentials

• Access to an NVIDIA GPU instance type (For example, A100, H100, L4 on GCP; p4d, p5, G6 on AWS)

Project Setup

Step 1: Install Packer

To get started, you'll install Packer with the steps below if you're on macOS (or you can follow the official documentation for Linux and Windows installation guides).

First, you'll install the official Packer formula from the terminal.

Install the HashiCorp tap, a repository of all Hashicorp packages.

$ brew tap hashicorp/tap

Now, install Packer with hashicorp/tap/packer.

$ brew install hashicorp/tap/packer

Step 2: Set Up Project Directory

With Packer installed, you'll create your project directory. For clean code and separation of concerns, your project directory should look like the below. Go ahead and create these files in your packer_demo folder using the command below:

mkdir -p packer_demo/script && touch packer_demo/{build.pkr.hcl,source.pkr.hcl,variable.pkr.hcl,local.pkr.hcl,plugins.pkr.hcl,values.pkrvars.hcl} packer_demo/script/base.sh

Your file directory should look like this:

packer_demo
├── build.pkr.hcl                 # Build pipeline — provisioner ordering
├── source.pkr.hcl                # GCP source definition (googlecompute)
├── variable.pkr.hcl              # Variable definitions with defaults
├── local.pkr.hcl                 # Local values
├── plugins.pkr.hcl                # Packer plugin requirements
├── values.pkrvars.hcl             # variable values (copy and customize)
├── script/
│   ├── base.sh                  # requirement script 

Step 3: Install Packer's Plugins

In your plugins.pkr.hcl file,, define your plugins in the packer block. The packer {} block contains Packer settings, including specifying a required plugin version. You'll find the required_plugins block in the Packer block, which specifies all the plugins required by the template to build your image. If you're on Azure or AWS, you can check for the latest plugin here.

packer {
  required_plugins {
    googlecompute = {
      source  = "github.com/hashicorp/googlecompute"
      version = "~> 1"
    }
  }
}

Then, initialize your Packer plugin with the command below:

packer init .

Step 4: Define Your Source

With your plugin initialized, you can now define your source block. The source block configures a specific builder plugin, which is then invoked by a build block. Source blocks contain your project ID, the zone where your machine will be created, the source_image_family (think of this as your base image, such as Debian, Ubuntu, and so on), and your source_image_project_id.

In GCP, each has an image project ID, such as "ubuntu-os-cloud" for Ubuntu. You'll set the machine type to a GPU machine type because you're building your base image for a GPU machine, so the machine on which it will be created needs to be able to run your commands.

source "googlecompute" "gpu-node" {
  project_id              = var.project_id
  zone                    = var.zone
  source_image_family     = var.image_family
  source_image_project_id = var.image_project_id
  ssh_username            = var.ssh_username
  machine_type            = var.machine_type



  image_name        = var.image_name
  image_description = var.image_description

  disk_size           = var.disk_size
  on_host_maintenance = "TERMINATE"

  tags = ["gpu-node"]

}

Setting on_host_maintenance = "TERMINATE" on Google Cloud Compute Engine ensures that a VM instance stops instead of live-migrating during infrastructure maintenance. This is important when using GPUs or specialized hardware that can't migrate, preventing data corruption.

You'll define all your variables in the variable.pkr.hcl file, and set the values in the values.pkrvars.hcl. Remember to always add your values.pkrvars.hcl file to Gitignore.

variable "image_name" {
  type        = string
  description = "The name of the resulting image"
}

variable "image_description" {
  type        = string
  description = "Description of the image"
}

variable "project_id" {
  type        = string
  description = "The GCP project ID where the image will be created"
}

variable "image_family" {
  type        = string
  description = "The image family to which the resulting image belongs"
}

variable "image_project_id" {
  type        = list(string)
  description = "The project ID(s) to search for the source image"
}

variable "zone" {
  type        = string
  description = "The GCP zone where the build instance will be created"
}

variable "ssh_username" {
  type        = string
  description = "The SSH username to use for connecting to the instance"
}
variable "machine_type" {
  type        = string
  description = "The machine type to use for the build instance"
}

variable "cuda_version" {
  type        = string
  description = "CUDA toolkit version"
  default     = "13.1"
}

variable "driver_version" {
  type        = string
  description = "NVIDIA driver version"
  default     = "590.48.01"
}

variable "disk_size" {
  type        = number
  description = "Boot disk size in GB"
  default     = 50
}

values.pkrvars.hcl

image_name        = "base-gpu-image-{{timestamp}}"
image_description = "Ubuntu 24.04 LTS with gpu drivers and health checks"
project_id        = "your gcp project id"
image_family      = "ubuntu-2404-lts-amd64"
image_project_id  = ["ubuntu-os-cloud"]
zone              = "us-central1-a"
ssh_username      = "packer"
machine_type      = "g2-standard-4"
disk_size        = 50
driver_version   = "590.48.01"
cuda_version      = "13.1" 

Step 5: Writing the Build Template

Create build.pkr.hcl. The build block creates a temporary instance, runs provisioners, and produces an image.

Provisioners in this template are organized as follows:

• First provisioner runs system updates and upgrades.

• Second provisioner reboots the instance (expect_disconnect = true).

• Third provisioner waits for the instance to come back (pause_before), then runs script/base.sh. This provisioner sets max_retries to handle transient SSH timeouts and pass environment variables for DRIVER_VERSION and CUDA_VERSION.

Lastly, you have the post-processor to tell you the image ID and completion status:

build {
  sources = ["source.googlecompute.gpu-node"]

  provisioner "shell" {
    inline = [
      "set -e",
      "sudo apt update",
      "sudo apt -y dist-upgrade"
    ]
  }

  provisioner "shell" {
    expect_disconnect = true
    inline            = ["sudo reboot"]
  }

  # Base: NVIDIA drivers, CUDA, DCGM
  provisioner "shell" {
    pause_before = "60s"
    script       = "script/base.sh"
    max_retries  = 2
    environment_vars = [
      "DRIVER_VERSION=${var.driver_version}",
      "CUDA_VERSION=${var.cuda_version}"
    ]
  }

  post-processor "shell-local" {
    inline = [
      "echo '=== Image Build Complete ==='",
      "echo 'Image ID: ${build.ID}'",
      "date"
    ]
  }
}

Step 6: Writing the GPU Provisioning Script

Now we'll go through the base script, and break down some parts of it.

Section 1: Pre-Installation (Kernel Headers)

Before installing NVIDIA drivers, the system needs kernel headers and build tools. The NVIDIA driver compiles a kernel module during installation via DKMS, so if the headers for your running kernel aren't present, the build will fail silently, and the driver won't load on boot.

log "Installing kernel headers and build tools..."
sudo apt-get install -qq -y \
  "linux-headers-$(uname -r)" \
  build-essential \
  dkms \
  curl \
  wget

Section 2: Installing NVIDIA's Apt Repository

This snippet downloads and installs NVIDIA’s official keyring package based on your OS Linux distribution, which adds the trusted signing keys needed for the system to verify CUDA packages.

log "Adding NVIDIA CUDA apt repository (${DISTRO})..."
wget -q "https://developer.download.nvidia.com/compute/cuda/repos/\({DISTRO}/\){ARCH}/cuda-keyring_1.1-1_all.deb" \
  -O /tmp/cuda-keyring.deb
sudo dpkg -i /tmp/cuda-keyring.deb
rm /tmp/cuda-keyring.deb
sudo apt-get update -qq

Section 3: Pinning NVIDIA Drivers Version

Pinning the NVIDIA driver to a specific version ensures that the system always installs and keeps using exactly that driver version, even when newer drivers appear in the repository.

NVIDIA drivers are tightly coupled with CUDA toolkit versions, Kernel versions, and container runtimes like Docker or NVIDIA Container Toolkit

A mismatch, such as the system auto‑upgrading to a newer driver, can cause CUDA to stop working, break GPU acceleration, or make the machine image inconsistent across deployments.

log "Pinning driver to version ${DRIVER_VERSION}..."
sudo apt-get install -qq -y "nvidia-driver-pinning-${DRIVER_VERSION}"

Section 4: Installing the Driver

The libnvidia-compute installs only the compute‑related user‑space libraries (CUDA driver components), while the nvidia-dkms-open; installs the open‑source NVIDIA kernel module, built locally via DKMS.

Together, these two packages give you a fully functional CUDA driver environment without any GUI or graphics dependencies.

Here, we're using NVIDIA’s compute‑only driver stack using the open‑source kernel modules, as it deliberately avoids installing any display-related components, which you don't need.

This method provides an installation module based on DKMS that's better aligned with Linux distros, as it's lightweight, and compute-focused.

log "Installing NVIDIA compute-only driver (open kernel modules)..."
sudo apt-get -V install -y \
  libnvidia-compute \
  nvidia-dkms-open

Section 5: CUDA Toolkit Installation

This part of the script installs the CUDA Toolkit for the specified version and then makes sure that CUDA’s executables and libraries are available system‑wide for every user and every shell session.

It adds CUDA binaries to PATH, so commands like nvcc, cuda-gdb, and cuda-memcheck work without specifying full paths. It also adds CUDA libraries to LD_LIBRARY_PATH, so applications can find CUDA’s shared libraries at runtime.

log "Installing CUDA Toolkit ${CUDA_VERSION}..."
sudo apt-get install -qq -y "cuda-toolkit-${CUDA_VERSION}"

# Persist CUDA paths for all users and sessions
cat <<'EOF' | sudo tee /etc/profile.d/cuda.sh
export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:${LD_LIBRARY_PATH:-}
EOF
echo "/usr/local/cuda/lib64" | sudo tee /etc/ld.so.conf.d/cuda.conf
sudo ldconfig

Section 6: NVIDIA Container Toolkit

This block installs the NVIDIA Container Toolkit and configures it so that containers (Docker or containerd) can access the GPU safely and correctly. It’s a critical step for Kubernetes GPU nodes, Docker GPU workloads, and any system that needs GPU acceleration inside containers.

log "Installing NVIDIA Container Toolkit..."
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
  | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg

curl -fsSL https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \
  | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' \
  | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update -qq
sudo apt-get install -qq -y nvidia-container-toolkit

# Configure for containerd (primary Kubernetes runtime)
sudo nvidia-ctk runtime configure --runtime=containerd

# Configure for Docker if present on this image
if systemctl list-unit-files | grep -q "^docker.service"; then
  sudo nvidia-ctk runtime configure --runtime=docker
fi

Section 7: Installing DCGM (Data Center GPU Manager)

This section covers the installation and validation of NVIDIA DCGM (Data Center GPU Manager), which is NVIDIA’s official management and telemetry framework for data center GPUs.

It offers health monitoring and diagnostics, telemetry (including temperature, clocks, power, and utilization), error reporting, and integration with Kubernetes, Prometheus, and monitoring agents. Your GPU monitoring stack relies on this.

The script extracts the installed version and checks that it meets the minimum required version for NVIDIA driver 590+. Then it enforces the version requirement. This prevents a mismatch between the GPU driver and DCGM, which would break monitoring and health checks. It also enables fabric manager for NVLink/NVswitches, if you're on a Multi‑GPU topologies like A100/H100 DGX or multi‑GPU servers.

log "Installing DCGM..."
sudo apt-get install -qq -y datacenter-gpu-manager

DCGM_VER=\((dpkg -s datacenter-gpu-manager 2>/dev/null | awk '/^Version:/{print \)2}' | sed 's/^[0-9]*://')
DCGM_MAJOR=\((echo "\){DCGM_VER}" | cut -d. -f1)
DCGM_MINOR=\((echo "\){DCGM_VER}" | cut -d. -f2)
if [[ "\({DCGM_MAJOR}" -lt 4 ]] || { [[ "\){DCGM_MAJOR}" -eq 4 ]] && [[ "${DCGM_MINOR}" -lt 3 ]]; }; then
  error "DCGM ${DCGM_VER} is below the 4.3 minimum required for driver 590+. Check your CUDA repo."
fi
log "DCGM installed: ${DCGM_VER}"

sudo systemctl enable nvidia-dcgm
sudo systemctl start  nvidia-dcgm

# Fabric Manager — only needed for NVLink/NVSwitch GPUs (A100/H100 multi-GPU nodes)
if systemctl list-unit-files | grep -q "^nvidia-fabricmanager.service"; then
  log "Enabling nvidia-fabricmanager for NVLink GPUs..."
  sudo systemctl enable nvidia-fabricmanager
  sudo systemctl start  nvidia-fabricmanager
fi

Section 8: Enabling Persistence Mode

The NVIDIA driver normally unloads itself when the GPU is idle. When a new workload starts, the driver must reload, reinitialize the GPU, and set up memory mappings. This adds a delay of a few hundred milliseconds to several seconds, depending on the GPU and system.

Enabling nvidia‑persistenced keeps the NVIDIA driver loaded in memory even when no GPU workloads are running.

log "Enabling nvidia-persistenced..."
sudo systemctl enable nvidia-persistenced
sudo systemctl start  nvidia-persistenced

Section 9: System Tuning for GPU Compute Workloads

This block applies a set of system‑level performance and stability tunings that are standard for high‑performance GPU servers, Kubernetes GPU nodes, and ML/AI workloads.

Each line targets a specific bottleneck or instability pattern that appears in real GPU production environments.

• Swap and memory behavior: Disabling swap and setting vm.swappiness=0 prevents the kernel from pushing GPU‑bound processes into swap. GPU workloads are extremely sensitive to latency, and swapping can cause CUDA context resets and GPU driver timeouts.

• Hugepages for large memory allocations: Setting vm.nr_hugepages=2048 allocates a pool of hugepages, which reduces TLB pressure for large contiguous memory allocations.

  CUDA, NCCL, and deep‑learning frameworks frequently allocate large buffers, and hugepages reduce page‑table overhead, improving memory bandwidth and lowering latency for large tensor operations. This is especially useful on multi‑GPU servers.

• CPU frequency governor: Installing cpupower and forcing the CPU governor to performance ensures the CPU stays at maximum frequency instead of scaling down.

  GPU workloads often become CPU‑bound during Data preprocessing, Kernel launches, and NCCL communication. Keeping CPUs at full speed reduces jitter and improves throughput.

• NUMA and topology tools: Installing numactl, libnuma-dev, and hwloc provides tools for pinning processes to NUMA nodes, understanding CPU–GPU affinity, and optimizing multi‑GPU placement.

• Disabling irqbalance: Stopping and disabling irqbalance it lets the NVIDIA driver manage interrupt affinity. For GPU servers, irqbalance can incorrectly move GPU interrupts to suboptimal CPUs, causing higher latency and lower throughput.

log "Applying system tuning..."

# Disable swap (critical for Kubernetes scheduler and ML stability)
sudo swapoff -a
sudo sed -i '/ swap / s/^/#/' /etc/fstab
echo "vm.swappiness=0"     | sudo tee /etc/sysctl.d/99-gpu-swappiness.conf

# Hugepages — reduces TLB pressure for large memory allocations
echo "vm.nr_hugepages=2048" | sudo tee /etc/sysctl.d/99-gpu-hugepages.conf

# CPU performance governor
sudo apt-get install -qq -y linux-tools-common "linux-tools-$(uname -r)" || true
sudo cpupower frequency-set -g performance || true

# NUMA and topology tools for GPU affinity tuning
sudo apt-get install -qq -y numactl libnuma-dev hwloc

# Disable irqbalance — let NVIDIA driver manage interrupt affinity
sudo systemctl disable irqbalance || true
sudo systemctl stop    irqbalance || true

# Apply all sysctl settings now
sudo sysctl --system

Full base.sh script here:

#!/bin/bash
set -euo pipefail

log()   { echo "[BASE] $1"; }
error() { echo "[BASE][ERROR] $1" >&2; exit 1; }

###############################################################
###############################################################
[[ -z "${DRIVER_VERSION:-}" ]] && error "DRIVER_VERSION is not set."
[[ -z "${CUDA_VERSION:-}"   ]] && error "CUDA_VERSION is not set."

log "DRIVER_VERSION : ${DRIVER_VERSION}"
log "CUDA_VERSION   : ${CUDA_VERSION}"

DISTRO=\((. /etc/os-release && echo "\){ID}${VERSION_ID}" | tr -d '.')
ARCH="x86_64"

export DEBIAN_FRONTEND=noninteractive

###############################################################
# 1. System update
###############################################################
log "Updating system packages..."
sudo apt-get update -qq
sudo apt-get upgrade -qq -y

###############################################################
# 2. Pre-installation — kernel headers
#    Source: https://docs.nvidia.com/datacenter/tesla/driver-installation-guide/ubuntu.html
###############################################################
log "Installing kernel headers and build tools..."
sudo apt-get install -qq -y \
  "linux-headers-$(uname -r)" \
  build-essential \
  dkms \
  curl \
  wget

###############################################################
# 3. NVIDIA CUDA Network Repository
###############################################################
log "Adding NVIDIA CUDA apt repository (${DISTRO})..."
wget -q "https://developer.download.nvidia.com/compute/cuda/repos/\({DISTRO}/\){ARCH}/cuda-keyring_1.1-1_all.deb" \
  -O /tmp/cuda-keyring.deb
sudo dpkg -i /tmp/cuda-keyring.deb
rm /tmp/cuda-keyring.deb
sudo apt-get update -qq

###############################################################
# 4. Pin driver version BEFORE installation (590+ requirement)
###############################################################
log "Pinning driver to version ${DRIVER_VERSION}..."
sudo apt-get install -qq -y "nvidia-driver-pinning-${DRIVER_VERSION}"

###############################################################
# 5. Compute-only (headless) driver — Open Kernel Modules
#    Source: NVIDIA Driver Installation Guide — Compute-only System (Open Kernel Modules)
#
#    libnvidia-compute  = compute libraries only (no GL/Vulkan/display)
#    nvidia-dkms-open   = open-source kernel module built via DKMS
#
#    Open kernel modules are the NVIDIA-recommended choice for
#    Ampere, Hopper, and Blackwell data centre GPUs (A100, H100, etc.)
###############################################################
log "Installing NVIDIA compute-only driver (open kernel modules)..."
sudo apt-get -V install -y \
  libnvidia-compute \
  nvidia-dkms-open

###############################################################
# 6. CUDA Toolkit
###############################################################
log "Installing CUDA Toolkit ${CUDA_VERSION}..."
sudo apt-get install -qq -y "cuda-toolkit-${CUDA_VERSION}"

# Persist CUDA paths for all users and sessions
cat <<'EOF' | sudo tee /etc/profile.d/cuda.sh
export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:${LD_LIBRARY_PATH:-}
EOF
echo "/usr/local/cuda/lib64" | sudo tee /etc/ld.so.conf.d/cuda.conf
sudo ldconfig

###############################################################
# 7. NVIDIA Container Toolkit
#    Required for GPU workloads in Docker / containerd / Kubernetes
###############################################################
log "Installing NVIDIA Container Toolkit..."
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
  | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg

curl -fsSL https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \
  | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' \
  | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update -qq
sudo apt-get install -qq -y nvidia-container-toolkit

# Configure for containerd (primary Kubernetes runtime)
sudo nvidia-ctk runtime configure --runtime=containerd

# Configure for Docker if present on this image
if systemctl list-unit-files | grep -q "^docker.service"; then
  sudo nvidia-ctk runtime configure --runtime=docker
fi

###############################################################
# 8. DCGM — DataCenter GPU Manager
###############################################################
log "Installing DCGM..."
sudo apt-get install -qq -y datacenter-gpu-manager
 
DCGM_VER=\((dpkg -s datacenter-gpu-manager 2>/dev/null | awk '/^Version:/{print \)2}' | sed 's/^[0-9]*://')
DCGM_MAJOR=\((echo "\){DCGM_VER}" | cut -d. -f1)
DCGM_MINOR=\((echo "\){DCGM_VER}" | cut -d. -f2)
if [[ "\({DCGM_MAJOR}" -lt 4 ]] || { [[ "\){DCGM_MAJOR}" -eq 4 ]] && [[ "${DCGM_MINOR}" -lt 3 ]]; }; then
  error "DCGM ${DCGM_VER} is below the 4.3 minimum required for driver 590+. Check your CUDA repo."
fi
log "DCGM installed: ${DCGM_VER}"

sudo systemctl enable nvidia-dcgm
sudo systemctl start  nvidia-dcgm

# Fabric Manager — only needed for NVLink/NVSwitch GPUs (A100/H100 multi-GPU nodes)
if systemctl list-unit-files | grep -q "^nvidia-fabricmanager.service"; then
  log "Enabling nvidia-fabricmanager for NVLink GPUs..."
  sudo systemctl enable nvidia-fabricmanager
  sudo systemctl start  nvidia-fabricmanager
fi

###############################################################
# 9. NVIDIA Persistence Daemon
#    Keeps the driver loaded between jobs — reduces cold-start
#    latency on the first CUDA call in each new workload
###############################################################
log "Enabling nvidia-persistenced..."
sudo systemctl enable nvidia-persistenced
sudo systemctl start  nvidia-persistenced

###############################################################
# 10. System tuning for GPU compute workloads
###############################################################
log "Applying system tuning..."

# Disable swap (critical for Kubernetes scheduler and ML stability)
sudo swapoff -a
sudo sed -i '/ swap / s/^/#/' /etc/fstab
echo "vm.swappiness=0"     | sudo tee /etc/sysctl.d/99-gpu-swappiness.conf

# Hugepages — reduces TLB pressure for large memory allocations
echo "vm.nr_hugepages=2048" | sudo tee /etc/sysctl.d/99-gpu-hugepages.conf

# CPU performance governor
sudo apt-get install -qq -y linux-tools-common "linux-tools-$(uname -r)" || true
sudo cpupower frequency-set -g performance || true

# NUMA and topology tools for GPU affinity tuning
sudo apt-get install -qq -y numactl libnuma-dev hwloc

# Disable irqbalance — let NVIDIA driver manage interrupt affinity
sudo systemctl disable irqbalance || true
sudo systemctl stop    irqbalance || true

# Apply all sysctl settings now
sudo sysctl --system

###############################################################
# Done
###############################################################
log "============================================"
log "Base layer provisioning complete."
log "  OS      : ${DISTRO}"
log "  Driver  : ${DRIVER_VERSION} (open kernel modules, compute-only)"
log "  CUDA    : cuda-toolkit-${CUDA_VERSION}"
log "  DCGM    : ${DCGM_VER}"
log "============================================"

Step 7: Assembling and Running the Build

Validate the template first, then run the build. Validation catches syntax or variable errors early, so the build doesn’t start on a broken config.

packer validate -var-file=values.pkrvars.hcl .

If validation succeeds, you’ll see a short confirmation like The configuration is valid.. After that, start the build. You should expect the process to create a temporary VM, run your provisioners, and produce an image:

packer build -var-file=values.pkrvars.hcl .

The build typically takes 15–20 minutes, depending on network speed and package installs. Watch the Packer log for three key checkpoints:

• Instance creation — confirms the temporary VM was provisioned.

• Provisioner output — shows each script step (updates, reboot, script/base.sh) and any errors.

• Image creation — indicates the build finished and an image artifact was written.

If the build fails, copy the failing provisioner’s log lines and re-run the build after fixing the script or variables. For quick troubleshooting, re-run the failing provisioner locally on a matching test VM to iterate faster.

googlecompute.gpu-node: output will be in this color.

==> googlecompute.gpu-node: Checking image does not exist...
==> googlecompute.gpu-node: Creating temporary RSA SSH key for instance...
==> googlecompute.gpu-node: no persistent disk to create
==> googlecompute.gpu-node: Using image: ubuntu-2404-noble-amd64-v20260225
==> googlecompute.gpu-node: Creating instance...
==> googlecompute.gpu-node: Loading zone: us-central1-a
==> googlecompute.gpu-node: Loading machine type: g2-standard-4
==> googlecompute.gpu-node: Requesting instance creation...
==> googlecompute.gpu-node: Waiting for creation operation to complete...
==> googlecompute.gpu-node: Instance has been created!
==> googlecompute.gpu-node: Waiting for the instance to become running...
==> googlecompute.gpu-node: IP: 34.58.58.214
==> googlecompute.gpu-node: Using SSH communicator to connect: 34.58.58.214
==> googlecompute.gpu-node: Waiting for SSH to become available...
systemd-logind.service
==> googlecompute.gpu-node:  systemctl restart unattended-upgrades.service
==> googlecompute.gpu-node:
==> googlecompute.gpu-node: No containers need to be restarted.
==> googlecompute.gpu-node:
==> googlecompute.gpu-node: User sessions running outdated binaries:
==> googlecompute.gpu-node:  packer @ session #1: sshd[1535]
==> googlecompute.gpu-node:  packer @ user manager service: systemd[1540]
==> googlecompute.gpu-node: Pausing 1m0s before the next provisioner...
==> googlecompute.gpu-node: Provisioning with shell script: script/base.sh
==> googlecompute.gpu-node: [BASE] DRIVER_VERSION : 590.48.01
==> googlecompute.gpu-node: [BASE] CUDA_VERSION   : 13.1
==> googlecompute.gpu-node: [BASE] Updating system packages...
==> googlecompute.gpu-node: [BASE] Installing kernel headers and build tools...
==> googlecompute.gpu-node: [BASE] Installing CUDA Toolkit 13.1...
==> googlecompute.gpu-node: [BASE] Installing DCGM...
==> googlecompute.gpu-node: [BASE] Enabling nvidia-persistenced...
==> googlecompute.gpu-node: [BASE] Applying system tuning...
==> googlecompute.gpu-node: vm.swappiness=0
==> googlecompute.gpu-node: vm.nr_hugepages=2048
==> googlecompute.gpu-node: Setting cpu: 0
==> googlecompute.gpu-node: Error setting new values. Common errors:
==> googlecompute.gpu-node: [BASE] ============================================
==> googlecompute.gpu-node: [BASE] Base layer provisioning complete.
==> googlecompute.gpu-node: [BASE]   OS      : ubuntu2404
==> googlecompute.gpu-node: [BASE]   Driver  : 590.48.01 (open kernel modules, compute-only)
==> googlecompute.gpu-node: [BASE]   CUDA    : cuda-toolkit-13.1
==> googlecompute.gpu-node: [BASE]   DCGM    : 1:3.3.9
==> googlecompute.gpu-node: [BASE] ============================================
==> googlecompute.gpu-node: Deleting instance...
==> googlecompute.gpu-node: Instance has been deleted!
==> googlecompute.gpu-node: Creating image...
==> googlecompute.gpu-node: Deleting disk...
==> googlecompute.gpu-node: Disk has been deleted!
==> googlecompute.gpu-node: Running post-processor:  (type shell-local)
==> googlecompute.gpu-node (shell-local): Running local shell script: 
==> googlecompute.gpu-node (shell-local): === Image Build Complete ===
==> googlecompute.gpu-node (shell-local): Image ID: packer-69b6c2ee-883a-3602-7bb5-059f1ba27c8b
==> googlecompute.gpu-node (shell-local): Sun Mar 15 15:50:09 WAT 2026
Build 'googlecompute.gpu-node' finished after 17 minutes 55 seconds.

==> Wait completed after 17 minutes 55 seconds

==> Builds finished. The artifacts of successful builds are:
--> googlecompute.gpu-node: A disk image was created in the 'my_project-00000' project: base-gpu-image-1773585134

Step 8: Test the Image and Verify the GPU Stack

Confirm the image exists in the GCP Console: Compute → Storage → Images and locate your newly created OS image.

Create a test VM from the image:

gcloud compute instances create my-gpu-vm \
  --machine-type=g2-standard-4 \
  --accelerator=count=1,type=nvidia-l4 \
  --image=base-gpu-image-1772718104 \
  --image-project=YOUR_PROJECT_ID \
  --boot-disk-size=50GB \
  --maintenance-policy=TERMINATE \
  --restart-on-failure \
  --zone=us-central1-a

Created [https://www.googleapis.com/compute/v1/projects/my-project-000/zones/us-central1-a/instances/my-gpu-vm].
NAME       ZONE           MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP    EXTERNAL_IP      STATUS
my-gpu-vm  us-central1-a  g2-standard-4               10.128.15.227  104.154.184.217  RUNNING

Once the instance is RUNNING, verify the NVIDIA driver and GPU are visible:

The nvidia-smi output confirms:

• Driver 590.48.01 loaded

• CUDA 13.1 available

• Persistence Mode is On

• The L4 GPU is detected with 23GB VRAM

• Zero ECC errors

• No running processes (clean idle state).

This is exactly what a healthy base image should look like. Notice Disp.A: Off? That confirms our compute-only driver choice is working — no display adapter is active.

Confirm the installed CUDA toolkit by running. nvcc --version. You can see that version 13.1 was installed as specified.

Let's confirm DCGM installation by running dcgmi discovery -l. Successful output indicates DCGM is running and communicating with the driver.

Conclusion

You now have a production‑grade, GPU‑optimized base image that includes the NVIDIA compute‑only driver built with open kernel modules, DCGM for monitoring, and the CUDA Toolkit. You also applied OS‑level tuning tailored to GPU compute workloads, providing a consistent, reproducible environment with no manual setup.

From here, you can extend the build by adding an application‑layer script to install frameworks such as PyTorch, TensorFlow, or vLLM, or create an instance template that uses this image to scale your GPU infrastructure.

The full Packer project includes additional scripts for training and inference workloads that you can use to extend your image.

References

• NVIDIA Driver Installation Guide (Ubuntu): https://docs.nvidia.com/datacenter/tesla/driver-installation-guide/

• NVIDIA CUDA Toolkit Documentation: https://docs.nvidia.com/cuda/

• NVIDIA Container Toolkit Installation Guide: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html

• NVIDIA DCGM Documentation: https://docs.nvidia.com/datacenter/dcgm/latest/index.html

• NVIDIA Persistence Daemon: https://docs.nvidia.com/deploy/driver-persistence/index.html

• HashiCorp Packer Documentation: https://developer.hashicorp.com/packer/docs

• Packer Google Compute Builder: https://developer.hashicorp.com/packer/integrations/hashicorp/googlecompute

NVIDIA's Blackwell architecture, succeeding Hopper in 2024, attacks this problem at the hardware level, rethinking not just how much memory a GPU has, but how it's structured and accessed entirely.

Running Llama 3 70B is no longer a concern – no parallelization or squeezing the model into tight memory limits. Instead, the same hardware footprint can now handle significantly larger parameter counts.

This article breaks down the memory enhancements that make Blackwell the most capable AI accelerator to date.

Prerequisites

This article assumes you're comfortable with a few GPU fundamentals. If any of these feel shaky, the linked resources will get you up to speed in 10–15 minutes each.

• GPU anatomy — what an SM is, and the role of registers, shared memory (L1), L2 cache, and memory controllers. [Memory Hierarchy of GPUs]

• The three memory metrics — capacity (how much fits), bandwidth (how fast data moves), and latency (how long a single access takes). These aren't interchangeable, and Blackwell improves all three differently. [GPU Memory Bandwidth]

• GPU memory types — HBM, GDDR, and LPDDR5X, and the bandwidth/capacity/power trade-offs between them. [Cuda GPU Memory Types]

• Chip interconnects — PCIe, NVLink, and the idea of a chip-to-chip (C2C) link. [The AI Systems Game]

If you're solid on all four, you're ready.

Table of Contents

• The Generational Leap

• The GB200 Superchip

  • Grace CPU

  • LPDDR5X (Low Power Double Data Rate 5x)

  • Blackwell GPU

  • High-Bandwidth Interface (NV-HBI)

  • NVLINK C-2-C (Chip-to-Chip)

• Memory Hierarchy and Bandwidth

  • The Hierarchy at a Glance

  • Registers and L1/Shared Memory

  • L2 Cache: Compensating for Smaller L1

  • HBM3e: The Main Memory Pool

  • LPDDR5X: The Extended Tier

  • Data Flow in Practice

  • Practical Example: Running Llama 3 70B

• Conclusion

  • References

The Generational Leap

Before diving into how Blackwell achieves its performance gains, here's what changed from the previous GPU generation:

The numbers tell a clear story: more memory, more bandwidth, larger caches. The rest of this article explains how these pieces fit together

The GB200 Superchip

The Grace Blackwell (GB200) extends the superchip design NVIDIA introduced with the Grace Hopper (GH200), where an ARM-based Grace CPU is paired with GPU chips in a single package to form one unified computing system.

In the Blackwell generation, the GB200 pairs one Grace CPU with two Blackwell GPUs, connected via NVLink Chip-to-Chip (NVLink-C2C), a high-bandwidth interface that lets the CPU and GPUs share memory and operate as a single system.

Grace CPU

The Grace CPU is an ARM Neoverse v2 designed by NVIDIA for bandwidth and efficiency. It handles general-purpose tasks, pre-processing, and tokenization, and feeds data to the GPU through NVLink C-2-C. The Grace CPU acts as extended storage for the GPU.

The Grace CPU runs at a moderate clock speed but compensates with a large memory bandwidth of up to 500GB/s to its LPDDR5X memory (Low Power Double Data Rate 5x – we'll discuss this more in a moment) with about 100MB of L3 Cache.

LPDDR5X (Low Power Double Data Rate 5x)

The LPDDR5X is a high-speed memory standard that delivers data up to 10.7 Gbps. The LPDDR5X offers low-power efficiency, making it ideal for this use case.

It strikes a perfect balance between performance and power efficiency, delivering up to 500 GB/s while using only about 16W, roughly one-fifth the power of conventional DDR5 memory.

Blackwell GPU

The Blackwell GPU made significant improvements over the previous Hopper GPU model, especially in terms of memory. The Blackwell GPUs are designed as dual-die GPUs, with two GPU dies in a single module.

Each die is connected by a super-fast NV-HBI (NVIDIA High-Bandwidth Interface) with a speed of 10TB/s, ensuring full performance. Each die contains 104 billion transistors, totaling 208 billion across the two dies. Each die also contains 96 GB of HBM3e memory, totaling 192 GB, with 180 GB usable (as 12 GB is used for error-correcting code (ECC), system firmware, and so on).

With this amount of memory, the Blackwell GPU's memory bandwidth is about 2.4 times faster than that of the Hopper generation.

The L2 cache was also increased to 126 MB. By increasing the L2 cache, Blackwell can store more neural network weights or intermediate results on-chip, avoiding extra trips out to HBM. This ensures the GPU’s compute units are rarely starved for data.

High-Bandwidth Interface (NV-HBI)

High Bandwidth Interconnect is a standard for die-to-die (or d2d) communication. The NVIDIA High-Bandwidth Interface (NV-HBI) offers a 10TB/s connection, combining the two GPU dies into a single, unified GPU.

NVLINK C-2-C (Chip-to-Chip)

The NVLink C-2-C provides a communication speed of up to ~900 GB/s between the Grace CPU and the Blackwell GPUs, eliminating the need to copy memory from the CPU to the GPU memory pool via the PCIe bus.

The NVLink C-2-C interconnect speed is faster than the typical PCIe bus. PCIe Gen6 is only about 128 GB/s per direction compared to the NVLink C-2-C's speed. It's also cache-coherent, meaning both the CPU and GPU share a coherent memory architecture, allowing the CPU to read and write to GPU memory and vice versa.

This unified memory architecture is called Unified CPU-GPU Memory or Extended GPU Memory (EGM) by NVIDIA.

Memory Hierarchy and Bandwidth

Understanding how data flows through Blackwell's memory system is key to optimizing AI workloads. The architecture follows a classic hierarchy principle: smaller, faster memory sits closest to the compute units, with progressively larger but slower memory tiers extending outward.

The Hierarchy at a Glance

Registers and L1/Shared Memory

A streaming multiprocessor (SM) executes compute instructions on the GPU. At the lowest level, each Streaming Multiprocessor (SM) contains a register file and configurable L1/Shared memory as illustrated in the diagram above. Registers hold the operands for active computations, that is, data that the GPU cores are working on right now.

An SM executes threads in fixed-size groups known as warps, with each warp containing exactly 32 threads that execute the same instructions in lockstep. The L1/Shared memory acts as a staging area, allowing threads within an SM to share data without going to slower memory tiers.

Blackwell's L1/Shared memory is 128 KB per SM by default, a reduction from Hopper's 256 KB. In specific configurations, this can extend to 228 KB per SM. The aggregate bandwidth across all SMs is approximately 40 TB/s.

Why the reduction? NVIDIA shifted capacity to TMEM for Tensor Core operations and compensated with a larger L2 cache. General-purpose shared memory workloads see less per-SM capacity, but the workloads that matter most, matrix multiplications, get dedicated, faster memory.

L2 Cache: Compensating for Smaller L1

The L2 cache sits between the SMs and HBM, shared across all compute units on a die. Blackwell provides 64-65 MB per die (roughly 126 MB total across the dual-die module). This represents a 2.5× increase over Hopper's 50 MB and compensates for the smaller per-SM L1.

In AI workloads, the same model weights are accessed repeatedly across different input batches. A larger L2 cache means more of these weights can stay on-chip between batches, reducing expensive trips to HBM. For inference serving, where the same model handles thousands of requests, this translates directly to lower latency and higher throughput.

The dual-die design does introduce complexity here. Each die has its own 63 MB L2 partition. Accessing data cached on the other die requires crossing the NV-HBI interconnect fast at 10 TB/s, but still slower than local L2 access. NVIDIA's software stack handles this transparently, but performance-conscious engineers should be aware that data placement across dies can affect cache efficiency.

HBM3e: The Main Memory Pool

High Bandwidth Memory (HBM3e) serves as the primary storage for model weights, activations, gradients, and input data. Blackwell's HBM3e delivers 8 TB/s of bandwidth per GPU, roughly 2.4× faster than Hopper's 3.35 TB/s HBM3.

The physical implementation uses an 8-Hi stack design: eight DRAM dies stacked vertically, each providing 3 GB, for 24 GB per stack. With eight stacks total (four per die), the B200 GPU provides 192 GB of on-package memory, though 180 GB is usable after accounting for ECC and system overhead.

This bandwidth increase is critical. Tensor Core operations can consume data at enormous rates. If HBM can't feed data fast enough, the compute units stall, leaving expensive silicon idle. Blackwell's 8 TB/s keeps the tensor cores fed even during the largest matrix multiplications.

LPDDR5X: The Extended Tier

Beyond the GPU's HBM sits the Grace CPU's LPDDR5X memory, approximately 480 GB accessible at up to 500 GB/s locally, or ~900 GB/s when accessed from the GPU via NVLink C-2-C.

Accessing LPDDR5X from the GPU has roughly 10× lower bandwidth and higher latency compared to HBM. But it remains far faster than NVMe SSDs or network storage.

LPDDR5X serves as a high-speed overflow tier. Data that doesn't fit in HBM, such as large embedding tables, KV caches for long-context inference, or checkpoint buffers, can reside in CPU memory without catastrophic performance penalties.

Data Flow in Practice

When a Blackwell GPU executes an AI workload, data flows through this hierarchy in stages:

1. Model loading: Weights move from storage → CPU memory → HBM (or stay in LPDDR5X if HBM is full)

2. Batch processing: Input data streams into HBM, then into L2 as SMs request it

3. Computation: Active data moves from L2 → L1/Shared → Registers as operations execute

4. Output: Results flow back down the hierarchy to HBM or CPU memory

Each tier serves as a buffer for the tier above it.

Practical Example: Running Llama 3 70B

Consider deploying Llama 3 70B for inference. In FP16 precision (Note with GB200, you can go as low as FP4), the model weights alone require approximately 140 GB of memory.

On a Hopper H100 (80 GB HBM3): The model doesn't fit. You must either quantize aggressively, use tensor parallelism across multiple GPUs, or offload layers to CPU memory over PCIe (slow at ~64 GB/s).

On a single GB200 Superchip (~360 GB usable HBM3e + ~480 GB LPDDR5X): The full 140 GB model fits easily within a single GPU's HBM, leaving the second GPU's HBM and all CPU memory available for KV cache, batching, or running multiple model instances. No model parallelism required. No aggressive quantization forced by memory limits. The GB200 Superchip provides roughly 10× the usable memory of a single H100, fundamentally changing what fits on one unit

This is the practical impact of Blackwell's memory architecture: models that previously required multi-GPU setups can now run on a single superchip, simplifying deployment and reducing inter-GPU communication overhead.

Conclusion

Memory has always been the limiting factor in AI hardware. Blackwell changes that equation.

By combining dual-die GPUs, HBM3e with 8 TB/s bandwidth, and unified CPU-GPU memory through NVLink C2C, NVIDIA has delivered a system where a single superchip offers roughly 10× the usable memory of its predecessor. Models that once demanded complex multi-GPU orchestration now fit on one unit.

For AI engineers, this means spending less time working around memory constraints and more time building better models. The architecture isn't just faster, it's fundamentally simpler to work with.

As models continue to grow, Blackwell's memory-first design philosophy points to where GPU architecture is heading: tighter integration, unified memory pools, and specialized hardware for the workloads that matter most.

References

1. NVIDIA Blackwell Architecture Technical Brief: https://resources.nvidia.com/en-us-blackwell-architecture

2. NVIDIA Blackwell Architecture: A Deep Dive: https://medium.com/@kvnagesh/nvidia-blackwell-architecture-a-deep-dive-into-the-next-generation-of-ai-computing-79c2b1ce3c1b

3. AI Systems Performance Engineering: https://learning.oreilly.com/library/view/ai-systems-performance/9798341627772/

4. Memory Hierarchy of GPUs**:** https://www.arccompute.io/arc-blog/gpu-101-memory-hierarchy

5. GPU Memory Bandwidth and Its Impact on Performance: https://www.digitalocean.com/community/tutorials/gpu-memory-bandwidth

6. The AI Systems Game: https://medium.com/@adi.fu7/the-ai-systems-game-are-chip-to-chip-interconnects-the-future-of-inference-ec3bbda53eb3

7. CUDA GPU Memory Types: https://medium.com/@jghaly00/cuda-gpu-memory-types-a07428b3eb16

If you’re new to Machine Learning and are just getting started, then a free Kaggle or Colab might be enough for you. But that won’t be the case when you want to go deeper. You’ll need a GPU, which can get costly if you’re continuously using it on the cloud.

But there’s some good news: you can utilize your computer’s Nvidia GPU (GTX/RTX) quite easily and perform machine learning-related tasks right on your local machine. The cool thing is, it won’t cost you anything other than the electricity it uses!

When you’re running Machine Learning models on your local machines, the most suitable operating system is a Linux-based one, like Ubuntu. But Windows has improved a lot for this purpose. If you’re using the latest Windows 11, you can leverage Windows Subsystem for Linux (WSL) and use your GPU directly for Machine Learning-related workflows.

This process can be quite tricky, though, as can making two popular Machine Learning frameworks, TensorFlow and PyTorch, compatible with your system GPU in Windows 11. That’s why I have written this comprehensive guide to ease your pain.

In it, I’ll help you set up CUDA on Windows Subsystem for Linux 2 (WSL2) so you can leverage your Nvidia GPU for machine learning tasks.

By following these steps, you’ll be able to run ML frameworks like TensorFlow and PyTorch with GPU acceleration on Windows 11.

Keep in mind that this guide assumes you have a compatible Nvidia GPU. Make sure to check Nvidia's official compatibility list before proceeding.

I have also prepared a video for you that’ll help you follow proper guidelines throughout this article.

Also, if this tutorial helps you, then don’t forget to add a star to the GitHub repository CUDA-WSL2-Ubuntu-v2. If you face any issues or have any suggestions/improvements, then please raise an issue in the GitHub repository. Currently, the live website is available at ml-win11-v2.fahimbinamin.com.

Table of Contents

1. Prerequisites

2. Windows Terminal

3. Windows PowerShell (Latest & Greatest)

4. Configure Windows Terminal

5. Configuration of my computer

6. CPU Virtualization

7. Install WSL2

8. Install Latest LTS Ubuntu via WSL2

9. Update & Upgrade Ubuntu Packages

10. Install and Configure Miniconda

11. Install Jupyter & Ipykernel

12. Nvidia Driver

13. Install CUDA dependencies

14. CUDA Toolkit

15. Add Path to Shell Profile for CUDA

16. nvcc Version

17. cuDNN SDK

18. TensorFlow GPU

    • Check TensorFlow GPU

19. PyTorch GPU

    • Check PyTorch GPU

    • Check PyTorch & TensorFlow GPU inside Jupyter Notebook

20. Conclusion

Prerequisites

Before you begin, make sure you have the following requirements met:

• Windows 11 operating system

• Nvidia GPU (GTX/RTX series)

• Administrator access to your PC

• At least 30 GB of free disk space

• Internet connection for downloads

• Latest Nvidia drivers installed

Windows Terminal

First, you’ll need to ensure that you have Windows Terminal installed properly in your operating system. It is the newest terminal application for users of command-line tools and shells like Command Prompt, PowerShell, and WSL. You can download it from the Microsoft Store.

After ensuring that it’s installed properly, you can proceed to the next steps.

Windows PowerShell (Latest & Greatest)

Windows PowerShell is a modern and updated command-line shell from Microsoft. You can use some Linux specific commands directly on it. It comes with built-in command suggestions. You can download it from the official GitHub page.

Download the latest x64 installer and install it. After ensuring that it is installed properly, you can proceed to the next steps.

Configure Windows Terminal

Now you’ll need to configure your Windows Terminal to use PowerShell as the default shell. It’s optional and you might skip this step. But I recommend doing it for a better experience.

Open Windows Terminal. Click on the down arrow icon in the title bar and select "Settings".

In the Settings tab, under "Startup", find the "Default profile" dropdown menu. Select "PowerShell" from the list.

Now for the "Default terminal application", select "Windows Terminal".

By default, Windows PowerShell always shows the version number in the title bar. If you want to disable it, select the "PowerShell" profile from the left sidebar. Click on the "Command Line" field and add an --nologo argument at the end of the command. After this, the line becomes "C:\Program Files\PowerShell\7\pwsh.exe" --nologo.

If you don’t use other shells frequently and want to hide them in the dropdown, then you’ll need to select those profiles one by one from the left sidebar. Scroll down to the bottom and find the "Hide profile from dropdown" toggle and enable it. It will hide that specific shell from the dropdown menu.

For example, I am hiding the Azure Cloud Shell profile as I don't use it frequently:

Now click on the "Save" button at the bottom right corner to apply the changes. Close the Windows Terminal for now.

Configuration of My Computer

I figured it’d be helpful to share my current computer’s configuration so you can have a clear idea of which setup I’m using in this guide. Here are the details:

Now that you have an idea about my computer’s configuration, we can proceed to the next steps.

CPU Virtualization

As we are going to use WSL2, we’ll need to make sure that the CPU virtualization is enabled. To check whether virtualization is enabled or not from Windows, simply open the Windows Task Manager. Go to the Performance tab and select CPU from the left sidebar. In the bottom right corner, you will see the Virtualization status. If it shows "Enabled", then you are good to go. If it shows "Disabled", then you need to enable it from the BIOS.

⚠️ You have to ensure that CPU Virtualization is enabled in your BIOS settings. Different manufacturers have different ways to access the BIOS. Usually, you can access the BIOS by pressing the Delete or F2 key during the boot process. Once in BIOS, look for settings related to "Virtualization Technology" or "Intel VT-x"/"AMD-V" and make sure it is enabled. Save the changes and exit the BIOS.

Install WSL2

Open the Windows Terminal or Windows PowerShell as an administrator. Run the following command to install WSL2 along with the latest Ubuntu LTS distribution:

wsl.exe --install

It will install Windows Subsystem for Linux 2 (WSL2). After the installation is complete, you will be prompted to restart your computer. Do so to finalize the installation.

⚠️ If you encounter any issues during installation, refer to the official Microsoft documentation for troubleshooting WSL installation problems.

Install Latest LTS Ubuntu via WSL2

Open the Windows Terminal or Windows PowerShell again with the administrator privileges. If you want to check the available Linux distributions to install via WSL, run the following command:

wsl.exe --list --online

For installing any specific distribution, run the following command:

wsl.exe --install <DistroName>

We are going to install the latest LTS Ubuntu distribution. As of now, the latest LTS version is Ubuntu 24.04. But I prefer to install the Ubuntu directly as it always points to the latest LTS version. So, run the following command:

wsl.exe --install Ubuntu

You need to give it a default user account name. For me, I am going with fahim.

It also comes with a nice GUI management tool for WSL.

You can configure a lot of stuff in it including restricting core, RAM, disk space and a lot of specifications from the settings GUI window.

Update & Upgrade Ubuntu Packages

Open your Ubuntu terminal from Windows Terminal. First, we need to update and upgrade the existing packages to their latest versions.

To update the Ubuntu system, simply use the following command:

sudo apt update -y

To upgrade all the packages at once, simply use the following command:

sudo apt upgrade -y

⚠️ Make sure that you have a stable internet connection during the update and upgrade process to avoid any interruptions.

Install and Configure Miniconda

In Machine Learning, we need to manage multiple environments with different package versions. Conda is a popular package and environment management system that makes it easy to create and manage isolated environments for different projects. We will install Miniconda, a minimal installer for Conda, to manage our Python environments. But if you prefer Anaconda, you can install it instead.

Go to the official website of Miniconda. Currently the Miniconda installer is inside Anaconda here. If the official website gets updated, you can always search for "Miniconda installer" on Google to find the latest version. Also, you can create an issue in the official GitHub repository of this project to notify me about it.

As we are installing it inside WSL, we have to select the macOS/Linux Installation. Then select Linux Terminal Installer and choose Linux x86 for downloading the installer.

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

It will download the installer to your WSL directory. Then use the following command to install it properly:

bash ~/Miniconda3-latest-Linux-x86_64.sh

⚠️ Make sure that you are in the correct directory where the installer is downloaded. If you downloaded it to a different location, adjust the path accordingly. Also, replace bash with zsh or sh if you are using a different shell.

Make sure to choose the initialization option properly. I prefer to keep the conda env active whenever I open a new shell. Therefore, I chose "Yes".

Make sure that the installation succeeds without any errors.

For the changes to take effect, you can close and reopen the current shell. But you can also do that without closing and reopening the shell by applying the command below.

source ~/.bashrc

⚠️ If you’re using a different shell like zsh or fish, make sure to source the appropriate configuration file (e.g., ~/.zshrc for zsh).

Install Jupyter & Ipykernel

I prefer to use Jupyter Notebook for running my machine learning experiments. It provides an interactive environment for coding and data analysis. We’ll install Jupyter Notebook and Ipykernel to run Jupyter notebooks in our conda environment. We will do that in all conda environments starting with the base environment. It also helps us to keep the conda environment kernel inside Jupyter Notebook.

First, make sure that you are in the base conda environment. You will see (base) on the left side of the terminal.

Now install Jupyter and Ipykernel both by applying the following command:

conda install jupyter ipykernel -y

Make sure that you accept the terms of service of Conda.

Now, I will create a separate conda environment for both TensorFlow and the PyTorch GPU. You can directly install them in the base environment or in any other environment as per your preference. I am not specifying any specific Python version while creating the environment. It will automatically install the latest stable version of Python.

conda create -name ml -y

To activate any specific conda environment, you have to use the following command:

conda activate <conda-env-name>

For example, if I want to activate my newly created ml environment, I will use this command:

conda activate ml

If you’re not sure which conda environments are installed in your system, you can check all available and installed conda environments in your system by running the following command:

conda env list

Nvidia Driver

Ensure that you have the latest Nvidia drivers installed on Windows. WSL2 uses the Windows driver, so no separate driver installation is needed in Ubuntu. You can download the latest drivers from the official Nvidia website.

If you are just installing the latest GPU driver, then after installing the drivers, restart your computer to ensure the changes take effect. You can either use the GeForce Game Ready Driver or the NVIDIA Studio Driver. But I recommend using the Studio Driver for better stability with creative and ML applications.

Install CUDA Dependencies

You might face some issues if you do not have the CUDA dependencies installed properly. I recommend that you install the required dependencies before proceeding further:

sudo apt install gcc g++ build-essential

After installing the dependencies, you can then verify the CUDA installation if you had any issues earlier.

CUDA Toolkit

TensorFlow GPU is very picky about the CUDA version. So we need to install a specific version of CUDA Toolkit that is compatible with the TensorFlow version we are going to install.

To understand exactly which CUDA version is compatible with which TensorFlow version, you can check the official TensorFlow GPU support matrix here.

At the time I’m writing this article, the TensorFlow GPU documentation says that we should have CUDA Toolkit 12.3. So I will ensure that I install exactly that version. You can simply click on that version link in the official docs and it will redirect you to the official Nvidia CUDA Toolkit download page. But if the link gets updated in the future, you can always search for "Nvidia CUDA Toolkit" on Google to find the latest version.

As TensorFlow GPU is asking for exact Version 12.3, I will select version 12.3.0 exactly.

In the CUDA Toolkit download page, make sure to choose the operating system as Linux, Architecture as x86_64, Distribution as WSL-Ubuntu, Version as 2.0 and the Installer type as runfile(local).

⚠️ As we are using Ubuntu in our WSL2, you can also choose Ubuntu as your operating system. But I prefer to choose WSL-Ubuntu for better compatibility.

After selecting those, it will give you the download commands. You have to apply them sequentially. Make sure that you don't keep the checkmark in "Kernel Objects" during installing CUDA.

⚠️ Make sure to copy and paste the commands one by one in your WSL Ubuntu terminal to download and install the CUDA Toolkit properly. If you face any issues related to CUDA dependency, then quickly go through the Install CUDA dependencies section, where I have explained how to install the CUDA dependencies properly.

Add Path to Shell Profile for CUDA

After installing CUDA Toolkit, we need to add the CUDA binaries to our shell profile for easy access. This will allow us to run CUDA commands from any directory in the terminal.

Note that, depending on the shell you are using (bash, zsh, and so on), you need to add the CUDA path to the appropriate configuration file. Make sure to replace .bashrc with .zshrc or other configuration files if you are using a different shell.

To add the CUDA binary path, follow the command below:

echo 'export PATH=/usr/local/cuda-12.3/bin:$PATH' >> ~/.bashrc

You have to use the updated path where you installed it. Your terminal will show it after installing the CUDA:

Now, you need to add the path inside the Library path. Just use the exact path where you installed CUDA. Your terminal will list the path properly.

echo 'export LD_LIBRARY_PATH=/usr/local/cuda-12.3/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc

After adding those paths, you need to source the shell profile for the changes to take effect. You can do that by running the following command:

source ~/.bashrc

nvcc Version

NVCC stands for Nvidia CUDA Compiler. It is basically a compiler driver for the CUDA platform that allows developers to write parallel programs to run on Nvidia GPUs. As we have already installed the CUDA toolkit, we need to see whether the compiler is also properly activated. To check that, we need to verify the version.

Verify that CUDA is properly installed by checking the version:

nvcc --version

If the output shows the correct CUDA version, then you have successfully installed CUDA Toolkit in your WSL2 Ubuntu environment.

cuDNN SDK

The cuDNN (CUDA Deep Neural Network) SDK is a GPU accelerated library of primitives for deep neural networks, developed by Nvidia. It provides highly optimized building blocks for common deep learning operations, significantly speeding up the training and inference processes of AI models on Nvidia GPUs.

Note: Even though TensorFlow GPU suggests a specific cuDNN version, it’s often compatible with multiple versions. Because of this, I recommend downloading the latest cuDNN version that is compatible with your installed CUDA version. You can find the cuDNN download page here.

Select the Operating System as Linux, Architecture as x86_64, Distribution as Ubuntu, Version as 24.04, Installer Type as deb (local), Configuration as FULL. After selecting those, it will give you the download commands. You have to apply them sequentially.

⚠️ Make sure to copy and paste the commands one by one in your WSL Ubuntu terminal to download and install the cuDNN SDK properly. If you face any issues related to CUDA dependency, then quickly go through the Install CUDA dependencies section, where I have explained how to install the CUDA dependencies properly.

TensorFlow GPU

Now, we are going to install TensorFlow GPU in our conda environment. Make sure that you have activated the conda environment where you want to install it. I’m going to install it in my previously created ml environment. To activate it, I’ll use the following command:

conda activate ml

⚠️ Make sure that you have activated the correct conda environment before installing TensorFlow GPU. You will see the environment name in the terminal prompt.

I will install ipykernel and jupyter in this new environment.

conda install jupyter ipykernel -y

Now, to install TensorFlow GPU, I will simply use the following command:

pip install tensorflow[and-cuda]

It might take a couple of minutes depending on the internet speed you have. Just have patience and wait for it to finish the installation.

Check TensorFlow GPU

After installing TensorFlow GPU, we need to verify that it is working properly with GPU support. Open a Python shell in your Ubuntu terminal and run the following commands:

python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"

If the output shows a list of available GPU devices, then TensorFlow GPU is successfully installed and working properly.

PyTorch GPU

Now, we’re going to install PyTorch GPU in our conda environment. Make sure that you have activated the conda environment where you want to install it. I’m going to install it in my previously created ml environment. To activate it, I will use the following command:

conda activate ml

Installing PyTorch GPU is very straightforward. You can use the official PyTorch installation command generator here.

Make sure to select PyTorch Build as the latest Stable one, Your OS as Linux, Package as Pip, Language as Python. For the Compute Platform, select the CUDA version that matches your installed CUDA Toolkit. For me, it is CUDA 12.3. But, if you can not find the exact one then choose the closest. As CUDA 12.3 is not available for me now, I am choosing CUDA 12.6.

After selecting those, it will give you the installation command. You have to apply it in your WSL Ubuntu terminal.

It might take a couple of minutes depending on the internet speed you have. Just have patience and wait for it to finish the installation.

Check PyTorch GPU

After installing PyTorch GPU, verify that it is working properly with GPU support. Open a Python shell in your Ubuntu terminal and run the following commands:

python3 - << 'EOF'
import torch
print(torch.cuda.is_available())
print(torch.cuda.device_count())
print(torch.cuda.current_device())
print(torch.cuda.device(0))
print(torch.cuda.get_device_name(0))
EOF

The output should look similar to the screenshot, showing:

• True: GPU is available for PyTorch

• 1: Number of detected CUDA devices

• 0: Index of the current active CUDA device

• A device object representation

• NVIDIA GeForce RTX 3060 (or your GPU name)

Check PyTorch & TensorFlow GPU inside Jupyter Notebook

Now that the environment is fully configured, we will verify GPU support directly inside Jupyter Notebook. This ensures both PyTorch and TensorFlow can successfully detect and use your GPU.

1. Test PyTorch GPU

Create a new Jupyter Notebook and run the following commands one by one:

import torch

print(torch.cuda.is_available())
print(torch.cuda.device_count())
print(torch.cuda.current_device())
print(torch.cuda.device(0))
print(torch.cuda.get_device_name(0))

If everything is configured correctly, you will see your GPU (for example NVIDIA GeForce RTX 3060) detected properly:

2. Test TensorFlow GPU

Next, run the following code to check whether TensorFlow detects your GPU:

import tensorflow as tf

print(tf.config.list_physical_devices('GPU'))

You can also check the number of GPUs detected:

print("Num GPUs Available:", len(tf.config.list_physical_devices('GPU')))

Finally, run TensorFlow’s built-in GPU validation (warnings are normal):

import tensorflow as tf

assert tf.test.is_gpu_available()
assert tf.test.is_built_with_cuda()

If TensorFlow logs show your GPU model (such as RTX 3060), then TensorFlow GPU is successfully installed and fully working inside Jupyter Notebook.

Conclusion

Thank you so much for reading all the way through. I hope you have been able to configure your Windows 11 computer properly for running almost any kind of Machine Learning-based experiments.

To get more content like this, you can follow me on LinkedIn and X. You can also check my website and follow me on GitHub if you are into open source and development.

What most don’t realize is that the real engine behind this AI revolution started in a place few expected: inside the humble gaming PC.

The same graphics cards once built to render smooth 3D visuals are now powering chatbots, image generators, and self-driving systems. The journey from pixels to predictions is one of the most fascinating stories in modern computing.

The CPU Era and Its Limits

In the early days of machine learning, researchers depended on CPUs to crunch data.

CPUs were versatile and great for handling a wide range of tasks, but they had one big limitation: they worked on problems in sequence.

That means they could process only a few operations at a time. For small models, this was fine. But as neural networks grew in complexity, training them on CPUs became painfully slow.

Imagine trying to teach a computer to recognize images. A neural network might have millions of parameters, and every single one needs to be adjusted again and again during training.

On CPUs, that could take days or even weeks. Researchers quickly realized that if AI was going to advance, it needed a completely different kind of hardware.

How GPUs Entered the Picture

Graphics processing units, or GPUs, were originally built to render the fast-moving images in video games. They were designed for parallelism, performing thousands of small calculations at the same time.

While a CPU might have a handful of cores, a GPU has thousands. This architecture made GPUs ideal for the kind of math used in machine learning, where the same operation needs to be applied to huge amounts of data simultaneously.

In a way, the GPU was built for games but destined for AI. What started as a chip to make lighting effects smoother and explosions look more realistic soon found a second life powering neural networks.

Around the early 2010s, researchers began experimenting with running deep learning algorithms on GPUs, and the results were stunning. Training times dropped from weeks to days, and accuracy improved.

It was a quiet revolution happening in research labs around the world.

The Role of Gaming PCs in Early AI Research

Here’s where the story gets even more interesting: many of the early breakthroughs in AI didn’t come from massive data centres or expensive supercomputers. They came from researchers using consumer-grade GPUs, often sitting inside regular gaming PCs.

These machines, built for entertainment, turned out to be powerful enough for deep learning experiments.

NVIDIA’s CUDA platform made this possible by allowing developers to program GPUs for tasks beyond graphics. Suddenly, a gaming GPU could handle complex scientific computations.

Researchers used their own rigs, sometimes the same computers they played games on at night, to train neural networks that recognized speech, images, and text. The gaming PC became a testbed for the future of artificial intelligence.

The Turning Point: AlexNet and the Deep Learning Boom

In 2012, a neural network called AlexNet stunned the world by winning the ImageNet competition, a major benchmark in computer vision.

What made AlexNet special wasn’t just its architecture but the hardware behind it. It ran on two NVIDIA GTX 580 GPUs, hardware you could buy for your low-cost gaming PC. That win marked a turning point. It proved that GPUs weren’t just for rendering graphics – they were the key to advancing AI.

After that, the AI world changed fast. Every major research lab and tech company started building GPU clusters. NVIDIA, sensing the opportunity, leaned into AI hardware development.

The same company that once catered mainly to gamers now powered Google, OpenAI, and Tesla. What started as a tool for better visuals had become the backbone of machine intelligence.

Why GPUs Are So Good at AI

GPUs excel at matrix math, the kind of computation that neural networks rely on.

When you train a model, you’re constantly multiplying and adding matrices of numbers. GPUs do this faster because they handle thousands of operations in parallel. They’re also designed with high memory bandwidth, meaning they can move large amounts of data in and out quickly.

This architecture fits perfectly with deep learning workloads. Whether it’s image recognition or language translation, GPUs can process huge batches of data at once.

CPUs, by contrast, get bottlenecked by sequential processing. The difference in performance is like comparing a single craftsman building a house to a team of thousands working at once.

The AI Hardware Race

As AI took off, the demand for GPUs exploded. What began in gaming PCs scaled up into massive data centers filled with thousands of cards.

Companies like NVIDIA developed new lines of GPUs specifically for AI, such as the Tesla and A100 series. Other players joined the race too, like AMD with its ROCm platform, and Google with its custom TPUs (Tensor Processing Units).

Yet, even today, the line between gaming and AI hardware remains blurred. The same RTX GPUs designed for gamers are still used by many AI researchers and small startups.

A powerful gaming PC equipped with a modern GPU can run local AI models, generate images, or even fine-tune small language models. The hardware that made virtual worlds come alive now brings intelligence to our real one.

The Future of GPUs and AI

As AI models grow larger, new challenges are emerging. GPUs are evolving to handle trillion-parameter models, but they’re also getting smarter about energy use and efficiency.

Technologies like chiplet design, optical interconnects, and AI-specific cores are pushing performance further while keeping costs down.

Meanwhile, local AI is making a comeback. With advancements in GPU efficiency, many users are experimenting with running models on their own machines.

A well-equipped gaming PC can now do what once required access to a cloud GPU cluster. This shift could democratize AI development, letting anyone with the right hardware explore the field from home.

Conclusion

The GPU’s journey from gaming to AI is one of the most unexpected transformations in tech history. What started as a chip to render virtual landscapes evolved into the heart of artificial intelligence. From early experiments on gaming PCs to the data centers powering today’s largest models, GPUs have bridged the worlds of creativity, computation, and cognition.

As we look ahead, it’s clear that the same technology that once made games more realistic is now making machines more intelligent. The story of the GPU reminds us that innovation often comes from unexpected places, and sometimes, the future of AI begins in the glow of a gaming screen.

Hope you enjoyed this article. Find me on Linkedin or visit my website.