I did. I spun up an EC2 instance, followed along for an hour, and then forgot to shut it down. A week later I had a $34 bill for a machine running nothing.

That was the last time I practiced on someone else's infrastructure.

Everything in this guide runs on your laptop. No cloud account, no credit card, no bill at the end of the month. By the end, you'll be able to spin up a multi-server environment from scratch, configure it automatically with Ansible, serve a site you wrote yourself, and diagnose what breaks when you intentionally destroy it.

That last part is where the actual learning happens.

Prerequisites

Before you start, make sure you have:

• A laptop with at least 8GB of RAM (16GB is better)

• At least 20GB of free disk space

• Windows, macOS, or Linux operating system

• Administrator access to your computer

• Virtualization enabled in your BIOS/UEFI settings

• A stable internet connection for the initial downloads

Knowledge and comfort level:

• You should be comfortable using a terminal (running commands, changing directories, and editing small text files with whatever editor you like).

• Basic familiarity with concepts like “a server,” “SSH,” and “a port” helps, but you don't need prior experience with Docker, Kubernetes, Vagrant, or Ansible. This guide introduces them as you go.

If you can follow step-by-step instructions and read error output without panicking, you're ready.

Table of Contents

1. What is DevOps?

2. Why Build a Local Lab?

3. How to Set Up Docker

4. How to Set Up Kubernetes

5. How to Install kubectl

6. How to Set Up Vagrant

7. How to Install Ansible

8. How to Build Your First DevOps Project

9. How to Break Your Lab on Purpose

10. What You Can Now Do

What is DevOps?

DevOps is the practice of breaking down the wall between software development and IT operations teams.

Traditionally, developers write code and hand it off to operations teams to deploy and maintain. That handoff causes delays, misunderstandings, and outages. DevOps is what happens when both teams work together from the start.

The tools you'll install in this guide each solve a specific part of that process:

• Docker packages your application and everything it needs into a portable container that runs the same way on any machine.

• Kubernetes manages multiple containers at scale, handling restarts, networking, and load balancing automatically.

• Vagrant creates and manages virtual machine environments so your whole team always works on identical setups.

• Ansible automates repetitive configuration tasks across many servers without writing a script for each one.

Why Build a Local Lab?

A local lab gives you a safe place to break things, fix them, and learn from that process without any cost or risk.

Here's what you get with a local setup:

• Zero cost. No cloud bills, no surprise charges, and no credit card required.

• Works offline. Practice anywhere, even without internet after the initial setup.

• Full control. You manage every layer from the OS up to the application.

• Safe experimentation. Break things freely. Nothing here affects production.

• Fast feedback. No waiting for cloud resources to spin up. Everything runs on your machine.

The tradeoff is resource limits. Your laptop's CPU and RAM are the ceiling. You can't simulate large-scale deployments, and some cloud-native services like AWS Lambda or S3 have no direct local equivalent. But for learning core DevOps workflows, none of that matters.

How to Set Up Docker

Docker is the foundation of this lab. Every other tool in this guide either runs inside Docker containers or works alongside them.

How to Install Docker on Windows

First, enable virtualization in your BIOS:

1. Restart your computer and enter BIOS/UEFI setup. The key is usually F2, F10, Del, or Esc during boot.

2. Find the virtualization setting. It's usually listed as Intel VT-x, AMD-V, SVM, or Virtualization Technology.

3. Enable it, save your changes, and exit.

Then install Docker Desktop:

1. Download Docker Desktop from Docker's official website.

2. Run the installer and follow the prompts.

3. Enable WSL 2 (Windows Subsystem for Linux) when asked.

4. Restart your computer.

5. Open Docker Desktop from the Start menu and wait for the whale icon in the taskbar to stop animating.

Troubleshooting: If Docker fails to start, run this in PowerShell as Administrator to verify virtualization is active:

systeminfo | findstr "Hyper-V Requirements"

All items should show "Yes". If they don't, revisit your BIOS settings.

How to Install Docker on Mac

1. Download Docker Desktop for Mac from Docker's website.

2. Open the downloaded .dmg file and drag Docker to your Applications folder.

3. Open Docker from Applications.

4. Enter your password when prompted.

5. Wait for the whale icon in the menu bar to stop animating.

How to Install Docker on Linux

Run these commands in order:

# Update your package lists
sudo apt-get update

# Install prerequisites
sudo apt-get install apt-transport-https ca-certificates curl software-properties-common

# Add Docker's official GPG key
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

# Add the Docker repository
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

# Update and install Docker
sudo apt-get update
sudo apt-get install docker-ce

# Start and enable Docker
sudo systemctl start docker
sudo systemctl enable docker

# Add your user to the docker group
sudo usermod -aG docker $USER

Log out and back in for the group change to take effect.

How to Test Docker

Run this command:

docker run hello-world

If you see "Hello from Docker!" then Docker is working correctly.

Docker is set up. Next, you'll install Kubernetes to manage containers at scale.

How to Set Up Kubernetes

Kubernetes manages containers at scale. For a local lab, you have four options. Here's how to choose:

If you're just starting out, use Minikube. It has the simplest setup and a visual dashboard that helps you understand what's happening inside the cluster.

If your laptop has 8GB RAM or less, use k3s. It runs lean and behaves closer to a real cluster than Minikube does.

Use kubeadm only if you want to understand how Kubernetes nodes join a cluster — it requires more manual steps and isn't beginner-friendly.

How to Install Minikube (Recommended for Beginners)

Minikube creates a single-node Kubernetes cluster on your laptop.

On Windows:

1. Download the Minikube installer from Minikube's GitHub releases page.

2. Run the .exe installer.

3. Open Command Prompt as Administrator and start Minikube:

minikube start --driver=docker

On Mac:

brew install minikube
minikube start --driver=docker

On Linux:

curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
chmod +x minikube-linux-amd64
sudo mv minikube-linux-amd64 /usr/local/bin/minikube
minikube start --driver=docker

Test your cluster:

minikube status
minikube dashboard

How to Install k3s (Recommended for Low-RAM Machines)

k3s is a lightweight version of Kubernetes that installs in under a minute. It runs lean and behaves like a real cluster — not a simplified demo version.

On Linux (and Mac via Multipass):

curl -sfL https://get.k3s.io | sh -

That single command installs k3s and runs it automatically in the background. Check that it is running:

sudo k3s kubectl get nodes

You should see one node with status Ready.

On Mac directly — k3s doesn't run natively on macOS. Use Multipass to spin up a lightweight Ubuntu VM first, then run the install command inside it.

On Windows — use WSL2 (Ubuntu), then run the install command inside your WSL2 terminal.

How to Install Kind (Kubernetes IN Docker)

Kind runs a full Kubernetes cluster inside Docker containers. It starts faster than Minikube and is useful if you want to run multiple clusters simultaneously.

# Mac or Linux
brew install kind

# Windows
choco install kind

Create a cluster:

kind create cluster --name my-local-lab

How to Install kubeadm (For Understanding Cluster Bootstrap)

kubeadm is the tool Kubernetes uses to initialize and join nodes in a real cluster. Use this when you want to understand what happens under the hood — not as your daily driver.

It requires at least two machines (or VMs). The setup is more involved than the options above. Follow the official kubeadm installation guide for your OS, then initialize your cluster:

sudo kubeadm init --pod-network-cidr=10.244.0.0/16

After init, join worker nodes using the command kubeadm prints at the end of the output.

How to Install kubectl

kubectl is the command-line tool you use to interact with any Kubernetes cluster.

On Windows:

Download kubectl.exe from Kubernetes' website and place it in a directory that is in your PATH. Or install with Chocolatey:

choco install kubernetes-cli

On Mac:

brew install kubectl

On Linux:

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x kubectl
sudo mv kubectl /usr/local/bin/kubectl

Test it:

kubectl get pods --all-namespaces

On a fresh cluster, you'll see system pods running in the kube-system namespace — things like coredns and storage-provisioner. That's the expected output. It means your cluster is up and kubectl can talk to it.

Kubernetes is running. Next is Vagrant. But before that, there's one important distinction worth making.

Docker vs Vagrant — they aren't the same thing

Docker creates containers: lightweight processes that share your operating system's kernel. Vagrant creates full virtual machines: isolated computers with their own OS running inside your laptop.

Containers are fast and small. VMs are heavier but behave exactly like real servers. You'll use both in this lab for different reasons.

How to Set Up Vagrant

Vagrant lets you create and manage reproducible virtual machine environments. It is ideal for simulating multi-server setups on a single laptop.

How to Install Vagrant on Windows

1. Download and install VirtualBox with default options.

2. Download and install Vagrant.

3. Restart your computer if prompted.

Note: VirtualBox and Hyper-V can't run at the same time on Windows. Check if Hyper-V is active:

systeminfo | findstr "Hyper-V"

If it's enabled, you have two options: switch to the Hyper-V Vagrant provider, or disable Hyper-V with:

Disable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V-All

Restart after disabling.

How to Install Vagrant on Mac and Linux

On Mac:

1. Download and install VirtualBox.

2. After installation, open System Preferences > Security & Privacy > General. You will see a message saying system software from Oracle was blocked. Click Allow and restart your Mac. Without this step, VirtualBox will not run.

3. Download and install Vagrant.

Note for Apple Silicon (M1/M2/M3) Macs: VirtualBox support on Apple Silicon is still limited. If you're on an M-series Mac, use UTM as your VM provider instead, or use Multipass which works natively on Apple Silicon.

On Linux:

1. Download and install VirtualBox.

2. Download and install Vagrant.

Verify both are installed:

vboxmanage --version
vagrant --version

How to Create Your First Vagrant Environment

Create a new directory for your project. Inside it, create a file named Vagrantfile with this content:

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/focal64"

  # Create a private network between VMs
  config.vm.network "private_network", type: "dhcp"

  # Forward port 8080 on your laptop to port 80 on the VM
  config.vm.network "forwarded_port", guest: 80, host: 8080

  # Install Nginx when the VM starts
  config.vm.provision "shell", inline: <<-SHELL
    apt-get update
    apt-get install -y nginx
    echo "Hello from Vagrant!" > /var/www/html/index.html
  SHELL
end

Start the VM:

vagrant up

Visit http://localhost:8080 in your browser. You should see "Hello from Vagrant!"

Troubleshooting SSH on Windows

If vagrant ssh fails, try:

vagrant ssh -- -v

Or connect manually:

ssh -i .vagrant/machines/default/virtualbox/private_key vagrant@127.0.0.1 -p 2222

How to Create a Local Vagrant Box Without Internet

Note: Most readers can skip this. Only do this if you want to work fully offline after the initial setup.

1. Download Ubuntu 20.04 LTS and save the .iso file locally.

2. Open VirtualBox and create a new VM: Name it ubuntu-devops, Type: Linux, Version: Ubuntu (64-bit).

3. Assign 2048MB RAM and a 20GB VDI disk.

4. Attach the .iso under Storage > Optical Drive.

5. Start the VM and complete the Ubuntu installation.

6. Once installed, shut down the VM and run:

VBoxManage list vms
vagrant package --base "ubuntu-devops" --output ubuntu2004.box
vagrant box add ubuntu2004 ubuntu2004.box

You now have a reusable local box that works without internet.

You can spin up virtual machines. Next is Ansible, which automates what goes inside them.

How to Install Ansible

Ansible automates configuration and software installation across multiple servers. Instead of SSH-ing into ten machines and running the same commands manually, you write a playbook once and Ansible handles the rest.

How to Install Ansible on Windows

Ansible doesn't run natively on Windows. You need to use it through WSL (Windows Subsystem for Linux).

1. Open PowerShell as Administrator and enable WSL:

dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart

1. Restart your computer.

2. Install Ubuntu from the Microsoft Store.

3. Open Ubuntu and install Ansible:

sudo apt update
sudo apt install software-properties-common
sudo apt-add-repository --yes --update ppa:ansible/ansible
sudo apt install ansible

How to Install Ansible on Mac

brew install ansible

How to Install Ansible on Linux

# Ubuntu/Debian
sudo apt update
sudo apt install software-properties-common
sudo apt-add-repository --yes --update ppa:ansible/ansible
sudo apt install ansible

# Red Hat/CentOS
sudo yum install ansible

How to Test Ansible

Create a file called hosts in your current directory:

[local]
localhost ansible_connection=local

Create a file called playbook.yml in the same directory:

---
- name: Test playbook
  hosts: local
  tasks:
    - name: Print a message
      debug:
        msg: "Ansible is working!"

Run the playbook, passing the local hosts file with -i:

ansible-playbook -i hosts playbook.yml

You should see the message "Ansible is working!" in the output.

Alright, all your tools are installed. Now you'll use them together to build something real.

How to Build Your First DevOps Project

You can find the entire code for this lab in this repo: https://github.com/Osomudeya/homelab-demo-article

Now you'll put these tools together in one project. Each tool will perform its actual job, and nothing is forced.

Before you start, create a fresh directory for this project. Don't run it inside the directory you used to test Vagrant earlier, as the Vagrantfile here is different and will conflict.

You'll be building a two-VM environment: one machine serves a web page you write yourself inside a Docker container, and the other runs a MariaDB database. Vagrant creates the machines and Ansible configures them. The page you see at the end is yours.

Step 1: Create the Project Directory

mkdir devops-lab-project && cd devops-lab-project

Step 2: Write Your Site Content

Create a file called index.html in the project directory. Write whatever you want on this page — it's what you'll see in your browser at the end:

<!DOCTYPE html>
<html>
  <head><title>My DevOps Lab</title></head>
  <body>
    <h1>My DevOps Lab</h1>
    <p>Provisioned by Vagrant. Configured by Ansible. Served by Docker.</p>
    <p>Built on a laptop. No cloud account needed.</p>
  </body>
</html>

Change the text to whatever you like. This is your page.

Step 3: Write the Vagrantfile

Create a file called Vagrantfile in the same directory:

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/focal64"

  config.vm.define "web" do |web|
    web.vm.network "private_network", ip: "192.168.33.10"
    web.vm.network "forwarded_port", guest: 80, host: 8080
  end

  config.vm.define "db" do |db|
    db.vm.network "private_network", ip: "192.168.33.11"
  end
end

Step 4: Start the Virtual Machines

vagrant up

The first run downloads the ubuntu/focal64 box, which is around 500MB.

Expect this to take 10–30 minutes depending on your connection. Subsequent runs will be much faster since the box is cached locally.

Step 5: Create the Ansible Inventory

Create a file called inventory in the same directory:

[webservers]
192.168.33.10 ansible_user=vagrant ansible_ssh_private_key_file=.vagrant/machines/web/virtualbox/private_key

[dbservers]
192.168.33.11 ansible_user=vagrant ansible_ssh_private_key_file=.vagrant/machines/db/virtualbox/private_key

Ansible uses the Vagrant-generated private keys so it can SSH in as the vagrant user. Host key checking for this lab is turned off in ansible.cfg (next step), not in the inventory.

Step 6: Create the Ansible Config File

Before running the playbook, create a file called ansible.cfg in the same directory:

[defaults]
inventory = inventory
host_key_checking = False

The inventory line tells Ansible to use the inventory file in this folder by default. host_key_checking = False tells Ansible not to verify SSH host keys when connecting to your Vagrant VMs. Without it, Ansible will fail with a Host key verification failed error on first connection because the VM's key is not yet in your known_hosts file.

These settings are for a local lab only. Do not use host_key_checking = False for production systems.

Step 7: Create the Ansible Playbook

Create a file called playbook.yml:

---
- name: Configure web server
  hosts: webservers
  become: yes
  tasks:

    - name: Install Docker
      apt:
        name: docker.io
        state: present
        update_cache: yes

    - name: Start Docker service
      service:
        name: docker
        state: started
        enabled: yes

    # Create the directory that will hold your site content
    - name: Create web content directory
      file:
        path: /var/www/html
        state: directory
        mode: '0755'

    # This copies your index.html from your laptop into the VM
    - name: Copy site content to web server
      copy:
        src: index.html
        dest: /var/www/html/index.html

    # This mounts that file into the Nginx container so it serves your page
    # The -v flag connects /var/www/html on the VM to /usr/share/nginx/html inside the container
    - name: Run Nginx serving your content
      shell: |
        docker rm -f webapp 2>/dev/null || true
        docker run -d --name webapp --restart always -p 80:80 \
          -v /var/www/html:/usr/share/nginx/html:ro nginx

- name: Configure database server
  hosts: dbservers
  become: yes
  tasks:

    # Hash sum mismatch on .deb downloads is often stale lists, a flaky mirror, or apt pipelining
    # behind NAT; fresh indices + Pipeline-Depth 0 usually fixes it on lab VMs.
    - name: Disable apt HTTP pipelining (mirror/proxy hash mismatch workaround)
      copy:
        dest: /etc/apt/apt.conf.d/99disable-pipelining
        content: 'Acquire::http::Pipeline-Depth "0";'
        mode: "0644"

    - name: Clear apt package index cache
      shell: apt-get clean && rm -rf /var/lib/apt/lists/* /var/lib/apt/lists/auxfiles/*
      changed_when: true

    - name: Update apt cache after reset
      apt:
        update_cache: yes

    - name: Install MariaDB
      apt:
        name: mariadb-server
        state: present
        update_cache: no

    - name: Start MariaDB service
      service:
        name: mariadb
        state: started
        enabled: yes

Two lines worth paying attention to:

• src: index.html — Ansible looks for this file in the same directory as the playbook. That is the file you wrote in Step 2.

• -v /var/www/html:/usr/share/nginx/html:ro — this mounts the directory from the VM into the Nginx container. The :ro means read-only. Nginx serves whatever is in that folder.

Step 8: Run the Playbook

ansible-playbook -i inventory playbook.yml

You'll see task-by-task output as Ansible connects to each VM over SSH and configures it. A green ok or yellow changed next to each task means it worked. Red fatal means something failed.

Step 9: Verify the Setup

Open http://localhost:8080 in your browser. You should see the page you wrote in Step 2 served from inside a Docker container, running on a Vagrant VM, configured automatically by Ansible.

If you see the page, every tool in this lab is working together.

Step 9: Clean Up (Optional)

When you're done:

vagrant destroy -f

This shuts down and deletes both VMs. Your Vagrantfile, inventory, playbook.yml, and index.html stay on disk — run vagrant up followed by ansible-playbook -i inventory playbook.yml any time to bring it all back.

Now that you have a working lab, let's use it properly.

How to Break Your Lab on Purpose

Following these steps has gotten you a running lab. Breaking things teaches you how everything actually works.

Here are five things to break and what to look for when you do.

Break 1: Crash the Main Process Inside the Container (and Watch It Come Back)

Doing this just proves that something inside the container can die (like a real bug or OOM), Docker can restart the container because of --restart always, and your site can come back without re-running Ansible.

After vagrant ssh web, every docker command below runs on the web VM. So keep your browser on your laptop at http://localhost:8080 (Vagrant forwards your host port to the VM’s port 80).

Troubleshooting: If Your Lab Isn't Ready

From your project folder on the host (your laptop) – unless the step says to run it on the VM:

• You ran vagrant destroy -f. Run vagrant up, then ansible-playbook -i inventory playbook.yml.

• docker ps shows webapp but status is Exited. On the web VM, run sudo docker start webapp, then sudo docker ps again.

• There's no webapp row in docker ps -a. Re-run ansible-playbook -i inventory playbook.yml on the host.

If the playbook is already applied and webapp is Up, skip this section and start at step 1 under Steps (happy path) below. (Don't skip SSH or docker ps. You need the VM shell and a quick check before you run docker exec.)

Steps (happy path)

1. SSH into the web VM:

vagrant ssh web

1. Confirm webapp is Up:

   sudo docker ps
   

2. Break it on purpose: kill the container’s main process from inside (PID 1). That ends the container the same way a crashing app would, not the same as docker stop on the host:

sudo docker exec webapp sh -c 'sleep 5 && kill 1'

The sleep 5 gives you a moment to switch to the browser. Right after you run the command, open or refresh http://localhost:8080. You may catch a brief error or blank page while nothing is listening on port 80.

1. Watch Docker restart the container:

watch sudo docker ps -a

Within a few seconds you should see Exited (137) become Up again. (Press Ctrl+C to exit watch.)

5. Refresh the browser. You should see the same HTML as before, because the files live on the VM under /var/www/html and are bind-mounted into the container; restarting only replaced the Nginx process, not those files.

Why not docker stop or docker kill on the host for this demo?

Those commands go through Docker’s API. On many setups (including recent Docker), Docker treats them as you choosing to stop the container (hasBeenManuallyStopped), and --restart always may not bring the container back until you docker start it or similar.

Killing PID 1 from inside the container is treated more like an internal crash, so the restart policy you set in the playbook is the one you actually get to observe here.

Kubernetes analogy: A pod whose containers exit can be restarted by the kubelet; a pod you delete does not come back by itself.

What to observe (three separate checks):

1. Exit code: After kill 1, docker ps -a should show the container exited with code 137, meaning the main process was killed by a signal. That confirms the container really died, not that you ran docker stop on the host.

2. Restart delay vs browser: Watch how many seconds pass between Exited and Up in docker ps -a; that interval is Docker applying --restart always. That's separate from what you see in the browser: the browser only shows whether something is accepting connections on port 80 on the VM, so it may show an error or blank page during the gap even while Docker is about to restart the container.

3. Content after recovery: After status is Up again, refresh the page. You should see the same HTML as before. That shows your content lives on the VM disk (mounted into the container with -v), not inside a file that vanishes when the container process restarts. The process was replaced, not your index.html on the host path.

Break 2: Cause a Container Name Conflict

On a single Docker daemon (here, on your web VM), a container name is a unique label. Two running (or stopped) containers can't share the same name. Scripts and playbooks that always use docker run --name webapp without cleaning up first hit this error constantly and recognizing it saves time in real work.

Before you start: Ansible already created one container named webapp.
Stay on the web VM (for example still inside vagrant ssh web) so the commands below run where that container lives.

So now, try to start a second container and also call it webapp. The image is plain nginx here on purpose – the point is the name clash, not matching your site’s ports or volume mounts.

sudo docker run -d --name webapp nginx

What actually happens here is that Docker doesn't create a second container. It returns an error immediately. Your original webapp is unchanged.

This is because the name webapp is already registered to the existing container (the error shows that container’s ID). Docker refuses to reuse the name until the old container is removed or renamed.

Example error (your ID will differ):

docker: Error response from daemon: Conflict. The container name "/webapp" is already in use by container "2e48b81a311c4b71cdc1e25e0df75a22296845c7eb53aab82f9ae739fb6410ec". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.

To fix it, free the name, then create webapp again the same way the playbook does (publish port 80, mount your HTML, restart policy):

sudo docker rm -f webapp
sudo docker run -d --name webapp --restart always -p 80:80 \
  -v /var/www/html:/usr/share/nginx/html:ro nginx

After that, your site should behave as before (refresh http://localhost:8080 from your laptop).

What to observe:

Read Docker’s Conflict message end to end. You should see that the name /webapp is already in use and a container ID pointing at the existing box. In production, that pattern means “something already claimed this name. Just remove it, rename it, or pick a different name before you run docker run again.”

Break 3: Make Ansible Fail to Reach a VM

Ansible separates “could not connect” from “connected, but a task broke.” The first is UNREACHABLE, the second is FAILED. Knowing which one you have tells you whether to fix network / SSH or playbook / packages / permissions.

On your laptop, in the project folder, edit inventory and change the web server address from 192.168.33.10 to an IP no VM uses, for example 192.168.33.99. Save the file.

[webservers]
192.168.33.99 ansible_user=vagrant ansible_ssh_private_key_file=.vagrant/machines/web/virtualbox/private_key

What you run (from the same project folder on the host):

ansible-playbook -i inventory playbook.yml

After this, Ansible tries to SSH to 192.168.33.99. Nothing on your lab network answers as that host (or SSH never succeeds), so Ansible never runs tasks on the web server. It stops that host with UNREACHABLE:

fatal: [192.168.33.99]: UNREACHABLE! => {"msg": "Failed to connect to the host via ssh"}

This is realistic because the same message shape appears when the IP is wrong, the VM isn't running, a firewall blocks port 22, or the network is misconfigured. The common thread is no working SSH session.

Now it's time to put it back: restore 192.168.33.10 in inventory and run ansible-playbook -i inventory playbook.yml again. The web play should reach the VM and complete (assuming your lab is up).

UNREACHABLE vs FAILED – what to observe:

• If Ansible prints UNREACHABLE, you should assume it never opened SSH on that host and never ran tasks there. Go ahead and fix the connection (IP, VM up, firewall, key path) before you debug playbook logic.

• If Ansible prints FAILED, you should assume SSH worked and a task returned an error. Read the task output for the real cause (package name, permissions, syntax), not the network first.

When you debug later, you should look at the keyword Ansible prints: UNREACHABLE points to reachability while FAILED points to task output and the first failed task under that host.

Break 4: Fill the VM's Disk

Databases and other services need free disk for logs, temp files, and data. When the filesystem is full or nearly full, a service may fail to start or fail at runtime. This break walks through the same diagnosis habit you would use on a real server: check space, then read systemd and journal output for the service.

All commands below run on the db VM after vagrant ssh db. MariaDB was installed there by your playbook.

What you do:

1. Open a shell on the db VM:

   vagrant ssh db
   

2. Allocate a large file full of zeros (here 1GB) to simulate something eating disk space:

   sudo dd if=/dev/zero of=/tmp/bigfile bs=1M count=1024
   
   df -h
   

   Use df -h to see how full the root filesystem (or relevant mount) is. Your Vagrant disk may be large enough that 1GB only raises usage. If MariaDB still starts, you still practiced the checks. To see a stronger effect, you can repeat with a larger count= only in a lab (never fill production disks on purpose without a plan).

3. Ask systemd to restart MariaDB and show status:

   sudo systemctl restart mariadb
   sudo systemctl status mariadb
   

   If the disk is critically full, restart may fail or the service may show failed or not running.

4. If something looks wrong, read recent logs for the MariaDB unit:

   sudo journalctl -u mariadb --no-pager | tail -20
   

   Errors often mention disk, space, read-only filesystem, or InnoDB being unable to write.

5. Clean up so your VM stays usable:

   sudo rm /tmp/bigfile
   

   Optionally run sudo systemctl restart mariadb again and confirm it is active (running).

What to observe:

• You should use df -h first to confirm whether the filesystem is actually tight. That avoids blaming the database when disk space is fine.

• You should read systemctl status mariadb to see whether systemd thinks the service is active, failed, or flapping.

• You should read journalctl -u mariadb when status is bad, so you can tie the failure to concrete errors from MariaDB or the kernel (often mentioning disk, space, or read-only filesystem). Space + status + logs is the same order you would use on a production server.

Break 5: Run Minikube Out of Resources

Kubernetes schedules pods onto nodes that have enough CPU and memory. If you ask for more than the cluster can place, some pods stay Pending and Events explain why (for example Insufficient cpu). That is not the same as a pod that starts and then crashes.

To do this, you'll need a local cluster (we're using Minikube in this guide) and kubectl on your laptop. This break doesn't use the Vagrant VMs. If you haven't installed Minikube yet, complete the "How to Set Up Kubernetes" section first, or skip this break until you do.

You'll run this on your Mac, Linux, or Windows terminal (host), not inside vagrant ssh. If you're still inside a VM, type exit until your prompt is back on the host.

What you do:

1. Check Minikube:

   minikube status
   

   If it's stopped, start it (Docker driver matches earlier sections):

   minikube start --driver=docker
   

2. Create a deployment with many replicas so your single Minikube node can't run them all at once:

   kubectl create deployment stress --image=nginx --replicas=20
   
   #watch pods start
   kubectl get pods -w
   

   Press Ctrl+C when you're done watching. Some pods may stay Pending while others are Running.

3. Pick one Pending pod name from kubectl get pods and inspect it:

   kubectl describe pod <pod-name>
   

   Under Events, look for FailedScheduling and a line similar to:

   Warning  FailedScheduling  0/1 nodes are available: 1 Insufficient cpu.
   

   You might see Insufficient memory instead, depending on your machine.

4. Fix the lab by scaling back so the cluster can catch up:

   kubectl scale deployment stress --replicas=2
   

   You can delete the deployment entirely when finished: kubectl delete deployment stress.

What to observe:

• You should see Pending pods stay unscheduled until capacity frees up. That means the scheduler hasn't placed them on any node yet, usually because the node is out of CPU or memory for that workload.

• You should read kubectl describe pod <pod-name> and scroll to Events. Messages like Insufficient cpu or Insufficient memory mean the cluster ran out of schedulable capacity, not that the container image image is corrupt.

• You should contrast that with a pod that reaches Running and then CrashLoopBackOff, which usually means the process inside the container keeps exiting. that is an application or config problem, not a “nowhere to run” problem.

What You Can Now Do

You didn't just install tools in this tutorial. You also used them.

You can now spin up two servers from a single file. You can write a playbook that installs software and deploys a container without touching either machine manually.

You can serve a page you wrote from inside a Docker container running on a Vagrant VM, and bring the whole thing back from scratch in one command.

You also broke it. You saw what a container conflict looks like, what Ansible prints when it can't reach a machine, what disk pressure does to a running service, and what a Kubernetes scheduler says when it runs out of resources. Those error messages aren't unfamiliar anymore.

That's the difference between someone who has read about DevOps and someone who has run it.

Here are four free projects you can run in this same lab to go further:

• DevOps Home-Lab 2026 — Build a multi-service app (frontend, API, PostgreSQL, Redis) end-to-end with Docker Compose, Kubernetes, Prometheus/Grafana monitoring, GitOps with ArgoCD, and Cloudflare for global exposure.

• KubeLab — Trigger real Kubernetes failure scenarios, pod crashes, OOMKills, node drains, cascading failures, and watch how the cluster responds using live metrics.

• K8s Secrets Lab — Build a full secret management pipeline from AWS Secrets Manager into your cluster, including rotation behavior and IRSA.

• DevOps Troubleshooting Toolkit — Structured debugging guides across Linux, containers, Kubernetes, cloud, databases, and observability with copy-paste commands for real incidents.

All free and open source: github.com/Osomudeya/List-Of-DevOps-Projects.

If you want to go deeper, you can find six full chapters covering Terraform, Ansible, monitoring, CI/CD, and a simulated three-VM production environment at Build Your Own DevOps Lab.

If DevOps feels out of reach because you’re also concerned about the cost, don't worry. I’ll guide you step-by-step through creating a production-ready pipeline without spending a dime. Let's get started!

Table of Contents

1. Prerequisites

2. Introduction

3. How to Set Up Your Source Control and Project Structure

4. How to Build Your CI Pipeline with GitHub Actions

5. How to Optimize Docker Builds for CI

6. Infrastructure as Code Using Terraform and Free Cloud Providers

7. How to Set Up Container Orchestration on Minimal Resources

8. How to Create a Free Deployment Pipeline

9. How to Build a Comprehensive Monitoring System

10. How to Implement Security Testing and Scanning

11. Performance Optimization and Scaling

12. Putting it All Together

13. Conclusion

🛠 Prerequisites

• Basic Git knowledge: Cloning repos, creating branches, committing code, and creating PRs

• Familiarity with command line: For Docker, Terraform, and Kubernetes

• Basic understanding of CI/CD: Continuous integration/delivery concepts and pipelines

Accounts needed:

• GitHub account

• At least one cloud provider: AWS Free Tier (recommended), Oracle Cloud Free Tier, or Google Cloud/Azure with free credits

• Terraform Cloud (free tier) for infrastructure state management

• Grafana Cloud (free tier) for monitoring

• UptimeRobot (free tier) for external availability checks

Tools to Install Locally

Optional but Helpful:

• VS Code or any good code editor

• Postman for testing APIs

• Understanding of YAML and Dockerfiles

Introduction

When people hear "DevOps," they often picture complex enterprise systems powered by pricey tools and premium cloud services. But the truth is, you don't actually need a massive budget to build a solid, professional-grade DevOps pipeline. The foundations of good DevOps – automation, consistency, security, and visibility – can be built entirely with free tools.

In this guide, you will learn how to build a production-ready DevOps pipeline using zero-cost resources. We will use a simple CRUD (Create, Read, Update, Delete) app with frontend, backend API, and database as our example project to demonstrate every step of the process.

How to Set Up Your Source Control and Project Structure

1. Create a Well-Structured Repository

A clean repo is the foundation of your pipeline. We will set up:

• Separate folders for frontend, backend, and infrastructure

• A .github folder to hold workflow configurations

• Clear naming conventions and a well-written README.md

🛠 Tip: Use semantic commit messages and consider adopting Conventional Commits for clarity in versioning and changelogs.

2. Set Up Branch Protection Without Paid Features

While GitHub's more advanced rules require Pro, you can still:

• Require pull requests before merging

• Enable status checks to prevent broken code from landing in main

• Enforce linear history for cleaner version control

💡 This makes your project safer and more collaborative, without needing GitHub Enterprise.

3. Implement PR Templates and Automated Checks

Make your reviews smoother:

• Add a PULL_REQUEST_TEMPLATE.md to guide contributors

• Use GitHub Actions (which we'll set up in the next part) for linting, tests, and formatting checks

✨ These tiny improvements add polish and professionalism.

4. Configure GitHub Issue Templates and Project Boards

Even solo developers benefit from issue tracking:

• Add issue templates for bugs and features

• Use GitHub Projects to manage work with a Kanban board, all free and native to GitHub

📌 Bonus: This setup lays the groundwork for GitOps practices later on.

5. Advanced Technique: Set Up Custom Validation Scripts as Pre-Commit Hooks

Before code ever hits GitHub, you can catch issues locally with Git hooks. Using a tool like Husky or pre-commit, you can:

• Lint code before it's committed

• Run tests or formatters automatically

• Prevent secrets from being accidentally committed

// Initialize Husky and install needed dependencies
// Then add a pre-commit hook that runs tests before allowing the commit
npx husky-init && npm install
npx husky add .husky/pre-commit "npm test"

6. Sample CRUD App Setup:

Our CRUD app manages users (create, read, update, delete). Below is the minimal code with comments to explain each part:

Backend (backend/):

// backend/package.json
{
  "name": "crud-backend", // Name of the backend project
  "version": "1.0.0", // Version for tracking changes
  "scripts": {
    "start": "node index.js", // Runs the server
    "test": "echo 'Add tests here'", // Placeholder for tests (update with Jest later)
    "lint": "eslint ." // Checks code style with ESLint
  },
  "dependencies": {
    "express": "^4.17.1", // Web framework for API endpoints
    "pg": "^8.7.3" // PostgreSQL client to connect to the database
  },
  "devDependencies": {
    "eslint": "^8.0.0" // Linting tool for code quality
  }
}

// backend/index.js
const express = require('express'); // Import Express for building the API
const { Pool } = require('pg'); // Import PostgreSQL client
const app = express(); // Create an Express app
app.use(express.json()); // Parse JSON request bodies

// Connect to PostgreSQL using DATABASE_URL from environment variables
const pool = new Pool({ connectionString: process.env.DATABASE_URL });

// Health check endpoint for Kubernetes probes and monitoring
app.get('/healthz', (req, res) => res.json({ status: 'ok' }));

// Get all users from the database
app.get('/users', async (req, res) => {
  const { rows } = await pool.query('SELECT * FROM users'); // Query the users table
  res.json(rows); // Send users as JSON
});

// Add a new user to the database
app.post('/users', async (req, res) => {
  const { name } = req.body; // Get name from request body
  // Insert user and return the new record
  const { rows } = await pool.query('INSERT INTO users(name) VALUES($1) RETURNING *', [name]);
  res.json(rows[0]); // Send the new user as JSON
});

// Start the server on port 3000
app.listen(3000, () => console.log('Backend running on port 3000'));

Frontend (frontend/):

// frontend/package.json
{
  "name": "crud-frontend", // Name of the frontend project
  "version": "1.0.0", // Version for tracking changes
  "scripts": {
    "start": "react-scripts start", // Runs the dev server
    "build": "react-scripts build", // Builds for production
    "test": "react-scripts test", // Runs tests (placeholder for Jest)
    "lint": "eslint ." // Checks code style with ESLint
  },
  "dependencies": {
    "react": "^17.0.2", // Core React library
    "react-dom": "^17.0.2", // Renders React to the DOM
    "react-scripts": "^4.0.3", // Scripts for React development
    "axios": "^0.24.0" // HTTP client for API calls
  },
  "devDependencies": {
    "eslint": "^8.0.0" // Linting tool for code quality
  }
}

// frontend/src/App.js
import React, { useState, useEffect } from 'react'; // Import React and hooks
import axios from 'axios'; // Import Axios for API requests

function App() {
  // State for storing users fetched from the backend
  const [users, setUsers] = useState([]);
  // State for the input field to add a new user
  const [name, setName] = useState('');

  // Fetch users when the component mounts
  useEffect(() => {
    axios.get('http://localhost:3000/users').then(res => setUsers(res.data));
  }, []); // Empty array means run once on mount

  // Add a new user via the API
  const addUser = async () => {
    const res = await axios.post('http://localhost:3000/users', { name }); // Post new user
    setUsers([...users, res.data]); // Update users list
    setName(''); // Clear input field
  };

  return (
    <div>
      <h1>Users</h1>
      {/* Input for new user name */}
      <input value={name} onChange={e => setName(e.target.value)} />
      {/* Button to add user */}
      <button onClick={addUser}>Add User</button>
      {/* List all users */}
      <ul>{users.map(user => <li key={user.id}>{user.name}</li>)}</ul>
    </div>
  );
}

export default App; // Export the component

Database Setup:

-- infra/db.sql
-- Create a table to store users
CREATE TABLE users (
  id SERIAL PRIMARY KEY, -- Auto-incrementing ID
  name VARCHAR(100) NOT NULL -- User name, required
);

crud-app/
├── backend/
│   ├── package.json
│   └── index.js
├── frontend/
│   ├── package.json
│   └── src/App.js
├── infra/
│   └── db.sql
├── .github/
│   └── workflows/
└── README.md

This app provides a /users endpoint (GET/POST) and a frontend to list/add users, stored in PostgreSQL. The /healthz endpoint supports monitoring. Save this code in your repo to follow the pipeline steps.

How to Build Your CI Pipeline with GitHub Actions

1. Set Up Your First GitHub Actions Workflow

First, let’s create a basic workflow that automatically builds, tests, and lints your app every time you push code or open a pull request. This ensures your app stays healthy and any issues are caught early.

Create a file at .github/workflows/ci.yml and add the following:

# CI workflow to build, test, and lint the CRUD app on push or pull request
name: CI Pipeline
on:
  push:
    branches: [main] # Trigger on pushes to main branch
  pull_request:
    branches: [main] # Trigger on PRs to main branch
jobs:
  build:
    runs-on: ubuntu-latest # Use GitHub's free Linux runner
    steps:
      - uses: actions/checkout@v3 # Check out the repository code
      - name: Set up Node.js # Install Node.js environment
        uses: actions/setup-node@v3
        with:
          node-version: '18' # Use Node.js 18 for consistency
      - name: Cache dependencies # Cache node_modules to speed up builds
        uses: actions/cache@v3
        with:
          path: ~/.npm # Cache npm’s global cache
          key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }} # Key based on OS and package-lock.json
      - run: npm ci # Install dependencies reliably using package-lock.json
      - run: npm test # Run tests defined in package.json
      - run: npm run lint # Run ESLint to ensure code quality

This workflow automatically runs on every push and pull request to the main branch. It installs dependencies, runs tests, and performs code linting, with dependency caching to make builds faster over time.

Common Issues and Fixes:

• “Secret not found”: Ensure AWS_ACCESS_KEY_ID is in repository secrets (Settings → Secrets).

• Tests fail: Check test/users.test.js for database connectivity.

Understanding GitHub Actions' Free Tier Limits

Before building more workflows, it is important to know what GitHub offers for free.

If you are working on private repositories, you get 2,000 free minutes per month. For public repositories, you get unlimited minutes.

To avoid hitting limits quickly:

• Cache your dependencies to cut down install times.

• Only trigger workflows on meaningful branches (like main or release).

• Skip unnecessary steps when you can.

2. Creating a Multi-Stage Build Pipeline

As your app grows, it is better to split your CI pipeline into clear stages like install, test, and lint. This structure makes workflows easier to maintain and speeds things up, because some jobs can run in parallel.

Here’s how you can split the work into multiple jobs for better clarity:

jobs:
  install:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: npm ci  # Clean install of dependencies

  test:
    needs: install  # This job depends on the install job finishing
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: npm test  # Run test suite

  lint:
    needs: install  # This job also depends on install but runs in parallel with test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: npm run lint  # Run linting checks

By breaking the pipeline into stages, you can quickly spot which step fails, and your test and lint jobs can run at the same time after dependencies are installed.

3. Implement Matrix Builds for Cross-Environment Testing

When you want your app to work across different Node.js versions or databases, matrix builds are your best bet. They let you test across multiple environments in parallel, without duplicating code.

Here’s how you can set up a matrix strategy, to test across multiple environments simultaneously:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        node-version: [14.x, 16.x, 18.x]  # Test on multiple Node versions
        database: [postgres, mysql]        # Test against different databases
    steps:
      - uses: actions/checkout@v3
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v3
        with:
          node-version: ${{ matrix.node-version }}
      - run: npm install
      - run: npm test  # This will run 6 different test combinations (3 Node versions × 2 databases)

Matrix builds save time and help you catch environment-specific bugs early.

4. Optimize Workflow with Dependency Caching

Every second counts in CI. Dependency caching can help save minutes in your workflow by reusing previously installed packages instead of reinstalling them from scratch every time.

Here’s how to set up smart caching to speed up your builds:

- name: Cache node modules
  uses: actions/cache@v3
  with:
    path: |  # Cache both global npm cache and local node_modules
      ~/.npm
      node_modules
    key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}  # Cache key based on OS and dependencies
    restore-keys: |  # Fallback keys if exact match isn't found
      ${{ runner.os }}-node-

This cache setup checks if your dependencies have changed. If not, it restores the cache, making builds significantly faster.

How to Optimize Docker Builds for CI

When you're building Docker images in CI, build time can quickly become a bottleneck. Especially if your images are large. Optimizing your Docker builds makes your pipelines much faster, saves bandwidth, and produces smaller, more efficient images ready for deployment.

In this section, I’ll walk through creating a basic Dockerfile, using multi-stage builds, caching layers, and enabling BuildKit for even faster builds.

1. Create a Baseline Dockerfile

First, start with a simple Dockerfile that installs your app’s dependencies and runs it. This is what you’ll be optimizing later.

# Simple Dockerfile for a Node.js application
FROM node:18-alpine  # Use Alpine for a smaller base image
WORKDIR /app         # Set working directory
COPY . .             # Copy all files to container
RUN npm ci           # Install dependencies (clean install)
CMD ["npm", "start"] # Start the application

Using an Alpine-based Node.js image helps keep your image small from the start.

2. Multi-Stage Docker Builds

Next, let's separate the build process from the production image. Multi-stage builds let you compile or build your app in one stage and only copy over the final product to a clean, smaller image. This keeps production images lean:

# Stage 1: Build the application
FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./  # Copy package files first for better caching
RUN npm ci             # Install all dependencies
COPY . .               # Then copy source code
RUN npm run build      # Build the application

# Stage 2: Production image with minimal footprint
FROM node:18-alpine
WORKDIR /app
# Only copy built assets and production dependencies
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/package*.json ./
RUN npm ci --production  # Install only production dependencies
CMD ["node", "dist/server.js"]  # Run the built application

This approach keeps your production images lightweight and secure by excluding unnecessary build tools and dev dependencies.

3. Optimizing Layer Caching

For even faster builds, order your Dockerfile instructions to maximize layer caching. Copy and install dependencies before copying your full source code.

This way, Docker reuses the cached npm install step if your dependencies haven't changed, even if you edit your app's code:

• First: COPY package*.json ./

• Then: RUN npm ci

• Finally: COPY . .

4. Enable BuildKit for Faster Builds

Docker BuildKit is a newer build engine that enables features like better caching, parallel build steps, and overall faster builds.

To enable BuildKit during your CI, run:

- name: Build Docker image
  run: |
    # Enable BuildKit for parallel and more efficient builds
    DOCKER_BUILDKIT=1 docker build -t myapp:latest .

Turning on BuildKit can significantly speed up complex Docker builds and is highly recommended for all CI pipelines.

Infrastructure as Code Using Terraform and Free Cloud Providers

Why Infrastructure as Code (IaC) Matters

When you manage infrastructure manually – that is, clicking around cloud dashboards or setting things up by hand – it’s easy to lose track of what you did and how to repeat it.

Infrastructure as Code (IaC) solves this by letting you define your infrastructure with code, version it just like application code, and track every change over time. This makes your setups easy to replicate across environments (development, staging, production), ensures changes are declarative and auditable, and reduces human error.

Whether you are spinning up a single server or scaling a complex system, IaC lays the foundation for professional-grade infrastructure from day one, letting you automate, document, and grow your environment systematically.

How to Provision Infrastructure with Terraform

Initialize a Terraform Project

First, define the providers and versions you need. Here, we’re using Render’s free cloud hosting service:

# Define required providers and versions
terraform {
  required_providers {
    render = {
      source  = "renderinc/render"  # Using Render's free tier
      version = "0.1.0"             # Specify provider version for stability
    }
  }
}

# Configure the Render provider with authentication
provider "render" {
  api_key = var.render_api_key  # Store API key as a variable
}

Then, configure the provider by authenticating with your API key. It is best practice to store secrets like API keys in variables instead of hardcoding them. This setup tells Terraform what platform you’re working with (Render) and how to authenticate to manage resources automatically.

Provision a Web App on Render

Next, define the infrastructure you want – in this case, a web service hosted on Render:

# Define a web service on Render's free tier
resource "render_service" "web_app" {
  name = "ci-demo-app"                                 # Service name
  type = "web_service"                                 # Type of service
  repo = "https://github.com/YOUR-USERNAME/YOUR-REPO"  # Source repo
  env = "docker"                                       # Use Docker environment
  plan = "starter"                                     # Free tier plan
  branch = "main"                                      # Deploy from main branch
  build_command = "docker build -t app ."              # Build command
  start_command = "docker run -p 3000:3000 app"        # Start command
  auto_deploy = true                                   # Auto-deploy on commits
}

This resource block describes exactly how your app should be deployed. Whenever you change this file and reapply, Terraform will update the infrastructure to match.

Provision PostgreSQL for Free

Most applications need a database, but you don't have to pay for one when you're getting started. Platforms like Railway offer free tiers that are perfect for development and small projects.

You can quickly create a free PostgreSQL instance by signing up on the platform and clicking "Create New Project". At the end, you'll get a DATABASE_URL a connection string that your app will use to talk to the database.

Connect App to DB

In Render (or whatever platform you're using), set an environment variable called DATABASE_URL and paste in the connection string from your PostgreSQL provider. This lets your application securely access the database without hardcoding credentials into your codebase.

Make it Reproducible

Once everything is defined, use Terraform to create and apply an infrastructure plan:

# Create execution plan and save it to a file
terraform plan -out=infra.tfplan
# Apply the saved plan exactly as planned
terraform apply infra.tfplan

Saving the plan to a file (infra.tfplan) ensures you’re applying exactly what you reviewed, so there will be no surprises.

Common Issues and Fixes:

• Provider not found: Run terraform init.

• API key error: Check render_api_key in Terraform Cloud variables.

How to Set Up Container Orchestration on Minimal Resources

When you're working with limited resources like a laptop, a small server, or a lightweight cloud VM, setting up full Kubernetes can be overwhelming. Instead, you can use K3d, a lightweight Kubernetes distribution that runs inside Docker containers. Here's how to set up a minimal, efficient cluster for local development or testing.

1. Install K3d for Local Kubernetes

First, install K3d. It's a super lightweight way to run Kubernetes clusters inside Docker without needing a heavy setup like Minikube.

# Download and install K3d - a lightweight K8s distribution
curl -s https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash

2. Create a Lightweight K3d Cluster

Once K3d is installed, you can spin up a cluster with minimal nodes to save resources.

# Create a minimal K8s cluster with 1 server and 2 agent nodes
k3d cluster create dev-cluster \
  --servers 1 \                        # Single server node to minimize resource usage
  --agents 2 \                         # Two worker nodes for pod distribution
  --volume /tmp/k3dvol:/tmp/k3dvol \   # Mount local volume for persistence
  --port 8080:80@loadbalancer \        # Map port 8080 locally to 80 in the cluster
  --api-port 6443                      # Set the API port

This setup gives you a tiny but real Kubernetes cluster that is perfect for experimentation.

3. Deploy with Optimized Kubernetes Manifests

Now that your cluster is running, you can deploy your app. It's important to define resource requests and limits carefully so your pods don’t consume too much memory or CPU.

# Resource-optimized deployment manifest
apiVersion: apps/v1
kind: Deployment
metadata:
  name: webapp  # Name of the deployment
spec:
  replicas: 1   # Single replica to save resources
  selector:
    matchLabels:
      app: webapp
  template:
    metadata:
      labels:
        app: webapp
    spec:
      containers:
        - name: app
          image: myapp:latest
          resources:
            # Set minimal resource requests
            requests:
              memory: "64Mi"   # Request only 64MB memory
              cpu: "50m"       # Request only 5% of a CPU core
            # Set reasonable limits
            limits:
              memory: "128Mi"  # Limit to 128MB memory
              cpu: "100m"      # Limit to 10% of a CPU core

This ensures Kubernetes knows how much to allocate and avoid overloading your lightweight environment.

4. Set up GitOps with Flux

To manage deployments automatically from your GitHub repository, you can set up GitOps using Flux.

# Install Flux CLI
brew install fluxcd/tap/flux

# Bootstrap Flux on your cluster connected to your GitHub repository
flux bootstrap github \
  --owner=YOUR_GITHUB_USERNAME \    # Your GitHub username
  --repository=YOUR_REPO_NAME \     # Repository to store Flux manifests
  --branch=main \                   # Branch to use
  --path=clusters/dev-cluster \     # Path within repo for cluster configs
  --personal                        # Flag for personal account

Flux watches your repo and applies updates to your cluster, keeping everything declarative and reproducible.

Common Issues and Fixes:

• Pods crash: Run kubectl logs pod-name or increase resources.

• Flux sync fails: Check GitHub token permissions.

How to Create a Free Deployment Pipeline

Like I said initially, not every project needs expensive infrastructure. If you're just getting started or building side projects, free tiers from cloud providers can cover a lot of ground.

1. Understanding Free Tier Limitations

Here’s a quick overview of popular cloud free tiers:

Knowing these limits helps you stay within budget.

2. Set Up Deployment Workflows

You can automate deployments with GitHub Actions. Here's an example of a deployment workflow to AWS:

# GitHub Action workflow for deploying to AWS
name: AWS Deployment

on:
  push:
    branches:
      - main  # Deploy on push to main branch

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3  # Check out code

      # Set up AWS credentials from GitHub secrets
      - name: Set up AWS credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1

      # Build the Docker image
      - name: Build Docker Image
        run: docker build -t myapp .

      # Push the image to AWS ECR
      - name: Push Docker Image to ECR
        run: |
          # Create repository if it doesn't exist (ignoring errors if it does)
          aws ecr create-repository --repository-name myapp || true

          # Login to ECR
          aws ecr get-login-password | docker login --username AWS --password-stdin <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com

          # Tag and push the image
          docker tag myapp:latest <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/myapp:latest
          docker push <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/myapp:latest

3. Implement Zero-Downtime Deployments

Zero downtime is crucial. Kubernetes makes this easy with rolling updates:

# Kubernetes deployment configured for zero-downtime updates
apiVersion: apps/v1
kind: Deployment
metadata:
  name: crud-app
spec:
  replicas: 3  # Multiple replicas for high availability
  selector:
    matchLabels:
      app: crud-app
  template:
    metadata:
      labels:
        app: crud-app
    spec:
      containers:
      - name: app
        image: <docker_registry>/crud-app:latest
        ports:
        - containerPort: 80  # Expose container port

By having multiple replicas, you ensure that some pods stay live during updates.

4. Create Cross-Cloud Deployment for Redundancy

If you want better reliability, you can deploy across different clouds in parallel:

# Deploy to multiple cloud providers for redundancy
name: Cross-Cloud Deployment

on:
  push:
    branches:
      - main

jobs:
  # Deploy to AWS
  aws-deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: AWS Setup & Deploy
        run: |
          # Configure AWS CLI with credentials
          aws configure set aws_access_key_id ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws configure set aws_secret_access_key ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          # AWS deployment commands...

  # Deploy to Oracle Cloud in parallel
  oracle-deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Oracle Setup & Deploy
        run: |
          # Configure Oracle Cloud CLI
          oci setup config
          # Oracle Cloud deployment commands...

Now if one cloud goes down, the other is still up.

5. Implement Automated Rollbacks with Health Checks

Set up health checks so Kubernetes can automatically rollback if something goes wrong:

# Deployment with health checks for automated rollbacks
apiVersion: apps/v1
kind: Deployment
metadata:
  name: crud-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: crud-app
  template:
    metadata:
      labels:
        app: crud-app
    spec:
      containers:
      - name: crud-app
        image: <docker_registry>/crud-app:latest
        ports:
        - containerPort: 80
        # Check if the container is alive
        livenessProbe:
          httpGet:
            path: /healthz  # Health check endpoint
            port: 80
          initialDelaySeconds: 5  # Wait before first check
          periodSeconds: 10       # Check every 10 seconds
        # Check if the container is ready to receive traffic
        readinessProbe:
          httpGet:
            path: /readiness  # Readiness check endpoint
            port: 80
          initialDelaySeconds: 5  # Wait before first check
          periodSeconds: 10       # Check every 10 seconds

How to Build a Comprehensive Monitoring System

Even with a small deployment, monitoring is key to spotting issues early. So now, I’ll walk through setting up a comprehensive monitoring system for your application.

You'll learn how to integrate Grafana Cloud for visualizing your metrics, use Prometheus for collecting data, and configure custom alerts to monitor your app's performance. I’ll also cover tracking Service Level Objectives (SLOs) and setting up external monitoring with UptimeRobot to make sure that your endpoints are always available.

1. Set Up Grafana Cloud's Free Tier

Create a Grafana Cloud account and connect Prometheus as a data source. They offer generous free usage, which is perfect for small teams.

2. Configure Prometheus for Metrics Collection

Prometheus collects metrics from your app.

# prometheus.yml - Basic Prometheus configuration
global:
  scrape_interval: 15s  # Collect metrics every 15 seconds
scrape_configs:
  - job_name: 'crud-app'  # Job name for the crud-app metrics
    static_configs:
      - targets: ['localhost:8080']  # Where to collect metrics from

This scrapes your app every 15 seconds for metrics.

3. Create Monitoring Dashboards

Grafana visualizes Prometheus data. You can create dashboards using queries like:

# Calculate average CPU usage rate per instance over 1 minute
avg(rate(cpu_usage_seconds_total[1m])) by (instance)

This calculates average CPU usage over the last minute per instance.

4. Write Custom PromQL Queries for Alerts

You can create smart alerts to detect increasing error rates, like the below:

# Calculate error rate as a percentage of total requests
# Alert when error rate exceeds 5%
sum(rate(http_requests_total{status=~"5.."}[5m])) by (service)
  / 
sum(rate(http_requests_total[5m])) by (service) > 0.05

This alerts if more than 5% of your traffic results in errors.

5. Implement SLO Tracking on a Budget

You can track Service Level Objectives (SLOs) with Prometheus for free:

# Calculate percentage of requests completed under 200ms
# Alert when it drops below 99%
rate(http_request_duration_seconds_bucket{le="0.2"}[5m]) 
  / rate(http_request_duration_seconds_count[5m]) 
> 0.99

This tracks if 99% of requests complete in under 200ms.

6. Set Up UptimeRobot for External Monitoring

Finally, you can use UptimeRobot to check if your endpoints are reachable externally, and get alerts if anything goes down.

How to Implement Security Testing and Scanning

Security should be integrated into your development pipeline from the start, not added as an afterthought. In this section, I’ll show you how to implement security testing and scanning at various stages of your workflow.

You’ll use GitHub CodeQL for static code analysis, OWASP ZAP for scanning web vulnerabilities, and Trivy for container image scanning. You’ll also learn how to enforce security thresholds directly in your CI pipeline.

1. Enable GitHub Code Scanning with CodeQL

GitHub has built-in code scanning with CodeQL. Here’s how to set it up:

# GitHub workflow for CodeQL security scanning
name: CodeQL

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

jobs:
  analyze:
    name: Analyze code with CodeQL
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      # Initialize the CodeQL scanning tools
      - name: Set up CodeQL
        uses: github/codeql-action/init@v2

      # Run the analysis and generate results
      - name: Analyze code
        uses: github/codeql-action/analyze@v2

This automatically checks your code for security vulnerabilities.

2. Integrate OWASP ZAP into Your CI Pipeline

You can also scan your deployed app with OWASP ZAP like this:

# Automated security scanning with OWASP ZAP
name: ZAP Scan

on:
  push:
    branches:
      - main

jobs:
  zap-scan:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      # Run the ZAP security scan against deployed application
      - name: Run ZAP security scan
        uses: zaproxy/action-full-scan@v0.3.0
        with:
          target: 'https://yourapp.com'  # URL to scan

This checks for common web vulnerabilities.

3. Set Up Trivy for Container Vulnerability Scanning

You can also check your container images for vulnerabilities with Trivy:

# Scan Docker images for vulnerabilities using Trivy
- name: Run Trivy vulnerability scanner
  uses: aquasecurity/trivy-action@master
  with:
    image-ref: 'crud-app:latest'   # Image to scan
    format: 'table'             # Output format
    exit-code: '1'              # Fail the build if vulnerabilities found
    ignore-unfixed: true        # Skip vulnerabilities without fixes
    severity: 'CRITICAL,HIGH'   # Only alert on critical and high severity

Your builds will fail if serious issues are found, keeping you safe by default.

4. Create Threshold-Based Pipeline Failures

You can configure your pipelines to fail automatically if vulnerabilities exceed a set threshold, enforcing strong security practices without manual effort. Here’s how that should look:

# Fail the pipeline if critical or high vulnerabilities are found
- name: Run Trivy vulnerability scanner
  uses: aquasecurity/trivy-action@master
  with:
    image-ref: 'crud-app:latest'   # Image to scan
    format: 'json'              # Output as JSON for parsing
    exit-code: '1'              # Fail the build if vulnerabilities found
    severity: 'CRITICAL,HIGH'   # Check for critical and high severity issues
    ignore-unfixed: true        # Skip vulnerabilities without fixes

This forces a no-compromise security posture – that is, if critical or high vulnerabilities are detected, the build stops immediately.

5. Implement Custom Security Checks

Sometimes you need to go beyond automated scanners. Here's a basic example of a custom security check you can add to your pipeline:

#!/bin/bash

# Custom script to check for hard-coded secrets in source code
# Check for hard-coded API keys in source files
if grep -r "API_KEY" ./src; then
  echo "Security issue: Found hard-coded API keys."
  exit 1  # Fail the build
else
  echo "No hard-coded API keys found."
fi

You can extend this script to scan for patterns like private keys, passwords, or other sensitive information, helping catch issues before they ever reach production.

Performance Optimization and Scaling

Optimizing early saves you pain later. Here’s how to make your pipelines faster, smarter, and more scalable:

1. Measure Pipeline Execution Times

Understanding how long each step takes is the first step to improving it:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      # Record the start time
      - name: Start timer
        run: echo "Start time: $(date)"

      - uses: actions/checkout@v3
      - run: npm install

      # Record the end time to calculate duration
      - name: End timer
        run: echo "End time: $(date)"

Later, you can automate time tracking for full reports and alerts.

2. Implement Parallelization Strategies

Split your jobs smartly to save time:

jobs:
  # First job to install dependencies
  install:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: npm ci

  # Run tests in parallel with linting
  test:
    runs-on: ubuntu-latest
    needs: install  # Depends on install job
    steps:
      - uses: actions/checkout@v3
      - run: npm test

  # Run linting in parallel with tests
  lint:
    runs-on: ubuntu-latest
    needs: install  # Also depends on install job
    steps:
      - uses: actions/checkout@v3
      - run: npm run lint

Result: Testing and linting run in parallel after installing dependencies, cutting pipeline time significantly.

3. Set Up Distributed Caching

Caching saves your workflow from repeating expensive tasks:

# Cache dependencies to speed up builds
- name: Cache node modules
  uses: actions/cache@v3
  with:
    path: |
      ~/.npm           # Cache global npm cache
      node_modules     # Cache local dependencies
    key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}  # Key based on OS and dependency hash
    restore-keys: |    # Fallback keys if exact match isn't found
      ${{ runner.os }}-node-

Tip: Also cache build artifacts, Docker layers, and Terraform plans when possible.

4. Create Performance Benchmarks

Track your build times over time with benchmarks:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      # Store the start time as an environment variable
      - name: Start timer
        id: start_time
        run: echo "start_time=$(date +%s)" >> $GITHUB_ENV

      - uses: actions/checkout@v3
      - run: npm install

      # Calculate and display the elapsed time
      - name: End timer and calculate elapsed time
        run: |
          end_time=$(date +%s)
          elapsed_time=$((end_time - ${{ env.start_time }}))
          echo "Build time: $elapsed_time seconds"

With benchmarks in place, you can monitor regressions and trigger optimizations automatically.

5. How to Plan for Growth Beyond Free Tiers

• Understand cloud pricing structures: AWS, Azure, GCP all offer generous free tiers, but know the limits to avoid surprise bills. (I have been there and it wasn’t pretty.)

• Consider scaling to more advanced CI/CD tools: Jenkins, CircleCI, GitLab can offer better performance or self-hosted control as you grow.

• Automate resource provisioning: Use Infrastructure as Code (IaC) with Terraform, Pulumi, or AWS CDK to dynamically scale your infrastructure when your team or traffic grows.

Complete CI/CD Pipeline Example

Here’s a full example tying everything together:

# Complete end-to-end CI/CD pipeline
name: CI/CD Pipeline

on:
  push:
    branches:
      - main

jobs:
  # Initial setup job
  setup:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v3

  # Build and test job
  build:
    runs-on: ubuntu-latest
    needs: setup  # Depends on setup job
    steps:
      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '16'
      - name: Install dependencies
        run: npm install
      - name: Run security scan
        run: npx eslint .  # Run ESLint for security rules

  # Deploy to Kubernetes job
  deploy:
    runs-on: ubuntu-latest
    needs: build  # Depends on successful build
    steps:
      - name: Setup K3d cluster
        run: k3d cluster create dev-cluster --servers 1 --agents 2 --port 8080:80@loadbalancer
      - name: Apply Kubernetes manifests
        run: kubectl apply -f k8s/  # Apply all K8s manifests in the k8s directory
      - name: Deploy app
        run: kubectl rollout restart deployment/webapp  # Restart deployment for zero-downtime update

  # Infrastructure provisioning job
  terraform:
    runs-on: ubuntu-latest
    needs: deploy  # Run after deployment
    steps:
      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v2
      - name: Terraform Init
        run: terraform init  # Initialize Terraform
      - name: Terraform Apply
        run: terraform apply -auto-approve  # Apply infrastructure changes automatically

Runbook: Failed Deployment:

Issue: Pods fail due to resource limits (for example, OOMKilled, CrashLoopBackOff).
Fix:

  kubectl top pod
  kubectl edit deployment crud-app
  kubectl apply -f deployment.yaml
  kubectl rollout status deployment/crud-app

Tip: Set realistic resource requests and limits early, it'll save you debugging time later.

Conclusion

By following along with this tutorial, you now know how to build a production-ready DevOps pipeline using free tools:

• CI/CD: GitHub Actions for testing, linting, and building.

• Infrastructure: Terraform for AWS/Render and PostgreSQL setup.

• Orchestration: K3d for local Kubernetes.

• Monitoring: Grafana, Prometheus, UptimeRobot.

• Security: CodeQL, OWASP ZAP, Trivy for vulnerability scanning.

This pipeline is scalable and secure, and it’s perfect for small projects. As your app grows, you might want to consider paid plans for more resources (for example, AWS larger instances, Grafana unlimited metrics). You can check AWS Free Tier, Terraform Docs, and Grafana Docs for more learning.

PS: I’d love to see what you build. Share your pipeline on FreeCodeCamp’s forum or tag me on X @Emidowojo with #DevOpsOnABudget, and tell me about the challenges you faced. You can also connect with me on LinkedIn if you’d like to stay in touch. If you made it to the end of this lengthy article, thanks for reading!