Cropping allows users to remove these unwanted areas and focus only on the important content.

In this tutorial, you'll build a browser-based PDF Crop Tool using JavaScript.

Users will be able to upload a PDF, preview pages, select a crop area visually, apply crop settings to specific pages, generate a cropped PDF, preview the final result, and download the updated document directly from the browser.

Everything runs locally without requiring a backend server.

Table of Contents

• Why PDF Cropping Is Useful

• How PDF Cropping Works

• Project Setup

• What Library Are We Using?

• Creating the Upload Interface

• Previewing Uploaded PDF Pages

• Configuring Crop Settings

• Applying the Crop

• Generating the Cropped PDF

• Why PDF Cropping Is Useful in Real-World Documents

• Demo: Cropping PDF Files in the Browser

• Important Notes from Real-World Use

• Common Mistakes to Avoid

• Conclusion

Why PDF Cropping Is Useful

PDF cropping is commonly used when working with scanned documents, invoices, reports, contracts, forms, ebooks, manuals, presentations, and academic documents.

Many PDFs contain unnecessary whitespace or scanning artifacts around the edges of the page. Cropping helps remove distractions and makes documents easier to read.

Businesses often crop invoices and reports before sharing them with clients. Students crop lecture notes and scanned study materials to focus on the important content. And designers frequently crop exported PDFs to remove unwanted margins before printing or publishing.

Cropping also reduces visual clutter and creates a cleaner, more professional document.

How PDF Cropping Works

A PDF crop tool loads document pages inside the browser and allows users to define a rectangular crop area.

Once selected, the crop coordinates are applied to the chosen pages. The browser then generates a new PDF using only the selected content area.

Everything happens locally inside the browser. This means uploaded documents never leave the user's device, improving privacy and security.

Project Setup

This project is intentionally simple: you'll only need an HTML file, a JavaScript file, and a PDF processing library.

No backend server or database is required, as everything runs directly inside the browser.

What Library Are We Using?

We'll use PDF-lib for PDF processing.

PDF-lib allows us to load PDF documents, modify page boundaries, and export updated PDF files directly in JavaScript.

Add the library using a CDN:

<script src="https://unpkg.com/pdf-lib/dist/pdf-lib.min.js"></script>

Once loaded, JavaScript can process PDF pages directly inside the browser.

Creating the Upload Interface

Users first upload a PDF document into the browser.

A simple file input works well:

<input type="file" id="pdfInput" accept=".pdf">

JavaScript can detect when a file is selected:

document.getElementById("pdfInput").addEventListener("change", (event) => {
  const file = event.target.files[0];
  console.log(file.name);
});

Here's what the upload section looks like:

Previewing Uploaded PDF Pages

After uploading a document, users can preview PDF pages directly inside the browser.

The preview area includes page navigation controls that allow users to move between pages before cropping.

A default crop selection area is displayed on the preview page to help users begin selecting content immediately. This makes it easier to verify the document before applying crop settings.

Here's what the preview section looks like:

Configuring Crop Settings

After users select a crop area on the PDF preview, they often need more precise control over how the crop should be applied.

A practical PDF crop tool should allow users to manually adjust crop coordinates, choose predefined page ratios, and decide which pages should receive the crop operation.

This flexibility is especially useful when working with scanned documents, forms, reports, ebooks, presentations, and multi-page PDFs where different pages may require different crop settings.

In this project, users can adjust the crop position, control the crop dimensions, choose predefined crop ratios, and decide whether the crop should be applied to the current page, all pages, or a specific page range.

The crop settings panel provides complete control before generating the final PDF.

Here's what the crop settings section looks like:

Reading Crop Coordinates

When users drag a selection area on the preview page, the application records the crop dimensions.

A crop object typically contains:

const cropArea = {
  x: 173,
  y: 141,
  width: 452,
  height: 309
};

These values determine which portion of the page will remain visible after cropping.

Applying Custom Coordinates

Users can manually modify crop coordinates for more accurate results.

For example:

const left = parseInt(document.getElementById("cropX").value);
const top = parseInt(document.getElementById("cropY").value);
const width = parseInt(document.getElementById("cropWidth").value);
const height = parseInt(document.getElementById("cropHeight").value);

These values are later used when applying the crop to the PDF page.

Supporting Predefined Crop Ratios

Many users don't want to manually enter crop coordinates every time they crop a document.

In real-world situations, documents often need to follow standard page dimensions. Instead of adjusting the crop area manually, users can quickly choose a predefined ratio and let the tool apply the appropriate crop settings automatically.

For example, a user preparing documents for printing may choose an A4 layout, while someone working with presentation slides may prefer a landscape format. Other users may simply want to remove margins while keeping the original page proportions intact.

A simple example looks like this:

function applyA4Portrait() {
  cropArea = {
    x: 0,
    y: 0,
    width: 595,
    height: 842
  };
}

This allows users to instantly apply a standard page size.

Selecting Pages to Crop

Not every page requires cropping. Some users may only want to crop a single page while leaving the rest of the document unchanged.

The tool supports three page selection modes:

Current page only:

const applyMode = "current";

All pages:

const applyMode = "all";

Specific page ranges:

const applyMode = "specific";
const pageRange = "1,3-5,10";

This gives users full control over where the crop should be applied.

Applying Crop Settings to PDF Pages

Once the crop values are finalized, the selected pages can be updated using PDF-lib.

A simplified example looks like this:

const pages = pdfDoc.getPages();

pages.forEach((page) => {
  page.setCropBox(
    cropArea.x,
    cropArea.y,
    cropArea.width,
    cropArea.height
  );
});

The crop box defines the visible area that will remain in the generated PDF.

Validating Crop Values

Before applying the crop, it's important to verify that users entered valid dimensions.

For example:

if (
  cropArea.width <= 0 ||
  cropArea.height <= 0
) {
  alert("Invalid crop size");
  return;
}

Validation helps prevent errors and ensures the final PDF is generated correctly.

After the crop settings are configured, users can proceed to generate the updated PDF and review the results before downloading the final document.

Applying the Crop

Once the crop settings are configured, users can apply the crop operation.

For example:

page.setCropBox(x, y, width, height);

The selected crop area is applied to the chosen pages before generating the updated document.

Here's what the crop action section looks like:

Generating the Cropped PDF

After cropping is complete, the browser generates a new PDF document containing only the selected page areas.

For example:

const pdfBytes = await pdfDoc.save();

The updated file can then be previewed and downloaded.

Why PDF Cropping Is Useful in Real-World Documents

PDF cropping is one of those features that seems simple at first, but it becomes incredibly useful once you start working with real-world documents.

Many PDFs contain content that users don't actually need. Scanned documents often include large white borders created by scanners. Screenshots converted into PDFs may contain unnecessary background areas. Reports and presentations frequently have oversized margins that waste space and make documents harder to read.

By cropping unwanted areas, users can focus attention on the content that actually matters. The final document becomes cleaner, easier to read, more professional, and often more suitable for printing.

PDF cropping is especially valuable in business environments where documents are processed in large quantities.

For example, many e-commerce sellers on platforms such as Flipkart, Amazon, Meesho, and other marketplaces regularly download shipping labels, invoices, and packing slips in PDF format.

Imagine receiving a PDF containing 100 shipping labels for customer orders. The downloaded file may include unnecessary margins, extra whitespace, instructions, or content outside the area that needs to be printed.

Instead of manually editing every page, users can define a crop area once and apply the same crop settings to all pages in the document. This automatically removes unwanted content from all 100 labels in a single operation.

The result is a cleaner PDF that contains only the information required for printing and packaging.

The same workflow is useful for:

• E-commerce packing slips

• Warehouse barcode sheets

• Courier documentation

• Invoices and billing documents

• Scanned contracts and agreements

• Academic research papers

• Government forms

• Business reports and presentations

• Construction drawings and engineering documents

• Training manuals and internal company documentation

Cropping can also significantly improve printing efficiency. When unnecessary margins are removed, the important content occupies more of the printable area, making labels, invoices, diagrams, and reports easier to read.

Another common use case involves scanned paperwork. Many scanner applications automatically capture extra background around a document. Cropping removes these unwanted edges and produces a cleaner digital copy without requiring image-editing software.

Because the crop area can be applied to the current page, all pages, or specific page ranges, users can process large PDF documents in seconds rather than manually editing pages one by one.

For businesses that handle hundreds of PDF files every week, this can save a significant amount of time while producing cleaner, more professional documents ready for sharing, printing, or archiving.

Demo: How the PDF Crop Tool Works

Step 1: Upload a PDF File

Users begin by uploading a PDF document into the browser.

The upload area supports drag-and-drop functionality and manual file selection.

Step 2: Preview the Uploaded PDF

After uploading the document, the browser displays a page preview.

Users can move between pages using the navigation controls.

A default crop selection area is displayed to simplify the cropping process.

Step 3: Configure Crop Settings

Users can fine-tune crop coordinates, choose predefined ratios, and select which pages should receive the crop.

The crop can be applied to a single page, all pages, or specific page ranges.

Step 4: Apply the Crop

Once everything is configured, users click the Crop PDF button.

The browser processes the selected pages and applies the crop settings.

Step 5: Preview the Cropped PDF

After processing is complete, users can preview the cropped document.

Page navigation controls allow users to review every cropped page before downloading.

Step 6: Download the Cropped PDF

The final section displays the generated file.

Users can rename the document, review file details such as total pages and file size, and download the cropped PDF.

A Start Over button is also available for processing another file.

Important Notes from Real-World Use

When working with large PDF files, processing may take longer depending on the number of pages.

Always validate uploaded files before loading them.

For example:

if (!file.name.endsWith(".pdf")) {
  alert("Please upload a PDF file");
  return;
}

Previewing pages before downloading helps catch cropping mistakes early.

Common Mistakes to Avoid

One common mistake is selecting crop coordinates that remove important document content.

Another mistake is applying a crop to all pages when only specific pages should be modified.

For example:

if (!cropArea) {
  alert("Select a crop area first");
  return;
}

Always review the cropped preview before downloading the final document.

Conclusion

In this tutorial, you built a browser-based PDF Crop Tool using JavaScript.

You learned how to upload PDF files, preview pages, define crop areas, configure crop settings, apply cropping operations, generate updated PDFs, and download the final document directly from the browser.

More importantly, you saw how modern browsers can handle PDF editing tasks locally without requiring a backend server. This approach keeps document processing fast, private, and easy to use.

If you'd like to see a working example, try the AllinoneTools- PDF Crop Tool and explore how PDF pages can be cropped directly in the browser.

Once you understand this workflow, you can extend it further with features like PDF rotation, page organization, watermarking, metadata editing, annotations, and advanced PDF editing tools.

Modern browsers make this much easier than before.

Instead of uploading documents to a server, we can process PDF files directly inside the browser using JavaScript. This keeps the tool fast, private, and easy to use.

In this tutorial, you’ll build a browser-based PDF to image converter using JavaScript.

The tool will support uploading PDF files, previewing pages, selecting image formats like JPG or PNG, adjusting image quality, and downloading converted images directly from the browser.

Everything runs entirely client-side without any backend.

Table of Contents

1. How PDF to Image Conversion Works

2. Project Setup

3. What Library Are We Using?

4. Creating the Upload Interface

5. Reading the PDF File

6. Rendering PDF Pages as Images

7. Selecting Image Format and Quality

8. Generating and Downloading Images

9. Demo: How the PDF to Image Tool Works

10. Important Notes from Real-World Use

11. Common Mistakes to Avoid

12. Conclusion

How PDF to Image Conversion Works

A browser can't directly convert PDF files into images on its own.

Instead, JavaScript libraries render PDF pages onto an HTML canvas, which can then be exported as image files like JPG or PNG.

The process starts when users upload a PDF document into the browser. JavaScript then reads the file, renders each PDF page visually onto a canvas, converts those rendered pages into image files, and finally makes them available for download.

Everything happens locally inside the browser.

This means users don't need to upload private documents to external servers, making the process faster and more privacy-friendly.

Project Setup

This project is intentionally simple. Everything runs directly inside the browser using JavaScript, so no backend or server setup is required.

You only need:

• an HTML file

• a JavaScript file

• the PDF.js library

What Library Are We Using?

We’ll use Mozilla’s PDF.js library to render PDF pages inside the browser.

Add it using a CDN:

<script src="https://cdnjs.cloudflare.com/ajax/libs/pdf.js/3.11.174/pdf.min.js"></script>

Once loaded, the browser can read and render PDF pages directly using JavaScript.

Creating the Upload Interface

Start with a simple upload area:

<input type="file" id="pdfUpload" accept="application/pdf">

<select id="format">
  <option>JPG</option>
  <option>PNG</option>
  <option>WEBP</option>
</select>

<input type="range" id="quality" min="10" max="100" value="90">

<button onclick="convertPDF()">
  Convert to Images
</button>

This allows users to upload PDF files directly into the browser.

Here’s what the upload section looks like inside the tool:

Reading the PDF File

After the file is uploaded, we need to read it using JavaScript.

For example:

const file = document.getElementById("pdfUpload").files[0];

const reader = new FileReader();

reader.onload = async function () {
  const typedArray = new Uint8Array(reader.result);

  const pdf = await pdfjsLib.getDocument(typedArray).promise;

  console.log(pdf.numPages);
};

reader.readAsArrayBuffer(file);

This loads the PDF document directly inside the browser.

You can then access each page individually.

Rendering PDF Pages as Images

Once the PDF is loaded, pages can be rendered onto a canvas.

For example:

const page = await pdf.getPage(1);

const viewport = page.getViewport({ scale: 2 });

const canvas = document.createElement("canvas");

const context = canvas.getContext("2d");

canvas.width = viewport.width;
canvas.height = viewport.height;

await page.render({
  canvasContext: context,
  viewport: viewport
}).promise;

This renders the selected PDF page visually inside the browser.

After rendering, the canvas can be converted into an image.

For example:

const imageData = canvas.toDataURL("image/jpeg", 0.9);

This creates a downloadable image version of the PDF page.

Selecting Image Format and Quality

Before generating the final images, users may want to customize output settings.

Different image formats work better for different situations.

For example:

• JPG works well for smaller file sizes

• PNG preserves better quality

• WEBP offers modern compression

Users can also control image quality using a slider.

For example:

canvas.toDataURL("image/jpeg", 0.8);

The value 0.8 controls compression quality.

Here’s an example of image format and quality settings inside the tool:

Generating and Downloading Images

Once pages are rendered, images can be downloaded directly from the browser.

For example:

const link = document.createElement("a");

link.href = imageData;

link.download = `page-${pageNumber}.jpg`;

link.click();

This downloads the generated image instantly.

When working with multi-page PDFs, the same process can run for every page automatically.

This allows users to export complete PDF documents as separate image files.

Demo: How the PDF to Image Tool Works

For this example, we’ll convert PDF pages into downloadable image files directly inside the browser.

Step 1: Upload PDF Files

Users upload one or more PDF files into the converter.

Step 2: Preview Uploaded Pages

The tool generates page previews before conversion.

This helps users verify the uploaded document visually.

Step 3: Configure Output Settings

Users can choose image format and quality settings before generating images.

This allows better control over output size and image clarity.

Step 4: Convert PDF Pages into Images

Once settings are configured, users click the convert button.

The browser processes the PDF locally and generates image files instantly.

Step 5: Download Generated Images

After conversion, every PDF page becomes a downloadable image.

Important Notes from Real-World Use

When working with large PDFs, performance and memory usage become important.

Documents with many pages can slow down rendering if everything is processed at once.

One practical optimization is processing pages step-by-step instead of rendering the entire document immediately.

For example:

for (let i = 1; i <= pdf.numPages; i++) {
  const page = await pdf.getPage(i);

  // render page
}

This keeps browser memory usage more stable.

Another useful optimization is reducing render scale for large documents.

For example:

const viewport = page.getViewport({
  scale: 1.5
});

Lower scale values generate smaller image files and improve performance.

You can also resize generated images before export.

For example:

canvas.width = viewport.width;
canvas.height = viewport.height;

This helps reduce unnecessary file size growth.

Since everything runs locally inside the browser, uploaded PDF files never leave the user’s device, which improves privacy and security.

Common Mistakes to Avoid

One common mistake is not validating uploaded files before processing them.

For example:

if (!file || file.type !== "application/pdf") {
  alert("Please upload a valid PDF file.");
  return;
}

This prevents unsupported files from breaking the tool.

Another issue is rendering extremely large pages at very high scale values.

Large canvas rendering can consume a lot of memory and slow down conversion significantly.

Using smaller scale values usually improves performance.

Another common mistake is forgetting to wait for page rendering before exporting the image.

For example:

await page.render({
  canvasContext: context,
  viewport: viewport
}).promise;

Without await, the image may export before rendering finishes.

Incorrect file naming can also confuse users when multiple pages are generated.

Adding page numbers to filenames improves organization:

link.download = `page-${pageNumber}.jpg`;

Conclusion

In this tutorial, you built a browser-based PDF to image converter using JavaScript.

You learned how to upload PDF files, render pages inside the browser, generate images, and download them directly without using a backend server.

More importantly, you saw how modern browsers can handle document processing tasks locally while keeping user files private.

This approach keeps the tool fast, lightweight, and easy to use.

Once you understand this workflow, you can extend it further with features like ZIP downloads, batch exports, page selection, watermarking, or image compression.

You can also try a real working version here:

https://allinonetools.net/pdf-to-image-converter/

And that’s where things start getting really interesting.

Most developers either rely on external tools or assume this requires backend processing. That’s usually where things get slower, more complex, and harder to maintain.

But modern browsers have quietly become powerful enough to handle this entirely on their own.

In this tutorial, you’ll build a barcode generator that runs completely in the browser. It won’t upload data anywhere, and it won’t require any server logic. Everything happens instantly on the client side.

Along the way, you’ll also learn how barcode formats work, how to validate inputs properly, and how to create a real-time preview experience that feels responsive and practical.

Table of Contents

1. How Barcode Generation Works

2. Project Setup

3. What Library Are We Using?

4. Creating the HTML Structure

5. Adding JavaScript for Barcode Generation

6. How the Barcode Is Generated

7. Types of Barcodes You Can Generate

8. Adding Real-Time Preview

9. How to Validate Input Properly

10. How to Download the Barcode

11. Important Notes from Real-World Use

12. Common Mistakes to Avoid

13. Demo: How the Barcode Generator Works

14. Conclusion

How Barcode Generation Works

A barcode is simply a visual encoding of data. Instead of displaying text directly, it represents that data using a pattern of lines and spaces.

Different barcode formats use different encoding rules. Some support only numbers, while others allow full text input. When you generate a barcode in the browser, you’re essentially converting user input into a structured visual pattern.

The key idea here is that we don’t draw these lines manually. A library takes care of encoding the data and rendering it as an SVG element, which the browser can display instantly.

Project Setup

We’ll keep this project intentionally simple so the focus stays on understanding how it works.

All you need is a basic HTML file, a small JavaScript file, and a barcode library. There’s no backend involved, and nothing gets stored or uploaded.

This makes the tool fast, private, and easy to integrate into other projects.

What Library Are We Using?

In this project, we use the JsBarcode library.

It’s a lightweight JavaScript library that can generate barcodes directly inside the browser using SVG. It supports multiple formats and works without any external dependencies.

You can include it using a CDN:

<script src="https://cdn.jsdelivr.net/npm/jsbarcode@3.11.5/dist/JsBarcode.all.min.js"></script>

Creating the HTML Structure

The interface is simple but practical. It includes an input field where users can enter data, a dropdown to choose the barcode format, and a preview area where the barcode is rendered.

<input type="text" id="text" placeholder="Enter text or number">

<select id="format">
  <option value="CODE128">Code128</option>
  <option value="EAN13">EAN13</option>
</select>

<button onclick="generateBarcode()">Generate</button>

<svg id="barcode"></svg>

This structure is enough to handle input, display output, and connect everything through JavaScript.

Adding JavaScript for Barcode Generation

Now we'll connect the user input to barcode generation.

function generateBarcode() {
  const text = document.getElementById("text").value;
  const format = document.getElementById("format").value;

  if (!text) {
    alert("Please enter a value");
    return;
  }

  JsBarcode("#barcode", text, {
    format: format,
    width: 2,
    height: 100,
    displayValue: true
  });
}

This function reads the input, checks if it exists, and then generates the barcode using the selected format.

How the Barcode Is Generated

When you call the JsBarcode function, the library handles everything behind the scenes.

It encodes the input into a barcode standard, converts that into a pattern of lines, and renders it as an SVG element. Because SVG is vector-based, the barcode remains sharp even when resized.

All of this happens instantly in the browser, which is why the experience feels fast.

Types of Barcodes You Can Generate

Different barcode formats are used in different industries, and understanding them helps you build more practical tools.

1. Code128 is the most flexible format. It supports letters, numbers, and special characters, which makes it ideal for general-purpose use.

2. EAN-13 is commonly used in retail products. It works only with 13-digit numbers, so it requires strict validation.

3. UPC is similar to EAN and is widely used in billing systems, especially in the US. It also expects numeric input with a fixed length.

4. Code39 is simpler and supports uppercase letters and numbers, but it’s less compact compared to Code128.

5. ITF-14 is mostly used in logistics and packaging. It’s designed for numeric data and is common in shipping environments.

In most cases, starting with Code128 is the safest option unless you have a specific requirement.

Adding Real-Time Preview

One of the biggest improvements you can make to a tool like this is real-time feedback.

Instead of requiring users to click a button every time, you can generate the barcode as they type.

document.getElementById("text").addEventListener("input", generateBarcode);
document.getElementById("format").addEventListener("change", generateBarcode);

This small change makes the tool feel much more responsive.

As soon as the user types or changes the format, the barcode updates automatically. This is the same kind of interaction you see in polished production tools.

How to Validate Input Properly

Validation is where many simple tools break.

Since different barcode formats have different rules, if you don’t validate input correctly, the barcode may fail silently or produce incorrect output.

Here’s a simple example:

function isValidInput(text, format) {
  if (format === "EAN13") {
    return /^\d{13}$/.test(text);
  }

  if (format === "UPC") {
    return /^\d{12}$/.test(text);
  }

  return text.length > 0;
}

Then use it inside your generator:

if (!isValidInput(text, format)) {
  alert("Invalid input for selected format");
  return;
}

This ensures users get immediate feedback instead of confusion.

How to Download the Barcode

Once the barcode is generated, you can allow users to download it.

function downloadBarcode() {
  const svg = document.getElementById("barcode");
  const serializer = new XMLSerializer();
  const source = serializer.serializeToString(svg);

  const blob = new Blob([source], { type: "image/svg+xml" });
  const url = URL.createObjectURL(blob);

  const link = document.createElement("a");
  link.href = url;
  link.download = "barcode.svg";
  link.click();
}

This converts the SVG into a file that can be downloaded directly from the browser.

Important Notes from Real-World Use

When building tools like this in production, small details matter.

Large input values can sometimes affect readability, so it’s important to test how dense the barcode becomes. Choosing the right format also makes a difference depending on whether you need flexibility or strict standards.

Another important detail is rendering quality. Using SVG instead of raster formats ensures that the barcode remains sharp even when printed.

Common Mistakes to Avoid

One common issue is skipping validation. This leads to broken or unreadable barcodes, especially with strict formats like EAN or UPC.

Another mistake is relying too much on button-based interactions. Real-time updates create a much better user experience.

Finally, developers sometimes forget to include the library correctly, which leads to silent failures. Always verify that your CDN is loaded.

Demo: How the Barcode Generator Works

To better understand how everything comes together, here’s a quick walkthrough of how the tool works in the browser.

Step 1: Select a Barcode Type

Start by choosing the barcode format. In most cases, Code128 is a good default since it supports both text and numbers.

Step 2: Enter Your Data

Next, enter the value you want to encode. This could be a product ID, URL, or any text depending on the selected format.

Step 3: Customize the Design

You can adjust things like bar width, height, and colors. These settings help control how the barcode looks and how readable it is in different use cases.

Step 4: Generate and Preview

As you type or change settings, the barcode updates instantly. This real-time preview makes it easier to experiment and see results immediately.

Step 5: Download the Barcode

Once you're satisfied with the result, you can download the barcode in formats like PNG, JPG, or SVG.

This entire process happens in the browser, without uploading any data to a server.

Conclusion

In this tutorial, you built a browser-based barcode generator using JavaScript.

More importantly, you learned how to think about building tools that run entirely on the client side. This approach reduces complexity, improves performance, and gives users a faster experience.

Once you understand this pattern, you can apply it to many other tools like QR generators, image converters, and file processors.

And that’s where things start to get interesting.

Most developers use online tools for this. But there’s a problem: many of those tools upload your image to a server. That can be slow, and sometimes you don’t want to upload private files at all.

The good news is that modern browsers are powerful enough to handle image conversion locally using JavaScript.

In this tutorial, you’ll learn how to build a browser-based image converter that runs entirely in the browser. The tool converts images using JavaScript without uploading files to a server, and allows users to download the converted file instantly.

By the end, you’ll understand how browser-based file processing works and how to use it in your own projects.

Table of Contents

1. How Browser-Based Image Conversion Works

2. Project Setup

3. How to Read the Image File in JavaScript

4. How the Canvas Converts the Image

5. How the Download Works

6. Why This Approach Is Powerful

7. Important Notes from Real-World Use

8. Common Mistakes to Avoid

9. How You Can Extend This Project

10. Why Browser-Based Tools Are Becoming More Popular

11. Demo: How the Image Converter Works

12. Conclusion

How Browser-Based Image Conversion Works

Before writing code, you should understand what’s happening behind the scenes.

Modern browsers provide several APIs that make this possible. JavaScript can read local files from a user’s device, draw images on a canvas element, and export the processed image in a different format.

The key pieces we’ll use are:

• File input – to select an image

• FileReader – to read the file

• Canvas API – to redraw and convert

• toDataURL or toBlob – to export the converted image

The important thing is that everything happens locally in the user’s browser. Nothing gets uploaded anywhere.

Project Setup

We’ll keep this simple with just HTML and JavaScript.

Create an index.html file:

<!DOCTYPE html>
<html>
<head>
  <title>Image Converter</title>
</head>
<body>

<h2>Browser Image Converter</h2>

<input type="file" id="upload" accept="image/*">

<select id="format">
  <option value="image/png">PNG</option>
  <option value="image/jpeg">JPEG</option>
  <option value="image/webp">WebP</option>
</select>

<button onclick="convertImage()">Convert</button>

<br><br>

<a id="download" style="display:none;">Download Converted Image</a>

<script src="script.js"></script>

</body>
</html>

This simple interface includes a file upload input for selecting the image, a format selector for choosing the output format, a convert button to start the process, and a download link that appears once the image has been converted.

Now let’s add the logic.

How to Read the Image File in JavaScript

Create a script.js file:

function convertImage() {

  const fileInput = document.getElementById("upload");
  const format = document.getElementById("format").value;

  if (!fileInput.files.length) {
    alert("Please select an image");
    return;
  }

  const file = fileInput.files[0];
  const reader = new FileReader();

  reader.onload = function(event) {

    const img = new Image();

    img.onload = function() {

      const canvas = document.createElement("canvas");
      const ctx = canvas.getContext("2d");

      canvas.width = img.width;
      canvas.height = img.height;

      ctx.drawImage(img, 0, 0);

      const converted = canvas.toDataURL(format);

      const link = document.getElementById("download");

      link.href = converted;
      link.download = "converted-image";
      link.style.display = "inline";
      link.innerText = "Download Converted Image";

    };

    img.src = event.target.result;

  };

  reader.readAsDataURL(file);

}

This is the core of the image converter. Let’s break down what’s happening.

How the Canvas Converts the Image

This line draws the image:

ctx.drawImage(img, 0, 0);

Now the image exists inside the canvas.

This line converts it:

canvas.toDataURL(format);

This exports the image in the selected format.

For example:

• PNG → image/png

• JPEG → image/jpeg

• WebP → image/webp

This is where the conversion actually happens.

How the Download Works

This part creates the download:

link.href = converted;
link.download = "converted-image";

The browser treats it as a downloadable file. No server needed.

Why This Approach Is Powerful

This technique has several advantages.

• It’s fast: There is no upload time, and everything runs locally.

• It’s private: Files never leave the user’s device. This matters for sensitive images.

• It reduces server costs: You don’t need backend processing. No storage, and no processing servers.

Important Notes from Real-World Use

If you plan to build tools like this, here are a few practical things I’ve learned.

Large Images Use More Memory

Very large images can slow down the browser. If needed, you can resize images using Canvas.

JPEG Supports Quality Settings

You can control quality:

canvas.toDataURL("image/jpeg", 0.8);

This reduces file size.

WebP Usually Gives the Best Compression

WebP often produces smaller files than PNG or JPEG. It’s a good default option.

How to Resize an Image Using Canvas

If you need to reduce the size of large images, you can resize them before exporting.

After loading the image, you can set a smaller width and height on the canvas:

const canvas = document.createElement("canvas");
const ctx = canvas.getContext("2d");

const maxWidth = 800;
const scale = maxWidth / img.width;

canvas.width = maxWidth;
canvas.height = img.height * scale;

ctx.drawImage(img, 0, 0, canvas.width, canvas.height);

Common Mistakes to Avoid

Trying to Upload Files Unnecessarily

If processing can happen in the browser, do it there. It’s faster and simpler.

Forgetting Browser Compatibility

Most modern browsers support Canvas and FileReader. But always test.

Not Validating File Input Properly

Before processing the image, it’s important to validate the input file.

For example, you can check if a file is selected and ensure it is an image:

const file = fileInput.files[0];

if (!file) {
  alert("Please select a file.");
  return;
}

if (!file.type.startsWith("image/")) {
  alert("Please upload a valid image file.");
  return;
}

How You Can Extend This Project

Once this basic converter works, you can expand it with additional features. For example, you could add image resizing so users can adjust dimensions before downloading the converted file. Another useful improvement is implementing drag-and-drop uploads, which makes the interface more user-friendly.

You might also support multiple file uploads so users can convert several images at once. Adding compression controls would allow users to balance image quality and file size. Finally, you could include an image preview before download so users can confirm the result before saving the file.

All of these improvements rely on the same browser APIs used in this tutorial, so once you understand the core logic, extending the project becomes much easier.

Why Browser-Based Tools Are Becoming More Popular

Browsers today are far more capable than they used to be. Modern browser APIs allow developers to handle tasks that previously required server-side processing.

For example, browsers can now perform image processing, generate PDFs, convert files into different formats, and even handle some types of video processing directly on the client side.

Because of these capabilities, developers can build tools that run entirely inside the browser without relying on a backend server. This approach improves performance since users don’t have to upload files and wait for a server to process them.

It also improves privacy because files stay on the user’s device instead of being sent to a remote server. At the same time, it simplifies system architecture and makes applications easier to scale since there is no server infrastructure needed for file processing.

Demo: How the Image Converter Works

After building the project, here is what the tool looks like in the browser.

Upload an Image

First, the user uploads an image using the file upload area.

Select the Output Format

After uploading the image, the tool displays a preview along with details such as the image name, format, and file size. This helps users confirm that they uploaded the correct file before converting it.

Next, the user can choose the desired output format from the dropdown menu. The tool supports formats such as PNG, JPEG, WebP, GIF, and BMP, allowing the image to be converted into the format that best fits the user's needs.

Convert the Image

Once the format is selected, clicking the Convert All Images button processes the image directly in the browser.

Download the Converted Image

After conversion is complete, the tool generates a downloadable file.

Conversion Results

The tool can also display useful information such as original size, converted size, and space saved after compression.

Because everything happens in the browser using JavaScript and the Canvas API, the image never leaves the user's device.

Conclusion

In this tutorial, you built a browser-based image converter using JavaScript.

In this tutorial, you learned how to read local image files using JavaScript, process images using the Canvas API, convert them into different formats, and allow users to download the result directly from the browser.

This pattern is useful far beyond image conversion.

You can use the same approach for many browser-based tools.

Understanding how to use browser APIs like this opens up a lot of possibilities for building fast, efficient web applications.

Most people think Jupyter is just for Python and data science stuff, but apparently you can run Rust in one, too.

In this tutorial, we’ll be taking a look at:

1. What is EvCxR?

2. How to Install the Rust Jupyter kernel

3. How to run your first Rust code in a notebook

4. Handy Tips and Tricks

5. Common Issues and Solutions

6. When NOT to Use Jupyter for Rust

Friendly Disclaimer: This tutorial assumes you know the basics of both Rust and Jupyter. If you break something, that's on you, mate 🙂.

So without further ado, let's jump in.

What is EvCxR?

EvCxR (pronounced "Evaluator" to my fellow linguists’ horror) is a Rust REPL and Jupyter kernel. It's basically the magic that lets you run Rust code interactively in Jupyter notebooks instead of the traditional compile-run-debug cycle.

The name stands for "Evaluation Context for Rust", and it’s an open source project actively maintained on GitHub. Here are a few things that make this terribly named tool absolutely brilliant:

1. Interactive development: It lets you test Rust snippets without creating a whole project 🧪

2. Prototyping: You can quickly try out ideas before committing to a full implementation 💡

3. Data visualisation: And yes, you can even plot charts with Rust (more on that later) 📊

How to Install the Rust Jupyter kernel

Prerequisites

Before we dive into the installation, make sure you have these sorted:

1. A Linux System: Or at least, Windows Subsystem for Linux (There’s a little note below for Windows users.)

2. The Rust toolchain: You can get it from rustup.rs if you haven't already

3. Jupyter: Install via pip – pip install jupyter

4. Patience: This might take a minute or two ⏱️

Once you’ve got all that, we can get rusty (pun intended).

Note: If you’re using Windows, you’ll need to do a little extra to get started. Here’s the quick rundown:

1. Go to https://visualstudio.microsoft.com/visual-cpp-build-tools/

2. Download and run the installer

3. Select "Desktop development with C++"

4. Install it (it's large, ~5GB)

Step 1: Install EvCxR

Open your terminal and run this command:

cargo install evcxr_jupyter

Now go grab a cup of joe ☕. This will take a few minutes as Cargo downloads and compiles everything. And don't panic if it seems stuck. Rust compilation is thorough but not particularly fast.

If you get any errors about missing system libraries, you might need to install some dependencies. On Ubuntu/Debian, try:

sudo apt install jupyter-notebook jupyter-core python-ipykernel
sudo apt install cmake

On macOS with Homebrew:

brew install cmake jupyter

Step 2: Install the Jupyter Kernel

Once the installation finishes, you’ll need to register the EvCxR kernel with Jupyter:

evcxr_jupyter --install

You should see output that looks something like this at the end:

Installation complete

Step 3: Launch Jupyter and Create a Rust Notebook

Let’s test out our baby. Fire up Jupyter:

jupyter notebook

Your browser should open automatically (if it doesn't, copy the URL from the terminal).

In the Jupyter interface:

1. Click New in the top right

2. Select Rust from the dropdown (or "evcxr" depending on your version)

3. A new notebook opens

Welcome to interactive Rust! 🦀

Step 4: Write Your First Rust Code

Let's start with a classic:

println!("Hello my fellow Rustaceans! 🦀");

Hit Shift + Enter to run the cell. You should see the output appear below the cell. Simple as that.

Note that notebooks execute code at the top level, so you don’t have to wrap it around the main() function. If you still want to do that, you’re going to have to call it like this:

fn main(){
    println!("Hello my fellow Rustaceans! 🦀");
}
//Calling the function
main()

Now let's try something more interesting:

fn fibonacci(n: u32) -> u32 {
    match n {
        0 => 0,
        1 => 1,
        _ => fibonacci(n - 1) + fibonacci(n - 2)
    }
}

for i in 0..10 {
    println!("fibonacci({}) = {}", i, fibonacci(i));
}

Run it and watch the Fibonacci sequence appear.

fibonacci(0) = 0
fibonacci(1) = 1
fibonacci(2) = 1
fibonacci(3) = 2
fibonacci(4) = 3
fibonacci(5) = 5
fibonacci(6) = 8
fibonacci(7) = 13
fibonacci(8) = 21
fibonacci(9) = 34

Handy Tips and Tricks

Functions aren’t the only things that behave differently when using Rust in notebooks. Here are a few other things you might want to keep in mind:

Variables Persist Between Cells

Unlike traditional Rust compilation, variables you define in one cell stick around for the next cells:

let mut counter = 0;

Then in the next cell:

counter += 1;
println!("Counter: {}", counter);

The output would be:

Counter: 1

This is great for building up complex examples step by step.

You Can Use External Crates

Add dependencies with the :dep command in one cell:

:dep serde = { version = "1.0", features = ["derive"] }
:dep serde_json = "1.0"

Then use them normally in the next:

use serde::{Serialize, Deserialize};

#[derive(Serialize, Deserialize, Debug)]
struct Person {
    name: String,
    age: u32,
}

let person = Person {
    name: "Amina".to_string(),
    age: 24,
};

let json = serde_json::to_string(&person).unwrap();
println!("{}", json);

Output:

{"name":"Amina","age":24}

Pretty neat, huh?

Visualisation Support

You can even create graphs. To get started, install the plotters crate:

:dep plotters = { version = "0.3", default-features = false, features = ["evcxr", "all_series", "bitmap_backend", "bitmap_encoder"] }

Then create a simple sine graph:

use plotters::prelude::*;

let root = SVGBackend::new("sine_wave.svg", (640, 480)).into_drawing_area();
root.fill(&WHITE).unwrap();

let mut chart = ChartBuilder::on(&root)
    .caption("Sine Wave", ("Arial", 20))
    .margin(5)
    .x_label_area_size(30)
    .y_label_area_size(30)
    .build_cartesian_2d(-3.14..3.14, -1.2..1.2)
    .unwrap();

chart.configure_mesh().draw().unwrap();

chart.draw_series(LineSeries::new(
    (-314..314).map(|x| {
        let x = x as f64 / 100.0;
        (x, x.sin())
    }),
    &RED,
)).unwrap();

root.present().unwrap();
println!("Plot saved to sine_wave.svg");

Output:

A word on plotting: You can actually display plots directly inline in your notebook. But if you're using WSL with VSCode (like I do), inline plotting may not work properly due to rendering issues on the notebook interface. That’s why I used it as an svg file that I can easily view in my text editor.

Checking Types

Not sure what type something is? Use :vars. This shows all variables and their types:

let x = vec![1, 2, 3];

:vars

Output:

Variable	    Type
       x	Vec<i32>

Common Issues and Solutions

Compilation Errors Everywhere

If you're getting weird compilation errors, remember:

• Each cell is compiled separately

• You might need to reimport things in each cell

Slow Execution

The first time you run code in a session, it's slow due to the compilation overhead. Subsequent runs are faster. If it's really slow, you might want to:

• Use release mode: :opt 2

• Reduce dependency features to only what you need

• Consider if Jupyter is the right tool for your use case

Dependencies Not Loading

If a crate won't load:

• Make sure the version exists on crates.io

• Check your internet connection (it needs to download)

• Try specifying features explicitly

• Clear the cargo cache if things get really wonky: rm -rf ~/.evcxr

When NOT to Use Jupyter for Rust

Jupyter notebooks are great for learning and experimenting, but they're not always the best choice in:

• Production code: Use proper projects with cargo

• Performance-critical code: The overhead isn't worth it

• Large applications: Notebooks get very messy, very fast

• Team collaboration: Version control with notebooks is quite the nightmare

Stick to notebooks for prototyping and quick experiments. For anything serious, fire up your favourite editor and create a proper Rust project.

Conclusion

Let's summarise what you've learned:

1. How to install the EvCxR Jupyter kernel

2. How to create and run Rust notebooks

3. How to use external crates in notebooks

4. Tips and tricks for interactive Rust development

Jupyter notebooks make Rust more accessible for learning and experimentation. Give it a go next time you want to try out a quick Rust snippet without the ceremony of creating a full project. And with that, we've come to the end of this tutorial.

Cheers.

Resources

1. EvCxR GitHub Repository

2. Rust Book

3. Jupyter Documentation

Acknowledgements

Thanks to Anuoluwapo Victor, Chinaza Nwukwa, Holumidey Mercy, Favour Ojo, Georgina Awani, and my family for the inspiration, support and knowledge used to put this post together.

And thanks to the EvCxR project maintainers for making this possible, the Rust community for being awesome, and to anyone reading this for wanting to learn. You inspire me daily.

This is where Transformers come in. Originally built for language, they’ve become state-of-the-art in vision tasks thanks to models like the Vision Transformer (ViT) and video-focused variants such as TimeSformer.

In this tutorial, we’ll use a Transformer backbone to create a lightweight real-time gesture recognition tool, optimized for small datasets and deployable on a regular laptop webcam.

Table of Contents

• Why Transformers for Gestures?

• What You’ll Learn

• Prerequisites

• Project Setup

• Generate a Gesture Dataset

• Option 1: Generate a Synthetic Dataset

• Training Script: train.py

• Export the Model to ONNX

• Evaluate Accuracy + Latency

• Option 2: Use Small Samples from Public Gesture Datasets

• Accessibility Notes & Ethical Limits

• Next Steps

• Conclusion

Why Transformers for Gestures?

Transformers are powerful because they use self-attention to model relationships across a sequence. For gestures, this means the model doesn’t just see isolated frames, but also learns how movements evolve over time. A wave, for example, looks different from a raised hand only when viewed as a sequence.

Vision Transformers process images as patches, while video Transformers extend this to multiple frames with temporal attention. Even a simple approach, like applying ViT to each frame and pooling across time, can outperform traditional CNN-based methods for small datasets.

Combined with Hugging Face’s pre-trained models and ONNX Runtime for optimization, Transformers make it possible to train on a modest dataset and still achieve smooth real-time recognition.

What You’ll Learn

In this tutorial, you’ll build a gesture recognition system using Transformers. By the end, you’ll know how to:

• Create (or record) a tiny gesture dataset

• Train a Vision Transformer (ViT) with temporal pooling

• Export the model to ONNX for faster inference

• Build a real-time Gradio app that classifies gestures from your webcam

• Evaluate your model’s accuracy and latency with simple scripts

• Understand the accessibility potential and ethical limits of gesture recognition

Prerequisites

To follow along, you should have:

• Basic Python knowledge (functions, scripts, virtual environments)

• Familiarity with PyTorch (tensors, datasets, training loops) – helpful but not required

• Python 3.8+ installed on your system

• A webcam (for the live demo in Gradio)

• Optionally: GPU access (training on CPU works, but is slower)

Project Setup

Create a new project folder and install the required libraries.

# Create a new project directory and navigate into it
mkdir transformer-gesture && cd transformer-gesture

# Set up a Python virtual environment
python -m venv .venv

# Activate the virtual environment
# Windows PowerShell
.venv\Scripts\Activate.ps1

# macOS/Linux
source .venv/bin/activate

The provided code snippet is a set of commands for setting up a new Python project with a virtual environment. Here's a breakdown of each part:

1. mkdir transformer-gesture && cd transformer-gesture: This command creates a new directory named "transformer-gesture" and then navigates into it.

2. python -m venv .venv: This command creates a new virtual environment in the current directory. The virtual environment is stored in a folder named ".venv".

3. Activating the virtual environment:

   • For Windows PowerShell, you can use .venv\Scripts\Activate.ps1 to activate the virtual environment.

   • For macOS/Linux, use source .venv/bin/activate to activate the virtual environment.

Activating a virtual environment ensures that the Python interpreter and any packages you install are isolated to this specific project, preventing conflicts with other projects or system-wide packages.

Create a requirements.txt file:

torch>=2.0
torchvision
torchaudio
timm
huggingface_hub

onnx
onnxruntime

gradio

numpy
opencv-python
pillow

matplotlib
seaborn
scikit-learn

The list provided is a set of package dependencies typically found in a requirements.txt file for a Python project. Here's a brief explanation of each package:

1. torch>=2.0: PyTorch is a popular open-source deep learning framework that provides a flexible and efficient platform for building and training neural networks. Version 2.0 and above includes improvements in performance and new features.

2. torchvision: This library is part of the PyTorch ecosystem and provides tools for computer vision tasks, including datasets, model architectures, and image transformations.

3. torchaudio: Also part of the PyTorch ecosystem, Torchaudio provides audio processing tools and datasets, making it easier to work with audio data in deep learning projects.

4. timm: The PyTorch Image Models (timm) library offers a collection of pre-trained models and utilities for computer vision tasks, facilitating quick experimentation and deployment.

5. huggingface_hub: This library allows easy access to models and datasets hosted on the Hugging Face Hub, a platform for sharing and collaborating on machine learning models and datasets.

6. onnx: The Open Neural Network Exchange (ONNX) format is used to represent machine learning models, enabling interoperability between different frameworks.

7. onnxruntime: This is a high-performance runtime for executing ONNX models, allowing for efficient deployment across various platforms.

8. gradio: Gradio is a library for creating user interfaces for machine learning models, making them accessible through a web interface for easy interaction and testing.

9. numpy: A fundamental package for numerical computing in Python, providing support for arrays and a wide range of mathematical functions.

10. opencv-python: OpenCV is a library for computer vision and image processing tasks, widely used for real-time applications.

11. pillow: A Python Imaging Library (PIL) fork, Pillow provides tools for opening, manipulating, and saving many different image file formats.

12. matplotlib: A plotting library for Python, Matplotlib is used for creating static, interactive, and animated visualizations in Python.

13. seaborn: Built on top of Matplotlib, Seaborn provides a high-level interface for drawing attractive and informative statistical graphics.

14. scikit-learn: A machine learning library in Python that provides simple and efficient tools for data analysis and modeling, including classification, regression, clustering, and dimensionality reduction.

Install dependencies:

pip install -r requirements.txt

The command pip install -r requirements.txt is used to install all the Python packages listed in a file named requirements.txt. This file typically contains a list of package dependencies required for a Python project, each specified with a package name and optionally a version number.

By running this command, pip, which is the Python package installer, reads the file and installs each package listed, ensuring that the project has all the necessary dependencies to run properly. This is a common practice in Python projects to manage and share dependencies easily.

Generate a Gesture Dataset

To train our Transformer-based gesture recognizer, we need some data. Instead of downloading a huge dataset, we’ll start with a tiny synthetic dataset you can generate in seconds. This makes the tutorial lightweight and ensures that everyone can follow along without dealing with multi-gigabyte downloads.

Option 1: Generate a Synthetic Dataset

We’ll use a small Python script that creates short .mp4 clips of a moving (or still) coloured box. Each class represents a gesture:

• swipe_left – box moves from right to left

• swipe_right – box moves from left to right

• stop – box stays still in the center

Save this script as generate_synthetic_gestures.py in your project root:

import os, cv2, numpy as np, random, argparse

def ensure_dir(p): os.makedirs(p, exist_ok=True)

def make_clip(mode, out_path, seconds=1.5, fps=16, size=224, box_size=60, seed=0, codec="mp4v"):
    rng = random.Random(seed)
    frames = int(seconds * fps)
    H = W = size

    # background + box color
    bg_val = rng.randint(160, 220)
    bg = np.full((H, W, 3), bg_val, dtype=np.uint8)
    color = (rng.randint(20, 80), rng.randint(20, 80), rng.randint(20, 80))

    # path of motion
    y = rng.randint(40, H - 40 - box_size)
    if mode == "swipe_left":
        x_start, x_end = W - 20 - box_size, 20
    elif mode == "swipe_right":
        x_start, x_end = 20, W - 20 - box_size
    elif mode == "stop":
        x_start = x_end = (W - box_size) // 2
    else:
        raise ValueError(f"Unknown mode: {mode}")

    fourcc = cv2.VideoWriter_fourcc(*codec)
    vw = cv2.VideoWriter(out_path, fourcc, fps, (W, H))
    if not vw.isOpened():
        raise RuntimeError(
            f"Could not open VideoWriter with codec '{codec}'. "
            "Try --codec XVID and use .avi extension, e.g. out.avi"
        )

    for t in range(frames):
        alpha = t / max(1, frames - 1)
        x = int((1 - alpha) * x_start + alpha * x_end)
        # small jitter to avoid being too synthetic
        jitter_x, jitter_y = rng.randint(-2, 2), rng.randint(-2, 2)
        frame = bg.copy()
        cv2.rectangle(frame, (x + jitter_x, y + jitter_y),
                      (x + jitter_x + box_size, y + jitter_y + box_size),
                      color, thickness=-1)
        # overlay text
        cv2.putText(frame, mode, (8, 24), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 0), 2, cv2.LINE_AA)
        cv2.putText(frame, mode, (8, 24), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (255, 255, 255), 1, cv2.LINE_AA)
        vw.write(frame)

    vw.release()

def write_labels(labels, out_dir):
    with open(os.path.join(out_dir, "labels.txt"), "w", encoding="utf-8") as f:
        for c in labels:
            f.write(c + "\n")

def main():
    ap = argparse.ArgumentParser(description="Generate a tiny synthetic gesture dataset.")
    ap.add_argument("--out", default="data", help="Output directory (default: data)")
    ap.add_argument("--classes", nargs="+",
                    default=["swipe_left", "swipe_right", "stop"],
                    help="Class names (default: swipe_left swipe_right stop)")
    ap.add_argument("--clips", type=int, default=16, help="Clips per class (default: 16)")
    ap.add_argument("--seconds", type=float, default=1.5, help="Seconds per clip (default: 1.5)")
    ap.add_argument("--fps", type=int, default=16, help="Frames per second (default: 16)")
    ap.add_argument("--size", type=int, default=224, help="Frame size WxH (default: 224)")
    ap.add_argument("--box", type=int, default=60, help="Box size (default: 60)")
    ap.add_argument("--codec", default="mp4v", help="Codec fourcc (mp4v or XVID)")
    ap.add_argument("--ext", default=".mp4", help="File extension (.mp4 or .avi)")
    args = ap.parse_args()

    ensure_dir(args.out)
    write_labels(args.classes, ".")  # writes labels.txt to project root

    print(f"Generating synthetic dataset -> {args.out}")
    for cls in args.classes:
        cls_dir = os.path.join(args.out, cls)
        ensure_dir(cls_dir)
        mode = "stop" if cls == "stop" else ("swipe_left" if "left" in cls else ("swipe_right" if "right" in cls else "stop"))
        for i in range(args.clips):
            filename = os.path.join(cls_dir, f"{cls}_{i+1:03d}{args.ext}")
            make_clip(
                mode=mode,
                out_path=filename,
                seconds=args.seconds,
                fps=args.fps,
                size=args.size,
                box_size=args.box,
                seed=i + 1,
                codec=args.codec
            )
        print(f"  {cls}: {args.clips} clips")

    print("Done. You can now run: python train.py, python export_onnx.py, python app.py")

if __name__ == "__main__":
    main()

The script generates a synthetic gesture dataset by creating video clips of a moving or stationary coloured box, simulating gestures like "swipe left," "swipe right," and "stop," and saves them in a specified output directory.

Now run it inside your virtual environment:

python generate_synthetic_gestures.py --out data --clips 16 --seconds 1.5

The command above runs a Python script named generate_synthetic_gestures.py, which generates a synthetic gesture dataset with 16 clips per gesture, each lasting 1.5 seconds, and saves the output in a directory named "data".

This creates a dataset like:

data/
  swipe_left/*.mp4
  swipe_right/*.mp4
  stop/*.mp4
labels.txt

Each folder contains short clips of a moving (or still) box that simulate gestures. This is perfect for testing the pipeline.

Training Script: train.py

Now that we have our dataset, let’s fine-tune a Vision Transformer with temporal pooling. This model applies ViT frame-by-frame, averages embeddings across time, and trains a classification head on your gestures.

Here’s the full training script:

# train.py
import torch, torch.nn as nn, torch.optim as optim
from torch.utils.data import DataLoader
import timm
from dataset import GestureClips, read_labels

class ViTTemporal(nn.Module):
    """Frame-wise ViT encoder -> mean pool over time -> linear head."""
    def __init__(self, num_classes, vit_name="vit_tiny_patch16_224"):
        super().__init__()
        self.vit = timm.create_model(vit_name, pretrained=True, num_classes=0, global_pool="avg")
        feat_dim = self.vit.num_features
        self.head = nn.Linear(feat_dim, num_classes)

    def forward(self, x):  # x: (B,T,C,H,W)
        B, T, C, H, W = x.shape
        x = x.view(B * T, C, H, W)
        feats = self.vit(x)                  # (B*T, D)
        feats = feats.view(B, T, -1).mean(dim=1)  # (B, D)
        return self.head(feats)

def train():
    device = "cuda" if torch.cuda.is_available() else "cpu"
    labels, _ = read_labels("labels.txt")
    n_classes = len(labels)

    train_ds = GestureClips(train=True)
    val_ds   = GestureClips(train=False)
    print(f"Train clips: {len(train_ds)} | Val clips: {len(val_ds)}")

    # Windows/CPU friendly
    train_dl = DataLoader(train_ds, batch_size=2, shuffle=True,  num_workers=0, pin_memory=False)
    val_dl   = DataLoader(val_ds,   batch_size=2, shuffle=False, num_workers=0, pin_memory=False)

    model = ViTTemporal(num_classes=n_classes).to(device)
    criterion = nn.CrossEntropyLoss()
    optimizer = optim.AdamW(model.parameters(), lr=3e-4, weight_decay=0.05)

    best_acc = 0.0
    epochs = 5
    for epoch in range(1, epochs + 1):
        # ---- Train ----
        model.train()
        total, correct, loss_sum = 0, 0, 0.0
        for x, y in train_dl:
            x, y = x.to(device), y.to(device)
            optimizer.zero_grad()
            logits = model(x)
            loss = criterion(logits, y)
            loss.backward()
            optimizer.step()

            loss_sum += loss.item() * x.size(0)
            correct += (logits.argmax(1) == y).sum().item()
            total += x.size(0)

        train_acc = correct / total if total else 0.0
        train_loss = loss_sum / total if total else 0.0

        # ---- Validate ----
        model.eval()
        vtotal, vcorrect = 0, 0
        with torch.no_grad():
            for x, y in val_dl:
                x, y = x.to(device), y.to(device)
                vcorrect += (model(x).argmax(1) == y).sum().item()
                vtotal += x.size(0)
        val_acc = vcorrect / vtotal if vtotal else 0.0

        print(f"Epoch {epoch:02d} | train_loss {train_loss:.4f} "
              f"| train_acc {train_acc:.3f} | val_acc {val_acc:.3f}")

        if val_acc > best_acc:
            best_acc = val_acc
            torch.save(model.state_dict(), "vit_temporal_best.pt")

    print("Best val acc:", best_acc)

if __name__ == "__main__":
    train()

Running the command python train.py initiates the training process for your gesture recognition model. Here's a breakdown of what happens:

1. Load your dataset from data/: The script will access and load the gesture dataset stored in the "data" directory. This dataset is used to train the model.

2. Fine-tune a pre-trained Vision Transformer: The training script will take a Vision Transformer model that has been pre-trained on a larger dataset and fine-tune it using your specific gesture dataset. Fine-tuning helps the model adapt to the nuances of your data, improving its performance on the specific task of gesture recognition.

3. Save the best checkpoint as vit_temporal_best.pt: During training, the script will evaluate the model's performance on a validation set. The best-performing version of the model (based on some metric like accuracy) will be saved as a checkpoint file named "vit_temporal_best.pt". This file can later be used for inference or further training.

What Training Looks Like

You should see logs similar to this:

Train clips: 38 | Val clips: 10
Epoch 01 | train_loss 1.4508 | train_acc 0.395 | val_acc 0.200
Epoch 02 | train_loss 1.2466 | train_acc 0.263 | val_acc 0.200
Epoch 03 | train_loss 1.1361 | train_acc 0.368 | val_acc 0.200
Best val acc: 0.200

Don’t worry if your accuracy is low at first, as with the synthetic dataset that’s normal. The key is proving that the Transformer pipeline works. You can boost results later by:

• Adding more clips per class

• Training for more epochs

• Switching to real recorded gestures

Figure 1. Example training logs from train.py, where the Vision Transformer with temporal pooling is fine-tuned on a tiny synthetic dataset.

Export the Model to ONNX

To make our model easier to run in real time (and lighter on CPU), we’ll export it to the ONNX format.

Note: ONNX, which stands for Open Neural Network Exchange, is an open-source format designed to facilitate the interchange of deep learning models between different frameworks. It lets you train a model in one framework, such as PyTorch or TensorFlow, and then deploy it in another, like Caffe2 or MXNet, without needing to completely rewrite the model. This interoperability is achieved by providing a standardized representation of the model's architecture and parameters.

ONNX supports a wide range of operators and is continually updated to include new features, making it a versatile choice for deploying machine learning models across various platforms and devices.

Create a file called export_onnx.py:

import torch
from train import ViTTemporal
from dataset import read_labels

labels, _ = read_labels("labels.txt")
n_classes = len(labels)

# Load trained model
model = ViTTemporal(num_classes=n_classes)
model.load_state_dict(torch.load("vit_temporal_best.pt", map_location="cpu"))
model.eval()

# Dummy input: batch=1, 16 frames, 3x224x224
dummy = torch.randn(1, 16, 3, 224, 224)

# Export
torch.onnx.export(
    model, dummy, "vit_temporal.onnx",
    input_names=["video"], output_names=["logits"],
    dynamic_axes={"video": {0: "batch"}},
    opset_version=13
)

print("Exported vit_temporal.onnx")

Run it with python export_onnx.py.

This generates a file vit_temporal.onnx in your project folder. ONNX lets us use onnxruntime, which is much faster for inference.

Create a file called app.py:

import os, tempfile, cv2, torch, onnxruntime, numpy as np
import gradio as gr
from dataset import read_labels

T = 16
SIZE = 224
MODEL_PATH = "vit_temporal.onnx"

labels, _ = read_labels("labels.txt")

# --- ONNX session + auto-detect names ---
ort_session = onnxruntime.InferenceSession(MODEL_PATH, providers=["CPUExecutionProvider"])
# detect first input and first output names to avoid mismatches
INPUT_NAME = ort_session.get_inputs()[0].name   # e.g. "input" or "video"
OUTPUT_NAME = ort_session.get_outputs()[0].name # e.g. "logits" or something else

def preprocess_clip(frames_rgb):
    if len(frames_rgb) == 0:
        frames_rgb = [np.zeros((SIZE, SIZE, 3), dtype=np.uint8)]
    if len(frames_rgb) < T:
        frames_rgb = frames_rgb + [frames_rgb[-1]] * (T - len(frames_rgb))
    frames_rgb = frames_rgb[:T]
    clip = [cv2.resize(f, (SIZE, SIZE), interpolation=cv2.INTER_AREA) for f in frames_rgb]
    clip = np.stack(clip, axis=0)                                    # (T,H,W,3)
    clip = np.transpose(clip, (0, 3, 1, 2)).astype(np.float32) / 255 # (T,3,H,W)
    clip = (clip - 0.5) / 0.5
    clip = np.expand_dims(clip, 0)                                   # (1,T,3,H,W)
    return clip

def _extract_path_from_gradio_video(inp):
    if isinstance(inp, str) and os.path.exists(inp):
        return inp
    if isinstance(inp, dict):
        for key in ("video", "name", "path", "filepath"):
            v = inp.get(key)
            if isinstance(v, str) and os.path.exists(v):
                return v
        for key in ("data", "video"):
            v = inp.get(key)
            if isinstance(v, (bytes, bytearray)):
                tmp = tempfile.NamedTemporaryFile(delete=False, suffix=".mp4")
                tmp.write(v); tmp.flush(); tmp.close()
                return tmp.name
    if isinstance(inp, (list, tuple)) and inp and isinstance(inp[0], str) and os.path.exists(inp[0]):
        return inp[0]
    return None

def _read_uniform_frames(video_path):
    cap = cv2.VideoCapture(video_path)
    frames = []
    total = int(cap.get(cv2.CAP_PROP_FRAME_COUNT)) or 1
    idxs = np.linspace(0, total - 1, max(T, 1)).astype(int)
    want = set(int(i) for i in idxs.tolist())
    j = 0
    while True:
        ok, bgr = cap.read()
        if not ok: break
        if j in want:
            rgb = cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB)
            frames.append(rgb)
        j += 1
    cap.release()
    return frames

def predict_from_video(gradio_video):
    video_path = _extract_path_from_gradio_video(gradio_video)
    if not video_path or not os.path.exists(video_path):
        return {}
    frames = _read_uniform_frames(video_path)

    # If OpenCV choked on the codec (common with recorded webm), re-encode once:
    if len(frames) == 0:
        tmp = tempfile.NamedTemporaryFile(delete=False, suffix=".mp4"); tmp_name = tmp.name; tmp.close()
        cap = cv2.VideoCapture(video_path)
        fourcc = cv2.VideoWriter_fourcc(*"mp4v")
        w = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)) or 640
        h = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)) or 480
        out = cv2.VideoWriter(tmp_name, fourcc, 20.0, (w, h))
        while True:
            ok, frame = cap.read()
            if not ok: break
            out.write(frame)
        cap.release(); out.release()
        frames = _read_uniform_frames(tmp_name)

    clip = preprocess_clip(frames)
    # >>> use the detected ONNX input/output names <<<
    logits = ort_session.run([OUTPUT_NAME], {INPUT_NAME: clip})[0]  # (1, C)
    probs = torch.softmax(torch.from_numpy(logits), dim=1)[0].numpy().tolist()
    return {labels[i]: float(probs[i]) for i in range(len(labels))}

def predict_from_image(image):
    if image is None:
        return {}
    clip = preprocess_clip([image] * T)
    logits = ort_session.run([OUTPUT_NAME], {INPUT_NAME: clip})[0]
    probs = torch.softmax(torch.from_numpy(logits), dim=1)[0].numpy().tolist()
    return {labels[i]: float(probs[i]) for i in range(len(labels))}

with gr.Blocks() as demo:
    gr.Markdown("# Gesture Classifier (ONNX)\nRecord or upload a short video, then click **Classify Video**.")
    with gr.Tab("Video (record or upload)"):
        vid_in = gr.Video(label="Record from webcam or upload a short clip")
        vid_out = gr.Label(num_top_classes=3, label="Prediction")
        gr.Button("Classify Video").click(fn=predict_from_video, inputs=vid_in, outputs=vid_out)
    with gr.Tab("Single Image (fallback)"):
        img_in = gr.Image(label="Upload an image frame", type="numpy")
        img_out = gr.Label(num_top_classes=3, label="Prediction")
        gr.Button("Classify Image").click(fn=predict_from_image, inputs=img_in, outputs=img_out)

if __name__ == "__main__":
    demo.launch()

Running the command python app.py launches a Gradio application in your web browser. Here's what happens:

1. Webcam feed streams live: The application accesses your webcam to provide a live video feed. This allows you to perform gestures in front of the camera in real-time.

2. Predictions update continuously: As you perform gestures, the model processes the video frames continuously, updating its predictions in real-time.

3. Top 3 gesture classes displayed with probabilities: The application displays the top three predicted gesture classes along with their probabilities, giving you an idea of the model's confidence in its predictions.

When you open the app in your browser, you'll find two tabs. In the Video tab, you can click Record from webcam to capture a short clip of your gesture, typically lasting 2–4 seconds. After recording, click Classify Video. The model will then process the captured frames using the Transformer model and display the predicted gesture probabilities. This setup allows for interactive testing and demonstration of the gesture recognition system.

Here’s an example where I raised my hand for a stop gesture, and the model predicts “stop” as the top class:

Figure 2. The Gradio app running locally. After recording a short clip, the Transformer model predicts the gesture with class probabilities.

Evaluate Accuracy + Latency

Now that the model runs in a demo app, let’s check how well it performs. There are two sides to this:

• Accuracy: does the model predict the right gesture class?

• Latency: how fast does it respond, especially on CPU vs GPU?

1. Quick Accuracy Check

Save this as eval.py in the same folder as your other scripts:

import torch
from dataset import GestureClips, read_labels
from train import ViTTemporal

labels, _ = read_labels("labels.txt")
n_classes = len(labels)

# Load validation data
val_ds = GestureClips(train=False)
val_dl = torch.utils.data.DataLoader(val_ds, batch_size=2, shuffle=False)

# Load trained model
model = ViTTemporal(num_classes=n_classes)
model.load_state_dict(torch.load("vit_temporal_best.pt", map_location="cpu"))
model.eval()

correct, total = 0, 0
all_preds, all_labels = [], []

with torch.no_grad():
    for x, y in val_dl:
        logits = model(x)
        preds = logits.argmax(dim=1)
        correct += (preds == y).sum().item()
        total += y.size(0)
        all_preds.extend(preds.tolist())
        all_labels.extend(y.tolist())

print(f"Validation accuracy: {correct/total:.2%}")

2. Confusion Matrix

Let’s also visualize which gestures are confused. Add this snippet at the bottom of eval.py:

import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.metrics import confusion_matrix

cm = confusion_matrix(all_labels, all_preds)

plt.figure(figsize=(6,6))
sns.heatmap(cm, annot=True, fmt="d", xticklabels=labels, yticklabels=labels, cmap="Blues")
plt.xlabel("Predicted")
plt.ylabel("True")
plt.title("Confusion Matrix")
plt.tight_layout()
plt.show()

When you run python eval.py, a heatmap like this will pop up:

Figure 3. Confusion matrix on the validation set. Correct predictions appear along the diagonal. Off-diagonal counts show gesture confusions.

3. Latency Benchmark

Finally, let’s see how fast inference runs. Save the following as benchmark.py:

import time, numpy as np, onnxruntime
from dataset import read_labels

labels, _ = read_labels("labels.txt")

ort = onnxruntime.InferenceSession("vit_temporal.onnx", providers=["CPUExecutionProvider"])
INPUT_NAME = ort.get_inputs()[0].name
OUTPUT_NAME = ort.get_outputs()[0].name

dummy = np.random.randn(1, 16, 3, 224, 224).astype(np.float32)

# Warmup
for _ in range(3):
    ort.run([OUTPUT_NAME], {INPUT_NAME: dummy})

# Benchmark
t0 = time.time()
for _ in range(50):
    ort.run([OUTPUT_NAME], {INPUT_NAME: dummy})
t1 = time.time()

print(f"Average latency: {(t1 - t0)/50:.3f} seconds per clip")

Run: python benchmark.py

On CPU, you might see ~0.05–0.15s per clip; on GPU it’s much faster.

Note: If latency is high, you can enable quantization in ONNX to shrink the model and speed up inference.

Option 2: Use Small Samples from Public Gesture Datasets

If you’d prefer to see your model trained on real gesture clips instead of synthetic moving boxes, you can grab a handful of videos from open datasets. You don’t need to download the entire dataset (which can be several GB) just a few .mp4 samples are enough to follow along.

Recommended sources

• 20BN Jester Dataset: Contains short clips of hand gestures like swiping, clapping, and pointing.

• WLASL: A large-scale dataset of isolated sign language words.

Both projects provide small .mp4 videos you can use as realistic training examples. I’ve linked them below.

Setting up your dataset folder

Once you download a few clips, place them in the data/ folder under subfolders named after each gesture class. For example:

data/
├── swipe_left/
│   ├── clip1.mp4
│   └── clip2.mp4
├── swipe_right/
│   ├── clip1.mp4
│   └── clip2.mp4
└── stop/
    ├── clip1.mp4
    └── clip2.mp4

And update labels.txt to match the folder names:

swipe_left
swipe_right
stop

Now your dataset is ready, and the same training scripts from earlier (train.py, eval.py) will work without modification.

Why choose this option?

• Gives more realistic results than synthetic coloured boxes

• Lets you see how the model handles actual human hand movements

• It just requires a bit more effort (downloading clips, trimming them if needed)

Tip: If downloading from these datasets feels too heavy, you can also record your own short gestures using your laptop webcam. Just save them as .mp4 files and organize them in the same folder structure.

Accessibility Notes & Ethical Limits

While this project shows the technical workflow for gesture recognition with Transformers, it’s important to step back and consider the human context:

• Accessibility first: Tools like this can help students with speech or motor difficulties, but they should always be co-designed with the people who will use them. Don’t assume one-size-fits-all.

• Dataset sensitivity: Using publicly available sign or gesture datasets is fine for prototyping, but deploying such a system requires careful consideration of consent and representation.

• Error tolerance: Even small misclassifications can have big consequences in accessibility contexts (for example, confusing stop with go). Always plan for fallback options (like manual input or confirmation).

• Bias and inclusivity: Models trained on narrow datasets may fail for different skin tones, lighting conditions, or cultural gesture variations. Broad and diverse training data is essential for fairness.

In other words: this demo is a teaching scaffold, not a production-ready accessibility tool. Responsible deployment requires collaboration with educators, therapists, and end users.

Next Steps

If you’d like to push this project further, here are some directions to explore:

• Better models: Try video-focused Transformers like TimeSformer or VideoMAE for stronger temporal reasoning.

• Larger vocabularies: Add more gesture classes, build your own dataset, or use portions of public datasets like 20BN Jester or WLASL.

• Pose fusion: Combine gesture video with human pose keypoints from MediaPipe or OpenPose for more robust predictions.

• Real-time smoothing: Implement temporal smoothing or debounce logic in the app so predictions are more stable during live use.

• Quantization + edge devices: Convert your ONNX model to an INT8 quantized version and deploy it on a Raspberry Pi or Jetson Nano for classroom-ready prototypes.

Conclusion

In this tutorial, you learned how to create a gesture recognition system using Transformer models, demonstrating the potential of cutting-edge machine learning techniques. By preparing a small dataset, training a Vision Transformer with temporal pooling, exporting the model to ONNX for efficient inference, and deploying a real-time Gradio app, you showcased a practical application of these technologies. The evaluation of accuracy and latency further highlighted the system's effectiveness and responsiveness.

This project illustrates how you can leverage advanced ML methods to enhance accessibility and communication, paving the way for more inclusive learning environments.

Remember: while this demo works with small datasets, real-world applications need larger, more diverse data and careful consideration of accessibility, inclusivity, and ethics.

Here’s the GitHub repo for full source code: transformer-gesture.

In this handbook, we’ll focus on endgame fundamentals. You’ll learn about some common checkmate patterns for beginners and how you can recognize these patterns more easily to increase your chances of winning.

This handbook is not for absolute beginners. You should understand these concepts before proceeding:

• How to identify each chess piece.

• How to set up a chess board.

• How each piece moves.

• Some basic chess terms like "capture", "castling", "check", "checkmate", "piece promotion", "fork", "pin", and so on.

If you're new to chess, or you're not yet familiar with the prerequisites, then this free resource is a good place to start.

In the first chapter, you'll learn about algebraic chess notations. You’ll learn about files and ranks, how to identify and denote each square on a board, and how to record chess moves using algebraic chess notations.

In the second chapter, you’ll learn about some checkmate patterns like queen and king vs king, two rooks vs king, opera mate, Anastasia’s mate, and so on.

I've provided some exercises in different sections to help you practice. You can find the answers to each exercise at the end of the handbook.

I’ve also included some practice positions for each endgame pattern. To use them, click on the link provided for each exercise, then click on “CONTINUE FROM HERE” in the options at the bottom-right of the page. That is:

This will let you practice the positions against an engine. The engine has varying levels of strength, so you should try beating each level. You can find links for all the practice positions at the end of the article. You can also use the board editor from the links to set up your own positions.

You can watch the video version of this handbook here:

Table of Contents

1. How to Use this Handbook

2. Chapter 1: What are Notations in Chess? Algebraic Chess Notation Explained

   • What are Chess Notations?

   • What is Algebraic Chess Notation?

   • What are Chess Notations Used For?

   • How to Write Algebraic Chess Notations

   • How to Identify Squares on a Chess Board

3. Chapter 2: Checkmate Patterns for Beginners

   • What is a Check and Checkmate in Chess?

   • Checkmate Patterns

   • Checkmate Patterns II

4. Conclusion

5. Exercise Answers

6. Practice Position URLs

How to Use this Handbook

This handbook is not meant to be read and understood just once. If you're a beginner, then chances are high that you'll reference it multiple times in order to understand certain concepts.

In order to get the best out of this resource, you'll need a chess board. You can use either a physical board or an online chessboard. I recommend using Lichess if you're going to use the latter option.

You should try and replicate the moves as you read. Remember, you're not meant to understand everything the first time you read it, so it’s totally fine to go over one concept again and again.

Chess books can be boring, but don't let that hinder your progress. It gets interesting when you're able to understand, practice, and see improvements from what you've learned.

Let's get started!

Chapter 1: What are Notations in Chess? Algebraic Chess Notation Explained

You can use chess notations to record chess moves, and this handbook will rely heavily on them to teach you difference concepts about chess as a beginner. The first time you learn about writing or reading notations may seem overwhelming, but don't worry – with constant practice and studying, you'll get the hang of it.

What are Chess Notations?

To put it simply, chess notation is a method of recording moves in a chess game. In a physical chess game, it usually involves two players writing down moves as they’re played.

While there are other forms of chess notations like descriptive, coordinate, numeric, and so on, we'll focus primarily on algebraic chess notation.

What is Algebraic Chess Notation?

Algebraic chess notation is the standard way of recording chess. Each notation shows the move number, the piece moved, and the position of the board where the piece was moved to. In some cases, the notations also show an action being performed, like a check, a checkmate, castling, and so on.

For example:

1. e4 d5

2. d4 dxe4

3. Nc3 e6

These may look strange if it's your first time looking at notations. But don't worry, you'll understand how to write and read/interpret them as we progress.

In online games, they're recorded automatically:

The image above shows a chess game played on lichess.org. On the right side of the image, you can see the notations of the game as each move was played, along with numbers corresponding with the moves. We'll talk more about the numbers later.

Now that you know what notations are, let's talk about why they're needed.

What are Chess Notations Used For?

Here are some of the uses of chess notations:

Competitive/Professional Chess

Algebraic chess notation is the standard way of recording a chess game in tournaments. In over the board tournaments, each player is given a piece of paper with columns for writing each move played. Note that this involves recording their opponent's moves as well.

Studying and Teaching Chess

You can think of chess notations as the language of chess, and to learn about something, there has to be a way to pass and communicate information.

Communication

With chess notations, you can communicate moves and share games with players across the world without needing a physical board.

How to Write Algebraic Chess Notations

Algebraic chess notation makes use of the board coordinates and the pieces to denote moves.

Let's start with the coordinates:

The image above shows a chess board without the pieces. On the right edge of the board are numbers 1 to 8, and the bottom edge of the board has letters a to h. You can use these numbers and letters to refer to a particular position of the board.

Each vertical column (top to bottom) in a chess board is called a file, while each horizontal row (left to right) is called a rank. You can also associate them with the letters and numbers: the letters can be used to reference files, while the numbers can be used to reference ranks.

Files a to f

There are eight files in a chess board. Another way to say this is that, from top to bottom, there are eight vertical lines (stack of squares from top to bottom) in a chess board. Each vertical line is associated with a letter, which is usually written at the bottom edge of the board. That is:

To help you understand files, let's try and identify some of them separately. The image below shows the a-file:

We can tell that this is the a-file because the all the squares in that vertical column fall under the part of the board labelled "a". Remember that files go from top to bottom.

Here's another example:

This arrow runs from top to bottom, but on a different part of the board. To know what file this is, simply look the the letter associated with the vertical column. Using that, we can say that the arrow is on the the f-file.

Here's an exercise for you (you can find the answers to each exercise at the end of the handbook):

In the image above, using the letters on the board, what file is the arrow on?

Ranks 1 to 8

Just like files, we have eight ranks in a chess board. These are the eight horizontal lines (stack of squares from left to right) in a chess board. That is:

Let's identify some of the ranks separately.

Using the position of the arrow, you can tell the rank. The arrow run from left to right on the part of the board with the number 4, so we can say that it is on the fourth rank.

Here's another example:

You may confuse files and ranks here since you can see the letters at the bottom. Remember, we're dealing with ranks here, and they go from left to right, not top to bottom. From left to right, in the image above, we can see the letter 1. So the arrow is on the first rank.

Here's an exercise for you:

In the image above, using the numbers on the board, what rank is the arrow on?

How to Identify Squares on a Chess Board

Now that you know what files and ranks are in chess, let's see how you can use them to identify a square. Identifying a square is important when writing notations. Each notation is made up of the move number, the piece moved, and the coordinate. In the case, a coordinate is the combination of a file and a rank.

Let's start with this image:

In the image above, we have a circle on a random square. The goal here is to identify that square, that is, to refer to it as "something".

To identify the square, you have to trace its file and the rank. Let' start with the file.

Using the arrow, we can see that the circle falls under the e-file. Next, let's trace the rank.

Great! We can tell that the circle is on the fifth rank.

So we can say that we have a circle on the e-file and the fifth rank. Using notations, you can write it as e5.

In the next example, we'll identify the position of chess pieces. In order to do that, you need to know the letters that represent each piece on a board:

King = K

Queen = Q

Rook = R

Bishop = B

Knight = N

Pawn = no letter (denoted by the square they move to).

In our last example, we identified the square as e5. If you replace that square with a chess piece, then the the notation would start with the letter of that piece, followed by its position on the board. Here's an example:

Now we have the queen on the e5 square. The notation will look like this: Qe5. This way of writing the letter of the piece, followed by the coordinate of the square applies to other pieces, except the pawns. For pawns, you only write the coordinate.

If you had a rook on e5, the notation would be Re5, Ke5 for a king, Ne5 for a knight, Be5 for a bishop, and e5 for a pawn.

How to Write Algebraic Chess Notations

We stated earlier that algebraic chess notations show the move number, the piece moved, and the position of the board where the piece is moved to. In our examples, you've seen how to write the piece moved, and where it was moved to using coordinate (files and ranks). In the examples for this section, we'll add the move number to complete the algebraic chess notation.

By move number, we simply mean the turn count. Each move number is associated with one White and one Black move. For instance, the first move of the game will have a move number of 1. Let's look at some examples.

Each piece on the board is in its starting position. Let's make the first move for White:

White has moved a pawn to e4. To record this, you'd start with the move number, followed by the piece and coordinate. That is:

1. e4

Next, Black's move.

Black responded by moving a pawn to e4. Since this is Black's first move, it'll also be recorded under move number 1:

1. e4 e5

If you look at the algebraic notation above, you can see that both notations are written on the same line, and are both associated with move number 1. To interpret this, the first coordinate belongs to White, while the second belongs to Black.

To make this even clearer, let's continue the game. It's White's turn to play.

The white knight was moved to f3, which can be denoted by Nf3. Since this is White's second move, it'll be recorded under move number 2:

1. e4 e5

2. Nf3

Now Black has to respond with a second move.

Black played Nc6 (knight to c6). Remember that this is the second move for Black, so it has to be recorded in move number 2, right after the notation for White's second move. So we'll have this:

1. e4 e5

2. Nf3 Nc6

This system of recording chess moves is known as algebraic chess notation. Here's a summary of how it works:

• You write the move number.

• You write piece moved (omitted for pawns).

• You write the coordinate (file and rank) that the piece moved to.

With this information, you can easily read and write chess notations! But we're not done yet. How do you write checks, checkmates, castling, and pawn promotions?

How to Write a Check Notation in Algebraic Chess Notations

You can use the + symbol to denote a check. This comes after the move notation. That is, you write the notation as you would for any other move, and then add the + symbol at the end. Here's an example:

Here, the queen moves to e7 to check the king. So the notation would be Qe7+. The + at the end denotes the check by the queen. If you write it as Qe7, without the check symbol, then you'd have an inaccurate notation of the move.

How to Write a Checkmate Notation in Algebraic Chess Notations

A checkmate is denoted by the # symbol. It also comes after the move notation. That is, you write the notation as you would for any other move, and then add the # symbol at the end. Here's an example:

In the image above, the rook moved to g8 to win the game. The notation would be Rg8#. The # symbol denotes the checkmate.

How to Write a Castling Notation in Algebraic Chess Notations

In a chess game, castling can either be to the kingside or queenside. Kingside castling is denoted using O-O, while queenside castling is denoted using O-O-O. Note that the symbol is not a zero. Rather, it's an O, as in orange.

How to Write a Capture Move in Algebraic Chess Notations

Capture moves are denoted using the x symbol. Here are some different variations:

• Qxg6 means that the queen captured a piece on g6.

• Qxg6+ means that the queen captured a piece on g6 and checked the king.

• Qxg8# means that the queen captured a piece on g8 and checkmated the king.

• dxe4 mean that a pawn captured a piece on e4.

In a case where you have two pieces that can move to the same square, you have to include the file of the piece that was moved. Here's an example:

In the board above, we have two knights: one on the f-file and another on the h-file. Both knights can capture the rook on g8, and it may be confusing for someone reading your notations if you don't specify which knight was moved.

To denote this, you have to add the file from where the knight was moved in your notation. So, if you capture with the knight on the f-file, you'd have Nfxg8 (knight on f-file captured a piece on g8). On the other hand, capturing with the knight on the h-file will result to a notation like this: Khxg8.

This is not only specific to capturing pieces. If you have two knights that can move to the same square, then you'd have to include the previous file of whichever knight that is moved. So, assuming that we didn't have a rook to capture on g8, you'd have Nfg8 to denote that the knight on f-file moved to g8, and Nhg8 to denote that the knight on the h-file moved to g8.

How to Denote Piece Promotion in Algebraic Chess Notations

You can use the = symbol to denote the promotion of a pawn. That is:

• e8=Q means that pawn moved to e8 and got promoted to a queen.

• e8=Q+ means that pawn moved to e8, got promoted to queen, and checked the king.

• e8=Q# means that pawn moved to e8, got promoted to queen, and checkmated the king.

• e8=N means that pawn moved to e8 and got promoted to a knight.

Here are two exercises for you:

• How do you write the notation if a pawn moves to d1 and gets promoted to a rook?

• How do you write the notation if a pawn moves to b8 and gets promoted to a bishop?

And here's the main exercise for this chapter. Show the final board position using these notations:

1. e4 e5

2. Nf3 Nc6

3. Bb5 Nf6

4. O-O Nxe4

5. Re1 Nd6

6. Nxe5 Be7

7. Bf1 Nxe5

8. Rxe5 O-O

9. d4 Bf6

10. Re1 Re8

11. c3 Rxe1

12. Qxe1 Nf5

13. Bf4 d6

14. a4 Qe7

15. Na3 g5

16. Bd2 Qxe1

17. Rxe1 Bd7

18. a5 Kg7

Chapter 2: Checkmate Patterns for Beginners

Before we look at the common checkmate patterns, let's talk a bit about what a check means. We'll do this using the attack pattern or capture method of the pieces.

What is a Check and Checkmate in Chess?

The main goal of a standard chess match is to capture your opponent's king. You can do this in multiple ways, with different combinations of pieces.

A check is simply an attack on a king. When the king is in check, it must move to a safe square. A safe square for a king in chess is a square where the king is not under attack.

This is an example of a check:

The king in the image above is being threatened by the queen. You can deliver checks based on how a piece moves and captures other pieces. A rook can deliver a check like this:

A bishop can deliver a check like this:

Same applies to the knights and pawns – they can check the king based on their movement and mode of attack.

So what is a checkmate? Well, a checkmate happens when the opponent's king has no safe squares to move to after a check. This can be the result of the king being trapped by its own pieces, the king being unable to move to another square because its opponent has control of all the possible squares it can move to, or not having any piece to block the attack on the king.

In this sections that follow, you'll learn about different checkmate patterns using a combination of different pieces.

Checkmate Patterns

How to Play the Two Rooks vs King Checkmate Pattern

This endgame pattern usually involves two rooks against a king. It is one of the easiest checkmate patterns, but it's also easy to get carried away as a beginner. Here, you'll learn how to effectively mate and end a game with two rooks.

Consider this position:

White can end this game in six moves. Here are some things to keep in mind when you have a position like this:

• Restrict the opposing king's movement by cutting off squares where it can move to.

• Keep your rooks away from the opposing king or/and close to each other.

Let's see how each step works using the position above.

Restrict the King:

Looking at the initial position above, you can see that the king is on the f-file. Our goal here is to isolate the king and restrict its movement. One common move for beginners would be to check the king with Rf2+. But this allows the king to move closer to the rook on d4. That is:

While this is still a winning position, you'll spend more moves trying to restrict the king's movement. If you're playing a timed game, you may end up with a forced draw, losing a rook, or even losing the game, depending on the pieces on the board. This makes it important to know where and when to move your rooks.

So, back to the starting position:

1. Re2 Kf5.

Playing Re2 cuts off the e-file, meaning that the black king can no longer get on any square in that file. Since the king can not pass through or get on the e-file, it means that the a to e files have become inaccessible to it. It is now left with the f to h files.

The king responds with Kf5, an attempt to get closer and capture a rook. At this point, you'd want to stay away from the king.

Move Your Rooks Away From The King:

This is the current position:

Be careful not to play Re5+, confusing the d-rook for a supporting queen. This forces you into a one rook and king checkmate endgame (you'll learn about that in the next section) after Kxe5. This is a better move:

2. Rd8

Which brings us to this position:

This keeps your rooks far away from the black king. Since the king is limited to one square per move, it'll have a long way to go before getting close to either rook. And in the process of getting close, you can continue limiting which files it can get on.

The king has five possible moves here: Kf6, Kf4, Kg6, Kg5, Kg4. The only thing these moves have in common is that they lead to nowhere. But let's assume the black king has some sort of miracle to perform here.

2. ... Kg6

The king moving to g6 gives you the opportunity to occupy the f-file with:

3. Rf8

You have the f-file, but your rook is closer to the king:

3. ... Kg7

The king moves closer to challenge the rook. Unfortunately, a checkmate is still inevitable. At the point, you've not only restricted the files that the king can move to, you've also lured it all the way to the seventh rank.

Keep Your Rooks Closer to Each Other:

4. Rf1

With one rook on f1 and the other on e2, you're far away from the king and have both rooks closer to each other:

With this position, it will take the king at least four moves to get to your rooks.

4. ... Kg6

All that's left is to do now is keep pushing the king away from accessing other files:

5. Rg2+

Checking the king with the g-rook forces it to the h-file. Since you have a rook occupying the e-file, the king can’t go there.

5. ... Kh5

6. Rh1#

There are multiple ways to checkmate the king from the starting position, and in some cases, you may already have enough space to bring your rooks together and trap the king, instead of trying to restrict it to a part of the board first.

Consider this position:

The black king is already in a bad position. If you move the b-rook to b6, the game ends after Ra7+ and Rb8#. This did not require you to restrict the king.

Another thing to keep in mind is that your rooks must not always be close to each other to win games like this. Here's another position:

Here, the king is already limited to only the sixth and seventh rank, and you have enough squares/space between it and the rooks to win. There’s no need to avoid the king by moving one rook to a different part of the board.

Playing Rh7+ forces the king into the eighth rank and Ra8# checkmates the king.

You'll also notice that the white king did not participate in these moves. Its contribution was staying out of the way of the rooks. In some cases, the king can be helpful in delivering a checkmate. But if it'll block your rooks from restricting an enemy king, then you should keep your king away.

Here's a replay of the moves, along with the notations:

1. Re2 Kf5

2. Rd8 Kg6

3. Rf8 Kg7

4. Rf1 Kg6

5. Rg2+ Kh5

6. Rh1#

You can replicate them on your own if you weren't doing so already, and practice with the positions provided below.

Practice Position for Two Rooks vs King Endgame/Checkmate Pattern

You can practice what you've learned in the last section using this position:

I’ve set up the position for you here: https://lichess.org/editor/1r4k1/8/8/5r2/8/3K4/8/8_b_-_-_0_1?color=black. Click on the link, then click on “CONTINUE FROM HERE” to practice the position against an engine.

How to Play the King and Rook vs King Checkmate Pattern

Before we talk about how to win this game, you need to learn about the concept of "direct opposition". This happens when two kings are facing each other on the same file or rank, with exactly one square between them. That is:

The kings in the image above are facing each other, have one square between them, and are both on the d-file. Both kings are in direct opposition.

Here's another example:

The kings are still facing each other, they also have a square between them, but are now on the seventh rank. There are other forms of opposition like the diagonal and distant opposition, but we won't discuss them here.

So why does direct opposition matter in our current position? Well, let's look at how the king moves.

The black king, in the image above, can move to all the squares with a circle on them. Now, here's what happens when the king is in direct opposition with the other king.

Now the black king has lost the ability to move to three squares in the direction of the opposition. This also implies that the black king, using the position in the image above, no longer has access to the fourth rank. So playing Ra5+ forces the black king to move to the sixth rank, restricting it, and allowing the white king to come closer and set up another direct opposition.

This is one way of winning with a rook and a king. An alternative method is the box method, which we'll look at later in this section. Both techniques can be used together, but I think it would be helpful to learn them separately at first.

How to Use the Direct Opposition to Win with a Rook and a King

Here's our starting position:

1. Rb5+

Forcing the black king to the sixth rank.

1. ... Kd6

Now the king is trying to get closer and attack your rook, so you have to bring your own king closer for support.

2. Kd4 Ke6

Creating direct opposition. If the black king moves backwards (seventh rank), it loses the opportunity to pass through or be on the sixth rank. If it moves to c6, it'll have fewer squares to work with. So the best move for the black king would be Ke6.

Now we'll use something called the box method - a technique used when playing with a rook and a king to shrink the space where the opposing king can move in. This is the current position of the game:

3. Rd5

This move blocks the black king from moving to the d-file and forces it into a box that will gradually shrink if you make the right moves. Here:

The image above shows squares where the black king has been trapped in. The best move for the black king would be Kf6. Playing Ke7 or Kf7 would allow the white king to move closer. Remember, this is a lost position for black, but if you play against an opponent who wants to rely on luck to force a draw through a stalemate or timeout, then knowing exactly where to move your rook becomes very important.

Also, checks with the rook will do you no good in this position. If you check the king with Re5+, then the king will just escape to the d-file. So when you're this close to the opposing king:

3. ... Kf6

4. Re5 Kf7

5. Kd5 Kf6

The black king has gotten closer to your rook again. If you check it, it escapes through the fifth rank. When you have all three pieces in a similar position, move your king instead of a check. This forces the opposing king to remain boxed in.

6. Kd6 Kf7

7. Rf5+ Kg6

8. Ke6

Providing support to the rook and ensuring that the black king remains in a smaller space. This is what the box looks like now:

8. ... Kg7

9. Rg5+ Kh6

10. Kf6 Kh7

11. Rh5+ Kg8

12. Rh6 Kf8

13. Rh8#

Here are things to keep in mind when playing with a king and rook against a king:

• Cut off the enemy king with your rook.

• Bring your king closer to support the rook and push the enemy king.

• Shrink the box using the two points above.

• Force the king to the edge of the board (first rank, eight rank, a-file, h-file). Checkmate is only possible when the enemy king is at the edge of the board and in opposition with your king.

This can be overwhelming for beginners, so don't worry if you don't fully understand the process from start to finish. With constant practice, it should become easier and faster for you to recognize the patterns. I'll also provide some puzzles to help you practice.

Here's a replay of all the move and the notations:

1. Rb5+ Kd6

2. Kd4 Ke6

3. Rd5 Kf6

4. Re5 Kf7

5. Kd5 Kf6

6. Kd6 Kf7

7. Rf5+ Kg6

8. Ke6 Kg7

9. Rg5+ Kh6

10. Kf6 Kh7

11. Rh5+ Kg8

12. Rh6 Kf8

13. Rh8#

Practice Position for One Rook and King vs King Endgame/Checkmate Pattern

You can practice what you learned in the last section using this position:

I’ve set up the position for you here: https://lichess.org/editor/6k1/8/8/5r2/8/3K4/8/8_w_-_-_0_1?color=black. Click on the link, then click on “CONTINUE FROM HERE” to practice the position against an engine.

How to Play the Queen and King vs King Checkmate Pattern

We’ll start with this board:

There are a couple of ways to play endgame like the one in the image above. A lot of beginners struggle with this checkmate pattern because they often end in a stalemate or just chase the king around the board until they or their opponent runs out of time.

We can approach this endgame using the direct opposition and box method like we did in the previous section, but there's a more efficient and straightforward way of winning this position, with minimal effort from your king.

We'll use the knight move method. This implies that your queen should only move to squares that imitate a knight attack on the enemy king. Using this method, you can push the king towards the edge of the board.

Here are the possible squares where the queen can go to in order to imitate a knight attack:

Let's start with the closest square:

1. ... Qd6

This brings us to this position:

We've pushed the king into a box. Now, no matter what the king plays, try to follow up with moves that imitate a knight attack (the L), and do not play a move that allows the king leave the shrinking box.

2. Ke3 Qd5

The keeps the king in the box and brings the queen closer.

3. Ke2 Qd4

4. Kf3 Qe5

The box has gotten smaller:

No checks. No direct opposition. Just follow the king with the queen, as though you were playing as a knight.

5. Kf2 Qe4

6. Kg3 Qf5

7. Kg2 Qf4

8. Kg1 Qf3

9. Kh2 Qg4

10. Kh1

You've successfully pushed the king to the edge of the box. At this point, don't be in a hurry to play another knight attack move with the queen – this will lead to a stalemate.

The queen's job is almost done. The white king only has two squares to move to: h2 and h1. It time to push the black king forward to support the queen in a checkmate. Since you've trapped the white king, it will continue to move between the two possible squares until the black king arrives.

10. ... Ke7

11. Kh2 Kf6

12. Kh1 Kf5

13. Kh2 Kf4

14. Kh1 Kf3

15. Kh2 Qg2#

Here's a replay and all the notations:

1. ... Qd6

2. Ke3 Qd5

3. Ke2 Qd4

4. Kf3 Qe5

5. Kf2 Qe4

6. Kg3 Qf5

7. Kg2 Qf4

8. Kg1 Qf3

9. Kh2 Qg4

10. Kh1 Ke7

11. Kh2 Kf6

12. Kh1 Kf5

13. Kh2 Kf4

14. Kh1 Kf3

15. Kh2 Qg2#

Practice Position for Queen and King vs King Endgame/Checkmate Pattern

You can practice what you learned in the last section using this position:

I’ve set up the position for you here: https://lichess.org/editor/1r4k1/8/8/5r2/8/3K4/8/8_b_-_-_0_1?color=black. Click on the link, then click on “CONTINUE FROM HERE” to practice the position against an engine.

For this position, choose to play as black before starting the game:

Checkmate Patterns II

In this section, we’ll deal with checkmate patterns that can happen at any point. That is, you don't have to be in an endgame position/situation to play them.

In some cases, these positions can happen by chance, and in other cases, you have to plan and set them up yourself. The main objective here is to get used to recognizing patterns.

Opera Mate

An Opera Mate usually involves the following:

• A rook or queen delivering a check along a back rank or file.

• A bishop covering a diagonal escape square.

• A king trapped by its own pieces or the edge of the board.

Here's an example:

In the position above, black played Bg7, threatening the white rook. But if we look closely at the position, white can deliver a checkmate by moving to c8.

With Rc8#, the king can't capture the rook because the bishop is protecting it, the king can't move to d7 because the bishop is attacking that square, and unfortunately for the king, its own pawn is blocking it from escaping through e7.

Here's another position:

Let's assume that it's white's turn to play again. Would it still be checkmate if the rook moves to c8? Yes it would – the position hasn’t really changed: the bishop is still protecting the rook on c8 and attacking the d7 square, and the pawn on e7 is still blocking the king.

Let's look at an example from Lichess:

This game still looks alive, as though both players are still setting up their strategies. It's black's turn to play. At this point, we'll assume that black has discovered a possible opera mate.

The black bishop is attacking the d1 and e2 squares, while the queen can check the king on d1, with additional support from the rook on d8. But, a closer look will reveal something: white has a bishop protecting the d1 square.

So, do you go on to attack or play it safe and find a different strategy? Well, let's evaluate the position, keeping the d1 square in mind.

For black:

• Queen can attack d1.

• Rook can support and attack d1.

• Bishop can support d1.

For white:

• Bishop is protecting d1.

From this evaluation, you can see that black has a very good advantage. In order to win this game, you must sacrifice the queen. Beginners often avoid sacrifices because we learn chess with the idea that the most power pieces must stay on the board. In this case, it’s the only path to victory.

So:

1. ... Qd1+

That brings us to this position:

Black has two options here:

• Capture the queen with the bishop.

• Resign.

Let's go with capturing the queen.

2. Bxd1

And now, we have this position:

Remember the rook on d8?

2. ... Rxd1#

The rook captures the bishop and wins the game. The king can't capture the rook because it has support from the bishop on g4, it can't escape through e2 because the bishop is attacking that square, and the pawn on f2 is blocking the king.

This shows that you don't have to play until the endgame to win in chess. If you get better at recognizing patterns, you'd be shocked at how many chances you've missed to increase your advantage or win a game.

Anastasia's Mate

Anastasia's Mate is a checkmate pattern that involves a knight and a queen (or rook) working together to trap the opposing king. It involves the following:

• The rook or queen delivering a check.

• The knight blocking the king's escape squares (usually two escape squares).

• The opposing king trapped by its own pieces.

Consider this position:

From the look of things, white has a stronger position. That would be totally true if black didn't have a mate in two opportunity. When black plays Ne2+, it sets up the pattern for Anastasia's mate. That is:

The king has two possible squares to move to: h1 and h2.

1. ... Ne2+

2. Kh2

Is the king safe?

2. ... Rh5#

The game ends:

The e2 knight blocks the king from going to either g1 or g3, while the rook attacks the h-file. It's easy to miss this pattern for beginners, because we're often used to endgame patterns that involve more "offensive" attacks.

Here's another example from Lichess:

It's not very obvious that this position leads to an Anastasia's mate. You have a bishop attacking the queen, and the queen can capture it and gain more material advantage. But what happens when you sacrifice the queen by capturing the h7 square?

You've sacrificed the queen to force the king to the h7 square, setting it up for Rh5#.

So, even though you may recognize what the Anastasia's Mate looks like, you may have to intentionally lose some pieces to achieve it. This is also applicable to other aspects of chess. Sacrificing pieces can be a perfect way to force your opponent into dangerous situations.

Back Rank Mate

This is a checkmate pattern that you'll encounter a lot in your chess journey. It involves trapping the king behind the pieces in front of it. That is:

From the image above, you can see the white king trapped behind the three pawns in front of it. To avoid this, you should always remember to create space for your king at some point after castling.

Here's an example from Lichess:

It's white's turn to play, and the black king still has three pieces blocking it. The white queen and the rook are both attacking the e8 square, so you can force an exchange that ends the game:

1. Rxe8+ Rxe8

2. Qxe8#

After the trade, you'll have the queen on e8, forcing a checkmate because the king has nowhere to escape to.

Smothered Mate

The smothered mate is a checkmate pattern where a knight delivers mate to the enemy king, which is completely trapped/surrounded (or “smothered”) by its own pieces.

Here's what it looks like:

This is a checkmate pattern that shows how powerful the knight can be in endgames. With any other piece, it would be difficult for black to make any significant moves in this situation. But since the knight can jump over pieces when moving and attacking, it can be used in unpredictable ways.

Let's look at an example from Lichess:

Again, we'll be sacrificing a piece to gain an advantage. You can sacrifice the white queen with:

1. Qxh7+

This forces the black queen to capture:

1. ... Qxh7

The problem with this is that the king is now trapped:

As beginners, it's often easier to capture a powerful piece than to analyze a position for potential mate patterns. It wouldn’t be surprising to see someone new to chess capturing the queen with the knight.

The best move here is Nf7#:

The king is under check and can't escape because it has been trapped by its own pieces.

Conclusion

The handbook showed you some of the common checkmate patterns in chess. But these are not all of them, and not all checkmate patterns are easy to understand for beginners.

While you improve other aspects of your game like strategy and tactics, opening principles, solving puzzles, and so on, it becomes easier to understand certain concepts that seemed too hard at first.

One way to test your understanding of different concepts is by playing against other people or engines. A good place to do this is on Lichess. It’s free, has an active community, and a ton of tools to help you improve your game.

I hope this handbook has been helpful for you on your chess journey. You can find me on Lichess @ IHECHIKARA.

You can watch the video version of this handbook here.

Exercise Answers

Here are the answers to the exercises and practice positions:

Files a to f Exercise

The arrow is on the c-file. You can tell by looking at the alphabet that corresponds with the vertical line.

Ranks 1 to 8 Exercise

The arrow is on the eight rank. You can tell by looking at the number that corresponds with the horizontal line.

How to Denote Piece Promotion in Algebraic Chess Notation

• The notation for a pawn that moves to d1 and gets promoted to a rook is d1=R.

• The notation for a pawn that moves to b8 and gets promoted to a bishop d1=B.

• Here is the final board position for the given notations:

Practice Position URLs

Here are the links for all the practice positions:

• Two Rooks vs King Endgame/Checkmate Pattern URL: https://lichess.org/editor/1r4k1/8/8/5r2/8/3K4/8/8_b_-_-_0_1?color=black.

• One Rook and King vs King Endgame/Checkmate Pattern URL: https://lichess.org/editor/6k1/8/8/5r2/8/3K4/8/8_w_-_-_0_1?color=black.

• Queen and King vs King Endgame/Checkmate Pattern URL: https://lichess.org/editor/1r4k1/8/8/5r2/8/3K4/8/8_b_-_-_0_1?color=black (choose to play as black).

• Practice many checkmate patterns here: https://lichess.org/practice/checkmates/checkmate-patterns-i/fE4k21MW/9rd7XwOw. Click on “Practice list” to access more patterns.

In Go, the official MongoDB driver provides a robust way to connect to and interact with MongoDB, a flexible NoSQL database that stores data in JSON-like documents.

In this tutorial, you won't just learn how to connect Go to MongoDB. You'll take it a step further by building a simple blog application. Along the way, you'll learn how to perform essential CRUD (Create, Read, Update, Delete) operations and display your results using the Gin web framework.

Table of Contents

• Prerequisites

• Create a New Go Project

• Basic MongoDB Operations

  • Insert data into the collection

  • Find documents in MongoDB

  • Update documents in MongoDB

  • Delete documents in MongoDB

• How to Build a Blog App with go-mongodb-driver and Gin

  • Initialize the Gin application

  • Create the HTML templates

  • Create the handlers

  • Run the application

• That's How to Use MongoDB with Go

Prerequisites

Before you proceed, ensure that you have the following:

• Basic knowledge of Go and its concepts

• Go (version 1.24 or higher) installed

• MongoDB Installed (running locally on port 27017)

• Basic knowledge of NoSQL

Create a New Go Project

First, create a new Go project, change into the new project directory, and initialize a new Go module by running the following commands:

mkdir go-mongodb-integration
cd go-mongodb-integrationgo 
mod init go-mongodb

Next, install the MongoDB Go driver by running the following command:

go get go.mongodb.org/mongo-driver/mongo
go get go.mongodb.org/mongo-driver/bson

The standard Go library includes the database/sql package for working with SQL databases, but it doesn't support MongoDB out of the box. To work with MongoDB in Go, you’ll use the official MongoDB driver, which provides everything you need to connect to and interact with a MongoDB database.

With the basic setup complete, let’s now examine basic operations in MongoDB.

Basic MongoDB Operations

In MongoDB, databases and collections are created automatically upon the first data insertion, adopting a "lazy creation" approach. Specifically, a database is created when you insert your first document, and a collection is likewise created when data is first inserted into it.

It's important to note that functions like client.Database() and db.Collection() only generate references to these structures – they don’t create the actual database or collection until data is inserted.

Insert data into the collection

Let’s walk through how to insert a document into a collection in MongoDB.

First, open your project in a code editor, create a main.go file, and add the following code:

package main

import (
    "context"
    "log"
    "time"

    "go.mongodb.org/mongo-driver/bson/primitive"
    "go.mongodb.org/mongo-driver/mongo"
    "go.mongodb.org/mongo-driver/mongo/options"
)

type User struct {
    ID   primitive.ObjectID `bson:"_id,omitempty"`
    Name string             `bson:"name"`
    Age  int                `bson:"age"`
}

func main() {
    clientOptions := options.Client().ApplyURI("mongodb://localhost:27017")

    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()

    client, err := mongo.Connect(ctx, clientOptions)
    if err != nil {
        log.Fatal(err)
    }
    defer client.Disconnect(ctx)

    err = client.Ping(ctx, nil)
    if err != nil {
        log.Fatal(err)
    }

    db := client.Database("test_db")
    usersCollection := db.Collection("users")

    newUser := User{
        Name: "John Doe",
        Age:  30,
    }

    result, err := usersCollection.InsertOne(ctx, newUser)
    if err != nil {
        log.Fatal(err)
    }

    log.Printf("Inserted user with ID: %v\n", result.InsertedID)
}

In the code above, you define a User struct that represents your document structure, then insert a new user document into the collection using the InsertOne method. When you run this insert operation, MongoDB automatically creates both the test_db database and the users collection if they don’t already exist.

Execute the code by running:

go run main.go

You should see the response below, indicating that a user was inserted successfully.

Find documents in MongoDB

Now that you've inserted some data, it's time to query the database and retrieve documents.

Update your main.go file with the following code:

package main

import (
    "context"
    "fmt"
    "log"
    "time"

    "go.mongodb.org/mongo-driver/bson"
    "go.mongodb.org/mongo-driver/bson/primitive"
    "go.mongodb.org/mongo-driver/mongo"
    "go.mongodb.org/mongo-driver/mongo/options"
)

type User struct {
    ID   primitive.ObjectID `bson:"_id,omitempty"`
    Name string             `bson:"name"`
    Age  int                `bson:"age"`
}

func main() {
    clientOptions := options.Client().ApplyURI("mongodb://localhost:27017")

    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()

    client, err := mongo.Connect(ctx, clientOptions)
    if err != nil {
        log.Fatal(err)
    }
    defer client.Disconnect(ctx)

    db := client.Database("test_db")
    usersCollection := db.Collection("users")

    cursor, err := usersCollection.Find(ctx, bson.M{})
    if err != nil {
        log.Fatal(err)
    }
    defer cursor.Close(ctx)

    var users []User
    if err = cursor.All(ctx, &users); err != nil {
        log.Fatal(err)
    }

    for _, user := range users {
        fmt.Printf("User: %s, Age: %d, ID: %s\n", user.Name, user.Age, user.ID.Hex())
    }
}

In the code above, you use the Find method with an empty filter (`bson.M{}`) to retrieve all documents from the users collection. Then, you use cursor.All to decode all the results into a slice of User structs.

Each document is printed to the terminal, showing the name, age, and ID of every user in the collection.

To run the code, use:

go run main.go

You should see the response below in your terminal.

Update documents in MongoDB

To update a document in your collection, modify your main.go file as shown below:

package main

import (
    "context"
    "log"
    "time"

    "go.mongodb.org/mongo-driver/bson"
    "go.mongodb.org/mongo-driver/bson/primitive"
    "go.mongodb.org/mongo-driver/mongo"
    "go.mongodb.org/mongo-driver/mongo/options"
)

type User struct {
    ID   primitive.ObjectID `bson:"_id,omitempty"`
    Name string             `bson:"name"`
    Age  int                `bson:"age"`
}

func main() {
    clientOptions := options.Client().ApplyURI("mongodb://localhost:27017")
    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()

    client, err := mongo.Connect(ctx, clientOptions)
    if err != nil {
        log.Fatal(err)
    }
    defer client.Disconnect(ctx)

    db := client.Database("test_db")
    usersCollection := db.Collection("users")

    var userToUpdate User
    err = usersCollection.FindOne(ctx, bson.M{"name": "John Doe"}).Decode(&userToUpdate)
    if err != nil {
        log.Println("No user found to update")
    } else {
        update := bson.M{
            "$set": bson.M{
                "name": "Jane Doe",
                "age":  25,
            },
        }
        result, err := usersCollection.UpdateOne(
            ctx,
            bson.M{"_id": userToUpdate.ID},
            update,
        )
        if err != nil {
            log.Fatal(err)
        }
        log.Printf("Updated %v document(s)\n", result.ModifiedCount)
    }
}

In the code above, you first search for a user named "John Doe" using the FindOne method. If a match is found, you use the UpdateOne method to update their name and age. The $set operator ensures that only the specified fields are updated, leaving the rest of the document unchanged.

Execute the code by running:

go run main.go

You should see output in your terminal indicating how many documents were updated.

Delete documents in MongoDB

To remove documents from a collection, you can use the DeleteOne method. Update your main.go file with the following code:

package main

import (
    "context"
    "log"
    "time"

    "go.mongodb.org/mongo-driver/bson"
    "go.mongodb.org/mongo-driver/bson/primitive"
    "go.mongodb.org/mongo-driver/mongo"
    "go.mongodb.org/mongo-driver/mongo/options"
)

type User struct {
    ID   primitive.ObjectID `bson:"_id,omitempty"`
    Name string             `bson:"name"`
    Age  int                `bson:"age"`
}

func main() {
    clientOptions := options.Client().ApplyURI("mongodb://localhost:27017")
    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()

    client, err := mongo.Connect(ctx, clientOptions)
    if err != nil {
        log.Fatal(err)
    }
    defer client.Disconnect(ctx)

    db := client.Database("test_db")
    usersCollection := db.Collection("users")

    result, err := usersCollection.DeleteOne(ctx, bson.M{"name": "Jane Doe"})
    if err != nil {
        log.Fatal(err)
    }
    log.Printf("Deleted %v document(s)\n", result.DeletedCount)
}

In the code above, you use the DeleteOne method to remove the first document that matches the filter { "name": "Jane Doe" }.

You should see the result below in your terminal.

How to Build a Blog App with go-mongodb-driver and Gin

Now that you understand how to perform basic CRUD operations with MongoDB in Go, you're ready to build a more complete application.

Start by creating a new directory for your project and initializing it as a Go module. Navigate to your chosen directory and run:

mkdir go-blog
cd go-blog
go mod init blog

Next, install the required dependencies:

go get github.com/gin-gonic/gin
go get go.mongodb.org/mongo-driver/mongo
go get go.mongodb.org/mongo-driver/bson

Your project will have the following structure:

go-blog/  
├── main.go  
├── handlers/  
│   └── main.go  
└── templates/  
    ├── index.html  
    ├── post.html  
    ├── create.html  
    └── edit.html

Initialize the Gin application

To initialize a new Gin application, create a new main.go file and add the below code snippet to it:

package main

import (
    "context"
    "log"
    "time"

    "blog/handlers"

    "github.com/gin-gonic/gin"
    "go.mongodb.org/mongo-driver/mongo"
    "go.mongodb.org/mongo-driver/mongo/options"
)

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()

    clientOptions := options.Client().ApplyURI("mongodb://localhost:27017")
    client, err := mongo.Connect(ctx, clientOptions)
    if err != nil {
        log.Fatal(err)
    }
    defer client.Disconnect(ctx)

    err = client.Ping(ctx, nil)
    if err != nil {
        log.Fatal(err)
    }
    log.Println("Connected to MongoDB!")

    db := client.Database("blog_db")
    h := handlers.NewHandler(db)

    router := gin.Default()
    router.LoadHTMLGlob("templates/*")

    router.GET("/", h.HomePage)
    router.GET("/post/:id", h.ViewPost)
    router.GET("/create", h.CreatePost)
    router.GET("/edit/:id", h.EditPost)
    router.POST("/save", h.SavePost)
    router.GET("/delete/:id", h.DeletePost)

    log.Println("Server starting on :8080...")
    router.Run(":8080")
}

The code above sets up the MongoDB connection, initializes the Gin router, and registers your routes.

Create the HTML templates

Now, create the HTML templates for displaying the blog UI.

First, create a templates directory and add the following files:

index.html:

<!DOCTYPE html>  
<html lang="en">  
<head>  
    <meta charset="UTF-8">  
    <meta name="viewport" content="width=device-width, initial-scale=1.0">  
    <title>Go Blog with MongoDB</title>  
    <script src="https://cdn.tailwindcss.com"></script>  
</head>  
<body class="bg-gray-100 min-h-screen">  
    <div class="container mx-auto px-4 py-8">  
        <header class="mb-8">  
            <h1 class="text-3xl font-bold text-center text-blue-600">Go Blog with MongoDB</h1>  
            <div class="flex justify-center mt-4">  
                <a href="/create" class="bg-blue-500 hover:bg-blue-600 text-white px-4 py-2 rounded">Create New Post</a>  
            </div>  
        </header>

        <div class="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-6">  
            {{range .}}  
            <div class="bg-white rounded-lg shadow-md overflow-hidden hover:shadow-lg transition-shadow duration-300">  
                <div class="p-6">  
                    <h2 class="text-xl font-semibold mb-2 text-gray-800">{{.Title}}</h2>  
                    <p class="text-gray-600 mb-4 line-clamp-3">  
                        {{if gt (len .Content) 150}}  
                            {{slice .Content 0 150}}...  
                        {{else}}  
                            {{.Content}}  
                        {{end}}  
                    </p>  
                    <div class="flex justify-between items-center text-sm text-gray-500">  
                        <span>{{.CreatedAt.Format "Jan 02, 2006"}}</span>  
                        <a href="/post/{{.ID.Hex}}" class="text-blue-500 hover:text-blue-700">Read More</a>  
                    </div>  
                </div>  
                <div class="flex border-t border-gray-200">  
                    <a href="/edit/{{.ID.Hex}}" class="w-1/2 py-2 text-center text-sm text-gray-600 hover:bg-gray-100 border-r border-gray-200">Edit</a>  
                    <a href="/delete/{{.ID.Hex}}" class="w-1/2 py-2 text-center text-sm text-red-600 hover:bg-gray-100" onclick="return confirm('Are you sure you want to delete this post?')">Delete</a>  
                </div>  
            </div>  
            {{else}}  
            <div class="col-span-3 text-center py-12">  
                <p class="text-gray-600 text-lg">No posts yet. <a href="/create" class="text-blue-500 hover:underline">Create one</a>!</p>  
            </div>  
            {{end}}  
        </div>  
    </div>  
</body>  
</html>

This template lists all blog posts and includes buttons to create, edit, or delete posts.

post.html:

<!DOCTYPE html>  
<html lang="en">  
<head>  
    <meta charset="UTF-8">  
    <meta name="viewport" content="width=device-width, initial-scale=1.0">  
    <title>{{.Title}} | Go Blog with MongoDB</title>  
    <script src="https://cdn.tailwindcss.com"></script>  
</head>  
<body class="bg-gray-100 min-h-screen">  
    <div class="container mx-auto px-4 py-8">  
        <header class="mb-8">  
            <h1 class="text-3xl font-bold text-center text-blue-600">Go Blog with MongoDB</h1>  
            <div class="flex justify-center mt-4">  
                <a href="/" class="bg-blue-500 hover:bg-blue-600 text-white px-4 py-2 rounded">Back to Home</a>  
            </div>  
        </header>

        <div class="max-w-3xl mx-auto bg-white rounded-lg shadow-md overflow-hidden">  
            <div class="p-6">  
                <h2 class="text-2xl font-bold mb-4 text-gray-800">{{.Title}}</h2>  

                <div class="flex items-center text-sm text-gray-500 mb-6">  
                    <span>Posted: {{.CreatedAt.Format "Jan 02, 2006"}}</span>  
                    {{if ne .CreatedAt .UpdatedAt}}  
                    <span class="mx-2">•</span>  
                    <span>Updated: {{.UpdatedAt.Format "Jan 02, 2006"}}</span>  
                    {{end}}  
                </div>  

                <div class="prose max-w-none">  
                    <p class="text-gray-700 whitespace-pre-line">{{.Content}}</p>  
                </div>  
            </div>  

            <div class="flex border-t border-gray-200">  
                <a href="/edit/{{.ID.Hex}}" class="w-1/2 py-3 text-center text-blue-600 hover:bg-gray-100 border-r border-gray-200">Edit Post</a>  
                <a href="/delete/{{.ID.Hex}}" class="w-1/2 py-3 text-center text-red-600 hover:bg-gray-100" onclick="return confirm('Are you sure you want to delete this post?')">Delete Post</a>  
            </div>  
        </div>  
    </div>  
</body>  
</html>

This template displays a single post.

create.html:

<!DOCTYPE html>  
<html lang="en">  
<head>  
    <meta charset="UTF-8">  
    <meta name="viewport" content="width=device-width, initial-scale=1.0">  
    <title>Create New Post | Go Blog with MongoDB</title>  
    <script src="https://cdn.tailwindcss.com"></script>  
</head>  
<body class="bg-gray-100 min-h-screen">  
    <div class="container mx-auto px-4 py-8">  
        <header class="mb-8">  
            <h1 class="text-3xl font-bold text-center text-blue-600">Go Blog with MongoDB</h1>  
            <div class="flex justify-center mt-4">  
                <a href="/" class="bg-blue-500 hover:bg-blue-600 text-white px-4 py-2 rounded">Back to Home</a>  
            </div>  
        </header>

        <div class="max-w-2xl mx-auto bg-white rounded-lg shadow-md overflow-hidden">  
            <div class="p-6">  
                <h2 class="text-2xl font-bold mb-6 text-gray-800">Create New Post</h2>  

                <form action="/save" method="POST">  
                    <div class="mb-4">  
                        <label for="title" class="block text-gray-700 font-medium mb-2">Title</label>  
                        <input type="text" id="title" name="title" required  
                            class="w-full px-3 py-2 border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500">  
                    </div>  

                    <div class="mb-6">  
                        <label for="content" class="block text-gray-700 font-medium mb-2">Content</label>  
                        <textarea id="content" name="content" rows="10" required  
                            class="w-full px-3 py-2 border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500"></textarea>  
                    </div>  

                    <div class="flex justify-end">  
                        <button type="submit" class="bg-blue-500 hover:bg-blue-600 text-white px-6 py-2 rounded-md">  
                            Save Post  
                        </button>  
                    </div>  
                </form>  
            </div>  
        </div>  
    </div>  
</body>  
</html>

This template allows you to create a new post.

edit.html:

<!DOCTYPE html>  
<html lang="en">  
<head>  
    <meta charset="UTF-8">  
    <meta name="viewport" content="width=device-width, initial-scale=1.0">  
    <title>Edit Post | Go Blog with MongoDB</title>  
    <script src="https://cdn.tailwindcss.com"></script>  
</head>  
<body class="bg-gray-100 min-h-screen">  
    <div class="container mx-auto px-4 py-8">  
        <header class="mb-8">  
            <h1 class="text-3xl font-bold text-center text-blue-600">Go Blog with MongoDB</h1>  
            <div class="flex justify-center mt-4">  
                <a href="/" class="bg-blue-500 hover:bg-blue-600 text-white px-4 py-2 rounded">Back to Home</a>  
            </div>  
        </header>

        <div class="max-w-2xl mx-auto bg-white rounded-lg shadow-md overflow-hidden">  
            <div class="p-6">  
                <h2 class="text-2xl font-bold mb-6 text-gray-800">Edit Post</h2>  

                <form action="/save" method="POST">  
                    <input type="hidden" name="id" value="{{.ID.Hex}}">  

                    <div class="mb-4">  
                        <label for="title" class="block text-gray-700 font-medium mb-2">Title</label>  
                        <input type="text" id="title" name="title" value="{{.Title}}" required  
                            class="w-full px-3 py-2 border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500">  
                    </div>  

                    <div class="mb-6">  
                        <label for="content" class="block text-gray-700 font-medium mb-2">Content</label>  
                        <textarea id="content" name="content" rows="10" required  
                            class="w-full px-3 py-2 border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500">{{.Content}}</textarea>  
                    </div>  

                    <div class="flex justify-between">  
                        <a href="/post/{{.ID.Hex}}" class="text-gray-600 hover:text-gray-800">Cancel</a>  
                        <button type="submit" class="bg-blue-500 hover:bg-blue-600 text-white px-6 py-2 rounded-md">  
                            Update Post  
                        </button>  
                    </div>  
                </form>  
            </div>  
        </div>  
    </div>  
</body>  
</html>

This template is used to edit a post.

Create the handlers

Next, set up the handlers to connect with MongoDB and render the templates. Create a new folder called handlers in your project's root directory, then add a main.go file inside it and insert the following code snippet:

package handlers

import (
    "context"
    "log"
    "net/http"
    "time"

    "github.com/gin-gonic/gin"
    "go.mongodb.org/mongo-driver/bson"
    "go.mongodb.org/mongo-driver/bson/primitive"
    "go.mongodb.org/mongo-driver/mongo"
)

type Post struct {
    ID        primitive.ObjectID `bson:"_id,omitempty" json:"id"`
    Title     string             `bson:"title" json:"title"`
    Content   string             `bson:"content" json:"content"`
    CreatedAt time.Time          `bson:"created_at" json:"created_at"`
    UpdatedAt time.Time          `bson:"updated_at" json:"updated_at"`
}

type Handler struct {
    db         *mongo.Database
    collection *mongo.Collection
}

func NewHandler(db *mongo.Database) *Handler {
    return &Handler{
        db:         db,
        collection: db.Collection("posts"),
    }
}

func (h *Handler) HomePage(c *gin.Context) {
    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()

    cursor, err := h.collection.Find(ctx, bson.M{})
    if err != nil {
        c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
        return
    }
    defer cursor.Close(ctx)

    var posts []Post
    if err = cursor.All(ctx, &posts); err != nil {
        c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
        return
    }

    c.HTML(http.StatusOK, "index.html", posts)
}

func (h *Handler) ViewPost(c *gin.Context) {
    id := c.Param("id")
    objID, err := primitive.ObjectIDFromHex(id)
    if err != nil {
        c.JSON(http.StatusBadRequest, gin.H{"error": "Invalid post ID"})
        return
    }

    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()

    var post Post
    err = h.collection.FindOne(ctx, bson.M{"_id": objID}).Decode(&post)
    if err != nil {
        c.JSON(http.StatusNotFound, gin.H{"error": "Post not found"})
        return
    }

    c.HTML(http.StatusOK, "post.html", post)
}

func (h *Handler) CreatePost(c *gin.Context) {
    c.HTML(http.StatusOK, "create.html", nil)
}

func (h *Handler) EditPost(c *gin.Context) {
    id := c.Param("id")
    objID, err := primitive.ObjectIDFromHex(id)
    if err != nil {
        c.JSON(http.StatusBadRequest, gin.H{"error": "Invalid post ID"})
        return
    }

    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()

    var post Post
    err = h.collection.FindOne(ctx, bson.M{"_id": objID}).Decode(&post)
    if err != nil {
        c.JSON(http.StatusNotFound, gin.H{"error": "Post not found"})
        return
    }

    c.HTML(http.StatusOK, "edit.html", post)
}

func (h *Handler) SavePost(c *gin.Context) {
    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()

    id := c.PostForm("id")
    title := c.PostForm("title")
    content := c.PostForm("content")

    now := time.Now()

    if id == "" {
        post := Post{
            Title:     title,
            Content:   content,
            CreatedAt: now,
            UpdatedAt: now,
        }

        result, err := h.collection.InsertOne(ctx, post)
        if err != nil {
            c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
            return
        }

        log.Printf("Created post with ID: %v\n", result.InsertedID)
    } else {
        objID, err := primitive.ObjectIDFromHex(id)
        if err != nil {
            c.JSON(http.StatusBadRequest, gin.H{"error": "Invalid post ID"})
            return
        }

        update := bson.M{
            "$set": bson.M{
                "title":      title,
                "content":    content,
                "updated_at": now,
            },
        }

        result, err := h.collection.UpdateOne(ctx, bson.M{"_id": objID}, update)
        if err != nil {
            c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
            return
        }

        log.Printf("Updated post with ID: %s (Modified %d documents)\n", id, result.ModifiedCount)
    }

    c.Redirect(http.StatusSeeOther, "/")
}

func (h *Handler) DeletePost(c *gin.Context) {
    id := c.Param("id")
    objID, err := primitive.ObjectIDFromHex(id)
    if err != nil {
        c.JSON(http.StatusBadRequest, gin.H{"error": "Invalid post ID"})
        return
    }

    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()

    result, err := h.collection.DeleteOne(ctx, bson.M{"_id": objID})
    if err != nil {
        c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
        return
    }

    log.Printf("Deleted %d document(s) with ID: %s\n", result.DeletedCount, id)
    c.Redirect(http.StatusSeeOther, "/")
}

The code above contains all the logic for managing blog posts. Here's what each component does:

• Post struct: Defines the structure of a blog post document with fields for ID, title, content, and timestamps. The bson tags specify how fields are stored in MongoDB, while json tags handle JSON serialization.

• Handler struct: Contains a reference to the MongoDB database and the posts collection, providing a centralized way to access the database throughout your handlers.

• NewHandler function: Creates and initializes a new handler instance with the database connection and sets up the posts collection reference.

• HomePage: Retrieves all blog posts from the database using Find() with an empty filter and renders them using the index.html template.

• ViewPost: Fetches a single post by its ObjectID using FindOne() and displays it with the post.html template.

• CreatePost & EditPost: Render the respective forms for creating new posts or editing existing ones.

• SavePost: Handles both creating new posts and updating existing ones. It checks if an ID is provided. If not, it creates a new post using InsertOne(). Otherwise, it updates the existing post using UpdateOne() with MongoDB's $set operator.

• DeletePost: Removes a post from the database using DeleteOne() and redirects back to the homepage.

Run the application

With everything set up, you can now launch your blog. Open your terminal and run:

go mod tidy && go run main.go

Then, visit http://localhost:8080 in your browser to see your blog in action.

That's How to Use MongoDB with Go

In this tutorial, you built a simple blog application using Go and MongoDB. You learned how to connect to a MongoDB database using the official Go driver, perform CRUD operations, and render your results with the Gin web framework.

MongoDB’s flexible, document-based structure makes it a great fit for applications where data models need to evolve over time. It allows you to iterate quickly and adapt as your app grows.

As you expand this project, consider adding features such as user authentication, tagging or categorization, comments, pagination, or search functionality to enhance the user experience.

Cheers to building more with Go and MongoDB!

In this article, you'll learn about:

• What a JSON Web Token (JWT) is

• How JWTs are structured and created

• Different JWT signing techniques and algorithms (Symmetric vs. Asymmetric)

• How JWTs are used in real-world authentication flows

• Important security best practices for using JWTs

Table of Contents

• What is a JWT?

  • JSON Web Tokens Are Made Of Three Elements

  • Asymmetric Signing (RS256) Explained

  • Analogy: The Lock and Key for Asymmetric Signatures

• Symmetric Signing: HS256 (HMAC with SHA-256)

  • How HS256 Verification Works

• JWTs in Action: A Typical Authentication Flow

• JWT Security Best Practices & Considerations

What Is a JWT?

JSON Web Token (JWT) is an open standard (RFC 7519) that defines a compact and self-contained way for securely transmitting information between parties as a JSON object. This information can be verified and trusted because it is digitally signed. JWTs can be signed using a secret (with the HMAC algorithm) or a public/private key pair using RSA or ECDSA.
— Introduction to JWT by jwt.io

While accurate, this definition can be a bit dense at first glance. Imagine you want to send someone a sealed, tamper-proof message. That's essentially what a JSON Web Token (JWT) is. It's a secure message, a special kind of message designed to be sent between two parties which can be assured it came from an expected sender.

Each JWT is digitally signed using either a secret code (for symmetric algorithms like HMAC) or a private key (for asymmetric algorithms like RSA or ECDSA).

This secret code or private key is known only to the system that issues the JWT (often called an authentication provider, like Auth0, AWS Cognito, or Firebase Auth, which handles user logins and identity).

This signature proves two things:

1. Authenticity: It proves the message really came from who it claims to be from.

2. Integrity: It proves that the message hasn't been changed or tampered with since it was signed. If even one character is altered, the signature won't match, and you'll know something is wrong, meaning the contents of the JWT can't be trusted.

JSON Web Tokens Are Made of Three Elements

JWTs are made up of 3 key parts:

1. Header

2. Payload

3. Signature

Header

The header contains metadata information about the token. Think of it like a label on a package – it tells you what’s inside and how it was prepared.

Typically, the header contains:

alg: This specifies the algorithm used to sign the JWT. Common algorithms are HS256 (HMAC with SHA-256) or RS256 (RSA with SHA-256).

typ: This specifies the type of token, which is almost always JWT for standard JSON Web Tokens.

Example (decoded):

{ 
  "alg": "RS256",
  "typ": "JWT" 
}

Payload

This is the second part of the JWT, and it's where the real data or "claims" are stored. Claims are statements about an entity (usually a user) and any other additional data. Using the previous analogy of a package, think of it as the “contents” of the package.

There are three types of claims:

1. Registered Claims: These are predefined claims that are recommended for common use cases. They are not mandatory but are very useful for interoperability. These include:

• iss – issuer, who issued the token (for example, your application’s domain)

• sub – subject, the subject of the token (for example, a User’s ID)

• aud – audience, the audience of the token (that is, who the token is intended for – for example, a specific API)

• exp – expiration, the expiry date as a timestamp

• iat – issued at, when the token was issued as a timestamp

• nbf – not before, when the token becomes valid (that is, the token cannot be used or deemed valid before this timestamp)

• jti – JWT ID, a unique identifier for the token, useful for preventing replay attacks or blacklisting

2. Public Claims: These can be defined by anyone using JWTs. To avoid naming conflicts, it's a good practice to register them or define them using a unique identifier like a URI.

3. Private Claims: These are custom claims created to share specific information between parties who agree on using them. They are entirely up to you and your application's needs.

Example payload:

{
  "sub": "1234567890", //  subject
  "name": "John Doe", // private claim
  "admin": true, // private claim / role
  "iat": 1678886400, // Issued at a specific timestamp
  "exp": 1678890000  // Expires at a specific timestamp
}

Like the header, this JSON object is also Base64Url encoded (a URL-safe variant of Base64 encoding) to form the second part of the JWT string.

Important Note: The payload is encoded*, not encrypted.* This means that anyone can easily decode the JWT and read its contents. Never put sensitive information (like passwords) directly into the payload unless the entire JWT itself is encrypted (which is a separate process called JWE - JSON Web Encryption). The security of a standard JWT comes entirely from the signature, which prevents tampering.

Signature

The signature, as we've already discussed, is the most important part of the JWT. Without it, there's no protection applied to the JWT, meaning no way to validate the origin of the token or its integrity.

The signature is created by taking the encoded header, the encoded payload, and a secret key (or a private key if using asymmetric algorithms like RSA). These are then run through the cryptographic algorithm specified in the header (alg field). For HS256, a shared secret key is used. For RS256, a private key is used to sign, and a corresponding public key is used to verify. We’ll get on to verification soon.

Think of it like a tamper-proof seal on your package, or even better, a wax seal on a letter. If you receive your letter and the wax seal has been broken, you'd naturally believe the contents of the letter may not be original and therefore can't be trusted.

In pseudo-code it would look like this:

Signature = Algorithm( Base64Url(Header) + "." + Base64Url(Payload), SecretKey )

The result of this signing process is the signature, which is also Base64Url encoded to form the third part of the JWT string.

At the end of the whole process your JWT would look like this:

base64EncodedHeader.base64EncodedPayload.base64EncodedSignature

Asymmetric Signing (RS256) Explained

When a JWT uses an algorithm like RS256 (RSA Signature with SHA-256), it employs an asymmetric cryptographic process involving a public and private key pair. This is where the core magic of proving authenticity and integrity happens without needing to share a secret.

The Signing Process (by the Issuer)

The sender (the server that issues the JWT, like Auth0) possesses the private key. This key is kept absolutely secret and secure. Here are the steps:

1. Prepare the data: The server takes the header (which includes the algorithm) and the payload. It Base64Url-encodes them, and then concatenates them with a dot: Base64Url(Header) + "." + Base64Url(Payload).

   For example, with this header:

    {
      "typ": "JWT",
      "alg": "RS256"
    }
   

   And this payload:

    {
      "sub": "1234567890",
      "name": "John Doe",
      "admin": true,
      "iat": 1751494086,
      "exp": 1751497686
    }
   

   This would create a Base64Url-encoded header.payload string like the animated example below:

   

2. Calculate the hash: It then calculates a hash (using SHA-256 in this case) of this combined header and payload string.

3. Sign the hash: Finally, it signs this hash using its private key. This cryptographically transformed hash is the signature part of the JWT.

The JWT is then formed by concatenating the Base64Url-encoded header, the Base64Url-encoded payload, and the Base64Url-encoded signature, separated by dots: header.payload.signature.

The top segment below shows the full JWT token (header.payload.signature):

The Verification Process

This is where the magic happens, and it's often a point of confusion. The public key doesn't "decrypt" the original data like a symmetric key does. Instead, it performs a unique verification process.

The receiver (the client or another server that needs to verify the JWT) possesses the public key. This key does not need to be kept secret – it can be freely distributed.

Here's a step-by-step explanation:

1. Separate the parts: The first thing the receiver does is split the incoming JWT string into its three Base64Url-encoded components: the Header, the Payload, and the Signature.

2. Obtain the public key: The verifier needs the public key that corresponds to the private key used by the issuer. Public keys are often available via a JWKS (JSON Web Key Set) endpoint (for example, your-domain.com/.well-known/jwks.json).

3. Re-create the data to be hashed: The receiver takes the received, Base64Url-encoded Header and the received, Base64Url-encoded Payload. It then combines them exactly as the issuer did: EncodedHeader.EncodedPayload.

4. Compute a local hash (Hash A): This combined string is then put through the same hashing algorithm (for example, SHA-256) that was specified in the JWT's header. This produces a new, locally computed hash (let's call this "Hash A"). This local hash represents what the content should look like if it hasn't been tampered with.

5. "Unsign" the received signature with the public key to get the original signed hash (Hash B): This is the core cryptographic step. The verifier uses the public key (obtained in step 2) to perform a mathematical operation on the received signature. This operation does not create a new signature for comparison. Instead, it effectively "unsigns" or "decrypts" the signature to reveal the original hash ("Hash B") that was produced by the issuer's private key.

   • Crucial Point: This process is for verifying authenticity, not decrypting confidential data. The public key confirms that the signature was indeed created by the corresponding private key, and as part of that confirmation, it returns the original hash that was signed.

6. Compare the hashes: The verifier now has two hashes:

   • Hash A: The hash it computed locally from the received header and payload (from step 4).

   • Hash B: The original hash that was extracted from the received Signature using the public key (from step 5)

7. If Hash A matches Hash B: It proves two critical things:

   1. Authenticity: The token was indeed signed by the legitimate holder of the corresponding private key (for example, Auth0).

   2. Integrity: The content of the header and payload has not been tampered with since it was originally signed. If even a single character in the header or payload were changed, Hash A would be different, and it would not match Hash B. In this case, the JWT is considered valid and its contents can be trusted.

8. If the hashes do NOT match: The token is considered invalid and must be rejected. This indicates either that the JWT was signed by an unauthorised party (a forged token) or that its header or payload has been altered after it was signed.

Analogy: The Lock and Key for Asymmetric Signatures

Private Key: A special, unique key that can lock a box (create a signature). Only the owner has this key.

Public Key: A widely distributed key that can test if a box was locked by the corresponding private key. It can't lock a new box, but it can confirm if an existing lock is authentic.

You don't re-lock the box with the public key. You use the public key to check if the existing lock (the signature) is genuine and corresponds to the contents of the box.

Symmetric Signing: HS256 (HMAC With SHA-256)

While RS256 uses a pair of keys (private for signing, public for verifying), many JWTs you'll encounter are signed symmetrically, most commonly with the HS256 algorithm. HS256 stands for HMAC (Hash-based Message Authentication Code) with SHA-256.

The fundamental difference here is the use of a single, shared secret key for both signing and verification.

How HS256 Signing Works

1. Shared secret key: The issuer (for example, your authentication provider) possesses a single, confidential secret key. This key is known only to the issuer and any parties (like your API) that need to verify the token.

2. Combine header and payload: Just like with asymmetric signing, the issuer takes the Base64Url-encoded Header (which specifies "alg": "HS256") and the Base64Url-encoded Payload, and joins them with a dot.

3. Apply HMAC-SHA256: This combined string is then fed into the HMAC-SHA256 algorithm along with the secret key. The HMAC algorithm uses the secret key to create a unique hash (the signature) of the data. In pseudo-code, it looks like this:

   Signature = HMAC-SHA256( Base64Url(Header) + "." + Base64Url(Payload), SecretKey )

4. Form the JWT: The resulting signature (which is also Base64Url-encoded) is appended to the header and payload with a dot, forming the complete JWT: base64EncodedHeader.base64EncodedPayload.base64EncodedSignature.

How HS256 Verification Works

When a receiver gets an HS256-signed JWT, it goes through a verification process.

First, it separates the parts. The JWT is split into its three Base64Url-encoded components: Header, Payload, and Signature, as we did with asymmetric JWTs.

Then, it obtains the shared secret key. The receiver must also possess the exact same secret key that the issuer used to sign the token. This key is not publicly distributed like a public key – it must be securely provisioned to any entity that needs to verify tokens.

Next, it re-calculates the signature. The receiver does this by taking the received Base64Url-encoded Header and Payload, combining them, and then re-applying the HMAC-SHA256 algorithm using the same secret key. This produces a new, locally computed signature.

Finally, the receiver compares the signature it just calculated locally with the signature it received as part of the JWT.

• If the two signatures match: The token is considered valid. This confirms its authenticity (it came from someone who knows the secret) and integrity (it hasn't been tampered with).

• If the signatures do NOT match: The token is invalid and must be rejected. This indicates either tampering or that it was signed with a different, unknown secret key.

Key Differences and Considerations:

• Key management: With HS256, the secret key must be securely shared and kept confidential by all parties involved in both signing and verifying. This can be more challenging to manage securely at scale compared to the public/private key model, where only the private key needs strict secrecy.

• Performance: HS256 is generally faster to compute than asymmetric algorithms like RS256, making it suitable for high-volume scenarios where the secret key can be securely distributed.

JWTs in Action: A Typical Authentication Flow

Now that you understand how JWTs are structured and signed, let's look at how they're typically used in a real-world web application. This authentication flow is a common pattern you'd encounter.

Step 1: User Logs In:

A user opens a client application (for example, a web browser, mobile app) and enters their login credentials (username and password).

The client sends these credentials securely (always over HTTPS!) to an authentication server (like Auth0, AWS Cognito, or your own backend's authentication endpoint).

Step 2: Authentication Server Issues JWT:

Then the authentication server verifies the user's credentials. If valid, it generates a new JWT. This JWT contains claims (like the user's ID, roles, expiration time) in its payload and is digitally signed by the server's private key (for asymmetric algorithms like RS256) or secret key (for symmetric algorithms like HS256).

The server then sends this signed JWT back to the client.

Step 3: Client Stores JWT:

The client receives the JWT and typically stores it in a secure location, such as browser memory storage, session storage, or an HTTP-only cookie. The method of storage depends on the client type and security considerations.

Step 4: Client Makes API Calls:

When the user wants to access a protected resource on a backend API (for example, their profile data, a private feed), the client includes the JWT in the request.

The standard way to do this is by sending the token in the Authorization header of the HTTP request, prefixed with the word Bearer:

Authorization: Bearer <your_jwt_here>

Step 5: API Verifies JWT & Authorises Request:

Now, the backend API receives the request and extracts the JWT from the Authorization header. The API then performs the JWT verification process depending on the algorithm:

• It checks the token's claims, especially the exp (expiration) claim, to ensure it's still valid.

• If the token is valid, the API trusts the claims within the payload (for example, the user's ID) and proceeds to fulfill the request, potentially using the user's roles to determine if they have permission to access the requested resource.

• If the token is invalid (bad signature, expired, and so on), the API rejects the request, typically with an HTTP 401 Unauthorised status.

This flow is powerful because JWTs are stateless: once issued, the authentication server doesn't need to keep a record of active sessions. The API can verify the token independently, which simplifies scaling and reduces server load.

JWT Security Best Practices and Considerations

While JWTs offer powerful authentication capabilities, using them securely requires careful attention to best practices. Misconfigurations or oversight can lead to significant vulnerabilities.

Always Use HTTPS/TLS:

Crucial: JWTs are encoded, not encrypted, by default. This means anyone who intercepts the token during transmission can easily read its payload. Therefore, JWTs (and all authentication traffic) must always be transmitted over HTTPS (TLS) to encrypt the communication channel itself and prevent eavesdropping.

Protect Your Signing Keys:

Whether it's a private key (for RS256) or a shared secret key (for HS256), these keys are paramount. If an attacker gains access to your signing key, they can forge valid JWTs, impersonate users, and compromise your system. Store these keys securely, preferably in dedicated key management services.

Keep Access Tokens Short-Lived (exp claim):

You should always set short expiration times (for example, 5-15 minutes) for your JWTs used as access tokens. This minimises the window of opportunity for an attacker if a token is compromised.

Since JWTs are stateless, they are hard to revoke immediately once issued. A short lifespan is your primary defense against compromised tokens.

Implement Refresh Tokens (for Longer Sessions):

To maintain user experience with short-lived access tokens, use refresh tokens. A refresh token is a separate, longer-lived token (usually stored more securely) that can be exchanged for a new, short-lived access token when the current one expires, without requiring the user to re-authenticate. Refresh tokens can be revoked by the server, offering better control.

Never Put Sensitive Data in the Payload:

Reiterating this crucial point: the JWT payload is Base64Url encoded, which is easily reversible. Do not put passwords, highly sensitive PII (Personally Identifiable Information), or confidential business data directly into the JWT payload. Only include non-sensitive or publicly available information, or data that's already encrypted by other means.

Validate ALL Claims on Verification:

When verifying a JWT, don't just check the signature. Always validate all relevant claims, including:

• exp (Expiration): Ensure the token hasn't expired.

• iss (Issuer): Verify the token came from the expected authentication server.

• aud (Audience): Ensure the token is intended for your specific API/application.

• nbf (Not Before): Check if the token is active yet.

Consider Token Revocation (for critical cases):

For situations requiring immediate revocation (for example, user password change, account deactivation), typical stateless JWTs are challenging. Strategies include:

• Short expiration times (as above).

• A blacklist/revocation list: Store the jti (JWT ID) of revoked tokens in a database, checking this list on every request. This adds a stateful lookup but provides immediate revocation.

Thanks for reading!

I hope you’ve found this tutorial useful, and as always if you want to ask any questions or hear about upcoming articles, you can always follow me on ‘X’, my handle is @grantdotdev and follow by clicking here.

In this article, I’ll teach you about the React Hook Form library, HTML attributes, and development considerations to make sure your site’s available for all, focusing on:

• blind or visually impaired users, who may use a screen reader

• better user feedback

• visual queues for all

• design considerations for all

Whilst following along with this tutorial, you can either pull down the code from the GitHub repo (by visiting this page), or you can use the inline code snippets within the article.

Pre-requisites for this article:

• Knowledge of React

• Knowledge of writing TypeScript and HTML / JSX.

• Familiarity with Tailwind CSS (not required in order to follow this tutorial)

Table of Contents

• The Initial Basic Form

• Error Handling With React-Hook-Form

• Hooking Up The useForm Methods To Our Form

• Showing Error Messages

• Adding aria-required

• Adding fieldset and legend

• Adding Labels and Using htmlFor

• Do Not Rely on Placeholders Only!

• Give Additional Information With aria-describedBy

• Avoid Tooltips for Critical Information

• Tell Me Something Important

• Focus States and Colouring

• Make Buttons Descriptive

• Final Thoughts

The Initial Basic Form

So if we take a look at the form in its current state, you may think it looks fine. But it’s actually not very accessible, nor does it offer a great user experience.

import { TvIcon } from "@heroicons/react/24/outline";

type FormData = {
    fullName: string;
    email: string;
    password: string;
    confirmPassword: string;
    agreeToTerms: boolean;
};

export const RegistrationForm = () => {
    const onSubmit = () => {
        alert(`Form submitted`);
    };

    return (
        <div className="flex justify-center items-center w-screen h-screen bg-gray-900">
            <div className="w-full max-w-md p-8 bg-black bg-opacity-75 rounded-lg">
                <div className="flex flex-row justify-center items-center gap-x-4">
                    <TvIcon className="h-12 w-12 text-white" />
                    <h1 className="text-7xl font-bold text-center text-red-600 mb-4">Getflix</h1>
                </div>
                <h2 className="text-3xl font-bold text-white mb-6 text-center">
                    Sign Up
                </h2>

                <form onSubmit={onSubmit} className="space-y-6">

                    {/* Full Name */}
                    <div>
                        <input
                            type="text"
                            placeholder="Full Name"
                            className="w-full p-3 rounded bg-gray-700 text-white placeholder-gray-400 "
                        />

                    </div>

                    {/* Email */}
                    <div>
                        <input
                            type="email"
                            placeholder="Email Address"
                            className="w-full p-3 rounded bg-gray-700 text-white placeholder-gray-400 "
                        />

                    </div>

                    {/* Password */}
                    <div>
                        <input
                            type="password"
                            placeholder="Password"
                            className="w-full p-3 rounded bg-gray-700 text-white placeholder-gray-400"
                        />

                    </div>

                    {/* Confirm Password */}
                    <div>
                        <input
                            type="password"
                            placeholder="Confirm Password"
                            className="w-full p-3 rounded bg-gray-700 text-white placeholder-gray-400 "
                        />

                    </div>

                    {/* Agree to Terms */}
                    <div className="flex items-center text-gray-400 text-sm">
                        <input

                            type="checkbox"
                            id="agreeToTerms"
                            className="mr-2"
                        />
                        <label htmlFor="agreeToTerms" className="select-none">
                            I agree to the Terms and Conditions
                        </label>
                    </div>


                    {/* Submit */}
                    <button
                        type="submit"
                        className="w-full py-3 bg-red-600 hover:bg-red-700 text-white rounded font-semibold transition"
                    >
                        Sign Up
                    </button>


                </form>
            </div>
        </div>
    );
};

What’s Wrong With The Form?

• Lack of action feedback – no user feedback means that users can become confused as to whether an action has happened or not. No error messages or feedback offers the user no insight into what they need to do to correct the form.

• No labels for form inputs – No labels for form inputs prevent screen readers from understanding their purpose. Some screen readers may miss placeholders, and once a user types within the input, the placeholder is replaced, losing context and making it hard to return to erroneous inputs.

• Lack of accessibility markup to make the form optimised for screen readers and accessibility tools.

So how do we make this better? Let’s jump right in.

Error Handling With React-Hook-Form

Error handling on forms is a critical aspect of any form submission flow. Without it, the process becomes both chaotic and frustrating for the user. We can alleviate this frustration by adding some useful error messages which explain the issues.

A popular library for working with forms in React is the react-hook-form library. It’s used by over 1.4 million people according to their GitHub statistics.

Go ahead and install it if you don’t have it already:

npm install react-hook-form

We will then implement the basic required functions from the react-hook-form package, using the useForm() hook like so:

// define our type structure to use within the form
type FormData = {
    fullName: string;
    email: string;
    password: string;
    confirmPassword: string;
    agreeToTerms: boolean;
};

// basic usage of `useForm()`
const {
    register,
    handleSubmit,
    watch,
    formState: { errors },
  } = useForm<Inputs>()

Quick Explanation:

• register: One of the key concepts in React Hook Form is to “register” your component / HTML element. This means you can access value of the element for both form validation and when submitting the form.

• handleSubmit: This is the key function needed to submit the form, run validation, and any other configured checks. It can take up to two arguments:

  1. handleSubmit(onSuccess) – called when the submission of the form is valid and can submit ok.

  2. handleSubmit(onSuccess, onFail) – here you can pass the handleSubmit() method two functions: the first will be run when React Hook Form deems the form to be valid, and allows you to continue. The second will be called when the form sees an error. This could be from validation, or another stipulation.

• watch: Watch is a function that monitors a specified element for changes and returns its value. For instance, if you’re watching an input element, you can output the user’s typing in real-time or have another element validate it against a predefined value. A good example is a confirm password matching the previous password field.

• formState: this is an object which holds information about your form. The formState object keeps track of the state of the form, like:

  1. isDirty – true if the user has changed any input.

  2. isValid – true if the form passes all validations.

  3. errors – an object holding any validation errors per field.

  4. isSubmitting – true while the form is being submitted (useful for showing loading spinners)

  5. isSubmitted – true after the form has been submitted.

  6. touchedFields – which fields the user has interacted with.

  7. dirtyFields – which fields the user has modified.

We can use any of these properties by including them in our form state object. We are destructing the errors property so we can use the errors later in our form to either show error messages, or validate that there no errors on the page.

Hooking Up the useForm Methods to Our Form

Now that we know more about the useForm() method and react-hook-form, we need to integrate this with our existing <form/> element. Doing so will allow us to use all the react-hook-form features we’ve discussed so far in our form.

import { TvIcon } from "@heroicons/react/24/outline";
import { useState } from "react";
import { useForm } from "react-hook-form";

type FormData = {
    fullName: string;
    email: string;
    password: string;
    confirmPassword: string;
    agreeToTerms: boolean;
};

export const RegistrationForm = () => {
    const {
        register,
        handleSubmit,
        formState: { errors },
        watch,
    } = useForm<FormData>();

    const onSubmit = () => {
        alert(`Form submitted`);
    };

    return (
        <div className="flex justify-center items-center w-screen h-screen bg-gray-900">
            <div className="w-full max-w-md p-8 bg-black bg-opacity-75 rounded-lg">
                <div className="flex flex-row justify-center items-center gap-x-4">
                    <TvIcon className="h-12 w-12 text-red-500" />
                    <h1 className="text-7xl font-bold text-center text-white mb-4">Getflix</h1>
                </div>
                <h2 className="text-3xl font-bold text-white mb-6 text-center">
                    Sign Up
                </h2>


                <form onSubmit={handleSubmit(onSubmit)} className="space-y-6">

                    {/* Full Name */}
                    <div>
                        <input
                            {...register("fullName", {
                                required: "Full Name is required"
                            })}
                            aria-required
                            type="text"
                            placeholder="Full name"
                            className="w-full p-3 rounded bg-gray-700 text-white placeholder-gray-400 focus:outline-none focus:ring-2 focus:ring-red-500"
                        />
                        {errors.fullName && (
                            <p className="text-red-500 text-sm mt-1">{errors.fullName.message}</p>
                        )}
                    </div>

                    {/* Email */}
                    <div>
                        <input
                            {...register("email", {
                                required: "Email is required",
                                pattern: {
                                    value: /^\S+@\S+$/i,
                                    message: "Invalid email address",
                                },
                            })}
                            type="email"
                            placeholder="Email Address"
                            className="w-full p-3 rounded bg-gray-700 text-white placeholder-gray-400 focus:outline-none focus:ring-2 focus:ring-red-500"
                        />
                        {errors.email && (
                            <p className="text-red-500 text-sm mt-1">{errors.email.message}</p>
                        )}

                    </div>

                    {/* Password */}
                    <div>
                        <input
                            {...register("password", {
                                required: "Please enter your password",
                            })}
                            type="password"
                            placeholder="Password"
                            className="w-full p-3 rounded bg-gray-700 text-white placeholder-gray-400 focus:outline-none focus:ring-2 focus:ring-red-500"
                        />
                        {errors.password && (
                            <p className="text-red-500 text-sm mt-1">{errors.password.message}</p>
                        )}
                    </div>

                    {/* Confirm Password */}
                    <div>
                        <input
                            {...register("confirmPassword", {
                                required: "Please enter your password",
                                validate: (value) =>
                                    value === watch("password") || "Passwords do not match",
                            })}
                            type="password"
                            placeholder="Confirm Password"
                            className="w-full p-3 rounded bg-gray-700 text-white placeholder-gray-400 focus:outline-none focus:ring-2 focus:ring-red-500"
                        />
                        {errors.confirmPassword && (
                            <p className="text-red-500 text-sm mt-1">{errors.confirmPassword.message}</p>
                        )}
                    </div>

                    {/* Agree to Terms */}
                    <div className="flex items-center text-gray-400 text-sm">
                        <input
                            {...register("agreeToTerms", {
                                required: "You must agree to the terms and conditions"
                            })}
                            type="checkbox"
                            id="agreeToTerms"
                            className="mr-2"
                        />
                        <label className="select-none">
                            I agree to the Terms and Conditions
                        </label>

                    </div>
                    {errors.agreeToTerms && (
                        <p className="text-red-500 text-sm mt-1">{errors.agreeToTerms.message}</p>
                    )}


                    {/* Submit */}
                    <button
                        type="submit"
                        className="w-full py-3 bg-red-600 hover:bg-red-700 text-white rounded font-semibold transition"
                    >
                        Sign Up
                    </button>

                    {/* Already have account */}
                    <p className="text-center text-gray-400 text-sm mt-4">
                        Already have an account?{" "}
                        <a href="#" className="text-red-500 hover:underline">
                            Sign In
                        </a>
                    </p>
                </form>
            </div>
        </div >
    );
};

So in the updated form code, we’ve made a few adjustments:

Registered Each Our Elements

For each of our elements we’ve added the register object, and configuring some overrides.

We added the required property to all input fields, which checks if the element has a value. If not, it records the provided name and marks the error as erroneous, updating the errors object with our name and the provided required message.

 {...register("fullName", {
    required: "Full Name is required"
  })}

We’ve added a pattern property on the email’s register object. This allows us to specify a criteria for the value of the input – perfect for passwords, email fields, and other inputs which may have value restrictions, or requirements.

// valid email pattern
pattern: {
    value: /^\S+@\S+$/i,
    message: "Invalid email address",
},

We have also added the validate property to the confirm password element. This is a given function that will run as the user types.

validate: (value) => value === watch("password") || "Passwords do not match"

The validate function inside register is run automatically based on the field's validationMode setting.

By default (if you do not specify the validationMode), React Hook Form runs validation on onChange and onBlur events. This means that:

• When the user types into the input → it triggers validate.

• When the user leaves (blurs) the input → it triggers validate again.

If you wanted to update the custom validation mode, you can override this using the mode setting within useForm() like so:

 const { register, handleSubmit, formState, trigger } = useForm({
    mode: "onSubmit",
  });

If you then want to go an extra step and update the mode per element, overriding the mode setting you just globally set for your form, you can use the trigger() method from useForm like so:

<input
  {...register("email", { required: "Email is required" })}
  onBlur={() => trigger("email")} // validate this field onBlur manually
/>

This allows you to have onSubmit validation set via mode, and then email is triggered via onBlur() too.

Just adding these simple settings within the react-hook-form library already gives us a much better user experience than before – but it isn’t everything. Let’s explore more settings, HTML, and attributes we can add to increase accessibility and user experience.

Showing Error Messages

Form errors can be stored within the formState object we mentioned earlier, but they’re no good there – we need to display them to our users. We can achieve this simply by accessing the destructed errors object, like below:

{errors.password && (
    <p className="text-red-500 text-sm mt-1">{errors.password.message}</p>
)}

The code uses conditional syntax to show the <p> tag only if the errors.password object has a value, indicating an error associated with the password field from useForm() checks. We can then display the error message from errors.password.message, combined with a commonly used erroneous colour like red, to highlight the form’s problems. This can then been applied to all other input fields as per the code above.

Adding aria-required

So we’ve informed the form that certain elements are required and these should be checked when submitting the form. But this alone doesn’t inform visually impaired users that the element is required.

To aid with screen-readers, we can add an aria attribute to our element which will be read by the screen-reader. This property is the aria-required property. This means that when the screen-reader reads out information about the element it will inform the user that this value is required for successful submission.

 <input
    {...register("fullName", {
        required: "Full Name is required"
    })}
    aria-required
    type="text"
    placeholder="Full name"
    className="w-full p-3 rounded bg-gray-700 text-white placeholder-gray-400 focus:outline-none focus:ring-2 focus:ring-red-500"
/>

Adding fieldset and legend

Fieldset elements group <form> controls together, while legend elements provide a description for the grouped controls.

Imagine you have one big form, but it spans two "sections" – for example, a "User Details" section for username, email, and passwords, and an "Address Details" section asking for your shipping and billing information.

In this tutorial, we’re using TailwindCSS, which provides a utility class called sr-only. You can apply sr-only to your legends so they are only visible to screen readers, and not actually visible on the page.

This way, the legend will be read aloud when users navigate into a section of the form, making it clear which part of the form they are interacting with.

Important Note: Legends must be placed inside fieldsets. You need to wrap your legends within a <fieldset> element for your HTML to be valid and accessible.

Here's an unrelated example (to keep it brief and simple):

  <fieldset>
    <legend>Payment Method</legend>    
    <label>
      <input type="radio" name="payment" value="card" />
      Credit Card
    </label>

    <label>
      <input type="radio" name="payment" value="paypal" />
      PayPal
    </label>
  </fieldset>

You can see that the payment option inputs have been grouped within a fieldset, and then described by the legend element, informing the user that these elements relate to “Payment Method”. You as the developer can then decide if you would like this shown to everyone, or if it’s only for visually impaired users.

For screen readers, they’d hear something like:

"Group: Payment Method. Credit Card radio button. PayPal radio button."

Do Not Rely on Placeholders Only!

Placeholders are a great addition to make it clear to the user what the input elements are used for, and show helpful information. But they aren’t that user friendly, especially in regards to screen-readers.

The main reasons for this are:

• Placeholders disappear when typing, meaning that if a user begins to type “Grant”, and then tabs away from the input when they go back, without a label it will simply read the value of the input, not what it relates to.

• Often developers utilise a grey-like colour for their placeholders, with a low opacity. This can mean it’s difficult for users to sometimes see the placeholder, especially those who are colour blind or visually impaired.

So what can we do instead ? Well this leads me onto our next point – we can use a common HTML element, the <label/>.

Adding Labels and Using htmlFor

Another accessibility feature we can add to boost our accessibility and user experience for all, is the htmlFor attribute combined with the <label/> element.

Labels are highly important for both sighted and visually impaired users. It offers clarity as to what the input is associated with, as well as a navigational tool for those using screen-readers.

The htmlFor attribute is used to link <label/> elements with their input.

Note: htmlFor attributes can only be used on labels and are not valid on any other element.

<label htmlFor="fullname" className="text-white">Full Name</label>
<input
    {...register("fullName", {
        required: "Full Name is required"
    })}
    id="fullname"
    aria-required
    type="text"
    placeholder="Full name"
    className="w-full p-3 rounded bg-gray-700 text-white placeholder-gray-400 focus:outline-none focus:ring-2 focus:ring-red-500"
/>

Why this is important for accessibility:

1. Screen readers:

When a screen reader lands on the <input>, it automatically reads the associated label ("Full Name"). Even if the label is not visually right next to the input, the screen reader still knows which text describes the input, giving you some freedom when designing your forms.

2. Click behaviour:

When you click the <label>, it automatically focuses the <input> when using htmlFor.

Users don’t have to click exactly on the tiny input field – and this can certainly be useful when dealing with checkboxes or radio buttons, for example.

In short, big click targets = better usability and faster form filling.

This is also very helpful for mobile users where precision tapping is hard, especially on smaller screens.

Give Additional Information With aria-describedBy

Now that we’ve added clear labels to our form fields, we can take accessibility a step further by providing additional guidance for users when errors occur. By using aria-describedby and aria-invalid, we can link helpful error messages to the input fields and ensure screen readers communicate validation issues clearly. Let’s look at how to implement this:

<div>
  <label htmlFor="email" className="text-white">Email</label>
  <input
    {...register("email", {
          required: "You must enter an email address",
      pattern: {
        value: /^\S+@\S+$/i,
        message: "Invalid email address",
      },
    })}
    id="email"
    type="email"
    aria-invalid={errors.email ? "true" : "false"}
    aria-describedby={errors.email ? "email-error" : undefined}
    placeholder="Enter your email address"
    className="w-full p-3 rounded bg-gray-700 text-white placeholder-gray-400 focus:outline-none focus:ring-2 focus:ring-red-500"
  />
  {errors.email && (
    <p id="email-error" className="text-red-500 text-sm mt-1">
      {errors.email.message}
    </p>
  )}
</div>

Notice the two new attributes we’ve added:

• aria-describedBy – this attribute links our error message with our input. Screen readers will therefore read out the error message whilst reading out other information when the input is focused.

• aria-invalid – this attribute again aids with screen readers, informing the user that the input’s value is invalid and they need to correct it. This combined with the describedBy attribute gives visually impaired users all the information they need in order to correct their mistake.

Avoid Tooltips for Critical Information

When developing your form, try to avoid tooltips (those little elements that show when you hover over another element for a period of time like below).

The problems with using tooltips are:

1. They often require mouse hover, which doesn't work on touch devices (for example mobile phones, or tablets).

2. They aren’t announced reliably by screen readers if proper aria labels aren’t added.

3. They disappear too quickly

Instead, we can use inline helper text or descriptions combined with aria-describedby like below:

<p id="passwordHint" className="text-xs text-gray-500">
  Must be at least 8 characters and include a number.
</p>

We can then reference this within our input using the aria-describedBy attribute. But wait, we already have a describedBy pointing at the error message – well, that’s ok! We can link multiple elements, like the brief example below:

// now references both passwordHint and the password error (we seperate the ids with a space)
<input 
  id="password"
  aria-describedby="passwordHint passwordError"
/>

<div id="passwordHint">
  Must be at least 8 characters long.
</div>

<div id="passwordError">
  Passwords do not match!
</div>

Tell Me Something Important

aria-live is an aria attribute you can add to an element to tell screen readers:

"Hey, if the content inside me changes, announce it automatically."

It makes dynamic content updates audible without needing the user to re-focus anything.

A basic example could look something like below, where a message which is updated upon submission is updated, it could contain something like:

“Loading” → “Hurray, registration complete”

or

““Pending” → “Registration failed due to many errors”

<p aria-live="polite">
  {formSubmissionResultMessage}
</p>

When formSubmissionResultMessage changes, screen readers will automatically announce the updated message.

The timing of when it is read out depends on the value of the aria-live attribute – with polite, the announcement waits for a natural pause. With assertive, it interrupts immediately.

Real-World Examples

Polite update: good for passive notifications

<p aria-live="polite" className="mt-2 text-green-500">
  Form saved successfully.
</p>

The screen reader waits for a good moment to say it.

Assertive update: good for urgent errors

<p aria-live="assertive" className="mt-2 text-red-500">
  Passwords do not match!
</p>

The screen reader immediately interrupts and announces it.

Good things to know:

• The element needs to already exist in the DOM when the update happens. So it’s smart to always render the <p aria-live> – just update its content.

• Don’t overuse assertive, or you’ll annoy users and make apps feel super noisy and overwhelming.

Focus States and Colouring

You may have noticed on the input elements that I have added some custom colouring with TailwindCSS classes focus:. But what is this doing?

Well, this allows us to control the focus colour of the inputs. Without this, the browser will apply its own default styling which may not be as accessible to our users, especially those with colour-blindness.

For example, within our form, without the styling the input with focus looks like this:

Here you can see it has applied a subtle white and blue outline – but its not that clear it’s being focused. You can argue it is different enough to other input elements, but for some users this may not be enough.

To combat this and improve usability, we can override this with our own custom colouring. When using TailwindCSS, we can apply the following class names:

focus:outline-none focus:ring-2 focus:ring-red-500

What Does This Do?

This now applies a much thicker red line (encompassing brand colours) as well as making it clearer against the darker background

If you’re not using TailwindCSS, you can accomplish the same with plain CSS like so:

input:focus {
  outline: none; /* no default browser outline */
  box-shadow: 0 0 0 2px #ef4444; /* 2px red ring around input */
}

Make Buttons Descriptive

A super simple way to level up your form’s user experience is to make sure your buttons use clear, descriptive text.

Let’s take a look at a few examples of buttons that don’t quite achieve this:

The above buttons are examples of poor input buttons because:

• “Click Here” doesn’t give any context. Screen reader users, and even sighted users, have no idea what "click here" does without reading nearby text.

• Icon Only: Sighted users might guess what the icon means, but screen readers see nothing unless you add aria-label. The point is, it is ambiguous and unclear as to what the button does. You may see websites that just use an icon, not surrounded by a button, which can be even more confusing.

• “Submit”: If you have several "Submit" buttons (for example, one for payment, one for contact form), users don't know which "submit" is doing what.

Improvements

Instead, we can improve those buttons to be more accessible and user-friendly by doing the following:

• Use descriptive button text – for example: "Pay Now", "Sign Up", or "Save Changes".

• Use both an icon and text – combining an icon with text can be the perfect blend for both accessibility and design.

• Use aria-label – if you really must use an icon-only button (like a basket or home icon in a navigation bar), make sure to add an aria-label attribute to clearly describe the button’s purpose, like so:

<button 
    type="submit"
    className="w-full py-3 px-6 rounded-lg bg-red-600 hover:bg-red-700 focus:outline-none focus:ring-2 focus:ring-red-500 text-white text-lg font-semibold transition"
> Pay Now <button>

<button
    type="submit"
    className="w-full py-3 px-6 rounded-lg bg-blue-600 hover:bg-blue-700 focus:outline-none focus:ring-2 focus:ring-blue-500 text-white text-lg font-semibold flex justify-center items-center gap-2 transition">
        <HomeIcon className="h-6 w-6" />
        Home
</button>

<button
    type="submit"
    aria-label="Go to homepage"
    className="w-full py-3 px-6 rounded-lg bg-blue-600 hover:bg-blue-700 focus:outline-none focus:ring-2 focus:ring-blue-500 text-white text-lg font-semibold flex justify-center items-center transition">
        <HomeIcon className="h-6 w-6" />
</button>

That code would generate the following:

Final Thoughts

In this tutorial, we’ve covered various ways to make your forms more accessible and user-friendly. From simple things like making button text clearer and using more user-friendly colours, to more complex HTML attributes like aria-describedBy and aria-live, you should be covered.

I hope you found this tutorial helpful, and now you’re ready to take your development skills to the next level. Making these simple changes can have a big impact on your users’ experience, and they’ll definitely stick around longer and be less frustrated.

As always, if you’d like to share feedback on the article, discuss it further, or just hear about future articles or content, you can drop me a follow on X (Twitter) via my handle @grantdotdev.

In this article, I’ll teach you how to use the TestContainers library to make running integration tests much easier and more manageable.

Table of Contents

• Prerequisites

• What Is TestContainers?

• How Does It All Work?

• How to Set Up Your First Test

• Key Behaviors of IAsyncLifetime in a Test Class

• How to Improve Performance

• Explanation of Differences

• How to Share Your Container Across Multiple Test Classes

• Summary of Approaches:

• How to Create Multiple Containers

• How to Make Your Setup Easier With Custom Images

• Final Thoughts

Prerequisites

• Understanding of Docker

• Understanding of xUnit and testing

• Installation of the following packages:

  • TestContainers

  • TestContainers.MsSql

  • xUnit

  • >= .Net 8

  • FluentAssertions

  • Microsoft.Data.SqlClient

What Is TestContainers?

TestContainers is an open source library that provides you with easily disposable container instances for things like database hosting, message brokers, browsers and more – basically anything that can run in a Docker container.

It removes the necessity to maintain hosted environments for testing in the cloud or on local machines. As long as the user’s machine and CI/CD host supports Docker, the testContainer tests can easily be run.

How Does It All Work?

You define the image you’re wanting to utilise, and specify a configuration.

The TestContainer library spins up a Docker Container with the configured image.

Provides Connection Details

After starting the container, TestContainers exposes connection strings (for example, a database connection URL), so your tests can use the real service, rather than having to configure this yourself.

Cleans Up Automatically

When the test finishes, TestContainers removes the container automatically, ensuring no leftover resources. This is one of the best things about using TestContainers: all the creation, tear down, and container setup is handled within the library itself, making it perfect for use within delivery pipelines.

How to Set Up Your First Test

For the purpose of this tutorial, we’re going to keep things simple, and only use a MS Sql Server image.

The first thing we’re going to do is configure our Microsoft SQL Server Docker container via the TestContainer fluid API.

Create your test class like below:

public class IntegrationTests: IAsyncLifetime 
{
    private MsSqlContainer _container;
    private FakeLogger _logger

    public async Task InitializeAsync()
    {
           _container = new MsSqlBuilder()
                .WithImage("mcr.microsoft.com/mssql/server:2022-latest")
                .WithPassword("P@ssw0rd123")
                .WithPortBinding(1443)
                .WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(1433))
                .Build();

            _logger = new FakeLogger();
    }

    public async Task DisposeAsync() => await _container.DisposeAsync();
}

Here we’re using xUnit’s IAsyncLifetime interface. It’s an interface in xUnit that provides a way to handle async setup and teardown for test classes. It's useful when you need to initialise and clean up resources asynchronously. We’re using the InitializeAsync() to setup and define our Microsoft SQL Database container as well as starting the container, then using the DisposeAsync() method to stop and dispose of our container.

Explanation of Builder Methods

• WithImage(): this allows us to specify the image we want Docker to pull down and run. We’ve opted for the latest version of SQL Server 2022.

• WithPassword(): This allows us to specify the password for the database (when creating most databases, a password is normally required).

• WithPortBinding(): This allows us to specify both the hosting port number on your machine, as well as the container port number

• WithWaitStrategy(): Here we can specify a wait strategy, which informs our container to wait for a condition before the container is ready to use. This is important because some services (like databases or APIs) take time to fully start up.

• Build()" This is the command that builds the test container based on the configuration. This does not run or start the container – you can do this using the container.StartAsync() method as mentioned previously.

Why Is WithWaitStrategy() Needed?

By default, TestContainers assumes the container is ready as soon as it starts running. But some services might:

• Take time to initialize.

• Require a specific log message before they are ready.

• Need a port to be accessible before you can connect.

Using WithWaitStrategy(), you can customise how TestContainers waits before considering the container "ready."

Adding the Test

public class IntegrationTests: IAsyncLifetime 
{
    private MsSqlContainer _container;
    private FakeLoger _logger;

    public async Task InitializeAsync()
    {
           _container = new MsSqlBuilder()
                .WithImage("mcr.microsoft.com/mssql/server:2022-latest")
                .WithPassword("P@ssw0rd123")
                .WithPortBinding(1443)
                .WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(1433))
                .Build();

            await _container.StartAsync();
            _logger = new FakeLogger();
    }

    public async Task DisposeAsync() => await _container.DisposeAsync();

    [Fact]
    public async Task Test_Database_Connection()
    {
        var connectionString = _container.GetConnectionString();
        using var conn = new SqlConnection(connectionString);
        await conn.OpenAsync();

        Assert.True(conn.State == System.Data.ConnectionState.Open);
    }
}

The above test, although it’s simple, illustrates how easy it is to spin up a container and create a simple test. The above test will work, but it can lead to low performing tests and high usage of machine resource when not used correctly. Let me explain:

Using IAsyncLifetime is necessary, as we’re calling async setup methods (StartAsync), for example. But the InitializeAsync() / DisposeAsync() methods when situated in a test class are run before and after every test (Fact in xUnit).

This means that every time a test begins, it is:

• creating a brand new Docker container,

• pulling the MS Sql image,

• creating the DB,

• running the tests, and

• tearing down the container.

You can test this by copying and pasting the above Test_Database_Connection() test multiple times, adding a number to each duplicate test (to keep the compiler happy), and opening Docker Desktop. Running all the tests, you will see a new container (with a different name) being created for each test run.

Now, this can be acceptable if you have a limited number of tests in your test class. But it can have negative outcomes on test classes with a larger number of tests, meaning test maintenance and planning is key. It’s useful, though, when you want to make sure that the database is in a completely clean state before each test, ensuring no data contamination from other tests running.

Key Behaviors of IAsyncLifetime in a Test Class

When your test class implements IAsyncLifetime, xUnit's default behaviour is:

1. Creates a new instance of the test class for each test method.
2. Calls InitializeAsync() before each test.
3. Calls DisposeAsync() after each test.

What Does This Mean for TestContainers?

• In our case, since InitializeAsync() sets up a new container, a new container is created for each test.

• DisposeAsync() stops the container after each test finishes.

• Ensures a completely fresh database state for every test, avoiding data contamination.

• Is slow and resource-intensive, especially if you have many test methods.

A more visual look on a test class could look like this:

🟢 InitializeAsync() -> New Container Created (For Test_1)

🧪 Running Test_1

🛑 DisposeAsync() -> Container Stopped (After Test_1)

🟢 InitializeAsync() -> New Container Created (For Test_2)

🧪 Running Test_2

🛑 DisposeAsync() -> Container Stopped (After Test_2)

When Is This Useful?

• You need a completely fresh database state or container for each test.

• Avoids test data contamination.

• Each test starts from a clean slate.

When Is This a Problem?

• It results in slow execution – a new container is started for every test.

• It’s resource-heavy – multiple containers run sequentially.

• And it’s not scalable – hundreds of tests will take a long time to complete.

How to Improve Performance

Ok, so we’ve seen how to create containers once per test, and explored scenarios where this would be useful, but what if performance and cost are a concern?

Here we can combine IClassFixture and IAsyncLiftetime to achieve a Once per test class approach, where we create one container and one database, and its lifecycle is the full length of the test class (that is, all tests run against the same DB).

How to Write This

We can utilise a TestFixture class which inherits the IAsyncLifetime interface, exposing the InitializeAsync() and DisposeAsync() methods as before.

using DotNet.Testcontainers.Builders;
using Microsoft.Extensions.Logging.Testing;
using Testcontainers.MsSql;

namespace IntegrationTests;

public class TestClassFixture : IAsyncLifetime
{
    public MsSqlContainer Container { get; set; }
    private FakeLogger _logger;

    public async Task InitializeAsync()
    {
        Container = new MsSqlBuilder()
            .WithImage("mcr.microsoft.com/mssql/server:2022-latest")
            .WithPassword("P@ssw0rd123")
            .WithPortBinding(1443)
            .WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(1433))
            .Build();

        _logger = new FakeLogger();
        await Container.StartAsync();
    }

    public async Task DisposeAsync()
    {
        await Container.DisposeAsync();
    }
}

Using xUnit’s IClassFixture interface, we can pass our TestClassFixture and have our test class inherit from this. A test fixture is only run once per test class, making it perfect for our scenario.

public class IntegrationFixtureTests : IClassFixture<TestClassFixture>
{
    private readonly string _connectionString;

    public IntegrationFixtureTests(TestClassFixture testClassFixture)
    {
        _connectionString = testClassFixture.Container.GetConnectionString();

        // other test class specific setup goes here
    }

    [Fact]
    public async Task Test_Database_Connection()
    {
        await using var conn = new SqlConnection(_connectionString);
        await conn.OpenAsync();

        Assert.True(conn.State == System.Data.ConnectionState.Open);
    }
}

We now have a much cleaner test class, and all our container logic is handled by the IClassFixture instead. Should you need to add test class specific code, for example seeding the database prior to running, or the mocking of any resources, you can place this code within the constructor.

Explanation of Differences

We set our Container property as public, rather than private so that our test class can access the container. The test fixture is injected by xUnit's own internal dependency injection mechanics when you use IClassFixture<T>.

xUnit automatically creates an instance of the fixture class and passes it into the test class constructor.

The container is started within the InitializeAsync() method on the TestFixture now, rather than the test class, meaning it only gets started once and is readily available for all the tests. This improves performance and test speeds (no more waiting for each container to spin up before each test).

The test flow would look something more like this now:

🟢 InitializeAsync() → Container Created → Container Started

🧪 Running Test_1

🧪 Running Test_2

🛑 DisposeAsync() -> Container Stopped → Container Disposed of

Advantages and Disadvantages

✅ Faster Execution

Significantly reduces setup/teardown overhead, especially when using slow-starting services like databases.

✅ Lower Resource Usage

Running a container once per test class consumes far fewer system resources compared to one container per test. This is especially beneficial when running integration tests in CI/CD pipelines where resource usage needs to be optimised to keep costs low.

✅ More Realistic Testing

In real-world scenarios, applications don’t restart their databases between API calls, so why should your integration tests?

❌ Data Contamination

Effective test data management is essential for maintaining reliable tests. If test data is not properly isolated, it can lead to unintended interference between tests.

For example, a test that creates a new record might introduce unexpected data, causing a retrieval test to fail if it runs afterward. This type of data contamination is a common issue when all tests in a test class share the same database setup. But,with careful test design—such as proper data isolation, cleanup strategies, or using transactional rollbacks—these issues can be mitigated or entirely avoided.

❌ More Care Needs To Be Taken Around Indempotency

“Indempotency” refers to the ability to run any test on its own in any order. If the test class is accessing data from the same areas, the assertions may fail when ran in certain orders than others. For example:

• Test_1 inserts a record.

• Test_2 assumes the table is empty and asserts QueryByName() should return 1 record

• Test_2 fails because Test_1 has already inserted its own record

How to Share Your Container Across Multiple Test Classes

So we’ve covered a container per test and a container per test class. But what about sharing a container for multiple test classes? Well, it’s as simple as using the ICollectionFixture interface instead of IClassFixture, and it can be used like so:

[CollectionDefinition("Database collection")]
public class DatabaseCollection : ICollectionFixture<TestClassFixture>
{
    // This class has no code, 
    // it’s just used to apply the [Collection] attribute to test classes.
}

The ICollectionFixture<T> mechanism in xUnit automatically ties the fixture instance to all test classes marked with the [Collection("Collection Name")] attribute, for example:

using IntegrationTests;
using Microsoft.Data.SqlClient;

[Collection("Database collection")]
public class IntegrationFixtureTests
{
    private readonly string _connectionString;

    public IntegrationFixtureTests(TestClassFixture testClassFixture)
    {
        _connectionString = testClassFixture.Container.GetConnectionString();
    }

    [Fact]
    public async Task Test_Database_Connection()
    {
        await using var conn = new SqlConnection(_connectionString);
        await conn.OpenAsync();

        Assert.True(conn.State == System.Data.ConnectionState.Open);
    }
}

[Collection("Database collection")]
public class AnotherIntegrationTest
{
    private readonly string _connectionString;

    public AnotherIntegrationTest(TestClassFixture testClassFixture)
    {
        _connectionString = testClassFixture.Container.GetConnectionString();
    }

    [Fact]
    public async Task Another_Database_Test()
    {
        await using var conn = new SqlConnection(_connectionString);
        await conn.OpenAsync();

        Assert.True(conn.State == System.Data.ConnectionState.Open);
    }
}

Now you can group your integration tests, whether it be all read tests or all write tests – making your tests much more maintainable.

Summary of Approaches:

How to Create Multiple Containers

Yes, you can create multiple containers which can host different images, making it perfect for when you have multiple systems you need to integrate with – for example Microsoft SQL Server and a Redis instance.

You can do this by calling the constructor of the relevant TestContainer package like below:

public class TestContainersFixture : IAsyncLifetime
{
    public MsSqlContainer SqlContainer { get; private set; }
    public RedisContainer RedisContainer { get; private set; }

    public async Task InitializeAsync()
    {
        // SQL Server Container
        SqlContainer = new MsSqlBuilder()
            .WithImage("mcr.microsoft.com/mssql/server:2022-latest")
            .WithPassword("P@ssw0rd123")
            .WithPortBinding(1433)
            .WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(1433))
            .Build();

        // Redis Container
        RedisContainer = new RedisContainerBuilder()
            .WithImage("redis:latest")
            .WithPortBinding(6379)
            .WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(6379))
            .Build();

        await Task.WhenAll(SqlContainer.StartAsync(), RedisContainer.StartAsync());
    }

    public async Task DisposeAsync()
    {
        await Task.WhenAll(SqlContainer.DisposeAsync(), RedisContainer.DisposeAsync());
    }
}

And just like that, we have a SQL Server and a Redis instance ready to integrate test against.

How to Make Your Setup Easier With Custom Images

To make testing easier, and leverage the power of Docker and TestContainers, here’s a great tip. TestContainers fully supports using custom images, including pre-configured ones with seeded databases. Instead of defining everything in the test setup, you can build and use a custom Docker image that already contains the required schema and test data.

When creating your own custom package to use, you can:

1. Upload your custom image to DockerHub and reference from there:

 SqlContainer = new MsSqlBuilder()
            .WithImage("your-dockerhub-username/custom-sql-image") 
            .WithPassword("P@ssw0rd123")
            .WithPortBinding(1433)
            .WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(1433))
            .Build();

2. Build your Docker image locally - f you're using a local image in TestContainers, you can simply reference the image name (e.g., my-custom-sql-image) in your code. TestContainers will first check your local Docker Desktop for the image before attempting to pull it from a registry like Docker Hub.

SqlContainer = new MsSqlBuilder()
    .WithImage("custom-sql-image") // Reference your local image
    .WithPassword("P@ssw0rd123")
    .WithPortBinding(1433)
    .WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(1433))
    .Build();

Having a pre-built image can speed up your tests especially in CI/CD pipelines, not to mention make them more readable by removing the seeding code.

To access your custom image in a CI/CD pipeline, you can upload it to DockerHub or GitHub Container Registry (GHCR) and access it from your tests. Build your DockerFile and push it to either system before accessing it in your tests.

Final Thoughts

Using TestContainers in .NET is a game-changer for integration testing. It’s a lightweight and automated way to manage external dependencies like databases, caching systems, and more. By using test containers in a test class, TestFixture, or ICollectionFixture, you can create cleaner, more reliable tests with isolated environments.

TestContainers can also save you money by eliminating the need for dedicated testing environments with long-lived dependencies. You can create and destroy them on the fly, or even integrate them into your CI/CD pipelines, especially in GitHub where Docker can be easily used.

As always I hope you’ve found this article helpful, and if you have any questions don’t hesitate to reach out on X / Twitter - @grantdotdev

Magic methods are special methods that let you define how your objects behave in response to various operations and built-in functions. They're what makes Python's object-oriented programming so powerful and intuitive.

In this guide, you'll learn how to use magic methods to create more elegant and powerful code. You'll see practical examples that show how these methods work in real-world scenarios.

Prerequisites

• Basic understanding of Python syntax and object-oriented programming concepts.

• Familiarity with classes, objects, and inheritance.

• Knowledge of built-in Python data types (lists, dictionaries, and so on).

• A working Python 3 installation is recommended to actively engage with the examples here.

Table of Contents

1. What are Magic Methods?

2. Object Representation

   • str vs repr

   • Practical Example: Custom Error Class

3. Operator Overloading

   • Arithmetic Operators

   • Comparison Operators

   • Practical Example: Money Class

4. Container Methods

   • Sequence Protocol

   • Mapping Protocol

   • Practical Example: Custom Cache

5. Attribute Access

   • getattr and getattribute

   • setattr and delattr

   • Practical Example: Auto-Logging Properties

6. Context Managers

   • enter and exit

   • Practical Example: Database Connection Manager

7. Callable Objects

   • call

   • Practical Example: Memoization Decorator

8. Advanced Magic Methods

   • new for Object Creation

   • slots for Memory Optimization

   • missing for Default Dictionary Values

9. Performance Considerations

10. Best Practices

11. Wrapping Up

12. References

What are Magic Methods?

Magic methods in Python are special methods that start and end with double underscores (__). When you use certain operations or functions on your objects, Python automatically calls these methods.

For example, when you use the + operator on two objects, Python looks for the __add__ method in the left operand. If it finds it, it calls that method with the right operand as an argument.

Here's a simple example that shows how this works:

class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def __add__(self, other):
        return Point(self.x + other.x, self.y + other.y)

p1 = Point(1, 2)
p2 = Point(3, 4)
p3 = p1 + p2  # This calls p1.__add__(p2)
print(p3.x, p3.y)  # Output: 4 6

Let's break down what's happening here:

1. We create a Point class that represents a point in 2D space

2. The __init__ method initializes the x and y coordinates

3. The __add__ method defines what happens when we add two points

4. When we write p1 + p2, Python automatically calls p1.__add__(p2)

5. The result is a new Point with coordinates (4, 6)

This is just the beginning. Python has many magic methods that let you customize how your objects behave in different situations. Let's explore some of the most useful ones.

Object Representation

When you work with objects in Python, you often need to convert them to strings. This happens when you print an object or try to display it in the interactive console. Python provides two magic methods for this purpose: __str__ and __repr__.

str vs repr

The __str__ and __repr__ methods serve different purposes:

• __str__: Called by the str() function and by the print() function. It should return a string that is readable for end-users.

• __repr__: Called by the repr() function and used in the interactive console. It should return a string that, ideally, could be used to recreate the object.

Here's an example that shows the difference:

class Temperature:
    def __init__(self, celsius):
        self.celsius = celsius

    def __str__(self):
        return f"{self.celsius}°C"

    def __repr__(self):
        return f"Temperature({self.celsius})"

temp = Temperature(25)
print(str(temp))      # Output: 25°C
print(repr(temp))     # Output: Temperature(25)

In this example:

• __str__ returns a user-friendly string showing the temperature with a degree symbol

• __repr__ returns a string that shows how to create the object, which is useful for debugging

The difference becomes clear when you use these objects in different contexts:

• When you print the temperature, you see the user-friendly version: 25°C

• When you inspect the object in the Python console, you see the detailed version: Temperature(25)

Practical Example: Custom Error Class

Let's create a custom error class that provides better debugging information. This example shows how you can use __str__ and __repr__ to make your error messages more helpful:

class ValidationError(Exception):
    def __init__(self, field, message, value=None):
        self.field = field
        self.message = message
        self.value = value
        super().__init__(self.message)

    def __str__(self):
        if self.value is not None:
            return f"Error in field '{self.field}': {self.message} (got: {repr(self.value)})"
        return f"Error in field '{self.field}': {self.message}"

    def __repr__(self):
        if self.value is not None:
            return f"ValidationError(field='{self.field}', message='{self.message}', value={repr(self.value)})"
        return f"ValidationError(field='{self.field}', message='{self.message}')"

# Usage
try:
    age = -5
    if age < 0:
        raise ValidationError("age", "Age must be positive", age)
except ValidationError as e:
    print(e)  # Output: Error in field 'age': Age must be positive (got: -5)

This custom error class provides several benefits:

1. It includes the field name where the error occurred

2. It shows the actual value that caused the error

3. It provides both user-friendly and detailed error messages

4. It makes debugging easier by including all relevant information

Operator Overloading

Operator overloading is one of the most powerful features of Python's magic methods. It lets you define how your objects behave when used with operators like +, -, *, and ==. This makes your code more intuitive and readable.

Arithmetic Operators

Python provides magic methods for all basic arithmetic operations. Here's a table showing which method corresponds to which operator:

Comparison Operators

Similarly, you can define how your objects are compared using these magic methods:

Practical Example: Money Class

Let's create a Money class that handles currency operations correctly. This example shows how to implement multiple operators and handle edge cases:

from functools import total_ordering
from decimal import Decimal

@total_ordering  # Implements all comparison methods based on __eq__ and __lt__
class Money:
    def __init__(self, amount, currency="USD"):
        self.amount = Decimal(str(amount))
        self.currency = currency

    def __add__(self, other):
        if not isinstance(other, Money):
            return NotImplemented
        if self.currency != other.currency:
            raise ValueError(f"Cannot add different currencies: {self.currency} and {other.currency}")
        return Money(self.amount + other.amount, self.currency)

    def __sub__(self, other):
        if not isinstance(other, Money):
            return NotImplemented
        if self.currency != other.currency:
            raise ValueError(f"Cannot subtract different currencies: {self.currency} and {other.currency}")
        return Money(self.amount - other.amount, self.currency)

    def __mul__(self, other):
        if isinstance(other, (int, float, Decimal)):
            return Money(self.amount * Decimal(str(other)), self.currency)
        return NotImplemented

    def __truediv__(self, other):
        if isinstance(other, (int, float, Decimal)):
            return Money(self.amount / Decimal(str(other)), self.currency)
        return NotImplemented

    def __eq__(self, other):
        if not isinstance(other, Money):
            return NotImplemented
        return self.currency == other.currency and self.amount == other.amount

    def __lt__(self, other):
        if not isinstance(other, Money):
            return NotImplemented
        if self.currency != other.currency:
            raise ValueError(f"Cannot compare different currencies: {self.currency} and {other.currency}")
        return self.amount < other.amount

    def __str__(self):
        return f"{self.currency} {self.amount:.2f}"

    def __repr__(self):
        return f"Money({repr(float(self.amount))}, {repr(self.currency)})"

Let's break down the key features of this Money class:

1. Precision handling: We use Decimal instead of float to avoid floating-point precision issues with money calculations.

2. Currency safety: The class prevents operations between different currencies to avoid errors.

3. Type checking: Each method checks if the other operand is of the correct type using isinstance().

4. NotImplemented: When an operation doesn't make sense, we return NotImplemented to let Python try the reverse operation.

5. @total_ordering: This decorator automatically implements all comparison methods based on __eq__ and __lt__.

Here's how to use the Money class:

# Basic arithmetic
wallet = Money(100, "USD")
expense = Money(20, "USD")
remaining = wallet - expense
print(remaining)  # Output: USD 80.00

# Working with different currencies
salary = Money(5000, "USD")
bonus = Money(1000, "USD")
total = salary + bonus
print(total)  # Output: USD 6000.00

# Division by scalar
weekly_pay = salary / 4
print(weekly_pay)  # Output: USD 1250.00

# Comparisons
print(Money(100, "USD") > Money(50, "USD"))  # Output: True
print(Money(100, "USD") == Money(100, "USD"))  # Output: True

# Error handling
try:
    Money(100, "USD") + Money(100, "EUR")
except ValueError as e:
    print(e)  # Output: Cannot add different currencies: USD and EUR

This Money class demonstrates several important concepts:

1. How to handle different types of operands

2. How to implement proper error handling

3. How to use the @total_ordering decorator

4. How to maintain precision in financial calculations

5. How to provide both string and representation methods

Container Methods

Container methods let you make your objects behave like built-in containers such as lists, dictionaries, or sets. This is particularly useful when you need custom behavior for storing and retrieving data.

Sequence Protocol

To make your object behave like a sequence (like a list or tuple), you need to implement these methods:

Mapping Protocol

For dictionary-like behavior, you'll want to implement these methods:

Practical Example: Custom Cache

Let's implement a time-based cache that automatically expires old entries. This example shows how to create a custom container that behaves like a dictionary but with additional functionality:

import time
from collections import OrderedDict

class ExpiringCache:
    def __init__(self, max_age_seconds=60):
        self.max_age = max_age_seconds
        self._cache = OrderedDict()  # {key: (value, timestamp)}

    def __getitem__(self, key):
        if key not in self._cache:
            raise KeyError(key)

        value, timestamp = self._cache[key]
        if time.time() - timestamp > self.max_age:
            del self._cache[key]
            raise KeyError(f"Key '{key}' has expired")

        return value

    def __setitem__(self, key, value):
        self._cache[key] = (value, time.time())
        self._cache.move_to_end(key)  # Move to end to maintain insertion order

    def __delitem__(self, key):
        del self._cache[key]

    def __len__(self):
        self._clean_expired()  # Clean expired items before reporting length
        return len(self._cache)

    def __iter__(self):
        self._clean_expired()  # Clean expired items before iteration
        for key in self._cache:
            yield key

    def __contains__(self, key):
        if key not in self._cache:
            return False

        _, timestamp = self._cache[key]
        if time.time() - timestamp > self.max_age:
            del self._cache[key]
            return False

        return True

    def _clean_expired(self):
        """Remove all expired entries from the cache."""
        now = time.time()
        expired_keys = [
            key for key, (_, timestamp) in self._cache.items()
            if now - timestamp > self.max_age
        ]
        for key in expired_keys:
            del self._cache[key]

Let's break down how this cache works:

1. Storage: The cache uses an OrderedDict to store key-value pairs along with timestamps.

2. Expiration: Each value is stored as a tuple of (value, timestamp). When accessing a value, we check if it has expired.

3. Container methods: The class implements all necessary methods to behave like a dictionary:

   • __getitem__: Retrieves values and checks expiration

   • __setitem__: Stores values with current timestamp

   • __delitem__: Removes entries

   • __len__: Returns number of non-expired entries

   • __iter__: Iterates over non-expired keys

   • __contains__: Checks if a key exists

Here's how to use the cache:

# Create a cache with 2-second expiration
cache = ExpiringCache(max_age_seconds=2)

# Store some values
cache["name"] = "Vivek"
cache["age"] = 30

# Access values
print("name" in cache)  # Output: True
print(cache["name"])    # Output: Vivek
print(len(cache))       # Output: 2

# Wait for expiration
print("Waiting for expiration...")
time.sleep(3)

# Check expired values
print("name" in cache)  # Output: False
try:
    print(cache["name"])
except KeyError as e:
    print(f"KeyError: {e}")  # Output: KeyError: 'name'

print(len(cache))  # Output: 0

This cache implementation provides several benefits:

1. Automatic expiration of old entries

2. Dictionary-like interface for easy use

3. Memory efficiency by removing expired entries

4. Thread-safe operations (assuming single-threaded access)

5. Maintains insertion order of entries

Attribute Access

Attribute access methods let you control how your objects handle getting, setting, and deleting attributes. This is particularly useful for implementing properties, validation, and logging.

getattr and getattribute

Python provides two methods for controlling attribute access:

1. __getattr__: Called only when an attribute lookup fails (that is, when the attribute doesn't exist)

2. __getattribute__: Called for every attribute access, even for attributes that exist

The key difference is that __getattribute__ is called for all attribute access, while __getattr__ is only called when the attribute isn't found through normal means.

Here's a simple example showing the difference:

class AttributeDemo:
    def __init__(self):
        self.name = "Vivek"

    def __getattr__(self, name):
        print(f"__getattr__ called for {name}")
        return f"Default value for {name}"

    def __getattribute__(self, name):
        print(f"__getattribute__ called for {name}")
        return super().__getattribute__(name)

demo = AttributeDemo()
print(demo.name)      # Output: __getattribute__ called for name
                      #        Vivek
print(demo.age)       # Output: __getattribute__ called for age
                      #        __getattr__ called for age
                      #        Default value for age

setattr and delattr

Similarly, you can control how attributes are set and deleted:

1. __setattr__: Called when an attribute is set

2. __delattr__: Called when an attribute is deleted

These methods let you implement validation, logging, or custom behavior when attributes are modified.

Practical Example: Auto-Logging Properties

Let's create a class that automatically logs all property changes. This is useful for debugging, auditing, or tracking object state changes:

import logging

# Set up logging
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s'
)

class LoggedObject:
    def __init__(self, **kwargs):
        self._data = {}
        # Initialize attributes without triggering __setattr__
        for key, value in kwargs.items():
            self._data[key] = value

    def __getattr__(self, name):
        if name in self._data:
            logging.debug(f"Accessing attribute {name}: {self._data[name]}")
            return self._data[name]
        raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{name}'")

    def __setattr__(self, name, value):
        if name == "_data":
            # Allow setting the _data attribute directly
            super().__setattr__(name, value)
        else:
            old_value = self._data.get(name, "<undefined>")
            self._data[name] = value
            logging.info(f"Changed {name}: {old_value} -> {value}")

    def __delattr__(self, name):
        if name in self._data:
            old_value = self._data[name]
            del self._data[name]
            logging.info(f"Deleted {name} (was: {old_value})")
        else:
            raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{name}'")

Let's break down how this class works:

1. Storage: The class uses a private _data dictionary to store attribute values.

2. Attribute access:

   • __getattr__: Returns values from _data and logs debug messages

   • __setattr__: Stores values in _data and logs changes

   • __delattr__: Removes values from _data and logs deletions

3. Special handling: The _data attribute itself is handled differently to avoid infinite recursion.

Here's how to use the class:

# Create a logged object with initial values
user = LoggedObject(name="Vivek", email="hello@wewake.dev")

# Modify attributes
user.name = "Vivek"  # Logs: Changed name: Vivek -> Vivek
user.age = 30         # Logs: Changed age: <undefined> -> 30

# Access attributes
print(user.name)      # Output: Vivek

# Delete attributes
del user.email        # Logs: Deleted email (was: hello@wewake.dev)

# Try to access deleted attribute
try:
    print(user.email)
except AttributeError as e:
    print(f"AttributeError: {e}")  # Output: AttributeError: 'LoggedObject' object has no attribute 'email'

This implementation provides several benefits:

1. Automatic logging of all attribute changes

2. Debug-level logging for attribute access

3. Clear error messages for missing attributes

4. Easy tracking of object state changes

5. Useful for debugging and auditing

Context Managers

Context managers are a powerful feature in Python that helps you manage resources properly. They ensure that resources are properly acquired and released, even if an error occurs. The with statement is the most common way to use context managers.

enter and exit

To create a context manager, you need to implement two magic methods:

1. __enter__: Called when entering the with block. It should return the resource to be managed.

2. __exit__: Called when exiting the with block, even if an exception occurs. It should handle cleanup.

The __exit__ method receives three arguments:

• exc_type: The type of the exception (if any)

• exc_val: The exception instance (if any)

• exc_tb: The traceback (if any)

Practical Example: Database Connection Manager

Let's create a context manager for database connections. This example shows how to properly manage database resources and handle transactions:

import sqlite3
import logging

# Set up logging
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s'
)

class DatabaseConnection:
    def __init__(self, db_path):
        self.db_path = db_path
        self.connection = None
        self.cursor = None

    def __enter__(self):
        logging.info(f"Connecting to database: {self.db_path}")
        self.connection = sqlite3.connect(self.db_path)
        self.cursor = self.connection.cursor()
        return self.cursor

    def __exit__(self, exc_type, exc_val, exc_tb):
        if exc_type is not None:
            logging.error(f"An error occurred: {exc_val}")
            self.connection.rollback()
            logging.info("Transaction rolled back")
        else:
            self.connection.commit()
            logging.info("Transaction committed")

        if self.cursor:
            self.cursor.close()
        if self.connection:
            self.connection.close()

        logging.info("Database connection closed")

        # Return False to propagate exceptions, True to suppress them
        return False

Let's break down how this context manager works:

1. Initialization:

   • The class takes a database path

   • It initializes connection and cursor as None

2. Enter method:

   • Creates a database connection

   • Creates a cursor

   • Returns the cursor for use in the with block

3. Exit method:

   • Handles transaction management (commit/rollback)

   • Closes cursor and connection

   • Logs all operations

   • Returns False to propagate exceptions

Here's how to use the context manager:

# Create a test database in memory
try:
    # Successful transaction
    with DatabaseConnection(":memory:") as cursor:
        # Create a table
        cursor.execute("""
            CREATE TABLE users (
                id INTEGER PRIMARY KEY,
                name TEXT,
                email TEXT
            )
        """)

        # Insert data
        cursor.execute(
            "INSERT INTO users (name, email) VALUES (?, ?)",
            ("Vivek", "hello@wewake.dev")
        )

        # Query data
        cursor.execute("SELECT * FROM users")
        print(cursor.fetchall())  # Output: [(1, 'Vivek', 'hello@wewake.dev')]

    # Demonstrate transaction rollback on error
    with DatabaseConnection(":memory:") as cursor:
        cursor.execute("""
            CREATE TABLE users (
                id INTEGER PRIMARY KEY,
                name TEXT,
                email TEXT
            )
        """)
        cursor.execute(
            "INSERT INTO users (name, email) VALUES (?, ?)",
            ("Wewake", "contact@wewake.dev")
        )
        # This will cause an error - table 'nonexistent' doesn't exist
        cursor.execute("SELECT * FROM nonexistent")
except sqlite3.OperationalError as e:
    print(f"Caught exception: {e}")

This context manager provides several benefits:

1. Resources are managed automatically (ex: connections are always closed).

2. With transaction safety, changes are committed or rolled back appropriately.

3. Exceptions are caught and handled gracefully

4. All operations are logged for debugging

5. The with statement makes the code clear and concise

Callable Objects

The __call__ magic method lets you make instances of your class behave like functions. This is useful for creating objects that maintain state between calls or for implementing function-like behavior with additional features.

call

The __call__ method is called when you try to call an instance of your class as if it were a function. Here's a simple example:

class Multiplier:
    def __init__(self, factor):
        self.factor = factor

    def __call__(self, x):
        return x * self.factor

# Create instances that behave like functions
double = Multiplier(2)
triple = Multiplier(3)

print(double(5))  # Output: 10
print(triple(5))  # Output: 15

This example shows how __call__ lets you create objects that maintain state (the factor) while being callable like functions.

Practical Example: Memoization Decorator

Let's implement a memoization decorator using __call__. This decorator will cache function results to avoid redundant computations:

import time
import functools

class Memoize:
    def __init__(self, func):
        self.func = func
        self.cache = {}
        # Preserve function metadata (name, docstring, etc.)
        functools.update_wrapper(self, func)

    def __call__(self, *args, **kwargs):
        # Create a key from the arguments
        # For simplicity, we assume all arguments are hashable
        key = str(args) + str(sorted(kwargs.items()))

        if key not in self.cache:
            self.cache[key] = self.func(*args, **kwargs)

        return self.cache[key]

# Usage
@Memoize
def fibonacci(n):
    """Calculate the nth Fibonacci number recursively."""
    if n <= 1:
        return n
    return fibonacci(n-1) + fibonacci(n-2)

# Measure execution time
def time_execution(func, *args, **kwargs):
    start = time.time()
    result = func(*args, **kwargs)
    end = time.time()
    print(f"{func.__name__}({args}, {kwargs}) took {end - start:.6f} seconds")
    return result

# Without memoization, this would be extremely slow
print("Calculating fibonacci(35)...")
result = time_execution(fibonacci, 35)
print(f"Result: {result}")

# Second call is instant due to memoization
print("\nCalculating fibonacci(35) again...")
result = time_execution(fibonacci, 35)
print(f"Result: {result}")

Let's break down how this memoization decorator works:

1. Initialization:

   • Takes a function as an argument

   • Creates a cache dictionary to store results

   • Preserves the function's metadata using functools.update_wrapper

2. Call method:

   • Creates a unique key from the function arguments

   • Checks if the result is in the cache

   • If not, computes the result and stores it

   • Returns the cached result

3. Usage:

   • Applied as a decorator to any function

   • Automatically caches results for repeated calls

   • Preserves function metadata and behavior

The benefits of this implementation include:

1. Better performance, as it avoids redundant computations

2. Better, transparency, as it works without modifying the original function

3. It’s flexible, and can be used with any function

4. It’s memory efficient and caches results for reuse

5. It maintains function documentation

Advanced Magic Methods

Now let's explore some of Python's more advanced magic methods. These methods give you fine-grained control over object creation, memory usage, and dictionary behavior.

new for Object Creation

The __new__ method is called before __init__ and is responsible for creating and returning a new instance of the class. This is useful for implementing patterns like singletons or immutable objects.

Here's an example of a singleton pattern using __new__:

class Singleton:
    _instance = None

    def __new__(cls, *args, **kwargs):
        if cls._instance is None:
            cls._instance = super().__new__(cls)
        return cls._instance

    def __init__(self, name=None):
        # This will be called every time Singleton() is called
        if name is not None:
            self.name = name

# Usage
s1 = Singleton("Vivek")
s2 = Singleton("Wewake")
print(s1 is s2)  # Output: True
print(s1.name)   # Output: Wewake (the second initialization overwrote the first)

Let's break down how this singleton works:

1. Class variable: _instance stores the single instance of the class

2. new method:

   • Checks if an instance exists

   • Creates one if it doesn't

   • Returns the existing instance if it does

3. init method:

   • Called every time the constructor is used

   • Updates the instance's attributes

slots for Memory Optimization

The __slots__ class variable restricts which attributes an instance can have, saving memory. This is particularly useful when you have many instances of a class with a fixed set of attributes.

Here's a comparison of regular and slotted classes:

import sys

class RegularPerson:
    def __init__(self, name, age, email):
        self.name = name
        self.age = age
        self.email = email

class SlottedPerson:
    __slots__ = ['name', 'age', 'email']

    def __init__(self, name, age, email):
        self.name = name
        self.age = age
        self.email = email

# Compare memory usage
regular_people = [RegularPerson("Vivek" + str(i), 30, "hello@wewake.dev") for i in range(1000)]
slotted_people = [SlottedPerson("Vivek" + str(i), 30, "hello@wewake.dev") for i in range(1000)]

print(f"Regular person size: {sys.getsizeof(regular_people[0])} bytes")  # Output: Regular person size: 48 bytes
print(f"Slotted person size: {sys.getsizeof(slotted_people[0])} bytes")  # Output: Slotted person size: 56 bytes
print(f"Memory saved per instance: {sys.getsizeof(regular_people[0]) - sys.getsizeof(slotted_people[0])} bytes")  # Output: Memory saved per instance: -8 bytes
print(f"Total memory saved for 1000 instances: {(sys.getsizeof(regular_people[0]) - sys.getsizeof(slotted_people[0])) * 1000 / 1024:.2f} KB")  # Output: Total memory saved for 1000 instances: -7.81 KB

Running this code produces an interesting result:

Regular person size: 48 bytes
Slotted person size: 56 bytes
Memory saved per instance: -8 bytes
Total memory saved for 1000 instances: -7.81 KB

Surprisingly, in this simple example, the slotted instance is actually 8 bytes larger than the regular instance! This seems to contradict the common advice about __slots__ saving memory.

So what's going on here? The real memory savings from __slots__ come from:

1. Eliminating dictionaries: Regular Python objects store their attributes in a dictionary (__dict__), which has overhead. The sys.getsizeof() function doesn't account for this dictionary's size.

2. Storing attributes: For small objects with few attributes, the overhead of the slot descriptors can outweigh the dictionary savings.

3. Scalability: The real benefit appears when:

   • You have many instances (thousands or millions)

   • Your objects have many attributes

   • You're adding attributes dynamically

Let's see a more complete comparison:

# A more accurate memory measurement
import sys

def get_size(obj):
    """Get a better estimate of the object's size in bytes."""
    size = sys.getsizeof(obj)
    if hasattr(obj, '__dict__'):
        size += sys.getsizeof(obj.__dict__)
        # Add the size of the dict contents
        size += sum(sys.getsizeof(v) for v in obj.__dict__.values())
    return size

class RegularPerson:
    def __init__(self, name, age, email):
        self.name = name
        self.age = age
        self.email = email

class SlottedPerson:
    __slots__ = ['name', 'age', 'email']

    def __init__(self, name, age, email):
        self.name = name
        self.age = age
        self.email = email

regular = RegularPerson("Vivek", 30, "hello@wewake.dev")
slotted = SlottedPerson("Vivek", 30, "hello@wewake.dev")

print(f"Complete Regular person size: {get_size(regular)} bytes")  # Output: Complete Regular person size: 610 bytes
print(f"Complete Slotted person size: {get_size(slotted)} bytes")  # Output: Complete Slotted person size: 56 bytes

With this more accurate measurement, you'll see that slotted objects typically use less total memory, especially as you add more attributes.

Key points about __slots__:

1. Real memory benefits: The primary memory savings come from eliminating the instance __dict__

2. Dynamic restrictions: You can't add arbitrary attributes to slotted objects

3. Inheritance considerations: Using __slots__ with inheritance requires careful planning

4. Use cases: Best for classes with many instances and fixed attributes

5. Performance bonus: Can also provide faster attribute access in some cases

missing for Default Dictionary Values

The __missing__ method is called by dictionary subclasses when a key is not found. This is useful for implementing dictionaries with default values or automatic key creation.

Here's an example of a dictionary that automatically creates empty lists for missing keys:

class AutoKeyDict(dict):
    def __missing__(self, key):
        self[key] = []
        return self[key]

# Usage
groups = AutoKeyDict()
groups["team1"].append("Vivek")
groups["team1"].append("Wewake")
groups["team2"].append("Vibha")

print(groups)  # Output: {'team1': ['Vivek', 'Wewake'], 'team2': ['Vibha']}

This implementation provides several benefits:

1. No need to check if a key exists, which is more convenient.

2. Automatic initialization creates default values as needed.

3. Reduces boilerplate for dictionary initialization.

4. It’s more flexible, and can implement any default value logic.

5. Only creates values when needed, making it more memory efficient.

Performance Considerations

While magic methods are powerful, they can impact performance if you don’t use them carefully. Let's explore some common performance considerations and how to measure them.

Impact of Magic Methods on Performance

Different magic methods have different performance implications:

Attribute Access methods:

• __getattr__, __getattribute__, __setattr__, and __delattr__ are called frequently

• Complex operations in these methods can significantly slow down your code

Container methods:

• __getitem__, __setitem__, and __len__ are called often in loops

• Inefficient implementations can make your container much slower than built-in types

Operator overloading:

• Arithmetic and comparison operators are used frequently

• Complex implementations can make simple operations unexpectedly slow

Let's measure the performance impact of __getattr__ vs. direct attribute access:

import time

class DirectAccess:
    def __init__(self):
        self.value = 42

class GetAttrAccess:
    def __init__(self):
        self._value = 42

    def __getattr__(self, name):
        if name == "value":
            return self._value
        raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{name}'")

# Measure performance
direct = DirectAccess()
getattr_obj = GetAttrAccess()

def benchmark(obj, iterations=1000000):
    start = time.time()
    for _ in range(iterations):
        x = obj.value
    end = time.time()
    return end - start

direct_time = benchmark(direct)
getattr_time = benchmark(getattr_obj)

print(f"Direct access: {direct_time:.6f} seconds")
print(f"__getattr__ access: {getattr_time:.6f} seconds")
print(f"__getattr__ is {getattr_time / direct_time:.2f}x slower")

Running this benchmark shows significant performance differences:

Direct access: 0.027714 seconds
__getattr__ access: 0.060646 seconds
__getattr__ is 2.19x slower

As you can see, using __getattr__ is more than twice as slow as direct attribute access. This might not matter for occasionally accessed attributes, but it can become significant in performance-critical code that accesses attributes in tight loops.

Optimization Strategies

Fortunately, there are various ways you can optimize magic methods.

1. Use slots for memory efficiency: This reduces memory usage and improves attribute access speed. It’s best for classes with many instances.

2. Cache computed values: You can store results of expensive operations and update the cache only when necessary. Use @property for computed attributes.

3. Minimize method calls: Make sure you avoid unnecessary magic method calls and use direct attribute access when possible. Consider using __slots__ for frequently accessed attributes.

Best Practices

When using magic methods, follow these best practices to ensure your code is maintainable, efficient, and reliable.

1. Be Consistent

When implementing related magic methods, maintain consistency in behavior:

from functools import total_ordering

@total_ordering
class ConsistentNumber:
    def __init__(self, value):
        self.value = value

    def __eq__(self, other):
        if not isinstance(other, ConsistentNumber):
            return NotImplemented
        return self.value == other.value

    def __lt__(self, other):
        if not isinstance(other, ConsistentNumber):
            return NotImplemented
        return self.value < other.value

2. Return NotImplemented

When an operation doesn't make sense, return NotImplemented to let Python try the reverse operation:

class Money:
    def __add__(self, other):
        if not isinstance(other, Money):
            return NotImplemented
        # ... rest of the implementation

3. Keep It Simple

Magic methods should be simple and predictable. Avoid complex logic that could lead to unexpected behavior:

# Good: Simple and predictable
class SimpleContainer:
    def __init__(self):
        self.items = []

    def __getitem__(self, index):
        return self.items[index]

# Bad: Complex and potentially confusing
class ComplexContainer:
    def __init__(self):
        self.items = []
        self.access_count = 0

    def __getitem__(self, index):
        self.access_count += 1
        if self.access_count > 100:
            raise RuntimeError("Too many accesses")
        return self.items[index]

4. Document Behavior

Clearly document how your magic methods behave, especially if they deviate from standard expectations:

class CustomDict(dict):
    def __missing__(self, key):
        """
        Called when a key is not found in the dictionary.
        Creates a new list for the key and returns it.
        This allows for automatic list creation when accessing
        non-existent keys.
        """
        self[key] = []
        return self[key]

5. Consider Performance

Be aware of the performance implications, especially for frequently called methods:

class OptimizedContainer:
    __slots__ = ['items']  # Use __slots__ for better performance

    def __init__(self):
        self.items = []

    def __getitem__(self, index):
        return self.items[index]  # Direct access is faster

6. Handle Edge Cases

Always consider edge cases and handle them appropriately:

class SafeContainer:
    def __getitem__(self, key):
        if not isinstance(key, (int, slice)):
            raise TypeError("Index must be integer or slice")
        if key < 0:
            raise ValueError("Index cannot be negative")
        # ... rest of the implementation

Wrapping Up

Python's magic methods provide a powerful way to make your classes behave like built-in types, enabling more intuitive and expressive code. Throughout this guide, we've explored how these methods work and how to use them effectively.

Key Takeaways

1. Object representation:

   • Use __str__ for user-friendly output

   • Use __repr__ for debugging and development

2. Operator overloading:

   • Implement arithmetic and comparison operators

   • Return NotImplemented for unsupported operations

   • Use @total_ordering for consistent comparisons

3. Container behavior:

   • Implement sequence and mapping protocols

   • Consider performance for frequently used operations

   • Handle edge cases appropriately

4. Resource management:

   • Use context managers for proper resource handling

   • Implement __enter__ and __exit__ for cleanup

   • Handle exceptions in __exit__

5. Performance optimization:

   • Use __slots__ for memory efficiency

   • Cache computed values when appropriate

   • Minimize method calls in frequently used code

When to Use Magic Methods

Magic methods are most useful when you need to:

1. Create custom data structures

2. Implement domain-specific types

3. Manage resources properly

4. Add special behavior to your classes

5. Make your code more Pythonic

When to Avoid Magic Methods

Avoid magic methods when:

1. Simple attribute access is sufficient

2. The behavior would be confusing or unexpected

3. Performance is critical and magic methods would add overhead

4. The implementation would be overly complex

Remember that with great power comes great responsibility. Use magic methods judiciously, keeping in mind their performance implications and the principle of least surprise. When used appropriately, magic methods can significantly enhance the readability and expressiveness of your code.

References and Further Reading

Official Python Documentation

1. Python Data Model - Official Documentation - Comprehensive guide to Python's data model and magic methods.

2. functools.total_ordering - Documentation for the total_ordering decorator that automatically fills in missing comparison methods.

3. Python Special Method Names - Official reference for special method identifiers in Python.

4. Collections Abstract Base Classes - Learn about abstract base classes for containers which define the interfaces that your container classes can implement.

Community Resources

5. A Guide to Python's Magic Methods - Rafe Kettler - Practical examples of magic methods and common use cases.

Further Reading

If you enjoyed this article, you might find these Python-related articles on my personal blog useful:

1. Practical Experiments for Django ORM Query Optimizations - Learn how to optimize your Django ORM queries with practical examples and experiments.

2. The High Cost of Synchronous uWSGI - Understand the performance implications of synchronous processing in uWSGI and how it affects your Python web applications.