In this tutorial, I'll walk you through exactly how I built it, from the backend serverless function that talks to Google's Gemini API, to the React chat widget that displays the AI's response as it's being generated.

By the end of this tutorial, you'll understand:

1. How to set up a Vercel Serverless function that calls Gemini and streams its response back to the browser in small text chunks.

2. Why that approach makes a chatbot feel faster and more responsive than waiting for the entire reply to come back at once.

3. How to build a React component that reads that streaming response and updates the chat window in real time as new text arrives.

We'll also test the endpoint with curl before wiring up the UI, and we'll cover how to actually deploy the whole thing to Vercel.

All of this revolves around a technique called plain-text chunk streaming. Rather than waiting for the AI to generate its entire response and then sending it all back to the browser at once, the serverless function streams the response as a series of small text chunks, writing each one to the browser as soon as it's ready.

This is what gives the chatbot that responsive, "typing" feel you've probably seen in modern AI chat interfaces, instead of a long pause followed by the full answer appearing all at once.

What We'll Cover:

• 🧩 Architecture Overview

• ✅ Prerequisites

• 📜 API Contract

• ⚙️ Backend (the Vercel Serverless Function)

• 🧪 Testing the Endpoint Before Wiring Up the UI

• 🎨 Frontend: Reading the Response Stream

• 🚀 Deploying

• ⚠️ A Note on CORS for Embedded Widgets

• 🎉 Conclusion

🧩 Architecture Overview

The application is built from two main pieces. The UI component is a React chat widget, where the user types a message and the AI's response gets rendered back to them as it streams in. The backend is a Vercel serverless function (for example, api/chat) that validates the incoming request, calls Gemini, and streams the output back to the browser.

You can read more about Vercel's serverless functions in their official documentation.

Here's how the whole flow works, from start to finish: the browser sends a request to the Vercel function, the Vercel function calls the Gemini API, and Gemini's response is streamed back to the browser chunk by chunk.

In practice, this means the backend sends the response to the frontend in small pieces as they become available, rather than making the user wait for the entire response to be generated before they see anything at all.

✅ Prerequisites

Before you get started, make sure you have the following in place:

• Node.js installed locally (Node 18 or later)

• A Gemini API key 🔑 from Google AI Studio

• A Vercel account and the Vercel CLI

• The @google/genai package installed in your function's project, which you can add by running npm install @google/genai

You'll also want to store your key in an environment variable called process.env.GOOGLE_API_KEY for local development, and add that same variable in your Vercel project's dashboard (under Settings and then Environment Variables) before deploying, so the API key gets picked up correctly once the application is hosted.

📜 API Contract

Before diving into the code, it's worth taking a moment to understand the "contract" between the frontend and backend – in other words, what the frontend is expected to send, and what the backend expects to receive in return.

When a user sends a message in the chat widget, the frontend makes a POST request to /api/chat with a JSON body containing two things: the message the user just typed, and the résumé data that's already loaded into the app (since this particular chatbot is acting as a résumé coach). The backend then uses both pieces of information together to generate a response that's actually relevant to that user's résumé, rather than a generic reply.

Request:

POST /api/chat
{
  "message": "How can I improve my resume summary?",
  "resume": { "name": "...", "experience": [...], "skills": [...] }
}

⚙️ Backend (the Vercel Serverless Function)

This is the heart of the application: a single Vercel serverless function that receives the chat request, validates it, passes it along to Gemini, and streams the AI's response back to the browser as it's generated.

Below is the full handler, and I'll walk through what each part of it is doing once you've had a look at the whole thing.

const { GoogleGenAI } = require("@google/genai");
const ai = new GoogleGenAI({
    apiKey: process.env.GOOGLE_API_KEY,  // GOOGLE_API_KEY can be configured in Vercel
});
const MAX_TEXT_LENGTH = 2000;
const MAX_ARRAY_LENGTH = 50;

function sanitizeString(input = "") {
    // sanitize your input string here
}

function sanitizeObject(obj = {}) {
    // sanitize your resume object
}

const allowCors = fn => async (req, res) => {

    res.setHeader('Access-Control-Allow-Origin', '<<your web app url goes here>>');
    res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS');
    res.setHeader('Access-Control-Allow-Headers', 'Content-Type, Authorization');

    // ✅ Handle preflight request
    if (req.method === 'OPTIONS') {
        return res.status(200).end();
    }
    // another option
    res.setHeader('Access-Control-Allow-Methods', 'GET, HEAD, OPTIONS, POST, PUT, DELETE')
    res.setHeader(
        'Access-Control-Allow-Headers',
        'X-CSRF-Token, X-Requested-With, Accept, Accept-Version, Content-Length, Content-MD5, Content-Type, Date, X-Api-Version'
    )
    if (req.method === 'OPTIONS') {
        res.status(200).end()
        return
    }
    return await fn(req, res)
};

const handler = async (req, res) => {

    if (req.url === '/api/chat' && req.method === 'POST') {

        const { message, resume } = req.body || {};

        const safeMessage = sanitizeString(message);
        const safeResume = sanitizeObject(resume);

        if (!safeMessage) {
            return res.status(400).json({ error: "Invalid message" });
        }

        if (!safeMessage || !safeResume) {
            return res.status(400).json({ error: 'Missing message or resume data' });
        }

        try {
            const stream = await ai.models.generateContentStream({
                model: "<<GEMINI MODEL GOES HERE>>",
                contents: `
            You are a professional Resume Coach AI.

            - Always respond clearly, politely, and professionally.
            - <<Add descriptive instruction for the prompt based on your requirement>>
            - Resume data:
${JSON.stringify(safeResume, null, 2)}

User question:
${safeMessage}
`,
            });
            res.setHeader("Content-Type", "text/plain; charset=utf-8");
            res.setHeader("Cache-Control", "no-cache");

            for await (const chunk of stream) {
                const text = chunk.text;
                if (text) {
                    res.write(text);
                }
            }

            res.end();
        } catch (error) {
            console.error('Gemini Error:', error);
            res.status(500).json({ error: 'AI request failed' });
        }
    }

    // I have added a health check for testing the handler
    if (req.url === '/api/chat?type=healthcheck' && req.method === 'GET') {
        res.status(200).json({ message: 'Hello from the chat endpoint!' });
    }
}

module.exports = allowCors(handler)

Now let's break that down piece by piece:

• Setting up the Gemini client: At the top of the file, we create a GoogleGenAI client using the API key stored in the GOOGLE_API_KEY environment variable. This client is what we'll use to actually talk to Gemini throughout the rest of the function.

• Sanitizing the input: Before we do anything with the incoming request, we run the message and résumé through sanitizeString and sanitizeObject. These functions strip out anything unexpected or oversized, so we're never passing untrusted user input straight through to the AI without a check first.

• Handling CORS: The allowCors wrapper function sits around our handler and takes care of Cross-Origin Resource Sharing. Since the chat widget might be embedded on a different domain than the Vercel function itself, we need to explicitly allow requests from that domain and handle the OPTIONS preflight request that browsers automatically send before the real POST request goes through.

• Validating the request: Inside the handler itself, we check that both message and resume made it through sanitization successfully. If either one is missing or invalid, we return a 400 error right away instead of wasting a call to Gemini on a request we already know is bad.

• Calling Gemini and streaming the response: This is the key part of the whole tutorial. Instead of calling a regular "generate content" method and waiting for the full response to come back, we call generateContentStream, which returns an async iterable. We loop over that stream with a for await...of loop, and every time a new chunk of text comes in, we immediately write it to the response with res.write(text). This is exactly what lets the browser start receiving text before Gemini has even finished generating the full answer.

• Error handling and the health check: If anything goes wrong while talking to Gemini, we catch the error, log it, and return a 500 response so the frontend knows something failed. There's also a simple GET health check endpoint you can hit to confirm the function is up and running before you start testing the actual chat flow.

🧪 Testing the Endpoint Before Wiring Up the UI

Before you start building out the frontend, it's a good idea to confirm that the serverless function is actually streaming chunks the way you expect. You can do that with a simple curl request.

curl -N -X POST "https://YOUR_APP.vercel.app/api/chat?type=chat" \
  -H "Content-Type: application/json" \
  -d '{"message":"Give me 3 resume summary tips","resume":{"name":"Test"}}'

The -N flag disables curl's output buffering, so you'll see the text appear incrementally in your terminal rather than all at once. That's your confirmation that the streaming is working end to end, before you've written a single line of frontend code.

🎨 Frontend: Reading the Response Stream

Here's a screenshot of the chat widget UI I built in React:

We won't walk through building that UI from scratch here. The layout, styling, and message list are really up to you and your own design preferences.

What we will cover is the function that actually powers it: the code behind the send button, which takes what the user typed, sends it off to our serverless function, and reads back the streaming response so the widget can display the AI's reply as it arrives, piece by piece, the way you see it happening in the screenshot above.

Before jumping into that function, here's the shell it lives inside, so you can see where it fits in the component:

function ChatWidget({ resume }) {
  const [messages, setMessages] = useState([]);
  const [input, setInput] = useState("");
  const [loading, setLoading] = useState(false);
  const [error, setError] = useState(null);

  async function sendMessage() {
    // ...actual function is added below
  }

  return (
    <div className="chat-widget">
      {/* message list, input box, and a send button that calls sendMessage() */}
    </div>
  );
}

That's the shell the function lives in: state for the message list, the input box, a loading flag, and an error slot. sendMessage is what fires when the send button is clicked.

Here's the full function, sendMessage, which runs whenever the user types a prompt and clicks send. This is the piece of code that connects the UI shown above to the backend we just built.

async function sendMessage() {
    const messageText = input.trim();
    if (!messageText || loading) return;

    const userMessage = { role: "user", text: messageText };
    setMessages((message) => [...message, userMessage]);
    setInput("");
    setLoading(true);
    setError(null);

    try {
      const res = await fetch("<<YOUR VERCEL SERVERLESS FUNCTION URL GOES HERE>>/api/chat", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ message: messageText, resume })
      });

      if (!res.ok) {
        throw new Error(`Failed to get response: ${res.status}`);
      }

      const reader = res.body.getReader();
      const decoder = new TextDecoder();

      let fullText = "";
      setMessages((message) => [...message, { role: "assistant", text: "" }]);

      while (true) {
        const { value, done } = await reader.read();
        if (done) break;

        const chunk = decoder.decode(value);
        fullText += chunk;

        setMessages((message) => {
          const updated = [...message];
          updated[updated.length - 1] = { role: "assistant", text: fullText };
          return updated;
        });
      }

    } catch (err) {
      setError("Failed to send message. Please try again.");
      setMessages((m) => [...m, {
        role: "assistant",
        text: "Sorry, I encountered an error. Please try again later."
      }]);
    } finally {
      setLoading(false);
    }
  }

Here's what's happening step by step:

First, when the user clicks send, we grab their typed message and immediately add it to the messages array so it shows up in the chat right away.

Then we send a POST request to our /api/chat endpoint with the message and the résumé data. Once we get a response back, we call res.body.getReader() to get a ReadableStreamDefaultReader on the response body, and we use a TextDecoder to convert each raw chunk of bytes into readable text.

From there, we loop, reading one chunk at a time and appending it to a growing fullText string, and on every iteration we update the last message in the chat with that text.

That loop is exactly what creates the "typing" effect you see in the screenshot above: the assistant's message in the chat widget visibly grows word by word as new chunks arrive, instead of popping in all at once. This pattern is a bit different from the typical fetch().then(res => res.json()) call you're probably used to, so it's worth pausing on if you haven't streamed a fetch response before.

🚀 Deploying

Once you've confirmed that the function and UI both work locally, deploying comes down to three steps:

1. Set your environment variable in Vercel. In your project's dashboard, go to Settings and then Environment Variables and add GOOGLE_API_KEY with the same value you used locally.

2. Update your CORS origin and fetch URL. In the backend, point allowCors's Access-Control-Allow-Origin header at your real deployed domain. In the frontend, point the fetch call at that same deployed function URL instead of localhost.

3. Deploy, using whichever of the two options below fits your workflow.

Option A: Deploy from the CLI

npm install -g vercel   # if you haven't already
vercel login
vercel --prod

vercel login authenticates your machine, and vercel --prod builds and ships straight to production from your project directory. This is a good fit for one-off deploys, or when you want full control over exactly when a deploy happens.

Option B: Deploy via GitHub integration

1. Push your project to a GitHub repository (if it isn't already).

2. In the Vercel dashboard, click Add New and then Project, then select your repository.

3. Vercel auto-detects your framework settings. Confirm them and click Deploy.

4. From then on, every push to your main branch triggers an automatic redeploy.

I've always preferred this option and it has been more convenient if you're actively iterating on the project, since you never have to remember to run a deploy command yourself.

Once the deploy finishes, Vercel will give you a live URL for your app. Test it with the same curl command from earlier, swapping in your production URL, to confirm that streaming is working correctly in production before you consider yourself done.

⚠️ A Note on CORS for Embedded Widgets

If you're planning to embed this chat widget on a website that lives on a different domain than your Vercel app (for example, embedding it as a widget on a marketing site while the function itself is deployed elsewhere), you'll run into CORS restrictions by default. Browsers block requests to a different origin unless the server explicitly allows them.

To handle this correctly, your serverless function needs to respond to the OPTIONS preflight request that the browser automatically sends before the real POST request, and it needs to set the Access-Control-Allow-Origin header to match the domain your widget is actually running on. That's exactly what the allowCors wrapper in the backend code above is doing for you.

🎉 Conclusion

Plain-text chunk streaming is what makes a chatbot feel responsive on Vercel. Because the user sees the response appear gradually, they aren't stuck waiting for the entire answer to be generated before anything shows up on screen.

If you'd like to keep building on this, here are a few natural next steps to consider: adding rate limiting to prevent abuse of the endpoint, adding conversation persistence with something like KV, Redis, or Postgres so chat history isn't lost on refresh, and adding a stop button to the widget so users can cancel a response that's already in the middle of generating.

If you build something with this approach, I'd love to hear about it! What are you planning to build with Gemini and Vercel?

Hope you have a blessed week! 😇

An AI assistant can help with that workflow, but only if it can query a real catalog and persist what you already decided. Chat history isn't a database. A hallucinated deadline is worse than no deadline at all.

The Model Context Protocol (MCP) is the standard way to give AI apps that kind of access. In this tutorial, you'll build a scholarship research MCP server with Node.js, Express, and MongoDB.

When you finish, Cursor, Claude Desktop, or any other MCP host will be able to search awards, match them to a student profile, bookmark a shortlist, and store research notes. The model stays the reasoning layer while your server owns the data.

You'll build:

• A MongoDB catalog of scholarships, plus saved-list and notes collections

• Nine MCP tools for search, matching, deadlines, comparison, and research tracking

• Resources so a client can read the catalog without calling a tool

• Prompts that turn a student profile into a research plan

• An Express app that serves MCP over Streamable HTTP

The sample catalog in this project is a teaching dataset. Amounts, dates, and eligibility rules are simplified. Always confirm details on the official application page before applying.

What You Need

You should be comfortable with JavaScript and basic Express routing. You don't need prior MCP experience.

Install:

• Node.js 20 or later

• MongoDB running locally, or a free MongoDB Atlas cluster

• An MCP client if you want to try the last section. Cursor and Claude Desktop both work.

Docker is enough for MongoDB:

docker run -d --name mongo -p 27017:27017 mongo:7

Table of Contents

• What You Need

• What Is the Model Context Protocol?

• Why a Scholarship Research Server?

• How the Architecture Fits Together

• How to Set Up the Project

• How to Connect MongoDB

• How to Model Scholarship Data

• How to Write the Scholarship Service

• How to Register MCP Tools

• How to Expose Resources and Prompts

• How to Serve MCP Over Express

• How to Seed the Catalog

• How to Test the Server

• How to Connect Cursor and Claude Desktop

• How a Research Session Runs

• How the Matching Logic Works

• What You Can Build Next

• Conclusion

What Is the Model Context Protocol?

MCP is an open protocol that lets an AI application talk to external tools and data sources through a shared contract. Anthropic introduced it in 2024, and it's now maintained as an open standard.

Anthropic described MCP as "a USB-C port for AI applications": one connector, many hosts. (Source: Introducing the Model Context Protocol) Instead of writing one integration for Cursor, another for Claude Desktop, and a third for a custom agent, you implement the protocol once.

The official architecture overview splits MCP into two layers:

• The data layer is JSON-RPC 2.0. Clients and servers exchange requests such as tools/list and tools/call.

• The transport layer moves those messages. Local servers usually use stdio. Remote or long-running servers use Streamable HTTP.

This tutorial uses Streamable HTTP, because Express is already an HTTP server and you want the catalog available to any client on your machine.

Hosts, Clients, and Servers

Three roles show up in every MCP setup:

• The host is the AI app. Cursor and Claude Desktop are hosts.

• The client lives inside the host. The host creates one client per connected server.

• The server is your program. It advertises tools, resources, and prompts, then handles calls.

Your scholarship app is the server. You never talk to the model SDK directly. The host does that.

Tools, Resources, and Prompts

MCP servers expose three primitives. You'll use all three.

Tools are actions. The model decides to call them, the way it might call a function in a regular tool-calling API. Search, save, and compare belong here.

Resources are data the host can read and attach as context. A catalog URI and a per-scholarship URI belong here. The model doesn't have to "take an action" to see them.

Prompts are named templates. People usually invoke them from a slash command or a menu. A "research plan" prompt belongs here, because it's a workflow you want to start on purpose.

That split matters. If you put everything in tools, the model has to guess when to look things up. Resources and prompts give the host better knobs.

Why a Scholarship Research Server?

A weather MCP demo is a single API call. Scholarship research is closer to a real product:

• The catalog must be queryable. Keyword search, GPA filters, and deadline windows all live in the database.

• The workflow must persist. A saved shortlist and research notes should survive a new chat.

• Eligibility is logic, not prose. A GPA minimum is a number. First-generation-only is a boolean. Put those checks in code so the model can't invent a match.

• The output must be inspectable. Students should be able to open the official URL and verify every claim.

MongoDB fits this well. Each scholarship is a document with nested arrays for fields of study, citizenship, and requirements. Saved items and notes are separate collections with references back to the catalog.

You could wrap a public API instead of storing documents. That's a good follow-up. Starting with your own catalog keeps the tutorial self-contained and makes the MCP contract obvious.

How the Architecture Fits Together

The finished project looks like this:

MCP host (Cursor or Claude Desktop)
        |
        |  Streamable HTTP  POST /mcp
        v
Express app  (createMcpExpressApp)
        |
        |  createMcpHandler factory
        v
McpServer  tools / resources / prompts
        |
        v
Scholarship service
        |
        v
MongoDB  scholarships, savedScholarships, researchnotes

A few design choices are worth calling out before you write code.

The MCP handler is stateless. The SDK runs your server factory once per HTTP request. That's the recommended v2 pattern for Streamable HTTP. Don't keep tool state on the McpServer instance. Keep it in MongoDB.

The database connection is process-wide. Connecting on every request would be slow and pointless. You connect once at startup and close over that pool from the tools.

The HTTP surface is small on purpose. /health is for you. /mcp is for the protocol. You don't need a REST API in front of the same data unless you want one later.

How to Set Up the Project

Create a folder and initialize a Node.js project. ESM is required because the MCP SDK is ESM-first.

mkdir scholarship-research-mcp-server
cd scholarship-research-mcp-server
npm init -y

Open package.json and set "type": "module". Then install the SDK, Express, Mongoose, Zod, and dotenv:

npm install @modelcontextprotocol/server @modelcontextprotocol/express @modelcontextprotocol/node express mongoose dotenv zod

The SDK split into packages in v2:

• @modelcontextprotocol/server is the McpServer class and createMcpHandler

• @modelcontextprotocol/express gives you createMcpExpressApp, including DNS rebinding protection

• @modelcontextprotocol/node adapts the web-standard handler to Node's req/res

Create a .env file:

MONGODB_URI=mongodb://127.0.0.1:27017/scholarship_research
PORT=3000
HOST=127.0.0.1

Add a .gitignore that excludes node_modules and .env.

Your source layout can stay small:

src/
  index.js
  config.js
  db.js
  models/
  services/
  mcp/
  utils/
data/
  scholarships.json
scripts/
  seed.js
  smoke-test.js

src/config.js reads environment variables with defaults:

export const config = {
  mongodbUri: process.env.MONGODB_URI ?? "mongodb://127.0.0.1:27017/scholarship_research",
  port: Number(process.env.PORT ?? 3000),
  host: process.env.HOST ?? "127.0.0.1",
};

Keep configuration in one file. Tools shouldn't read process.env directly.

How to Connect MongoDB

Mongoose 9 works cleanly with ESM. A short src/db.js is enough:

import mongoose from "mongoose";
import { config } from "./config.js";

export async function connectDatabase() {
  mongoose.set("strictQuery", true);
  await mongoose.connect(config.mongodbUri);
  return mongoose.connection;
}

Call this once in src/index.js before app.listen. If the connection fails, the process should exit. A running Express server with a dead database is harder to debug than a failed startup.

How to Model Scholarship Data

You need three collections.

Scholarship

This is the catalog. Store the fields a matching engine actually uses, not a blob of markdown.

import mongoose from "mongoose";

const scholarshipSchema = new mongoose.Schema(
  {
    title: { type: String, required: true, trim: true },
    provider: { type: String, required: true, trim: true },
    description: { type: String, required: true },
    amountMin: { type: Number, default: 0 },
    amountMax: { type: Number, default: 0 },
    currency: { type: String, default: "USD" },
    deadline: { type: Date, default: null },
    rolling: { type: Boolean, default: false },
    educationLevels: { type: [String], default: ["undergraduate"] },
    fieldsOfStudy: { type: [String], default: ["any"] },
    gpaMinimum: { type: Number, default: null },
    citizenship: { type: [String], default: ["any"] },
    countries: { type: [String], default: ["any"] },
    firstGenerationOnly: { type: Boolean, default: false },
    womenOnly: { type: Boolean, default: false },
    numberOfAwards: { type: Number, default: 1 },
    renewable: { type: Boolean, default: false },
    applicationUrl: { type: String, required: true },
    applicationRequirements: { type: [String], default: [] },
  },
  { timestamps: true },
);

scholarshipSchema.index({
  title: "text",
  provider: "text",
  description: "text",
  fieldsOfStudy: "text",
});
scholarshipSchema.index({ deadline: 1 });
scholarshipSchema.index({ amountMax: -1 });

export const Scholarship = mongoose.model("Scholarship", scholarshipSchema);

A few field choices are doing real work:

• fieldsOfStudy: ["any"] means the award is field-open. The matcher treats any as a wildcard.

• deadline: null plus rolling: true covers programs that accept applications year-round.

• applicationUrl is mandatory. Every tool result should point back to a human-verifiable source.

• gpaMinimum: null means the provider didn't publish a cutoff. That's different from 0.

Saved Scholarship

A research list is a join between a person and a catalog row.

const savedScholarshipSchema = new mongoose.Schema(
  {
    researcherId: { type: String, required: true, default: "default" },
    scholarship: {
      type: mongoose.Schema.Types.ObjectId,
      ref: "Scholarship",
      required: true,
    },
    status: {
      type: String,
      enum: ["saved", "applying", "submitted", "won", "rejected"],
      default: "saved",
    },
  },
  { timestamps: true },
);

savedScholarshipSchema.index({ researcherId: 1, scholarship: 1 }, { unique: true });

The unique index makes save_scholarship idempotent. Saving the same award twice updates the status instead of creating duplicates.

researcherId is a plain string. For a tutorial, that's enough. In production you would take it from an auth token.

Research Note

Notes are a separate collection so one scholarship can have many of them.

const researchNoteSchema = new mongoose.Schema(
  {
    researcherId: { type: String, required: true, default: "default" },
    scholarship: {
      type: mongoose.Schema.Types.ObjectId,
      ref: "Scholarship",
      required: true,
    },
    body: { type: String, required: true, trim: true },
  },
  { timestamps: true },
);

You now have a catalog, a shortlist, and a notebook. That's the whole product surface the MCP tools will wrap.

How to Write the Scholarship Service

Keep MongoDB queries out of the MCP layer. Tools should call a service, get plain objects back, and format text. That makes the same functions reusable from a seed script, a smoke test, or a future REST route.

Search

Search is a filter builder. Each optional argument adds a clause. Open or rolling awards stay in the result set. Closed deadlines drop out.

const OPEN_DEADLINE_FILTER = {
  $or: [{ rolling: true }, { deadline: null }, { deadline: { $gte: new Date() } }],
};

export async function searchScholarships(filters) {
  const query = { ...OPEN_DEADLINE_FILTER };
  const and = [query];

  if (filters.keyword) {
    and.push({
      $or: [
        { title: { $regex: escapeRegex(filters.keyword), $options: "i" } },
        { provider: { $regex: escapeRegex(filters.keyword), $options: "i" } },
        { description: { $regex: escapeRegex(filters.keyword), $options: "i" } },
        { fieldsOfStudy: { $regex: escapeRegex(filters.keyword), $options: "i" } },
      ],
    });
  }

  if (filters.fieldOfStudy) {
    and.push({
      $or: [
        { fieldsOfStudy: { $regex: `^any$`, $options: "i" } },
        { fieldsOfStudy: { $regex: escapeRegex(filters.fieldOfStudy), $options: "i" } },
      ],
    });
  }

  // educationLevel, citizenship, country, minAmount, gpa, flags...

  const results = await Scholarship.find({ $and: and })
    .sort({ deadline: 1, amountMax: -1 })
    .limit(100)
    .lean();

  return rankByFieldMatch(results, filters.fieldOfStudy).slice(0, filters.limit ?? 10);
}

Two details are easy to skip and worth keeping.

Escape user input before you drop it into $regex. A keyword of ( shouldn't become a broken regular expression.

When a student searches for computer science, field-open awards (any) are eligible, but the specific CS scholarships should appear first. Rank in memory after the query. MongoDB already did the eligibility filter. You're only adjusting display order.

Match

Matching is not the same as search. Search is "find documents that look like this." Matching is "here is a student, score every open award."

The service loads open scholarships, then applies hard filters and a score:

Hard filters prevent false hope. Soft scores rank the rest. Each result also returns a reasons array, so the model can explain the match instead of inventing one.

That last point is the whole reason to put matching on the server. If you only return raw documents, the model will sometimes "helpfully" include an award the student can't apply for. Returning score and reasons keeps the explanation grounded in code.

Save, Notes, Deadlines, Compare

The remaining functions are thin:

• saveScholarship upserts by (researcherId, scholarshipId)

• addResearchNote inserts a note after confirming the scholarship exists

• getUpcomingDeadlines queries deadline between now and now + N days

• compareScholarships loads two or three documents and returns the same fields for each

Validate MongoDB ids with mongoose.Types.ObjectId.isValid before you query. An LLM will occasionally pass a title where you asked for an id. Fail clearly. Don't throw a CastError into the MCP transport.

How to Register MCP Tools

Create src/mcp/server.js as a factory. The HTTP handler will call it on every request.

import { McpServer } from "@modelcontextprotocol/server";
import { registerPrompts } from "./prompts.js";
import { registerResources } from "./resources.js";
import { registerTools } from "./tools.js";

export function createScholarshipServer() {
  const server = new McpServer({
    name: "scholarship-research",
    version: "1.0.0",
  });

  registerTools(server);
  registerResources(server);
  registerPrompts(server);

  return server;
}

Keep this factory cheap. No database connections, no file reads, and no caches that belong at module scope. The HTTP serving guide is explicit about this: create connection pools once at startup, and close over them.

One Tool, Fully

registerTool takes a name, a config object, and a handler. The inputSchema is a Zod object. The SDK turns that schema into JSON Schema for tools/list, validates arguments before your handler runs, and infers types if you're on TypeScript.

import * as z from "zod/v4";

server.registerTool(
  "search_scholarships",
  {
    title: "Search scholarships",
    description:
      "Search the scholarship catalog by keyword, field of study, education level, citizenship, country, GPA, and award amount.",
    inputSchema: z.object({
      keyword: z.string().min(1).optional().describe("Free-text search across title, provider, description, and fields"),
      fieldOfStudy: z.string().optional().describe("For example computer science, public health, or engineering"),
      educationLevel: z.enum(["undergraduate", "graduate", "doctoral"]).optional(),
      citizenship: z.string().optional(),
      country: z.string().optional(),
      minAmount: z.number().nonnegative().optional(),
      gpa: z.number().min(0).max(4).optional(),
      firstGeneration: z.boolean().optional(),
      womenOnly: z.boolean().optional(),
      limit: z.number().int().min(1).max(25).optional(),
    }),
    annotations: { readOnlyHint: true, openWorldHint: false },
  },
  async (args) => {
    const results = await searchScholarships(args);
    return toolText(formatScholarshipList(results));
  },
);

Write descriptions as if the model is the only docs the tool will ever get. .describe() on a Zod field survives conversion to JSON Schema. That's how the host tells the model what fieldOfStudy means.

title is the human label. description is the model-facing contract. They're not the same string.

Annotations

Annotations don't change how the SDK runs the tool. Hosts use them to decide how cautious to be.

• readOnlyHint: true for search, get, match, list, compare, and deadlines

• readOnlyHint: false for save and add-note

• idempotentHint: true on save, because of the unique index

• openWorldHint: false because this server talks to your database, not the open web

A host can auto-approve a read-only search and ask the user before a write. That's worth five extra keys in the config.

Return Shape

Every tool returns MCP content blocks:

export function toolText(text, isError = false) {
  return {
    content: [{ type: "text", text }],
    isError,
  };
}

Return isError: true for domain failures such as "scholarship not found." Throw only for unexpected failures. The spec treats those differently. A validation error from Zod never reaches your handler. The SDK already converts it into an isError result.

Format lists for a person who is skimming a chat transcript. Include the MongoDB id on every row. Later tools need that id, and the model can't invent a valid ObjectId.

The Full Tool Set

The server registers nine tools:

That's enough for a research loop: find, inspect, match, save, annotate, and compare.

Resist the urge to add a delete_everything tool. Destructive tools need extra confirmation and aren't part of this workflow.

How to Expose Resources and Prompts

Tools aren't the only way a host gets context.

Resources

A static resource is a fixed URI. The catalog fits that:

server.registerResource(
  "scholarship-catalog",
  "scholarship://catalog",
  {
    title: "Scholarship catalog",
    description: "Open scholarships currently stored in MongoDB",
    mimeType: "application/json",
  },
  async (uri) => {
    const catalog = await listCatalog();
    return {
      contents: [
        {
          uri: uri.href,
          mimeType: "application/json",
          text: JSON.stringify(catalog, null, 2),
        },
      ],
    };
  },
);

A resource template covers a family of URIs. Use scholarship://item/{id} rather than scholarship://{id}. If the pattern is scholarship://{id}, the URI scholarship://catalog becomes ambiguous.

import { ResourceTemplate } from "@modelcontextprotocol/server";

server.registerResource(
  "scholarship-record",
  new ResourceTemplate("scholarship://item/{id}", {
    list: async () => {
      const catalog = await listCatalog(20);
      return {
        resources: catalog.map((scholarship) => ({
          uri: `scholarship://item/${scholarship._id}`,
          name: scholarship.title,
          mimeType: "text/plain",
        })),
      };
    },
  }),
  {
    title: "Scholarship record",
    description: "Full details for one scholarship",
    mimeType: "text/plain",
  },
  async (uri, { id }) => {
    const scholarship = await getScholarshipById(id);
    return {
      contents: [
        {
          uri: uri.href,
          mimeType: "text/plain",
          text: scholarship
            ? formatScholarship(scholarship)
            : `No scholarship found with id ${id}.`,
        },
      ],
    };
  },
);

list is required on a template. Pass undefined if you can't enumerate instances. Here you can, so the host can show a picker.

Prompts

Prompts are workflows you want a person to start. The research plan prompt doesn't query MongoDB itself. It tells the model to use the tools, then structure the answer.

server.registerPrompt(
  "research-plan",
  {
    title: "Scholarship research plan",
    description: "Build a week-by-week research and application plan from a student profile.",
    argsSchema: z.object({
      fieldOfStudy: z.string(),
      educationLevel: z.string(),
      citizenship: z.string(),
      gpa: z.string(),
      weeks: z.string().optional(),
    }),
  },
  ({ fieldOfStudy, educationLevel, citizenship, gpa, weeks }) => ({
    messages: [
      {
        role: "user",
        content: {
          type: "text",
          text: `Create a ${weeks || "6"}-week scholarship research plan for this student.

Field of study: ${fieldOfStudy}
Education level: ${educationLevel}
Citizenship: ${citizenship}
GPA: ${gpa}

Use the scholarship research tools to find real awards first. Then produce a shortlist, a week-by-week plan, and risks. Name actual scholarships and dates from the tool results.`,
        },
      },
    ],
  }),
);

Notice the instruction "use the scholarship research tools." A prompt isn't a substitute for tools. It's a script that makes tool use more likely and the output shape more consistent.

A second prompt, application-checklist, takes a scholarship id and asks for a document list and a backward calendar. That's the kind of repetitive work MCP prompts are good at.

Prompt arguments are strings in many hosts, even when the value is a number. Typing gpa as a string avoids a frustrating expected number, received string error from a slash-command form.

How to Serve MCP Over Express

This is the part that used to be a page of session-handling code. In SDK v2 it's a factory plus one route.

import "dotenv/config";
import { createMcpExpressApp } from "@modelcontextprotocol/express";
import { toNodeHandler } from "@modelcontextprotocol/node";
import { createMcpHandler } from "@modelcontextprotocol/server";
import { config } from "./config.js";
import { connectDatabase } from "./db.js";
import { createScholarshipServer } from "./mcp/server.js";

const mcpHandler = createMcpHandler(() => createScholarshipServer());
const nodeHandler = toNodeHandler(mcpHandler);

const app = createMcpExpressApp({
  host: config.host,
  allowedHosts: ["127.0.0.1", "localhost"],
});

app.get("/health", (_req, res) => {
  res.json({
    status: "ok",
    service: "scholarship-research-mcp",
    transport: "streamable-http",
  });
});

app.all("/mcp", (req, res) => {
  void nodeHandler(req, res, req.body);
});

async function start() {
  await connectDatabase();
  app.listen(config.port, config.host, () => {
    console.log(`Scholarship research MCP server listening on http://${config.host}:${config.port}/mcp`);
  });
}

start();

Walk through what each helper is doing.

createMcpHandler takes a function that returns a fresh McpServer. It exposes a web-standard fetch. That's the same handler you would export from a Cloudflare Worker.

toNodeHandler adapts that fetch to Express (req, res). You pass req.body as the third argument because createMcpExpressApp already ran express.json(). If you omit the body, the adapter tries to read a stream Express already consumed.

createMcpExpressApp is express() with two extras: JSON parsing, and Host/Origin checks. Those checks exist because of DNS rebinding. A malicious page can point its own domain at 127.0.0.1 and, without a Host check, your browser would treat the local MCP server as same-origin. The default bind is 127.0.0.1 for that reason. The Express serving guide covers this in more detail.

app.all("/mcp", ...) is intentional. Streamable HTTP uses POST for JSON-RPC, and GET for SSE streams. Registering only POST will break some clients.

Shut the handler down on SIGINT:

process.on("SIGINT", async () => {
  await mcpHandler.close();
  process.exit(0);
});

close() waits for in-flight requests. Then you can exit.

Add npm scripts:

{
  "scripts": {
    "start": "node src/index.js",
    "dev": "node --watch src/index.js",
    "seed": "node scripts/seed.js",
    "smoke": "node scripts/smoke-test.js"
  }
}

node --watch is enough for local development. You don't need nodemon for this project.

How to Seed the Catalog

MCP tools against an empty database will work and return "no scholarships matched." That's correct, and also a bad first impression.

Put 20 to 30 realistic records in data/scholarships.json. Mix:

• Undergraduate and graduate awards

• STEM and field-open awards

• Country-specific programs and global ones

• Rolling deadlines and hard dates

• First-generation and women-only flags

A seed script should replace the catalog, not append:

import "dotenv/config";
import { readFile } from "node:fs/promises";
import path from "node:path";
import { fileURLToPath } from "node:url";
import { connectDatabase } from "../src/db.js";
import { Scholarship } from "../src/models/index.js";

const __dirname = path.dirname(fileURLToPath(import.meta.url));

async function seed() {
  await connectDatabase();
  const raw = await readFile(path.join(__dirname, "..", "data", "scholarships.json"), "utf8");
  await Scholarship.deleteMany({});
  const inserted = await Scholarship.insertMany(JSON.parse(raw));
  console.log(`Seeded ${inserted.length} scholarships.`);
  process.exit(0);
}

seed();

Run it:

npm run seed

Treat the JSON as sample data. Names of well-known programs help the tutorial feel real. They also create a duty to say, clearly, that students must verify every number and date on the official site. The applicationUrl field exists so that reminder has somewhere to point.

If you later replace the JSON with a live source, keep the same schema. The MCP tools shouldn't care where the documents came from.

How to Test the Server

Start MongoDB, seed, then start the process:

npm run seed
npm start

You should see:

Scholarship research MCP server listening on http://127.0.0.1:3000/mcp

Health Check

curl -s http://127.0.0.1:3000/health

A JSON status: ok means Express is up. It doesn't mean MCP is wired correctly. For that, send a JSON-RPC request.

List Tools

curl -s -X POST http://127.0.0.1:3000/mcp \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

The response is an SSE event whose data: line contains the JSON-RPC result. You should see all nine tools, each with a JSON Schema derived from Zod.

The Accept header matters. MCP Streamable HTTP can return JSON or an event stream. Asking for both is the compatible choice.

Call a Tool

curl -s -X POST http://127.0.0.1:3000/mcp \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{
    "jsonrpc":"2.0",
    "id":2,
    "method":"tools/call",
    "params": {
      "name": "search_scholarships",
      "arguments": {
        "fieldOfStudy": "computer science",
        "limit": 3
      }
    }
  }'

You should get a numbered list with ids, deadlines, and GPA minimums. Copy one id and pass it to get_scholarship.

A small Node smoke test is nicer than raw curl once you're calling several methods. Parse the data: line, then print result.content[0].text. The repo includes scripts/smoke-test.js for that.

Inspector

The MCP Inspector is the official GUI for servers. Point it at http://127.0.0.1:3000/mcp and you can list tools, fill in arguments, and read resources without a host app in the way.

Use Inspector when a host "can't see" your server. If Inspector works and the host doesn't, the bug is in the host config. If Inspector fails, the bug is in your process.

How to Connect Cursor and Claude Desktop

Keep npm start running. MCP over HTTP is a live server, not a one-shot CLI.

Cursor

Add a server entry in Cursor's MCP settings. A Streamable HTTP server looks like this:

{
  "mcpServers": {
    "scholarship-research": {
      "url": "http://127.0.0.1:3000/mcp"
    }
  }
}

Restart the MCP session if Cursor had a previous failed connection cached. Then ask:

I am a first-generation undergraduate studying computer science in the United States, GPA 3.6. Use the scholarship research tools to build a shortlist and a six-week plan.

You should see the host call match_scholarships or search_scholarships, then get_scholarship for the interesting rows, then maybe save_scholarship. If it never calls a tool, the server isn't actually connected. Check the MCP logs in Cursor before you change code.

You can also invoke the research-plan prompt from the host's prompt menu if it surfaces prompts.

Claude Desktop

Claude Desktop's config file lives at:

• macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

• Windows: %APPDATA%\Claude\claude_desktop_config.json

Claude Desktop prefers stdio. Bridge your HTTP server with mcp-remote:

{
  "mcpServers": {
    "scholarship-research": {
      "command": "npx",
      "args": ["-y", "mcp-remote", "http://127.0.0.1:3000/mcp"]
    }
  }
}

Restart Claude Desktop after you save the file. The scholarship tools should appear in the tool list.

Don't put MongoDB credentials in the Claude config. The Node process already loaded .env. The host only needs the URL.

How a Research Session Runs

Here's a realistic session against the seed catalog. The student is a first-generation undergraduate in computer science, a US citizen, GPA 3.6.

The host calls match_scholarships with that profile. The server returns ranked rows. A women-in-technology award and a first-generation program both score well, for different reasons. A graduate-only award never appears.

The host then calls get_scholarship on the top two ids. Each result includes the official URL, the requirement list, and the deadline. That's the moment to tell the student to open the URL. The model shouldn't be the last word on eligibility.

If an award is worth pursuing, the host calls save_scholarship with a researcherId such as ada and status saved. Later it can set applying. list_saved_scholarships is how a new chat picks up the shortlist. Persistence is the whole point of MongoDB here. Without it, every conversation starts from zero.

add_research_note is for the messy human details: "Ask Dr. Chen for a recommendation by October 1." get_upcoming_deadlines is the weekly sweep. compare_scholarships is for the moment the student has two finalists and needs amount, GPA, and requirements in one view.

The research-plan prompt packages that loop. It injects the profile into a user message that tells the model to call tools first and then produce a week-by-week plan. If you invoke the prompt without a connected server, you get a generic essay. If you invoke it with this server running, you get named awards and real dates.

That's the product: a catalog the model can query, a shortlist it can't forget, and prompts that make the workflow repeatable.

How the Matching Logic Works

It's worth slowing down on matching, because this is the part people are tempted to hand to the model.

Suppose the student is:

• GPA 3.6

• Computer science

• Undergraduate

• United States citizen

• First-generation

• A woman

The matcher walks every open award.

A women-in-technology scholarship with a 3.3 GPA minimum, CS as a listed field, and US eligibility scores high: education, GPA, citizenship, women-only flag, and field all hit. A first-generation program that is field-open also scores high, because any counts as a field match. A graduate-only award is skipped, even if the title looks relevant. A 3.8 GPA cutoff is skipped, even if everything else fits.

The tool then returns ranked rows with reasons:

1. Palantir Women in Technology Scholarship — score 90
   Why: education level matches; GPA 3.6 meets the 3.3 minimum; citizenship is eligible; women-only award matches; field of study matches

The model can still write a warm paragraph around that. It shouldn't be the component that decided eligibility.

If you extend this later, keep the same split. New eligibility rules belong in the service. New prose belongs in the prompt.

What You Can Build Next

The server you have is complete enough to use. It's also a base for a more serious research tool.

First, you could replace the seed file with a live source. Official feeds such as Grants.gov and college-maintained lists are safer than scraping commercial aggregators. Keep your schema. Write an importer that upserts by a stable external id.

You could also add authentication. createMcpExpressApp works with requireBearerAuth. Map researcherId from the verified token instead of a tool argument. The Express adapter documents that middleware.

Try adding full-text search. The schema already has a text index. For a large catalog, Atlas Search or a dedicated search engine will beat a pile of regex filters.

You can track documents, not just notes. A documents collection for transcripts, recommendation status, and essay drafts turns the shortlist into an application tracker.

You could also write tests around the matcher. Eligibility code is where silent bugs hurt people. A table of profiles and expected include/exclude lists will pay for itself.

And you could deploy it. Bind to 127.0.0.1 on your laptop. If you put this on a network, set allowedHosts, terminate TLS, and require a bearer token. An open MCP server is an open database with a helpful English interface.

Conclusion

In this guide, you built a scholarship research MCP server that's more than a toy tool list.

You stored awards, shortlists, and notes in MongoDB. You exposed search, matching, and research-tracking as MCP tools, with Zod schemas the host can advertise to a model. You added resources for the catalog and prompts for repeatable workflows. You served the whole thing over Streamable HTTP with Express, including the Host header checks the SDK enables for localhost.

The pattern transfers. Any research workflow with a catalog and a personal working set can use the same three layers: a service that owns the rules, an McpServer factory that registers primitives, and a small Express app that speaks the protocol.

If you take one idea from this tutorial, take this one: let the model write the plan, and let your server decide what's true.

The sample catalog is for learning. Confirm every scholarship on its official application page before you apply, and before you tell someone else to apply.

Then production traffic arrives.

The email API takes 2 seconds instead of 200 milliseconds. A few requests time out. During a provider outage, every signup starts returning 500. Support asks why users can create accounts but never get the confirmation link.

The bug isn't your email template. The bug is where you decided to send the email.

In this article, you'll learn why sending email inside the HTTP request path is a production trap, what goes wrong as you scale, and how to move email (and other side effects) into a background job queue so your API stays fast and reliable.

Prerequisites

You'll get more out of this article if you know:

• How REST APIs work (request, response, status codes)

• Basic Node.js or any backend language that can call an email API

• What a background job or queue is at a high level

You don't need prior experience with BullMQ, Redis, or a specific email provider.

Table of Contents

• Prerequisites

• 1. What "Inside the Request" Really Means

• 2. The Tempting Code That Causes the Problem

• 3. Why This Breaks in Production

• 4. The Rule: Your API Should Enqueue Work, Not Do It

• 5. How to Move Email to a Background Queue

• 6. How to Handle Retries, Failures, and Idempotency

• 7. What Else Should Leave the Request Path

• 8. A Practical Checklist

• Conclusion

1. What "Inside the Request" Really Means

When a client hits your API, the request path is everything that must finish before you send the HTTP response.

That usually includes:

• validating input

• checking auth

• writing to your database

• returning JSON

It should almost never include:

• calling an SMTP server

• waiting on Resend, SendGrid, SES, Mailgun, or Postmark

• rendering a big HTML template and uploading attachments

• retrying a flaky third-party network call

Email providers are external systems. They have their own latency, rate limits, and outages. If your API waits on them, their problems become your users' problems.

Bad path (synchronous):

Client ──▶ API ──▶ Database ──▶ Email provider ──▶ Client
                         ▲
                         └── user waits for all of this

2. The Tempting Code That Causes the Problem

Here's the pattern most teams start with. It's clear, short, and works on your laptop.

app.post('/signup', async (c) => {
  const body = await c.req.json();

  const user = await db.users.create({
    email: body.email,
    passwordHash: await hash(body.password),
  });

  // This is the trap.
  await emailClient.send({
    to: user.email,
    subject: 'Confirm your account',
    html: renderConfirmEmail(user),
  });

  return c.json({ id: user.id }, 201);
});

The request can't finish until the email provider answers. If that call is slow, your response is slow. If that call throws, your signup either fails or you catch the error and silently skip the email.

Neither option is good.

3. Why This Breaks in Production

Your Response Time Becomes Someone Else's SLA

A healthy signup endpoint should often respond in tens or hundreds of milliseconds. Email APIs regularly take longer than that. Under load, they take much longer.

Your p95 latency is no longer owned by your code. It's owned by the slowest third-party call on the path.

Failures Create Ugly Tradeoffs

If send() throws, you have two bad choices:

1. Fail the whole request: the user sees an error even though the account may already exist

2. Swallow the error: the user is created, but never gets the email

That second case is especially painful with confirmation or password-reset flows. The database says success but the inbox says nothing.

Timeouts Cascade

API gateways, load balancers, and browsers all have timeouts. A slow email call can turn a successful business action into a client-side failure. The user retries. Now you may create duplicate users or send duplicate emails.

Rate Limits Punish Traffic Spikes

Email providers rate-limit you. Product launches, digests, and password-reset spikes can all hit those limits. If sending happens inside the request, a rate limit error becomes a user-facing API error.

You Can't Retry Cleanly

Background jobs can retry with backoff. HTTP requests can't. Once you've returned a response, the request is over. If the email failed and you already told the client "success," you need another system to repair that. That system is a queue.

What the user experiences during a provider slowdown:

Signup click
   │
   ▼
API waits on email provider .............. 2s ... 5s ... timeout
   │
   ▼
"Something went wrong"
   │
   ▼
User clicks again → duplicate attempts, confused support tickets

4. The Rule: Your API Should Enqueue Work, Not Do It

Treat email as asynchronous work that must happen because of the request, not inside the request.

The request path should:

1. validate the input

2. commit the business change

3. enqueue a job that describes the email

4. return quickly

A worker somewhere else should:

1. pick up the job

2. render the template

3. call the email provider

4. retry on failure

5. record the result

Good path (asynchronous):

Client ──▶ API ──▶ Database ──▶ Queue ──▶ Client (fast response)
                                   │
                                   ▼
                                Worker ──▶ Email provider

This is the same idea behind processing uploads, generating PDFs, calling AI APIs, or syncing data to a CRM. Email is just the most common example.

5. How to Move Email to a Background Queue

Here's a minimal Node.js shape using BullMQ and Redis. The exact library matters less than the boundary.

Define the Job

// jobs/email.ts
export type SendEmailJob = {
  to: string;
  template: 'confirm-account' | 'password-reset' | 'welcome';
  variables: Record<string, string>;
  idempotencyKey: string;
};

Enqueue from the API

import { emailQueue } from '../queues/email-queue';

app.post('/signup', async (c) => {
  const body = await c.req.json();

  const user = await db.users.create({
    email: body.email,
    passwordHash: await hash(body.password),
  });

  await emailQueue.add(
    'send-email',
    {
      to: user.email,
      template: 'confirm-account',
      variables: {
        name: user.name,
        confirmUrl: `https://example.com/confirm/${user.confirmToken}`,
      },
      idempotencyKey: `confirm-account:${user.id}`,
    },
    {
      jobId: `confirm-account:${user.id}`,
      attempts: 5,
      backoff: { type: 'exponential', delay: 2000 },
      removeOnComplete: 1000,
      removeOnFail: 5000,
    },
  );

  return c.json({ id: user.id }, 201);
});

Notice what changed. The API still creates the user. It no longer waits for SMTP. It waits for Redis to accept a job, which is usually far cheaper and more predictable than waiting for an email provider.

Process the Job in a Worker

// worker/email-worker.ts
import { Worker } from 'bullmq';
import { emailClient } from '../lib/email-client';
import { renderTemplate } from '../lib/templates';

new Worker(
  'email',
  async (job) => {
    const { to, template, variables } = job.data;

    await emailClient.send({
      to,
      subject: subjectFor(template),
      html: renderTemplate(template, variables),
    });
  },
  {
    connection: redisConnection,
    concurrency: 5,
  },
);

Run the worker as a separate process from the API. That way an email outage slows the worker, not every HTTP response.

Return the Right Status

For signup, 201 Created is still correct if the account exists and the confirmation email is queued.

For endpoints that only trigger work, prefer 202 Accepted and return a job or delivery id when the client needs to track progress.

6. How to Handle Retries, Failures, and Idempotency

Moving email to a queue isn't enough. You also need failure rules.

Retry Transient Errors

Network blips, 429 Too Many Requests, and 503 Service Unavailable should retry with backoff. Permanent errors like an invalid recipient should fail fast and land in a dead-letter queue or failed-jobs table.

Make Jobs Idempotent

Workers can run more than once. Use a stable jobId or idempotencyKey so a retry doesn't send five welcome emails for one signup.

A practical approach:

• derive the key from the business event: confirm-account:user_123

• store provider message ids when a send succeeds

• skip work if that key was already delivered

Don't Lose the Business Event

If the process crashes between db.users.create() and emailQueue.add(), you can lose the email.

There are two common ways to reduce that risk:

1. Outbox pattern: write the email intent into an outbox table in the same database transaction as the user row. A separate publisher reads the outbox and enqueues jobs.

2. Transactional enqueue where supported: keep the write and the enqueue as close together as possible, and run reconciliation jobs for missed sends.

For many apps, a simple queue plus monitoring is enough at first. Add an outbox when missed emails become a real reliability requirement.

Separate Urgent Mail from Bulk Mail

Password reset and OTP emails shouldn't sit behind a newsletter blast.

Use priorities or separate queues:

• high priority: OTP, password reset, confirmation

• normal priority: receipts, onboarding

• low priority: digests, marketing

That keeps user-blocking email fast even when bulk traffic spikes.

7. What Else Should Leave the Request Path

If a task calls a third-party network API, can take more than a few hundred milliseconds, or it needs retries, keep it out of the request path. Same if it can fail without undoing the main business action or it does CPU-heavy work like image or PDF generation.

Here are some common examples of these types of tasks:

• emails and SMS

• webhook fan-out

• search indexing

• thumbnail generation

• invoice PDF creation

• CRM sync

• AI transcription or summarization

• slow report generation

Your API should confirm the intent and persist the state. Workers should do the slow parts.

8. A Practical Checklist

Before you ship a feature that sends email, ask:

1. Does the user need the email to finish before seeing success?

2. Can the email provider being down break this endpoint?

3. If sending fails after the database write, how do we recover?

4. If the job runs twice, will the user get duplicate messages?

5. Do OTP and password-reset emails have a faster lane than bulk email?

6. Can we see failed jobs in logs, metrics, or a dashboard?

7. Do we alert when the queue depth or failure rate spikes?

If you can't answer those questions, the design isn't ready for production.

Conclusion

Sending email inside your API request couples your response time and your error rate to a third-party system you don't control. It feels simple in a demo and becomes expensive in production.

The better default is simple:

• commit the business change

• enqueue the email

• return quickly

• let a worker send, retry, and report failures

That pattern makes your API faster, your email delivery more reliable, and your failure modes much easier to reason about.

Once you make that split for email, you'll start seeing the same boundary everywhere: if the user shouldn't wait for it, it doesn't belong in the request.

I ran into this while building Temploracraft, a résumé tool that takes an uploaded document and turns it into structured data the application can edit.

The input is genuinely unpredictable: two-column PDFs, tables that aren't really tables, and dates written in about fourteen different formats. The output has to be strict, because every extracted field lands in a form that a person is going to look at. When the model gets a date wrong, the user notices in about two seconds.

This article is about the layer that sits between "the model returned some text" and "my application has data it can trust." It covers the three mechanisms for constraining output, why designing the schema first beats writing a longer prompt, how to build a retry loop that doesn't set money on fire, and what to do about the failures that no retry will ever fix.

Table of Contents

• Prerequisites

• Why Prompting for JSON Isn't Enough

• The Three Ways to Constrain Output

• Start with the Schema, Not the Prompt

• Validation Is Two Jobs, Not One

• Building a Retry Loop That Doesn't Burn Tokens

• Streaming Structured Output

• The Failures You Can't Retry Away

• What This Actually Costs

• When You Don't Need Any of This

• Wrapping Up

• References

Prerequisites

To follow along comfortably, you'll want a few things in place:

• Working knowledge of TypeScript: The examples use type inference and generics lightly, and you should be able to read a type annotation without pausing.

• You have called a language model API at least once: You don't need to be an expert, but you should know what a system prompt is and roughly what a token is.

• Familiarity with JSON Schema is helpful but not required: I explain the parts that matter as they come up.

• Node.js 18 or later if you want to run the examples, since they use the native fetch and async iterators.

The code samples use Zod for schema definition and the Anthropic SDK for model calls, but every technique here translates directly to other validation libraries and other providers. The ideas matter more than the specific packages.

Why Prompting for JSON Isn't Enough

The first instinct when you need structured data is to ask for it politely. You write something like "Respond with valid JSON matching this shape, and do not include any other text," you paste an example, and it works. It keeps working through development. It works in your demo.

The problem is that a language model generates one token at a time based on probability, and your instruction is only one influence among many. It competes with the model's training, the shape of the input document, and whatever the model produced in the preceding few hundred tokens. Most of the time your instruction wins. Occasionally it does not.

Here are failures I've actually collected from a production extraction pipeline, all from a model that was explicitly told to return strict JSON:

• The model wrapped the response in a Markdown code fence, despite being told twice not to.

• It returned "2019 - Present" as a single string where the schema defined separate startDate and endDate fields.

• It invented an endDate of "2024-12-31" for a role the document clearly marked as current.

• It returned the string "null" instead of an actual null.

• It emitted seven bullet points for a role where the schema set a maximum of five.

• It truncated mid-object because the response hit the output token limit.

Notice that these aren't all the same kind of failure. The code fence and the truncation are syntax problems, and you can often fix them locally without calling the model again. The merged date string and the extra bullets are schema problems, where the JSON parses fine but doesn't match the shape you need. The invented end date is a semantic problem, where the output is both valid JSON and schema-conformant but factually wrong about the source document.

Those three categories need three different responses, and conflating them is the most common architectural mistake I see in extraction code. The rest of this article is largely about separating them.

The Three Ways to Constrain Output

Before writing any validation code, it's worth understanding what the API itself can enforce for you. There are three mechanisms, and they give you meaningfully different guarantees.

JSON mode is the weakest of the three. You set a flag such as response_format: { type: "json_object" }, and the provider guarantees the response will be syntactically valid JSON. That's genuinely useful, because it eliminates the code fence and truncation problems in one move. What it doesn't do is guarantee anything about the shape. You can get valid JSON with the wrong keys, the wrong types, or a completely different structure than you asked for.

Tool calling is the mechanism I reach for most often. You describe a function with a JSON Schema for its parameters, then force the model to call it. The model returns arguments matching that schema rather than free text. Support is broad, the schema travels with the request so you're not duplicating it in the prompt, and providers tend to constrain the output more aggressively than they do in plain JSON mode.

Here's what that looks like with the Anthropic SDK:

const response = await client.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 4096,
  tools: [
    {
      name: "emit_resume",
      description: "Return the parsed resume as structured data.",
      input_schema: jsonSchema,
    },
  ],
  tool_choice: { type: "tool", name: "emit_resume" },
  messages: [{ role: "user", content: resumeText }],
});

const block = response.content.find((b) => b.type === "tool_use");
const candidate = block?.input;

The tool_choice field is the important part. Without it the model decides whether to call the tool, and sometimes it will just answer in prose instead. Forcing the specific tool removes that branch entirely.

Constrained decoding is the strongest option and the least widely available. Instead of asking the model to follow a schema, the runtime masks the token sampler at every step so that tokens which would produce invalid output can't be selected at all. Invalid output becomes impossible rather than unlikely.

OpenAI exposes a version of this through strict structured outputs, and if you run models locally you can use GBNF grammars in llama.cpp or a library such as Outlines.

The tradeoff is that constrained decoding can push the model into awkward corners. If the schema demands a field the document genuinely doesn't contain, the model can't decline, so it fills the slot with something. You've traded a parsing failure for a hallucination, which is harder to detect. I make required fields nullable for exactly this reason, which I'll come back to shortly.

My default is tool calling with a nullable-heavy schema, plus validation on top. That combination gives most of the benefit without the issues.

Start with the Schema, Not the Prompt

The instinct when output quality is poor is to write a longer prompt. More examples, more emphasis, and more capital letters. This helps a little and scales badly, because the prompt is prose and prose isn't enforceable.

A better approach is to treat the schema as the primary artifact and let everything else derive from it. Define it once, and use that single definition to generate the TypeScript type, the JSON Schema sent to the API, and the runtime validator. When the shape changes, all three change together, and there's no way for them to drift apart.

With Zod that looks like this:

import { z } from "zod";

const YearMonth = z
  .string()
  .regex(/^\d{4}-\d{2}$/, "Expected a YYYY-MM date");

const Role = z.object({
  company: z.string().min(1),
  title: z.string().min(1),
  startDate: YearMonth,
  endDate: YearMonth.nullable(),
  bullets: z.array(z.string().min(1)).min(1).max(5),
});

const Resume = z.object({
  name: z.string().min(1),
  email: z.string().email().nullable(),
  roles: z.array(Role),
});

export type Resume = z.infer<typeof Resume>;

Three details in there are doing real work.

The YearMonth regex is narrower than z.string(), and that narrowness is the point. A bare string field invites the model to return "January 2019" or "2019 - Present" or "01/2019", and all of those pass validation. Constraining the format at the schema level means the mismatch surfaces immediately instead of three layers deeper in your date-handling code.

The endDate field is nullable rather than optional. This is the single change that most improved extraction quality for me. An optional field lets the model quietly omit it, and you can't distinguish "the document did not say" from "the model forgot." A nullable field forces an explicit decision, and null is a meaningful answer that means the role is current.

The max(5) on bullets encodes a product constraint directly into the contract rather than trimming the array afterward. If the model exceeds it, you want to know, because it usually means the model is padding rather than extracting.

To send the schema to the API, derive the JSON Schema from the same definition:

import { zodToJsonSchema } from "zod-to-json-schema";

const jsonSchema = zodToJsonSchema(Resume, { target: "openApi3" });

One habit worth adopting: write a .describe() on any field where the name alone is ambiguous. Those descriptions end up in the JSON Schema, which means they reach the model as part of the tool definition, which means they function as targeted, structured prompt instructions attached to exactly the field they concern.

const Role = z.object({
  company: z.string().min(1),
  title: z.string().min(1).describe("The person's job title, not the team name"),
  startDate: YearMonth,
  endDate: YearMonth.nullable().describe("null if this role is current"),
  bullets: z
    .array(z.string().min(1))
    .min(1)
    .max(5)
    .describe("Verbatim from the document. Do not rewrite or summarise."),
});

That last description eliminated an entire class of failure for me, where the model would helpfully improve the wording of a person's own bullet points.

Validation Is Two Jobs, Not One

Once the response comes back, it's tempting to run schema.parse() and consider the job done. Zod will tell you whether the shape is right, and if it is, you move on.

But shape validation and semantic validation are different jobs, and only the first one is free. Zod can tell you that endDate is a string matching YYYY-MM. It can't tell you that the end date falls before the start date, or that the date is in the future, or that a role listed as current also has an end date. Those are all schema-valid and all wrong.

So run two passes. The first is structural and comes from the schema. The second is a plain function that encodes what you know about the domain:

function findSemanticProblems(resume: Resume): string[] {
  const problems: string[] = [];
  const currentMonth = new Date().toISOString().slice(0, 7);

  for (const role of resume.roles) {
    if (role.startDate > currentMonth) {
      problems.push(`${role.company}: start date is in the future`);
    }
    if (role.endDate && role.endDate < role.startDate) {
      problems.push(`${role.company}: end date precedes start date`);
    }
  }

  const currentRoles = resume.roles.filter((r) => r.endDate === null);
  if (currentRoles.length > 1) {
    problems.push("More than one role is marked as current");
  }

  return problems;
}

None of this is clever code, and that's rather the point. It's ordinary business logic that happens to be checking a model's work rather than a user's. Write it as you discover failures, and treat each new check as a permanent regression test for a mistake the model made once.

There's a useful asymmetry here. Semantic problems are often better surfaced to the user than retried, because the model frequently can't do better with the information available. If a document genuinely lists two current roles, that's what the document says, and asking the model again won't change it.

Building a Retry Loop That Doesn't Burn Tokens

When validation fails, the naïve fix is to call the model again with the same prompt and hope for a different sample. That works often enough to feel reasonable and is wasteful enough to notice on the bill, because you're paying full input cost for a request that has already mostly succeeded.

Two changes make a large difference.

The first is to attempt a local repair before spending anything. A meaningful share of failures are cosmetic, and you can fix them with string handling:

function salvage(raw: string): string {
  let text = raw.trim();

  // Strip a Markdown fence the model added despite instructions.
  const fenced = text.match(/^```(?:json)?\s*([\s\S]*?)\s*```$/);
  if (fenced) text = fenced[1];

  // Drop any prose before the first brace or after the last one.
  const start = text.indexOf("{");
  const end = text.lastIndexOf("}");
  if (start !== -1 && end > start) text = text.slice(start, end + 1);

  return text;
}

The second is that when you do call the model again, you should send a repair request rather than a fresh attempt. Include the original prompt, the output that failed, and the specific validation errors. The model has already done the hard extraction work, and you're asking it to fix a small number of named problems rather than start over. Repairs converge faster and produce shorter outputs, which means they cost less.

async function requestRepair(
  originalPrompt: string,
  badOutput: string,
  error: z.ZodError,
): Promise<string> {
  const issues = error.issues
    .map((issue) => `${issue.path.join(".") || "root"}: ${issue.message}`)
    .join("\n");

  const response = await client.messages.create({
    model: "claude-sonnet-5",
    max_tokens: 4096,
    messages: [
      { role: "user", content: originalPrompt },
      { role: "assistant", content: badOutput },
      {
        role: "user",
        content:
          `That output failed validation with these problems:\n${issues}\n\n` +
          `Return the corrected JSON only. Keep everything that was already correct.`,
      },
    ],
  });

  return response.content[0].type === "text" ? response.content[0].text : "";
}

Putting it together, the full loop separates the failure classes and caps the spend:

type Outcome<T> =
  | { ok: true; value: T; attempts: number }
  | { ok: false; error: string; attempts: number };

async function extract<T>(
  schema: z.ZodType<T>,
  prompt: string,
  maxAttempts = 3,
): Promise<Outcome<T>> {
  let lastRaw = "";
  let lastError: z.ZodError | null = null;

  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    lastRaw =
      attempt === 1
        ? await callModel(prompt)
        : await requestRepair(prompt, lastRaw, lastError!);

    let candidate: unknown;
    try {
      candidate = JSON.parse(salvage(lastRaw));
    } catch {
      continue; // Syntax failure. Try again without a schema error to report.
    }

    const result = schema.safeParse(candidate);
    if (result.success) {
      return { ok: true, value: result.data, attempts: attempt };
    }
    lastError = result.error;
  }

  return {
    ok: false,
    error: lastError?.message ?? "Output was never parseable",
    attempts: maxAttempts,
  };
}

Two things about this loop are deliberate. It returns the attempt count, which you should log, because attempts per success is the single most useful health metric for an extraction pipeline. And it caps attempts at three. If three tries haven't produced valid output, a fourth rarely helps, and by then you're better off degrading gracefully than continuing to pay.

One more distinction worth respecting: a rate limit error and a schema validation error aren't the same failure and shouldn't share a retry policy. Rate limits want exponential backoff, because the problem is timing. Schema failures want an immediate repair request, because the problem is content, and waiting changes nothing.

Streaming Structured Output

Streaming and structured output pull against each other. Streaming exists so the user sees progress before the response finishes, but you can't parse a JSON object until the closing brace arrives. If your extraction takes twelve seconds, the best options are to show a spinner for twelve seconds or find a way to stream something meaningful.

There are two reasonable approaches.

The first is a partial JSON parser, which takes an incomplete string and returns the largest valid structure it can infer, closing open braces and dropping the trailing incomplete value. Libraries such as best-effort-json-parser do this. It works, and it's the right choice when the output is genuinely one large object. The cost is that intermediate states can be misleading, because a field can appear with a truncated value that looks complete.

The second approach, which I prefer when the output is a list, is to change the output format so that streaming is natural. Instead of asking for one array of objects, ask for one object per line. Each line is independently parseable, so you can validate and emit items the moment they complete:

const stream = client.messages.stream({
  model: "claude-sonnet-5",
  max_tokens: 4096,
  messages: [{ role: "user", content: prompt }],
});

let buffer = "";

for await (const event of stream) {
  if (event.type !== "content_block_delta") continue;
  buffer += event.delta.text ?? "";

  const lines = buffer.split("\n");
  buffer = lines.pop() ?? ""; // Keep the incomplete tail for the next chunk.

  for (const line of lines) {
    if (!line.trim()) continue;

    try {
      const parsed = Role.safeParse(JSON.parse(line));
      if (parsed.success) onRole(parsed.data);
    } catch {
      // A malformed line is dropped rather than failing the whole stream.
    }
  }
}

The buffer handling is the part that people get wrong. Network chunks don't align with line boundaries, so a chunk will frequently end mid-object. Popping the final element back into the buffer and carrying it forward is what makes the loop correct.

This pattern gives you per-item validation for free, which is a real advantage over parsing one large object. One bad item doesn't invalidate the rest of the extraction.

The Failures You Can't Retry Away

Everything so far assumes the model can produce the right answer if you ask correctly. Some failures don't work that way, and treating them as retry candidates wastes money while producing confidently wrong data.

The most important of these is extraction from something the source doesn't contain. If a résumé genuinely has no email address and your schema requires one, the model will produce something plausible.

Retrying produces a different plausible thing. This is the failure mode that constrained decoding makes worse rather than better, because the grammar guarantees a well-formed value in a slot that should've been empty. The fix is at the schema level, which is why nearly every field in my extraction schemas is nullable.

Ambiguity in the source is a similar case. When a document lists a date range next to two different job titles, there's no correct extraction, only a guess. Retrying gives you a different guess with the same confidence. These situations want a confidence signal in the schema and a review step in the interface, not another API call.

Then there's silent truncation. If the response hits the output token limit mid-object, you get syntactically broken JSON, and a naïve retry loop treats it as a transient parse failure and tries again with the same limit, failing identically each time. Check the stop reason on the response. If the model stopped because it ran out of tokens, retrying without raising the limit or splitting the input is guaranteed to fail again.

The general principle is that your retry policy should ask what kind of failure this is before deciding whether repeating the call could possibly help.

What This Actually Costs

Every layer above has a price, and it's worth measuring rather than assuming.

The main figure to track is attempts per successful extraction. Log it on every request and watch the p95 rather than the mean, because the mean hides the tail where the money goes. When a schema change causes a quality regression, this number moves before anything else does.

Prompt caching matters more here than in most workloads. Extraction prompts are unusually cache-friendly because the system prompt, the schema, and any few-shot examples are byte-identical on every request, and only the document changes.

On a pipeline where the schema and instructions ran to a couple of thousand tokens, moving that block into a cached prefix cut input cost substantially, and the saving compounds with every retry because repairs re-send the same prefix.

Repair requests are cheaper than fresh attempts for a reason worth understanding. The input grows, since you're now sending the original prompt plus the failed output plus the error list. But the output shrinks considerably, because the model is correcting a handful of fields rather than generating the full document again. Output tokens are the expensive direction on most providers, so the trade is usually favourable.

Two other things are worth instrumenting from the start: the distribution of validation error paths, which tells you exactly which schema fields are causing trouble and is far more actionable than an overall failure rate, and your local-repair hit rate, since if salvage() is fixing a large fraction of responses you have a prompt problem you can solve once rather than paying for repeatedly.

When You Don't Need Any of This

This machinery earns its place in a specific set of circumstances, and it's genuinely overkill outside them.

If the model's output goes straight to a human who will read it as prose, you don't need structured output at all. A chat interface, a summary, or a draft email all have a person as the validator, and adding a schema in front of that just constrains the model for no benefit.

If you're extracting one or two fields rather than a document's worth, a well-targeted prompt with a light safeParse and a single retry will serve you fine. The retry loop, semantic validation layer, and streaming parser are answers to problems that appear at scale and with schema complexity.

If your volume is low and a human reviews every result anyway, the validation layer is duplicating work someone is already doing. Surface the raw output, let the reviewer correct it, and spend the engineering time elsewhere.

And if you're still exploring what the feature should be, resist building this early. The schema is the most expensive thing to change once extraction code, validation rules, and stored data all depend on it. Prototype loosely, learn what fields you actually need, and tighten afterwards.

The safest rule is to start with a schema and safeParse, and add each subsequent layer only when a real failure justifies it. Every technique in this article came from a specific production incident rather than from a design document.

Wrapping Up

The gap between a working demo and a reliable feature is almost entirely in this layer. The model isn't the hard part any more, and the prompt isn't usually the hard part either. The hard part is deciding what you'll accept, detecting when you didn't get it, and responding sensibly when that happens.

Three ideas carry most of the weight:

1. The schema is the contract, and everything derives from it. One definition producing your type, your API schema, and your validator means those three can never drift apart.

2. Separate syntax failures, schema failures, and semantic failures. They have different causes and different fixes, and a retry loop that treats them identically will waste money on problems that repeating the call can't solve.

3. Make fields nullable rather than optional. Forcing the model to say "this was not present" rather than allowing it to quietly omit the field converts a whole class of silent hallucination into an explicit, checkable value.

Get those right and the rest is ordinary engineering. You're writing validation code and retry logic, which developers have been doing against unreliable inputs for decades. A language model is just a new kind of unreliable input, and it responds well to the same discipline.

References

• Zod: the schema library used throughout these examples.

• zod-to-json-schema: converts a Zod schema into the JSON Schema an API expects.

• Anthropic tool use documentation: forcing structured arguments through a tool definition.

• OpenAI structured outputs: strict schema conformance through constrained decoding.

• Outlines: grammar-constrained generation for locally hosted models.

• best-effort-json-parser: parsing incomplete JSON while a response is still streaming.

A Slack read uses that user's workspace. A GitHub issue is created as that user, in a repository they can access. An agent can make the wrong call, but it must never act with the wrong user's access.

The fix has two parts, and both appear in the first half of this tutorial:

1. Each user grants access separately. Alice authorizes Slack for herself. Bob authorizes it for himself.

2. Your agent passes an identifier, not a token. A string like alice@example.com selects whose grant to use. One function turns it into a token at the moment of the call, and that token never reaches your model inputs, your tool schemas, or your logs.

Most agent tutorials stop before either point. They hand you an API key, wire up one function, and the model calls it. The design works until a second person shows up.

To make the pattern concrete, you'll build a command-line agent that watches a Slack channel, decides on its own which messages describe real work, files a GitHub issue for those, and replies in the Slack thread with the issue link. Every call runs as one user's own OAuth grant.

You'll write the OAuth flow yourself: the consent redirect, the state check, the token exchange, an encrypted store, and the refresh path. None of it is long, and seeing it whole is what makes the identity argument checkable instead of a claim you take on faith.

Two topics stay out of scope here: we won't cover Model Context Protocol servers or voice or realtime hosts. The identity pattern holds in both settings, but the surrounding plumbing deserves its own article.

Table of Contents

• What You'll Build

• Prerequisites

• What Are AI Agent Tools?

• Why a Shared Token Breaks

• Architecture Overview

• How to Register the Slack and GitHub OAuth Apps

• How to Run the Consent Flow

• How to Store Tokens Encrypted, Keyed by User

• How to Run Tool Calls as the Current User

• How to Handle Refresh and Revocation

• How to Add a Second Provider

• Full Walkthrough

• How to Apply the Pattern to Other Use Cases

• What Went Wrong When I Built This

• Conclusion

What You'll Build

The agent is called channel-watcher-agent. Each run does four things:

1. Reads recent messages from a Slack channel.

2. Asks a model, message by message, whether the text describes a bug or a concrete action item.

3. Files a GitHub issue for the messages that qualify.

4. Replies in the original Slack thread with a link to the new issue.

Nobody clicks a button to start any of it. Slack already ships a "create an issue from this message" action, which is a different product. Here the agent reads the channel, forms its own judgment, and acts only on what it judges worth acting on.

The stack stays small on purpose:

Three of those five ship with Node. The only packages you install are the AI SDK and its friends.

By the end you'll have:

• Two OAuth apps, Slack and GitHub, that a user consents to once.

• An encrypted token store keyed by user and provider.

• An agent that resolves the current user to an identifier and never lets a token reach the model.

• A tool loop where the model decides whether to file an issue at all.

• A demonstration that a second user's run stops instead of reading the first user's data.

The finished code lives at github.com/saif-shines/channel-watcher-agent.

Prerequisites

Accounts and tools:

• Node.js 22.13 or newer, plus npm. The token store uses node:sqlite, which is stable from that version on.

• A Slack workspace where you can install apps, and a channel to watch. A throwaway channel works best.

• A GitHub account and a repository that can absorb test issues.

• An API key for a model provider the AI SDK supports. Anthropic is used in the examples.

• mkcert, to issue a local HTTPS certificate. How to Register the Slack and GitHub OAuth Apps explains why an ordinary http://localhost callback will not do.

Useful background, though none of it is a hard requirement:

• async and await, and reading a small Node script.

• OAuth 2.0 at a high level: an app redirects a user to a provider, the user consents, the app receives a token.

• Tool calling, sometimes called function calling. The next section covers what the tutorial needs.

One warning before starting: The agent writes to real systems. It opens real GitHub issues and posts real Slack messages. Use a test Slack channel and a throwaway GitHub repository while you're still checking that it only acts on messages you intend.

What Are AI Agent Tools?

A tool is a function you hand the model along with your input. The model can't run that function itself. It can only ask: call fileGithubIssue with this title and this body. Your code performs the call, returns the result, and the model uses that result to choose the next step.

Request, execute, return. The exchange is the whole mechanism, and everything called an "agent" is a loop around it.

How a Tool Differs from an API

Tools and APIs wrap the same call but are written for different readers.

An API is written for you. It assumes you read the documentation, and that you know thread_ts is the field that turns a Slack message into a threaded reply.

A tool is written for a model that has read nothing. So a tool carries its own explanation:

• A name the model can reason about, like fileGithubIssue.

• A description in plain language, including when not to use the tool.

• A schema for the inputs, so the model knows title is a required string.

Below is one tool from the project. Most of the code is explanation rather than logic:

const fileGithubIssue = tool({
  description: 'File a GitHub issue for an actionable Slack message',
  inputSchema: z.object({
    title: z.string(),
    body: z.string(),
  }),
  execute: async ({ title, body }) => {
    // ... the actual API call goes here
  },
});

The description and inputSchema are the parts the model sees. The execute function is yours alone. Identity gets settled inside execute, so the model never learns which account the call ran against.

Why Models Handle Tools Better Than Raw API Calls

Pasting a curl command into the input and asking the model to fill in the blanks is possible. But this approach fails in predictable ways.

Tools work better for three reasons:

1. The schema is enforced before your code runs. A malformed tool call gets rejected and retried by the SDK. A malformed URL fails at runtime instead.

2. Results return to the model. After fileGithubIssue returns, the model can read the new issue URL and use it in the Slack reply. The chaining is what makes the second step possible.

3. Credentials stay out of the conversation. The model asks for an action by name and never sees a token. A token it never sees can't leak into a completion, a log line, or a prompt-injection payload.

Reason three is what the rest of this tutorial builds toward. You'll keep tokens out of the model on purpose: the agent holds an identifier, and a token appears only at the moment of the provider call.

Most Agents Need More Than One App

Few useful agents talk to a single app. A support agent reads Zendesk and updates Salesforce. A standup agent reads GitHub and posts to Slack. A scheduling agent reads Gmail and writes to Google Calendar.

Each app brings its own OAuth registration, scope names, token lifetime, and refresh behavior. Multiply the list by every user of the agent, and the real problem appears.

Why a Shared Token Breaks

One shared credential for everybody works in a demo and fails once a second person shows up. Picture the quick version of the Slack half: create a Slack app, install it, copy the bot token into .env, and let every tool call use it.

Three problems arrive together.

First, every run uses the same permissions. The bot sees every channel it was invited to, no matter who triggered the run. Ask the agent about a channel you were never in, and the bot reads it anyway. The agent has become a way around your own workspace permissions.

Second, the audit trail is also wrong. Every GitHub issue says the bot opened it. Every Slack reply comes from the bot. Asked why an issue exists, the honest answer is "an agent filed it for somebody, and we can't tell who."

And third, revocation stops working. A user leaves the company and their Slack account is deactivated. The agent keeps running, because it never used their credentials.

The alternative is per-user grants. Each user authorizes the apps for themselves. That creates a new requirement, though: somewhere to keep those grants.

The Distinction is One Field in One Response

Slack makes the difference unusually easy to see. When a user finishes the consent screen, the token exchange returns both kinds of token in the same JSON object:

{
  "ok": true,
  "access_token": "xoxb-REDACTED-BOT-TOKEN",
  "token_type": "bot",
  "authed_user": {
    "id": "U0A1B2C3D",
    "scope": "channels:history,chat:write,users:read",
    "access_token": "xoxp-REDACTED-USER-TOKEN",
    "token_type": "user"
  }
}

The top-level access_token is the bot. The nested authed_user.access_token is the person who just consented. Reading conversations.history with the first one returns every channel the app was invited to. Reading it with the second returns only the channels that users can already see. The same split governs writes: chat.postMessage with a user token posts under that person's name.

Two fields, one letter apart in the prefix, and the entire permission model of your agent hangs on which one you store. This tutorial requests only user scopes, so Slack issues no bot token at all.

Tokens Must Stay Out of the Model and the Logs

Per-user tokens become the most sensitive data in the system. Two destinations are off limits:

• The model: Keep tokens out of inputs, tool descriptions, and tool return values. A model that has seen a token can repeat it, and prompt injection turns any tool result into untrusted input.

• Your logs: Tool inputs and outputs are exactly what you want to log while debugging an agent. Tokens traveling in those payloads land in your log store permanently.

This tutorial keeps tokens on one narrow path. Your code passes an identifier, a stable reference to one user. One helper turns that identifier into a token, and from there the token goes straight into a provider call and nowhere else. It's never named in a tool schema, never attached to anything the model can read, and never returned from a tool.

Why You Own the OAuth Apps and the Store

The point of writing the flow yourself isn't the plumbing. It's control over who may use whose grant.

In this tutorial the users are teammates. Each person connects their own Slack and GitHub, and the agent acts as whoever triggered the run. The same design holds when those users are customers of your product: each person still has their own grant, and a wrong mapping means one person's run using someone else's access. Only the source of the identifier changes. A session for teammates, a tenant record for customers.

Architecture Overview

Two flows matter, and they happen at different times. Keeping them separate is most of the work.

Connection time happens once per user, per app. The user consents, and tokens land in your store. The agent isn't running.

Runtime happens on every execution. The agent resolves the current user to an identifier and does its work. No consent screens and no browser.

CONNECTION TIME (once per user, per app)

  Your user              connect.js              Slack / GitHub
     |                       |                         |
     |-- "connect Slack" --->|                         |
     |<--- consent link -----|                         |
     |----------------------- OAuth consent ---------->|
     |                       |<--- redirect + code ----|
     |                       |---- exchange code ----->|
     |                       |<---- tokens ------------|
     |                       |                         |
     |                  [encrypt, store                |
     |                   under (identifier,            |
     |                   provider)]                    |
     |                       |                         |


RUNTIME (every agent run)

  Your agent             Token store             Slack / GitHub
     |                       |                         |
  [resolve identifier        |                         |
   from your own session]    |                         |
     |                       |                         |
     |-- getAccessToken( --->|                         |
     |     identifier,       |                         |
     |     provider )        |                         |
     |<---- token -----------|                         |
     |                       |                         |
     |------------------ API call as user ------------>|
     |<----------------- result -----------------------|
     |                       |                         |
  [model sees result,        |                         |
   never a token]            |                         |

Three properties follow from the shape.

The identifier replaces the token in your agent code. Everything above the token store handles a string like alice@example.com or user_8f21c. The string is worthless on its own: without the store and its encryption key, it opens nothing.

One identity spans many apps. A single identifier has a Slack row and a GitHub row beneath it. A third app doesn't create a third identity to reconcile.

Authorization stays in your code. The store answers which tokens belong to an identifier. The store can't know whether the request deserved an answer. Deciding that the caller may act as that identifier happens before any call.

One rule follows, and bending it defeats the whole design: resolve the identifier server-side from an authenticated session. Never accept an identifier from a request body, a query parameter, or a browser. An identifier accepted from a client is an "act as any user" endpoint.

How to Register the Slack and GitHub OAuth Apps

The walkthrough uses Slack and GitHub as the two providers end to end. Both need the same three things: a registered app, a redirect URI, and a set of scopes. The details differ enough to be worth walking through separately.

The Redirect URI Has to Use HTTPS

Most tutorials that touch OAuth hand you http://localhost:3000/callback and move on. Slack rejects it. Slack's documentation states flatly that "a Redirect URL must also use HTTPS", and it makes no exception for localhost. GitHub is more relaxed and accepts either, so a single HTTPS callback satisfies both.

The rule looks pedantic, because on localhost the request never leaves your machine and there's nothing on the wire to intercept. Slack applies it uniformly anyway, and a uniform rule with no exemptions is a defensible choice for a provider handing out credentials: every exemption is a branch somebody has to get right, and "is this really localhost" is a question that has been answered incorrectly before.

mkcert issues a certificate signed by a local authority it adds to your system trust store, so the browser accepts it without a warning:

mkcert -install
mkcert localhost

That writes localhost.pem and localhost-key.pem into the current directory. A tunneling service such as ngrok also works, but its free URLs rotate, which means re-editing both app registrations every session.

The Slack App, and the One Setting That Matters

At api.slack.com/apps, create an app in your workspace. Then open OAuth & Permissions and set two things.

Add https://localhost:3000/callback under Redirect URLs.

Then find the scopes. The page has two sections, and choosing the wrong one silently rebuilds the shared-bot design:

Under User Token Scopes, add:

• channels:history: read messages in public channels the user belongs to

• chat:write: post as the user

• users:read: turn user IDs into names

Leave Bot Token Scopes empty. Copy the Client ID and Client Secret from Basic Information.

The GitHub OAuth App

Under Settings → Developer settings → OAuth Apps → New OAuth App, set the Authorization callback URL to the same https://localhost:3000/callback, then generate a client secret. GitHub documents the web application flow in full if you want the surrounding detail.

GitHub's scope for issue creation depends on the repository:

• repo covers private repositories, and grants read and write access to code along with it.

• public_repo is the narrower choice, and enough when your test repository is public.

Take the narrower one when you can. A scope you didn't need is a scope you have to explain later.

The Environment File

Both apps produce a client ID and a client secret, and the store needs an encryption key. Generate the key first:

node -e "console.log(require('node:crypto').randomBytes(32).toString('base64'))"

Then fill in .env:

OAUTH_REDIRECT_URI=https://localhost:3000/callback
TLS_CERT_PATH=./localhost.pem
TLS_KEY_PATH=./localhost-key.pem

SLACK_CLIENT_ID=
SLACK_CLIENT_SECRET=
GITHUB_CLIENT_ID=
GITHUB_CLIENT_SECRET=

TOKEN_ENCRYPTION_KEY=

SLACK_CHANNEL_ID=C0XXXXXXXXX
GITHUB_REPO=your-name/your-test-repo

Those client secrets authenticate your application to the providers. They're not user credentials, and they never belong in a browser.

How to Run the Consent Flow

Everything provider-specific belongs in one place, so that adding a third provider later means adding an entry rather than a branch.

Step 1: Describe Each Provider Once

const REDIRECT_URI = process.env.OAUTH_REDIRECT_URI;

export const providers = {
  slack: {
    label: 'Slack',
    authorizeUrl: 'https://slack.com/oauth/v2/authorize',
    tokenUrl: 'https://slack.com/api/oauth.v2.access',

    // These go in `user_scope`, not `scope`. Scopes listed under `scope` grant
    // a bot token, and a bot token is what this project exists to avoid.
    userScopes: ['channels:history', 'chat:write', 'users:read'],

    buildAuthorizeUrl(state) {
      const url = new URL(this.authorizeUrl);
      url.searchParams.set('client_id', process.env.SLACK_CLIENT_ID);
      url.searchParams.set('user_scope', this.userScopes.join(','));
      url.searchParams.set('redirect_uri', REDIRECT_URI);
      url.searchParams.set('state', state);
      return url.toString();
    },
    // exchangeCode and refresh follow below
  },
};

The user_scope parameter is the whole argument in one line. Slack reads scope for bot permissions and user_scope for user permissions. This project sets only the second, so the response comes back with no bot token in it at all.

The state parameter isn't optional. It's a random string you generate, send to the provider, and check on the way back. Without it, any page on the internet can point a browser at your callback URL with an attacker's code attached, and your server will happily exchange it and store the attacker's token under your user's identifier.

Step 2: Exchange the Code, and Take the Right Token

async exchangeCode(code) {
  const response = await fetch(this.tokenUrl, {
    method: 'POST',
    headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
    body: new URLSearchParams({
      code,
      client_id: process.env.SLACK_CLIENT_ID,
      client_secret: process.env.SLACK_CLIENT_SECRET,
      redirect_uri: REDIRECT_URI,
    }),
  });

  const json = await response.json();

  // Slack answers HTTP 200 even when the exchange failed. The `ok` field
  // is the real status.
  if (!json.ok) {
    throw new Error(`Slack token exchange failed: ${json.error}`);
  }

  return normalizeSlackTokens(json.authed_user);
}

Two details in that function cost real debugging time when missed.

Slack returns HTTP 200 for failures. Checking response.ok tells you the HTTP request succeeded, which it did. The json.ok field is the one that reports whether the OAuth exchange worked.

json.authed_user, not json. This is the fork from the section above, expressed as one property access. Reading json.access_token here would compile, run, store a token, and quietly give every user of your agent the same bot identity.

Normalizing the result keeps the rest of the codebase provider-agnostic:

function normalizeSlackTokens(authedUser) {
  return {
    accessToken: authedUser.access_token,
    refreshToken: authedUser.refresh_token ?? null,
    expiresAt: authedUser.expires_in
      ? Date.now() + authedUser.expires_in * 1000
      : null,
    scope: authedUser.scope,
  };
}

GitHub's version of the same function differs in two ways worth noting:

async exchangeCode(code) {
  const response = await fetch(this.tokenUrl, {
    method: 'POST',
    // Without this header GitHub answers with a form-encoded body.
    headers: {
      'Content-Type': 'application/x-www-form-urlencoded',
      Accept: 'application/json',
    },
    body: new URLSearchParams({
      code,
      client_id: process.env.GITHUB_CLIENT_ID,
      client_secret: process.env.GITHUB_CLIENT_SECRET,
      redirect_uri: REDIRECT_URI,
    }),
  });

  const json = await response.json();
  if (json.error) {
    throw new Error(
      `GitHub token exchange failed: ${json.error_description ?? json.error}`
    );
  }

  // OAuth App tokens carry no expiry, so there is nothing to refresh.
  return {
    accessToken: json.access_token,
    refreshToken: null,
    expiresAt: null,
    scope: json.scope,
  };
}

The Accept: application/json header is easy to skip and produces a confusing failure: response.json() throws on a body that came back as access_token=gho_...&scope=repo.

Step 3: Catch the Redirect

OAuth needs somewhere to land. For a command-line tool, a server that starts, handles one callback per provider, and exits is enough. Because Slack demands HTTPS, the scheme in OAUTH_REDIRECT_URI decides which kind of server to start:

function createCallbackServer(handler) {
  if (redirect.protocol !== 'https:') {
    return createHttpServer(handler);
  }

  try {
    return createHttpsServer(
      {
        cert: readFileSync(process.env.TLS_CERT_PATH),
        key: readFileSync(process.env.TLS_KEY_PATH),
      },
      handler
    );
  } catch (err) {
    throw new Error(
      `Could not read the TLS certificate (${err.code ?? err.message}).\n` +
        'Generate a locally-trusted one with mkcert:\n' +
        '  mkcert -install\n' +
        '  mkcert localhost\n' +
        'then point TLS_CERT_PATH and TLS_KEY_PATH at the two files it writes.'
    );
  }
}

A missing certificate is going to happen to somebody, and ENOENT on its own explains nothing about OAuth. The catch block spends four lines saying what to run instead.

The handler itself is where state gets checked:

const pending = new Map();

function handleCallback(request, response) {
  const url = new URL(request.url, redirect.origin);

  if (url.pathname !== redirect.pathname) {
    response.writeHead(404).end('Not found');
    return;
  }

  const state = url.searchParams.get('state');
  const entry = pending.get(state);

  if (!entry) {
    response.writeHead(400).end('State mismatch. Start the flow again.');
    return;
  }

  pending.delete(state);

  const error = url.searchParams.get('error');
  if (error) {
    response.writeHead(400).end(`Authorization denied: ${error}`);
    entry.reject(new Error(`[${entry.provider}] authorization denied: ${error}`));
    return;
  }

  entry.finish(url.searchParams.get('code'), response);
}

The pending map is the state check. A state value gets into that map only when this process generated it, and it's deleted the moment it's used. An unrecognized state means the callback didn't come from a flow you started, and a state that arrives twice means a replay. Both fall out of one Map lookup.

Generating the state and waiting for its callback:

function connect(providerName) {
  const provider = providers[providerName];
  const state = randomBytes(16).toString('hex');

  console.log(`\n[${providerName}] authorize as "${IDENTIFIER}":`);
  console.log(provider.buildAuthorizeUrl(state));

  return new Promise((resolve, reject) => {
    pending.set(state, {
      provider: providerName,
      reject,
      async finish(code, response) {
        const tokens = await provider.exchangeCode(code);
        saveGrant(IDENTIFIER, providerName, tokens);
        response
          .writeHead(200, { 'Content-Type': 'text/html' })
          .end(`<p>${provider.label} connected. You can close this tab.</p>`);
        resolve();
      },
    });
  });
}

randomBytes(16) and not Math.random(). A predictable state parameter is the same as no state parameter.

Running it walks each unconnected provider in turn:

[slack] authorize as "alice@example.com":
https://slack.com/oauth/v2/authorize?client_id=123.456&user_scope=channels%3Ahistory%2Cchat%3Awrite%2Cusers%3Aread&redirect_uri=https%3A%2F%2Flocalhost%3A3000%2Fcallback&state=1159699dbf1a808fd33ba31c7b643505

Notice what that URL doesn't contain: any scope parameter. Slack has no instruction to mint a bot token, so it won't.

How to Store Tokens Encrypted, Keyed by User

The store answers one question: which token belongs to this user, for this provider? Everything else about it follows from keeping that answer safe.

node:sqlite has shipped with Node since v22.5, and stopped requiring a flag in v22.13. That makes a real database available with nothing to install:

import { DatabaseSync } from 'node:sqlite';
import { createCipheriv, createDecipheriv, randomBytes } from 'node:crypto';

const KEY = Buffer.from(process.env.TOKEN_ENCRYPTION_KEY ?? '', 'base64');

if (KEY.length !== 32) {
  throw new Error(
    'TOKEN_ENCRYPTION_KEY must be 32 bytes, base64-encoded. ' +
      `Got ${KEY.length} bytes.`
  );
}

const db = new DatabaseSync(
  process.env.TOKEN_DB_PATH ?? new URL('../tokens.db', import.meta.url).pathname
);

// One row per user, per provider. expires_at stays outside the ciphertext so
// a token's freshness can be checked without decrypting it.
db.exec(`
  CREATE TABLE IF NOT EXISTS grants (
    identifier TEXT    NOT NULL,
    provider   TEXT    NOT NULL,
    ciphertext BLOB    NOT NULL,
    iv         BLOB    NOT NULL,
    auth_tag   BLOB    NOT NULL,
    expires_at INTEGER,
    PRIMARY KEY (identifier, provider)
  )
`);

The composite primary key is the isolation guarantee, written down. (identifier, provider) means Alice's Slack row and Bob's Slack row can't collide, and no query that supplies both parts can return somebody else's grant.

expires_at sits outside the ciphertext deliberately. Checking whether a token needs refreshing is something you do before every call. Decrypting to find out would mean decrypting constantly, so the one field that isn't a secret stays readable.

Encryption is AES-256-GCM, which authenticates as well as encrypts:

function encrypt(payload) {
  const iv = randomBytes(12);
  const cipher = createCipheriv('aes-256-gcm', KEY, iv);
  const ciphertext = Buffer.concat([
    cipher.update(JSON.stringify(payload), 'utf8'),
    cipher.final(),
  ]);
  return { ciphertext, iv, authTag: cipher.getAuthTag() };
}

function decrypt({ ciphertext, iv, authTag }) {
  const decipher = createDecipheriv('aes-256-gcm', KEY, iv);
  decipher.setAuthTag(authTag);
  const plaintext = Buffer.concat([
    decipher.update(ciphertext),
    decipher.final(),
  ]);
  return JSON.parse(plaintext.toString('utf8'));
}

Three rules govern that pair, and breaking any one of them is worse than not encrypting at all, because it looks like it worked:

1. A fresh IV per encryption: Reusing an initialization vector with GCM is a catastrophic failure, not a minor one. randomBytes(12) on every call, stored beside the ciphertext.

2. Keep the auth tag: GCM produces a tag that proves the ciphertext wasn't altered. Without setAuthTag on the way back, you have encryption without integrity, and decipher.final() won't complain.

3. Encrypt the whole token object, not each field. One ciphertext for { accessToken, refreshToken, scope } means one IV and one tag to manage rather than three of each.

Writing and reading are then unremarkable:

export function saveGrant(identifier, provider, tokens) {
  const { ciphertext, iv, authTag } = encrypt(tokens);
  db.prepare(
    `INSERT INTO grants (identifier, provider, ciphertext, iv, auth_tag, expires_at)
     VALUES (?, ?, ?, ?, ?, ?)
     ON CONFLICT (identifier, provider) DO UPDATE SET
       ciphertext = excluded.ciphertext,
       iv         = excluded.iv,
       auth_tag   = excluded.auth_tag,
       expires_at = excluded.expires_at`
  ).run(identifier, provider, ciphertext, iv, authTag, tokens.expiresAt ?? null);
}

The ON CONFLICT clause matters more than it looks. Re-consenting has to replace a grant rather than fail or duplicate it, and re-consenting is exactly what a user does after a revocation or a scope change.

The encryption key itself lives in .env here, which is right for a tutorial and wrong for production, where it belongs in a secrets manager or a KMS. Losing it makes every stored grant unreadable and forces every user to consent again. That is a real outage, but it's a better one than the alternative: a stolen database file that hands over working tokens for every user of your agent.

How to Run Tool Calls as the Current User

Runtime has three moves: resolve the identifier, fetch a token with it, and wrap the whole thing as a tool.

Step 1: Resolve the Identifier, Then Authorize

An identifier is any stable string that represents one user, an email address, a user ID, a tenant-scoped key.

// In a real app this comes from your authenticated session, resolved
// server-side. Never accept it from client input.
const IDENTIFIER = process.argv[2] ?? 'channel-watcher-agent';

Reading the identifier from argv keeps the demo runnable without a login, and it makes the isolation test later in this tutorial a single command. A real application replaces the line:

// Real app: resolve from your authenticated session, server-side.
const session = await getSession(request);                  // your auth
const identifier = await lookupIdentifier(session.userId);  // your database

Order matters in those two lines. Authenticate the caller first, then look up which identifier the caller may act as. An identifier arriving from a client turns the endpoint into a reader of any user's Slack.

Step 2: Turn the Identifier into a Token, Late

One function stands between the identifier and every provider call:

const REFRESH_WINDOW_MS = 60_000;

export async function getAccessToken(identifier, providerName) {
  const grant = readGrant(identifier, providerName);

  if (!grant) {
    throw new Error(
      `[${providerName}] no grant for "${identifier}".\n` +
        `Connect it first: node src/connect.js ${identifier}`
    );
  }

  const expiringSoon =
    grant.expiresAt !== null &&
    grant.expiresAt !== undefined &&
    grant.expiresAt - Date.now() < REFRESH_WINDOW_MS;

  if (!expiringSoon) {
    return grant.accessToken;
  }

  if (!grant.refreshToken) {
    throw new Error(
      `[${providerName}] token for "${identifier}" expired and no refresh ` +
        'token is stored. The user has to consent again.'
    );
  }

  const refreshed = await providers[providerName].refresh(grant.refreshToken);
  saveGrant(identifier, providerName, refreshed);
  return refreshed.accessToken;
}

Call this immediately before the API call, not once at startup. A long agent run can outlive a twelve-hour token, and resolving tokens up front means discovering that at the least convenient moment. Fetching late costs one cheap database read and removes the whole class of problem.

Also, the sixty-second window isn't padding for its own sake. A token with four seconds left passes a naive expiry check and then expires in flight. Refreshing anything inside the window means the token handed back is good for at least a minute of work.

Finally, a missing grant raises an error rather than falling back. There's nothing sensible to fall back to. The correct outcome for an unconnected user is a stop, with a message saying how to connect.

Step 3: Wrap Provider Calls as Tools

Identity gets injected here, one layer below anything the model can influence:

export function buildTools(identifier) {
  const [owner, repo] = process.env.GITHUB_REPO.split('/');

  const fileGithubIssue = tool({
    description: 'File a GitHub issue for an actionable Slack message',
    inputSchema: z.object({
      title: z.string(),
      body: z.string(),
    }),
    execute: async ({ title, body }) => {
      const token = await getAccessToken(identifier, 'github');
      return createIssue(token, owner, repo, { title, body });
    },
  });

  const replyInSlackThread = tool({
    description:
      'Reply in the original Slack thread (e.g. with the created issue link)',
    inputSchema: z.object({
      text: z.string(),
      thread_ts: z.string(),
    }),
    execute: async ({ text, thread_ts }) => {
      const token = await getAccessToken(identifier, 'slack');
      return postThreadReply(
        token,
        process.env.SLACK_CHANNEL_ID,
        text,
        thread_ts
      );
    },
  });

  return { fileGithubIssue, replyInSlackThread };
}

Compare what the model controls against what it can't. The model chooses title, body, and text. The model can't choose the user. identifier is a closure argument, fixed before the model ran, and it appears in no inputSchema. There's no input that makes the model file an issue as somebody else, because the account isn't one of its inputs.

Return values deserve one audit each. createIssue returns the issue number, URL, and title. postThreadReply returns a timestamp. Neither returns a token, and neither returns the raw provider response, which is where a token would hide if one were going to.

The provider calls themselves are ordinary HTTP:

export async function createIssue(token, owner, repo, { title, body }) {
  const response = await fetch(
    `https://api.github.com/repos/${owner}/${repo}/issues`,
    {
      method: 'POST',
      headers: {
        Authorization: `Bearer ${token}`,
        Accept: 'application/vnd.github+json',
        'X-GitHub-Api-Version': '2022-11-28',
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({ title, body }),
    }
  );

  const json = await response.json();

  if (!response.ok) {
    // 403 here usually means the grant is missing the `repo` scope.
    throw new Error(
      `GitHub issue creation failed (${response.status}): ${json.message}`
    );
  }

  return { number: json.number, url: json.html_url, title: json.title };
}

Step 4: Read the Channel

Slack's conversations.history returns clean JSON, with one gap: messages carry a user ID, never a display name. Turning those into names means a users.info call each, which is what users:read was in the scope list for.

export async function readChannel(token, channelId, limit = 20) {
  const { messages } = await slackCall(token, 'conversations.history', {
    channel: channelId,
    limit: String(limit),
  });

  const authors = await resolveAuthors(
    token,
    messages.filter((m) => m.user).map((m) => m.user)
  );

  return messages
    .filter((message) => message.text)
    .map((message) => ({
      author: authors.get(message.user) ?? 'unknown',
      userId: message.user,
      text: message.text,
      ts: message.ts,
    }))
    .reverse(); // oldest first
}

Three small decisions in that function:

1. Names cost one users.info call per unique author. Caching them per run keeps a channel full of one person's messages from producing twenty identical lookups. A lookup that fails falls back to the user ID rather than throwing, since an unresolvable name isn't a reason to abandon the run.

2. Messages without text get dropped. Channel joins and purpose changes arrive as message objects with no body, and there's nothing for the model to triage in them.

3. .reverse() isn't cosmetic. Slack returns newest first. A model reading a conversation backwards will misread which message answered which.

The ts field then does double duty. It identifies a message, which makes it both the thread anchor for replies and the key for remembering what the agent already handled:

const state = await loadState();
const processed = new Set(state[IDENTIFIER]?.processedTs ?? []);
const newMessages = messages.filter((m) => !processed.has(m.ts));

Key that state by identifier, as the snippet does. A single flat list lets one user's processed messages hide another's, which reintroduces cross-user bleed in the one place the whole design exists to prevent.

Step 5: Run the Tool Loop

Hand the model both tools and let it decide:

const { text } = await generateText({
  model: anthropic(process.env.MODEL),
  tools,
  stopWhen: stepCountIs(5),
  prompt: `You triage messages from a dev team's Slack channel.

Message from ${message.author}: "${message.text}"
Message timestamp (thread_ts): ${message.ts}

Decide if this message is actionable (a bug report or concrete action item) or just noise (chit-chat, join notices, already-resolved chatter).

If actionable: file a GitHub issue with a clear title and body drafted from the message, then reply in the original Slack thread (use the exact thread_ts above) with a short note and the created issue's URL.

If not actionable: do nothing and briefly say why.`,
});

The loop is what makes the second step possible. The model reads the message and may call fileGithubIssue. The AI SDK runs the tool, feeds the result back into context along with the new issue URL, and calls the model again. Now the model can reply in the thread with a URL it couldn't have known on the first pass. Then it stops.

stopWhen: stepCountIs(5) caps the rounds. Without a bound, a confused model can retry a failing tool indefinitely. Five rounds is generous for two tools.

A deterministic version is also reasonable: classify with a structured-output call, then call both tools yourself in a fixed order when the message qualifies.

The fixed sequence is easier to test and gives up real flexibility. A loop lets the model skip the reply, or file without replying, and adding a third tool needs no new branching. Choose the loop when the set of actions varies per input, and the fixed sequence when it never does.

One note on the provider line, for accuracy about what ran. The snippet above uses @ai-sdk/anthropic, which suits a direct Anthropic API key. My own tests went through an OpenAI-compatible gateway, which changes only the provider construction:

import { createOpenAICompatible } from '@ai-sdk/openai-compatible';

const gateway = createOpenAICompatible({
  name: 'gateway',
  baseURL: `${process.env.GATEWAY_BASE_URL}/v1`,
  apiKey: process.env.GATEWAY_API_KEY,
});
// then: model: gateway(process.env.MODEL)

The tools, the loop, and the token handling are identical either way. Only the model argument changes.

How to Handle Refresh and Revocation

Tokens end in two different ways, and only one of them is your code's problem**.** Expiry is routine and recoverable. Revocation is a decision somebody made, and the correct response is to ask for consent again.

The two providers in this tutorial sit at opposite ends of the range, which makes them a useful pair.

GitHub: Tokens That Don't Expire, Until They Do

An OAuth App user token has no expiry timestamp. There's no refresh token to store and no refresh call to make, which is why github.refresh() in this project does nothing but explain itself:

async refresh() {
  throw new Error(
    'GitHub OAuth App tokens do not expire. A failure here means the ' +
      'grant was revoked — send the user through consent again.'
  );
}

"Does not expire" is not the same as "lasts forever," and GitHub revokes tokens for several reasons worth knowing:

• The user revokes the authorization from their account settings.

• The token goes unused for one year.

• The token gets pushed to a public repository or gist, at which point GitHub revokes it automatically.

• The app accumulates more than ten tokens for the same user and scope combination, and the oldest are revoked.

The third one deserves a moment. GitHub scans public pushes for its own token formats and kills what it finds. That is a safety net, not a strategy, and the one thing it can't protect is a token in a private repository or a log file.

GitHub Apps behave differently from OAuth Apps, which is a common source of confusion when reading GitHub's documentation. A GitHub App's user access token expires in eight hours and comes with a refresh token good for six months. If you build on GitHub Apps instead, the Slack-shaped refresh path below is the one you want.

Slack: Rotation is Opt-in and Permanent

By default, a Slack user token doesn't expire either. Token rotation changes that, and it comes with a warning worth repeating: rotation can't be turned off once it's turned on. Enable it on a test app first.

With rotation on, tokens live twelve hours and arrive with a refresh token. The refresh call reuses the same endpoint as the initial exchange, with a different grant type:

async refresh(refreshToken) {
  const response = await fetch(this.tokenUrl, {
    method: 'POST',
    headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
    body: new URLSearchParams({
      grant_type: 'refresh_token',
      refresh_token: refreshToken,
      client_id: process.env.SLACK_CLIENT_ID,
      client_secret: process.env.SLACK_CLIENT_SECRET,
    }),
  });

  const json = await response.json();
  if (!json.ok) {
    throw new Error(`Slack token refresh failed: ${json.error}`);
  }

  return normalizeSlackTokens(json.authed_user ?? json);
}

Store the new refresh token, not just the new access token. Refresh tokens rotate too. Writing back only the access token leaves you holding a spent refresh token, and the failure arrives twelve hours later, which is a long time to wait to learn something.

That write-back is why getAccessToken calls saveGrant after refreshing rather than returning the token and moving on.

Treat a Dead Grant as a Normal State

A revoked grant isn't an exception in the exceptional sense. Users leave, administrators tighten scopes, and people change their minds about what an agent may do.

The shape that works is the one getAccessToken already uses: catch the failure, and surface a fresh authorization link rather than a stack trace. connect.js with the same identifier lets the user re-consent, ON CONFLICT overwrites the dead row, and nothing else in your user record changes.

How to Add a Second Provider

A second provider costs one OAuth app, one entry in the providers object, and one tool. Keeping identity in a single string is what buys the discount.

The agent has used two providers all along. Worth noticing is what the second one didn't require: no second identity, no second consent server, and no second token table.

export const providers = {
  slack: { /* ... */ },
  github: { /* ... */ },
};

Google Calendar as a third means a third entry with its own authorizeUrl, tokenUrl, scopes, and exchangeCode. The consent server loops over Object.keys(providers), so it picks the new one up without modification. The store already keys on (identifier, provider), so it needs no migration. Then one more tool:

const createCalendarEvent = tool({
  description: 'Create a calendar event',
  inputSchema: z.object({ summary: z.string(), start: z.string() }),
  execute: async ({ summary, start }) => {
    const token = await getAccessToken(identifier, 'google-calendar');
    // ...one more provider call
  },
});

The identifier doesn't change, your user table doesn't change, and the model's view of the world grows by exactly one tool.

The cost that doesn't scale down is the provider-specific knowledge. Each new provider brings its own scope vocabulary, its own error format, and its own answer to whether tokens expire. Slack and GitHub disagreed on all three, and a third will disagree differently. The registry pattern contains that knowledge in one object per provider rather than spreading it through your agent, but it doesn't make the knowledge unnecessary.

One caveat on consent: A grant is per user, per provider. Alice connecting Slack but not Calendar means her calendar tool calls fail, and failure is correct there, since she never consented. Treat it as a prompt to connect rather than an error, a point the Failure Modes section returns to.

Full Walkthrough

Clone the repository, install, and fill in .env:

git clone https://github.com/saif-shines/channel-watcher-agent.git
cd channel-watcher-agent
npm install
cp .env.example .env
# fill in both client IDs and secrets, the encryption key, channel ID, repo

Then connect. The command starts the callback server and prints one link per unconnected provider:

npm run connect

[slack] authorize as "channel-watcher-agent":
https://slack.com/oauth/v2/authorize?client_id=123.456&user_scope=channels%3Ahistory%2Cchat%3Awrite%2Cusers%3Aread&redirect_uri=https%3A%2F%2Flocalhost%3A3000%2Fcallback&state=1159699dbf1a808fd33ba31c7b643505
[slack] connected.

[github] authorize as "channel-watcher-agent":
https://github.com/login/oauth/authorize?client_id=Iv1.abc&scope=repo&redirect_uri=https%3A%2F%2Flocalhost%3A3000%2Fcallback&state=e6d13461099c391367266235f8313630
[github] connected.

All providers connected. Run: node src/index.js channel-watcher-agent

Open each link, consent, and the tab confirms. The state parameter in those URLs is checked on the way back. A callback carrying anything else gets a 400 and never reaches the token exchange.

Then run the agent against a channel holding ordinary chatter:

node src/index.js

The output from that run, against a channel with three unremarkable messages:

[channel-watcher-agent] 3 messages fetched, 3 new.

--- Alex: "Sending draft message" ---
The message "Sending draft message" is noise — it appears to be a test or
accidental send, not a bug report or concrete action item.

...

No tools called, and no issues filed. The negative case matters more than it looks. An agent with write access that can't say no is a liability, and a run over ordinary chatter is the cheapest available test of its restraint.

Now post an actual bug report in the channel:

hey the /export endpoint is timing out for any file over 50MB, been happening since yesterday's deploy

Run the agent again, and the state file keeps the earlier messages from being triaged twice.

Authorship is the part that matters. The GitHub account behind the identifier opens the issue, using that user's own OAuth grant, not a shared bot. The Slack reply comes from that person too. Revoke their access and the next run fails at getAccessToken, which is the correct outcome.

What Changes for a Second User

The identifier comes from the command line, so isolation is testable without building a login first:

node src/index.js                     # the identifier you already authorized
node src/index.js alice@example.com   # a different user entirely

The second command never reads the channel. It stops:

[slack] no grant for "alice@example.com".
Connect it first: node src/connect.js alice@example.com

The refusal is the whole point. Nothing about the agent changed between the two commands. Same providers, tools, and code. Only the identifier differed, and Alice hasn't consented, so no row exists to decrypt and the run stops before touching Slack.

A shared-bot version behaves differently. The second command would read the channel and file an issue as the bot, because no per-user grant was ever involved.

Once Alice consents, everything downstream follows her grant. readGrant returns her row. getAccessToken decrypts her token. The Slack read returns the channels she can see, and her GitHub account authors the issue.

Production replaces argv with a session lookup:

const identifier = await lookupIdentifier(session.userId);

Testing the Isolation Without Credentials

The repository includes a test suite that replaces fetch with stand-in Slack and GitHub endpoints, so the request building, response parsing, storage, and refresh logic all run without a single OAuth app registered:

npm test

Three of those tests are worth naming, because they check the claims this tutorial makes rather than the code's internals:

• Slack exchange keeps the user token and discards the bot token. The fixture returns both. The test asserts the stored value is the xoxp- one.

• Two users get two different tokens from identical tool inputs. Same text, same thread_ts, two identifiers, two different Authorization headers reaching the provider.

• A tool built for an unconnected user fails instead of falling back. It also asserts that zero provider calls were attempted, since failing after leaking a request isn't much of a failure.

Tests that pass on the first run are worth distrusting, so I checked these by breaking the code on purpose. Substituting the bot token for the user token, ignoring the identifier in buildTools, and exposing identifier in the model-visible schema each fail at least one test.

How to Apply the Pattern to Other Use Cases

Nothing in the pattern is specific to Slack triage. The shape is: read from one app, decide with a model, write to another app, all as one user.

Swapping the providers produces a different product:

Every row uses the same three pieces: a provider entry, a token lookup by identifier, and a tool. Only three things change: the OAuth app registrations, the API calls inside execute, and the input you write for the model.

The input is where your product lives. OAuth is plumbing. Deciding which messages deserve an issue, and what the issue should say, is judgment, and judgment is the part worth your weeks.

The same code supports two deployment shapes:

• Internal team agent: The identifier is the teammate who triggered the run. Runs on a schedule or a command.

• Customer-facing agent: The identifier comes from your tenant and user records. Runs on customer data, inside customer accounts.

The code stays identical. The consequences of a wrong identifier do not.

What Went Wrong When I Built This

These are problems I hit while building the project, in roughly the order they showed up. If you hit the same ones, the fix is usually small.

Slack Won't Save the Redirect URL

The symptom arrives before any code runs: the Slack app configuration page refuses to accept http://localhost:3000/callback.

Slack requires HTTPS on redirect URLs with no exception for localhost. Issue a local certificate with mkcert, register the https:// form, and point TLS_CERT_PATH and TLS_KEY_PATH at the files it wrote. GitHub accepts either scheme, so the same HTTPS URL works for both apps.

The Browser Warns That the Certificate Isn't Trusted

mkcert -install is the step that adds mkcert's local authority to your system trust store, and skipping it leaves a certificate no browser recognises.

Running it once fixes every certificate mkcert issues afterwards. A self-signed certificate made with openssl will always warn, since nothing trusts it.

The Redirect URI Doesn't Match

Both providers compare the redirect_uri you send against the one registered with the app, and the comparison is exact. A trailing slash, 127.0.0.1 in place of localhost, http where you registered https, or a different port all fail.

The error arrives before consent, on the provider's own page, which at least makes it easy to spot. Keep OAUTH_REDIRECT_URI as the single source and pass it in both the authorize URL and the token exchange, as the provider registry does.

The Callback Port is Already in Use

connect.js binds the port from OAUTH_REDIRECT_URI, and port 3000 is popular. An unhandled EADDRINUSE produces a stack trace that says nothing about OAuth, so the project catches it and says what to do instead.

Changing the port means changing it in three places: .env, the Slack app's redirect URLs, and the GitHub app's callback URL. Missing one produces the previous failure.

The State Check Rejects a Legitimate Callback

State values live in memory and are deleted once used. Restarting connect.js after opening the link, or refreshing the callback tab, both produce a state that's no longer in the map.

Both are correct rejections. Generate a fresh link and start again.

Tool Calls Return Permission Errors or Empty Results

A missing scope or a revoked grant causes both.

GitHub answers 403 with "Resource not accessible" when the grant lacks repo. Slack answers 200 with ok: false and an error like missing_scope. Fix the scope list, then send the user through consent again, since an existing grant doesn't gain scopes retroactively.

A partially-scoped grant fails at the point of use rather than at connection time, which is what makes the symptom look mysterious. The consent screen succeeded, the token stored fine, and the failure arrives during a tool call hours later.

The Agent Reads Channels it Shouldn't

The single most likely cause is storing json.access_token instead of json.authed_user.access_token during the Slack exchange. Both are strings, both are truthy, and both work (one works as the app rather than the person).

The tell is the scope of what comes back. A user token returns only that person's channels. If conversations.history returns a channel the current user was never in, a bot token is in the store.

A Tool Call Runs as the Wrong User

Passing a token or identifier belonging to somebody else will do the wrong thing correctly.

Two habits prevent it. Resolve the identifier server-side after authenticating the caller, never from client input. Then take the identifier as a closure argument in buildTools and let each execute fetch its own token, so no code path can pass a stray credential.

Refresh Works Once and Then Stops

Refresh tokens rotate. A refresh that writes back the new access token but keeps the old refresh token succeeds immediately and fails on the following cycle, which puts twelve hours between the bug and its symptom.

saveGrant takes the whole normalized token object for this reason. Write back everything the refresh returned.

The Agent Files Duplicate Issues

Two causes. A missing or unwritten state file makes every run triage everything again. Or stopWhen allows enough rounds for a confused model to retry a tool that already succeeded.

Check the state file first. Then check whether the tool's return value clearly signals success, because an ambiguous result invites a retry.

Conclusion

You've built an agent that reads a Slack channel, judges which messages describe real work, files GitHub issues for those, and closes the loop with a threaded reply. Every call ran as one specific user's own OAuth grant, through an OAuth flow and a token store you wrote yourself.

Five ideas carry over to any provider:

• A tool is an API call plus an explanation for a model, and the explanation is most of the work.

• The identifier replaces the token in your agent code. Everything above one small function handles a reference to a user rather than a credential, so tokens never reach your model inputs or your logs.

• Connection time and runtime are separate flows. Consent happens once per user, per app. Runtime resolves an identifier and fetches a token late.

• Authorization stays yours. A token store answers which tokens belong to an identifier. Whether a caller may act as that identifier is a question only your code can answer.

• Multi-provider support is a registry problem, not an architecture problem, once identity lives in one string.

The detail that carries the most weight is also the smallest: authed_user.access_token rather than access_token. One property access decides whether your agent respects the permissions your workspace already has or quietly routes around them.

From here, keep the shape and swap the providers. Point the read half at Gmail and the write half at Linear, then rewrite the input for the model. The identity plumbing doesn't change.

The full source is at github.com/saif-shines/channel-watcher-agent.

This write-up reconstructs what we learned building Scalekit, a hosted version of the token vault you just built.

The order service handles the checkout, saves the order to its database, and then publishes an order.created event to a message queue so every downstream system can react independently.

This is a common and reasonable design, but it has a reliability problem that's easy to miss until something goes wrong in production.

When a customer places an order and the payment goes through, the application needs to do two things: save the order to the database and publish the event to the queue. These are two separate writes to two separate systems, and there's no way to make them share a single atomic transaction. If the process crashes, the network hiccups, or a deployment rolls out between the two writes, one side commits and the other does not. The order sits confirmed on the customer's screen while the warehouse has no idea it exists.

The transactional outbox pattern is the standard solution to this problem. In this article, we'll build it from scratch in Node.js, using PostgreSQL for the order service database, SQS for the queue, and DynamoDB as the fulfillment service's database. For local development, we'll use floci, a free open-source AWS emulator that runs all three with a single Docker container.

What We'll Cover

• Prerequisites

• The Problem with Two Writes

• The Outbox Pattern

• What We'll Build

• Project Setup

• Database Schema

• The Request Handler

• The Relay Worker

• The Consumer

• Running the Whole Thing

• Going to Production

• Conclusion

Prerequisites

To follow along, you should be comfortable with:

• Node.js and async/await

• Database transactions (BEGIN, COMMIT, ROLLBACK)

• The general concept of a message queue

You don't need prior experience with AWS, SQS, or DynamoDB. We'll be running everything locally.

You will need Node.js 20 or later and Docker installed on your machine.

The Problem with Two Writes

The order service scenario from the intro is one place this problem appears, but the same pattern comes up in many other contexts.

A user registers and the app inserts their account record, then sends a message to trigger the welcome email and the onboarding workflow. A file is uploaded and the API writes the metadata to the database, then publishes a message to kick off a processing worker for virus scanning or thumbnail generation. A payment webhook arrives, the handler records it in the database, then notifies downstream services that the payment is confirmed.

In every case, the application needs two writes to succeed together: one to the database and one to a queue or external system. If the second one is lost, the first one has no way of knowing.

If you want a deeper look at what database transactions actually guarantee and where they stop helping, see Beyond Happy Path Engineering: Databases.

The naïve implementation looks straightforward:

await db.query('INSERT INTO orders (customer_id, amount_cents) VALUES ($1, $2)', [customerId, amountCents]);
await sqs.send(new SendMessageCommand({ QueueUrl: QUEUE_URL, MessageBody: JSON.stringify({ customerId, amountCents }) }));

The database write happens first, then the queue write. Under normal conditions this works fine. The problem is what happens when something goes wrong between the two.

If the process crashes, runs out of memory, or gets killed mid-deployment after the database write but before sqs.send is called, the order record exists in the database but no event is ever published. The warehouse, email service, and fraud checker never find out the order happened. From the customer's perspective the order went through. From every downstream system's perspective it doesn't exist.

The failure can also go the other way. If sqs.send succeeds but the database write is later rolled back due to a constraint violation or an error in a subsequent step, you've published an event for an order that doesn't actually exist. A consumer acting on that event may try to fulfill an order with no corresponding record, or charge a customer for something that was never saved.

There's also a timing window even when both writes eventually succeed. Between the database commit and the successful sqs.send, a consumer that queries the database after receiving the event may not find the order yet, depending on transaction isolation and replication lag. These are two separate systems with no shared transaction boundary, and no amount of careful sequencing fully closes the gap.

These aren't edge cases that only happen under extraordinary circumstances. Deploys restart processes mid-request. Out-of-memory kills happen without warning. Networks drop connections at any point. Any of these can interrupt the two-write sequence, and the result is a system that's silently inconsistent with no error logged and no alert fired.

A variation I've seen a few times that looks safer but is actually worse is wrapping both operations in a database transaction:

// PLEASE DO NOT EVER DO THIS
const client = await pool.connect();
await client.query('BEGIN');
await client.query('INSERT INTO orders (customer_id, amount_cents) VALUES ($1, $2)', [customerId, amountCents]);
await sqs.send(new SendMessageCommand({ QueueUrl: QUEUE_URL, MessageBody: JSON.stringify({ customerId, amountCents }) }));
await client.query('COMMIT');

The intent is to make the two writes feel like a unit, but a database transaction has no authority over SQS. The transaction can only roll back database operations. If sqs.send succeeds and then COMMIT fails, the message is already in the queue and can't be taken back. If the process crashes after COMMIT but before the function returns, the transaction committed and the message was sent, but the caller may retry, potentially inserting a duplicate order.

Beyond the correctness problems, this pattern holds an open database connection and any row locks for the entire duration of the SQS network call. SQS is normally fast, but under load, retries, or a degraded queue, that call can take seconds. Every other request trying to read or write the same rows has to wait. In a busy application, this is a reliable way to exhaust the connection pool and bring down unrelated parts of the service.

The Outbox Pattern

The core idea is to stop treating the queue publish as a second write that happens after the database write, and instead make it part of the same database transaction.

Rather than calling sqs.send directly, the application inserts a row into an outbox table in the same transaction as the business record. A separate relay process reads the outbox table and publishes the messages to SQS. On the other end, a consumer receives the messages and writes to its own data store. In our case that is a fulfillment service writing to DynamoDB, completely separate from the order service's PostgreSQL database.

If the transaction rolls back for any reason, the outbox row disappears with it. There's no orphaned message in the queue because the message was never sent. If the application crashes after committing but before the relay runs, the outbox row is still there with status='pending', and the relay will pick it up on its next iteration.

The only guarantee the pattern relies on is the one the database already provides: atomicity within a single transaction.

The relay worker is responsible for the eventual delivery guarantee. It runs on an interval, selects pending rows, publishes them to SQS, and marks them as sent only after SQS confirms receipt. If the relay crashes mid-run, it will reprocess the same rows on the next iteration, which means SQS may receive some messages more than once.

That's why the consumer needs to be idempotent: it must handle receiving the same message twice without creating duplicate fulfillment records. We'll cover how to implement that when we build the consumer.

This separation of concerns is what makes the pattern practical. The request handler commits one atomic database transaction and returns. The relay handles the network call to SQS asynchronously, at its own pace, with its own retry logic, without holding database connections open or blocking request handling. The consumer is fully decoupled from the order service and owns its own data store.

The diagram below illustrates the flow:

What We'll Build

Now let's see the whole thing in practice. We'll implement a simple order placement API. When a customer sends a request to place an order, the order service saves it to PostgreSQL and inserts a row into the outbox table, all in one atomic transaction. A relay worker wakes up periodically, reads the pending outbox rows, and publishes each one as a message to SQS. A separate fulfillment service receives those messages from the queue and creates fulfillment records in DynamoDB.

By the end, you'll have an HTTP endpoint you can call, and you'll be able to verify that placing an order triggers the creation of a fulfillment record in a completely separate database, owned by a completely separate service, without either service ever talking to the other directly.

You can find the complete working code at github.com/gkoos/article-outbox.

Project Setup

Before you can run any code, you need to get floci running so you have local instances of PostgreSQL, SQS, and DynamoDB. You'll also need Node.js 20 or later and Docker installed.

Start by cloning the repository and installing dependencies:

git clone https://github.com/gkoos/article-outbox
cd article-outbox
npm install

Next, start floci. This command pulls the latest floci image and starts a Docker container that exposes a local AWS API endpoint (make sure Docker is running):

npm run floci:start

On Linux and macOS, this just works. On Windows with Docker Desktop, you need to edit the floci:start script in your package.json to change the Docker socket mount from /var/run/docker.sock to //var/run/docker.sock.

The floci container is now listening on port 4566 and can spin up RDS (PostgreSQL), SQS, and DynamoDB instances on demand.

Now provision the AWS resources with a single setup command:

npm run setup

This script creates an RDS PostgreSQL database instance, an SQS queue named orders, and a DynamoDB table named fulfillments. It waits for RDS to become available and then writes a .env file with the correct connection details. The environment variables PG_PORT, SQS_QUEUE_URL, and DYNAMODB_TABLE_NAME now point to the local emulated services.

Finally, create the PostgreSQL tables:

npm run migrate

This creates the orders table and the outbox table in PostgreSQL. You now have a fully functional local environment ready to build against.

Database Schema

The two tables are simple. orders holds the business records: each order has a customer ID, an amount in cents, and a timestamp. The outbox table is the heart of the pattern: it's where the application writes the event that needs to be published.

CREATE TABLE orders (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  customer_id TEXT NOT NULL,
  amount_cents INTEGER NOT NULL,
  created_at TIMESTAMPTZ DEFAULT now()
);

CREATE TABLE outbox (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  event_type TEXT NOT NULL,
  payload JSONB NOT NULL,
  status TEXT NOT NULL DEFAULT 'pending',
  created_at TIMESTAMPTZ DEFAULT now(),
  sent_at TIMESTAMPTZ
);

CREATE INDEX ON outbox (status, created_at) WHERE status = 'pending';

The orders table needs nothing special. The outbox table stores the event metadata: what type of event it is (event_type), what data it contains (payload as JSON), and whether it has been sent yet (status).

The status starts as pending. When the relay publishes it to SQS, it will mark it as sent and record the timestamp. The index on (status, created_at) WHERE status = 'pending' lets the relay quickly find the next batch of unsent events without scanning the entire table.

The Request Handler

This is where the pattern starts. The request handler receives an HTTP POST, inserts an order into the database, inserts a corresponding row into the outbox table, and commits everything in a single atomic transaction. The key insight is that neither write succeeds unless both succeed.

const client = await pool.connect();
try {
  await client.query('BEGIN');

  // Insert the order record
  const { rows } = await client.query(
    'INSERT INTO orders (customer_id, amount_cents) VALUES ($1, $2) RETURNING *',
    [customerId, amountCents]
  );
  const order = rows[0];

  // Insert the outbox record in the same transaction
  await client.query(
    `INSERT INTO outbox (event_type, payload)
     VALUES ($1, $2)`,
    ['order.created', JSON.stringify({ orderId: order.id, customerId: order.customer_id, amountCents: order.amount_cents, createdAt: order.created_at })],
  );

  await client.query('COMMIT');
  res.status(201).json(order);
} catch (err) {
  await client.query('ROLLBACK');
  next(err);
} finally {
  client.release();
}

The handler gets customerId and amountCents from the request body, starts an explicit transaction with BEGIN, and inserts the order. Then it inserts an outbox row with the order data as the payload.

Everything commits atomically. If anything fails, everything rolls back and the client gets an error. If the process crashes between the commit and the response, the client won't get a 201, but the order and the outbox row are still safely committed to the database and the relay will eventually pick it up. The handler doesn't call SQS at all. That is the relay's job.

The Relay Worker

The relay worker is a separate process that polls the outbox table every second and publishes pending rows to SQS. It runs independently of the HTTP server and has no shared state with it.

async function relay() {
  const client = await pool.connect();
  try {
    await client.query('BEGIN');

    const { rows } = await client.query(`
      SELECT *
      FROM outbox
      WHERE status = 'pending'
      ORDER BY created_at
      LIMIT 10
      FOR UPDATE SKIP LOCKED -- prevents multiple relays from processing the same rows
    `);

    for (const row of rows) {
      await sqsClient.send(new SendMessageCommand({
        QueueUrl: QUEUE_URL,
        MessageBody: JSON.stringify(row.payload),
        MessageAttributes: {
          EventType: { DataType: 'String', StringValue: row.event_type },
        },
      }));

      await client.query(
        `UPDATE outbox SET status = 'sent', sent_at = now() WHERE id = $1`,
        [row.id],
      );
    }

    await client.query('COMMIT');
  } catch (err) {
    await client.query('ROLLBACK');
    console.error('Relay error:', err.message);
  } finally {
    client.release();
  }
}

setInterval(relay, 1000);

FOR UPDATE SKIP LOCKED is the key to running multiple relay instances safely: when a relay picks up a batch of rows, it locks them. Any other relay instance trying to select the same rows will skip them and move to the next available ones, so you never get two relays publishing the same message from the same run.

The relay marks each row as sent only after sqsClient.send returns. If the relay crashes after sending to SQS but before updating the row, the row stays pending and the relay will resend it on the next iteration.

Note that the UPDATE happens inside the same transaction as the SELECT FOR UPDATE, so if the relay crashes mid-batch, the entire batch rolls back and all rows in it will be retried, including any that were already successfully sent to SQS.

The at-least-once delivery guarantee applies at the batch level, not the individual row level. You can read about this problem in Beyond Happy Path Engineering: the Network: when a response is lost, the caller can't know whether the operation succeeded, so it retries, and the receiver may see the same request twice. This means the consumer may see the same message more than once, which is why idempotency matters on the consumer side.

The Consumer

The consumer is a completely separate service. It knows nothing about the order service's PostgreSQL database. Its only input is the SQS queue, and its only output is the DynamoDB fulfillments table. This is the point of the pattern: the two services are decoupled by the queue, and each owns its own data store.

As we saw earlier, because SQS delivers at least once (meaning a message might be delivered more than once), the consumer must be idempotent. The PutItem call uses a ConditionExpression that makes the write a no-op if a fulfillment record for that order already exists, so redelivered messages are handled safely.

async function consume() {
  const { Messages } = await sqsClient.send(new ReceiveMessageCommand({
    QueueUrl:              QUEUE_URL,
    WaitTimeSeconds:       20,   // long-poll: wait up to 20s for messages
    MaxNumberOfMessages:   10,
    MessageAttributeNames: ['All'],
  }));

  for (const msg of Messages ?? []) {
    const event = JSON.parse(msg.Body);

    try {
      await dynamoClient.send(new PutItemCommand({
        TableName: 'fulfillments',
        Item: {
          orderId:     { S: event.orderId },
          customerId:  { S: event.customerId },
          amountCents: { N: String(event.amountCents) },
          status:      { S: 'received' },
          createdAt:   { S: new Date().toISOString() },
        },
        ConditionExpression: 'attribute_not_exists(orderId)', // idempotency check
      }));
    } catch (err) {
      if (err.name !== 'ConditionalCheckFailedException') throw err;
      // already processed, safe to continue
    }

    // delete the message only after the write succeeds (or was already done)
    await sqsClient.send(new DeleteMessageCommand({
      QueueUrl:      QUEUE_URL,
      ReceiptHandle: msg.ReceiptHandle,
    }));
  }
}

ConditionExpression: 'attribute_not_exists(orderId)' tells DynamoDB to reject the write if a record with that orderId already exists. When that happens, DynamoDB throws a ConditionalCheckFailedException. The consumer catches that specific error and ignores it, then deletes the message from the queue and moves on. Any other error is rethrown and the message stays in the queue to be retried.

The DeleteMessage call happens after the DynamoDB write, not before. If the process crashes between the write and the delete, SQS will redeliver the message and the condition check will handle it. If the process crashes before the write, the message stays in the queue and will be processed normally on the next delivery.

Running the Whole Thing

With floci running and the resources provisioned, open three terminal tabs and start each process:

node src/server.js    # the order API on port 3000
node src/relay.js     # the outbox relay
node src/consumer.js  # the fulfillment consumer

Now place an order:

curl -X POST localhost:3000/orders \
  -H 'Content-Type: application/json' \
  -d '{"customerId":"c1","amountCents":4999}'

You should get back a 201 with the new order record:

{"id":"1768d35b-083d-45f1-adb5-4063d8d7fcab","customer_id":"c1","amount_cents":4999,"created_at":"2026-07-30T20:27:10.628Z"}

Within a second the relay will pick up the outbox row and publish it to SQS. The consumer will receive the message and write a fulfillment record to DynamoDB. The repo includes a convenience script to verify this:

npm run check

You should see a fulfillment record with the orderId from the order you just placed:

{
  orderId: 'c335640e-bc4a-47e4-afed-484c95fbd6d3',
  customerId: 'c1',
  amountCents: '4999',
  status: 'received',
  createdAt: '2026-07-30T19:02:54.929Z'
}

Going to Production

Because the local setup uses floci to emulate AWS, switching to real AWS requires no code changes at all. The AWS SDK reads the endpoint from AWS_ENDPOINT_URL in the environment. In production, you simply don't set that variable and the SDK talks to real AWS using the credentials and region from the standard environment variables (AWS_REGION, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, or an IAM role if you are running on EC2 or ECS).

Running multiple relay instances is safe out of the box because of FOR UPDATE SKIP LOCKED. You can scale the relay horizontally and each instance will pick up a different set of rows without duplicating messages.

One thing worth adding before going to production is handling permanent failures in the relay. Right now the relay only uses pending and sent. You should add a failed status and a retry counter: after a row has failed N times, mark it failed and stop retrying it. Then configure a dead-letter queue on the orders SQS queue as well, so that messages the consumer can't process after the maximum number of retries land somewhere you can inspect rather than disappearing silently.

For high-throughput systems where polling latency matters, change data capture (CDC) is a common alternative to the polling relay. Tools like Debezium read directly from the PostgreSQL write-ahead log and publish changes to Kafka or SQS without any polling delay. The outbox table and the consumer stay exactly the same, only the relay is replaced.

This is a bigger operational commitment than a polling worker, so polling is the right starting point for most systems.

Conclusion

The dual-write problem is easy to overlook because the naïve implementation works correctly most of the time. It only fails in the gaps between two separate system writes, and those gaps only become visible when something goes wrong at exactly the wrong moment. By the time you notice it in production, data is already inconsistent and there is no clean way to recover.

The transactional outbox pattern closes that gap at the database level. The outbox row is part of the same atomic commit as the business record, so the two are always in sync. The relay handles the network call to SQS independently, with its own retry logic, without touching the request lifecycle. The consumer handles at-least-once delivery with a single condition check on the write.

Each piece is simple on its own, and together they give you reliable, decoupled event delivery without distributed transactions.

I had the first version running in a day. Single questions worked. But a week in, a tester typed: "What is the weather in Berlin, and how much would 500 EUR convert to in USD right now?"

The model called the weather function, returned that answer, and ignored the second half of the question entirely.

That is the gap between a chatbot and an agent. A chatbot works from training data. That's its limit. An agent doesn't have that limit. It calls a tool, reads what came back, and decides whether to keep going.

Most questions resolve in one or two tool calls. Multi-step ones take a few more. Remove that loop, and they all break.

This tutorial shows you how to build that loop with Google Gemini's function calling API and Node.js. You'll build an agent that can call real external tools: Open-Meteo for live weather, frankfurter.app for live currency rates, and a math evaluator for calculations. All three are completely free. The only API key you need is Gemini, which is also free on Google AI Studio at 1,500 requests per day.

Everything is on GitHub: github.com/ziaongit/nodejs-gemini-agent.

Table of Contents

• How Function Calling Works

• What We're Building

• Prerequisites

• Project Setup

• Defining the Tools

• Implementing the Tool Functions

• Building the Agentic Loop

• The CLI Entry Point

• Adding an Express HTTP Server

• Testing the Agent

• Troubleshooting

• What to Build Next

How Function Calling Works

Most LLM tutorials show function calling as: define a function, the model calls it, done. That framing skips the part that actually matters.

The model doesn't call your function. It can't. What happens is more like a negotiation.

You send the model a message along with a list of tool descriptions. Each description is a JSON schema: the function name, what it does, and what arguments it needs. Gemini reads those at request time to decide which tool, if any, fits what the user asked.

Here's the part that surprises people. Gemini doesn't run your code. It sends back a structured object that says: call get_weather, city = Berlin. Your code picks that up, runs the actual function, and sends the result back. Gemini checks whether that's enough to answer. If not, it requests another tool.

That exchange is the loop:

User message
      │
      ▼
Model + tool schemas
      │
      ▼
Response: functionCall?
      │
   YES │                          NO
      ▼                            ▼
Run the function(s)         Return text answer
      │
      ▼
Send result(s) back to model
      │
      └──── loop back ────────────┘

The loop keeps running until the model decides it has enough to answer. That's what allows it to chain calls: check the weather, see the temperature is above 25°C, then decide it should also fetch the exchange rate before answering.

There's one detail that trips people up the first time. When Gemini requests multiple tools in the same response, you run all of them and return all results in a single message. Returning them one at a time in separate messages breaks the model's turn-tracking and produces unreliable output.

What We're Building

In this tutorial, we'll build an AI agent with three working tools:

• get_weather — fetches current weather for any city via Open-Meteo (free, no API key)

• calculate — evaluates a math expression safely in JavaScript

• get_exchange_rate — fetches live currency rates via frankfurter.app (free, no API key)

There are two ways to run it: a readline CLI for quick local testing, and an Express HTTP endpoint to wire into a real application.

Full tech stack:

• Node.js 20: runtime (Node 18 minimum for native fetch)

• @google/generative-ai: official Gemini SDK

• dotenv: environment variable loading

• Express: HTTP server for the API endpoint

• Open-Meteo API: free weather and geocoding, no key required

• frankfurter.app: free currency exchange rates, no key required

Architecture:

┌─────────────────────────────────────────────────┐
│                   Client                         │
│         CLI (readline) / HTTP POST               │
└──────────────────────┬──────────────────────────┘
                       │  user message
                       ▼
┌─────────────────────────────────────────────────┐
│               agent.js — Agentic Loop            │
│                                                  │
│  1. Send message + tool schemas to Gemini        │
│  2. Receive response                             │
│  3. functionCalls() present?                     │
│      YES → execute tools in parallel             │
│            send all results back                 │
│            go to step 2                          │
│      NO  → return final text answer              │
└──────────────────────┬──────────────────────────┘
                       │  tool calls
                       ▼
┌─────────────────────────────────────────────────┐
│                  Tool Handlers                   │
│                                                  │
│  get_weather(city)                               │
│    └─► geocoding-api.open-meteo.com              │
│        api.open-meteo.com                        │
│                                                  │
│  calculate(expression)                           │
│    └─► JS safe evaluator (no external call)      │
│                                                  │
│  get_exchange_rate(from, to, amount?)            │
│    └─► api.frankfurter.app                       │
└─────────────────────────────────────────────────┘

Prerequisites

Before you start, you should have:

• Node.js 18 or higher — run node --version to check

• A Gemini API key from aistudio.google.com — free, no card. The free tier gives you 1,500 requests a day.

• You should also know how async/await works in Node.js. That's about it.

Project Setup

mkdir nodejs-gemini-agent && cd nodejs-gemini-agent
npm init -y
npm install @google/generative-ai dotenv express
mkdir src

Add a .gitignore, as you don't want .env in your repo:

node_modules/
.env

Drop a .env at the root:

GEMINI_API_KEY=your_api_key_here
PORT=3000

# Optional: override the default model (gemini-2.0-flash)
# Uncomment if you hit free-tier quota limits
# GEMINI_MODEL=gemini-2.0-flash-lite

Project structure:

nodejs-gemini-agent/
├── src/
│   ├── tools.js        ← tool schemas for Gemini
│   ├── functions.js    ← actual implementations
│   ├── agent.js        ← the agentic loop
│   ├── index.js        ← CLI entry point
│   └── server.js       ← Express HTTP server
├── .env
├── .env.example
├── .gitignore
└── package.json

Defining the Tools

Gemini can't see your code. It picks tools based entirely on the JSON schemas you pass in. Each schema has a name, a description, and the parameter definitions.

The description is what drives routing. Gemini reads it at request time to decide which tool fits the user's question. Focus on when to call the function, not just what it does.

// src/tools.js

const toolDefinitions = [
  {
    name: 'get_weather',
    description:
      'Get the current weather for a city. Returns temperature in Celsius, ' +
      'humidity percentage, and wind speed. Use this when the user asks about ' +
      'weather, temperature, or climate conditions in any location.',
    parameters: {
      type: 'OBJECT',
      properties: {
        city: {
          type: 'STRING',
          description: 'The city name, e.g. Tokyo, London, New York',
        },
      },
      required: ['city'],
    },
  },
  {
    name: 'calculate',
    description:
      'Evaluate a mathematical expression and return the numeric result. ' +
      'Use this for arithmetic, percentage calculations, or any numeric computation ' +
      'the user asks for. Do not guess at math — always call this tool.',
    parameters: {
      type: 'OBJECT',
      properties: {
        expression: {
          type: 'STRING',
          description:
            'A valid mathematical expression, e.g. "47.50 * 0.18" or "1500 / 12"',
        },
      },
      required: ['expression'],
    },
  },
  {
    name: 'get_exchange_rate',
    description:
      'Get the current exchange rate between two currencies. Can also convert ' +
      'a specific amount. Use this when the user asks about currency conversion, ' +
      'exchange rates, or how much a foreign currency amount is worth.',
    parameters: {
      type: 'OBJECT',
      properties: {
        from: {
          type: 'STRING',
          description: 'The source currency code, e.g. USD, EUR, JPY, GBP',
        },
        to: {
          type: 'STRING',
          description: 'The target currency code, e.g. USD, EUR, JPY, GBP',
        },
        amount: {
          type: 'NUMBER',
          description:
            'Amount to convert. Optional — defaults to 1 if not provided.',
        },
      },
      required: ['from', 'to'],
    },
  },
];

module.exports = { toolDefinitions };

Vague descriptions work most of the time. The problems show up at the edges. "Does currency stuff" routes fine on a simple question. It falls apart on anything ambiguous. "Get the current exchange rate between two currencies. Use this when the user asks about currency conversion." holds up. The extra words cost basically nothing. Debugging bad routing costs a lot more.

Implementing the Tool Functions

These are the actual functions that run when the model requests them. Each receives the arguments the model decided to pass, does real work, and returns a plain JavaScript object.

// src/functions.js

async function get_weather({ city }) {
  // Open-Meteo uses a two-step approach: geocode the city first, then fetch weather.
  // Both APIs are free with no key required.
  const geoUrl = `https://geocoding-api.open-meteo.com/v1/search?name=${encodeURIComponent(city)}&count=1`;
  const geoRes  = await fetch(geoUrl);
  const geoData = await geoRes.json();

  if (!geoData.results?.length) {
    return { error: `City not found: ${city}` };
  }

  const { latitude, longitude, name, country } = geoData.results[0];

  const weatherUrl =
    `https://api.open-meteo.com/v1/forecast` +
    `?latitude=${latitude}&longitude=${longitude}` +
    `&current=temperature_2m,relative_humidity_2m,wind_speed_10m,weather_code`;

  const weatherRes  = await fetch(weatherUrl);
  const weatherData = await weatherRes.json();
  const current     = weatherData.current;

  return {
    city:        `${name}, ${country}`,
    temperature: `${current.temperature_2m}°C`,
    humidity:    `${current.relative_humidity_2m}%`,
    wind_speed:  `${current.wind_speed_10m} km/h`,
  };
}

function calculate({ expression }) {
  try {
    // Strip anything that is not a number or basic operator before eval.
    // This is not a complete sandbox — use a proper math parser like mathjs
    // in production if expressions come from untrusted users.
    const safe = expression.replace(/[^0-9+\-*/.() %]/g, '');
    if (!safe.trim()) return { error: 'Invalid or empty expression' };

    const result = Function('"use strict"; return (' + safe + ')')();
    return { expression, result };
  } catch {
    return { error: `Could not evaluate: ${expression}` };
  }
}

async function get_exchange_rate({ from, to, amount = 1 }) {
  const url  = `https://api.frankfurter.app/latest?from=${from.toUpperCase()}&to=${to.toUpperCase()}`;
  const res  = await fetch(url);
  const data = await res.json();

  if (data.error) return { error: data.error };

  const rate      = data.rates[to.toUpperCase()];
  if (!rate) return { error: `No rate found for ${from} → ${to}` };

  const converted = parseFloat((amount * rate).toFixed(4));

  return { from: from.toUpperCase(), to: to.toUpperCase(), rate, amount, converted };
}

module.exports = { get_weather, calculate, get_exchange_rate };

There are a few things worth pointing out here.

Open-Meteo uses geocoding before the weather fetch. Passing latitude and longitude directly to the weather endpoint is more reliable than a city name string, and the geocoding API handles misspellings reasonably well. There are two fetch calls, but that's the trade-off for accuracy.

The calculate function strips everything except digits and operators before evaluation. It narrows the injection risk, but it's not a real sandbox. If you expose this over HTTP with anonymous users, use a proper math parser like mathjs.

Frankfurter converts currency codes to uppercase before the request. Users will type "usd" or "Usd" and both should work. The API is case-sensitive on its end.

Building the Agentic Loop

This is the file everything else supports. The loop itself is about 20 lines. The rest is logging and error handling.

// src/agent.js
const { GoogleGenerativeAI } = require('@google/generative-ai');
const { toolDefinitions }    = require('./tools');
const { get_weather, calculate, get_exchange_rate } = require('./functions');

const genAI = new GoogleGenerativeAI(process.env.GEMINI_API_KEY);

// Map tool names to handler functions
const toolHandlers = { get_weather, calculate, get_exchange_rate };

async function runAgent(userMessage) {
  const model = genAI.getGenerativeModel({
    model: process.env.GEMINI_MODEL || 'gemini-2.0-flash',
    tools: [{ functionDeclarations: toolDefinitions }],
  });

  const chat = model.startChat();

  console.log(`\nUser: ${userMessage}`);
  console.log('---');

  let response = await chat.sendMessage(userMessage);
  let iterations = 0;
  const MAX_ITERATIONS = 10; // safety cap against infinite loops

  // Agentic loop
  while (iterations < MAX_ITERATIONS) {
    iterations++;
    const calls = response.response.functionCalls();

    // No tool calls — model is done, return the answer
    if (!calls || calls.length === 0) break;

    // Run all requested tools, collect results
    const toolResults = await Promise.allSettled(
      calls.map(async (call) => {
        console.log(`Calling tool: ${call.name}(${JSON.stringify(call.args)})`);

        const handler = toolHandlers[call.name];

        if (!handler) {
          return {
            functionResponse: {
              name:     call.name,
              response: { error: `Unknown tool: ${call.name}` },
            },
          };
        }

        try {
          const result = await handler(call.args);
          console.log(`Tool result: ${JSON.stringify(result)}`);
          return {
            functionResponse: {
              name:     call.name,
              response: result,
            },
          };
        } catch (err) {
          return {
            functionResponse: {
              name:     call.name,
              response: { error: err.message },
            },
          };
        }
      })
    );

    // Extract values from allSettled (fulfilled only — errors already caught above)
    const parts = toolResults
      .filter(r => r.status === 'fulfilled')
      .map(r => r.value);

    // Send all results back to the model in one message
    response = await chat.sendMessage(parts);
  }

  return response.response.text();
}

module.exports = { runAgent };

The MAX_ITERATIONS cap isn't paranoia. A model can get into a loop if a tool keeps returning an error and the model keeps retrying. 10 iterations is more than enough for any real query. A complex multi-tool question typically resolves in two or three turns.

Promise.allSettled runs all requested tools in parallel rather than sequentially. When the model requests both weather and exchange rate in the same response, they fetch simultaneously. Individual tool failures get caught inside the map rather than letting one failure abort the others.

The logging is intentional. When you're building and testing an agent, watching the tool calls happen in real time tells you whether the routing is working. Production code would route these to a structured logger instead of console output, but the information is the same.

The CLI Entry Point

require('dotenv').config() runs first so your API key loads before anything else. After that it's a standard readline loop: each line you type goes to runAgent, the response prints, and the loop waits for the next input.

// src/index.js
require('dotenv').config();
const readline     = require('readline');
const { runAgent } = require('./agent');

const rl = readline.createInterface({
  input:  process.stdin,
  output: process.stdout,
});

function ask(prompt) {
  return new Promise(resolve => rl.question(prompt, resolve));
}

async function main() {
  console.log('Gemini Agent — type your question, or "exit" to quit\n');

  while (true) {
    const input = await ask('You: ');
    if (input.toLowerCase() === 'exit') break;
    if (!input.trim()) continue;

    try {
      const answer = await runAgent(input);
      console.log(`\nAgent: ${answer}\n`);
    } catch (err) {
      console.error(`Error: ${err.message}`);
    }
  }

  rl.close();
}

main();

Adding an Express HTTP Server

The CLI is useful for testing. For integrating into an app, you need an HTTP endpoint.

// src/server.js
require('dotenv').config();
const express      = require('express');
const { runAgent } = require('./agent');

const app  = express();
const PORT = process.env.PORT || 3000;

app.use(express.json());

app.post('/agent', async (req, res) => {
  const { message } = req.body;

  if (!message || typeof message !== 'string') {
    return res.status(400).json({ error: 'message is required and must be a string' });
  }

  try {
    const answer = await runAgent(message);
    res.json({ answer });
  } catch (err) {
    console.error('[agent error]', err.message);
    res.status(500).json({ error: 'Agent failed to process the request' });
  }
});

app.listen(PORT, () => {
  console.log(`Agent server running on http://localhost:${PORT}`);
});

Start it:

node src/server.js

Call it:

curl -X POST http://localhost:3000/agent \
  -H "Content-Type: application/json" \
  -d '{"message": "What is the weather in Paris?"}'

Response:

{
  "answer": "The current weather in Paris, France is 19°C with 65% humidity and wind speeds of 12 km/h."
}

The POST body is a plain { message } object. The response is a plain { answer } string. The agent handles the rest.

Testing the Agent

Start the CLI:

node src/index.js

The You: prompt comes from the readline in index.js. The User: line and --- separator are logged by agent.js at the start of each run. This is the same logging discussed in the agentic loop section.

Single tool — weather:

You: What's the weather in Tokyo?

User: What's the weather in Tokyo?
---
Calling tool: get_weather({"city":"Tokyo"})
Tool result: {"city":"Tokyo, JP","temperature":"31°C","humidity":"72%","wind_speed":"8 km/h"}

Agent: The current weather in Tokyo, Japan is 31°C with 72% humidity and wind speeds of 8 km/h.

Two tools — chained reasoning:

You: What is the weather in Tokyo? If it's above 20°C, convert 10000 JPY to EUR.

User: What is the weather in Tokyo? If it's above 20°C, convert 10000 JPY to EUR.
---
Calling tool: get_weather({"city":"Tokyo"})
Tool result: {"city":"Tokyo, JP","temperature":"31°C","humidity":"72%","wind_speed":"8 km/h"}

Calling tool: get_exchange_rate({"from":"JPY","to":"EUR","amount":10000})
Tool result: {"from":"JPY","to":"EUR","rate":0.006,"amount":10000,"converted":60.0}

Agent: The current temperature in Tokyo is 31°C, which is above 20°C.
Converting 10,000 JPY to EUR at the current exchange rate gives approximately 60.00 EUR.

Notice what happened: the model called get_weather, read the result (31°C), applied the conditional logic from the user's question on its own, and then called get_exchange_rate. You didn't write any of that branching logic. The model handled it from the description alone.

Calculator:

You: How much is 18% tip on a $47.50 restaurant bill?

User: How much is 18% tip on a $47.50 restaurant bill?
---
Calling tool: calculate({"expression":"47.50 * 0.18"})
Tool result: {"expression":"47.50 * 0.18","result":8.55}

Agent: An 18% tip on a $47.50 bill is $8.55, making your total $56.05.

Three tools in one query:

You: What's the weather in London and Berlin? And what is 250 GBP in EUR?

User: What's the weather in London and Berlin? And what is 250 GBP in EUR?
---
Calling tool: get_weather({"city":"London"})
Calling tool: get_weather({"city":"Berlin"})
Calling tool: get_exchange_rate({"from":"GBP","to":"EUR","amount":250})
Tool result: {"city":"London, GB","temperature":"16°C","humidity":"78%","wind_speed":"20 km/h"}
Tool result: {"city":"Berlin, DE","temperature":"22°C","humidity":"55%","wind_speed":"14 km/h"}
Tool result: {"from":"GBP","to":"EUR","rate":1.17,"amount":250,"converted":292.5}

Agent: London is currently 16°C with 78% humidity and 20 km/h winds.
Berlin is warmer at 22°C with 55% humidity and lighter winds of 14 km/h.
250 GBP converts to approximately 292.50 EUR at the current exchange rate.

All three tools ran in parallel. Promise.allSettled is why. A sequential loop would have made three serial network requests. Parallel gives you the same result in roughly the time of the slowest single request.

Troubleshooting

Here are a few common issues you might encounter, and how to fix them:

1. [404 Not Found] models/gemini-1.5-flash is not found for API version v1beta

The model name is outdated. Google deprecates older aliases over time. Swap it out for gemini-2.0-flash in agent.js. To check what models your key can actually access, run:

node -e "
const { GoogleGenerativeAI } = require('@google/generative-ai');
require('dotenv').config();
const g = new GoogleGenerativeAI(process.env.GEMINI_API_KEY);
g.listModels().then(r => r.models.forEach(m => console.log(m.name)));
"

2. [429 Too Many Requests] You exceeded your current quota

The gemini-2.0-flash free tier caps you at 1,500 requests a day. Hit that and every call returns a 429 until midnight Pacific resets the counter.

The error names the quota ID directly. GenerateRequestsPerDayPerProjectPerModel-FreeTier means you hit the daily cap. GenerateRequestsPerMinutePerProjectPerModel-FreeTier means the per-minute rate.

For the per-minute limit, the error includes a retryDelay field. Wait that many seconds and retry. For the daily limit, the quota is per-project. All models under the same project share it.

There are three ways out:

• New project (fastest): head to aistudio.google.com, spin up a new project, grab a new API key, and swap it into .env. You get a fresh quota immediately.

• Enable billing: billing-enabled projects get much higher limits while keeping the free usage tier. Set up at aistudio.google.com.

• Wait: resets daily at midnight Pacific.

Because agent.js reads the model name from process.env.GEMINI_MODEL, you can also switch models without touching code. Add this to your .env to test with a lighter model:

GEMINI_MODEL=gemini-2.0-flash-lite

Remove the line when your quota resets and the agent goes back to gemini-2.0-flash.

3. Error: GEMINI_API_KEY is not set

Nine times out of ten, require('dotenv').config() is either missing or buried below other requires. Drag it to the very top of index.js. Your .env also needs to live at the project root with your actual key in it, not your_api_key_here.

4. GoogleGenerativeAIError: 400 INVALID_ARGUMENT

Almost always a malformed tool schema. Gemini uses uppercase type strings: 'OBJECT', 'STRING', 'NUMBER'. JSON Schema uses lowercase. Check your parameters.type values.

5. Model Answers Without Calling Any Tools

The description is too vague or the user's question doesn't match well enough for the model to route it. Add more context to the description about when the tool should be used. The phrase "use this when the user asks about X" directly improves routing accuracy.

6. TypeError: fetch is not a function

Node 17 and below don't have native fetch. It was added in Node 18. Run node --version to check yours.

If you can't upgrade, install it with npm install node-fetch. Every file that calls fetch then needs const fetch = require('node-fetch') as its first line.

7. Tool Works in Isolation but Agent Loop Doesn't Call it

The name in toolDefinitions must exactly match the key in toolHandlers. Case matters in JavaScript. get_Weather and get_weather are two different things.

8. Exchange Rate Returns No rate found

The currency code you passed isn't supported by frankfurter.app. The API covers ~30 major currencies. Check supported codes at frankfurter.app.

What to Build Next

The three tools here are a foundation. The loop works the same way regardless of how many tools you add.

Database lookup tool: A search_products function that queries your PostgreSQL table turns the agent into a product assistant. Point it at your catalog, and it can answer questions about availability, pricing, and specs without you writing any routing logic.

Write tools: get_* functions make the agent read-only. Add a create_ticket or send_notification function and the agent can take actions: file a support request, trigger a workflow, update a record. Once you add write tools, think carefully about which queries should require confirmation before executing.

Memory across sessions: Right now model.startChat() creates a fresh conversation on every call. Pass a history array when starting the chat and the model remembers prior turns. Store that history in PostgreSQL or Redis keyed to the user ID, and the agent carries context across sessions.

Streaming responses: For a UI that shows the answer as it types rather than waiting for the full response, replace chat.sendMessage with chat.sendMessageStream. The tool call loop stays the same. Only the final response delivery changes.

Swap the model: The model string in getGenerativeModel is the only thing that pins you to Gemini 2.0 Flash. gemini-2.0-flash-lite is lighter and faster for simpler queries. For stronger reasoning on complex tasks, run the listModels script from the Troubleshooting section to find the latest available models. The function calling interface is identical across all Gemini models, so swapping takes one line.

The full source code for this article is on GitHub at github.com/ziaongit/nodejs-gemini-agent.

I pulled up the query logs expecting something subtle. It was not. The list endpoint had no tenant_id filter at all. Every tenant in the database could read every other tenant's projects. The application never threw an error. It just returned whatever was there.

Missing tenant filters don't throw errors. They return the wrong data without any complaint, and nothing in your logs will flag it. I've seen this run in production for weeks before a support ticket pointed anyone at the query logs.

When it does surface, who finds it first matters a lot. A customer noticing it is bad. A compliance auditor noticing it during a SOC 2 review is a different kind of problem.

Isolation built in from the start is a day of work. The time I spent helping a team retrofit it after a compliance review was considerably longer than that, and involved more customer emails than anyone wanted to write.

The stack is Node.js with PostgreSQL. CRUD is the easy part. Tenant isolation, RBAC, and audit logging take more care, and where those checks run in the stack matters. I put all three in middleware, before any route handler fires. A handler that never calls the isolation logic directly can't accidentally skip it.

Prerequisites

• Node.js 18+

• PostgreSQL 14+

• Basic knowledge of Express.js and JWT

What We Will Build

A multi-tenant Express REST API that enforces:

1. Tenant isolation: every database query scopes to the tenant_id from the verified JWT. The client can't influence which tenant the query runs against.

2. RBAC: four roles, each with a numeric level (SuperAdmin is highest, Viewer lowest). Middleware checks the level before the handler runs.

3. Audit logging: any write or sensitive read appends a row to the audit table. The app can't modify those rows afterward. The database enforces this directly. If a bug in the app tries to UPDATE an audit row, the database refuses it. Application-level enforcement alone can't give you that guarantee.

4. Per-tenant rate limiting: request counts in Redis, keyed to the tenant. I've seen IP-based limiting break an enterprise rollout when fifty users came through a single corporate proxy.

5. Tenant isolation tests: a dedicated test file that proves cross-tenant data can't leak. Wire it into CI and it catches broken isolation before it ships.

Table of Contents

1. How Multi-Tenancy Works

2. Architecture Overview

3. Database Schema Design

4. Project Setup

5. JWT Design for Multi-Tenancy

6. Auth and RBAC Middleware

7. The Tenant-Safe Repository Layer

8. Audit Logging Service

9. Per-Tenant Rate Limiting

10. Building the Routes

11. Testing Tenant Isolation

12. Troubleshooting

13. Wrapping Up

How Multi-Tenancy Works

This tutorial uses a shared database with row-level isolation: a tenant_id column on every table, a filter on every query. The database holds everyone's data together. The application decides what each tenant can see.

Two other approaches exist: schema-per-tenant and database-per-tenant. I've talked to teams on schema-per-tenant who ended up spending more engineering time on migration tooling than on their actual product. Database-per-tenant gives stronger guarantees but a connection pool that balloons with every new customer signup.

Neither scales cheaply. Row-level isolation scales further than most teams expect. The ones I know who moved off it did so years in, usually under specific regulatory pressure, not because the approach stopped working.

The one thing in this design that can't be optional: tenant_id must always come from the verified JWT. Not from the request body, not from the URL. Users control what they put in both of those. They don't control what gets signed into a JWT on your server.

Architecture Overview

HTTP Request
     │
     ▼
┌─────────────────────────────────────────┐
│           Express Middleware Stack       │
│                                         │
│  1. Rate Limiter (per tenant_id)        │
│  2. Auth Middleware (verify JWT)        │
│     └─► Extracts: userId, tenantId,    │
│          role, permissions              │
│  3. RBAC Middleware (check role)        │
└──────────────┬──────────────────────────┘
               │
               ▼
┌─────────────────────────────────────────┐
│           Route Handler                  │
│                                         │
│  1. Call Repository (tenant-safe query) │
│  2. Call Audit Service (fire & forget)  │
│  3. Return response                     │
└──────────────┬──────────────────────────┘
               │
     ┌─────────┴──────────┐
     ▼                    ▼
┌─────────┐        ┌────────────┐
│ Projects│        │ Audit Logs │
│  Table  │        │   Table    │
│(+tenant)│        │(append only│
└─────────┘        └────────────┘

Rate limiting, auth, and RBAC all run before any handler sees the request. Writes pass through the audit service. The repository takes tenantId from req.user and the handler never touches tenant scoping directly, so there's no path around it.

Database Schema Design

-- Tenants table
CREATE TABLE tenants (
  id          UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  name        VARCHAR(255) NOT NULL,
  plan        VARCHAR(50) NOT NULL DEFAULT 'free', -- 'free', 'pro', 'enterprise'
  created_at  TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

-- Users table
CREATE TABLE users (
  id          UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  tenant_id   UUID NOT NULL REFERENCES tenants(id) ON DELETE CASCADE,
  email       VARCHAR(255) NOT NULL,
  role        VARCHAR(50) NOT NULL DEFAULT 'Member', -- 'SuperAdmin','TenantAdmin','Member','Viewer'
  created_at  TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  UNIQUE(tenant_id, email)
);

CREATE INDEX idx_users_tenant ON users(tenant_id);

-- Projects table (example resource — replace with your domain entity)
CREATE TABLE projects (
  id          UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  tenant_id   UUID NOT NULL REFERENCES tenants(id) ON DELETE CASCADE,
  name        VARCHAR(255) NOT NULL,
  description TEXT,
  created_by  UUID NOT NULL REFERENCES users(id),
  created_at  TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  updated_at  TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

CREATE INDEX idx_projects_tenant ON projects(tenant_id);

-- Audit log table (append-only — never UPDATE or DELETE rows here)
CREATE TABLE audit_logs (
  id          UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  tenant_id   UUID NOT NULL,
  user_id     UUID NOT NULL,
  user_email  TEXT NOT NULL,
  user_role   TEXT NOT NULL,        -- role at time of action
  action      TEXT NOT NULL,        -- 'CREATE', 'UPDATE', 'DELETE', 'VIEW'
  resource    TEXT NOT NULL,        -- table name
  resource_id TEXT,
  old_values  JSONB,
  new_values  JSONB,
  ip_address  INET,
  user_agent  TEXT,
  created_at  TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

CREATE INDEX idx_audit_tenant ON audit_logs(tenant_id);
CREATE INDEX idx_audit_created ON audit_logs(created_at DESC);

-- Protect audit log at database level
-- Use a DO block so this runs safely in Docker where app_user is the superuser
DO $$
BEGIN
  IF current_user <> 'app_user' THEN
    REVOKE DELETE, UPDATE ON audit_logs FROM app_user;
  END IF;
END $$;

The REVOKE matters. Application bugs happen. If something in your codebase accidentally tries to UPDATE an audit row, you want the database to refuse it outright, not silently comply.

Project Setup

mkdir nodejs-multitenant-saas-api
cd nodejs-multitenant-saas-api
npm init -y
npm install express pg jsonwebtoken bcryptjs express-rate-limit rate-limit-redis ioredis dotenv
npm install --save-dev jest supertest

Starting PostgreSQL and Redis with Docker

Skip the local installs. One docker-compose.yml in the project root brings up both PostgreSQL and Redis:

services:
  postgres:
    image: postgres:16-alpine
    environment:
      POSTGRES_DB: saas_api
      POSTGRES_USER: app_user
      POSTGRES_PASSWORD: app_password
    ports:
      - "5432:5432"
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - ./schema.sql:/docker-entrypoint-initdb.d/01_schema.sql

  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"

volumes:
  postgres_data:

That schema.sql mount runs your SQL automatically when the container first starts. No psql required.

docker compose up -d

.env in the project root:

DATABASE_URL=postgresql://app_user:app_password@localhost:5432/saas_api
REDIS_URL=redis://localhost:6379
JWT_SECRET=your_random_secret_here
PORT=3000
NODE_ENV=development

Don't type a JWT_SECRET by hand. Run this to generate one:

node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"

File structure:

nodejs-multitenant-saas-api/
├── src/
│   ├── middleware/
│   │   ├── auth.js          # JWT verification + tenant extraction
│   │   ├── rbac.js          # Role enforcement
│   │   └── rateLimiter.js   # Per-tenant rate limiting
│   ├── services/
│   │   └── auditService.js  # Append-only audit logger
│   ├── repositories/
│   │   └── projectRepo.js   # Tenant-safe DB queries
│   ├── routes/
│   │   └── projects.js      # Route handlers
│   └── utils/
│       └── token.js         # JWT token generation
├── db/
│   ├── index.js             # PostgreSQL pool
│   └── redis.js             # Redis client
├── docker-compose.yml
├── app.js
├── server.js
└── tests/
    └── tenantIsolation.test.js

Boilerplate Files

There are four files the tutorial doesn't cover in detail, but the test file needs all of them to run:

// db/index.js
const { Pool } = require('pg');

const pool = new Pool({ connectionString: process.env.DATABASE_URL });

pool.on('error', (err) => console.error('PostgreSQL error:', err.message));

module.exports = { pool };

// db/redis.js
const Redis = require('ioredis');

const redisClient = new Redis(process.env.REDIS_URL);

redisClient.on('error', (err) => console.error('Redis error:', err.message));

module.exports = { redisClient };

// app.js
require('dotenv').config();
const express = require('express');
const projectsRouter = require('./src/routes/projects');

const app = express();
app.use(express.json());

app.use('/api/projects', projectsRouter);

// Global error handler — must have 4 parameters to be recognised by Express
app.use((err, req, res, next) => {
  console.error(err.stack);
  res.status(500).json({ error: 'Internal server error' });
});

module.exports = app;

// server.js
const app = require('./app');

const PORT = process.env.PORT || 3000;
app.listen(PORT, () => console.log(`Server running on port ${PORT}`));

bcryptjs is included for a login endpoint with proper password hashing. That part isn't covered here, but the GitHub repo has a working /api/auth/login example.

JWT Design for Multi-Tenancy

Both tenantId and role go into the JWT payload. Everything downstream reads from these two fields. Get them wrong, and nothing behaves correctly.

// Example JWT payload
{
  "userId": "usr_abc123",
  "tenantId": "ten_xyz789",
  "email": "alice@acme.com",
  "role": "TenantAdmin",
  "iat": 1720000000,
  "exp": 1720086400
}

The roles in order of privilege:

• SuperAdmin: cross-tenant access for your internal team only

• TenantAdmin: full access within their tenant

• Member: read and write within their tenant

• Viewer: read-only within their tenant

Generate a token (used for testing and your auth endpoint):

// src/utils/token.js
const jwt = require('jsonwebtoken');

function generateToken({ userId, tenantId, email, role }) {
  return jwt.sign(
    { userId, tenantId, email, role },
    process.env.JWT_SECRET,
    { expiresIn: '24h' }
  );
}

module.exports = { generateToken };

Auth and RBAC Middleware

The auth middleware does two things: verifies the JWT signature and extracts the tenant context into req.user.

That second part is what the entire system depends on. Every query downstream reads req.user.tenantId. The client has no say in what that value is. They send a token the server signed, and the server reads back what it put in.

// src/middleware/auth.js
const jwt = require('jsonwebtoken');

function authMiddleware(req, res, next) {
  const authHeader = req.headers.authorization;
  if (!authHeader?.startsWith('Bearer ')) {
    return res.status(401).json({ error: 'Missing or malformed Authorization header' });
  }

  const token = authHeader.split(' ')[1];

  try {
    const decoded = jwt.verify(token, process.env.JWT_SECRET);

    // tenantId always comes from the verified token — never req.body or req.params
    req.user = {
      userId:   decoded.userId,
      tenantId: decoded.tenantId,
      email:    decoded.email,
      role:     decoded.role,
    };

    next();
  } catch (err) {
    return res.status(401).json({ error: 'Invalid or expired token' });
  }
}

module.exports = { authMiddleware };

The RBAC middleware is separate from auth by design. Auth runs on every route. Role enforcement only applies where a minimum role is required. You pass the allowed roles to requireRole() and it compares the user's level against the hierarchy. A Viewer trying to delete something hits the 403 before the handler ever runs.

// src/middleware/rbac.js
const ROLE_HIERARCHY = {
  SuperAdmin:   4,
  TenantAdmin:  3,
  Member:       2,
  Viewer:       1,
};

// requireRole('TenantAdmin') — user must be TenantAdmin or higher
function requireRole(...roles) {
  return (req, res, next) => {
    const userLevel = ROLE_HIERARCHY[req.user?.role] ?? 0;
    const requiredLevel = Math.min(...roles.map(r => ROLE_HIERARCHY[r] ?? 999));

    if (userLevel < requiredLevel) {
      return res.status(403).json({
        error: 'Insufficient permissions',
        required: roles,
        current: req.user?.role,
      });
    }

    next();
  };
}

module.exports = { requireRole };

The Tenant-Safe Repository Layer

Isolation lives here. Every function takes tenantId as a required argument, pulled from req.user by the handler. There's no way to call these without providing a tenant scope. I've watched teams try to handle this with a URL parameter instead (GET /api/projects?tenantId=xyz) and call it isolated. It is not. Any client sends whatever it wants in a query string.

// src/repositories/projectRepo.js
const { pool } = require('../../db');

// List all projects for a tenant — tenantId is ALWAYS from the JWT
async function listProjects(tenantId) {
  const result = await pool.query(
    `SELECT id, name, description, created_by, created_at
     FROM projects
     WHERE tenant_id = $1
     ORDER BY created_at DESC`,
    [tenantId]
  );
  return result.rows;
}

// Get a single project — returns null if it belongs to a different tenant
// NOTE: Returns 404 (not 403) intentionally — don't reveal the resource exists
async function getProject(id, tenantId) {
  const result = await pool.query(
    `SELECT id, name, description, created_by, created_at
     FROM projects
     WHERE id = $1 AND tenant_id = $2`,
    [id, tenantId]
  );
  return result.rows[0] || null;
}

async function createProject({ tenantId, name, description, createdBy }) {
  const result = await pool.query(
    `INSERT INTO projects (tenant_id, name, description, created_by)
     VALUES ($1, $2, $3, $4)
     RETURNING *`,
    [tenantId, name, description, createdBy]
  );
  return result.rows[0];
}

async function updateProject(id, tenantId, updates) {
  const result = await pool.query(
    `UPDATE projects
     SET name = COALESCE($3, name),
         description = COALESCE($4, description),
         updated_at = NOW()
     WHERE id = $1 AND tenant_id = $2
     RETURNING *`,
    [id, tenantId, updates.name, updates.description]
  );
  return result.rows[0] || null;
}

async function deleteProject(id, tenantId) {
  const result = await pool.query(
    `DELETE FROM projects WHERE id = $1 AND tenant_id = $2 RETURNING id`,
    [id, tenantId]
  );
  return result.rows[0] || null;
}

module.exports = { listProjects, getProject, createProject, updateProject, deleteProject };

Notice what getProject does when Tenant A tries to fetch a Tenant B resource. The query runs with Tenant A's tenantId. The condition id = $1 AND tenant_id = $2 matches nothing, null comes back, and the handler sends a 404. Not a 403. A 403 tells the caller the resource exists, but they can't access it, which is information they shouldn't have.

Audit Logging Service

// src/services/auditService.js
const { pool } = require('../../db');

async function log({
  tenantId,
  userId,
  userEmail,
  userRole,          // role at time of action — roles change, log should not
  action,            // 'CREATE' | 'UPDATE' | 'DELETE' | 'VIEW'
  resource,          // table name
  resourceId = null,
  oldValues = null,
  newValues = null,
  ipAddress = null,
  userAgent = null,
}) {
  const query = `
    INSERT INTO audit_logs
      (tenant_id, user_id, user_email, user_role, action, resource,
       resource_id, old_values, new_values, ip_address, user_agent)
    VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11)
  `;

  const values = [
    tenantId, userId, userEmail, userRole, action, resource,
    resourceId,
    oldValues  ? JSON.stringify(oldValues)  : null,
    newValues  ? JSON.stringify(newValues)  : null,
    ipAddress,
    userAgent,
  ];

  // Fire-and-forget — audit logging must never block or fail a user request
  pool.query(query, values).catch((err) => {
    console.error('[AuditService] Failed to write log:', err.message);
  });
}

module.exports = { log };

Capturing userRole at write time matters more than it looks. User roles change after the fact: someone gets demoted, a permission is revoked. If the log only records the user ID, you lose the context of what privilege they held when the action happened. Store the role at the time of the action, and you always know.

Per-Tenant Rate Limiting

IP-based rate limiting breaks down in SaaS. A corporate customer might route hundreds of users through a single NAT gateway, sharing one IP address. One heavy tenant throttles everyone else on that address.

I've watched teams discover this the hard way when an enterprise customer suddenly floods the API, and their other tenants start getting 429s with no explanation. Scope limits to tenant_id instead.

// src/middleware/rateLimiter.js
const rateLimit = require('express-rate-limit');
const { RedisStore } = require('rate-limit-redis');
const { redisClient } = require('../../db/redis');

// Rate limits by plan — extend as needed
const PLAN_LIMITS = {
  free:       { max: 100,  windowMs: 15 * 60 * 1000 }, // 100 req / 15 min
  pro:        { max: 500,  windowMs: 15 * 60 * 1000 }, // 500 req / 15 min
  enterprise: { max: 2000, windowMs: 15 * 60 * 1000 }, // 2000 req / 15 min
};

function createTenantRateLimiter(plan = 'free') {
  const limits = PLAN_LIMITS[plan] || PLAN_LIMITS.free;

  return rateLimit({
    windowMs: limits.windowMs,
    max: limits.max,
    // Key = tenant_id from verified JWT — NOT the IP address
    keyGenerator: (req) => `tenant:${req.user?.tenantId || req.ip}`,
    store: new RedisStore({
      sendCommand: (...args) => redisClient.call(...args),
    }),
    handler: (req, res) => {
      res.status(429).json({
        error: 'Too many requests',
        retryAfter: Math.ceil(limits.windowMs / 1000),
      });
    },
  });
}

// Default limiter for all API routes
const defaultLimiter = createTenantRateLimiter('free');

module.exports = { defaultLimiter, createTenantRateLimiter };

Building the Routes

This is where everything connects. Auth and rate limiting apply to the whole router. Role checks go on individual routes. The audit log fires after every write. tenantId never comes from the request body or URL. req.user.tenantId is the only source, set by the auth middleware from the verified token, so there's no path around it.

One practical detail for Express 4: it doesn't catch async errors automatically. Every handler wraps its logic in try/catch and passes failures to next(err). Skip that and an unhandled promise rejection returns a blank 500 with no log entry and no audit trail. The comment at the top of the router is a reminder that the pattern is intentional.

// src/routes/projects.js
const express = require('express');
const { authMiddleware }  = require('../middleware/auth');
const { requireRole }     = require('../middleware/rbac');
const { defaultLimiter }  = require('../middleware/rateLimiter');
const audit               = require('../services/auditService');
const repo                = require('../repositories/projectRepo');

const router = express.Router();

// All routes require authentication
router.use(authMiddleware);
router.use(defaultLimiter);

// Express 4 does not catch async errors automatically.
// Every handler must wrap await calls in try/catch and pass errors to next().
// Without this, an unhandled promise rejection silently returns 500
// with no useful message and no audit log entry.

// GET /api/projects — list all (Viewer and above)
router.get('/', async (req, res, next) => {
  try {
    const projects = await repo.listProjects(req.user.tenantId);

    audit.log({
      tenantId:   req.user.tenantId,
      userId:     req.user.userId,
      userEmail:  req.user.email,
      userRole:   req.user.role,
      action:     'VIEW',
      resource:   'projects',
      ipAddress:  req.ip,
      userAgent:  req.headers['user-agent'],
    });

    res.json(projects);
  } catch (err) {
    next(err);
  }
});

// GET /api/projects/:id — single project (Viewer and above)
router.get('/:id', async (req, res, next) => {
  try {
    const project = await repo.getProject(req.params.id, req.user.tenantId);
    if (!project) return res.status(404).json({ error: 'Not found' });
    res.json(project);
  } catch (err) {
    next(err);
  }
});

// POST /api/projects — create (Member and above)
router.post('/', requireRole('Member', 'TenantAdmin', 'SuperAdmin'), async (req, res, next) => {
  try {
    const { name, description } = req.body;
    if (!name) return res.status(400).json({ error: 'name is required' });

    const project = await repo.createProject({
      tenantId:    req.user.tenantId,
      name,
      description,
      createdBy:   req.user.userId,
    });

    audit.log({
      tenantId:    req.user.tenantId,
      userId:      req.user.userId,
      userEmail:   req.user.email,
      userRole:    req.user.role,
      action:      'CREATE',
      resource:    'projects',
      resourceId:  project.id,
      newValues:   project,
      ipAddress:   req.ip,
      userAgent:   req.headers['user-agent'],
    });

    res.status(201).json(project);
  } catch (err) {
    next(err);
  }
});

// PUT /api/projects/:id — update (Member and above)
router.put('/:id', requireRole('Member', 'TenantAdmin', 'SuperAdmin'), async (req, res, next) => {
  try {
    const oldProject = await repo.getProject(req.params.id, req.user.tenantId);
    if (!oldProject) return res.status(404).json({ error: 'Not found' });

    const updated = await repo.updateProject(req.params.id, req.user.tenantId, req.body);

    audit.log({
      tenantId:    req.user.tenantId,
      userId:      req.user.userId,
      userEmail:   req.user.email,
      userRole:    req.user.role,
      action:      'UPDATE',
      resource:    'projects',
      resourceId:  req.params.id,
      oldValues:   oldProject,
      newValues:   updated,
      ipAddress:   req.ip,
      userAgent:   req.headers['user-agent'],
    });

    res.json(updated);
  } catch (err) {
    next(err);
  }
});

// DELETE /api/projects/:id — TenantAdmin and above only
router.delete('/:id', requireRole('TenantAdmin', 'SuperAdmin'), async (req, res, next) => {
  try {
    const project = await repo.getProject(req.params.id, req.user.tenantId);
    if (!project) return res.status(404).json({ error: 'Not found' });

    await repo.deleteProject(req.params.id, req.user.tenantId);

    audit.log({
      tenantId:    req.user.tenantId,
      userId:      req.user.userId,
      userEmail:   req.user.email,
      userRole:    req.user.role,
      action:      'DELETE',
      resource:    'projects',
      resourceId:  req.params.id,
      oldValues:   project,
      ipAddress:   req.ip,
      userAgent:   req.headers['user-agent'],
    });

    res.json({ deleted: true });
  } catch (err) {
    next(err);
  }
});

module.exports = router;

Testing Tenant Isolation

Skip the isolation tests and you're flying blind. The application keeps running, nothing throws an error, but two customers are reading each other's data.

I've watched this sit undetected in production for months because nothing actually broke. The wrong data just showed up quietly. Automated tests on every pull request are the only reliable way to catch it early.

// tests/tenantIsolation.test.js
require('dotenv').config();  // must be first — loads DATABASE_URL and REDIS_URL
const request = require('supertest');
const app     = require('../app');
const { generateToken } = require('../src/utils/token');
const { pool }        = require('../db');
const { redisClient } = require('../db/redis');

// Test fixture: two isolated tenants, one project in Tenant B
async function seedTestData() {
  // Clean up from any previous run to avoid unique-constraint failures
  await pool.query(`DELETE FROM projects WHERE name LIKE 'TEST-%'`);
  await pool.query(`DELETE FROM tenants WHERE name IN ('Tenant A', 'Tenant B')`);

  const tenantA = (await pool.query(
    `INSERT INTO tenants (name, plan) VALUES ('Tenant A', 'pro') RETURNING id`
  )).rows[0].id;

  const tenantB = (await pool.query(
    `INSERT INTO tenants (name, plan) VALUES ('Tenant B', 'pro') RETURNING id`
  )).rows[0].id;

  const userA = (await pool.query(
    `INSERT INTO users (tenant_id, email, role) VALUES ($1, 'usera@a.com', 'Member') RETURNING id`,
    [tenantA]
  )).rows[0].id;

  // userB owns the project in Tenant B — satisfies the created_by FK constraint
  const userB = (await pool.query(
    `INSERT INTO users (tenant_id, email, role) VALUES ($1, 'userb@b.com', 'Member') RETURNING id`,
    [tenantB]
  )).rows[0].id;

  const projectB = (await pool.query(
    `INSERT INTO projects (tenant_id, name, created_by)
     VALUES ($1, 'TEST-Secret Project', $2) RETURNING id`,
    [tenantB, userB]
  )).rows[0].id;

  return { tenantA, tenantB, userA, projectB };
}

describe('Tenant Isolation', () => {
  let data;

  beforeAll(async () => {
    data = await seedTestData();
  });

  afterAll(async () => {
    await pool.query(`DELETE FROM tenants WHERE name IN ('Tenant A', 'Tenant B')`);
    await pool.end();
    await redisClient.quit();  // close Redis connection so Jest exits cleanly
  });

  test('Tenant A user cannot read Tenant B project', async () => {
    const token = generateToken({
      userId:   data.userA,
      tenantId: data.tenantA,   // ← Tenant A token
      email:    'usera@a.com',
      role:     'Member',
    });

    const res = await request(app)
      .get(`/api/projects/${data.projectB}`)  // ← Tenant B's project ID
      .set('Authorization', `Bearer ${token}`);

    // Must be 404, not 200 or 403
    expect(res.status).toBe(404);
  });

  test('Tenant A user cannot list Tenant B projects', async () => {
    const token = generateToken({
      userId:   data.userA,
      tenantId: data.tenantA,
      email:    'usera@a.com',
      role:     'TenantAdmin',
    });

    const res = await request(app)
      .get('/api/projects')
      .set('Authorization', `Bearer ${token}`);

    expect(res.status).toBe(200);
    // Response must contain zero Tenant B projects
    const names = res.body.map(p => p.name);
    expect(names).not.toContain('TEST-Secret Project');
  });

  test('Viewer cannot delete a project', async () => {
    const token = generateToken({
      userId:   data.userA,
      tenantId: data.tenantA,
      email:    'usera@a.com',
      role:     'Viewer',         // ← Viewer role
    });

    const res = await request(app)
      .delete(`/api/projects/${data.projectB}`)
      .set('Authorization', `Bearer ${token}`);

    expect(res.status).toBe(403);
  });
});

Run the tests:

npm test

Three tests, three boundaries confirmed. Wire these into CI so they run on every pull request. A future refactor that quietly drops the tenant_id filter will get caught before it ships.

Troubleshooting

Tenant A can see Tenant B's data

One query is missing the AND tenant_id = $N clause. Search every repository file for SELECT statements and check each one. It's almost always this.

403 Forbidden on a route that should be accessible

The role string in the JWT doesn't match what requireRole() is checking. Check the exact string in the token payload. 'member' and 'Member' aren't the same thing. Paste your token into jwt.io and look at the role field directly.

Rate limiter isn't working

Redis is probably not connected. Log redisClient.status before the server starts. If it's not ready, the limiter has fallen back to in-memory, which means restarts reset all counters and tenant-scoped limiting stops working.

Audit log table growing very large

Expected behaviour. Audit tables grow, that's the point. Once it gets large, ship rows older than a year to S3 or Azure Blob and keep querying against a smaller hot table. Most compliance requirements want at least 12 months of accessible logs anyway. Just don't DELETE from the table itself.

jwt.verify throws JsonWebTokenError: invalid signature

The secret that signed the token doesn't match JWT_SECRET in the environment where you're verifying it. This comes up most when switching between environments or when a second service has a different value in its .env. Every service that calls jwt.verify needs the exact same secret. Copy it across, don't retype it.

Wrapping Up

The system you've built: row-level isolation in the repository, role checks before the handler runs, an audit table the app can't touch, and rate limits per tenant. That's the whole thing.

The tests are what I see dropped most often. Teams build the isolation, ship it, and never write something that actually proves cross-tenant data can't leak. Then a query gets refactored six months later and the tenant_id filter quietly disappears. CI catches it. Manual code review rarely does.

Schema-per-tenant comes up eventually if your product grows large enough. But not at the start. Row-level isolation handles more scale than most teams will ever hit, and it costs a fraction of the operational overhead.

The full working code is available on GitHub: nodejs-multitenant-saas-api

IBM's 2024 breach cost report put the average data breach at $4.88 million — and that's the average, not the worst cases.

Exposed credentials are consistently near the top of root causes. GitHub found over a million secrets leaked in public repos in 2023 alone, before you even count the private ones nobody ever discovered.

It's not a people problem. The developers I've worked with aren't careless — the architecture is just set up to fail them. A .env file gets committed once by accident. Credentials get copied and pasted into a Slack message to unblock a teammate. A Docker image gets published with secrets baked into a layer. A server gets shut down, and nobody rotates the credentials it was holding.

Azure Key Vault solves this differently. Your application fetches credentials at runtime from a centralized, encrypted service — the .env file stops being a liability because it stops holding anything worth stealing.

What you'll build is a Node.js Express API that fetches every secret from Azure Key Vault at startup. No passwords in the code. When someone quits, there's nothing in the repo to rotate. The .env ends up with one line — the vault name.

Prerequisites

• Node.js 18+

• An Azure account (free tier works)

• Azure CLI installed and logged in (az login)

• Basic knowledge of Express.js

• Docker (optional — only needed for the local database test section)

What We Will Build

A Node.js Express API that:

1. Connects to PostgreSQL using credentials fetched from Key Vault at startup

2. Uses Managed Identity for authentication — no client secrets or passwords anywhere

3. Caches secrets in memory, so Key Vault isn't called on every request

4. Works locally via Azure CLI auth and in production via Managed Identity — same code, zero changes

Table of Contents

1. How the Architecture Works

2. What Is Azure Key Vault?

3. Set Up the Key Vault

4. Create the Node.js Project

5. Connect to Key Vault with Managed Identity

6. Cache Secrets at Startup

7. Use Secrets in Your Express API

8. Test Locally

9. Deploy to Azure App Service

10. Grant Key Vault Access to the App

11. Rotate Secrets Without Redeploying

12. Troubleshooting

13. Wrapping Up

How the Architecture Works

Before writing any code, it helps to see the full picture:

 LOCAL DEVELOPMENT
.-------------------------------------------------------.
|                                                        |
|   [Node.js App]                                        |
|        |                                               |
|        v                                               |
|   [DefaultAzureCredential] ---> az login session       |
|        |                                               |
|        v                                               |
|   [Azure Key Vault]  ---> Returns secrets              |
|        |                                               |
|        v                                               |
|   [In-memory cache]  ---> App uses secrets at runtime  |
'-------------------------------------------------------'

 PRODUCTION (Azure)
.-------------------------------------------------------.
|                                                        |
|   [Azure App Service]                                  |
|        |                                               |
|        v                                               |
|   [DefaultAzureCredential] ---> Managed Identity       |
|        |                                               |
|        v                                               |
|   [Azure Key Vault]  ---> Returns secrets              |
|        |                                               |
|        v                                               |
|   [In-memory cache]  ---> App uses secrets at runtime  |
'-------------------------------------------------------'

Both environments run the exact same code. DefaultAzureCredential figures out where it is — locally it picks up your az login session, on Azure it uses Managed Identity. You don't switch config files and you don't manage credentials. It just works.

What Is Azure Key Vault?

Azure Key Vault is Microsoft's managed secret store — it handles secrets, keys, and certificates. For this tutorial, we're only using the secrets part: database passwords, API keys, JWT signing keys, anything your app needs to run but has no business being in your Git history.

Compared to .env files, the practical differences are worth understanding before you write any code.

Rotation is the one I notice most on real projects. Update a secret in Key Vault and every app picks it up on the next restart — no hunting down five different environment configs across staging and production.

Access control is the other big one. Each application only gets permission to read the secrets it actually needs. If one service gets compromised, it can't read credentials belonging to other services.

And every read gets logged. When something goes wrong — and eventually something will — you can see exactly which app accessed which secret, and when. That log is what auditors actually want to see.

I've sat in enough security reviews to know that "we use .env files and tell people not to commit them" doesn't satisfy an auditor. SOC 2, HIPAA, GDPR — they all want demonstrable controls. A vault with an access log is demonstrable.

Set Up the Key Vault

Run these commands. The vault name has to be globally unique across all of Azure — not just your own subscription — so pick something specific. Letters, numbers, and hyphens, 3 to 24 characters.

# Create a resource group (skip if you already have one)
az group create \
  --name keyvault-demo-rg \
  --location eastus

# Create the Key Vault (RBAC enabled by default — required for the role assignment later)
az keyvault create \
  --name your-vault-name \
  --resource-group keyvault-demo-rg \
  --location eastus

# Grant yourself permission to manage secrets (required with RBAC — creators are not auto-assigned)
az role assignment create \
  --role "Key Vault Secrets Officer" \
  --assignee-object-id $(az ad signed-in-user show --query id -o tsv) \
  --scope $(az keyvault show \
    --name your-vault-name \
    --resource-group keyvault-demo-rg \
    --query id -o tsv)

# Add your secrets
az keyvault secret set \
  --vault-name your-vault-name \
  --name "DB-HOST" \
  --value "your-db-host.postgres.database.azure.com"

az keyvault secret set \
  --vault-name your-vault-name \
  --name "DB-PASSWORD" \
  --value "your-super-secret-password"

az keyvault secret set \
  --vault-name your-vault-name \
  --name "JWT-SECRET" \
  --value "your-jwt-signing-secret"

Verify the secrets were stored:

az keyvault secret list --vault-name your-vault-name --query "[].name" -o tsv

You should see:

DB-HOST
DB-PASSWORD
JWT-SECRET

Create the Node.js Project

Set up the project structure:

mkdir nodejs-azure-keyvault
cd nodejs-azure-keyvault
npm init -y
npm install express pg jsonwebtoken @azure/keyvault-secrets @azure/identity dotenv

The two Azure packages do all the work:

• @azure/keyvault-secrets — connects to your vault and pulls secrets out

• @azure/identity — handles auth. Locally, it uses your az login session, in production, it switches to Managed Identity automatically

Add a start script to package.json:

npm pkg set scripts.start="node server.js"

Create the following file structure:

nodejs-azure-keyvault/
|-- src/
|   |-- config/
|   |   `-- secrets.js   # Key Vault client and secret loader
|   |-- db/
|   |   `-- index.js     # PostgreSQL pool using secrets
|   `-- routes/
|       `-- users.js     # Example route
|-- app.js               # Express app
`-- server.js            # Entry point -- loads secrets first

Connect to Key Vault with Managed Identity

Create the secrets config file:

// src/config/secrets.js
const { SecretClient } = require('@azure/keyvault-secrets');
const { DefaultAzureCredential } = require('@azure/identity');

const VAULT_URL = `https://${process.env.KEY_VAULT_NAME}.vault.azure.net`;

const credential = new DefaultAzureCredential();
const client = new SecretClient(VAULT_URL, credential);

async function getSecret(name) {
  const secret = await client.getSecret(name);
  return secret.value;
}

module.exports = { getSecret };

DefaultAzureCredential is the most important part of this setup. It tries a chain of authentication methods in order:

1. Environment variables (for CI/CD pipelines)

2. Azure CLI credentials (for local development — az login)

3. Managed Identity (for deployed apps on Azure)

This means the exact same code works locally and in production with zero changes. Locally, it uses your az login session. In production, it uses the app's Managed Identity. You never touch credentials.

Cache Secrets at Startup

Calling Key Vault on every request adds latency and costs money. Load all secrets once at startup and cache them in memory. Replace src/config/secrets.js with this complete version:

// src/config/secrets.js
const { SecretClient } = require('@azure/keyvault-secrets');
const { DefaultAzureCredential } = require('@azure/identity');

const VAULT_URL = `https://${process.env.KEY_VAULT_NAME}.vault.azure.net`;

const credential = new DefaultAzureCredential();
const client = new SecretClient(VAULT_URL, credential);

// In-memory cache
const cache = {};

async function getSecret(name) {
  if (cache[name]) return cache[name];
  const secret = await client.getSecret(name);
  cache[name] = secret.value;
  return secret.value;
}

async function loadAllSecrets() {
  console.log('Loading secrets from Azure Key Vault...');
  const secretNames = ['DB-HOST', 'DB-PASSWORD', 'JWT-SECRET'];

  await Promise.all(
    secretNames.map(async (name) => {
      cache[name] = await getSecret(name);
      console.log(`  ✓ ${name} loaded`);
    })
  );

  console.log('All secrets loaded successfully.');
}

function getFromCache(name) {
  if (!cache[name]) throw new Error(`Secret "${name}" not loaded. Did loadAllSecrets() run?`);
  return cache[name];
}

module.exports = { loadAllSecrets, getFromCache };

The loadAllSecrets function runs once when the application starts. After that, all secrets are served from the in-memory cache with zero latency and zero Key Vault calls.

Use Secrets in Your Express API

Set up the database connection using the cached secrets:

// src/db/index.js
const { Pool } = require('pg');
const { getFromCache } = require('../config/secrets');

let pool;

function getPool() {
  if (!pool) {
    pool = new Pool({
      host:     getFromCache('DB-HOST'),
      database: process.env.DB_NAME || 'myapp',
      user:     process.env.DB_USER || 'dbadmin',
      password: getFromCache('DB-PASSWORD'),
      port:     parseInt(process.env.DB_PORT || '5432'),
      ssl:      process.env.NODE_ENV === 'production'
                  ? { rejectUnauthorized: false }
                  : false,
    });

    pool.on('error', (err) => {
      console.error('Unexpected database pool error:', err.message);
    });
  }

  return pool;
}

module.exports = { getPool };

Notice the distinction: DB-HOST and DB-PASSWORD come from Key Vault because they're sensitive. The database name, username, and port are not — they don't need to be protected, so they use environment variables with sensible defaults. Key Vault is for credentials, not all configuration.

The SSL flag is environment-aware: forced on in production, off locally so Docker connections work without a certificate. The rejectUnauthorized: false setting accepts Azure Database for PostgreSQL's certificate without verifying the CA chain — this is standard for Azure-managed databases. For stricter environments, you can download the Azure root CA and pass it via the ca option in the pool config instead.

Create a sample route that uses JWT verification with the secret from Key Vault:

// src/routes/users.js
const express = require('express');
const jwt     = require('jsonwebtoken');
const { getFromCache } = require('../config/secrets');
const { getPool }      = require('../db');

const router = express.Router();

// Auth middleware — JWT secret comes from Key Vault, not process.env
function authMiddleware(req, res, next) {
  const authHeader = req.headers.authorization;
  if (!authHeader?.startsWith('Bearer ')) {
    return res.status(401).json({ error: 'Missing or malformed Authorization header' });
  }

  const token = authHeader.split(' ')[1];

  try {
    req.user = jwt.verify(token, getFromCache('JWT-SECRET'));
    next();
  } catch (err) {
    return res.status(401).json({ error: 'Invalid or expired token' });
  }
}

// GET /api/users — list users (authenticated)
router.get('/', authMiddleware, async (req, res) => {
  try {
    const result = await getPool().query(
      'SELECT id, email, created_at FROM users ORDER BY created_at DESC LIMIT 20'
    );
    res.json(result.rows);
  } catch (err) {
    console.error('Database error:', err.message);
    res.status(500).json({ error: 'Internal server error' });
  }
});

// GET /api/users/:id — single user (authenticated)
router.get('/:id', authMiddleware, async (req, res) => {
  try {
    const result = await getPool().query(
      'SELECT id, email, created_at FROM users WHERE id = $1',
      [req.params.id]
    );
    if (!result.rows[0]) return res.status(404).json({ error: 'User not found' });
    res.json(result.rows[0]);
  } catch (err) {
    console.error('Database error:', err.message);
    res.status(500).json({ error: 'Internal server error' });
  }
});

module.exports = router;

Notice the error handler returns 'Internal server error' instead of err.message. Database errors are surprisingly chatty — they'll hand an attacker your table names, column names, and query structure if you let them through.

Set up the Express application. Both files define authMiddleware locally — yes, it's duplicated. In production, I'd pull this into a shared middleware file. For this tutorial, keeping it local means you can read either file without bouncing between three others:

// app.js
const express = require('express');
const jwt = require('jsonwebtoken');
const { getFromCache } = require('./src/config/secrets');
const usersRouter = require('./src/routes/users');

const app = express();
app.use(express.json());

// Auth middleware — JWT secret comes from Key Vault, not process.env
function authMiddleware(req, res, next) {
  const authHeader = req.headers.authorization;
  if (!authHeader?.startsWith('Bearer ')) {
    return res.status(401).json({ error: 'Missing or malformed Authorization header' });
  }
  const token = authHeader.split(' ')[1];
  try {
    req.user = jwt.verify(token, getFromCache('JWT-SECRET'));
    next();
  } catch (err) {
    return res.status(401).json({ error: 'Invalid or expired token' });
  }
}

// Health check — no auth required
app.get('/health', (req, res) => {
  res.json({ status: 'healthy', timestamp: new Date().toISOString() });
});

// Status endpoint — proves Key Vault integration without needing a database
app.get('/api/status', authMiddleware, (req, res) => {
  res.json({
    message: 'All secrets loaded from Azure Key Vault',
    vault: process.env.KEY_VAULT_NAME,
    secrets_loaded: ['DB-HOST', 'DB-PASSWORD', 'JWT-SECRET'],
    authenticated_as: req.user.email,
    timestamp: new Date().toISOString()
  });
});

app.use('/api/users', usersRouter);

app.use((req, res) => res.status(404).json({ error: 'Route not found' }));
app.use((err, req, res, next) => {
  console.error('Unhandled error:', err.message);
  res.status(500).json({ error: 'Internal server error' });
});

module.exports = app;

The entry point loads secrets before starting the server. The server doesn't start unless all secrets load successfully:

// server.js
require('dotenv').config();
const app = require('./app');
const { loadAllSecrets } = require('./src/config/secrets');

const PORT = process.env.PORT || 3000;

async function start() {
  try {
    await loadAllSecrets();
    app.listen(PORT, () => {
      console.log(`Server running on port ${PORT}`);
    });
  } catch (err) {
    console.error('Failed to start server:', err.message);
    console.error('Hint: Run "az login" for local development, or check Managed Identity for Azure deployments.');
    process.exit(1);
  }
}

start();

That process.exit(1) is deliberate. I'd rather the app crash loudly at startup than limp along with missing credentials and fail on the first real request two hours later.

Test Locally

Create a .env file for local development. This only contains the Key Vault name, nothing sensitive:

# .env
KEY_VAULT_NAME=your-vault-name
PORT=3000

Add .env and the deployment zip to .gitignore:

echo ".env" >> .gitignore
echo "app.zip" >> .gitignore

Make sure you're logged into Azure CLI:

az login

Start the application:

npm start

You should see:

Loading secrets from Azure Key Vault...
  ✓ JWT-SECRET loaded
  ✓ DB-PASSWORD loaded
  ✓ DB-HOST loaded
All secrets loaded successfully.
Server running on port 3000

The order secrets load may vary — Promise.all fetches them in parallel and resolves as each one completes. What matters is that all three are confirmed before the server starts.

Test the health endpoint:

curl http://localhost:3000/health
# {"status":"healthy","timestamp":"2026-07-14T19:38:11.659Z"}

Now prove the integration end-to-end. Grab the value you stored as JWT-SECRET and use it to sign a test token — paste it in for YOUR-JWT-SECRET-VALUE. Then hit /api/status with it:

node -e "const jwt = require('jsonwebtoken'); console.log(jwt.sign({id:1, email:'test@test.com'}, 'YOUR-JWT-SECRET-VALUE', {expiresIn:'1h'}));"

On Linux/macOS:

curl -H "Authorization: Bearer YOUR_TOKEN" http://localhost:3000/api/status

On Windows PowerShell:

Invoke-RestMethod -Uri "http://localhost:3000/api/status" -Headers @{Authorization = "Bearer YOUR_TOKEN"}

You should see:

{
  "message": "All secrets loaded from Azure Key Vault",
  "vault": "your-vault-name",
  "secrets_loaded": ["DB-HOST", "DB-PASSWORD", "JWT-SECRET"],
  "authenticated_as": "test@test.com",
  "timestamp": "2026-07-14T19:50:08.687Z"
}

If you got that response, the whole chain worked. The JWT was signed and verified using a secret that lived only in Key Vault — not in your code, not in your.env, not anywhere in the repo. Your az login session handled the auth locally. In production, Managed Identity takes over. Same code, nothing changes.

Test the Full Database Flow with Docker

The app reads DB-HOST and DB-PASSWORD from Key Vault, so those secrets need to match your local Docker container. Update them now:

az keyvault secret set --vault-name your-vault-name --name "DB-HOST" --value "localhost"
az keyvault secret set --vault-name your-vault-name --name "DB-PASSWORD" --value "demopassword123"

Docker up a Postgres container. The password has to match demopassword123 — that's what you just put in Key Vault:

docker run --name pg-demo \
  -e POSTGRES_USER=dbadmin \
  -e POSTGRES_PASSWORD=demopassword123 \
  -e POSTGRES_DB=myapp \
  -p 5432:5432 \
  -d postgres:15

Get the table created and throw in some test rows:

docker exec -it pg-demo psql -U dbadmin -d myapp -c \
  "CREATE TABLE IF NOT EXISTS users (id SERIAL PRIMARY KEY, email VARCHAR(255) UNIQUE NOT NULL, created_at TIMESTAMPTZ DEFAULT NOW());"

docker exec -it pg-demo psql -U dbadmin -d myapp -c \
  "INSERT INTO users (email) VALUES ('alice@example.com'), ('bob@example.com'), ('carol@example.com');"

Kill the server and bring it back up — secrets load at startup, so it needs a fresh run to pick up what you just changed in Key Vault:

npm start

Call the users endpoint with a valid JWT:

# Generate a token (use the same value you stored as JWT-SECRET in Key Vault)
node -e "const jwt = require('jsonwebtoken'); console.log(jwt.sign({id:1, email:'test@test.com'}, 'YOUR-JWT-SECRET-VALUE', {expiresIn:'1h'}));"

On Linux/macOS:

curl -H "Authorization: Bearer YOUR_TOKEN" http://localhost:3000/api/users

On Windows PowerShell:

Invoke-RestMethod -Uri "http://localhost:3000/api/users" -Headers @{Authorization = "Bearer YOUR_TOKEN"}

You should see:

[
  { "id": 1, "email": "alice@example.com", "created_at": "2026-07-14T19:59:21.064Z" },
  { "id": 2, "email": "bob@example.com",   "created_at": "2026-07-14T19:59:21.064Z" },
  { "id": 3, "email": "carol@example.com", "created_at": "2026-07-14T19:59:21.064Z" }
]

That query ran using a password that came straight from Key Vault. It's not in your .env, not hardcoded anywhere, and not in a local variable. The repo has nothing worth stealing.

Before you deploy, put the real production values back in Key Vault:

az keyvault secret set --vault-name your-vault-name --name "DB-HOST" --value "your-db-host.postgres.database.azure.com"
az keyvault secret set --vault-name your-vault-name --name "DB-PASSWORD" --value "your-super-secret-password"

If you skip this, the deployed app will try to connect to localhost and fail immediately — localhost doesn't exist on App Service.

Deploy to Azure App Service

Note: This section creates the App Service infrastructure. The actual code deployment (zip upload) happens at the end of the next section — the app must have Key Vault access configured before its first startup, or it will fail immediately and exit.

Create the App Service:

# Create an App Service Plan (B1 is the cheapest paid tier)
az appservice plan create \
  --name keyvault-demo-plan \
  --resource-group keyvault-demo-rg \
  --sku B1 \
  --is-linux

# Create the Web App
az webapp create \
  --name my-keyvault-node-app \
  --resource-group keyvault-demo-rg \
  --plan keyvault-demo-plan \
  --runtime "NODE:18-lts"

# Set app settings — KEY_VAULT_NAME tells the app which vault to use
# NODE_ENV=production enables SSL for the database connection
az webapp config appsettings set \
  --name my-keyvault-node-app \
  --resource-group keyvault-demo-rg \
  --settings KEY_VAULT_NAME=your-vault-name NODE_ENV=production

Grant Key Vault Access to the App

Enable Managed Identity on the app. This gives it an identity in Microsoft Entra ID that Key Vault can trust:

# Enable system-assigned managed identity
az webapp identity assign \
  --name my-keyvault-node-app \
  --resource-group keyvault-demo-rg

The following commands capture the principalId automatically and use it to grant the role:

# Get the principal ID
PRINCIPAL_ID=$(az webapp identity show \
  --name my-keyvault-node-app \
  --resource-group keyvault-demo-rg \
  --query principalId \
  --output tsv)

# Get the Key Vault resource ID
KV_ID=$(az keyvault show \
  --name your-vault-name \
  --resource-group keyvault-demo-rg \
  --query id \
  --output tsv)

# Grant the app the "Key Vault Secrets User" role
az role assignment create \
  --role "Key Vault Secrets User" \
  --assignee-object-id $PRINCIPAL_ID \
  --scope $KV_ID

The Key Vault Secrets User role allows the app to read secrets. It can't create, update, or delete them. This is the principle of least privilege — the application can only do what it needs to do.

Time to ship it. Linux/macOS can run this directly — Windows users, open Git Bash (it ships with Git for Windows):

zip -r app.zip . -x "node_modules/*" ".git/*" ".env" "app.zip"

Then deploy:

az webapp deployment source config-zip \
  --name my-keyvault-node-app \
  --resource-group keyvault-demo-rg \
  --src app.zip

The deployed application authenticates to Key Vault using its Managed Identity automatically. No passwords, no client secrets, no credentials of any kind in the deployment.

Check the health endpoint to confirm it's running:

curl https://my-keyvault-node-app.azurewebsites.net/health
# {"status":"healthy","timestamp":"..."}

If it won't start, pull the logs:

az webapp log tail --name my-keyvault-node-app --resource-group keyvault-demo-rg

Nine times out of ten, it's that the Key Vault role assignment has not been propagated yet. Give it 2–3 minutes, then restart:

az webapp restart --name my-keyvault-node-app --resource-group keyvault-demo-rg

Rotate Secrets Without Redeploying

One of the biggest practical benefits of Key Vault is secret rotation. When a database password needs to change, you update it in Key Vault — not in your app:

az keyvault secret set \
  --vault-name your-vault-name \
  --name "DB-PASSWORD" \
  --value "new-rotated-password"

The cache builds at startup, so you don't need a redeploy — a restart is enough:

az webapp restart \
  --name my-keyvault-node-app \
  --resource-group keyvault-demo-rg

No code change. No new deployment. The secret is rotated, and the app is using the new value in seconds.

If you need zero-downtime rotation, add a /refresh-secrets endpoint behind admin auth that clears the cache and then calls loadAllSecrets(). The order matters — loadAllSecrets() uses getSecret() which returns cached values if they exist, so you must clear the cache first, or it will reload nothing. This is optional but useful for long-running processes that can't afford a restart.

Troubleshooting

CredentialUnavailableError: DefaultAzureCredential failed to retrieve a token

You're not logged into Azure CLI. Run az login and try again. On Azure App Service, check that Managed Identity is enabled and the role assignment was created correctly.

RestError: Forbidden — The user does not have secrets get permission

The Managed Identity isn't wired up to Key Vault yet. Go back and run the az role assignment create command. If you already did, it might just need time. Azure can take 2–3 minutes to propagate role assignments, so give it a moment before you dig further.

Error: Secret "DB-PASSWORD" not loaded. Did loadAllSecrets() run?

getFromCache() ran before loadAllSecrets() finished, meaning the startup sequence is out of order. Open server.js and confirm await loadAllSecrets() comes before app.listen(). If the order's fine, the secret might just not be in the vault yet. Run az keyvault secret list --vault-name YOUR_VAULT to double-check. (A name mismatch — wrong case, typo — throws SecretNotFound instead, which is the entry below.)

App starts locally but fails on Azure App Service

Almost always, the app setting. Either KEY_VAULT_NAME isn't in App Service configuration at all, or the vault name has a typo. Run az webapp log tail to see the actual startup error — that'll tell you which one.

AuthorizationFailed when running az role assignment create

You are a guest user in your Azure tenant and lack the Owner role needed to assign roles. Switch the existing vault to the access policy model — no need to recreate it or lose your secrets:

az keyvault update \
  --name your-vault-name \
  --resource-group keyvault-demo-rg \
  --enable-rbac-authorization false

If this happened during Set Up the Key Vault (granting yourself access), run:

az keyvault set-policy \
  --name your-vault-name \
  --object-id $(az ad signed-in-user show --query id -o tsv) \
  --secret-permissions get set list delete

If this happened during Grant Key Vault Access to the App (granting the Managed Identity access), run:

az keyvault set-policy \
  --name your-vault-name \
  --object-id $PRINCIPAL_ID \
  --secret-permissions get list

Key Vault returns SecretNotFound

The secret was never added, was deleted, or its name doesn't match exactly what your code requests — Key Vault secret names are case-sensitive. A secret named db-password and a request for DB-PASSWORD are different names. Run az keyvault secret list --vault-name YOUR_VAULT and compare what's actually in the vault against what loadAllSecrets() is asking for in src/config/secrets.js. Usually, it's a casing issue or a stray hyphen.

Wrapping Up

The .env file in this project contains exactly one value: the Key Vault name. That's not sensitive. Every actual secret — database passwords, API keys, signing secrets — lives in Key Vault and never touches your codebase or your deployment pipeline.

This is the pattern I use on Azure projects now. The startup check is the part I find most useful in practice: if Key Vault is unreachable or a secret is missing, the server exits immediately with a clear error instead of starting up broken and failing on the first real request. You find out right away, rather than getting an obscure database connection error two hours later.

To add another secret, put it in Key Vault and drop its name into the secretNames array — that's it. Everything else scales with it.

The full working code is on GitHub: nodejs-azure-keyvault

This guide covers six of those patterns. For each one, you'll see a real code example that creates the vulnerability, an explanation of what makes it dangerous, and a corrected version with notes on exactly what changed and why.

No security background is required to follow along here. It'll just help to have some familiarity with Node.js and SQL.

Prerequisites

The examples assume you're comfortable with:

• Node.js and Express.js basics

• SQL queries

• How HTTP requests and responses work

• Basic authentication concepts (sessions, tokens)

Note on code examples: Throughout this tutorial, db.query() is a fictional database helper that returns a single row object for SELECT queries (or null if not found), and a result object for INSERT/UPDATE queries. The connection.query() in the race conditions section uses the mysql2 promise API directly, where query() returns [rows, fields]. Adapt the syntax to the database driver you use.

Table of Contents

• 1. Broken Access Control and IDOR

• 2. Mass Assignment

• 3. Prototype Pollution

• 4. Race Conditions

• 5. Business Logic Flaws

• 6. JWT Misconfiguration

• Summary

1. Broken Access Control and IDOR

Broken Access Control has topped the OWASP Top 10 since 2021, and it's not hard to see why. The most common form is Insecure Direct Object Reference (IDOR): the application exposes a database ID in a URL, a user changes the number, and suddenly they're looking at someone else's data.

The fix seems obvious in hindsight. But it keeps appearing in production code because authentication and authorization get conflated. Confirming that a user is logged in is not the same as confirming they're allowed to access a specific resource.

How to Identify This Vulnerability in Your Code

Here's a typical user profile endpoint:

// Express.js - Vulnerable
app.get('/api/users/:id/profile', authenticate, async (req, res) => {
  const userId = req.params.id;

  const user = await db.query(
    'SELECT id, name, email, address FROM users WHERE id = ?',
    [userId]
  );

  if (!user) {
    return res.status(404).json({ error: 'User not found' });
  }

  res.json(user);
});

The authenticate middleware confirms that the request includes a valid token. But it doesn't confirm whether the authenticated user is allowed to access the requested resource.

Any authenticated user can request /api/users/1/profile, /api/users/2/profile, and so on and retrieve other users' data.

Why This Matters

Authentication confirms who you are. Authorization confirms what you're allowed to do. The code above does the first and skips the second entirely.

With a sequential numeric ID, a curious user doesn't need any special tools. They can just change 1 to 2 in the URL. But the same problem exists with UUIDs or slugs if the ownership check is missing.

How to Fix This Vulnerability

Verify server-side that the authenticated user owns — or is explicitly authorized to access — the requested resource:

// Express.js - Secure
app.get('/api/users/:id/profile', authenticate, async (req, res) => {
  // Reject anything that isn't a string of digits — parseInt("12abc") would return 12
  if (!/^\d+$/.test(req.params.id)) {
    return res.status(400).json({ error: 'Invalid user ID' });
  }

  const requestedId = Number(req.params.id);

  if (requestedId < 1) {
    return res.status(400).json({ error: 'Invalid user ID' });
  }

  // The authenticated user's ID is set by the authenticate middleware
  const authenticatedId = req.user.id;

  // Enforce ownership: users can only access their own profile
  if (requestedId !== authenticatedId) {
    return res.status(403).json({ error: 'Forbidden' });
  }

  const user = await db.query(
    'SELECT id, name, email, address FROM users WHERE id = ?',
    [requestedId]
  );

  if (!user) {
    return res.status(404).json({ error: 'User not found' });
  }

  res.json(user);
});

For admin endpoints that legitimately need to access any user, enforce role-based authorization explicitly:

// Admin endpoint with explicit role check
app.get('/api/admin/users/:id', authenticate, requireRole('admin'), async (req, res) => {
  if (!/^\d+$/.test(req.params.id)) {
    return res.status(400).json({ error: 'Invalid user ID' });
  }

  const userId = Number(req.params.id);

  const user = await db.query(
    'SELECT id, name, email, role FROM users WHERE id = ?',
    [userId]
  );

  if (!user) {
    return res.status(404).json({ error: 'User not found' });
  }

  res.json(user);
});

Here's what changed and why:

• /^\d+$/.test(req.params.id) rejects anything that isn't a pure string of digits. parseInt("12abc", 10) would silently return 12 and pass further checks. The regex prevents this.

• Number(req.params.id) converts the already-validated string to a number safely.

• The comparison requestedId !== authenticatedId enforces ownership.

• Admin functionality is a separate endpoint with its own authorization check.

Never infer authorization from a URL parameter. Derive it from the authenticated session.

2. Mass Assignment

Mass assignment is one of those vulnerabilities that's almost invisible when you write it. You're just being efficient, right? Why iterate over fields manually when you can pass the whole object?

The problem is that your database table knows about fields your users were never supposed to touch.

How to Identify This Vulnerability in Your Code

Here's a user profile update endpoint:

// Express.js - Vulnerable
app.put('/api/users/me', authenticate, async (req, res) => {
  const userId = req.user.id;

  // req.body contains everything the client sends
  const updates = req.body;

  await db.query(
    'UPDATE users SET ? WHERE id = ?',
    [updates, userId]
  );

  res.json({ success: true });
});

The users table has these columns:

CREATE TABLE users (
  id           INT PRIMARY KEY,
  name         VARCHAR(100),
  email        VARCHAR(100),
  bio          TEXT,
  role         ENUM('user', 'moderator', 'admin') DEFAULT 'user',
  credits      INT DEFAULT 0,
  is_banned    BOOLEAN DEFAULT false
);

The developer intended users to update name, email, and bio. But role, credits, and is_banned are also in the table — and the query updates whatever fields the client sends.

Why This Matters

That request body goes straight into the SQL query. The users table also has role, credits, and is_banned — and the query doesn't know or care which fields the developer "intended" to expose.

A user who sends this:

{
  "name": "Alice",
  "role": "admin",
  "credits": 100000,
  "is_banned": false
}

Has just promoted themselves to admin, cleared their ban, and given themselves a hundred thousand credits.

How to Fix This Vulnerability

Build the update object yourself, field by field, using an explicit allowlist:

// Express.js - Secure
app.put('/api/users/me', authenticate, async (req, res) => {
  const userId = req.user.id;

  // Only these fields may be updated by the user
  const ALLOWED_FIELDS = ['name', 'email', 'bio'];
  const updates = {};

  for (const field of ALLOWED_FIELDS) {
    if (req.body[field] !== undefined) {
      updates[field] = req.body[field];
    }
  }

  if (Object.keys(updates).length === 0) {
    return res.status(400).json({ error: 'No valid fields provided' });
  }

  // Validate individual fields
  // isValidEmail is a simple helper: /^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(email)
  if (updates.email && !isValidEmail(updates.email)) {
    return res.status(400).json({ error: 'Invalid email format' });
  }

  if (updates.name && (typeof updates.name !== 'string' || updates.name.length > 100)) {
    return res.status(400).json({ error: 'Name must be a string of 100 characters or fewer' });
  }

  await db.query(
    'UPDATE users SET ? WHERE id = ?',
    [updates, userId]
  );

  res.json({ success: true });
});

For admin operations that legitimately update sensitive fields, use a separate endpoint with its own authorization:

// Admin-only endpoint for changing user roles
app.put('/api/admin/users/:id/role', authenticate, requireRole('admin'), async (req, res) => {
  if (!/^\d+$/.test(req.params.id)) {
    return res.status(400).json({ error: 'Invalid user ID' });
  }

  const userId = Number(req.params.id);
  const { role } = req.body;

  const VALID_ROLES = ['user', 'moderator', 'admin'];

  if (!VALID_ROLES.includes(role)) {
    return res.status(400).json({ error: 'Invalid role' });
  }

  await db.query(
    'UPDATE users SET role = ? WHERE id = ?',
    [role, userId]
  );

  res.json({ success: true });
});

Don't spread req.body into a database query. Build the update object field by field.

3. Prototype Pollution

Prototype pollution occurs when untrusted data is merged into an object recursively, allowing an attacker to inject properties into Object.prototype (the base object that every plain JavaScript object inherits from).

The OWASP Top 10 covers this under Software and Data Integrity Failures (A08:2021).

How to Identify This Vulnerability in Your Code

A configuration merge utility that processes user-supplied settings:

// Vulnerable recursive merge function
function mergeConfig(target, source) {
  for (const key of Object.keys(source)) {
    if (typeof source[key] === 'object' && source[key] !== null) {
      if (!target[key]) target[key] = {};
      mergeConfig(target[key], source[key]); // recursive call
    } else {
      target[key] = source[key]; // triggers __proto__ setter via bracket notation
    }
  }
}

app.post('/api/settings', authenticate, (req, res) => {
  const userSettings = {};
  mergeConfig(userSettings, req.body); // merge untrusted input
  applySettings(userSettings);
  res.json({ success: true });
});

Why This Matters

An attacker sends this request body:

{
  "__proto__": {
    "isAdmin": true
  }
}

The recursive mergeConfig function reaches the __proto__ key and executes target['__proto__']['isAdmin'] = true. Because __proto__ is JavaScript's prototype accessor, this writes directly to Object.prototype. After the merge:

const anyObject = {};
console.log(anyObject.isAdmin); // true — inherited from Object.prototype

Every plain object in the running application now inherits isAdmin: true. If any authorization check looks like this:

if (user.isAdmin) { /* grant admin access */ }

That check now passes for every user, regardless of their actual role.

How to Fix This Vulnerability

Store user state server-side and validate each field individually. Never recursively merge untrusted input:

// Express.js - Secure
app.post('/api/settings', authenticate, async (req, res) => {
  // Load current settings from the database — never from the client
  const current = await db.query(
    'SELECT theme, language, notifications FROM user_settings WHERE user_id = ?',
    [req.user.id]
  );

  // Validate each field against an explicit allowlist
  const safeSettings = {
    theme: validateEnum(req.body.theme, ['light', 'dark'], current.theme),
    language: validateEnum(req.body.language, ['en', 'es', 'fr', 'de'], current.language),
    notifications: typeof req.body.notifications === 'boolean'
      ? req.body.notifications
      : current.notifications
  };

  await db.query(
    'UPDATE user_settings SET ? WHERE user_id = ?',
    [safeSettings, req.user.id]
  );

  res.json({ success: true });
});

function validateEnum(value, allowed, defaultValue) {
  return allowed.includes(value) ? value : defaultValue;
}

If you need a merge utility, use Object.create(null) as the base — it has no prototype, so __proto__ can't be polluted — and allowlist keys explicitly:

// Safe merge: base object with no prototype
function safeMerge(allowedKeys, source) {
  const result = Object.create(null); // no prototype = no pollution possible

  for (const key of allowedKeys) {
    if (key in source && typeof source[key] !== 'object') {
      result[key] = source[key];
    }
  }

  return result;
}

Rules:

• Never recursively merge untrusted input into a plain object.

• Store application state server-side. Don't trust clients to carry it.

• Use Object.create(null) for data containers that will hold untrusted keys.

• Validate each field by type and allowed values before using it.

4. Race Conditions

Race conditions are tricky because the code is perfectly correct. It's the timing that breaks it. Two requests arrive at almost the same moment, both check the same condition, both see a valid result, and both proceed. The result is something that should only happen once, happening twice.

This is called a Time-of-Check to Time-of-Use (TOCTOU) problem: the state you checked is no longer the state you're acting on.

How to Identify This Pattern in Your Code

A single-use coupon redemption endpoint:

// Express.js - Vulnerable
app.post('/api/redeem-coupon', authenticate, async (req, res) => {
  const { couponCode } = req.body;
  const userId = req.user.id;

  // Step 1: Check if the coupon is still valid
  const coupon = await db.query(
    'SELECT id, discount_amount, used FROM coupons WHERE code = ? AND used = false',
    [couponCode]
  );

  if (!coupon) {
    return res.status(400).json({ error: 'Invalid or already used coupon' });
  }

  // Time gap: another request can pass Step 1 here before Step 3 runs

  // Step 2: Apply the discount
  await applyDiscountToOrder(userId, coupon.discount_amount);

  // Step 3: Mark coupon as used
  await db.query(
    'UPDATE coupons SET used = true, used_by = ? WHERE code = ?',
    [userId, couponCode]
  );

  res.json({ success: true });
});

Why This Matters

Two requests arrive with the same coupon code at nearly the same time. Both hit Step 1 before either reaches Step 3. Both read used = false. Both apply the discount. One coupon, used twice.

The same window exists anywhere you read-then-write: balance checks before deductions, inventory checks before reservations, vote checks before incrementing. Any of them can be exploited the same way.

How to Fix This Vulnerability

Replace the check-then-act pattern with an atomic operation. An atomic database update guarantees that the condition check and the write happen as a single indivisible unit:

// Express.js - Secure
app.post('/api/redeem-coupon', authenticate, async (req, res) => {
  const { couponCode } = req.body;
  const userId = req.user.id;

  const connection = await db.getConnection();

  try {
    await connection.beginTransaction();

    // Atomic: only one request can update a row where used = false.
    // The database row lock ensures only one request succeeds.
    const [result] = await connection.query(
      `UPDATE coupons
       SET used = true, used_by = ?, used_at = NOW()
       WHERE code = ? AND used = false`,
      [userId, couponCode]
    );

    if (result.affectedRows === 0) {
      await connection.rollback();
      return res.status(400).json({ error: 'Invalid or already used coupon' });
    }

    const [couponRows] = await connection.query(
      'SELECT discount_amount FROM coupons WHERE code = ?',
      [couponCode]
    );
    const coupon = couponRows[0];

    await applyDiscountToOrder(userId, coupon.discount_amount, connection);

    await connection.commit();

    res.json({ success: true, discount: coupon.discount_amount });

  } catch (error) {
    await connection.rollback();
    console.error('Coupon redemption failed:', error);
    res.status(500).json({ error: 'Could not process the coupon' });
  } finally {
    connection.release();
  }
});

The key change is UPDATE ... WHERE code = ? AND used = false. The database acquires a row lock during the update. Only one concurrent request can succeed. The second request finds affectedRows = 0 and returns an error — correctly.

Any time you read a value to make a decision before writing — that's a potential race condition. Make the check and the write atomic.

5. Business Logic Flaws

Business logic flaws are the hardest category to catch. Automated scanners won't find them, and code review might miss them too, because the code works exactly as written. The problem is in what the code was designed to do, not how it does it.

The most common form: trusting the client to send sensible numeric values.

How to Identify This Vulnerability in Your Code

An e-commerce checkout endpoint:

// Express.js - Vulnerable
app.post('/api/checkout', authenticate, async (req, res) => {
  const { items } = req.body;

  let total = 0;
  const processedItems = [];

  for (const item of items) {
    const product = await db.query(
      'SELECT id, price FROM products WHERE id = ?',
      [item.productId]
    );

    if (!product) {
      return res.status(400).json({ error: `Product not found: ${item.productId}` });
    }

    // Trust the quantity value from the client
    const itemTotal = product.price * item.quantity;
    total += itemTotal;

    processedItems.push({ productId: product.id, quantity: item.quantity, price: product.price });
  }

  if (total > 100) {
    total = total * 0.9; // 10% discount
  }

  await createOrder(req.user.id, processedItems, total);
  res.json({ success: true, total });
});

Why This Matters

Problem 1 — Negative quantity: The code multiplies the server's price by the client's quantity without checking that the quantity is positive. A user sends "quantity": -5 on an expensive item. Its contribution to the total becomes negative, reducing the overall total.

Problem 2 — Floating point arithmetic: 0.1 + 0.2 in JavaScript is 0.30000000000000004. Without rounding, financial calculations accumulate errors over time.

Problem 3 — Discount manipulation: If a separate endpoint allows modifying an order after checkout without recalculating the total, a user could add items to earn the discount, then remove items while keeping the discounted price.

How to Fix This Vulnerability

Validate every numeric input and recalculate all totals server-side. Use integer arithmetic (cents) for money to avoid floating point errors:

// Express.js - Secure
app.post('/api/checkout', authenticate, async (req, res) => {
  const { items } = req.body;

  if (!Array.isArray(items) || items.length === 0) {
    return res.status(400).json({ error: 'Cart must contain at least one item' });
  }

  if (items.length > 50) {
    return res.status(400).json({ error: 'Cart cannot contain more than 50 items' });
  }

  let totalCents = 0; // Integer arithmetic avoids floating point errors
  const processedItems = [];

  for (const item of items) {
    if (!/^\d+$/.test(String(item.productId))) {
      return res.status(400).json({ error: `Invalid product ID: ${item.productId}` });
    }

    const productId = Number(item.productId);

    // Validate quantity: must be a string of digits only, between 1 and 10
    // parseInt("3abc", 10) returns 3 — we use regex to prevent this
    if (!/^\d+$/.test(String(item.quantity))) {
      return res.status(400).json({
        error: `Invalid quantity for product ${productId}`
      });
    }

    const quantity = Number(item.quantity);

    if (quantity < 1 || quantity > 10) {
      return res.status(400).json({
        error: `Quantity for product ${productId} must be between 1 and 10`
      });
    }

    const product = await db.query(
      'SELECT id, name, price_cents, stock FROM products WHERE id = ? AND active = true',
      [productId]
    );

    if (!product) {
      return res.status(400).json({ error: `Product not found: ${productId}` });
    }

    if (product.stock < quantity) {
      return res.status(400).json({
        error: `Insufficient stock for "${product.name}"`
      });
    }

    // Use the server's price — never trust a price from the client
    totalCents += product.price_cents * quantity;

    processedItems.push({
      productId: product.id,
      name: product.name,
      quantity,
      unitPriceCents: product.price_cents
    });
  }

  // Integer arithmetic for the discount calculation
  const discountMultiplier = totalCents > 10000 ? 90 : 100; // 10000 cents = $100
  const finalTotalCents = Math.round(totalCents * discountMultiplier / 100);

  await createOrder(req.user.id, processedItems, finalTotalCents);

  res.json({
    success: true,
    total: (finalTotalCents / 100).toFixed(2),
    currency: 'USD'
  });
});

What changed and why:

• Integer (cents) arithmetic eliminates floating point errors. 10000 + 3000 in integer cents is always exact.

• quantity < 1 || quantity > 10 prevents negative quantities and unreasonably large orders.

• items.length > 50 prevents oversized requests.

• product.stock < quantity ensures the order doesn't exceed available inventory.

• All totals, discounts, and final prices are calculated server-side from server-side prices.

Define the valid range for every numeric input. Recalculate every total server-side. The client isn't a trusted source for prices or quantities.

6. JWT Misconfiguration

JWTs are everywhere in Node.js APIs, and the jsonwebtoken library makes them easy to use. That's both good and bad: they're easy to use correctly, but they're also easy to use in ways that look fine until they're not.

The OWASP Top 10 classifies authentication failures under Identification and Authentication Failures (A07:2021). Three misconfigurations show up repeatedly.

How to Identify This Vulnerability in Your Code

Mistake 1 — Algorithm not specified in verify():

// Vulnerable
const decoded = jwt.verify(token, process.env.JWT_SECRET);

Without an algorithms option, some JWT implementations can be tricked into accepting tokens that declare "alg": "none" in their header — meaning no signature is required at all.

Mistake 2 — Weak or hardcoded secrets:

// Vulnerable
const token = jwt.sign({ userId: user.id }, 'secret');

A short or predictable HS256 secret can be brute-forced offline if an attacker obtains a valid token.

Mistake 3 — Sensitive data in the payload:

// Vulnerable
const token = jwt.sign({
  userId: user.id,
  passwordHash: user.passwordHash, // never do this
  role: user.role
}, secret);

JWT payloads are base64-encoded, not encrypted. Anyone who holds the token can decode the payload and read its contents.

How to Fix These Issues

// Secure JWT implementation
const jwt = require('jsonwebtoken');
const crypto = require('crypto');

// Generate a strong secret once and store it as an environment variable:
// node -e "console.log(crypto.randomBytes(64).toString('hex'))"
const JWT_SECRET = process.env.JWT_SECRET;

if (!JWT_SECRET || Buffer.from(JWT_SECRET, 'hex').length < 32) {
  throw new Error('JWT_SECRET must be at least 32 random bytes');
}

function signToken(userId) {
  return jwt.sign(
    { sub: userId },         // 'sub' is the standard claim for the user identifier
    JWT_SECRET,
    {
      algorithm: 'HS256',    // always declare the algorithm explicitly
      expiresIn: '15m',      // short-lived tokens reduce the window if a token is stolen
      issuer: 'your-app-name',
      audience: 'your-app-name'
    }
  );
}

function verifyToken(token) {
  return jwt.verify(token, JWT_SECRET, {
    algorithms: ['HS256'],   // whitelist only the expected algorithm
    issuer: 'your-app-name',
    audience: 'your-app-name'
  });
}

function authenticate(req, res, next) {
  const authHeader = req.headers.authorization;

  if (!authHeader || !authHeader.startsWith('Bearer ')) {
    return res.status(401).json({ error: 'No token provided' });
  }

  const token = authHeader.split(' ')[1];

  try {
    const decoded = verifyToken(token);
    req.user = { id: decoded.sub };
    next();
  } catch (err) {
    return res.status(401).json({ error: 'Invalid or expired token' });
  }
}

A note on token revocation: JWTs are stateless, meaning your server keeps no record of them. This means you can't immediately invalidate a token on logout or account compromise without extra infrastructure.

Common solutions include short-lived access tokens (15 minutes) paired with refresh tokens stored server-side, or a token denylist in a fast store like Redis. Choose the approach that matches your application's requirements.

JWT security checklist:

• Always pass algorithms: ['HS256'] to verify().

• Use a secret of at least 32 random bytes generated with crypto.randomBytes.

• Set short expiration times.

• Store only non-sensitive identifiers (user ID) in the payload — not emails, passwords, or roles.

• Use standard claims: sub, iss, aud, exp.

• Plan your revocation strategy before going to production.

Summary

Here's a quick reference for the six vulnerability categories covered in this tutorial:

None of these vulnerabilities requires a clever attacker. They just need code that trusts the wrong thing at the wrong time.

The patterns worth internalizing: always derive authorization from the session, never from the request. Validate every numeric input with a range. Make financial writes atomic. Build your JWT configuration explicitly.

Understanding how to fix these vulnerabilities is one side of the picture. Understanding how they're actually identified and exploited during a real security assessment is the other.

If you want to go deeper on the offensive side — how penetration testers approach web application targets, what they look for, and how they chain multiple flaws together — this guide to web application attack techniques covers that perspective in detail.

The doc was accurate. It was even well-written. But nobody could find anything in it fast enough for it to be useful.

That's the problem RAG, or Retrieval-Augmented Generation, solves.

The naïve approach is to stuff your entire PDF into a prompt and let the model figure it out. That breaks down fast: context windows overflow, costs spike on every request, and the model loses the thread somewhere in the wall of text.

RAG takes a different approach. Your documents get broken into small chunks upfront. Ask it a question and it digs out the 3 or 4 chunks that best match it — those are what the model actually sees. The model gets a tight, focused context. The answer comes from what your document actually says — not from whatever the LLM memorized during training.

In this tutorial, you'll build that from scratch. Upload any PDF — an API reference, an internal spec, a research paper — and ask questions about it in plain English. The system finds the relevant sections and answers from the document itself, not from general training data.

The stack: Node.js with Express, Google Gemini for embeddings, Groq for text generation, and pgvector running in Docker. Every piece of it is free — no credit card, no trial period.

The complete code is on GitHub at nodejs-rag-chatbot.

Table of Contents

• How RAG Works

• What We're Building

• Prerequisites

• Project Setup

• Set Up Postgres with pgvector Using Docker

• Connect to the Database

• Build the Ingestion Pipeline

• Build the Query Pipeline

• Build the Chat API with Express

• Test the Chatbot

• Troubleshooting

• How to Swap in OpenAI

• What to Build Next

How RAG Works

RAG has two phases, and the code maps directly to both.

Ingestion phase — runs once when you upload a document:

1. Pull the raw text out of the PDF

2. Break it into chunks of 400 to 600 characters each, with a bit of overlap so nothing important gets cut at a boundary

3. Run each chunk through an embedding model, which turns it into a vector (a long list of numbers that captures what the text means)

4. Store each chunk and its vector in Postgres

Query phase — runs every time someone asks a question:

1. Embed the user's question using the same model

2. Search the database for chunks whose vectors are closest to the question vector

3. Take the top 5 matching chunks and assemble them into a context block

4. Send context + question to the LLM and return its answer

The reason this works better than keyword search: the embedding model captures meaning, not just exact words. If your doc says "terminate the process" and the user asks "how do I stop it?", vector similarity finds that match. Regular string matching doesn't.

One thing that trips people up: you must use the same embedding model at query time as you did at ingestion. The model defines the geometric space those vectors live in. Switch models halfway through and the coordinates stop meaning the same thing — you'd be comparing apples to completely different apples.

What We're Building

The architecture is intentionally minimal: two endpoints, with nothing you don't need:

• POST /ingest: accepts a PDF upload, chunks it, embeds each chunk, stores vectors in pgvector

• POST /chat: accepts a question, retrieves the most relevant chunks, returns an LLM-generated answer

The full tech stack:

• Node.js + Express — API layer

• Google Gemini free API — gemini-embedding-001 for embeddings (3,072 dimensions per chunk)

• Groq free API — llama-3.1-8b-instant for text generation

• PostgreSQL + pgvector — vector storage and cosine similarity search, running in Docker

• pdf-parse — extracts raw text from PDF buffers

Gemini handles embeddings and Groq handles generation. Splitting them across two providers isn't arbitrary. Gemini's generation API has a quota limit of zero in certain regions (including Pakistan), while Groq works everywhere with no restrictions. Using Groq for generation means this tutorial runs the same way regardless of where you are.

Prerequisites

Before you start:

• Node.js 20+ installed on your machine

• Docker Desktop running (this is how we'll run Postgres locally)

• A free Google Gemini API key (for embeddings)

• A free Groq API key (for text generation)

How to Get Your Free Gemini API Key

1. Go to aistudio.google.com/app/apikey and sign in with a Google account

2. Click "Create API key"

3. Select "Create API key in new project"

4. Copy the key — it starts with AIzaSy...

No credit card or billing required.

How to Get Your Free Groq API Key

1. Go to console.groq.com and sign up with Google

2. Click "API Keys" in the left sidebar

3. Click "Create API Key", give it a name, copy the key — it starts with gsk_...

Groq is free with generous rate limits and works in all regions.

Project Setup

Create the project directory and initialize it:

mkdir nodejs-rag-chatbot
cd nodejs-rag-chatbot
npm init -y

Install dependencies:

npm install express pg pdf-parse uuid dotenv multer
npm install --save-dev nodemon

A quick note on the packages: multer is what makes file uploads work on the /ingest endpoint. Without it, Express can't parse multipart form data.

pdf-parse does the heavy lifting on PDFs, though watch out for scanned PDFs. Those are just images with no text layer underneath, so you'll get back an empty string.

pg talks to Postgres, uuid gives each row a unique ID, and dotenv loads your keys before the app does anything.

Create a .env in the project root. It needs seven values:

GEMINI_API_KEY=AIzaSy...         ← your Gemini key from Google AI Studio
GROQ_API_KEY=gsk_...             ← your Groq key from console.groq.com
POSTGRES_USER=rag_user
POSTGRES_PASSWORD=rag_pass       ← choose any password, this is local only
POSTGRES_DB=rag_db
DATABASE_URL=postgresql://rag_user:rag_pass@localhost:5432/rag_db
PORT=3000

One thing: the password in POSTGRES_PASSWORD and the one in DATABASE_URL must match exactly. I changed just one of them once and spent way too long debugging a "password authentication failed" error before realising the two values were out of sync.

Update package.json scripts:

"scripts": {
  "start": "node src/index.js",
  "dev": "nodemon src/index.js"
}

Create the src directory:

mkdir src

Your final folder structure will look like this:

nodejs-rag-chatbot/
├── src/
│   ├── index.js        ← Express app entry point
│   ├── db.js           ← Postgres connection and schema setup
│   ├── embeddings.js   ← Gemini embedding + Groq generation
│   ├── ingest.js       ← Document ingestion pipeline
│   └── query.js        ← RAG query pipeline
├── docker-compose.yml
├── .env
└── package.json

Set Up Postgres with pgvector Using Docker

pgvector adds a vector column type to Postgres and the operators needed to search it by similarity. Normally you'd have to install it yourself, but the pgvector/pgvector Docker image ships with it already baked in. Just pull the image and you're good.

Now add docker-compose.yml to the project root:

services:
  postgres:
    image: pgvector/pgvector:pg16
    environment:
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: ${POSTGRES_DB}
    ports:
      - "5432:5432"
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

Those ${VARIABLE} references get swapped out from .env when Compose starts — so docker-compose.yml itself stays clean. This is worth doing from day one. I've seen people skip this and regret it after a repo goes public.

Start it:

docker compose up -d

Connect to the Database

Create src/db.js. This sets up the connection pool and creates the documents table on first run:

const { Pool } = require('pg');

const pool = new Pool({
  connectionString: process.env.DATABASE_URL,
});

async function initDb() {
  await pool.query(`CREATE EXTENSION IF NOT EXISTS vector`);

  await pool.query(`
    CREATE TABLE IF NOT EXISTS documents (
      id UUID PRIMARY KEY,
      content TEXT NOT NULL,
      source TEXT NOT NULL,
      embedding VECTOR(3072)
    )
  `);

  console.log('Database ready');
}

module.exports = { pool, initDb };

The VECTOR(3072) dimension matches the output of Gemini's gemini-embedding-001 model exactly. If you use a different embedding model in the future, check its output dimensions and update this number to match.

Build the Ingestion Pipeline

Start with embeddings.js. This file is the bridge to both external APIs — Gemini for turning text into vectors, Groq for generating the final answer. Keeping both in one place means a single file to touch if you ever swap providers.

src/embeddings.js:

const GEMINI_KEY = process.env.GEMINI_API_KEY;
const GEMINI_BASE = 'https://generativelanguage.googleapis.com/v1/models';

async function embedText(text) {
  const res = await fetch(
    `${GEMINI_BASE}/gemini-embedding-001:embedContent?key=${GEMINI_KEY}`,
    {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ content: { parts: [{ text }] } }),
    }
  );
  const data = await res.json();
  if (!res.ok) throw new Error(JSON.stringify(data));
  return data.embedding.values;
}

async function generateAnswer(context, question) {
  const res = await fetch(
    'https://api.groq.com/openai/v1/chat/completions',
    {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'Authorization': `Bearer ${process.env.GROQ_API_KEY}`,
      },
      body: JSON.stringify({
        model: 'llama-3.1-8b-instant',
        messages: [
          {
            role: 'system',
            content: 'You are a helpful assistant. Answer the question using only the context provided. If the context does not contain enough information, say so clearly.',
          },
          {
            role: 'user',
            content: `Context:\n${context}\n\nQuestion: ${question}`,
          },
        ],
      }),
    }
  );
  const data = await res.json();
  if (!res.ok) throw new Error(JSON.stringify(data));
  return data.choices[0].message.content;
}

module.exports = { embedText, generateAnswer };

We're calling both APIs directly with Node.js's built-in fetch rather than the official SDKs. The reason is practical: Google's Node.js SDK routes requests through the v1beta endpoint by default, and gemini-embedding-001 isn't available there — only on v1. Direct fetch sidesteps that entirely and keeps the dependency count low.

src/ingest.js:

const pdfParse = require('pdf-parse');
const { v4: uuidv4 } = require('uuid');
const { pool } = require('./db');
const { embedText } = require('./embeddings');

function chunkText(text, chunkSize = 500, overlap = 50) {
  const chunks = [];
  let start = 0;

  while (start < text.length) {
    const end = Math.min(start + chunkSize, text.length);
    chunks.push(text.slice(start, end).trim());
    start += chunkSize - overlap;
  }

  return chunks.filter(chunk => chunk.length > 50);
}

async function ingestDocument(buffer, filename) {
  const { text } = await pdfParse(buffer);
  const chunks = chunkText(text);

  console.log(`Processing ${chunks.length} chunks from "${filename}"`);

  for (const chunk of chunks) {
    const embedding = await embedText(chunk);

    await pool.query(
      `INSERT INTO documents (id, content, source, embedding)
       VALUES ($1, $2, $3, $4::vector)`,
      [uuidv4(), chunk, filename, JSON.stringify(embedding)]
    );
  }

  return chunks.length;
}

module.exports = { ingestDocument };

500 characters per chunk, with 50 characters of overlap between neighbours.

Why the overlap? Without it, a sentence that straddles a boundary gets split, half in one chunk, half in the next — and neither piece makes sense on its own when retrieved. The overlap keeps those boundary sentences intact.

For most technical docs, 500 is a good starting point. Dense legal or financial text tends to need something closer to 300.

Build the Query Pipeline

src/query.js:

const { pool } = require('./db');
const { embedText, generateAnswer } = require('./embeddings');

async function queryDocuments(question) {
  const questionEmbedding = await embedText(question);

  const { rows } = await pool.query(
    `SELECT content, source,
            1 - (embedding <=> $1::vector) AS similarity
     FROM documents
     ORDER BY embedding <=> $1::vector
     LIMIT 5`,
    [JSON.stringify(questionEmbedding)]
  );

  if (rows.length === 0) {
    return { answer: 'No relevant documents found.', sources: [] };
  }

  const context = rows.map(r => r.content).join('\n\n---\n\n');
  const answer = await generateAnswer(context, question);

  return {
    answer,
    sources: [...new Set(rows.map(r => r.source))],
    topSimilarity: parseFloat(rows[0].similarity).toFixed(3),
  };
}

module.exports = { queryDocuments };

The <=> operator is pgvector's cosine distance. Semantically similar text produces vectors that point in the same direction — so the distance between them is small. Flip that with 1 - distance and you get a similarity score, where anything close to 1 means a strong match.

I found 0.7 to be a reliable threshold in my testing — chunks above that were almost always relevant. Anything below 0.5 and the retrieval was really stretching, pulling chunks that shared a keyword or two but weren't actually answering the question.

When that happens, the system prompt instruction ("if the context does not contain enough information, say so clearly") becomes important. A well-behaved model will tell the user it doesn't know rather than guess.

We also surface the source filename. Once you've ingested more than one document, users need to know whether that answer came from the architecture spec or the incident report.

Build the Chat API with Express

src/index.js:

require('dotenv').config();
const express = require('express');
const multer = require('multer');
const { initDb } = require('./db');
const { ingestDocument } = require('./ingest');
const { queryDocuments } = require('./query');

const app = express();
const upload = multer({ storage: multer.memoryStorage() });

app.use(express.json());

app.post('/ingest', upload.single('file'), async (req, res) => {
  if (!req.file) {
    return res.status(400).json({ error: 'No file uploaded' });
  }

  if (!req.file.mimetype.includes('pdf')) {
    return res.status(400).json({ error: 'Only PDF files are supported' });
  }

  try {
    const count = await ingestDocument(req.file.buffer, req.file.originalname);
    res.json({ message: `Ingested ${count} chunks from "${req.file.originalname}"` });
  } catch (err) {
    console.error(err);
    res.status(500).json({ error: 'Ingestion failed', detail: err.message });
  }
});

app.post('/chat', async (req, res) => {
  const { question } = req.body;

  if (!question || typeof question !== 'string') {
    return res.status(400).json({ error: 'question is required' });
  }

  try {
    const result = await queryDocuments(question);
    res.json(result);
  } catch (err) {
    console.error(err);
    res.status(500).json({ error: 'Query failed', detail: err.message });
  }
});

const PORT = process.env.PORT || 3000;

initDb().then(() => {
  app.listen(PORT, () => {
    console.log(`RAG chatbot running on port ${PORT}`);
  });
});

memoryStorage() keeps the uploaded file in a buffer instead of writing it to disk. We parse it and store the chunks immediately, so there's nothing to save.

Test the Chatbot

Start the server:

npm run dev

You should see:

Database ready
RAG chatbot running on port 3000

Upload a PDF. Any PDF works. I tested with a copy of a Node.js best practices guide:

# Linux / macOS
curl -X POST http://localhost:3000/ingest -F "file=@your-document.pdf"

# Windows PowerShell
curl.exe -X POST http://localhost:3000/ingest -F "file=@your-document.pdf"

Response:

{ "message": "Ingested 142 chunks from \"your-document.pdf\"" }

Now ask a question:

# Linux / macOS
curl -X POST http://localhost:3000/chat \
  -H "Content-Type: application/json" \
  -d '{ "question": "How should I handle errors in async functions?" }'

# Windows PowerShell
curl.exe -X POST http://localhost:3000/chat -H "Content-Type: application/json" -d "{\"question\": \"How should I handle errors in async functions?\"}"

Response:

{
  "answer": "For async functions in Node.js, wrap your logic in a try/catch block to handle rejected promises. In Express, pass the caught error to next(err) to trigger your error-handling middleware. Alternatively, you can create a wrapper function that wraps any async route handler in a promise and calls next on rejection, keeping your route handlers clean...",
  "sources": ["your-document.pdf"],
  "topSimilarity": "0.841"
}

The topSimilarity score tells you how well the retrieval went. Above 0.7 and the chunks pulled were genuinely relevant. Below 0.5, and the search was struggling: it found something, but not something that actually answers the question.

Try asking about something your PDF doesn't mention. If the system prompt is doing its job, the model should say it doesn't have enough information rather than making something up. That's the behaviour you want in production.

The repo includes two diagnostic scripts that are useful if anything isn't working:

• node test-keys.js — tests both API keys live and reports whether each one succeeds

• node list-models.js — fetches the full list of Gemini models available to your API key

Run these before diving into the troubleshooting section below.

Troubleshooting

Everything in this section is a real error I hit while building this. Nothing hypothetical.

Port 5432 is already in use

Error: bind: address already in use

Something else — probably a local Postgres install — is already on that port. Two fixes are needed. First, in docker-compose.yml:

ports:
  - "5433:5432"

Second, update DATABASE_URL in .env:

DATABASE_URL=postgresql://rag_user:rag_pass@localhost:5433/rag_db

The container itself still listens on 5432 internally. You're just changing which port your machine uses to reach it.

Password authentication failed for user "rag_user"

Error: password authentication failed for user "rag_user"

The password Postgres was initialized with doesn't match what your app is sending. Open .env and compare POSTGRES_PASSWORD with the password embedded in DATABASE_URL. They need to be character-for-character identical.

After fixing the mismatch, the old volume still has the wrong password baked into it. You must destroy it and start fresh:

docker compose down -v
docker compose up -d

The -v flag deletes the data volume. Postgres reinitializes on the next start with the credentials from your current .env.

Gemini model not found (404)

{ "error": { "code": 404, "message": "models/text-embedding-004 is not found" } }

The Google AI model naming has changed. Older tutorials and blog posts reference model names that no longer exist on the v1 endpoint. The correct model for this stack is gemini-embedding-001. That's what this repo uses.

If you want to see every model available to your API key, run:

node list-models.js

That script fetches the live list directly from the API so you're not guessing.

Vector dimension mismatch

ERROR: expected 768 dimensions, not 3072

This error appears when your database table was created with a different dimension count than what your embedding model produces. gemini-embedding-001 outputs 3,072-dimensional vectors. The documents table in this tutorial uses VECTOR(3072) to match.

If you get this error, it means either an old table exists with the wrong dimension, or you changed embedding models without recreating the table. Drop the data volume and restart:

docker compose down -v
docker compose up -d

Vector index dimension limit

ERROR: ivfflat index type only supports up to 2000 dimensions

pgvector's ivfflat and hnsw index types have a maximum dimension of 2000. Since gemini-embedding-001 produces 3,072-dimensional vectors, neither index type works.

This tutorial drops the index and lets pgvector do a full scan — fine for development and any reasonably sized corpus. Scaling to thousands of documents in production? Pick a model under 2000 dimensions. OpenAI's text-embedding-3-small outputs 1536 and plays nicely with both index types.

Port 3000 is already in use

Error: EADDRINUSE: address already in use :::3000

Some other process got there first. Swap the port number in .env:

PORT=3002

Save it and restart the server.

Gemini generation returns quota exceeded (limit: 0)

{ "error": { "status": "RESOURCE_EXHAUSTED", "message": "Quota exceeded for quota metric ... with limit 0" } }

That limit 0 means Google has switched off free generation in your country entirely — not that you've used it up. I hit this myself while testing from Pakistan.

That's exactly why this tutorial uses Groq instead. Make sure GROQ_API_KEY is in your .env and that generateAnswer in src/embeddings.js is pointing at api.groq.com.

To verify both keys work, run:

node test-keys.js

It tests the Gemini embedding endpoint and the Groq generation endpoint independently and reports whether each succeeds.

nodemon doesn't pick up changes to .env

nodemon only watches .js files — .env changes don't trigger a restart. Switch to the terminal running the server and type rs, then hit Enter. That forces a restart and picks up whatever you changed.

curl doesn't work in Windows PowerShell

curl : The term 'curl' is not recognized

or

curl : Cannot bind parameter because parameter 'Method' is specified more than once

PowerShell has a built-in curl alias that points to Invoke-WebRequest — completely different flags, completely different behaviour. Add .exe and you bypass the alias and hit the real binary.

So instead of curl, type curl.exe:

# Ingest
curl.exe -X POST http://localhost:3000/ingest -F "file=@your-document.pdf"

# Chat
curl.exe -X POST http://localhost:3000/chat -H "Content-Type: application/json" -d "{\"question\": \"How do I handle async errors?\"}"

That .exe is the whole fix.

Docker Desktop stopped running

Docker Desktop doesn't start automatically after a reboot on most setups. If your Docker commands suddenly fail with connection errors, that's probably why. Open Docker Desktop, wait until it says "Engine running", then try again.

How to Swap in OpenAI

If you want to use the OpenAI API instead of Gemini, it's three changes.

1. Install the OpenAI SDK:

npm install openai

2. Replace src/embeddings.js entirely:

const OpenAI = require('openai');

const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

async function embedText(text) {
  const result = await client.embeddings.create({
    model: 'text-embedding-3-small',
    input: text,
  });
  return result.data[0].embedding;
}

async function generateAnswer(context, question) {
  const result = await client.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [
      { role: 'system', content: 'Answer only from the context provided. If the context is insufficient, say so.' },
      { role: 'user', content: `Context:\n${context}\n\nQuestion: ${question}` },
    ],
  });
  return result.choices[0].message.content;
}

module.exports = { embedText, generateAnswer };

3. Update the vector dimension in src/db.js:

Open db.js and swap VECTOR(3072) for VECTOR(1536) — that's the output size of text-embedding-3-small. Then kill the volume so the table gets recreated with the right dimensions:

docker compose down -v
docker compose up -d

Nothing else needs touching. The ingestion and query logic works the same regardless of which model you plugged in.

What to Build Next

What you've built works. But there are some gaps that come up quickly once you put it in front of real users.

The most noticeable one is streaming. Right now /chat holds the connection open until Groq finishes generating the full answer, then returns everything at once. On a short question that's fine. On a longer one, the user stares at nothing for a few seconds and wonders if the request hung.

The Groq API supports streaming — add stream: true to the request body and tokens start coming back as they're generated. Piping those through Express with res.write() is maybe 15 minutes of work and the difference in feel is immediate.

Metadata filtering is the second thing you'll want. Once you've loaded more than a few documents, queries bleed across everything: ask about the API spec and you'll get chunks from the onboarding guide too.

The fix is a metadata JSONB column where you store the document ID on ingest, then add WHERE metadata->>'doc_id' = $1 to the similarity query. Expose it as an optional body field on /chat: { "question": "...", "docId": "api-spec-v2" }. Users get scoped results, and you get much cleaner answers.

When your corpus grows into the hundreds of documents, look at re-ranking. Vector similarity retrieval is fast but approximate — it finds chunks that are semantically close to the question, not necessarily the ones that most directly answer it.

The pattern is: retrieve the top 20 by cosine distance, then run a cross-encoder over them to re-score by actual relevance, then take the best 5 from that second pass. LangChain.js has a cross-encoder wrapper if you don't want to implement it yourself.

The last thing most people forget until they actually need it is document management — the ability to list what's ingested, delete a specific file, and re-ingest an updated version.

A DELETE FROM documents WHERE source = $1 handles the delete case. Add a GET /documents endpoint that queries SELECT DISTINCT source FROM documents and you have a complete enough API for real use.

RAG isn't magic. It's a well-scoped retrieval problem combined with a language model that's been told to stay within its lane.

The quality of your answers depends on three things: how cleanly your PDFs parse, how well your chunk size fits the content type, and how clearly your system prompt instructs the model to say "I don't know" rather than guess. Get those right and you've built something genuinely useful: the kind of thing that saves a new engineer's first two weeks.

Maybe it's a different Node version, maybe an environment variable is missing, or maybe a system dependency doesn't match. You spend an hour debugging something that was never actually a code problem.

Docker fixes this at the root. With Docker, you stop shipping just code. The Node version, dependencies, and config all travel inside the container. Your laptop, a CI server, a production VM — it behaves the same on all of them. No more environment surprises.

In this tutorial, we'll go through all this step by step: a multi-stage Dockerfile, using Docker Compose with PostgreSQL for local development, and a GitHub Actions workflow that pushes a fresh image to Docker Hub on every merge to main.

The complete code for this tutorial is available on GitHub.

Table of Contents

1. Prerequisites

2. The Sample Application

3. Writing the Dockerfile

4. The .dockerignore File

5. The .gitignore File

6. Build and Test the Image Locally

7. Docker Compose for Local Development

8. Automate the Build with GitHub Actions

9. Deploying the Image

10. Wrapping Up

Prerequisites

• Node.js 18+

• Docker Desktop, which you can download at docs.docker.com/get-docker. Windows users need WSL 2 before Docker starts. Open PowerShell as Administrator and run wsl --install. After the restart, Docker Desktop will install without issues.

• A GitHub account

• A Docker Hub account (free at hub.docker.com)

• Some Express.js experience helps, but isn't required

The Sample Application

We're building a task management API with Express and PostgreSQL. Keep in mind the app is just a vehicle to teach you how this works. The Dockerfile and pipeline we set up here work the same way for any Node.js project.

Create the project:

mkdir nodejs-docker-cicd && cd nodejs-docker-cicd
npm init -y
npm install express pg dotenv
npm install --save-dev nodemon

Create src/index.js:

const express = require('express');
const { Pool } = require('pg');
require('dotenv').config();

const app = express();
app.use(express.json());

const pool = new Pool({
  host: process.env.DB_HOST,
  port: process.env.DB_PORT,
  database: process.env.DB_NAME,
  user: process.env.DB_USER,
  password: process.env.DB_PASSWORD,
});

// Create table on startup
pool.query(`
  CREATE TABLE IF NOT EXISTS tasks (
    id SERIAL PRIMARY KEY,
    title VARCHAR(255) NOT NULL,
    completed BOOLEAN DEFAULT FALSE,
    created_at TIMESTAMP DEFAULT NOW()
  )
`).catch(console.error);

// Health check — required for Docker HEALTHCHECK and load balancers
app.get('/health', (req, res) => {
  res.json({ status: 'ok', timestamp: new Date().toISOString() });
});

app.get('/tasks', async (req, res) => {
  try {
    const result = await pool.query('SELECT * FROM tasks ORDER BY created_at DESC');
    res.json(result.rows);
  } catch (err) {
    res.status(500).json({ error: err.message });
  }
});

app.post('/tasks', async (req, res) => {
  const { title } = req.body;
  if (!title) return res.status(400).json({ error: 'Title is required' });
  try {
    const result = await pool.query(
      'INSERT INTO tasks (title) VALUES ($1) RETURNING *',
      [title]
    );
    res.status(201).json(result.rows[0]);
  } catch (err) {
    res.status(500).json({ error: err.message });
  }
});

app.patch('/tasks/:id', async (req, res) => {
  const { id } = req.params;
  const { completed } = req.body;
  try {
    const result = await pool.query(
      'UPDATE tasks SET completed = $1 WHERE id = $2 RETURNING *',
      [completed, id]
    );
    if (result.rows.length === 0) return res.status(404).json({ error: 'Task not found' });
    res.json(result.rows[0]);
  } catch (err) {
    res.status(500).json({ error: err.message });
  }
});

const PORT = process.env.PORT || 3000;
app.listen(PORT, () => console.log(`Server running on port ${PORT}`));

Open package.json and update the "scripts" section:

"scripts": {
  "start": "node src/index.js",
  "dev": "nodemon src/index.js"
}

npm start runs the app directly with Node. npm run dev uses nodemon so the server restarts automatically when you edit a file.

For running without Docker, create a .env file:

DB_HOST=localhost
DB_PORT=5432
DB_NAME=tasksdb
DB_USER=postgres
DB_PASSWORD=yourpassword
PORT=3000

Notice that all database credentials come from environment variables rather than being hardcoded. Swap the variables, and the same image runs against your local database or a production one — no code changes needed. The /health endpoint is what Docker pings to know the app is actually handling requests.

Writing the Dockerfile

Before touching the Dockerfile, there are two terms you'll keep seeing. An image is a packaged, immutable version of your app — Node runtime, code, dependencies, everything together in one artifact. A container is a running instance of that image. One image, many containers, any machine.

Here's the Dockerfile we'll use:

# ── Stage 1: Install dependencies ──────────────────────────────────────────
FROM node:18-alpine AS builder

WORKDIR /app

# Copy package files first — Docker caches this layer separately.
# If you only change src code (not package.json), Docker skips npm ci on rebuild.
COPY package*.json ./
RUN npm ci

COPY . .


# ── Stage 2: Production image ───────────────────────────────────────────────
FROM node:18-alpine AS production

# Create a non-root user — running as root inside a container is a security risk
RUN addgroup -g 1001 -S nodejs && \
    adduser -S nodeuser -u 1001

WORKDIR /app

COPY package*.json ./
RUN npm ci --only=production

# Copy only the source code from the builder stage (not node_modules or dev files)
COPY --from=builder /app/src ./src

RUN chown -R nodeuser:nodejs /app
USER nodeuser

EXPOSE 3000

# Docker will ping /health every 30s. If it fails 3 times, the container is marked unhealthy.
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
  CMD wget --no-verbose --tries=1 --spider http://localhost:3000/health || exit 1

CMD ["node", "src/index.js"]

This is a multi-stage build. The first stage (builder) installs everything, including dev dependencies. The second stage (production) starts fresh and only copies what the app needs to run. Nodemon, test frameworks, and anything else dev-only never make it into the final image.

The size difference is real. A node:18 Debian image is over 950MB. Switch to node:18-alpine and cut out the dev dependencies, and the final image lands around 150–200MB instead. A smaller image means faster pushes and faster deploys.

npm ci instead of npm install is a deliberate choice for CI/CD. It reads exact versions from package-lock.json and fails hard if the lockfile doesn't match package.json. Every build on every machine installs the exact same versions — no surprises from a dependency that quietly updated overnight.

The nodeuser account exists because containers run as root by default. That's fine until something goes wrong. A non-root user means that an attacker who gets inside the container can't just do whatever they want.

The .dockerignore File

Create .dockerignore before building:

node_modules
npm-debug.log
.env
.git
.gitignore
README.md
Dockerfile
.dockerignore

The node_modules exclusion is the critical one. Your local modules were compiled for your operating system — macOS or Windows binaries won't work inside a Linux container. Excluding them means Docker installs fresh modules during the build, compiled for the correct platform. Without this exclusion, you'd either copy broken binaries into the image or waste time uploading hundreds of megabytes to the build context.

Never put .env in an image. Passwords, API keys, anything sensitive — those go in at runtime as environment variables, never inside the image itself.

The .gitignore File

One more thing before the first commit: a .gitignore. You don't want node_modules or .env tracked:

node_modules/
.env
.env.local
npm-debug.log*
logs/
.DS_Store
Thumbs.db
.vscode/
.idea/
dist/
build/

Build and Test the Image Locally

Open Docker Desktop first and give it a moment. On Windows, you'll see a whale icon in the taskbar that animates while the engine is starting up. Once it goes still, you're good to run Docker commands. If you try to run Docker before the engine is up, you'll hit this:

ERROR: Error response from daemon: Docker Desktop is unable to start

If that happens, quit Docker Desktop. Open PowerShell as Administrator, run wsl --update, and restart. Then go to Control Panel → Programs → Turn Windows features on or off. Both Hyper-V and Virtual Machine Platform need to be checked. After the restart, Docker Desktop should come up fine.

It's worth knowing about this error too:

docker : The term 'docker' is not recognized as the name of a cmdlet, function,
script file, or operable program.

This means that Docker Desktop isn't running or isn't installed. Open it from the Start menu and wait.

Run the build:

docker build -t nodejs-docker-cicd:latest .

The first time takes roughly 30 seconds since Docker has to pull node:18-alpine from the internet. Once that's cached, subsequent builds are much quicker. Both stages will scroll by:

[+] Building 33.1s (17/17) FINISHED
 => [builder 1/5] FROM docker.io/library/node:18-alpine       20.9s
 => [builder 4/5] RUN npm ci                                   3.5s
 => [production 5/7] RUN npm ci --only=production              3.2s
 => [production 7/7] RUN chown -R nodeuser:nodejs /app         3.2s
 => exporting to image                                         1.5s
 => => naming to docker.io/library/nodejs-docker-cicd:latest     0.0s

When you see (17/17) FINISHED the image is built. Check the size:

docker images nodejs-docker-cicd

IMAGE                     ID             DISK USAGE   CONTENT SIZE
nodejs-docker-cicd:latest   c9eed311d999        198MB         47.5MB

CONTENT SIZE (47.5MB) is the compressed size that gets pushed to Docker Hub. DISK USAGE (198MB) is what it takes up on disk locally. Compare that to a node:18 Debian image at 950MB+, and you can see why the Alpine base and multi-stage approach matter.

On subsequent builds, Docker reuses cached layers. Edit only your source files without touching package.json and the npm ci step gets skipped completely. That 33-second first build becomes 3 seconds.

Docker Compose for Local Development

The app needs a database. Setting up PostgreSQL locally means every developer who clones the repo has to do it, too. Docker Compose handles this: one file defines both services, and one command starts them.

Create docker-compose.yml:

services:
  app:
    build:
      context: .
      target: production
    ports:
      - '3000:3000'
    environment:
      DB_HOST: postgres
      DB_PORT: 5432
      DB_NAME: tasksdb
      DB_USER: postgres
      DB_PASSWORD: postgres
      PORT: 3000
    depends_on:
      postgres:
        condition: service_healthy
    restart: unless-stopped

  postgres:
    image: postgres:15-alpine
    environment:
      POSTGRES_DB: tasksdb
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
    ports:
      - '5432:5432'
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ['CMD-SHELL', 'pg_isready -U postgres']
      interval: 5s
      timeout: 5s
      retries: 5

volumes:
  postgres_data:

A few things worth pointing out. DB_HOST is set to postgres. That's the service name, not localhost. Containers on the same Docker network reach each other by service name. Put localhost there and the app tries to connect to itself.

depends_on with condition: service_healthy holds the app back until Postgres actually passes its health check. Skip this and the app starts, tries to connect to a database that isn't ready yet, and crashes. The health check pings pg_isready every 5 seconds. Once it gets a green response, the app container starts.

The named volume postgres_data keeps your data alive between restarts. Run docker compose down and the data is still there next time. Add --volumes to wipe it clean.

Start both services:

docker compose up --build

You'll see PostgreSQL initialize and then the app start. Once you see Server running on port 3000 in the logs, the stack is up.

Open a second terminal to test — leave the compose logs running in the first one.

Linux/macOS:

curl -X POST http://localhost:3000/tasks \
  -H "Content-Type: application/json" \
  -d '{"title": "Learn Docker"}'

curl http://localhost:3000/tasks

curl http://localhost:3000/health

Windows PowerShell: Typing curl in PowerShell runs Invoke-WebRequest, not actual curl. Run curl.exe instead. For JSON bodies, write to a file first:

'{"title": "Learn Docker"}' | Set-Content body.json
curl.exe -X POST http://localhost:3000/tasks -H "Content-Type: application/json" --data `@body.json

curl.exe http://localhost:3000/tasks

curl.exe http://localhost:3000/health

The backtick before @body.json is necessary. PowerShell would otherwise try to interpret @ as a splatting operator rather than passing it to curl as a filename prefix.

You should see responses like these:

# POST /tasks
{"id":1,"title":"Learn Docker","completed":false,"created_at":"2026-07-09T22:21:17.073Z"}

# GET /tasks
[{"id":1,"title":"Learn Docker","completed":false,"created_at":"2026-07-09T22:21:17.073Z"}]

# GET /health
{"status":"ok","timestamp":"2026-07-09T22:11:44.700Z"}

The task hit PostgreSQL in one container and came back through the app. Ctrl+C in the compose terminal stops both.

Automate the Build with GitHub Actions

The image works locally, so it's time to stop doing this by hand.

Step 1: Create a Docker Hub Access Token

Go to hub.docker.com and then Account Settings → Security → New Access Token. Set permission to Read & Write, as read-only breaks the push. The token appears once, so copy it before closing the page.

Security warning: Don't paste this token into a chat, email, or commit. If you expose it by accident, delete it immediately, then make a new one.

Step 2: Add Secrets to Your GitHub Repository

Head to Settings → Secrets and variables → Actions in your repo and add:

• DOCKERHUB_USERNAME — your Docker Hub username

• DOCKERHUB_TOKEN — paste the token here, nowhere else

If you ran into Error: Username and password required, the secrets either aren't saved yet or the names are typed wrong. Both are case-sensitive.

A Node 20 deprecation warning in the logs is normal. It comes from the Docker actions internally, not your code.

Step 3: Create the Workflow File

Create .github/workflows/docker-publish.yml:

name: Build and Push Docker Image

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

env:
  IMAGE_NAME: ${{ secrets.DOCKERHUB_USERNAME }}/nodejs-docker-cicd

jobs:
  build-and-push:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Log in to Docker Hub
        if: github.event_name != 'pull_request'
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}

      - name: Extract metadata
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: ${{ env.IMAGE_NAME }}
          tags: |
            type=sha,prefix=sha-
            type=raw,value=latest,enable={{is_default_branch}}

      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          target: production
          push: ${{ github.event_name != 'pull_request' }}
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

The login step has if: github.event_name != 'pull_request'. This skips authentication on pull requests. PRs from forks don't have access to your secrets, so trying to log in would just fail. The build still runs on PRs to validate your Dockerfile, but the image isn't pushed.

The metadata action generates two tags on every merge to main: latest and a short commit SHA like sha-a1b2c3d. The SHA tag is what makes rollbacks practical. If latest breaks in production, you can pull any previous sha- tag and you're back to a known-good state in seconds.

The cache-from/cache-to: type=gha lines store Docker's layer cache in GitHub Actions' built-in cache. The first run builds everything from scratch. After that, unchanged layers are pulled from cache rather than rebuilt. On a typical Node.js app this brings build time from 2–3 minutes down to under 30 seconds.

Push and Watch it Run

git add .
git commit -m "Add Docker configuration and GitHub Actions workflow"
git push origin main

Go to your repo's Actions tab. You'll see the workflow running in real time. Each step turns green as it completes:

✅ Checkout code
✅ Set up Docker Buildx
✅ Log in to Docker Hub
✅ Extract metadata
✅ Build and push

Green across the board means your image is live on Docker Hub — two tags, latest and a commit SHA like sha-a1b2c3d. Every push to main from here builds and ships automatically.

Deploying the Image

With your image on Docker Hub, you can deploy it to any infrastructure:

Any VPS or server:

docker pull yourusername/nodejs-docker-cicd:latest
docker run -d -p 3000:3000 \
  -e DB_HOST=your-db-host \
  -e DB_NAME=tasksdb \
  -e DB_USER=postgres \
  -e DB_PASSWORD=yourpassword \
  yourusername/nodejs-docker-cicd:latest

Railway — Connect your Docker Hub image in the Railway dashboard and it deploys on the next push.

Fly.io — Run fly launch pointing at your Dockerfile and Fly handles the rest.

Render — Paste your Docker Hub image URL into the Render service settings.

Each push to main runs the workflow. New image goes to Docker Hub, platform picks it up — that's your deployment handled.

Wrapping Up

What started as a local Node.js app now runs in a container. You get the same behavior on any machine, real PostgreSQL in development, and a pipeline that builds and ships to Docker Hub without you doing anything after the push.

The multi-stage build keeps the image lean — dev tools stay out, non-root user, health check baked in. Compose gets the full stack up with one command for anyone who clones the repo. The SHA tag on every GitHub Actions build means rolling back is just a matter of pulling an older tag.

These same patterns (multi-stage builds, Compose for local development, automated image publishing) are used across the industry for production Node.js deployments. Pick up these patterns once and they follow you to every project.

From here, you can extend the pipeline: drop a test step in before the build, or add multi-platform support if you're targeting ARM. Once Docker Compose starts feeling limiting in production, that's usually when Kubernetes enters the picture.

Role-Based Access Control sounds fancy, but the idea is simple: what you can do depends on who you are, not just that you're logged in. An admin deletes users. An editor creates posts. A regular user just reads. Same app, completely different experience depending on who's asking.

That's what we're building here. A REST API with three roles: JWT to carry those roles on every request, and a pair of middleware functions that check permissions before your route handlers even run. There's no database hit per request, and no if/else soup in your business logic.

By the end, you'll have three working roles (admin, editor, user) each locked to their own endpoints. More importantly, the pattern is transferable: once it clicks, you'll wire it into your next project without needing a tutorial.

Full source code on GitHub: github.com/ziaongit/nodejs-rbac-jwt-api

Table of Contents

• What You'll Learn

• Prerequisites

• What We'll Build

• Project Setup

• Setting Up the In-Memory Data Store

• Building the Auth Routes

• Building the RBAC Middleware

• Building the Protected Routes

• Putting It All Together

• Testing the API

• Key Takeaways

• Conclusion

What You'll Learn

• What RBAC is and how it differs from basic authentication

• How to embed roles in JWT payloads

• How to write reusable Express middleware for token verification and role checking

• How to protect API routes based on user roles

Prerequisites

• Node.js (v18+) installed

• Basic knowledge of Express.js

• Familiarity with how JWTs work (we'll cover the relevant parts)

• npm installed

What We'll Build

We'll build a REST API for a simple content management system with three user roles:

The API will expose these endpoints:

Project Setup

Create a new folder and initialize the project:

mkdir nodejs-rbac-jwt-api
cd nodejs-rbac-jwt-api
npm init -y

Install the dependencies:

npm install express jsonwebtoken bcryptjs dotenv
npm install --save-dev nodemon

Here's what each package does:

• express: web framework for building the API

• jsonwebtoken: creates and verifies JWTs

• bcryptjs: securely hashes passwords

• dotenv: reads your .env file so you're not hardcoding secrets in your source code

Update package.json to add start scripts:

"scripts": {
  "start": "node src/app.js",
  "dev": "nodemon src/app.js"
}

Create the project structure:

nodejs-rbac-jwt-api/
├── src/
│   ├── middleware/
│   │   └── auth.js
│   ├── routes/
│   │   ├── auth.js
│   │   ├── content.js
│   │   └── admin.js
│   ├── data/
│   │   └── users.js
│   └── app.js
├── .env
├── .env.example
└── package.json

Create your .env file:

JWT_SECRET=your_super_secret_key_change_this_in_production
PORT=3000

Important: Never commit your .env file to version control. Add it to .gitignore.

Setting Up the In-Memory Data Store

We don't have a database here, just an array in memory. The point was to keep the focus on RBAC, not spend half the tutorial on database config. In a real project, swap the array for whatever database you're already using.

Create src/data/users.js:

// In-memory users store
// In production, replace this with a real database (MongoDB, PostgreSQL, etc.)
const users = [];

const findUserByEmail = (email) => users.find((u) => u.email === email);
const findUserById = (id) => users.find((u) => u.id === id);
const createUser = (user) => {
  users.push(user);
  return user;
};
const getAllUsers = () => users.map(({ password, ...user }) => user);

module.exports = { findUserByEmail, findUserById, createUser, getAllUsers };

One thing worth noting: getAllUsers uses destructuring to drop the password before returning anything. Never send password fields in API responses, even hashed ones.

Building the Auth Routes

The auth routes handle registration and login. Login is where roles first enter the picture — we embed the user's role directly into the JWT payload.

Create src/routes/auth.js:

const express = require('express');
const bcrypt = require('bcryptjs');
const jwt = require('jsonwebtoken');
const { findUserByEmail, createUser } = require('../data/users');

const router = express.Router();

// POST /api/auth/register
router.post('/register', async (req, res) => {
  const { name, email, password, role } = req.body;

  if (!name || !email || !password) {
    return res.status(400).json({ message: 'Name, email, and password are required' });
  }

  if (findUserByEmail(email)) {
    return res.status(409).json({ message: 'Email already registered' });
  }

  // Only allow valid roles — default to 'user' if none provided
  const validRoles = ['user', 'editor', 'admin'];
  const assignedRole = validRoles.includes(role) ? role : 'user';

  const hashedPassword = await bcrypt.hash(password, 10);

  const newUser = {
    id: Date.now().toString(),
    name,
    email,
    password: hashedPassword,
    role: assignedRole,
  };

  createUser(newUser);

  res.status(201).json({
    message: 'User registered successfully',
    user: {
      id: newUser.id,
      name: newUser.name,
      email: newUser.email,
      role: newUser.role,
    },
  });
});

// POST /api/auth/login
router.post('/login', async (req, res) => {
  const { email, password } = req.body;

  if (!email || !password) {
    return res.status(400).json({ message: 'Email and password are required' });
  }

  const user = findUserByEmail(email);
  if (!user) {
    return res.status(401).json({ message: 'Invalid credentials' });
  }

  const isMatch = await bcrypt.compare(password, user.password);
  if (!isMatch) {
    return res.status(401).json({ message: 'Invalid credentials' });
  }

  // Issue JWT — embed role in the payload
  const token = jwt.sign(
    {
      id: user.id,
      email: user.email,
      role: user.role,   // ← This is the key part for RBAC
    },
    process.env.JWT_SECRET,
    { expiresIn: '24h' }
  );

  res.json({
    message: 'Login successful',
    token,
  });
});

module.exports = router;

The most important line is the JWT payload:

jwt.sign({ id, email, role }, process.env.JWT_SECRET, { expiresIn: '24h' })

By embedding role in the token, every subsequent request carries the user's permissions without requiring a database lookup. The server just verifies the token and reads the role from the payload.

Building the RBAC Middleware

This is the core of the system. We need two separate middleware functions:

1. verifyToken confirms the JWT is valid and attaches the decoded payload to req.user

2. checkRole confirms the user has the required role for a specific route

Keeping them separate gives you flexibility. Some routes only need authentication. Others need both authentication and a specific role.

Create src/middleware/auth.js:

const jwt = require('jsonwebtoken');

// Middleware 1: Verify the JWT token
const verifyToken = (req, res, next) => {
  const authHeader = req.headers['authorization'];
  const token = authHeader && authHeader.split(' ')[1]; // Expects: Bearer <token>

  if (!token) {
    return res.status(401).json({ message: 'Access denied. No token provided.' });
  }

  try {
    const decoded = jwt.verify(token, process.env.JWT_SECRET);
    req.user = decoded; // Attach decoded payload (including role) to request
    next();
  } catch (err) {
    return res.status(403).json({ message: 'Invalid or expired token.' });
  }
};

// Middleware 2: Check if user has one of the required roles
const checkRole = (...allowedRoles) => {
  return (req, res, next) => {
    if (!req.user) {
      return res.status(401).json({ message: 'Not authenticated.' });
    }

    if (!allowedRoles.includes(req.user.role)) {
      return res.status(403).json({
        message: `Access denied. Required role: ${allowedRoles.join(' or ')}. Your role: ${req.user.role}`,
      });
    }

    next();
  };
};

module.exports = { verifyToken, checkRole };

checkRole uses a rest parameter (...allowedRoles) so you can pass in one or multiple roles:

checkRole('admin')                  // only admin
checkRole('editor', 'admin')        // editor or admin
checkRole('user', 'editor', 'admin') // all roles

This makes route definitions clean and readable — the permissions are visible right at the route level.

Building the Protected Routes

Now let's wire up routes that use the middleware.

Create src/routes/content.js:

const express = require('express');
const { verifyToken, checkRole } = require('../middleware/auth');

const router = express.Router();

// In-memory content store
const content = [
  { id: '1', title: 'Getting Started with Node.js', author: 'admin' },
  { id: '2', title: 'Express Middleware Explained', author: 'editor' },
];

// GET /api/content — all authenticated users
router.get('/', verifyToken, checkRole('user', 'editor', 'admin'), (req, res) => {
  res.json({ content });
});

// POST /api/content — editors and admins only
router.post('/', verifyToken, checkRole('editor', 'admin'), (req, res) => {
  const { title } = req.body;

  if (!title) {
    return res.status(400).json({ message: 'Title is required' });
  }

  const newItem = {
    id: Date.now().toString(),
    title,
    author: req.user.email,
  };

  content.push(newItem);
  res.status(201).json({ message: 'Content created', item: newItem });
});

// DELETE /api/content/:id — admin only
router.delete('/:id', verifyToken, checkRole('admin'), (req, res) => {
  const index = content.findIndex((c) => c.id === req.params.id);

  if (index === -1) {
    return res.status(404).json({ message: 'Content not found' });
  }

  content.splice(index, 1);
  res.json({ message: 'Content deleted successfully' });
});

module.exports = router;

Notice how readable each route is:

router.delete('/:id', verifyToken, checkRole('admin'), handler)

You can understand the access control without reading the handler body. This is one of the key advantages of middleware-based RBAC: permissions live at the routing layer, not buried in business logic.

Create src/routes/admin.js:

const express = require('express');
const { verifyToken, checkRole } = require('../middleware/auth');
const { getAllUsers } = require('../data/users');

const router = express.Router();

// GET /api/admin/users — admin only
router.get('/users', verifyToken, checkRole('admin'), (req, res) => {
  res.json({ users: getAllUsers() });
});

module.exports = router;

Putting It All Together

Create src/app.js:

require('dotenv').config();
const express = require('express');

const authRoutes = require('./routes/auth');
const contentRoutes = require('./routes/content');
const adminRoutes = require('./routes/admin');

const app = express();

app.use(express.json());

// Routes
app.use('/api/auth', authRoutes);
app.use('/api/content', contentRoutes);
app.use('/api/admin', adminRoutes);

// Health check
app.get('/', (req, res) => {
  res.json({ message: 'RBAC API is running' });
});

const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
  console.log(`Server running on port ${PORT}`);
});

Testing the API

Start the server:

npm run dev

Step 1: Register Users with Different Roles

Register an admin:

curl -X POST http://localhost:3000/api/auth/register \
  -H "Content-Type: application/json" \
  -d '{"name": "Admin User", "email": "admin@example.com", "password": "password123", "role": "admin"}'

Register an editor:

curl -X POST http://localhost:3000/api/auth/register \
  -H "Content-Type: application/json" \
  -d '{"name": "Editor User", "email": "editor@example.com", "password": "password123", "role": "editor"}'

Register a regular user (no role specified — defaults to user):

curl -X POST http://localhost:3000/api/auth/register \
  -H "Content-Type: application/json" \
  -d '{"name": "Regular User", "email": "user@example.com", "password": "password123"}'

Step 2: Log in and Get a Token

curl -X POST http://localhost:3000/api/auth/login \
  -H "Content-Type: application/json" \
  -d '{"email": "user@example.com", "password": "password123"}'

You'll get a response like:

{
  "message": "Login successful",
  "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."
}

Copy the token.

Step 3: Test Role-based Access

Read content as a regular user (should succeed):

curl http://localhost:3000/api/content \
  -H "Authorization: Bearer YOUR_TOKEN_HERE"

Try creating content as a regular user (should fail — 403):

curl -X POST http://localhost:3000/api/content \
  -H "Authorization: Bearer YOUR_TOKEN_HERE" \
  -H "Content-Type: application/json" \
  -d '{"title": "New Article"}'

Response:

{
  "message": "Access denied. Required role: editor or admin. Your role: user"
}

Now log in as an editor and try the same POST request. It succeeds. Log in as admin and try the DELETE route. Only the admin token will work.

Step 4: Decode the JWT to See the Role

You can paste any token into jwt.io to inspect the payload. You'll see something like:

{
  "id": "1720300000000",
  "email": "admin@example.com",
  "role": "admin",
  "iat": 1720300000,
  "exp": 1720386400
}

The role field is exactly what checkRole reads on every protected request.

Key Takeaways

Roles live in the JWT payload. The role travels with the token — no extra DB call needed every time someone hits a protected route. It gets embedded at login and verified cryptographically on each request.

Middleware is composable. verifyToken and checkRole are separate, reusable functions. You can chain them on any route in any combination.

Permissions are visible at the route level. router.delete('/:id', verifyToken, checkRole('admin'), handler) tells you everything about access control before you even read the handler.

Before you ship this to production:

• The in-memory array was just to keep this tutorial focused — replace it with a real database before anything goes near production. A server restart wipes all your users right now.

• That 24h token expiry is too long. Cut it to 15 minutes and add refresh token rotation. A stolen token becomes useless fast.

• Re-validate roles from the DB on sensitive operations. A role change won't reflect in an existing token until it expires

• HTTPS, always

• If your permission logic grows beyond "check a role", look at casl. It handles attribute-level rules cleanly

Conclusion

The core of it fits in two middleware functions and a JWT payload. I've used this same pattern across several projects. And once you've built it yourself, you'll start spotting it everywhere, because almost every multi-user app needs some version of it.

Full source code on GitHub: github.com/ziaongit/nodejs-rbac-jwt-api

The boring stuff is what gets you, because the boring stuff is what everyone agrees to fix "later," and later has a way of never arriving.

My favorite personal example (favorite in the way a scar is your favorite) was an internal API that compared an access token with a plain equality check and had no limit on request size. It ran fine for a year. It ran fine right up until someone curious discovered they could both fingerprint the token comparison and post a body large enough to make the server sweat.

Neither bug was sophisticated. Both would have been a complete non-event if something had simply refused to let me do the wrong thing in the first place.

This tutorial shows you how to add practical guardrails around every HTTP API you build, regardless of framework. You'll write them by hand, in plain Node.js, with no dependencies. This will let you see exactly what each one does and why.

By the end, you'll have a small server that survives a lot more contact with the public internet than the version most of us shipped early in our careers.

This tutorial uses plain JavaScript so anyone can copy and run it. If you use TypeScript, you can add types afterward. You need Node 22 or newer, a basic understanding of HTTP requests and responses, and a terminal for testing the examples.

What We'll Cover:

1. Prerequisites

2. What You'll Build

3. How to Start with the Naïve Server

4. How to Limit the Request Body

5. How to Time Out Slow Requests

6. How to Parse JSON Safely and Block Prototype Pollution

7. How to Set Security Headers on Every Response

8. How to Compare Secrets in Constant Time

9. How to Validate Input as a Gate, Not a Suggestion

10. How to Fail Without Leaking and Log So You Can See It

11. How to Put It All Together

12. How to Handle CORS Correctly

13. What This Tutorial Doesn't Cover

14. Why Defaults Beat Checklists

15. An Honest Note on Frameworks

16. The Takeaway Checklist

Prerequisites

• Node.js 22 or newer

• Basic familiarity with HTTP requests and responses

• A terminal and curl, Postman, or a similar client

This isn't a guide to making your API unhackable. It's a practical guide to avoiding the easy attacks and building safer defaults.

What You'll Build

A plain Node.js API with request size limits, request timeouts, safe JSON parsing, security headers, timing-safe secret comparison, validation, and error handling.

How to Start with the Naïve Server

Here's the kind of server I wrote when I was younger and braver and wrong. It reads a JSON body and echoes it back. Pretend it's the start of a real API.

import http from "node:http";

const server = http.createServer((req, res) => {
  let body = "";
  req.on("data", (chunk) => (body += chunk));
  req.on("end", () => {
    const data = JSON.parse(body || "{}");
    res.writeHead(200, { "Content-Type": "application/json" });
    res.end(JSON.stringify({ youSent: data }));
  });
});

server.listen(3000, () => console.log("listening on http://localhost:3000"));

It works. You can curl it and it answers. It's also a small disaster, and here's the incomplete list of why:

• It reads an unbounded body into memory. Send it a few gigabytes and you have a denial-of-service with no effort.

• JSON.parse throws on bad input, which here means an uncaught exception that can take the process down.

• There's no timeout. A client that sends one byte per minute can hold a connection hostage.

• It sets no security headers and happily advertises that it's a Node server.

• It parses untrusted JSON straight into an object with no checks, which opens the door to prototype pollution downstream.

You're going to fix each of these issues. The fixes are small. The point is to make them habits, not heroics.

How to Limit the Request Body

The first rule of accepting input from strangers is to decide, in advance, how much input you're willing to accept. If you don't set a limit, the limit is "however much RAM the server has," and someone will find that out for you.

There are two layers here. The first is the Content-Length header, which the client sends to declare how big the body is. You can reject early based on it. But you must never trust it alone, because a client can lie or simply not send it.

The real defense is to count bytes as they stream in and stop the moment they cross your line.

const MAX_BODY_BYTES = 100 * 1024; // 100 KB is plenty for most JSON APIs

function readBody(req, limit = MAX_BODY_BYTES) {
  return new Promise((resolve, reject) => {
    // Cheap early rejection if the client is honest about being too big.
    const declared = Number(req.headers["content-length"]);
    if (Number.isFinite(declared) && declared > limit) {
      reject(httpError(413, "Payload too large"));
      return;
    }

    let size = 0;
    const chunks = [];

    req.on("data", (chunk) => {
      size += chunk.length;
      if (size > limit) {
        reject(httpError(413, "Payload too large"));
        req.destroy(); // stop reading; we are done with this client
        return;
      }
      chunks.push(chunk);
    });

    req.on("end", () => resolve(Buffer.concat(chunks)));
    req.on("error", reject);
  });
}

function httpError(statusCode, message) {
  return Object.assign(new Error(message), { statusCode });
}

A few things worth noticing here. You accumulate Buffer chunks and only join them at the end, rather than concatenating strings, because string concatenation forces an early decode and can mangle multibyte UTF-8 characters that happen to land on a chunk boundary.

You also call req.destroy() as soon as you cross the limit, so you don't keep pulling bytes you've already decided to refuse.

Pick a limit that matches the route. A JSON API that creates a user doesn't need a 50 MB body. A file upload endpoint is a different conversation, and there you would stream to disk or object storage instead of buffering in memory at all. The mistake is having no limit, not having the wrong one.

How to Time Out Slow Requests

Once you have a body limit, the next trick an attacker reaches for is to be slow instead of large. This is the family of attacks named after slowloris, a sad-looking primate that moves very slowly, which is rude to the animal but accurate about the attack.

The idea is to open many connections and feed them bytes at a glacial pace, never finishing, so the server keeps each one alive waiting politely. Do that enough times and you've exhausted the connection pool without sending anything that looks malicious.

Node has built-in defenses for this, and the defaults are generous and worth tightening for an API.

const server = http.createServer(handler);

// Total time allowed to receive the entire request (headers + body).
server.requestTimeout = 30_000; // 30 seconds

// Time allowed to receive just the headers. Slowloris lives here.
server.headersTimeout = 10_000; // 10 seconds

// Idle socket timeout: kill connections that go quiet.
server.setTimeout(60_000);

Those three lines handle the network layer. But there's a second kind of slow: your own handler. A database query that hangs, an outbound call to a third party that never answers, a regular expression that decided to think about its life choices. You want a ceiling on how long a single request is allowed to occupy a worker, and you want to be able to cancel the work when that ceiling is hit.

The modern tool for cancellation in Node is AbortController. Here's a small wrapper that gives every handler a deadline and a signal it can pass down to anything that supports cancellation, like fetch.

function withTimeout(handler, ms = 15_000) {
  return async (req, res) => {
    const controller = new AbortController();
    const timer = setTimeout(() => controller.abort(), ms);
    try {
      await handler(req, res, controller.signal);
    } finally {
      clearTimeout(timer);
    }
  };
}

Now a handler can do await fetch(url, { signal }) and the request gets cut off if it blows the deadline, instead of camping on a worker forever.

The discipline to learn here is that any time you talk to something outside your process, you give it a deadline. Networks fail in the most boring way possible: by hanging, not by erroring. A timeout turns a hang into a clean error you can handle.

How to Parse JSON Safely and Block Prototype Pollution

This is the one people skip because it sounds theoretical, and then it shows up in a CVE with their stack in it.

First, the easy half. JSON.parse throws a SyntaxError on malformed input. In the naïve server, that throw was uncaught and could crash the process. So we wrap parsing and turn a parse failure into a clean 400.

function parseJson(buffer) {
  if (buffer.length === 0) return {};
  let text = buffer.toString("utf8");
  try {
    return JSON.parse(text, reviver);
  } catch {
    throw httpError(400, "Invalid JSON body");
  }
}

Now the interesting half: that reviver argument. Prototype pollution is an attack where a request payload reaches up and modifies Object.prototype, the object that almost every object in your program inherits from. If an attacker can set a property there, they can set it on effectively everything at once.

It's easier to believe once you see it. Here's a recursive merge function, the kind people write all the time to apply updates onto an existing record:

function merge(target, source) {
  for (const key in source) {
    if (source[key] && typeof source[key] === "object") {
      if (!target[key]) target[key] = {};
      merge(target[key], source[key]);
    } else {
      target[key] = source[key];
    }
  }
}

Looks harmless. Now feed it a payload an attacker controls:

const evil = JSON.parse('{"__proto__": {"isAdmin": true}}');
const account = {};
merge(account, evil);

console.log(account.isAdmin);   // undefined, account itself is fine
console.log(({}).isAdmin);      // true  <-- every object is now "admin"

That second line is the horror. You never touched ({}). You polluted the shared prototype, so a brand new empty object now reports isAdmin: true. If somewhere later your code does if (user.isAdmin) on an object that didn't explicitly set that field, congratulations! Everyone is an admin. The __proto__ key tricked the merge into walking up into the prototype that all objects share.

The defense is to refuse the dangerous keys before they ever get into your data. The cleanest way at parse time is the reviver, a function JSON.parse calls for every key as it builds the result. Return undefined for a key and it gets dropped.

const FORBIDDEN_KEYS = new Set(["__proto__", "constructor", "prototype"]);

function reviver(key, value) {
  if (FORBIDDEN_KEYS.has(key)) return undefined;
  return value;
}

That is it. Three key names, blocked at the door, and the merge above becomes harmless because the payload never carries __proto__ past the parser.

For extra defense in depth, you can also build internal lookup objects with Object.create(null), which creates an object with no prototype at all, or use a Map when the keys are user-controlled. And if you want a belt to go with the suspenders, Object.freeze(Object.prototype) early in your process start will make the whole class of attack fail loudly.

I wouldn't rely on freezing alone, because some libraries get unhappy about it, but blocking the keys costs you nothing and should be the default.

How to Set Security Headers on Every Response

Browsers will defend your users for you, but only if you tell them to. That instruction comes as a small set of response headers. For an API that returns JSON, the list is short and the defaults are strict, which is exactly how you want it.

function secureHeaders(res) {
  // Do not let the browser guess content types. Stops a JSON response
  // from being treated as HTML or a script.
  res.setHeader("X-Content-Type-Options", "nosniff");

  // Clickjacking defense: do not allow this response inside a frame.
  res.setHeader("X-Frame-Options", "DENY");
  res.setHeader("Content-Security-Policy", "default-src 'none'; frame-ancestors 'none'");

  // Do not leak the full URL (which may contain ids or tokens) on navigation.
  res.setHeader("Referrer-Policy", "no-referrer");

  // Only meaningful over HTTPS: force HTTPS for two years, including subdomains.
  res.setHeader("Strict-Transport-Security", "max-age=63072000; includeSubDomains");

  // Stop advertising what you are running. Free reconnaissance for nobody.
  res.removeHeader("X-Powered-By");
}

A quick tour, because cargo-culting headers is how you end up with a Content Security Policy that does nothing.

X-Content-Type-Options: nosniff stops the browser from second-guessing your Content-Type, which closes a sneaky path where a response gets reinterpreted as something executable. X-Frame-Options and the frame-ancestors directive both refuse to let your responses be embedded in a frame, which is the heart of clickjacking. For a pure JSON API, default-src 'none' is a strong and appropriate CSP, because an API has no business loading scripts, styles, or images.

Referrer-Policy: no-referrer keeps your URLs, which sometimes carry ids you would rather not gossip about, from being sent to other sites. Strict-Transport-Security only matters once you're on HTTPS, but once you are, it stops downgrade attacks by telling the browser to refuse plain HTTP.

And removing X-Powered-By is a tiny thing that just denies an attacker a free hint about what to throw at you.

The reason to wrap this in a function and call it on every response is that "every response" is the part humans forget. Make it one call you do at the top of the request, before you can get distracted.

How to Compare Secrets in Constant Time

Here's a bug that looks completely fine and is completely broken:

if (providedApiKey === expectedApiKey) {
  // grant access
}

The problem is that === on strings is allowed to be fast. It compares character by character and returns false the instant it finds a mismatch. That means a wrong guess that gets the first character right takes very slightly longer to reject than one that gets it wrong immediately.

That timing difference is tiny, but it's measurable over enough requests, and it lets an attacker recover a secret one character at a time. This is a real attack, it has a name (a timing attack), and the fix is built into Node.

You want a comparison whose running time doesn't depend on where the first difference is. Node gives you crypto.timingSafeEqual for exactly this. It has one sharp edge: it throws if the two buffers have different lengths, and length itself is a leak.

The clean way to handle both problems at once is to hash both inputs to a fixed size first, then compare the hashes.

import { timingSafeEqual, createHash } from "node:crypto";

function safeCompare(a, b) {
  // Hashing normalizes length (so timingSafeEqual is happy) and hides
  // the length of the real secret from a timing observer.
  const ha = createHash("sha256").update(String(a)).digest();
  const hb = createHash("sha256").update(String(b)).digest();
  return timingSafeEqual(ha, hb);
}

Use this anywhere you compare a value a stranger supplied against a secret you hold: API keys, webhook signatures, password reset tokens, session identifiers. The rule of thumb is simple: if being wrong about the comparison would let someone in, don't use ===.

One important caveat so you don't misuse this. For user passwords, don't store them and compare with this function. Passwords get hashed with a slow, purpose-built algorithm so that even if your database leaks, the hashes are expensive to crack.

Node ships scrypt for this in node:crypto, and bcrypt and argon2 are popular libraries. The safeCompare above is for comparing high-entropy secrets like tokens and keys, not for human-chosen passwords.

How to Validate Input as a Gate, Not a Suggestion

Everything so far has been about surviving hostile input at the transport level. Validation is about refusing input that doesn't match the shape your code expects, before that input reaches your business logic.

A surprising amount of "weird production behavior" is just a handler that assumed a field was a string and got an array, or assumed a number and got the string "NaN".

You can hand-roll validation for a small API, and it's healthy to see what that looks like before you reach for a library:

function expect(condition, message) {
  if (!condition) throw httpError(400, message);
}

function parseCreateUser(data) {
  expect(typeof data.email === "string" && data.email.includes("@"), "email is required");
  expect(typeof data.password === "string", "password is required");
  expect(data.password.length >= 12, "password must be at least 12 characters");
  // Return only the fields you actually want. Ignore everything else.
  return { email: data.email, password: data.password };
}

Notice the last line. You build a fresh object with only the fields you asked for, rather than passing data straight through. This quietly closes a mass-assignment hole, where a client sends {"email": "...", "password": "...", "role": "admin"} and a careless handler writes the whole object into the database, role included. If you only copy the fields you meant to accept, the extra ones never matter.

For anything beyond a few routes, a schema library pays for itself fast. Zod and Valibot are the popular choices, and both let you describe the shape once and get validation plus inferred types out of it.

import { z } from "zod";

const CreateUser = z
  .object({
    email: z.string().email(),
    password: z.string().min(12),
  })
  .strict(); // reject unknown keys instead of ignoring them

const result = CreateUser.safeParse(data);
if (!result.success) throw httpError(400, "Validation failed");
const user = result.data;

That .strict() call is doing the same mass-assignment defense as our hand-rolled version, but declaratively. Whether you hand-roll it or use a library, the principle is the same: input is guilty until proven to match a shape you defined on purpose.

How to Fail Without Leaking and Log So You Can See It

Errors are going to happen. The real question is what your server says when they do, and whether you can reconstruct what went wrong afterward.

There are two common ways to get this wrong, and they are opposites. Either you hand attackers a map of your internals, or you blind yourself during an incident.

The leaking version looks like this, and yes, I've shipped it:

catch (err) {
  res.writeHead(500);
  res.end(err.stack); // please do not
}

That stack trace can include file paths, library versions, query fragments, and sometimes secrets that got interpolated into an error message. It's a free briefing for whoever is poking at you.

The rule is blunt: a 500 should tell the client nothing useful, and tell you everything, through your logs.

The opposite failure is hiding the error so thoroughly that when it happens in production at 2am you have nothing to go on.

The fix for both problems is the same small idea: a request id. It's a short unique value you attach to each request, return to the client in a header, and include in every log line for that request. When a user reports "I got an error and it said request abc123," you can find exactly that request in your logs in seconds.

import { randomUUID } from "node:crypto";

function withRequestId(req, res) {
  const requestId = req.headers["x-request-id"] ?? randomUUID();
  res.setHeader("X-Request-Id", requestId);
  return requestId;
}

function log(level, requestId, message, extra = {}) {
  // Structured logs: one JSON object per line, easy to search and ship.
  console.log(
    JSON.stringify({ level, requestId, message, ...extra, at: new Date().toISOString() }),
  );
}

function sendError(res, err, requestId) {
  const status = err.statusCode ?? 500;
  const message = status === 500 ? "Internal Server Error" : err.message;
  if (status === 500) {
    log("error", requestId, "unhandled error", { stack: err.stack });
  }
  if (!res.headersSent) {
    res.writeHead(status, { "Content-Type": "application/json" });
  }
  res.end(JSON.stringify({ error: message, requestId }));
}

The client gets the requestId but never the details. They can quote it to support, but they can't read your stack trace. Accept an incoming X-Request-Id when a trusted upstream set one, so a single request keeps the same id as it moves across your services, but generate your own whenever it is missing. Structured logs, one JSON object per line, are worth the slight ugliness, because they're trivial to filter and feed into a log aggregator, which a pile of freeform console.log calls is not.

One note on environments. It's fine, even helpful, to return richer error detail when you run locally. Just gate it on an explicit environment check and make production the strict default, so the worst outcome of a misconfiguration is too little information leaked, never too much.

How to Put It All Together

None of these guardrails is impressive on its own. The power is in having all of them on, by default, on every route, so that the safe path is the path of least resistance.

Here is the naïve server from the start, rebuilt with everything we covered. It's still tiny. It's just no longer naïve.

import http from "node:http";
import { timingSafeEqual, createHash } from "node:crypto";

const MAX_BODY_BYTES = 100 * 1024;
const FORBIDDEN_KEYS = new Set(["__proto__", "constructor", "prototype"]);

function httpError(statusCode, message) {
  return Object.assign(new Error(message), { statusCode });
}

function reviver(key, value) {
  return FORBIDDEN_KEYS.has(key) ? undefined : value;
}

function readBody(req, limit = MAX_BODY_BYTES) {
  return new Promise((resolve, reject) => {
    const declared = Number(req.headers["content-length"]);
    if (Number.isFinite(declared) && declared > limit) {
      return reject(httpError(413, "Payload too large"));
    }
    let size = 0;
    const chunks = [];
    req.on("data", (chunk) => {
      size += chunk.length;
      if (size > limit) {
        reject(httpError(413, "Payload too large"));
        req.destroy();
        return;
      }
      chunks.push(chunk);
    });
    req.on("end", () => resolve(Buffer.concat(chunks)));
    req.on("error", reject);
  });
}

function parseJson(buffer) {
  if (buffer.length === 0) return {};
  try {
    return JSON.parse(buffer.toString("utf8"), reviver);
  } catch {
    throw httpError(400, "Invalid JSON body");
  }
}

function secureHeaders(res) {
  res.setHeader("X-Content-Type-Options", "nosniff");
  res.setHeader("X-Frame-Options", "DENY");
  res.setHeader("Content-Security-Policy", "default-src 'none'; frame-ancestors 'none'");
  res.setHeader("Referrer-Policy", "no-referrer");
  res.setHeader("Strict-Transport-Security", "max-age=63072000; includeSubDomains");
  res.removeHeader("X-Powered-By");
}

function safeCompare(a, b) {
  const ha = createHash("sha256").update(String(a)).digest();
  const hb = createHash("sha256").update(String(b)).digest();
  return timingSafeEqual(ha, hb);
}

function sendJson(res, status, payload) {
  if (!res.headersSent) {
    res.writeHead(status, { "Content-Type": "application/json" });
  }
  res.end(JSON.stringify(payload));
}

function sendError(res, err) {
  const status = err.statusCode ?? 500;
  // Never leak internal error details on a 500. Log them, do not ship them.
  const message = status === 500 ? "Internal Server Error" : err.message;
  if (status === 500) console.error(err);
  sendJson(res, status, { error: message });
}

const API_KEY = process.env.API_KEY ?? "dev-only-key";

async function handler(req, res) {
  secureHeaders(res);

  // A single protected route, as an example.
  if (req.method === "POST" && req.url === "/users") {
    const provided = req.headers["x-api-key"] ?? "";
    if (!safeCompare(provided, API_KEY)) {
      throw httpError(401, "Unauthorized");
    }

    const data = parseJson(await readBody(req));

    if (typeof data.email !== "string" || !data.email.includes("@")) {
      throw httpError(400, "email is required");
    }
    if (typeof data.password !== "string" || data.password.length < 12) {
      throw httpError(400, "password must be at least 12 characters");
    }

    // Only the fields we asked for. No mass assignment.
    const user = { email: data.email };
    return sendJson(res, 201, { created: user });
  }

  throw httpError(404, "Not found");
}

const server = http.createServer((req, res) => {
  handler(req, res).catch((err) => sendError(res, err));
});

server.requestTimeout = 30_000;
server.headersTimeout = 10_000;
server.setTimeout(60_000);

server.listen(3000, () => console.log("listening on http://localhost:3000"));

Read that top to bottom and notice how the security isn't a separate "security middleware" bolted on at the end. It's woven into the normal flow.

The body has a limit. The JSON is parsed safely. The headers go out every time. The API key check is timing-safe. The validation runs before any logic. The error handler refuses to leak internals. And the whole thing is still small enough to hold in your head, which matters, because security you can't understand is security you'll eventually disable by accident.

Try breaking it. Send a huge body and watch the 413. Send {"__proto__": {"isAdmin": true}} and confirm ({}).isAdmin is still undefined afterward. Send a wrong API key and notice you can't tell from the response time how close you were. That last one is invisible by design, which is the whole point.

How to Handle CORS Correctly

CORS, which stands for Cross-Origin Resource Sharing, is one of the most misunderstood security features in web development, and the misunderstanding is the dangerous kind.

Here's the part people get wrong: CORS doesn't protect your server. It's not a firewall. It's a browser feature that decides whether JavaScript running on one website is allowed to read the response from your API on another. Your server stays perfectly reachable from curl, from Postman, and from any other server, CORS headers or not.

What that means in practice is that the most common "fix" people apply is also the most common mistake:

res.setHeader("Access-Control-Allow-Origin", "*"); // understand this before you ship it

A wildcard says "any website's JavaScript may read my responses." For a genuinely public, read-only API with no credentials, that can be perfectly fine. For anything that uses cookies or returns data tied to a logged-in user, it's a mistake, and browsers will refuse to combine * with credentials anyway.

The correct approach is to allow only the origins you actually trust:

const ALLOWED_ORIGINS = new Set(["https://app.example.com"]);

function applyCors(req, res) {
  const origin = req.headers.origin;
  if (origin && ALLOWED_ORIGINS.has(origin)) {
    res.setHeader("Access-Control-Allow-Origin", origin);
    res.setHeader("Vary", "Origin"); // so a cache does not mix origins up
    res.setHeader("Access-Control-Allow-Credentials", "true");
  }
}

Keep this mental model: CORS loosens the browser's default protection in a controlled way. Setting it to * doesn't make your API more exposed to attacks from other servers, because servers were never restricted in the first place. It makes your data readable by any web page a victim happens to visit, which is a privacy and data-exposure decision, not a "make the console error go away" decision. Decide it on purpose, origin by origin.

What This Tutorial Doesn't Cover

Honesty time, because a tutorial that pretends to be totally complete is doing you a disservice. The guardrails above are the baseline, not the finish line.

Here's what's deliberately out of scope and where to look next:

• Authentication and authorization: You checked one API key. Real apps need sessions or tokens, and a real story for who is allowed to do what. That is a whole topic on its own.

• Rate limiting: A single client shouldn't be able to hammer your login route ten thousand times a minute. In-memory counters work for one instance. Behind a load balancer you need a shared store like Redis.

• Outbound request safety (SSRF): The moment your server makes requests to URLs a user supplied, you have a new attack surface: someone can point you at internal addresses or the cloud metadata endpoint. That deserves its own article.

• TLS: Everything HSTS-related assumes you actually terminate HTTPS somewhere, whether that's the runtime, a reverse proxy, or your platform.

• Logging and monitoring: You can't respond to what you can't see. Structured logs with request ids are the unglamorous foundation of every incident response that went well.

Each of these is a future tutorial, and each one follows the same philosophy as this one: make the safe choice the default, and make the unsafe choice something you have to go out of your way to do.

Why Defaults Beat Checklists

You might be wondering why I keep saying "by default" instead of just handing you a checklist and wishing you luck. The reason is that I've watched a lot of checklists lose to a deadline.

A checklist is a list of things a human has to remember to do, correctly, every single time, forever. That includes the junior dev who joined last week, and the senior dev who's exhausted and shipping a hotfix at midnight.

Security that depends on perfect human memory is security that quietly degrades the moment the team gets busy, which is precisely the moment an attacker is hoping for.

A default is a different kind of thing. A default is what happens when nobody does anything at all. If the safe behavior is the default, then forgetting produces a safe app. If the unsafe behavior is the default, then forgetting produces a vulnerability, and people forget constantly, because they're human and they have forty other things on their plate.

This is exactly why the helpers in this article are wrappers you call once at the top of a request, instead of steps you sprinkle through your handlers and hope you got them all. It's the same reason frameworks that take security seriously turn protections on and make you opt out, rather than leaving them off and making you opt in.

The wording sounds like a small difference. Measured across a real team over a real year, the difference in outcomes is enormous. Design it so the lazy path and the safe path are the same path, and you'll be surprised how secure "lazy" can be.

An Honest Note on Frameworks

If wiring all of this by hand every time sounds tedious, that's exactly the right instinct, and it's why some frameworks ship these protections on by default so you don't have to remember them.

Full disclosure: I maintain one of them, an open-source project called DaloyJS. I'm not here to sell it, and everything in this article is plain Node that works the same no matter what you build on.

I mention it only because the lesson that produced it is the lesson of this whole piece: the defaults are the product. A framework that makes you opt into safety will, statistically, be run by someone who forgot to.

Whether you use a framework or roll your own, copy the helpers above into your project today. They're dependency-free and they will quietly prevent a category of bad days.

The Takeaway Checklist

If you remember nothing else, remember this list and put it somewhere your team will see it:

• Limit the body: Count bytes as they stream, reject past your cap, never trust Content-Length alone.

• Time out everything: Tighten Node's request and header timeouts, and give every outbound call a deadline with AbortController.

• Parse JSON defensively: Catch parse errors into a clean 400, and strip __proto__, constructor, and prototype with a reviver.

• Set security headers on every response: Wrap them in one function so "every response" actually means every response.

• Compare secrets in constant time: Use crypto.timingSafeEqual on hashed inputs, never ===, and use a real password hash for passwords.

• Validate input as a gate: Define the shape on purpose, reject what doesn't match, and copy only the fields you asked for.

None of this is clever. That's the best thing about it. The boring stuff is what gets you, so make the boring stuff automatic, and go spend your cleverness on the parts of your product that actually need it.

But back then, accessing these models wasn't a walk in the park. Our primary option was Midjourney, which meant you had to struggle on Discord, and sometimes you couldn't do anything due to rate limits and servers being very busy.

Accessing image generation back then felt like trying to order a coffee during a flash mob.

Thankfully, the landscape has completely shifted. Today, not only can we run state-of-the-art models like Stable Diffusion on consumer hardware, but we can do it locally, offline, and completely free of charge. We don't need any API keys, there aren't any subscription rate limits, and there's no Discord channels to deal with.

In this tutorial, we'll build a local web application using Node.js, Express, Socket.io, and the QVAC SDK to run a quantized Stable Diffusion 2.1 model.

Table of Contents

• Prerequisites

• What is QVAC?

• How Stable Diffusion Works Under the Hood

• GPU Limitations: Metal, AMD, and the Intel Mac Trap

• The Image Generation Pipeline

• Complete Implementation

• Codebase Breakdown

• Conclusion

• Resources and Further Reading

Prerequisites

To get the most out of this tutorial, you should have a solid foundation in web backend and frontend basics:

• Node.js and ES Modules: Basic familiarity with modern JavaScript modules (import/export), async loops, and event listeners.

• Express and WebSockets: Familiarity with routing static files and sending real-time messages over WebSockets with socket.io.

• HTML and Vanilla CSS: Understanding of basic DOM manipulation and style bindings.

• Development environment: A local machine with Node.js installed.

What is QVAC?

Developed by Tether, QVAC is a family of local inference tools designed to execute machine learning models directly on client hardware.

Instead of routing inference requests to expensive cloud-hosted APIs (such as DALL-E or Midjourney), QVAC bundles pre-compiled machine learning runtimes (like llama.cpp for text, whisper.cpp for transcription, and custom diffusion backends) directly into Node.js, mobile, and desktop runtimes.

Running local AI models with QVAC offers several practical advantages:

• Zero API costs: Generate as many images as your hardware can handle without recurring costs.

• Privacy-first: Prompts and generated images are kept entirely in memory on your local machine.

• Offline independence: Run your application in isolated networks, on flights, or in regions without internet access.

How Stable Diffusion Works Under the Hood

To execute image generation locally without running out of RAM, QVAC leverages a quantized Stable Diffusion 2.1 GGUF model (SD_V2_1_1B_Q8_0).

But how does this actual image generation process work conceptually? Let's make one thing clear: this is not a scientific paper. We aren't going to dive into the underlying multivariable calculus, probability distributions, or stochastic differential equations because I'm not a low-level machine learning researcher (and let's be honest, neither of us wants to stare at Greek symbols and linear algebra formulas on a screen when we could be writing clean JavaScript).

Instead, let's understand how these models work conceptually, using some intuitive developer analogies.

The World-Class Sculptor Analogy

At its core, modern AI image generation turns randomness into reality. Instead of "painting" an image from scratch, pixel-by-pixel, like a human illustrator with a brush, the AI essentially acts as a world-class sculptor, carving an image out of a block of digital static.

The most dominant technology behind this today is Diffusion, which powers models like Stable Diffusion, Midjourney, and Google's Imagen series.

Here is the conceptual step-by-step breakdown of how this block of static turns into art:

1. The Training Phase (Learning the Patterns)

Before a model can generate anything, it has to look at billions of images and their corresponding text descriptions. During this phase, developers do something counterintuitive: they intentionally ruin the images.

• Adding noise: The system takes a clear picture (for example, of a cat) and gradually adds random digital static (noise) pixel-by-pixel until the original image is completely unrecognizable.

• Learning to reverse it: The AI's job is to look at a noisy image and predict exactly how much noise was added at that specific step. By doing this billions of times, it becomes an expert at denoising – that is, turning chaos back into order.

2. Connecting Words to Visuals (CLIP)

To make sure the AI knows what a "cat wearing a top hat" looks like, it uses a text-to-image bridge, often powered by a system called CLIP (Contrastive Language-Image Pre-training).

• CLIP translates human language into a mathematical map (called an embedding).

• In this map, the words "cat" and the actual pixels of a cat sit very close together. This ensures that when you type a prompt, the AI knows exactly which visual concepts to pull from its memory.

3. The Generation Phase (The Reverse Diffusion Loop)

When you type a prompt and hit "Generate," the magic happens in reverse:

• The blank canvas: The AI starts with a canvas of pure, 100% random digital noise (it looks like old television static).

• The prompt guidance: The AI looks at your prompt and uses its text embedding to guide its eye. It looks at the random static and asks, "Where in this mess can I start to see a cat?"

• Step-by-step denoising: The AI subtracts a little bit of noise, sharpening the image slightly. It repeats this loop 20 to 50 times. With every step, fuzzy shapes turn into rough outlines, textures appear, and eventually, a crisp, clean, brand-new image emerges.

Fun fact about seeds: Because the process starts with completely random static every single time, typing the exact same prompt twice will always give you a completely different image (unless you lock down the starting randomness using a specific number called a Seed).

Here's an illustration of denoising with diffusion models:

Latent Diffusion: Keeping it Fast (The VAE)

Generating high-resolution images pixel-by-pixel requires massive computing power. If we tried to do this directly in pixel space on consumer hardware, our computers would melt, and a single generation would take hours.

To fix this, modern models use Latent Diffusion.

Instead of working with the full-sized image, a component called an encoder compresses the image into a smaller, abstract mathematical space (the "latent space"). Think of it as a shrunken playground where all the noisy/denoising math happens. Because this playground is so small, the computations are incredibly fast.

Once the denoising loop finishes in the latent space, another component called the decoder (specifically, a Variational Autoencoder, or VAE) blows it back up into a sharp, high-resolution image for you to see.

Architectures Supported by QVAC

When you run local inference with QVAC, the SDK hooks into optimized, community-maintained C++ backends. QVAC manages the hardware bindings and model lifecycles for different AI modalities:

1. Text generation (llama.cpp): Used for large language models (LLMs) like Llama 3 or Mistral, executing auto-regressive token prediction.

2. Audio transcription (whisper.cpp): Used for highly optimized speech-to-text transcription.

3. Image Generation (stable-diffusion.cpp / sdcpp-generation): Our focus in this tutorial. QVAC supports two distinct approaches for image generation depending on the model architecture you choose:

   • The Bundled Model Approach (Stable Diffusion 1.5/2.1/XL): The traditional approach where the entire pipeline (Text Encoders, VAE, and the main Diffusion UNet) is baked into a single, unified GGUF file (for example, SD_V2_1_1B_Q8_0).

     This is incredibly convenient for local deployments because you only need to manage and load one file to start generating images.

   • The Modular Multi-Model Approach (Flux): Modern architectures like FLUX.1 use a much more complex setup. Instead of a single file, Flux splits its computational brain into separate components. You load a core Diffusion Transformer (DiT) model, but you must also separately load large text encoders (like T5-v1.1-xxl and CLIP-L) and an independent VAE model.

     While this requires more complex orchestration to load multiple GGUF files simultaneously, it provides vastly superior prompt adherence and photorealism by utilizing dedicated, massive text-understanding models.

4. Speech synthesis (TTS): Specialized architectures like Chatterbox (transformer-based zero-shot voice cloning) and Supertonic (diffusion-based speech denoising).

GPU Limitations: Metal, AMD, and the Intel Mac Trap

When running machine learning models locally on Apple Mac hardware, QVAC will try to automatically accelerate execution by compiling compute pipelines for the Metal API to utilize the system's GPU.

If you're on an Apple Silicon Mac (M1, M2, M3, M4, or M5 chip), this works seamlessly, and generation will compile on the Apple Neural Engine and Unified GPU memory in seconds.

But if you're running on an older Intel-based Mac with a discrete AMD Radeon GPU (such as the AMD Radeon Pro 5500M commonly found in 16-inch MacBook Pros), you'll run into a major driver-level limitation:

• The macOS Metal driver for older AMD discrete GPUs doesn't support the modern machine learning compute shaders and matrix reduction operators used by stable-diffusion.cpp.

• When the inference worker attempts to run these unsupported operations, the driver fails to compile the pipeline and triggers a hard C++ crash (SIGABRT) inside the ggml-metal-ops.cpp shader encoder, abruptly exiting the background worker process.

If you hit this hardware roadblock, the default GPU configuration will crash the application every time you trigger an image generation.

To resolve this, you should configure the model to run on the CPU instead by setting the model configuration parameter device to "cpu" and specifying the threads (for example, threads: 4). While generating images on the CPU takes longer than on a GPU, it runs successfully on any machine, regardless of how old or limited its GPU is.

The Image Generation Pipeline

To coordinate the local execution lifecycle, our app sets up a real-time event pipeline:

[Browser Client]                                  [Node.js Server]
       |                                                 |
       | ------ 1. Connects & Checks Model --------->    |
       | <----- 2. Downloads & Loads Model ----------     | (Model Cached locally)
       |                                                 |
       | ------ 3. Submits prompt ("Cozy cabin...") ->  |
       |                                                 |
       |                                                 | === [ QVAC Inference Engine ] ===
       |                                                 | 
       | <----- 4. Denoising Step Updates (e.g. 5/20) -- | (Streams steps in real time)
       |                                                 |
       | <----- 5. Sends final image (Base64 DataURL) -- | (Direct in-memory payload)
       |                                                 |

Complete Implementation

Let's look at the implementation. You can clone the full project repository to follow along, or build it from scratch by creating a project folder, running npm init -y, installing the dependencies (@qvac/sdk, express, socket.io, concurrently), and configuring "type": "module" in your package.json.

1. Server Configuration (server.js)

Create a file named server.js and paste the following implementation:

import express from 'express';
import path from 'path';
import http from 'http';
import { Server } from 'socket.io';
import fs from 'fs';
import { fileURLToPath } from 'url';
import { loadModel, unloadModel, getLoadedModelInfo, diffusion, SD_V2_1_1B_Q8_0 } from "@qvac/sdk";

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

const app = express();
const server = http.createServer(app);
const io = new Server(server);

const PORT = process.env.PORT || 3000;

app.use(express.json());
app.use(express.static(path.join(__dirname, 'public')));

const CONFIG_PATH = path.join(__dirname, '.device-preference.json');

function getPreferredDevice() {
  try {
    if (fs.existsSync(CONFIG_PATH)) {
      const data = JSON.parse(fs.readFileSync(CONFIG_PATH, 'utf8'));
      return data.device || null;
    }
  } catch (err) {
    console.error('Failed to read device preference:', err.message);
  }
  return null;
}

function setPreferredDevice(device) {
  try {
    fs.writeFileSync(CONFIG_PATH, JSON.stringify({ device }), 'utf8');
  } catch (err) {
    console.error('Failed to write device preference:', err.message);
  }
}

// Global model state
let loadedModelId = process.modelId || null;
let modelLoadPercent = 0;
let modelLoadStatus = 'Awaiting trigger...';
let isModelLoading = false;

const modelSize = (SD_V2_1_1B_Q8_0.expectedSize / (1024 * 1024 * 1024)).toFixed(2) + ' GB';

function broadcastModelProgress(percent, status) {
  io.emit('model-download-progress', { percent, status, size: modelSize });
}

io.on('connection', (socket) => {
  console.log('Client connected:', socket.id);

  socket.on('disconnect', () => {
    console.log('Client disconnected:', socket.id);
  });

  // Trigger model download
  socket.on('trigger-model-download', async () => {
    // If already loaded, verify it's still alive in the worker
    if (loadedModelId) {
      try {
        await getLoadedModelInfo({ modelId: loadedModelId });
        socket.emit('model-download-progress', {
          percent: 100,
          status: 'Model fully loaded locally.',
          size: modelSize
        });
        return;
      } catch (err) {
        console.log('Model ID was stale/not found, resetting state and reloading...', err.message);
        loadedModelId = null;
        process.modelId = null;
      }
    }

    // If currently loading, report current progress
    if (isModelLoading) {
      socket.emit('model-download-progress', {
        percent: Math.round(modelLoadPercent),
        status: modelLoadStatus,
        size: modelSize
      });
      return;
    }

    isModelLoading = true;
    modelLoadPercent = 0;
    modelLoadStatus = 'Initiating model download...';
    broadcastModelProgress(modelLoadPercent, modelLoadStatus);

    try {
      console.log('Starting model download...');
      const preferredDevice = getPreferredDevice();
      const loadConfig = { prediction: "v" };
      if (preferredDevice) {
        loadConfig.device = preferredDevice;
        if (preferredDevice === 'cpu') {
          loadConfig.threads = 4;
        }
        console.log(`Using cached device preference: ${preferredDevice}`);
      }

      loadedModelId = await loadModel({
        modelSrc: SD_V2_1_1B_Q8_0,
        modelType: "sdcpp-generation",
        modelConfig: loadConfig,
        onProgress: (p) => {
          modelLoadPercent = p.percentage;
          modelLoadStatus = p.percentage >= 100 ? 'Model fully loaded locally.' : `Downloading model weights... (${p.percentage.toFixed(1)}%)`;
          broadcastModelProgress(Math.round(modelLoadPercent), modelLoadStatus);
        }
      });
      process.modelId = loadedModelId;

      isModelLoading = false;
      console.log('Model loaded successfully. ID:', loadedModelId);
    } catch (err) {
      isModelLoading = false;
      modelLoadPercent = 0;
      modelLoadStatus = 'Failed to load model: ' + err.message;
      console.error('Failed to load model:', err);
      broadcastModelProgress(0, modelLoadStatus);
      socket.emit('error_event', { message: 'Failed to load model: ' + err.message });
    }
  });

  socket.on('generate', async (data) => {
    const { prompt, ratio } = data;
    if (!prompt || prompt.trim() === '') {
      socket.emit('error_event', { message: 'Prompt is required' });
      return;
    }

    if (!loadedModelId) {
      socket.emit('error_event', { message: 'Model is not loaded yet' });
      return;
    }

    const runDiffusion = async (modelIdToUse) => {
      socket.emit('progress', {
        percent: 0,
        status: 'Starting diffusion process...',
        sub: 'DIFFUSION INITIALIZING'
      });

      console.log(`Generating image for prompt: "\({prompt}" with ratio: \){ratio} using model ID: ${modelIdToUse}`);

      const { progressStream, outputs, stats } = diffusion({
        modelId: modelIdToUse,
        prompt,
      });

      // Stream progress steps
      for await (const { step, totalSteps } of progressStream) {
        const percent = Math.round((step / totalSteps) * 100);
        socket.emit('progress', {
          percent,
          status: `Denoising step \({step}/\){totalSteps}...`,
          sub: 'RUNNING DIFFUSION'
        });
      }

      // Resolve output buffers
      const buffers = await outputs;
      if (!buffers || buffers.length === 0) {
        throw new Error('No image buffer returned from diffusion model.');
      }

      // Convert image buffer to a base64 Data URL instead of saving to disk
      const base64Data = Buffer.from(buffers[0]).toString('base64');
      const dataUrl = `data:image/png;base64,${base64Data}`;

      // Emit success
      socket.emit('success', {
        url: dataUrl,
        prompt,
        seed: (await stats).seed || -1
      });

      console.log(`Image generated and emitted successfully as base64 Data URL.`);
    };

    try {
      await runDiffusion(loadedModelId);
    } catch (err) {
      console.error('Image generation failed:', err);

      const isCrash = err.code === 50205 || (err.message && err.message.includes('WORKER_CRASHED'));
      if (isCrash) {
        console.log('Worker crashed during GPU execution. Attempting CPU fallback...');

        // Save device preference so we load CPU directly next time and prevent double loading
        setPreferredDevice('cpu');

        // Reset the stale model state
        loadedModelId = null;
        process.modelId = null;

        socket.emit('progress', {
          percent: 0,
          status: 'GPU driver crashed. Automatically falling back to CPU mode...',
          sub: 'CPU FALLBACK LOADING'
        });

        try {
          console.log('Loading model on CPU...');
          isModelLoading = true;
          modelLoadPercent = 0;
          modelLoadStatus = 'Loading CPU model weights...';
          broadcastModelProgress(modelLoadPercent, modelLoadStatus);

          loadedModelId = await loadModel({
            modelSrc: SD_V2_1_1B_Q8_0,
            modelType: "sdcpp-generation",
            modelConfig: { prediction: "v", device: 'cpu', threads: 4 },
            onProgress: (p) => {
              modelLoadPercent = p.percentage;
              modelLoadStatus = `Loading CPU model weights... (${p.percentage.toFixed(1)}%)`;
              broadcastModelProgress(Math.round(modelLoadPercent), modelLoadStatus);
            }
          });
          process.modelId = loadedModelId;
          isModelLoading = false;
          console.log('Model loaded successfully on CPU. ID:', loadedModelId);

          // Retry diffusion on CPU
          await runDiffusion(loadedModelId);
        } catch (cpuErr) {
          console.error('CPU fallback execution failed:', cpuErr);
          isModelLoading = false;
          socket.emit('error_event', { message: 'Image generation failed on CPU: ' + cpuErr.message });
        }
      } else {
        if (err.message && (err.message.includes('MODEL_NOT_FOUND') || err.message.includes('not found'))) {
          loadedModelId = null;
          process.modelId = null;
          broadcastModelProgress(0, 'Model state lost. Please re-trigger download.');
        }
        socket.emit('error_event', { message: 'Image generation failed: ' + err.message });
      }
    }
  });
});

app.get('*', (req, res) => {
  res.sendFile(path.join(__dirname, 'public', 'index.html'));
});

server.listen(PORT, () => {
  console.log(`Server is running at http://localhost:${PORT}`);
});

// Clean exit handler
async function handleCleanup() {
  const modelId = process.modelId || loadedModelId;
  if (modelId && modelId !== 'mock-model-id') {
    try {
      await unloadModel({ modelId, clearStorage: false });
    } catch (err) {}
  }
  process.exit(0);
}

process.on('SIGINT', handleCleanup);
process.on('SIGTERM', handleCleanup);

2. Frontend Architecture Summary

Since our application runs completely locally, the frontend is a single-page web app built with vanilla HTML, CSS, and client-side JavaScript that communicates with our Express server over Socket.io WebSockets.

Rather than cluttering this tutorial with hundreds of lines of UI templates and style sheets, we'll keep the focus entirely on the backend orchestration. You can grab the complete HTML layout, Tailwind styles, and client script from the GitHub Repository.

Here is a summary of how the client communicates with the server under the hood:

1. Preflight sync (trigger-model-download): As soon as the page loads, the client establishes a WebSocket connection and emits trigger-model-download. The server intercepts this to check if the model is cached/loading, and begins broadcasting progress.

2. Denoising stream (progress): During image generation, the server constantly streams progress events containing denoising statistics (for example Denoising step 12/20...). The client updates the visual progress bar and status labels accordingly.

3. Data URL delivery (success): When the diffusion steps are completed, the server converts the binary image buffer into a Base64 string and emits a success event. The client binds this Base64 Data URL directly to the source of the <img> element for direct local display and instant download.

Codebase Breakdown

Let’s lift the hood on the key mechanisms that make our local offline image generator work smoothly.

1. Multi-Client Model ID Binding (process.modelId)

Quantized weights take a significant amount of memory. Every time we call loadModel(), QVAC boots a separate C++ background process (a Bare worker) to host the GGML runtime.

To prevent spawning multiple processes or loading the 2.3 GB GGUF model multiple times when a client refreshes a page or opens another browser tab, we store the loaded model ID globally on Node’s process object:

let loadedModelId = process.modelId || null;
// ...
process.modelId = loadedModelId;

This acts as a process-wide singleton registry. But using a global variable introduces a challenge: stale worker processes. If a client triggers a model load, gets an ID, and the background worker process later crashes or is killed, process.modelId remains populated with a dead reference.

To resolve this, every time a new client connects and requests a model download trigger, we preflight the model ID using getLoadedModelInfo:

if (loadedModelId) {
  try {
    await getLoadedModelInfo({ modelId: loadedModelId });
    socket.emit('model-download-progress', { percent: 100, status: 'Model fully loaded locally.' });
    return;
  } catch (err) {
    console.log('Model ID was stale, resetting state...', err.message);
    loadedModelId = null;
    process.modelId = null;
  }
}

If the background worker is dead, getLoadedModelInfo throws an error. The catch block intercepts this, wipes the stale references, and safely restarts the loading routine.

[!IMPORTANT] Process singleton integrity: Always preflight model state visibility before initiating inference. Without validation checks, attempting diffusion() on a stale model ID will trigger immediate client-side connection timeouts and silent backend worker failures.

2. In-Memory Image Serialization (Zero Disk Writes)

Writing generated images to the server's hard drive creates significant I/O overhead. It forces you to write custom cron cleanup scripts to delete old image files, and runs the risk of running out of disk space on systems with high user traffic.

Since QVAC’s diffusion() function outputs generated PNG files directly as in-memory binary buffers (Uint8Array), we bypass the local file system entirely. We serialize the binary array into a Base64 string directly in memory:

const base64Data = Buffer.from(buffers[0]).toString('base64');
const dataUrl = `data:image/png;base64,${base64Data}`;

This Data URL is transmitted over WebSockets to the client, which immediately binds it to the image element:

• Zero disk overhead: The server doesn't write a single byte to the hard drive, preserving SSD life and preventing storage bloat.

• Instant delivery: Transmission is handled entirely within network memory buffers, bypassing disk serialization latency.

• Effortless client integration: The client doesn't need to request a static image URL path. It directly renders the Base64 Data URL, allowing users to save or download the image instantly.

3. GPU-to-CPU Fallback & Preference Cache Strategy

One of the biggest challenges with local-first AI is client hardware heterogeneity. For example, older Intel Macs with discrete AMD Radeon GPUs support Apple's Metal framework, but lack the modern tensor reduction operators used by the Stable Diffusion engine, causing a hard C++ crash (SIGABRT) inside ggml-metal-ops.cpp.

To keep the application running and ensure we don't trigger the model loading twice (once on the incompatible GPU on startup, and once on the CPU fallback after the first prompt crash), we use a persistent device preference cache file (.device-preference.json) alongside our C++ worker crash interceptor:

try {
  await runDiffusion(loadedModelId);
} catch (err) {
  const isCrash = err.code === 50205 || err.message.includes('WORKER_CRASHED');
  if (isCrash) {
    // 1. Cache the CPU preference on disk
    setPreferredDevice('cpu');

    // 2. Reset stale references
    loadedModelId = null;
    process.modelId = null;

    // 3. Automatically load the model on CPU with multi-threading
    loadedModelId = await loadModel({
      modelSrc: SD_V2_1_1B_Q8_0,
      modelType: "sdcpp-generation",
      modelConfig: { prediction: "v", device: "cpu", threads: 4 }
    });
    process.modelId = loadedModelId;

    // 4. Transparently retry generation
    await runDiffusion(loadedModelId);
  }
}

This approach utilizes a two-layered defense:

1. Dynamic recovery: If a GPU driver error triggers a crash, the app intercepts it, saves "device": "cpu" to the .device-preference.json file, dynamically reloads the weights into CPU threads, and retries the generation. The client simply sees a status update indicating CPU fallback is occurring, surviving what would otherwise be a fatal crash.

2. Preference persistence: The next time the server starts or a page is loaded, the preflight loading routine reads the cached preference from the disk and loads the CPU model immediately:

const preferredDevice = getPreferredDevice(); // Reads .device-preference.json
const loadConfig = { prediction: "v" };
if (preferredDevice) {
  loadConfig.device = preferredDevice;
  if (preferredDevice === 'cpu') {
    loadConfig.threads = 4;
  }
}
loadedModelId = await loadModel({
  modelSrc: SD_V2_1_1B_Q8_0,
  modelType: "sdcpp-generation",
  modelConfig: loadConfig,
  // ...
});

This prevents the server from making redundant GPU load attempts on subsequent sessions, ensuring that the model is loaded only once and directly onto the correct hardware execution target.

[!WARNING] CPU Fallback Latency: While CPU mode guarantees resilience across older hardware, it uses sequential multi-threaded calculations instead of GPU hardware cores. Consequently, generation times will be significantly longer (typically 1 to 2 minutes on CPU compared to 10 to 15 seconds on a compatible GPU). Make sure to design responsive progress loaders in the UI to manage user expectations during fallback.

Conclusion

Running local-first Stable Diffusion with QVAC gives you absolute control over your inference costs and data privacy. By coupling on-device GGML models with a simple Node.js WebSocket backend, you can build responsive web tools that run completely offline without ever spending money on cloud APIs.

As mobile and desktop system-on-chip architectures continue to pack more neural engines, local-first AI architectures will become an increasingly powerful option for modern developers.

Resources and Further Reading

• QVAC Image-to-Image Generation Documentation

• QVAC SDK GitHub Repository

• Stable Diffusion GGUF models on Hugging Face

• Socket.io WebSockets Guide

• Full codebase