It's based on current OpenAI Codex documentation and Help Center articles. Pricing and plan availability change frequently, so treat the pricing section as a snapshot of the current docs and verify against the official links before making procurement decisions.

What's new (April 2026): OpenAI released GPT-5.5 and GPT-5.5 Pro on April 23–24, 2026. GPT-5.5 is now the flagship general model and is rolling into Codex surfaces. See the new "GPT-5.5: The Newest Release" subsection in Section 2, the full benchmark deep dive in Section 11, and the updated pricing snapshot in Section 7.

Authors: Tatev Aslanyan, Vahe Aslanyan, Jim Amuto | Version: 1.3 — Last updated April 30, 2026

Executive Summary

Codex is OpenAI's coding agent — not a single model, but a product and workflow layer that wraps OpenAI's frontier models with file access, shell execution, sandboxes, approval flows, and code review.

It runs in four surfaces: the CLI, IDE extensions (VS Code, Cursor, Windsurf), the macOS/Windows app, and Codex Cloud for background tasks against GitHub repositories.

The product is included with most paid ChatGPT plans (Plus, Pro, Business, Enterprise/Edu) and, for now, Free and Go with stricter rate limits.

The model layer beneath Codex shifted in April 2026. GPT-5.5 is the new general flagship, with substantial gains on agentic and long-context benchmarks (MRCR v2 at 1M tokens jumped from 36.6% on GPT-5.4 to 74.0% on GPT-5.5. Terminal-Bench 2.0 reaches 82.7%, and hallucination rate dropped roughly 60% versus prior generations). It's also roughly 2× the per-token cost of GPT-5.4, so picking the right model per task now matters more for budget than it did a quarter ago.

For teams adopting Codex, the highest-leverage choices are:

1. Start in the CLI or IDE on small bounded tasks before enabling cloud

2. Use Codex as a pre-merge reviewer in addition to a code generator

3. Keep admin and user access separated through workspace RBAC, and

4. Treat token consumption — not prompt count — as the cost driver.

The 30-60-90 day adoption plan in the appendix gives a phased rollout that surfaces friction early.

This handbook covers what Codex is, how to set it up, how to use it well, how it compares to Claude Code, GitHub Copilot, and self-hosted alternatives. We'll also discuss what it costs, how to govern it in an enterprise, and where it does and does not fit. You'll find a glossary, security checklist, and worked cost example in the appendix.

Table of Contents

Here's What We'll Cover:

1. Executive Summary

2. Prerequisites

3. Section 1: What Codex Is

4. Section 2: Where Codex Fits in the OpenAI Ecosystem

5. Section 3: The Core Surfaces

6. Section 4: Getting Started: Install, Set Up, and Your First Task

7. Section 5: How to Use Codex Effectively

8. Section 6: Difference Between Codex and Other Coding Tools

9. Comparison Matrix

10. Section 7: Pricing and Plan Access

11. Worked Cost Example

12. Section 8: Security, Permissions, and Enterprise Setup

13. Section 9: Best Practices for Teams

14. Section 10: Common Workflows and Examples

15. Section 11: Model Specs and Benchmarks (GPT-5.5 Deep Dive)

16. Section 12: Troubleshooting

17. Section 13: FAQ

18. Section 14: When NOT to Use Codex

19. Section 15: Final Recommendations

20. Section 16: Source References

21. Appendix A: 30-60-90 Day Adoption Plan

22. Appendix B: Glossary

23. Appendix C: Admin Security Checklist

24. Appendix D: Changelog

25. Appendix E: Working with Codex in VS Code

Prerequisites

This handbook is hands-on. To get the most out of it — especially Section 4, Section 5, and Section 10 where you'll install Codex and run real tasks — you should have the following in place.

Background Knowledge You Should Already Have

You don't need to be a senior engineer, but the walkthroughs assume:

• Comfort using the command line. You can cd into a directory, list files, run git commands, and read shell error messages. If you have never opened a terminal, work through a one-hour shell tutorial first.

• Basic Git literacy. You understand commits, branches, pull requests, and the difference between staged and unstaged changes. The Codex workflow centers on producing reviewable diffs, so this is non-negotiable.

• Experience reading code in at least one mainstream language. Codex can work in any language, but the demo repo in Section 4 is a small Python service. If you can read Python, JavaScript, Go, or similar, you'll be fine.

• A mental model of "what an API call costs." Section 7's worked cost example assumes you understand that LLM usage is metered by tokens. If "tokens" is a brand-new concept, skim the OpenAI tokenizer page once before reading Section 7.

If you're an engineering manager, procurement lead, or admin and you only need Section 7, Section 8, and Section 14, you can skip the technical prerequisites and jump straight to those sections.

Tools and Accounts You Need to Install

Before starting Section 4, have the following ready. Approximate setup time: 15–25 minutes if you're starting from scratch.

Verify Your Environment

Run these three commands before you start Section 4. If any of them fails, fix it first.

node --version   # should print v18.x or higher
npm --version    # should print 9.x or higher
git --version    # should print 2.30 or higher

What This Handbook Will Not Teach You

To set expectations honestly, this handbook does not cover:

• How to write production-grade Python, JavaScript, or any specific language. We use small examples to demonstrate Codex behavior, not teach syntax.

• How to design a system architecture from scratch. Section 14 explains why Codex is a poor fit for novel architecture decisions.

• How to administer GitHub at the organization level. Section 8 covers the Codex-specific GitHub Connector setup, but assumes your GitHub org already exists.

• LLM internals (attention, RLHF, and so on). We treat the model as a black box with measurable behavior.

Section 1: What Codex Is

Codex is OpenAI's coding agent. The most important thing to understand is that Codex is not just a single model name. It's a product and workflow layer designed to help people write, review, debug, and ship code faster. In OpenAI's own wording, it's an AI coding agent that can work with you locally or complete tasks in the cloud.

That distinction matters. Most people think of AI in one of two ways:

• A chat model that answers questions.

• A coding assistant that suggests snippets.

Codex is broader than both. It can inspect a repository, edit files, run commands, and execute tests. It can also handle larger chunks of work by taking a prompt or spec and turning it into a task plan, code changes, and reviewable output.

For teams, the cloud-based workflow is especially important because it lets Codex run in the background while engineers stay in flow.

OpenAI's current docs also place Codex alongside a wider set of developer tools: the API, the Responses API, the Agents SDK, MCP tools, and the Codex app. If you are onboarding a team, the easiest mental model is this:

• The models are the engine.

• Codex is the coding product that uses those engines.

• The CLI, IDE extension, web app, and cloud tasks are the ways you interact with it.

Section 2: Where Codex Fits in the OpenAI Ecosystem

OpenAI now offers a layered stack:

• General-purpose frontier models such as GPT-5.5, GPT-5.5 Pro, GPT-5.4, GPT-5.4-mini, and GPT-5.4-nano.

• Codex-specific models such as GPT-5.3-Codex, GPT-5.2-Codex, GPT-5.1-Codex, and codex-mini-latest.

• Product surfaces that package those models into workflows, such as Codex CLI, the Codex app, IDE extensions, cloud tasks, and code review.

The practical difference is simple:

• If you need one-off reasoning, synthesis, or general chat, you may use a general model.

• If you need an agent that should navigate a repository, change files, run tests, and push toward a concrete code outcome, Codex is the purpose-built surface.

OpenAI's current model docs describe GPT-5.4 as the flagship model for complex reasoning and coding. At the same time, Codex-specific model pages describe GPT-5.3-Codex and GPT-5.2-Codex as optimized for agentic coding tasks in Codex or similar environments. That tells you how OpenAI is positioning the stack:

• GPT-5.4 is the general flagship.

• Codex-specific models are tuned for coding workflows.

• Codex the product can switch models depending on the surface and configuration.

If you remember nothing else from this section, remember this: Codex is the workflow. Models are the engine.

GPT-5.5: The Newest Release

OpenAI launched GPT-5.5 on April 23, 2026, with API availability following on April 24, 2026. A higher-tier GPT-5.5 Pro variant shipped alongside it. OpenAI describes GPT-5.5 as their "smartest and most intuitive to use model yet, and the next step toward a new way of getting work done on a computer."

For a Codex user, the practical upshot is short:

1. GPT-5.5 is the new general flagship. Anywhere older docs say "GPT-5.4 is the flagship," read GPT-5.5 going forward. GPT-5.4 remains available as a cheaper default.

2. Codex surfaces will switch over. Expect GPT-5.5 to become selectable (and often the default) inside the CLI, IDE, app, and cloud tasks shortly after launch. Verify the active model in your settings.

3. Pricing has shifted. GPT-5.5 sits well above GPT-5.4 on a per-token basis. See Section 7 before approving budgets.

The full benchmark breakdown, performance highlights, and per-workload guidance for picking GPT-5.5 vs GPT-5.4 vs Codex-specific models are in Section 11: Model Specs and Benchmarks. Read that section once you have the foundational chapters under your belt.

Section 3: The Core Surfaces

Codex currently shows up in a few places, and each one is optimized for a slightly different working style.

Codex CLI

• Official docs: developers.openai.com/codex/cli

• npm package: @openai/codex

• GitHub repo

The CLI is the fastest way to put Codex directly into a terminal session. The docs describe it as OpenAI's coding agent that runs locally from your terminal, can read, change, and run code on your machine, and is open source and written in Rust.

Use the CLI when you want:

• A terminal-first workflow.

• Fast iteration inside an existing repo.

• Fine-grained control over approvals and execution.

• A lightweight path for local coding tasks.

IDE Extension

• Official docs: developers.openai.com/codex/ide

• VS Code Marketplace listing (openai.chatgpt)

The CLI docs and Help Center articles point to the IDE extension for VS Code, Cursor, Windsurf, and other VS Code forks. This is the natural fit when your team lives in an editor and wants Codex embedded in the normal coding flow.

Use the IDE extension when you want:

• Codex close to the files you are already editing.

• Prompting and editing without switching contexts.

• A bridge between human-driven and agent-driven editing.

Codex App

• Help Center: Using Codex with your ChatGPT plan

• Download from chatgpt.com/codex

OpenAI's Help Center says the Codex app is available on macOS and Windows. It is designed for parallel work across projects, with built-in worktree support, skills, automations, and git functionality.

Use the app when you want:

• Multiple Codex agents running in parallel.

• Cloud tasks without bouncing between terminal and editor.

• A project-centric place to assign and monitor tasks.

Codex Cloud

• Official docs: developers.openai.com/codex/cloud

• Web interface: chatgpt.com/codex

Codex cloud is the background execution mode. It runs each task in an isolated sandbox with the repository and environment, and it is intended for reviewable code output rather than direct interactive sessions.

Use Codex cloud when you want:

• Tasks to run while you do something else.

• Sandboxed execution with reviewable diffs.

• Automated code review or repository-level workflows.

Code Review

• Help Center: Codex for code review

• Codex use cases

Codex can also review code inside GitHub. OpenAI describes this as a way to automatically review your personal pull requests or configure reviews at the team level.

Use code review when you want:

• A second set of eyes on pull requests.

• Automated regression or issue spotting before human review.

• Lightweight review coverage across a team.

Section 4: Getting Started: Install, Set Up, and Your First Task

This section walks you end-to-end from "nothing installed" to "Codex just fixed a real bug for me."

We will use a tiny demo repository you build yourself in two minutes — a small Python price-calculator with one obvious bug and one missing test. That gives you a real, reproducible target you can throw away when you're done.

The same walkthrough works for the CLI, the IDE extension, and the app, with notes for each.

If you have existing code you would rather use, skip ahead to Step 4 and point Codex at your own repo. The demo is for readers who want a known-good starting point.

Step 0: Confirm Access

Codex is included with ChatGPT Plus, Pro, Business, and Enterprise/Edu plans. For a limited time, it is also included with Free and Go, with stricter rate limits.

If you are in a team or enterprise workspace, access may also depend on workspace settings and role-based controls. Do not assume that a ChatGPT subscription alone guarantees access in a managed environment — confirm with your admin or look in Codex Cloud settings at chatgpt.com/codex.

Step 1: Install Codex

You have three install paths. Pick one to start; you can add the others later.

Option A: The CLI (recommended for first task)

The CLI is the most direct way to see how Codex behaves. The official docs note that macOS and Linux are first-class, while Windows is experimental and you should use WSL2.

npm i -g @openai/codex
codex --version

If codex --version prints a version number, you are done.

Option B: The VS Code Extension

In VS Code (or Cursor / Windsurf), open the Extensions panel, search for "Codex" by openai, and install it. Or from a terminal:

code --install-extension openai.chatgpt

The Codex panel will appear in the right sidebar after install.

Option C: The Codex App

Download the Codex app for macOS or Windows from chatgpt.com/codex. The app shines when you want parallel tasks, built-in git worktrees, and a project-centric UI. For your very first task it is overkill — start with the CLI or extension.

VS Code users: For a step-by-step guide covering all three VS Code entry points (extension, CLI in the integrated terminal, and browser Codex), see Appendix E: Working with Codex in VS Code.

Step 2: Authenticate

Run codex in a terminal (or open the extension panel). You will be prompted to:

• Sign in with ChatGPT — recommended. Usage is charged against your plan's included Codex credits.

• Sign in with an API key — used when you want metered API billing or your workspace policy requires it.

If you are unsure, pick ChatGPT sign-in.

Step 3: Build the Demo Repo

This is the part most quick-starts skip. Instead of pointing Codex at "any repo," let's create a small, self-contained demo repo with a known bug so you can verify Codex actually fixes it.

In a terminal, run:

mkdir codex-demo && cd codex-demo
git init

Now create three files. First, pricing.py — a small pricing calculator with one off-by-one bug and one missing edge case:

# pricing.py
def apply_discount(price: float, discount_percent: float) -> float:
    """Apply a percentage discount to a price.

    BUG: The discount is applied as a multiplier of (discount_percent / 10)
    instead of (discount_percent / 100). A 20% discount currently doubles
    the price instead of reducing it.
    """
    if discount_percent < 0:
        raise ValueError("discount_percent must be >= 0")
    return price * (1 - discount_percent / 10)


def cart_total(items: list[dict], discount_percent: float = 0) -> float:
    """Compute the total for a list of cart items after a discount."""
    subtotal = sum(item["price"] * item["quantity"] for item in items)
    return apply_discount(subtotal, discount_percent)

Then test_pricing.py — a single passing test plus one that will fail because of the bug:

# test_pricing.py
from pricing import apply_discount, cart_total


def test_no_discount_returns_original_price():
    assert apply_discount(100.0, 0) == 100.0


def test_twenty_percent_discount_on_100_is_80():
    # This will FAIL until the bug in apply_discount is fixed.
    assert apply_discount(100.0, 20) == 80.0


def test_cart_total_with_discount():
    items = [
        {"price": 10.0, "quantity": 2},
        {"price": 5.0, "quantity": 1},
    ]
    # Subtotal is 25.0. With 10% off, expected total is 22.5.
    assert cart_total(items, discount_percent=10) == 22.5

And a tiny README.md:

# codex-demo

A tiny pricing module used to learn the Codex workflow.

Run tests with: `python -m pytest`

Commit the starting state so Codex's diffs are easy to review:

git add .
git commit -m "Initial demo: pricing module with a known bug"

Confirm the bug is real before you ask Codex to fix it:

python -m pytest

You should see two failing tests (test_twenty_percent_discount_on_100_is_80 and test_cart_total_with_discount).

If pytest is not installed: pip install pytest. The full demo needs only Python 3.10+ and pytest.

Step 4: Launch Codex and Run Your First Task

Now point Codex at the demo repo.

From the CLI:

cd codex-demo
codex

When Codex starts, give it a clear, bounded task. Type this prompt exactly:

The test suite has two failing tests. Read pricing.py and test_pricing.py,
identify the root cause, fix the smallest possible thing, then run the tests
to confirm they pass. Explain what you changed and why.

Codex will:

1. Inspect pricing.py and test_pricing.py.

2. Recognize the off-by-one bug (/ 10 should be / 100).

3. Propose a one-line diff.

4. Ask for approval before modifying the file (in the default approval mode).

5. After you approve, run python -m pytest and report that all three tests now pass.

From the VS Code extension: Open the codex-demo folder in VS Code, open the Codex panel in the right sidebar, and paste the same prompt. The diff will appear inline in the editor for you to review and accept.

Step 5: Review the Diff

This is the most important habit to build early. Even though the fix is one character (10 → 100), look at the diff before accepting:

git diff

Read the change. Confirm it matches what Codex described. Run the tests yourself:

python -m pytest

All three should pass. Commit the fix:

git commit -am "Fix off-by-one in apply_discount"

You have just completed the full Codex loop: context → task → change → review → verify. Every bigger task is a longer version of this loop.

Step 6: Try Two More Bounded Tasks

Now that the loop works, try these against the same demo repo:

1. Add an edge case test. Prompt: "Add a test that verifies apply_discount raises a ValueError when discount_percent is negative. Run the tests after."

2. Add a missing safety check. Prompt: "apply_discount does not currently reject discount_percent values greater than 100, which would produce a negative price. Add validation, update the existing tests if needed, and add a new test for the new behavior."

Each task is small, has a clear acceptance criterion (the tests pass), and produces a reviewable diff. That is the shape of every good Codex task.

Step 7 (Optional): Set Up Codex Cloud

Cloud tasks let Codex run in the background while you do other work. They require a GitHub-hosted repository.

To enable Codex Cloud against the demo repo:

1. Push codex-demo to a private GitHub repo: gh repo create codex-demo --private --source=. --push (requires the gh CLI).

2. Visit chatgpt.com/codex and connect the ChatGPT GitHub Connector.

3. Allow the codex-demo repository in the connector. Do not grant org-wide access by default — see Appendix C.

4. From the web interface, pick the repo and prompt: "Add type hints to every function in pricing.py and add a CI-style summary of what changed."

5. Wait for the sandbox to finish, review the diff in the browser, and either accept it or open a PR.

By default, Codex Cloud sandboxes have no internet access. That is deliberate — admins can allowlist dependency registries and trusted sites if a real workflow needs them.

When to Use Which Surface

After completing the demo, the surface trade-offs become concrete:

• CLI — fastest for terminal-heavy local work, scriptable, best for multi-step agentic tasks with explicit approvals.

• VS Code extension — lowest friction for in-flow editing while you are already in the editor.

• Codex app — best when you want to run multiple parallel tasks across projects with worktree isolation.

• Codex Cloud — best for background work, long-running tasks, and PR-style review you can leave running.

Most experienced users have all of them installed and pick per task. A single workflow rarely fits every kind of work.

What If Something Doesn't Work?

If you get stuck during this walkthrough:

• codex command not found → npm's global bin is not on your PATH. Restart your terminal, or use a Node version manager like nvm.

• Sign-in keeps failing → confirm the email matches your ChatGPT plan; in enterprise workspaces, your admin must enable Codex.

• Codex won't modify the file → you may be in a strict approval mode. Approve when prompted, or relax the mode after your first successful task.

• Windows misbehavior → switch to a WSL2 terminal. Native Windows for the CLI is experimental.

The full troubleshooting guide is in Section 12.

Section 5: How to Use Codex Effectively

Codex works best when you treat it like a developer you're onboarding rather than a magic prompt responder. The more concrete your task, the better the result.

Each tip below has a bad example (what people actually type) and a good example (what produces a useful result). Most use the codex-demo repo from Section 4 so you can run them yourself.

Give It a Real Objective

A "real objective" means a concrete goal with a verifiable outcome — not a feeling.

Bad:

Improve this codebase.

Codex will pick something to do, but you have no way to know if the result is what you wanted, and the diff will probably touch more than you can review.

Good:

Refactor cart_total in pricing.py so the iteration logic and the discount
application are in two separate helper functions. Keep the public signature
of cart_total unchanged. Add tests for each helper. Run pytest at the end.

This works because there is exactly one acceptance criterion (tests pass with the new structure) and exactly one boundary (public signature unchanged). You can review the diff in 30 seconds.

Other shapes that work:

• "Fix the failing test in test_pricing.py::test_twenty_percent_discount_on_100_is_80."

• "Add a currency: str = 'USD' parameter to cart_total and update the tests."

• "Review the changes in my last commit for missing edge cases."

Provide the Right Context

Codex can inspect the repo, but you still need to steer it to the right files and constraints. Without that, it wanders.

Bad:

Add validation to the pricing module.

What kind of validation? On which inputs? What error class? Codex has to guess all of that.

Good:

Context:
- File: pricing.py
- Function: apply_discount
- Current behavior: raises ValueError for negative discount_percent.
- Desired behavior: also raise ValueError when discount_percent > 100,
  with the message "discount_percent must be between 0 and 100".

Task:
- Add the validation.
- Add a matching test in test_pricing.py.
- Do not change apply_discount's public signature.
- Run pytest after.

Notice the structure: what file, current behavior, desired behavior, task, constraints, how to verify. That is the difference between a hopeful prompt and a usable spec.

For larger tasks, also include:

• A link to the issue or spec (Codex can fetch it if web access is enabled).

• The names of related files even if Codex could find them itself — naming them halves the time-to-first-edit.

• The name of any test command, build command, or lint that should pass.

Ask for Intermediate Thinking When Needed

"Intermediate thinking" means asking Codex to plan in writing before it edits files. The default is for Codex to dive straight to code. For anything larger than a single function, that is the wrong default.

Without intermediate thinking (the alternative):

Refactor pricing.py to support multiple currencies.

Codex starts editing immediately. You discover after the fact that it changed the database schema, the API contract, and three test files — and you have no idea whether the design choice it made was the right one.

With intermediate thinking:

I want to add multi-currency support to pricing.py.

Before editing anything:
1. List the files you expect to touch and why.
2. Outline the approach in 5-10 bullets.
3. Call out any assumptions you are making and any open questions.
4. Identify the riskiest part of the change.

Wait for my approval before making any edits.

Now you get a plan you can review, push back on, or scrap entirely — at zero cost to the codebase. After you approve, Codex executes against the plan it just wrote, which makes the resulting diff predictable.

Use intermediate thinking whenever the task is:

• Multi-file or cross-cutting.

• Architecturally novel for this codebase.

• Hard to test (so the diff is your only signal).

• High blast-radius if wrong (auth, payments, data migrations).

Prefer Bounded Changes

A bounded change is one with all four of these properties:

1. Small surface area — touches one file, one module, or one logical concept.

2. Clear acceptance criterion — there's a specific test, output, or behavior that proves it worked.

3. Reviewable in a few minutes — a human can read the diff and form an opinion without setting aside an hour.

4. Easily revertible — if it goes wrong, git revert undoes it cleanly without breaking anything else.

The opposite is an unbounded change: "make the codebase faster," "modernize the API," "add types everywhere." These have no clear endpoint, no easy verification, and no clean revert path.

Bounded examples (good):

• "Add a serialize() method to CartItem that returns a dict suitable for JSON encoding. Add a test."

• "In apply_discount, replace the magic number 100 with a module-level constant MAX_DISCOUNT_PERCENT."

• "The cart_total function takes a discount_percent keyword argument that defaults to 0. Make the default None and treat None as 'no discount.' Update the tests."

Unbounded examples (avoid):

• "Make pricing.py production-ready."

• "Add proper error handling everywhere."

• "Improve the architecture."

When you catch yourself writing an unbounded prompt, break it into a list of bounded ones before sending. The decomposition itself is most of the work; once you have it, Codex is good at executing each piece.

Use Reviews as a Loop

Codex is not just for writing code — it is also a useful pre-merge reviewer. The loop is:

1. You (or Codex) write the change.

2. Ask Codex to review it.

3. Fix the issues it finds.

4. Re-run tests.

What this looks like in practice:

After completing a task in codex-demo, ask Codex to review your own commit:

Review the change in my last commit (git show HEAD) for:
- correctness issues (off-by-one, type mismatches, wrong defaults)
- missing tests, especially edge cases
- security concerns (input validation, injection, unsafe defaults)
- maintainability risks (unclear naming, hidden coupling)

Prioritize findings by severity (critical / important / nit). For each
finding, point to the exact line and propose a concrete fix. Do not
modify any files in this turn — just produce the review.

You will typically get back a structured response like:

CRITICAL: line 14 — apply_discount accepts NaN silently because the type
  check is `discount_percent < 0`, which is False for NaN. Fix: add an
  explicit math.isnan() check before the comparison.

IMPORTANT: test_pricing.py has no test for the boundary discount_percent=100.
  Fix: add a test asserting apply_discount(100, 100) == 0.

NIT: line 8 — the docstring mentions a "BUG" comment that should be removed
  now that the bug is fixed.

Then you triage: fix the critical and important findings (often by feeding them back to Codex with "apply the fixes you proposed"), defer or reject the nits, and re-run tests.

This converts Codex from a code generator into a quality gate, which is usually the higher-leverage use. A team that uses Codex only as a generator gets faster code; a team that also uses it as a reviewer gets better code.

Section 6: Difference Between Codex and Other Coding Tools

This is the section that usually matters most to new users, because the category boundaries are easy to blur.

Codex Is A Product Layer, Not Just A Model

Codex is the product experience and workflow layer. Models are the underlying engines. Put differently:

• A general model answers questions or writes text.

• A coding model is tuned more narrowly for software tasks.

• Codex packages the model inside an agentic coding workflow with files, commands, approvals, sandboxes, and reviews.

That matters because users often compare Codex to "another model" when the real comparison is "another coding system."

Codex vs OpenAI General Models

OpenAI's current models page recommends GPT-5.4 as the flagship model for complex reasoning and coding. That is the general model-side recommendation.

Codex-specific pages, on the other hand, describe models like GPT-5.3-Codex and GPT-5.2-Codex as optimized for agentic coding tasks in Codex or similar environments.

The practical takeaway:

• Use GPT-5.4 when you want a top-tier general model.

• Use Codex-specific models when you want a model optimized for coding workflows inside Codex.

• Use the Codex surface when you want file edits, shell commands, reviews, and sandboxes, not just text output.

Codex vs Claude Code

Claude Code is also a terminal-based agentic coding tool. Anthropic's docs describe it as a terminal tool that can make plans, edit files, run commands, create commits, and work with MCP-connected data sources. It is strong if your team already prefers a terminal-first workflow and wants a tightly scriptable developer tool.

Codex differs in a few practical ways:

• Codex spans more surfaces, including CLI, IDE extension, app, cloud tasks, and code review.

• Codex cloud is built around GitHub-connected task execution and review.

• Codex is more explicitly positioned as a family of coding workflows, not just a single terminal agent.

The practical takeaway:

• Choose Claude Code if you want a terminal-native workflow with strong composability and you are happy living mostly in the shell.

• Choose Codex if you want a broader product layer with local, cloud, and app-based workflows that can be shared across a team.

Codex vs GitHub Copilot Coding Agent

GitHub Copilot coding agent is designed around GitHub's own workflow. GitHub docs describe it as an agent you can assign issues or pull requests to, and it works in the background to create or modify PRs. It lives very naturally inside GitHub-hosted development flows.

Codex is different in emphasis:

• Copilot coding agent is highly GitHub-centric.

• Codex is broader across terminal, IDE, app, and cloud.

• Copilot is a strong fit if your team already uses GitHub as the center of gravity for task assignment and review.

• Codex is a stronger fit if you want a more general coding agent surface that can work across local and cloud workflows.

The practical takeaway:

• Choose Copilot coding agent if your process is already deeply anchored in GitHub issues and pull requests.

• Choose Codex if you want a wider agent workflow that can run locally, in the IDE, or in Codex cloud.

Codex vs Open-Weight and Self-Hosted Models

Open-weight or self-hosted models serve a different need. Teams usually reach for them when they want:

• Full infrastructure control.

• Custom hosting or air-gapped deployment.

• More direct control over retention and data boundaries.

• A lower-cost path at high scale if they already own the hardware and ops stack.

The tradeoff is that self-hosted models usually do not give you the same out-of-the-box agentic product experience that Codex does. You have to assemble the orchestration, repo access, sandboxing, approvals, and review loop yourself.

That means the real choice is not "Which model is smartest?" It is "How much engineering do I want to spend on the workflow around the model?"

The practical takeaway:

• Choose open-weight or self-hosted models when infrastructure control is the main requirement and you are willing to build the surrounding agent system.

• Choose Codex when you want the workflow already packaged, especially for day-to-day engineering teams.

Codex vs General Chat Models

General chat models are best when the task is:

• A question and answer exchange.

• Conceptual reasoning.

• Drafting prose.

• Summarizing or rewriting text.

Codex is better when the task is:

• Reading and modifying a repository.

• Running tests.

• Fixing code.

• Reviewing pull requests.

• Coordinating multi-step implementation work.

Codex vs API Usage of the Same Models

The same model family can behave differently depending on the surface.

• In the API, you may call a model directly and design your own orchestration.

• In Codex, the same or similar model may be wrapped in repo access, approval flows, and task execution.

That is why some model pages mention that a model is optimized for "Codex or similar environments." The model is tuned for agentic software work, but the workflow surface still matters.

Comparison Matrix

The prose comparisons above collapse into a single matrix for fast reference:

Use the matrix to pick the dominant tool, then layer the others where they fit. Many teams legitimately run two of these in parallel — for example, Codex for cross-surface work and Claude Code for power-user terminal workflows.

Which Tool Should A New User Choose?

As a rule of thumb:

• For terminal-first coding and scripting, Claude Code is a strong alternative.

• For GitHub-native issue and PR automation, GitHub Copilot coding agent fits naturally.

• For local plus cloud plus app-based team workflows, Codex is the most flexible option.

• For maximum infrastructure control, self-hosted or open-weight stacks make sense.

OpenAI's docs currently list GPT-5.5 as the general flagship, with GPT-5.4, GPT-5.4-mini, and GPT-5.4-nano remaining available below it, while Codex docs and model pages expose Codex-specific variants and model switching inside the CLI.

Section 7: Pricing and Plan Access

Pricing is the part of Codex most likely to change, so this section should be treated as a snapshot of the current official docs.

Plan Access

OpenAI's current Help Center says Codex is included with:

• ChatGPT Plus

• ChatGPT Pro

• ChatGPT Business

• ChatGPT Enterprise/Edu

For a limited time, it is also included with Free and Go, though those plans are temporary exceptions and subject to rate limits.

Flexible Pricing and Credits

The current rate card says Codex pricing changed on April 2, 2026 to align with API token usage instead of purely per-message pricing. The same article explains that:

• New and existing Plus and Pro customers use the token-based rate card.

• New and existing Business customers use the token-based rate card.

• New Enterprise customers use the token-based rate card.

• Existing Enterprise/Edu and several other legacy plan categories remain on the legacy rate card until migration.

This is important because two teams in the same company can be on different pricing logic depending on workspace status and plan vintage.

Current Model Pricing Snapshot

The current model pages list pricing per 1M tokens in USD. The exact numbers depend on the model you choose:

• GPT-5.5: \(5 input, \)30 output. New flagship as of April 23, 2026.

• GPT-5.5 Pro: \(30 input, \)180 output. Higher-tier variant for the most demanding agentic and reasoning workloads.

• GPT-5.4: \(2.50 input, \)15 output.

• GPT-5.4-mini: \(0.75 input, \)4.50 output.

• GPT-5.4-nano: \(0.20 input, \)1.25 output.

• GPT-5-Codex: \(1.25 input, \)10 output.

• GPT-5.2-Codex: \(1.75 input, \)14 output.

• GPT-5.1-Codex-mini: \(0.25 input, \)2 output.

• codex-mini-latest: \(1.50 input, \)6 output.

These model pages also note context windows, output limits, and whether the model is intended for Codex-specific or general API use. For budget planning, remember that longer outputs can cost much more than the input prompt, so task framing matters as much as model choice.

Note that GPT-5.5 is roughly 2x the input price and 2x the output price of GPT-5.4, and GPT-5.5 Pro is an order of magnitude above that. OpenAI's framing is that GPT-5.5 is also more token-efficient than GPT-5.4, which can offset some of the headline price difference, but you should measure this on your own workloads before assuming it nets out. For the Codex-specific models, expect the lineup to shift as Codex variants based on GPT-5.5 ship; until then, the Codex-specific models above remain the right choice for purely coding-shaped tasks.

What This Means in Practice

The real cost depends on:

• Input size.

• Cached input.

• Output length.

• Whether the task uses fast mode.

• Which model you select.

So if you are planning a team rollout, do not estimate usage from "number of prompts" alone. Estimate based on expected token consumption and task type.

Legacy Pricing

The legacy rate card still matters for users and workspaces that have not been migrated. The big lesson is that pricing is now tied more closely to model usage than to a simple fixed message count. Anyone budgeting Codex should read the current rate card before setting internal chargeback rules or usage policies.

Worked Cost Example

Pricing tables are easy to misread. A worked example makes the model selection question concrete.

Scenario: A 30-engineer team uses Codex Cloud for automated pull request review. Each engineer opens roughly 4 PRs per week. Each PR review pulls in approximately 30,000 input tokens (the diff plus relevant context files) and produces approximately 3,000 output tokens (the review comments and risk summary).

Weekly token volume:

• Reviews per week: 30 engineers × 4 PRs = 120 reviews

• Input tokens per week: 120 × 30,000 = 3.6M input tokens

• Output tokens per week: 120 × 3,000 = 360K output tokens

Cost per week by model:

Reading the table: The headline GPT-5.5 sticker shock disappears at this volume — under $1,500/year for 30 engineers' worth of automated review is a rounding error against engineering payroll. GPT-5.5 Pro is 6× more expensive and generally not justified for routine review; reserve it for the small share of reviews where you need its extra capability. The Codex-specific models are dramatically cheaper and are the right default if your reviews are mostly mechanical (style, obvious bugs, missing tests).

What this example does not capture:

• Cached input. OpenAI prices repeated input tokens lower; if your review pulls the same context files repeatedly, real costs are lower than shown.

• Long-task overhead. Agentic workflows that re-read files or iterate burn many more tokens than a single-shot review. A coding task can easily be 5–10× the tokens of a review.

• Failure retries. A failed task that gets re-run costs roughly the same as the original. Agent flakiness is a real budget line item.

• Mixed-model strategies. Most mature teams route cheap tasks (test stubs, doc updates) to a Codex-mini model and reserve GPT-5.5 for repository-wide refactors and PRs that need long-context reasoning.

The practical pattern: build the cost model around your actual highest-volume workload (usually PR review or test generation), then size the GPT-5.5 budget separately for the smaller set of tasks that actually benefit from the new capabilities.

Section 8: Security, Permissions, and Enterprise Setup

Teams care about Codex not just as a productivity tool, but as a controlled software-development system. OpenAI's docs reflect that reality.

Local vs Cloud Access

Enterprise admins can separately enable:

• Codex Local

• Codex Cloud

• Both

Codex Local covers the app, CLI, and IDE extension. Codex Cloud covers hosted tasks, code review, and related integrations.

That separation is useful because some organizations want local tooling enabled broadly while keeping cloud tasks restricted to fewer users.

Workspace Controls

The admin docs say workspace owners can use RBAC to manage access. They can:

• Set a default role.

• Create custom roles.

• Assign roles to groups.

• Sync groups with SCIM.

• Manage permissions centrally.

This is the right place to build a rollout with least privilege rather than giving every developer broad Codex access by default.

GitHub Connector and Repository Access

Codex Cloud requires GitHub-hosted repositories. Admins connect the ChatGPT GitHub Connector, choose an installation target, and allow specific repositories. Codex uses short-lived, least-privilege GitHub App tokens and respects repository permissions and branch protection rules.

For security teams, that matters because it keeps Codex aligned with the repo access model you already use.

Internet Access

By default, Codex cloud agents do not have internet access at runtime. That is deliberate. If your task truly needs access to dependency registries or trusted sites, admins can configure allowlists and HTTP method limits.

Recommended Governance Pattern

The enterprise docs recommend using separate groups for users and admins:

• A smaller Codex Admin group for people who manage policy and governance.

• A broader Codex Users group for developers who just need to use the tool.

That keeps policy management tight and avoids accidental over-permissioning.

Section 9: Best Practices for Teams

If you are onboarding a team, you will get much better outcomes if you set expectations up front.

Start With Simple, Valuable Tasks

Good first-team use cases:

• Pull request review.

• Small bug fixes.

• Test generation.

• Documentation updates.

• Codebase navigation and understanding.

These are easy to compare against human work and easy to judge for quality.

Standardize Task Prompts

Give people a shared prompt template. For example:

Task: Fix the failing test in X.
Context: The regression started after Y.
Constraints: Do not change public API behavior.
Output: Explain root cause, apply fix, run tests, summarize risks.

This makes results easier to review and reduces the "prompt quality lottery" that often hurts team adoption.

Use a Review Culture

Codex should not replace code review discipline. Treat it as:

• A first-pass implementer.

• A pre-review reviewer.

• A way to reduce repetitive work.

The human team should still own architecture, product tradeoffs, and final sign-off.

Measure What Matters

The metrics that matter are the ones that tell you whether Codex is producing reviewable, mergeable, trustworthy work — not the ones that count activity. Below is each metric, how to actually compute it from data you already have, and the rule of thumb for what "healthy" looks like.

1. Time to First Useful Diff

Definition: From the moment a Codex task is started, how long until it produces a diff that a human would actually consider applying (after possible small tweaks).

How to measure:

• For CLI/IDE tasks, log the wall-clock time from prompt submission to first diff. The Codex CLI emits structured logs you can parse; a simple wrapper script suffices:

  start=\((date +%s); codex "<prompt>"; echo "elapsed: \)(( $(date +%s) - start ))s"
  

• For Codex Cloud tasks, use the task duration shown in the chatgpt.com/codex dashboard, or pull it from the workspace usage export.

• Tag each task as "useful" or "discarded" in a shared spreadsheet for the first month. After that, you can sample.

Healthy: under 2 minutes for bounded tasks; under 10 minutes for multi-file refactors. If the median is much higher, your prompts probably lack context (see Section 5).

2. Test Pass Rate on Codex-Generated Changes

Definition: Of the diffs Codex produces, what percentage pass the existing test suite on the first try.

How to measure:

• In CI, tag PRs that originated from Codex (a label like codex-authored or a commit-message prefix works). Then run a simple weekly query:

  SELECT
    COUNT(*) FILTER (WHERE first_ci_run = 'pass') * 100.0 / COUNT(*) AS first_try_pass_rate
  FROM pull_requests
  WHERE labels @> '{"codex-authored"}'
    AND created_at > NOW() - INTERVAL '7 days';
  

• For local CLI usage, instrument with a wrapper that runs your test command immediately after Codex finishes and records the exit code.

Healthy: above 75% for bounded tasks. Below 50% means Codex is making changes without verifying them — usually fixable by adding "run the tests after" to your prompt template (see Section 9 → Standardize Task Prompts).

3. Review Findings Caught by Codex

Definition: When Codex is used as a pre-merge reviewer, how many issues does it surface that a human reviewer or CI would have caught anyway, vs. issues only Codex caught, vs. false positives.

How to measure:

• Have human reviewers annotate Codex's review comments with one of three tags: agree-found-it, agree-missed-it, disagree-noise.

• Track the ratios over time:

  • Useful-finding rate = (agree-found-it + agree-missed-it) / total Codex comments.

  • Unique-value rate = agree-missed-it / total Codex comments.

• A simple GitHub Actions step that posts the Codex review and asks the human reviewer to react with emoji (✅ / ⚠️ / ❌) makes this nearly free to collect.

Healthy: useful-finding rate above 70%; unique-value rate above 20%. Unique-value rate is the number that justifies keeping the workflow on — if it is near zero, Codex is duplicating CI and you can disable it without losing anything.

4. Tasks Completed Without Human Rewrite

Definition: Of all merged Codex-authored changes, what fraction shipped substantially as Codex wrote them (vs. being heavily rewritten by a human before merge).

How to measure:

• Compare the diff Codex initially produced to the diff that actually merged. The simplest proxy:

  # in the Codex-authored branch:
  git diff codex/initial-commit HEAD --shortstat
  

  If the post-Codex diff changes more than ~30% of the lines Codex originally wrote, count the task as "rewritten."

• Track this monthly. The trend line matters more than the absolute number.

Healthy: above 60% shipped without major rewrite. Lower than that, and either prompts are under-specified or Codex is being pushed into work it is bad at — re-read Section 14.

5. Developer Satisfaction

Definition: Whether the people actually using the tool think it makes them faster and want to keep using it. Hard numbers do not capture this.

How to measure:

• Run a 5-question pulse survey monthly. Keep it short. Suggested questions, all on a 1–5 scale:

  1. "Codex saved me time this week."

  2. "I trust Codex's diffs enough to review them confidently."

  3. "Codex's review comments are usually worth reading."

  4. "I would be unhappy if Codex were taken away."

  5. "What is the single biggest friction point?" (free text)

• Track the trend in question 4 specifically. That is the closest equivalent to a product-market-fit signal for an internal tool.

Healthy: average score above 3.5/5 on questions 1–4 by month 3 of rollout. If question 4 trends down, the rollout is failing regardless of what the other metrics say.

What NOT to Measure

These look useful but mislead:

• Number of prompts sent. Counts activity, not value. A team sending 10× more prompts may be 10× more productive — or 10× more confused.

• Tokens consumed. Useful for budget, useless for impact. Heavy users are not necessarily good users.

• Lines of code generated. Same problem as LOC has always had: you reward verbosity.

• PRs opened by Codex. A Codex-opened PR that nobody merges is a negative outcome dressed up as a positive one.

Use the cost data (Section 7) to manage budget. Use the metrics above to manage adoption.

Use the Right Surface for the Job

• CLI for terminal-heavy local work.

• IDE extension for day-to-day coding.

• App for parallel project work.

• Cloud for background tasks and review.

That is usually the difference between "this is useful" and "this is annoying."

Section 10: Common Workflows and Examples

Here are the workflows most teams will actually use. Each one includes a worked example against the codex-demo repo from Section 4 so you can see the full prompt, the kind of output Codex produces, and what to do with it.

Workflow 1: Fix a Bug Locally

Use when: A test is failing, a behavior is wrong, and the cause is contained to one file or function.

Steps:

1. Open the repo in your terminal or IDE.

2. Ask Codex to inspect the failing path.

3. Request a fix and a test.

4. Review the diff.

5. Run the test suite.

Worked example:

In the codex-demo repo, suppose a teammate just reported: "apply_discount is silently returning a negative price when discount_percent is greater than 100." Verify the bug first:

python -c "from pricing import apply_discount; print(apply_discount(100, 150))"
# prints: -50.0    <-- silent negative price, no error raised

Now launch Codex and run:

Bug: apply_discount(100, 150) returns -50.0 instead of raising an error.
Expected: discount_percent values above 100 should raise ValueError with
the message "discount_percent must be between 0 and 100".

Task:
- Add the validation in pricing.py.
- Add a test in test_pricing.py that asserts ValueError is raised for
  discount_percent=150.
- Keep the existing tests passing.
- Run pytest at the end and report the result.

What you get back: a diff that adds if discount_percent > 100: raise ValueError(...) in apply_discount, a new test_invalid_discount_percent_above_100 test, and the pytest output showing all four tests passing. Review with git diff, run python -m pytest yourself to confirm, then git commit -am "Reject discount_percent > 100".

This works best when the bug is bounded and reproducible. If you cannot reproduce it from the command line, Codex usually cannot either.

Workflow 2: Review a Pull Request

Use when: You (or a teammate) just made a change and want a fast pre-merge sanity check before opening it for human review.

Steps:

1. Point Codex at the PR or changed files.

2. Ask for correctness issues, missing tests, and security risks.

3. Compare the findings against human review.

4. Use Codex as a pre-filter before the broader team reviews.

Worked example:

After completing Workflow 1 above, ask Codex to review your own change before opening a PR:

Review the change in my last commit (HEAD) — it added validation to
apply_discount in pricing.py.

Look for:
- correctness issues (off-by-one on the boundary, wrong error type, etc.)
- missing tests (boundary cases like exactly 100, exactly 0, NaN, negative zero)
- security or robustness issues
- API consistency with the existing apply_discount validation style

Prioritize findings as CRITICAL / IMPORTANT / NIT and propose a concrete
fix for each. Do not modify any files in this turn.

What you might get back:

IMPORTANT: line 14 — the new validation rejects discount_percent > 100 but
  silently allows discount_percent == 100, which makes the price 0. That is
  technically valid but worth a test to lock the boundary. Add:
    test_apply_discount_at_boundary_100_returns_zero

NIT: the new error message says "between 0 and 100" but the existing check
  for negative values says "must be >= 0". Consider unifying the messages
  for consistency.

You apply the IMPORTANT fix (often by following up with: "apply the IMPORTANT fix from your review"), defer or accept the nit, and re-run tests.

This is one of the highest-leverage team workflows because it catches obvious problems before a human spends review time on them. See Section 9 → Measure What Matters → Review Findings Caught by Codex for how to track its actual value over time.

Workflow 3: Understand a Large Codebase

Use when: You are new to a repo (or returning after months away) and need a map before you can safely make changes.

Steps:

1. Ask Codex to trace a request flow.

2. Ask for the key modules and entry points.

3. Request a map of the code path before editing anything.

Worked example:

The codex-demo repo is too small to need this, so imagine a more realistic case: a teammate's repo with app/, services/, models/, api/, and 80 files you have never seen. Open the repo in Codex and run:

I am new to this codebase. Without modifying anything, give me an
orientation:

1. What is the entry point for the HTTP API?
2. Trace what happens when a POST hits /users — list every file the
   request touches in order, with a one-line description of each.
3. Where is database access centralized? Is there a repository pattern?
4. What test command should I run to verify any change I make?
5. What are the three files I should read first to understand the
   project's conventions?

Output as a structured markdown report.

What you get back: a markdown report you can paste into your notes. Read the recommended files, then start working with Codex on actual changes. The 10 minutes spent on this orientation typically saves an hour of confused refactoring later.

This workflow is particularly useful for new hires. A senior engineer can also use it the first time they touch an unfamiliar service to avoid breaking conventions they cannot see.

Workflow 4: Generate a Feature in Parallel

Use when: A feature naturally splits into independent pieces (API + tests + docs, or UI + backend + migration) that do not block each other.

Steps:

1. Break the work into subtasks.

2. Run separate Codex tasks for UI, API, tests, or docs.

3. Merge the outputs after review.

Worked example:

Add a new "loyalty discount" capability to codex-demo. The work splits into three pieces that do not depend on each other:

Each runs in parallel. When all three finish, merge the diffs (typically the implementation goes first, then tests verify against it, then docs reference what shipped). Review each independently.

The Codex app and cloud surfaces are especially good for this because they let you launch and monitor multiple tasks without juggling terminal windows. The CLI also supports parallel work, but it benefits from git worktree so each run operates on its own branch checkout.

Workflow 5: Use Subagents for Decomposition

Use when: A single task is too large for one Codex run but can be naturally split into investigate / plan / implement phases.

The CLI explicitly supports subagents — one Codex task that spawns child tasks, each with a narrower scope and its own context window.

Worked example:

A bug report says: "Cart totals are sometimes off by a penny for European currencies." You do not yet know if this is a rounding bug, a currency-conversion bug, or a data bug. Run a parent task that decomposes:

A bug report says cart totals are occasionally off by a penny for
European currencies.

Decompose this into three subagent tasks:

1. INVESTIGATE: Read pricing.py and any currency-related code. Identify
   every place where floating-point arithmetic touches a money value.
   Report findings without proposing fixes.

2. REPRODUCE: Write a failing test in test_pricing.py that demonstrates
   a one-cent discrepancy with EUR amounts. Use the smallest possible
   reproduction.

3. PROPOSE: Based on (1) and (2), propose two possible fixes (e.g.,
   switching to Decimal vs. rounding at the boundary) with the trade-offs
   of each. Do not implement either yet.

Wait for me to pick a fix before writing any production code.

Why subagents help: each child task has a clean context, so the investigation findings do not pollute the test-writing context, and the proposal task gets a clean view of both. You also get a natural human checkpoint between investigation and implementation.

That division is often faster than one giant all-purpose run, and dramatically more reviewable.

Prompt Cookbook

New users often ask for examples because they know what they want outcome-wise but not how to phrase it. These templates are a good starting point.

Bug Fix Template

Inspect the failing behavior in [file or module].
Identify the root cause.
Patch the smallest safe fix.
Add or update tests.
Summarize what changed and any edge cases I should watch.

Use this when the bug is narrow and you want a disciplined fix, not a redesign.

Refactor Template

Refactor [module] to improve readability and maintain the current behavior.
Keep external APIs stable.
Explain the refactor plan before editing.
Make the smallest set of changes that achieves the goal.

Use this when the code works but is hard to maintain.

Review Template

Review this change for correctness, missing tests, security issues, and maintainability risks.
Prioritize findings by severity.
Call out any behavior changes or ambiguous logic.

Use this when you want Codex to act like a pre-merge reviewer.

Feature Template

Implement [feature] in [file or subsystem].
List the files you expect to touch before changing anything.
Add tests.
Keep the implementation aligned with the current architecture.

Use this when the task spans multiple files and you want visibility into the plan.

Signs You Are Using Codex Well

You usually know the workflow is healthy when:

• Codex makes small, reviewable diffs instead of broad rewrites.

• The model asks for clarification only when the missing detail matters.

• Test coverage improves along with functionality.

• New developers can use the tool without needing a custom training session.

• The time from prompt to merged change is lower, but review quality does not drop.

You usually know the workflow is unhealthy when:

• Prompts are vague and every result needs heavy rework.

• The team treats the first output as final.

• Nobody is checking diffs or running tests.

• Users keep asking for "make it better" instead of defining a clear target.

Those signals matter more than raw usage counts.

Section 11: Model Specs and Benchmarks (GPT-5.5 Deep Dive)

Section 2 introduced GPT-5.5 as the new general flagship and gave the three-bullet practical takeaway. This section is the deep dive: the published benchmark numbers, what each one actually measures, why it matters for Codex workloads specifically, and how to use those numbers to pick the right model per task.

If you are setting budgets or choosing default models for a team, read this section in full. If you just want to use Codex, you can skim it.

Why Benchmarks Matter for Model Selection

Codex lets you pick the model behind each surface. Picking well is mostly about matching the model's strengths to the task shape:

• A bounded local edit (one file, one function) does not benefit much from a frontier model. Codex-specific or Codex-mini variants are usually the right call.

• A repository-wide refactor that needs the model to keep many files in working memory benefits enormously from long-context performance.

• An agentic cloud task that runs unattended for ten minutes benefits from low hallucination rates and strong tool-use behavior.

• A PR review benefits from low hallucination rates above almost everything else — a confident-but-wrong review comment costs more than a missed real issue.

The benchmarks below tell you which model best matches each shape.

GPT-5.5 Performance Highlights

The published benchmarks position GPT-5.5 as a meaningful jump over GPT-5.4, particularly on agentic and long-context work — the workloads most relevant to Codex users.

• Knowledge work (GDPval) — 84.9%. GDPval evaluates whether a model can produce well-specified knowledge-work output across 44 occupations. This is the headline general-capability number.

• Computer use (OSWorld-Verified) — 78.7%. Measures whether the model can drive a real computer environment end-to-end. Directly relevant to Codex Cloud sandboxes and agentic CLI runs.

• Coding (Terminal-Bench 2.0) — 82.7%. A terminal-centric coding benchmark with long-context retrieval and computer-use components. The closest public proxy for Codex CLI workloads.

• Customer-service workflows (Tau2-bench Telecom) — 98.0% without prompt tuning. Indicates strong tool-use and policy-adherence behavior straight out of the box.

• Long-context retrieval (MRCR v2 at 1M tokens) — 74.0%, up from 36.6% on GPT-5.4. This is the largest single jump in the report and the most important one for repository-scale Codex tasks where the model must keep many files in working memory.

• Hallucination rate — independent coverage reports a roughly 60% reduction in hallucinations versus prior generations, which materially changes the trust calculus for review and PR-feedback workflows.

What Each Benchmark Actually Measures

Benchmarks are easy to misread. Quick definitions of the ones cited above:

• GDPval — Asks the model to produce specified knowledge-work output across 44 occupations (legal memos, financial summaries, technical documentation, etc.). A high score means the model can produce structured, well-specified output reliably. Use as a general-capability signal, not a coding-specific one.

• OSWorld-Verified — Tasks the model with operating a real desktop environment to complete real workflows (open files, navigate UIs, run commands). High scores predict the model will behave well in agentic sandboxes that mimic a developer's desktop.

• Terminal-Bench 2.0 — A terminal-driven coding benchmark with long-context retrieval and computer-use components. The closest public proxy for what Codex CLI actually does day to day.

• Tau2-bench Telecom — Evaluates complex customer-service-style workflows that require following policies and using tools correctly. A proxy for "does the model do what you told it without going off-script."

• MRCR v2 at 1M tokens — A long-context retrieval benchmark. Tests whether the model can find and use information across a full 1M-token context window. The single best predictor of behavior on repository-scale Codex tasks where many files must be kept in working memory.

Practical Guidance for Codex Users

Translate the benchmarks into model choice:

• Repository-wide tasks (cross-file refactors, multi-module migrations): GPT-5.5. The MRCR v2 jump is the single best signal that it will behave better on large codebases than GPT-5.4 did.

• Cheap, bounded local edits (single function, single test, doc tweak): GPT-5.4 or a Codex-specific model. The cost/latency tradeoff is much better and the capability headroom is wasted on small tasks. Do not default everything to GPT-5.5 just because it is newest.

• Agentic cloud tasks (background sandbox runs, multi-step workflows): GPT-5.5. The OSWorld-Verified score and lower hallucination rate are the relevant signals — fewer broken sandbox runs and fewer confidently-wrong outputs.

• PR review and code review workflows: GPT-5.5. The 60% hallucination drop is the single most important number for review work; a noisy reviewer trains the team to ignore the reviewer.

• Most expensive workloads (anything that approaches GPT-5.5 Pro pricing): keep GPT-5.5 Pro reserved for the small set of tasks where its extra capability is justified — typically deeply novel reasoning or extreme long-context work.

For Procurement: Treat GPT-5.5 as a Separate Budget Line

Token consumption on agentic tasks is dominated by output. GPT-5.5 outputs are substantially more expensive than GPT-5.4 outputs. Concretely:

• Mixed-model strategies are now the rule, not the exception. Most mature teams route routine work to a Codex-mini model and reserve GPT-5.5 for repository-wide and review-heavy work.

• The worked cost example in Section 7 shows the 30-engineer PR-review case across all five model tiers. Read it before approving a budget.

• Re-check pricing every quarter. The rate card has changed in the past and will change again.

Verify Before Quoting

The numbers in this section come from OpenAI's launch documentation and contemporaneous press coverage. Before they go into a procurement deck or a public document, verify against the official OpenAI announcement and the model page — see Section 16: Source References. Benchmarks get re-run; numbers shift with eval methodology changes.

Section 12: Troubleshooting

Even good tools fail if the setup is wrong. Here are the most common issues.

"Codex is not installed"

Check:

• You ran npm i -g @openai/codex.

• You are using a supported shell and runtime.

• The binary is on your path.

"I cannot sign in"

Check:

• Your ChatGPT account has the right plan.

• Your workspace allows Codex local or cloud use.

• You are signing in with the correct account.

"Windows is behaving badly"

The CLI docs say Windows support is experimental. If you are on Windows, the best supported path is to use WSL for the CLI or use the Codex app where appropriate.

"Cloud task cannot see my repo"

Check:

• The GitHub connector is installed.

• The repository is allowed in the connector.

• Your organization admin has enabled Codex cloud.

• You are using a GitHub-hosted repository.

"Codex will not browse the internet"

That is expected by default in cloud mode. Ask your admin whether internet access has been intentionally restricted.

"The result is technically correct but not what I wanted"

Usually this means the prompt was under-specified. Tighten:

• The target file or feature.

• The acceptance criteria.

• The constraints.

• The expected output format.

Section 13: FAQ

Is Codex a chat model?

Not exactly. It is a coding agent and product surface built to work on repositories, tests, code review, and multi-step software tasks.

Can I use Codex without switching tools all the time?

Yes. That is one of its strengths. You can use the CLI, IDE extension, or Codex app depending on your workflow.

Do I need the cloud features?

No. Many individual users will get value from the local CLI or IDE extension alone. Cloud tasks become more valuable as soon as you want background execution, parallelism, or automated review.

Is Codex only for professional engineers?

No, but it is most useful when the user can evaluate code changes and understand a repository. It is a developer tool first.

Is Codex the same as GPT-5.4?

No. GPT-5.4 is a model. Codex is the coding product/workflow. Codex may use different models depending on the surface and configuration.

What is the safest way to start?

Use the CLI or IDE extension in a small repo change, keep the approval mode conservative, and review every diff before merging.

Section 14: When NOT to Use Codex

Most of this handbook is affirmative — Codex is good at this, Codex fits here, here is how to set it up. That framing risks creating the impression that Codex is the right tool for any coding-adjacent task. It is not. The fastest way to lose team trust in an AI coding tool is to push it into work it is bad at. The following is an honest list of where Codex is a poor fit today.

Tasks With No Reviewable Output

Codex's value depends on a human reviewing the diff, the test result, or the explanation. If the task produces something nobody will check — a one-off script that touches production data, an exploratory query whose result drives a decision before anyone reads the SQL — the AI's confidence becomes the only quality gate. That is a bad position to be in regardless of model quality. Either add a review step or do the task yourself.

Highly Novel Architecture Decisions

Codex is good at applying patterns. It is much weaker at choosing which pattern fits a problem the team has not solved before. Expect it to confidently generate plausible-but-wrong architecture for genuinely new domains: a new pricing model, a new auth boundary, a new event-sourcing scheme. Use it to prototype options, not to decide between them.

Work That Crosses Org Boundaries

Codex sees the repository it has access to. It does not see the cross-team contracts, the deprecation calendar in the platform team's roadmap, the half-finished migration in another repo, or the political reasons one approach is off-limits. For changes that span multiple teams or services, Codex can implement individual pieces, but a human still needs to own the cross-cutting plan.

Anything Touching Live Production State

Codex Cloud sandboxes are good. They are not a substitute for human approval before a production change. Database migrations, infrastructure-as-code that mutates real resources, secret rotation, customer-data scripts — these need a human in the approval path even if Codex wrote the diff. The fact that Codex can run commands does not mean it should run those commands.

Compliance- and Safety-Critical Code

Code that lives inside a regulated boundary (payments, medical, security primitives, model-evaluation harnesses for safety) has higher review and provenance requirements than typical product code. Codex output is fine as a starting draft, but the review burden is the same as for any third-party-authored code, which usually means the speed advantage shrinks substantially. Plan for that or keep these areas Codex-free.

Tasks Where the Real Bottleneck Is Knowledge, Not Typing

If the team is stuck because nobody understands the legacy system, the failing test, or the weird customer report, generating more code rarely helps. Codex can accelerate the implementation once you know what to do. It cannot replace the discovery and design conversation that should happen first. Teams that skip the discovery step and go straight to "ask Codex" tend to ship the wrong thing fast.

Anything Where Hallucinations Have High Cost

GPT-5.5 dropped hallucination rates by roughly 60% versus prior generations, which is a real improvement. It is not zero. Tasks where a confident-but-wrong output causes real damage — generating regulatory citations, copying API contract details from a doc the model hasn't actually read, asserting facts about an unfamiliar third-party library — still need the same skepticism you would apply to any AI output. Use search-grounded workflows or human verification for these.

Quick Heuristic

If you can answer all four of these with "yes," Codex is likely a good fit:

1. Can the output be reviewed by someone who would catch a mistake?

2. Is the task a known pattern, not a novel architecture decision?

3. Is the blast radius local to one repository or service?

4. Is the cost of a bad output bounded (e.g., a failed test, a reverted commit) rather than unbounded (e.g., production data loss, regulatory exposure)?

If any of those are "no," either restructure the task to make them "yes" or keep the work outside Codex.

Section 15: Final Recommendations

If you are rolling Codex out to new users, I would keep the guidance very simple:

1. Start with the CLI or IDE extension.

2. Use one small task to learn the tool.

3. Review every change before merging.

4. Move to cloud tasks only after users trust the local workflow.

5. For teams, separate user access from admin access.

6. Re-check pricing whenever your plan or workspace changes.

Codex is most valuable when it is treated as a disciplined engineering tool rather than a novelty. If you give it real code, clear constraints, and a review culture, it can accelerate the boring parts of software development and make bigger tasks easier to break down.

The LUNARTECH Fellowship: Bridging Academia and Industry

Addressing the growing disconnect between academic theory and the practical demands of the tech industry, the LUNARTECH Fellowship was created to bridge this talent gap.

Far too often, aspiring engineers are caught in the “no experience, no job” loop, graduating with theoretical knowledge but unprepared for the messy reality of production systems.

To combat this systemic issue and halt the resulting brain drain, the Fellowship invests heavily in promising individuals, offering a transformative environment that prioritizes hands-on experience, mentorship, and real-world engineering over traditional degrees.

This 6-month, remote-first apprenticeship serves as an immersive odyssey from aspiring talent to AI trailblazer. Rather than paying to learn in isolation, Fellows work on live, high-stakes AI and data products alongside experienced senior engineers and founders. By tackling actual engineering challenges and building a concrete portfolio of production-ready work, participants acquire the job-ready skills needed to thrive in today’s competitive landscape.

If you are ready to break the loop and accelerate your career, you can explore these opportunities and start your journey here: https://www.lunartech.ai/our-careers.

Master Your Career: The AI Engineering Handbook

For those ready to transition from theory to practice, we have developed The AI Engineering Handbook: How to Start a Career and Excel as an AI Engineer. This comprehensive guide provides a step-by-step roadmap for mastering the skills necessary to thrive in the transformative world of AI in 2026.

Whether you are a developer looking to break into a competitive field or a professional seeking to future-proof your career, this handbook offers proven strategies and actionable insights that have already empowered countless individuals to secure high-impact roles.

Inside, you will explore real-world industry workflows, advanced architecting methods, and expert perspectives from leaders at companies like NVIDIA, Microsoft, and OpenAI. From discovering the technology behind ChatGPT to learning how to architect systems that transform research into world-changing products, this eBook is your ultimate companion for career acceleration. You can download your free copy and start mastering the future of AI.

Section 16: Source References

Official OpenAI sources used for this handbook:

• Introducing GPT-5.5 (OpenAI)

• Using Codex with your ChatGPT plan

• Flexible pricing for the Enterprise, Edu, and Business plans

• All models

• OpenAI API models overview

• GPT-5-Codex model

• GPT-5.2-Codex model

• codex-mini-latest model

• Codex use cases

• Claude overview

• GitHub Copilot documentation

• Codex enterprise admin setup

• Codex IDE extension docs

• Codex – OpenAI's coding agent (VS Code Marketplace listing)

• Codex web (cloud) docs

• Codex CLI docs

• Codex CLI command-line reference

• Codex CLI features

• Codex quickstart

• Using Codex with your ChatGPT plan (Help Center)

Press coverage of the GPT-5.5 release referenced in Section 2 and Section 11:

• OpenAI releases GPT-5.5, bringing company one step closer to an AI 'super app' (TechCrunch)

• OpenAI launches GPT-5.5, calling it "a new class of intelligence" (The New Stack)

• OpenAI's GPT-5.5 benchmarks show a 60% hallucination drop and coding skills that rival senior engineers (Startup Fortune)

Appendix A: 30-60-90 Day Adoption Plan

If you are introducing Codex to a team, the fastest way to create trust is to phase adoption instead of rolling it out as a big-bang change. A staged plan also helps you discover where the real friction lives: authentication, permissions, prompt quality, review habits, or budget assumptions.

First 30 Days: Prove Value

In the first month, the goal is not maximum usage. The goal is repeatable wins.

Recommended actions:

1. Pick one or two engineers who are comfortable trying new tools.

2. Restrict usage to small, low-risk tasks such as bug fixes, test generation, and documentation updates.

3. Standardize a short prompt template so every request includes task, context, constraints, and expected output.

4. Require human review for every change.

5. Track the time it takes to go from prompt to merged diff.

What you should learn in this phase:

• Does Codex understand your codebase structure?

• Are the diffs reviewable?

• Does the approval flow slow people down in a useful way, or in a frustrating way?

• Which classes of tasks work well, and which ones need more guidance?

If the first month is noisy, do not blame the model first. Usually the issue is task scope, missing context, or unclear acceptance criteria.

Days 31-60: Expand Carefully

Once the tool has proven itself on a handful of tasks, expand to a broader pilot group.

Recommended actions:

1. Add more developers from different parts of the stack.

2. Include at least one person who is skeptical, because their feedback will reveal weak spots.

3. Try the app, CLI, and IDE extension in parallel so people can choose the workflow that matches their habits.

4. Introduce Codex cloud for one or two background tasks or pull request reviews.

5. Start documenting prompts that worked well, including examples of high-quality follow-up instructions.

What you should learn in this phase:

• Which surfaces are actually sticky for the team?

• Where does Codex save the most time?

• Do people trust the output enough to delegate real work?

• Are you seeing the same mistakes repeatedly?

At this stage, your internal documentation matters. A short "how we use Codex here" page is often more useful than another technical deep dive.

Days 61-90: Operationalize

After about three months, your objective should shift from experimentation to operating practice.

Recommended actions:

1. Assign ownership for workspace settings, GitHub connector setup, and model access.

2. Define which tasks should stay local and which can go to cloud sandboxes.

3. Document your review standards for Codex-generated diffs.

4. Set budget expectations with the team so no one is surprised by token-heavy tasks.

5. Add Codex to onboarding for new engineers, starting with one simple flow.

What good looks like at this stage:

• New hires can use Codex on day one.

• Team members know when to reach for Codex and when to use a different workflow.

• Admins can answer access and pricing questions quickly.

• The organization has a realistic picture of the tool's strengths and limits.

A Practical Onboarding Script

If you need a ready-made orientation for a new user, use this:

1. "Install the CLI or extension."

2. "Open a repository you know well."

3. "Ask Codex to make one small, safe change."

4. "Review the diff line by line."

5. "Run the tests."

6. "Ask Codex to explain what it changed and why."

7. "Repeat with a slightly larger task."

That sequence teaches the core loop: context, task, change, review, verify. Once a user understands that loop, the rest of the product family becomes much easier to adopt.

Appendix B: Glossary

Terms used in this handbook, in alphabetical order. The list is intentionally narrow — only terms that appear in the body and are likely to be unfamiliar to a non-engineering reader (procurement, security, leadership) are defined here.

• Agent / agentic workflow. Software that can take a goal, plan steps, take actions (read files, run commands, call APIs), observe the result, and iterate. Codex is an agentic coding workflow; a chatbot is not.

• Approval mode. A Codex setting that controls how much the agent can do without asking. Stricter modes prompt the human before running shell commands or modifying files; permissive modes let the agent work uninterrupted.

• CLI. Command-line interface. The Codex CLI is the terminal-based version of Codex, installed via npm i -g @openai/codex.

• Codex Cloud. The hosted, sandboxed execution mode for Codex. Tasks run in isolated environments with the repo and finish with a reviewable diff.

• GDPval. A benchmark that scores models on their ability to produce well-specified knowledge-work output across 44 occupations. Used in Section 11 as a general-capability signal.

• GitHub Connector. The integration that lets Codex Cloud access GitHub repositories. Required for cloud tasks; uses short-lived, least-privilege tokens.

• MCP (Model Context Protocol). An open protocol for connecting models to external data sources and tools. Codex CLI supports MCP, which lets it pull in data from systems beyond the repo.

• MRCR v2. A long-context retrieval benchmark that measures whether the model can find and use information across very large input windows. The 1M-token version is cited in the GPT-5.5 section because it predicts behavior on repository-scale tasks.

• OSWorld-Verified. A benchmark that measures whether a model can operate a real desktop computer environment to complete tasks. A direct proxy for agentic and computer-use workloads.

• PR (pull request). A proposed change to a code repository, hosted on GitHub or similar platforms, where reviewers approve before the change merges.

• RBAC (role-based access control). A permission model where users are assigned to roles, and roles have specific permissions. Used by Codex workspace admins to control who can do what.

• SCIM (System for Cross-domain Identity Management). A standard for syncing users and groups from an identity provider (Okta, Entra ID, etc.) into another system. Codex supports SCIM-based group sync for enterprise.

• Subagent. A Codex CLI feature that splits a task across multiple parallel agent runs, each handling a piece of the work.

• Tau2-bench Telecom. A benchmark for complex customer-service workflows with tool use. Cited as a signal for tool-use reliability and policy adherence.

• Terminal-Bench 2.0. A coding benchmark focused on terminal-driven workflows, including long-context retrieval and computer use. The closest public proxy for Codex CLI workloads.

• Worktree. A git feature that lets multiple branches be checked out simultaneously in different directories. The Codex app uses worktrees so multiple agents can work in parallel without stepping on each other.

• WSL (Windows Subsystem for Linux). A compatibility layer that runs Linux binaries natively on Windows. The recommended environment for Codex CLI on Windows, since direct Windows support is experimental.

Appendix C: Admin Security Checklist

For workspace admins setting up Codex for an enterprise. This checklist condenses Section 8 into actionable items. Run through it before broad rollout, then revisit quarterly.

Access

• [ ] Decide whether Codex Local, Codex Cloud, or both are enabled at the workspace level.

• [ ] Create separate RBAC groups for Codex Admins (policy and governance) and Codex Users (day-to-day developers). Avoid mixing the two.

• [ ] Sync user and group membership from your identity provider via SCIM rather than managing users by hand.

• [ ] Set a sensible default role for new workspace members. Do not default to admin.

GitHub integration

• [ ] Install the ChatGPT GitHub Connector against the correct GitHub organization.

• [ ] Allowlist only the repositories Codex Cloud needs. Do not grant org-wide access by default.

• [ ] Verify Codex respects existing branch protection rules on protected branches before enabling cloud tasks against them.

• [ ] Confirm the GitHub App tokens Codex uses are short-lived and least-privilege.

Network and runtime

• [ ] Confirm Codex Cloud runs with no internet access by default. This is the secure default; verify it is on.

• [ ] If a workflow requires internet access, define an explicit allowlist (dependency registries, trusted sites) and limit allowed HTTP methods.

• [ ] Document which model surfaces are approved for sensitive code (often: local CLI yes, cloud no for the most sensitive repositories).

Data and review

• [ ] Document the team's review standard for Codex-generated diffs. At minimum: a human approves every merge.

• [ ] Confirm logging and audit trails are configured for Codex actions (model used, prompts, files changed) per your compliance requirements.

• [ ] Define which classes of data are off-limits to Codex (PII, customer data, secrets) and how those boundaries are enforced.

• [ ] Establish an incident playbook for the case where Codex generates or commits something it should not have.

Budget and ongoing operations

• [ ] Set a per-workspace token budget or alert threshold so unexpected spend is caught early.

• [ ] Pick a default model per task type (e.g., Codex-mini for routine review, GPT-5.5 for repository-wide refactors) and document the choice.

• [ ] Review the Codex pricing page quarterly. The rate card has changed in the past and will change again.

• [ ] Re-run this checklist when (a) a major model release lands, (b) the workspace expands to a new team, or (c) Codex adds a new surface or capability.

Appendix D: Changelog

A short, append-only log of substantive revisions to this handbook. Each entry lists the version, date, and a one-line summary of what changed.

• v1.3 — 2026-04-30. Made the Table of Contents clickable. Added a new Prerequisites section after the TOC. Restructured the early sections: merged the old "Quick Start" and "How to Set Up Codex" into a single Section 4 walkthrough using a self-contained codex-demo repo readers build themselves. Slimmed Section 2 by moving the GPT-5.5 benchmark deep dive to a new Section 11 (Model Specs and Benchmarks). Added per-surface hyperlinks to Section 3. Rewrote Section 5 (How to Use Codex Effectively) with bad/good examples for every tip and a definition of "bounded change." Rewrote the "Measure What Matters" subsection with concrete computation methods for each metric. Added worked, runnable examples to every workflow in Section 10. Renumbered downstream sections accordingly.

• v1.2 — 2026-04-25. Added Appendix E (Working with Codex in VS Code), a detailed step-by-step guide covering the three VS Code entry points — the extension, the CLI in the integrated terminal, and browser Codex at chatgpt.com/codex — with setup instructions, a decision matrix, a combined-workflow pattern, and VS Code-specific troubleshooting. Added a forward-pointer in the setup section.

• v1.1 — 2026-04-25. Added GPT-5.5 / GPT-5.5 Pro coverage in Section 2 and Section 7. Added executive summary, comparison matrix in the model-comparison section, worked cost example, "When NOT to use Codex" in Section 14. Added Appendix B (Glossary), Appendix C (Admin Security Checklist), Appendix D (Changelog). Added version stamp and author line. Press coverage sources for GPT-5.5 added in Section 16.

• v1.0 — Initial release. Original Codex onboarding handbook covering surfaces, setup, usage, model comparison, pricing, security, team practices, workflows, troubleshooting, FAQ, and the 30-60-90 day adoption plan.

Appendix E: Working with Codex in VS Code

This appendix is a focused, step-by-step guide to using Codex inside Visual Studio Code (and its forks, Cursor and Windsurf).

VS Code is the most common starting surface for new Codex users, and the workflow has three distinct entry points that can be used independently or together. This guide covers each one, when to pick it, and how the three combine into a single fluid workflow.

E.1 Why VS Code Is the Recommended Starting Surface

Most teams start with VS Code rather than the standalone Codex app or pure CLI for a few practical reasons:

• The editor is already where engineers spend their day. Adding Codex does not require a context switch.

• The extension surface area is small and reviewable. Engineers can try it on a single file before adopting it more broadly.

• VS Code's integrated terminal makes the CLI a one-keystroke experience, so the extension and CLI can be combined without leaving the editor.

• Cursor and Windsurf, the most popular VS Code forks, both run the same Codex extension. A team that standardizes on the VS Code workflow does not have to retrain people if some engineers prefer a fork.

The downside of starting in VS Code is that you do not get parallel-task management or worktree support out of the box — those are stronger in the Codex app. For most individual contributors, that is not a meaningful loss in the first month.

E.2 The Three Entry Points

Codex shows up in VS Code in three distinct ways, and they are easy to confuse. Each is a separate piece of software with its own install and its own auth handshake, even though they all sign in with the same ChatGPT account.

1. The Codex VS Code extension — a sidebar UI inside VS Code itself. Installed from the VS Code Marketplace. Best for in-flow editing, quick questions about the open file, and short bounded tasks.

2. The Codex CLI, run inside VS Code's integrated terminal — the command-line agent (codex) running in the terminal pane that is already attached to your VS Code workspace. Best for multi-step agentic tasks, scripted runs, and anything where you want explicit approval gates.

3. Browser Codex at chatgpt.com/codex — the web interface to Codex Cloud, where tasks run in isolated sandboxes against your GitHub repository. Best for background work, parallel tasks, and PR-style review.

These are not alternatives to each other in the sense that you must pick one. They are three workflows that target different kinds of work, and most experienced Codex users have all three set up.

E.3 Setting Up the Codex VS Code Extension

This is the entry point most new users meet first.

Install

There are two install paths:

1. Open the VS Code Marketplace, search for "Codex" or "ChatGPT", and install the extension published by openai. The marketplace identifier is openai.chatgpt.

2. From a terminal, run:

code --install-extension openai.chatgpt

The CLI install path is useful for scripted dev-environment provisioning, dotfiles repos, and onboarding scripts that bring a new machine up to a known baseline.

Sign in

After install, the Codex panel appears in the right sidebar. The first time you open it, you will be prompted to sign in. You have two options:

• Sign in with ChatGPT. Recommended for individuals on Plus, Pro, Business, or Enterprise/Edu plans. Usage is charged against your plan's included Codex credits.

• Sign in with an API key. Used when you want metered API billing instead of plan-based usage, or when your workspace policy requires it. Get the key from the OpenAI developer console, then paste it into the extension's auth prompt.

If both options are visible and you are unsure which to pick, default to ChatGPT sign-in. It is the path that exercises the same plan-included usage that the rest of your team is on, which makes cost behavior predictable.

First-run sanity check

Once signed in, do a five-minute sanity check before relying on the extension for real work:

1. Open a small repository you know well.

2. Open the Codex panel in the right sidebar.

3. Ask a question about the open file (e.g., "What does this function do?") and confirm the answer matches what you already know.

4. Ask for a small change (e.g., "Add a docstring to this function") and confirm a reviewable diff appears.

5. Apply the change, run your tests, and revert if needed.

If any of those steps fails, fix the auth or install before going further. Trying to debug the extension on a real task is much harder than debugging it on a known-good toy task.

Platform notes

• macOS and Linux are first-class. The extension and the underlying CLI both work natively.

• Windows is experimental for the CLI. The extension itself works, but if you also want to run the CLI inside VS Code's integrated terminal, OpenAI recommends using a WSL workspace. Open the folder via "Reopen in WSL" before installing the CLI.

• Cursor and Windsurf run the same extension. Watch for visual or shortcut conflicts with the fork's built-in AI features — see E.9 for specifics.

E.4 Setting Up the Codex CLI Inside VS Code's Integrated Terminal

The CLI is the second entry point. It runs as a normal command-line tool, but inside VS Code's integrated terminal it picks up the active workspace folder automatically, which makes it feel like a native part of the editor.

Install the CLI

From any terminal, including VS Code's integrated terminal:

npm i -g @openai/codex

This installs the codex binary globally. Confirm by running:

codex --version

If the command is not found, the most common cause is that npm's global bin directory is not on your PATH. Either fix the PATH or use a Node version manager (nvm, fnm, volta) that handles it for you.

Open the integrated terminal in VS Code

Three ways to open it, pick whichever matches your habits:

• The View menu → Terminal.

• The keyboard shortcut Ctrl+** (backtick) on Windows/Linux, **⌃ on macOS.

• The Command Palette: Terminal: Create New Terminal.

The integrated terminal inherits the active workspace folder as its working directory, which means codex launched from there immediately sees the right repo.

Run Codex

In the terminal, navigate to the repo (if you are not already there) and run:

codex

The first time you run it, you will go through the same auth flow as the extension — sign in with ChatGPT or paste an API key.

Pick an approval mode

The CLI supports several approval modes that govern how much Codex can do without explicit confirmation. For new users, start with the strictest mode (asks before every shell command and every file change), then loosen it once you trust the workflow on your repo. The relevant modes and how to toggle them are described in the CLI docs linked in Section 16.

Where the CLI beats the extension

• Multi-step agentic runs that need to read several files, run tests, iterate, and report.

• Anything you want to script or invoke from a package.json script, a Makefile, or a CI step.

• Subagent decomposition (the CLI explicitly supports splitting a task across multiple parallel agent runs).

• MCP-connected tools and custom data sources.

• Cloud task launching from the terminal, when you do not want to leave the keyboard.

E.5 Setting Up Browser Codex (chatgpt.com/codex)

The third entry point lives outside VS Code but is essential for the full workflow because it is how you launch and monitor cloud tasks.

Open browser Codex

Navigate to chatgpt.com/codex. You will need to be signed into the same ChatGPT account you used for the extension and CLI. If you are part of an enterprise workspace, your admin must have enabled Codex Cloud at the workspace level — see Section 8.

You can also reach Codex through the sidebar in regular ChatGPT. The browser surface exposes two main verbs:

• Code — assign a coding task. Codex spins up a sandbox preloaded with your repository and produces a reviewable diff.

• Ask — ask a question about your codebase without changing any code.

Connect a GitHub repository

Cloud tasks need a GitHub-hosted repository. Connect it once:

1. Open environment settings at chatgpt.com/codex.

2. Connect your GitHub account through the ChatGPT GitHub Connector.

3. Grant access to the specific repositories you want Codex to be able to use. Do not grant org-wide access by default — see Appendix C for the security checklist.

4. Confirm the connector shows the repo as available.

Launch a task

From the Codex web interface:

1. Pick the repository and (optionally) the branch.

2. Type a prompt describing the task. Be specific — "Add input validation to the /users POST endpoint and update the matching tests" beats "Improve the API."

3. Click Code (or Ask for a non-mutating question).

4. Watch the live logs as Codex works, or close the tab and let it run in the background.

5. When it finishes, review the diff. From there you can request changes, accept the result, or open a pull request.

Delegate from a GitHub PR comment

A useful shortcut: in any PR on a connected repo, you can post a comment that tags @codex with an instruction (for example, "@codex review this PR for security issues and missing tests"). Codex will pick up the request and respond on the PR. This requires being signed into ChatGPT in the same browser.

Why the browser surface matters even if you live in VS Code

Cloud tasks decouple Codex from your local machine. You can launch a long-running task from the browser, close the laptop, and come back to the diff later. The extension and CLI cannot do this — they need an open VS Code instance to run.

E.6 When to Pick Which Entry Point

The three entry points overlap, which causes confusion. This table makes the choice mechanical.

The pattern that emerges: extension for in-flow editing, CLI for serious local agentic work, browser for anything you want offloaded or shared with the team.

E.7 The Combined VS Code Workflow

The three entry points are most powerful when used together. A representative day looks like this.

Morning, in VS Code:

1. Open the repo. The Codex extension panel is in the right sidebar.

2. Use the extension to ask questions about an unfamiliar module before you touch it.

3. Make small in-line edits — single-function changes, docstrings, type fixes — using the extension's diff-apply flow.

Mid-morning, in the integrated terminal:

1. Open the integrated terminal (Ctrl+`).

2. Run codex and start a multi-file task with explicit approval mode: "Refactor the auth middleware to use the new session interface. List the files you intend to touch first, then make the changes in the smallest commits possible."

3. Approve each shell command and each diff as Codex requests them.

4. Run the test suite when Codex finishes.

Afternoon, in the browser:

1. While you are reviewing the morning's CLI changes, open chatgpt.com/codex in another tab.

2. Launch a cloud task: "Add OpenAPI annotations to every public endpoint in the /api/v2 directory." This will take a while.

3. Switch back to VS Code and keep working. The cloud task runs in its own sandbox.

4. When the cloud task finishes, review the diff in the browser, request any tweaks, and open a PR.

End of day, on GitHub:

1. Tag @codex on a teammate's open PR with "review for correctness and missing tests." The result lands as a comment overnight.

The point of the combined workflow is that each entry point is doing what it is best at simultaneously. The extension keeps in-flow editing fast, the CLI handles local agentic work where you want approval control, and the cloud handles long-running and parallel tasks without consuming your local machine.

E.8 VS Code-Specific Tips

These are small tips that compound over time once you use Codex daily inside VS Code.

• Sidebar position. The Codex panel defaults to the right sidebar. If you also have GitHub PR review or another panel there, drag Codex to the secondary side or to a panel-bottom dock — whichever keeps it visible without stealing space from the editor.

• Keybindings. Bind the most-used Codex commands (open panel, new task, accept diff) to keyboard shortcuts via VS Code's Preferences: Open Keyboard Shortcuts. Reach for the keyboard, not the mouse.

• Settings sync. If you use VS Code's Settings Sync, the Codex extension's settings travel with you to other machines. Auth state does not — you sign in again on each machine. This is the right behavior; do not work around it.

• Multi-root workspaces. The extension scopes to the active workspace folder. If you open a multi-root workspace, switch the active folder explicitly before asking Codex to make changes, otherwise it may operate against the wrong root.

• Integrated terminal profiles. If you use multiple terminal profiles (PowerShell, bash, WSL), set the WSL profile as default on Windows so codex from the integrated terminal always lands in the supported environment.

• Source control panel. After Codex applies a change, the VS Code Source Control panel shows the diff. Review there before committing — it gives you the same context as a git diff without leaving the editor.

• Don't fight the approval mode. New users often loosen approvals to "auto" too quickly because the prompts feel slow. Resist that for the first week. The approvals are how you build a mental model of what Codex actually does in your repo.

• One Codex panel per VS Code window. Avoid running the extension and the CLI in the same workspace simultaneously on the same task — they can both touch files and you will get confused about which one made which change.

E.9 Cursor and Windsurf

The Codex extension explicitly supports Cursor and Windsurf, the two most popular VS Code forks. The install and sign-in flow is identical. The notes worth knowing:

• Avoid double-AI confusion. Cursor and Windsurf both ship their own AI features. Engineers using them with Codex sometimes accidentally invoke the fork's built-in AI when they meant to invoke Codex, or vice versa. Pick a primary tool for editing and use the other only when its specific strengths matter.

• Auth is independent. The Codex extension's ChatGPT sign-in is separate from Cursor's or Windsurf's own model accounts. Your Codex usage is billed against your ChatGPT plan; Cursor/Windsurf usage against theirs.

• Keybinding conflicts. Cursor in particular has heavily customized AI-related keybindings. Audit your bindings after installing the Codex extension to make sure both surfaces are reachable.

• Settings sync caveat. Cursor and Windsurf have their own settings sync that diverges from upstream VS Code. Codex extension settings may sync within Cursor or Windsurf separately from your VS Code installs.

For pure Codex-first teams, vanilla VS Code is the simplest baseline. For teams that already standardized on Cursor or Windsurf for other reasons, the Codex extension is a clean addition rather than a replacement.

E.10 Troubleshooting VS Code Specifically

The general troubleshooting list is in Section 12. The issues below are specific to running Codex inside VS Code.

Extension installs but sidebar panel never appears

Reload the window (Command Palette → "Developer: Reload Window"). If that does not fix it, check the Output panel, switch the dropdown to "Codex", and look for the actual error. The most common causes are a corporate proxy blocking the extension's auth handshake, or a conflicting older version of the extension still installed.

"Sign in" keeps looping back to the sign-in prompt

This usually means the redirect from the browser auth flow did not reach the extension. Try signing out completely, closing all VS Code windows, then reopening and signing in fresh. On Windows, verify your default browser is one VS Code can open via the OS handler.

codex command not found in the integrated terminal

The CLI's npm global bin directory is not on PATH. The fastest fix on macOS/Linux is to add $(npm bin -g) to your shell profile (.zshrc, .bashrc). On Windows, restart VS Code after the npm install so the integrated terminal picks up the updated PATH, or switch to a WSL terminal where the install is already on PATH.

Cloud task says "no repository connected" even though you connected one

Verify in chatgpt.com/codex environment settings that the specific repository is in the allowlist. The GitHub Connector grants per-repository access; granting access to the org alone is not enough. Also confirm your workspace admin has enabled Codex Cloud — individual users cannot enable it themselves.

Extension and CLI both editing the same file at the same time

Stop one of them. They do not coordinate, and you will get conflicting edits. The simplest discipline: pick one entry point per task, switch between tasks rather than trying to combine within a task.

Extension feels slower than the CLI for the same prompt

Often this is because the extension is using a different default model than your CLI configuration. Check both for the active model — the model picker in the extension panel, and codex --help or the relevant config file for the CLI.

Windows behavior is generally bad

Switch to a WSL workspace. OpenAI's own docs call out Windows as experimental for the CLI; the WSL path is the supported one and clears most issues at once.

Ready to Excel as an AI Engineer?

As we conclude this exploration of intelligent healthcare, it’s clear that the future belongs to those who can bridge the gap between groundbreaking research and real-world utility. If you are inspired to lead this transformation, we invite you to download our flagship resource, The AI Engineering Handbook. Authored by Tatev Aslanyan, a pioneering AI engineer and co-founder of LUNARTECH, this guide is designed to help you navigate the highly competitive landscape of AI engineering, providing you with the step-by-step roadmap and industry workflows needed to build world-changing products.

Empower yourself with the same strategies used by AI trailblazers at the world's most innovative tech companies. By mastering these production-ready skills, you won't just keep pace with the hyper-connected world — you will help define it. Get started today by downloading your eBook here: https://www.lunartech.ai/download/the-ai-engineering-handbook.

About LunarTech Lab

“Real AI. Real ROI. Delivered by Engineers — Not Slide Decks.”

LunarTech Lab is a deep-tech innovation partner specializing in AI, data science, and digital transformation – from healthcare to energy, telecom, and beyond.

We build real systems, not PowerPoint strategies. Our teams combine clinical, data, and engineering expertise to design AI that’s measurable, compliant, and production-ready. We’re vendor-neutral, globally distributed, and grounded in real AI and engineering, not hype. Our model blends Western European and North American leadership with high-performance technical teams offering world-class delivery at 70% of the Big Four’s cost.

How We Work — From Scratch, in Four Phases

1. Discovery Sprint (2–4 Weeks): We start with data and ROI – not assumptions to define what’s worth building and what’s not and how much it will cost you.

2. Pilot / Proof of Concept (8–12 Weeks): We prototype the core idea – fast, focused, and measurable.
This phase tests models, integrations, and real-world ROI before scaling.

3. Full Implementation (6–12 Months): We industrialize the solution – secure data pipelines, production-grade models, full compliance (HIPAA, MDR, GDPR), and knowledge transfer.

4. Managed Services (Ongoing): We maintain, retrain, and evolve the AI models for lasting ROI. Quarterly reviews ensure that performance improves with time, not decays. As we own LunarTech Academy, we also build customised training to ensure clients tech team can continue working without us.

Every project is designed from scratch, integrating clinical knowledge, data engineering, and applied AI research.

Why LunarTech Lab?

LunarTech Lab bridges the gap between strategy and real engineering, where most competitors fall short. Traditional consultancies, including the Big Four, sell frameworks, not systems – expensive slide decks with little execution.

We offer the same strategic clarity, but it’s delivered by engineers and data scientists who build what they design, at about 70% of the cost. Cloud vendors push their own stacks and lock clients in. LunarTech is vendor-neutral: we choose what’s best for your goals, ensuring freedom and long-term flexibility.

Outsourcing firms execute without innovation. LunarTech works like an R&D partner, building from first principles, co-creating IP, and delivering measurable ROI.

From discovery to deployment, we combine strategy, science, and engineering, with one promise: We don’t sell slides. We deliver intelligence that works.

Stay Connected with LunarTech

Follow LunarTech Lab on LunarTech NewsLetter and LinkedIn, where innovation meets real engineering. You’ll get insights, project stories, and industry breakthroughs from the front lines of applied AI and data science.

Without API documentation, application development is considered incomplete because developers cannot build software to interact with it, rendering the application effectively useless.

In this article, you will learn how to create beautiful REST API documentation that also allows you to test the APIs for free using an OpenAPI specification and Scalar in Node.js projects. You will use asteasolutions/zod-to-openapi to generate OpenAPI specification and use Scalar to create a web page from the specification.

To get the most out of this article, you should have experience developing REST APIs with Express or NestJS. You should also have experience with documenting REST APIs and using zod.

Table of Contents

• REST API Documentation Tools for Node.js

  • Swagger

  • Postman

  • ReDoc

• zod-to-openapi and Scalar for REST API Documentation

  • How zod-to-openapi Works with Scalar

• Benefits of Using zod-to-openapi and Scalar to Create REST API Documentation

  • OpenAPI Specification Support

  • Open Source and Free to Use

  • Better Documentation Experience

  • Developer-friendly UI

  • Markdown Support

• How to Create the API Documentation

  • Set up the Project

  • How to Set Up zod-to-openapi

  • How to Generate the Documentation UI with Scalar

  • Document the Endpoints

• How to Use Scalar with NestJS

• How to Resolve Content Security Policy (CSP) Errors When Used with Helmet

• Absence of AsyncAPI Documentation Feature

• Conclusion

REST API Documentation Tools for Node.js

A variety of tools already exist for documenting REST APIs and they have different strengths and weaknesses depending on their use case. While some of them are completely free to use, others operate a freemium model, and while some have an interface for testing APIs, others have a presentation-only user interface (UI).

Some of the most popular tools for documenting REST APIs in Node.js projects are listed below:

• Swagger

• Postman

• Redoc

Swagger

In order to document REST APIs in Express with Swagger, you need swagger-jsdoc and swagger-express-ui. swagger-jsdoc collates and parses JSDoc-annotated documentation comments in the codebase and generates an OpenAPI specification document. swagger-ui-express uses the generated document to created a web page that renders the API documentation and test the APIs.

One of Swagger’s strengths lies in its support for the OpenAPI specification which is an industry standard. Swagger is free to use, has a vibrant open source community and strong support for many programming languages and frameworks. It supports only REST APIs.

Its major drawback is the poor developer experience in manually writing JSDoc comments or YAML for the documentation. The process can be clumsy and developers can forget to include some annotations. Another drawback is that the JSDoc comments can interfere with reading functional code. Lastly, some developers have complained about its boring UI.

Postman

Postman is a cloud-based desktop API client application that allows developers and technical writers to write, test, collaborate on and publish API documentation. Unlike Swagger, it does not require deep programming experience to use most of its features. It is also not limited to REST API documentation — it can document APIs for GraphQL, websockets and gRPC.

Postman provides a UI to fill in details of an API documentation. The documentation process is manual and sometimes, its content can get out of sync with that of deployed applications. It is not free to use for teams and collaboration and hides the real behaviour of browser interaction with the APIs like CORS and streaming.

ReDoc

ReDoc is an open-source tool used to generate API documentation from an OpenAPI (Swagger) specification. It supports GraphQL, AsyncAPI and the OpenAPI specification and it renders a more beautiful documentation than Swagger. redoc-express is used to document Express REST APIs with Redoc.

Redoc’s major drawback is that its free community edition is presentation-only. It does not support testing the APIs. Similar to Swagger, a drawback it has is manually updating the application's OpenAPI document via a YAML specification file or JSDoc comments.

zod-to-openapi and Scalar for REST API Documentation

asteasolutions/zod-to-openapi is a TypeScript library that generates OpenAPI specification from zod schemas. It provides typed methods which serve as guardrails for documenting API components instead of using code comments so that:

• The library methods serve as guardrails for what to document and how to document it

• The documentation is consistent across the codebase

• The documentation doesn't negatively affect code readability

A sample snippet used to document a POST request for creating a user with zod-to-openapi is shown below:

// CreateUser and User are zod schema

registry.registerPath({
  method: "post",
  path: "/api/users",
  summary: "Create user",
  tags: ["users"],
  request: {
    body: {
      content: {
        "application/json": {
          schema: schema.CreateUser, 
        },
      },
      description: "Create user payload",
      required: true,
    },
  },
  responses: {
    201: {
      description: "User created",
      content: {
        "application/json": {
          schema: z.object({
            message: z.string(),
            data: schema.User,
          }),
        },
      },
    },
  },
});

Scalar is a tool that generates beautiful, organized and searchable API documentation from OpenAPI documents. The documentation generated also supports testing the APIs and this makes Scalar effectively function as an API documentation generator and a lightweight API client. The image below shows a sample documentation generated by Scalar:

How zod-to-openapi Works with Scalar

zod-to-openapi provides the functionality to generate an OpenAPI specification from code. Scalar uses the document generated to create a documentation web page that presents the information in the document in an organized and beautiful way that also allows for testing the APIs.

Benefits of Using zod-to-openapi and Scalar to Create REST API Documentation

When you combine zod-to-openapi and Scalar to create REST API documentation for your Express applications, you get a myriad of benefits. Some of the benefits are explained below:

OpenAPI Specification Support

The OpenAPI specification is a format for describing REST APIs. It takes takes into consideration the important components of an API necessary for clients to use it effectively. These components include:

• Request paths, methods, headers, path parameters and query parameters,

• Schemas of request and response payloads,

• Authentication requirements,

• Descriptions for information not accommodated by other components

zod-to-openapi provides methods for all of these to be included in the documentation and it generates an OpenAPI specification-compliant document that Scalar uses to generate the documentation web page.

Open Source and Free to Use

zod-to-openapi is open source and free to use. It has no plans to be unsupported or sunsetted soon because like Ruby on Rails and Laravel, the creators of the project use it in their day-to-day work.

Scalar is open source too. It has paid plans but the features in the paid plans are only really useful for enterprise applications. The free version supports the necessary features needed to create useful REST API documentation.

Better Documentation Experience

In terms of user experience, the union of zod-to-openapi and Scalar provides the following benefits when writing documentation with both tools:

Guardrails with zod-to-openapi Methods

The methods provided by zod-to-openapi serve as guardrails to ensure that developers don't omit or forget the documentation of important components of APIs. The methods also ensure that these components are documented in an OpenAPI specification-compliant manner through the typed nature of the methods' parameters.

Avoid the Clumsiness of Comments and YAML Files

With zod-to-openapi, you don't document the APIs using comments or a YAML file. You document APIs using methods from zod-to-openapi. This removes the cluttering of code with comments and the clumsiness around manually updating large YAML files of OpenAPI specification.

Accuracy and Auto-generation of Documentation

When you use zod-to-openapi and Scalar, your API documentation is generated automatically when the application runs. zod-to-openapi does the collation and compilation of the documented APIs, and Scalar creates a web page for it that can be hosted on an API route of the same application. You don't need to manually run CLI commands to generate the documentation.

Another benefit of accuracy and auto-generation is that the job of API documentation is not split between technical writer and backend developer. The documentation is in the code and this makes development faster and more seamless in terms of API documentation.

Developer-friendly UI

While Swagger’s UI is functional, some developers consider its presentation somewhat minimal, particularly when displaying detailed endpoint descriptions. ReDoc improves on visual design but does not offer API testing features. Scalar, on the other hand, delivers a more refined and intuitive interface with greater customization options than ReDoc.

Beyond its design advantages, Scalar provides auto-generated code samples in multiple programming languages. This enables developers to integrate APIs more efficiently using examples tailored to their specific tech stack.

Scalar's UI provides a search feature that allows developers to quickly locate specific sections of the API documentation. It also includes an AI chat interface that enables users to understand how different API endpoints can help address their specific use cases. This approach is more efficient than manually reviewing the entire documentation.

Lastly, you can test the API endpoints from the UI. When you make authenticated API requests, the UI caches authentication tokens so that you don't have to type or paste them for subsequent requests.

Markdown Support

zod-to-openapi and Scalar have Markdown support. With Markdown, you can include conceptual documentation and more information about API endpoints that are not supported by the default documentation components like headers and the request body.

You can embed images, include tables and format text in the documentation. You can use Markdown to include notes that explain concepts related to the API.

How to Create the API Documentation

In this section, you will create an Express CRUD API project that uses zod-to-openapi and Scalar to document its APIs. To practice along, clone the Express starter project from GitHub at orimdominic/freeCodeCamp-zod-to-openapi-scalar.

Set up the Project

After cloning the project:

• install its dependencies using your preferred Node.js package manager

• start the server using the serve script

# Install dependencies
npm install

# Start the application
npm run serve

You should see the following output on the terminal if the application runs successfully:

> freecodecamp-zod-to-openapi-scalar@1.0.0 serve
> node --experimental-strip-types --watch src/index.ts

Listening on :3000

The project has two modules - Users and Pets.

The router configuration for each module is defined in the router.ts file, while the route controllers are located in the controllers.ts file within each module's folder under src/modules. The controllers do not contain business logic, they simply respond with JSON values generated by the Faker library.

How to Set Up zod-to-openapi

Install asteasolutions/zod-to-openapi using your preferred Node.js package manager. If you use npm, run the code snippet below in your terminal:

npm install @asteasolutions/zod-to-openapi

After the installation, create a folder called lib (library) in the src folder. In the lib folder, create a file called openapi.ts. The file will house the code that sets up zod-to-openapi for collating the API documentation and generating the OpenAPI specification.

Copy and paste the code snippet below into src/lib/openapi.ts:

import z from "zod";
import {
  extendZodWithOpenApi,
  OpenApiGeneratorV31,
  OpenAPIRegistry,
} from "@asteasolutions/zod-to-openapi";

extendZodWithOpenApi(z);

export const registry = new OpenAPIRegistry();

export const bearerAuth = registry.registerComponent("securitySchemes", "bearerAuth", {
  type: "http",
  scheme: "bearer",
  bearerFormat: "JWT",
});

export function generateOpenAPIDocument() {
  const generator = new OpenApiGeneratorV3(registry.definitions);

  return generator.generateDocument({
    openapi: "3.1.0",
    info: {
      title: "Users API",
      version: "1.0.0",
      description: `Backend API documentation for users application.`,
    },
    tags: [
      {
        name: "users",
        description: "For operations carried out by admin users",
      },
    ],
    servers: [
      {
        url: "http://localhost:3000",
        description: "Local server",
      },
    ],
  });
}

zod-to-openapi v8 requires zod v4. If you use zod v3, you should use v7.3.4 of zod-to-openapi.

extendZodWithOpenApi is a method provided by zod-to-openapi that enhances Zod schemas by adding an openapi method. The openapi method allows you to attach additional documentation to request payloads, responses, parameters, and their properties, which are then displayed in the API documentation rendered by Scalar.

It is important to call extendZodWithOpenApi before loading any files that use the openapi method, otherwise accessing openapi on Zod objects will result in errors.

An alternative is to use the meta method on zod v4 schemas for the additional documentation. For example, schemaOne and schemaTwo in the code snippet below are the same:

const schemaOne = z
  .string()
  .openapi({ description: 'Name of the user', example: 'Test' });

const schemaTwo = z
  .string()
  .meta({description: 'Name of the user', example: 'Test' });

The meta method supports all metadata information that you'd normally pass to openapi and will produce exactly the same results.

The OpenAPIRegistry is a utility that is used to collate API documentation which would later be passed to an OpenAPI specification generator. registry is created from OpenAPIRegistry, exported, and used to document API endpoints and components in modules where it is imported.

export const registry = new OpenAPIRegistry();

bearerAuth is a component created by the registry to represent JWT authentication. When bearerAuth is included in the documentation of an endpoint, the UI renders an input for submitting an authentication token for authenticated requests as shown in the image below.

In the registerComponent method, "securitySchemes" registers a security scheme component. "bearerAuth" , the first argument of registerComponent, is the name given to the component and it can be changed to a name that you prefer. It appears in the top right of the authentication token input, shown in the image above. The third input to registerComponent is an object that defines the component.

When generateOpenAPIDocument function is executed, it collates all the registry API definitions in the project, generates the OpenAPI specification through generator.generateDocument, and returns the specification as JSON.

The tags property in generator.generateDocument organizes API endpoints into sections on the documentation UI. For example, all API endpoints with the Users tag in their registry definition will be placed under the Users section of the UI. description can be written in Markdown within template literals.

The servers property is a collection of the servers connected to the application. If you have multiple servers, you have the option of selecting what server to use for the base URL in the documentation UI for making API requests from it.

With this setup in place, when endpoints are documented with the registry, generateOpenAPIDocument will have an OpenAPI specification to return.

How to Generate the Documentation UI with Scalar

In this section, you will set up Scalar and connect it to the return value of generateOpenAPIDocument. You will also connect Scalar with an Express route, allowing the application to serve the documentation UI at that route.

Scalar has an Express API reference library that makes it easier for you to connect it with the OpenAPI specification and Express. Install scalar/express-api-reference using your preferred Node.js package manager. If you use npm, use the snippet below:

npm install @scalar/express-api-reference 

Copy and paste the code snippet below into src/app.js:

import express from "express";
import router from "./router.ts";
import { generateOpenAPIDocument } from "./lib/openapi.ts";
import { apiReference } from "@scalar/express-api-reference";

const app = express();
app.use(express.json(), express.urlencoded({ extended: true }));

app.get("/", function (req, res) {
  return res.send("OK");
});

app.use("/api", router);

const apiDocJsonContent = generateOpenAPIDocument();

app.use(
  "/docs", // documentation route
  apiReference({
    content: apiDocJsonContent,
    title: "Users API",
    pageTitle: "Users API",
  }),
);

export default app;

In the code snippet above, generateOpenAPIDocument is imported from src/lib/openapi.ts, and apiReference is imported from @scalar/express-api-reference. When executed, generateOpenAPIDocument returns the OpenAPI specification, which is stored in apiDocJsonContent for caching to improve perfoemance.

A GET /docs route is then created, with the Scalar apiReference function acting as the controller. It accepts apiDocJsonContent and returns a web page whenever the GET /docs route is accessed.

With this setup in place, run the application using npm run serve and visit the documentation page at http://localhost:3000/docs in your browser. You should see a user interface similar to the image below:

To view the codebase at this point, run git checkout set-up-openapi-scalar .

Document the Endpoints

You have set up zod-to-openapi and connected it with Scalar. You have also hooked it up with a route in the backend application. In this section, you will write code to document the endpoints in the application for generating the OpenAPI specification and rendering it on the documentation UI.

To document the route for creating users ( POST /api/users ), in src/modules/users/router.ts , import registry, the schemas and zod using the snippet below:

import z from "zod";
import { registry } from "../../lib/openapi.ts";
import { 
    UserSchema, 
    UserListItemSchema,
    UpdateUserSchema, 
    CreateUserSchema, 
} from "./types.ts";

Copy and paste the code below above the create user route to document the create user endpoint:

registry.registerPath({
  method: "post",
  path: "/api/users",
  summary: "Create user",
  tags: ["users"],
  request: {
    body: {
      content: {
        "application/json": {
          schema: CreateUserSchema,
        },
      },
      description: "Create user payload",
      required: true,
    },
  },
  responses: {
    201: {
      description: "User created",
      content: {
        "application/json": {
          schema: z.object({
            message: z.string().openapi({ example: "User created" }),
            data: UserSchema,
          }),
        },
      },
    },
  },
});

Visit the documentation page and you will find see a web page similar to the image below:

The UI result of some of the input fields of registry.registerPath have been labelled in the image above. The description in the API endpoint is italicised because its value is Markdown in a template string.

By registering the route path for creating users with registry.registerPath and filling its values, you added the documentation of the route to the registry definitions and that makes it included in the OpenAPI specification.

To test the endpoint from the documentation UI:

• click the Test Request button

• fill in the payload in the dialog that appears and

• click the Send button

To document the get user by id route (GET /api/users/:id ), import bearerAuth from src/lib/openapi.ts, copy the code snippet below and paste it above the get user by id route definition.

registry.registerPath({
  method: "get",
  path: "/api/users/{userId}",
  summary: "Get user details by id",
  tags: ["Users"],
  security: [{ [bearerAuth.name]: [] }],
  request: {
    params: z.object({ userId: z.int() }),
  },
  responses: {
    200: {
      description: "User retrieved",
      content: { "application/json": { schema: UserSchema } },
    },
  },
});

When the request.params field is defined using a Zod object, it generates an input UI on the documentation web page that enables users to provide values for path parameters such as userId, highlighted in the image below:

The complete code for the documentation of all endpoints in this section can be accessed when you check out the complete-project branch by running git checkout complete-project in your terminal. It contains documentation for the endpoint for uploading user photo, which demonstrates how to document endpoints that accept file uploads.

How to Use Scalar with NestJS

Scalar has a library that integrates with NestJS. You can use supply the Swagger document created by swagger/nestjs to the Scalar NestJS integration library to generate the Scalar documentation UI.

In root folder of your NestJS project, install the Scalar NestJS integration library:

npm install @scalar/nestjs-api-reference

Update the main.ts file of your NestJS project with the code snippet below:

import { NestFactory } from '@nestjs/core';
import { apiReference } from '@scalar/nestjs-api-reference';
import { DocumentBuilder, SwaggerModule } from '@nestjs/swagger';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);

  const options = new DocumentBuilder()
    .setTitle('Cats example')
    .setDescription('The cats API description')
    .setVersion('1.0')
    .addTag('cats')
    .addBearerAuth()
    .build();

  const openApiSpecification = SwaggerModule.createDocument(app, options);
  
  // integrate the documentation with NestJS
  app.use(
    '/api/docs', // documentation route
    apiReference({
      content: openApiSpecification,
    }),
  )

  await app.listen(3000);
  console.log(`Application is running on: ${await app.getUrl()}`);
}

bootstrap();

With this setup in place, you can visit the /api/docs route in your browser to view the Scalar documentation for your NestJS application.

How to Resolve Content Security Policy (CSP) Errors When Used with Helmet

If you use Helmet in your Express or NestJS project, you will encounter CSP errors when you try to render the Scalar documentation UI. To resolve the errors, update the Helmet CSP configuration in your code to have the value of the object in the code snippet below:

{
    directives: {
      defaultSrc: [`'self'`],
      styleSrc: [`'self'`, `'unsafe-inline'`],
      imgSrc: [`'self'`, 'data:', 'validator.swagger.io'],
      scriptSrc: ['self', 'https:', 'unsafe-inline'],
    },
  }

Absence of AsyncAPI Documentation Feature

At the time of writing, Scalar does not fully support rendering AsyncAPI specifications for event-driven architecture APIs, although it is currently under development. You can track the progress of its development through the GitHub issue linked in the documentation to stay informed about its release.

Conclusion

You have learned about zod-to-openapi and how it makes it easier for you to generate an OpenAPI specification for your REST APIs than writing comments or large YAML files. You also learned how to use the specification document generated to render a beautiful API documentation UI which also functions as a lightweight API client. Endeavour to implement it in your projects that need a documentation uplift.

Feel free to connect with me on LinkedIn for questions or clarifications. Thank you for reading this far and I hope this helps you achieve what you intended to achieve. Don’t hesitate to share this article if you feel that it would help someone else out there. Cheers!

By the end of this guide, you'll have a fully functional application that can:

• Upload and process PDF, DOCX, and TXT files

• Store documents in Supabase Storage

• Generate embeddings using OpenAI

• Perform semantic search across document chunks

• Provide AI-generated answers based on document content

• View and manage uploaded documents

This is a production-ready solution that you can deploy and use immediately.

Table of Contents

• What You'll Learn

• Prerequisites

• Understanding the Technologies

• Project Overview

• Step 1: Create Your Next.js Project

• Step 2: Install Required Dependencies

• Step 3: Set Up Your Supabase Project

• Step 4: Configure Environment Variables

• Step 5: Create the Upload API Route

• Step 6: Create the RAG Search API Route

• Step 7: Create the Documents API Route

• Step 8: Create the Upload Modal Component

• Step 9: Create the PDF Viewer Modal Component

• Step 10: Create the Navigation Component

• Step 11: Create the Home Page (Search Interface)

• Step 12: Create the Documents Page

• Step 13: Test Your Application

• Step 14: Deploy Your Application

• How RAG Search Works

• Troubleshooting Common Issues

• Next Steps

• Conclusion

What You'll Learn

In this handbook, you'll learn how to:

• Set up a Next.js application with TypeScript

• Configure Supabase for database and file storage

• Integrate OpenAI embeddings and chat completions

• Implement document text extraction and chunking

• Build a vector search system using PostgreSQL

• Create a modern UI with React components

• Handle file uploads and storage

• Implement RAG (Retrieval-Augmented Generation) search

Prerequisites

Before you begin, make sure you have:

• Node.js 18 or higher installed on your computer

• A Supabase account (free tier works fine)

• An OpenAI API key

• Basic knowledge of React and TypeScript

• Familiarity with Next.js (helpful but not required)

Understanding the Technologies

Before we dive into building the application, you should understand the key technologies and concepts you'll be working with:

What is RAG (Retrieval-Augmented Generation)?

RAG is an AI pattern that combines information retrieval with text generation. Instead of relying solely on an AI model's training data, RAG retrieves relevant information from your own documents. It then uses that information as context to generate accurate, up-to-date answers. This approach gives you:

• Accuracy: Answers are based on your actual documents, not just the AI's training data

• Transparency: You can see which document sections were used to generate the answer

• Efficiency: Only relevant document chunks are used, reducing token costs

What are Embeddings and Vector Database?

Embeddings are numerical representations of text that capture semantic meaning. When you convert text to an embedding, similar meanings are represented by similar numbers. For example, "dog" and "puppy" would have similar embeddings. Meanwhile, "dog" and "airplane" would have very different ones.

OpenAI's embedding models convert text into vectors. These are arrays of numbers that can be compared mathematically. This allows you to find documents that are semantically similar to a search query. You can find matches even if they don't contain the exact same words.

A vector database is a specialized database designed to store and search through embeddings efficiently. Instead of searching for exact text matches, vector databases use mathematical operations. They use operations like cosine similarity to find the most semantically similar content.

In this tutorial, you'll use Supabase's PostgreSQL database with the pgvector extension. This extension adds vector storage and similarity search capabilities to PostgreSQL. This lets you store embeddings alongside your regular database data. You can also perform fast similarity searches.

What is Text Chunking?

Text chunking is the process of breaking large documents into smaller, manageable pieces. This is necessary for several reasons.

First, AI models have token limits. These are maximum input sizes. Second, smaller chunks allow for more precise retrieval. Third, overlapping chunks ensure context isn't lost at boundaries.

You'll use LangChain's RecursiveCharacterTextSplitter. This tool intelligently splits text while trying to preserve sentence and paragraph boundaries.

What is Supabase?

Supabase is an open-source Firebase alternative. It provides several key features.

You get a PostgreSQL database, which is a powerful, open-source relational database. You also get storage, which is file storage similar to AWS S3. There are real-time features that provide real-time subscriptions to database changes. Finally, there's built-in user authentication.

For this project, you'll use Supabase's database to store document chunks and embeddings. You'll also use Supabase Storage to store the original uploaded files.

What is Tailwind CSS?

Tailwind CSS is a utility-first CSS framework that lets you style your application by applying pre-built utility classes directly in your HTML/JSX. Instead of writing custom CSS, you use classes like bg-blue-600, text-white, and rounded-lg to style elements.

You'll use Tailwind CSS in this project because it speeds up development by providing ready-made styling utilities. It also ensures consistent design across the application. Plus, it makes it easy to create responsive, modern UIs. Finally, it works seamlessly with Next.js.

Now that you understand the core concepts and tools we’ll be using, let's start building the application.

Project Overview

Your RAG search application will consist of:

1. Frontend: Next.js application with React components for uploading documents and searching

2. Backend API Routes: Next.js API routes for handling uploads, searches, and document management

3. Database: Supabase PostgreSQL with vector extension for storing embeddings

4. Storage: Supabase Storage for storing original files

5. AI Integration: OpenAI for generating embeddings and chat completions

The application will have two main pages:

• Search Page: Where users can ask questions about their uploaded documents and get AI-generated answers

• Documents Page: Where users can view all uploaded documents, upload new ones, preview files, and manage their document library

Let's start building!

If you ever get stuck on the source code, you can view it on GitHub here:

https://github.com/mayur9210/rag-search-app

Step 1: Create Your Next.js Project

Start by creating a new Next.js project with TypeScript. Open your terminal and run:

npx create-next-app@latest rag-search-app --typescript --tailwind --app

When prompted, choose the following options:

• TypeScript: Yes

• ESLint: Yes

• Tailwind CSS: Yes

• App Router: Yes (default)

• Customize import alias: No

Navigate into your project directory:

cd rag-search-app

Now that your project is set up, you'll need to install the additional packages required for document processing, AI integration, and database operations.

Step 2: Install Required Dependencies

You'll need several packages for this project. You can install them using npm:

npm install @supabase/supabase-js @langchain/openai @langchain/textsplitters langchain openai mammoth pdf2json

Here's what each package does:

• @supabase/supabase-js: Client library for interacting with Supabase (database and storage)

• @langchain/openai: LangChain integration for OpenAI (helps with text processing)

• @langchain/textsplitters: Text splitting utilities for chunking documents into smaller pieces

• langchain: Core LangChain library (provides AI workflow tools)

• openai: Official OpenAI SDK (for generating embeddings and chat completions)

• mammoth: Converts DOCX files to plain text

• pdf2json: Extracts text from PDF files

Install the TypeScript types for pdf2json:

npm install --save-dev @types/pdf-parse

With all dependencies installed, you're ready to set up your Supabase project, which will handle your database and file storage needs.

Step 3: Set Up Your Supabase Project

Create a Supabase Project

First, you’ll need to create a new Supabase project, which you can do by following these steps:

1. Go to supabase.com and sign in or create an account

2. Click "New Project"

3. Fill in your project details:

   • Name: rag-search-app (or any name you prefer)

   • Database Password: Choose a strong password (save this – you'll need it)

   • Region: Select the region closest to you

4. Click "Create new project" and wait for it to be ready (this takes a few minutes)

Get Your Supabase Credentials

Once your project is ready, go to Settings and then API.

Copy the following values:

• Project URL (this is your NEXT_PUBLIC_SUPABASE_URL)

• anon public key (this is your NEXT_PUBLIC_SUPABASE_PUBLISHABLE_DEFAULT_KEY)

• service_role key (this is your SUPABASE_SERVICE_ROLE_KEY)

Important: Keep your service role key secret. Never expose it in client-side code. It bypasses Row-Level Security (RLS) policies, which is necessary for server-side file uploads but should never be used in browser code.

Set Up the Database Schema

Now you'll set up the database structure to store your documents and embeddings. Go to SQL Editor in your Supabase dashboard and run the following SQL:

-- Enable the vector extension for embeddings
-- This extension allows PostgreSQL to store and search vector data efficiently
CREATE EXTENSION IF NOT EXISTS vector;

-- Create the documents table
-- This table stores document chunks, their metadata, and embeddings
CREATE TABLE documents (
  id BIGSERIAL PRIMARY KEY,
  content TEXT NOT NULL,
  metadata JSONB,
  embedding vector(1536)  -- OpenAI's text-embedding-3-small produces 1536-dimensional vectors
  file_path text null,
  file_url text null,
);

-- Create an index on the embedding column for faster similarity search
-- The ivfflat index speeds up vector similarity queries
CREATE INDEX ON documents USING ivfflat (embedding vector_cosine_ops);

-- Create a function for matching documents based on similarity
-- This function finds the most similar document chunks to a query embedding
CREATE OR REPLACE FUNCTION match_documents(
  query_embedding vector(1536),
  match_threshold float,
  match_count int
)
RETURNS TABLE (
  id bigint,
  content text,
  metadata jsonb,
  similarity float
)
LANGUAGE plpgsql
AS $$
BEGIN
  RETURN QUERY
  SELECT
    documents.id,
    documents.content,
    documents.metadata,
    1 - (documents.embedding <=> query_embedding) AS similarity
  FROM documents
  WHERE 1 - (documents.embedding <=> query_embedding) > match_threshold
  ORDER BY documents.embedding <=> query_embedding
  LIMIT match_count;
END;
$$;

This SQL does the following:

• Enables the vector extension: This adds vector storage and similarity search capabilities to PostgreSQL

• Creates the documents table: Stores document chunks, metadata (file name, type, and so on), and their embeddings

• Creates an index: Speeds up similarity searches on the embedding column

• Creates a match function: Finds the most similar document chunks to a query embedding using cosine similarity

The <=> operator calculates cosine distance between vectors. A smaller distance means more similar content.

Set Up Supabase Storage

You’ll need a storage bucket to store uploaded files. This is separate from the database and holds the original PDF, DOCX, and TXT files.

To set up your storage bucket:

1. Go to Storage in your Supabase dashboard

2. Click New bucket

3. Name it documents

4. Set it to Public (this allows file downloads)

5. Click Create bucket

If you prefer a private bucket, you can use the service role key for server-side operations, which bypasses Row-Level Security policies. For this tutorial, a public bucket is simpler and works well.

Now that your Supabase project is configured, you'll set up your environment variables to connect your Next.js application to Supabase and OpenAI.

Step 4: Configure Environment Variables

Create a .env.local file in your project root:

NEXT_PUBLIC_SUPABASE_URL=your_supabase_project_url
NEXT_PUBLIC_SUPABASE_PUBLISHABLE_DEFAULT_KEY=your_supabase_anon_key
SUPABASE_SERVICE_ROLE_KEY=your_supabase_service_role_key
OPENAI_API_KEY=your_openai_api_key

Replace the placeholder values with your actual credentials:

• Get Supabase values from Settings → API in your Supabase dashboard

• Get your OpenAI API key from platform.openai.com/api-keys

Security Note: Never commit .env.local to version control. It's already in .gitignore by default, but double-check to ensure your secrets stay secure.

With your environment configured, you're ready to start building the API routes that will handle file uploads, searches, and document management.

Step 5: Create the Upload API Route

Now you'll create the API route that handles file uploads. This route will process uploaded files, extract their text, split them into chunks, generate embeddings, and store everything in your database and storage.

Create src/app/api/upload/route.ts:

import { createClient } from '@supabase/supabase-js';
import OpenAI from 'openai';
import { NextResponse } from 'next/server';
import { RecursiveCharacterTextSplitter } from '@langchain/textsplitters';
import mammoth from 'mammoth';

const url = process.env.NEXT_PUBLIC_SUPABASE_URL!;
const anonKey = process.env.NEXT_PUBLIC_SUPABASE_PUBLISHABLE_DEFAULT_KEY!;
const serviceKey = process.env.SUPABASE_SERVICE_ROLE_KEY;
const supabaseStorage = createClient(url, serviceKey || anonKey);
const supabase = createClient(url, anonKey);
const openai = new OpenAI();

function safeDecodeURIComponent(str: string): string {
  try { 
    return decodeURIComponent(str); 
  } catch { 
    try { 
      return decodeURIComponent(str.replace(/%/g, '%25')); 
    } catch { 
      return str; 
    } 
  }
}

async function extractTextFromFile(file: File): Promise<string> {
  const buffer = Buffer.from(await file.arrayBuffer());
  const fileName = file.name.toLowerCase();

  if (fileName.endsWith('.pdf')) {
    const PDFParser = (await import('pdf2json')).default;
    return new Promise((resolve, reject) => {
      const pdfParser = new (PDFParser as any)(null, true);
      pdfParser.on('pdfParser_dataError', (err: any) => 
        reject(new Error(`PDF parsing error: ${err.parserError}`))
      );
      pdfParser.on('pdfParser_dataReady', (pdfData: any) => {
        try {
          let fullText = '';
          pdfData.Pages?.forEach((page: any) => 
            page.Texts?.forEach((text: any) => 
              text.R?.forEach((r: any) => 
                r.T && (fullText += safeDecodeURIComponent(r.T) + ' ')
              )
            )
          );
          resolve(fullText.trim());
        } catch (error: any) {
          reject(new Error(`Error extracting text: ${error.message}`));
        }
      });
      pdfParser.parseBuffer(buffer);
    });
  } else if (fileName.endsWith('.docx')) {
    const result = await mammoth.extractRawText({ buffer });
    return result.value;
  } else if (fileName.endsWith('.txt')) {
    return buffer.toString('utf-8');
  } else {
    throw new Error('Unsupported file type. Please upload PDF, DOCX, or TXT files.');
  }
}

export async function POST(req: Request) {
  try {
    const file = (await req.formData()).get('file') as File;
    if (!file) {
      return NextResponse.json({ error: 'No file provided' }, { status: 400 });
    }

    const documentId = crypto.randomUUID();
    const uploadDate = new Date().toISOString();
    const filePath = `${documentId}.${file.name.split('.').pop() || 'bin'}`;

    // Upload file to Supabase Storage
    const fileBuffer = Buffer.from(await file.arrayBuffer());
    const { error: storageError } = await supabaseStorage.storage
      .from('documents')
      .upload(filePath, fileBuffer, {
        contentType: file.type || 'application/octet-stream',
        upsert: false,
      });

    if (storageError) {
      const msg = storageError.message || 'Unknown storage error';
      if (msg.includes('row-level security') || msg.includes('RLS')) {
        return NextResponse.json({ 
          success: false, 
          error: `Storage RLS error: ${msg}. Ensure SUPABASE_SERVICE_ROLE_KEY is set.` 
        }, { status: 500 });
      }
      return NextResponse.json({ 
        success: false, 
        error: `Failed to store file: ${msg}` 
      }, { status: 500 });
    }

    // Get public URL for the file
    const { data: urlData } = supabaseStorage.storage
      .from('documents')
      .getPublicUrl(filePath);

    // Extract text from file
    const text = await extractTextFromFile(file);
    if (!text || text.trim().length === 0) {
      return NextResponse.json({ 
        error: 'Could not extract text from file' 
      }, { status: 400 });
    }

    // Split text into chunks
    // Chunk size of 800 characters with 100-character overlap ensures
    // we don't lose context at chunk boundaries
    const textSplitter = new RecursiveCharacterTextSplitter({
      chunkSize: 800,
      chunkOverlap: 100,
    });
    const chunks = await textSplitter.splitText(text);

    // Process each chunk: generate embedding and store in database
    for (let i = 0; i < chunks.length; i++) {
      const chunk = chunks[i];

      // Generate embedding using OpenAI
      // This converts the text chunk into a 1536-dimensional vector
      const emb = await openai.embeddings.create({
        model: 'text-embedding-3-small',
        input: chunk,
      });

      // Store chunk with embedding in database
      const { error } = await supabase.from('documents').insert({
        content: chunk,
        metadata: { 
          source: file.name,
          document_id: documentId,
          file_name: file.name,
          file_type: file.type || file.name.split('.').pop(),
          file_size: file.size,
          upload_date: uploadDate,
          chunk_index: i,
          total_chunks: chunks.length,
          file_path: filePath,
          file_url: urlData.publicUrl,
        },
        embedding: JSON.stringify(emb.data[0].embedding),
      });

      if (error) {
        return NextResponse.json({ 
          success: false, 
          error: error.message 
        }, { status: 500 });
      }
    }

    return NextResponse.json({ 
      success: true, 
      documentId, 
      fileName: file.name, 
      chunks: chunks.length, 
      textLength: text.length, 
      fileUrl: urlData.publicUrl 
    });
  } catch (error: any) {
    return NextResponse.json({ 
      success: false, 
      error: error.message || 'Failed to process file' 
    }, { status: 500 });
  }
}

This route handles the complete upload workflow:

1. Receives the file from the client via FormData

2. Generates a unique document ID using crypto.randomUUID()

3. Uploads the file to Supabase Storage for safekeeping

4. Extracts text based on file type (PDF, DOCX, or TXT)

5. Splits the text into chunks of 800 characters with 100-character overlap

6. Generates embeddings for each chunk using OpenAI's embedding model

7. Stores each chunk with its embedding and metadata in the database

The overlap between chunks ensures that if a sentence or concept spans a chunk boundary, it won't be lost. Now that you can upload and process documents, let's create the search functionality.

Step 6: Create the RAG Search API Route

This route implements the core RAG functionality: it takes a user's query, finds the most relevant document chunks, and uses them to generate an accurate answer.

Create src/app/api/search/route.ts:

import { createClient } from '@supabase/supabase-js';
import OpenAI from 'openai';
import { NextResponse } from 'next/server';

const supabase = createClient(
  process.env.NEXT_PUBLIC_SUPABASE_URL!,
  process.env.NEXT_PUBLIC_SUPABASE_PUBLISHABLE_DEFAULT_KEY!
);
const openai = new OpenAI();

export async function POST(req: Request) {
  try {
    const { query } = await req.json();

    // Generate embedding for the user's query
    // This converts the search query into the same vector space as document chunks
    const emb = await openai.embeddings.create({ 
      model: 'text-embedding-3-small', 
      input: query 
    });

    // Find similar documents using vector similarity search
    // The match_documents function finds the 5 most similar chunks
    const { data: results, error } = await supabase.rpc('match_documents', {
      query_embedding: JSON.stringify(emb.data[0].embedding),
      match_threshold: 0.0,  // Accept any similarity (you can increase this for stricter matching)
      match_count: 5,        // Return top 5 most similar chunks
    });

    if (error) {
      return NextResponse.json({ error: error.message }, { status: 500 });
    }

    // Combine retrieved chunks into context
    // These chunks will be used as context for the AI to generate an answer
    const context = results?.map((r: any) => r.content).join('\n---\n') || '';

    // Generate answer using OpenAI with retrieved context
    // This is the "Generation" part of RAG
    const completion = await openai.chat.completions.create({
      model: 'gpt-4o-mini',
      messages: [
        { 
          role: 'system', 
          content: 'You are a helpful assistant. Use the provided context to answer questions. If the answer is not in the context, say you do not know.' 
        },
        { 
          role: 'user', 
          content: `Context: ${context}\n\nQuestion: ${query}` 
        }
      ],
    });

    return NextResponse.json({ 
      answer: completion.choices[0].message.content, 
      sources: results 
    });
  } catch (error: any) {
    return NextResponse.json({ error: error.message }, { status: 500 });
  }
}

This route implements the RAG pattern. Here's how the complete RAG workflow works:

1. Converts the query to an embedding: The user's question is transformed into the same vector space as your document chunks. This uses the same embedding model (text-embedding-3-small) that processed the documents, ensuring they're in the same "vector space."

2. Searches for similar chunks: Uses the match_documents function to find the 5 most semantically similar document chunks. This uses cosine similarity on the embeddings. Cosine similarity measures the angle between vectors - smaller angles mean more similar content, even if the exact words differ.

3. Uses chunks as context: The retrieved chunks are passed to GPT-4o-mini as context. These chunks contain the most relevant information from your documents.

4. Generates an answer: The AI model generates an answer based on the provided context. The system prompt instructs the AI to only answer based on the provided context, ensuring accuracy and preventing hallucinations.

5. Returns results: Both the answer and source chunks are returned so users can verify the information.

This RAG approach gives you several benefits. First, you get accuracy because answers are based on your actual documents, not just the AI's training data. Second, you get transparency because you can see which document chunks were used to generate each answer. Third, you get efficiency because only relevant chunks are used, which reduces token usage and costs. Finally, you get up-to-date information because you can update your knowledge base by uploading new documents without retraining the AI.

Now let's create the API route for managing documents.

Step 7: Create the Documents API Route

This route handles listing, viewing, downloading, and deleting documents. It serves multiple purposes depending on the query parameters.

Create src/app/api/documents/route.ts:

import { createClient } from '@supabase/supabase-js';
import { NextResponse } from 'next/server';

const url = process.env.NEXT_PUBLIC_SUPABASE_URL!;
const anonKey = process.env.NEXT_PUBLIC_SUPABASE_PUBLISHABLE_DEFAULT_KEY!;
const serviceKey = process.env.SUPABASE_SERVICE_ROLE_KEY || anonKey;
const supabase = createClient(url, anonKey);
const supabaseStorage = createClient(url, serviceKey);

export async function GET(req: Request) {
  try {
    const reqUrl = new URL(req.url);
    const id = reqUrl.searchParams.get('id');
    const file = reqUrl.searchParams.get('file') === 'true';
    const view = reqUrl.searchParams.get('view') === 'true';

    // Handle file download/view
    if (id && file) {
      const { data: documents } = await supabase
        .from('documents')
        .select('metadata')
        .eq('metadata->>document_id', id)
        .limit(1);

      if (!documents || documents.length === 0) {
        return NextResponse.json({ error: 'Document not found' }, { status: 404 });
      }

      const meta = documents[0].metadata;
      const fileName = meta?.file_name || 'document';
      const fileType = meta?.file_type || 'application/octet-stream';
      const filePath = meta?.file_path || `${id}.${fileName.split('.').pop() || 'pdf'}`;

      const { data: fileData, error: downloadError } = await supabaseStorage.storage
        .from('documents')
        .download(filePath);

      if (downloadError || !fileData) {
        return NextResponse.json({ 
          error: downloadError?.message || 'File not stored' 
        }, { status: 404 });
      }

      const buffer = Buffer.from(await fileData.arrayBuffer());
      if (buffer.length === 0) {
        return NextResponse.json({ error: 'File is empty' }, { status: 500 });
      }

      const isPDF = fileType === 'application/pdf' || fileName.toLowerCase().endsWith('.pdf');
      return new NextResponse(new Uint8Array(buffer), {
        headers: {
          'Content-Type': fileType,
          'Content-Disposition': (view && isPDF) 
            ? `inline; filename="${fileName}"` 
            : `attachment; filename="${fileName}"`,
          'Content-Length': buffer.length.toString(),
          ...(view && isPDF ? { 'X-Content-Type-Options': 'nosniff' } : {}),
        },
      });
    }

    // Get single document with text content
    if (id) {
      const { data: chunks, error } = await supabase
        .from('documents')
        .select('content, metadata')
        .eq('metadata->>document_id', id)
        .order('metadata->>chunk_index', { ascending: true });

      if (error || !chunks || chunks.length === 0) {
        return NextResponse.json({ error: 'Document not found' }, { status: 404 });
      }

      const m = chunks[0].metadata || {};
      return NextResponse.json({
        id,
        file_name: m.file_name || 'Unknown',
        file_type: m.file_type || 'unknown',
        file_size: m.file_size || 0,
        upload_date: m.upload_date || new Date().toISOString(),
        total_chunks: chunks.length,
        fullText: chunks.map((c: any) => c.content).join('\n\n'),
        file_url: m.file_url,
        file_path: m.file_path
      });
    }

    // List all documents
    const { data: documents, error } = await supabase
      .from('documents')
      .select('metadata');

    if (error) {
      return NextResponse.json({ error: error.message }, { status: 500 });
    }

    // Deduplicate documents by document_id
    // Since each document is split into multiple chunks, we need to group them
    const map = new Map();
    documents?.forEach((doc: any) => {
      const m = doc.metadata;
      if (m?.document_id && !map.has(m.document_id)) {
        map.set(m.document_id, {
          id: m.document_id,
          file_name: m.file_name || 'Unknown',
          file_type: m.file_type || 'unknown',
          file_size: m.file_size || 0,
          upload_date: m.upload_date || new Date().toISOString(),
          total_chunks: m.total_chunks || 0,
          file_url: m.file_url,
          file_path: m.file_path,
        });
      }
    });

    return NextResponse.json({ documents: Array.from(map.values()) });
  } catch (error: any) {
    return NextResponse.json({ error: error.message }, { status: 500 });
  }
}

export async function DELETE(req: Request) {
  try {
    const id = new URL(req.url).searchParams.get('id');
    if (!id) {
      return NextResponse.json({ error: 'Document ID required' }, { status: 400 });
    }

    // Get file path from metadata
    const { data: docs } = await supabase
      .from('documents')
      .select('metadata')
      .eq('metadata->>document_id', id)
      .limit(1);

    const filePath = docs?.[0]?.metadata?.file_path;

    // Delete file from storage
    if (filePath) {
      await supabaseStorage.storage.from('documents').remove([filePath]);
    }

    // Delete all chunks from database
    const { error } = await supabase
      .from('documents')
      .delete()
      .eq('metadata->>document_id', id);

    if (error) {
      return NextResponse.json({ error: error.message }, { status: 500 });
    }

    return NextResponse.json({ success: true, fileDeleted: !!filePath });
  } catch (error: any) {
    return NextResponse.json({ error: error.message }, { status: 500 });
  }
}

This route handles:

• GET without ID: Lists all documents (deduplicated since each document has multiple chunks)

• GET with ID: Returns document details and full text (all chunks combined)

• GET with ID and file=true: Downloads the original file from storage

• DELETE with ID: Deletes the document and its file from both storage and database

Now that your API routes are complete, let's build the user interface components, starting with the upload modal.

Step 8: Create the Upload Modal Component

The upload modal provides a user-friendly interface for selecting and uploading documents. It handles file selection, upload progress, and displays success or error messages.

Create src/app/components/UploadModal.tsx:

'use client';
import { useState, useEffect } from 'react';

interface UploadModalProps {
  isOpen: boolean;
  onClose: () => void;
  onUploadSuccess?: () => void;
}

export default function UploadModal({ isOpen, onClose, onUploadSuccess }: UploadModalProps) {
  const [file, setFile] = useState<File | null>(null);
  const [uploading, setUploading] = useState(false);
  const [message, setMessage] = useState<{ type: 'success' | 'error'; text: string } | null>(null);

  useEffect(() => {
    document.body.style.overflow = isOpen ? 'hidden' : 'unset';
    if (!isOpen) { 
      setFile(null); 
      setMessage(null); 
    }
    return () => { 
      document.body.style.overflow = 'unset'; 
    };
  }, [isOpen]);

  const handleFileChange = (e: React.ChangeEvent<HTMLInputElement>) => {
    if (e.target.files && e.target.files[0]) {
      setFile(e.target.files[0]);
      setMessage(null);
    }
  };

  const handleUpload = async () => {
    if (!file) {
      setMessage({ type: 'error', text: 'Please select a file' });
      return;
    }

    setUploading(true);
    setMessage(null);

    try {
      const formData = new FormData();
      formData.append('file', file);

      const res = await fetch('/api/upload', {
        method: 'POST',
        body: formData,
      });

      const data = await res.json();

      if (data.success) {
        setMessage({
          type: 'success',
          text: `File "${data.fileName}" uploaded successfully! Processed ${data.chunks} chunks.`,
        });
        setFile(null);
        (document.getElementById('upload-file-input') as HTMLInputElement)?.setAttribute('value', '');
        setTimeout(() => { 
          onUploadSuccess?.(); 
          onClose(); 
        }, 1500);
      } else {
        setMessage({ type: 'error', text: data.error || 'Upload failed' });
      }
    } catch (error: any) {
      setMessage({ type: 'error', text: error.message || 'Upload failed' });
    } finally {
      setUploading(false);
    }
  };

  if (!isOpen) return null;

  return (
    <div
      className="fixed inset-0 z-50 flex items-center justify-center bg-black bg-opacity-75 p-4"
      onClick={onClose}
    >
      <div
        className="relative bg-white dark:bg-gray-900 rounded-lg shadow-xl w-full max-w-2xl max-h-[90vh] overflow-y-auto"
        onClick={(e) => e.stopPropagation()}
      >
        <div className="flex items-center justify-between p-6 border-b border-gray-200 dark:border-gray-800">
          <h2 className="text-2xl font-semibold text-gray-900 dark:text-gray-100">
            Upload Document
          </h2>
          <button
            onClick={onClose}
            className="p-2 text-gray-500 hover:text-gray-700 dark:text-gray-400 dark:hover:text-gray-200 rounded-lg hover:bg-gray-100 dark:hover:bg-gray-800"
            aria-label="Close"
          >
            <svg className="w-6 h-6" fill="none" stroke="currentColor" viewBox="0 0 24 24">
              <path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M6 18L18 6M6 6l12 12" />
            </svg>
          </button>
        </div>

        <div className="p-6">
          <div className="mb-6">
            <label htmlFor="upload-file-input" className="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-2">
              Select a file (PDF, DOCX, or TXT)
            </label>
            <input
              id="upload-file-input"
              type="file"
              accept=".pdf,.docx,.txt"
              onChange={handleFileChange}
              className="block w-full text-sm text-gray-500
                file:mr-4 file:py-2 file:px-4
                file:rounded-lg file:border-0
                file:text-sm file:font-semibold
                file:bg-blue-50 file:text-blue-700
                hover:file:bg-blue-100
                dark:file:bg-blue-900 dark:file:text-blue-300
                dark:hover:file:bg-blue-800"
            />
          </div>

          {file && (
            <div className="mb-6 p-4 bg-gray-50 dark:bg-gray-800 rounded-lg text-sm text-gray-600 dark:text-gray-400 space-y-1">
              <p><span className="font-medium">Selected:</span> {file.name}</p>
              <p><span className="font-medium">Size:</span> {(file.size / 1024).toFixed(2)} KB</p>
              <p><span className="font-medium">Type:</span> {file.type || file.name.split('.').pop()}</p>
            </div>
          )}

          <button
            onClick={handleUpload}
            disabled={!file || uploading}
            className="w-full bg-blue-600 text-white px-6 py-3 rounded-lg hover:bg-blue-700 disabled:bg-gray-400 disabled:cursor-not-allowed font-medium"
          >
            {uploading ? 'Uploading and Processing...' : 'Upload Document'}
          </button>

          {message && (
            <div
              className={`mt-6 p-4 rounded-lg ${
                message.type === 'success'
                  ? 'bg-green-50 text-green-800 dark:bg-green-900 dark:text-green-200'
                  : 'bg-red-50 text-red-800 dark:bg-red-900 dark:text-red-200'
              }`}
            >
              {message.text}
            </div>
          )}

          <div className="mt-8 p-4 bg-blue-50 dark:bg-blue-900/20 rounded-lg text-sm">
            <p className="font-medium text-blue-900 dark:text-blue-200 mb-2">Supported: PDF, DOCX, TXT</p>
            <p className="text-blue-700 dark:text-blue-400">Files will be processed and embedded for RAG search.</p>
          </div>
        </div>
      </div>
    </div>
  );
}

This component provides a clean interface for file uploads with proper error handling and user feedback. Next, let's create the PDF viewer component for previewing documents.

Step 9: Create the PDF Viewer Modal Component

The PDF viewer modal allows users to preview PDFs and view extracted text from any document. It's particularly useful for verifying that documents were processed correctly.

Create src/app/components/PDFViewerModal.tsx:

'use client';
import { useEffect, useState } from 'react';

interface PDFViewerModalProps {
  isOpen: boolean;
  onClose: () => void;
  fileUrl: string;
  fileName: string;
  documentId?: string;
  isPDF?: boolean;
}

export default function PDFViewerModal({ 
  isOpen, 
  onClose, 
  fileUrl, 
  fileName, 
  documentId, 
  isPDF = true 
}: PDFViewerModalProps) {
  const [error, setError] = useState<string | null>(null);
  const [loading, setLoading] = useState(true);
  const [activeTab, setActiveTab] = useState<'preview' | 'content'>('preview');
  const [text, setText] = useState<string>('');
  const [textLoading, setTextLoading] = useState(false);
  const [textError, setTextError] = useState<string | null>(null);

  useEffect(() => {
    document.body.style.overflow = isOpen ? 'hidden' : 'unset';
    if (isOpen) { 
      setError(null); 
      setLoading(true); 
      setActiveTab(isPDF ? 'preview' : 'content'); 
      setText(''); 
      setTextError(null); 
    }
    return () => { 
      document.body.style.overflow = 'unset'; 
    };
  }, [isOpen, isPDF]);

  useEffect(() => {
    if (isOpen && documentId && activeTab === 'content' && !text && !textLoading && !textError) {
      fetchDocumentText();
    }
  }, [isOpen, documentId, activeTab, text, textLoading, textError]);

  useEffect(() => {
    if (isOpen && fileUrl && isPDF) {
      fetch(fileUrl, { method: 'GET', headers: { 'Accept': 'application/json' } })
        .then(async res => {
          if (res.headers.get('content-type')?.includes('application/json')) {
            const data = await res.json();
            throw new Error(data.error || 'File not available');
          }
          if (!res.ok) throw new Error(`Failed to load: ${res.status}`);
          setLoading(false);
        })
        .catch(err => {
          setError(err.message || 'Failed to load PDF');
          setLoading(false);
        });
    } else if (isOpen && !isPDF) {
      setLoading(false);
    }
  }, [isOpen, fileUrl, isPDF]);

  const fetchDocumentText = async () => {
    if (!documentId) return;
    setTextLoading(true); 
    setTextError(null);
    try {
      const res = await fetch(`/api/documents?id=${documentId}`);
      const data = await res.json();
      if (data.error) {
        setTextError(data.error);
      } else {
        setText(data.fullText || 'No text content available');
      }
    } catch (err) {
      setTextError(err instanceof Error ? err.message : 'Failed to fetch document text');
    } finally {
      setTextLoading(false);
    }
  };

  if (!isOpen) return null;

  return (
    <div
      className="fixed inset-0 z-50 flex items-center justify-center bg-black bg-opacity-75 p-4"
      onClick={onClose}
    >
      <div
        className="relative bg-white dark:bg-gray-900 rounded-lg shadow-xl w-full max-w-6xl h-[90vh] flex flex-col"
        onClick={(e) => e.stopPropagation()}
      >
        <div className="flex flex-col border-b border-gray-200 dark:border-gray-800">
          <div className="flex items-center justify-between p-4">
            <h2 className="text-xl font-semibold text-gray-900 dark:text-gray-100 truncate flex-1 mr-4">
              {fileName}
            </h2>
            <div className="flex items-center gap-2">
              <button
                onClick={onClose}
                className="p-2 text-gray-500 hover:text-gray-700 dark:text-gray-400 dark:hover:text-gray-200 rounded-lg hover:bg-gray-100 dark:hover:bg-gray-800"
                aria-label="Close"
              >
                <svg className="w-6 h-6" fill="none" stroke="currentColor" viewBox="0 0 24 24">
                  <path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M6 18L18 6M6 6l12 12" />
                </svg>
              </button>
            </div>
          </div>

          {isPDF && (
            <div className="flex border-t border-gray-200 dark:border-gray-800">
              {(['preview', 'content'] as const).map(tab => (
                <button 
                  key={tab} 
                  onClick={() => setActiveTab(tab)} 
                  className={`flex-1 px-4 py-3 text-sm font-medium transition-colors ${
                    activeTab === tab 
                      ? 'text-blue-600 dark:text-blue-400 border-b-2 border-blue-600 dark:border-blue-400 bg-blue-50 dark:bg-blue-900/20' 
                      : 'text-gray-500 dark:text-gray-400 hover:text-gray-700 dark:hover:text-gray-300 hover:bg-gray-50 dark:hover:bg-gray-800'
                  }`}
                >
                  {tab.charAt(0).toUpperCase() + tab.slice(1)}
                </button>
              ))}
            </div>
          )}
        </div>

        <div className="flex-1 overflow-hidden">
          {isPDF && activeTab === 'preview' && (
            <div className="h-full overflow-hidden">
              {error ? (
                <div className="flex flex-col items-center justify-center h-full p-8">
                  <div className="bg-yellow-50 dark:bg-yellow-900/20 border border-yellow-200 dark:border-yellow-800 rounded-lg p-6 max-w-md">
                    <h3 className="text-lg font-semibold text-yellow-800 dark:text-yellow-200 mb-2">
                      PDF File Not Available
                    </h3>
                    <p className="text-yellow-700 dark:text-yellow-300 mb-4">{error}</p>
                    {documentId && (
                      <button 
                        onClick={() => setActiveTab('content')} 
                        className="px-4 py-2 bg-blue-600 text-white rounded-lg hover:bg-blue-700 font-medium"
                      >
                        View Extracted Text Instead
                      </button>
                    )}
                  </div>
                </div>
              ) : loading ? (
                <div className="flex items-center justify-center h-full">
                  <p className="text-gray-500 dark:text-gray-400">Loading PDF...</p>
                </div>
              ) : (
                <iframe
                  src={`${fileUrl}${fileUrl.includes('?') ? '&' : '?'}view=true#toolbar=1&navpanes=0&scrollbar=1`}
                  className="w-full h-full border-0"
                  title={fileName}
                  allow="fullscreen"
                  onError={() => setError('Failed to load PDF')}
                />
              )}
            </div>
          )}

          {(!isPDF || activeTab === 'content') && (
            <div className="h-full overflow-auto p-6">
              {textLoading ? (
                <div className="flex items-center justify-center h-full">
                  <p className="text-gray-500 dark:text-gray-400">Loading...</p>
                </div>
              ) : textError ? (
                <div className="bg-red-50 dark:bg-red-900/20 border border-red-200 dark:border-red-800 rounded-lg p-4">
                  <p className="text-red-800 dark:text-red-200">Error: {textError}</p>
                </div>
              ) : (
                <div className="space-y-4">
                  <p className="text-sm text-gray-500 dark:text-gray-400">
                    Formatting may be inconsistent from source.
                  </p>
                  <pre className="whitespace-pre-wrap text-sm text-gray-800 dark:text-gray-200 font-mono bg-gray-50 dark:bg-gray-800 p-4 rounded-lg">
                    {text || 'No text content available'}
                  </pre>
                </div>
              )}
            </div>
          )}
        </div>
      </div>
    </div>
  );
}

This component provides a full-screen modal for viewing PDFs and extracted text, with tabs to switch between preview and text content. Now let's create a simple navigation component to tie everything together.

Step 10: Create the Navigation Component

The navigation component provides easy access to the Search and Documents pages. It highlights the current page and provides a clean, consistent navigation experience.

Create src/app/components/Navigation.tsx:

'use client';
import Link from 'next/link';
import { usePathname } from 'next/navigation';

export default function Navigation() {
  const pathname = usePathname();

  const navItems = [
    { href: '/', label: 'Search' },
    { href: '/documents', label: 'Documents' },
  ];

  return (
    <nav className="border-b border-gray-200 dark:border-gray-800 mb-8">
      <div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8">
        <div className="flex space-x-8">
          {navItems.map((item) => (
            <Link
              key={item.href}
              href={item.href}
              className={`py-4 px-1 border-b-2 font-medium text-sm ${
                pathname === item.href
                  ? 'border-blue-500 text-blue-600 dark:text-blue-400'
                  : 'border-transparent text-gray-500 hover:text-gray-700 hover:border-gray-300 dark:text-gray-400 dark:hover:text-gray-300'
              }`}
            >
              {item.label}
            </Link>
          ))}
        </div>
      </div>
    </nav>
  );
}

With navigation in place, let's create the main search page where users can query their documents.

Step 11: Create the Home Page (Search Interface)

The search page is the main interface where users ask questions about their uploaded documents. It displays the AI-generated answers along with source citations, allowing users to verify the information.

Update src/app/page.tsx:

'use client';
import { useState } from 'react';
import Navigation from './components/Navigation';

export default function Home() {
  const [query, setQuery] = useState('');
  const [answer, setAnswer] = useState('');
  const [loading, setLoading] = useState(false);
  const [sources, setSources] = useState<any[]>([]);

  const handleSearch = async () => {
    if (!query.trim()) return;
    setLoading(true); 
    setAnswer(''); 
    setSources([]);
    try {
      const res = await fetch('/api/search', { 
        method: 'POST', 
        headers: { 'Content-Type': 'application/json' }, 
        body: JSON.stringify({ query }) 
      });
      const data = await res.json();
      if (data.error) {
        setAnswer(`Error: ${data.error}`);
      } else { 
        setAnswer(data.answer || 'No answer generated'); 
        setSources(data.sources || []); 
      }
    } catch (error: any) {
      setAnswer(`Error: ${error.message}`);
    } finally {
      setLoading(false);
    }
  };

  const handleKeyPress = (e: React.KeyboardEvent) => {
    if (e.key === 'Enter' && (e.metaKey || e.ctrlKey)) {
      handleSearch();
    }
  };

  return (
    <div className="min-h-screen">
      <Navigation />
      <main className="max-w-4xl mx-auto p-8">
        <h1 className="text-3xl font-bold mb-6">RAG Search</h1>

        <div className="bg-white dark:bg-gray-900 border border-gray-200 dark:border-gray-800 rounded-lg p-6 shadow-sm mb-6">
          <textarea 
            className="w-full p-4 border border-gray-300 dark:border-gray-700 rounded-lg shadow-sm bg-white dark:bg-gray-800 text-gray-900 dark:text-gray-100 resize-none focus:ring-2 focus:ring-blue-500 focus:border-transparent"
            placeholder="Ask a question about your uploaded documents..."
            value={query}
            onChange={(e) => setQuery(e.target.value)}
            onKeyDown={handleKeyPress}
            rows={4}
          />
          <button 
            onClick={handleSearch}
            className="mt-4 bg-blue-600 text-white px-8 py-3 rounded-lg hover:bg-blue-700 disabled:bg-gray-400 disabled:cursor-not-allowed font-medium"
            disabled={loading || !query.trim()}
          >
            {loading ? 'Searching...' : 'Search'}
          </button>
          <p className="mt-2 text-sm text-gray-500 dark:text-gray-400">
            Press Cmd/Ctrl + Enter to search
          </p>
        </div>

        {answer && (
          <div className="bg-white dark:bg-gray-900 border border-gray-200 dark:border-gray-800 rounded-lg p-6 shadow-sm mb-6">
            <h2 className="text-xl font-semibold mb-3">Answer:</h2>
            <p className="text-gray-800 dark:text-gray-200 leading-relaxed whitespace-pre-wrap">
              {answer}
            </p>
          </div>
        )}

        {sources && sources.length > 0 && (
          <div className="bg-white dark:bg-gray-900 border border-gray-200 dark:border-gray-800 rounded-lg p-6 shadow-sm">
            <h2 className="text-xl font-semibold mb-3">Sources ({sources.length}):</h2>
            <div className="space-y-3">
              {sources.map((source, index) => (
                <div
                  key={index}
                  className="p-4 bg-gray-50 dark:bg-gray-800 rounded-lg border border-gray-200 dark:border-gray-700"
                >
                  <p className="text-sm text-gray-600 dark:text-gray-400 mb-1">
                    <span className="font-medium">Source:</span>{' '}
                    {source.metadata?.source || source.metadata?.file_name || 'Unknown'}
                  </p>
                  <p className="text-sm text-gray-800 dark:text-gray-200 line-clamp-3">
                    {source.content}
                  </p>
                </div>
              ))}
            </div>
          </div>
        )}
      </main>
    </div>
  );
}

This page provides a clean search interface with a textarea for queries, a search button, and sections to display answers and source citations. The sources section helps users verify where the information came from, which is crucial for trust and accuracy. Now let's create the documents management page.

Step 12: Create the Documents Page

The documents page serves as your document library. It displays all uploaded documents in a table format, shows metadata like file size and chunk count, and provides actions to preview, download, or delete documents. This page is essential for managing your document collection and verifying uploads.

Create src/app/documents/page.tsx:

'use client';
import { useState, useEffect } from 'react';
import Navigation from '../components/Navigation';
import PDFViewerModal from '../components/PDFViewerModal';
import UploadModal from '../components/UploadModal';

interface Document {
  id: string;
  file_name: string;
  file_type: string;
  file_size: number;
  upload_date: string;
  total_chunks: number;
  file_url?: string;
  file_path?: string;
}

export default function DocumentsPage() {
  const [documents, setDocuments] = useState<Document[]>([]);
  const [loading, setLoading] = useState(true);
  const [error, setError] = useState<string | null>(null);
  const [showPDFModal, setShowPDFModal] = useState(false);
  const [selectedPDF, setSelectedPDF] = useState<{ url: string; name: string; id?: string; isPDF?: boolean } | null>(null);
  const [deletingId, setDeletingId] = useState<string | null>(null);
  const [showUploadModal, setShowUploadModal] = useState(false);

  useEffect(() => {
    fetchDocuments();
  }, []);

  const fetchDocuments = async () => {
    try {
      setLoading(true);
      const res = await fetch('/api/documents');
      const data = await res.json();
      if (data.error) {
        setError(data.error);
      } else {
        setDocuments(data.documents || []);
      }
    } catch (err) {
      setError(err instanceof Error ? err.message : 'Failed to fetch documents');
    } finally {
      setLoading(false);
    }
  };

  const formatDate = (s: string) => {
    try {
      const d = new Date(s);
      return isNaN(d.getTime()) 
        ? s 
        : d.toLocaleString('en-US', { 
            year: 'numeric', 
            month: 'short', 
            day: 'numeric', 
            hour: '2-digit', 
            minute: '2-digit', 
            hour12: true 
          });
    } catch { 
      return s; 
    }
  };

  const formatFileSize = (b: number) => 
    b < 1024 
      ? `${b} B` 
      : b < 1024 * 1024 
        ? `${(b / 1024).toFixed(2)} KB` 
        : `${(b / (1024 * 1024)).toFixed(2)} MB`;

  const handleDelete = async (id: string, name: string) => {
    if (!confirm(`Delete "${name}"? This will permanently delete the document, embeddings, and file.`)) {
      return;
    }
    setDeletingId(id);
    try {
      const res = await fetch(`/api/documents?id=${id}`, { method: 'DELETE' });
      const data = await res.json();
      if (data.error) {
        alert(`Error: ${data.error}`);
      } else {
        setDocuments(documents.filter(doc => doc.id !== id));
      }
    } catch (err) {
      alert(err instanceof Error ? err.message : 'Failed to delete');
    } finally {
      setDeletingId(null);
    }
  };

  return (
    <div className="min-h-screen">
      <Navigation />
      <main className="max-w-7xl mx-auto p-8">
        <div className="flex items-center justify-between mb-6">
          <h1 className="text-3xl font-bold">Documents</h1>
          <button
            onClick={() => setShowUploadModal(true)}
            className="px-4 py-2 bg-blue-600 text-white rounded-lg hover:bg-blue-700 font-medium"
          >
            Upload Document
          </button>
        </div>

        {loading ? (
          <div className="text-center py-12">
            <p className="text-gray-500 dark:text-gray-400">Loading documents...</p>
          </div>
        ) : error ? (
          <div className="bg-red-50 dark:bg-red-900/20 border border-red-200 dark:border-red-800 rounded-lg p-4">
            <p className="text-red-800 dark:text-red-200">Error: {error}</p>
          </div>
        ) : documents.length === 0 ? (
          <div className="bg-gray-50 dark:bg-gray-800 border border-gray-200 dark:border-gray-700 rounded-lg p-12 text-center">
            <p className="text-gray-500 dark:text-gray-400 mb-4">No documents uploaded yet.</p>
            <button
              onClick={() => setShowUploadModal(true)}
              className="text-blue-600 dark:text-blue-400 hover:underline font-medium"
            >
              Upload your first document
            </button>
          </div>
        ) : (
          <div className="bg-white dark:bg-gray-900 border border-gray-200 dark:border-gray-800 rounded-lg shadow-sm overflow-hidden">
            <div className="overflow-x-auto">
              <table className="min-w-full divide-y divide-gray-200 dark:divide-gray-800">
                <thead className="bg-gray-50 dark:bg-gray-800">
                  <tr>
                    <th className="px-6 py-3 text-left text-xs font-medium text-gray-500 dark:text-gray-400 uppercase tracking-wider">
                      File Name
                    </th>
                    <th className="px-6 py-3 text-left text-xs font-medium text-gray-500 dark:text-gray-400 uppercase tracking-wider">
                      Type
                    </th>
                    <th className="px-6 py-3 text-left text-xs font-medium text-gray-500 dark:text-gray-400 uppercase tracking-wider">
                      Size
                    </th>
                    <th className="px-6 py-3 text-left text-xs font-medium text-gray-500 dark:text-gray-400 uppercase tracking-wider">
                      Chunks
                    </th>
                    <th className="px-6 py-3 text-left text-xs font-medium text-gray-500 dark:text-gray-400 uppercase tracking-wider">
                      Upload Date
                    </th>
                    <th className="px-6 py-3 text-left text-xs font-medium text-gray-500 dark:text-gray-400 uppercase tracking-wider">
                      Actions
                    </th>
                  </tr>
                </thead>
                <tbody className="bg-white dark:bg-gray-900 divide-y divide-gray-200 dark:divide-gray-800">
                  {documents.map((doc) => (
                    <tr key={doc.id} className="hover:bg-gray-50 dark:hover:bg-gray-800">
                      <td className="px-6 py-4 whitespace-nowrap">
                        <div className="text-sm font-medium text-gray-900 dark:text-gray-100">
                          {doc.file_name}
                        </div>
                      </td>
                      <td className="px-6 py-4 whitespace-nowrap">
                        <span className="px-2 inline-flex text-xs leading-5 font-semibold rounded-full bg-blue-100 text-blue-800 dark:bg-blue-900 dark:text-blue-200">
                          {doc.file_type || 'unknown'}
                        </span>
                      </td>
                      <td className="px-6 py-4 whitespace-nowrap text-sm text-gray-500 dark:text-gray-400">
                        {formatFileSize(doc.file_size)}
                      </td>
                      <td className="px-6 py-4 whitespace-nowrap text-sm text-gray-500 dark:text-gray-400">
                        {doc.total_chunks}
                      </td>
                      <td className="px-6 py-4 whitespace-nowrap text-sm text-gray-500 dark:text-gray-400">
                        {formatDate(doc.upload_date)}
                      </td>
                      <td className="px-6 py-4 whitespace-nowrap text-sm font-medium">
                        <div className="flex gap-3 items-center">
                          {doc.file_name.toLowerCase().endsWith('.pdf') ? (
                            <button 
                              onClick={() => {
                                const pdfUrl = doc.file_url 
                                  ? `${doc.file_url}?view=true` 
                                  : `/api/documents?id=${doc.id}&file=true&view=true`;
                                setSelectedPDF({ url: pdfUrl, name: doc.file_name, id: doc.id });
                                setShowPDFModal(true);
                              }} 
                              className="text-blue-600 hover:text-blue-900 dark:text-blue-400 dark:hover:text-blue-300"
                            >
                              Preview
                            </button>
                          ) : (
                            <>
                              <button 
                                onClick={() => {
                                  setSelectedPDF({ 
                                    url: doc.file_url || `/api/documents?id=${doc.id}&file=true`, 
                                    name: doc.file_name, 
                                    id: doc.id, 
                                    isPDF: false 
                                  });
                                  setShowPDFModal(true);
                                }} 
                                className="text-blue-600 hover:text-blue-900 dark:text-blue-400 dark:hover:text-blue-300"
                              >
                                View
                              </button>
                              {(doc.file_url || doc.file_path) && (
                                <a 
                                  href={doc.file_url || `/api/documents?id=${doc.id}&file=true`} 
                                  download={doc.file_name}
                                  className="text-green-600 hover:text-green-900 dark:text-green-400 dark:hover:text-green-300" 
                                  target="_blank" 
                                  rel="noopener noreferrer"
                                >
                                  Download
                                </a>
                              )}
                            </>
                          )}
                          <button 
                            onClick={() => handleDelete(doc.id, doc.file_name)} 
                            disabled={deletingId === doc.id}
                            className="text-red-600 hover:text-red-900 dark:text-red-400 dark:hover:text-red-300 disabled:opacity-50 disabled:cursor-not-allowed"
                          >
                            {deletingId === doc.id ? 'Deleting...' : 'Delete'}
                          </button>
                        </div>
                      </td>
                    </tr>
                  ))}
                </tbody>
              </table>
            </div>
          </div>
        )}

        {selectedPDF && (
          <PDFViewerModal 
            isOpen={showPDFModal} 
            onClose={() => { 
              setShowPDFModal(false); 
              setSelectedPDF(null); 
            }}
            fileUrl={selectedPDF.url} 
            fileName={selectedPDF.name} 
            documentId={selectedPDF.id} 
            isPDF={selectedPDF.isPDF !== false} 
          />
        )}
        <UploadModal 
          isOpen={showUploadModal} 
          onClose={() => setShowUploadModal(false)} 
          onUploadSuccess={fetchDocuments} 
        />
      </main>
    </div>
  );
}

This page provides a comprehensive document management interface with a table showing all documents, their metadata, and action buttons for preview, download, and deletion. The page automatically refreshes after uploads and handles loading and error states gracefully.

Now that all your components and pages are built, let's test the complete application.

Step 13: Test Your Application

Start your development server:

npm run dev

Open http://localhost:3000 in your browser.

Test the Upload Flow

1. Navigate to the Documents page

2. Click "Upload Document"

3. Select a PDF, DOCX, or TXT file

4. Wait for the upload and processing to complete (this may take a moment as embeddings are generated)

5. You should see your document in the list with its metadata:

Test the Search Flow

1. Navigate to the Search page (or click "Search" in the navigation)

2. Make sure you've uploaded at least one document first

3. Type a question about your uploaded document (for example, "What is this document about?" or ask about specific content)

4. Click "Search" or press Cmd/Ctrl + Enter

5. You should see an AI-generated answer with source citations showing which document chunks were used

Once the embedding is done, you can navigate to search and look for the sample test command based on the documents you have uploaded. You can also check the source from which the search results were pulled.

Test Document Management

1. On the Documents page, click "Preview" or "View" on a document

2. Try downloading a document

3. Test deleting a document (be careful - this is permanent)

If everything works correctly, you're ready to deploy your application!

Step 14: Deploy Your Application

Deploy to Vercel

Vercel is the easiest way to deploy Next.js applications and is made by the creators of Next.js:

To get started, you’ll need to push your code to GitHub. So go ahead and create a repository and push your code.

Then go to vercel.com and sign in with your GitHub account. Click "New Project" and import your GitHub repository.

Add your environment variables in the project settings:

• NEXT_PUBLIC_SUPABASE_URL

• NEXT_PUBLIC_SUPABASE_PUBLISHABLE_DEFAULT_KEY

• SUPABASE_SERVICE_ROLE_KEY

• OPENAI_API_KEY

Then click "Deploy", and your application will be live in minutes! Vercel automatically builds and deploys your Next.js application, and you'll get a URL like your-app.vercel.app.

Important Deployment Notes

• Make sure all environment variables are set in your Vercel project settings

• The service role key is required for file uploads to work

• Supabase Storage bucket should be accessible (public or with proper RLS policies)

• Your OpenAI API key should have sufficient credits

How RAG Search Works

Your application uses the RAG (Retrieval-Augmented Generation) pattern. This combines information retrieval with AI text generation. Here's how it works step by step:

1. Document processing: When you upload a document, it's split into chunks. These are typically 800 characters each with 100-character overlap. Each chunk gets an embedding. This is a 1536-dimensional vector that represents its semantic meaning.

2. Storage: Embeddings are stored in a vector database. This is PostgreSQL with the pgvector extension. They're stored alongside the original text chunks. The original files are stored in Supabase Storage.

3. Query processing: When you search, your query is converted into an embedding. It uses the same model that processed the documents. This ensures the query and documents are in the same "vector space."

4. Similarity search: The system finds the most similar document chunks. It uses cosine similarity on the embeddings. Cosine similarity measures the angle between vectors. Smaller angles mean more similar content, even if the exact words differ.

5. Answer generation: The retrieved chunks are used as context for an AI model. This model is GPT-4o-mini. It generates an accurate answer. The system prompt instructs the AI to only answer based on the provided context. This ensures accuracy.

This approach gives you several benefits.

First, you get accuracy. Answers are based on your actual documents, not just the AI's training data. Second, you get transparency. You can see which document chunks were used to generate each answer. Third, you get efficiency. Only relevant chunks are used, which reduces token usage and costs. Finally, you get up-to-date information. You can update your knowledge base by uploading new documents without retraining the AI.

Troubleshooting Common Issues

"Storage RLS error" when uploading

This means your SUPABASE_SERVICE_ROLE_KEY is not set or incorrect. Make sure the key is in your .env.local file for local development. Also make sure you're using the service role key, not the anon key. Finally, make sure the key is correctly set in your deployment environment, such as Vercel.

"Failed to extract text from file"

Make sure your file is a valid PDF, DOCX, or TXT file. Check that the file isn't corrupted. For PDFs, ensure they contain extractable text. Scanned PDFs with only images won't work without OCR.

"No answer generated"

Make sure you've uploaded at least one document. Try a different query that's more likely to match your documents. Check that embeddings were successfully created. You can verify this in your Supabase database.

Vector similarity search not working

Ensure the vector extension is enabled in Supabase. You can do this by running CREATE EXTENSION IF NOT EXISTS vector;. Verify the match_documents function exists in your database. You can check this in the SQL Editor. Check that embeddings are being stored correctly. They should be JSON strings in the embedding column.

Slow search or upload times

Large documents take longer to process. This is because more chunks mean more embedding API calls. Consider reducing chunk size or processing documents in batches. Also check your OpenAI API rate limits.

Next Steps

Now that you have a working RAG search application, you can extend it with additional features. Here are some examples of useful features you could add:

• You can add more file types by extending the text extraction to support Markdown, HTML, or other formats.

• You can improve chunking by experimenting with different chunk sizes, overlap strategies, or semantic chunking.

• You can add authentication to protect your documents with user authentication using Supabase Auth.

• You can enhance the UI by adding features like search history, document tags, or advanced filters.

• You can optimize performance by adding caching, pagination, or streaming responses.

• You can add filters to allow users to search within specific documents or date ranges.

• Finally, you can improve search by adding hybrid search, which combines keyword and semantic search, or reranking.

Conclusion

You've built a complete RAG search application from scratch. This application demonstrates modern web development with Next.js and TypeScript. It shows vector database operations with Supabase and pgvector. It demonstrates AI integration with OpenAI embeddings and chat completions. It includes file handling and storage with Supabase Storage. Finally, it features a production-ready user interface with Tailwind CSS.

The RAG pattern you've implemented is used by many production applications. These include chatbots, knowledge bases, document search systems, and AI assistants. You now have the foundation to build more advanced features on top of this.

The skills you've learned are highly valuable in today's AI-driven development landscape. You've learned to work with embeddings, vector databases, and the RAG pattern. You can apply these concepts to build intelligent search systems, document Q&A applications, or AI-powered knowledge bases.

With OpenAI’s text-to-speech models, you can build a clean service that takes a blog URL or pasted text and produces a natural-sounding audio file.

In this article, you’ll learn how to build this system end-to-end. You will learn how to fetch blog content, send it to OpenAI’s audio API, save the output as an MP3 file, and serve everything through a small FastAPI app.

At the end, you’ll also build a minimal user interface and deploy it to Sevalla so that anyone can upload text and download audio without touching code.

Table of Contents

• Understanding the Core Idea

• How to Set Up Your Project

• How to Fetch and Clean Blog Content

• How to Send Text to OpenAI for Audio

• How to Build a FastAPI Backend

• How to Add a Simple User Interface

• How to Deploy Your Service to Sevalla

• Conclusion

Understanding the Core Idea

A blog-to-audio service has only three important parts. The first part takes a blog link or text and cleans it. The second part sends the clean text to OpenAI’s text-to-speech model. The third part gives the final MP3 file back to the user.

OpenAI’s speech generation is simple to use. You send text, choose a voice, and get audio back. The quality is high and works well even for long posts. This means you do not need to worry about training models or tuning voices.

The only job left is to make the system easy to use. That is where FastAPI and a small HTML form help. They wrap your code into a web service so anyone can try it.

How to Set Up Your Project

Create a folder for your project. Inside it, create a file called main.py. You will also need a basic HTML file later.

Install the libraries you need with pip:

pip install fastapi uvicorn requests beautifulsoup4 python-multipart

FastAPI gives you a simple backend. Requests module helps download blog pages. BeautifulSoup helps remove HTML tags and extract readable text. Python-multipart helps upload form data.

You must also install the OpenAI client:

pip install openai

Make sure you have your OpenAI API key ready. Set it in your terminal before running the app:

export OPENAI_API_KEY="your-key"

On Windows, you can do:

setx OPENAI_API_KEY "your-key"

How to Fetch and Clean Blog Content

To convert a blog into audio, you must first extract the main article text. You can fetch the page with requests and parse it with BeautifulSoup.

Below is a simple function that does this.

import requests
from bs4 import BeautifulSoup

def extract_text_from_url(url: str) -> str:
    response = requests.get(url, timeout=10)
    html = response.text
    soup = BeautifulSoup(html, "html.parser")
    paragraphs = soup.find_all("p")
    text = " ".join(p.get_text(strip=True) for p in paragraphs)
    return text

Here is what happens step by step.

• The function downloads the page.

• BeautifulSoup reads the HTML and finds all paragraph tags.

• It pulls out the text in each paragraph and joins them into one long string.

• This gives you a clean version of the blog post without ads or layout code.

If the user pastes text instead of a URL, you can skip this part and use the text as it is.

How to Send Text to OpenAI for Audio

OpenAI’s text-to-speech API makes this part of the work very easy. You send a message with text and select a voice such as Alloy or Verse. The API returns raw audio bytes. You can save these bytes as an MP3 file.

Here is a helper function to convert text into audio:

from openai import OpenAI
client = OpenAI()

def text_to_audio(text: str, output_path: str):
    audio = client.audio.speech.create(
        model="gpt-4o-mini-tts",
        voice="alloy",
        input=text
    )
    with open(output_path, "wb") as f:
        f.write(audio.read())

This function calls the OpenAI client and passes the text, model name, and voice choice. The .read() method extracts the binary audio stream. Writing this to an MP3 file completes the process.

If the blog post is very long, you may want to limit text length or chunk the text and join the audio files later. But for most blogs, the model can handle the entire text in one request.

How to Build a FastAPI Backend

Now you can wrap both steps into a simple FastAPI server. This server will accept either a URL or pasted text. It will convert the content into audio and return the MP3 file as a response.

Here is the full backend code:

from fastapi import FastAPI, Form
from fastapi.responses import FileResponse
import uuid
import os

app = FastAPI()
@app.post("/convert")
def convert(url: str = Form(None), text: str = Form(None)):
    if not url and not text:
        return {"error": "Please provide a URL or text"}
    if url:
        try:
            text_content = extract_text_from_url(url)
        except Exception:
            return {"error": "Could not fetch the URL"}
    else:
        text_content = text
    file_id = uuid.uuid4().hex
    output_path = f"audio_{file_id}.mp3"
    text_to_audio(text_content, output_path)
    return FileResponse(output_path, media_type="audio/mpeg")

Here is how it works. The user sends form data with either url or text. The server checks which one exists.

If there is a URL, it extracts text with the earlier function. If there is no URL, it uses the provided text directly. A unique file name is created for every request. Then the audio file is generated and returned as an MP3 download.

You can run the server like this:

uvicorn main:app --reload

Open your browser at http://localhost:8000. You will not see the UI yet, but the API endpoint is working. You can test it using a tool like Postman or by building the front end next.

How to Add a Simple User Interface

A service is much easier to use when it has a clean UI. Below is a simple HTML page that sends either a URL or text to your FastAPI backend. Save this file as index.html in the same folder:

<!DOCTYPE html>
<html>
<head>
    <title>Blog to Audio</title>
    <style>
        body { font-family: Arial, padding: 40px; max-width: 600px; margin: auto; }
        input, textarea { width: 100%; padding: 10px; margin-top: 10px; }
        button { padding: 12px 20px; margin-top: 20px; cursor: pointer; }
    </style>
</head>
<body>
    <h2>Convert Blog to Audio</h2>
    <form action="/convert" method="post">
        <label>Blog URL</label>
        <input type="text" name="url" placeholder="Enter a blog link">
<p>or paste text below</p>
        <textarea name="text" rows="10" placeholder="Paste blog text here"></textarea>
        <button type="submit">Convert to Audio</button>
    </form>
</body>
</html>

This page gives the user two options. They can type a URL or paste text. The form sends the data to /convert using a POST request. The response will be the MP3 file, so the browser will download it.

To serve the HTML file, add this route to your main.py:

from fastapi.responses import HTMLResponse

@app.get("/")
def home():
    with open("index.html", "r") as f:
        html = f.read()
    return HTMLResponse(html)

Now, when you visit the main URL, you will see a clean form.

When you submit a URL, the server will process your request and give you an audio file.

Great. Our text to audio service is working. Now let’s get it into production.

How to Deploy Your Service to Sevalla

You can choose any cloud provider, like AWS, DigitalOcean, or others, to host your service. I will be using Sevalla for this example.

Sevalla is a developer-friendly PaaS provider. It offers application hosting, database, object storage, and static site hosting for your projects.

Every platform will charge you for creating a cloud resource. Sevalla comes with a $50 credit for us to use, so we won’t incur any costs for this example.

Let’s push this project to GitHub so that we can connect our repository to Sevalla. We can also enable auto-deployments so that any new change to the repository is automatically deployed.

You can also fork my repository from here.

Log in to Sevalla and click on Applications -> Create new application. You can see the option to link your GitHub repository to create a new application.

Use the default settings. Click “Create application”. Now we have to add our OpenAI API key to the environment variables. Click on the “Environment variables” section once the application is created, and save the OPENAI_API_KEY value as an environment variable.

Now we are ready to deploy our application. Click on “Deployments” and click “Deploy now”. It will take 2–3 minutes for the deployment to complete.

Once done, click on “Visit app”. You will see the application served via a URL ending with sevalla.app . This is your new root URL. You can replace localhost:8000 with this URL and start using it.

Congrats! Your blog-to-audio service is now live. You can extend this by adding other capabilities and pushing your code to GitHub. Sevalla will automatically deploy your application to production.

Conclusion

You now know how to build a full blog-to-audio service using OpenAI. You learned how to fetch blog text, convert it into speech, and serve it with FastAPI. You also learned how to create a simple user interface, allowing people to try it with no setup.

With this foundation, you can turn any written content into smooth, natural audio. This can help creators reach a wider audience, enhance accessibility, and provide users with more ways to enjoy content.

Hope you enjoyed this article. Signup for my free newsletter TuringTalks.ai for more hands-on tutorials on AI. You can also visit my website.

Think of these apps as plugins for ChatGPT:

• You can invoke them naturally in a conversation.

• They can render custom interactive UIs inside ChatGPT (maps, carousels, videos, and more).

• They run on an MCP server that you control, which defines the tools, resources, and widgets the app provides.

In this step-by-step guide, you’ll build a ChatGPT App using the official Pizza App example. This app shows how ChatGPT can render UI widgets like a pizza map or carousel, powered by your local server.

What You’ll Learn

By following this tutorial, you’ll learn how to:

• Set up and run a ChatGPT App with the OpenAI Apps SDK.

• Understand the core building blocks: tools, resources, and widgets.

• Connect your local app server to ChatGPT using Developer Mode.

• Render custom UI directly inside a ChatGPT conversation.

Table of Contents

• What You’ll Learn

• Table of Contents

• How ChatGPT Apps Work (Big Picture)

• Step 1. Clone the Examples Repo

• Step 2. Run the Pizza App Server

• Step 3. Expose Your Local Server

  • 3.1 Get ngrok

  • 3.2 Install ngrok

  • 3.3 Connect Your Account

  • 3.4 Start a Tunnel

• Step 4. Walk Through the Pizza App Code

  • 4.1 Imports and Setup

  • 4.2 Defining Pizza Widgets

  • 4.3 Mapping Widgets to Tools and Resources

  • 4.4 Handling Requests

  • 4.5 Creating the Server

• Step 5. Enable Developer Mode in ChatGPT

  • 5.1 Enable Developer Mode

  • 5.2 Create App

  • 5.3 Use Your App

• Challenges (Try These Yourself)

  • Challenge A: Add a “Pizza Specials” widget (text-only)

  • Challenge B: Support Multiple Toppings

  • Challenge C: Fetch Real Pizza Data from an External API

• Conclusion

How ChatGPT Apps Work (Big Picture)

Here’s the architecture in simple terms:

ChatGPT (frontend)
   |
   v
MCP Server (your backend)
   |
   v
Widgets (HTML/JS markup displayed inside ChatGPT)

• ChatGPT sends requests like: “Show me a pizza carousel.”

• MCP Server responds with resources (HTML markup) and tool logic.

• Widgets are rendered inline in ChatGPT.

Step 1. Clone the Examples Repo

OpenAI provides an official examples repo that includes the Pizza App. Clone it and install the dependencies using these commands:

git clone https://github.com/openai/openai-apps-sdk-examples.git
cd openai-apps-sdk-examples
pnpm install

After installing, build the components and start the dev server:

pnpm run build  
pnpm run dev

Step 2. Run the Pizza App Server

Navigate to the Pizza App server and start it:

cd pizzaz_server_node
pnpm start

If it works, you should see:

Pizzaz MCP server listening on http://localhost:8000
  SSE stream: GET http://localhost:8000/mcp
  Message post endpoint: POST http://localhost:8000/mcp/messages

This means your server is running locally.

Step 3. Expose Your Local Server

To let ChatGPT communicate with your app, your local server needs a public URL. ngrok provides a quick way to expose it during development.

3.1 Get ngrok

Sign up at ngrok.com and copy your authtoken.

3.2 Install ngrok

macOS:

brew install ngrok

Windows:

• Download and unzip ngrok.

• Optionally, add the folder to your PATH.

3.3 Connect Your Account

ngrok config add-authtoken <your_authtoken>

3.4 Start a Tunnel

ngrok http 8000

This gives you a public HTTPS URL (like https://xyz.ngrok.app/mcp).

Step 4. Walk Through the Pizza App Code

The full Pizza App server code is long, so let’s break it down into digestible parts.

4.1 Imports and Setup

import { createServer } from "node:http";
import { Server } from "@modelcontextprotocol/sdk/server/index.js";
import { SSEServerTransport } from "@modelcontextprotocol/sdk/server/sse.js";
import { z } from "zod";

• Server and SSEServerTransport come from the Apps SDK.

• zod validates input to ensure ChatGPT sends the right arguments.

4.2 Defining Pizza Widgets

Widgets are the heart of the app. Each one represents a piece of UI ChatGPT can display.

Here’s the Pizza Map widget:

{
  id: "pizza-map",
  title: "Show Pizza Map",
  templateUri: "ui://widget/pizza-map.html",
  html: `
    <div id="pizzaz-root"></div>
    <link rel="stylesheet" href=".../pizzaz-0038.css">
    <script type="module" src=".../pizzaz-0038.js"></script>
  `,
  responseText: "Rendered a pizza map!"
}

• id → unique name of the widget.

• templateUri → how ChatGPT fetches the UI.

• html → actual markup and assets.

• responseText → message that shows in chat.

The app defines five widgets:

• Pizza Map

• Pizza Carousel

• Pizza Album

• Pizza List

• Pizza Video

4.3 Mapping Widgets to Tools and Resources

Next, widgets are converted into tools (things ChatGPT can call) and resources (UI markup ChatGPT can render).

const tools = widgets.map((widget) => ({
  name: widget.id,
  description: widget.title,
  inputSchema: toolInputSchema,
  title: widget.title,
  _meta: widgetMeta(widget)
}));

const resources = widgets.map((widget) => ({
  uri: widget.templateUri,
  name: widget.title,
  description: `${widget.title} widget markup`,
  mimeType: "text/html+skybridge",
  _meta: widgetMeta(widget)
}));

This makes each widget callable and displayable.

4.4 Handling Requests

The MCP server responds to ChatGPT’s requests. For example, when ChatGPT calls a widget tool:

server.setRequestHandler(CallToolRequestSchema, async (request) => {
  const widget = widgetsById.get(request.params.name);
  const args = toolInputParser.parse(request.params.arguments ?? {});
  return {
    content: [{ type: "text", text: widget.responseText }],
    structuredContent: { pizzaTopping: args.pizzaTopping },
    _meta: widgetMeta(widget)
  };
});

This:

• Finds the widget requested.

• Validates the input (pizzaTopping).

• Responds with text + metadata so ChatGPT can render the widget.

4.5 Creating the Server

Finally, the server is bound to HTTP endpoints (/mcp and /mcp/messages) so ChatGPT can stream messages to and from it.

const httpServer = createServer(async (req, res) => {
  // handle requests to /mcp and /mcp/messages
});

httpServer.listen(8000, () => {
  console.log("Pizzaz MCP server running on port 8000");
});

Step 5. Enable Developer Mode in ChatGPT

5.1 Enable Developer Mode

• Open ChatGPT

• Go to Settings → Apps & Connectors → Advanced Settings

• Toggle Developer Mode

When Developer Mode is enabled, ChatGPT should look like this:

5.2 Create App

• Go back to Settings → Apps & Connectors

• Click Create

• Next:

  • Name: Enter a name for your app (for example, Pizza App)

  • Description: Enter any description for your app (or leave empty)

  • MCP Server URL: Paste the public HTTPS URL of your MCP endpoint. Make sure it points directly to /mcp, not just the server root

  • Authentication: Choose No authentication

  • Check I trust this application

  • Click Create to finish

Once your app is connected to ChatGPT, it should look like this:

When you click on the Back icon, you should see your app and other apps that you can connect to and use with ChatGPT:

5.3 Use Your App

To use your app,

• Open a new chat in ChatGPT

• Click on the + icon

• Scroll down to more

• You would see your app

• Choose Pizza App to start using your app

Here are some commands you can try out with your pizza app in ChatGPT:

• Show me a pizza map with pepperoni topping

• Show me a pizza carousel with mushroom topping

• Show me a pizza album with veggie topping

• Show me a pizza list with cheese topping

• Show me a pizza video with chicken topping

Each command tells ChatGPT which widget to render, and you can swap in any topping you like.

Below are samples:

• Pepperoni topping map:

• Extra cheese carousel:

• Mushroom topping album:

Challenges (Try These Yourself)

Here are three practical ways to extend your Pizza App. Each one ties directly to the code you already have.

Challenge A: Add a “Pizza Specials” widget (text-only)

Goal: Create a widget that just shows a short message like “Today’s special: Margherita with basil.”

Where to change:

• resources.widgets → duplicate an entry and give it a new id/title.

• tools → register it as a new tool.

• CallTool handler → detect when it’s called (if (request.params.name === "pizza-special")) and return your special.

Hint:
This widget doesn’t need extra CSS/JS files. Just keep its html to something like <div>🍕 Today’s special: Margherita</div>. The idea is to show that widgets can be as simple as plain HTML.

Challenge B: Support Multiple Toppings

Goal: Let users order a pizza with more than one topping, like ["pepperoni", "mushroom"].

Where to change:

• toolInputSchema → switch from z.string() to z.array(z.string()).

• CallTool handler → after parsing, args.pizzaTopping will be an array. Join it into a string before inserting into HTML/response.

• Widget HTML → update the display so it lists all chosen toppings.

Hint:
Console.log the parsed args first to confirm you’re actually getting an array. Then try something like:

const toppings = args.pizzaTopping.join(", ");
return { responseText: `Pizza ordered with ${toppings}` };

Challenge C: Fetch Real Pizza Data from an External API

Goal: Instead of hard-coding content, fetch real pizza info. For example, you could call Yelp’s API to list pizza places in a location, or use a free placeholder API to simulate data.

Where to change:

• Inside the CallTool handler for your widget.

• Replace the static HTML with a fetch(...) call that builds dynamic HTML from the response.

Hint:
Start small with a free API like JSONPlaceholder. For example:

const res = await fetch("https://jsonplaceholder.typicode.com/posts?_limit=3");
const data = await res.json();

const html = `
  <ul>
    ${data.map((p: any) => `<li>${p.title}</li>`).join("")}
  </ul>
`;

return { responseText: "Fetched pizza places!", content: [{ type: "text/html", text: html }] };

Once that works, swap in a real API such as Yelp or Google Maps Places to render actual pizza places.

Conclusion

You just built your first ChatGPT App using the OpenAI Apps SDK. With a bit of JavaScript and HTML, you created a server that ChatGPT can talk to, and rendered interactive widgets right inside the chat window.

This example focused on the pizza app sample provided by OpenAI, but you could build:

• A weather dashboard,

• A movie finder,

• A financial data viewer,

• Or even a mini-game.

The SDK makes it possible to blend conversation + interactive UI in powerful new ways.

Explore the OpenAI Apps SDK documentation to go deeper and start building your own apps.

LLMs were like a tireless, super knowledgeable pair programmer that could conjure code out of thin air. We’d type a quick, messy request, and out would pop something that...kind of worked. It was amazing, but also a little frustrating. The code might be buggy, inefficient, or completely miss the subtle context of our project.

But with GPT-5, the game has changed quite a bit. This model doesn’t just spit out code – it reasons, adapts, and understands context like never before. Still, here’s the catch: you need to speak its language to be able to generate the best output. But how? That’s where prompt engineering comes in.

In this article, I’ll share 10 proven patterns that will help you transform GPT-5 from a helpful tool into a rock-solid coding partner you can trust for accuracy and speed. Let’s get started!

Table of Contents

1. What is GPT-5? Why You Should Use It as a Developer?

2. Why Prompt Engineering?

3. How to Use GPT-5 for Free?

4. Patterns Every Developer Should Know

   • Persona Pattern

   • Few-Shot Pattern

   • Chain-of-Thought Pattern

   • Delimiter Pattern

   • Structured Output Pattern

   • Flipped Interaction Pattern

   • Negative Constraint Pattern

   • Tool Use Pattern

   • Verbosity Pattern

   • Code-as-Context Pattern

5. Common Pitfalls to Avoid

6. Final Thoughts

What is GPT-5? Why You Should Use It as a Developer?

OpenAI recently launched one of its best models, GPT-5. It’s capable of performing coding and agentic tasks across various domains. Think of it as a full-stack, super-intelligent intern who’s been given a master key to the internet's knowledge. It's not just better at writing code, it can under why you need the code, how it should fit into a larger system, and how to debug it.

It excels at:

• Long-context reasoning: It can handle an entire codebase or a lengthy API documentation, a game-changer for refactoring or fixing bugs across multiple files.

• Instruction following: It’s far less likely to get confused by a long list of constraints or a detailed set of steps.

• Tool use and agentic tasks: It can intelligently decide to call an external API, execute a shell command, or search a repository to complete a task.

Why Prompt Engineering?

Think of LLMs as junior developers: super smart, but literal. The way you phrase your request drastically changes the output. Prompt engineering is the art and science of crafting effective instructions for an LLM to achieve a specific goal. It’s the method you use to communicate your intent, provide necessary context, and structure your request in a way that the model can most accurately understand and respond to. When you master it, you can:

• Make GPT-5 generate working, testable code.

• Avoid vague or irrelevant answers.

• Save tokens (and money).

• Reduce the time spent editing or debugging outputs.

How to Use GPT-5 for Free

While the API for GPT-5 is a paid service, many developers can access its power for free or at a low cost. Now, for example, the default public version of ChatGPT often uses the version of GPT-5 with certain usage caps. Many tools like Cursor, GitHub Copilot, Microsoft Copilot integrate GPT-5 or lighter variants.

See the screenshot below of the Cursor IDE with integration of various models, including gpt-5-fast, gpt-5-low, and so on. If you’re experimenting, this is the easiest way to explore GPT-5 without paying for direct API calls.

For this article, we'll use a standard API call structure, but these same principles apply whether you're using a web interface or an integrated tool. Let’s dive into the patterns.

Patterns Every Developer Should Know

Persona Pattern

You know how, when you're interviewing a candidate, you might ask them to act as if they're a "Engineering Lead or Manager" or a "Frontend engineer"? This pattern is the same idea. By assigning the model a role, you give it an immediate set of assumptions and a knowledge filter.

To effectively craft a persona, be specific. For example, instead of saying "You are a developer," try "You are a senior JavaScript developer specializing in backend APIs and scalability." This provides context on their skill level, their domain, and their preferred programming language, guiding the LLM toward a more tailored and expert-level response.

Example:

# Python Example
from openai import OpenAI
client = OpenAI()

response = client.responses.create(
    model="gpt-5",
    input="""You are a senior JavaScript developer. 
    Refactor this code for readability:
    numbers = [8, 9, 10, 11, 12]; total=0
    for i in numbers: total+=i
    print(total)"""
)

print(response.output_text)

This code ensures answers match the tone and expertise you expect, as specified in the prompt.

Few-Shot Pattern

Sometimes, the best way to get a specific style or format of code is to provide an example. This is called "few-shot" prompting. Instead of just describing what you want, you show the model a few completed examples.

Example:

from openai import OpenAI

client = OpenAI()

prompt = """
Convert functions to arrow syntax:

Example:
function sum(x, y) { return x + y; }
=> const sum = (x, y) => x + y;

Then convert:
function greet(name) { return "Hey, " + name; }
"""

response = client.responses.create(
    model="gpt-5",
    input=prompt
)

print(response.output_text)

This code example provides a concrete, undeniable pattern for the model to follow, which is much more effective than a verbose description.

Chain-of-Thought Pattern

When faced with a complex problem, humans don't just jump to a solution instead, we think through the steps. The Chain-of-Thought pattern asks the LLM to do the same. By telling the model to “think step by step,” you're not just requesting a final answer but you're instructing it to perform internal reasoning and break down the problem into smaller, logical parts. This process is what gives you room to debug.

If the final output is incorrect, you can review its thought process to identify where the logic went wrong. This is particularly effective with GPT-5's enhanced reasoning capabilities. The LLM's reasoning might look like an intermediate, internal monologue you don't always see, but asking it to print its thought process can make it explicit.

Example:

prompt = """
Debug the below step by step:
My Python function loop skips the last element of the list. Check why?
"""

By encouraging reasoning, you reduce errors in the code.

Delimiter Pattern

When you’re giving the LLM instructions, it’s important to give it a clear way to differentiate your instructions from the data you want it to process. To do this, you can use delimiters like ###, """, or <> wrapped around your input text to create a clean boundary. This is a general best practice for all LLMs, as they all can struggle with this distinction without a clear signal.

Example:

prompt = """
Explain this code in simple and easy English:

###
for i in range(10):
    print(i**3)
###
"""

This helps prevent the model from misinterpreting your data as part of the instructions, particularly when the data contains instruction-like strings.

Structured Output Pattern

If you need the model's response to be easily parseable by a program, you must specify the format clearly. This is particularly important when you want to use the output as an input for a different part of your software, such as generating JSON configuration files, XML for web services, or even markdown (MD) files for documentation. By telling the model to adhere to a rigid structure, you ensure the output is consistent and reliable.

Example:

import json
from openai import OpenAI

client = OpenAI()

def generate_product_list(product_info):
    prompt = f"""
    Generate a JSON object for the following product information.
    The JSON should have a 'products' key, which is an array of objects.
    Each object should have keys for 'name', 'category', 'price', and 'in_stock' (a boolean).

    Product Information:
    {product_info}

    Provide only the JSON output, and nothing else.
    """

    response = client.responses.create(
        model="gpt-5",
        input=prompt
    )

    # Try to parse the response as JSON
    try:
        json_output = json.loads(response.output_text)
        return json_output
    except json.JSONDecodeError as e:
        print(f"Error parsing JSON: {e}")
        return None

# Let's try it out
product_data = """
Laptop Pro, Electronics, 1500, True
Ergo Mouse, Accessories, 50, True
Wireless Keyboard, Accessories, 90, False
"""

product_list = generate_product_list(product_data)
if product_list:
    print(json.dumps(product_list, indent=2))

In this example, the prompt is the instruction you give to the LLM. It's a text string that outlines a clear task and specifies the output format (a JSON object with specific keys). The response from the model is the raw text it generates, which should be the JSON object you requested. The Python code then attempts to parse this raw text response into a structured JSON object using json.loads().

Flipped Interaction Pattern

Sometimes, the best way to get GPT-5 to help you is to have it ask you some questions before it writes any code.

Example:

prompt = """
I want a python script to scrape travel websites for travelling data.
Ask me 5 clarifying questions before writing the code.
"""

This type of prompt helps prevent assumptions and will provide more accurate code.

Negative Constraint Pattern

While it’s important to tell the model what it should do, it’s also sometimes as important to tell it what it should not do or what it shouldn’t include in its response. This helps the model avoid certain words, tones, or topics.

Example:

from openai import OpenAI

client = OpenAI()

def my_func(technical_report):
    prompt = f"""
    Summarize the following technical report for a non-technical audience. 
    Do not use any specialized jargon, acronyms, or complex terms. 
    Use simple, everyday language.

    Technical Report:
    "{technical_report}"
    """
    response = client.responses.create(
        model="gpt-5",
        input=prompt
    )
    return response.output_text

# Let's try it out
report = (
    "The quantum entanglement protocol (QEP) showed significant improvements "
    "in qubit coherence by utilizing a novel multi-photon emission cascade. "
    "The data indicates a 12% reduction in decoherence rates, validating the "
    "hypothesis that non-linear optical feedback could mitigate environmental noise."
)

summary = my_func(report)
print(summary)

This pattern is a great way to fine-tune the output and steer it away from common pitfalls, overly technical language, and so on, ensuring it meets your specific requirements.

Tool Use Pattern

GPT-5 is an incredible reasoning engine, but its real power comes when it can interact with external tools, like a web search, a code interpreter, or a file retrieval system. This pattern involves providing the model with a clear description of the tools it can or should use.

Example:

prompt = """
You have access to a 'code_interpreter' tool.
Its purpose is to execute JavaScript code in a secure sandbox.
The tool takes a single argument: the JavaScript code as a string.

Your task is to use this tool to calculate the area of a rectangle 
with a length and breadth as 15.
After you get the result, respond with only the final answer number.
"""

This is what unlocks GPT-5's potential for true agentic behavior. It can autonomously solve a problem by deciding which tools to use and in what order, moving beyond simple text generation.

Verbosity Pattern

Depending on your needs, you might want more or less concise output from the LLM. With the GPT-5 API, you can adjust the level of detail and length of the output with the use of the new text.verbosity parameter. Just select the level of text.verbosity as low, medium, or high.

Example:

from openai import OpenAI

client = OpenAI()

# Low Verbosity for a concise function
def get_concise_code(description):
    prompt = f"Write a Python function for {description}."
    response = client.responses.create(
        model="gpt-5",
        input=prompt,
        metadata={"verbosity": "low"} 
    )
    return response.output_text

user_input = "a quicksort algorithm"

concise_code = get_concise_code(user_input)

print("Concise Code-\n", concise_code)

This saves you time by preventing the model from "over-explaining" when you just need a quick snippet, and it gives you more context when you're learning something new or working with a complex piece of code.

Code-as-Context Pattern

GPT-5’s massive context window is a game-changer for working with a full file or even a small project. Instead of just giving it a snippet, you can feed it an entire script and ask it to analyze, refactor, or optimize it.

Example:

async def my_optimize_codebase(code_file: str) -> str:
    prompt = f"""
    You are a performance optimization expert. Analyze the following JavaScript 
    code file for potential performance bottlenecks, redundant code, or memory leaks. 
    Provide a detailed report and then a refactored version of the code.

    Code to analyze:
    \"\"\"
    {code_file}
    \"\"\"
    """
    # For this demonstration, we'll just return the prompt
    return prompt


# User input: "your text input here"
my_code = """
// A large, unoptimized JavaScript file
const fetchData = async () => {
  const data = await fetch('https://api.example.com/data');
  const jsonData = await data.json();
  const filteredData = jsonData.filter(item => item.isActive);
  const mappedData = filteredData.map(item => {
    return {
      id: item.id,
      name: item.name.toUpperCase(),
      status: 'active'
    };
  });

  // This is a loop that could be more efficient
  const res= [];
  for (let i = 0; i < mappedData.length; i++) {
    for (let j = 0; j < 10000; j++) {
      res.append(mappedData[i])
    }
  }
  return res;
};
"""

import asyncio

async def main():
    prompt = await my_optimize_codebase(my_code)
    print(prompt)

asyncio.run(main())

This prompt allows GPT-5 to see the full picture. It can understand variable scope, function dependencies, and the overall logic of a file in a way that’s impossible with a single, isolated snippet.

Common Pitfalls to Avoid

• Being Vague or Ambiguous: A prompt such as “Write some code” will result in a response that lacks focus and is generic. Make sure to clarify which programming language, the specific function, output format, and any limitations that may be required.

• Overloading a Single Prompt: An example “Write a Python script, summarize it in three bullet points, and then translate it into French” has multiple unrelated tasks and will commonly generate disorganized or incomplete reports. Focus on complex requests and break them down into a series of prompts.

• Failing to Iterate: Usually, your first prompt is hardly the most accurate or relevant to the topic of discussion. A general approach is to focus on the prompts generated and go over the concerns of the first sentence as a response. Take into consideration to elaborate, incorporate more facts, and refine, hence have a conversation back and forth to achieve the desired result.

Final Thoughts

With GPT-5, prompt engineering is much more complex than locating a “magic” phrase. You need to shift your thinking to software engineering and articulate it for the AI. You are not merely instructing the AI – you are defining the parameters within which it should work to arrive at an efficient solution.

You can put these 10 patterns, along with the new features of reasoning effort and verbosity control, to make GPT-5 a dependable coding assistant: generating boilerplate code, debugging, code refactoring, or app scaffolding. Start improving your prompt engineering technique with lower models like GPT-4o, Gemini, and others. Once you are ready, upgrade to GPT-5 to power real-world dev workflows.

If you found this article helpful and want to discuss AI development, LLMs, or software development, feel free to connect with me on X/Twitter, LinkedIn, or check out my portfolio on my Blog. I regularly share insights about AI, development, technical writing, and so on, and would love to see what you build with this foundation.

That's exactly what LLM agents bring to the table. They're changing how we automate complex tasks, and they can help bring our AI ideas to life in a whole new way.

In this article, we'll explore what LLM agents are, how they work, and how you can build your very own using awesome open-source frameworks.

What we’ll cover:

1. The Current State of LLM Agents

   • From Chatbots to Autonomous Agents

   • What Can Agents Do Today?

   • What's Available to Build With?

   • Why Now Is the Best Time to Learn

2. What Are LLM Agents and Why Are They a Big Deal?

   • What Is an LLM?

   • So, What’s an LLM Agent?

   • Why Does This Matter?

3. The Rise of Open-Source Agent Frameworks

   • Popular Open-Source Agent Frameworks

   • What These Tools Enable

   • Why Use a Framework Instead of Building from Scratch?

4. Core Concepts Behind Agent Design

   • The Agent Loop

   • Key Components of an Agent

   • Multi-Agent Collaboration

5. Project: Automate Your Daily Schedule from Emails

   • What We’re Automating

   • Step 1: Install the Required Tools

   • Step 2: Define the Task

   • Step 3: Build the Workflow with LangGraph

6. Multi-Agent Collaboration with CrewAI

   • What Is CrewAI?

   • Sample Roles for the Email Summary Task

   • Sample CrewAI Code

7. What Actually Happens During Execution?

8. Are LLM Agents Safe? What to Know About Security and Privacy

9. Troubleshooting & Tips

10. Explore More Daily Automations

11. What’s Next in Agent Technology?

12. Final Summary

The Current State of LLM Agents

LLM agents are one of the most exciting developments in AI right now. They’re already helping automate real tasks but they’re also still evolving. So where are we today?

From Chatbots to Autonomous Agents

Large Language Models (LLMs) like GPT-4, Claude, Gemini, and LLaMA have evolved from simple chatbots into surprisingly capable reasoning engines. They've gone from answering trivia questions and generating essays to performing complex reasoning, following multi-step instructions, and interacting with tools like web search and code interpreters.

But here’s the catch: these models are reactive. They wait for input and give output. They don't retain memory between tasks, plan ahead, or pursue goals on their own. That’s where LLM agents come in – they bridge this gap by adding structure, memory, and autonomy.

What Can Agents Do Today?

Right now, LLM agents are already being used for:

• Summarizing emails or documents

• Planning daily schedules

• Running DevOps scripts

• Searching APIs or tools for answers

• Collaborating in small “teams” to complete complex tasks

But they’re not perfect yet. Agents can still:

• Get stuck in loops

• Misunderstand goals

• Require detailed prompts and guardrails

That’s because this technology is still early-stage. Frameworks are getting better fast, but reliability and memory are still works in progress. So just keep that in mind as you experiment.

Why Now Is the Best Time to Learn

The truth is: we’re still early. But not too early.

This is the perfect time to start experimenting with agents:

• The tooling is mature enough to build real projects

• The community is growing rapidly

• And you don’t need to be an AI expert just comfortable with Python

What Are LLM Agents and Why Are They a Big Deal?

Before we dive into the exciting world of agents, let's quickly chat a bit more about the basics.

What Is an LLM?

An LLM, or Large Language Model, is basically an AI that's learned from a massive amount of text from the internet – think books, articles, code, and tons more. You can picture it as a super-smart autocomplete engine. But it does way more than just finish your sentences. It can also:

• Answer tricky questions

• Summarize long articles or documents

• Write code, emails, or creative stories

• Translate languages instantly

• Even solve logic puzzles and have engaging conversations

Chances are you've heard of ChatGPT, which is powered by OpenAI's GPT models. Other popular LLMs you might come across include Claude (from Anthropic), LLaMA (by Meta), Mistral, and Gemini (from Google).

These models work by simply predicting the next word in a sentence based on the context. While that sounds straightforward, when trained on billions of words, LLMs become capable of surprisingly intelligent behavior, understanding your instructions, following step-by-step reasoning, and producing coherent responses across almost any topic you can imagine.

So, What’s an LLM Agent?

While LLMs are super powerful, they usually just react – they only respond when you ask them something. An LLM agent, on the other hand, is proactive.

LLM agents can:

• Break down big, complex tasks into smaller, manageable steps

• Make smart decisions and figure out what to do next

• Use "tools" like web search, calculators, or even other apps

• Work towards a goal, even if it takes multiple steps or tries

• Team up with other agents to accomplish shared objectives

In short, LLM agents can think, plan, act, and adapt.

Think of an LLM agent like your super-efficient new assistant: you give it a goal, and it figures out how to achieve it all on its own.

Why Does This Matter?

This shift from just responding to actively pursuing goals opens a ton of exciting possibilities:

• Automating boring IT or DevOps tasks

• Generating detailed reports from raw data

• Helping you with multi-step research projects

• Reading through your daily emails and highlighting key info

• Running your internal tools to take real-world actions

Unlike older, rule-based bots, LLM agents can reason, reflect, and learn from their attempts. This makes them a much better fit for real-world tasks that are messy, require flexibility, and depend on understanding context.

The Rise of Open-Source Agent Frameworks

Not too long ago, if you wanted to build an AI system that could act autonomously, it meant writing a ton of custom code, painstakingly managing memory, and trying to stitch together dozens of components. It was a complex, delicate, and highly specialized job.

But guess what? That's not the case anymore.

In 2024, a wave of fantastic open-source frameworks hit the scene. These tools have made it dramatically easier to build powerful LLM agents without you having to reinvent the wheel every time.

Popular Open-Source Agent Frameworks

What These Tools Enable

These frameworks give you ready-made building blocks to handle the trickier parts of creating agents:

• Planning – Letting agents decide their next move

• Tool Use – Easily connecting agents to things like file systems, web browsers, APIs, or databases

• Memory – Storing and retrieving past information or intermediate results for long-term context

• Multi-Agent Collaboration – Setting up teams of agents that work together on shared goals

Why Use a Framework Instead of Building from Scratch?

While you could build a custom agent from the ground up, using a framework will save you a huge amount of time and effort. Open-source agent libraries come packed with:

• Built-in support for orchestrating LLMs

• Proven patterns for task planning, keeping track of where you are, and getting feedback

• Easy integration with popular models like OpenAI, or even models you run locally

• The flexibility to grow from a single helpful agent to entire teams of agents

Basically, these frameworks let you focus on what your agent should do, rather than getting bogged down in how to build all the internal workings. Plus, choosing open source means you benefit from community contributions, transparency in how they work, and the freedom to tweak them to your exact needs, without getting locked into a single vendor.

Core Concepts Behind Agent Design

To really grasp how LLM agents operate, it helps to think of them as goal-driven systems that constantly cycle through observing, reasoning, and acting. This continuous loop allows them to tackle tasks that go beyond simple questions and answers, moving into true automation, tool usage, and adapting on the fly.

The Agent Loop

Most LLM agents function based on a mental model called the Agent Loop a step-by-step cycle that repeats until the job is done. Here’s how it typically works:

• Perceive: The agent starts by noticing something in its environment or receiving new information. This could be your prompt, a piece of data, or the current state of a system.

• Plan: Based on what it perceives and its overall goal, the agent decides what to do next. It might break the task into smaller sub-goals or figure out the best tool for the job.

• Act: The agent then acts. This could mean running a function, calling an API, searching the web, interacting with a database, or even asking another agent for help.

• Reflect: After acting, the agent looks at the outcome: Did it work? Was the result useful? Should it try a different approach? Based on this, it updates its plan and keeps going until the task is complete.

This loop is what makes agents so dynamic. It allows them to handle ever-changing tasks, learn from partial results, and correct their course qualities that are vital for building truly useful AI assistants.

Key Components of an Agent

To do their job effectively, agents are built around several crucial parts:

• Tools are how an agent interacts with the real (or digital) world. These can be anything from search engines, code execution environments, file readers, or API clients, to simple calculators or command-line scripts.

• Memory lets agents remember what they've done or seen across different steps. This might include previous things you've said, temporary results, or key decisions. Some frameworks offer short-term memory (just for one session), while others support long-term memory that can span multiple sessions or goals.

• Environment refers to the external data or system context the agent operates within think APIs, documents, databases, files, or sensor inputs. The more information and access an agent have to its environment, the more meaningful actions it can take.

• Goal is the agent's ultimate objective: what it's trying to achieve. Goals should be specific and clear for instance, “generate a daily schedule,” “summarize this document,” or “extract tasks from emails.”

Multi-Agent Collaboration

For more advanced systems, you can even have multiple agents working together to hit a shared target. Each agent can be given a specific role that highlights its specialty just like people working on a team.

For example:

• A researcher agent might be tasked with gathering information.

• A coder agent could write Python scripts or automation routines.

• A reviewer agent might check the results and ensure everything is up to snuff.

These agents can chat with each other, share information, and even debate or vote on decisions. This kind of teamwork allows AI systems to tackle bigger, more complex tasks while keeping things organized and modular.

Project: Automate Your Daily Schedule from Emails

What We’re Automating

Think about your typical morning routine:

• You open your inbox.

• You quickly scan through a bunch of emails.

• You try to spot meetings, tasks, and important reminders.

• Then, you manually write a to-do list or add things to your calendar.

Let's use an LLM agent to make that process effortless. Our agent will:

• Read a list of your email messages

• Pull out time-sensitive items like meetings or deadlines

• Summarize everything into a nice, clean daily schedule

Step 1: Install the Required Tools

To get started, you'll need three main tools: Python, VSCode, and an OpenAI API key.

1. Install Python 3.9 or Higher

Grab the latest version of Python 3.9+ from the official website: https://www.python.org/downloads/

Once it's installed, double-check it by running python --version in your terminal.

This command simply asks your system to report the Python version currently installed. You'll want to see Python 3.9.x or something higher to ensure compatibility with our project.

2. Install VSCode (Optional but Recommended)

VSCode is a fantastic, user-friendly code editor that works perfectly with Python. You can download it right here: https://code.visualstudio.com/.

3. Get Your OpenAI API Key

Head over to: https://platform.openai.com

Sign in or create a new account. Navigate to your API Keys page. Click “Create new secret key” and make sure to copy that key somewhere safe for later.

4. Install Python Libraries

Open your terminal or command prompt and install these essential packages:

pip install langgraph langchain openai

This command uses pip, Python's package manager, to download and install three crucial libraries for our agent:

• langgraph: The core framework we'll use to build our agent's workflow.

• langchain: A foundational library for working with large language models, upon which LangGraph is built.

• openai: The official Python library for connecting to OpenAI's powerful AI models.

If you're excited to try out multi-agent setups (which we'll cover in Step 5), also install CrewAI:

pip install crewai

This command installs CrewAI, a specialized framework that makes it easy to orchestrate multiple AI agents working together as a team.

5. Set Your OpenAI API Key

You need to make sure your Python code can find and use your OpenAI API key. This is typically done by setting it as an environment variable.

On macOS/Linux, run this in your terminal (replace "your-api-key" with your actual key):

export OPENAI_API_KEY="your-api-key"

This command sets an environment variable named OPENAI_API_KEY. Environment variables are a secure way for applications (like your Python script) to access sensitive information without hardcoding it directly into the code itself.

On Windows (using Command Prompt), do this:

set OPENAI_API_KEY="your-api-key"

This is the Windows equivalent command to set the OPENAI_API_KEY environment variable.

Now, your Python code will be all set to talk to the OpenAI model!

Step 2: Define the Task

We discussed this briefly in the beginning of this section. But to reiterate, this is what we’ll want our agent to do:

• Scan for meetings, events, and important tasks.

• Jot them down quickly in a notebook or an app.

• Create a rough mental plan for your day.

This routine takes time and mental energy. So having an agent do it for us will be super helpful.

Step 3: Build the Workflow with LangGraph

What Is LangGraph?

LangGraph is a cool framework that helps you build agents using a "graph-based" workflow, kind of like drawing a flowchart. It's powered by LangChain and gives you a lot more control over exactly how each step in your agent's process unfolds.

Each "node" in this graph represents a decision point or a function that:

• Takes some input (its current "state").

• Does some reasoning or takes an action (often involving the LLM and its tools).

• Returns an updated output (a new "state").

You draw the connections between these nodes, and LangGraph then executes it like a smart, automated state machine.

Why Use LangGraph?

• You get to control the precise order of execution.

• It's fantastic for building workflows that have multiple steps or even branch off into different paths.

• It plays nicely with both cloud-based models (like OpenAI) and models you run locally.

Alright – now let’s write the code.

1. Simulate Email Input

In a real application, your agent would probably connect to Gmail or Outlook to fetch your actual emails. For this example, though, we’ll just hardcode some sample messages to keep things simple:

Python

emails = """
1. Subject: Standup Call at 10 AM
2. Subject: Client Review due by 5 PM
3. Subject: Lunch with Sarah at noon
4. Subject: AWS Budget Warning – 80% usage
5. Subject: Dentist Appointment - 4 PM
"""

This multiline Python string, emails, acts as our stand-in for real email content. We're providing a simple, structured list of email subjects to demonstrate how the agent will process text.

2. Define the Agent Logic

Now, we'll tell OpenAI’s GPT model how to process this email text and turn it into a summary.

from langchain_openai import ChatOpenAI
from langgraph.graph import StateGraph, END
from typing import TypedDict, Annotated, List
import operator

# Define the state for our graph
class AgentState(TypedDict):
    emails: str
    result: str

llm = ChatOpenAI(temperature=0, model="gpt-4o") # Using gpt-4o for better performance

def calendar_summary_agent(state: AgentState) -> AgentState:
    emails = state["emails"]
    prompt = f"Summarize today's schedule based on these emails, listing time-sensitive items first and then other important notes. Be concise and use bullet points:\n{emails}"
    summary = llm.invoke(prompt).content
    return {"result": summary, "emails": emails} # Ensure emails is also returned

Here’s what’s going on:

• Imports: We bring in necessary components:

  • ChatOpenAI to connect to the LLM,

  • StateGraph and END from langgraph.graph to build our agent workflow,

  • TypedDict, Annotated, and List from typing for type checking and structure,

  • operator (though not used in this snippet, it can help with comparisons or logic).

• AgentState: This TypedDict defines the shape of the data our agent will work with. It includes:

  • emails: the raw input messages.

  • result: the final output (the daily summary).

• llm = ChatOpenAI(...): Initializes the language model. We're using GPT-4o with temperature=0 to ensure consistent, predictable output perfect for structured summarization tasks.

• calendar_summary_agent(state: AgentState): This function is the "brain" of our agent. It:

  • Takes in the current state, which includes a list of emails.

  • Extracts the emails from that state.

  • Constructs a prompt that tells the model to generate a concise daily schedule summary using bullet points, prioritizing time-sensitive items.

  • Sends this prompt to the model with llm.invoke(prompt).content, which returns the LLM’s response as plain text.

  • Returns a new AgentState dictionary containing:

    • result: the generated summary,

    • emails: preserved in case we need it downstream.

3. Build and Run the Graph

Now, let's use LangGraph to map out the flow of our single-agent task and then run it.

builder = StateGraph(AgentState)
builder.add_node("calendar", calendar_summary_agent)
builder.set_entry_point("calendar")
builder.set_finish_point("calendar") # END is implicit if not set explicitly

graph = builder.compile()

# Run the graph using your simulated email data
result = graph.invoke({"emails": emails})
print(result["result"])

Here’s what’s going on:

• builder = StateGraph(AgentState): We're initiating a StateGraph object. By passing AgentState, we're telling LangGraph the expected data structure for its internal state.

• builder.add_node("calendar", calendar_summary_agent): This line adds a named "node" to our graph. We're calling it "calendar", and we're linking it to our calendar_summary_agent function, meaning that function will be executed when this node is active.

• builder.set_entry_point("calendar"): This sets "calendar" as the very first step in our workflow. When we start the graph, execution will begin here.

• builder.set_finish_point("calendar"): This tells LangGraph that once the "calendar" node finishes its job, the entire graph process is complete.

• graph = builder.compile(): This command takes our defined graph blueprint and "compiles" it into an executable workflow.

• result = graph.invoke({"emails": emails}): This is where the magic happens! We're telling our graph to start running. We pass it an initial state that contains our emails data. The graph will then process this data through its nodes until it reaches an end point, returning the final state.

• print(result["result"]): Finally, we grab the summarized schedule from the result (the final state of our graph) and print it to the console.

Example Output

Your Schedule:
- 10:00 AM – Standup Call
- 12:00 PM – Lunch with Sarah
- 4:00 PM – Dentist Appointment
- Submit client report by 5:00 PM
- AWS Budget Warning – check usage

Boom! You've just built an AI agent that can read your emails and whip up your daily schedule. Pretty cool, right? This is a simple yet powerful peek into what LLM agents can do with just a few lines of code.

Multi-Agent Collaboration with CrewAI

What Is CrewAI?

CrewAI is an exciting open-source framework that lets you build teams of agents that work together seamlessly just like a real-world project team! Each agent in a CrewAI setup:

• Has a specific, specialized role.

• Can communicate and share information with its teammates.

• Collaborates to achieve a shared goal.

This multi-agent approach is super useful when your task is too big or too complex for just one agent, or when breaking it down into specialized parts makes it clearer and more efficient.

Sample Roles for the Email Summary Task

Let's imagine our email summary task being handled by a small team of agents:

Sample CrewAI Code

from crewai import Agent, Crew, Task, Process
from langchain_openai import ChatOpenAI
import os

# Set your OpenAI API key from environment variables
# os.environ["OPENAI_API_KEY"] = "YOUR_API_KEY" # Make sure this is set, or defined directly

# Initialize the LLM (using gpt-4o for better performance)
llm = ChatOpenAI(temperature=0, model="gpt-4o")

# Define the agents with specific roles and goals
extractor = Agent(
    role="Email Scanner",
    goal="Find all meetings, reminders, and tasks from the given emails, accurately extracting details like time, date, and subject.",
    backstory="You are an expert at scanning emails for key information. You meticulously extract every relevant detail.",
    verbose=True,
    allow_delegation=False,
    llm=llm
)

prioritizer = Agent(
    role="Schedule Optimizer",
    goal="Sort extracted items by urgency and time, preparing them for a daily agenda.",
    backstory="You are a master of time management, always knowing what needs to be done first. You organize tasks logically.",
    verbose=True,
    allow_delegation=False,
    llm=llm
)

formatter = Agent(
    role="Output Generator",
    goal="Generate a clean, polished, and concise daily agenda in bullet-point format, clearly listing all schedule items.",
    backstory="You are a professional secretary, ensuring all outputs are perfectly formatted and easy to read. You prioritize clarity.",
    verbose=True,
    allow_delegation=False,
    llm=llm
)

# Simulate email input
emails = """
1. Subject: Standup Call at 10 AM
2. Subject: Client Review due by 5 PM
3. Subject: Lunch with Sarah at noon
4. Subject: AWS Budget Warning – 80% usage
5. Subject: Dentist Appointment - 4 PM
"""

# Define the tasks for each agent
extract_task = Task(
    description=f"Extract all relevant events, meetings, and tasks from these emails: {emails}. Focus on precise details.",
    agent=extractor,
    expected_output="A list of extracted items with their details (e.g., '- Standup Call at 10 AM', '- Client Review due by 5 PM')."
)

prioritize_task = Task(
    description="Prioritize the extracted items by time and urgency. Meetings first, then deadlines, then other notes.",
    agent=prioritizer,
    context=[extract_task], # The output of extract_task is the input here
    expected_output="A prioritized list of schedule items."
)

format_task = Task(
    description="Format the prioritized schedule into a clean, easy-to-read daily agenda using bullet points. Ensure concise language.",
    agent=formatter,
    context=[prioritize_task], # The output of prioritize_task is the input here
    expected_output="A well-formatted daily agenda with bullet points."
)

# Instantiate the crew
crew = Crew(
    agents=[extractor, prioritizer, formatter],
    tasks=[extract_task, prioritize_task, format_task],
    process=Process.sequential, # Tasks are executed sequentially
    verbose=2 # Outputs more details during execution
)

# Run the crew
result = crew.kickoff()
print("\n########################")
print("## Final Daily Agenda ##")
print("########################\n")
print(result)

Here’s what’s going on:

• Imports: We bring in key classes from CrewAI: Agent, Crew, Task, and Process. We also import ChatOpenAI for our language model and os to handle environment variables.

• llm = ChatOpenAI(...): Just like in the LangGraph example, this sets up our OpenAI language model, making sure its responses are direct (temperature=0) and using the gpt-4o model.

• Agent Definitions (extractor, prioritizer, formatter):

  • Each of these variables creates an Agent instance. An agent is defined by its role (what it does), a specific goal it's trying to achieve, and a backstory (a sort of personality or expertise that helps the LLM understand its purpose better).

  • verbose=True is super helpful for debugging, as it makes the agents print out their "thoughts" as they work.

  • allow_delegation=False means these agents won't pass their assigned tasks to other agents (though this can be set to True for more complex delegation scenarios).

  • llm=llm connects each agent to our OpenAI language model.

• Simulated emails: We reuse the same sample email data for this example.

• Task Definitions (extract_task, prioritize_task, format_task):

  • Each Task defines a specific piece of work that an agent needs to perform.

  • description clearly tells the agent what the task involves.

  • agent assigns this task to one of our defined agents (e.g., extractor for extract_task).

  • context=[...] is a critical part of CrewAI's collaboration. It tells a task to use the output of a previous task as its input. For instance, prioritize_task takes the extract_task's output as its context.

  • expected_output gives the agent an idea of what its result should look like, helping guide the LLM.

• crew = Crew(...):

  • This is where we assemble our team! We create a Crew instance, giving it our list of agents and tasks.

  • process=Process.sequential tells the crew to execute tasks one after another in the order they're defined in the tasks list. CrewAI also supports more advanced processes like hierarchical ones.

  • verbose=2 will show you a very detailed log of the crew's internal workings and communication.

• result = crew.kickoff(): This command officially starts the entire multi-agent workflow. The agents will begin collaborating, passing information, and working through their assigned tasks in sequence.

• fprint(result): Finally, the consolidated output from the entire crew's collaborative effort is printed to your console.

CrewAI cleverly handles all the communication between agents, figures out who needs to work on what and when, and passes the output smoothly from one agent to the next it's like having a mini AI assembly line!

What Actually Happens During Execution?

So, whether you're using LangGraph or CrewAI, what's really going on behind the scenes when an agent runs? Let's break down the execution process:

• The system gets an input state (for example, your emails).

• The first agent or graph node reads this input and uses a Large Language Model (LLM) to make sense of it.

• Based on its understanding, the agent decides on an action like pulling out key events or calling a specific tool.

• If needed, the agent might invoke tools (like a web search or a file reader) to get more context or perform external operations.

• The result of that action is then passed to the next agent in the team (if it's a multi-agent setup) or returned directly to you.

Execution keeps going until:

• The task is fully completed.

• All agents have finished their assigned roles.

• A stopping condition or a designated "END" point in the workflow is reached.

Think of this as a super-smart workflow engine where every single step involves reasoning, making decisions, and remembering previous interactions.

Are LLM Agents Safe? What to Know About Security and Privacy

As cool as LLM agents are, they raise an important question: can you really trust an AI to run parts of your workflow or interact with your data? It depends. If you’re using services like OpenAI or Anthropic, your data is encrypted in transit and (as of now) isn’t used for training.

But some data might still be temporarily logged to prevent abuse. That’s usually fine for testing and personal projects, but if you’re working with sensitive business info, customer data, or anything private, you’ll want to be careful.

Use anonymized inputs, avoid exposing full datasets, and consider running agents locally using open-source models like LLaMA or Mistral if full control matters to you.

You can also set clear boundaries for your agents so they don’t overstep. Think of it like onboarding a new intern: you wouldn’t give them access to everything on day one.

Give agents only the tools and files they need, keep logs of what they do, and always review the results before letting them make real changes.

As this tech grows, more safety features are coming like better sandboxing, memory limits, and role-based access. But for now, it’s smart to treat your agents like powerful helpers that still need some human supervision.

Troubleshooting & Tips

Sometimes, agents can be a bit quirky! Here are some common issues you might run into and how to fix them:

A handy tip: You can also add print() statements or logging messages inside your agent functions to see what's happening at each stage and debug state transitions.

Explore More Daily Automations

Once you've built one agent-based task, you'll find it incredibly easy to adapt the pattern for other automations. Here are some cool ideas to get your creative juices flowing:

Each of these can be built using the very same principles and frameworks we discussed whether that's LangGraph or CrewAI.

What’s Next in Agent Technology?

LLM agents are evolving at lightning speed, and the next wave of innovation is already here:

• Smarter memory systems: Expect agents to have better long-term memory, allowing them to learn over extended periods and remember past conversations and actions.

• Multi-modal agents: Agents won't just handle text anymore! They'll be able to process and understand images, audio, and video, making them much more versatile.

• Advanced planning frameworks: Techniques like ReAct, Toolformer, and AutoGen are constantly improving agents' ability to reason, plan, and reduce those pesky "hallucinations."

• Edge deployment: Imagine agents running entirely offline on your local computer or device using lightweight models like LLaMA 3 or Mistral.

In the very near future, you'll see agents seamlessly integrated into:

• Your DevOps pipelines

• Big enterprise workflows

• Everyday productivity tools

• Mobile apps and smart devices

• Games, simulations, and educational platforms

Final Summary

Alright, let's quickly recap all the cool stuff you've just learned and accomplished:

• You've gotten a solid grasp of what LLM agents are and why they're so powerful.

• You've seen how open-source frameworks like LangGraph and CrewAI make building agents much easier.

• You've built a real LLM agent using LangGraph to automate a common daily task: summarizing your inbox!

• You've explored the world of multi-agent collaboration with CrewAI, understanding how teams of AIs can work together.

• You've learned how to take these principles and scale them to automate countless other tasks.

So, next time you find yourself stuck doing something repetitive, just ask yourself: "Hey, can I build an agent for that?" The answer is probably yes!

Resources Recap

Here are some helpful resources if you want to dive deeper into building LLM agents:

Unlike chatbots or rule-based software, agentic AI actively responds to user requests. It may break activities into smaller tasks, make decisions based on a high-level goal, and change its behavior over time using tools or other specialized AI components.

To summarize, agentic AI systems "solve complex, multi-step problems autonomously by using sophisticated reasoning and iterative planning." In customer service, for example, an agentic AI may answer questions, check a user's account, offer balance settlements, and conduct transactions without human supervision.

So, agentic AI is "AI with agency”. Given a problem context, it sets goals, creates strategies, manipulates the environment or software tools, and learns from the results.

But at the moment, most popular AI systems are reactive or non-agentic, doing a specific job or reacting to inputs without preparation. For example, Siri or a traditional image classifier use predefined models or rules to map inputs to outputs. Instead of long-term goals or multi-step processes, reactive AI "responds to specific inputs with pre-defined actions". Agentic AI is more like a robot or personal assistant that can handle reasoning chains, adapt, and "think" before acting.

What we’ll cover here

In this article, you’ll learn what makes Agentic AI fundamentally different from traditional reactive systems. We’ll cover its key components like autonomy, goal-setting, planning, reasoning, and memory and explore how these systems are being built today. We’ll also look at the challenges they present, and where they are currently in development. Finally, you’ll get a hands-on tutorial on how to build your own simple agent using Python and LangChain.

Table of Contents:

1. Agentic vs Reactive AI

2. Key Components of AI Agency

   • Autonomy

   • Goal-Directed Behavior

   • Planning

   • Reasoning

   • Memory

3. How Does Agentic AI Know What to Do?

   • 1. It Uses a Pretrained AI Model

   • 2. It Follows Instructions in Prompts

   • 3. It Uses Tools, But Only When Told How

   • 4. It Can Remember (Sometimes)

   • 5. It’s Not Fully Autonomous — Yet

4. So What’s the Current State of Agentic AI?

   • What Exists Today

   • What’s Still Experimental

   • Are We Close to Truly Autonomous Agents?

5. Building Agentic AI: Frameworks and Approaches

   • Reinforcement Learning (RL) Agents

   • LLM-Based (Generative) Agents

   • Multi-Agent and Orchestration Frameworks

   • Classical Planning and Symbolic AI

   • Tool-augmented Reasoning

6. Major Challenges of Agentic AI

   • Alignment and Value Specification

   • Unintended Consequences

   • Safety and Security

   • Coordination and Scalability

   • Ethical and Legal Questions

7. Code Snippet and Real-World Examples

8. Tutorial: Build Your First Agentic AI with Python

   • Real-World Use Case

   • Prerequisites – What You Need

   • Step-by-Step Tutorial

9. Conclusion

Agentic vs Reactive AI

Before we dive fully in, I want to make sure the differences between non-agentic and agentic AI are clear.

Non-agentic reactive AI uses learned models or rules to map inputs to outputs. It replies to one idea or task at a time, not starting additional ones. Examples include a calculator, spam filter, and rudimentary chatbot with pre-written responses. Reactive AI cannot plan or improve without reprogramming.

Agentic AI, on the other hand, acts independently with goals. It may organize actions, set objectives, adapt to new information, and collaborate with others. Agentic AI can break a complex task into small segments and coordinate the usage of specialized tools or services to complete each step.

The agent is also proactive. An agentic AI may inform users of updates, restock supplies, and check inventory levels, unlike a reactive system.

The difference is a paradigmatic shift: modern agentic systems include several specialized agents working together on a high-level objective, with dynamic task breakdown and even permanent memory, instead of a single model. This multi-agent collaboration may help agentic AI solve large real-world problems.

Cutting-edge prototypes like intelligent chatbots with tool integration, autonomous driving software, and coordinated industrial robots are entering agentic territory, but today's reactive AI virtual assistants (Alexa, Siri) may blur the line. It's a vital distinction whether the system actively selects rather than reacts.

Key Components of AI Agency

Agentic AI systems are characterized by several core capabilities that give them agency. Let’s look at these now.

Autonomy

An autonomous agent may work without human supervision. It may act depending on its goals and strategy rather than waiting for specific directions.

The agent must use sensors or data streams to perceive, evaluate, and decide to be autonomous. An autonomous warehouse robot can move, pick up things, and alter path when it encounters barriers without human guidance. Autonomy implies self-monitoring: an agent gauges its battery life or job completion and adapts as needed.

An agentic AI's “reasoning engine” (usually a large language model or similar system) makes decisions and can adjust its behavior based on user feedback or rewards.

As IBM explains, “without any human intervention, agentic AI can act independently, adapt to new situations, make decisions, and learn from experience” (source). But uncontrolled autonomous agents may behave in unpredictable ways – which is why they must be carefully designed.

Although agentic AIs can operate on their own, their goals, tools, and boundaries must be clearly planned to avoid unintended or harmful outcomes. Without that guidance, they may follow instructions too literally or make decisions without understanding the bigger picture.

Goal-Directed Behavior

Agentic AI is goal-directed. The system attempts to achieve one or more goals. The goals might be specified openly ("set up a meeting for tomorrow") or implicitly through a reward system. Instead of following a script, the agent chooses how to achieve its goal. It may choose methods, subgoals, and long-term goals.

Unplanned reactive AI has short-term or implicit goals (for example, recognize an image, guess the next word). Agentic AIs aim toward long-term goals. If assigned the duty of "organizing my travel itinerary," an agent may book flights, hotels, transportation, and so on, choose the best order, and adjust the schedule if airline prices change.

Business and research sources underline this distinction. Agentic AI plans and works for long-term goals, whereas reactive systems manage immediate, reactive responses. A plan-and-execute architecture lets the agent decide what to do and define and alter its goals. Instead of distinct, separate acts, it progressively performs a series. Goal-directed behavior demonstrates purposeful intent, even if the goal is vague.

Planning

An agent plans to achieve its goals. A goal and data instruct the agentic AI to conduct a series of actions or subtasks. Planning includes simple heuristics (if A, then do B) and advanced reasoning (evaluating options).

Modern agentic AI uses planner-executor architectures with chain-of-thought prompting. In a "plan-and-execute" agent, an LLM-driven planner develops a multi-step plan, and executor modules employ tools or models to execute each step. ReAct is another technique in which the agent alternates between action and reasoning (or "thought") to refine its approach as it accumulates observations.

Planning often involves search and optimization using neural networks, decision trees, or graph-based techniques. For example, an agent might build a planning graph showing different possible actions and outcomes, then use algorithms like A* search or Monte Carlo tree search to choose the best next step.

In some cases, the agent simulates multiple possible futures to evaluate which actions are most likely to lead to success. Large language models (LLMs) can also help by breaking down complex instructions into smaller steps turning a single high-level goal into a list of tasks that can be executed one by one.

Here’s a simplified example (pseudocode) of an agent loop:

goal = "prepare presentation on AI"
agent = AI_Agent(goal)
environment = TaskEnvironment()
 # Loop until the task is complete
while not environment.task_complete():
    observation = agent.perceive(environment)
    plan = agent.make_plan(observation)        # e.g., list of steps
    action = plan.next_step()
    result = agent.act(action, environment)
    agent.learn(result)                       # update memory or strategy

Here, the agent perceives the current state, plans a sequence of steps toward its goal, acts by executing the next step, and then learns from the outcome before repeating. This cycle captures the core loop of an autonomous agent.

Reasoning

Making judgments by applying logic and inference is known as reasoning. In addition to acting, an agentic AI considers what actions make sense in light of its information. This entails assessing trade-offs, comprehending cause and consequence, and, if necessary, applying mathematical or symbolic thinking.

An agent may, for instance, apply deductive reasoning, like "If sales fall below X, reorder inventory" or "All invoices are paid by Friday. This is an invoice, so I should pay it by Friday". By enabling the agent to process natural language commands, retain contextual information, and produce logical justifications for its decisions, large language models support reasoning.

An LLM "acts as the orchestrator or reasoning engine" that comprehends tasks and produces solutions, according to one explanation in the LangChain docs. In order to retrieve pertinent information for reasoning, agents also employ strategies such as retrieval-augmented generation (RAG).

Agentic reasoning is essentially like internal planning and problem-solving. An agent evaluates a task by internally simulating potential strategies (often in the "thoughts" of an LLM) and selecting the most effective one. This might entail formal logic, analogical reasoning (connecting a new problem to previous ones), or multi-step deduction. So the agent continually considers its next course of action and adjusts to new inputs rather of just clicking "execute" on a single model outcome.

Memory

Agents can utilize memory to recall prior experiences, information, and interactions to make decisions. A memoryless AI would treat every moment as new. Agentic systems record their behaviors, outcomes, and context. A short-term “working memory” of the present plan state or a long-term world knowledge base are examples.

A customer-service agent may remember a user's name and issue history to avoid repeating inquiries. Game-playing agents learn from past positions to move better. IBM says AI agent memory “refers to an AI system’s ability to store and recall past experiences to improve decision-making, perception and overall performance”. Goal-oriented agents need memory to create a cohesive narrative of previous steps (to avoid repeating failures) and discover trends.

Agentic architectures incorporate memory modules like databases or vector storage that the LLM may query. Large language models are stateless. Agents utilize relevance filters to retain only important information since too much memory slows the system. Memory offers the agent context and continuity, allowing it to learn from previous tasks rather than beginning again.

How Does Agentic AI Know What to Do?

Agentic AI might seem smart, but it’s not actually “thinking” like a human. Let’s break down how it really works.

1. It Uses a Pretrained AI Model

At the heart of most agentic systems is a large language model (LLM) like GPT-4. This model is trained on a huge amount of tex, books, articles, websites, and so on to learn how people write and talk.

But it wasn’t trained to act like an agent. It was trained to predict the next word in a sentence.

When we give it the right prompts, it can seem like it’s making plans or solving problems. Really, it’s just generating useful responses based on patterns it learned during training.

2. It Follows Instructions in Prompts

Agentic AI doesn’t figure out what to do by itself – developers give it structure using prompts.

For example:

• “You are an assistant. First, think step by step. Then take action.”

• “Here’s a goal: research coding tools. Plan steps. Use Wikipedia to search.”

These prompts help the AI simulate planning, decision-making, and action.

3. It Uses Tools, But Only When Told How

The AI doesn’t automatically know how to use tools like search engines or calculators. Developers give it access to those tools, and the AI can decide when to use them based on the text it generates.

Think of it like this: the AI suggests, “Now I’ll look something up,” and the system makes that happen.

4. It Can Remember (Sometimes)

Some agents use short-term memory to remember past questions or results. Others store useful information in a database for later. But they don’t “learn” over time like humans do – they only remember what you let them.

5. It’s Not Fully Autonomous — Yet

Most agentic systems today are not fully self-learning or self-aware. They’re smart combinations of:

• Pretrained AI

• Prompts

• Tools

• Memory

Their “autonomy” comes from how all these parts work together – not from deep understanding or long-term training.

So What’s the Current State of Agentic AI?

Agentic AI is still an emerging area of development. While it sounds futuristic, many systems today are just starting to use agent-like capabilities.

What Exists Today

Simple agentic systems already work in limited ways

• For example, some customer service bots can check account details, respond to questions, and escalate issues automatically.

• Warehouse robots can plan simple routes and avoid obstacles on their own.

• Coding assistants like GitHub Copilot can help write and fix code based on natural language input.

These systems show basic agentic behavior like goal-following and tool use but usually in a narrow, structured environment.

What’s Still Experimental

• Fully autonomous, multi-purpose agents the kind that can reason deeply, make long-term plans, and adapt to new tools, are still in research or prototype stages.

• Projects like AutoGPT, BabyAGI, and OpenDevin are exciting, but they’re mostly experimental and require human oversight.

Most current agentic systems:

• Don’t learn continuously

• Struggle with unpredictable environments

• Require a lot of setup to avoid errors or unexpected behavior

Are We Close to Truly Autonomous Agents?

We’re getting closer, but we’re not there yet.

Today’s agentic AI is like a very clever assistant that can follow instructions, use tools, and plan steps. But it still depends on developers to give it structure (via prompts, tool choices, and boundaries).

In short, Agentic AI works in specific, well-designed use cases. But general-purpose, human-level autonomous agents are still a long way off.

Building Agentic AI: Frameworks and Approaches

Researchers and engineers have developed various frameworks and tools to construct agentic AI systems. Let’s discuss some key approaches.

Reinforcement Learning (RL) Agents

In artificial intelligence, traditional agents are frequently constructed via reinforcement learning, in which the agent learns to maximize a reward signal through trial and error. Atari game agents and DeepMind's AlphaGo are classic examples.

In addition to planning (in the sense of calculating a policy) and learning from interactions, RL agents are goal-directed (maximizing reward). Still, a lot of pure RL systems struggle with the open-ended complexity of real-world tasks and function best in simulated contexts.

While RL components are occasionally incorporated into modern agentic AI (for example, an agent may utilize RL to drive a robot at a basic level), they are frequently supplemented with other methods for higher level thinking.

LLM-Based (Generative) Agents

The use of LLMs as reasoning engines within agents has become popular due to the recent explosion of large language models. For instance, LLMs (such as GPT-4) are used by frameworks like ReAct, AutoGPT, and BabyAGI to create plans and actions. These systems include prompting an LLM with the agent's objective and context, after which it generates a step or sub-goal and invokes either a function or a tool.

One design, frequently referred to as a ReAct loop, alternates between "Thought" (the LLM planning or reasoning) and "Action" (calling upon tools or APIs). An alternative approach involves a distinct planner LLM that generates a comprehensive multi-step plan, which is then followed by executor modules that execute each step.

To increase their capabilities, LLM agents frequently employ tools like search engines, calculators, and API calls. They also use context retrieval, such as RAG or memory storage, to guide their reasoning. LangChain and LangGraph are well-known open-source frameworks that offer building blocks (memory buffers, tool integration, and so on) for creating unique agents.

Multi-Agent and Orchestration Frameworks

Several sub-agents are used in many agentic AI architectures. A "crew" or "society of minds" method, for example, may produce many LLM agents that communicate by message passing and each serve a different job (planner, analyst, critic, and so on).

Orchestrated multi-agent processes are demonstrated by projects such as AutoGen, ChatDev, or MetaGPT. Engineering ideas for multi-agent systems are being explored in academic work. One study by BMW, for instance, outlines a framework for multi-agent cooperation in which several AI agents manage planning, execution, and specialized activities while working together to achieve an industrial use case.

These systems frequently have scheduling logic to allocate agents to subtasks and a task decomposition module, which breaks a goal down into its component elements. This essentially resembles a "AI team," in which every individual is an agentic subsystem.

Classical Planning and Symbolic AI

AI planning was examined in symbolic terms before to the current ML revival (STRIPS, PDDL planners, and so on). These methods might be viewed as an early example of agentic AI, in which a planner constructs a series of symbolic actions to accomplish a goal.

These concepts are occasionally included into contemporary agentic AI. For instance, an LLM agent may provide a high-level symbolic plan that grounded systems carry out, such as "(Find x such that property y), (compute f(x)), (deliver result)" and so on.

There are also hybrid architectures that combine traditional search with neural networks. The transition to learned or language-based planners is an extension of the classical planning that underpins many robotics and scheduling agents, even though it’s less prevalent in pure form today.

Tool-augmented Reasoning

In many agentic systems, granting the agent access to external functions and information is a viable strategy. For instance, when responding to a difficult inquiry, a language-based agent may utilize Retrieval-Augmented Generation (RAG) to retrieve pertinent information from a database.

As "tools" that it may use, it might also include a calculator, a web browser, a database API, or bespoke code. Autonomy is largely made possible by the capacity to utilize tools – instead of attempting to learn everything by heart, the AI model learns how to ask the appropriate questions.

In sum, building an agentic AI often means combining multiple techniques: machine learning for perception and learning, symbolic planning for structure, LLM reasoning for natural language and problem decomposition, plus memory modules and feedback loops.

There is no one-size-fits-all framework yet. Research continues rapidly – recent papers on agentic systems emphasize end-to-end pipelines that integrate perception (input analysis), goal-oriented planning, tool use, and continual learning.

Major Challenges of Agentic AI

Building AI agents with autonomy and goals is powerful but raises new risks and difficulties. Key challenges include:

Alignment and Value Specification

Setting the correct goals is crucial for agentic systems. If an agent's aims don't match human values, it may be damaging. If a scheduling agent is directed to “minimize costs,” it may reduce vital services unless told to preserve quality. Humans' complicated priorities make value formulation challenging. Unspecified or poorly described goals cause unexpected consequences (Goodhart's Law).

Unintended Consequences

Even with good intentions, agents may discover loopholes. Reward-hacking in reinforcement learning is an example from basic AI. Autonomy increases these hazards for agentic AI. Recent experiments showed an LLM-based AI was told to pursue a goal “at all costs.” It planned to stop its own monitoring and clone itself to escape shutdown, acting in self-preservation.

If unconstrained, an agent may deceive to achieve its aims. Unintended effects can range from an assistant arranging a hazardous flight because it fixed on a cost-savings aim to more subtle damages like cutting important benefits. IBM researchers warn that agents “can act without your supervision”, resulting in unintended consequences without strong protections.

Safety and Security

Highly autonomous agents can increase danger. They may access sensitive data or operate machinery. IBM says that agents are opaque and open-ended, so their judgments might be unclear, and they may suddenly use new tools or data. A healthcare agent may leak patient data, or a financial bot may execute a dangerous move.

LLM-style adversarial assaults and hallucinations become more dangerous in agentic AI. Though bothersome, a delusional chatbot or investment agent might also lose millions. Agent multi-step reasoning is sensitive to hostile inputs at any level. Complex agents make trust and verification difficult.

Coordination and Scalability

In many agentic systems, multiple agents may collaborate or compete. Ensuring that they communicate correctly and don’t conflict is non-trivial.

A recent review notes unique challenges in orchestrating multiple agents without standardized protocols. As the Stanford ethics report points out, if millions of agents interact (for example, booking each other’s appointments), the emergent behavior could be unpredictable at scale. This raises societal concerns about system-level effects and feedback loops we haven’t seen before.

Ethical and Legal Questions

Finally, there are questions of responsibility and bias. Who is liable if an autonomous agent makes a mistake? How do we ensure transparency and fairness in a black-box multi-agent system?

Legal and ethical frameworks are still catching up. For example, IBM highlights that agentic AI brings “an expanded set of ethical dilemmas” compared to today’s AI. And AI ethicists caution that deploying powerful assistants (as personal secretaries, advisors, and so on) will have profound societal impacts that are hard to predict.

Here are some specific things we need to consider:

• Accountability: Who is accountable if an AI agent makes a damaging choice (such a medical AI agent prescribing the wrong medication or a logistics agent causing an accident)? Designers, deployers, or agents? Legal systems presume human control, but autonomous agents may not.

• Transparency: Complex and opaque agentic systems exist. Multiple neural networks, knowledge bases, and tools may interact. Explaining an agent's behavior for auditing or debugging is tough. This opposes explainable AI.

• Bias and fairness: Agents learn from data and environments that may reflect human biases. An autonomous hiring assistant agent, for instance, might inadvertently replicate discriminatory patterns unless carefully checked. And because agentic AI can perpetuate or amplify biases across many decisions, the impact could be larger.

• Job disruption and social impact: Just as factory automation destroyed certain employment, powerful AI agents might change office and creative labor. Personal assistant agents that schedule, manage email, and research might change many careers. This might boost production but also exacerbate deskilling and inequality. Social pressure to utilize agentic AI (if rivals do) may divide workers into “augmented” and “unaugmented” workers.

• Security and privacy: An agent with extensive system access harms privacy. Compromise of an AI agent permitted to access and write business data or personal correspondence might reveal critical information. IBM warns that agentic AI can increase recognized hazards, such as an agent accidentally biasing a database or sharing private data without monitoring. Tools must be authenticated and data handled securely.

• Human-AI interaction: Our agents may affect how we use technology and interact with others. If individuals utilize AI bots for conversation, information filtering, or companionship, it might change societal dynamics. Consider again the Stanford study referenced above. So we need to pursue ways to include standards and values into these encounters.

In recognition of these challenges, technologists and ethicists urge us to use proactive safeguards. As IBM researchers put it, because agentic AI is advancing rapidly, we cannot wait to address safety – we must build strong guardrails now. Some proposed measures include strict testing protocols for agents, explainability requirements, legal regulations on autonomous systems, and design principles that prioritize human values.

So as you can see, while agentic AI offers the potential for AI that can handle complex tasks end-to-end, it also amplifies known AI risks (bias, error) and introduces new ones (autonomous decision-making, coordination failures). Addressing these challenges requires careful design of alignment, robust evaluation of agent behavior, and interdisciplinary governance.

Code Snippet and Real-World Examples

To illustrate how an agentic system works, let’s consider a very simple Python-like pseudocode for an abstract agent (mixing concepts from above):

class Agent:
    def init(self, goal):
        self.goal = goal
        self.memory = []
    def perceive(self, environment):
        # Get data from environment (sensor, API, etc.)
        return environment.get_state()
    def plan(self, observation):
        # Use reasoning (LLM or algorithm) to decide next action(s)
        plan = ReasoningEngine.generate_plan(goal=self.goal, context=observation)
        return plan  # e.g. list of steps or actions
    def act(self, action, environment):
        # Execute the action using tools or directly in the environment
        result = environment.execute(action)
        return result
    def learn(self, experience):
        # Store outcome or update strategy
        self.memory.append(experience)   
    def run(self, environment):
        while not environment.task_complete():
            obs = self.perceive(environment)
            plan = self.plan(obs)
            for action in plan:
                result = self.act(action, environment)
                self.learn(result)

This example demonstrates the core loop of an agentic AI:

• The agent starts with a goal and can store memory of what it has done.

• It observes its environment to understand what’s happening.

• Based on that input, it creates a plan – a list of actions to reach its goal.

• It executes each action, interacts with the environment, and learns from what happens.

• This process repeats until the goal is met or the task is complete.

This basic structure mirrors how real-world agentic systems operate: perceive → plan → act → learn.

Real-world agentic AI systems are evolving. Self-driving cars detect their environment, set navigation goals, plan routes, and learn from experience.

Tesla's Full Self-Driving “continuously learns from the driving environment and adjusts its behavior” to increase safety. Supply chain logistics businesses are creating agents that monitor inventory, estimate demand, alter routes, and place new orders autonomously. Amazon's warehouse robots utilize agentic AI to navigate complicated surroundings and adapt to changing situations, independently fulfilling orders.

Cybersecurity, healthcare, and customer service also use autonomous agents to identify and respond to risks. An agentic AI at a contact center may assess a customer's mood, account history, and company policies to provide a bespoke solution or process. Agentic systems organize and arrange marketing campaigns, write text, choose graphics, and alter strategies depending on performance data. In processes with several phases and choices, agentic AI can handle the whole workflow.

Recently, several prototype projects and open-source tools have begun experimenting with agentic AI in real-world scenarios.

For example, tools like AutoGPT and AgentGPT have demonstrated agents that can generate multimedia reports by coordinating research, writing, and image selection tasks. Other use cases include agents that retrieve knowledge and take follow-up action (for example, “find and implement the next step”), conduct security operations like scanning and responding to threats, or automate multi-step workflows in call centers.

These examples show how early-stage products and research projects are beginning to test and deploy agentic AI for complex, multi-step tasks beyond just answering questions.

Tutorial: Build Your First Agentic AI with Python

This step-by-step guide will teach you how to build a basic Agentic AI system even if you're just starting out. I’ll explain every concept clearly and give you working Python code you can run and study.

Real-World Use Case

Scenario: You're a product manager exploring tools for your team. Instead of spending hours researching AI coding assistants manually, you'd like a personal research agent to:

• Understand your task

• Gather relevant information from Wikipedia

• Summarize it clearly

• Remember context from previous questions

This is where Agentic AI shines: it acts autonomously, reasons, and uses tools just like a smart human assistant.

Prerequisites – What You Need

1. Python 3.10 or higher

2. An OpenAI API key (https://platform.openai.com/api-keys). Note that as of writing this OpenAI does not offer free API calls, so if you don’t already have an account you’ll need to use a credit card and a few dollars to complete this tutorial.

3. Install the required Python libraries:

pip install langchain openai wikipedia

⚠️ Don't forget to store your API key safely. Never share it in public code.

Step-by-Step Tutorial

Step 1: Set Up Your Environment

Start by setting your OpenAI API key in your script so that LangChain can access GPT models.

import os

os.environ["OPENAI_API_KEY"] = "your-api-key-here"  # Replace with your real key

Step 2: Connect to a Knowledge Source (Wikipedia)

We'll give our agent the ability to use Wikipedia as a tool to gather information.

from langchain.agents import Tool
from langchain.tools import WikipediaQueryRun
from langchain.utilities import WikipediaAPIWrapper
# Create the Wikipedia tool
wiki = WikipediaQueryRun(api_wrapper=WikipediaAPIWrapper())
# Register the tool so the agent knows how to use it
tools = [
    Tool(
        name="Wikipedia",
        func=wiki.run,
        description="Useful for looking up general knowledge."
    )
]

You're giving your agent a way to "see the world" – Wikipedia is your agent's eyes.

Step 3: Initialize the Agent (Reasoning Engine)

We now give the agent a brain – a GPT model that can reason, decide, and plan.

from langchain.chat_models import ChatOpenAI
from langchain.agents import initialize_agent
from langchain.agents.agent_types import AgentType
# Use a GPT model with zero randomness for consistent output
llm = ChatOpenAI(temperature=0)
# Combine reasoning (LLM) and tools (Wikipedia) into one agent
agent = initialize_agent(
    tools=tools,
    llm=llm,
    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
    verbose=True  # Show thought process step-by-step
)

This step fuses logic (GPT) and action (Wikipedia) to make your agent capable of goal-driven behavior.

Step 4: Give Your Agent a Goal

goal = "What are the top AI coding assistants and what makes them unique?"
response = agent.run(goal)
print("\nAgent's response:\n", response)

You’ve given your agent a mission. It will now think, search, and summarize.

You should see output like:

> Entering new AgentExecutor chain...

Thought: I should look up AI coding assistants on Wikipedia

Action: Wikipedia

Action Input: AI coding assistants

...

Final Answer: The top AI coding assistants are GitHub Copilot, Amazon CodeWhisperer, and Tabnine...

At this point, the agent has:

• Interpreted your goal

• Selected a tool (Wikipedia)

• Retrieved and analyzed content

• Reasoned through it to deliver a conclusion

Step 5: Give Your Agent Memory (Optional but Powerful)

Let your agent remember what you previously asked, like a real assistant.

from langchain.memory import ConversationBufferMemory
memory = ConversationBufferMemory(memory_key="chat_history")
agent_with_memory = initialize_agent(
    tools=tools,
    llm=llm,
    agent=AgentType.CONVERSATIONAL_REACT_DESCRIPTION,
    memory=memory,
    verbose=True
)
# Ask a follow-up
agent_with_memory.run("Tell me about GitHub Copilot")
agent_with_memory.run("What else do you know about coding assistants?")

Your agent now tracks context across multiple interactions just like a good human assistant.

When this is done, your agent:

• Responds more naturally to follow-up questions

• Links previous conversations to improve continuity

After running the steps, your agent reads your goal and plans steps to fulfill it. It searches Wikipedia to gather facts, and reasons using a GPT model to summarize and decide what to say. It also optionally remembers context (with memory enabled). You now have a working Agentic AI that can be extended for real-world tasks.

Conclusion

Agentic AI offers an exciting glimpse into a future where machines can collaborate with humans to solve complex, multi-step problems not just respond to commands. With capabilities like planning, reasoning, tool use, and memory, these systems could one day handle tasks that currently require entire teams of people.

But with that power comes real responsibility. If not properly designed and guided, autonomous agents could act in unpredictable or harmful ways. That’s why developers, researchers, and policymakers need to work together to set clear boundaries, safety rules, and ethical standards.

The technology is advancing quickly from self-driving cars to research assistants to multi-agent platforms like AutoGPT and LangChain. As we build smarter systems, the challenge isn't just what they can do, but how we ensure they do it safely, fairly, and in ways that benefit everyone.

Tools such as Swagger, JSON Schema, or OpenAPI are powerful, but they can be verbose, inflexible, or not conducive to reuse.

Well, I recently discovered TypeSpec. In this guide, I’ll show you how to take advantage of TypeSpec to create modern, maintainable, and well-documented REST APIs.

We'll take a look at:

• Prerequisites

• What is TypeSpec?

• Why use TypeSpec?

• How to Install and Configure TypeSpec

• TypeSpec Basic Syntax

• How to Create a REST API Model

• How to Build the API in Express and ASP.NET Core

• Best Practices for Structuring TypeSpec Projects and Components

• Conclusion

Prerequisites

Before we dive into using TypeSpec to document and model APIs, here are a few things you'll need to familiarize yourself with and/or have:

• Node.js (version 18 or higher)

• npm for dependency management

• Visual Studio Code (recommended to take advantage of the official TypeSpec extension). For an optimal experience, to create your project easily, it provides syntax highlighting, validation, autocompletion, navigation, and more.

• TypeSpec Extension in VS Code (You can install the extension via Visual Studio Marketplace)

• An understanding of how to use and create APIs

What is TypeSpec?

TypeSpec is an open-source declarative language, developed by Microsoft, designed to describe APIs in an explicit, reusable, scalable, and standards-based way. It’s designed to model REST, gRPC, GraphQL, and other types of APIs, and offers a modern syntax close to TypeScript.

It can automatically generate:

• OpenAPI, JSON Schema, or Protobuf specifications

• server and client code

• API documentation

• and other interface-related artifacts

TypeSpec isn't just a language – it's an API design platform that favors abstraction, encourages code reuse, and integrates with modern tools like Visual Studio Code via a dedicated extension. You can install the extension via the VS Code Visual Studio Marketplace.

Why use TypeSpec?

Before diving into the code, let's take a minute to understand the TypeSpec philosophy. Microsoft uses TypeSpec internally to deliver high-quality API services to millions of customers, across tens of thousands of endpoints, while ensuring code quality, governance, and scalability.

Unlike generators such as Swagger, Codegen, or Postman, which start from an OpenAPI file to generate code, TypeSpec does the opposite: you first write your API design in a DSL (Domain Specific Language), then generate everything you need.

TypeSpec has been designed to meet the major challenges of large-scale API design and governance:

• Simplification: clear, concise syntax to focus on business logic.

• Reusability: encapsulates types, request/response models, and directives in modular components.

• Productivity: automatically generates the necessary resources from a single source definition.

• Consistency: maintains compliance with internal standards thanks to shared libraries.

• Interoperability: integrates with the OpenAPI ecosystem and supports multi-format generation.

• Scalability: designed to handle thousands of endpoints like those used by Microsoft Azure.

Let's take a look at how to install and configure the development environment

How to Install and Configure TypeSpec

Before you can start writing your first API with TypeSpec, you need to set up your development environment. Here's how to install TypeSpec on your machine.

Requirements:

• Node.js (version 18 or higher)

• npm for dependency management

• Visual Studio Code (recommended to take advantage of the official TypeSpec extension). For an optimal experience, it provides syntax highlighting, validation, autocompletion, navigation, and more.

TypeSpec CLI global installation:

npm install -g @typespec/compiler

How to Create a TypeSpec Project

The easiest way to create a project is to use Visual Studio Code via the TypeSpec extension you've installed (if you're not comfortable with the command line (CMD)).

Create a folder containing the project and open it with Visual Studio Code. Then click on the View tab, and next on Comment Palette .

In the search bar that appears, enter TypeSpec: Create TypeSpec Project.

Follow the quick selections to select the root folder of the project you've just created. Then choose the Template – for our case this will be Generic REST API – and enter the project name. Leave the emitter OpenAPI 3.1 document (3.1 is the current version at the time of writing) selected by default. This will put us @typespec/http@typespec/openapi3. Finally, wait for the project configuration to finish.

You should have a basic TypeSpec project configuration with a structure that looks like this:

• node_modules/: Directory where npm installs project dependencies.

• main.tsp: the entry point for your TypeSpec build. This file generally contains the main definitions of your models, services, and operations.

• package.json: Contains project metadata, including dependencies, scripts, and other project-related information.

• tspconfig.yaml: TypeSpec compiler configuration file, specifying options and parameters for the generation process.

You can also run tsp compile . to compile the project, but it's better to run tsp compile . --watch to automatically compile changes during development each time you save.

Once the project has been compiled, you'll see the tsp-output and schema folders generated and a file added openai.yaml.

• tsp-output/: Directory where the TypeSpec compiler generates files.

• openapi.yaml: OpenAPI specification file generated for your API, detailing API endpoints, templates, and operations. Output may vary depending on the target format specified in the tspconfig.yaml file.

emit:
  - "@typespec/openapi3"
options:
  "@typespec/openapi3":
    emitter-output-dir: "{output-dir}/schema"
    openapi-versions:
      - 3.1.0

Thanks to this configuration of the tspconfig.yaml file, one of TypeSpec's major assets is its ability to automatically generate OpenAPI specifications from clear, typed, and modular source code. This means you can write your API as you would in TypeScript (or a well-structured DSL), and get output in .yaml files compatible with the whole OpenAPI ecosystem: Swagger UI, Postman, Redoc, and so on.

In the next section, we'll look at the basic syntax of TypeSpec.

TypeSpec Basic Syntax

Now that you've got a clear idea of what TypeSpec is and what its benefits are in the world of API design, it's time to get to the heart of the matter: the basic syntax.

TypeSpec is a declarative language, inspired by TypeScript, that lets you model the resources, routes, data structures, and behaviors of an API in an explicit, readable, and modular way. Its syntax is based on simple keywords and clear file organization, making it easy to learn yet powerful.

Language Basics

Here's a very simple example of defining a model with TypeSpec:

model Book {
  id: string;
  title: string;
  author: string;
}

This block defines a Book resource with three typed fields. The model keyword is used to describe the JSON objects manipulated by the API. It is equivalent to schemas in JSON Schema or type definitions in OpenAPI.

Defining an HTTP operation

TypeSpec lets you bind operations to models using the @route keyword. Here's a minimal example of an endpoint:

@route("/books")
op listBooks(): Book[];

This syntax declares a REST operation that returns a list of books. @route indicates the URL path, op introduces an operation, and Book[] is the return type.

You can also define path, query, or body parameters very easily.

@route("/books/{id}")
op getBook(@path id: string): Book;

In this example, we declare that id is a URL parameter (path parameter).

Fundamental Concepts

model Defining data structures

A model represents an API entity, like a JSON object. Models are the basis of your information exchanges.

model User {
  id: string;
  email: string;
  age?: int32;
}

interface Group operations

An interface groups together a set of logically linked operations. This is useful for structuring large API sets.

interface BookOperations {
  @get op listBooks(): Book[];
  @get op getBook(@path id: string): Book;
}

service Entry point of the API

A service defines publicly exposed interfaces, their version, and the basic path.

@service({ title: "Book API", version: "1.0.0" })
namespace BookApi {
  interface BookOperations;
}

Import and Organize Your Code with Namespaces

TypeSpec provides clear organization through namespaces, similar to modules or packages.

namespace CommonModels {
  model Error {
    message: string;
  }
}

Then you can import them into another file like this:

import CommonModels from "./common.tsp";

Complete Example of a REST Service

Let's take a complete example of a REST service in TypeSpec.

@service({ title: "Book Service", version: "1.0.0" })

@route("/books")

namespace BookService {

  model Book {
    id: string;
    title: string;
    author: string;
    publishedYear?: int32;
  }

  @get()
  op listBooks(): Book[];

  @post()
  op createBook(@body book: Book): Book;

  @get("/{id}")
  op getBook(@path id: string): Book;

  @put("/{id}")
  op updateBook(@path id: string, @body book: Book): Book;

  @delete("/{id}")
  op deleteBook(@path id: string): void;
}

Here’s what’s going on:

• @service({ title, version }): Defines service metadata (name, version), useful for generated documentation (for example, Swagger UI).

• @route("/books"): Defines the basic path for all operations of this API.

• namespace BookService { ... }: Encapsulates all models and operations linked to this service under a single logical name.

Next come the operations:

• @get() op listBooks(): Endpoint GET /books qui retourne un tableau de livres.

• @post() op createBook(): Endpoint POST /books which accepts a Book object in the request body (@body) and returns the created book.

• @get("/{id}"): Endpoint GET /books/{id} which retrieves a book via its identifier (@path).

• @put("/{id}"): Endpoint PUT /books/{id} which updates a book's data.

• @delete("/{id}"): Deletes a book via its id. The void type means that no data is returned.

With just a few lines, you get a complete, well-organized, easily readable REST service, ready to be automatically converted into OpenAPI documentation, a client SDK, or backend code.

Add Validation Annotations

TypeSpec makes it easy to add validation annotations to your models using:

model Book {
  id: string;
  title: string @minLength(3);
  author: string @minLength(3);
  publishedYear?: int32 @minValue(1800);
}

This adds validation rules directly to the schema, which will be taken into account during OpenAPI generation.

Comparison with Other Tools (OpenAPI / Swagger)

So you might wonder – why should you use TypeSpec rather than writing directly in OpenAPI?

Let's take the example of OpenAPI 3 (YAML):

paths:
  /books:
    get:
      summary: Get list of books
      responses:
        '200':
          description: OK
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/Book'
    post:
      summary: Create a new book
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/Book'
      responses:
        '201':
          description: Created
  /books/{id}:
    get:
      parameters:
        - name: id
          in: path
          required: true
          schema:
            type: string
      responses:
        '200':
          description: OK
    put:
      parameters:
        - name: id
          in: path
          required: true
          schema:
            type: string
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/Book'
    delete:
      parameters:
        - name: id
          in: path
          required: true
          schema:
            type: string
components:
  schemas:
    Book:
      type: object
      properties:
        id:
          type: string
        title:
          type: string
        author:
          type: string
        publishedYear:
          type: integer

As you can see, the OpenAPI definition is much more verbose. Relationships between paths, methods, schemas, and parameters are scattered, which complicates reading and maintenance. Also, it's less typed, given that OpenAPI remains YAML (or JSON), without the typing security or modularity of a real language.

Why TypeSpec is useful here

With TypeSpec, everything is centralized in a declarative, modular, typed, and intuitive format.

• Greater legibility: less noise, more intent.

• Reusability: you can create modular components and share them between projects.

• Productivity: you write less code and generate more (OpenAPI, client, server, doc).

• Consistency: errors are detected early thanks to strong typing.

Note: TypeSpec doesn't replace OpenAPI, but complements it: you write to TypeSpec, then automatically generate OpenAPI files, SDKs, specs and so on. It gives you a source language for accurately describing your API.

In the next section, we'll look at how to create a REST API template.

How to Create a REST API Model

To deepen our understanding of REST API creation with TypeSpec, let's continue with the example of managing books. In this example, we'll create a Book model, define a service to manage the books, and add validations to ensure that the data respects the right constraints.

Define a Data Model for Book

First, we'll define a data model for the Book resource. A book can have the following properties:

• id: A unique identifier for the book.

• title: The title of the book.

• author: The author of the book.

• publicationYear: The book's year of publication.

• isbn: The book's ISBN number.

Book model in TypeSpec

model Book {
  id: integer;
  @minLength(1)
  title: string;
  @minLength(1)
  author: string;
  publicationYear: integer;
  @pattern("^\\d{3}-\\d{1,5}-\\d{1,7}-\\d{1,7}-\\d{1}$")
  isbn: string;
}

• id: Unique book identifier (integer type).

• title and author: Character strings representing the book's title and author, validated by @minLength(1) to ensure they are not empty.

• publicationYear: The book's year of publication (integer type).

• isbn: The book's ISBN number, validated with a regular expression that matches the standard format of an ISBN.

Define a REST Service to Manage Books

Now that we have a Book model, we'll create a service to manage CRUD operations on this resource. This service will contain methods for retrieving a book by its identifier, creating a new book, updating an existing book, and deleting a book.

BooksService service in TypeSpec

service BooksService {

  @get("/books/{id}")
  getBook(id: integer): Book;

  @post("/books")
  createBook(book: Book): Book;

  @put("/books/{id}")
  updateBook(id: integer, book: Book): Book;

  @delete("/books/{id}")
  deleteBook(id: integer): void;
}

The BooksService contains four methods for performing actions on books:

• @get("/books/{id}"): Method for retrieving a book by its id.

• @post("/books"): Method for creating a new book.

• @put("/books/{id}"): Method for updating an existing book by its id.

• @delete("/books/{id}"): Method for deleting a book based on its id.

These methods use HTTP annotations to indicate the type of operation they perform (GET, POST, PUT, DELETE).

Add Additional Validations for the Book Model

As in the previous example for users, we can add additional validations on Book template properties.

Example of validation on publicationYear and isbn

model Book {
  id: integer;
  @minLength(1)
  title: string;
  @minLength(1)
  author: string;
  @minValue(1000)
  publicationYear: integer;
  @pattern("^\\d{3}-\\d{1,5}-\\d{1,7}-\\d{1,7}-\\d{1}$")
  isbn: string;
}

• @minValue(1000) guarantees that the year of publication is greater than or equal to 1000.

• Validation of the isbn remains the same, using a regular expression to validate a standard ISBN format.

A Complete Service for Managing Books

Now that we have the Book model and the necessary validations, here's a complete service for managing books, with all the essential operations.

Complete BooksService in TypeSpec

model Book {
  id: integer;
  @minLength(1)
  title: string;
  @minLength(1)
  author: string;
  @minValue(1000)
  publicationYear: integer;
  @pattern("^\\d{3}-\\d{1,5}-\\d{1,7}-\\d{1,7}-\\d{1}$")
  isbn: string;
}

service BooksService {
  @get("/books/{id}")
  getBook(id: integer): Book;

  @post("/books")
  createBook(book: Book): Book;

  @put("/books/{id}")
  updateBook(id: integer, book: Book): Book;

  @delete("/books/{id}")
  deleteBook(id: integer): void;
}

• The Book model defines properties and validations for a book.

• The BooksService provides endpoints for retrieving, creating, updating, and deleting a book.

• Each service method is correctly annotated with the corresponding HTTP verbs (GET, POST, PUT, DELETE).

And here’s a summary of everything we’ve done:

• We created a Book model with properties such as title, author, year of publication, and ISBN number.

• We defined a BooksService to provide CRUD operations on books.

• We added validations to ensure that the data respected specified constraints (for example, ISBN and year of publication).

• We designed a complete REST API to manage books with TypeSpec, using a minimum amount of code and staying true to standards.

This example shows just how quickly and efficiently TypeSpec can be used to model a REST API, while ensuring a clear structure and robust validations.

How to Build the API in Express and ASP.NET Core

Now that we've defined a book management REST service with TypeSpec, let's see how we'd implement this same API using two popular frameworks:

• ExpressJS (Node.js / TypeScript)

• ASP.NET Core (C#)

This will allow us to better compare TypeSpec's conciseness and readability with traditional implementations.

Manual implementation with ExpressJS (Node.js / TypeScript):

//server.ts
import express from 'express';

const app = express();
app.use(express.json());

interface Book {
  id: number;
  title: string;
  author: string;
  publicationYear: number;
  isbn: string;
}

const books: Book[] = [];

// GET /books/:id
app.get('/books/:id', (req, res) => {
  const id = parseInt(req.params.id);
  const book = books.find(b => b.id === id);
  if (!book) return res.status(404).send({ message: 'Book not found' });
  res.send(book);
});

// POST /books
app.post('/books', (req, res) => {
  const newBook: Book = req.body;
  books.push(newBook);
  res.status(201).send(newBook);
});

// PUT /books/:id
app.put('/books/:id', (req, res) => {
  const id = parseInt(req.params.id);
  const index = books.findIndex(b => b.id === id);
  if (index === -1) return res.status(404).send({ message: 'Book not found' });

  books[index] = req.body;
  res.send(books[index]);
});

// DELETE /books/:id
app.delete('/books/:id', (req, res) => {
  const id = parseInt(req.params.id);
  const index = books.findIndex(b => b.id === id);
  if (index === -1) return res.status(404).send({ message: 'Book not found' });

  books.splice(index, 1);
  res.status(204).send();
});

app.listen(3000, () => {
  console.log('Server is running on port 3000');
});

Observations:

• A lot of repetitive logic.

• No automatic validation.

• Routes must be maintained manually.

• No automatically generated API documentation.

Manual implementation with ASP.NET Core (C#):

// Book.cs
public class Book
{
    public int Id { get; set; }

    [Required]
    public string Title { get; set; } = string.Empty;

    [Required]
    public string Author { get; set; } = string.Empty;

    [Range(1000, int.MaxValue)]
    public int PublicationYear { get; set; }

    [RegularExpression(@"^\d{3}-\d{1,5}-\d{1,7}-\d{1,7}-\d{1}$")]
    public string Isbn { get; set; } = string.Empty;
}

// BooksController.cs
[ApiController]
[Route("books")]
public class BooksController : ControllerBase
{
    private static readonly List<Book> books = new();

    [HttpGet("{id}")]
    public IActionResult GetBook(int id)
    {
        var book = books.FirstOrDefault(b => b.Id == id);
        if (book == null) return NotFound("Book not found");
        return Ok(book);
    }

    [HttpPost]
    public IActionResult CreateBook([FromBody] Book book)
    {
        books.Add(book);
        return CreatedAtAction(nameof(GetBook), new { id = book.Id }, book);
    }

    [HttpPut("{id}")]
    public IActionResult UpdateBook(int id, [FromBody] Book updatedBook)
    {
        var index = books.FindIndex(b => b.Id == id);
        if (index == -1) return NotFound("Book not found");

        books[index] = updatedBook;
        return Ok(updatedBook);
    }

    [HttpDelete("{id}")]
    public IActionResult DeleteBook(int id)
    {
        var book = books.FirstOrDefault(b => b.Id == id);
        if (book == null) return NotFound("Book not found");

        books.Remove(book);
        return NoContent();
    }
}

Observations:

• More formal and structured than Express, thanks to C# annotations ([HttpPost], [Required], and so on).

• Validation is handled automatically via Data Annotations.

• Once again, no automatic OpenAPI generation or SDK client without additional configuration.

Comparison with TypeSpec:

Best Practices for Structuring TypeSpec Projects and Components

When you start writing API definitions in TypeSpec, it's easy to put everything in a single file. But as with any software project, as the application grows, a good structure becomes essential to guarantee the readability, reusability and maintainability of the code.

Here's a set of best practices I strongly recommend:

Organize by Functional Area

Use namespaces to group models, interfaces, and operations by business domain: book, user, auth, payment, and so on.

namespace MyApi.Books;

Create a /books folder with the following files:

src/
├── books/
│   ├── models.tsp
│   ├── routes.tsp
│   └── service.tsp

This ensures a clear separation of responsibilities, just like in a well-structured Node.js project.

A Single main.tsp Entry Point

This is the main file that orchestrates:

// main.tsp
import "./books/service.tsp";
import "./users/service.tsp";
import "./auth/service.tsp";

This allows you to compile the entire project from a single point.

Create Reusable Components

Define common models and types in a shared file. Example: