In this guide, you’ll learn how to set up the Langbase Docs MCP server inside Cursor (an AI code editor), and build a summary AI agent that uses Langbase docs as live, on-demand context.

Here’s what we’ll cover:

• Prerequisites

• What is Model Context Protocol (MCP)?

• Anthropic’s role in launching MCP

• Cursor AI code editor

• What is Langbase and why is its Docs MCP server useful?

• How to set up the Langbase Docs MCP server in Cursor?

• How to use Langbase Docs MCP server in Cursor AI?

• Use case: Build a summary AI Agent with Langbase Docs MCP server

Prerequisites

Before we begin creating the agent, you’ll need to have some things setup and some tools ready to go.

In this tutorial, I’ll be using the following tech stack:

• Langbase – the platform to build and deploy your serverless AI agents.

• Langbase SDK – a TypeScript AI SDK, designed to work with JavaScript, TypeScript, Node.js, Next.js, React, and the like.

• Cursor – An AI code editor just like VS Code.

You’ll also need to:

• Sign up on Langbase to get access to the API key.

What is Model Context Protocol (MCP)?

Model Context Protocol (MCP) is an open protocol that standardizes how applications provide external context to large language models (LLMs). With MCP, developers can connect AI models to various tools and data sources like documentation, APIs, and databases – in a clean, consistent way.

Instead of relying solely on prompts, MCP allows LLMs to call custom tools (like documentation fetchers or API explorers) during a conversation.

MCP General Architecture

At its core, MCP follows a client-server architecture where a host application can connect to multiple servers.

Here’s the general architecture of what it looks like:

The Model Context Protocol architecture lets AI clients (like Claude, IDEs, and developer tools) securely connect to multiple local or remote data sources in real time. MCP clients communicate with one or more MCP servers, which act as bridges to structured data – whether from local files, databases, or remote APIs.

This setup allows AI models to retrieve fresh, relevant context from different sources seamlessly, without embedding data directly into the model.

Anthropic’s Role in Launching MCP

Anthropic introduced MCP as part of their vision to make LLMs tool-augmented by default. MCP was originally built to expand Claude’s capabilities, but it's now available more broadly and supported in developer-friendly environments like Cursor and Claude Desktop.

By standardizing how tools integrate into LLM workflows, MCP makes it easier for developers to extend AI systems without custom plugins or API hacks.

Cursor AI Code Editor

Cursor is a developer-first AI code editor that integrates LLMs (like Claude, GPT, and more) directly into your IDE. Cursor supports MCP, meaning you can quickly attach custom tool servers and make them accessible as AI-augmented tools while you code.

Think of Cursor as VS Code meets AI agents – with built-in support for smart tools like docs fetchers and code examples retrievers.

What is Langbase and Why is its Docs MCP Server Useful?

Langbase is a powerful serverless AI platform for building AI agents with memory. It helps developers build AI-powered apps and assistants by connecting LLMs directly to their data, APIs, and documentation.

The Langbase Docs MCP Server provides access to the Langbase documentation and API reference. This server allows you to use the Langbase documentation as context for your LLMs.

By connecting this server to Cursor (or any MCP-supported IDE), you can make Langbase documentation available to your AI agents on demand. This means less context-switching, faster workflows, and smarter assistance when building serverless agentic applications.

How to Set Up the Langbase Docs MCP Server in Cursor

Let’s walk through setting up the server step-by-step.

1. Open Cursor Settings

Launch Cursor and open Settings. From the left sidebar, select MCP.

2. Add a New MCP Server

Click the yellow + Add new global MCP server button.

3. Configure the Langbase Docs MCP Server

Paste the following configuration into the mcp.json file:

{
    "mcpServers": {
        "Langbase": {
        "command": "npx",
        "args": ["@langbase/cli","docs-mcp-server"]
        }
    }
}

4. Start the Langbase Docs MCP Server

In your terminal, run:

pnpm add @langbase/cli

And then run this command:

pnpm dlx @langbase/cli docs-mcp-server

5. Enable the MCP Server in Cursor

In the MCP settings, make sure the Langbase server is toggled to Enabled.

How to Use Langbase Docs MCP Server in Cursor AI

Once everything’s all set up, Cursor’s AI agent can now call Langbase docs tools like:

• docs_route_finder

• sdk_documentation_fetcher

• examples_tool

• guide_tool

• api_reference_tool

For example, you can ask the Cursor agent:

“Show me the API reference for Langbase Memory”
 or
 “Find a code example of creating an AI agent pipe in Langbase”

The AI will use the Docs MCP server to fetch precise documentation snippets – directly inside Cursor.

Use Case: Build a Summary AI Agent with Langbase Docs MCP Server

Let’s build a summary agent that summarizes context using the Langbase SDK, powered by the Langbase Docs MCP server inside the Cursor AI code editor.

1. Open an empty folder in Cursor and launch the chat panel (Cmd+Shift+I on Mac or Ctrl+Shift+I on Windows).

2. Switch to Agent mode from the mode selector and pick your preferred LLM (we’ll use Claude 3.5 Sonnet for this demo).

3. In the chat input, enter the following prompt:
   “In this directory, using Langbase SDK, create the summary pipe agent. Use TypeScript and pnpm to run the agent in the terminal.“

4. Cursor will automatically invoke MCP calls, generate the required files and code using Langbase Docs as context, and suggest changes. Accept the changes, and your summary agent will be ready. You can run the agent using the commands provided by Cursor and view the results.

Here’s a demo video of creating this summary agent with a single prompt and Langbase Docs MCP server:

By combining Langbase’s Docs MCP server with Cursor AI, you’ve learned how to build serverless AI agents in minutes – all without leaving your IDE.

If you’re building AI agents, tools, or apps with Langbase, this is one of the fastest ways to simplify your development process.

Happy building! 🚀

Connect with me by 🙌:

• Subscribing to my YouTube Channel. If you are willing to learn about AI and agents.

• Subscribing to my free newsletter “The Agentic Engineer” where I share all the latest AI and agents news/trends/jobs and much more.

• Follow me on X (Twitter).

You can build powerful agentic systems without heavyweight frameworks or orchestration engines. One of the simplest and most effective ways to do that is to use Langbase agentic architectures (built with AI primitives that don't require a framework to ship scalable AI agentic systems).

In this article, we’ll dive into one of Langbase's agentic architectures: prompt chaining. We’ll look at why it’s useful and how to implement it by building a prompt chaining agent.

Table of Contents

1. Prerequisites

2. AI primitives (agentic architecture)

3. What is prompt chaining?

4. Prompt chaining architecture

5. Langbase SDK

6. Building a prompt chaining agent using Langbase Pipes

   • Step 1: Setup your project

   • Step 2: Get Langbase API Key

   • Step 3: Add LLM API keys

   • Step 4: Add logic in prompt-chaining.ts file

   • Step 5: Run the file

7. The result

Prerequisites

Before we begin creating a prompt chaining agent, you’ll need to have the following setup and tools ready to go.

In this tutorial, I’ll be using the following tech stack:

• Langbase – the platform to build and deploy your serverless AI agents.

• Langbase SDK – a TypeScript AI SDK, designed to work with JavaScript, TypeScript, Node.js, Next.js, React, and the like.

• OpenAI – to get the LLM key for the preferred model.

You’ll also need to:

• Sign up on Langbase to get access to the API key.

• Sign up on OpenAI to generate the LLM key for the model you want to use (for this demo, I’ll be using the openai:gpt-4o-mini model). You can generate the key here.

AI Primitives (Agentic Architecture)

An AI primitive level approach means building AI systems using the most basic building blocks – without relying on heavy abstractions, orchestration engines, or full-blown frameworks.

Langbase Pipe and Memory agents serve as these building blocks.

Pipe agents on Langbase are different from other agents. They are serverless AI agents with agentic tools that can work with any language or framework. Pipe agents are easily deployable, and with just one API they let you connect 250+ LLMs to any data to build any developer API workflow.

Langbase memory agents (long-term memory solution) are designed to acquire, process, retain, and retrieve information seamlessly. They dynamically attach private data to any LLM, enabling context-aware responses in real time and reducing hallucinations. Memory, when connected to a pipe agent, becomes a memory agent.

With these building blocks (AI primitives) you can build entire agentic workflows. For this, Langbase agentic architectures serves as a boilerplate in building, deploying, and scaling autonomous agents.

Let’s look at one of the agentic architectures: prompt chaining.

What is Prompt Chaining?

Prompt chaining is an agent architecture where a task is broken down into a sequence of prompts. Each step passes its output to the next, enabling the LLM to handle more complex workflows with higher accuracy.

This is particularly useful for structured tasks like:

• Document summarization and analysis

• Multi-step content generation

• Data transformation and cleanup

• Content validation and refinement

Rather than relying on a single prompt to do everything, you split the work into focused steps. This makes it easier to debug, improves output quality, and introduces natural "checkpoints" in your AI workflow.

Prompt Chaining Architecture

Here’s a reference architecture explaining the workflow:

This diagram is a visual reference for how prompt chaining can be used to build a lightweight agentic system using just LLM calls and conditional logic – without any heavyweight frameworks.

Here’s a breakdown of what’s happening in the flow:

1. In → LLM Call

• Takes the initial input and runs the first LLM call.

• Produces Output 1.

2. Gate

• Evaluates Output 1 to decide the next step.

• Acts as a conditional checkpoint (for example, success/failure, intent validation, confidence threshold).

3. If Gate passes:

• Proceeds to LLM Call 2 with Output 1 as input.

• LLM Call 2 produces Output 2.

• Output 2 goes into LLM Call 3, which generates the final result.

• Final output flows into the Out.

4. If Gate fails:

• The flow terminates early at Exit.

• Skips further LLM calls, saving compute and avoiding invalid outputs.

Langbase SDK

The Langbase SDK makes it easy to build powerful AI agents using TypeScript. It gives you everything you need to work with any LLM, connect your own embedding models, manage document memory, and build AI agents that can reason and respond.

The SDK is designed to work with Node.js, Next.js, React, or any modern JavaScript stack. You can use it to upload documents, create semantic memory, and run AI workflows (called Pipes agents) with just a few lines of code.

Langbase is an API-first AI platform, and its TypeScript SDK smooths out the experience – making it easy to get started without dealing with infrastructure. Just drop in your API key, write your logic, and you're good to go.

Now that you know about Langbase SDK, let’s start building the prompt chaining agent.

Building a Prompt Chaining Agent using Langbase Pipes

Let’s walk through a real prompt chaining agentic system built using Langbase Pipe agents (serverless AI agents with unified APIs for every LLM). For this, we’ll be setting up a basic Node.js project.

We’ll be implementing a sequential product marketing content pipeline that transforms a raw product description into polished marketing copy through three stages (that is, the creation of three Pipe agents):

First Stage (Summary Agent):

• Takes a raw product description

• Condenses it into two concise sentences

• Has a quality gate that checks if the summary is detailed enough (at least 10 words)

Second Stage (Features Agent):

• Takes the summary from stage 1

• Extracts and formats key product features as bullet points

Final Stage (Marketing Copy Agent):

• Takes the bullet points from stage 2

• Generates refined marketing copy for the product

All stages will be using the OpenAI 4o-mini model through the Langbase SDK. The best part is that you can use different LLM models for each stage/Pipe agent creation as well.

What makes this interesting is its pipeline approach. Each stage builds upon the output of the previous stage, with a quality check after the summary stage to ensure the pipeline maintains high standards.

Let’s begin with the creation of this prompt chaining agentic system.

Step 1: Setup Your Project

I’ll be building a basic Node.js app in TypeScript that uses the Langbase SDK to create a scalable prompt chaining agentic system. It will work without any framework, following an AI primitive level approach.

To get started with that, create a new directory for your project and navigate to it:

mkdir agentic-architecture && cd agentic-architecture

Then initialize a Node.js project and create a TypeScript file by running this command in your terminal:

npm init -y && touch prompt-chaining.ts

The prompt-chaining.ts file will contain code of all the agent creations in it.

After this, we will be using the Langbase SDK to create the agents and dotenv to manage environment variables. So, let's install these dependencies.

npm i langbase dotenv

Step 2: Get Langbase API Key

Every request you send to Langbase needs an API key. You can generate API keys from the Langbase studio by following these steps:

1. Switch to your user or org account.

2. From the sidebar, click on the Settings menu.

3. In the developer settings section, click on the Langbase API keys link.

4. From here you can create a new API key or manage existing ones.

For more details, you can visit the Langbase API keys documentation.

After generating the API key, create an .env file in the root of your project and add your Langbase API key in it:

LANGBASE_API_KEY=xxxxxxxxx

Replace xxxxxxxxx with your Langbase API key.

Step 3: Add LLM API keys

Once you have the Langbase API key, you’ll be needing the LLM key as well to run the RAG agent. If you have set up LLM API keys in your profile, the AI memory and agent pipe will automatically use them. Otherwise navigate to the LLM API keys page and add keys for different providers like OpenAI, Anthropic, and so on.

Follow these steps to add the LLM keys:

1. Add LLM API keys in your account using Langbase studio

2. Switch to your user or org account.

3. From the sidebar, click on the Settings menu.

4. In the developer settings section, click on the LLM API keys link.

5. From here you can add LLM API keys for different providers like OpenAI, TogetherAI, Anthropic, and so on.

Step 4: Add logic in prompt-chaining.ts file

In the prompt-chaining.ts file you created in Step 1, add the following code:

import dotenv from 'dotenv';
import { Langbase } from 'langbase';


dotenv.config();


const langbase = new Langbase({
   apiKey: process.env.LANGBASE_API_KEY!
});


async function main(inputText: string) {
   // Prompt chaining steps
   const steps = [
       {
           name: `summary-agent-${Date.now()}`,
           model: 'openai:gpt-4o-mini',
           description:
               'summarize the product description into two concise sentences',
           prompt: `Please summarize the following product description into two concise
           sentences:\n`
       },
       {
           name: `features-agent-${Date.now()}`,
           model: 'openai:gpt-4o-mini',
           description: 'extract key product features as bullet points',
           prompt: `Based on the following summary, list the key product features as
           bullet points:\n`
       },
       {
           name: `marketing-copy-agent-${Date.now()}`,
           model: 'openai:gpt-4o-mini',
           description:
               'generate a polished marketing copy using the bullet points',
           prompt: `Using the following bullet points of product features, generate a
           compelling and refined marketing copy for the product, be precise:\n`
       }
   ];


   //  Create the pipe agents
   await Promise.all(
       steps.map(step =>
           langbase.pipes.create({
               name: step.name,
               model: step.model,
               messages: [
                   {
                       role: 'system',
                       content: `You are a helpful assistant that can ${step.description}.`
                   }
               ]
           })
       )
   );


   // Initialize the data with the raw input.
   let data = inputText;


   try {
       // Process each step in the workflow sequentially.
       for (const step of steps) {
           // Call the LLM for the current step.
           const response = await langbase.pipes.run({
               stream: false,
               name: step.name,
               messages: [{ role: 'user', content: `${step.prompt} ${data}` }]
           });


           data = response.completion;


           console.log(`Step: ${step.name} \n\n Response: ${data}`);


           // Gate on summary agent output to ensure it is not too brief.
           // If summary is less than 10 words, throw an error to stop the workflow.
           if (step.name === 'summary-agent' && data.split(' ').length < 10) {
               throw new Error(
                   'Gate triggered for summary agent. Summary is too brief. Exiting workflow.'
               );
               return;
           }
       }
   } catch (error) {
       console.error('Error in main workflow:', error);
   }


   // The final refined marketing copy
   console.log('Final Refined Product Marketing Copy:', data);
}


const inputText = `Our new smartwatch is a versatile device featuring a high-resolution display,
long-lasting battery life,fitness tracking, and smartphone connectivity. It's designed for
everyday use and is water-resistant. With cutting-edge sensors and a sleek design, it's
perfect for tech-savvy individuals.`;


main(inputText);

Here’s a breakdown of the above code:

Setup and initialization:

• dotenv loads env variables from the .env file for secure API key access.

• Langbase is imported from the SDK to interact with the API.

• A Langbase client instance is created using your API key.

Define the AI steps (prompt chain):

• Three AI agents (steps) are defined for a pipeline:

  1. Summarization Agent: Summarizes the input product description into 2 sentences.

  2. Feature Extraction Agent: Extracts key features from the summary as bullet points.

  3. Marketing Copy Agent: Turns bullet points into polished marketing copy.

• Each agent uses openai:gpt-4o-mini as the LLM.

Create Langbase Pipes (agents):

• Langbase pipes are created for each step using langbase.pipes.create(...).

• Each pipe has a unique name (timestamped) and a system message guiding its purpose.

Run the workflow (sequential processing):

• Input text flows through each step one by one:

  • The output of one step becomes the input for the next.

  • Pipes are run using langbase.pipes.run(...).

• Intermediate outputs are logged after each step.

Validation check (gatekeeping):

• If the summary output is too short (less than 10 words), the workflow stops with an error.

Final Output:

• After all steps, the final result is a refined marketing copy printed to the console.

For this article, we’re using a demo smartwatch product description to view the result in the inputText field.

Step 5: Run the file

To run the prompt-chaining.ts file to view the results, you need to:

• Add TypeScript as a dependency

• Add a script to run TypeScript files

• Add a TypeScript configuration file

For it lets first install pnpm by running this command in your terminal:

pnpm install

Then in your terminal again, run this command to add relevant dependencies and configuration files:

pnpm add -D typescript ts-node @types/node

After that, create a TypeScript configuration file tsconfig.json:

pnpm exec tsc --init

And update the package.json to add the relevant script. This is what your package.json should look like after updating:

{
 "name": "agentic-architectures",
 "version": "1.0.0",
 "main": "index.js",
 "scripts": {
   "test": "echo \"Error: no test specified\" && exit 1",
   "prompt-chaining": "ts-node prompt-chaining.ts"
 },
 "keywords": [],
 "author": "",
 "license": "ISC",
 "description": "",
 "dependencies": {
   "dotenv": "^16.5.0",
   "langbase": "^1.1.55"
 },
 "devDependencies": {
   "@types/node": "^22.14.1",
   "ts-node": "^10.9.2",
   "typescript": "^5.8.3"
 }
}

Now let’s run the project by pnpm run prompt-chaining

The Result

After running the project, you’ll see the result of the example smartwatch product description in your console as follows:

Step: summarize-description
Response: This smartwatch combines fitness tracking and smartphone connectivity with a high-resolution display and long-lasting battery. Designed for everyday use with a sleek, water-resistant build, it's ideal for tech enthusiasts.

Step: extract-features
Response: Okay, here are the key product features extracted from the summary:

Fitness Tracking
Smartphone Connectivity
High-Resolution Display
Long-Lasting Battery
Sleek Design
Water-Resistant Build
Designed for Everyday Use
Step: refine-marketing-copy
Response: ## Elevate Your Everyday with Seamless Connectivity and Unrivaled Performance.

Experience the perfect fusion of style and functionality with our revolutionary device, designed to seamlessly integrate into your active lifestyle. Stay motivated and informed with comprehensive Fitness Tracking, while effortlessly staying connected via Smartphone Connectivity.

Immerse yourself in vibrant clarity with the stunning High-Resolution Display, and power through your day without interruption thanks to the Long-Lasting Battery. Encased in a Sleek Design, this device is as stylish as it is practical.

Built to withstand the rigors of daily life, the Water-Resistant Build ensures worry-free wear, rain or shine. Engineered for comfort and performance, this device is Designed for Everyday Use, empowering you to live your best life, effortlessly.

This is how you can build a prompt chaining agentic system with AI primitives (no framework) using the Langbase SDK and Langbase agentic architectures.

Thank you for reading!

Connect with me by 🙌:

• Subscribing to my YouTube Channel. If you are willing to learn about AI and agents.

• Subscribing to my free newsletter “The Agentic Engineer” where I share all the latest AI and agents news/trends/jobs and much more.

• Follow me on X (Twitter).

In this guide, you'll build a RAG-based AI agent in TypeScript using Langbase SDK. You'll plug in your own data as memory, use any embedding model, retrieve relevant context, and call an LLM to generate a precise response.

By the end of this tutorial, you'll have a working RAG system that:

• Stores and retrieves documents with semantic memory

• Uses custom embeddings for vector search

• Handles user queries with relevant context

• Generates responses via OpenAI, Anthropic, or any LLM

Table of Contents

• Prerequisites

• What is agentic RAG?

• Langbase SDK

• Step 1: Setup your project

• Step 2: Get Langbase API Key

• Step 3: Add LLM API keys

• Step 4: Create an agentic AI memory

• Step 5: Add documents to AI memory

• Step 6: Perform RAG retrieval

• Step 7: Create support pipe agent

• Step 8: Generate RAG responses

• The result

Prerequisites

Before we begin creating a RAG-based AI agent, you’ll need to have some tools ready to go.

In this tutorial, I’ll be using the following tech stack:

• Langbase – the platform to build and deploy your serverless AI agents.

• Langbase SDK – a TypeScript AI SDK, designed to work with JavaScript, TypeScript, Node.js, Next.js, React, and the like.

• OpenAI – to get the LLM key for the preferred model.

You’ll also need to:

• Sign up on Langbase to get access to the API key.

• Sign up on OpenAI to generate the LLM key for the model you want to use (for this demo, I’ll be using the openai:text-embedding-3-large model). You can generate the key here.

What is Agentic RAG?

Retrieval augmented generation (RAG) is an architecture for optimizing the performance of an artificial intelligence (AI) model by connecting it with external knowledge bases. RAG helps large language models (LLMs) deliver more relevant responses at a higher quality.

When we use AI agents to facilitate RAG, it becomes Agentic RAG. Agentic RAG systems add AI agents to the RAG pipeline to increase adaptability and accuracy. Compared to traditional RAG systems, agentic RAG allows LLMs to conduct information retrieval from multiple sources and handle more complex workflows.

Here’s the tabular comparison of RAG vs Agentic RAG:

Langbase SDK

The Langbase SDK makes it easy to build powerful AI tools using TypeScript. It gives you everything you need to work with any LLM, connect your own embedding models, manage document memory, and build AI agents that can reason and respond.

The SDK is designed to work with Node.js, Next.js, React, or any modern JavaScript stack. You can use it to upload documents, create semantic memory, and run AI workflows (called Pipe agents) with just a few lines of code.

Langbase is an API-first AI platform. Its TypeScript SDK smooths out the experience, making it easy to get started without dealing with infrastructure. Just drop in your API key, write your logic, and you're good to go.

Now that you know about Langbase SDK, let’s start building the RAG agent.

Step 1: Setup Your Project

We’ll be building a basic Node.js app in TypeScript that uses the Langbase SDK to create an agentic RAG system. For that, create a new directory for your project and navigate to it.

mkdir agentic-rag && cd agentic-rag

Then initialize a Node.js project and create different TypeScript files by running this command in your terminal:

npm init -y && touch index.ts agents.ts create-memory.ts upload-docs.ts create-pipe.ts

Here’s a breakdown of what each file will do in the project:

• index.ts: This is typically the entry point of a TypeScript project. It orchestrates agent creation, memory setup, and document upload.

• agents.ts: This file handles AI agent creation and configuration.

• create-memory.ts: This sets up Langbase Memory (RAG) for storing and retrieving context.

• upload-docs.ts: This file will upload documents to Memory so agents can access and use them.

• create-pipe.ts: This file sets up a Langbase Pipe agent which is a serverless AI agent with unified APIs for every LLM.

After this, we will be using the Langbase SDK to create RAG agents and dotenv to manage environment variables. So, let's install these dependencies.

npm i langbase dotenv

Step 2: Get Langbase API Key

Every request you send to Langbase needs an API key. You can generate API keys from the Langbase studio by following these steps:

1. Switch to your user or org account.

2. From the sidebar, click on the Settings menu.

3. In the developer settings section, click on the Langbase API keys link.

4. From here you can create a new API key or manage existing ones.

For more details, check out the Langbase API keys documentation.

After generating the API key, create an .env file in the root of your project and add your Langbase API key in it:

LANGBASE_API_KEY=xxxxxxxxx

Replace xxxxxxxxx with your Langbase API key.

Step 3: Add LLM API Keys

Once you have the Langbase API key, you’ll need the LLM key as well to run the RAG agent. If you have set up LLM API keys in your profile, the AI memory and agent pipe will automatically use them. Otherwise, navigate to the LLM API keys page and add keys for different providers like OpenAI, Anthropic, and so on.

Follow these steps to add the LLM keys:

1. Add LLM API keys in your account using Langbase studio

2. Switch to your user or org account.

3. From the sidebar, click on the Settings menu.

4. In the developer settings section, click on the LLM API keys link.

5. From here you can add LLM API keys for different providers like OpenAI, TogetherAI, Anthropic, and so on.

Step 4: Create an Agentic AI Memory

Let’s now use the Langbase SDK to create an AI memory (Langbase memory agent) where your agent can store and retrieve context.

Langbase serverless memory agents (long-term memory solution) are designed to acquire, process, retain, and retrieve information seamlessly. They dynamically attach private data to any LLM, enabling context-aware responses in real time and reducing hallucinations.

These agents combine vector storage, RAG, and internet access to create a powerful managed context search API. You can use them to build smarter, more capable AI applications.

In a RAG setup, memory – when connected directly to a Langbase Pipe Agent – becomes a memory agent. This pairing gives the LLM the ability to fetch relevant data and deliver precise, contextually accurate answers – addressing the limitations of LLMs when it comes to handling private data.

To create it, add the following code to create-memory.ts file you created in Step 1:

import 'dotenv/config';
import {Langbase} from 'langbase';

const langbase = new Langbase({
    apiKey: process.env.LANGBASE_API_KEY!,
});

async function main() {
    const memory = await langbase.memories.create({
        name: 'knowledge-base',
        description: 'An AI memory for agentic memory workshop',
        embedding_model: 'openai:text-embedding-3-large'
    });

    console.log('AI Memory:', memory);
}

main();

Here’s what’s happening in the above code:

• Import the dotenv package to load environment variables.

• Import the Langbase class from the langbase package.

• Create a new instance of the Langbase class with your API key.

• Use the memories.create method to create a new AI memory.

• Set the name and description of the memory.

• Use the openai:text-embedding-3-large model for embedding.

• Log the created memory to the console.

After this, let's create the agentic memory by running the create-memory.ts file.

npx tsx create-memory.ts

This will create an AI memory and log the memory details to the console.

Step 5: Add Documents to AI Memory

Now that you’ve created an AI memory agent, the next step is to add documents in it. These documents will serve as the context your agent can reference during interactions.

First, create a docs directory in your project root, and add two sample text files:

• agent-architectures.txt

• langbase-faq.txt

Next, open the upload-docs.ts file created in Step 1 and paste in the following code:

import 'dotenv/config';
import { Langbase } from 'langbase';
import { readFile } from 'fs/promises';
import path from 'path';

const langbase = new Langbase({
    apiKey: process.env.LANGBASE_API_KEY!,
});

async function main() {
    const cwd = process.cwd();
    const memoryName = 'knowledge-base';

    // Upload agent architecture document
    const agentArchitecture = await readFile(path.join(cwd, 'docs', 'agent-architectures.txt'));
    const agentResult = await langbase.memories.documents.upload({
        memoryName,
        contentType: 'text/plain',
        documentName: 'agent-architectures.txt',
        document: agentArchitecture,
        meta: { category: 'Examples', topic: 'Agent architecture' },
    });

    console.log(agentResult.ok ? '✓ Agent doc uploaded' : '✗ Agent doc failed');

    // Upload FAQ document
    const langbaseFaq = await readFile(path.join(cwd, 'docs', 'langbase-faq.txt'));
    const faqResult = await langbase.memories.documents.upload({
        memoryName,
        contentType: 'text/plain',
        documentName: 'langbase-faq.txt',
        document: langbaseFaq,
        meta: { category: 'Support', topic: 'Langbase FAQs' },
    });

    console.log(faqResult.ok ? '✓ FAQ doc uploaded' : '✗ FAQ doc failed');
}

main();

Let’s break down what’s happening in this code:

• dotenv/config is used to load environment variables from your .env file.

• Langbase is imported from the SDK to interact with the API.

• readFile from the fs/promises module reads each document file asynchronously.

• path.join() ensures file paths work across different operating systems.

• A Langbase client instance is created using your API key.

• memories.documents.upload is used to upload each .txt file to the AI memory.

• Each upload includes metadata like category and topic to help organize the content.

• Upload success or failure is logged to the console.

This step ensures your AI agent will have actual content to pull from – FAQs, architecture docs, or anything else you upload into memory.

Then, run the upload-docs.ts file to upload the documents to the AI memory by this command in your terminal. This will upload the documents to the AI memory:

npx tsx upload-docs.ts

Step 6: Perform RAG Retrieval

In this step, we’ll perform RAG against a query using the documents we uploaded to AI memory.

Add the following code to your agents.ts file created in Step 1:

import 'dotenv/config';
import { Langbase } from 'langbase';

const langbase = new Langbase({
    apiKey: process.env.LANGBASE_API_KEY!,
});

export async function runMemoryAgent(query: string) {
    const chunks = await langbase.memories.retrieve({
        query,
        topK: 4,
        memory: [
            {
                name: 'knowledge-base',
            },
        ],
    });

    return chunks;
}

Let’s break down what this does:

• Import the Langbase class from the Langbase SDK.

• Initialize the Langbase client using your API key from environment variables.

• Define an async function runMemoryAgent that takes a query string as input.

• Use memories.retrieve to query the memory for the most relevant chunks, retrieving the top 4 results (topK: 4) from the memory named "knowledge-base".

• Return the retrieved memory chunks.

Now let's add the following code to the index.ts file created in Step 1 to run the memory agent:

import { runMemoryAgent } from './agents';

async function main() {
    const chunks = await runMemoryAgent('What is agent parallelization?');
    console.log('Memory chunk:', chunks);
}

main();

This code runs a Langbase memory query for “What is agent parallelization?” It uses runMemoryAgent to retrieve the top matching chunks from your AI memory and logs the results. It’s how you fetch relevant knowledge with RAG.

After this, run the index.ts file to perform RAG retrieval against the query by this command in terminal:

npx tsx index.ts

You will see the memory agent output as retrieved memory chunks in the console as follows:

[
  {
    text: '---\n' +
      '\n' +
      '## Agent Parallelization\n' +
      '\n' +
      'Parallelization runs multiple LLM tasks at the same time to improve speed or accuracy. It works by splitting a task into independent parts (sectioning) or generating multiple responses for comparison (voting).\n' +
      '\n' +
      'Voting is a parallelization method where multiple LLM calls generate different responses for the same task. The best result is selected based on agreement, predefined rules, or quality evaluation, improving accuracy and reliability.\n' +
      '\n' +
      "`This code implements an email analysis system that processes incoming emails through multiple parallel AI agents to determine if and how they should be handled. Here's the breakdown:",
    similarity: 0.7146744132041931,
    meta: {
      docName: 'agent-architectures.txt',
      documentName: 'agent-architectures.txt',
      category: 'Examples',
      topic: 'Agent architecture'
    }
  },
  {
    text: 'async function main(inputText: string) {\n' +
      '\ttry {\n' +
      '\t\t// Create pipes first\n' +
      '\t\tawait createPipes();\n' +
      '\n' +
      '\t\t// Step A: Determine which agent to route to\n' +
      '\t\tconst route = await routerAgent(inputText);\n' +
      "\t\tconsole.log('Router decision:', route);\n" +
      '\n' +
      '\t\t// Step B: Call the appropriate agent\n' +
      '\t\tconst agent = agentConfigs[route.agent];\n' +
      '\n' +
      '\t\tconst response = await langbase.pipes.run({\n' +
      '\t\t\tstream: false,\n' +
      '\t\t\tname: agent.name,\n' +
      '\t\t\tmessages: [\n' +
      "\t\t\t\t{ role: 'user', content: `${agent.prompt} ${inputText}` }\n" +
      '\t\t\t]\n' +
      '\t\t});\n' +
      '\n' +
      '\t\t// Final output\n' +
      '\t\tconsole.log(\n' +
      '\t\t\t`Agent: ${agent.name} \\n\\n Response: ${response.completion}`\n' +
      '\t\t);\n' +
      '\t} catch (error) {\n' +
      "\t\tconsole.error('Error in main workflow:', error);\n" +
      '\t}\n' +
      '}\n' +
      '\n' +
      '// Example usage:\n' +
      "const inputText = 'Why days are shorter in winter?';\n" +
      '\n' +
      'main(inputText);\n' +
      '```\n' +
      '\n' +
      '\n' +
      '---\n' +
      '\n' +
      '## Agent Parallelization\n' +
      '\n' +
      'Parallelization runs multiple LLM tasks at the same time to improve speed or accuracy. It works by splitting a task into independent parts (sectioning) or generating multiple responses for comparison (voting).',
    similarity: 0.5911030173301697,
    meta: {
      docName: 'agent-architectures.txt',
      documentName: 'agent-architectures.txt',
      category: 'Examples',
      topic: 'Agent architecture'
    }
  },
  {
    text: "`This code implements a sophisticated task orchestration system with dynamic subtask generation and parallel processing. Here's how it works:\n" +
      '\n' +
      '1. Orchestrator Agent (Planning Phase):\n' +
      '   - Takes a complex task as input\n' +
      '   - Analyzes the task and breaks it down into smaller, manageable subtasks\n' +
      '   - Returns both an analysis and a list of subtasks in JSON format\n' +
      '\n' +
      '2. Worker Agents (Execution Phase):\n' +
      '   - Multiple workers run in parallel using Promise.all()\n' +
      '   - Each worker gets:\n' +
      '     - The original task for context\n' +
      '     - Their specific subtask to complete\n' +
      '   - All workers use Gemini 2.0 Flash model\n' +
      '\n' +
      '3. Synthesizer Agent (Integration Phase):\n' +
      '   - Takes all the worker outputs\n' +
      '   - Combines them into a cohesive final result\n' +
      '   - Ensures the pieces flow together naturally',
    similarity: 0.5393730401992798,
    meta: {
      docName: 'agent-architectures.txt',
      documentName: 'agent-architectures.txt',
      category: 'Examples',
      topic: 'Agent architecture'
    }
  },
  {
    text: "`This code implements an email analysis system that processes incoming emails through multiple parallel AI agents to determine if and how they should be handled. Here's the breakdown:\n" +
      '\n' +
      '1. Three Specialized Agents running in parallel:\n' +
      '   - Sentiment Analysis Agent: Determines if the email tone is positive, negative, or neutral\n' +
      '   - Summary Agent: Creates a concise summary of the email content\n' +
      '   - Decision Maker Agent: Takes the outputs from the other agents and decides:\n' +
      '     - If the email needs a response\n' +
      "     - Whether it's spam\n" +
      '     - Priority level (low, medium, high, urgent)\n' +
      '\n' +
      '2. The workflow:\n' +
      '   - Takes an email input\n' +
      '   - Runs sentiment analysis and summary generation in parallel using Promise.all()\n' +
      '   - Feeds those results to the decision maker agent\n' +
      '   - Outputs a final decision object with response requirements\n' +
      '\n' +
      '3. All agents use Gemini 2.0 Flash model and are structured to return parsed JSON responses',
    similarity: 0.49115753173828125,
    meta: {
      docName: 'agent-architectures.txt',
      documentName: 'agent-architectures.txt',
      category: 'Examples',
      topic: 'Agent architecture'
    }
  }
]

Step 7: Create Support Pipe Agent

In this step, we will create a support agent using the Langbase SDK. Go ahead and add the following code to the create-pipe.ts file created in Step 1:

import 'dotenv/config';
import { Langbase } from 'langbase';

const langbase = new Langbase({
    apiKey: process.env.LANGBASE_API_KEY!,
});

async function main() {
    const supportAgent = await langbase.pipes.create({
        name: `ai-support-agent`,
        description: `An AI agent to support users with their queries.`,
        messages: [
            {
                role: `system`,
                content: `You're a helpful AI assistant.
                You will assist users with their queries.
                Always ensure that you provide accurate and to the point information.`,
            },
        ],
    });

    console.log('Support agent:', supportAgent);
}

main();

Let's go through the above code:

• Initialize the Langbase SDK with your API key.

• Use the pipes.create method to create a new pipe agent.

• Log the created pipe agent to the console.

Now run the create-pipe.ts file to create the pipe agent by this command in your terminal:

npx tsx create-pipe.ts

This will create a support agent and log the agent details to the console.

Step 8: Generate RAG Responses

Up until now, we’ve created a Langbase memory agent, added documents in it, performed RAG retrieval against a query, and created a support agent using Langbase Pipe agent. The only thing left in creating this complete RAG agent is generating comprehensive responses using LLMs.

To do this, add the following code to the agents.ts file created in Step 1:

import 'dotenv/config';
import { Langbase, MemoryRetrieveResponse } from 'langbase';

const langbase = new Langbase({
    apiKey: process.env.LANGBASE_API_KEY!,
});

export async function runAiSupportAgent({
    chunks,
    query,
}: {
    chunks: MemoryRetrieveResponse[];
    query: string;
}) {
    const systemPrompt = await getSystemPrompt(chunks);

    const { completion } = await langbase.pipes.run({
        stream: false,
        name: 'ai-support-agent',
        messages: [
            {
                role: 'system',
                content: systemPrompt,
            },
            {
                role: 'user',
                content: query,
            },
        ],
    });

    return completion;
}

async function getSystemPrompt(chunks: MemoryRetrieveResponse[]) {
    let chunksText = '';
    for (const chunk of chunks) {
        chunksText += chunk.text + '\n';
    }

    const systemPrompt = `
    You're a helpful AI assistant.
    You will assist users with their queries.

    Always ensure that you provide accurate and to the point information.
    Below is some CONTEXT for you to answer the questions. ONLY answer from the CONTEXT. CONTEXT consists of multiple information chunks. Each chunk has a source mentioned at the end.

For each piece of response you provide, cite the source in brackets like so: [1].

At the end of the answer, always list each source with its corresponding number and provide the document name. like so [1] Filename.doc. If there is a URL, make it hyperlink on the name.

 If you don't know the answer, say so. Ask for more context if needed.
    ${chunksText}`;

    return systemPrompt;
}

export async function runMemoryAgent(query: string) {
    const chunks = await langbase.memories.retrieve({
        query,
        topK: 4,
        memory: [
            {
                name: 'knowledge-base',
            },
        ],
    });

    return chunks;
}

The above code:

• Creates a function runAiSupportAgent that takes chunks and query as input.

• Uses the pipes.run method to generate responses using the LLM.

• Creates a function getSystemPrompt to generate a system prompt for the LLM.

• Combines the retrieved chunks to create a system prompt.

• Returns the generated completion.

Now, let's run the support agent with AI memory chunks. Add the following code to the index.ts file:

import { runMemoryAgent, runAiSupportAgent } from './agents';

async function main() {
    const query = 'What is agent parallelization?';
    const chunks = await runMemoryAgent(query);

    const completion = await runAiSupportAgent({
        chunks,
        query,
    });

    console.log('Completion:', completion);
}

main();

This code runs two agents: one to retrieve memory chunks relevant to a query (runMemoryAgent), and another (runAiSupportAgent) to generate a final answer using those chunks.

Then, run the index.ts file to generate responses using the LLM.

npx tsx index.ts

The Result

After running the support agent, you’ll see the following output generated in your console:

Completion: Agent parallelization is a process that runs multiple LLM (Language Model) tasks simultaneously to enhance speed or accuracy. This technique can be implemented in two main ways:

1. **Sectioning**: A task is divided into independent parts that can be processed concurrently.
2. **Voting**: Multiple LLM calls generate different responses for the same task, and the best result is selected based on agreement, predefined rules, or quality evaluation. This approach improves accuracy and reliability by comparing various outputs.

In practice, agent parallelization involves orchestrating multiple specialized agents to handle different aspects of a task, allowing for efficient processing and improved outcomes.

If you need more detailed examples or further clarification, feel free to ask!

This is how you can build an agentic RAG system with TypeScript using the Langbase SDK.

Thank you for reading!

Connect with me by 🙌:

• Subscribing to my YouTube Channel if you want to learn about AI and agents.

• Subscribing to my free newsletter The Agentic Engineer where I share all the latest AI and agents news/trends/jobs and much more.

• Follow me on X (Twitter).

In 2025, the difference between AI agents that work and those that don’t comes down to how well they’re prompted. Whether you’re a developer, product manager, or just building with AI, getting really good at prompt engineering will make you significantly more effective.

Langbase lets you craft high-performance prompts and deploy serverless AI agents optimized for the latest models. In this article, we’ll break down tips and tricks to help you design effective prompts. We’ll also look at some advanced prompt engineering techniques for building serverless agents, and how to fine-tune LLM parameters to get the best results.

Prerequisites

To get the most out of this article, you’ll need:

• A Langbase account – Sign up if you haven’t already.

• Basic knowledge of LLMs, AI agents, and RAG (retrieval-augmented generation).

Here’s what I’ll cover:

• Prompt engineering fundamentals

• Tips and tricks for effective prompt design

• Langbase pipe agent prompts

• How to prompt engineer your AI agent

• Fine-tune LLM model’s response parameters

Let’s get started!

Prompt Engineering Fundamentals

A prompt tells the AI what to do—it sets the context, guides the response, and shapes the conversation. Prompt engineering is about designing prompts that make AI agents actually useful in real-world applications.

Here’s how to write good prompts:

1. Define your goal clearly

Before crafting a prompt, be clear about what you want to achieve—just like planning logic before writing code. Consider whether dynamic inputs are needed and how they’ll be handled. Define the ideal output format, whether JSON, XML, or plain text. Determine if the model requires additional context or if its training data is enough.

Set any constraints on response length, structure, or tone. Fine-tune LLM parameters if necessary to improve control. The more precise your goals, the better the results. And remember, effective prompt engineering is often a team effort.

Here’s an example prompt to help you out. If you're building a customer support bot, your goal should look like this:

"Generate concise, polite responses in plain text, pulling details from the company knowledge base."

• Define the output format (JSON, XML, plain text).

  • Example: "Respond in JSON format with 'answer' and 'source' fields."

• Decide if extra context is needed.

  • Example: "Use this document as reference: [URL]."

• Set constraints on length, tone, or structure.

  • Example: "Limit response to 50 words, use a friendly tone."

2. Experiment relentlessly

LLMs aren’t perfect, and neither is prompt engineering. Test everything. Try different formats, tweak parameters, and provide examples. AI models vary in capability—refining prompts through iteration is the only way to ensure reliable outputs.

Suppose your AI isn’t giving useful answers. You could:

• Rephrase the prompt: Instead of "Explain this topic," try "Summarize this in one paragraph with key takeaways."

• Add constraints: "Limit response to three bullet points."

• Give examples: "For example, if asked about Python, reply like this: 'Python is a versatile language used for AI, web development, and automation.'”

3. Treat LLMs like machines, not humans

LLMs don’t think. They follow instructions—precisely. Ambiguity confuses them. Over-explaining can be just as bad as under-explaining. And remember: LLMs will generate an answer, even if it’s wrong. You have to manage this risk.

Here’s a comparison between over and under-explaining prompts:

Over-explaining: "Can you please, if possible, provide a very detailed yet concise explanation about how neural networks work, but not too technical, and try to be engaging, but also keep it short?"
Better prompt: "Explain neural networks in simple terms, under 100 words, with an analogy."

Under-explaining: "Tell me about neural networks."
Better prompt: "Describe neural networks in two sentences with an example."

Tips and Tricks for Effective Prompt Design

Here are a few tips and tricks to help you effectively prompt engineer your AI and agents:

• Be specific – Vague prompts lead to bad outputs. Define the format, tone, and level of detail you want. If needed, break complex tasks into smaller steps and chain your prompts.

• Control response length – If you need a concise response, specify the word or character limit. For example: “Summarize this in 50 words.”

• Provide context – LLMs don’t know everything. If the model needs specific knowledge, include it in your prompt. For dynamic context, use a RAG-based approach to inject relevant information on demand.

• Use step-by-step reasoning – If a task requires logical reasoning, instruct the model explicitly: “Think step by step before answering.” This improves accuracy.

• Separate instructions from context – Long prompts can get messy. Start with clear instructions, then separate additional info.

• Tell it what to do, not what to avoid – Instead of saying, “Don’t explain the answer,” say, “Only output the final answer.” Positive instructions work better.

• Set constraints – Define limits on tone, length, or complexity. Example: “Write in a professional tone, under 3 sentences.”

• Assign a role – LLMs perform better with a defined persona. Start with, “You are an expert in X,” for example, to guide the model’s behavior.

• Use examples – If precision matters, show the model what you expect. Techniques like few-shot and chain-of-thought (CoT) prompting help improve complex reasoning.

Langbase Pipe Agent Prompts

AI agents aren’t just chatbots—they reason, plan, and take action based on user inputs. Unlike simple LLM queries, AI agents operate autonomously, making decisions and interacting with external tools to complete tasks.

Langbase Pipe Agents are serverless AI agents with unified APIs for every LLM. They let developers define structured prompts to control agent behavior across different models. In this section, you'll learn how to structure prompts effectively by creating an AI agent Pipe to get reliable and useful responses.

The three key prompts in Langbase Pipe agents

To make AI agents work effectively, you need three types of prompts:

1. System prompt: Defines the LLM model's role, tone, and guidelines before processing user input.

2. User prompt: The input given by the user to request a response from the model.

3. AI assistant prompt: The model's generated response based on the user’s input.

To learn how to use these three prompts with the UI using Langbase AI studio, you’ll find clear and concise instructions in this guide. It explains exactly where to go/what to click to write these prompts.

Let’s learn how to create an AI agent Pipe using the Langbase AI studio:

Step 1: Create a Pipe agent

After logging into your Langbase account, you can always go to pipe.new to create a new Pipe.

1. Give your Pipe a name. Let’s call it AI support agent.

2. Click on the [Create Pipe] button. And just like that, you have created your first Pipe.

Step 2: Using an LLM model

If you have set up LLM API keys in your profile, the Pipe will automatically use them. If not, just hit the LLM API Keys button or head over to Settings to add Pipe-level LLM API keys.

Let's add an LLM provider API key now.

1. Click on the LLM keys button. It will open a side panel.

2. Select Pipe level keys. Choose any LLM. For example, you can use OpenAI (for GPT) or any other 250+ supported models on Langbase.

3. Click on OpenAI [ADD KEY] button, add your LLM API key. Inside each key modal, you'll find a link Get a new key from here click it to create a new API key on any API provider's website.

Step 3: Build your Pipe: Configure LLM model

Let's start building our pipe. Go back to the Pipe tab and follow these steps:

1. Click on the gpt-4o-mini button to select and configure the LLM model for your Pipe.

2. By default OpenAI gpt-4o-mini is selected. You can also pick any LLM model.

3. Choose one of the pre-configured presets for your model.

4. You can also modify any of the model params. Learn more with the icon, next to param name.

Step 4: Build your Pipe: Configure the Pipe's Meta

Use the Meta section to configure how your AI support agent Pipe should work. There are multiple ways you can configure it.

To start, you can set the output format of the Pipe to JSON. You can also turn on moderation mode to filter out inappropriate content as a requirement by OpenAI.

Then you can turn the streaming mode on and off, and turn off storing messages (input prompt and generated completion) for sensitive data like emails.

Step 5: Design a Prompt

Now that you have your LLM model and Pipe meta configured, it's time to design your prompt.

Prompt: System Instructions

Let's add a system instruction message to this agent. You can add this: You're a helpful AI assistant. You will assist users with their queries about {{company}}. Always ensure that you provide accurate and to the point information.

Prompt: User Message

Now let's add a user message. Click on the USER button to add a new message. You can add this: How to request payment API?.

Prompt: Variables

Any text written between double curly brackets {{}} becomes a variable. A variables section will display all your variable keys and values.

Since you added a variable {{company}}, you can see it appear in variables. Now you’re assessing the company variable value as ACME. This pipe will now replace {{company}} with its value in all messages.

✨ Variables allow you to use the same pipe with different data.

Prompt as Code

We're not writing code here, but if you were to write this prompt as code, it would look like this:

1. Prompt is a messages array. Inside it are message objects.

2. Each message object typically consists of two properties:

   1. role either "system", "user", or "assistant".

   2. content that you're sending or expecting to be generated from the AI LLM.

// Prompt example:
{
    messages: [
        { role: 'system', content: 'You are a helpful assistant.' },
        { role: 'user', content: 'How to request payment API?' },
        { role: 'assistant', content: 'Sure, here you go … …' }
    ];
}

If you’re using the Langbase SDK to build serverless AI Pipe agents to define these three prompts, you need to send the prompt content in the messages object array as follows:

interface Message {
    role: 'user' | 'assistant' | 'system'| 'tool';
    content: string | null;
    name?: string;
    tool_call_id?: string;
    tool_calls?: ToolCall[];
}

You can learn more about creating a pipe agent using the Langbase SDK here.

Now that you know about creating a Pipe agent and its prompts, let's discuss a few effective techniques to prompt engineer your AI Pipe agents that’ll prove useful for a vast majority of LLMs.

How to Prompt Engineer Your AI Agent

1. Few-shot training

Few-shot prompting improves an AI agent's ability to generate accurate responses by providing it with a few examples before asking it to perform a task. Instead of relying purely on pre-trained knowledge, the model learns from sample interactions, helping it generalize patterns and reduce errors.

For instance, in a customer support AI, showing examples of refund requests and troubleshooting responses allows the model to infer how to handle similar queries effectively.

You are a customer support AI. Use the examples below to understand how to respond.

Example 1:
Customer: "I want a refund for my order."
AI: "Our refund policy allows returns within 30 days. Please provide your order number, and I'll assist you further."

Example 2:
Customer: "My product isn't working. What should I do?"
AI: "I'm sorry to hear that! Can you describe the issue? Meanwhile, you can check our troubleshooting guide [link]."

Now, respond to the following query:
Customer: "I received the wrong item. What should I do?"

2. Memory-augmented prompting (RAG-based)

Memory-Augmented Prompting (RAG-Based) enhances AI responses by retrieving relevant external data instead of relying solely on pre-trained knowledge. This approach is particularly useful when dealing with dynamic or domain-specific information.

Using Langbase, you can create memory agents for this. Langbase memory agents are a managed context search API for developers. They’re a helpful long-term memory solution that can acquire, process, retain, and later retrieve information. Memory agents combine vector storage, RAG (Retrieval-Augmented Generation), and internet access to help you build powerful AI features and products.

By incorporating Langbase with a Retrieval Augmented Generation (RAG) system, Memory is used with a Pipe agent to retrieve relevant data for queries.

The process involves:

• Creating query embeddings.

• Retrieving matching data from Memory.

• Augmenting the query with this data of 3-20 chunks.

• Using it to generate accurate, context-aware responses.

RAG prompt

When a memory is attached to a Pipe agent, by default a RAG prompt appears which is fed to LLM to utilize the memory. Default prompt works fine in most cases, but you can customize the prompt based on your use case.

You can learn how to build RAG by following this step-by-step guide.

3. Chain of Thought (CoT) prompting

CoT prompting helps AI agents break down complex problems into logical steps before answering. Instead of jumping to conclusions, the model is guided to reason through the problem systematically.

This prompting technique is great when you need the "how" behind the answer. It is especially useful for tasks requiring multi-step reasoning, such as debugging code.

For example, a coding AI agent can analyze an issue with the following prompt:

Analyze the following error message and identify possible causes. Then, break down the debugging steps to fix the issue.

This approach leads to more accurate and reliable responses by encouraging deeper reasoning rather than generating a hasty answer.

4. Role-based prompting

Role-based prompting helps AI agents generate more precise and context-aware responses by assigning them a specific identity. Instead of providing generic answers, the model adopts the characteristics of a domain expert, leading to better accuracy and relevance.

For example, in a cybersecurity AI agent, defining its role as a security expert ensures its responses prioritize risk assessment and best practices. A sample prompt could be:

You are a cybersecurity expert. Identify vulnerabilities in the given code and suggest fixes.

This approach narrows the LLM model’s focus, helping it analyze threats more effectively rather than offering broad, unstructured advice.

5. ReACT (Reasoning + Acting) prompting

This enables AI agents to make decisions by alternating between logical reasoning and real-world actions. Instead of generating static responses, the model interacts dynamically with tools, APIs, or databases to fetch and process information.

For example, a personal assistant AI booking flights might use a prompt like:

Check flight availability for [destination] on [date]. If no flights are found, suggest alternative dates.

This approach ensures the agent doesn’t hallucinate results—it retrieves real data, evaluates it, and adjusts its actions accordingly, making responses more reliable and grounded in actual outcomes. This technique integrates reasoning with real-time decision-making in agents. It’s perfect for dynamic, on-the-fly problem-solving.

6. Safety prompts

Langbase AI studio has a separate section that lets you define safety prompts inside a Pipe agent. For instance, do not answer questions outside of the given context.

One of its use cases can be to ensure the LLM does not provide any sensitive information in its response from the provided context.

Learn how to define safety instructions for any LLM inside a pipe here.

How to Fine-Tune the LLM’s Response Parameters

Now that you know the techniques to design strong prompts for your Pipe agents, let's take this further by adjusting model parameters like temperature, maximum tokens, top_p, and others to refine how the model responds to user queries.

Here are the LLM parameters that you can tweak to build efficient Pipe agents:

• Precise: Tuned for precise and accurate responses.

• Balanced: Strikes a balance between accuracy and creativity.

• Creative: Prioritizes creativity and diversity in the generated responses.

• Custom: Allows you to manually configure the response parameters.

• JSON_mode: Ensures the model will always output valid JSON.

• Temperature: Control how creative the LLM is with the outputs.

• Max_tokens: Specifies the maximum number of tokens that can be generated in the output.

• Frequency Penalty: Prevents the model from repeating a word that was too recently used/used too often.

• Presence Penalty: Prevents the model from repeating a word.

• Top_p: Generate tokens until the cumulative probability exceeds the chosen threshold.

Wrapping Up

Building effective serverless AI agents becomes easier if you use these prompt engineering techniques. You can give it a try by creating your own Pipe agent by visiting pipe.new.

Thank you for reading!

Connect with me by 🙌:

• Subscribing to my YouTube Channel. If you are willing to learn about AI and agents.

• Subscribing to my free newsletter “The Agentic Engineer” where I share all the latest AI and agents news/trends/jobs and much more.

• Follow me on X (Twitter).

This guide will walk you through building a cold email generator agent using serverless memory agents by Langbase to automate this entire process. We’ll integrate the memory agent into a Node.js project, enabling it to read your résumé, analyze the job description, and generate a personalized, high-impact cold email in seconds.

Here’s what I’ll cover:

1. Large language models (LLMs) are stateless by nature

2. What are Memory agents?

3. Prerequisites

4. Reference Architecture

5. Step 1: Create a directory and initialize npm

6. Step 2: Create a serverless Pipe agent

7. Step 3: Add a .env file

8. Step 4: Create a serverless memory agent

9. Step 5: Add documents to the memory agent

10. Step 6: Generate memory embeddings

    • Understanding memory embeddings

    • How to generate embeddings?

11. Step 7: Integrate memory in Pipe agent

12. Step 8: Integrate the memory agent in Node.js

13. Step 9: Start the BaseAI server

14. Step 10: Run the memory agent

15. The result

Large Language Models (LLMs) Are Stateless by Nature

LLMs (Large Language Models) are stateless because they don’t retain any memory of previous interactions or the context of past queries beyond the input they're given in a session. Each time an LLM processes a prompt, it operates on that specific prompt without any history from prior ones.

This stateless nature allows the model to treat each request as independent, which simplifies its architecture and training process. But this also means that without mechanisms like RAG (Retrieval-Augmented Generation) or memory (long-term), LLMs can't carry forward information from one interaction to the next.

To introduce continuity or context, developers can implement external systems to manage and inject context, but the model itself doesn't "remember" anything between requests.

How do we solve this?

By integrating Memory Agents by Langbase, we can give LLMs long-term memory—allowing them to store, retrieve, and use information dynamically, making them much more useful for real-world applications.

What Are Memory Agents?

Langbase serverless memory agents (long-term memory solution) are designed to acquire, process, retain, and retrieve information seamlessly. They dynamically attach private data to any LLM, enabling context-aware responses in real time and reducing hallucinations.

These agents combine vector storage, Retrieval-Augmented Generation (RAG), and internet access to create a powerful managed context search API. Developers can use them to build smarter, more capable AI applications.

In a RAG setup, memory – when connected directly to a Langbase Pipe Agent – becomes a memory agent. This pairing gives the LLM the ability to fetch relevant data and deliver precise, contextually accurate answers—addressing the limitations of LLMs when it comes to handling private data.

Memory agents ensure secure local memory storage. Data used to create memory embeddings stays protected, processed within secure environments, and only sent externally if explicitly configured. Access is strictly controlled via API keys, ensuring sensitive information remains safe.

Note that pipe is a serverless AI agent. It has agentic memory and tools.

Prerequisites

Before we begin creating a cold email generator agent, you’ll need to have the following setup and tools ready to go.

In this tutorial, I’ll be using this tech stack:

• BaseAI — the web framework for building AI agents locally.

• Langbase — the platform to build and deploy your serverless AI agents.

• OpenAI — to get the LLM key for the preferred model.

You’ll also need to:

• Sign up on Langbase to get access to the API key.

• Sign up on OpenAI to generate the LLM key for the model you want to use (for this demo, I’ll be using GPT-4o mini). You can generate the key here.

Reference Architecture

Here’s a diagrammatic representation of the entire process of building a serverless AI agent to generate cold emails for job applications:

Let’s start building the agent!

Step 1: Create a Directory and Initialize npm

To start creating a serverless AI agent that generates cold emails for a job opening, you need to create a directory in your local machine and install all the relevant dev dependencies in it. You can do this by navigating to it and running the following command in the terminal:

mkdir my-project
npm init -y
npm install dotenv

This command will create a package.json file in your project directory with default values. It will also install the dotenv package to read environment variables from the .env file.

Step 2: Create a Serverless Pipe Agent

Next, we’ll be creating a pipe agent. Pipes are different from other agents, as they are serverless AI agents with agentic tools that can work with any language or framework. They are easily deployable, and with just one API they let you connect more than 250 LLMs to any data to build any developer API workflow.

To create your AI agent pipe, navigate to your project directory. Run the following command:

npx baseai@latest pipe

Upon running, you’ll see the following prompts:

BaseAI is not installed but required to run. Would you like to install it? Yes/No
Name of the pipe? email-generator-agent
Description of the pipe? Generates emails for your dream job in seconds
Status of the pipe? Public/Private
System prompt? You are a helpful AI assistant

Once you are done with the name, description, and status of the AI agent pipe, everything will be set up automatically for you. Your pipe will be created successfully at /baseai/pipes/email-generator-agent.ts.

Step 3: Add a .env File

Create a .env file in the root directory of your project and add the OpenAI and Langbase API keys in it. You can access your Langbase API key from here.

Step 4: Create a Serverless Memory Agent

Next, we’ll be creating a memory and then attaching it with the Pipe to make it a memory agent. To do this, run this command in your terminal:

npx baseai@latest memory

Upon running this command, you’ll see the following prompts:

Name of the memory? email-generator-memory
Description of the memory? Contains my resume
Do you want to create memory from the current project git repository? Yes/No

After this, everything will be set up automatically for you and you can access your memory created successfully at /baseai/memory/email-generator-memory.ts.

Step 5: Add Documents to the Memory Agent

Inside /baseai/memory/email-generator-memory.ts you’ll see another folder called documents. This is where you’ll store the files you want your AI agent to access. Let’s save your résumé as either a .pdf or .txt file. Then, I’ll convert it to a markdown file and place it in the /baseai/memory/email-generator-memory/documents directory.

This step ensures that the memory agent can process and retrieve information from your documents, making the AI agent capable of generating accurate cold emails based on the experiences and skills provided in the résumé attached.

Step 6: Generate Memory Embeddings

With your documents added to memory, the next step is generating memory embeddings. But before that, let me quickly explain what embeddings are and why they matter.

Understanding memory embeddings

Memory embeddings are numerical representations of your documents that enable an AI to grasp context, relationships, and meaning within text. They act as a bridge, converting raw data into a structured format AI can process for semantic search and retrieval.

Without embeddings, AI agents wouldn’t effectively connect user queries with relevant content. Generating embeddings creates a searchable index, allowing the memory agent to deliver accurate, context-aware responses efficiently.

How to generate embeddings

To generate embeddings for your documents, run the following command in your terminal:

npx baseai@latest embed -m email-generator-memory

Your memory is now ready to be connected with a Pipe (memory agent), enabling your AI agent to fetch precise, context-aware responses from your documents.

Step 7: Integrate Memory in Pipe Agent

Next, you have to attach the memory you created to your Pipe agent to make it a memory agent. For that, go to /baseai/pipes/email-generator-agent.ts. This is what it will look like at the moment:

import { PipeI } from '@baseai/core';

const pipePipeWithMemory = (): PipeI => ({
    apiKey: process.env.LANGBASE_API_KEY!, // Replace with your API key https://langbase.com/docs/api-reference/api-keys
    name: 'email-generator-agent',
    description: 'Generates emails for your dream job in seconds',
    status: 'public',
    model: 'openai:gpt-4o-mini',
    stream: true,
    json: false,
    store: true,
    moderate: true,
    top_p: 1,
    max_tokens: 1000,
    temperature: 0.7,
    presence_penalty: 1,
    frequency_penalty: 1,
    stop: [],
    tool_choice: 'auto',
    parallel_tool_calls: false,
    messages: [
        { role: 'system', content: You are a helpful AI assistant. }],
    variables: [],
    memory: [],
    tools: []
});

export default pipePipeWithMemory;

Now integrate the memory in the pipe by importing it at the top and calling it as a function in the memory array. Also, add the following in the messages content:

Based on the job description and my resume attached, write a compelling cold email tailored to the job, highlighting my most relevant skills, achievements, and experiences. Ensure the tone is professional yet approachable, and include a strong call to action for a follow-up or interview.

This is what the code will look like after doing all of this:

import { PipeI } from '@baseai/core';
import emailGeneratorMemoryMemory from '../memory/email-generator-memory';

const pipeEmailGeneratorAgent = (): PipeI => ({
 // Replace with your API key https://langbase.com/docs/api-reference/api-keys
 apiKey: process.env.LANGBASE_API_KEY!,
 name: 'email-generator-agent',
 description: 'Generates emails for your dream job in seconds',
 status: 'private',
 model: 'openai:gpt-4o-mini',
 stream: true,
 json: false,
 store: true,
 moderate: true,
 top_p: 1,
 max_tokens: 1000,
 temperature: 0.7,
 presence_penalty: 1,
 frequency_penalty: 1,
 stop: [],
 tool_choice: 'auto',
 parallel_tool_calls: true,
 messages: [{ role: 'system', content: Based on the job description and my resume attached, write a compelling cold email tailored to the job, highlighting my most relevant skills, achievements, and experiences. Ensure the tone is professional yet approachable, and include a strong call to action for a follow-up or interview. }],
 variables: [],
 memory: [emailGeneratorMemoryMemory()],
 tools: []
});

export default pipeEmailGeneratorAgent;

Step 8: Integrate the Memory Agent in Node.js

Now we’ll integrate the memory agent you created into the Node.js project to build an interactive command-line interface (CLI) for the document attached. This Node.js project will serve as the base for testing and interacting with the memory agent (in the beginning of the tutorial, we set up a Node.js project by initializing npm).

Now, create an index.ts file:

touch index.ts

In this TypeScript file, import the pipe agent you created. We will use the pipe primitive from @baseai/core to run the pipe.

Add the following code to the index.ts file:

import 'dotenv/config';
import { Pipe } from '@baseai/core';
import inquirer from 'inquirer';
import ora from 'ora';
import chalk from 'chalk';
import pipeEmailGeneratorAgent from './baseai/pipes/email-generator-agent';

const pipe = new Pipe(pipeEmailGeneratorAgent());

async function main() {

   const initialSpinner = ora('Conversation with Memory agent...').start();
   try {
       const { completion: calculatorTool} = await pipe.run({
           messages: [{ role: 'user', content: 'Hello' }],
       });
       initialSpinner.stop();
       console.log(chalk.cyan('Report Generator Agent response...'));
       console.log(calculatorTool);
   } catch (error) {
       initialSpinner.stop();
       console.error(chalk.red('Error processing initial request:'), error);
   }

   while (true) {
       const { userMsg } = await inquirer.prompt([
           {
               type: 'input',
               name: 'userMsg',
               message: chalk.blue('Enter your query (or type "exit" to quit):'),
           },
       ]);


       if (userMsg.toLowerCase() === 'exit') {
           console.log(chalk.green('Goodbye!'));
           break;
       }


       const spinner = ora('Processing your request...').start();


       try {
           const { completion: reportAgentResponse } = await pipe.run({
               messages: [{ role: 'user', content: userMsg }],
           });


           spinner.stop();
           console.log(chalk.cyan('Agent:'));
           console.log(reportAgentResponse);
       } catch (error) {
           spinner.stop();
           console.error(chalk.red('Error processing your request:'), error);
       }
   }
}

main();

This code creates an interactive CLI for chatting with an AI agent, using a pipe from the @baseai/core library to process user input. Here's what happens:

• It imports necessary libraries such as dotenv for environment configuration, inquirer for user input, ora for loading spinners, and chalk for colored output. Make sure you install these libraries first using this command in your terminal: npm install ora inquirer.

• A pipe object is created from the BaseAI library using a predefined memory called email-generator-agent.

In the main() function:

• A spinner starts while an initial conversation with the AI agent is initiated with the message 'Hello'.

• The response from the AI is displayed.

• A loop runs to continually ask the user for input and send queries to the AI agent.

• The AI's responses are shown, and the process continues until the user types "exit”.

Step 9: Start the BaseAI Server

To run the memory agent locally, you need to start the BaseAI server first. Run the following command in your terminal:

npx baseai@latest dev

Step 10: Run the Memory Agent

Run the index.ts file using the following command:

npx tsx index.ts

The Result

In your terminal, you’ll be prompted to "Enter your query." For example, let’s paste a job description and ask to generate an email from our end showing interest. And it will give us the response with correct sources/citations as well.

With this setup, we’ve built a Cold Email Generator agent that uses the power of LLMs and Langbase memory agents to overcome LLMs' limitations, ensuring accurate responses without hallucinating on private data.

Here’s a demo of the end result:

Thank you for reading!

Connect with me by 🙌:

• Subscribing to my YouTube Channel. If you are willing to learn about AI and agents.

• Subscribing to my free newsletter “The Agentic Engineer” where I share all the latest AI and agents news/trends/jobs and much more.

• Follow me on X (Twitter).

While LangChain has enabled rapid development of LLM-powered apps with tools for chaining, agents, and memory, its heavy abstraction, complex debugging, and difficulty with real-world use often make it more suited for prototyping than production-level applications.

In this article, we’ll explore some powerful LangChain alternatives you can try out that’ll help you build effective AI and agentic workflows. I’ll go through each one’s key features and best use cases so you can get a good idea of how it might help you.

Table of Contents

1. What is an AI and Agentic Workflow?

2. Langbase

3. LlamaIndex

4. AG2

5. Braintrust

6. FlowiseAI

7. Wrap Up 🙌

What is an AI and Agentic Workflow?

An AI workflow refers to a series of tasks executed by AI systems, typically following a predefined sequence. It handles tasks like data extraction, processing, and generating output based on clear instructions.

An agentic workflow goes a step further. It involves AI taking initiative, making decisions, and managing tasks autonomously. In agentic workflows, AI adapts its actions based on its environment or predefined goals, often without human intervention.

In short, an AI workflow becomes more "agentic" as it begins to think, decide, and act independently, acting like an intelligent agent. The more decisions AI can make on its own, the less it needs to be prompted by a human.

Now that it’s clear what an AI and agentic workflow is, let’s look at some other tools and frameworks that could serve as LangChain alternatives, each offering unique capabilities and approaches that you can use to build your AI and agentic workflows.

Langbase

Langbase is a Serverless Composable AI Developer platform with multi-agent orchestration and advanced long-term memory. It’s designed for seamless AI development and deployment. Langbase provides support to over 100+ LLMs through one API ensuring a unified developer experience, with easy model switching and optimization.

💡 Multi-agent orchestration refers to coordinating multiple AI agents to work together on tasks. It involves controlling the flow of tasks, ensuring agents work in the right sequence, and coordinating their actions to maximize efficiency.

Langbase Products

The platform offers the following products:

1. Pipe Agents: Pipe agents on Langbase are different from other agents. They are serverless AI agents with agentic tools that can work with any language or framework. Pipe agents are easily deployable, and with just one API they let you connect 100+ LLMs to any data to build any developer API workflow.

2. Memory Agents: Langbase memory agents (long-term memory solution) are designed to acquire, process, retain, and retrieve information seamlessly. They dynamically attach private data to any LLM, enabling context-aware responses in real time and reducing hallucinations. Memory, when connected to a pipe agent, becomes a memory agent.

3. BaseAI.dev: BaseAI is the open-source first web AI framework. With it you can start building local-first, agentic pipes, tools, and memory and deploy serverless with one command.

4. AI Studio: Langbase AI Studio provides a playground to collaborate on AI agents, memory and tools. With it you can build, collaborate, test and deploy pipe and memory (RAG) agents.

5. LangUI: LangUI is a free, open-source Tailwind library with ready-made components designed for AI and GPT projects.

6. Langbase SDK: Langbase offers a TypeScript AI SDK that simplifies development. It helps you easily integrate LLMs, create memory agents, and chain them together into pipelines—all with minimal code. It supports JavaScript, TypeScript, Node.js, Next.js, React, and more, enabling faster development with a great developer experience.

Key Features

• API-first platform: Simple APIs (Pipe and memory agent APIs) for easy integration with clear documentation and community support.

  💡 Use the Pipe API to manage the pipe agents in your Langbase account. It has create, update, list and run endpoints. The Langbase Memory API lets you manage memories and documents in your Langbase account programmatically.

• Serverless environment: Langbase operates in a fully serverless environment, removing the need for developers to manage infrastructure. This simplifies scaling and deployment, empowering developers of all skill levels—not just AI/ML experts—to build, scale, and deploy AI agents seamlessly.

• Composable infrastructure: Langbase is the first composable AI platform. It is built for flexibility and modularity. Developers can combine models into pipelines, each focused on a specific task. This makes development easier, shows the cost of each step, and lets you create highly personalized experiences. By choosing the best model for each task, Langbase helps you build efficient workflows that fit different needs.

  💡 Composable AI means combining different AI models like building blocks to create custom solutions. It’s simple, flexible, and adapts to your needs.

• Cost efficiency: Langbase offers significantly more value at a lower cost than LangChain, with overage costs just $2 per 1,000 runs compared to LangChain’s $5.

Use Cases

Langbase is perfect for developers seeking cost-efficient solutions with seamless model switching through a single API. It’s well-suited for projects that require composable/modular AI infrastructure and advanced long-term memory features. It also excels in building autonomous workflows with multi-agent collaboration.

Here are a few specific applications that you can build using Langbase:

• Customer Support Agent: Build customer support agents/apps that can handle complex, context-aware conversations across support tickets, emails, and chats, providing accurate, efficient resolutions. Check out the customer support agents here.

• Coding Agent: Create multi-agentic apps that assist developers by generating code snippets, debugging, and reviewing code in real time, improving productivity in software development workflows. Here’s an example coding agent demo.

Getting Started with Langbase

• To get started with Langbase signup for free here.

• To create a pipe agent simply type pipe.new in the search bar.

• To create a memory agent type rag.new in the search bar.

LlamaIndex

LlamaIndex is an open-source framework built for RAG applications and agent-based systems. It provides essential tools to ingest, structure, and connect private or domain-specific data to LLMs, enabling more accurate and reliable text generation.

With its support for building agents and integrating RAG pipelines as part of a broader toolset, LlamaIndex offers the flexibility to handle complex tasks.

Key Features

• Data loading: LlamaIndex makes data import seamless with support for 150+ sources, including APIs, PDFs, documents, and SQL databases. Using data connectors (LlamaHub), developers can integrate diverse data into their LLM applications effortlessly. Examples include pulling real-time data from APIs, loading structured information from MySQL or PostgreSQL, and ingesting text from PDFs or reports.

  💡 Data Loaders are utilities that allow you to easily ingest data for search and retrieval by a large language model.

• Indexing: Indexing organizes and stores data for easy and fast retrieval, creating structures like vector or document indexes. With LlamaIndex, you can store and index data across multiple providers (for example, vector, document, graph, and SQL databases).

• Querying: Querying retrieves specific information from indexed data, enabling searches and advanced workflows like RAG pipelines for context-aware responses. For this, LlamaIndex lets you build advanced query workflows with retrieval, post-processing, and response synthesis for prompt chains and RAG pipelines.

  💡 A Query Pipeline in LlamaIndex is a simple way to design query workflows for different tasks like RAG and structured data extraction. It helps you define how queries interact with your data, making it easy to handle both basic and advanced workflows. Read about LlamaIndex Query Pipelines here.

• Evaluations: Includes modules for evaluating retrieval and response quality, enhancing performance monitoring.

Use Cases

LlamaIndex is preferred for seamless data indexing and quick retrieval, making it more suitable for production-ready RAG applications. On the other hand, LangChain provides more out-of-the-box components, making it easier to create diverse LLM architectures.

Here are a few specific RAG applications that you can build using LlamaIndex:

• Financial Insights Assistant: Build a knowledge assistant for financial analysts to retrieve real-time insights from market data, earnings reports, and internal financial documents, enabling quicker decision-making and risk assessment.

• Manufacturing Advisor: Create an AI-powered assistant to streamline production workflows by accessing equipment manuals, maintenance logs, and supply chain data, improving operational efficiency and reducing downtime.

Getting Started with LlamaIndex

You can get started with LlamaIndex in Python or TypeScript in just 5 lines of code.

1. Set the OPENAI_API_KEY environment variable with your OpenAI API key.

2. Install the Python library: pip install llama-index

3. Place your documents in a folder named data, then use this starter code to query them:

pythonCopyEditfrom llama_index.core import VectorStoreIndex, SimpleDirectoryReader

documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
response = query_engine.query("Your question here")
print(response)

For more details, check out their documentation.

AG2

AG2 (formerly AutoGen) is an open-source framework for building AI agents and enabling multi-agent collaboration. AG2 provides a framework for building autonomous workflows and agent collaboration, simplifying the creation of specialized agents that can work together seamlessly.

💡 Multi-agent collaboration refers to multiple agents working together toward a common goal, each performing tasks and sharing information as needed. The agents can be independent and specialized, but they collaborate to complete a task.

Key Features

• Agent collaboration: Supports multi-agent orchestration for seamless communication and task management.

• Flexible agent roles: Define agent behaviors, roles, and workflows with intuitive code. Assign specific roles to agents—such as data collector, analyzer, or decision-maker—and have them interact in conversations or work independently. For example, one agent might gather information, while another processes it and provides insights. These agent conversations can drive task completion, with each agent contributing based on its designated role, making workflows more dynamic and efficient.

• Human-in-the-loop support: AG2 enables seamless human involvement in the workflow by allowing for customizable input methods, such as manual overrides or feedback loops. It offers context-aware handoff, meaning the system can pass tasks to a human at the right moment, based on specific conditions or requirements. Additionally, interactive interfaces are provided, enabling humans to review, approve, or adjust agent actions in real-time, ensuring the system remains aligned with human judgment and oversight.

• Conversation patterns: Built-in patterns automate coordination tasks like message routing, state management, and dynamic speaker selection.

Use Cases

AG2 stands out for its ability to handle complex agent interactions, making it a great choice for multi-agent workflows that require human collaboration.

Here are a few AI applications that you can build using AG2:

• Content Creation and Review Pipelines: Build collaborative workflows where one agent generates written or visual content, another ensures compliance with guidelines, and a human reviewer provides creative inputs or final approval.

• Personalized Education Platforms: Create learning assistants where one agent curates educational content, another designs custom learning paths, and a third monitors student progress. Teachers or mentors can step in to provide personalized feedback or adjustments to the curriculum.

Getting Started with AG2

AG2 requires Python version >= 3.9, < 3.14. It can be installed from pip:

pip install ag2

For more details visit the documentation.

Braintrust

Braintrust is an end-to-end platform for evaluating, improving, and deploying large language models (LLMs) with tools for prompt engineering, data management, and continuous evaluation. Designed to make building AI applications more robust and iterative, Braintrust helps you prototype rapidly with different prompts and models, evaluate performance with built-in tools, and monitor real-world interactions in real time.

Key Features

• Iterative experimentation: Rapidly prototype and test prompts with different models in the integrated playground. You can experiment with real dataset inputs, compare responses across models (OpenAI, Anthropic, Mistral, Google, Meta, and more), and fine-tune performance in the playground.

• Performance insights: Evaluate model and prompt performance with built-in tools like the prompt playground, dataset imports, and scoring functions. You can test outputs against real-world data, compare models, and refine prompts iteratively. Use heuristics or LLM-based scoring to assess accuracy, track results, and improve performance over time within Braintrust's UI or SDK.

• Real-time monitoring: Track AI interactions with detailed logs, capturing inputs, outputs, and metadata for each request. Braintrust logs traces of AI calls, breaking them down into spans to pinpoint issues, monitor user behavior, and refine performance. Logs integrate seamlessly with evaluations, creating a feedback loop for continuous model improvement.

• Centralized data management: Braintrust integrates data from production, staging, and evaluations, allowing you to track changes, compare iterations, and refine models over time. Versioning ensures you can roll back, audit, and pin evaluations to specific dataset versions, supporting structured experimentation and human-in-the-loop reviews for continuous improvement.

  💡 Datasets allow you to collect data from production, staging, evaluations, and even manually, and then use that data to run evaluations and track improvements over time.

Use Cases

Braintrust is best suited for iterative model development and evaluation, especially for projects that demand robust testing and deployment pipelines. It stands out for building scalable LLM applications, offering data-driven insights that enable precise optimization and continuous improvement.

Here are a few apps you can build with Braintrust:

• Evaluating a Chat Assistant: With Braintrust you can evaluate a chat assistant by ensuring conversational AI maintains context for accurate responses. It enables automated evaluations to assess response quality, manages datasets to refine test cases, and tracks performance for continuous improvement.

• AI Search Bar: Braintrust helps optimize AI-powered search by ensuring accuracy and context awareness. It logs queries to identify gaps, benchmarks search results for relevance, and compares model versions to track improvements.

Getting Started with Braintrust

• To get started signup on Braintrust.

• Once you’re signed up, you’ll be asked to create an organization for free.

• To run your first eval, you can either use the UI or the starter code available. Install the Braintrust SDK using this command:

    npm install braintrust autoevals
  

  For more details visit the documentation.

FlowiseAI

FlowiseAI is an open source low-code tool for developers to build customized LLM orchestration flows & AI agents. With its intuitive drag-and-drop interface, Flowise makes LLM technology accessible to a wider audience, including those with little to no coding experience.

Key Features

• Fast iteration: The low-code approach enables quick iterations, making it easy to move from testing to production in a fraction of the time.

• Chatflow and LLM orchestration: Seamlessly connect LLMs with memory, data loaders, caching, and moderation tools to manage how models process inputs, retrieve relevant data, and generate responses. This ensures controlled interactions between models, user inputs, and external data sources for optimal performance.

• 100+ integrations: Easily connect with tools like Langchain and LlamaIndex to improve your workflows. These integrations help you link data sources, manage tasks, and add extra features, allowing you to build customized AI applications. Use them to automate work, improve model performance, or expand what your system can do based on your needs.

• Agents and assistants: Build autonomous agents that execute tasks using tools like multi-agents or sequential agents, enhancing your application’s capabilities. These agents can interact with external data sources and tools, enabling them to perform specialized tasks efficiently. For example, Flowise provides two approaches for creating agent-based systems: Multi-Agents, which work together in a specialized, collaborative way, and Sequential Agents, which process tasks in a structured, step-by-step manner. By integrating these systems, you can automate complex workflows and improve task execution within your app.

• Developer-friendly: Extend and integrate with your apps using APIs, SDKs, and embedded chat options, including React SDK and embedded widgets.

Use Cases

Flowise is great for developers with little coding experience building LLM workflows and teams needing quick updates without losing functionality. It makes advanced AI workflows easy to use, even for non-experts.

It integrates with frameworks like LangChain and LlamaIndex, making it ideal for simplified AI development. But it may pose challenges for those new to LLMs, and code-first approaches might be better suited for highly specialized tasks.

Here are a few practical examples that you can build using Flowise:

• Query Multiple Documents: With Flowise, you can build systems that query multiple documents by uploading them to Pinecone with metadata. Tool agents help the LLM select the appropriate document based on the question.

• Personal Assistants: Develop assistants who can handle tasks, schedule appointments, and provide reminders with Flowise.

Getting Started with FlowiseAI

• To get started, install Flowise locally using NPM.

    npm install -g flowise
  

  💡 Prerequisite: ensure NodeJS is installed on machine. Node v18.15.0 or v20 and above is supported.

• Start Flowise using this command and open localhost:3000:

    npx flowise start
  

For more details check out these getting started steps.

Wrapping Up 🙌

AI and agentic workflows are moving fast, and LangChain isn’t the only option anymore. Choosing the right tool comes down to your project’s needs—flexible agent orchestration, cutting costs, or seamless integration. As we push into 2025, these alternatives deserve your attention while building the future of AI.

Thanks for reading!

Memory agents solve this by securely linking your private data to any LLM in real time. In this tutorial, I’ll walk you through turning an LLM into a conversational AI that chats with your personal documents using Langbase memory agents.

Here’s what we’ll cover:

1. What are Memory Agents?

2. Securing Your Data with Memory Agents

3. Use Cases for Memory Agents

4. Prerequisites

5. Step 1: Create a Directory and Initialize npm

6. Step 2: Create a Pipe Agent

7. Step 3: Add a .env File

8. Step 4: Create a Memory Agent

9. Step 5: Add Documents to the Memory Agent

10. Step 6: Generate Memory Embeddings

    • What Are Memory Embeddings?

    • Why Do You Need Memory Embeddings?

    • How to Generate Embeddings

11. Step 7: Integrate Memory in Pipe Agent

12. Step 8: Integrate the Memory Agent in Node.js

13. Step 9: Start the BaseAI Server

14. Step 10: Run the Memory Agent

15. The Result

What are Memory Agents?

Memory is what makes interactions meaningful. It’s how systems can remember what came before, a key aspect for building truly intelligent AI agents.

Here’s the thing: LLMs might seem human-like, but they don’t have memory built in. They’re stateless by design. To make them useful for real-world tasks, you need to add memory. That’s where memory agents step in.

Langbase memory agents (long-term memory solution) are designed to acquire, process, retain, and retrieve information seamlessly. They dynamically attach private data to any LLM, enabling context-aware responses in real time and reducing hallucinations.

These agents combine vector storage, Retrieval-Augmented Generation (RAG), and internet access to create a powerful managed context search API. Developers can use them to build smarter, more capable AI applications.

In a RAG setup, memory – when connected directly to a Langbase Pipe Agent – becomes a memory agent. This pairing gives the LLM the ability to fetch relevant data and deliver precise, contextually accurate answers—addressing the limitations of LLMs when it comes to handling private data.

Here’s a diagrammatic representation of the entire process:

Securing Your Data with Memory Agents

Memory agents prioritize data security by keeping private information isolated and processed locally or within secure environments. The data used to create memory embeddings is not sent to external servers unless explicitly configured, ensuring sensitive information remains protected.

Also, access to the memory system is strictly controlled through API keys and permissions, preventing unauthorized access. This setup not only enhances AI capabilities but also maintains user trust by safeguarding their data.

Use Cases for Memory Agents

Here are some practical applications of these agents:

• Customer Support: Deliver personalized, context-aware assistance by recalling interaction history.

• Document Search: Enable fast, semantic search in large datasets, manuals, or FAQs.

• Code Assistance: Provide project-specific documentation and debugging tips for developers.

• Knowledge Management: Centralize and retrieve internal information for teams efficiently.

• Education & Training: Provide students or employees with customized training materials, track progress, and answer questions based on stored resources.

• Healthcare: Securely retrieve patient records or medical history for accurate support.

• Collaborative Workflows: Track project history and integrate with tools for team alignment.

• Legal Compliance: Reference guidelines to ensure accurate and regulation-compliant decisions.

The many use cases enabled by memory agents are opening up new possibilities and changing what Artificial General Intelligence (AGI) can do.

Prerequisites

Before we begin creating a memory agent that chats with your documents, you’ll need to have the following setup and tools ready to go.

In this tutorial, I’ll be using the following tech stack:

• BaseAI — the web framework for building AI agents locally.

• Langbase — the platform to build and deploy your serverless AI agents.

• OpenAI — to get the LLM key for the preferred model.

You’ll also need to:

• Sign up on Langbase to get access to the API key.

• Sign up on OpenAI to generate the LLM key for the model you want to use (for this demo, I’ll be using GPT-4o mini).

Let’s get started!

Step 1: Create a Directory and Initialize npm

To start creating a memory agent that chats with your documents, you need to create a directory in your local machine and install all the relevant dev dependencies in it. You can do this by navigating to it and running the following command in the terminal:

mkdir my-project
npm init -y
npm install dotenv

This command will create a package.json file in your project directory with default values. It will also install the dotenv package to read environment variables from the .env file.

Step 2: Create a Pipe Agent

Next, we’ll be creating a pipe agent. Pipes are different from other agents, as they are serverless AI agents with agentic tools that can work with any language or framework. They are easily deployable, and with just one API they let you connect 100+ LLMs to any data to build any developer API workflow.

To create your AI agent pipe, navigate to your project directory. Run the following command:

npx baseai@latest pipe

Upon running, you’ll see the following prompts:

BaseAI is not installed but required to run. Would you like to install it? Yes/No
Name of the pipe?  pipe-with-memory
Description of the pipe? Pipe attached to a memory
Status of the pipe? Public/Private
System prompt? You are a helpful AI assistant

Once you are done with the name, description, and status of the AI agent pipe, everything will be set up automatically for you. Your pipe will be created successfully at /baseai/pipes/pipe-with-memory.ts.

Step 3: Add a .env File

Create a .env file in the root directory of your project and add the OpenAI and Langbase API key in it. You can access your Langbase API key from here.

Step 4: Create a Memory Agent

Next, we’ll be creating a memory and then attaching it with the Pipe to make it a memory agent. To do this, run this command in your terminal:

npx baseai@latest memory

Upon running this command, you’ll see the following prompts:

Name of the memory?  chat-with-docs-agent
Description of the pipe? FAQs docs
Do you want to create memory from the current project git repository? Yes/No

After this, everything will be set up automatically for you and you can access your memory created successfully at /baseai/memory/chat-with-docs-agent.ts.

Step 5: Add Documents to the Memory Agent

Inside /baseai/memory/chat-with-docs-agent.ts you’ll see another folder called documents. This is where you’ll store the files you want your AI agent to access. For this demo, I’ll save the Pipe FAQs page as either a .pdf or .txt file. Then, I’ll convert it to a markdown file and place it in the baseai/memory/chat-with-docs/documents directory.

This step ensures the memory agent can process and retrieve information from your documents, making the AI agent capable of answering queries based on the content you provide.

Step 6: Generate Memory Embeddings

Now that you’ve added documents to the memory, the next step is to generate memory embeddings. But first, what exactly are embeddings, and why are they essential?

What Are Memory Embeddings?

Embeddings are numerical representations of your documents that allow the AI to understand the context and relationships between words, phrases, and sentences. Think of embeddings as a way to translate your documents into a language the AI can process for semantic search and retrieval.

Why Do You Need Memory Embeddings?

Without embeddings, the AI agent wouldn’t be able to match user queries with relevant content from your documents. By generating embeddings, you’re essentially creating a searchable index that enables accurate and efficient responses from the memory agent.

How to Generate Embeddings

To generate embeddings for your documents, run the following command in your terminal:

npx baseai@latest embed -m chat-with-docs-agent

Your memory is now ready to be connected with a Pipe (memory agent), enabling your AI agent to fetch precise, context-aware responses from your documents.

Step 7: Integrate Memory in Pipe Agent

Next, you have to attach the memory you created to your Pipe agent to make it a memory agent. For that, go to /baseai/pipes/pipe-with-memory.ts. This is what it will look like at the moment:

import { PipeI } from '@baseai/core';

const pipePipeWithMemory = (): PipeI => ({
    apiKey: process.env.LANGBASE_API_KEY!, // Replace with your API key https://langbase.com/docs/api-reference/api-keys
    name: 'pipe-with-memory',
    description: 'Pipe attached to a memory',
    status: 'public',
    model: 'openai:gpt-4o-mini',
    stream: true,
    json: false,
    store: true,
    moderate: true,
    top_p: 1,
    max_tokens: 1000,
    temperature: 0.7,
    presence_penalty: 1,
    frequency_penalty: 1,
    stop: [],
    tool_choice: 'auto',
    parallel_tool_calls: false,
    messages: [
        { role: 'system', content: `You are a helpful AI assistant.` }],
    variables: [],
    memory: [],
    tools: []
});

export default pipePipeWithMemory;

Now integrate the memory in the pipe by importing it at the top and calling it as a function in the memory array. This is what the code will look like after doing all of this:

import { PipeI } from '@baseai/core';
import chatWithDocsAgentMemory from '../memory/chat-with-docs-agent';

const pipePipeWithMemory = (): PipeI => ({
    apiKey: process.env.LANGBASE_API_KEY!, // Replace with your API key https://langbase.com/docs/api-reference/api-keys
    name: 'pipe-with-memory',
    description: 'Pipe attached to a memory',
    status: 'public',
    model: 'openai:gpt-4o-mini',
    stream: true,
    json: false,
    store: true,
    moderate: true,
    top_p: 1,
    max_tokens: 1000,
    temperature: 0.7,
    presence_penalty: 1,
    frequency_penalty: 1,
    stop: [],
    tool_choice: 'auto',
    parallel_tool_calls: false,
    messages: [
        { role: 'system', content: `You are a helpful AI assistant.` }],
    variables: [],
    memory: [chatWithDocsAgentMemory()],
    tools: []
});

export default pipePipeWithMemory;

Step 8: Integrate the Memory Agent in Node.js

Now we’ll integrate the memory agent you created into the Node.js project to build an interactive command-line interface (CLI) for the document attached. This Node.js project will serve as the base for testing and interacting with the memory agent (in the beginning of the tutorial, we set up a Node.js project by initializing npm).

Now, create an index.ts file:

touch index.ts

In this TypeScript file, import the pipe agent you created. We will use the pipe primitive from @baseai/core to run the pipe.

Add the following code to the index.ts file:

import 'dotenv/config';
import { Pipe } from '@baseai/core';
import inquirer from 'inquirer';
import ora from 'ora';
import chalk from 'chalk';
import pipePipeWithMemory from './baseai/pipes/pipe-with-memory';

const pipe = new Pipe(pipePipeWithMemory());

async function main() {

   const initialSpinner = ora('Conversation with Memory agent...').start();
   try {
       const { completion: calculatorTool} = await pipe.run({
           messages: [{ role: 'user', content: 'Hello' }],
       });
       initialSpinner.stop();
       console.log(chalk.cyan('Report Generator Agent response...'));
       console.log(calculatorTool);
   } catch (error) {
       initialSpinner.stop();
       console.error(chalk.red('Error processing initial request:'), error);
   }


   while (true) {
       const { userMsg } = await inquirer.prompt([
           {
               type: 'input',
               name: 'userMsg',
               message: chalk.blue('Enter your query (or type "exit" to quit):'),
           },
       ]);


       if (userMsg.toLowerCase() === 'exit') {
           console.log(chalk.green('Goodbye!'));
           break;
       }


       const spinner = ora('Processing your request...').start();


       try {
           const { completion: reportAgentResponse } = await pipe.run({
               messages: [{ role: 'user', content: userMsg }],
           });


           spinner.stop();
           console.log(chalk.cyan('Agent:'));
           console.log(reportAgentResponse);
       } catch (error) {
           spinner.stop();
           console.error(chalk.red('Error processing your request:'), error);
       }
   }
}

main();

This code creates an interactive CLI for chatting with an AI agent, using a pipe from the @baseai/core library to process user input. Here's what happens:

• It imports necessary libraries such as dotenv for environment configuration, inquirer for user input, ora for loading spinners, and chalk for colored output. Make sure you install these libraries first using this command in your terminal: npm install ora inquirer.

• A pipe object is created from the BaseAI library using a predefined memory called pipe-with-memory.

In the main() function:

• A spinner starts while an initial conversation with the AI agent is initiated with the message 'Hello'.

• The response from the AI is displayed.

• A loop runs to continually ask the user for input and send queries to the AI agent.

• The AI's responses are shown, and the process continues until the user types "exit”.

Step 9: Start the BaseAI Server

To run the memory agent locally, you need to start the BaseAI server first. Run the following command in your terminal:

npx baseai@latest dev

Step 10: Run the Memory Agent

Run the index.ts file using the following command:

npx tsx index.ts

The Result

In your terminal, you’ll be prompted to "Enter your query." For example, let’s ask: "What is a pipe on Langbase?" And it will give us the response with correct sources/citations as well.

With this setup, we’ve built a "Chat with Your Document" agent that uses the power of LLMs and Langbase memory agents to overcome LLMs' limitations, ensuring accurate responses without hallucinating on private data.

Here’s a demo of the end result:

Thank you for reading!

LLMs are built to handle natural language. While generally being good at generating words and stringing together language, when it comes to math, they often need help.

Unlike a calculator or math library, LLMs cannot sometimes reason or process symbolic logic. So, while they can manage basic arithmetic, especially if it's something familiar from their training data, they typically struggle with more complex problems, particularly word problems.

The main question is how to fix this LLM limitation?

No doubt, LLMs have evolved with the launch of reasoning models like GPT-o1 or Llama 3.3. But they still hallucinate, lack real-time data access, struggle with complex math, and produce non-deterministic outputs. Fortunately, we can solve this problem using AI agents.

What is an AI Agent?

AI agents are autonomous software that use LLMs to perform tasks beyond simple text generation.

They make decisions and execute actions. AI agents rely on LLMs for language understanding but add capabilities like memory, real-time interaction, and decision-making.

How AI Agents Solve LLM Limitations

Agents augment the capabilities of LLMs in the following ways:

• Memory: AI agents help LLMs retain context from past interactions, improving long-term conversation coherence.

• Asynchronous processing: Agents handle multiple tasks at once, enhancing efficiency.

• Fact-checking: They connect to real-time data sources to verify information.

• Enhanced math: They integrate tools to handle complex calculations.

• Consistent output: Agents standardize LLM outputs for uniform formatting.

To help address some of the math limitations LLMs experience, let’s create an AI agent that builds a calculator using MathJS and BaseAI tool calls.

Prerequisites

In this tutorial, I’ll be using the following tech stack:

• MathJS — an extensive math library for JavaScript and Node.js.

• BaseAI — the web framework for building AI agents locally.

• Langbase — the platform to build and deploy your serverless AI agents.

• OpenAI — to get the LLM key for the preferred model.

You’ll also need to:

• Sign up on Langbase to get access to the API key.

• Sign up on OpenAI to generate the LLM key for the model you want to use (for this demo, I’ll be using GPT-4o mini).

Let’s get started!

Step 1: Create a Directory and Initialize npm

To start creating an AI agent, you need to create a directory in your local machine and install all the relevant dev dependancies in it. You can do this by navigating to it and running the following command in the terminal:

mkdir my-project

npm init -y

npm install dotenv mathjs

This command will create a package.json file in your project directory with default values. It will also install the dotenv package to read environment variables from the .env file, and mathjs to handle math operations.

Step 2: Create an AI Agent Pipe

Next, we’ll be creating an AI agent pipe. Pipes are different from other agents, as they are serverless AI agents with agentic tools that can work with any language or framework. They are easily deployable, and with just one API they let you connect 100+ LLMs to any data to build any developer API workflow.

To create your AI agent pipe, navigate to your project directory. Run the following command:

npx baseai@latest pipe

Upon running that command, you’ll see the following prompts:

BaseAI is not installed but required to run. Would you like to install it? Yes/No

Name of the pipe? pipe-with-tool

Description of the pipe? An AI agent pipe that can call tools

Status of the pipe? Public/Private

System prompt? You are a helpful AI assistant

Once you are done with the name, description, and status of the AI agent pipe, everything will be set up automatically for you. Your pipe will be created successfully at /baseai/pipes/pipe-with-tool.ts.

Step 3: Add a .env File

Create a .env file in the root directory of your project and add the OpenAI and Langbase API key in it. You can access your Langbase API key from here.

Step 4: Configure the AI Agent Pipe

In this step, we’ll configure the AI agent pipe created according to our needs.

Navigate to your project directory and open the AI agent pipe you created. You can add a system prompt to the pipe if you want. I’m sticking to You are a helpful AI assistant that will work as a calculator. This is what it will look like:

import { PipeI } from '@baseai/core';

const pipePipeWithTool = (): PipeI => ({
   apiKey: process.env.LANGBASE_API_KEY!,
   name: 'pipe-with-tool',
   description: 'An AI agent pipe that can call tools',
   status: 'private',
   model: 'openai:gpt-4o-mini',
   stream: true,
   json: false,
   store: true,
   moderate: true,
   top_p: 1,
   max_tokens: 1000,
   temperature: 0.7,
   presence_penalty: 1,
   frequency_penalty: 1,
   stop: [],
   tool_choice: 'auto',
   parallel_tool_calls: true,
   messages: [{ role: 'system', content: `You are a helpful AI assistant that will work as a calculator.` }],
   variables: [],
   memory: [],
   tools: []
});

export default pipePipeWithTool;

Step 5: Create a Calculator Tool

Tool calling lets an LLM use external tools, such as functions, APIs, or other resources, to get information or perform tasks beyond its built-in knowledge.

In this step, we'll create a Calculator Tool using BaseAI tools. This tool will handle all mathematical computations in your project, ensuring they are error-free and trustworthy. The tool is versatile and suitable for both simple calculations (e.g., 5+7) and more advanced ones (e.g., sin(pi/4) + log(10)).

It will also be particularly helpful in reducing hallucinations, which it can do by offloading computations to an external tool This avoids incorrect or fabricated answers that LLMs might otherwise generate. It also reduces the likelihood of getting incorrect responses from the LLM by rechecking or gathering additional data to ensure accuracy.

By using BaseAI's smart tool-calling and memory features, we can reduce AI hallucinations by 21% while improving the model's ability to self-correct its outputs.

These enhancements are useful when dealing with complex mathematical expressions or formula evaluations and should really improve the quality and accuracy of the LLM’s answers.

To create a calculator tool in your project that will be responsible for doing all the calculations without errors, run this command in your terminal:

npx baseai@latest tool

You’ll be asked to provide a name and description of the tool in your terminal. This is what I’m providing:

Name of the tool? Calculator

Description of the tool? Evaluate mathematical expressions

Your tool will be created at /baseai/tools/calculator.ts.

Step 6: Configure the Calculator Tool

To configure the tool, navigate to your project directory and open the tool you created. You can find it at /baseai/tools/calculator.ts.

This is what the code will look like:

import { ToolI } from '@baseai/core';

export async function calculator() {
   // Add your tool logic here
   // This function will be called when the tool is executed
}

const toolCalculator = (): ToolI => ({
   run: calculator,
   type: 'function' as const,
   function: {
       name: 'toolCalculator',
       description: 'Evaluate mathematical expressions',
       parameters: {}
   }
});

export default toolCalculator;

The run key in the toolCalculator object is the function that will be executed when the tool is called. You can write your logic to get the mathematical calculations for a given function.

Update the calculator tool’s description and code by adding parameters to the calculator function. The LLM will give values to these parameters when it calls the tool. And it’ll even import math from mathjs. This is the final code:

import * as math from 'mathjs';

export async function calculator({expression}: {expression: string}) {
   return math.evaluate(expression);
}

const toolCalculator = () => ({
   run: calculator,
   type: 'function' as const,
   function: {
       name: 'calculator',
       description:
           `A tool that can evaluate mathematical expressions. ` +
           `Example expressions: ` +
           `'5.6 * (5 + 10.5)', '7.86 cm to inch', 'cos(80 deg) ^ 4'.`,
       parameters: {
           type: 'object',
           required: ['expression'],
           properties: {
               expression: {
                   type: 'string',
                   description: 'The mathematical expression to evaluate.',
               },
           },
       },
   },
});

export default toolCalculator;

Step 7: Integrate the Tool in the AI Agent Pipe

In this step, we’ll integrate the tool in the AI agent pipe we created. For that, open the pipe file present at /baseai/pipes/pipe-with-tool.ts and import the calculator tool at the top of the file. We will also call the calculator tool in the tools array of the pipe.

import {PipeI} from '@baseai/core';
import toolCalculator from '../tools/calculator';

const pipeWithTools = (): PipeI => ({
   apiKey: process.env.LANGBASE_API_KEY!,
   name: 'pipe-with-tool',
   description: 'An AI agent pipe that can call tools',
   status: 'public',
   model: 'openai:gpt-4o-mini',
   stream: false,
   json: false,
   store: true,
   moderate: true,
   top_p: 1,
   max_tokens: 1000,
   temperature: 0.7,
   presence_penalty: 1,
   frequency_penalty: 1,
   stop: [],
   tool_choice: 'auto',
   parallel_tool_calls: true,
   messages: [{role: 'system', content: `You are a helpful AI assistant that will work as a calculator.`}],
   variables: [],
   memory: [],
   tools: [ toolCalculator()],
});

export default pipeWithTools;

Step 8: Integrate AI Agent Pipe in Node.js

Now we’ll integrate the AI agent pipe you created into the Node.js project to build an interactive command-line interface (CLI) for the calculator tool. This Node.js project will serve as the base for testing and interacting with the AI agent pipe (in the beginning of the tutorial, we set up a Node.js project by initializing npm).

Now, create an index.ts file:

touch index.ts

In this TypeScript file, import the AI agent pipe you created. We will use the pipe primitive from @baseai/core to run the pipe.

Add the following code to the index.ts file:

import 'dotenv/config';
import { Pipe } from '@baseai/core';
import inquirer from 'inquirer';
import ora from 'ora';
import chalk from 'chalk';
import pipePipeWithTool from './baseai/pipes/pipe-with-tool';

const pipe = new Pipe(pipePipeWithTool());

async function main() {

   const initialSpinner = ora('Conversation with Math agent...').start();
   try {
       const { completion: calculatorTool} = await pipe.run({
           messages: [{ role: 'user', content: 'Hello' }],
       });
       initialSpinner.stop();
       console.log(chalk.cyan('Report Generator Agent response...'));
       console.log(calculatorTool);
   } catch (error) {
       initialSpinner.stop();
       console.error(chalk.red('Error processing initial request:'), error);
   }

   while (true) {
       const { userMsg } = await inquirer.prompt([
           {
               type: 'input',
               name: 'userMsg',
               message: chalk.blue('Enter your query (or type "exit" to quit):'),
           },
       ]);

       if (userMsg.toLowerCase() === 'exit') {
           console.log(chalk.green('Goodbye!'));
           break;
       }

       const spinner = ora('Processing your request...').start();

       try {
           const { completion: reportAgentResponse } = await pipe.run({
               messages: [{ role: 'user', content: userMsg }],
           });

           spinner.stop();
           console.log(chalk.cyan('Agent:'));
           console.log(reportAgentResponse);
       } catch (error) {
           spinner.stop();
           console.error(chalk.red('Error processing your request:'), error);
       }
   }
}

main();

This code creates an interactive CLI for chatting with an AI agent, using a pipe from the @baseai/core library to process user input. Here's what happens:

• It imports necessary libraries such as dotenv for environment configuration, inquirer for user input, ora for loading spinners, and chalk for colored output. Make sure you install these libraries first using this command in your terminal npm install ora inquirer.

• A pipe object is created from the BaseAI library using a predefined tool called pipe-with-tool.

In the main() function:

• A spinner starts while an initial conversation with the AI agent is initiated with the message 'Hello'.

• The response from the AI is displayed.

• A loop runs to continually ask the user for input and send queries to the AI agent.

• The AI's responses are shown, and the process continues until the user types "exit”.

Step 9: Start the BaseAI Server

To run the AI agent pipe locally, you need to start the BaseAI server. Run the following command in your terminal:

npx baseai@latest dev

Step 10: Run the AI Agent Pipe

Run the index.ts file using the following command:

npx tsx index.ts

Result

In your terminal, you’ll be prompted to "Enter your query." For example, let’s ask: "What is 120 cm in feet?" LLMs usually hallucinate when converting to feet. But because of the self-healing tool calling of the BaseAI framework, the tool detects and corrects its own errors.

With this setup, we’ve successfully built an AI agent that uses MathJS and BaseAI tool calls to eliminate the mathematical limitations of LLMs.

Here’s a demo of the end result:

Wrapping Up

As Large Language Models (LLMs) often struggle with mathematical reasoning due to their focus on language, leading to frequent errors in calculations, especially with complex math problems.

AI agents extend LLM capabilities by integrating tool calls. They handle real-time data, ensure more consistent outputs, and reduce hallucination.

By incorporating MathJS and tool calls via the BaseAI framework, developers can create custom serverless AI agents called pipes that serve as reliable calculators and address LLMs' inherent limitations.