Shrijal Acharya - freeCodeCamp.org

Figma MCP vs Kombai: Cloning the Front End from Figma with AI Tools

Shrijal Acharya — Mon, 08 Dec 2025 16:58:51 +0000

Frontend automation is moving fast. Tools like Figma MCP and Kombai can read design context and generate working UI code. I wanted to see what you actually get in practice, so I decided to compare them.

Figma MCP exposes design metadata to AI clients, while Kombai is a frontend-first agent that integrates with editors and existing stacks.

In this article, we’ll feed the same two Figma files into both tools, review how close the output is to the designs, and look at the code structure in a real editor.

What's the Deal?
Meet the Tools
- Kombai
- Figma MCP
Frontend Comparison with Figma
Test 1: Simple Portfolio Design
- Figma MCP
- Kombai
Test 2: Complex Learning Dashboard
- Figma MCP
- Kombai
What You Should Know Before Using These Tools
Final Verdict and What's Next?
Conclusion

What's the Deal?

Cloning complex Figma designs by hand isn’t fun anymore, nor is writing your CSS line by line with exact precision.

And sure, you can attach a screenshot or whatever to GPT, but it often ends up with something that barely looks like your design. That's where Kombai or the Figma MCP come in.

They actually get your Figma design metadata and give you frontend code that's super close to the real thing.

So now, instead of spending hours rebuilding what's already in your design file, you can focus more on small tweaks and what actually matters.

Meet the Tools

Kombai

Kombai is an AI agent designed for frontend work. It takes input from Figma (like text, images, or your existing code), understands your stack, and converts it into clean, production-ready UI.

💡 It’s made specifically for frontend work, so you can expect it to be very good at that (unlike more generic tools like ChatGPT or Claude).

Kombai also handles large repositories easily. It doesn't just convert Figma designs into code. It actually understands your entire frontend codebase, even if it's huge.

So, even if you're working on a small side project or a very large production app, it can read, change, and write code that fits perfectly into your existing project.

Note: Kombai isn’t just good at cloning Figma designs and writing clean code. It actually understands your whole repo, too. You can chat with it like GPT, but it already knows your frontend. It can help refactor code, clean things up, or make changes without ever touching your backend logic.

Pretty handy, right?

No backend code is ever touched, which ensures none of your business logic is mistakenly changed.

You can also add Kombai right inside your editor. It works with VSCode, Cursor, Windsurf, and Trae. Just grab it from the extension marketplace, launch it, and you’re ready to go.

With Kombai, you can:

Turn Figma designs into code (React, HTML, CSS, and so on) using the component library your project already uses.
Work with a frontend-smart engine that understands 30+ libraries including Next.js, MUI, and Chakra UI.
Stay in your editor, follow your own conventions, and ship faster with good accuracy.
And most importantly, preview the changes in a sandbox so you can approve or reject the change before committing it to the files.

You can be up and running in under a minute. Here are the steps to get started:

Install the extension for your editor
Sign in and connect your project
Paste a Figma link or describe what you want to build
Review the output and commit your code

You can find it in the Extension marketplace of your IDE.

Now, using it is just as simple as accessing it from the left sidebar and having a chat similar to how you would with ChatGPT. (Optionally, you can add your tech stack, but Kombai handles it automatically.)

Head to the docs to get started and find the setup for your editor.

Pricing Note: Kombai is a paid tool but gives you a free plan with 300 credits per month, which is great for personal projects. For more advanced workflows, you can move up to the Pro plan or the Enterprise plan.

If you spend most of your time on the frontend, Kombai may be a good fit.

Figma MCP

Figma MCP (Model Context Protocol) lets AI agents connect directly to your Figma files. It closes the gap between your designs and your AI tools by giving them structured access to real design data instead of relying on screenshots or rough estimates.

It works by exposing your design's node tree, styles, layout rules, and component structure so the model can build the UI with actual design data.

That means tools like Claude Code, Gemini CLI, Cursor, and VSCode can actually read your designs, including layers, components, colors, spacing, and text, and use that context to generate accurate, production-ready code or design updates.

With Figma MCP, you can:

Let AI tools pull live data from your Figma files, so your code suggestions always match your latest designs
Ask your AI assistant to inspect components, layouts, or styles directly from Figma
Generate UI code that reflects real design and structure instead of guessing from an image
Keep designers and developers in sync without constantly sending files back and forth.

Setting it up is simple:

Run the Figma MCP server locally
Authorize your Figma workspace
Connect your editor or AI tool (Cursor, Claude Code, Gemini CLI, and so on)

For this test, I'll be using Figma MCP inside Claude Code in Linux, and setting it up is as simple as adding the following JSON in your Claude configuration file ~/.claude.json:

{
  "mcpServers": {
    "Framelink MCP for Figma": {
      "command": "npx",
      "args": ["-y", "figma-developer-mcp", "--figma-api-key=YOUR-KEY", "--stdio"]
    }
  }
}

For Windows users:

{
  "mcpServers": {
    "Framelink MCP for Figma": {
      "command": "cmd",
      "args": ["/c", "npx", "-y", "figma-developer-mcp", "--figma-api-key=YOUR-KEY", "--stdio"]
    }
  }
}

Pricing Note: To use Figma MCP, you need to have a paid Figma plan, either Professional, Organization, or Enterprise. But there's a community-maintained open-source MCP server, Figma-Context-MCP, that you can test out for free – which I'll be using for this test.

Once it’s running, any MCP-supported tool can understand your design files, making frontend coding development much more accurate.

Check the Figma MCP Guide to get started.

Frontend Comparison with Figma

For this test, we'll be comparing Kombai with Figma MCP using two Figma designs: one is a simple portfolio design, and the other is a more complex learner dashboard.

NOTE: For this test with Figma MCP, I'll be using Sonnet 4, which, in my experience, has been the best model for coding the frontend. I've also tested with the recent GPT-5 and Opus 4, but Sonnet 4 seems to be the best for frontend work. If you want to try other models, feel free to do so and see if you notice much difference in the results.

💁 Prompt: Clone this Figma design from this Figma frame link attached. Write clean, maintainable, and responsive code that matches the design closely. Keep components simple, reusable, and production-ready.

Quick note about the videos in the next section: The demo recordings are pretty long because I kept them raw. The idea is to show how the tools behave in real time. If you only care about the final output, feel free to skip to the end of each video.

Test 1: Simple Portfolio Design

Let's start with a simpler design that doesn't have much going on in the UI.

You can find the Figma design template here: Personal Portfolio Template

Figma MCP

Here's the response from Figma MCP:

This is pretty decent. The overall UI looks good, and the colors and fonts are all accurate. The biggest visual issues are with the hero image and a few icon placements, which are a bit off compared to the original Figma file.

The overall implementation took just about 5 minutes of coding and achieved this entire result in one go, as you see in the video demo. The time it takes isn't really dependent on the MCP itself but mostly on the model, so the timings will vary based on the model you choose to work with. The timing is something you can simply ignore here.

The whole page is split into sensible components (Header, Hero, Projects, ProjectCard, Footer) and composed in a clean page.tsx.

export default function Home() {
  return (
    
      
      
        
        
      
      
    
  );
}

That is a nice, readable starting point for a Next app.

You can find the code it generated here.

But here are some issues I noticed right away:

The hero decoration is positioned with pretty brittle absolute values:

This achieves the desired look at one screen size, but it can easily become misaligned when you resize. When compared side by side with the Figma frame, the hero image and yellow shape do not align as they should.

Fixed Header

For a simple portfolio page with a short hero, a fixed header is not always worth the complexity.

The problem here is that since the header is fixed to the top, the rest of the content also starts from the top. On smaller devices, this might cover parts of the content when scrolling.

return (
  
    {/* ... */}
    
    {/* more buttons */}
  
);

This is still a great head start, though it is not quite at the level where I would add it to a production repo without tidying up some of the layout changes.

Kombai

Here's the response from Kombai:

Visually, this one is extremely close to the Figma template. Apart from the hero image being slightly off from the Figma design, I see no other differences. It actually feels like the design is exactly copy-pasted.

Notice that the font, images, and icons are exactly the same, which to me is insane.

You can find the code it generated here.

Here are the specific things it does better in this simple example.

It mirrors the Figma typography and colors as real tokens

Kombai sets up globals.css with Figma-like tokens and even defines utility classes for the text styles:

:root {
  /* ... */
}

@theme inline {
  /* ... */
}

@utility text-heading-large {
  /* ... */
}

@utility text-subtitle {
  /* ... */
}

That is very similar to how a designer would set up styles in Figma, and it means you can reuse these utilities in new screens instead of retyping Tailwind font sizes everywhere.

Components are cleaner and more reusable

All the other components, like Hero or some smaller button components, use the same styles set up in styles.css.

const baseClasses =
  "text-button px-6 py-3 rounded-sm transition-all hover:opacity-90";

const variantClasses =
  variant === "primary"
    ? "bg-(--primary-yellow) text-(--foreground)"
    : "bg-transparent border-2 border-(--foreground) text-(--foreground) hover:bg-(--foreground) hover:text-white";

The footer pulls each icon into its own component:

import InstagramIcon from "./icons/InstagramIcon";
import LinkedInIcon from "./icons/LinkedInIcon";
import MailIcon from "./icons/MailIcon";

In practice, that means if the designer swaps the mail icon or tweaks the size, there is a single place to update it.

So for this simple test, Kombai’s output is both closer to the visual design and a bit nicer structurally for a real project. I would still tweak naming and some minor details, but I would happily keep most of this as is. How crazy is that?

Test 2: Complex Learner Dashboard

So, for the second one, let's create a slightly more complex design with a lot happening in the UI.

You can find the Figma design template here: Learning Dashboard

Figma MCP

Here's the response from Figma MCP:

This is good, considering the complexity of the design. It’s able to put all the images and assets in place. This is much better than what I expected. But there's a slight inconsistency in the placement of images between the original design and the implementation, as you can see for yourself.

If I compare the time, this got it done super fast, in just about 8 minutes, whereas Kombai took over 15 minutes to get it done (but with a better result).

You can find the code it generated here.

Here's what I like and dislike about a few things it did here:

Great smaller components, but everything is still quite page-centric

It does break things into logical components like Sidebar, Input, Button, StatCard, CourseCard, and Icons. The main page then stitches them together:

export default function Home() {
  const mentors = [
    {
      id: 1,
      name: "John Doe",
      subject: "UI/UX Design",
      color: "bg-purple-500",
    },
    // ...
  ];

  return (
    
      {/* Sidebar */}
      

      {/* Main content */}
      
        {/* Search, hero, cards, mentor table */}
      
    
  );
}

The separation into components is nice, but everything is still wired directly inside one big page component with inline mock data. For a real app, I would want that data in its own module, ideally typed, so it is not mixed with layout logic.

Hard-coded dimensions tied to the original frame

The outer container is pinned to a specific height:

That’s fine if you are literally recreating a 1440 by 933 frame for a screenshot, but in a live app, it means:

You get weird empty space on taller screens.
Anything that grows vertically (longer course titles, more mentors) will either overflow or get clipped.

The hero banner has the same kind of pixel-exact positioning:


  
  {/* four more star images with fixed top/left */}

This is great for matching the specific Figma design, but as soon as the width changes, these positions stop lining up perfectly.

So overall, I would call this result surprisingly good for a single prompt, but a bit rigid and template-like once you start thinking about real data and using it in production.

Kombai

Here's the response from Kombai:

You will see in the video that I had to fix a small error with an extra prompt, but after that, it produced a fully working dashboard. The visual match is very strong, given how complex the layout is.

You can find the code it generated here.

Here is what stands out compared to the MCP output.

It treats the Figma file like a real product, not just a static screen.

Instead of wiring everything in a single page with inline arrays, Kombai creates proper domain types and a mock-data.ts:

import { UserProfile, Friend, Course, ProgressCard, Mentor } from "./types";

export const courses: Course[] = [
  {
    id: "1",
    title: "Beginner's Guide to becoming a professional frontend developer",
    category: "Frontend",
    thumbnail: "/images/course-coding.jpg",
    instructor: {
      name: "Prashant Kumar singh",
      role: "software Developer",
      avatar: "/images/avatar-prashant.jpg",
    },
  },
  // ...
];

That looks much closer to what you would expect in a production codebase: clear types, data separated from layout, and a page component that just composes everything.

Better mapping of the smaller UI pieces

The course card is similar to the MCP one, but now it is fully driven by a Course object:

export function CourseCard({ course }: { course: Course }) {
  return (
    
      
        
        
      
        
          {course.category}
        
        
          {course.title}
        
        
          
        
        {/* instructor avatar and name */}
      
    
  );
}

The structure and text styles are very close to the original design, and because the card is fully data-driven, you can plug in real data without touching the JSX.

Design tokens and typography utilities again

Just like in the portfolio example, Kombai sets up a proper token layer for the dashboard:

:root {
  /* ... */
}

@utility heading-section {
  /* ... */
}

@utility text-caption {
  /* ... */
}

The components then reuse these utilities, which keeps the code close to the design system instead of scattering font sizes and colors everywhere.

Things I would still tweak

It is not perfect:

The Next layout.tsx is still using the default Geist fonts and “Create Next App” metadata, so you would want to align that with the Inter font and real app title.
Some of the mock data has inconsistent casing in names and roles, which you would clean up in a real project.
The play button on the course card is just a white dot button for now, so you would still plug in the real icon.

But even with those issues, it is very close to something I would actually keep in a production repo after a quick pass.

Now, this is not as perfect as the previous Kombai implementation, and it did not run into errors. But considering how complex this design is, with multiple different cards with images and all, it's still really impressive to me.

For this one, it took a bit longer to code, but in my opinion, the extra time was worth it.

Imagine you're building something similar and get a response this good already. Then it's not that big of a deal to iterate a little bit, right? You don't have to start from scratch. Just make a few changes if required, and you're done.

What You Should Know Before Using These Tools

As good as these tools are, they’re not something you can just trust blindly. They’ll get you off to a solid start, but you’ll still need to tweak a few things before calling it production-ready.

Kombai does a great job cloning Figma designs and writing clean, modular code. It breaks components into smaller files and generally follows good structure.

The only issue I noticed is that it sometimes slips on naming conventions. Since it scans your entire codebase to stay consistent with your setup, it can be a bit slower to generate code, but that’s also what makes it smarter. You’re not just getting a Figma cloner, you’re getting an assistant that actually understands your frontend.

Figma MCP is fast and does a decent job matching the UI, although the results depend a lot on the model you use for generation. If your main goal is to clone Figma designs quickly and you don’t mind refining the output, it’s a good option.

In short, both tools can save you a ton of time, but they’re not plug-and-play replacements for a frontend workflow. Treat them as part of your toolkit, and you’ll get the best results.

Final Verdict, and What's Next?

Now that you’ve got the gist of what these tools can do, go ahead and try them out. You can turn your Figma designs into working frontends in just a few minutes without all the endless play with CSS.

To sum up, here’s the quick rundown:

If you want production-ready code that actually looks like your Figma design and you mostly live in VS Code, Cursor, or any GUI IDE, go with Kombai. It nails the details and even understands your codebase, which is completely missing in Figma MCP.
If you just want to clone a Figma design quickly and don’t mind if things are slightly off, Figma MCP is totally fine. It gets the job done pretty well.

Basically, choose Kombai if you care about precision and code quality with codebase understanding.

Choose Figma MCP if you want something quick, that works and looks decent enough. 🤷‍♂️

Conclusion

So, what do you think? Pretty cool, right? This was a fun little experiment to see how close tools like Figma MCP and Kombai can get to cloning real frontends straight from Figma.

If you’re into building frontends and want to save yourself a few hours of CSS pain, definitely give them a try. Just don’t expect them to be perfect in one try – their output still needs review and likely a little refining.

That’s all for this one. Thank you for reading! ✌️

How to Build a Google Sheet AI Agent with Composio and Gemini TTS Support

Shrijal Acharya — Fri, 26 Sep 2025 14:21:26 +0000

With the rise of AI agents and agentic systems, we’re no longer just generating text or images, we’re teaching AI how to take actions. Instead of asking, “Can AI write this for me?” you can now ask, “Can AI do this for me?” From updating CRMs to managing tasks, agents can now connect to real tools and get things done.

In this article, you’ll build an AI agent that can talk, think, and even update your Google Sheets using Composio, Next.js, and Gemini TTS.

What's Covered?

In this tutorial, you'll learn how to build your own AI agent for Google Sheets with voice support that can use tools from Composio. You’ll learn these along the way:

What an AI Agent is
How to use Composio to add integrations to your agent.
How to stream responses from a Next.js API route with Vercel AI SDK.
How to work with the Gemini text-to-speech API.

What's Covered?
What’s this Sheet Agent?
How to Set Up the Project
Core Components in the Application
Google Sheet Agent in Action
Conclusion

What’s this Sheet Agent?

First, what is an AI Agent? An AI agent is a system that can act independently to achieve goals. For example, it can book a flight, send an email, or search a database.

Generative AI, like ChatGPT, mainly focuses on creating output such as text, images, or code. An agent is different because it can make decisions, plan, and take actions in the real world, not just generate content.

Large language models (LLMs) often power these agents. The LLM provides reasoning and conversation skills, while the agent layer adds tools that enable it to act beyond simple generation.

So, you might have guessed it. Today, we're building an AI agent that can access real data from Google Sheets and even make changes to it.

How to Set Up the Project

It's fairly simple to get this project up and running. Follow these steps:

First, you need to clone the repository:

git clone https://github.com/shricodev/google-sheet-super-agent.git
cd google-sheet-super-agent

Next, you need to install the dependencies:

npm install

Then set up the environment variables and run the development server:

# API key for Google Gemini (direct access)
GEMINI_API_KEY=

# API key for Composio to access tool integrations (especially Google Sheets)
COMPOSIO_API_KEY=

# Composio user ID (get this from your Composio dashboard after login)
COMPOSIO_GOOGLE_SHEET_USER_ID=

# Auth config ID for Google Sheets inside Composio
GOOGLE_SHEETS_AUTH_CONFIG_ID=

# API key for Google Generative AI SDK (Gemini SDK client)
GOOGLE_GENERATIVE_AI_API_KEY=

# Secret key for signing/encrypting sessions.
# Generate with `openssl rand -base64 32`
SESSION_SECRET=

To get the Composio API key, create an account and log in to the dashboard. You can find the API key in your default project settings.

For the COMPOSIO_GOOGLE_SHEET_USER_ID, you can obtain it after connecting an account in the Google Sheets Auth configuration in Composio.

Core Components in the Application

There are mainly three core logical components in this project:

1. Initiate Connection

This is fairly straightforward. You need to initiate a connection with Composio to use the integrations, which in our case is Google Sheets.

// ...Rest of the code

const connection = await composio.connectedAccounts.initiate(
  userID,
  googleSheetAuthConfigID,
  // Comment this out if you want to allow multiple accounts
  // {
  //   allowMultiple: true,
  // },
);

infoLog(
  "Please visit the following URL to authorize: ",
  connection.redirectUrl ? connection.redirectUrl : "Something went wrong!",
);

2. Set up TTS with Gemini API

For this project, I decided to go with Gemini for TTS generation instead of OpenAI only because they recently (end of August 2025) launched their TTS API.

You can read more about it here: Gemini Speech Generation (text-to-speech).

import { errorLog } from "@/lib/logger";
import { ttsSchema } from "@/lib/validators/tts";
import { GoogleGenAI } from "@google/genai";
import { StatusCodes } from "http-status-codes";
import { NextRequest, NextResponse } from "next/server";
import { Readable } from "stream";
import wav from "wav";

const ai = new GoogleGenAI({
  apiKey: process.env.GEMINI_API_KEY,
});

async function convertL16ToWav(pcmBuffer: Buffer): Promise {
  return new Promise((resolve, reject) => {
    const chunks: Buffer[] = [];

    const writer = new wav.Writer({
      channels: 1,
      sampleRate: 24000,
      bitDepth: 16,
    });

    writer.on("data", (chunk) => {
      chunks.push(chunk);
    });

    writer.on("end", () => {
      resolve(Buffer.concat(chunks));
    });

    writer.on("error", reject);

    const readable = new Readable({
      read() {
        this.push(pcmBuffer);
        this.push(null); // End the stream
      },
    });

    readable.pipe(writer);
  });
}

export async function POST(req: NextRequest) {
  try {
    const body = await req.json();
    const parsedBody = ttsSchema.safeParse(body);

    if (!parsedBody.success) {
      return NextResponse.json(
        {
          error: parsedBody.error.message,
        },
        { status: StatusCodes.BAD_REQUEST },
      );
    }

    const { text } = parsedBody.data;

    const result = await ai.models.generateContent({
      model: "gemini-2.5-flash-preview-tts",
      contents: [{ parts: [{ text: text }] }],
      config: {
        responseModalities: ["AUDIO"],
        speechConfig: {
          voiceConfig: {
            prebuiltVoiceConfig: { voiceName: "Kore" },
          },
        },
      },
    });

    const data = result.candidates?.[0]?.content?.parts?.[0]?.inlineData?.data;
    const mimeType =
      result.candidates?.[0]?.content?.parts?.[0]?.inlineData?.mimeType;

    if (typeof data !== "string") {
      errorLog("Invalid audio data received:", { data, mimeType });
      return NextResponse.json(
        { error: "Audio data is not a string." },
        { status: StatusCodes.INTERNAL_SERVER_ERROR },
      );
    }

    if (!data || data.length === 0) {
      errorLog("Empty audio data received:", { data, mimeType });
      return NextResponse.json(
        { error: "Empty audio data received." },
        { status: StatusCodes.INTERNAL_SERVER_ERROR },
      );
    }

    try {
      const audioBuffer = Buffer.from(data, "base64");

      console.log("Generated audio:", {
        bufferSize: audioBuffer.length,
        contentType: mimeType || "unknown",
        mimeType,
        textLength: text.length,
      });

      // Check if it's L16 PCM format that needs conversion
      if (
        mimeType?.startsWith("audio/L16") ||
        mimeType?.startsWith("audio/l16")
      ) {
        const wavBuffer = await convertL16ToWav(audioBuffer);

        return new NextResponse(new Uint8Array(wavBuffer), {
          headers: {
            "Content-Type": "audio/wav",
            "Content-Length": wavBuffer.length.toString(),
            "Cache-Control": "no-cache",
            "Accept-Ranges": "bytes",
          },
        });
      }

      return new NextResponse(new Uint8Array(audioBuffer), {
        headers: {
          "Content-Type": mimeType || "audio/mpeg",
          "Content-Length": audioBuffer.length.toString(),
          "Cache-Control": "no-cache",
          "Accept-Ranges": "bytes",
        },
      });
    } catch (bufferError) {
      errorLog(bufferError, "API /tts (buffer error)");
      return NextResponse.json(
        { error: "Invalid base64 audio data." },
        { status: StatusCodes.INTERNAL_SERVER_ERROR },
      );
    }
  } catch (error) {
    errorLog(error, "API /tts");
    return NextResponse.json(
      { message: "Error generating audio." },
      { status: 500 },
    );
  }
}

This one's a bit more involved. For some reason, Gemini's API returns the data in the audio/L16 format and not in the mp3 or wav format that we're used to using.

And you can't really play this audio format directly in your browser. So, first, we need to convert it to wav format using the convertL16ToWav function. Then, we can return the wav buffer as the response.

This took me forever to implement. I didn't know there was something like audio/L16 that I couldn't play in my browser. I had to do a lot of googling to figure this one out.

All in all, all it's doing is wrap the raw audio in a WAV file that looks like mono, 24kHz, 16-bit PCM.

And if you want to use the OpenAI package, which is much easier to use as it returns the speech in mp3 format, check out this project of mine: shricodev/voice-chat-ai-agent (TTS).

3. Handle User Queries

This is the last piece of the puzzle. Here's where the actual tool call logic happens.

import { google } from "@ai-sdk/google";
import { streamText } from "ai";
import { Composio } from "@composio/core";
import { NextResponse } from "next/server";
import { chatSchema } from "@/lib/validators/chat";
import { StatusCodes } from "http-status-codes";
import { errorLog } from "@/lib/logger";
import { VercelProvider } from "@composio/vercel";

// ...Rest of the code

const tools = await composio.tools.get(userID, {
  toolkits: ["GOOGLESHEETS"],
});

let conversationContext = "";
if (conversationHistory && conversationHistory.length > 0) {
  conversationContext = conversationHistory
    .map((conversation) => {
      return `${conversation.role}: ${conversation.content}`;
    })
    .join("\n");
}

const systemPrompt = `
You are an intelligent Google Sheets assistant. You can help users analyze, query, and manipulate data in their Google Sheets.

Sheet ID: ${sheetID}
User ID: ${userID}

Guidelines:
- Always use the Google Sheets tools to access real data from the spreadsheet
- Provide clear, actionable insights based on the actual data
- If you need to read data, use the appropriate Google Sheets tools first
- Format your responses in a clear, professional manner
- If asked about calculations, use the actual data from the sheet

Always generate a short summary of what you got done. like if the user asked
you to make changes, then write in short about what all changes you did. If
they asked you to summarize the data, then write in short about what the data
is all about.

---

Previous conversation in this document:

${conversationContext}
`;

const result = streamText({
  model: google("gemini-2.5-pro"),
  system: systemPrompt,
  prompt,
  tools: tools,
  toolChoice: "auto",
});

return result.toUIMessageStreamResponse({ sendReasoning: true });

This code lives in the Next.js app router. First, we fetch the tools from Composio using the composio.tools.get function. We use auto as the tool choice, which means that the agent will use the tools it has the most confidence in.

Then, we create the system prompt that will guide the agent on how to behave.

Finally, we call the streamText function, which streams the response instead of waiting for the entire response before sending it to the client, passing in the tools, system prompt, and the model to use. Then, we send the response in the UIMessageStreamResponse format so it can be easily displayed on the UI.

Google Sheet Agent in Action

Here's a quick demo of the agent in action:

Conclusion

So, what do you think of the project so far? This was a really fun project for me to work on.

Go ahead, clone the repository, and give it a try with your Google Sheet. Even after all of this, it is a fairly small project with super simple logic, which I believe you've already understood completely.

Do I suggest you use it on an important Google Sheet? Not at all. Remember, it's just an AI model that can access tools from Composio. You can never be 100% sure with AI. While building this project, I did run into cases where the AI picked the wrong tools and even messed up the sheet entirely. But, you can always try it on a not-so-important sheet to see how it all works.

You can find the entire source code here: shricodev/google-sheet-super-agent.

How to Build a LangGraph and Composio-Powered Discord Bot

Shrijal Acharya — Tue, 24 Jun 2025 21:03:03 +0000

With the rise of AI tools over the past couple years, most of us are learning how to use them in our projects. And in this article, I’ll teach you how to build a quick Discord bot with LangGraph and Composio.

You’ll use LangGraph nodes to build a branching flow that processes incoming messages and detects intent like chat, support, or tool usage. It’ll then route them to the right logic based on what the user says.

I know it may sound a bit weird to use LangGraph for a Discord bot, but you’ll soon see that this project is a pretty fun way to visualize how node-based AI workflows actually run.

For now, the workflow is simple: you’ll figure out if the user is just chatting, asking a support question, or requesting that the bot perform an action, and respond based on that.

What you will learn: 👀

How to use LangGraph to create an AI-driven workflow that powers your bot’s logic.
How you can integrate Composio to let your bot take real-world actions using external tools.
How you can use Discord.js and handle different message types like replies, threads, and embeds.
How you can maintain per-channel context using message history and pass it into AI.

By the end of this article, you’ll have a quite decent and functional Discord bot that you can add to your server. It replies to users based on message context and even has tool-calling support! (And there’s a small challenge for you to implement something yourself.) 😉

Prerequisites

Make sure you have Discord installed on your machine so you can test the bot easily.

This project is designed to demonstrate how you can build a bot powered by LangGraph and Composio. Before proceeding, it is helpful to have a basic understanding of:

How to work with Node.js
Rough idea of what LangGraph is and how it works
How to work with Discord.js
What AI Agents are

If you’re not confident about any of these, try following along anyway. You might pick things up just fine. And if it ever gets confusing, you can always check out the full source code here.

How to Set Up the Environment
Build the Application Logic
Wrapping Up

How to Set Up the Environment

In this section, we will get everything set up for building the project.

Initialize the Project

Initialize a Node.js application with the following command:

💁 Here I'm using Bun, but you can choose any package manager of your choice.

mkdir discord-bot-langgraph && cd discord-bot-langgraph \
&& bun init -y

Now, that our Node.js application is ready, let's install some dependencies.

Install Dependencies

We'll be using the following main packages and some other helper packages:

discord.js: Interacts with the Discord API
composio: Adds tools integration support to the bot
openai: Enables AI-powered responses
langchain: Manages LLM workflows
zod: Validates and parses data safely

bun add discord.js openai @langchain/core @langchain/langgraph \
langchain composio-core dotenv zod uuid

Configure Composio

💁 You’ll use Composio to add integrations to your application. You can choose the integration of your choice, but here I'm using Google sheets.

First, before moving forward, you need to get access to a Composio API key.

Go ahead and create an account on Composio, get your API key, and paste it in the .env file in the root of the project:

COMPOSIO_API_KEY=

Authenticate yourself with the following command:

composio login

Once that’s done, run the composio whoami command, and if you see something like the below, you’re successfully logged in.

You're almost there: now you just need to set up integrations. Here, I’ll use Google sheets, but again you can set up any integration you like.

Run the following command to set up the Google Sheets integration:

composio add googlesheets

You should see an output similar to this:

Head over to the URL that’s shown, and you should be authenticated like so:

That's it. You’ve successfully added the Google Sheets integration and can access all its tools in your application.

Once finished, run the composio integrations command to verify if it worked. You should see a list of all your integrations:

Configure Discord Integration

This is a bit off topic for this tutorial, but basically, you’ll create an application/bot on Discord and add it to your server.

You can find a guide on how to create and add a bot to your server in the Discord.js documentation.

And yes, it’s free if you’re wondering whether any step here requires a pro account or anything. 😉

Make sure you populate these three environment variables:

DISCORD_BOT_TOKEN=
DISCORD_BOT_GUILD_ID=
DISCORD_BOT_CHANNEL_ID=

Add Environment Variables

You’ll require a few other environment variables, including the OpenAI API key, for the bot to work.

Your final .env file should look something like this:

OPENAI_API_KEY=

COMPOSIO_API_KEY=

DISCORD_BOT_TOKEN=
DISCORD_BOT_GUILD_ID=
DISCORD_BOT_CHANNEL_ID=

Build the Application Logic

Now that you’ve laid all the groundwork, you can finally start coding the project.

Define Types and Utility Helpers

Let’s start by writing some helper functions and defining the types of data you’ll be working with.

It's important in any application, especially ones like the one we're building – which is prone to errors due to multiple API calls – that we set up decent logging so we know when and how things go wrong.

Create a new file named logger.ts inside the utils directory and add the following lines of code:

// 👇 discord-bot-langgraph/utils/logger.ts

export const DEBUG = "DEBUG";
export const INFO = "INFO";
export const WARN = "WARN";
export const ERROR = "ERROR";

export type LogLevel = typeof DEBUG | typeof INFO | typeof WARN | typeof ERROR;

// eslint-disable-next-line  @typescript-eslint/no-explicit-any
export function log(level: LogLevel, message: string, ...data: any[]) {
  const timestamp = new Date().toLocaleString();
  const prefix = `[${timestamp}] [${level}]`;

  switch (level) {
    case ERROR:
      console.error(prefix, message, ...data);
      break;
    case WARN:
      console.warn(prefix, message, ...data);
      break;
    default:
      console.log(prefix, message, ...data);
  }
}

This is already looking great. Why not write a small environment variables validator? Run this during the initial program startup, and if something goes wrong, the application will exit with clear logs so users know if any environment variables are missing.

Create a new file named env-validator.ts in the utils directory and add the following lines of code:

// 👇 discord-bot-langgraph/utils/env-validator.ts

import { log, ERROR } from "./logger.js";

export const OPENAI_API_KEY = "OPENAI_API_KEY";

export const DISCORD_BOT_TOKEN = "DISCORD_BOT_TOKEN";
export const DISCORD_BOT_GUILD_ID = "DISCORD_BOT_GUILD_ID";
export const DISCORD_BOT_CLIENT_ID = "DISCORD_BOT_CLIENT_ID";

export const COMPOSIO_API_KEY = "COMPOSIO_API_KEY";

export const validateEnvVars = (requiredEnvVars: string[]): void => {
  const missingVars: string[] = [];

  for (const envVar of requiredEnvVars) {
    if (!process.env[envVar]) {
      missingVars.push(envVar);
    }
  }

  if (missingVars.length > 0) {
    log(
      ERROR,
      "missing required environment variables. please create a .env file and add the following:",
    );
    missingVars.forEach((envVar) => console.error(`- ${envVar}`));
    process.exit(1);
  }
};

Now, let's also define the type of data you'll be working with:

Create a new file named types.ts inside the types directory and add the following lines of code:

// 👇 discord-bot-langgraph/types/types.ts

export const QUESTION = "QUESTION";
export const HELP = "HELP";
export const SUPPORT = "SUPPORT";
export const OTHER = "OTHER";
export const TOOL_CALL_REQUEST = "TOOL_CALL_REQUEST";

export type FinalAction =
  | { type: "REPLY"; content: string }
  | { type: "REPLY_IN_THREAD"; content: string }
  | {
      type: "CREATE_EMBED";
      title: string;
      description: string;
      roleToPing?: string;
    };

export type MessageChoice =
  | typeof SUPPORT
  | typeof OTHER
  | typeof TOOL_CALL_REQUEST;

export type SupportTicketType = typeof QUESTION | typeof HELP;

export type Message = {
  author: string;
  content: string;
};

export type SupportTicketQuestion = {
  description: string;
  answer: string;
};

export type SupportTicket = {
  type?: SupportTicketType;
  question?: SupportTicketQuestion;
};

export type ToolCallRequestAction = {
  // actionLog is not intended to be shown to the end-user.
  // This is solely for logging purpose.
  actionLog: string;
  status: "success" | "failed" | "acknowledged";
};

The types are pretty self-explanatory, but here’s a quick overview.

Message holds the user's input and author. Each message can be marked as support, a tool call request, or just other, like spam or small talk.

Support messages are further labeled as either help or a question using SupportTicketType.

The graph returns a FinalAction, which can be a direct reply, a reply in a thread, or an embed. If it's CREATE_EMBED and has roleToPing set, it denotes support help, so we can ping the mod.

For tool-based responses, ToolCallRequestAction stores the status and an internal log used for debugging.

Now, you need one last helper function to use in your nodes to extract the response from the LLM. Create a new file named helpers.ts and add the following code:

// 👇 discord-bot-langgraph/utils/helpers.ts

import type { AIMessage } from "@langchain/core/messages";

export function extractStringFromAIMessage(
  message: AIMessage,
  fallback: string = "No valid response generated by the LLM.",
): string {
  if (typeof message.content === "string") {
    return message.content;
  }

  if (Array.isArray(message.content)) {
    const textContent = message.content
      .map((item) => (typeof item === "string" ? item : ""))
      .join(" ");
    return textContent.trim() || fallback;
  }

  return fallback;
}

You're all set for now with these helper functions in place. Now, you can start coding the logic.

Implement LangGraph Workflow

Now that you have the types defined, structure your graph and connect it with some edges.

Create a new file named graph.ts inside the src directory and add the following lines of code:

// 👇 discord-bot-langgraph/src/graph.ts

import { Annotation, END, START, StateGraph } from "@langchain/langgraph";
import {
  type FinalAction,
  type ToolCallRequestAction,
  type Message,
  type MessageChoice,
  type SupportTicket,
} from "../types/types.js";
import {
  processToolCall,
  processMessage,
  processOther,
  processSupport,
  processSupportHelp,
  processSupportQuestion,
} from "./nodes.js";
import { processMessageEdges, processSupportEdges } from "./edges.js";

const state = Annotation.Root({
  message: Annotation(),
  previousMessages: Annotation(),
  messageChoice: Annotation(),
  supportTicket: Annotation(),
  toolCallRequest: Annotation(),
  finalAction: Annotation(),
});

export type State = typeof state.State;
export type Update = typeof state.Update;

export function initializeGraph() {
  const workflow = new StateGraph(state);

  workflow
    .addNode("process-message", processMessage)
    .addNode("process-support", processSupport)
    .addNode("process-other", processOther)

    .addNode("process-support-question", processSupportQuestion)
    .addNode("process-support-help", processSupportHelp)
    .addNode("process-tool-call", processToolCall)

    // Edges setup starts here....
    .addEdge(START, "process-message")

    .addConditionalEdges("process-message", processMessageEdges)
    .addConditionalEdges("process-support", processSupportEdges)

    .addEdge("process-other", END)
    .addEdge("process-support-question", END)
    .addEdge("process-support-help", END)
    .addEdge("process-tool-call", END);

  const graph = workflow.compile();

  // To get the graph in png
  // getGraph() is deprecated though
  // Bun.write("graph/graph.png", await graph.getGraph().drawMermaidPng());

  return graph;
}

The initializeGraph function, as the name suggests, returns the graph you can use to execute the workflow.

The process-message node is the starting point of the graph. It takes in the user’s message, processes it, and routes it to the appropriate next node: process-support, process-tool-call, or process-other.

The process-support node further classifies the support message and decides whether it should go to process-support-help or process-support-question.

The process-tool-call node handles messages when the user tries to trigger some kind of tool or action.

The process-other node handles everything that doesn’t fall into the support or tool call categories. These are general or fallback responses.

To help you visualize how things will shape up, here’s how the graph looks with all the different nodes (yet to work on!):

To wire everything together, you need to define edges between nodes, including conditional edges that dynamically decide the next step based on the state.

Create a new file named edges.ts inside the src directory and add the following lines of code:

// 👇 discord-bot-langgraph/src/edges.ts

import { END } from "@langchain/langgraph";
import { type State } from "./graph.js";
import { QUESTION, OTHER, SUPPORT, TOOL_CALL_REQUEST } from "../types/types.js";
import { log, WARN } from "../utils/logger.js";

export const processMessageEdges = (
  state: State,
): "process-support" | "process-other" | "process-tool-call" | "__end__" => {
  if (!state.messageChoice) {
    log(WARN, "state.messageChoice is undefined. Returning...");
    return END;
  }

  switch (state.messageChoice) {
    case SUPPORT:
      return "process-support";
    case TOOL_CALL_REQUEST:
      return "process-tool-call";
    case OTHER:
      return "process-other";
    default:
      log(WARN, "unknown message choice. Returning...");
      return END;
  }
};

export const processSupportEdges = (
  state: State,
): "process-support-question" | "process-support-help" | "__end__" => {
  if (!state.supportTicket?.type) {
    log(WARN, "state.supportTicket.type is undefined. Returning...");
    return END;
  }

  return state.supportTicket.type === QUESTION
    ? "process-support-question"
    : "process-support-help";
};

These are the edges that connect different nodes in your application. They direct the flow in your graph.

Things are really shaping up – so let’s finish the core logic by implementing all the nodes for your application.

Create a new file named nodes.ts inside the src directory and add the following lines of code:

// 👇 discord-bot-langgraph/src/nodes.ts

import { type State, type Update } from "./graph.js";
import { ChatOpenAI } from "@langchain/openai";
import { z } from "zod";
import {
  HELP,
  TOOL_CALL_REQUEST,
  OTHER,
  QUESTION,
  SUPPORT,
} from "../types/types.js";
import { extractStringFromAIMessage } from "../utils/helpers.js";
import { OpenAIToolSet } from "composio-core";
import type { ChatCompletionMessageToolCall } from "openai/resources/chat/completions.mjs";
import { v4 as uuidv4 } from "uuid";
import { DEBUG, ERROR, INFO, log, WARN } from "../utils/logger.js";
import {
  SystemMessage,
  HumanMessage,
  ToolMessage,
  BaseMessage,
} from "@langchain/core/messages";

// feel free to use any model. Here I'm going with gpt-4o-mini
const model = "gpt-4o-mini";

const toolset = new OpenAIToolSet();
const llm = new ChatOpenAI({
  model,
  apiKey: process.env.OPENAI_API_KEY,
  temperature: 0,
});

export const processMessage = async (state: State): Promise => {
  log(DEBUG, "message in process message:", state.message);

  const llm = new ChatOpenAI({
    model,
    apiKey: process.env.OPENAI_API_KEY,
    temperature: 0,
  });

  const structuredLlm = llm.withStructuredOutput(
    z.object({
      type: z.enum([SUPPORT, OTHER, TOOL_CALL_REQUEST]).describe(`
Categorize the user's message:
- ${SUPPORT}: Technical support, help with problems, or questions about AI.
- ${TOOL_CALL_REQUEST}: User asks the bot to perform tool action (e.g., "send an email", "summarize chat", "summarize google sheets").
- ${OTHER}: General conversation, spam, or off-topic messages.
`),
    }),
  );

  const res = await structuredLlm.invoke([
    [
      "system",
      `You are an expert message analyzer AI. You need to categorize the message into
one of these categories:

- ${SUPPORT}: If the message asks for technical support, help with a problem, or questions about AIs and LLMs.
- ${TOOL_CALL_REQUEST}: If the message is a direct command or request for the bot to perform an action using external tools/services. Examples: "Summarize a document or Google Sheet", "Summarize the last hour of chat", "Send an email to devteam about this bug", "Create a Trello card for this feature request". Prioritize this if the user is asking the bot to *do* something beyond just answering.
- ${OTHER}: For general chit-chat, spam, off-topic messages, or anything not fitting ${SUPPORT} or ${TOOL_CALL_REQUEST}.
`,
    ],
    ["human", state.message.content],
  ]);

  return {
    messageChoice: res.type,
  };
};

export const processSupport = async (state: State): Promise => {
  log(DEBUG, "message in support:", state.message);

  const llm = new ChatOpenAI({
    model,
    apiKey: process.env.OPENAI_API_KEY,
    temperature: 0,
  });

  const structuredLlm = llm.withStructuredOutput(
    z.object({
      type: z.enum([QUESTION, HELP]).describe(`
Type of support needed:
- ${QUESTION}: User asks a specific question seeking information or an answer.
- ${HELP}: User needs broader assistance, guidance, or reports an issue requiring intervention/troubleshooting.
`),
    }),
  );

  const res = await structuredLlm.invoke([
    [
      "system",
      `
You are a support ticket analyzer. Given a support message, categorize it as ${QUESTION} or ${HELP}.
- ${QUESTION}: For specific questions.
- ${HELP}: For requests for assistance, troubleshooting, or problem reports.
`,
    ],
    ["human", state.message.content],
  ]);

  return {
    supportTicket: {
      ...state.supportTicket,
      type: res.type,
    },
  };
};

export const processSupportHelp = async (state: State): Promise => {
  log(DEBUG, "message in support help:", state.message);

  return {
    supportTicket: {
      ...state.supportTicket,
    },
    finalAction: {
      type: "CREATE_EMBED",
      title: "🚨 Help Needed!",
      description: `A new request for help has been raised by **@${state.message.author}**.\n\n**Query:**\n> ${state.message.content}`,
      roleToPing: process.env.DISCORD_SUPPORT_MOD_ID,
    },
  };
};

export const processSupportQuestion = async (state: State): Promise => {
  log(DEBUG, "message in support question category:", state.message);

  const llm = new ChatOpenAI({
    model,
    apiKey: process.env.OPENAI_API_KEY,
    temperature: 0,
  });

  const systemPrompt = `
You are a helpful AI assistant specializing in AI, and LLMs. Answer
the user's question concisely and accurately based on general knowledge in
these areas. If the question is outside this scope (e.g., personal advice,
non-technical topics), politely state you cannot answer. User's question:
`;

  const res = await llm.invoke([
    ["system", systemPrompt],
    ["human", state.message.content],
  ]);

  const llmResponse = extractStringFromAIMessage(res);
  return {
    supportTicket: {
      ...state.supportTicket,
      question: {
        description: state.message.content,
        answer: llmResponse,
      },
    },
    finalAction: {
      type: "REPLY",
      content: llmResponse,
    },
  };
};

export const processOther = async (state: State): Promise => {
  log(DEBUG, "message in other category:", state.message);

  const response =
    "This seems to be a general message. I'm here to help with technical support or perform specific actions if you ask. How can I assist you with those?";

  return {
    finalAction: {
      type: "REPLY_IN_THREAD",
      content: response,
    },
  };
};

There’s not much to explain for these nodes. Each node in the flow functions as a message classifier. It spins up a Chat LLM instance and uses structured output to ensure the model returns a specific label from a predefined set like QUESTION or HELP for support messages. The system prompt clearly defines what each label means, and your user message is passed in for classification.

You’re almost there. But there’s one piece missing. Can you spot it?

The process-tool-call node that’s supposed to handle the workflow when the user asks to use a tool. This is a big piece of the workflow.

It’s a bit longer, so I’ll explain it separately.

Modify the above nodes.ts file to add the missing node:

// 👇 discord-bot-langgraph/src/nodes.ts

// Rest of the code...
export const processToolCall = async (state: State): Promise => {
  log(DEBUG, "message in tool call request category:", state.message);

  const structuredOutputType = z.object({
    service: z
      .string()
      .describe("The target service (e.g., 'email', 'discord')."),
    task: z
      .string()
      .describe(
        "A concise description of the task (e.g., 'send email to X', 'summarize recent chat', 'create task Y').",
      ),
    details: z
      .string()
      .optional()
      .describe(
        "Any specific details or parameters extracted from the message relevant to the task.",
      ),
  });

  const structuredLlm = llm.withStructuredOutput(structuredOutputType);

  let parsedActionDetails: z.infer<typeof structuredOutputType> = {
    service: "unknown",
    task: "perform a requested action",
  };

  try {
    const res = await structuredLlm.invoke([
      [
        "system",
        `Parse the user's request to identify an action. Extract the target service, a description of the task, and any relevant details or parameters.
      Examples:
      - "Remind me to check emails at 5 PM": service: calendar/reminder, task: set reminder, details: check emails at 5 PM
      - "Send a summary of this conversation to #general channel": service: discord, task: send summary to channel, details: channel #general
      - "Create a bug report for 'login fails on mobile'": service: project_manager, task: create bug report, details: title 'login fails on mobile'`,
      ],
      ["human", state.message.content],
    ]);

    parsedActionDetails = res;
    log(INFO, "initial parsing action details:", parsedActionDetails);
  } catch (error) {
    log(ERROR, "initial parsing error:", error);
    return {
      toolCallRequest: {
        actionLog: `Failed to parse user request: ${state.message.content}`,
        status: "failed",
      },
      finalAction: {
        type: "REPLY_IN_THREAD",
        content:
          "I'm sorry, I had trouble understanding that action. Could you please rephrase it?",
      },
    };
  }

  try {
    log(INFO, "fetching composio tools");
    const tools = await toolset.getTools({
      apps: ["GOOGLESHEETS"],
    });

    log(INFO, `fetched ${tools.length} tools. Errors if > 128 for OpenAI:`);

    if (tools.length === 0) {
      log(WARN, "no tools fetched from Composio. skipping...");
      return {
        toolCallRequest: {
          actionLog: `Service: ${parsedActionDetails.service}, Task: ${parsedActionDetails.task}. No composio tools found`,
          status: "failed",
        },
        finalAction: {
          type: "REPLY_IN_THREAD",
          content: "Couldn't find any tools to perform your action.",
        },
      };
    }

    log(DEBUG, "starting iterative tool execution loop");

    const conversationHistory: BaseMessage[] = [
      new SystemMessage(
        "You are a helpful assistant that performs tool calls. Your task is to understand the user's request and use the available tools to fulfill the request completely. You can use multiple tools in sequence to accomplish complex tasks. Always provide a brief, conversational summary of what you accomplished after using tools.",
      ),
      new HumanMessage(state.message.content),
    ];

    let totalToolsUsed = 0;
    let finalResponse: string | null = null;

    const maxIterations = 5;
    let iteration = 0;

    while (iteration < maxIterations) {
      iteration++;
      log(
        DEBUG,
        `Iteration ${iteration}: calling LLM with ${tools.length} tools`,
      );

      const llmResponse = await llm.invoke(conversationHistory, {
        tools: tools,
      });

      log(DEBUG, `Iteration ${iteration} LLM response:`, llmResponse);

      const toolCalls = llmResponse.tool_calls;

      if ((!toolCalls || toolCalls.length === 0) && llmResponse.content) {
        finalResponse =
          typeof llmResponse.content === "string"
            ? llmResponse.content
            : JSON.stringify(llmResponse.content);
        log(
          INFO,
          `Final response received after ${iteration} iterations:`,
          finalResponse,
        );
        break;
      }

      if (toolCalls && toolCalls.length > 0) {
        log(
          INFO,
          `Iteration ${iteration}: executing ${toolCalls.length} tool(s)`,
        );
        totalToolsUsed += toolCalls.length;

        conversationHistory.push(llmResponse);

        for (const toolCall of toolCalls) {
          log(
            INFO,
            `Executing tool: ${toolCall.name} with args:`,
            toolCall.args,
          );

          const composioCompatibleToolCall: ChatCompletionMessageToolCall = {
            id: toolCall.id || uuidv4(),
            type: "function",
            function: {
              name: toolCall.name,
              arguments: JSON.stringify(toolCall.args),
            },
          };

          let toolOutputContent: string;
          try {
            const executionResult = await toolset.executeToolCall(
              composioCompatibleToolCall,
            );
            log(
              INFO,
              `Tool ${toolCall.name} execution result:`,
              executionResult,
            );
            toolOutputContent = JSON.stringify(executionResult);
          } catch (toolError) {
            log(ERROR, `Tool ${toolCall.name} execution error:`, toolError);
            const errorMessage =
              toolError instanceof Error
                ? toolError.message
                : String(toolError);

            toolOutputContent = `Error: ${errorMessage}`;
          }

          conversationHistory.push(
            new ToolMessage({
              content: toolOutputContent,
              tool_call_id: toolCall.id || uuidv4(),
            }),
          );
        }

        continue;
      }

      log(
        WARN,
        `Iteration ${iteration}: LLM provided no tool calls or content`,
      );
      break;
    }

    let userFriendlyResponse: string;

    if (totalToolsUsed > 0) {
      log(DEBUG, "Generating user-friendly summary using LLM");

      try {
        const summaryResponse = await llm.invoke([
          new SystemMessage(
            "You are tasked with creating a brief, friendly summary for a Discord user about what actions were just completed. Keep it conversational, under 2-3 sentences, and focus on what was accomplished rather than technical details. Start with phrases like 'Done!', 'Successfully completed', 'All set!', etc.",
          ),
          new HumanMessage(
            `The user requested: "${state.message.content}"

I used ${totalToolsUsed} tools across ${iteration} iterations to complete their request. ${finalResponse ? `My final response was: ${finalResponse}` : "The task was completed successfully."}

Generate a brief, friendly summary of what was accomplished.`,
          ),
        ]);

        userFriendlyResponse =
          typeof summaryResponse.content === "string"
            ? summaryResponse.content
            : `Done! I've completed your request using ${totalToolsUsed} action${totalToolsUsed > 1 ? "s" : ""}.`;

        log(INFO, "Generated user-friendly summary:", userFriendlyResponse);
      } catch (summaryError) {
        log(ERROR, "Failed to generate summary:", summaryError);
        userFriendlyResponse = `All set! I've completed your request using ${totalToolsUsed} action${totalToolsUsed > 1 ? "s" : ""}.`;
      }
    } else {
      userFriendlyResponse =
        finalResponse ||
        `I understood your request about '${parsedActionDetails.task}' but couldn't find the right tools to complete it.`;
    }

    const actionLog = `Service: ${parsedActionDetails.service}, Task: ${parsedActionDetails.task}. Used ${totalToolsUsed} tools across ${iteration} iterations.`;

    return {
      toolCallRequest: {
        actionLog,
        status: totalToolsUsed > 0 ? "success" : "acknowledged",
      },
      finalAction: {
        type: "REPLY_IN_THREAD",
        content: userFriendlyResponse,
      },
    };
  } catch (error) {
    log(ERROR, "processing tool call with Composio:", error);
    const errorMessage = error instanceof Error ? error.message : String(error);

    return {
      toolCallRequest: {
        actionLog: `Error during tool call (Service: ${parsedActionDetails.service}, Task: ${parsedActionDetails.task}). Error: ${errorMessage}`,
        status: "failed",
      },
      finalAction: {
        type: "REPLY_IN_THREAD",
        content: "Sorry, I encountered an error while processing your request.",
      },
    };
  }
};

The part up until the first try-catch block is the same. Up until then, you're figuring out the tool the user is trying to call. Now comes the juicy part: actually handling tool calls.

At this point, you need to fetch the tools from Composio. Here, I’m just passing in Google Sheets as the option for demo purposes, but you could use literally anything once you authenticate yourself as shown above.

After fetching the tools, you enter a loop where the LLM can use them. It reviews the conversation history and decides which tools to call. You execute these calls, feed the results back, and repeat for up to 5 iterations or until the LLM gives a final answer.

This loop runs up to 5 times as a safeguard so the LLM doesn’t get stuck in an endless back-and-forth.

If tools were used, you ask the LLM to write a friendly summary for the user instead of dumping the raw JSON response. If no tools worked or none matched, just let the user know you couldn’t perform the action.

Now with that, you’re done with the difficult part (I mean, it was pretty easy though, right?). From here on, you just need to set up and work with the Discord API using Discord.js.

Set Up Discord.js Client

In this application, you’re using slash commands. To use slash commands in Discord, you need to register them first. You can do this manually, but why not automate it as well? 😉

Create a new file named slash-deploy.ts inside the utils directory and add the following lines of code:

// 👇 discord-bot-langgraph/utils/slash-deploy.ts

import { REST, Routes } from "discord.js";
import dotenv from "dotenv";
import { log, INFO, ERROR } from "./logger.js";
import {
  DISCORD_BOT_TOKEN,
  DISCORD_BOT_GUILD_ID,
  OPENAI_API_KEY,
  DISCORD_BOT_CLIENT_ID,
  validateEnvVars,
} from "./env-validator.js";

dotenv.config();

const requiredEnvVars = [
  DISCORD_BOT_TOKEN,
  DISCORD_BOT_GUILD_ID,
  DISCORD_BOT_CLIENT_ID,
  OPENAI_API_KEY,
];
validateEnvVars(requiredEnvVars);

const commands = [
  {
    name: "ask",
    description: "Ask the AI assistant a question or give it a command.",
    options: [
      {
        name: "prompt",
        type: 3,
        description: "Your question or command for the bot",
        required: true,
      },
    ],
  },
];

const rest = new REST({ version: "10" }).setToken(
  process.env.DISCORD_BOT_TOKEN!,
);

(async () => {
  try {
    log(INFO, "deploying slash(/) commands");
    await rest.put(
      Routes.applicationGuildCommands(
        process.env.DISCORD_BOT_CLIENT_ID!,
        process.env.DISCORD_BOT_GUILD_ID!,
      ),
      {
        body: commands,
      },
    );

    log(INFO, "slash(/) commands deployed");
  } catch (error) {
    log(ERROR, "deploying slash(/) commands:", error);
  }
})();

See your validateEnvVars function in action? Here, you’re specifying the environment variables that must be set before running the program. If any are missing and you try to run the program, you’ll get an error.

The way you deploy the slash commands to Discord is using the REST API provided by discord.js, specifically by calling rest.put with your command data and target guild.

Now, simply run the commands:deploy bun script and you should have /ask registered as a slash command in your Discord.

At this point, you should see the /ask slash command available in your server. All that’s left is to create the index.ts file, which will be the entry point to your Discord bot.

Create a new file named index.ts inside the src directory and add the following lines of code:

// 👇 discord-bot-langgraph/src/index.ts

import dotenv from "dotenv";
import {
  Client,
  Events,
  GatewayIntentBits,
  EmbedBuilder,
  type Interaction,
} from "discord.js";
import { initializeGraph } from "./graph.js";
import { type Message as ChatMessage } from "../types/types.js";
import { ERROR, INFO, log } from "../utils/logger.js";
import {
  DISCORD_BOT_TOKEN,
  DISCORD_BOT_GUILD_ID,
  OPENAI_API_KEY,
  validateEnvVars,
  DISCORD_BOT_CLIENT_ID,
  COMPOSIO_API_KEY,
} from "../utils/env-validator.js";

dotenv.config();

const requiredEnvVars = [
  DISCORD_BOT_CLIENT_ID,
  DISCORD_BOT_TOKEN,
  DISCORD_BOT_GUILD_ID,

  OPENAI_API_KEY,

  COMPOSIO_API_KEY,
];
validateEnvVars(requiredEnvVars);

const graph = initializeGraph();

const client = new Client({
  intents: [
    GatewayIntentBits.Guilds,
    GatewayIntentBits.GuildMessages,
    GatewayIntentBits.MessageContent,
  ],
});

// use a map to store history per channel to make it work properly with all the
// channels and not for one specific channel.
const channelHistories = new Map<string, ChatMessage[]>();

client.on(Events.ClientReady, async (readyClient) => {
  log(INFO, `logged in as ${readyClient.user.tag}. ready to process commands!`);
});

client.on(Events.InteractionCreate, async (interaction: Interaction) => {
  if (!interaction.isChatInputCommand()) return;
  if (interaction.commandName !== "ask") return;

  const userPrompt = interaction.options.getString("prompt", true);
  const user = interaction.user;
  const channelId = interaction.channelId;

  if (!channelHistories.has(channelId)) channelHistories.set(channelId, []);

  const messageHistory = channelHistories.get(channelId)!;

  const currentUserMessage: ChatMessage = {
    author: user.username,
    content: userPrompt,
  };

  const graphInput = {
    message: currentUserMessage,
    previousMessages: [...messageHistory],
  };

  messageHistory.push(currentUserMessage);
  if (messageHistory.length > 20) messageHistory.shift();

  try {
    await interaction.reply({
      content: "Hmm... processing your request! 🐀",
    });

    const finalState = await graph.invoke(graphInput);

    if (!finalState.finalAction) {
      log(ERROR, "no final action found");
      await interaction.editReply({
        content: "I'm sorry, I couldn't process your request.",
      });
      return;
    }

    const userPing = `<@${user.id}>`;
    const action = finalState.finalAction;

    const quotedPrompt = `🗣️ "${userPrompt}"`;

    switch (action.type) {
      case "REPLY":
        await interaction.editReply({
          content: `${userPing}\n\n${quotedPrompt}\n\n${action.content}`,
        });
        break;

      case "REPLY_IN_THREAD":
        if (!interaction.channel || !("threads" in interaction.channel)) {
          await interaction.editReply({
            content: "Cannot create a thread in this channel",
          });
          return;
        }

        try {
          const thread = await interaction.channel.threads.create({
            name: `Action: ${userPrompt.substring(0, 50)}...`,
            autoArchiveDuration: 60,
          });

          await thread.send(
            `${userPing}\n\n${quotedPrompt}\n\n${action.content}`,
          );
          await interaction.editReply({
            content: `I've created a thread for you: ${thread.url}`,
          });
        } catch (threadError) {
          log(ERROR, "failed to create or reply in thread:", threadError);
          await interaction.editReply({
            content: `${userPing}\n\n${quotedPrompt}\n\nI tried to create a thread but failed. Here is your response:\n\n${action.content}`,
          });
        }
        break;

      case "CREATE_EMBED": {
        const embed = new EmbedBuilder()
          .setColor(0xffa500)
          .setTitle(action.title)
          .setDescription(action.description)
          .setTimestamp()
          .setFooter({ text: "Support System" });

        const rolePing = action.roleToPing ? `<@${action.roleToPing}>` : "";

        await interaction.editReply({
          content: `${userPing} ${rolePing}`,
          embeds: [embed],
        });
        break;
      }
    }
  } catch (error) {
    log(ERROR, "generating AI response or processing graph:", error);
    const errorMessage =
      "sorry, I encountered an error while processing your request.";
    if (interaction.replied || interaction.deferred) {
      await interaction.followUp({ content: errorMessage, ephemeral: true });
    } else {
      await interaction.reply({ content: errorMessage, ephemeral: true });
    }
  }
});

const token = process.env.DISCORD_BOT_TOKEN!;
client.login(token);

At the core of our bot is the Client object from discord.js. This represents your bot and handles everything from connecting to Discord’s API to listening for events like user messages or interactions.

What’s with that intent? Discord uses intents as a way for bots to declare what kind of data they want access to. In our case:

Guilds lets the bot connect to servers
GuildMessages allows it to see messages
MessageContent gives access to the actual content of messages

These are quite standard, and there are many more based on different use cases. You can always check them all out here.

You also keep a Map to store per-channel message history so the bot can respond with context across multiple channels:

const channelHistories = new Map<string, ChatMessage[]>();

Discord.js provides access to a few events that you can listen to. When you work with slash commands, it registers an Events.InteractionCreate, which is what you’re listening to.

With every /ask command, you take the user's prompt and any previous messages. If channelHistories does not have a key with that specific channelId, meaning it's being used for the first time, you initialize it with an empty array and feed them into the AI state.

const finalState = await graph.invoke({
  message: currentUserMessage,
  previousMessages: [...messageHistory],
});

Depending on what the graph finalAction.type returns, you either:

reply directly,
create a thread and respond there,
or send an embed (for support-type replies).

If a thread can’t be created, you fall back to replying in the main channel. Message history is capped at 20 to keep things lightweight.

Note that we’re not really using previousMessages much at the moment in the application, but I’ve prepared everything you need to handle querying previous conversations. You could easily create a new LangGraph node that queries or reasons over history if the bot needs to reference past conversations. (Take this as your challenge!)

This project should give you a basic idea of how you can use LangGraph + Composio to build a somewhat useful bot that can already handle decent stuff. There’s a lot more you could improve. I’ll leave that up to you. ✌️

Here’s a quick demo of what we’ve built so far:

Wrapping Up

By now you should have a good idea of how LangGraph works and also how to power the bot with integrations using Composio.

This is just a fraction of what you can do. Try adding more features and more integration support to the bot to fit your workflow. This can come in really handy.

If you got lost somewhere while coding along, you can find the source code here.

So, that is it for this article. Thank you so much for reading! See you next time. 🫡

Love to build cool stuff like this? I regularly build such stuff every few weeks. Feel free to reach out to me here:

How to Host Local LLMs in a Docker Container on Azure

Shrijal Acharya — Wed, 26 Mar 2025 16:10:45 +0000

Have you ever run into a situation where you want to test some local AI models, but your computer doesn't have enough specs to run them? Or maybe you just don't like bloating your computer with a ton of AI models?

You're not alone in this. I’ve faced this exact issue, and I was able to solve it with the help of a spare VM. So the only thing you'll need is a spare PC somewhere that you can access.

Here, I'm using Azure, but the process should be fairly simple for other cloud providers as well. Even if you have a homelab with your old PC or something that you can SSH into, the only thing you'll have to change is the commands that deal with Azure. Everything else should work just fine.

And the best part? We will be doing everything inside a Docker container. So if you ever want to remove all the AI models, just remove the container, and you're all set. Even your VM is not going to install anything locally, pure Docker! 👌

In this article, I will show you how to host local LLMs with Ollama in a Docker container on an Azure Virtual Machine.

What you will learn: 👀

How to use Ollama to run multiple LLMs on a single machine.
How to set up ollama and ollama-webui containers with docker-compose.
How to create a VM on Azure and configure everything using the Azure CLI.
How to restrict Azure VM access to your public IP using Azure CLI.

By the end of the article, you will have a fully functional Azure VM capable of running all your chosen AI models (dependent on its specs, of course).

How to Set Up the Environment
How to Set Up the Azure Virtual Machine
How to Set Up the VM for Running AI Models
Wrapping Up

How to Set Up the Environment

💁 You will mostly be writing Bash, so make sure that you know some basics of shell scripting before moving forward.

Create a folder to keep all your source code for the project:

mkdir run-local-ai-models-docker-azure \
    && cd run-local-ai-models-docker-azure
mkdir azure scripts

Here, the azure directory will hold all the scripts required to work with the Azure VM, and the scripts directory will hold everything needed to set up the VM to run the LLMs.

Create a new .env file in the root of the project with the following environment variables. Make sure you change the size, name, location and the models as you like.

RESOURCE_GROUP="ollama-vm-rg"
LOCATION="eastus"
VM_NAME="ollama-vm"
VM_SIZE="Standard_D2s_v3"
USERNAME=""

# This will be used as a backup when we can't fetch the IP
# from the 'api.ipify.org'
IP_ADDRESS=""

# Change it to whatever models you like.
OLLAMA_DEFAULT_MODEL="qwen2.5-coder:3b"

# Make sure to use "," when separating multiple models.
OLLAMA_ADDITIONAL_MODELS="deepseek-r1:1.5b,tinyllama:1.1b"

WEBUI_PORT=3000
OLLAMA_PORT=11434

And finally, you need to have Azure CLI installed. Follow the installation instructions shown here to install it locally on your machine.

Once, you have it installed, authenticate the CLI with your Azure account using the following command:

az login

Once you are logged in, the setup is complete, and you can start coding the project. 🎉

How to Set Up the Azure Virtual Machine

In this section, I'll show you how to set up your Azure VM using the Azure CLI az. You'll do everything from making a separate resource group to setting up the network to only allow access from your IP address, and finally, creating the VM.

When creating a new file for this section, make sure you do it in the azure directory.

Create a Resource Group

First, begin by creating a new resource group for your Virtual Machine. But what’s a resource group? Basically, a resource group is like a container that holds related resources, such as the VM, storage, NSGs and all. It helps organize these resources quickly, making it easier to deploy, update, and delete them all at once. In short, just think of it as a way to group related stuff together.

Create a new file called create-resource-group.sh and add the following lines of code:

#!/usr/bin/env bash
set -e

SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"

source "$PROJECT_ROOT/.env"

echo "Creating resource group '$RESOURCE_GROUP' in location '$LOCATION'..."
az group create --name $RESOURCE_GROUP --location $LOCATION

echo "Resource group created successfully."

Notice the set -e command. By adding this, its telling the script to stop if any step causes an error. Without this, the script would keep running even if a step fails, which would lead to errors. Remember this command, as it is in all the scripts you’ll write.

Next, it sources the .env file to access the environment variables. After that, it runs the az command to create a resource group with the given name and location.

Now, run the following command:

bash create-resource-group.sh

To check if it worked, go to your Azure account and look under the Resource groups section for your new resource group.

If you see your resource group on the list, you’re good to go.

Create the Virtual Machine

Now that the resource group is created, you can now move forward to creating the virtual machine itself. This script is also going to be fairly similar to the earlier one.

Create a new file called create-vm.sh and add the following lines of code:

#!/usr/bin/env bash
set -e

SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"

source "$PROJECT_ROOT/.env"

VM_EXISTS=$(az vm show --resource-group $RESOURCE_GROUP --name $VM_NAME --query "name" -o tsv 2>/dev/null || echo "")

if [ -n "$VM_EXISTS" ]; then
    echo "VM '$VM_NAME' already exists in resource group '$RESOURCE_GROUP'."
    echo "Please choose a different VM name or use the existing VM."
    exit 1
fi

echo "Creating VM '$VM_NAME'..."
az vm create \
  --resource-group $RESOURCE_GROUP \
  --name $VM_NAME \
  --image Ubuntu2204 \
  --admin-username $USERNAME \
  --generate-ssh-keys \
  --size $VM_SIZE \
  --public-ip-sku Standard

# The above command generates the ssh key as "id_rsa", if you already
# have a key called id_rsa in the ~/.ssh directory, make sure to name it to something else
ssh-add ~/.ssh/id_rsa

echo "VM created successfully."

First, it checks if a VM with the same name is already in the resource group. If it isn't, then it creates a new VM there, providing details like size, image, username, and asking it to generate an SSH key which it will use to log in to the VM.

I chose Ubuntu because, in another section where you’ll set up Docker, I’ve used Debian steps. If you want to use a different distro, make sure to update that script, too.

Now, run the following command:

bash create-vm.sh

After running the command, under the VM section, you should see your newly created VM.

There you have it, the VM is up and running. You could manually go into the VM and set it up, but I will show you how to automate all of these steps as well, because we are devs, remember? 😉

Get the Virtual Machine Public IP

So, now that you have the VM created perfectly, it’s time to fetch the public IP of the VM so you can then use SSH to login.

Create a new file called get-vm-details.sh and add the following lines of code:

#!/usr/bin/env bash
set -e

SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"

source "$PROJECT_ROOT/.env"

echo "Getting VM details..."

PUBLIC_IP=$(az vm show --resource-group $RESOURCE_GROUP --name $VM_NAME --show-details --query publicIps -o tsv)

if [ -z "$PUBLIC_IP" ]; then
    echo "Error: Could not retrieve public IP for VM '$VM_NAME'"
    exit 1
fi

echo "VM Public IP: $PUBLIC_IP"
echo "Ollama API endpoint: http://$PUBLIC_IP:$OLLAMA_PORT"
echo "Web UI: http://$PUBLIC_IP:$WEBUI_PORT"

echo "PUBLIC_IP=$PUBLIC_IP" > "$PROJECT_ROOT/.vm_details.env"

echo "VM details retrieved successfully."

All it is doing is sourcing the .env and then using the az command to fetch the public IP of the VM and store it in a separate file called .vm_details.env. This way, in other scripts, it doesn't need to fetch the public IP repeatedly and can simply source the file to access it.

Now, run the following command:

bash get-vm-details.sh

Configure the VM Network to Allow Access Only from the User's IP

Now that you have the VM working, there's a slight security issue with it. If you check your VM network settings, you'll see that there is no IP restriction. This means anyone can easily try to SSH into your VM.

Allowing access from any source IP address is definitely not a good idea. You need to configure this to allow access only from your public IP, and also need to configure the port for ollama and ollama-webui.

Create a new file called configure-network.sh and add the following lines of code:

#!/usr/bin/env bash
set -e

SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"

source "$PROJECT_ROOT/.env"

echo "Configuring network security..."

# Get current public IP address with fallback to .env one.
echo "Retrieving current public IP address..."
IP_ADDRESS_CURRENT=$(curl -s https://api.ipify.org || echo "")
if [ -z "$IP_ADDRESS_CURRENT" ]; then
    echo "Warning: Could not retrieve IP from api.ipify.org, falling back to .env value"
    IP_ADDRESS_CURRENT=$IP_ADDRESS
fi

echo "Using IP address: $IP_ADDRESS_CURRENT"

NSG_NAME=$(az network nsg list --resource-group $RESOURCE_GROUP --query "[?contains(name, '${VM_NAME}')].name" -o tsv)
if [ -z "$NSG_NAME" ]; then
    echo "Error: Could not find NSG for VM '$VM_NAME'"
    exit 1
fi

create_or_update_nsg_rule() {
    local RULE_NAME=$1
    local PORT=$2
    local PRIORITY=$3

    RULE_EXISTS=$(az network nsg rule list --resource-group $RESOURCE_GROUP --nsg-name $NSG_NAME --query "[?name=='$RULE_NAME'].name" -o tsv)

    if [ -z "$RULE_EXISTS" ]; then
        echo "Creating new rule: $RULE_NAME for port $PORT..."
        az network nsg rule create --resource-group $RESOURCE_GROUP --nsg-name $NSG_NAME \
            --name "$RULE_NAME" \
            --protocol tcp --direction inbound --priority $PRIORITY \
            --source-address-prefix $IP_ADDRESS_CURRENT --source-port-range "*" \
            --destination-address-prefix "*" --destination-port-range $PORT \
            --access allow
    else
        echo "Updating existing rule: $RULE_NAME with new IP address..."
        az network nsg rule update --resource-group $RESOURCE_GROUP --nsg-name $NSG_NAME \
            --name "$RULE_NAME" \
            --source-address-prefix $IP_ADDRESS_CURRENT
    fi
}

# Check if 'default-allow-ssh' exists
SSH_RULE_EXISTS=$(az network nsg rule list --resource-group $RESOURCE_GROUP --nsg-name $NSG_NAME --query "[?name=='default-allow-ssh'].name" -o tsv)
if [ -n "$SSH_RULE_EXISTS" ]; then
    echo "Updating existing SSH rule (default-allow-ssh) with restricted IP..."
    az network nsg rule update --resource-group $RESOURCE_GROUP --nsg-name $NSG_NAME \
        --name "default-allow-ssh" \
        --source-address-prefix $IP_ADDRESS_CURRENT
else
    # If no default SSH rule, create our own with a different priority
    create_or_update_nsg_rule "SSH_Restricted" 22 1010
fi

# Configure rules for Ollama and Web UI
echo "Opening ports for Ollama API and Web UI (restricted to your IP)..."

create_or_update_nsg_rule "Port_${OLLAMA_PORT}_Restricted" $OLLAMA_PORT 1001
create_or_update_nsg_rule "Port_${WEBUI_PORT}_Restricted" $WEBUI_PORT 1002

echo "Network security configured successfully."
echo "Note: If your IP address changes, you'll need to run this script again to update the rules."

Don't be scared by this script. It might seem complex, but it's actually pretty straightforward. The first thing it does is try to get the user's IP address. It first attempts to fetch the user's IP from a service called api.ipify.org (because public IPs can change frequently), which returns your current public IP. If there's an error, it falls back to using the IP address stored in the .env file.

Next, it tries to get the VM's NSG (Network Security Group) because it's needed when creating a new NSG rule. If there’s an error, there's no point in continuing with the script, so it exits with an error status.

There's also a function called create_or_update_nsg_rule, which is used to create a new NSG rule if it doesn't exist. If it does exist, it simply updates it to allow access only from the user's IP address.

Finally, it creates or updates the SSH rule depending on whether it exists. It also sets up the rules for ollama and ollama-webui so you can access the exposed port on your local machine to test the models with ollama-webui.

Now, run the following command:

bash configure-network.sh

After running this command, you can view the changed network settings. There should be two new rules for port 11434 and 3000, and most importantly, you should see that the source IP is limited to your public IP.

Every configuration on the Azure side is now complete. All you need to do next is write a few more scripts to set up running AI models on your configured VM. ✌️

How to Set Up the VM for Running AI Models

Now that all the configuration on the Azure side is complete, in this section, I will show you how you can configure everything on the VM side, from installing Docker to setting up containers, and deploying those containers in the Azure VM you just created.

When creating a new file for this section, ensure you do it in the scripts directory, except for the docker-compose.yaml file.

Set Up the Virtual Machine

Okay, so now that you have a VM up and running with some network settings configured, let’s set this thing up to install and set up Docker and start the service.

Create a new file called setup-vm.sh and add the following lines of code:

#!/usr/bin/env bash
set -e

SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"

source "$PROJECT_ROOT/.env"
source "$PROJECT_ROOT/.vm_details.env"

echo "Setting up VM with Docker and dependencies..."

ssh $USERNAME@$PUBLIC_IP << 'EOF'
  # Install Docker Engine
  echo "Installing Docker..."

  # From Docker documentation for debian based distros
  # Add Docker's official GPG key:
  sudo apt-get update
  sudo apt-get install ca-certificates curl
  sudo install -m 0755 -d /etc/apt/keyrings
  sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
  sudo chmod a+r /etc/apt/keyrings/docker.asc

  echo \
    "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
    $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \
    sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
  sudo apt-get update

  sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
  # End of docker debian installation instructions

  sudo usermod -aG docker $USER

  sudo apt-get install -y docker-compose

  sudo systemctl start docker.service

  sudo docker volume create ollama_data

  # Here we will place our docker-compose.yaml file
  mkdir -p ~/ollama-project
EOF

echo "VM setup completed successfully."

And as I said earlier, in this example, I’m using Ubuntu, so I’m following Debian instructions to install Docker. If you are using a different distro, make sure to change the docker setup commands. Then it simply creates a Docker volume because you will want to persist the container state and not lose it every single time.

Now, run the following command:

bash setup-vm.sh

If everything goes well, you should have the Docker engine installed on your machine. You can verify it with the following command:

ssh @ << 'EOF'
    docker --version
EOF

Set Up Docker Compose

Create a new file called docker-compose.yaml at the root of the project – not inside the scripts directory this time – and add the following lines of code:

version: "3.9"
services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ~/ollama_data:/root/.ollama
    environment:
      - OLLAMA_HOST=0.0.0.0
    restart: unless-stopped

  ollama-webui:
    image: ghcr.io/ollama-webui/ollama-webui:main
    container_name: ollama-webui
    ports:
      - "${WEBUI_PORT:-3000}:8080"
    environment:
      - OLLAMA_API_BASE_URL=http://ollama:11434/api
    depends_on:
      - ollama
    restart: unless-stopped

The Docker Compose file should be fairly straightforward to understand. It sets up two services, or rather two containers: ollama and ollama-webui. You need to expose the container port so that you can access it in the VM, whose port is already exposed during the VM network configuration, allowing you to access it on your local machine. Finally, it specifies the Docker volumes and a few environment variables, and that's it.

For the ollama-webui container, it’s necessary for the ollama service to be up and running first, so it depends on the ollama container. After all, what's the point of starting the UI if the service itself is not running, right?

Deploy the Containers

Now that Docker is installed on the VM, it's time to copy the docker-compose.yaml file into the VM and start the containers.

Create a new file called deploy-containers.sh and add the following lines of code:

#!/usr/bin/env bash
set -e

SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"

source "$PROJECT_ROOT/.env"
source "$PROJECT_ROOT/.vm_details.env"

echo "Deploying Docker containers to the VM..."

scp "$PROJECT_ROOT/docker-compose.yaml" $USERNAME@$PUBLIC_IP:~/ollama-project/

ssh $USERNAME@$PUBLIC_IP << 'EOF'
  cd ~/ollama-project
  export WEBUI_PORT=3000
  sudo docker-compose up -d
  echo "Docker containers started successfully."
EOF

echo "Deployment completed successfully."
echo "Web UI available at: http://$PUBLIC_IP:$WEBUI_PORT"

It’s pretty simple here as well. First, it copies the docker-compose file into the ~/ollama-project directory and then spins up the container in detached mode.

Now, the moment of truth. Run the following command, and if everything goes well, you should have two Docker containers running in your VM.

bash deploy-containers.sh

To see if it worked, run the following command to ssh into the VM and execute the docker ps command.

ssh @ << 'EOF'
    docker ps
EOF

Along with some SSH output, you should see something like this, and if both containers' statuses say Up, you're all good.

By now, you should be able to visit this URL (http://:3000) to view the Web UI running. But there are no AI models to chat with, so let's fix that.

Run the LLMs Locally Inside the Docker Container

You’re almost there. All that is left is to install some models in Ollama. To install any model, all you need to do is ollama run . So, let’s do that inside a script that runs that command inside a Docker container, because remember you have Ollama running in a Docker container.

💡 GOOD TO KNOW: You can run any command inside a Docker container from outside using docker exec . This is perfect for our situation because there's no need to be inside the Docker container. You just need to run one command, and that's all.

Create a new file called run-models.sh and add the following lines of code:

#!/usr/bin/env bash
set -e

SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"

source "$PROJECT_ROOT/.env"
source "$PROJECT_ROOT/.vm_details.env"

if [ -n "$OLLAMA_DEFAULT_MODEL" ]; then
  echo "Running default model $OLLAMA_DEFAULT_MODEL..."
  ssh $USERNAME@$PUBLIC_IP << EOF
    sudo docker exec ollama ollama run $OLLAMA_DEFAULT_MODEL
EOF
  echo "Default model $OLLAMA_DEFAULT_MODEL run successfully."
fi

if [ -n "$OLLAMA_ADDITIONAL_MODELS" ]; then
  echo "additional models $OLLAMA_ADDITIONAL_MODELS..."
  IFS=',' read -ra MODELS <<< "$OLLAMA_ADDITIONAL_MODELS"

  for MODEL in "${MODELS[@]}"; do
    # trim whitespace
    MODEL=$(echo "$MODEL" | xargs)
    echo "Running additional model $MODEL..."
    ssh $USERNAME@$PUBLIC_IP << EOF
      sudo docker exec ollama ollama run $MODEL
EOF
    echo "Additional model $MODEL run successfully."
  done
fi

echo "All models have been processed successfully."

All this script does is first check if the default model is set up in the env with OLLAMA_DEFAULT_MODEL. If it is, the script runs it and also runs any other models separated by commas in the OLLAMA_ADDITIONAL_MODELS env variable.

Now, run the following command:

bash run-models.sh

If everything goes well and you see the final echo message, then hurray! 🎉 You’ve successfully set up running LLMs inside a Docker container in an Azure VM.

Go ahead and refresh the Web UI, and you should see all your LLMs appearing in the list of available models. Choose any one you like and start chatting! 🔥

Wrapping Up

That is it for this one. I hope you enjoyed it and, better yet, understood everything we did together. I build such stuff every other week and document them with blogs. Feel free to check out some of my previous tutorials on DEV and freeCodeCamp.

You can find the complete source code here.

And hey, if you agree with the response from the above qwen2.5-coder model, here are my socials 😉:

GitHub: github.com/shricodev
Portfolio: techwithshrijal.com
LinkedIn: linkedin.com/in/iamshrijal

How to Host Static Sites on Azure Static Web Apps for Free

Shrijal Acharya — Tue, 18 Jun 2024 10:59:18 +0000

In this article, I will show you how you can host your React/Next.js app or any static web app on Azure Static Web Apps.

I will be showing you both ways of doing it – through the GUI and with the CLI.

I assume you already have built a project and optionally pushed it to GitHub or any other alternative, like GitLab or Bitbucket.

It's time to go live with your project and showcase it to the world. 💪

Whether it is your first time working with Azure or you are already a champ, feel free to follow along.

What is CI/CD?

Before we dive into Azure Static Web App, let me give you a brief overview of what CI/CD is.

Imagine you're building your cool new web app. Now, to make sure it's always up-to-date and running, you need something called Continuous Integration/Continuous Deployment (CI/CD).

Here's how it works:

Continuous Integration (CI) is the process of automatically building and testing your code whenever you make changes and ensuring it is working as you expect.
Continuous Deployment (CD) is the process of automatically deploying your tested code to production.

You might have already came across CI/CD if you've ever hosted your project on Vercel, Netlify, or any other site hosting platform, and noticed that once you push your local changes to your remote repository, those changes are reflected on the original hosted site in just a few minutes. Magic, huh? This is all possible with the help of CI/CD.

In short, CI/CD ensures that your project is tested and deployed whenever you push new commits to the repository hosting your project.

How to Push your Project to GitHub

In this step, I will use GitHub as an example. If your project is already pushed to GitHub, feel free to skip this step. Otherwise, follow the steps shown below.

Log in to GitHub CLI

Did you know you can create a GitHub repository right from your command line?

Let's get started. Firstly, make sure you have GitHub CLI installed. For installation instructions, follow the steps shown here.

To authenticate your GitHub CLI to your account, run the below command:

gh auth login

Follow the steps displayed in your terminal to authenticate with GitHub. Once done, you can proceed to the next step.

Create a repository

Once you're logged in, you can run the below command to start an interactive repository creation mode:

gh repo create

Or, directly specify flags to it like so:

gh repo create  --public --license mit --description

Running, this command will create a repository with all the specified options in your GitHub account.

Push your changes

Now, that you have already authenticated yourself and have created a repository in your GitHub account, it's time to push your local changes to the remote repository.

Run the following commands to push your local changes to the remote:

git branch -M main
git remote add origin https://github.com//.git
git push -u origin main

Note that if you have SSH authentication set up, you need to make sure you change the origin URL to follow SSH protocol.

That concludes the initial preparation. Now, let's proceed with creating the Azure Static Web App. 🚀

Host your project with Azure SWA CLI

In this section, you'll learn how to host your static project, whether it built with Next.js, React, or any other static site, using Azure SWA CLI.

First, you need to install the @azure/static-web-apps-cli package as a dev dependency.

If using pnpm as a package manager, run the below command. The command might vary based on different package managers.

pnpm install -g @azure/static-web-apps-cli

Now, build your project with the appropriate build command:

pnpm run build

It's time to deploy the application with the build folder. Run the below command to deploy it to Azure SWA:

swa deploy  --env production

If your application uses an API, then you need to pass the api folder location as a flag to the above command. So, your final command would be:

swa deploy  --api-location  --env production

Deploying the project with Azure Static Web App CLI.

That is it. Your site is deployed to Azure Static Web App through CLI. 🎉 You can view it in the shown URL.

There are further configuration which can be specified. For additional configuration, follow this link.

How to Host your project with Azure Portal GUI

In this section, we will create another brand new Azure SWA through the Azure portal.

Head over to the Azure portal at https://portal.azure.com and search for the Azure Static Web App.

Azure portal search results for Static Web App.

Hit "Create" and you should be asked with a bunch of extra configurations.

Creating a new Azure Static Web App.

Setting up the configurations for the Azure Static Web App.

For the Resource Group, create one and name it whatever you want. Also, give your web application a name.

If your project is hosted on GitHub, choose GitHub as the option and choose the repository from the dropdown menu. For the branch, select 'main' if you created the project from scratch with default settings.

If you are using any other alternatives, then select the one which satisfies your criteria.

Leave everything else as default and click on "Review + create" and wait till it is done hosting your project to Azure SWA.

In the Overview section of the app, you should see URL for your built project. Here is the overview of mine:

Azure Static Web App dashboard overview.

Visit the URL and you should have your site ready. 🥳

Deployed project hero section.

Wrapping Up

By now, you should have a general idea of how to host any static web app in Azure Static Web Apps.

While we could cover additional topics like adding a custom domain to our site, this may not apply to most of you, so I'm not including the steps in this article. To learn more, visit this link.

So, that is it for this article. Thank you so much for reading! See you next time. 🫡

Now that you've had a sneak peek at my portfolio in the image above, why not get in touch? 😉 Feel free to connect with me here:

GitHub: https://github.com/shricodev
LinkedIn: https://linkedin.com/in/iamshrijal
Twitter: https://twitter.com/shricodev

Shrijal Acharya - freeCodeCamp.org

Figma MCP vs Kombai: Cloning the Front End from Figma with AI Tools

Table of Contents

What's the Deal?

Meet the Tools

Kombai

Figma MCP

Frontend Comparison with Figma

Test 1: Simple Portfolio Design

Figma MCP

Kombai

Test 2: Complex Learner Dashboard

Figma MCP

Kombai

What You Should Know Before Using These Tools

Final Verdict, and What's Next?

Conclusion

How to Build a Google Sheet AI Agent with Composio and Gemini TTS Support

What's Covered?

Table of Contents

What’s this Sheet Agent?

How to Set Up the Project

Core Components in the Application

1. Initiate Connection

2. Set up TTS with Gemini API

3. Handle User Queries

Google Sheet Agent in Action

Conclusion

How to Build a LangGraph and Composio-Powered Discord Bot

Prerequisites

Table of Contents

How to Set Up the Environment

Initialize the Project

Install Dependencies

Configure Composio

Configure Discord Integration

Add Environment Variables

Build the Application Logic

Define Types and Utility Helpers

Implement LangGraph Workflow

Set Up Discord.js Client

Wrapping Up

How to Host Local LLMs in a Docker Container on Azure

Table of Contents

How to Set Up the Environment

How to Set Up the Azure Virtual Machine

Create a Resource Group

Create the Virtual Machine

Get the Virtual Machine Public IP

Configure the VM Network to Allow Access Only from the User's IP

How to Set Up the VM for Running AI Models

Set Up the Virtual Machine

Set Up Docker Compose

Deploy the Containers

Run the LLMs Locally Inside the Docker Container

Wrapping Up

How to Host Static Sites on Azure Static Web Apps for Free

What is CI/CD?

How to Push your Project to GitHub

Log in to GitHub CLI

Create a repository

Push your changes

Host your project with Azure SWA CLI

How to Host your project with Azure Portal GUI

Wrapping Up