This data holds immense value, promising to improve decision-making, streamline operations, and unlock deep customer insights.

But data often remains siloed and inaccessible. It may be spread across different departments and systems, and it can be challenging to understand and utilize effectively.

This is where the concept of Retrieval-Augmented Generation (RAG) technology comes in. By combining the power of retrieval-based techniques and modern generative AI tools, you can build Retrieval-Augmented Generation (RAG) chat applications that allow you to interact with your data using a simple chat interface.

What is Retrieval-Augmented Generation (RAG)?

But before you can chat about your data, a lot of “legwork” is involved. Setting up the infrastructure – the pipeline, vector database, message broker, and knowledge retrieval – is a complex and time-consuming process. This is where the open source tool Agent Cloud comes in.

In this guide, you'll learn all about Agent Cloud and what it can do. We'll start by looking at some background info and the current problems we're dealing with. Then, we'll see how Agent Cloud can help solve them.

How I Started Working with Agent Cloud

I'm passionate about new technology and developer tools, and I sit somewhere between Product Marketing, Growth, and Developer Advocacy. I specialize in the creation of high-quality, technical written content for educational purposes.

I've been involved with the web for ~14 years, the last 4 of which have been documented in punishing detail on my website.

I liked being a Software Engineer but, what I really love to do is code, design, develop, and then write.

Earlier this year I met Andrew (founder of Agent cloud) in a private Slack group. He was seeking someone who could not only write about the product but also discuss and teach people about what they're building. I reached out to him, and after two rounds of discussions, we began working together.

I started with building some cool RAG chatbots in my local and later wrote a couple of comprehensive guides on "How to Build a RAG Chatbot with Agent Cloud".

In this article, I'll teach you how to build a RAG chat app using Agent Cloud to privately and securely talk with your Google Sheets data. I'll also talk about why I think Agent Cloud is good open source developer tool.

Table of Contents:

• What is Agent Cloud?
• What is RAG?
• Challenges of Building a RAG Chatbot Without Agent Cloud
• Prerequisites
• How to Set Up Agent Cloud via Docker
• How to Add Models in Agent Cloud
• How to Create Your GCP Service Account Key
• How to Enable Google Sheets API
• How to Set Up your Data Sources
• How to Set Up tools
• How to Set Up an Agent
• How to Create a Task
• How to Set Up your App
• Conclusion

What is Agent Cloud?

Agent Cloud is an open-source platform that lets you build private, secure chat applications powered by large language models (think ChatGPT).

It streamlines this process by providing a "RAG as a service" offering and a built-in pipeline that allows you to split, chunk, and embed data from over 300 sources (including Google Sheets, Salesforce, Atlassian Confluence, BigQuery, MongoDB, Postgres Data, SharePoint, and OneDrive).

Data sources

What is Retrieval-Augmented Generation?

RAG is a process for enhancing the accuracy of large language models. It does this through the on-demand retrieval of external data and by injecting context into the prompt, at runtime.

This data can come from various sources, such as your customers' documentation/web pages (through scraping), and data or documents from dozens (if not hundreds) of 3rd party applications like their Databases, Google BigQuery, HubSpot, Google Ads, Google Analytics 4 (GA4) and so on.

For those who want to dive deeper into Retrieval-Augmented Generation and understand its broader applications and significance, I highly recommend reading this comprehensive blog by NVIDIA. It offers valuable insights and context that complement the practical aspects covered in this article.

Challenges of Building a RAG Chatbot Without Agent Cloud

If you're working with these AI tools on a daily basis, it becomes easy to understand the value they bring and realize the significance of Agent Cloud in simplifying the chatbot development process.

But to fully appreciate its benefits, you should understand how chatbot development was handled before such tools were available.

Before tools like Agent Cloud, creating a RAG (Retrieval-Augmented Generation) chatbot was a daunting and resource-intensive task. You had to manually integrate various components, which required significant expertise in multiple areas.

Here are some challenges faced and the solutions the Agent Cloud team had to devise:

Data Retrieval and Management:

List of data sources

• Problem: Ensuring that the chatbot could efficiently retrieve and manage data from sources like Google Sheets, Databases, and so on.
• Without Agent Cloud: Developers had to write custom scripts for data retrieval, often using APIs to fetch data from Google Sheets. This involved handling data formatting, error checking, and real-time updates manually. It was a time-consuming process prone to errors.
• Agent Cloud's Solution: Automates data retrieval and management, ensuring seamless and accurate integration with minimal manual intervention.

Natural Language Processing (NLP):

• Problem: Implementing NLP to understand user queries and generate human-like responses.
• Without Agent Cloud: Developers needed to integrate standalone NLP engines such as TensorFlow. This required training models on vast datasets, fine-tuning them for accuracy, and constantly updating them to handle new queries effectively.
• Agent Cloud's Solution: Offers built-in NLP capabilities, significantly reducing setup time and providing high-quality language understanding out of the box.

Scalability and Maintenance:

• Problem: Ensuring the chatbot could handle increasing data loads and user interactions.
• Without Agent Cloud: Building a scalable architecture meant investing in robust server infrastructure, writing efficient code, and continuously monitoring and maintaining the system to handle growth.
• Agent Cloud's Solution: Designed to scale effortlessly, allowing developers to focus on improving functionality rather than managing infrastructure.

User Interaction and Experience:

Agent cloud app UI

• Problem: Creating an engaging and user-friendly interface.
• Without Agent Cloud: Developers had to build custom interfaces, often from scratch, which required additional design and development resources. Ensuring smooth interactions and responsiveness was a major challenge.
• Agent Cloud's Solution: Provides pre-built templates and easy integration options, enhancing the user experience with minimal effort.

By understanding these challenges, you can see how a tool like Agent Cloud helps the process of building RAG chatbots. It addresses the pain points of manual data handling, complex NLP integration, scalability issues, and user interface design, providing an all-in-one solution that saves time and resources.

Prerequisites

You don't need any prior knowledge of RAG chat apps or Google Sheets to follow along because Agent Cloud handles the data splitting, encoding, storage, and synchronization. This allows you to focus on talking to your data and interpreting the results.

How to Set Up Agent Cloud via Docker

First, you'll need to install Docker on your system if you don't have it already. Docker is a platform for running containerized applications.

Open your terminal and run the following command to clone the Agent Cloud repository from GitHub:

Use the following command to move into the newly cloned agentcloud directory:

To run Agent Cloud locally, execute this command:

This command will grant executable permissions to the install.sh script and then run it. The script will download the necessary Docker images and start the Agent Cloud containers within your Docker environment.

local dev setup- agent cloud

Once the installation script finishes successfully, you can view the running Agent Cloud containers in the Docker application.

local docker services

Once everything is up and running, you can access the Agent Cloud user interface in your web browser.

Navigate to the URL:

This will typically open the registration page where you can create an account to use Agent Cloud.

signup- page- agent cloud

You can now log in using the credentials you created during signup.

Once you've successfully registered and logged in, you'll be greeted by the Agent Cloud user interface. This interface provides a central hub for managing your data sources, agents, tasks, models, applications, credentials, and so on.

Home screen - agent cloud

Congratulations! You've successfully set up Agent Cloud. Now let's move on to the next step and build our RAG chat application using Google Sheets as the data source.

How to Add Models in Agent Cloud

Go to the Models tab in Agent Cloud. Click the Add Models button to add two types of models.

• A fast embed model is a lightweight model that runs locally on your machine. It splits and chunks your data before embedding it.
• OpenAI is a popular cloud-based LLM provider.

Models screen- agent cloud

How to Create Your GCP Service Account Key

Agent Cloud offers two authentication methods for accessing your Google Sheets data:

• Google (Auth) – This method involves directly authorizing Agent Cloud through your Google account.
• Service Key Account – This approach utilizes a service key, a credential specifically created for application access to Google Cloud Platform (GCP) services, including Google Sheets.

For this guide, I'll focus on the service key account method, which is generally considered a more secure and streamlined approach for application-to-application communication.

Here is what you’ll need:

• A Google Cloud Platform (GCP) project with the Google Sheets API enabled.
• A service account key is created within your GCP project. This key will be used to authenticate Agent Cloud.

I'll walk you through creating a service account key in your GCP project and configuring Agent Cloud to use it to access your Google Sheets data.

First, create a Google Cloud Platform account. Then create a new project. You can give your project any name that you prefer.

How to Create a GCP Account

Navigate to the "IAM & Admin" section and select "Service Accounts."

IAM and Admin Screen

Click "Create Service Account" and provide a name and description.

How to add service account

Enter your service account name and unique service ID, then click Done.

How to create a service account in GCP

Under the actions tab, click the manage keys option.

Project

Click the ADD KEY button and select the Create Key option.

Select JSON as the key type, then click on Create New Key.

Add keys in GCP

Select JSON as the key type, then click on Create. Your service account JSON or key JSON will automatically download.

Create private key in GCP

Keep this file safe we will use it later for authentication.

How to Enable Google Sheets API

You'll also need to enable the Google Sheets API in your Google Cloud project.

Open the left navigation menu and navigate to "APIs & Services" and then "Library."

APIs and Services in GCP

Type "Google Sheets API" in the search bar and press Enter.

The Google Sheets API will appear in the search results. Click on it.

Google Sheets API

On the API details page, click the "ENABLE" button.

Enable- Google Sheets API

How to Set Up your Data Sources

Agent Cloud empowers you to process data from a wide range of sources.

In this guide, I'll use Google Sheets as the data source. Sheets is a popular web-based spreadsheet application included in Google Workspace. Google Sheets allows you to create, edit, and collaborate on spreadsheets in real-time.

For this example, I'll be working with a financial consumer complaints dataset stored in a Google Sheet.

This dataset contains several columns representing key sales stages and details, potentially including:

• Complaint ID
• Submitted Via
• Date Submitted
• Date Received
• State
• Product
• Sub-product
• Issue
• Sub-issue
• Company
• Public Response
• Company Response to Consumer
• Timely Response?

Here's how to connect your Google Sheets data:

Navigate to the Data Sources tab within the Agent Cloud interface.

Click the button labeled New Connection. This will initiate the process of adding a new data source.

Data Sources- Screen- Agent Cloud

Search and select “Google sheets” as the data source.

Create a Datasource

I've named our data source Financial Consumer Complaints, which you can name however you want. Add a clear and concise description. The data sync will be manual. This means you'll need to initiate the data refresh process whenever you want the latest information from your Google Sheets to be reflected in Agent Cloud.

Enter the appropriate row batch size. Row batch size means how many rows are processed from the Google sheet. For example, the default value 200 would process rows 1-201, then 201-401, and so on.

Choose the authentication method as “Service Account Key Authentication”.

Enter the JSON key we downloaded earlier above under the Service Account Information field.

Enter the link to the Google spreadsheet you want to sync. To copy the link, click the 'Share' button in the top-right corner of the spreadsheet, then click 'Copy link'.

Adding details under datasource

Click the submit button.

Select the collection and fields you want to sync to the vector database and enter their descriptions.

Model pop-up

Click continue and Choose the field you want to embed then choose the model.

Data sources

The connection to your Google Sheet is now established.

During the initial run, Agent Cloud will process your spreadsheet data and convert it into a format suitable for efficient retrieval. This processed data will then be stored within Agent Cloud's vector store.

New- datasource- screen

If you're comfortable with technical details, you can verify the data's presence in the Qdrant vector store running locally on port 6333.

This can be accessed through: http://localhost:6333/dashboard#/collections.

Look for a new collection corresponding to your Google Sheet data.

collection-screen

You can click the collection to view the payloads and the fields populated in the collection.

Payload

How to Set Up Tools

Tools play a crucial role in facilitating effective interaction between the AI agent and its environment, enabling seamless information processing and action execution aligned with its objectives.

These tools can include functions, APIs, data sources, and resources tailored to empower the agent to autonomously and efficiently undertake diverse tasks. While you have the flexibility to craft your own tools as per their requirements, Agent Cloud also streamlines the process by automatically generating a tool upon the addition of a new data source,

Click the Tools tab tools to switch to the Tools page and click the New button.

Enter the tools' names and add a description. A verbose and detailed description helps agents better understand when to use each tool.

Choose the data source and click the Save button.

Edit tool screen

How to Set Up an Agent

AI Agents are intelligent systems that excel at handling routine or repetitive tasks by perceiving their environment, collecting data, and using that data to make decisions.

Click the Agents tab and then click on the New Agent button.

New Agent Screen

Define an Agent's Name, Role, Goal, and Backstory as shown below.

In the "Model" section, choose the AI engine to power your agent's reasoning capabilities. For this example, we've selected "OpenAI GPT-4" as both the Model and Function Calling Model.

Choose the tool we set up earlier under Tools (Optional).

Adding Details in Agent Screen

If "OpenAI GPT-4" isn't already configured within Agent Cloud, you can easily add it. Click on the "Model" field and select "Add new model." A new window will appear, allowing you to define the model's name, type, credentials (your OpenAI API key), and the specific LLM model.

Model- popup

Click the Save button. Once you save this information, "GPT-4" will be available for future agent creation.

Agents-Screen

How to Create a Task

Tasks are specific actions assigned to agents for completion. To create a new task, navigate to the Tasks tab and click the "Add Task" button.

Create Tasks Screen

Enter the following details in the below pop-up window.

• Task name: Give your task a clear and concise title that reflects its purpose.
• Detailed description of what the task entails.
• Choose the tool we created earlier, which is “Financial Consumer Complaint” in my case.
• Select the preferred Agent we created earlier we named it “Customer Complaints Agent”

Adding Details in Tasks Screen

Click the Save button to save the task.

How to Set Up Your App

Now, buckle up because we're about to embark on the exciting part: creating a conversational app. This app will transform our data source into a chat partner, allowing you to have interactive conversations and unlock insights through natural language.

So far, we've laid the foundation for our app. We've created:

• A data source.
• A data retrieval tool.
• An Agent.
• Tasks.

Click the Apps tab and then click the New App Button, and then enter the details below:

New App Screen

• App name – Enter the name you’d like to give to your app.
• Enter a description of what our App.
• Select a Tasks
• Select an Agent
• Choose a Chat Manager Model – Pick the Open AI model we configured earlier.

App Screen

Now, let's test our app.

Clicking the play button will open a chat window for us where we can have a conversation.

Play Button in App Screen

A chat interface window should be opened for us where we can chat with our data. For instance, with the data I have used, I can prompt the app to summarize some of the issues raised by customers related to the Mortgage product.

Live Chat with Data

Or I can have it summarize company responses related to different issues.

Answer Screen

Conclusion

In this tutorial, we explored building a RAG chat application using Agent Cloud and Google Sheets.

We covered setting up Google Sheets as a data source, embedding the data for efficient retrieval, and storing it within a vector store like Qdrant. We also learned how to create tools for Agents (chatbot components) and build an app chat interface where users can interact with the data without writing a single line of code.

If you want to read more interesting articles about developer tools, React, Next.js, AI and more, then I'll encourage you to checkout my blog.

Some of the fresh articles I've written this year.

• How I Build a Blog with Next.js and Firebase
• How I Build a Task Management App with React and Appwrite
• How I Build a Notes App Using React and Supabase – The Complete Guide
• How I Build a RAG Chat App With Agent Cloud and BigQuery

You can get in touch if you have any questions or corrections. I’m expecting them.

And if you found this tutorial useful, please share it with your friends and colleagues who might benefit from it as well. Your support enables me to continue producing useful content for the tech community.

Now it’s time to take the next step by subscribing to my newsletter and following me on Twitter.

With rapid advancements in Large Language Models (LLMs), chatbots have grown to become pivotal tools adopted across many websites and services. From FAQ bots to live chat support, they can provide users with information and assistance.

If you have a website, a sleek chatbot interface can offer support to your users. And you'll want to present a modern chatbot that can captivate your users and leave an impression.

React is one of the most popular tools for developing websites, and React-powered sites and apps are great candidates for chatbots. In this short guide, you'll see how easy it can be to integrate a chatbot into your React website.

Prerequisites

Before we start setting up our chatbot, note that this guide assumes knowledge of the following:

• React
• Node.js/npm
• Linux Commands

A basic understanding of the above is sufficient, and you should be able to set up your own React project. If you're completely unfamiliar with the above, you can check out the linked tutorials (as well as any other helpful resources you find) for them first. Otherwise, let's begin!

Step 1: Create a Project

Before we can setup our chatbot, let's create a new blank React project. Head over to a project folder of your choice and run the following commands in your terminal:

npm create vite@latest

You will be prompted to enter a project name, framework, and variant. For the purpose of this tutorial, we will go with the following selections:

• Project Name: MyChatBot
• Framework: React
• Variant: JavaScript

Once your setup is complete, head into your project folder and run the following commands:

npm install && npm run dev

If you visit http://localhost:5173 on your browser, you should be greeted with the following screen:

React app set up

Step 2: Install React ChatBotify

Now that we have our project setup, it's time to install React ChatBotify. React ChatBotify is a highly customizable React library that simplifies the process of integrating a chatbot into your application. We'll be using it to include a chatbot so install it with the command below:

npm install react-chatbotify

Once the installation is complete, we can proceed to import the library and have it rendered. Within your src folder, open up your App.jsx file with an editor of your choice. The default file should look similar to this:

import { useState } from 'react'
import reactLogo from './assets/react.svg'
import viteLogo from '/vite.svg'
import './App.css'

function App() {
  const [count, setCount] = useState(0)

  return (
    <>
      <div>
        <a href="https://vitejs.dev" target="_blank">
          <img src={viteLogo} className="logo" alt="Vite logo" />
        </a>
        <a href="https://react.dev" target="_blank">
          <img src={reactLogo} className="logo react" alt="React logo" />
        </a>
      </div>
      <h1>Vite + React</h1>
      <div className="card">
        <button onClick={() => setCount((count) => count + 1)}>
          count is {count}
        </button>
        <p>
          Edit <code>src/App.jsx</code> and save to test HMR
        </p>
      </div>
      <p className="read-the-docs">
        Click on the Vite and React logos to learn more
      </p>
    </>
  )
}

export default App

Next, with just adding 2 lines of code, we will see the chatbot rendered in our application. At the top of your file, add the line:

• import ChatBot from 'react-chatbotify'

Above your <div> element in the return statement, add the following line:

• <ChatBot/>

Your edited file should look like this:

import ChatBot from 'react-chatbotify'
import { useState } from 'react'
import reactLogo from './assets/react.svg'
import viteLogo from '/vite.svg'
import './App.css'

function App() {
  const [count, setCount] = useState(0)

  return (
    <>
      <ChatBot/>
      <div>
        <a href="https://vitejs.dev" target="_blank">
          <img src={viteLogo} className="logo" alt="Vite logo" />
        </a>
        <a href="https://react.dev" target="_blank">
          <img src={reactLogo} className="logo react" alt="React logo" />
        </a>
      </div>
      <h1>Vite + React</h1>
      <div className="card">
        <button onClick={() => setCount((count) => count + 1)}>
          count is {count}
        </button>
        <p>
          Edit <code>src/App.jsx</code> and save to test HMR
        </p>
      </div>
      <p className="read-the-docs">
        Click on the Vite and React logos to learn more
      </p>
    </>
  )
}

export default App

Try running your application again and you'll be greeted with a chatbot on the bottom right corner as shown in the screenshot below:

Chatbot in the React app

Pretty neat, isn't it?

Step 3: Craft Your "Hello World" Message

It's nice to have the chatbot setup easily, but it's not great to only have the default welcome message. Let's quickly add a hello world message of our own.

The <ChatBot/> component takes in a flow prop to define conversations. By default, the entry point of a conversation is always named the start block as shown in this example below:

const flow = {
  "start": {
    "message": "Hello world!"
  }
}

We will go ahead and pass the above flow into our chatbot:

import { useState } from 'react'
import reactLogo from './assets/react.svg'
import viteLogo from '/vite.svg'
import ChatBot from "react-chatbotify"
import './App.css'

function App() {
  const [count, setCount] = useState(0)

  const flow = {
    "start": {
      "message": "Hello world!"
    }
  }
  return (
    <>
      <div>
        <a href="https://vitejs.dev" target="_blank">
          <img src={viteLogo} className="logo" alt="Vite logo" />
        </a>
        <a href="https://react.dev" target="_blank">
          <img src={reactLogo} className="logo react" alt="React logo" />
        </a>
      </div>
      <h1>Vite + React</h1>
      <div className="card">
        <button onClick={() => setCount((count) => count + 1)}>
          count is {count}
        </button>
        <p>
          Edit <code>src/App.jsx</code> and save to test HMR
        </p>
      </div>
      <p className="read-the-docs">
        Click on the Vite and React logos to learn more
      </p>
      <ChatBot flow={flow}/>
    </>
  )
}

export default App

When you look at your application again, you will notice that the default message has disappeared and is replaced with Hello world! instead. Not too hard, isn't it?

Conclusion

In this guide, you've seen how easy it can be to setup a modern React chatbot.

While the example above is simple, there are plenty of other properties within a flow that can help you build your conversations. These are documented on the library website which also comes with live playground examples for you to explore and find out more.

Claude is one of the leading Large Language Models (LLMs) in the market, developed by Anthropic — an AI startup that was co-founded by ex OpenAI employees. The company is known for its stringent set of AI ethics and is currently backed by tech giants such as Google and Amazon.

In this article, we're going to delve into Claude AI, compare it to ChatGPT, and provide a quick example of how to work with it via the API.

What is Claude?

Similar to AI chatbots such as ChatGPT and Gemini, Claude is a chatbot powered by Claude 3  —  Anthropic's latest large language model. It is capable of taking user input and generating human-like output. Besides conversations, you can upload images and documents to Claude and have it summarize them or answer questions about specific points.

What distinguishes Claude from other competitors is Anthropic's claim that it is safer and less likely to produce harmful and offensive output due to "Constitutional AI" – a unique training approach pioneered by Anthropic aimed at developing AI systems that adhere to a set of ethical principles.

The first model was released on March 2023, which was later followed by updated versions with enhanced capabilities, more advanced training techniques, and more focus on safety.

In March 2024, Anthropic launched Claude 3, its most advanced, state-of-the-art suite of models: Haiku, Sonnet and Opus. Each having their own unique capabilities, with Opus being the most powerful.

Claude 3 offers image processing, lower rates of hallucination and a significantly larger context window. Claude chatbot, currently powered by Claude 3, demonstrated superior performance over ChatGPT in standardized benchmarks.

Apart from the chatbot, Claude is also available through API where developers can build applications on top of it.

Claude AI Capabilities

Here are the key areas where Claude excels:

• Conversation: Claude is highly capable of engaging in natural conversations, understanding user's context and providing thoughtful responses.
• Content Creation: Claude can generate high quality content tailored to requirements set by the user.
• Language Translation: In this day and age, global communication is crucial. Claude has multilingual capabilities, allowing translation between different languages in real-time and multi-lingual content creation.
• Visual processing: Claude can analyze and transcribe images, including photographs and handwritten notes.
• Code Generation: Code generation has become an attractive feature and a key competitive advantage with every new AI model release. Claude can generate code snippets, understand different programming languages, explain code functionality, and assist in debugging.

Claude vs ChatGPT

Let's compare the differences between ChatGPT and Claude:

LLM

• Claude 3 is the latest model lineup released on March 2024.
• GPT-4 is the current model since March 2023.

Performance

At the time of writing, Claude excels in terms of factual accuracy and can maintain context over longer conversations. Claude Opus is shown to outperform GPT-4 in all evaluation benchmarks of AI systems, especially in terms of knowledge and language understanding.

Context Window

Context window represents the maximum number of tokens an AI system can process in a single input or output. Larger context means the LLM can handle longer text and maintain the context while processing that text.

• GPT-4 has 8,192 tokens, while a newer version (GPT-4-32k) has 32,768 tokens.
• All three of Claude 3 versions have a context window of 200k tokens which is significantly larger than GPT-4.

Safety

GPT-4:

• While safety is a critical aspect of ChatGPT and efforts have been made to mitigate misinformation, the chatbot tends to generate some incorrect output.
• ChatGPT saves conversations with the user to further train and improve the model.

Claude 3:

• Claude was developed with safety in mind. Anthropic emphasizes on ethical use of AI in both the training and how the chatbot processes input and generates output. The model strictly follows Constitutional AI.
• Claude does not retain user data.

Accessibility

Both Claude 3 and GPT-4 models are available directly through the chatbots as well as through API. Additionally, they're accessible in other platforms:

• GPT-4: Microsoft invested heavily on OpenAI to integrate their latest LLMs into Microsoft platforms. As of today, GPT-4 is available via Microsoft's Copilot for free.
• Claude 3: Anthropic has partnered with companies like Notion, Amazon, and DuckDuckGo to integrate Claude 3 into their products.

How to Interact with Claude AI

If you're familiar with ChatGPT (by now, that is most likely the case), you should have a similar experience with the Claude chatbot. Once you create an account here, it'll be as simple as typing your queries.

In this section, we'll focus on interacting with the AI model through the provided API using Python to ask it to explain the concept of neural networks.

To get started, sign up here, then navigate here to create your first API key. Make sure you copy and store the API key in a secure file.

Here's an example of a Python script that instructs Claude to explain the concept of neural networks:

  # import anthropic library
  import anthropic

  # create a client instance
  client = anthropic.Anthropic(
      api_key="your_api_key",
  )

  # create the prompt and call the API
  message = client.messages.create(
      model="claude-3-opus-20240229",
      max_tokens=1000,
      temperature=0.0,
      system="Respond in short and clear sentences.",
      messages=[
          {
            "role": "user",
            "content": "Can you explain the concept of neural networks?"
          }
      ]
  )

  print(message.content)

Make sure you replace your_api_key with the actual API key that you created.

Let's quickly discuss the parameters defined above:

• model="claude-3-opus-20240229" specifies the model to be used.
• max_tokens=1000 sets the maximum number of tokens that the generated response can have.
• temperature=0.0 the temperature controls the level of randomness of the generated response. 0.0 means the response will be more consistent and less varied.
• system="Provide short and clear responses." specifies how the system should generate the response.
• messages=[{"role": "user", "content": Can you explain the concept of neural networks?"}] defines the role and input message based on which the output will be generated.

Here's a sample response in JSON:

{
    "id": "msg_01H4xwvAZnb6XTz8cHPoerBS",
    "type": "message",
    "role": "assistant",
    "content": [
        {
            "type": "text",
            "text": "Neural networks are a type of machine learning algorithm inspired by the structure and function of the human brain. They consist of interconnected nodes, or \"neurons,\" organized in layers. Each neuron receives input, processes it, and passes the output to neurons in the next layer. Through training on large datasets, neural networks learn to recognize patterns and make predictions or decisions.\n\nKey points about neural networks:\n\n1. Structure: Input layer, hidden layer(s), and output layer of neurons\n2. Weights and biases: Each connection has a weight that determines the importance of the input\n3. Activation functions: Determine the output of a neuron based on its input\n4. Training: Networks learn by adjusting weights through backpropagation and optimization algorithms\n5. Applications: Used for tasks like image recognition, natural language processing, and prediction\n\nNeural networks excel at learning complex, non-linear relationships in data and have revolutionized fields like computer vision and speech recognition. However, they require large amounts of training data and computational resources."
        }
    ],
    "model": "claude-3-opus-20240229",
    "stop_reason": "end_turn",
    "stop_sequence": null,
    "usage": {
        "input_tokens": 23,
        "output_tokens": 219
    }
}

Conclusion

Claude represents a significant leap in the realm of artificial intelligence that outmatches various competitors in the market.

Claude provides a unique view, a set of standards regarding safety and security, and offers a diverse range of applications, from content creation to code generation.

Well, how about building a WhatsApp bot that can give you the meaning of words you want to know?

In this tutorial, you will learn how to build a chatbot that can serve as your dictionary. It'll be easily accessible on WhatsApp using the Twilio MessagingX WhatsApp API to send and receive messages. We'll use Fast API to create the web server and interact with the database, and MongoDB to store the words and their meanings into a database.

By the end of this tutorial, you will have developed a functional chatbot that can define words in real-time while you're conversing on WhatsApp.

Prerequisites

• Python 3.9+ installed on your machine.

• MongoDB free account – if you don’t have one, you can set it up here.

• Twilio free account – you can set up one here.

• Merriam-Webster Developer’s account – you can set up one here.

• An IDE or text editor, such as VS code.

Set Up Your Development Environment

Before you start, you need to set up your development environment by creating the required directory and files. Here's the commands for that:

mkdir whatsappDictionary
cd whatsappDictionary
touch requirements.txt models.py utils.py main.py .env

• requirements.txt contains the required libraries to get the chatbot up and running.

• model.py contains the code connecting your chatbot to the MongoDB server

• utils.py includes the code to connect to Twilio MessagingX WhatsApp API

• main.py contains the code to build the Fast API server and connect to the Merriam-Webster API.

• whatsappDictionary is the directory for all the files.

Next, you'll create and activate a virtual environment and update the Python package manager pip to the most recent version using the following command:

python -m venv venv; ./venv/Scripts/Activate; pip --upgrade pip

If you are on a Linux machine, use this command:

pyton -m venv venv; venv\\Scripts\\activate.bat; pip --upgrade pip

To learn more about virtual environments and their benefits, you can read this tutorial.

Next you'll need to populate the requirements.txt file with the following dependencies:

fastapi
uvicorn
twilio
pymongo
pyngrok
requests
dotenv

• fastapi is a Python framework for building APIs quickly and easily

• uvicorn is a lightning-fast server implementation for Python

• twilio allows you to interact with the Twilio MessagingX WhatsApp API

• pymongo is the driver you will use to connect to the MongoDB server

• pyngrok enables you to tunnel a local server to a public URL

• requests allow you to send HTTP requests using Python

• dotenv loads the environment variables from the .env file.

Install these dependencies by running the following command on your terminal:

pip install -r requirements.txt

Configure the Database

You now want to set up a database to store words and their definitions. You are going to use MongoDB, which is a NoSQL language and is easy to set up.

You will need to create a free account on the MongoDB website (if you don't have one already). Once you have an account, log in to create a new Shared cluster and database.

Image showing how to create a shared cluster on Mongo DB

Afterwards, go to Security_,_ then ADD NEW DATABASE USER to add a new user with read/write access to the database.

This image shows the tab where you can add a new database

This image shows the window of adding a new database user

Go back to the cluster dashboard, click on Connect and then on Drivers.

This image shows you the final process while trying to connect to the cluster

Copy the code shown to you into model.py:

This image shows the code on how to connect to the MongoDB server using the pymongo driver.

from pymongo.mongo_client import MongoClient
from pymongo.server_api import ServerApi

uri = f"mongodb+srv://adejumoridwan:<password>@cluster0.d9jr4ev.mongodb.net/?retryWrites=true&w=majority"

# Send a ping to confirm a successful connection
try:
    client.admin.command('ping')
    print("Pinged your deployment. You successfully connected to MongoDB!")
except Exception as e:
    print(e)

Give your password to run the code and connect to the MongoDB server. You don’t want anyone to see this. Go to the .env file you created and store your password there.

MONGO_SECRET=<password>

Next, update model.py to access the .env file.

from pymongo.mongo_client import MongoClient
from pymongo.server_api import ServerApi
from dotenv import load_dotenv
import os

load_dotenv()
password = os.environ.get('MONGO_SECRET')

uri = f"mongodb+srv://adejumoridwan:{password}@cluster0.d9jr4ev.mongodb.net/?retryWrites=true&w=majority"

# Send a ping to confirm a successful connection
try:
    client.admin.command('ping')
    print("Pinged your deployment. You successfully connected to MongoDB!")
except Exception as e:
    print(e)

• load_dotenv() loads the variables in .env

• os.environ.get('MONGO_SECRET') receives the password from .env, which stores the variable password

Run the script to connect to the MongoDB server. You can create a collection by clicking on your Cluster name and going to Collections. Collections are NoSQL versions of SQL tables.

Click on Create Database to create your database:

This shows information and tabs regarding the created clusters such as Overview, Collections and so on

Give the database and collection names. The database and collection names for this tutorial are MY_DB and dictionary, respectively.

The window shows you the option to fill in when you want to create a database, such as a database name, collection name, and additional preferences.

Go to models.py and update the code to create a new client and connect to the server.

from pymongo.mongo_client import MongoClient
from pymongo.server_api import ServerApi
from dotenv import load_dotenv
import os

load_dotenv()
password = os.environ.get('MONGO_SECRET')

uri = f"mongodb+srv://adejumoridwan:{password}@cluster0.d9jr4ev.mongodb.net/?retryWrites=true&w=majority"

# Create a new client and connect to the server
client = MongoClient(uri, server_api=ServerApi('1'))

dictionary_collection = client["MY_DB"]["dictionary"]

# Send a ping to confirm a successful connection
try:
    client.admin.command('ping')
    print("Pinged your deployment. You successfully connected to MongoDB!")
except Exception as e:
    print(e)

dictionary_collection is what you will use to update entries into the server later.

How to Configure Twilio Sandbox for WhatsApp

To configure the Twilio sandbox for WhatsApp, go to Twilio Console. Under Develop, click on Messaging, then Try it out. Under Try it out, click on Send a WhatsApp Message.

This sandbox allows you to send WhatsApp messages to your number

To connect to the WhatsApp Sandbox, save the number provided on the sandbox on your device and send the message join manner-free to the number, or you can scan the QR code on your device.

Here you see the sandbox to connect to whatsapp sandbox to your number

Once the connection is successful, copy the code and paste it into the file utils.py:

Here you can see how to connect to WhatsApp API in various languages. Currently, the image is showing how to connect to WhatsApp API so that you can send messages from Twilio.

from twilio.rest import Client

account_sid = '<account_sid>'
auth_token = '[AuthToken]'
client = Client(account_sid, auth_token)

message = client.messages.create(
  from_='whatsapp:+14155238886',
  body='Your appointment is coming up on July 21 at 3PM',
  to='whatsapp:<to_number>
)

print(message.sid)

The client.messages.create() function allows you to send messages to your WhatsApp from the WhatsApp sandbox. It takes three parameters:

• from_ is where the message is coming from, that is, from the WhatsApp Sandbox

• body takes in the body of your message

• to is the WhatsApp number you are sending to

How to Connect to the Twilio API

Go to the .env file to store your Twilio authentication token, account SID, Twilio sandbox number, and WhatsApp number.

MONGO_SECRET="<password>"
TWILIO_ACCOUNT_SID="<account_sid>"
TWILIO_AUTH_TOKEN="<auth_token>"
TWILIO_NUMBER="<twilio_number>"
TO_NUMBER="<to_number>"

Update the utils.py file to access these variables:

from twilio.rest import Client
from dotenv import load_dotenv
import os

load_dotenv()

account_sid = os.getenv("TWILIO_ACCOUNT_SID")
auth_token = os.getenv("TWILIO_AUTH_TOKEN")
client = Client(account_sid, auth_token)
twilio_number = os.getenv('TWILIO_NUMBER')
to_number = os.getenv(“TO_NUMBER”)

message = client.messages.create(
  from_=f'whatsapp:{twilio_number}',
  body='Your appointment is coming up on July 21 at 3PM',
  to=f'whatsapp:{to_number}
)

load_dotenv() loads the environment variables, and os.getenv gets these variables from the environment.

Next is to define a send_message function. This function is going to have two arguments: to_number and text. The function will send a message defined in text to to_number.

from twilio.rest import Client
from dotenv import load_dotenv
import os

load_dotenv()

account_sid = os.getenv("TWILIO_ACCOUNT_SID")
auth_token = os.getenv("TWILIO_AUTH_TOKEN")
client = Client(account_sid, auth_token)
twilio_number = os.getenv('TWILIO_NUMBER')

def send_message(to_number, text):
      message = client.messages.create(
          from_=f"whatsapp:{twilio_number}",
          body=text,
          to=f"whatsapp:{to_number}"
          )

Update the send_message function to configure logging in case errors are encountered when sending messages.

import logging
from dotenv import load_dotenv
import os
from twilio.rest import Client

load_dotenv()

account_sid = os.getenv("TWILIO_ACCOUNT_SID")
auth_token = os.getenv("TWILIO_AUTH_TOKEN")
client = Client(account_sid, auth_token)
twilio_number = os.getenv('TWILIO_NUMBER')

# Set up logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

# Sending message logic through Twilio Messaging API
def send_message(to_number, text):
    try:
        message = client.messages.create(
            from_=f"whatsapp:{twilio_number}",
            body=text,
            to=f"whatsapp:{to_number}"
            )
        logger.info(f"Message sent to {to_number}: {message.body}")
    except Exception as e:
        logger.error(f"Error sending message to {to_number}: {e}")

How to Build the FastAPI Backend

Inside the main.py, set up a basic FastAPI application.

from fastapi import FastAPI

app = FastAPI()

@app.get("/")
async def index():
    return {"message": "I love FreeCodeCamp"}

The code below sets up a basic FastAPI backend, creating a new instance of the FastAPI class and assigning it to the app variable.

The @app.get decorator creates a new endpoint that you can access with an HTTP GET request. The endpoint is at the root URL / and returns a JSON response with a single key-value pair: "message": "I love FreeCodeCamp".

To run the app, run the following command in your terminal:

uvicorn main:app --reload

On your browser, open the host http://127.0.0.1:8000. You will see a JSON response of {"message": "I love FreeCodeCamp"}. You can also access an interactive API doc provided by swagger on the host http://127.0.0.1:8000/doc which allows you to interact with your API and see if you have any errors.

This interactive documentation provided by swagger shows how to interact with your API to check for errors.

How to Set Up ngrok

To receive Twilio messages on the backend, you will use ngrok to host the local host on a public server. Read this post to learn how to set up ngrok on your machine.

On ngrok administrator, run the command ngrok http 8000. This makes your host public, and you can receive messages on your backend.

Here you can see ngrok up and running on your local machine

Go to your WhatsApp Sandbox, under Sandbox settings, and paste the forwarding URL https://b5a6-105-112-120-51.ngrok-free.app appending it with /message under When a message comes in and click Save.

This image shows how to configure your sandbox and link to the ngrok url

How to Connect to the Merriam-Webster API

To set up a Merriam-Webster dictionary account, go here and fill out your credentials:

Image showing page to register for Merriam Webster Developer Center

You can register for two keys: Collegiate Dictionary and Collegiate Thesaurus, though this tutorial uses only Collegiate Dictionary.

This shows the various keys one can apply for on Merriam-Webster

After filling in all your details, click on Register and Login.

In the home tab, go to Keys to get your API keys.

This key tab shows the keys you have registered for

Update the .env file, saving the key as DICTIONARY_KEY.

MONGO_SECRET="<password>"
TWILIO_ACCOUNT_SID="<account_sid>"
TWILIO_AUTH_TOKEN="<auth_token>"
TWILIO_NUMBER="<twilio_number>"
TO_NUMBER="<to_number>"
DICTIONARY_API_KEY="<dictionary_key>"

Update the main.py as follows:

from fastapi import FastAPI, Form
import requests
from utils import send_message
from dotenv import load_dotenv
import os
from typing import List

load_dotenv()

app = FastAPI()
whatsapp_number = os.getenv("TO_NUMBER")
api_key = os.getenv("DICTIONARY_API_KEY")

@app.post("/message")
async def reply(Body: str = Form()):
    url = f"<https://www.dictionaryapi.com/api/v3/references/collegiate/json/{Body}?key={api_key}>"
    response = requests.get(url)
    # Extract the JSON data from the response
    data = response.json()

    definition = data[0]["shortdef"][0]

    send_message(whatsapp_number, definition)

@app.post("/message") is a decorator in the fastAPI framework that defines a POST request route to the URL /message. The reply function defined above is called when a POST request is sent to this URL.

The reply function takes in a Body parameter in the request body, which is the message sent to the chatbot (the word you want to get the definition of). It then sends an HTTP request to the Merriam-Webster API to retrieve the word's meaning.

The url variable stores the link to Merriam-Webster API, which takes the Body and the api_key to get details regarding the provided word.

You can make requests from url using requests from the request library and storing request.get(url) into the response variable.

You then extract the JSON data from the response using response.json() and store it in variable data.

data[0]["shortdef"][0] allows you to access the short definition of a word stored in the variable definition.

send_message() takes the definition and sends it to whatsapp_number.

Next, you'll need to handle situations where someone sends a sentence instead of a word, or a word containing punctuations or characters. So update main.py as follows:

from fastapi import FastAPI, Form
import requests
from utils import send_message
from dotenv import load_dotenv
import os
from typing import List
from models import dictionary_collection

load_dotenv()

app = FastAPI()
whatsapp_number = os.getenv("TO_NUMBER")
api_key = os.getenv("DICTIONARY_API_KEY")

@app.post("/message")
async def reply(Body: str = Form()):
    url = f"<https://www.dictionaryapi.com/api/v3/references/collegiate/json/{Body}?key={api_key}>"
    flag="Please give a valid word"

    if Body.isalpha():
        response = requests.get(url)
        # Extract the JSON data from the response
        data = response.json()

        definition = data[0]["shortdef"][0]

        send_message(whatsapp_number, definition)
    else:
        return send_message(whatsapp_number, flag)

    return ""

flag is a variable storing the message to give if you provide a sentence or a word with characters.

The if condition checks if a message is a word through Body.isaplha(), if true it gets the definition from Merriam-Webster API if false it returns the function send_message() telling the user to Please give a valid word.

To store the words and their meanings to the MongoDB database, update main.py as follows:

from fastapi import FastAPI, Form
import requests
from utils import send_message
from dotenv import load_dotenv
import os
from typing import List
from models import dictionary_collection

load_dotenv()

app = FastAPI()
whatsapp_number = os.getenv("TO_NUMBER")
api_key = os.getenv("DICTIONARY_API_KEY")

@app.post("/message")
async def reply(Body: str = Form()):
    url = f"<https://www.dictionaryapi.com/api/v3/references/collegiate/json/{Body}?key={api_key}>"
    flag="Please give a valid word"

    if Body.isalpha():
        response = requests.get(url)
        # Extract the JSON data from the response
        data = response.json()

        definition = data[0]["shortdef"][0]

        send_message(whatsapp_number, definition)

        dictionary_db = {"word":Body, "definition":definition}
        dictionary_collection.insert_one(dictionary_db)

    else:
        return send_message(whatsapp_number, flag)

    return ""

dictionary_db = {"word": Body, "definition": definition} creates a dictionary with two keys, word and definition, and values Body and definition, respectively.

dictionary_collection.insert_one(dictionary_db) inserts the dictionary into the MongoDB collection, named dictionary_collection.

You can go to your dashboard and view the items added to the collection.

This contains the items that have been sent and received on your application

Test the ChatBot

Now you can chat with the chatbot and ask for definitions of words:

Image showing a chat with the dictionary chatbot

Conclusion

In this tutorial, you learned how to create a WhatsApp dictionary chatbot. You learned how to use FastAPI to power the backend of your application, how to interact with Twilio MessagingX WhatsApp API, and how to use NoSQL database like MongoDB to store your data.

You can extend the chatbot to get synonyms and definitions of words with more than one explanation by accessing more metadata from the Merriam-Webster API.

You can check out the Merriam-Webster API documentation for the various responses you can get. Ensure you read the Twilio WhatsApp API docs for more advanced functionalities like getting media replies and word pronunciation.

Recently, the OpenAI team expanded their API by giving developers access to their pretrained AI models (DALL-E, Codex, and GPT-3). This means that you can send a question to the API, get the response, and use the data in your application, all within seconds.

In this article, you'll learn how to create an OpenAI account, retrieve your API keys and query OpenAI's GPT-3 model from your Node.js application. Let's dive right in!

How to Sign Up for a ChatGPT Account

The first thing you need to do is sign up for an OpenAI account if you don't already have one. Once you're in, you'll be redirected back to the homepage.

At the top right corner of the page, click on your profile image, then click on Manage Account. On the sidebar, click API Keys and then click the create new secret key button to generate a secret key:

Screenshot of secret key

Copy the secret key and paste it somewhere safe and accessible because you'll need it later to connect your application with the OpenAI API.

With the key safely stored, the next step is to create a Node.js project and spin up an Express server on top of it. Let's start with the installation and basic setup.

How to Set up the Project

To follow along with this project, you need to have Node.js and npm installed on your local machine. The latest version of Node.js comes with npm, and it's available on the official Node.js website.

Start by creating an empty directory in your computer. Next, launch the command prompt and cd into the folder you just created:

cd path/to/project

Once pointed to the directory, run the following command to create a Node project:

npm init -y

Next, run the following command to install express and openai libraries from npm:

npm i express openai

The next step is to create the server.

How to Create an Express Server

For now, this server will only serve the static files. We'll implement the chat API towards the end of this article.

Start by creating a file named server.js inside the root folder of your project. Next, open the file with a code editor and add the following code:

const express =  require('express')
const app = express()

app.use(express.static('public'))

app.listen(5000, ()=> {
    console.log("Server is active")
})

With this code, you've created a web server that serves static files (i.e. HTML, CSS) from the /public folder.

Next, we'll create the HTML file that renders the chat interface on the web page, as well as the stylesheet file and the JavaScript file.

How to Create the Chatbot

Start by creating a folder named public inside the root of your project. Then inside the /public directory, create a file named index.html.

Open the file with a text editor and add the following markup in the file:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Document</title>
</head>
<body>
    <div id="chat-area">

    </div>

    <div class="submit-form">
        <div class="input">
            <textarea name="input" id="input" cols="40" rows="3"></textarea>
            <button id="btn">Submit</button>
        </div>    
    </div>    

</body>
</html>

As you can see above, the page comprises of the chat area (where the messages are displayed) and the submit area (comprising the text area and the submit button).

To style the page, add the following stylesheet between the opening and closing <head> tags in your /public/index.html file:

    <style>
        #chat-area {
            margin: 0 auto;
            width: 80%;
            height: 500px;
            overflow:scroll;
            border: 1px solid gray;
            border-radius: 4px;
        }

        .input {            
            width: 100%;
        }

        .submit-area{
            justify-content: center;
            display: flex;
            margin: 20px auto;
            width: 80%;            
        }       

        textarea {
            width: 100%;
        }

        .box {
            width: 96%;
            margin: 0 auto;
            padding: 10px 10px;
            background-color: #C4DBFE;
            margin: 10px auto;            
        }

        .answer {
            background-color: aquamarine;
        }

        button {
            background-color: #004089;
            color: white;
            padding: 10px 10px;
            border-radius: 5px;
            border: none;
        }
    </style>

If you save the file and check the browser, you should find your page like this:

Screenshot of the page

The chat area is empty for now because we haven't submitted any messages yet. To do that, we need to bring in JavaScript.

How to Add Front-end JavaScript

When the user inputs a message in the text area and clicks on the submit button, we'll send the message to the backend, get the response from the API and display it on the page.

Start by adding an empty script element within the <body> tags in index.html:

<script>

</script>

Inside the script tags, we're to call the getResponse() function whenever the user clicks on the submit button:

const btn = document.getElementById("btn")

btn.addEventListener('click', getResponse)

The getResponse function essentially gets the user's question, sends it to our Node.js backend to fetch the answer, and displays the response on the page.

Inside the function, we start by accessing the prompt submitted by the user:

async function getResponse() {                  
  var inputText = document.getElementById("input").value           
  const parentDiv = document.getElementById("chat-area") 

  // The remaining code goes inside this function
}

If the value of the text area is empty, we simply return nothing:

if(inputText === '') { return }

Otherwise, we first add the question to the chat area in the UI:

const question = document.createElement('div')
question.innerHTML = inputText
question.classList.add("box")
parentDiv.appendChild(question)

Next, we reset the text area so it's blank:

document.getElementById("input").value = ''

Then we send the question to our server so that the server can send it to the OpenAI API and send us back a response:

let res = await fetch('http://localhost:5000/chat', 
  {
    method: 'POST',
    headers: {
      "Content-Type": 'application/json'                
    },
    body: JSON.stringify({
      question: inputText          
    })
  }
)

const data = await res.json()

If the response has a message property, we add the message content to the chat area in the UI:

if(data.message) {
  const answer = document.createElement('div')
  answer.innerHTML = data.message
  answer.classList.add("box", "answer")
  parentDiv.appendChild(answer)
}

Now the frontend is all set. Let's move our focus back to the backend.

How to Send the API Response to the Client

Our backend will serve as the middleman between the frontend and OpenAI's API. Basically, we'll get the prompt from the client, send it to the API, and send the response back to the client.

In server.js, import these at the top of the file:

const { OpenAI } = require("openai")

Next, create an instance of the openai connection using the API key you generated earlier:

const openai = new OpenAI({
    // replace your-api-key with your API key from ChatGPT
    apiKey: 'your-api-key'
})

Finally, create the route:

app.post('/chat', async (req, res)=> {   
  try {
    const resp = await openai.chat.completions.create({
      model: "gpt-3.5-turbo",
        messages: [
          { role: "user", content: req.body.question}
        ]  
    })           

    res.status(200).json({message: resp.choices[0].message.content})
  } catch(e) {
      res.status(400).json({message: e.message})
  }
})

Save the file changes, then go to your browser and submit a question. You should get back a response after a few seconds.

You can ask as many questions as you want, but you'll have to wait for a response to each question from the backend.

Screenshot of the questions and corresponding answers from ChatGPT

If any error is encountered, check the console in your browser to inspect the error message.

Conclusion

OpenAI's API offers you a way to include AI-powered chatbots in your application using JavaScript or even HTMX (if you're knowledgeable of HTML but not JavaScript).

Connect with me on Twitter and LinkedIn.

Chatbots are transforming the way we interact online. Thanks to the OpenAI API, crafting intelligent, context-aware chatbots is now well within the reach of any budding web developer.

In this tutorial, I will teach you everything you need to know to build your own chatbot using the GPT-4 API.

For a deeper dive into the OpenAI API, I have created a 4.5 hour course, "Build AI Apps with ChatGPT, DALL-E, and GPT-4", which you can find on FreeCodeCamp’s YouTube Channel and Scrimba.

Table of Contents

• The app we're building
• Prerequisites
• The HTML and CSS for the app
• How to store the API key
• How to import the API key to index.js
• How to install the OpenAI dependency
• How to use the OpenAI dependency
• The flow of this app
• An array to store the conversation
• How to handle the user's input
• Time for the AI
• How to implement the typewriter effect
• Conclusion

The App We’re Building

Screenshot of the app you'll build

Meet KnowItAll, a ChatBot with extraordinary conversational powers. You can ask it questions, have it create content, correct language, suggest edits, or translate. It can even write code upon request.

Prerequisites

In this tutorial you will be using HTML, CSS and vanilla JavaScript. A basic understanding of JavaScript is enough – you don’t need to be super advanced.

You’ll also need a free OpenAI account, which you can get here. The complimentary credits you get on signing up should be more than enough to complete this tutorial. As you go through the sign-up process, be sure to copy and paste your API key somewhere safe, as you will need it soon.

This tutorial uses the GPT-4 model. At time of writing, there is a waiting list for GPT-4 (you can join it here). But don’t worry if you haven’t got access to it yet, the GPT-3.5-turbo model is fully compatible with everything we do in this tutorial, and it is available to all now.

And if you don’t know what an OpenAI model is, don’t worry, we will be talking more about them in just a moment.

OK, let’s get down to some code!

The HTML and CSS for the App

Before we dive into the JavaScript and AI components, we need to establish the HTML and CSS foundations of this app. Below, I’ve embedded a Scrim, which is an interactive code editor/screencast. In this scrim, you can:

• Browse the HTML and CSS used in this project
• Click on PREVIEW to see the project in a mini browser
• Watch a walkthrough explanation of the HTML and CSS and pause anytime to play with the code.

And if you want to run this code locally, you can click the gear icon (⚙️) bottom right and select Download as zip. You will get a zipped folder with all of the HTML, CSS and the image assets. You can unzip that folder and open it in VS Code or whichever dev environment you favour.

Click here for a full-screen version of this scrim.

I want to draw your attention to lines 22-24 of index.html which contain a hard-coded speech bubble from the chatbot to get the conversation started:

<div class="speech speech-ai">
    How can I help you?
</div>

As you can see from the screenshot near the top of this article, each conversation starts with the chatbot asking How can I help you? Note the two CSS classes speech and speech-ai, which style the speech bubble.

Apart from that, there’s nothing particularly unusual going on with this HTML and CSS and we won’t be referring to it much in this tutorial. But do take a moment to either go through it or watch the walkthrough before moving on.

How to Store the API Key

As the OpenAI API is central to this project, you need to store the OpenAI API key in the app.

⚠️ Remember – your API key is vulnerable in this front-end only project. When you run this app in a browser, your API key will be visible in dev tools, under the network tab. That means you should only run this project locally.

If you’d like to deploy this project so you can share it and include it in your portfolio, in the third section of the full YouTube/Scrimba course I show how you can use Netlify Serverless Functions to keep the API key safely hidden when deploying.

Ok, with that warning out of the way, let’s move forward. In your project folder, create a new file called env.js to hold your API key.

Inside env.js set up a const process which will hold an object with a single property env. This will store your API key in a key/value pair, where the key is OPENAI_API_KEY and the value is the API key itself. (Please note, the API key in the code below is not real!)

export const process = {
    env: {
        OPENAI_API_KEY: "sk-123456789123456789123456789123456789123456789123"
    }
}

Here’s an updated scrim showing all of the project code so far:

How to Import the API Key to index.js

Next, at the top of index.js, import your API key from env.js with this import statement.

import { process } from '/env.js'

This is a named import which means you include the name of the entity you are importing in curly braces. Now, the entire process object will be available in index.js.

As you are using JavaScript from more than one file, you need to tell the browser to expect modular JavaScript. So add type=”module” to the script tag in index.html.

<script src="index.js" type="module"></script>

And just to check you haven’t made any typos that cause you bugs down the line, log out process from within index.js.

import { process } from '/env.js'

console.log(process)
//{env: {OPENAI_API_KEY: "sk-123456789123456789123456789123456789123456789123"}}

Here’s a scrim with the code so far:

How to Install the OpenAI Dependency

You can access the OpenAI API with a regular fetch request, but it’s much easier to use the OpenAI dependency. If you’re working locally you can add it using NPM:

$ npm install openai

Or if you are working in Scrimba, hover your cursor over DEPENDENCIES and click on the three dot icon which appears.

The three dot menu adjacent to dependencies

From the dropdown, select Add module.

Dropdown menu with 'add module' option

This will bring up a dialogue box. Type in openai, click ADD, and Scrimba will do the rest.

The Add NPM package dialogue box

You will then see the OpenAI dependency listed in the sidebar.

Whether you’re working locally or in Scrimba, you now have the OpenAI dependency installed.

How to Use the OpenAI Dependency

You need to import two constructors from the dependency. They are needed to configure the app to use the API. They are Configuration and OpenAIApi and you can import them by adding the following line of code to index.js:

import { Configuration, OpenAIApi } from 'openai'

To interact with the API you need to set up your own configuration (note the lowercase ‘c’) object using the Configuration constructor.

const configuration = new Configuration()

Next, pass it an object with a key/value pair. The key will be apiKey and the value will be our API key which you have imported from process and can access with process.env.OPENAI_API_KEY.

const configuration = new Configuration({
    apiKey: process.env.OPENAI_API_KEY
})

Then, you need to pass configuration to the OpenAIApi constructor. Save this new instance of OpenAIApi to a const openai (note the lowercase ‘o’).

const openai = new OpenAIApi(configuration)

And to check it’s working, log out openai.

console.log(openai)
//OpenAIApi$1 {basePath: "<https://api.openai.com/v1>", configuration: {apiKey: "sk-12345678912345678912345678...", baseOptions: {headers: {User-Agent: "OpenAI/NodeJS/3.2.1", Authorization: "Bearer sk-1234567891234567891..."}}}}

Here’s a scrim with all the code so far:

Remember you can click the gear icon (⚙️) to download all of the code in one zipped folder.

Now that you have finished setting up the OpenAI API dependency, you can proceed to its usage. But before you continue writing more code, let's take a moment to envision how this chatbot will work.

The Flow of this App

One major challenge when working with OpenAI models is that they don't retain the queries or replies from previous interactions. In fact, they have no memory of their past conversations with you at all. This can lead to disjointed and confusing conversations like this:

Human: Who was the first person to walk on the moon?

AI: Neil Armstrong.

Human: How old was he at the time?

AI: How old was who?

Human: ???

Therefore, to create a chatbot capable of engaging in a coherent conversation, we need to provide the OpenAI model with a form of memory.

Fortunately, there is a straightforward way of doing that: we include the entire conversation as it currently stands with each API call. This allows the API to utilise the conversation's context when forming completions to better understand the questions it is being asked.

So the app works like this:

The user types a question into the input field and hits send.

The hard-coded message How can I help you? is displayed. The user types in a question or a request and hits enter or presses the send button.

The question is rendered to the DOM.

The question is rendered to the DOM in a green speech bubble and the input is cleared.

The question is stored in an array. This array is the single source of truth for the conversation.

The question is stored in an array. This is the array that will hold the entire conversation and acts as a single source of truth. This allows the app to have a "memory" of the conversation so it can understand requests and contextualise its responses.

The array is sent to the OpenAI API.

The array is sent off to the API. As the conversation grows, this array will hold more and more elements.

The OpenAI API sends back a response with the answer. (This is known as a completion.)

The OpenAI API sends back a response. Within that response is the actual language generated by the AI model. This is called the completion.

The completion is stored in the array and rendered to the DOM.

The completion is added to the array holding the conversation so that it can be used to contextualise any future requests to the API. The completion is also rendered to the DOM so the user can see it.

The user continues the conversation.

The user now continues the conversation. And whatever they type in is rendered to the DOM.

The user’s input is added to the conversation array and the entire array is sent off to the API.

❗️Step 8 is particularly important because here the question How many people live there? does not mention Paris, so the API can only answer correctly if it is getting the context of the conversation from the array we are sending with each request.

The OpenAI API’s response shows it understands the context of the question.

From its response, we can see that the API does have the context of the conversation from the array – it knew we were talking about Paris even though Paris was not mentioned in the question How many people live there? So now we can be sure that we will be able to have a logical, flowing conversation with the chatbot.

Now you have an overview of how the app is going to work, let’s get into the nuts and bolts of the AI.

An Array to Store the Conversation

As mentioned previously, the OpenAI API needs to be provided with the conversation as it exists at that time with each API call. The conversation should be structured as an array of objects, with each object following a specific format.

First, set up an array and call it conversationArr.

const conversationArr = []

Each element in this array will be an object with two key/value pairs. The first key will be role and the second key will be content. This structure will be consistent for all objects stored in the array throughout the project.

The first object in the array will contain instructions for the chatbot. This object, known as the instruction object, allows you to control the chatbot's personality and provide behavioural instructions, specify response length, and more.

The instruction object’s role property should contain the string ‘system' and the content should hold your instruction in a string.

Here are a few example instructions:

‘You are a useful assistant'

‘You reply in French'

‘You translate whatever I say into Spanish'

‘You are a useful assistant that gives long, detailed answers'

So conversationArr with the instruction object looks like this.

const conversationArr = [
    { 
        role: 'system',
        content: 'You are a useful assistant.' // this is the instruction
    }
]

And as the instruction object won’t change, let’s hard code it and put it in index.js.

So far, our code looks like this:

Before we move on, let’s look at the other two types of Object that you will be storing in conversationArr. And just to be clear, you won’t be hard-coding these in index.js now, but adding them programmatically as needed.

When the user submits some text, that text will be stored in an object in conversationArr and it will look like this, with the role being ‘user' and the content being the text the user has submitted.

{ 
    role: 'user',
    content: 'What is the capital of France?' // your question
}

And the API’s response will also be stored in an Object. Here the role will be ‘assistant' and the content will be the completion text, for example:

{ 
    role: 'assistant',
    content: 'The capital of France is Paris.' // the completion
}

All of the objects that end up in conversationArr as it grows will follow this same pattern, with role and content properties.

So now let’s deal with what happens when the user types in some text and presses the submit button.

How to Handle the User’s Input

Your next task is to take the user’s input and render it to the DOM. The div that holds the conversation in index.html has the id of chatbot-conversation. So in index.js take control of that div and save it to a const chatbotConversation.

const chatbotConversation = document.getElementById('chatbot-conversation')

And now you need to hook up an event listener which fires when a user submits some text. Another quick look at index.html confirms that the button is inside a form element:

<form id="form" class="chatbot-input-container">
    <input name="user-input" type="text" id="user-input" required>
        <button id="submit-btn" class="submit-btn">
            <img 
            src="images/send-btn-icon.png" 
            class="send-btn-icon"
            >
    </button>
</form>

So clicking on the button or hitting enter will trigger a submit event. That’s what you need the event listener to listen out for. And as you want to prevent the browser reloading with a query string in the URL, you need to add e.preventDefault().

document.addEventListener('submit', (e) => {
    e.preventDefault()
})

Rendering the User’s Message

When a user submits a message, you need to render it. There are five stages to that process:

1. Take control of the text input field, which has the id user-input.
2. Use createElement to create a new div.
3. Add the CSS classes the div needs: speech which is the generic speech bubble class, and speech-human which applies styles that differentiate the human speech bubble from the AI speech bubble.
4. Append that speech bubble to chatbotConversation.
5. Set the speech bubble’s textContent to the user’s input, which you can get from userInput.value.

document.addEventListener('submit', (e) => {
    e.preventDefault()
    // 1. take control of the text input field
    const userInput = document.getElementById('user-input')  
    // 2. create the new element
    const newSpeechBubble = document.createElement('div')
    // 3. give it CSS classes
    newSpeechBubble.classList.add('speech', 'speech-human')
    // 4. append the speech bubble to the conversation
    chatbotConversation.appendChild(newSpeechBubble)
    // 5. add the user's inputted text to the speech bubble
    newSpeechBubble.textContent = userInput.value
})

Updating conversationArr with the User's Input

Having rendered the message to the DOM, you now need to push an object to conversationArr in the format we looked at previously. This object will have a role of ‘user’ and the content property will hold whatever the user has typed in the text input.

document.addEventListener('submit', (e) => {
    e.preventDefault()

    const userInput = document.getElementById('user-input')  
    const newSpeechBubble = document.createElement('div')
    newSpeechBubble.classList.add('speech', 'speech-human')
    chatbotConversation.appendChild(newSpeechBubble)
    newSpeechBubble.textContent = userInput.value

    // push object to conversationArr
    conversationArr.push({ 
        role: 'user',
        content: userInput.value
    })

})

Clearing the input field and scrolling to the Bottom

There are two final jobs this event listener’s function needs to do.

Firstly, let’s clear the text input by setting it to an empty string. And secondly, let’s scroll the conversation to the bottom so the user doesn’t have to manually scroll down. You could use the scrollIntoView method to do that, but because I coded this for a mini browser and scrollIntoView was causing problems, I used a slightly different technique as shown below:

document.addEventListener('submit', (e) => {
    e.preventDefault()

    const userInput = document.getElementById('user-input')  
    const newSpeechBubble = document.createElement('div')
    newSpeechBubble.classList.add('speech', 'speech-human')
    chatbotConversation.appendChild(newSpeechBubble)
    newSpeechBubble.textContent = userInput.value

    conversationArr.push({ 
        role: 'user',
        content: userInput.value
    })

    // empty the text input
    userInput.value = ''
    // scroll the conversation to the bottom
    chatbotConversation.scrollTop = chatbotConversation.scrollHeight
})

So our code so far looks like this:

Now, if you ask the question What is the capital of France? and log out conversationArr from within the event listener’s function, you get this:

console.log(conversationArr)
//[{role: "system", content: "You are a useful assistant."}, {role: "user", content: "What is the capital of France?"}]

Time for the AI

It’s time to use the OpenAI API to actually generate some text. The dependency makes it a really easy API to use – you just need three pieces of information.

1. An endpoint
2. A model
3. Our conversation in an array

You’ve already set up conversationArr to deal with number 3 on that list, but before you make a request to the API, let’s look at 1 and 2 in more detail.

Endpoints

The OpenAI API has various endpoints available. Which one you use depends on what you want the AI to do (generate language, generate code, create images from text prompts, and so on).

For this chatbot, we will be using the chat/completion endpoint, which at the time of writing is the most advanced endpoint for natural language generation in the OpenAI stable.

Models

The model (sometimes called the engine) is what actually creates the language. Our model for this project is GPT-4. GPT-4 is on limited release via a waiting list at present, so if you can’t access it right now, you can use GPT-3.5-turbo instead. All code in this project works with both models, and GPT-3.5-turbo is also highly capable.

OK, let’s make a call to the API.

The fetchReply Function

We need a function to take on the job of making a request to the API. Let's call this function fetchReply. The basics of the function look like this:

async function fetchReply(){
    const response = await openai.createChatCompletion() 
}

Inside the body of the function, we’ve got a const response and we’ve awaited a call to the chat/completion endpoint which you can reach by taking the instance of the OpenAiAPI constructor you stored in the const openai earlier and calling the createChatCompletion method on it. (createChatCompletion is an OpenAI API method that gives us access to the chat/completion endpoint. For more info, check the OpenAI API docs.)

Because the dependency is making a fetch request, you need to use the await keyword and make this an async function.

Next, you need to pass createChatCompletion an object, and that object needs two properties: model and messages.

Model

Our model is GPT-4. You can specify this by putting it in a string in lowercase in a key/value pair: model: 'gpt-4'. This is where you could also use model: ’gpt-3.5-turbo' if you don’t have access to GPT-4 yet.

Messages

The messages property just needs to hold our conversation, which you have stored as an array of objects in the const conversationArr.

For now, log out the response. So the finished code looks like this:

async function fetchReply(){
    const response = await openai.createChatCompletion({
        model: 'gpt-4', // or 'gpt-3.5-turbo'
        messages: conversationArr,
    })
    console.log(response)
}

Now, call fetchReply from within the event listener’s function.

document.addEventListener('submit', (e) => {
    e.preventDefault()
    const userInput = document.getElementById('user-input')   
    conversationArr.push({ 
        role: 'user',
        content: userInput.value
    })
    const newSpeechBubble = document.createElement('div')
    newSpeechBubble.classList.add('speech', 'speech-human')
    chatbotConversation.appendChild(newSpeechBubble)
    newSpeechBubble.textContent = userInput.value
    userInput.value = ''
    chatbotConversation.scrollTop = chatbotConversation.scrollHeight
    console.log(conversationArr)

    // call fetch reply to trigger the API call
    fetchReply()
})

Chatbot with question from the user.

And when you type in What is the capital of France? and hit send, you get back this massive object: (Feel free to scroll through it, but don't be intimidated!)

{data: {id: "chatcmpl-7MuziItZYyiFpPG2KHQawd19rD54U", object: "chat.completion", created: 1685696658, model: "gpt-4-0314", usage: {prompt_tokens: 28, completion_tokens: 36, total_tokens: 64}, choices: [{message: {role: "assistant", content: "The capital of France is Paris."}, finish_reason: "stop", index: 0}]}, status: 200, statusText: "", headers: {cache-control: "no-cache, must-revalidate", content-type: "application/json"}, config: {transitional: {silentJSONParsing: true, forcedJSONParsing: true, clarifyTimeoutError: false}, adapter: xhrAdapter(e), transformRequest: [transformRequest(e,t)], transformResponse: [transformResponse(e)], timeout: 0, xsrfCookieName: "XSRF-TOKEN", xsrfHeaderName: "X-XSRF-TOKEN", maxContentLength: -1, maxBodyLength: -1, validateStatus: validateStatus(e), headers: {Accept: "application/json, text/plain, */*", Content-Type: "application/json", User-Agent: "OpenAI/NodeJS/3.2.1", Authorization: "Bearer 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"}, method: "post", data: "{"model":"gpt-4","messages":[{"role":"system","content":"You are a highly knowledgeable assistant that is always happy to help."},{"role":"user","content":"What is the capital of france?"}]}", url: "https://api.openai.com/v1/chat/completions"}, request: XMLHttpRequest {}}

There’s lots of useful info here, but we need to focus in on this one part, which I have formatted for readability:

choices: [
    {
        message: {
            role: "assistant", 
            content: "The capital of France is Paris."
        }, 
        finish_reason: "stop", index: 0
    }
]

This is where we can see the completion: The capital of France is Paris. And that is what you need to render to the DOM in a speech bubble. And you can use dot and bracket notation to get to that text.

Let’s log that out.

console.log(response.data.choices[0].message.content)
//The capital of France is Paris.

But before you render anything, remember you also need to include each piece of dialogue in conversationArr. And the format that you need for that is an object with two key/value pairs where one key is role and has the value ’assistant’, and the other is content and holds the completion as its value.

And that object is exactly what is given to you in response.data.choices[0].message – yes, the object that you need to add to conversationArr is actually provided to you by the API!

You can adjust the above console.log to prove it:

console.log(response.data.choices[0].message)
//{role: "assistant", content: "The capital of France is Paris."}

Now you can go ahead and make fetchReply push this object to conversationArr.

And let’s log out conversationArr to check that it works:

async function fetchReply(){
    const response = await openai.createChatCompletion({
        model: 'gpt-4',
        messages: conversationArr,
    })
    conversationArr.push(response.data.choices[0].message)
    console.log(conversationArr)
}

// [{role: "system", content: "You are a useful assistant."}, {role: "user", content: "What is the capital of France?"}, {role: "assistant", content: "The capital of France is Paris."}]

The project code should now look like this:

Now all that’s left to do is render out the completion.

How to Implement the Typewriter Effect

The last task is to make our chatBot type out its response. There are thousands of ways you could do this, and it is possible to do it only with CSS. We are going to do it with JavaScript.

Create a function called renderTypewriterText. This function will take in a parameter which will be the text string you get from the response.

The renderTypewriterText function needs to create a new speech bubble element, give it CSS classes, and append it to chatbotConversation. This is almost the same code as you used before for the user’s input, but note here that you will also need to give the speech bubble the blinking-cursor class, which uses a CSS animation to create a cursor effect while the text is being rendered. See index.css lines 151 onwards in the above scrim for the CSS.

function renderTypewriterText(text) {
    const newSpeechBubble = document.createElement('div')
    newSpeechBubble.classList.add('speech', 'speech-ai', 'blinking-cursor')
    chatbotConversation.appendChild(newSpeechBubble)
}

Now add some logic to render each character one by one:

function renderTypewriterText(text) {
    const newSpeechBubble = document.createElement('div')
    newSpeechBubble.classList.add('speech', 'speech-ai', 'blinking-cursor')
    chatbotConversation.appendChild(newSpeechBubble)

    // render each character one by one 
    let i = 0
    const interval = setInterval(() => {
        newSpeechBubble.textContent += text.slice(i-1, i)
        if (text.length === i) {
            clearInterval(interval)
            newSpeechBubble.classList.remove('blinking-cursor')
        }
        i++
        chatbotConversation.scrollTop = chatbotConversation.scrollHeight
    }, 50)
}

That’s quite a jumble of JavaScript, so let’s work through it step by step.

1. let i = 0: This initialises a variable i with a value of 0. It will be used to keep track of the current index of the text string.
2. const interval = setInterval(() => { ... }, 50): This creates an interval that repeatedly executes the arrow function every 50 milliseconds. The arrow function contains the code that will be executed at each interval.
3. newSpeechBubble.textContent += text.slice(i-1, i): This line appends a portion of the text string to the content of the newSpeechBubble element. It uses the slice method to extract a single character from the text string based on the current value of i.
4. if (text.length === i) { ... }: This condition checks if the entire text string has been appended to the speech bubble. If the length of the text string is equal to i, it means that all characters have been appended.
5. clearInterval(interval): This line clears the interval, stopping the execution of the function.
6. newSpeechBubble.classList.remove('blinking-cursor'): This removes the CSS class 'blinking-cursor' from the newSpeechBubble element. It removes the blinking cursor effect once the entire text string has been displayed. You only want the blinking cursor while the typewriter is running.
7. i++: This increments the value of i by 1, moving to the next character in the text string for the next interval execution.
8. chatbotConversation.scrollTop = chatbotConversation.scrollHeight: This scrolls the conversation container to the bottom, ensuring that the new content is always visible.

To finish wiring this up, call renderTypewriterText from inside fetchReply, remembering to pass in the text completion that comes back from the API.

async function fetchReply(){
    const response = await openai.createChatCompletion({
        model: 'gpt-4',
        messages: conversationArr,
    }) 
    conversationArr.push(response.data.choices[0].message)

    // call renderTypewriterText passing in the completion
    renderTypewriterText(response.data.choices[0].message.content)
}

And you are done!

The finished app with a conversation between a user and the AI chatbot

We now have a fully working chatbot using the GPT-4 API and you can continue the conversation for as long as you want!

Well, that’s not quite true. There is a theoretical limit to how long the conversation can be, but you would have to carrying on chatting for a long time to reach it. We talk about that more in the full course. Also, it's important to note that at some point, you may hit your credit limit.

Here’s the finished code. And as before, you can hit the gear icon ⚙️ and download it.

Conclusion

Congratulations on successfully building your own chatbot using the GPT-4 API! With GPT-4, you've unlocked a world of possibilities in natural language processing and conversation generation.

As you continue on your AI journey, remember to stay curious, keep learning, and explore the evolving field of artificial intelligence. Share your creations, collaborate with others, and be part of the AI community. Happy coding!

Always feel free to reach out to me on Twitter. I am @tpchant.

You may have read about the large number of AI apps that have been released over the last couple of months. You may have even started using some of them.

AI tools such as ChatPDF and CustomGPT AI have become very useful to people – and for good reason. Gone are the days where you need to scroll through a 50-page document just to find a simple answer. Instead, you can rely on AI to do the heavy lifting.

But how exactly are all these developers creating and using these tools? Well, many of them are using an open source framework called LangChain.

In this article, I'm going to introduce you to LangChain and show you how it's being used in combination with OpenAI's API to create these game-changing tools. Hopefully, I'll inspire one of you to come up with one of your own. So let's jump in!

What Is LangChain?

LangChain is an open source framework that allows AI developers to combine Large Language Models (LLMs) like GPT-4 with external data. It's offered in Python or JavaScript (TypeScript) packages.

As you may know, GPT models have been trained on data up until 2021, which can be a significant limitation. And while these models' general knowledge is great, being able to connect them to custom data and computations opens up many doors. That's exactly what LangChain does.

Essentially, it allows your LLM to reference entire databases when coming up with its answers. So you can now have your GPT models access up-to-date data in the form of reports, documents, and website info.

Recently, LangChain has experienced a significant surge in popularity, especially after the launch of GPT-4 in March. This was thanks to its versatility and the many possibilities it opens up when paired with a powerful LLM.

How Does LangChain Work?

While you may be thinking that LangChain sounds pretty complicated, it's actually quite approachable.

In short, LangChain just composes large amounts of data that can easily be referenced by a LLM with as little computation power as possible. It works by taking a big source of data, take for example a 50-page PDF, and breaking it down into "chunks" which are then embedded into a Vector Store.

Simple Diagram of creating a Vector Store

Now that we have vectorized representations of the large document, we can use this in conjunction with the LLM to retrieve only the information we need to be referenced when creating a prompt-completion pair.

When we insert a prompt into our new chatbot, LangChain will query the Vector Store for relevant information. Think of it as a mini-Google for your document. Once the relevant information is retrieved, we use that in conjunction with the prompt to feed to the LLM to generate our answer.

How LangChain Works With OpenAI's LLMs

LangChain also allows you to create apps that can take actions – such as surf the web, send emails, and complete other API-related tasks. Check out AgentGPT, a great example of this.

There are many possible use-cases for this – here are just a few off the top of my head:

• Personal AI Email Assistant
• AI Study Buddy
• AI Data Analytics
• Custom Company Customer Service Chatbots
• Social Media Content Creation Assistant

And the list goes on. I will cover proper build tutorials in future articles, so stay tuned for that.

How to Get Started with LangChain

A LangChain application consists of 5 main components:

1. Models (LLM Wrappers)
2. Prompts
3. Chains
4. Embeddings and Vector Stores
5. Agents

I am going to give you an overview of each, so that you can get a high-level understanding of how LangChain works. Moving forward, you should be able to apply the concepts to start to craft your own use-cases and create your own apps.

I'll be explaining everything with short code snippets from Rabbitmetrics (Github). He provides great tutorials on this topic. These snippets should allow you to get all set up and ready to use LangChain.

First, let's get our environment set up. You can pip install 3 libraries that you'll need for this:

pip install -r requirements.txt

python-dotenv==1.0.0
langchain==0.0.137
pinecone-client==2.2.1

Pinecone is the Vector Store that we will be using in conjunction with LangChain. With these, make sure to store your API keys for OpenAI, Pinecone Environment, and Pinecone API into your environment file. You will be able to find this info at their respective websites. Then we just load that environment file with the following:

# Load environment variables

from dotenv import load_dotenv,find_dotenv
load_dotenv(find_dotenv())

Now, we're ready to get started!

Models (LLM Wrappers)

To interact with our LLMs, we are going to instantiate a wrapper for OpenAI's GPT models. In this case, we are going to use OpenAI's GPT-3.5-turbo, as it's the most cost-efficient. But if you have access, feel free to use the more powerful GPT4.

To import these, we can use the following code:

# import schema for chat messages and ChatOpenAI in order to query chatmodels GPT-3.5-turbo or GPT-4

from langchain.schema import (
    AIMessage,
    HumanMessage,
    SystemMessage
)
from langchain.chat_models import ChatOpenAI


chat = ChatOpenAI(model_name="gpt-3.5-turbo",temperature=0.3)
messages = [
    SystemMessage(content="You are an expert data scientist"),
    HumanMessage(content="Write a Python script that trains a neural network on simulated data ")
]
response=chat(messages)

print(response.content,end='\n')

In essence, the SystemMessage provides context to the GPT-3.5-turbo module that it will reference for each prompt-completion pair. The HumanMessage refers to what you would type into the ChatGPT interface – your prompt.

But with a custom-knowledge chatbot, we often abstract away the repetitive components of a prompt. For example, if I was creating a Tweet generator app, I wouldn't want to keep typing "Write me a Tweet about...". In fact, that's how simple AI writing tools are developed!

So let's look at how we can abstract that away with prompt templates.

Prompts

LangChain provides PromptTemplates that allow you to dynamically change the prompts with user input, similar to how regex are used.

# Import prompt and define PromptTemplate

from langchain import PromptTemplate

template = """
You are an expert data scientist with an expertise in building deep learning models. 
Explain the concept of {concept} in a couple of lines
"""

prompt = PromptTemplate(
    input_variables=["concept"],
    template=template,
)


# Run LLM with PromptTemplate

llm(prompt.format(concept="autoencoder"))
llm(prompt.format(concept="regularization"))

You can vary these in different ways to fit your use-case. If you are familiar with using ChatGPT, this should be comfortable for you.

Chains

Chains allow you to take simple PromptTemplates and build functionality on top of them. Essentially, chains are like composite functions that allow you to integrate your PromptTemplates and LLMs together.

Using the wrappers and PromptTemplates from earlier, we can run the same prompts with a single chain that takes a PromptTemplate and composes it with an LLM:

# Import LLMChain and define chain with language model and prompt as arguments.

from langchain.chains import LLMChain
chain = LLMChain(llm=llm, prompt=prompt)

# Run the chain only specifying the input variable.
print(chain.run("autoencoder"))

On top of that, as the name suggests, we can chain these together to create even bigger compositions.

For example, I can take the result from one chain and pass it into another chain. In this snippet, Rabbitmetrics takes the completion from the first chain and passes it into the second chain to explain it to a 5 year old.

You can then combine those chains into a larger chain and run that.

# Define a second prompt 

second_prompt = PromptTemplate(
    input_variables=["ml_concept"],
    template="Turn the concept description of {ml_concept} and explain it to me like I'm five in 500 words",
)
chain_two = LLMChain(llm=llm, prompt=second_prompt)

# Define a sequential chain using the two chains above: the second chain takes the output of the first chain as input

from langchain.chains import SimpleSequentialChain
overall_chain = SimpleSequentialChain(chains=[chain, chain_two], verbose=True)

# Run the chain specifying only the input variable for the first chain.
explanation = overall_chain.run("autoencoder")
print(explanation)

With chains, you can create a huge array of functionality, which is what makes LangChain so versatile. But where it really shines is using it in conjunction with a Vector Store as discussed earlier. Let's introduce that component.

Embeddings and Vector Stores

This is where we incorporate the custom data aspect of LangChain. As mentioned earlier, the idea behind embeddings and Vector Stores is to break large data into chunks and store those to be queried when relevant.

LangChain has a text splitter function to do this:

# Import utility for splitting up texts and split up the explanation given above into document chunks

from langchain.text_splitter import RecursiveCharacterTextSplitter

text_splitter = RecursiveCharacterTextSplitter(
    chunk_size = 100,
    chunk_overlap  = 0,
)

texts = text_splitter.create_documents([explanation])

Splitting up text requires two parameters: How big a chunk is (chunk_size) and how much each chunk overlaps (chunk_overlap). Having an overlap between each chunk is important to help identify relevant adjoining chunks.

Each of those chunks can be retrieved as such:

texts[0].page_content

After we have those chunks, we need to turn them into embeddings. This allows the Vector Store to find and return each chunk when queried. We'll use OpenAI's embedding model to do this.

# Import and instantiate OpenAI embeddings

from langchain.embeddings import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(model_name="ada")


# Turn the first text chunk into a vector with the embedding

query_result = embeddings.embed_query(texts[0].page_content)
print(query_result)

And finally, we need to have a place to store these vectorized embeddings. As mentioned earlier, we will be using Pinecone for this. Using the API keys from the environment file earlier, we can initialize Pinecone to store our embeddings.

# Import and initialize Pinecone client

import os
import pinecone
from langchain.vectorstores import Pinecone


pinecone.init(
    api_key=os.getenv('PINECONE_API_KEY'),  
    environment=os.getenv('PINECONE_ENV')  
)


# Upload vectors to Pinecone

index_name = "langchain-quickstart"
search = Pinecone.from_documents(texts, embeddings, index_name=index_name)


# Do a simple vector similarity search

query = "What is magical about an autoencoder?"
result = search.similarity_search(query)

print(result)

And now we are able to query relevant information from our Pinecone Vector Store! All that's left to do is to combine what we have learned to create our specific use case – giving us our specialized AI "Agent".

Agents

An agent is essentially an autonomous AI that takes in inputs and completes those as tasks sequentially until the end goal is reached. This involves our AI using other APIs that allows it to complete tasks such as sending emails or doing math problems. Used in conjunction with our LLM + prompt chains, we can string together a proper AI app.

Now, explaining this part will be extensive, so here's a simple example of how a Python agent can be used in LangChain to solve a simple mathematical problem. This agent in this case solves the problem by connecting our LLM to run Python code, and finding the roots with NumPy:

# Import Python REPL tool and instantiate Python agent

from langchain.agents.agent_toolkits import create_python_agent
from langchain.tools.python.tool import PythonREPLTool
from langchain.python import PythonREPL
from langchain.llms.openai import OpenAI

agent_executor = create_python_agent(
    llm=OpenAI(temperature=0, max_tokens=1000),
    tool=PythonREPLTool(),
    verbose=True
)


# Execute the Python agent

agent_executor.run("Find the roots (zeros) if the quadratic function 3 * x**2 + 2*x -1")

A custom-knowledge chatbot is essentially an agent that chains together prompts and actions that query the Vectorized Storage, take the results, and chain that with the original question!

If you would like to read more about AI agents, this is a great resource.

Other Variations

Even with your newfound basic understanding of the functionality of LangChain, I'm sure you are bubbling with ideas at this point.

But we've only looked at one OpenAI model so far, and that's the text-based GPT-3.5-turbo. OpenAI has an array of models that you could use with LangChain – including image generation with Dall-E. Applying the same concepts we've discussed, we can create AI Art Generator agents, Website Builder agents, and much more.

Take some time to explore the AI landscape and I'm confident that you'll start getting more and more ideas.

Conclusion

I hope you've learnt a bit more on what's going on behind the scenes of all of these new AI tools. Understanding how LangChain works is a valuable skill to have as a programmer these days and can open up the possibilities for your AI development.

If you enjoyed this article and you would like to find out more about the cool new tools AI creators are building, you can stay up-to-date with my Byte-Sized AI Newsletter. There are tons of exciting stories of what people are building in the AI space, and I'd love for you to join our community.

You can also drop me a follow on Twitter, where we can also get in touch.

Other than that, start experimenting with LangChain and create some nifty AI projects.

Artificial Intelligence (AI) has been making waves lately, with ChatGPT revolutionizing the internet with the chat completion functionality.

You can do a lot with it: drafting an email or other piece of writing, answering questions about a set of documents, creating conversational agents, giving your software a natural language interface, tutoring in various subjects, translating languages, and so on.

This article will teach the basics of building a chat application using the chat completion functionality to make it easy for every programmer to get on board. It is not as tough as it looks. You will see this as you follow this tutorial.

You will learn the following:

• How to create a CLI chat app with Node.js only.
• How to build a chat application using just React.
• How to combine React and Node.js to create better chat AI software.

This tutorial will be based on the gpt-3.5-turbo model.

Prerequisites

This tutorial requires basic knowledge of JavaScript, CSS, React, and Node.js.

You also need an account on the OpenAI platform where chatGPT is hosted. It is free, so you can create one here.

How to Create a CLI Chat AI App With Node.js

This section will focus on creating a chat application that will run only on the terminal using Node.js.

Begin by creating a directory for the project:

mkdir nodejs-chatgpt-tutorial

Navigate into the folder:

cd nodejs-chatgpt-tutorial

Initialize the project:

npm init -y

This will create a package.json file to keep track of the project details

Add the following line of code to the file:

"type": "module"

This will enable you to use the ES6 module import statement.

Install OpenAI with the command below:

npm i openai

Create a file where all the code will live. Name it index.js:

touch index.js

Import Configuration and OpenAIApi from the OpenAI module and readline from the readline module:

import { Configuration, OpenAIApi } from "openai";
import readline from "readline";

Build the OpenAI configuration like this:

const configuration = new Configuration({
  organization: "org-0nmrFWw6wSm6xIJXSbx4FpTw",
  apiKey: "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",
});

This code creates a new instance of the Configuration object. Inside it, you'll enter values for your organization and apiKey. You can find details of your organization in settings and your apiKey info in API keys. If you do not have an existing API Key, you can create it.

Enter the following code after the configuration to create a new instance of the OpenAI API:

const openai = new OpenAIApi(configuration);

You'll use this throughout the project.

Type the code below to test the createChatCompletion function:

openai
  .createChatCompletion({
    model: "gpt-3.5-turbo",
    messages: [{ role: "user", content: "Hello" }],
  })
  .then((res) => {
    console.log(res.data.choices[0].message.content);
  })
  .catch((e) => {
    console.log(e);
  });

This code calls the createChatCompletion function that triggers an endpoint (https://api.openai.com/v1/chat/completions). The function accepts an object of arguments (the model of chatGPT in use and an array of messages between the user and the AI. We will look at how to use the messages array to keep a history of the chat and improve the app in the next section).

Each message is an object containing the role (that is, who sent the message. The value can be assistant if it is from the AI or user when the message originates from a human) and the content (the information sent).

Finally, the code prints the response (res.data.choices[0].message.content) from the AI. Run the file in the terminal with this command:

node index

This will return a response from the AI after some seconds.

And that is all it takes to create the chatbot!

But it will be helpful to make the application more interactive by requesting input from the user instead of hardcoding the message content into the code. The readline module will help us out in this regard.

To make it interactive, remove the last code you typed and add the following:

const userInterface = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
});

This code creates a UI in the terminal that allows users to type in their questions.

Next, prompt the user to enter a message using the code below:

userInterface.prompt();

Finally, enter the following code:

userInterface.on("line", async (input) => {
  await openai
    .createChatCompletion({
      model: "gpt-3.5-turbo",
      messages: [{ role: "user", content: input }],
    })
    .then((res) => {
      console.log(res.data.choices[0].message.content);
      userInterface.prompt();
    })
    .catch((e) => {
      console.log(e);
    });
});

In the code above,

• When a user types something and hits Enter, the code above triggers a callback function.
• It passes whatever was typed by the user as input.
• The input is now used as the content.
• After the response of the AI is displayed, the user is prompted for another message in the then block.

See all the code on GitHub.

Run the file and have a conversation with the AI. It will look like the image below:

CLI chat with AI

Great! That is an interactive CLI chat.

This is useful to a few people (like engineers), but it has good security because it is on the server side.

But how about others who might not understand how to use a CLI application? They will need something easier to use with a better user interface (UI) and user experience (UX). The following section will focus on building that kind of application using React.

How to Create a Chat Application Using React

This section is aimed at helping frontend developers get up to speed with the ChatGPT API for creating a chat app and building a better user interface to give users better experiences. You can apply the knowledge that you gain here to other frontend frameworks or libraries.

The first thing to do is to set up a basic React boilerplate. I will use Vite for this purpose. You can use Vite to scaffold any modern JavaScript frontend project. Use the command below:

npm create vite@latest

This command will prompt you to create a name and folder for your project and choose a framework or library (this tutorial uses React). After that, you will navigate into the folder and run the following command:

npm install
npm run dev

These commands will install the necessary dependencies and start the local server on port 5173

Next, install OpenAI with the command below:

npm i openai

This module gives access to all we need to create the chat app.

Now we are set to start writing the code!

Navigate into the src/App.jsx file and delete all of its content. Then add the following import statements:

import { useState } from "react";
import { Configuration, OpenAIApi } from "openai";

The code above imports the Configuration for setting up config values and OpenAIApi to give us access to the chat completion functionalities.

After that, build the configuration like this:

const configuration = new Configuration({
  organization: "org-0nmrFWw6wSm6xIJXSbx4FpTw",
  apiKey: "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",
});

This code creates a new instance of the Configuration object. Inside it, you enter values for your organization and apiKey. You can find details of your organization in settings and your apiKey info in API keys. If you do not have an existing API Key, you can create it.

Enter the following code after the configuration to create a new instance of the OpenAI API:

const openai = new OpenAIApi(configuration);

We'll use this throughout the project.

Create and export a default function:

function App() {

  return (
    <main>
      <h1>Chat AI Tutorial</h1>
    <main/>
  );
}
export default App;

This function will house the rest of the code.

Set up the following states before the return statement:

  const [message, setMessage] = useState("");
  const [chats, setChats] = useState([]);
  const [isTyping, setIsTyping] = useState(false);

• The message will hold the information sent from the app to the AI.
• The chats array will keep track of all the messages sent by both parties (user and AI).
• The isTyping variable will notify the user whether the bot is typing or not.

Type the following lines of code under the h1 tag

      <div className={isTyping ? "" : "hide"}>
        <p>
          <i>{isTyping ? "Typing" : ""}</i>
        </p>
      </div>

The code above will display Typing whenever the user is waiting for a response from the AI.

Create a form in which a user can type in a message by adding the code below into the main element:

      <form action="" onSubmit={(e) => chat(e, message)}>
        <input
          type="text"
          name="message"
          value={message}
          placeholder="Type a message here and hit Enter..."
          onChange={(e) => setMessage(e.target.value)}
        />
      </form>

This code creates a form with one input. Whenever the form is submitted by hitting the Enter key, it triggers the chat function.

The chat function will take two (2) arguments (e and message) like this:

const chat = async (e, message) => {

}

Enter the following lines in the function:

    e.preventDefault();

    if (!message) return;
    setIsTyping(true);

The code above prevents the form from reloading the web page, checks if a message was typed before submission, and sets isTyping to true to indicate that the app has started working on the input provided.

ChatGPT has a format in which messages should be. It takes the following pattern:

{role: user | assistant, content: message to be sent

Every message (content) must show who sent it. The role is assistant when the chat is from the AI but user if it is from a human. So, before sending the message, be sure to format it properly and add it to the array (chats) like this:

    let msgs = chats;
    msgs.push({ role: "user", content: message });
    setChats(msgs);

    setMessage("");

The last line above clears the input for a user to type another note.

Now we will call the createChatCompletion endpoint by triggering the createChatCompletion function using the code below:

  await openai
      .createChatCompletion({
        model: "gpt-3.5-turbo",
        messages: [
          {
            role: "system",
            content:
              "You are a EbereGPT. You can help with graphic design tasks",
          },
          ...chats,
        ],
      })

The createChatCompletion function takes at least two (2) arguments (model and messages):

• The model specifies the version of chatGPT in use.
• The messages is a list of all the messages between a user and the AI so far and a system message that gives the AI an idea of what sort of assistance it can provide.

          {
            role: "system",
            content:
              "You are a EbereGPT. You can help with graphic design tasks",
          }

You can change the content to whatever suits you.

The messages don’t have to contain more than one object in the array. It can be just one message. But when it is an array, it provides a message history that the AI can rely on to give better replies in the future, and it makes the user type less since there might be no need to be overly descriptive all the time.

The createChatCompletion function returns a promise. So use a then…catch… block to get the response.

      .then((res) => {
        msgs.push(res.data.choices[0].message);
        setChats(msgs);
        setIsTyping(false);
      })
      .catch((error) => {
        console.log(error);
      });

This code adds the message returned from the AI into the chats array and sets isTyping to false, indicating that the AI is done replying.

You should now receive feedback (Typing) each time you send a message:

Chat application giving feedback when the AI is about to respond

It is time to display the chat history for the user to see.

Type the following code just below the h1 tag:

      <section>
        {chats && chats.length
          ? chats.map((chat, index) => (
              <p key={index} className={chat.role === "user" ? "user_msg" : ""}>
                <span>
                  <b>{chat.role.toUpperCase()}</b>
                </span>
                <span>:</span>
                <span>{chat.content}</span>
              </p>
            ))
          : ""}
      </section>

The code above loops through the chats and displays them one after another to the user. It outputs the role in upper case and the content of the message side by side.

Here is what the output should look like:

ChatBot Working As Expected without CSS

That looks cool!

But adding some styling will give it an engaging look like WhatsApp or Messenger.

Replace the content of the src/index.css file with the following:

:root {
  font-family: Inter, system-ui, Avenir, Helvetica, Arial, sans-serif;
  line-height: 1.5;
  font-weight: 400;
  color-scheme: light dark;
  color: rgba(255, 255, 255, 0.87);
  background-color: #242424;
  font-synthesis: none;
  text-rendering: optimizeLegibility;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  -webkit-text-size-adjust: 100%;
}
h1 {
  font-size: 3.2em;
  line-height: 1.1;
  text-align: center;
  position: sticky;
  top: 0;
  background-color: #242424;
}
main{
  max-width: 500px;
  margin: auto;
}
p{
  background-color: darkslategray;
  max-width: 70%;
  padding: 15px;
  border-radius: 50px;
}
p span{
  margin: 5px;
}
p span:first-child{
  margin-right: 0;
}
.user_msg{
  text-align: right;
  margin-left: 30%;
  display: flex;
  flex-direction: row-reverse;
}
.hide {
  visibility: hidden;
  display: none;
}
form{
  text-align: center;
  position: sticky;
  bottom: 0;
}
input{
  width: 100%;
  height: 40px;
  border: none;
  padding: 10px;
  font-size: 1.2rem;
}
input:focus{
  outline: none;
}

And remove all the styles from the src/App.css file.

You can find the complete code on GitHub.

The application should now have a new look:

ChatBot Working As Expected with CSS

That wraps it up for creating a chatbot with React and ChatGPT. It isn’t as tough as it sounds.

But frontend applications like this one are best for demonstration, not production. The problem with creating the application this way is that the frontend exposes the API Key to cyber attacks.

To fix this problem, it may be wise to save the API Key and Organisation Id somewhere safe in the cloud and reference it or build a backend for your application with better security.

The following section will work on the problem.

How to Combine React and Node.js to Make a Fullstack Chat AI Software

This section will now join the powers of the previous sections to build a more secure application while exhibiting better UI and UX.

We will improve the Node section by using a server to expose an endpoint for the frontend’s consumption and simplify the frontend to interact with the backend instead of contacting OpenAI directly.

How to Setup the Project

This part will create the folders and files necessary for the project.

Create the project directory:

mkdir react-node-chatgpt-tutorial

Navigate into the folder:

cd react-node-chatgpt-tutorial

Install React using Vite and name the folder frontend. Use this command:

npm create vite@latest

After that, you will navigate into the folder and run the following command:

npm install
npm run dev

These commands will install the necessary dependencies and start the local server on port 5173.

Create the backend folder:

mkdir backend

Now navigate into the backend folder and initialize the project with this command:

npm init -y

This will create a package.json file to keep track of the project details.

Add the following line of code to the file:

"type": "module"

This will enable the use of the ES6 module import statement.

Install OpenAI and other dependencies with the command below:

npm i openai body-parser cors express

Create a file where all the code will live. Name it index.js:

touch index.js

That completes the project setup. There are now two folders (frontend and backend).

How to Build a Server

This part will focus on creating a local server to listen on port 8000.

The first thing to do is to import the necessary modules like this:

import { Configuration, OpenAIApi } from "openai";
import express from "express";
import bodyParser from "body-parser";
import cors from "cors";

Next, set up express, a port to listen to, the body-parser for receiving input, and cors to allow free communications between the frontend and backend. Use the code below:

const app = express();
const port = 8000;
app.use(bodyParser.json());
app.use(cors());

Finally, type the following code:

app.listen(port, () => {
  console.log(`listening on port ${port}`);
});

This completes the server setup.

When you run the index.js, you should get the following output:

listening on port 8000

How to Create an Endpoint

In this part, we will build an endpoint that will receive messages from the frontend using the request body and return a response to the caller.

Begin by establishing the configuration parameters as we did in previous sections:

const configuration = new Configuration({
  organization: "org-0nmrFWw6wSm6xIJXSbx4FpTw",
  apiKey: "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",
});
const openai = new OpenAIApi(configuration);

Next, create an asynchronous POST route using the code below:

app.post("/", async (request, response) => {

});

This endpoint will be called using http://localhost:8000/

In the callback function, enter the code below to receive the chats input from the request body (request.body):

const { chats } = request.body;

Now call the createChatCompletion endpoint as we did in the React section:

  const result = await openai.createChatCompletion({
    model: "gpt-3.5-turbo",
    messages: [
      {
        role: "system",
        content: "You are a EbereGPT. You can help with graphic design tasks",
      },
      ...chats,
    ],
  });

The difference here is that instead of using the then…catch… block, we assigned it to a variable (result) and returned the response using response.json() as in the following code:

  response.json({
    output: result.data.choices[0].message,
  });

Find the code for this part on GitHub here.

Here is the output when tested on Postman:

Output From Postman

That concludes the backend portion of the code. The next part will connect the frontend to the backend using the endpoint ( http://localhost:8000/) just created.

How to Connect to the Backend from the Frontend.

This part takes us to the frontend, where we will create a form. The form will send a message to the backend via the API endpoint and receive a response through the same medium.

Navigate to the frontend/src/App.jsx file and type the following code:

import { useState } from "react";

function App() {
  const [message, setMessage] = useState("");
  const [chats, setChats] = useState([]);
  const [isTyping, setIsTyping] = useState(false);

  const chat = async (e, message) => {
    e.preventDefault();

    if (!message) return;
    setIsTyping(true);

    let msgs = chats;
    msgs.push({ role: "user", content: message });
    setChats(msgs);

    setMessage("");

    alert(message);
  };

  return (
    <main>
      <h1>FullStack Chat AI Tutorial</h1>

      <section>
        {chats && chats.length
          ? chats.map((chat, index) => (
              <p key={index} className={chat.role === "user" ? "user_msg" : ""}>
                <span>
                  <b>{chat.role.toUpperCase()}</b>
                </span>
                <span>:</span>
                <span>{chat.content}</span>
              </p>
            ))
          : ""}
      </section>

      <div className={isTyping ? "" : "hide"}>
        <p>
          <i>{isTyping ? "Typing" : ""}</i>
        </p>
      </div>

      <form action="" onSubmit={(e) => chat(e, message)}>
        <input
          type="text"
          name="message"
          value={message}
          placeholder="Type a message here and hit Enter..."
          onChange={(e) => setMessage(e.target.value)}
        />
      </form>
    </main>
  );
}
export default App;

This code is similar to the code from the previous section. But we have removed the OpenAI configurations as we will no longer need them in this section.

At this point, an alert pops up whenever the form is submitted. This will change in a moment.

In the chat function, get rid of the alert message and type the following:

fetch("http://localhost:8000/", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        chats,
      }),
    })
      .then((response) => response.json())
      .then((data) => {
        msgs.push(data.output);
        setChats(msgs);
        setIsTyping(false);
      })
      .catch((error) => {
        console.log(error);
      });

The code above calls the endpoint we created and passes in the chats array for it to process. It then returns a response that is added to the chats and displayed in the UI.

The following is how the UI looks at the moment:

Fullstack Chat UI before Styling

The UI can look better if you add the following styles to the frontend/src/index.css file:

:root {
  font-family: Inter, system-ui, Avenir, Helvetica, Arial, sans-serif;
  line-height: 1.5;
  font-weight: 400;

  color-scheme: light dark;
  color: rgba(255, 255, 255, 0.87);
  background-color: #242424;

  font-synthesis: none;
  text-rendering: optimizeLegibility;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  -webkit-text-size-adjust: 100%;
}

html, body{
  scroll-behavior: smooth;
}

h1 {
  font-size: 3.2em;
  line-height: 1.1;
  text-align: center;
  position: sticky;
  top: 0;
  background-color: #242424;
}

main{
  max-width: 800px;
  margin: auto;
}

p{
  background-color: darkslategray;
  max-width: 70%;
  padding: 15px;
  border-radius: 50px;
}

p span{
  margin: 5px;
}

p span:first-child{
  margin-right: 0;
}

.user_msg{
  text-align: right;
  margin-left: 30%;
  display: flex;
  flex-direction: row-reverse;
}

.hide {
  visibility: hidden;
  display: none;
}

form{
  text-align: center;
  position: sticky;
  bottom: 0;
}

input{
  width: 100%;
  height: 40px;
  border: none;
  padding: 10px;
  font-size: 1.2rem;
  background-color: rgb(28, 23, 23);
}

input:focus{
  outline: none;
}

And remove all the styles from the frontend/src/App.css file.

The code for this part is on GitHub.

Now here is the final output:

FullStack ChatBot Working As Expected with CSS

Congratulations on finishing this project!

The full stack chatbot was more work, but it helped us separate concerns, build a more secure and attractive application, and offer a better experience to users. So it was worth the effort.

You can find the code for this section on GitHub.

Conclusion

This tutorial hopefully showed you that anyone with basic programming knowledge can build AI-powered software. You learned how to build a chatbot using React and Nodejs, and we discussed the pros and cons of each technology. Finally, we built a solution that was both functional, secure, and visually appealing.

After reading this tutorial, you can now explore AI’s functionalities like image manipulation and audio interaction. Take your time to go through the documentation and see how you can expand on what we covered here.

But some significant barriers remain that make it difficult for them to do this:

• OpenAI provides a simple stateless API for ChatGPT. The developer needs to keep track of the history and context of each conversation in a cache or database managed by the application. The developer also needs to manage and safeguard the API keys. There is a lot of boilerplate code unrelated to the application’s business logic.

• The “natural” UI for the ChatGPT API application is a threaded chat. But it is difficult to create a “chat UI” in a traditional web or app framework. In fact, the most commonly used chat UI already exists in messaging apps like Slack, Discord, and even forums (for example, GitHub Discussions). We need a simple way to connect ChatGPT API responses to an existing messaging service.

In this article, I will show you how to create a serverless GitHub bot. The bot allows GitHub users to chat with ChatGPT and each other in GitHub Issues. You can try it by asking a question, or joining another conversation thread by leaving a comment. In other words, this project uses GitHub Issues’ threaded messages UI as its own chat UI.

Figure 1. Learning Rust with ChatGPT. see https://github.com/second-state/chat-with-chatgpt/issues/31

The bot is a serverless function written in Rust. Just fork the example, deploy your fork on flows.network, and configure it to interact with your own GitHub repos and OpenAI keys. You will have a fully functional GitHub bot in 10 minutes. There is no need to set up a web server, or a webhook for GitHub API, or a cache / database server.

How to Fork the Template Repo

First, fork this template repo from GitHub.

The src/lib.rs file contains the bot application (also known as the flow function). The run() function is called upon starting up. It listens for issue_comment and issues events from the GitHub repo owner/repo. Those events are emitted when a new issue comment or a new issue is created in the repo.

#[no_mangle]
#[tokio::main(flavor = "current_thread")]
pub async fn run() {
    // Setup variables for
    //   ower: GitHub org to install the bot
    //   repo: GitHub repo to install the bot
    //   openai_key_name: Name for your OpenAI API key
    // All the values can be set in the source code or as env vars

    listen_to_event(&owner, &repo, vec!["issue_comment", "issues"], |payload| {
        handler(&owner, &repo, &openai_key_name, payload)
    })
    .await;
}

The handler() function processes the events received by listen_to_event(). If the event is a new comment in an issue, the bot calls OpenAI's ChatGPT API to add the comment text into an existing conversation identified by the issue.number. It receives a response from ChatGPT, and adds a comment in the issue.

The flow function here automatically and transparently manages the conversation history with the ChatGPT API in a local storage. The OpenAI API key is also stored in the local storage so that instead of putting the secret text in the source code, the key can be identified by a string name in openai_key_name.

EventPayload::IssueCommentEvent(e) => {
    if e.comment.user.r#type != "Bot" {
        if let Some(b) = e.comment.body {
            if let Some(r) = chat_completion (
                    openai_key_name,
                    &format!("issue#{}", e.issue.number),
                    &b,
                    &ChatOptions::default(),
            ) {
                if let Err(e) = issues.create_comment(e.issue.number, r.choice).await {
                    write_error_log!(e.to_string());
                }
            }
        }
    }
}

If the event is a new issue, the flow function creates a new conversation identified by issue.number, and requests a response from ChatGPT.

EventPayload::IssuesEvent(e) => {
    if e.action == IssuesEventAction::Closed {
        return;
    }

    let title = e.issue.title;
    let body = e.issue.body.unwrap_or("".to_string());
    let q = title + "\n" + &body;
    if let Some(r) = chat_completion (
            openai_key_name,
            &format!("issue#{}", e.issue.number),
            &q,
            &ChatOptions::default(),
    ) {
        if let Err(e) = issues.create_comment(e.issue.number, r.choice).await {
            write_error_log!(e.to_string());
        }
    }
}

How to Deploy the Serverless Flow Function

As we can see, the flow function code calls SDK APIs to perform complex operations. For example,

• The listen_to_event() function registers a webhook URL through GitHub API so that the handler() function will be called when certain events occur in GitHub.

• The chat_completion() function calls the ChatGPT API with the named API key and past history / context of the specified conversation. The API key and conversation history are stored in a Redis cache.

The webhook server and the Redis cache are both external services the SDK depends on. That means the flow function must run inside a managed host environment that provides such external services. Flows.network is a PaaS (Platform as a Service) host for the flow function SDKs.

In order to deploy the flow function on flows.network, you simply need to import its source code to the PaaS.

First, sign into flows.network from your GitHub account. Import your forked GitHub repo that contains the flow function source code and choose "With Environment Variables".

Note that this is NOT the GitHub repo where you want to deploy the bot. This is the repo for your forked flow function source code.

Figure 2. Import the GitHub repo you forked from the flow function template into flows.network.

Set the environment variables to point the flow function to the OpenAI API key name (open_ai_key) and GitHub repo (owner and repo).

The GitHub owner and repo variables here point to the GitHub repo where you want to deploy the bot, NOT the repo for the flow function source code.

Figure 3. Set the environment variables for the GitHub repo where you want to deploy the bot, as well as the OpenAI API key name.

Flows.network will fetch the source code and build the Rust source code into Wasm bytecode using the standard cargo toolchain. It will then run the Wasm flow function in the WasmEdge Runtime.

How to Connect the Flow Function to GitHub and OpenAI

While the flow function requires connections to the OpenAI and GitHub APIs, the source code has no hardcoded API keys, access tokens, or OAUTH logic. The flows function SDKs have made it easy and safe for developers to interact with external SaaS API services.

Flows.network discovers that the flow function requires connections to the OpenAI and GitHub APIs. It presents UI workflows for the developers to:

• Log into GitHub, authorize access to events, and register the flow function as the webhook for receiving those events.

• Associate an OpenAI API key with the name openai_key_name.

Figure 4. The external services required by the flow function are connected and turned green.

Once the external SaaS APIs are connected and authorized, they turn green on the flow function dashboard. The flow function will now receive the events it listen_to_event() for. It will also get transparent access to Redis for the named OpenAI API key and the cached conversation context to support the chat_completion() SDK function.

What's next

The GitHub bot is just one of many bot types the flows.network can support. By connecting the flow function to a Slack channel, you can get ChatGPT to participate in your group discussion. Here is an example of a Slack-based ChatGPT bot.

Figure 5. The Slack ChatGPT bot.

Another example is to have ChatGPT answering legal questions in a Slack channel. The flow function prepends the legal question with a prompt.

Figure 6. The Slack robo lawyer bot.

Besides GitHub and Slack, there are many SaaS products you can integrate into flows.network through their APIs.

While the example flow functions are written in Rust, we aim to support JavaScript-based flow function SDKs. In another word, platform SDK functions such as listen_to_event() and chat_completion() will have a JavaScript version. The JavaScript flow function runs inside the WasmEdge Runtime on the flows.network platform through the WasmEdge-QuickJS module.

In this article, I'll teach you how to build a Telegram chatbot that will tell you your horoscope. So, let’s get started!

How to Get Your Bot Token

To set up a new bot, you will need to talk to BotFather. No, he’s not a person – he’s also a bot, and he's the boss of all the Telegram bots.

1. Search for @botfather in Telegram.

BotFather Telegram Bot

2. Start a conversation with BotFather by clicking on the Start button.

Click on Start Button

3. Type /newbot, and follow the prompts to set up a new bot. The BotFather will give you a token that you will use to authenticate your bot and grant it access to the Telegram API.

Getting access token

Note: Make sure you store the token securely. Anyone with your token access can easily manipulate your bot.

How to Set Up Your Coding Environment

Let’s set up the coding environment. While there are various libraries available to create a Telegram bot, we’ll use the pyTelegramBotAPI library. It is a simple but extensible Python implementation for the Telegram Bot API with both synchronous and asynchronous capabilities.

Install the pyTelegramBotAPI library using pip:

pip install pyTelegramBotAPI

Next, open your favorite code editor and create a .env file to store your token as below:

export BOT_TOKEN=your-bot-token-here

After that, run the source .env command to read the environment variables from the .env file.

How to Create Your First Bot

All the API implementations are stored in a single class called TeleBot. It offers many ways to listen for incoming messages as well as functions like send_message(), send_document(), and others to send messages.

Create a new bot.py file and paste the following code there:

import os

import telebot

BOT_TOKEN = os.environ.get('BOT_TOKEN')

bot = telebot.TeleBot(BOT_TOKEN)

In the above code, we use the os library in order to read the environment variables stored in our system.

If you remember, we exported an environment variable called BOT_TOKEN in the previous step. The value of BOT_TOKEN is read in a variable called BOT_TOKEN. Further, we use the TeleBot class to create a bot instance and passed the BOT_TOKEN to it.

We then need to register message handlers. These message handlers contain filters that a message must pass. If a message passes the filter, the decorated function is called and the incoming message is supplied as an argument.

Let's define a message handler that handles incoming /start and /hello commands.

@bot.message_handler(commands=['start', 'hello'])
def send_welcome(message):
    bot.reply_to(message, "Howdy, how are you doing?")

Any name is acceptable for a function that is decorated by a message handler, but it can only have one parameter (the message).

Let’s add another handler that echoes all incoming text messages back to the sender.

@bot.message_handler(func=lambda msg: True)
def echo_all(message):
    bot.reply_to(message, message.text)

The above code uses a lambda expression to test a message. Since we need to echo all the messages, we always return True from the lambda function.

You now have a simple bot that responds to the /start and /hello commands with a static message and echoes all the other sent messages. Add the following to the end of your file to launch the bot:

bot.infinity_polling()

That’s it! We have a Telegram bot ready. Let’s run the Python file and go to Telegram to test the bot.

Search for the bot using its username if you’re unable to find it. You can test it by sending the commands like /hello and /start and other random texts.

Testing the bot

Note: All the message handlers are tested in the order in which they were declared in the source file.

For more information on using the pyTelegramBotAPI library, you can refer to their documentation.

How to Code the Horoscope Bot

Let’s shift our attention to building our Horoscope Bot now. We will use message chaining in the bot. The bot will first ask for your zodiac sign, and then the day, and then it will respond with the horoscope for that particular day.

Under the hood, the bot interacts with an API to get the horoscope data.

We are going to use the Horoscope API that I built in another tutorial. If you wish to learn how to build one, you can go through this tutorial. Make sure you explore the APIs here before getting started.

How to fetch the horoscope data

Let’s create a utility function to fetch the horoscope data for a particular day.

import requests

def get_daily_horoscope(sign: str, day: str) -> dict:
    """Get daily horoscope for a zodiac sign.
    Keyword arguments:
    sign:str - Zodiac sign
    day:str - Date in format (YYYY-MM-DD) OR TODAY OR TOMORROW OR YESTERDAY
    Return:dict - JSON data
    """
    url = "https://horoscope-app-api.vercel.app/api/v1/get-horoscope/daily"
    params = {"sign": sign, "day": day}
    response = requests.get(url, params)

    return response.json()

In the above Python code, we created a function that accepts two string arguments – sign and day – and returns JSON data. We send a GET request on the API URL and pass sign and day as the query parameters.

If you test the function, you will get an output similar to below:

{
   "data":{
      "date": "Dec 15, 2022",
      "horoscope_data": "Lie low during the day and try not to get caught up in the frivolous verbiage that dominates the waking hours. After sundown, feel free to speak your mind. You may notice that there is a sober tone and restrictive sensation today that leaves you feeling like you will never be able to break free from your current situation. Don't get caught in this negative mindset."
   },
   "status": 200,
   "success": true
}

Note: You can explore more about the requests library in Python in this tutorial.

How to add a message handler

Now that we have a function that returns the horoscope data, let’s create a message handler in our bot that asks for the zodiac sign of the user.

@bot.message_handler(commands=['horoscope'])
def sign_handler(message):
    text = "What's your zodiac sign?\nChoose one: *Aries*, *Taurus*, *Gemini*, *Cancer,* *Leo*, *Virgo*, *Libra*, *Scorpio*, *Sagittarius*, *Capricorn*, *Aquarius*, and *Pisces*."
    sent_msg = bot.send_message(message.chat.id, text, parse_mode="Markdown")
    bot.register_next_step_handler(sent_msg, day_handler)

The above function is a bit different from the other functions we defined earlier. The bot’s horoscope functionality will be invoked by the /horoscope command. We are sending a text message to the user, but notice that we have set the parse_mode to Markdown while sending the message.

Since we’ll use message chaining, we used the register_next_step_handler() method. This method accepts two parameters: the message sent by the user and the callback function which should be called after the message. Thus, we pass the sent_msg variable and a new day_handler function that we’ll define next.

Let’s define the day_handler() function that accepts the message.

def day_handler(message):
    sign = message.text
    text = "What day do you want to know?\nChoose one: *TODAY*, *TOMORROW*, *YESTERDAY*, or a date in format YYYY-MM-DD."
    sent_msg = bot.send_message(
        message.chat.id, text, parse_mode="Markdown")
    bot.register_next_step_handler(
        sent_msg, fetch_horoscope, sign.capitalize())

We fetch the zodiac sign from the message.text attribute. Similar to the previous function, it also asks the day for which you want to know the horoscope.

In the end, we use the same register_next_step_handler() method and pass the sent_msg, the fetch_horoscope callback function, and the sign.

Let’s now define the fetch_horoscope() function that accepts the message and the sign.

def fetch_horoscope(message, sign):
    day = message.text
    horoscope = get_daily_horoscope(sign, day)
    data = horoscope["data"]
    horoscope_message = f'*Horoscope:* {data["horoscope_data"]}\\n*Sign:* {sign}\\n*Day:* {data["date"]}'
    bot.send_message(message.chat.id, "Here's your horoscope!")
    bot.send_message(message.chat.id, horoscope_message, parse_mode="Markdown")

This is the final function where we get the sign from the function parameter and the day from the message.text attribute.

Next, we fetch the horoscope using the get_daily_horoscope() function and construct our message. In the end, we send the message with the horoscope data.

Bot Demo

Once you run the Python file, you can test this functionality. Here’s the demo:

Recommended Next Steps

As of now, the bot stops working as soon as we stop our Python application. In order to make it run always, you can deploy the bot on platforms like Heroku, Render, and so on.

Here's a link to the GitHub repo for this project - feel free to check it out.

You can also add more functionalities to the bot by exploring the Telegram APIs.

Thanks for reading! You can follow me on Twitter.

In order to build a working full-stack application, there are so many moving parts to think about. And you'll need to make many decisions that will be critical to the success of your app.

For example, what language will you use and what platform will you deploy on? Are you going to deploy a containerised software on a server, or make use of serverless functions to handle the backend? Do you plan to use third-party APIs to handle complex parts of your application, like authentication or payments? Where do you store the data?

In addition to all this, you'll also need to think about the user interface, design and usability of your application, and much more.

This is why complex large applications require a multifunctional development team collaborating to build the app.

One of the best ways to learn how to develop full stack applications is to build projects that cover the end-to-end development process. You'll go through designing the architecture, developing the API services, developing the user interface, and finally deploying your application.

So this tutorial will take you through the process of building an AI chatbot to help you learn these concepts in depth.

Some of the topics we will cover include:

• How to build APIs with Python, FastAPI, and WebSockets
• How to build real-time systems with Redis
• How to build a chat User Interface with React

Important Note: This is an intermediate full stack software development project that requires some basic Python and JavaScript knowledge.

I've carefully divided the project into sections to ensure that you can easily select the phase that is important to you in case you do not wish to code the full application.

You can download the full repository on My Github here.

Table of Contents

Section 1

• Application Architecture
• How to Set Up the Development Environment

  Section 2

• How to Build a Chat Server with Python, FastAPI, and WebSockets
  • How to Set Up the Python Environment
  • FastAPI Server Setup
  • How to Add Routes to the API
  • How to Generate a Chat Session Token with UUID
  • How to Test the API with Postman
  • Websockets and Connection Manager
  • Dependency Injection in FastAPI

    Section 3

• How to build Real-Time Systems with Redis
  • Redis and Distributed Messaging Queues
  • How to Connect to a Redis Cluster in Python with a Redis Client
  • How to Work with Redis Streams
  • How to Model the Chat Data
  • How to Work with Redis JSON
  • How to Update the Token Dependency

    Section 4

• How to Add Intelligence to Chatbots with AI models
  • How to Get Started with Huggingface
  • How to Interact with the Language Model
  • How to Simulate Short-term Memory for the AI Model
  • Stream Consumer and Real-timeDdata Pull from the Message Queue
  • How to Update the Chat Client with the AI Response
  • Refresh Token
  • How to Test the Chat with Multiple Clients in Postman

Application Architecture

Sketching out a solution architecture gives you a high-level overview of your application, the tools you intend to use, and how the components will communicate with each other.

I have drawn up a simple architecture below using draw.io:

Fullstack chatbot architecture

Let's go over the various parts of the architecture in more detail:

Client/User Interface

We will use React version 18 to build the user interface. The Chat UI will communicate with the backend via WebSockets.

GPT-J-6B and Huggingface Inference API

GPT-J-6B is a generative language model which was trained with 6 Billion parameters and performs closely with OpenAI's GPT-3 on some tasks.

I have chosen to use GPT-J-6B because it is an open-source model and doesn’t require paid tokens for simple use cases.

Huggingface also provides us with an on-demand API to connect with this model pretty much free of charge. You can read more about GPT-J-6B and Hugging Face Inference API.

Redis

When we send prompts to GPT, we need a way to store the prompts and easily retrieve the response. We will use Redis JSON to store the chat data and also use Redis Streams for handling the real-time communication with the huggingface inference API.

Redis is an in-memory key-value store that enables super-fast fetching and storing of JSON-like data. For this tutorial, we will use a managed free Redis storage provided by Redis Enterprise for testing purposes.

Web Sockets and the Chat API

To send messages between the client and server in real-time, we need to open a socket connection. This is because an HTTP connection will not be sufficient to ensure real-time bi-directional communication between the client and the server.

We will be using FastAPI for the chat server, as it provides a fast and modern Python server for our use. Check out the FastAPI documentation) to learn more about WebSockets.

How to Set Up the Development Environment

You can use your desired OS to build this app – I am currently using MacOS, and Visual Studio Code. Just make sure you have Python and NodeJs installed.

To set up the project structure, create a folder namedfullstack-ai-chatbot. Then create two folders within the project called client and server. The server will hold the code for the backend, while the client will hold the code for the frontend.

Next within the project directory, initialize a Git repository within the root of the project folder using the "git init" command. Then create a .gitignore file by using "touch .gitignore":

git init
touch .gitignore

In the next section, we will build our chat web server using FastAPI and Python.

How to Build a Chat Server with Python, FastAPI and WebSockets

In this section, we will build the chat server using FastAPI to communicate with the user. We will use WebSockets to ensure bi-directional communication between the client and server so that we can send responses to the user in real-time.

How to Set Up the Python Environment

To start our server, we need to set up our Python environment. Open the project folder within VS Code, and open up the terminal.

From the project root, cd into the server directory and run python3.8 -m venv env. This will create a virtual environment for our Python project, which will be named env. To activate the virtual environment, run source env/bin/activate

Next, install a couple of libraries in your Python environment.

pip install fastapi uuid uvicorn gunicorn WebSockets python-dotenv aioredis

Next create an environment file by running touch .env in the terminal. We will define our app variables and secret variables within the .env file.

Add your app environment variable and set it to "development" like so: export APP_ENV=development. Next, we will set up a development server with a FastAPI server.

FastAPI Server Setup

At the root of the server directory, create a new file named main.py then paste the code below for the development sever:

from fastapi import FastAPI, Request
import uvicorn
import os
from dotenv import load_dotenv

load_dotenv()

api = FastAPI()

@api.get("/test")
async def root():
    return {"msg": "API is Online"}


if __name__ == "__main__":
    if os.environ.get('APP_ENV') == "development":
        uvicorn.run("main:api", host="0.0.0.0", port=3500,
                    workers=4, reload=True)
    else:
      pass

First we import FastAPI and initialize it as api. Then we import load_dotenv from the python-dotenv library, and initialize it to load the variables from the .env file,

Then we create a simple test route to test the API. The test route will return a simple JSON response that tells us the API is online.

Lastly, we set up the development server by using uvicorn.run and providing the required arguments. The API will run on port 3500.

Finally, run the server in the terminal with python main.py. Once you see Application startup complete in the terminal, navigate to the URL http://localhost:3500/test on your browser, and you should get a web page like this:

API Test Page

How to Add Routes to the API

In this section, we will add routes to our API. Create a new folder named src. This is the directory where all our API code will live.

Create a subfolder named routes, cd into the folder, create a new file named chat.py and then add the code below:

import os
from fastapi import APIRouter, FastAPI, WebSocket,  Request

chat = APIRouter()

# @route   POST /token
# @desc    Route to generate chat token
# @access  Public

@chat.post("/token")
async def token_generator(request: Request):
    return None


# @route   POST /refresh_token
# @desc    Route to refresh token
# @access  Public

@chat.post("/refresh_token")
async def refresh_token(request: Request):
    return None


# @route   Websocket /chat
# @desc    Socket for chatbot
# @access  Public

@chat.websocket("/chat")
async def websocket_endpoint(websocket: WebSocket = WebSocket):
    return None

We created three endpoints:

• /token will issue the user a session token for access to the chat session. Since the chat app will be open publicly, we do not want to worry about authentication and just keep it simple – but we still need a way to identify each unique user session.
• /refresh_token will get the session history for the user if the connection is lost, as long as the token is still active and not expired.
• /chat will open a WebSocket to send messages between the client and server.

Next, connect the chat route to our main API. First we need to import chat from src.chat within our main.py file. Then we will include the router by literally calling an include_router method on the initialized FastAPI class and passing chat as the argument.

Update your api.py code as shown below:

from fastapi import FastAPI, Request
import uvicorn
import os
from dotenv import load_dotenv
from routes.chat import chat

load_dotenv()

api = FastAPI()
api.include_router(chat)


@api.get("/test")
async def root():
    return {"msg": "API is Online"}


if __name__ == "__main__":
    if os.environ.get('APP_ENV') == "development":
        uvicorn.run("main:api", host="0.0.0.0", port=3500,
                    workers=4, reload=True)
    else:
        pass

How to Generate a Chat Session Token with UUID

To generate a user token we will use uuid4 to create dynamic routes for our chat endpoint. Since this is a publicly available endpoint, we won't need to go into details about JWTs and authentication.

If you didn't install uuid initially, run pip install uuid. Next in chat.py, import UUID, and update the /token route with the code below:

from fastapi import APIRouter, FastAPI, WebSocket,  Request, BackgroundTasks, HTTPException
import uuid

# @route   POST /token
# @desc    Route generating chat token
# @access  Public

@chat.post("/token")
async def token_generator(name: str, request: Request):

    if name == "":
        raise HTTPException(status_code=400, detail={
            "loc": "name",  "msg": "Enter a valid name"})

    token = str(uuid.uuid4())

    data = {"name": name, "token": token}

    return data

In the code above, the client provides their name, which is required. We do a quick check to ensure that the name field is not empty, then generate a token using uuid4.

The session data is a simple dictionary for the name and token. Ultimately we will need to persist this session data and set a timeout, but for now we just return it to the client.

How to Test the API with Postman

Because we will be testing a WebSocket endpoint, we need to use a tool like Postman that allows this (as the default swagger docs on FastAPI does not support WebSockets).

In Postman, create a collection for your development environment and send a POST request to localhost:3500/token specifying the name as a query parameter and passing it a value. You should get a response as shown below:

Token Generator Postman

Websockets and Connection Manager

In the src root, create a new folder named socket and add a file named connection.py. In this file, we will define the class that controls the connections to our WebSockets, and all the helper methods to connect and disconnect.

In connection.py add the code below:

from fastapi import WebSocket

class ConnectionManager:
    def __init__(self):
        self.active_connections: List[WebSocket] = []

    async def connect(self, websocket: WebSocket):
        await websocket.accept()
        self.active_connections.append(websocket)

    def disconnect(self, websocket: WebSocket):
        self.active_connections.remove(websocket)

    async def send_personal_message(self, message: str, websocket: WebSocket):
        await websocket.send_text(message)

The ConnectionManager class is initialized with an active_connections attribute that is a list of active connections.

Then the asynchronous connect method will accept a WebSocket and add it to the list of active connections, while the disconnect method will remove the Websocket from the list of active connections.

Lastly, the send_personal_message method will take in a message and the Websocket we want to send the message to and asynchronously send the message.

WebSockets are a very broad topic and we only scraped the surface here. This should however be sufficient to create multiple connections and handle messages to those connections asynchronously.

You can read more about FastAPI Websockets and Sockets Programming.

To use the ConnectionManager, import and initialize it within the src.routes.chat.py, and update the /chat WebSocket route with the code below:

from ..socket.connection import ConnectionManager

manager = ConnectionManager()

@chat.websocket("/chat")
async def websocket_endpoint(websocket: WebSocket):
    await manager.connect(websocket)
    try:
        while True:
            data = await websocket.receive_text()
            print(data)
            await manager.send_personal_message(f"Response: Simulating response from the GPT service", websocket)

    except WebSocketDisconnect:
        manager.disconnect(websocket)

In the websocket_endpoint function, which takes a WebSocket, we add the new websocket to the connection manager and run a while True loop, to ensure that the socket stays open. Except when the socket gets disconnected.

While the connection is open, we receive any messages sent by the client with websocket.receive_test() and print them to the terminal for now.

Then we send a hard-coded response back to the client for now. Ultimately the message received from the clients will be sent to the AI Model, and the response sent back to the client will be the response from the AI Model.

In Postman, we can test this endpoint by creating a new WebSocket request, and connecting to the WebSocket endpoint localhost:3500/chat.

When you click connect, the Messages pane will show that the API client is connected to the URL, and a socket is open.

To test this, send a message "Hello Bot" to the chat server, and you should get an immediate test response "Response: Simulating response from the GPT service" as shown below:

Postman Chat Test

Dependency Injection in FastAPI

To be able to distinguish between two different client sessions and limit the chat sessions, we will use a timed token, passed as a query parameter to the WebSocket connection.

In the socket folder, create a file named utils.py then add the code below:

from fastapi import WebSocket, status, Query
from typing import Optional

async def get_token(
    websocket: WebSocket,
    token: Optional[str] = Query(None),
):
    if token is None or token == "":
        await websocket.close(code=status.WS_1008_POLICY_VIOLATION)

    return token

The get_token function receives a WebSocket and token, then checks if the token is None or null.

If this is the case, the function returns a policy violation status and if available, the function just returns the token. We will ultimately extend this function later with additional token validation.

To consume this function, we inject it into the /chat route. FastAPI provides a Depends class to easily inject dependencies, so we don't have to tinker with decorators.

Update the /chat route to the following:

from ..socket.utils import get_token

@chat.websocket("/chat")
async def websocket_endpoint(websocket: WebSocket, token: str = Depends(get_token)):
    await manager.connect(websocket)
    try:
        while True:
            data = await websocket.receive_text()
            print(data)
            await manager.send_personal_message(f"Response: Simulating response from the GPT service", websocket)

    except WebSocketDisconnect:
        manager.disconnect(websocket)

Now when you try to connect to the /chat endpoint in Postman, you will get a 403 error. Provide a token as query parameter and provide any value to the token, for now. Then you should be able to connect like before, only now the connection requires a token.

Postman Chat Test with Token

Congratulations on getting this far! Your chat.py file should now look like this:

import os
from fastapi import APIRouter, FastAPI, WebSocket, WebSocketDisconnect, Request, Depends, HTTPException
import uuid
from ..socket.connection import ConnectionManager
from ..socket.utils import get_token


chat = APIRouter()

manager = ConnectionManager()

# @route   POST /token
# @desc    Route to generate chat token
# @access  Public


@chat.post("/token")
async def token_generator(name: str, request: Request):
    token = str(uuid.uuid4())

    if name == "":
        raise HTTPException(status_code=400, detail={
            "loc": "name",  "msg": "Enter a valid name"})

    data = {"name": name, "token": token}

    return data


# @route   POST /refresh_token
# @desc    Route to refresh token
# @access  Public


@chat.post("/refresh_token")
async def refresh_token(request: Request):
    return None


# @route   Websocket /chat
# @desc    Socket for chatbot
# @access  Public

@chat.websocket("/chat")
async def websocket_endpoint(websocket: WebSocket, token: str = Depends(get_token)):
    await manager.connect(websocket)
    try:
        while True:
            data = await websocket.receive_text()
            print(data)
            await manager.send_personal_message(f"Response: Simulating response from the GPT service", websocket)

    except WebSocketDisconnect:
        manager.disconnect(websocket)

In the next part of this tutorial, we will focus on handling the state of our application and passing data between client and server.

How to Build Real-Time Systems with Redis

Our application currently does not store any state, and there is no way to identify users or store and retrieve chat data. We are also returning a hard-coded response to the client during chat sessions.

In this part of the tutorial, we will cover the following:

• How to connect to a Redis Cluster in Python and set up a Redis Client
• How to store and retrieve data with Redis JSON
• How to set up Redis Streams as message queues between a web server and worker environment

Redis and Distributed Messaging Queues

Redis is an open source in-memory data store that you can use as a database, cache, message broker, and streaming engine. It supports a number of data structures and is a perfect solution for distributed applications with real-time capabilities.

Redis Enterprise Cloud is a fully managed cloud service provided by Redis that helps us deploy Redis clusters at an infinite scale without worrying about infrastructure.

We will be using a free Redis Enterprise Cloud instance for this tutorial. You can Get started with Redis Cloud for free here and follow This tutorial to set up a Redis database and Redis Insight, a GUI to interact with Redis.

Once you have set up your Redis database, create a new folder in the project root (outside the server folder) named worker.

We will isolate our worker environment from the web server so that when the client sends a message to our WebSocket, the web server does not have to handle the request to the third-party service. Also, resources can be freed up for other users.

The background communication with the inference API is handled by this worker service, through Redis.

Requests from all the connected clients are appended to the message queue (producer), while the worker consumes the messages, sends off the requests to the inference API, and appends the response to a response queue.

Once the API receives a response, it sends it back to the client.

During the trip between the producer and the consumer, the client can send multiple messages, and these messages will be queued up and responded to in order.

Ideally, we could have this worker running on a completely different server, in its own environment, but for now, we will create its own Python environment on our local machine.

You might be wondering – why do we need a worker? Imagine a scenario where the web server also creates the request to the third-party service. This means that while waiting for the response from the third party service during a socket connection, the server is blocked and resources are tied up till the response is obtained from the API.

You can try this out by creating a random sleep time.sleep(10) before sending the hard-coded response, and sending a new message. Then try to connect with a different token in a new postman session.

You will notice that the chat session will not connect until the random sleep times out.

While we can use asynchronous techniques and worker pools in a more production-focused server set-up, that also won't be enough as the number of simultaneous users grow.

Ultimately, we want to avoid tying up the web server resources by using Redis to broker the communication between our chat API and the third-party API.

Next open up a new terminal, cd into the worker folder, and create and activate a new Python virtual environment similar to what we did in part 1.

Next, install the following dependencies:

pip install aiohttp aioredis python-dotenv

How to Connect to a Redis Cluster in Python with a Redis Client

We will use the aioredis client to connect with the Redis database. We'll also use the requests library to send requests to the Huggingface inference API.

Create two files .env, and main.py. Then create a folder named src. Also, create a folder named redis and add a new file named config.py.

In the .env file, add the following code – and make sure you update the fields with the credentials provided in your Redis Cluster.

export REDIS_URL=<REDIS URL PROVIDED IN REDIS CLOUD>
export REDIS_USER=<REDIS USER IN REDIS CLOUD>
export REDIS_PASSWORD=<DATABASE PASSWORD IN REDIS CLOUD>
export REDIS_HOST=<REDIS HOST IN REDIS CLOUD>
export REDIS_PORT=<REDIS PORT IN REDIS CLOUD>

In config.py add the Redis Class below:

import os
from dotenv import load_dotenv
import aioredis

load_dotenv()

class Redis():
    def __init__(self):
        """initialize  connection """
        self.REDIS_URL = os.environ['REDIS_URL']
        self.REDIS_PASSWORD = os.environ['REDIS_PASSWORD']
        self.REDIS_USER = os.environ['REDIS_USER']
        self.connection_url = f"redis://{self.REDIS_USER}:{self.REDIS_PASSWORD}@{self.REDIS_URL}"

    async def create_connection(self):
        self.connection = aioredis.from_url(
            self.connection_url, db=0)

        return self.connection

We create a Redis object and initialize the required parameters from the environment variables. Then we create an asynchronous method create_connection to create a Redis connection and return the connection pool obtained from the aioredis method from_url.

Next, we test the Redis connection in main.py by running the code below. This will create a new Redis connection pool, set a simple key "key", and assign a string "value" to it.

from src.redis.config import Redis
import asyncio

async def main():
    redis = Redis()
    redis = await redis.create_connection()
    print(redis)
    await redis.set("key", "value")

if __name__ == "__main__":
    asyncio.run(main())

Now open Redis Insight (if you followed the tutorial to download and install it) You should see something like this:

Redis Insight Test

How to Work with Redis Streams

Now that we have our worker environment setup, we can create a producer on the web server and a consumer on the worker.

First, let's create our Redis class again on the server. In server.src create a folder named redis and add two files, config.py and producer.py.

In config.py, add the code below as we did for the worker environment:

import os
from dotenv import load_dotenv
import aioredis

load_dotenv()

class Redis():
    def __init__(self):
        """initialize  connection """
        self.REDIS_URL = os.environ['REDIS_URL']
        self.REDIS_PASSWORD = os.environ['REDIS_PASSWORD']
        self.REDIS_USER = os.environ['REDIS_USER']
        self.connection_url = f"redis://{self.REDIS_USER}:{self.REDIS_PASSWORD}@{self.REDIS_URL}"

    async def create_connection(self):
        self.connection = aioredis.from_url(
            self.connection_url, db=0)

        return self.connection

In the .env file, also add the Redis credentials:

export REDIS_URL=<REDIS URL PROVIDED IN REDIS CLOUD>
export REDIS_USER=<REDIS USER IN REDIS CLOUD>
export REDIS_PASSWORD=<DATABASE PASSWORD IN REDIS CLOUD>
export REDIS_HOST=<REDIS HOST IN REDIS CLOUD>
export REDIS_PORT=<REDIS PORT IN REDIS CLOUD>

Finally, in server.src.redis.producer.py add the following code:

from .config import Redis

class Producer:
    def __init__(self, redis_client):
        self.redis_client = redis_client

    async def add_to_stream(self,  data: dict, stream_channel):
        try:
            msg_id = await self.redis_client.xadd(name=stream_channel, id="*", fields=data)
            print(f"Message id {msg_id} added to {stream_channel} stream")
            return msg_id

        except Exception as e:
            print(f"Error sending msg to stream => {e}")

We created a Producer class that is initialized with a Redis client. We use this client to add data to the stream with the add_to_stream method, which takes the data and the Redis channel name.

The Redis command for adding data to a stream channel is xadd and it has both high-level and low-level functions in aioredis.

Next, to run our newly created Producer, update chat.py and the WebSocket /chat endpoint like below. Notice the updated channel name message_channel.

from ..redis.producer import Producer
from ..redis.config import Redis

chat = APIRouter()
manager = ConnectionManager()
redis = Redis()


@chat.websocket("/chat")
async def websocket_endpoint(websocket: WebSocket, token: str = Depends(get_token)):
    await manager.connect(websocket)
    redis_client = await redis.create_connection()
    producer = Producer(redis_client)

    try:
        while True:
            data = await websocket.receive_text()
            print(data)
            stream_data = {}
            stream_data[token] = data
            await producer.add_to_stream(stream_data, "message_channel")
            await manager.send_personal_message(f"Response: Simulating response from the GPT service", websocket)

    except WebSocketDisconnect:
        manager.disconnect(websocket)

Next, in Postman, create a connection and send any number of messages that say Hello. You should have the stream messages printed to the terminal like below:

Terminal Channel Messages Test

In Redis Insight, you will see a new mesage_channel created and a time-stamped queue filled with the messages sent from the client. This timestamped queue is important to preserve the order of the messages.

Redis Insight Channel

How to Model the Chat Data

Next, we'll create a model for our chat messages. Recall that we are sending text data over WebSockets, but our chat data needs to hold more information than just the text. We need to timestamp when the chat was sent, create an ID for each message, and collect data about the chat session, then store this data in a JSON format.

We can store this JSON data in Redis so we don't lose the chat history once the connection is lost, because our WebSocket does not store state.

In server.src create a new folder named schema. Then create a file named chat.py in server.src.schema add the following code:

from datetime import datetime
from pydantic import BaseModel
from typing import List, Optional
import uuid


class Message(BaseModel):
    id = uuid.uuid4()
    msg: str
    timestamp = str(datetime.now())


class Chat(BaseModel):
    token: str
    messages: List[Message]
    name: str
    session_start = str(datetime.now())

We are using Pydantic's BaseModel class to model the chat data. The Chat class will hold data about a single Chat session. It will store the token, name of the user, and an automatically generated timestamp for the chat session start time using datetime.now().

The messages sent and received within this chat session are stored with a Message class which creates a chat id on the fly using uuid4. The only data we need to provide when initializing this Message class is the message text.

How to Work with Redis JSON

In order to use Redis JSON's ability to store our chat history, we need to install rejson provided by Redis labs.

In the terminal, cd into server and install rejson with pip install rejson. Then update your Redis class in server.src.redis.config.py to include the create_rejson_connection method:

import os
from dotenv import load_dotenv
import aioredis
from rejson import Client

load_dotenv()

class Redis():
    def __init__(self):
        """initialize  connection """
        self.REDIS_URL = os.environ['REDIS_URL']
        self.REDIS_PASSWORD = os.environ['REDIS_PASSWORD']
        self.REDIS_USER = os.environ['REDIS_USER']
        self.connection_url = f"redis://{self.REDIS_USER}:{self.REDIS_PASSWORD}@{self.REDIS_URL}"
        self.REDIS_HOST = os.environ['REDIS_HOST']
        self.REDIS_PORT = os.environ['REDIS_PORT']

    async def create_connection(self):
        self.connection = aioredis.from_url(
            self.connection_url, db=0)

        return self.connection

    def create_rejson_connection(self):
        self.redisJson = Client(host=self.REDIS_HOST,
                                port=self.REDIS_PORT, decode_responses=True, username=self.REDIS_USER, password=self.REDIS_PASSWORD)

        return self.redisJson

We are adding the create_rejson_connection method to connect to Redis with the rejson Client. This gives us the methods to create and manipulate JSON data in Redis, which are not available with aioredis.

Next, in server.src.routes.chat.py we can update the /token endpoint to create a new Chat instance and store the session data in Redis JSON like so:

@chat.post("/token")
async def token_generator(name: str, request: Request):
    token = str(uuid.uuid4())

    if name == "":
        raise HTTPException(status_code=400, detail={
            "loc": "name",  "msg": "Enter a valid name"})

    # Create new chat session
    json_client = redis.create_rejson_connection()

    chat_session = Chat(
        token=token,
        messages=[],
        name=name
    )

    # Store chat session in redis JSON with the token as key
    json_client.jsonset(str(token), Path.rootPath(), chat_session.dict())

    # Set a timeout for redis data
    redis_client = await redis.create_connection()
    await redis_client.expire(str(token), 3600)


    return chat_session.dict()

NOTE: Because this is a demo app, I do not want to store the chat data in Redis for too long. So I have added a 60-minute time out on the token using the aioredis client (rejson does not implement timeouts). This means that after 60 minutes, the chat session data will be lost.

This is necessary because we are not authenticating users, and we want to dump the chat data after a defined period. This step is optional, and you don't have to include it.

Next, in Postman, when you send a POST request to create a new token, you will get a structured response like the one below. You can also check Redis Insight to see your chat data stored with the token as a JSON key and the data as a value.

Token Generator Updated

How to Update the Token Dependency

Now that we have a token being generated and stored, this is a good time to update the get_token dependency in our /chat WebSocket. We do this to check for a valid token before starting the chat session.

In server.src.socket.utils.py update the get_token function to check if the token exists in the Redis instance. If it does then we return the token, which means that the socket connection is valid. If it doesn't exist, we close the connection.

The token created by /token will cease to exist after 60 minutes. So we can have some simple logic on the frontend to redirect the user to generate a new token if an error response is generated while trying to start a chat.

from ..redis.config import Redis

async def get_token(
    websocket: WebSocket,
    token: Optional[str] = Query(None),
):

    if token is None or token == "":
        await websocket.close(code=status.WS_1008_POLICY_VIOLATION)

    redis_client = await redis.create_connection()
    isexists = await redis_client.exists(token)

    if isexists == 1:
        return token
    else:
        await websocket.close(code=status.WS_1008_POLICY_VIOLATION, reason="Session not authenticated or expired token")

To test the dependency, connect to the chat session with the random token we have been using, and you should get a 403 error. (Note that you have to manually delete the token in Redis Insight.)

Now copy the token generated when you sent the post request to the /token endpoint (or create a new request) and paste it as the value to the token query parameter required by the /chat WebSocket. Then connect. You should get a successful connection.

Chat Session with Token

Bringing it all together, your chat.py should look like the below.

import os
from fastapi import APIRouter, FastAPI, WebSocket, WebSocketDisconnect, Request, Depends
import uuid
from ..socket.connection import ConnectionManager
from ..socket.utils import get_token
import time
from ..redis.producer import Producer
from ..redis.config import Redis
from ..schema.chat import Chat
from rejson import Path

chat = APIRouter()
manager = ConnectionManager()
redis = Redis()


# @route   POST /token
# @desc    Route to generate chat token
# @access  Public


@chat.post("/token")
async def token_generator(name: str, request: Request):
    token = str(uuid.uuid4())

    if name == "":
        raise HTTPException(status_code=400, detail={
            "loc": "name",  "msg": "Enter a valid name"})

    # Create nee chat session
    json_client = redis.create_rejson_connection()
    chat_session = Chat(
        token=token,
        messages=[],
        name=name
    )

    print(chat_session.dict())

    # Store chat session in redis JSON with the token as key
    json_client.jsonset(str(token), Path.rootPath(), chat_session.dict())

    # Set a timeout for redis data
    redis_client = await redis.create_connection()
    await redis_client.expire(str(token), 3600)

    return chat_session.dict()


# @route   POST /refresh_token
# @desc    Route to refresh token
# @access  Public


@chat.post("/refresh_token")
async def refresh_token(request: Request):
    return None


# @route   Websocket /chat
# @desc    Socket for chat bot
# @access  Public

@chat.websocket("/chat")
async def websocket_endpoint(websocket: WebSocket, token: str = Depends(get_token)):
    await manager.connect(websocket)
    redis_client = await redis.create_connection()
    producer = Producer(redis_client)
    json_client = redis.create_rejson_connection()

    try:
        while True:
            data = await websocket.receive_text()
            stream_data = {}
            stream_data[token] = data
            await producer.add_to_stream(stream_data, "message_channel")
            await manager.send_personal_message(f"Response: Simulating response from the GPT service", websocket)

    except WebSocketDisconnect:
        manager.disconnect(websocket)

Well done on reaching it this far! In the next section, we will focus on communicating with the AI model and handling the data transfer between client, server, worker, and the external API.

How to Add Intelligence to Chatbots with AI Models

In this section, we will focus on building a wrapper to communicate with the transformer model, send prompts from a user to the API in a conversational format, and receive and transform responses for our chat application.

How to Get Started with Huggingface

We will not be building or deploying any language models on Hugginface. Instead, we'll focus on using Huggingface's accelerated inference API to connect to pre-trained models.

The model we will be using is the GPT-J-6B Model provided by EleutherAI. It's a generative language model which was trained with 6 Billion parameters.

Huggingface provides us with an on-demand limited API to connect with this model pretty much free of charge.

To get started with Huggingface, Create a free account. In your settings, generate a new access token. For up to 30k tokens, Huggingface provides access to the inference API for free.

You can Monitor your API usage here. Make sure you keep this token safe and don't expose it publicly.

Note: We will use HTTP connections to communicate with the API because we are using a free account. But the PRO Huggingface account supports streaming with WebSockets see parallelism and batch jobs.

This can help significantly improve response times between the model and our chat application, and I'll hopefully cover this method in a follow-up article.

How to Interact with the Language Model

First, we add the Huggingface connection credentials to the .env file within our worker directory.

export HUGGINFACE_INFERENCE_TOKEN=<HUGGINGFACE ACCESS TOKEN>
export MODEL_URL=https://api-inference.huggingface.co/models/EleutherAI/gpt-j-6B

Next, in worker.src create a folder named model then add a file gptj.py. Then add the GPT class below:

import os
from dotenv import load_dotenv
import requests
import json

load_dotenv()

class GPT:
    def __init__(self):
        self.url = os.environ.get('MODEL_URL')
        self.headers = {
            "Authorization": f"Bearer {os.environ.get('HUGGINFACE_INFERENCE_TOKEN')}"}
        self.payload = {
            "inputs": "",
            "parameters": {
                "return_full_text": False,
                "use_cache": True,
                "max_new_tokens": 25
            }

        }

    def query(self, input: str) -> list:
        self.payload["inputs"] = input
        data = json.dumps(self.payload)
        response = requests.request(
            "POST", self.url, headers=self.headers, data=data)
        print(json.loads(response.content.decode("utf-8")))
        return json.loads(response.content.decode("utf-8"))

if __name__ == "__main__":
    GPT().query("Will artificial intelligence help humanity conquer the universe?")

The GPT class is initialized with the Huggingface model url, authentication header, and predefined payload. But the payload input is a dynamic field that is provided by the query method and updated before we send a request to the Huggingface endpoint.

Finally, we test this by running the query method on an instance of the GPT class directly. In the terminal, run python src/model/gptj.py, and you should get a response like this (just keep in mind that your response will certainly be different from this):

[{'generated_text': ' (AI) could solve all the problems on this planet? I am of the opinion that in the short term artificial intelligence is much better than human beings, but in the long and distant future human beings will surpass artificial intelligence.\n\nIn the distant'}]

Next, we add some tweaking to the input to make the interaction with the model more conversational by changing the format of the input.

Update the GPT class like so:

class GPT:
    def __init__(self):
        self.url = os.environ.get('MODEL_URL')
        self.headers = {
            "Authorization": f"Bearer {os.environ.get('HUGGINFACE_INFERENCE_TOKEN')}"}
        self.payload = {
            "inputs": "",
            "parameters": {
                "return_full_text": False,
                "use_cache": False,
                "max_new_tokens": 25
            }

        }

    def query(self, input: str) -> list:
        self.payload["inputs"] = f"Human: {input} Bot:"
        data = json.dumps(self.payload)
        response = requests.request(
            "POST", self.url, headers=self.headers, data=data)
        data = json.loads(response.content.decode("utf-8"))
        text = data[0]['generated_text']
        res = str(text.split("Human:")[0]).strip("\n").strip()
        return res


if __name__ == "__main__":
    GPT().query("Will artificial intelligence help humanity conquer the universe?")

We updated the input with a string literal f"Human: {input} Bot:". The human input is placed in the string and the Bot provides a response. This input format turns the GPT-J6B into a conversational model. Other changes you may notice include

• use_cache: you can make this False if you want the model to create a new response when the input is the same. I suggest leaving this as True in production to prevent exhausting your free tokens if a user just keeps spamming the bot with the same message. Using cache does not actually load a new response from the model.
• return_full_text: is False, as we do not need to return the input – we already have it. When we get a response, we strip the "Bot:" and leading/trailing spaces from the response and return just the response text.

How to Simulate Short-term Memory for the AI Model

For every new input we send to the model, there is no way for the model to remember the conversation history. This is important if we want to hold context in the conversation.

But remember that as the number of tokens we send to the model increases, the processing gets more expensive, and the response time is also longer.

So we will need to find a way to retrieve short-term history and send it to the model. We will also need to figure out a sweet spot - how much historical data do we want to retrieve and send to the model?

To handle chat history, we need to fall back to our JSON database. We'll use the token to get the last chat data, and then when we get the response, append the response to the JSON database.

Update worker.src.redis.config.py to include the create_rejson_connection method. Also, update the .env file with the authentication data, and ensure rejson is installed.

Your worker.src.redis.config.py should look like this:

import os
from dotenv import load_dotenv
import aioredis
from rejson import Client


load_dotenv()


class Redis():
    def __init__(self):
        """initialize  connection """
        self.REDIS_URL = os.environ['REDIS_URL']
        self.REDIS_PASSWORD = os.environ['REDIS_PASSWORD']
        self.REDIS_USER = os.environ['REDIS_USER']
        self.connection_url = f"redis://{self.REDIS_USER}:{self.REDIS_PASSWORD}@{self.REDIS_URL}"
        self.REDIS_HOST = os.environ['REDIS_HOST']
        self.REDIS_PORT = os.environ['REDIS_PORT']

    async def create_connection(self):
        self.connection = aioredis.from_url(
            self.connection_url, db=0)

        return self.connection

    def create_rejson_connection(self):
        self.redisJson = Client(host=self.REDIS_HOST,
                                port=self.REDIS_PORT, decode_responses=True, username=self.REDIS_USER, password=self.REDIS_PASSWORD)

        return self.redisJson

While your .env file should look like this:

export REDIS_URL=<REDIS URL PROVIDED IN REDIS CLOUD>
export REDIS_USER=<REDIS USER IN REDIS CLOUD>
export REDIS_PASSWORD=<DATABASE PASSWORD IN REDIS CLOUD>
export REDIS_HOST=<REDIS HOST IN REDIS CLOUD>
export REDIS_PORT=<REDIS PORT IN REDIS CLOUD>
export HUGGINFACE_INFERENCE_TOKEN=<HUGGINGFACE ACCESS TOKEN>
export MODEL_URL=https://api-inference.huggingface.co/models/EleutherAI/gpt-j-6B

Next, in worker.src.redis create a new file named cache.py and add the code below:

from .config import Redis
from rejson import Path

class Cache:
    def __init__(self, json_client):
        self.json_client = json_client

    async def get_chat_history(self, token: str):
        data = self.json_client.jsonget(
            str(token), Path.rootPath())

        return data

The cache is initialized with a rejson client, and the method get_chat_history takes in a token to get the chat history for that token, from Redis. Make sure you import the Path object from rejson.

Next, update the worker.main.py with the code below:

from src.redis.config import Redis
import asyncio
from src.model.gptj import GPT
from src.redis.cache import Cache

redis = Redis()

async def main():
    json_client = redis.create_rejson_connection()
    data = await Cache(json_client).get_chat_history(token="18196e23-763b-4808-ae84-064348a0daff")
    print(data)

if __name__ == "__main__":
    asyncio.run(main())

I have hard-coded a sample token created from previous tests in Postman. If you don't have a token created, just send a new request to /token and copy the token, then run python main.py in the terminal. You should see the data in the terminal like so:

{'token': '18196e23-763b-4808-ae84-064348a0daff', 'messages': [], 'name': 'Stephen', 'session_start': '2022-07-16 13:20:01.092109'}

Next, we need to add an add_message_to_cache method to our Cache class that adds messages to Redis for a specific token.

  async def add_message_to_cache(self, token: str, message_data: dict):
      self.json_client.jsonarrappend(
          str(token), Path('.messages'), message_data)

The jsonarrappend method provided by rejson appends the new message to the message array.

Note that to access the message array, we need to provide .messages as an argument to the Path. If your message data has a different/nested structure, just provide the path to the array you want to append the new data to.

To test this method, update the main function in the main.py file with the code below:

async def main():
    json_client = redis.create_rejson_connection()

    await Cache(json_client).add_message_to_cache(token="18196e23-763b-4808-ae84-064348a0daff", message_data={
        "id": "1",
        "msg": "Hello",
        "timestamp": "2022-07-16 13:20:01.092109"
    })

    data = await Cache(json_client).get_chat_history(token="18196e23-763b-4808-ae84-064348a0daff")
    print(data)

We are sending a hard-coded message to the cache, and getting the chat history from the cache. When you run python main.py in the terminal within the worker directory, you should get something like this printed in the terminal, with the message added to the message array.

{'token': '18196e23-763b-4808-ae84-064348a0daff', 'messages': [{'id': '1', 'msg': 'Hello', 'timestamp': '2022-07-16 13:20:01.092109'}], 'name': 'Stephen', 'session_start': '2022-07-16 13:20:01.092109'}

Finally, we need to update the main function to send the message data to the GPT model, and update the input with the last 4 messages sent between the client and the model.

First let's update our add_message_to_cache function with a new argument "source" that will tell us if the message is a human or bot. We can then use this arg to add the "Human:" or "Bot:" tags to the data before storing it in the cache.

Update the add_message_to_cache method in the Cache class like so:

  async def add_message_to_cache(self, token: str, source: str, message_data: dict):
      if source == "human":
          message_data['msg'] = "Human: " + (message_data['msg'])
      elif source == "bot":
          message_data['msg'] = "Bot: " + (message_data['msg'])

      self.json_client.jsonarrappend(
          str(token), Path('.messages'), message_data)

Then update the main function in main.py in the worker directory, and run python main.py to see the new results in the Redis database.

async def main():
    json_client = redis.create_rejson_connection()

    await Cache(json_client).add_message_to_cache(token="18196e23-763b-4808-ae84-064348a0daff", source="human", message_data={
        "id": "1",
        "msg": "Hello",
        "timestamp": "2022-07-16 13:20:01.092109"
    })

    data = await Cache(json_client).get_chat_history(token="18196e23-763b-4808-ae84-064348a0daff")
    print(data)

Next, we need to update the main function to add new messages to the cache, read the previous 4 messages from the cache, and then make an API call to the model using the query method. It'll have a payload consisting of a composite string of the last 4 messages.

You can always tune the number of messages in the history you want to extract, but I think 4 messages is a pretty good number for a demo.

In worker.src, create a new folder schema. Then create a new file named chat.py and paste our message schema in chat.py like so:

from datetime import datetime
from pydantic import BaseModel
from typing import List, Optional
import uuid


class Message(BaseModel):
    id = str(uuid.uuid4())
    msg: str
    timestamp = str(datetime.now())

Next, update the main.py file like below:

async def main():

    json_client = redis.create_rejson_connection()

    await Cache(json_client).add_message_to_cache(token="18196e23-763b-4808-ae84-064348a0daff", source="human", message_data={
        "id": "3",
        "msg": "I would like to go to the moon to, would you take me?",
        "timestamp": "2022-07-16 13:20:01.092109"
    })

    data = await Cache(json_client).get_chat_history(token="18196e23-763b-4808-ae84-064348a0daff")

    print(data)

    message_data = data['messages'][-4:]

    input = ["" + i['msg'] for i in message_data]
    input = " ".join(input)

    res = GPT().query(input=input)

    msg = Message(
        msg=res
    )

    print(msg)
    await Cache(json_client).add_message_to_cache(token="18196e23-763b-4808-ae84-064348a0daff", source="bot", message_data=msg.dict())

In the code above, we add new message data to the cache. This message will ultimately come from the message queue. Next we get the chat history from the cache, which will now include the most recent data we added.

Note that we are using the same hard-coded token to add to the cache and get from the cache, temporarily just to test this out.

Next, we trim off the cache data and extract only the last 4 items. Then we consolidate the input data by extracting the msg in a list and join it to an empty string.

Finally, we create a new Message instance for the bot response and add the response to the cache specifying the source as "bot"

Next, run python main.py a couple of times, changing the human message and id as desired with each run. You should have a full conversation input and output with the model.

Open Redis Insight and you should have something similar to the below:

Conversational Chat

Stream Consumer and Real-time Data Pull from the Message Queue

Next, we want to create a consumer and update our worker.main.py to connect to the message queue. We want it to pull the token data in real-time, as we are currently hard-coding the tokens and message inputs.

In worker.src.redis create a new file named stream.py. Add a StreamConsumer class with the code below:

class StreamConsumer:
    def __init__(self, redis_client):
        self.redis_client = redis_client

    async def consume_stream(self, count: int, block: int,  stream_channel):

        response = await self.redis_client.xread(
            streams={stream_channel:  '0-0'}, count=count, block=block)

        return response

    async def delete_message(self, stream_channel, message_id):
        await self.redis_client.xdel(stream_channel, message_id)

The StreamConsumer class is initialized with a Redis client. The consume_stream method pulls a new message from the queue from the message channel, using the xread method provided by aioredis.

Next, update the worker.main.py file with a while loop to keep the connection to the message channel alive, like so:

from src.redis.config import Redis
import asyncio
from src.model.gptj import GPT
from src.redis.cache import Cache
from src.redis.config import Redis
from src.redis.stream import StreamConsumer
import os
from src.schema.chat import Message


redis = Redis()


async def main():
    json_client = redis.create_rejson_connection()
    redis_client = await redis.create_connection()
    consumer = StreamConsumer(redis_client)
    cache = Cache(json_client)

    print("Stream consumer started")
    print("Stream waiting for new messages")

    while True:
        response = await consumer.consume_stream(stream_channel="message_channel", count=1, block=0)

        if response:
            for stream, messages in response:
                # Get message from stream, and extract token, message data and message id
                for message in messages:
                    message_id = message[0]
                    token = [k.decode('utf-8')
                             for k, v in message[1].items()][0]
                    message = [v.decode('utf-8')
                               for k, v in message[1].items()][0]
                    print(token)

                    # Create a new message instance and add to cache, specifying the source as human
                    msg = Message(msg=message)

                    await cache.add_message_to_cache(token=token, source="human", message_data=msg.dict())

                    # Get chat history from cache
                    data = await cache.get_chat_history(token=token)

                    # Clean message input and send to query
                    message_data = data['messages'][-4:]

                    input = ["" + i['msg'] for i in message_data]
                    input = " ".join(input)

                    res = GPT().query(input=input)

                    msg = Message(
                        msg=res
                    )

                    print(msg)

                    await cache.add_message_to_cache(token=token, source="bot", message_data=msg.dict())

                # Delete messaage from queue after it has been processed

                await consumer.delete_message(stream_channel="message_channel", message_id=message_id)


if __name__ == "__main__":
    asyncio.run(main())

This is quite the update, so let's take it step by step:

We use a while True loop so that the worker can be online listening to messages from the queue.

Next, we await new messages from the message_channel by calling our consume_stream method. If we have a message in the queue, we extract the message_id, token, and message. Then we create a new instance of the Message class, add the message to the cache, and then get the last 4 messages. We set it as input to the GPT model query method.

Once we get a response, we then add the response to the cache using the add_message_to_cache method, then delete the message from the queue.

How to Update the Chat Client with the AI Response

So far, we are sending a chat message from the client to the message_channel (which is received by the worker that queries the AI model) to get a response.

Next, we need to send this response to the client. As long as the socket connection is still open, the client should be able to receive the response.

If the connection is closed, the client can always get a response from the chat history using the refresh_token endpoint.

In worker.src.redis create a new file named producer.py, and add a Producer class similar to what we had on the chat web server:

class Producer:
    def __init__(self, redis_client):
        self.redis_client = redis_client

    async def add_to_stream(self,  data: dict, stream_channel) -> bool:
        msg_id = await self.redis_client.xadd(name=stream_channel, id="*", fields=data)
        print(f"Message id {msg_id} added to {stream_channel} stream")
        return msg_id

Next, in the main.py file, update the main function to initialize the producer, create a stream data, and send the response to a response_channel using the add_to_stream method:

from src.redis.config import Redis
import asyncio
from src.model.gptj import GPT
from src.redis.cache import Cache
from src.redis.config import Redis
from src.redis.stream import StreamConsumer
import os
from src.schema.chat import Message
from src.redis.producer import Producer


redis = Redis()


async def main():
    json_client = redis.create_rejson_connection()
    redis_client = await redis.create_connection()
    consumer = StreamConsumer(redis_client)
    cache = Cache(json_client)
    producer = Producer(redis_client)

    print("Stream consumer started")
    print("Stream waiting for new messages")

    while True:
        response = await consumer.consume_stream(stream_channel="message_channel", count=1, block=0)

        if response:
            for stream, messages in response:
                # Get message from stream, and extract token, message data and message id
                for message in messages:
                    message_id = message[0]
                    token = [k.decode('utf-8')
                             for k, v in message[1].items()][0]
                    message = [v.decode('utf-8')
                               for k, v in message[1].items()][0]

                    # Create a new message instance and add to cache, specifying the source as human
                    msg = Message(msg=message)

                    await cache.add_message_to_cache(token=token, source="human", message_data=msg.dict())

                    # Get chat history from cache
                    data = await cache.get_chat_history(token=token)

                    # Clean message input and send to query
                    message_data = data['messages'][-4:]

                    input = ["" + i['msg'] for i in message_data]
                    input = " ".join(input)

                    res = GPT().query(input=input)

                    msg = Message(
                        msg=res
                    )

                    stream_data = {}
                    stream_data[str(token)] = str(msg.dict())

                    await producer.add_to_stream(stream_data, "response_channel")

                    await cache.add_message_to_cache(token=token, source="bot", message_data=msg.dict())

                # Delete messaage from queue after it has been processed
                await consumer.delete_message(stream_channel="message_channel", message_id=message_id)


if __name__ == "__main__":
    asyncio.run(main())

Next, we need to let the client know when we receive responses from the worker in the /chat socket endpoint. We do this by listening to the response stream. We do not need to include a while loop here as the socket will be listening as long as the connection is open.

Note that we also need to check which client the response is for by adding logic to check if the token connected is equal to the token in the response. Then we delete the message in the response queue once it's been read.

In server.src.redis create a new file named stream.py and add our StreamConsumer class like this:

from .config import Redis

class StreamConsumer:
    def __init__(self, redis_client):
        self.redis_client = redis_client

    async def consume_stream(self, count: int, block: int,  stream_channel):
        response = await self.redis_client.xread(
            streams={stream_channel:  '0-0'}, count=count, block=block)

        return response

    async def delete_message(self, stream_channel, message_id):
        await self.redis_client.xdel(stream_channel, message_id)

Next, update the /chat socket endpoint like so:

from ..redis.stream import StreamConsumer

@chat.websocket("/chat")
async def websocket_endpoint(websocket: WebSocket, token: str = Depends(get_token)):
    await manager.connect(websocket)
    redis_client = await redis.create_connection()
    producer = Producer(redis_client)
    json_client = redis.create_rejson_connection()
    consumer = StreamConsumer(redis_client)

    try:
        while True:
            data = await websocket.receive_text()
            stream_data = {}
            stream_data[str(token)] = str(data)
            await producer.add_to_stream(stream_data, "message_channel")
            response = await consumer.consume_stream(stream_channel="response_channel", block=0)

            print(response)
            for stream, messages in response:
                for message in messages:
                    response_token = [k.decode('utf-8')
                                      for k, v in message[1].items()][0]

                    if token == response_token:
                        response_message = [v.decode('utf-8')
                                            for k, v in message[1].items()][0]

                        print(message[0].decode('utf-8'))
                        print(token)
                        print(response_token)

                        await manager.send_personal_message(response_message, websocket)

                    await consumer.delete_message(stream_channel="response_channel", message_id=message[0].decode('utf-8'))

    except WebSocketDisconnect:
        manager.disconnect(websocket)

Refresh Token

Finally, we need to update the /refresh_token endpoint to get the chat history from the Redis database using our Cache class.

In server.src.redis, add a cache.py file and add the code below:

from rejson import Path

class Cache:
    def __init__(self, json_client):
        self.json_client = json_client

    async def get_chat_history(self, token: str):
        data = self.json_client.jsonget(
            str(token), Path.rootPath())

        return data

Next, in server.src.routes.chat.py import the Cache class and update the /token endpoint to the below:

from ..redis.cache import Cache

@chat.get("/refresh_token")
async def refresh_token(request: Request, token: str):
    json_client = redis.create_rejson_connection()
    cache = Cache(json_client)
    data = await cache.get_chat_history(token)

    if data == None:
        raise HTTPException(
            status_code=400, detail="Session expired or does not exist")
    else:
        return data

Now, when we send a GET request to the /refresh_token endpoint with any token, the endpoint will fetch the data from the Redis database.

If the token has not timed out, the data will be sent to the user. Or it'll send a 400 response if the token is not found.

How to Test the Chat with multiple Clients in Postman

Finally, we will test the chat system by creating multiple chat sessions in Postman, connecting multiple clients in Postman, and chatting with the bot on the clients.

Lastly, we will try to get the chat history for the clients and hopefully get a proper response.

Recap

Let's have a quick recap as to what we have achieved with our chat system. The chat client creates a token for each chat session with a client. This token is used to identify each client, and each message sent by clients connected to or web server is queued in a Redis channel (message_chanel), identified by the token.

Our worker environment reads from this channel. It does not have any clue who the client is (except that it's a unique token) and uses the message in the queue to send requests to the Huggingface inference API.

When it gets a response, the response is added to a response channel and the chat history is updated. The client listening to the response_channel immediately sends the response to the client once it receives a response with its token.

If the socket is still open, this response is sent. If the socket is closed, we are certain that the response is preserved because the response is added to the chat history. The client can get the history, even if a page refresh happens or in the event of a lost connection.

Congratulations on getting this far! You have been able to build a working chat system.

In follow-up articles, I will focus on building a chat user interface for the client, creating unit and functional tests, fine-tuning our worker environment for faster response time with WebSockets and asynchronous requests, and ultimately deploying the chat application on AWS.

This Article is part of a series on building full-stack intelligent chatbots with tools like Python, React, Huggingface, Redis, and so on. You can follow the full series on my blog: blog.stephensanwo.dev - AI ChatBot Series**

You can download the full repository on My Github Repository

I wrote this tutorial in collaboration with Redis. Need help getting started with Redis? Try the following resources:

• Try Redis Cloud free of charge
• Watch this video on the benefits of Redis Cloud over other Redis providers
• Redis Developer Hub - tools, guides, and tutorials about Redis
• RedisInsight Desktop GUI

In case you've seen my previous tutorial on this topic, stick with me as this version features lots of updates.

You can follow along with this tutorial using the code on my GitHub:

If you want, you can dive right into my video tutorial on YouTube – or read on for more details. 😎

What to Expect from this Tutorial

Here is an example of the Discord AI chatbot that we will have built by the end of this tutorial.

Chat demo with my bot in Discord. I drew the bot's icon 😊

My chatbot project started as a joke with a friend when we were playing video games.

I'm honestly surprised by how popular it became – there were 5.9k views of my previous tutorial, plus, when I deployed my bot to a 1k+ user server, people flooded it with 300+ messages in an hour, effectively crashing the bot. 😳 You can read more about my deployment post-mortem in this post.

Since a lot of people are interested in building their own bots based on their favorite characters, I updated my tutorial to include an in-depth explanation on how to gather text data for any character, fictional or non-fictional.

You may also create a custom dataset that captures the speech between you and your friends and build a chatbot that speaks like yourself!

Other updates in this tutorial address changes in Hugging Face's model hosting services, including API changes that affect how we push the model to Hugging Face's model repositories.

Outline of this Tutorial

The video version of this tutorial runs for a total of one hour and features the following topics:

1. Gather text data for your character using one of these two methods: find pre-made datasets on Kaggle or make custom datasets from raw transcripts.

2. Train the model in Google Colab, a cloud-based Jupyter Notebook environment with free GPUs.

3. Deploy the model to Hugging Face, an AI model hosting service.

4. Build a Discord bot in either Python or JavaScript, your choice! 🤩

5. Set up the Discord bot's permissions so they don't spam non-bot channels

6. Host the bot on Repl.it.

7. Keep the bot running indefinitely with Uptime Robot.

To learn more about how to build Discord bots, you may also find these two freeCodeCamp posts useful – there's a Python version and a JavaScript version.

How to Prepare the Data

For our chatbot to learn to converse, we need text data in the form of dialogues. This is essentially how our chatbot is going to respond to different exchanges and contexts.

Is Your Favorite Character on Kaggle?

There are a lot of interesting datasets on Kaggle for popular cartoons, TV shows, and other media. For example:

• Rick and Morty

• Harry Potter

• The Big Bang Theory

• Game of Thrones

We only need two columns from these datasets: character name and dialogue line.

Example dataset: Harry Potter movie transcript

Can't Find Your Favorite Character on Kaggle?

Can't find your favorite character on Kaggle? No worries. We can create datasets from raw transcripts. A great place to look for transcripts is Transcript Wiki. For example, check out this Peppa Pig transcript.

Example: Peppa Pig transcript

Using a regular expression like ([a-zA-Z|\s]+): (.+), we can extract out the two columns of interest, character name, and dialogue line.

Try it out on this Python regex website yourself!

How to Train the Model

Under the hood, our model will be a Generative Pre-trained Transfomer (GPT), the most popular language model these days.

Instead of training from scratch, we will load Microsoft's pre-trained GPT, DialoGPT-small, and fine-tune it using our dataset.

My GitHub repo for this tutorial contains the notebook file named model_train_upload_workflow.ipynb to get you started. All you need to do is the following: (please refer to the video for a detailed walkthrough)

1. Upload the file to Google Colab

2. Select GPU as the runtime, which will speed up our model training.

3. Change the dataset and the target character in code snippets like:

data = pd.read_csv('MY-DATASET.csv')
CHARACTER_NAME = 'MY-CHARACTER'

Running through the training section of the notebook should take less than half an hour. I have about 700 lines and the training takes less than ten minutes. The model will be stored in a folder named output-small .

Want an even smarter and more eloquent model? Feel free to train a larger model like DialoGPT-medium or even DialoGPT-large. Model size here refers to the number of parameters in the model. More parameters will allow the model to pick up more complexity from the dataset.

You may also increase the number of training epochs by searching for num_train_epochs in the notebook. This is the number of times that the model will cycle through the training dataset. The model will generally get smarter when it has more exposure to the dataset.

However, do take care not to overfit the model: If the model is trained for too many epochs, it may memorize the dataset and recite back lines from the dataset when we try to converse with it. This isn't ideal as we want the conversation to be more organic.

How to Host the Model

We will host the model on Hugging Face, which provides a free API for us to query the model.

Sign up for Hugging Face and create a new model repository by clicking on New model. Obtain your API token by going to Edit profile > API Tokens. We will need this token when we build the Discord bot.

Follow along with this section in my video to push the model. Also, remember to tag it as conversational in its Model Card (equivalently its README.md):

---
tags:
- conversational
---

# My Awesome Model

You will know that everything works fine if you are able to chat with the model in the browser.

How to Build the Discord Bot

Go to the Discord Developer's page, create an application, and add a bot to it. Since our chatbot is only going to respond to user messages, checking Text Permissions > Send Messgaes in the Bot Permissions Setting is sufficient. Copy the bot's API token for later use.

Sign up for Repl.it and create a new Repl, Python or Node.js for JavaScript, whichever you are working with.

Let's store our API tokens for Hugging Face and Discord as environment variables, named HUGGINGFACE_TOKEN and DISCORD_TOKEN respectively. This helps keep them secret.

Copy my Python script for a Python bot and my JS script for a JS bot. Note that for the JS bot, because of a version incompatibility with Repl.it's Node and NPM, we will need to explicitly specify a lower version of the Discord API in package.json.

"dependencies": {
    "discord.js": "^12.5.3",
}

With that, our bot is ready to go! Start the Repl script by hitting Run, add the bot to a server, type something in the channel, and enjoy the bot's witty response.

How to Keep the Bot Online

One problem with our bot is that it halts as soon as we stop the running Repl (equivalently, if we close the Repl.it browser window).

To get around this and keep our bot running indefinitely, we will set up a web server to contain the bot script, and use a service like Uptime Robot to pin our server every five minutes so that our server stays alive.

In my video tutorial, I copied the server code from these two freeCodeCamp posts (Python version, JavaScript version). Then, I set up the monitor on Uptime Robot. Now my bot continues to reply to my messages even if I close the browser (or shut down my computer all together).

Congratulations on reaching the end of this tutorial! I hope you enjoyed creating the bot and have fun chatting with your favorite character! 🥳

Tutorial Video Link

More About Me and My Chatbot Project

I'm Lynn, a software engineer at Salesforce. I graduated from the University of Chicago in 2021 with a joint BS/MS in Computer Science, specializing in Machine Learning. Come say hi on my personal website!

I post fun project tutorials like this on my YouTube channel. Feel free to subscribe to catch up on my latest content. 😃

Want to learn more about my bot? Check out this 15-minute real-time chat demo featuring me, my friend, and my bot!

Interested in the model I trained? Check it out on Hugging Face:

My chatbot was so popular on a 1k+ user server that... it crashed. 🤯 Read about my deployment post-mortem in this post:

Thanks for reading!

By popular demand, we recently built a Discord bot that helps people track their progress in the challenge.

Today I am going to show you how to build your own 100 Days of Code bot.

How to Create a Discord Bot Application

Your first step is to set up a Discord bot application. Head over to the Discord Developer Portal, sign in if needed, and select "Applications" from the sidebar.

Screenshot of the Developer Portal. If this is your first bot, you will not have any applications here.

Click the "New Application" button. Give it a name, and set it as a "Personal" application. You will now be taken to the application's settings. Here you can change the name, or give it an avatar.

Select "Bot" from the side bar, then click the "Add Bot" button. This will create a Discord Bot account for your application.

Screenshot of the Bot settings page. If you did not set an avatar, you will see a default based on your bot's name.

This is the screen where you will get the bot token. It is very important to keep this token secret, as the token allows your code to connect to your bot. Keep it safe and do not share it with anyone.

Now you need to add the bot to a server to interact with it. Click the "OAuth2" option on the side bar. You should see a form under the "OAuth2 URL Generator" section. Leave the "Select Redirect URL" dropdown blank, and check the box for the "bot" scope.

An option to select permissions will appear. Check the boxes for the following permissions:

• Send Messages
• Manage Messages
• Embed Links
• Read Message History
• View Channels

Above that section, you should see a URL generated. Click the "Copy" button to copy it, then paste it into your browser and go.

This will take you through Discord's process to add your new bot to a server. Note that you must have the Manage Server permission in the server you want to add the bot to. If you do not have this permission in any servers, you can create a server to test your bot in.

Screenshot of the OAuth screen with the correct settings marked.

Now you are ready to write some code!

How to Set Up Your Project

You need to set up the infrastructure and tooling for your project.

Prepare the package.json

Create a directory, or folder, for your project. Open your terminal pointing to that new folder. Run the command npm init to set up your package.json file. For this tutorial, the default values are sufficient, but feel free to edit them as you wish.

You should end up with a package.json similar to this:

{
  "name": "tutorial",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC"
}

Now you need to make a couple of changes to get ready for the TypeScript implementation.

First, replace the main value of index.js with ./prod/index.js – you will be setting your TypeScript to compile to a prod directory.

Then, remove the test script and add the following two scripts:

"build": "tsc",
"start": "node -r dotenv/config ./prod/index.js"

The build script will compile your TypeScript into JavaScript so node can run it, and the start script will run the index.js entrypoint file.

Adding the -r dotenv/config here will dynamically import and run the config method in the dotenv package, which loads your environment variables from the .env file.

Speaking of packages, your next step is to install dependencies. Using npm install, install these dependencies:

• discord.js – this is the library that will handle connecting to the gateway and managing the Discord API calls.
• dotenv – a package that loads .env values into the node process.
• mongoose – A wrapper for the MongoDB connection which offers tools for structuring your data.

Finally, install the development dependencies with npm install --save-dev. Development dependencies are packages that are required for working on your project in a development environment, but not required for running the codebase in production.

• typescript – This is the package for the TypeScript language, which includes everything needed to write code in TypeScript and compile it into JavaScript.
• @types/node – TypeScript relies on type definitions to help understand the code you write. This package defines the types for the Node.js runtime environment, such as the process.env object.

With these packages installed, you should now have a package.json similar to this:

{
  "name": "tutorial",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "build": "tsc",
    "start": "node -r dotenv/config ./prod/index.js"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "discord.js": "^12.5.3",
    "dotenv": "^10.0.0",
    "mongoose": "^5.12.14"
  },
  "devDependencies": {
    "@types/node": "^15.12.2",
    "typescript": "^4.3.4"
  }
}

Prepare TypeScript

TypeScript's compiler offers a number of different settings to maximize your control over the resulting JavaScript.

You can typically modify compiler settings through a tsconfig.json file at the root of your project. You can generate the default boilerplate for this file with npx tsc --init, use an existing one if you set one up in another project, or even write one from scratch.

Because the compiler settings can significantly change the behaviour of TypeScript, it is best to use the same settings when following this tutorial. Here are settings you should use:

{
  "compilerOptions": {
    "target": "ES6",
    "module": "CommonJS",
    "rootDir": "./src",
    "outDir": "./prod",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "forceConsistentCasingInFileNames": true,
    "resolveJsonModule": true
  }
}

The most important settings here are the rootDir and outDir settings. These tell the compiler that all of your code will be in the src directory, and the resulting JavaScript should go in the prod directory.

If you would like to test your settings, create a src directory and place an index.ts file inside. Write some code (such as a console.log statement) and run npm run build in your terminal. You should see a prod directory get created, with an index.js containing your compiled code.

Additional Setup Notes

If you are using git as a version control, you want to avoid pushing secrets and unnecessary code to your repository. Create a .gitignore file in your root project directory, and add the following content:

/node_modules/
/prod/
.env

The .gitignore file tells git not to track files/folders that match the patterns you enter. Ignoring the node modules folder keeps your repository from becoming bloated.

Pushing the compiled JavaScript is also unnecessary, as your project is typically compiled in production before runtime. .env files typically contain your secret values, such as API keys and tokens, so they should not be committed to a repository.

If you are using a UNIX based environment (such as Linux, or Git Bash on Windows), you can also add a prebuild script to your package.json. The prebuild script will automatically run before the build script when you use npm run build. I set mine to clean up the existing prod directory with rm -r ./prod.

How to Create the Discord Bot

Your next step is to prepare the initial bot connection. If you did not do so earlier, create a src directory and an index.ts file within.

Start with an anonymous immediately-invoked function expression (IIFE) to allow for top-level await use:

(async () => {

})();

Within this function you are going to instantiate your Discord bot. At the top of the file, import the Client class with import { Client } from "discord.js;". The Client class represents your Discord bot's session.

Inside your function, construct a new Client instance and assign it to a BOT variable with const BOT = new Client();. Now the BOT variable will represent your bot.

To connect your bot to Discord's gateway and begin receiving events, you will need to use the .login() method on your bot instance. The .login() method takes a single argument, which is the token for the bot application you created earlier.

Many of the methods in discord.js are asynchronous, so you will need to use await here. Add the line await BOT.login(process.env.BOT_TOKEN); to your IIFE.

The process.env object will contain the environment variables for your Node.js runtime environment. With the dotenv package, this will also include any variables you set in your .env secrets file.

Create that .env file in the root of your project, and add BOT_TOKEN="" as the first line. Between the quotes, paste the bot token from the bot page on the Discord Developer Portal.

Your index.ts file should now look like this:

import { Client } from "discord.js";

(async () => {
  const BOT = new Client();

  await BOT.login(process.env.BOT_TOKEN);
})();

Assuming you added your new bot to a server, if you run npm run build and npm start you should see your bot come online in the server. However, the bot will not respond to anything yet, because we have not started listening to events.

Gateway Events in Discord

Gateway "events" are generated when an action happens on Discord, and are typically sent to clients (including your bot) as JSON payloads. You can listen to those events with the .on() method, allowing you to write logic for your bot to follow when specific events occur.

The first event to listen to is the "ready" event. This event fires when your bot has connected to the gateway and is ready to process events. Above your .login() call, add BOT.on("ready", () => console.log("Connected to Discord!"));.

For your changes to take effect, use npm run build again to compile the new code. Now if you try npm run start, you should see "Connected to Discord!" print in your terminal.

How to Connect to the Database

You'll be using the mongoose package to connect to a MongoDB instance. If you prefer, you can run MongoDB locally, or you can use the MongoDB Atlas free tier for a cloud-based solution.

If you do not have a MongoDB Atlas account, freeCodeCamp has a great tutorial on setting one up.

Grab your connection string for your database and add it to your .env file as MONGO_URI="", with the connection string going between the quotes. For the database name, use oneHundredDays.

Create a directory called database to hold the files that contain your database logic. Within that directory, create a file called connectDatabase.ts. You will be writing your logic to initiate the database connection here.

Start with an exported function declaration:

export const connectDatabase = async () => {

}

mongoose offers a connect method for connecting to the database. Import it with import { connect } from "mongoose"; at the top of your file.

Then use the method inside your function with await connect(process.env.MONGO_URI);. Add a console.log statement after that so you can identify that your bot has connected to the database.

Your connectDatabase.ts file should now look something like this:

import { connect } from "mongoose";

export const connectDatabase = async () => {
    await connect(process.env.MONGO_URI);
    console.log("Database Connected!")
}

Now, within your index.ts file, import this function with import { connectDatabase } from "./database/connectDatabase" and add await connectDatabase() to your IIFE, just before the .login() method. Go ahead and run npm run build again.

A compiler error, indicating that: Argument of type string or undefined is not assignable to parameter of type string.

Oh no – an error!

Environment Variable Validation

The problem with environment variables is that they can all be undefined. This often happens if you make a typo in your environment variable name, or mix the name up with another name (a mistake I made when writing this tutorial, using TOKEN instead of BOT_TOKEN in some places).

TypeScript is warning you that the connect method takes a string, and that an undefined value will break things. You can fix this, but first you will want to write a function to handle validating your environment variables.

Within your src directory, create a utils directory to contain your utility functions. Add a validateEnv.ts file there.

Create a function in the file called validateEnv. This function will be synchronous and does not need the async keyword. Within that function, add conditions to check for your two environment variables. If either one is missing, return false. Otherwise, return true.

Your code might look something like this:

export const validateEnv = () => {
  if (!process.env.BOT_TOKEN) {
    console.warn("Missing Discord bot token.");
    return false;
  }

  if (!process.env.MONGO_URI) {
    console.warn("Missing MongoDB connection.");
    return false;
  }
  return true;
};

Head back to your index.ts file and import this validation function with import { validateEnv } from "./utils/validateEnv". Then at the beginning of your IIFE, use an if statement to return early if the function returns false. Your index.ts should look like:

import { Client } from "discord.js";
import { connectDatabase } from "./database/connectDatabase";
import { validateEnv } from "./utils/validateEnv";

(async () => {
  if (!validateEnv()) return;

  const BOT = new Client();

  BOT.on("ready", () => console.log("Connected to Discord!"));

  await connectDatabase();

  await BOT.login(process.env.BOT_TOKEN);
})();

If you try npm run build again, you will see the same error message as before. This is because while we know the environment variable exists, TypeScript still cannot infer it. The validation function is set up to exit the process if the environment variable is missing, so we are going to tell TypeScript that it is definitely a string.

Back in your connectDatabase.ts file, within the connect function use process.env.MONGO_URI as string to coerce the type into string. The error should go away, and you can now run npm run build and npm start.

You should see the messages you wrote for both the Discord and MongoDB connections print in your terminal.

The "message" Event

While you are making great progress on your bot, it still does not do anything. In order for the bot to respond to messages, you will need another event handler.

The logic in this one will be a bit more complicated, so you should create a separate module for it. Create an events folder in the src directory.

Within your events folder, create an onMessage.ts file. At the top, import the Message class from discord.js with import { Message } from "discord.js";. This class will serve as your type definition.

Then create an exported function called onMessage. The function should be asynchronous and take a message parameter with the Message type. Your function will look like this:

import { Message } from "discord.js";

export const onMessage = async (message: Message) => {

};

Before diving in to the logic for this function, you should attach it to the event listener. Back in your index.ts file, import your new function with import { onMessage } from "./events/onMessage";.

Next to your existing .on("ready") listener, add a BOT.on("message") listener. For the "message" event, the callback takes a message argument which you can pass to your new onMessage function:

BOT.on("message", async (message) => await onMessage(message));

We should test that this works. Head back to your onMessage.ts file. Inside your onMessage function, add console.log(message.content). The .content property on the Message class contains the text content sent in the message.

Use npm run build and npm start to get your bot running again, and then send "Hello" in a Discord channel the bot can see. You should see "Hello" print to your terminal.

How to Prepare for Commands

I maintain a few Discord bots, and one thing I've discovered that helps keep code maintainable and readable is making the components modular.

Define an Interface

You will first need to define a common structure for your commands. Create an interfaces folder in src. Then inside interfaces create a file CommandInt.ts.

Now you are going to create an interface. In TypeScript, an interface is often used to define the structure of an object, and is one of many tools available for declaring a variable's type.

In your CommandInt.ts file, import the Message class from Discord, then declare an interface called CommandInt with this syntax:

import { Message } from "discord.js";

export interface CommandInt {

}

Inside this interface, you are going to add three properties:

• name: string; – the name value will be your command's name. You will use this to trigger the command in the Discord server.
• description: string; – the description value explains what the command does. You will used this in one of the commands.
• run: (message: Message) => Promise<void> – this is the property that will hold the command's logic.

The run type definition is a bit tricky, so let's break it down. You have typed it as a function which takes one argument, message. That argument should be the Message type. You then set the function's return type to Promise<void>. This means your function will be asynchronous (this is important later) and does not return a value.

Create a Command List

Next you need a place to store all of your commands. Create a folder called commands in the src directory, and add a file called _CommandList.ts. The underscore here will keep this file at the top of the list.

The _CommandList.ts file will need two lines. First, import your CommandInt interface, then declare a CommandList array. The array will be empty for now, but give it a CommandInt[] type so TypeScript knows it will eventually hold your command objects. The file should look like:

import { CommandInt } from "../interfaces/CommandInt";

export const CommandList: CommandInt[] = [];

The purpose of this file is to create an array of your bot's commands which you will iterate over in the message event listener. There are ways to automate this, but they tend to be unnecessarily complex for smaller bots.

Check for Commands

Back in your onMessage.ts file, you should start working on the logic to check messages for commands.

The first step is to ensure that your bot ignores its own messages, as well as the messages of other bots. This helps prevent endless cycles where the bot is responding to itself.

The message object has an author property, which represents the Discord user that sent the message. The author property has a bot property, which is a Boolean that indicates the author is a bot account. Add a step to check if this property is true:

if (message.author.bot) {
  return;
}

You also want to prevent people from accidentally calling your bot's commands. For example, if you have a help command, you would not want the bot to respond when someone says "help me please".

This can be avoided by setting a prefix for the bot to detect. Most bots use !, but you are welcome to choose whichever prefix you would like.

Declare a variable prefix and assign it your chosen prefix, such as const prefix = "!";. Then add a condition to check if the message.content does not start with that prefix, and if so return.

const prefix = "!";

if (!message.content.startsWith(prefix)) {
  return;
}

Now that you have verified that the message came from a user and is intentionally triggering your bot, you can check to see if the command is valid.

Using the (currently empty) CommandList array will facilitate this process, so import it at the top of your file with import { CommandList } from "../commands/_CommandList";.

There are a few ways to iterate through an array – for the live bot, I used a for..of loop. Regardless of the loop approach, you will want to check each command in the array against the message content. Here is a loop example:

  for (const Command of CommandList) {
    if (message.content.startsWith(prefix + Command.name)) {
      await Command.run(message);
      break;
    }
  }

This loop iterates through the list of commands, and if the message content starts with the prefix and command name, the bot will call the command's run method.

Remember that you declared the run property to be an async function that took the message as an argument. Then, to save on compute time, the loop breaks when it finds a matching command.

Database Model

There's one more step before you are ready to start writing commands. This bot will track your community members' 100 Days of Code progress. And you need to store that progress in the database.

mongoose helps structure your MongoDB records to prevent you from passing malformed or incomplete data into your database.

Start by creating a models folder in your database directory. In that models folder, create a CamperModel.ts file. This will be your structure for the user objects.

You first need to import the necessary values from the mongoose library. Add import { Document, model, Schema } from "mongoose"; at the top of the file.

Because you are using TypeScript, you need to create a type definition for your database objects. Create another interface, like you did for your commands, named CamperInt.

export interface CamperInt {

}

Your database model will have four properties. Add these to your interface:

• discordId: string; – Every user object in Discord has a unique identifier, called a Snowflake, which is used to distinguish them from other users. Unlike a username or discriminator (the four digit number after the username), the id value cannot be changed. This makes it the ideal value for linking your stored data to a Discord user.
• round: number; – This will represent the "round" the user is on in the challenge. When someone completes 100 days of the challenge, they may choose to undertake the challenge again. When they do, they often refer to it as "round 2", for example.
• day: number; – This represents the day the user is on in the challenge.
• timestamp: number; – You will use this value to track when the user last submitted a 100 Days of Code post.

Great! Now you need to define the Schema for your database entries. mongoose uses a Schema object to define the shape of the documents that go in to your database collection. The Schema import has a constructor, which you will assign to a variable.

export const Camper = new Schema();

This constructor takes an object as its argument, and that object defines the database keys and types. Go ahead and pass in an object similar to what your interface looks like.

export const Camper = new Schema({
    discordId: String,
    round: Number,
    day: Number,
    timestamp: Number,
})

Next you need to create the model. In mongoose, the model object serves to create, read, and update your documents in the MongoDB database. Add export default model(); at the bottom of your file.

The model function takes a few parameters. The first is a string, and is the name to use for the documents in your database. For this collection, use "camper". The second argument is the schema to use for the data – use your Camper schema there.

By default, mongoose will use the plural version of your model name for the collection. In our case, that would be "campers". If you want to change that, you can pass in a third argument of { collection: "name" } to set the collection to name.

If you were using JavaScript, this would be enough to get your database model set up. However, because you are using TypeScript, you should take advantage of the type safety. model() by default returns a Document type of any.

To resolve this, you can pass a generic type into the model function. Generic types serve as variables for type definitions, in a sense. You need to set the generic type for your model to use your interface. Add the generic type by changing model to model<CamperInt>.

Just one more step here. Your CamperInt interface only defines the properties you set in the MongoDB document, but doesn't include the standard properties.

Change your export interface CamperInt to export interface CamperInt extends Document. This tells TypeScript that your type definition is an extension of the existing Document type definition – you are essentially adding properties to that structure.

Your final file should look like this:

import { Document, model, Schema } from "mongoose";

export interface CamperInt {
  discordId: string;
  round: number;
  day: number;
  timestamp: number;
}

export const Camper = new Schema({
  discordId: String,
  round: Number,
  day: Number,
  timestamp: Number,
});

export default model<CamperInt>("camper", Camper);

As a safety check, use npm run build again. You should not see any errors in the terminal.

How to Write Bot Commands

You are finally ready to start writing some commands! As this is a 100 Days of Code bot, you should start with the command for creating a 100 Days of Code update.

100 command

Within your commands folder, create a oneHundred.ts file. This will hold your 100 Days of Code command. Import your command interface with import { CommandInt } from "../interfaces/CommandInt";.

Now declare an exported variable oneHundred and give it the CommandInt type:

import { CommandInt } from "../interfaces/CommandInt";

export const oneHundred: CommandInt = {

}

Set the name property to "100", give it a description property similar to "Creates a 100 Days of Code update", and set up the run function as async (message) => {}.

Now for the logic within the function. Your logic will need a few properties from the message object to work, so go ahead and destructure those out: const{ author, channel, content } = message;.

When a user calls this command, it should look something like this:

!100 Here is my 100 Days of Code update.

You will want to extract that text without the !100 part. There are a few was to do this – we are going to slice it out with const text = content.split(" ").slice(1).join(" "). Using the previous example, text would now hold the string "Here is my 100 Days of Code update.".

Time for some database work. Import your CamperModel with import CamperModel from "../database/models/CamperModel". Note that you are importing the default export, instead of a module.

Now you need to see if the user has a record in your database. Use let targetCamperData = await CamperModel.findOne() to prepare for this.

The .findOne() method is used to query the collection for a single record, and takes an object to filter the query. These queries support MongoDB's syntax for advanced searching, but in this case you only need to find the record by the user's discordId. Add {discordId: author.id} as the parameter for the findOne().

What happens if the user's record does not exist yet? If this is their first time using the command, they will not have a document in the database. Add an if condition to check if targetCamperData does not exist:

if (!targetCamperData) {

}

In this block, you are going to reassign targetCamperData to a new document with targetCamperData = await CamperModel.create(). You use the .create() method to generate and save a new document. The method takes an object as the first argument – that object is the document to create. Pass the following object to the method:

targetCamperData = await CamperModel.create({
  discordId: author.id,
  round: 1,
  day: 0,
  timestamp: Date.now()
});

Whether the record exists already or has just been created, your next step is to update it. After your if block, add a line to increment the day value: targetCamperData.day++.

What happens if the user is on day 100? They should not be able to go to day 101, as the challenge is only a hundred days long. You will need to add logic for that. If the user is above day 100, you want to set their day to 1 and increase their round:

targetCamperData.day++;
if (targetCamperData > 100) {
  targetCamperData.day = 1;
  targetCamperData.round++;
}

Now update the timestamp with targetCamperData.timestamp = Date.now();. This may seem redundant, since you did this step in the create method, but this ensures that the timestamp is updated if the data already existed.

You need to save the changes you made to the document. Add await targetCamperData.save(); to do this – mongoose will then save your changes to the document in MongoDB.

Now you will construct the message the bot should send. To do this, you are going to use a message embed. Message embeds are special message formats that are available to Discord bots, which offer additional formatting options and styling.

Start by adding the MessageEmbed class to your imports with import { MessageEmbed } from "discord.js";. Then, after your database logic, create a new message embed with const oneHundredEmbed = new MessageEmbed();. Time to start setting the values of the embed.

The embed title appears as large text at the top of the embed. Set the title to "100 Days of Code" with oneHundredEmbed.setTitle("100 Days of Code");.

The embed description appears as standard text below the title. Set this to the user-provided text with oneHundredEmbed.setDescription(text);.

The embed author appears above the title, and is used to indicate who generated the embed. You will set this with oneHundredEmbed.setAuthor().

This method takes a few arguments, and you will use the first two. The first argument is the author's name. Set this to author.username + "#" + author.discriminator. This will display in the same way that you see a user in Discord: nhcarrigan#0001.

Set the second argument to author.displayAvatarUrl(). This is a method provided by discord.js to fetch the URL for the author's avatar image.

Embed fields are additional title-description pairs that can be nested within the embed, and optionally inlined. These can be created with the .addField() method, which takes up to three arguments. The first argument is the field title, the second argument is the field description, and the third argument is an optional Boolean to set the field as inline.

Add two fields to your embed. The first is oneHundredEmbed.addField("Round", targetCamperData.round, true);, and the second is oneHundredEmbed.addField("Day", targetCamperData.day, true);.

You can add a footer to an embed and appears at the bottom in small text. Set the footer to the data's timestamp with oneHundredEmbed.setFooter("Day completed: " + new Date(targetCamperData.timestamp).toLocaleDateString();. The toLocaleDateString() method will take a Date object and convert it to a locale-specific string based on the location of your bot's server.

Now you need to send that message embed. The channel property you extracted from the message value earlier represents the Discord channel in which the message was sent. This object has a .send() method, which you can use to have the bot send a message back to that channel. Use await channel.send(oneHundredEmbed) to send your new embed to that channel.

To keep the channel clean, add an await message.delete() to have the bot delete the message that triggered the command. Your final code should look like this:

import { CommandInt } from "../interfaces/CommandInt";
import CamperModel from "../database/models/CamperModel";
import { MessageEmbed } from "discord.js";

export const oneHundred: CommandInt = {
  name: "100",
  description: "Creates a 100 Days of Code update",
  run: async (message) => {
    const { author, channel, content } = message;
    const text = content.split(" ").slice(1).join(" ");

    let targetCamperData = await CamperModel.findOne({ discordId: author.id });

    if (!targetCamperData) {
      targetCamperData = await CamperModel.create({
        discordId: author.id,
        round: 1,
        day: 0,
        timestamp: Date.now(),
      });
    }

    targetCamperData.day++;
    if (targetCamperData.day > 100) {
      targetCamperData.day = 1;
      targetCamperData.round++;
    }
    targetCamperData.timestamp = Date.now();
    await targetCamperData.save();

    const oneHundredEmbed = new MessageEmbed();
    oneHundredEmbed.setTitle("100 Days of Code");
    oneHundredEmbed.setDescription(text);
    oneHundredEmbed.setAuthor(
      author.username + "#" + author.discriminator,
      author.displayAvatarURL()
    );
    oneHundredEmbed.addField("Round", targetCamperData.round, true);
    oneHundredEmbed.addField("Day", targetCamperData.day, true);
    oneHundredEmbed.setFooter(
      "Day completed: " +
        new Date(targetCamperData.timestamp).toLocaleDateString()
    );

    await channel.send(oneHundredEmbed);
    await message.delete();
  },
};

If you remember, you created a list to hold all of your commands. You need to add your new command to that list. Head back to your _CommandList.ts file. Import your new command with import { oneHundred } from "./oneHundred";, then add oneHundred to your empty CommandList array:

import { CommandInt } from "../interfaces/CommandInt";
import { oneHundred } from "./oneHundred";

export const CommandList: CommandInt[] = [oneHundred];

Now you can test it out! Use npm run build and npm start to get the bot started. Try sending !100 This is my first post! in the channel. The bot should respond with an embed and delete your message.

You can see the embed, with the author, title, description, fields, and footer.

View command

What happens if a user forgets if they submitted or not, or wants to see what day they are on? You should add a command to view current 100 Days of Code progress.

In your commands directory, create a view.ts file. Like before, import your command interface and CamperModel, and create a new command called view. Set the name to "view", the description to something like "View your current 100 Days of Code progress", and the run command to async (message) => {}.

You won't need the message content for this command, so extract the author and channel values from the message like you did before: const { author, channel } = message;.

Just like the 100 command, you need to fetch the user's data from the database. This time, however, if the data does not exist you will not be creating it – so you can use const here instead of let: const targetCamperData = await CamperModel.findOne({ discordId: author.id });

Now, if the user doesn't have a data record yet, they haven't started the challenge with the bot. You should send a message to let them know how to do this.

if (!targetCamperData) {
  await channel.send("You have not started the challenge yet.");
  return;
}

Construct an embed, similar to the one you built for the 100 command. Don't forget to import the MessageEmbed class!

    const camperEmbed = new MessageEmbed();
    camperEmbed.setTitle("My 100DoC Progress");
    camperEmbed.setDescription(
      `Here is my 100 Days of Code progress. I last reported an update on ${new Date(
        targetCamperData.timestamp
      ).toLocaleDateString()}.`
    );
    camperEmbed.addField("Round", targetCamperData.round, true);
    camperEmbed.addField("Day", targetCamperData.day, true);
    camperEmbed.setAuthor(
      author.username + "#" + author.discriminator,
      author.displayAvatarURL()
    );

A couple of key differences here. Instead of taking a text input from the user, you are using a fixed description value to indicate this is a view embed instead of a 100 embed. Since you use the timestamp in the description, you do not need to add a footer.

Just like before, send the embed to the message channel and delete the original message. Your final file should be:

import { CommandInt } from "../interfaces/CommandInt";
import CamperModel from "../database/models/CamperModel";
import { MessageEmbed } from "discord.js";

export const view: CommandInt = {
  name: "view",
  description: "Views your 100 Days of Code progress.",
  run: async (message) => {
    const { author, channel } = message;

    const targetCamperData = await CamperModel.findOne({
      discordId: author.id,
    });

    if (!targetCamperData) {
      await channel.send("You have not started the challenge yet.");
      return;
    }

    const camperEmbed = new MessageEmbed();
    camperEmbed.setTitle("My 100DoC Progress");
    camperEmbed.setDescription(
      `Here is my 100 Days of Code progress. I last reported an update on ${new Date(
        targetCamperData.timestamp
      ).toLocaleDateString()}.`
    );
    camperEmbed.addField("Round", targetCamperData.round, true);
    camperEmbed.addField("Day", targetCamperData.day, true);
    camperEmbed.setAuthor(
      author.username + "#" + author.discriminator,
      author.displayAvatarURL()
    );

    await channel.send(camperEmbed)
    await message.delete();
  },
};

Add your new view command to your _CommandList.ts file with an import, and put the command in the CommandList array. Then use npm run build and npm start to test your new changes. Send "!view" in your channel and you should see the bot respond:

Edit command

Unfortunately, if a user makes a typo in their 100 Days of Code post, they can't edit the message because the bot sent it. But you can add a command that will allow them to do this.

Create an edit.ts file in your commands directory. Import your command interface and declare a new command called edit. Set the name to "edit", the description to something like "Edits a previous 100 Days of Code post", and prepare the run function as you have before.

Within the function, extract the author, channel, and content properties from the message object.

The edit command will take a Discord message id, followed by the updated text to use. You can destructure those from the message content with const [, targetId, ...text] = content.split(" ");.

The first element in the array would be the !edit command call, which is not needed for this command so you do not need to assign it to a value. The targetId element would be the id of the message to edit. ...text that uses the spread operator to assign the remaining message content to the text variable, as an array.

Now you need to use the targetId to get the actual message from Discord. The channel value has a messages property which represents all of the messages sent in that channel. You can use the fetch method on that messages property to get a specific message (or multiple messages). Set this up as const targetMessage = await channel.messages.fetch().

The .fetch() method can take an object containing the options for the fetch request, or it can take a string as the id of the message to fetch. Because you have the id, and are only fetching one message, you can pass targetId to the .fetch() method as the only parameter.

It is possible that the targetMessage does not exist. For example, if the user provided an invalid id string (or no id string at all). You'll need to add logic to check if the targetMessage is not found:

    if (!targetMessage) {
        await channel.send("That does not appear to be a valid message ID.");
        return;
    }

Now that you have asserted that the message exists, you can start working with the properties. Because your bot sends the message as an embed, the content property you are used to working with will be empty. Instead, you can find the embed within the embeds property.

The embeds property is an array of MessageEmbed objects. Since you wrote the bot's code to only send one embed, you can access that embed with const targetEmbed = targetMessage.embeds[0];.

Now that you have the embed, you need to confirm that the embed is from one of that user's 100 Days of Code posts. Thankfully, you set the user as the author of the embed. You can check if the embed's author information does not match the message author's information:

    if (
      targetEmbed.author?.name !==
      author.username + "#" + author.discriminator
    ) {
      await channel.send(
        "This does not appear to be your 100 Days of Code post. You cannot edit it."
      );
      return;
    }

You have accounted for the message belonging to a different user (or not having the correct embed entirely), so now you can edit the embed.

Like you did before, set the description of the embed with the .setDescription() method. You'll need to use .join(" ") on the text variable this time, since it is currently an array. targetEmbed.setDescription(text.join(" "));

Rather than sending a new message, you need to edit the existing message. You have the existing message stored in targetMessage, so you can use the .edit() method to change that message directly.

await targetMessage.edit(targetEmbed); will change the message's embed to your modified version. Then delete the message that triggered this command with await message.delete();. Your command should look like this:

import { CommandInt } from "../interfaces/CommandInt";

export const edit: CommandInt = {
  name: "edit",
  description: "Edits a previous 100 Days of Code post.",
  run: async (message) => {
    const { author, channel, content } = message;
    const [, targetId, ...text] = content.split(" ");

    const targetMessage = await channel.messages.fetch(targetId);

    if (!targetMessage) {
      await channel.send("That does not appear to be a valid message ID.");
      return;
    }

    const targetEmbed = targetMessage.embeds[0];

    if (
      targetEmbed.author?.name !==
      author.username + "#" + author.discriminator
    ) {
      await channel.send(
        "This does not appear to be your 100 Days of Code post. You cannot edit it."
      );
      return;
    }

    targetEmbed.setDescription(text.join(" "));

    await targetMessage.edit(targetEmbed);
    await message.delete();
  },
};

Add the command to your _CommandList.ts file, importing it and adding the variable to the array. Then use npm run build and npm start to run the bot again.

To grab a message ID, you should have Developer Mode enabled in your Discord client. If you have not done so, visit your settings and select the "Advanced" section. Toggle "Developer Mode" on:

Then head back to your channel and right click on your original 100 Days of Code message. You should see an option in the context menu to copy the message ID:

Select that option and you will get an ID (mine was 855559921666621441). Then in the same channel, use !edit 855559921666621441 This is an edited post!, replacing my value with the one you got from the "Copy ID" option. The bot should edit the existing embed with your new content.

Help command

You are almost there! One more command to go. Many bots have a help command, which returns a list of available commands. You should add one to your bot as well.

One last time, create a help file in your commands directory. Import your CommandInt interface and set up your command as help. Set the name to "help", and the description to something like "Returns information on the bot's available commands.". Set up your run function.

This time you only need the message's channel property, so no need to destructure anything here. Instead, import the MessageEmbed class from discord.js, and go ahead and import your command list too: import { CommandList } from "./_CommandList";.

Construct a new MessageEmbed and assign it to a helpEmbed variable. Set the title to "Available Commands:" and the description to something similar to "These are the available commands for this bot.".

Now you need to add a field to the embed and dynamically generate the list of commands. Start by adding the field with helpEmbed.addField(). Use the first parameter to set the field name to "Commands:". For the description (the second parameter), you will use the CommandList array to generate a readable list of commands.

    helpEmbed.addField(
      "Commands:",
      CommandList.map((el) => `\`!${el.name}\`: ${el.description}`).join("\n")
    );

The process here is two-part. First, using the built-in array method .map, you are creating a new array from your array of CommandInt objects. This array contains strings formatted using Markdown so the command name and description are readable. The string for your help command would look like:

!help: Returns information on the bot's available commands.

You are then joining that array of strings with a new-line separator, which will create a vertical list of commands in a single string (embed fields require strings for the description).

Send the embed to the channel. Because you did not destructure the channel property out of the message object, you will need to use message.channel.send(helpEmbed); directly.

This time, do not delete the original message – you did not add an author to the help embed, so preserving the original message helps moderators see who used the command. Your help command should look like:

import { CommandInt } from "../interfaces/CommandInt";
import { MessageEmbed } from "discord.js";
import { CommandList } from "./_CommandList";

export const help: CommandInt = {
  name: "help",
  description: "Returns information on the bot's available commands.",
  run: async (message) => {
    const helpEmbed = new MessageEmbed();
    helpEmbed.setTitle("Available Commands!");
    helpEmbed.setDescription(
      "These are the available commands for this bot."
    );
    helpEmbed.addField(
      "Commands:",
      CommandList.map((el) => `\`!${el.name}\`: ${el.description}`).join("\n")
    );

    await message.channel.send(helpEmbed);
  },
};

Import your help command into your _CommandList.ts file and add the command to your array. With this final command, your _CommandList.ts file should be:

import { CommandInt } from "../interfaces/CommandInt";
import { oneHundred } from "./oneHundred";
import { view } from "./view";
import { edit } from "./edit";
import { help } from "./help";

export const CommandList: CommandInt[] = [oneHundred, view, edit, help];

Use npm run build and npm start one last time to test this feature. Send !help in your channel and the bot should respond:

Conclusion

Congratulations! You have successfully built a Discord bot for the 100 Days of Code challenge.

If you are interested in exploring further, you can view the source code for the live bot that inspired this tutorial, which includes custom error logging, external error reporting, and a documentation site.

In the first hour, Deployment Hell struck and I had to take the bot down for maintenance. Another hour passed and I managed to patch up my bot and send it back.

Now my bot is up and running and more resilient than ever. As for me, I survived and even thrived in Deployment Hell 🔥. In this deployment postmortem, I'm going to show you how.

I sent my AI chatbot into the server at 2 pm on June 2nd, and hype started gathering around it. The chat went on for a while and all was nice and smooth.

At 3:20 pm, everything started falling apart: My free-tier API key reached its hourly request limit, and I had no choice but to take down the bot and the server for maintenance.

Never mind the typos in my message 😅 This incident caught me totally unprepared

Despite being struck by Deployment Hell in the first hour on the first day of my project debut, I knew I need to sit down, take a deep breath, and recover from this incident.

What Went Well with Deployment

As a first step, I didn't forget to congratulate myself on what went well despite this mishap. Clearly, people were enthusiastic about chatting with my chatbot, to the extent that we went over rate limit.

Moreover, in the brief hour when I observed people interact with my bot in real time, I discovered several good design choices that I've consciously, unconsciously, or subconsciously made.

Avoid Feature Creep At All Costs During Development

I originally developed my code for a tutorial, so I kept my code as simple and readable as possible, without complicated features that won't serve my bot's main use case: chatting.

That said, I marked down TODOs and stretch goals in my code, hoping to get back to those if necessary. For example:

 # TODO: cache chat history in DB and load
 # TODO: after each user input and bot input,
 # append them to conversation history for the next query
 ...
 # FIXME: better make this try-except block more fine-grained

As I observed users interacting with the bot, I'm actually relieved that I didn't implement the memory cache feature. Several users were talking with the bot at once, each along their different conversation thread. If I were to keep track of the conversation history, I would have to create a unique log for each user, further complicating database operations.

Abide by the Principle of Least Privilege

One thing I learned in my Computer Security class is the Principle of Least Privilege (PoLP) – granting an application the minimum amount of access it needs to do its job.

My chatbot only needs two low-level security permissions: View Channel for reading users' messages, and Send Text Messages for replying to users.

Of course I could have given it more fancy permissions like those shown below in the image, just in case it needs any. But that would have violated PoLP and who knows whether my malfunctioning bot will bring down anything else with it when it fails?

Permission settings for Discord bots

Other Observations from Deploying My Bot

Despite its ephemeral lifespan, my bot offered me an opportunity to conduct user research in the real world. This is a server with 1000+ users, not my cozy dev server where my friends and I hang out and take turns to politely exchange lines with the bot.

Here, I observed several interesting user behaviors:

• People tend to ask the bot open-ended questions instead of factual ones. Because I built my bot based off a video game character, when developing the AI model, I was keen on making sure the model learns the canon information about the character, like name, age, and role in the game. I was relieved when I saw that people are much more curious about the bot character's preference for ice cream flavor than their factual place of birth.

• People use a lot of emoticons :), emojis 😃 and GIFs as they text. However, these will most likely be treated as <UNKNOWN> tokens in the AI model's tokenizer, which means I should sanitize user inputs.

I was also very fortunate to receive direct feedback from friendly people on the server. One feature request that I got was to make the bot attach its response to a user's message thread, instead of just dumping its response in the channel.

Buzzing with excitement from people's enthusiasm and armed with insights from user research, I was ready to patch up my bot and send it back as soon as possible.

What I Needed to Fix

As a good development habit, I kept my code well-organized and modularized, so switching from the production bot back to my development bot requires no more than copy-pasting the dev bot's API key.

Once I was running on my dev server, I sat down to identify the types of problems I needed to tackle.

Fatal Crashes

The fatal bug that caused me to take down my bot was that I hit an hourly API rate limit. I took the obvious approach of keeping redundancy in my system: Keep an alternative API key, and once the primary one runs out, switch over to the alternative, and then switch back at the turn of the hour.

Workaround aside, I noted that this is a short-term solution. If I need to properly scale up my system, I should make an estimate of the number of requests per hour, as I'll discuss at the end of this section.

New Features for Usability

I decided on several new features that will make my bot more user-friendly. Some highlights are:

• As some server people suggested, I re-programmed the bot so that instead of dumping responses to different user messages into the channel, it directly responds to each user message in the message's thread.

• I sanitized user inputs by removing Unicode emojis and Discord-specific <:some_hilarious_gif> tags. This will limit <UNKNOWN> tokens that my AI model will receive and help it generate better responses.

• I implemented a magic command $ignore [message] that allow users to send a message to the channel without triggering a bot response. This feature comes from my observation that, whenever the bot says something funny or smart (or both!), users will remark on that by sending a text intended for their friends (and not the bot) to the channel. It'd be annoying to still receive a bot response on a remark intended for a friend. Hence, I hope that this magic command will address this user pain point.

• I implemented magic commands for the server moderators to interface with my bot (stopping or rebooting) so that they can keep the bot under control without having to access my Repl.it server. This makes both my job and theirs easier.

Future-proof Logging

In my cozy dev server, there is little complexity, whereas on this 1000+ user server where 40% of users are online at any given moment, complexity explodes.

There are multiple channels besides the chat channel dedicated to my bot, multiple user roles and permission levels, multiple users typing at the same time, and so on.

While I certainly cannot prevent all possible failure scenarios, what I could do is to protect the important part of my code with a try-except block and log out all information that might reveal the cause of a failure. Since real-time system bugs are subtle and difficult to reproduce, logging will save me lots of headaches down the road.

except Exception as e:
            print(e, 'Offending channel', message.channel, 
            'Offending message', message.content, 
            'Offending bot response', bot_response, 
            sep='\n', end='\n\n')

Scalability

Estimating under some system constraints is where basic statistics and heuristics come in. Hugging Face's model Inference API imposes two limits on the scalability of my system:

1. A 10k tokens (characters) per hour rate limit, which is about 300 queries.

2. A 30k tokens per month quota for free-tier accounts, which is about 900 queries.

Wonder how I get the numbers? Fun fact: 1) 10k characters is between 1430 and 2500 words. We will take 2100 since Discord messages usually use simple, short words. 2) The average length of a text message is 7 words. 2100 / 7 = 300 messages

After processing these numbers, the fact that I hit the per-hour rate limit during the first hour of my bot debut is quite a remarkable feat. People are clearly hyped about my witty chatbot. 🥳

Suppose the hype recedes and life goes on, consider now a hypothetical scenario where 20 users (2% of the 1000+ on the server) regularly chat with my bot, each for 25 lines, in the two hours following dinner. This produces a total of 500 queries in two hours (or 250 per hour), meaning that my bot is safe from the per-hour rate limit of 300.

However, in a month, 500 * 30 = 15,000 queries, 15 times more than my quota of 900. If my bot is indeed this popular, I would need to switch to a higher-tier subscription plan to ensure that it remains available.

From Tutorial Code to Production Code

Compared to my tutorial code which strives to be simple, readable, and educative, my production code is longer, more complicated, but also more robust.

What Makes a Great Side Project

Having emerged from Deployment Hell, like my bot, I'm more resilient than ever and have gained new insights about the principles and challenges in real-world software engineering.

As a final takeaway, I reflect on what makes my Discord AI chatbot this popular. (On June 3rd, a day when it's up 24 hours, it had already busted the monthly 30k quota on both my account and one I borrowed from a friend, totaling 2,000+ messages. 🤓)

I have completed and polished various side projects that received positive feedback, but none of them were as half popular as this one. In retrospect, it's not too hard to see why.

Among projects that I'm most proud of, I built a chess engine and a rhythm game engine for the Ren'Py Visual Novel (VN) game development engine. Both are rated 5-star on itch.io, a popular platform for publishing indie games.

These projects, however, are open-source engines intended for developers to integrate into their VN games more than standalone playables that can entertain players for hours.

In comparison, my Discord AI chatbot manages to capture each of the following elements that distinguishes a great side project from a good one:

• Audience: I'm fortunate to have this friendly server with 1000+ users who are open to experimenting with the bot and provide me with helpful feedback.

• Accessibility: For people to enjoy my chabot, there is nothing special they need to add to their routine - not even opening up a new web app - they just log into Discord as usual, and voilà, the bot is here to chat!

• Interactivity: Without interactive components, even the most visually-astonishing game will fail to retain players' attention. Nothing to worry about for my chatbot though: Like a loyal friend, it always has something to quip about whenever you need a good chat.

If you'd like to learn more about my methodology for working on side projects, check out my previous blog post:

Also check out my chatbot tutorial!

You can also try this out in JavaScript:

This is the same process I used to build freeCodeCamp's moderation chat bot for our community's self-hosted chat server. This code is now running in production, and lots of people are using it.

How to Set up a Rocket.Chat Server

Your first step is to get an instance of Rocket.Chat running locally – you will need this to test the bot's functionality.

You can use freeCodeCamp's docker file, which will spin up both Rocket.Chat and MongoDB automatically for a development environment. This will save you a lot of time.

You can either clone this repository, or manually create your own docker file based on our configuration. This tutorial will assume that you are using our existing docker file.

Note: If you do not have docker installed, you will need to install it. The installation process is different for each operating system. I personally use Windows 10, so I installed the Docker desktop client and had to enable hardware virtualisation in my BIOS.

Within your Rocket.Chat directory, create a .env file and insert the following contents:

COMPOSE_FILE=docker-compose.dev.yml
PORT=3000
ROOT_URL=http://localhost:3000
ROCKETCHAT_VERSION=latest

Then open your terminal pointed at that same directory and run:

docker-compose up -d

You should see three success messages in your terminal:

Image showing the console output for the docker-compose up -d command. Three docker images were created, and each shows done.

Now if you open your browser and navigate to localhost:3000 you should see your local Rocket.Chat instance. The first screen you see will be the Setup Wizard, which will walk you through creating your Admin account.

Most developers use the Admin account for root-level access to configure their chat. Because this is a local instance, your credentials' security is less important than in a live instance.

Fill in your information to create the admin account:

Image showing the Admin Info modal, with inputs for Name set to Nicholas Carrigan, Username set to nhcarrigan, Organization Email set to nick@freecodecamp.org, and Password which is obfuscated. Below the input fields is a button labelled Continue.

The next screen is the Organization Info screen. This information is optional. For this tutorial, we will leave this information blank.

Clicking Continue will take you to the Server Info page. Here you set the name for your chat server (which will appear in the title metadata), your default language, the server type, and the 2FA setting.

Be sure to turn off the automatic 2FA setup for your local instance or you could be locked out of your own server.

Image showing Server Info modal, with inputs for Site Name set to fCC ChatBot tutorial, Language set to Default, Server Type with no selection made, and Auto opt in new users for Two Factor via Email set to No. Below the input fields are buttons labelled "Back" and "Continue".

The final step is to optionally register your server and gain access to Rocket.Chat's services such as push notifications. Note that these are paid services.

For the purpose of this tutorial, you can select the Keep standalone option. Then you can decide whether you want any paid services later.

After clicking Continue, you'll see a modal indicating that your workspace is ready to use. Then you should see your new chat room. The default channel created by the Setup Wizard is general.

Image showing Rocket.Chat after completing the setup wizard. Sidebar on the left shows a general channel, and primary windows shows a system message that "nhcarrigan has joined the channel".

If you see this, congratulations. You are half way there and now have a functional chat server.

How to Set up a Bot Account in Rocket.Chat

Now we need to create a bot user in our local chat server for our code to connect to.

Select the three dots at the top of the sidebar and choose Administration. Then select Users from the new sidebar that appears, and click the +New button in the top right. This opens a pane for creating a new user account.

Fill in the information and credentials for your bot account.

A few key things to note:

• Leave Require password change and Set random password and send by email set to off.
• Leave Send welcome email set to off.
• Select bot from the Roles dropdown menu.

Image showing the Rocket.Chat settings screen. The left sidebar shows the list of settings - Users has been selected. The center screen shows a list of user accounts - nhcarrigan and Rocket.Cat. The right sidebar shows the Add User interface, with inputs for Name set to Tutorial Bot, Username set to tutorial-bot, Email set to nhcarrigan@gmail.com, Verified toggled off, Status Message with no value, Bio with no value, Nickname with no value, Password which is obfuscated, Require Password Change which is toggled off, Set random password and send by email which is toggled off, Roles with "bot" selected, Join default channels which is toggled on, and Send Welcome Email which is toggled off.

Rocket.cat is a built-in account used for system notifications (i.e. Rocket.Chat updates).

Save the changes, and your bot account should now be created! Keep a note of the username and password, as we will need these for the code.

How to Code your Rocket.Chat Chatbot

Now it's time to create the code. Start with a new, empty folder for your project.

Initial Rocket.Chat Chatbot Project Setup

We will begin with initializing a node.js project. You are welcome to use npm init to generate a package.json, or you may create one manually.

Either way, you will need to add some specific values to the scripts section:

  "scripts": {
    "prebuild": "rm -rf ./prod",
    "build": "tsc",
    "start": "node ./prod/bot.js"
  },

Next you will install your necessary dependencies. First, install the development dependencies:

npm install --save-dev typescript @types/node

Then, install your primary dependencies:

npm install @rocket.chat/sdk dotenv

Your next step is to set up the TypeScript configuration.

If you have installed TypeScript globally, you'll be able to call tsc --init and automatically generate a configuration file. Otherwise, you'll need to manually create a tsconfig.json file in your project's root directory.

Either way, these are the settings you will need for this project:

{
  "compilerOptions": {
    "target": "ES5",
    "module": "CommonJS",
    "rootDir": "./src",
    "outDir": "./prod",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "forceConsistentCasingInFileNames": true,
    "resolveJsonModule": true,
    "noImplicitAny": false,
  }
}

If you are using git for version control, you will need to create a .gitignore file. This file tells git which files/folders to ignore. In this case, you want to ignore:

• the compiled JavaScript in prod
• your Node modules
• your .env secrets.

Add these to your .gitignore:

/node_modules/
/prod/
.env

Speaking of secrets, you should set those up now. Create a .env file, and add the following values:

ROCKETCHAT_URL="localhost:3000"
ROCKETCHAT_USER="tutorial-bot"
ROCKETCHAT_PASSWORD="********"
ROCKETCHAT_USE_SSL=""

View the code at this point.

How to Write the Primary Rocket.Chat ChatBot Code

Now it is time to write the initial bot code. Create a src folder within your project directory, and inside that src folder create a bot.ts file. Your file structure should now look like this:

_Image showing a file tree. From top to bottom: A node_modules folder, which is collapsed, a src folder which contains a bot.ts file, a .env file, a .gitignore file, a .package-lock.json file, a package.json file, and a tsconfig.json file. The files show they are being tracked by git, except the node_modules folder and .env file._

The package-lock.json file is created/updated by npm whenever you run install. This should be committed to your repository too, as it is required for the npm ci command.

Within your bot.ts file you will write the basic code that powers your bot. Start with your necessary imports:

import { api, driver } from "@rocket.chat/sdk";
import dotenv from "dotenv";

Because node doesn't load environment variables automatically, you need to call dotenv's config() method to bring your .env values into the node process:

dotenv.config();

Now you can extract those variables from the node environment. Use destructuring to grab the values:

const {
  ROCKETCHAT_URL,
  ROCKETCHAT_USER,
  ROCKETCHAT_PASSWORD,
  ROCKETCHAT_USE_SSL,
} = process.env;

Aside from ROCKETCHAT_USE_SSH, these environment values are required. Missing one will cause the code you write to error out, so you need to add a step to verify that all of these values are present.

if (!ROCKETCHAT_URL || !ROCKETCHAT_USER || ROCKETCHAT_PASSWORD) {
  console.error("Missing required environment variables.");
  process.exit(1);
}

Now you can use the Rocket.Chat SDK to connect your bot to the account you created.

Because the methods in the SDK are asynchronous, you will use an anonymous immediately-invoked function expression (IIFE) to enable async/await features.

(async () => {
  // Nothing here yet
})();

These next steps will be written inside this function.

First, determine if your bot should use SSL to connect to the chat server. If your chat server uses HTTPS://, this should be set to true. Because you are developing locally, this is set to false as localhost does not have an HTTPS protocol.

To ensure your code would work in a production environment as well, you can dynamically set this value based on your environment variables:

const ssl = !!ROCKETCHAT_USE_SSL;

Next, use the SDK driver to interface with your chat server. Connect the driver to your server:

await driver.connect({ host: ROCKETCHAT_URL, useSsl: ssl });

Login as the bot account:

await driver.login({
    username: ROCKETCHAT_USER,
    password: ROCKETCHAT_PASSWORD,
  });
await api.login({ username: ROCKETCHAT_USER, password: ROCKETCHAT_PASSWORD });

Have the bot join your general room, to handle instances where the bot isn't already in the room. Also tell the bot to listen for messages with the subscribeToMessages() method.

  await driver.joinRooms(["general"]);
  await driver.subscribeToMessages();

Finally, have the bot send a message when it comes online (so you can confirm connection status).

  await driver.sendToRoom("I am alive!", "general");

Now, build and run the code. Call these necessary scripts in your terminal:

npm run build
npm run start

After some built-in logging from the SDK, you should see the bot send its online message in your chat server.

Image depicting a Rocket.Chat message. The message was sent by tutorial-bot, which has the bot role by its name. The message text reads I am alive!.

View the code at this point.

How to Write the Command Handler

Your bot will now connect to the chat server and listen for messages, but it does not have any functions.

Before you can add commands, you need to build the infrastructure to handle those commands.

First, tell the bot to handle messages. Just after your subscribeToMessages() call, add a line to handle responding to messages:

driver.reactToMessages();

You'll see an error in your Intellisense, because the reactToMessages() method expects a callback function.

You could write the callback function within this method directly, but instead you will modularise your code and create an exported handler. This keeps your code cleaner and more maintainable.

Create a folder called commands within your src folder, and add two files: CommandHandler.ts and CommandList.ts. Within the CommandList.ts file, we are going to add a single line for now:

export const CommandList = [];

As you build commands, you will add them to this array to be able to iterate through them in our handler.

Now you need to write your handler's logic in the CommandHandler.ts file.

Start with your required imports:

import { driver } from "@rocket.chat/sdk";
import { IMessage } from "@rocket.chat/sdk/dist/config/messageInterfaces";
import { CommandList } from "./CommandList";

Define the command handler function:

export const CommandHandler = async (
    err: unknown,
    messages: IMessage[]
): Promise<void> => {
    // Code will go here.
}

Add some error handling.

  if (err) {
    console.error(err);
    return;
  }
  const message = messages[0];
  if (!message.msg || !message.rid) {
    return;
  }

If you see an error in the err parameter, you need to return early.

The messages parameter takes an array of messages, but you want to react to the first message, so we extract it from that array as message.

Then, for TypeScript's assertion handling, you need to exit early if certain properties are missing or undefined. In this case, message.msg is the text content of the message, and message.rid is the ID of the room the message was received in.

For cleaner/more readable code, you can destructure some values out of the message object. Get the room's name from the rid value - the SDK includes a method for doing just this. Also get the prefix and the command that is called.

  const roomName = await driver.getRoomName(message.rid);
  const [prefix, commandName] = message.msg.split(" ");

Add the logic to iterate through our array of commands. TypeScript will identify some errors due to missing structures, but you can ignore those for now as you have not written the commands yet.

  if (prefix === "!fCC") {
    for (const Command of CommandList) {
      if (commandName === Command.name) {
        await Command.command(message, roomName);
        return;
      }
    }
    await driver.sendToRoom(
      `I am sorry, but \`${commandName}\` is not a valid command.`,
      roomName
    );
  }

This block of code might be a bit confusing as we have not established how commands work yet.

First, the bot determines if the message begins with the correct prefix. If it does not, the bot will ignore the message.

Then the bot iterates through the list of commands, and if it finds a command for which the name value matches the command name sent in the message, it will run that command.

If it does not find any matching commands, it will send a response in the room that the command was not valid.

Before you move on to a command, head back to the reactToMessages() call in the bot.ts file and pass your new handler as the callback:

  driver.reactToMessages(CommandHandler);

You may need to manually import it:

import { CommandHandler } from "./commands/CommandHandler";

View the code at this point.

How to Write a Command

TypeScript offers an interface feature which can be used to define an object structure.

In your src folder, create an interfaces folder, and create a CommandInt.ts file.

Inside that file, you will define your command type. First, import the message type again.

import { IMessage } from "@rocket.chat/sdk/dist/config/messageInterfaces";

Now build the exported interface for the command definitions.

export interface CommandInt {
    name: string;
    description: string;
    command: (message: IMessage, room: string) => Promise<void>
}

Congratulations! You are now ready to build the ping command.

Within your src/commands folder, create a ping.ts file. Start with your necessary imports: the Rocket.Chat driver and your new command interface.

import { driver } from "@rocket.chat/sdk";
import { CommandInt } from "../interfaces/CommandInt";

Define and export the command:

export const ping: CommandInt = {
    name: "ping",
    description: "Pings the bot.",
    command: async (message, room) => {
        // Code will go here.
    }
}

Let's have the bot respond with "Pong!" when this command is called. Inside the function, replace the comment with:

await driver.sendToRoom("Pong!", room);

Now, load this command in your list of commands. Open the CommandList.ts file, where you will import our new command and include it in the array.

import { ping } from "./ping";

export const CommandList = [ping];

With this, you should see the errors in the CommandHandler.ts file disappear as well, because TypeScript is inferring that the CommandList array contains CommandInt types.

For extra type safety, and to ensure you do not accidentally add values to your CommandList that aren't proper CommandInt objects, explicitly type this variable.

import { CommandInt } from "../interfaces/CommandInt";
import { ping } from "./ping";

export const CommandList: CommandInt[] = [ping];

Run your build and start scripts again to test this new feature.

Call your ping command in the chat room. You should see a successful response:

Image depicting a Rocket.Chat conversation. The first message was sent by nhcarrigan, who has the Admin role by his name. The first message content reads "!fCC ping". The second message was sent by tutorial-bot, which has the Bot role by its name. The second message content reads "Pong!".

Call a pong command. You should see that the bot identifies it is not a valid command:

Image depicting a Rocket.Chat conversation. The first message was sent by nhcarrigan, who has the Admin role by his name. The first message content reads "!fCC pong". The second message was sent by tutorial-bot, which has the Bot role by its name. The second message content reads "I am sorry, but pong is not a valid command".

View our final code.

Further Exploration

Congratulations! You have now successfully built a basic Rocket.Chat chatbot.

If you would like to explore further features and command implementations, feel free to browse our live bot's codebase.