Instead of interacting with the agent through a terminal, we’ll expose it over HTTP so multiple users can access it through a chat-style frontend interface. Each session will maintain its own conversation history and streamed responses.

The local AI agent will be built with LangChain v1, Ollama, Qwen, and Python, running on your own machine and ready to plug into larger applications without any per-call model API charges.

Table of Contents

• Background

• What is FastAPI?

• What is Streamlit?

• What Is Multi-User Support?

• Motivation and Architecture

• Step 1: Install Ollama and Pull the Model

• Step 2: Install Python Dependencies

• Step 3: Build the agent and API layer with FastAPI

• Step 4: Build Streamlit UI

• Step 5: Run the backend app

• Step 6: Run the frontend app

• Sample Output

• What to Improve Before Production

• Conclusion

Background

Many AI agents start out as simple Python scripts that run in a command-line terminal. You type a message, the agent responds, and everything happens in a single local session.

That setup is great for development and testing, but it becomes limiting when you want other people or applications to interact with the agent.

To make an AI agent truly useful, we need to expose it through an interface that other users can access. A REST API is a practical way to do that.

To follow this tutorial, you'll need Ollama installed on your machine. The tutorial works on macOS, Windows, and Linux. I'm using a MacBook Pro with 32 GB of RAM, but you can run this on a lower-memory machine by choosing a smaller Qwen model from Ollama.

What is FastAPI?

FastAPI is a Python web framework for building APIs. In this tutorial, it gives us a simple way to expose the agent over HTTP so other apps, scripts, or services can call it.

FastAPI is a good fit for AI apps because it gives us a clean boundary around the system. We define the request and response models in Python, FastAPI validates them automatically, and it turns HTTP requests into Python objects and Python objects back into JSON. It also generates interactive API docs for free and supports async endpoints, which is useful for AI workloads that may take longer to respond.

What is Streamlit?

Streamlit is a Python framework for building lightweight web interfaces with minimal frontend work. It lets us create interactive browser-based apps using normal Python code instead of HTML, CSS, and JavaScript.

In this tutorial, Streamlit sits on top of the FastAPI backend as a thin client. FastAPI exposes the AI agent over HTTP, and Streamlit gives us a simple UI for calling that API and displaying the results. That separation keeps the backend reusable while still making the agent easy to use in the browser.

What Is Multi-User Support?

Multi-user support means the AI agent can handle requests from more than one user while keeping each user’s session separate.

For example, User 1 asks the agent one question and User 2 asks a different question. The agent should remember the correct context for each user independently. Without multi-user support, all users may end up sharing the same conversation state, which can lead to mixed responses, incorrect memory, or overwritten context.

Motivation and Architecture

Turning an AI agent into an API is the natural next step after building it locally. A Python script is great for experimenting, but an API makes the agent reusable. And adding multi-user support makes the agent extensible to be used by others.

To keep things simple, we’ll use a small local agent powered by Ollama and Qwen. The agent has two tools: one for checking the current time and another for counting words.

FastAPI provides the HTTP layer by exposing one endpoint called /chat/stream. When the request comes in with a user message, Pydantic validates the request, LangChain handles the agent loop and tool calling, and the final answer is returned as stream. Streamlit sits on top of that API and acts as a frontend that sends requests to the API and displays the results.

Example request:

{ 
    "message": "How many words are in: LangChain makes tool calling easier",
    "user_id":"123e4567-e89b-12d3-a456-426614174000"
 }

Example response:

{
  "answer": "There are **5** words in LangChain makes tool calling easier."
}

The model runs locally through Ollama, so there are no per-call model API charges.

Step 1: Install Ollama and Pull the Model

To get started, install the Ollama application for your platform.

We’ll use Qwen as the chat model. I’m using qwen3.5:4b. If your machine has less RAM, you can use qwen3.5:0.8b instead.

ollama pull qwen3.5:4b

Step 2: Install Python Dependencies

Create a virtual environment and install the required packages:

python3 -m venv venv
source venv/bin/activate

pip install fastapi uvicorn streamlit requests langchain langchain-core langchain-ollama langgraph

If tutorial requires LangChain >= 1.0.0.

Step 3: Build the Agent and API Layer with FastAPI

This application has three main responsibilities. FastAPI exposes the HTTP endpoint, Pydantic validates the incoming request data, and LangChain runs the agent, including tool calling and short-term memory.

The user_id sent with each request is used as the thread identifier, allowing the checkpointer to keep each user’s conversation history separate. This memory is per session. So every new session will have its own memory.

Another important detail is that the agent is created only once at startup with agent = build_agent(). Reusing the same agent instance avoids rebuilding the model and tool list for every request, which reduces overhead and improves response times while still supporting multiple users.

Inside the /chat/stream endpoint, the backend uses LangChain’s stream_events(..., version="v3") to generate the response as a stream instead of waiting for the full answer all at once. FastAPI then wraps that stream in a StreamingResponse, so the frontend can receive the output gradually as it's produced. This makes the app feel much more interactive, because users can start reading the answer immediately while the rest is still being generated.

Put together, this gives you a lightweight backend that validates input, preserves separate memory for each user, and streams responses to the UI in real time.

Save the following code as app.py:

from datetime import datetime
from uuid import UUID

from fastapi import FastAPI, HTTPException
from fastapi.responses import StreamingResponse

from pydantic import BaseModel

from langchain.agents import create_agent
from langchain_core.tools import tool
from langchain_ollama import ChatOllama
from langgraph.checkpoint.memory import InMemorySaver

CHAT_MODEL = "qwen3.5:4b"

SYSTEM_PROMPT = (
    "You are a helpful assistant with access to tools for getting the current time "
    "and counting words in text. "
    "Use tools when needed. If the question does not need a tool, answer directly."
)

# -----------------------------
# Request model
# -----------------------------

class ChatRequest(BaseModel):
    user_id: UUID
    message: str

# -----------------------------
# Tools
# -----------------------------

@tool
def current_time() -> str:
    """Return the current local date and time."""
    return datetime.now().strftime("%Y-%m-%d %H:%M:%S")


@tool
def word_count(text: str) -> int:
    """Count the number of words in a piece of text."""
    return len(text.split())


# -----------------------------
# Agent + checkpoint memory
# -----------------------------

# Store conversation history in short term memory
checkpointer = InMemorySaver()

def build_agent():
    model = ChatOllama(model=CHAT_MODEL, temperature=0)
    return create_agent(
        model=model,
        tools=[current_time, word_count],
        system_prompt=SYSTEM_PROMPT,
        checkpointer=checkpointer,
    )


agent = build_agent()

# -----------------------------
# Streaming endpoint
# -----------------------------

app = FastAPI()

@app.post("/chat/stream")
def chat_stream(req: ChatRequest):
    def generate():
        run = agent.stream_events(
            {
                "messages": [{"role": "user", "content": req.message}],
            },
            config={
                "configurable": {
                    # Keep each user's short-term memory isolated
                    # by using their user_id as the thread ID.
                    "thread_id": str(req.user_id),
                }
            },
            version="v3",
        )

        for message in run.messages:
            for token in message.text:
                yield token

    return StreamingResponse(generate(), media_type="text/plain")

Step 4: Build Streamlit UI

The Streamlit code creates a simple chat interface for the AI agent and keeps each browser session tied to a unique user_id.

When the app first loads, it generates and stores a UUID in st.session_state, which is later sent to the backend so the agent can keep that user’s conversation history separate from other users. It also creates a chat_history list in session state so previous messages remain visible every time Streamlit reruns the script. The app then loops through that saved history and displays each message in a chat-style format using st.chat_message().

When the user enters a new message through st.chat_input(), the app immediately saves and displays it, then sends it to the backend API with a POST request to http://127.0.0.1:8001/chat/stream along with the session’s user_id.

The request is made with stream=True, which allows the response to arrive gradually instead of all at once. As each chunk of text is received from the backend, the code appends it to full_answer and updates a placeholder on the page, creating a live streaming effect. Once the response is complete, the final assistant message is stored in chat_history so it remains part of the conversation on the page

Save the below as streamlit_app.py

import uuid
import requests
import streamlit as st

API_URL = "http://127.0.0.1:8001/chat/stream"

st.title("Local AI Agent")

if "user_id" not in st.session_state:
    st.session_state.user_id = str(uuid.uuid4())

if "chat_history" not in st.session_state:
    st.session_state.chat_history = []

# Show previous messages
for item in st.session_state.chat_history:
    with st.chat_message(item["role"]):
        st.markdown(item["content"])

message = st.chat_input("Enter a message")

if message:
    # Save and show user message
    st.session_state.chat_history.append({"role": "user", "content": message})
    with st.chat_message("user"):
        st.markdown(message)

    # Stream assistant response
    full_answer = ""
    with st.chat_message("assistant"):
        placeholder = st.empty()

        # Send the reqeust to backend API via POST request
        with requests.post(
            API_URL,
            json={
                "message": message,
                "user_id": st.session_state.user_id,
            },
            stream=True,
        ) as response:
            response.raise_for_status()

            for chunk in response.iter_content(chunk_size=None, decode_unicode=True):
                if chunk:
                    full_answer += chunk
                    placeholder.markdown(full_answer)

    # Save final assistant response
    st.session_state.chat_history.append(
        {"role": "assistant", "content": full_answer}
    )

Step 5: Run the Backend App

Start the server with Uvicorn:

uvicorn app:app --reload --port 8001

Once the application starts, open:

• http://127.0.0.1:8001/

• http://127.0.0.1:8001/docs

The /docs endpoint is automatically generated by FastAPI using your Pydantic models. It provides an interactive interface where you can test the API without writing any client code.

You can send requests directly from curl. In your terminal, run these commands to invoke the API for the AI agent and check the output:

$ curl -X POST http://127.0.0.1:8001/chat/stream \
  -H "Content-Type: application/json" \
  -d '{"message":"What time is it?","user_id":"123e4567-e89b-12d3-a456-426614174000"}'

$ curl -X POST http://127.0.0.1:8001/chat/stream \
  -H "Content-Type: application/json" \
  -d '{"message":"How many words are in: LangChain makes tool calling easier","user_id":"123e4567-e89b-12d3-a456-426614174000"}'

$ curl -X POST "http://127.0.0.1:8001/chat/stream" \
-H "Content-Type: application/json" \
-d '{"message":"What is the capital of France?","user_id":"123e4567-e89b-12d3-a456-426614174000"}'

To stop the server, press Ctrl+C in the terminal.

Step 6: Run the Frontend App

In another terminal, go to the project directory:

source venv/bin/activate
streamlit run streamlit_app.py

That opens the frontend in your browser at http://localhost:8501/. Try the example prompts like "What is the capital of France". You should see the answer in a chat style interface.

The UI is calling the FastAPI endpoint and invoking the AI agent. You now have a working end to end application for your local AI agent that you can play with.

To stop the server, press Ctrl+C in the terminal.

Sample Output

The image below show two browser sessions of the app running side by side on the same endpoint. Each session is assigned a unique id, which allows the backend to maintain a separate conversation history for each user.

Even though both users ask the same question, “Who am I?”, the responses are different because each session’s answer is based on its own prior messages.

What to Improve Before Production

Although this application is fully functional, it's still intentionally minimal. It already supports a reusable FastAPI backend, a Streamlit chat interface, per-user conversation history, and streaming responses.

If you wanted to take it further, the next steps would be adding authentication, persistent storage, structured logging, monitoring, and more robust deployment setup.

It's also worth noting that if your goal is simply to get a polished self-hosted chat UI up and running quickly, you may not need to build the frontend yourself. Projects like LibreChat and Open WebUI already provide richer interfaces and broader features out of the box.

This tutorial takes a different approach: instead of adopting a full platform, it shows how to build a lightweight custom stack yourself so you can better understand the architecture and have more control over how the agent is exposed.

Conclusion

In this tutorial, we took a local AI agent, wrapped it in a FastAPI app, and used Streamlit UI on top of it.

This transforms the AI agent from a standalone script into a reusable service. Instead of only working in a terminal, it can now be accessed through a simple HTTP endpoint by other apps, scripts, or internal tools.

By assigning each session a unique id, the service can also maintain separate conversation history for multiple users, making it possible to support a chat-style interface with isolated memory per session.

From here, you can continue extending the same service by adding authentication or production-ready features. Happy tinkering!

If you enjoyed this tutorial, you can find more of my writing on my blog (recent posts include system design paper series), my work on my personal website, and updates on LinkedIn.

In this article, you’ll build a résumé screening system using pure Python, focusing on core programming concepts and the power of multiprocessing. You’ll create a custom system that automates the evaluation process by transforming unstructured résumé documents into a ranked leaderboard.

By the end of this guide, you will:

• Parse documents by extracting text from PDF and DOCX résumés using Python

• Extract information by identifying skills and keywords from résumé content

• Design a scoring algorithm using weighted logic to rank candidates objectively

• Build a web interface using Streamlit

• Deploy the application on Streamlit Cloud for public access

By following this tutorial, you’ll build a tool capable of processing hundreds of résumés in seconds.

Here’s the source code: GitHub Repository

Table of Contents

• Prerequisites

• Project Overview

• How the System Works

• System Architecture

• Project Structure

• Step 1: Set Up the Project

• Step 2: Build the Résumé Parser

• Step 3: Build the Keyword Extractor

• Step 4: Implement the Scoring Engine

• Step 5: Build the Web Interface

• Step 6: Test the System

• Step 7: Deploy the Application

• Conclusion

Prerequisites

To follow along with this tutorial, you should have:

• Basic knowledge of Python (functions, loops, dictionaries)

• Python 3.8 or higher installed

• Familiarity with installing packages using pip

• A code editor such as VS Code, PyCharm, or any editor you prefer

Project Overview

In this guide, you’ll develop a system that takes a folder of résumés and a Job Description (JD) as input. The system processes each résumé, extracts relevant information, and calculates a score based on how well the candidate matches the job requirements.

How the System Works

The project consists of four core components:

• Résumé Parser: Reads PDF and DOCX files and extracts text

• JD Parser: Analyses the job description to identify required skills

• Keyword Extractor: Matches résumé content against a skills taxonomy

• Scoring Engine: Ranks candidates using a weighted algorithm

Scoring Formula

Here’s the scoring formula we’ll use:

Total Score =
(Required Skills × 50%) +
(Preferred Skills × 25%) +
(Experience × 15%) +
(Keywords × 10%)

This approach ensures that essential skills carry more weight than secondary keywords.

How This Approach Helps Reduce Bias

This system evaluates résumés using predefined criteria instead of subjective judgment. Each résumé is scored based on the same set of required skills, preferred skills, experience indicators, and keywords.

Because all candidates are evaluated using the same weighted formula, personal factors such as writing style, formatting, or unconscious preferences don’t influence the ranking. The scoring logic focuses only on how closely a résumé matches the job requirements.

By normalising the evaluation process, the system promotes more consistent and objective screening, which helps reduce bias during the initial résumé review stage.

System Architecture

Input                    Processing                     Output
─────                    ──────────                     ──────

Résumés ──► Résumé Parser ──► Keyword Extractor ──┐
(PDF/DOCX)                                        │
                                                  ├──► Scoring Engine ──► Ranked Results
Job Description ──► JD Parser ────────────────────┘
(TXT/PDF)

The system follows a simple input–process–output flow.

Résumés and the job description are provided as inputs. The Résumé Parser extracts text from each résumé, while the JD Parser identifies required and preferred skills from the job description.

The extracted résumé text is then passed to the Keyword Extractor, which matches skills and keywords using a predefined taxonomy.

Finally, the Scoring Engine applies a weighted formula to calculate a score for each candidate and outputs a ranked list of résumés.

Project Structure

resume_screening_system/
├── app.py                    # Streamlit web interface
├── main.py                   # Command-line interface
├── parsers/
│   ├── resume_parser.py      # PDF/DOCX text extraction
│   └── jd_parser.py          # Job description parsing
├── extractors/
│   └── keyword_extractor.py  # Skills and experience extraction
├── matcher/
│   └── scorer.py             # Scoring algorithm
├── data/
│   ├── config.json           # Scoring weights
│   └── skills_taxonomy.json  # Skills database
└── requirements.txt          # Dependencies

The project is organised into clear, modular directories. Parsing logic, keyword extraction, and scoring are separated into their own folders, while configuration files and data are kept isolated. This structure keeps the codebase easy to navigate, maintain, and extend.

Step 1: Set Up the Project

Create the folder structure and set up a virtual environment:

mkdir resume_screening_system
cd resume_screening_system
mkdir parsers extractors matcher data input output
python -m venv venv

Then go ahead and activate the virtual environment:

# Windows
source venv/Scripts/activate

# macOS / Linux
source venv/bin/activate

Install the required dependencies like this:

pip install PyPDF2 python-docx streamlit pandas

Step 2: Build the Résumé Parser

The résumé parser handles different file formats by using a separate extraction method for each type.

For PDF files, the parser opens the document page by page and extracts text from each page using a PDF reader. The extracted text is combined into a single string for further processing.

For DOCX files, the parser reads each paragraph in the document and joins the paragraph text into one block. This ensures consistent text output regardless of the résumé format.

By combining all résumés into plain text, the parser allows components such as keyword extraction and scoring to work efficiently.

File: parsers/resume_parser.py

def _extract_pdf(self, file_path: Path) -> str:
    text = ""
    with open(file_path, "rb") as file:
        pdf_reader = PyPDF2.PdfReader(file)
        for page in pdf_reader.pages:
            page_text = page.extract_text()
            if page_text:
                text += page_text + "\\n"
    return text.strip()

def _extract_docx(self, file_path: Path) -> str:
    from docx import Document
    doc = Document(file_path)
    return "\\n".join(
        para.text for para in doc.paragraphs
    ).strip()

Step 3: Build the Keyword Extractor

This project uses a résumé dataset from Kaggle to ensure the logic works with real-world professional data. The keyword extractor identifies skills by scanning the résumé text.

The résumé text is first converted to lowercase so that matching is case-insensitive. A predefined skills taxonomy stores each skill along with its possible variations. The extractor checks the résumé text against these variations to find matches.

Word boundaries are used during matching to avoid partial matches, such as matching “Java” inside “JavaScript”. Matched skills are stored in a set to prevent duplicates.

This approach ensures consistent and controlled skill detection across all résumés.

File: extractors/keyword_extractor.py

def extract_skills(self, text: str) -> Set[str]:
    text_lower = text.lower()
    found_skills = set()

    for category, skills_dict in self.skills_taxonomy.items():
        for skill_name, variations in skills_dict.items():
            for variation in variations:
                # Prevent "Java" matching "JavaScript"
                pattern = r"\\b" + re.escape(variation) + r"\\b"
                if re.search(pattern, text_lower):
                    found_skills.add(skill_name)
                    break

    return found_skills

Step 4: Implement the Scoring Engine

To produce objective rankings, the system uses a weighted scoring formula.

Total Score =
(S_req × 0.50) +
(S_pref × 0.25) +
(E_exp × 0.15) +
(K_key × 0.10)

The scoring engine calculates a final score for each résumé using weighted values.

It counts how many required skills, preferred skills, experience indicators, and keywords appear in a résumé. Each count is multiplied by its assigned weight, with required skills contributing the most.

The weighted values are summed to produce a single score. Résumés are then sorted by this score to generate a ranked list of candidates.

Step 5: Build the Web Interface

Streamlit provides a simple web interface for interacting with the résumé screening system.

The text area allows users to input a job description, while the file uploader lets them upload multiple résumé files. When the button is clicked, Streamlit triggers the backend logic to parse résumés, extract data, and calculate scores.

The results are then displayed in the browser, allowing users to run the screening process without using the command line.

File: app.py

import streamlit as st

jd_text = st.text_area(
    "Paste the job description here:",
    height=300
)

uploaded_files = st.file_uploader(
    "Upload resume files:",
    type=["pdf", "docx", "txt"],
    accept_multiple_files=True
)

if st.button("Screen Resumes", type="primary"):
    st.success("Processing resumes...")

Run the application:

streamlit run app.py

The app will be available at http://localhost:8500.

Step 6: Test the System

Sample Job Description Input

Below is an example of a job description you can use to test the system:

We are looking for a Senior Python Developer with strong experience in backend development.

Required Skills:
- Python
- Django
- REST APIs
- SQL

Preferred Skills:
- PostgreSQL
- Docker
- AWS

Experience:
- 3+ years of professional Python development
- Experience building web applications

This input helps the system identify required skills, preferred skills, and experience keywords, which are then used by the scoring engine to rank résumés.

python main.py

Sample Output

============================================================
SCREENING RESULTS
============================================================
Rank #1: Alice Johnson | Score: 85.42/100 | Matched: python, django, postgresql
Rank #2: Carol Davis   | Score: 72.50/100 | Matched: python, django

Step 7: Deploy the Application

To make the system publicly accessible:

1. Push the code to GitHub

2. Go to share.streamlit.io

3. Select your app.py file

4. Deploy the application

Your app will be live at:

<https://your-app-name.streamlit.app>

Conclusion

In this tutorial, you’ve built a complete résumé screening system from scratch using Python. By combining text processing, structured scoring, and automation, this project demonstrates how manual résumé screening can be transformed into an efficient and objective workflow.

This system helps reduce bias, save time, and evaluate candidates more consistently. Happy coding!