Chirag PatilAphrodite / Field Notes
Menu

By Lordpatil · · 5 min read

How to Build and Deploy a LangGraph Chatbot with FastAPI - Complete Guide (Part 1)

A comprehensive guide showing how to create a stateless chatbot using LangGraph and deploy it as a production-ready API using FastAPI

AI agentslanggraphfastapideploymentchatbotai

How to Build and Deploy a LangGraph Chatbot with FastAPI: Complete Guide (Part 1)

Building conversational AI applications has become increasingly accessible with modern frameworks like LangGraph. This comprehensive guide shows you how to create a stateless chatbot using LangGraph and deploy it as a production-ready API using FastAPI. Whether you're looking to deploy LangGraph applications or integrate chatbots into your existing systems, this tutorial provides a complete solution.

Note: This is Part 1 of a series on LangGraph tutorials focused on deployment with FastAPI. Future parts will cover adding memory, tools, and advanced agent capabilities.

How This Code Was Created

This implementation was one-shotted using Cursor with the following prompt:


write a fastapi code in main.py file that uses code from chatbotdocs to create a simple chatbot using langgraph and exposes api endpoints of input from user and output from chatbot.

GOOGLE_API_KEY is already present in the .env file at the root so load it using load_dotenv()

use context7 for fastapi docs. only refer fastapi docs for implementing fastapi related functionality and best practices are noted down in the docs.

the code should be simple and easy to understand, do not add the any testing functionality just yet.

if you are planning to add anything else apart from what is mentioned above first take permission from me.

Alternative Approaches:

  • If you're not using Cursor, you can use VS Code with access to Context7 MCP
  • Alternatively, pass the LangGraph documentation directly to any LLM and reproduce similar results

⚠️ Important Security Warning: Context7 MCP

While Context7 MCP is a powerful tool for accessing documentation, be extremely cautious as it is community-managed with inherent security risks:

The Problem: Context7 is community-managed, meaning anyone can add documentation. Suppose someone adds docs for "Next.ja" (a common typo for Next.js) containing prompt-injected commands designed to transfer user data via command line. Since Cursor and other IDE agents have system access, and many developers use "YOLO mode," this creates a significant security vulnerability.

Key Security Principles:

  • Do not trust anything from the internet that will be added into your prompt
  • Treat your prompts as your password
  • This is still the early era of chat-based interfaces, but soon you'll be using agents for everything with tool access
  • Trust only MCPs that are open source and backed by corporations (e.g., Neon for secure database access)

If infiltration occurs through Context7 and you also have Neon MCP installed... well, you get the point.

Recommendation: Use Context7 MCP cautiously and verify the source of any documentation before incorporating it into your workflow.

Table of Contents

Prerequisites

Before starting this tutorial, ensure you have:

  • Python 3.8 or higher installed
  • Access to a language model API (OpenAI, Anthropic, or Google Gemini)
  • Basic understanding of Python and REST APIs
  • Familiarity with environment variables for API keys

Required Dependencies

Install the necessary packages:

pip install langgraph langsmith fastapi uvicorn python-dotenv

For this tutorial, we'll use Google Gemini, so also install:

pip install langchain[google-genai]

Understanding LangGraph Fundamentals

LangGraph is a framework for building stateful, multi-agent applications with language models. At its core, LangGraph uses a state machine approach where:

  • State: Defines the data structure that flows through your application
  • Nodes: Represent units of work (functions that process the state)
  • Edges: Define how the application transitions between nodes
  • Graph: The compiled workflow that orchestrates everything

Key Concepts for Chatbot Development

  1. StateGraph: The main class for defining your workflow
  2. State Management: How data persists and updates throughout the conversation
  3. Message Handling: LangGraph's built-in message management system
  4. Graph Compilation: Converting your workflow into an executable format

For detailed concepts, refer to the official LangGraph documentation.

Building the Chatbot Logic

Step 1: Define the State

LangGraph applications start with defining the state structure. For a chatbot, we need to manage messages:

from typing import Annotated
from typing_extensions import TypedDict
from langgraph.graph.message import add_messages

class State(TypedDict):
    messages: Annotated[list, add_messages]

The add_messages annotation ensures new messages are appended to the list rather than overwriting existing ones.

Step 2: Initialize the Language Model

Set up your chosen language model. This example uses Google Gemini:

from langchain.chat_models import init_chat_model

llm = init_chat_model("google_genai:gemini-2.0-flash")

Step 3: Create the Chatbot Node

Nodes in LangGraph are Python functions that receive the current state and return updates:

def chatbot(state: State):
    return {"messages": [llm.invoke(state["messages"])]}

This function takes the conversation history and generates a response from the LLM.

Step 4: Build and Compile the Graph

from langgraph.graph import StateGraph, START, END

graph_builder = StateGraph(State)
graph_builder.add_node("chatbot", chatbot)
graph_builder.add_edge(START, "chatbot")
graph_builder.add_edge("chatbot", END)

graph = graph_builder.compile()

This creates a simple linear flow: START → chatbot → END.

Creating the FastAPI Wrapper

FastAPI provides an excellent framework for creating production-ready APIs. Here's how to wrap your LangGraph chatbot:

Step 1: Define Data Models

Create Pydantic models for request/response handling:

from pydantic import BaseModel

class ChatRequest(BaseModel):
    message: str

class ChatResponse(BaseModel):
    response: str

class HealthResponse(BaseModel):
    status: str
    message: str

Step 2: Initialize FastAPI Application

from fastapi import FastAPI

app = FastAPI(
    title="LangGraph Chatbot API",
    description="A simple chatbot API using LangGraph and Google GenAI",
    version="1.0.0"
)

Step 3: Create API Endpoints

@app.get("/", response_model=HealthResponse)
async def read_root():
    """Health check endpoint"""
    return HealthResponse(
        status="healthy",
        message="LangGraph Chatbot API is running!"
    )

@app.post("/chat", response_model=ChatResponse)
async def chat_with_bot(request: ChatRequest):
    """Send a message to the chatbot and get a response"""
    try:
        input_data = {"messages": [{"role": "user", "content": request.message}]}
        result = graph.invoke(input_data)
        bot_response = result["messages"][-1].content
        return ChatResponse(response=bot_response)
    except Exception as e:
        return ChatResponse(response=f"Error: {str(e)}")

Complete Implementation

Here's the complete implementation combining LangGraph and FastAPI:

import os
from typing import Annotated
from dotenv import load_dotenv

from fastapi import FastAPI
from pydantic import BaseModel
from typing_extensions import TypedDict

from langchain.chat_models import init_chat_model
from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messages

# Load environment variables
load_dotenv()

# Pydantic models for API
class ChatRequest(BaseModel):
    message: str

class ChatResponse(BaseModel):
    response: str

class HealthResponse(BaseModel):
    status: str
    message: str

# LangGraph State definition
class State(TypedDict):
    messages: Annotated[list, add_messages]

# Initialize FastAPI app
app = FastAPI(
    title="LangGraph Chatbot API",
    description="A simple chatbot API using LangGraph and Google GenAI",
    version="1.0.0"
)

# Initialize the LLM
llm = init_chat_model("google_genai:gemini-2.0-flash")

# Create the chatbot node function
def chatbot(state: State):
    return {"messages": [llm.invoke(state["messages"])]}

# Build the graph
graph_builder = StateGraph(State)
graph_builder.add_node("chatbot", chatbot)
graph_builder.add_edge(START, "chatbot")
graph_builder.add_edge("chatbot", END)

# Compile the graph
graph = graph_builder.compile()

# API Endpoints
@app.get("/", response_model=HealthResponse)
async def read_root():
    """Health check endpoint"""
    return HealthResponse(
        status="healthy",
        message="LangGraph Chatbot API is running!"
    )

@app.post("/chat", response_model=ChatResponse)
async def chat_with_bot(request: ChatRequest):
    """Send a message to the chatbot and get a response"""
    try:
        input_data = {"messages": [{"role": "user", "content": request.message}]}
        result = graph.invoke(input_data)
        bot_response = result["messages"][-1].content
        return ChatResponse(response=bot_response)
    except Exception as e:
        return ChatResponse(response=f"Error: {str(e)}")

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8000)

Environment Configuration

Create a .env file for your API keys:

GOOGLE_API_KEY=your_google_api_key_here

Testing Your Deployment

Running Locally

Start your FastAPI server:

python main.py

or using uvicorn directly:

uvicorn main:app --host 0.0.0.0 --port 8000 --reload

Testing the API

  1. Health Check: Visit http://localhost:8000 or http://localhost:8000/docs
  2. Chat Endpoint: Send POST requests to http://localhost:8000/chat

Example using curl:

curl -X POST "http://localhost:8000/chat" \
     -H "Content-Type: application/json" \
     -d '{"message": "Hello, how are you?"}'

Interactive API Documentation

FastAPI automatically generates interactive documentation at:

  • Swagger UI: http://localhost:8000/docs
  • ReDoc: http://localhost:8000/redoc

Production Deployment Considerations

1. Environment Variables

Ensure all sensitive information (API keys) are stored as environment variables, not hardcoded.

2. Error Handling

Implement comprehensive error handling for:

  • API rate limits
  • Network timeouts
  • Invalid inputs
  • LLM service failures

3. Logging and Monitoring

Add logging for debugging and monitoring:

import logging

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

@app.post("/chat", response_model=ChatResponse)
async def chat_with_bot(request: ChatRequest):
    logger.info(f"Received message: {request.message}")
    # ... rest of the function

4. Rate Limiting

Consider implementing rate limiting to prevent abuse:

pip install slowapi

5. CORS Configuration

For web applications, configure CORS:

from fastapi.middleware.cors import CORSMiddleware

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],  # Configure appropriately for production
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

6. Deployment Platforms

Deploy your FastAPI application to:

  • Docker: Containerize your application
  • Railway, Render, or Heroku: Simple deployment platforms
  • AWS, GCP, or Azure: Cloud platforms for scalable deployment

Conclusion

This guide demonstrated how to build and deploy a LangGraph chatbot using FastAPI, creating a production-ready API that can handle conversational AI workloads. The combination of LangGraph's powerful state management with FastAPI's robust web framework provides an excellent foundation for deploying AI applications.

Key Benefits of This Approach:

  • Scalable: FastAPI's async support handles concurrent requests efficiently
  • Maintainable: Clear separation between chatbot logic and API layer
  • Extensible: Easy to add features like memory, tools, or authentication
  • Production-Ready: Built-in documentation, validation, and error handling

Next Steps:

  • Add conversation memory for stateful interactions
  • Implement user authentication and authorization
  • Add tools and external API integrations
  • Scale with load balancers and multiple instances

For more advanced LangGraph features, explore the official tutorials including adding memory, tools, and human-in-the-loop workflows.

Additional Resources:

Authored by Chirag Patil

Keep following the thread