How to Build an AI Chatbot

An AI chatbot is a web application that allows users to communicate with a language model through a conversational interface. The basic implementation can be surprisingly small: a text input, a backend endpoint, an AI API request, and a component that displays the response.

A production chatbot requires considerably more. It needs conversation history, authentication, secure API credentials, input validation, streaming, rate limiting, error handling, context management, usage tracking, and safeguards against misuse.

This guide explains how to build an AI chatbot step by step, starting with the basic architecture and gradually adding the components required for a reliable application.

How an AI Chatbot Works

A typical chatbot does not run the language model inside the browser. Instead, the browser sends a user's message to the application's backend. The backend prepares the request, sends it to an AI provider, receives the generated response, and returns it to the browser.

User
  ↓
Chat interface
  ↓ Message
Backend
  ↓ Conversation + prompt
AI provider
  ↓ Generated response
Backend
  ↓
Chat interface

The backend acts as the main security and application-logic boundary. It can protect the AI API key, authenticate users, enforce quotas, validate messages, and control which model and features are available.

Core Components

A basic AI chatbot can be divided into several components:

Not every chatbot needs every component immediately. A prototype can start with a frontend, backend, and AI API, while persistence, authentication, and usage management can be added as the application grows.

Choose a Model and AI API

The chatbot needs access to a language model. Most web applications use a hosted AI API because it avoids the infrastructure required to run a large model themselves.

When selecting a model, consider the chatbot's requirements rather than simply choosing the largest available model.

• Response quality.
• Input and output context limits.
• Response latency.
• API cost.
• Supported capabilities.
• Structured output support.
• Tool or function calling support.
• Availability and rate limits.

A simple customer-support chatbot may have very different requirements from a coding assistant or an agent that performs actions through external tools.

Create the Chat Interface

The frontend needs a message list, text input, submit action, loading state, and error handling. A minimal React component can maintain the current conversation in state.

type Message = {
  role: "user" | "assistant";
  content: string;
};

const [messages, setMessages] = useState<Message[]>([]);
const [input, setInput] = useState("");

The UI should clearly distinguish user messages from assistant messages and preserve the conversation order.

Send Messages to the Backend

When the user submits a message, the frontend can send it to a server-side endpoint. The request can contain the new message and, depending on the architecture, either the existing conversation or an identifier for a stored conversation.

const response = await fetch("/api/chat", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    message: input,
  }),
});

const data = await response.json();

For a prototype, sending the conversation from the browser can be acceptable. In a production application, storing conversations on the server usually provides better control over authorization, context size, and persistence.

Build the Backend Endpoint

The backend receives the request and prepares the AI API call. A simplified endpoint might look like this:

export async function POST(request: Request) {
  const body = await request.json();
  const message = body.message;

  if (typeof message !== "string" || !message.trim()) {
    return Response.json(
      { error: "Invalid message" },
      { status: 400 }
    );
  }

  // Call the AI provider here.

  return Response.json({
    message: "AI response",
  });
}

The actual provider SDK or HTTP request depends on the service you choose. The surrounding architecture remains largely the same.

Keep the AI API Key Private

The AI provider's private API key should remain on the server. Never put the provider key into client-side JavaScript, browser storage, or a public repository.

const apiKey = process.env.AI_API_KEY;

if (!apiKey) {
  throw new Error("AI_API_KEY is not configured");
}

Environment variables or a production secret manager can be used to provide the credential to the backend.

Design the Conversation Format

Chat models generally work with a sequence of messages or an equivalent conversation representation. Each message typically has a role and content.

[
  {
    "role": "user",
    "content": "What is TypeScript?"
  },
  {
    "role": "assistant",
    "content": "TypeScript is a typed superset of JavaScript..."
  },
  {
    "role": "user",
    "content": "Why should I use it?"
  }
]

The exact message schema varies between APIs. Some providers also support system or developer instructions, tool calls, multimodal content, and other message types.

Add System Instructions

A chatbot often needs behavior rules that should remain separate from ordinary user messages. These instructions can define the assistant's role, response style, constraints, or domain-specific behavior.

You are a technical support assistant.

Rules:
- Give concise and accurate answers.
- Ask for clarification when required.
- Do not invent product information.
- Use code examples when they are useful.

System or developer instructions should not be treated as a replacement for application-level security. Important authorization and business rules must remain enforced by the backend.

Maintain Conversation History

A chatbot needs conversation context if later messages should take earlier messages into account. Without previous messages, every request can appear to the model as a new conversation.

Message 1
Message 2
Message 3
    ↓
Conversation context
    ↓
AI model

For short conversations, history can be kept in frontend or backend state. For persistent chat applications, store conversations in a database and associate each conversation with the authenticated user.

Understand Context Limits

A model cannot necessarily process an unlimited conversation. The messages sent with each request consume part of the available context window.

As a conversation grows, sending every previous message can increase latency and cost and eventually exceed the model's context limit.

• Limit the maximum conversation size.
• Remove irrelevant older messages when appropriate.
• Summarize older conversation history.
• Retrieve only relevant historical information.
• Keep system instructions concise.
• Monitor input token usage.

Add Message Persistence

If users should be able to return to previous conversations, messages need to be persisted. A basic database model might contain users, conversations, and messages.

The exact schema depends on the product. The important relationship is that users should only be able to access conversations they are authorized to access.

Stream the AI Response

A chatbot can feel significantly faster when generated text is streamed to the browser instead of waiting for the entire response.

AI generates:
  "Hello" ──→ Browser
  "! How" ──→ Browser
  " can I" ──→ Browser
  " help?" ──→ Browser

The frontend can append incoming chunks to the current assistant message. This creates the familiar effect of the answer appearing progressively.

Handle Streaming on the Frontend

When the backend returns a streaming response, the browser can read the response body incrementally rather than waiting for a complete JSON object.

const response = await fetch("/api/chat", {
  method: "POST",
  body: JSON.stringify({ message: input }),
});

const reader = response.body?.getReader();

if (!reader) {
  throw new Error("Streaming is not supported");
}

const decoder = new TextDecoder();

while (true) {
  const { value, done } = await reader.read();

  if (done) break;

  const chunk = decoder.decode(value, { stream: true });
  // Append chunk to the assistant message.
}

Production implementations should also account for provider-specific streaming formats, partial chunks, connection interruptions, cancellation, and errors.

Add Authentication

Authentication becomes important when conversations, quotas, or paid AI features are associated with individual users. The backend should determine which account is making the request before processing it.

Browser
  ↓ Session / access token
Chat API
  ├── Verify user
  ├── Check permissions
  ├── Check quota
  ↓
AI provider

Authentication also prevents users from simply changing a conversation identifier and accessing another user's messages. Every conversation lookup should be authorized on the server.

Add Rate Limiting

Without rate limiting, a public chatbot endpoint can be abused by automated clients or a small number of users sending excessive requests.

• Limit requests per minute.
• Limit concurrent generations.
• Use stricter limits for anonymous users.
• Apply per-account quotas.
• Limit expensive models or features separately.

Control Chatbot Costs

Every chatbot request can consume input and output tokens or another billable unit. Conversation history can make input usage grow significantly because previous messages may be included in subsequent requests.

• Choose a model appropriate for the task.
• Limit maximum output length.
• Limit conversation history.
• Summarize old context when appropriate.
• Cache suitable repeated operations.
• Apply user quotas.
• Track usage and estimated cost.

Cost control should be designed into the chatbot from the beginning rather than added only after unexpected usage appears.

Validate and Sanitize Input

User messages should be treated as untrusted input. The backend should validate their size and structure before sending them to the AI provider.

• Reject empty messages.
• Limit maximum message length.
• Validate request structure.
• Restrict unsupported content types.
• Apply account and application rules.
• Avoid forwarding unnecessary metadata.

Handle Prompt Injection

A chatbot may receive messages that attempt to manipulate its instructions. This is especially important when the chatbot has access to tools, private documents, databases, or external services.

A model should not be the final authority for sensitive operations. The backend must enforce permissions independently of what the model says.

• Treat user messages as untrusted.
• Keep authorization logic in application code.
• Restrict model access to tools.
• Validate tool arguments before execution.
• Require confirmation for high-impact actions when appropriate.
• Avoid exposing sensitive internal instructions or data.

Add Function Calling and Tools

A basic chatbot only generates text. More advanced chatbots can use tools or function calling to retrieve information or perform actions.

User: "What's the weather in Paris?"
        ↓
AI model decides to call weather tool
        ↓
Backend executes tool
        ↓
Weather result
        ↓
AI model generates final answer

Tool execution should happen on the server. The model can request a tool call, but the backend should validate the requested operation, arguments, permissions, and resulting action before executing it.

Handle Errors Gracefully

A chatbot depends on multiple systems that can fail: the browser connection, your backend, authentication, the database, the AI provider, and external tools.

Avoid displaying raw server errors, API keys, stack traces, or provider-specific credentials to users.

Support Message Cancellation

Users may decide that they no longer want an answer while the model is generating it. Supporting cancellation can reduce unnecessary work and improve the interface.

const controller = new AbortController();

fetch("/api/chat", {
  method: "POST",
  signal: controller.signal,
  body: JSON.stringify({ message: input }),
});

// Cancel the request
controller.abort();

Cancellation behavior depends on the backend and AI provider. A browser-side abort does not automatically guarantee that every downstream operation has stopped, so the server should handle cancellation where supported.

Improve the Chat Experience

A technically correct chatbot can still feel poor if the interface is slow or difficult to use. Common UX improvements include:

• Automatically scroll to new messages.
• Allow Enter to send and another shortcut for a new line.
• Show a clear generation state.
• Display streamed responses progressively.
• Provide a stop-generation action.
• Allow users to retry failed messages.
• Preserve the conversation after page refreshes.
• Make code blocks and long responses easy to read.

Store Generated Messages Carefully

Persistent chat history can contain personal, confidential, or otherwise sensitive information. Store only what the application actually needs and apply access controls to every conversation.

• Authorize every conversation lookup.
• Protect stored data appropriately.
• Define retention rules.
• Avoid unnecessary logging of complete conversations.
• Delete data when the product's retention policy requires it.
• Be clear about how conversations are used.

Evaluate Chatbot Quality

A chatbot can appear to work correctly while producing unreliable answers. Testing should therefore include representative conversations rather than only checking whether the API request succeeds.

• Test common user questions.
• Test ambiguous requests.
• Test long conversations.
• Test malformed input.
• Test adversarial prompts.
• Test tool calls if supported.
• Check factual accuracy for domain-specific tasks.
• Measure latency and error rates.

When prompts, models, retrieval logic, or tool definitions change, run the same evaluation cases again to detect regressions.

A Production Chatbot Architecture

AI Provider
    ↑
AI Service
    ↑
    ├── Authentication
    ├── Rate Limiter
    ├── Database
    ↓
Chat API
    ↓
Chat UI

This architecture separates responsibilities so that authentication, storage, usage control, and AI integration can evolve independently.

Minimal Implementation vs Production Implementation

Common Mistakes

• Putting the AI API key in frontend code.
• Sending unlimited conversation history with every request.
• Allowing unauthenticated users unlimited AI usage.
• Not implementing rate limits.
• Assuming model output is always correct.
• Trusting the model to enforce authorization.
• Executing tool calls without server-side validation.
• Storing conversations without proper access controls.
• Logging sensitive conversations unnecessarily.
• Retrying every provider error indefinitely.
• Ignoring streaming failures and request cancellation.
• Failing to monitor AI usage and costs.

Recommended Development Order

Building the chatbot incrementally makes development easier. A practical order is:

• Create the chat interface.
• Create a server-side chat endpoint.
• Connect the endpoint to an AI provider.
• Keep the provider credential server-side.
• Add conversation history.
• Add streaming responses.
• Add authentication.
• Add rate limiting and usage quotas.
• Persist conversations.
• Add monitoring and cost tracking.
• Add evaluation tests.
• Add tools or function calling if the product requires them.

Starting with a small working chatbot makes it easier to isolate problems before adding persistence, authentication, tools, and other production features.

Frequently Asked Questions

Conclusion

A basic AI chatbot can be built with a chat interface, a backend endpoint, and an AI API. However, a production-ready chatbot needs much more than a successful model request.

Conversation history, context management, streaming, authentication, rate limiting, usage quotas, secure data storage, error handling, monitoring, and evaluation all contribute to a reliable implementation. If the chatbot can perform actions, server-side validation and authorization become even more important.

The best approach is to build the chatbot incrementally: start with the basic request-response flow, then add persistence, streaming, security, usage controls, and advanced capabilities as the product requires them.