Building a chat application

Chat is where most users first meet generative AI, and building a chat application is mostly a discipline of knowing what to reuse, what to customize, and what to watch after launch. This page walks through the distinction that shapes every design decision, the first API call, the UX issues AI introduces, domain customization, and the metrics and responsible-AI principles that keep a chat app trustworthy.

Chatbot or chat application?

The two terms are not interchangeable. A chatbot automates specific conversational tasks — answering FAQs, tracking a package — and is typically governed by rule-based logic or complex AI algorithms ^[1]. An AI-powered chat application is a much more expansive environment for text, voice, and video communication; its defining feature is an integrated generative AI model that simulates nuanced, human-like conversations, adapts to evolving contexts, and can produce creative or complex dialogue ^[1].

The practical differences follow from that: a chatbot is task-focused and rule-based, often integrated into larger systems, and limited to programmed functions; a generative AI chat application is context-aware, may host one or multiple chatbots, and handles open-domain discussions ^[1]. Decide which one you are building first — it determines your architecture, your team's skills, and your failure modes.

Start with what exists

A good first step is assessing what is already out there. Building on well-documented SDKs and APIs pays off four ways: development is faster and overhead lower, since you focus on business logic instead of plumbing; performance is better, because maintained SDKs and APIs have built-in answers to scaling questions like a sudden influx of users; maintenance is easier, since newer versions usually require just a library update; and you get access to cutting-edge technology — models fine-tuned on extensive datasets ^[1].

Access typically starts with a unique key or authentication token. In Python, a first call needs only a few lines ^[1]:

import os
from openai import OpenAI

API_KEY = os.getenv("OPENAI_API_KEY","")

client = OpenAI(
    api_key=API_KEY
    )

response = client.responses.create(model="gpt-5-mini", input="Suggest two titles for an instructional lesson on chat applications for generative AI.", store=False)
print(response.output_text)

This uses the GPT-5 mini model with the Responses API; the key must be set beforehand or the call errors out ^[1].

UX issues AI introduces

General UX principles still apply, but machine-learning components add three considerations ^[1]:

1. Ambiguity. Generative models occasionally produce ambiguous answers; give users a way to ask for clarification.
2. Context retention. Models can remember context within a conversation, which users value — but letting them control and manage that context risks retaining sensitive information. A retention policy balances context against privacy.
3. Personalization. User profiles and similar features make interactions more efficient and satisfying. ChatGPT's "Custom instructions" settings are the canonical example: a profile with professional background produces a more in-depth lesson plan on linked lists without changing the prompt ^[1].

The system message is the main lever for shaping behavior. Microsoft's framework for effective system messages covers four areas: who the model is for, with its capabilities and limitations; the output format; specific examples demonstrating intended behavior; and additional behavioral guardrails ^[1].

Accessibility deserves explicit attention rather than an afterthought: high contrast themes, resizable text, and screen reader compatibility for visual impairment; text-to-speech, speech-to-text, and visual cues for auditory impairment; keyboard navigation and voice commands for motor impairment; simplified language options for cognitive impairment ^[1].

Customizing for a domain

Two approaches bring a chat application into a specialized field: leveraging DSL (domain-specific language) models trained on a particular domain, and fine-tuning — further training a model with specific data ^[1]. DSL options range from training one from scratch, to using pre-existing models through SDKs and APIs, to fine-tuning an existing pre-trained model ^[1].

Fine-tuning earns its cost when a pre-trained model falls short in a specialized domain. The medical case is illustrative: a chat application giving practitioners quick references to treatment guidelines, drug interactions, and recent research will struggle with highly specific cases — think best practices for managing drug-resistant epilepsy in pediatric patients — and with incorporating the newest neurology and pharmacology findings ^[1]. Fine-tuning on a specialized medical dataset improves this, but it requires access to a large and relevant dataset ^[1].

What to monitor after launch

Quality has to be measured. The core metrics: uptime, response time, precision (true positives over all positive predictions), recall or sensitivity (true positives over actual positives), and the F1 score that balances the two; perplexity, measuring how well the model's predicted probability distribution aligns with the actual data distribution; user satisfaction from surveys; error rate; retraining cycles; and anomaly detection ^[1]. Each metric comes with an operational question — how you minimize downtime, what triggers a retraining cycle, how you respond to anomalies — that is worth answering in writing before launch ^[1].

Responsible AI belongs in the same planning loop. Microsoft's framework names six principles: fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability — with concrete developer actions from implementing fail-safes and encryption to providing clear reasoning for AI responses and establishing a process for auditing decisions ^[1].

Adapted from lesson 7 of Microsoft's open-source course *Generative AI for Beginners*^[1], released under the MIT License.