Building Text Generation Applications with LLM APIs

Who this is for and what you will get

This is for readers who want to put a large language model into their own app or script but do not know where to start. After reading it, you can write a Python program with the openai library that generates text, turns user input into part of the prompt, and adjusts the length and randomness of the output.

What a text generation app is

Most apps have a command-line form, where you type fixed commands, or a graphical form, where you click buttons and fill in forms. A command-line app only runs the commands it supports, and apps are often built for one specific language by default, even when more languages can be added.^[1] A text generation app works differently. Users interact with it in natural language, without a fixed set of commands or a single input language. It is connected to a model trained on a vast corpus of information, while a traditional app may only reach what is stored in its database.^[1]

Typical examples include a chatbot that answers questions about a company and its products, a helper that summarizes text or produces text such as resumes, and a code assistant.^[1]

Preparation: account, Python and an API key

There are two common ways to connect to an LLM. You can call the API directly, building web requests that carry the prompt and return generated text. Or you can use a library that wraps those calls. This guide uses the openai Python library.^[1]

For Azure OpenAI, you need an Azure account, access to Azure OpenAI (at the time of writing, you must apply for access), Python installed, and an Azure OpenAI Service resource.^[1][2] Copy the value of "Key 1" from the "Keys and Endpoint" section of the resource. That is your API key.^[1]

Keep the key out of your code. Store it in an environment variable, such as OPENAI_API_KEY:^[1]

export OPENAI_API_KEY='sk-...'

Install and configure

Install the library with pip:^[1]

pip install openai

Azure OpenAI is now part of Microsoft Foundry. For it, use the standard OpenAI client pointed at the Azure OpenAI /openai/v1/ endpoint. This works with the Responses API and needs no separate api_version management.^[1]

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    base_url=f"{os.environ['AZURE_OPENAI_ENDPOINT'].rstrip('/')}/openai/v1/",
)

Here, api_key is the key found in the Azure Portal or the Microsoft Foundry portal, and base_url is your Foundry resource endpoint with /openai/v1/ appended.^[1] Without Azure, leave out base_url and pass a model name such as gpt-5-mini in place of a deployment name.^[1]

Your first program

Text is generated with the responses.create method of the Responses API, and the result is read from response.output_text.^[1] This code asks the model to continue a sentence:

prompt = "Complete the following: Once upon a time there was a"

response = client.responses.create(
    model="gpt-5-mini",  # this is your model deployment name
    input=prompt,
    store=False,
)
print(response.output_text)

The exercise saves the program as app.py, with placeholders for the key, the endpoint and the deployment name:^[1]

import os
from openai import OpenAI

client = OpenAI(
    api_key="<replace this value with your Azure OpenAI key>",
    base_url="<endpoint found in Azure Portal>/openai/v1/",
)
deployment_name = "<deployment name>"

# add your completion code
prompt = "Complete the following: Once upon a time there was a"

# make a request using the Responses API
response = client.responses.create(model=deployment_name, input=prompt, store=False)

# print response
print(response.output_text)

For multi-turn conversations, the Responses API accepts a list of messages in input. This is covered in a later chapter.^[1]

from openai import OpenAI

client = OpenAI(api_key="sk-...")

response = client.responses.create(model="gpt-5-mini", input="Hello world", store=False)
print(response.output_text)

From one prompt to a recipe generator

Prompts can produce many kinds of output: a type of text such as a poem or quiz questions, a lookup of information, or code.^[1] A recipe example shows how a prompt becomes a small tool. The first step asks the model for five recipes using chicken, potatoes and carrots, with all the ingredients listed for each recipe.^[1]

prompt = "Show me 5 recipes for a dish with the following ingredients: chicken, potatoes, and carrots. Per recipe, list all the ingredients used"

The same program can return different results on different runs, because the LLM is nondeterministic.^[1]

To make the inputs adjustable, the program reads the number of recipes and the ingredients with input() and fills them into the prompt:^[1]

no_recipes = input("No of recipes (for example, 5): ")

ingredients = input("List of ingredients (for example, chicken, potatoes, and carrots): ")

# interpolate the number of recipes into the prompt an ingredients
prompt = f"Show me {no_recipes} recipes for a dish with the following ingredients: {ingredients}. Per recipe, list all the ingredients used"

A filter comes next. The program reads a filter value such as vegetarian, vegan or gluten-free, and appends it to the prompt as "no" followed by the filter:^[1]

filter = input("Filter (for example, vegetarian, vegan, or gluten-free): ")

prompt = f"Show me {no_recipes} recipes for a dish with the following ingredients: {ingredients}. Per recipe, list all the ingredients used, no {filter}"

A filter has to be specific. In the example, a filter of "no milk" still returned a recipe with shredded Parmesan cheese. A lactose-intolerant user may also need to exclude cheese.^[1]

The shopping list is handled in a second prompt. The first result is passed in as context, along with an instruction to leave out ingredients the user already has.^[1]

old_prompt_result = response.output_text
prompt = "Produce a shopping list for the generated recipes and please don't include ingredients that I already have."

new_prompt = f"{old_prompt_result} {prompt}"
response = client.responses.create(model=deployment_name, input=new_prompt, max_output_tokens=1200, store=False)

# print response
print("Shopping list:")
print(response.output_text)

Adjust length, randomness and secrets

Keep secrets out of code. It is recommended to load variables from a .env file with python-dotenv:^[1]

import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

Control output length. The max_output_tokens parameter limits how many tokens the model generates. The example sets it to 100:^[1]

response = client.responses.create(model=deployment, input=prompt, max_output_tokens=100, store=False)

Tokens cost money, so it is best to be economical with them, for example by phrasing prompts so they need fewer tokens.^[1]

Control randomness. The higher the temperature, the more random the output; the lower it is, the more predictable the output. The example uses 0.5, and values closer to 1.0 give more varied output.^[1]

response = client.responses.create(model=deployment, input=prompt, temperature=0.5, store=False)

Reasoning models do not accept temperature. Reasoning models, such as the GPT-5 family and the o-series, do not support temperature or top_p, and max_tokens is replaced by max_output_tokens. Sending temperature to gpt-5-mini produces a "parameter not supported" error.^[1] For these models, output is steered with prompt engineering and reasoning controls, which trade depth of reasoning against latency and cost.^[1]

To experiment with temperature, switch to a model that still supports sampling controls. The Microsoft Foundry model catalog includes the open Llama model Llama-3.3-70B-Instruct. Temperature and top_p also remain valid on models such as Mistral and Phi.^[1]

Limits and common questions

• Results are not stable: the same program can give different output on each run, so results need checking.^[1]
• Filters can be incomplete: a vague condition such as "no milk" does not exclude every dish with dairy.^[1]
• Cost: tokens cost money, and both the prompt and the output length affect the bill.^[1]

Practice ideas

Three assignments are suggested: improve the recipe generator with different temperatures and prompts, build a study buddy that answers questions about a Python topic, or build a history bot that plays a historical figure and answers questions about that person’s life.^[1] The challenge is to compare temperatures of 0, 0.5 and 1. Zero is the least varied and 1 is the most varied.^[1]

Next steps

For the Azure OpenAI access request and resource setup, read the Microsoft Learn overview.^[2]

This article is adapted from lesson 6, “Building Text Generation Applications,” of Microsoft’s open-source course *Generative AI for Beginners*^[1], released under the MIT License.