Prompt engineering fundamentals

The same question can get a brilliant answer from a model or a useless one — the difference is usually the prompt. Prompt engineering is the practice of designing and refining the text you send to a model so responses stay consistent and high-quality for your objective. This page covers how models actually read your prompt, why outputs vary, how a well-built prompt is structured, and the best practices worth applying immediately.

What prompt engineering is

Prompt engineering designs and optimizes text inputs to deliver consistent, quality responses for a given application objective and model ^[1]. It is naturally a two-step process: design the initial prompt for the model and objective, then refine it iteratively until the response quality is good enough ^[1].

Generative AI creates new content — text, images, audio, code — in response to user requests, powered by large language models like OpenAI's GPT series, trained on natural language and code ^[1]. The interaction model is deliberately simple: you send a prompt, the model returns a completion, and you can keep refining in a multi-turn conversation until the result matches your expectations ^[1]. That simplicity is exactly why the prompt itself becomes the primary programming interface of a generative AI app, and why writing it well matters.

How a model reads your prompt

Three concepts explain most of the surprising model behavior.

Tokenization is how text gets converted into smaller units called tokens that the model can process. Different models — or different versions of the same model — can tokenize the same prompt differently, and since LLMs are trained on tokens rather than raw text, tokenization directly affects response quality ^[1]. Tools like the OpenAI Tokenizer let you paste a prompt and see exactly how whitespace and punctuation turn into tokens ^[1].

A base LLM, or foundation model, does one thing: predict the next token in the sequence. It does not understand the meaning of the words — it sees a statistical pattern it can complete ^[1].

An instruction-tuned LLM starts from that foundation model and is fine-tuned with input/output pairs containing clear instructions, using techniques like Reinforcement Learning with Human Feedback (RLHF) so it follows instructions and learns from feedback ^[1]. The difference is tangible: give a summarization task the system message "Summarize content you are provided with for a second-grade student. Keep the result to one paragraph with 3-5 bullet points." and the output comes back already shaped for the target audience ^[1].

Why prompts need engineering

Current LLMs pose three challenges that make reliable completions hard without deliberate prompt construction ^[1]:

1. Responses are stochastic. The same prompt may produce different answers across models, model versions, or even the same model at different times. Better prompts act as guardrails that minimize the variation.
2. Models fabricate. Pre-training datasets are large but finite, so outside their scope models can produce inaccurate or imaginary completions. Asking for citations or reasoning helps identify and mitigate this.
3. Capabilities vary. Newer models bring richer abilities but also new quirks and cost tradeoffs; prompt techniques abstract away the differences.

Fabrication deserves a demonstration. Ask three providers the same deliberately fictional prompt — "generate a lesson plan on the Martian War of 2076" — and GPT-35 on OpenAI, GPT-35 on Azure OpenAI, and LLama-2 on Hugging Face each produced lesson plans convincing enough to fool an uninformed reader ^[1]. Techniques like metaprompting and temperature configuration reduce fabrications somewhat, and newer prompt architectures fold in tools that mitigate them further ^[1].

The three layers of a prompt

A basic prompt is just text with no other context; the Completion API completes it like a prediction ^[1]. Real applications build prompts with the Chat Completion API as a collection of messages: user/assistant input-output pairs plus a system message that sets the assistant's behavior or personality ^[1]:

response = client.responses.create(
    model="gpt-5-mini",
    input=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Who won the world series in 2020?"},
        {"role": "assistant", "content": "The Los Angeles Dodgers won the World Series in 2020."},
        {"role": "user", "content": "Where was it played?"}
    ]
)

Instruction prompts specify the task in detail. Escalating the detail changes the output: "Write a description of the Civil War" returns a paragraph; adding "provide key dates and events" adds a dated list; asking for "3 bullet points ... Return the output as a JSON file" returns structured JSON you can validate programmatically ^[1].

The primary content pattern divides the input into an instruction and relevant content that influences the action ^[1]. Supporting content adds further context — formatting rules, topic taxonomies, parameters — that shapes the output toward the user's objective ^[1]. In a course-catalog summarization, the instruction says what to do, examples show the desired format, and secondary content identifies which tags matter most ^[1].

Three techniques worth learning first

Examples. Provide zero, one, or a few demonstrations of the desired output. Zero-shot needs an explicit instruction like "Translate to Spanish"; with one or more examples the task gets inferred, and more examples mean more accurate inference with no added instructions ^[1].

Cues. Start the completion for the model with a snippet reflecting the desired format. Ending the prompt with "What we learned is that Jupiter" produces a sentence; ending with "Top 3 Facts We Learned:" produces a numbered list ^[1].

Templates. A prompt template is a pre-defined recipe stored and reused for consistent experiences at scale; complex templates hold placeholders filled from user input, system context, or external data to generate prompts programmatically ^[1]. Domain prompt libraries like Prompts For Edu show the end state: curated templates optimized for a specific audience ^[1].

Eight best practices

Practitioners at OpenAI and Azure OpenAI recommend: evaluate the latest models; separate instructions and context with delimiters; be specific and clear; be descriptive and use examples; use cues to jumpstart completions; double down — repeat instructions before and after the primary content; watch the order, since recency bias means what you present last can weigh more; and give the model an "out", a fallback response for when it cannot do the task, which reduces fabrications ^[1]. None of these guarantees a result: prompting stays a trial-and-error process, so record what works and build your own prompt library ^[1].

Adapted from lesson 4 of Microsoft's open-source course *Generative AI for Beginners*^[1], released under the MIT License.