Getting started with function calling

Chat models return prose, but applications need structure — and data the model was never trained on. Function calling solves both: you describe functions to the model, it answers with a structured JSON call instead of free text, and your code runs the real work. This page builds a working example end to end: a chatbot that turns "find me a good Azure course for a beginner" into a real catalog API call and a natural-language answer.

What problem function calling solves

Before function calling, LLM responses were unstructured and inconsistent. Developers wrote complex validation code to handle every variation, and questions like "What is the current weather in Stockholm?" were unanswerable because models were limited to their training data ^[1].

Function Calling, a feature of the Azure OpenAI Service, addresses two limitations ^[1]:

1. Consistent response format — a controlled format is far easier to integrate downstream.
2. External data — data from other parts of your application can be used in a chat context.

One crucial clarification: with function calling, the LLM never actually calls or runs anything — it acts as a dispatcher that decides what to call and with which arguments, while execution, data access, and permission boundaries stay inside your code. You define a structure for its responses; the structured response tells your application which function to run, and what the function returns can be sent back to the LLM so it can answer the user in natural language ^[1].

Where it fits

Three patterns cover most uses ^[1]:

1. Calling external tools: a chatbot that sends email when asked, mapping the request to send_email(to: string, body: string).
2. API or database queries from natural language: "Who completed the last assignment?" becomes get_completed(student_name: string, assignment: int, current_status: string).
3. Creating structured data: turning a text block or CSV into structured output, such as get_important_facts(agreement_name: string, date_signed: string, parties_involved: list).

The first function call, in three steps

The full loop has three steps: call the Responses API with your functions and a user message; read the model's response and perform the action; call the API again with the function's result to produce the final answer ^[1].

Step one, a user message. The role can be system (rules), assistant (the model), or user (the end user); distinct roles give the model a conversation history to build on ^[1]:

messages= [ {"role": "user", "content": "Find me a good course for a beginner student to learn Azure."} ]

Step two, define functions. Each tool carries type, name, description, and parameters; the optional required array lists what a call needs ^[1]:

functions = [
   {
      "type":"function",
      "name":"search_courses",
      "description":"Retrieves courses from the search index based on the parameters provided",
      "parameters":{
         "type":"object",
         "properties":{
            "role":{
               "type":"string",
               "description":"The role of the learner (i.e. developer, data scientist, student, etc.)"
            },
            "product":{
               "type":"string",
               "description":"The product that the lesson is covering (i.e. Azure, Power BI, etc.)"
            },
            "level":{
               "type":"string",
               "description":"The level of experience the learner has prior to taking the course (i.e. beginner, intermediate, advanced)"
            }
         },

         "required":[
            "role"
         ]
      }
   }
]

Mind the token budget: functions are included in the system message and count against your available tokens ^[1].

Step three, make the call with tools=functions; tool_choice="auto" hands the decision of which function to call to the model ^[1]:

response = client.responses.create(model=deployment,
                                        input=messages,
                                        tools=functions,
                                        tool_choice="auto",
                                        store=False)

print(response.output)

The response contains a function_call item ^[1]:

{
  "type": "function_call",
  "name": "search_courses",
  "call_id": "call_abc123",
  "arguments": "{\n  \"role\": \"student\",\n  \"product\": \"Azure\",\n  \"level\": \"beginner\"\n}"
}

Look at what just happened: the model extracted student, Azure, and beginner from the user message and filled the arguments itself — function calling doubles as prompt-based information extraction with a guaranteed shape ^[1].

Wiring it into an application

First, pick the function-call items out of the response ^[1]:

response_items = response.output
tool_calls = [item for item in response_items if item.type == "function_call"]

Then write the real function — a call to the Microsoft Learn Catalog API that returns the top five matching modules ^[1]:

import requests

def search_courses(role, product, level):
  url = "https://learn.microsoft.com/api/catalog/"
  params = {
     "role": role,
     "product": product,
     "level": level
  }
  response = requests.get(url, params=params)
  modules = response.json()["modules"]
  results = []
  for module in modules[:5]:
     title = module["title"]
     url = module["url"]
     results.append({"title": title, "url": url})
  return str(results)

The dispatch loop maps the model's chosen name to your Python functions, parses the JSON arguments, and calls the function ^[1]:

# Check if the model wants to call a function
if tool_calls:
 for tool_call in tool_calls:
  print("Recommended Function call:")
  print(tool_call.name)
  print()

  # Call the function.
  function_name = tool_call.name

  available_functions = {
          "search_courses": search_courses,
  }
  function_to_call = available_functions[function_name]

  function_args = json.loads(tool_call.arguments)
  function_response = function_to_call(**function_args)

  print("Output of function call:")
  print(function_response)
  print(type(function_response))

Three lines carry the core: look up the function, parse the arguments, call it ^[1]:

function_to_call = available_functions[function_name]

function_args = json.loads(tool_call.arguments)
function_response = function_to_call(**function_args)

Finally, append the function call and its output back to the conversation — the model's function_call item must come before its output — and ask for a second response ^[1]:

print("Messages in next request:")
print(messages)
print()

second_response = client.responses.create(
   input=messages,
   model=deployment,
   tool_choice="auto",
   tools=functions,
   temperature=0,
   store=False,
      )  # get a new response from the model where it can see the function response


print(second_response.output_text)

The user now sees a natural-language list of course links instead of raw JSON ^[1]. For production, add the robustness this example omits: handle a model naming a function you never registered, argument parsing failures, and empty API results, each with an explicit fallback path.

What makes a good function definition

The description fields do the heavy lifting: the model reads them to decide when and how to call, so be specific and clear ^[1]. Give every parameter a precise type and description, and use required to enforce the arguments a call cannot do without ^[1]. And budget for context: definitions ride along in the system message and consume tokens on every call ^[1].

Adapted from lesson 11 of Microsoft's open-source course *Generative AI for Beginners*^[1], released under the MIT License.