Connecting an LLM to MCP tools

Who this is for and what you will get

You can already write an MCP client that lists a server's tools and calls them by hand. The next step is to let users ask a question in plain language and have a model decide which tool to call. By the end you should be able to convert a server's tools into a format the model accepts and build the full flow from a user question to a tool call. ^[1]

Why add an LLM to the client

The starting point is that making users call MCP tools explicitly is impractical. Users expect to communicate in natural language and do not care whether the capabilities are stored in MCP. The solution is to add a large language model to the client. ^[1] This guide has three goals: create a client with an LLM, interact with an MCP server through the model, and give end users a better experience. ^[1]

The overall flow

There are four steps. Connect to the server. List its capabilities, meaning prompts, resources and tools, and save their schemas. Add an LLM and pass the saved capabilities and schemas to it in a format it understands. Then handle a user prompt by passing it to the LLM together with the tools the client listed. ^[1]

Setup: a model service and environment variables

The exercise uses Microsoft Foundry. Create a Foundry resource, deploy an active model such as gpt-5.1, and set three environment variables: AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY and AZURE_OPENAI_DEPLOYMENT. ^[1]

Two points matter here. GitHub Models was retired on July 30, 2026, so it can no longer be used. The deployment name used in API calls may differ from the underlying model name. Before choosing a model, check the Microsoft Foundry model retirement schedule. ^[1]

Step 1: connect to the server

In the TypeScript version, the constructor creates the OpenAI client with a baseURL made from the endpoint plus /openai/v1/, and throws an error if AZURE_OPENAI_ENDPOINT or AZURE_OPENAI_API_KEY is missing. The same class holds an MCP Client named example-client. ^[1]

The Python version imports the same MCP pieces as the earlier client, and starts the server with mcp run server.py. ^[1]

Step 2: list capabilities and keep their schemas

The Python version lists resources and tools, and prints each tool's name and the properties of its inputSchema. The schemas have to be kept so they can be handed to the model:

# List available resources
resources = await session.list_resources()
print("LISTING RESOURCES")
for resource in resources:
    print("Resource: ", resource)

# List available tools
tools = await session.list_tools()
print("LISTING TOOLS")
for tool in tools.tools:
    print("Tool: ", tool.name)
    print("Tool", tool.inputSchema["properties"])

The TypeScript version calls this.client.listTools() and passes each tool's inputSchema as input_schema to the conversion function. ^[1]

Step 3: convert the tools for the model

The model needs OpenAI-style function tool definitions. The TypeScript openAiToolAdapter returns an object whose type is function, with parameters built from the tool's input_schema:

    return {
        type: "function" as const, // Explicitly set type to "function"
        function: {
        name: tool.name,
        description: tool.description,
        parameters: {
        type: "object",
        properties: tool.input_schema.properties,
        required: tool.input_schema.required,
        },
        },
    };

The Python version does the same with convert_to_llm_tool:

def convert_to_llm_tool(tool):
    tool_schema = {
        "type": "function",
        "function": {
            "name": tool.name,
            "description": tool.description,
            "type": "function",
            "parameters": {
                "type": "object",
                "properties": tool.inputSchema["properties"]
            }
        }
    }

    return tool_schema

After conversion, the Python version collects the tools into a functions list:

functions = []
for tool in tools.tools:
    print("Tool: ", tool.name)
    print("Tool", tool.inputSchema["properties"])
    functions.append(convert_to_llm_tool(tool))

Step 4: handle the user's prompt

The TypeScript run method sends the user message What is the sum of 2 and 3? to the model. The model name comes from AZURE_OPENAI_DEPLOYMENT and defaults to gpt-5.1, max_completion_tokens is 1000, and the converted tools go in tools:

        let response = this.openai.chat.completions.create({
            model: process.env.AZURE_OPENAI_DEPLOYMENT ?? "gpt-5.1",
            max_completion_tokens: 1000,
            messages,
            tools: tools,
        });

The model's tool calls are handled by callTools, which reads each call's name and arguments and runs the tool through the MCP client:

        // 2. Call the server's tool 
        const toolResult = await this.client.callTool({
            name: toolName,
            arguments: JSON.parse(args),
        });

In Python, the model call is wrapped in call_llm, which creates an OpenAI client from the endpoint and key and passes the converted functions as tools:

def call_llm(prompt, functions):
    client = OpenAI(
        base_url=(
            f"{os.environ['AZURE_OPENAI_ENDPOINT'].rstrip('/')}/openai/v1/"
        ),
        api_key=os.environ["AZURE_OPENAI_API_KEY"],
    )

    print("CALLING LLM")
    response = client.chat.completions.create(
        messages=[
            {
            "role": "system",
            "content": "You are a helpful assistant.",
            },
            {
            "role": "user",
            "content": prompt,
            },
        ],
        model=os.getenv("AZURE_OPENAI_DEPLOYMENT", "gpt-5.1"),
        tools=functions,
        max_completion_tokens=1000,
    )

The rest of the function reads the model's tool calls, parses the arguments with json.loads, and returns a list of name and args pairs:

    response_message = response.choices[0].message

    functions_to_call = []

    if response_message.tool_calls:
        for tool_call in response_message.tool_calls:
            print("TOOL: ", tool_call)
            name = tool_call.function.name
            args = json.loads(tool_call.function.arguments)
            functions_to_call.append({ "name": name, "args": args })

    return functions_to_call

The main code sends one prompt, then calls each suggested function:

prompt = "Add 2 to 20"

# ask LLM what tools to all, if any
functions_to_call = call_llm(prompt, functions)

# call suggested functions
for f in functions_to_call:
    result = await session.call_tool(f["name"], arguments=f["args"])
    print("TOOLS result: ", result.content)

Each suggested function is run with session.call_tool, and its result is printed. ^[1]

Common problems and limits

• The tool result is not returned to the model. The TypeScript snippet leaves the handling of the result as a TODO comment, and the Python snippet ends after printing. To get a final natural-language answer, the result must be passed back to the model as a message, and the snippets do not show that step. ^[1]
• Tool arguments arrive as strings. The TypeScript code parses them with JSON.parse and the Python code with json.loads before passing them to a tool. ^[1]
• Mind the SDK and transport versions. The Java examples use the legacy HTTP+SSE transport and target 2025-11-25 SDK APIs. For new remote clients, a 2026-07-28-compatible SDK and Streamable HTTP are recommended. ^[1]

Other languages

The .NET version takes the properties of each MCP tool's JSON schema and builds an OpenAI tool with ChatTool.CreateFunctionTool. It then loops over the model's ToolCalls and passes each name and set of arguments to mcpClient.CallToolAsync. ^[1]

For Rust, the example uses the async-openai crate, which is maintained by the community, because there is no official OpenAI library for Rust. ^[1]

The Java version uses LangChain4j. McpToolProvider discovers the MCP client's tools, and AiServices binds the chat model to them, so schema conversion and function calling are handled by the framework. The example's model service is MiniMax, with MiniMax-M3.1-Flash as the default model and MiniMax-M3 as the other current model. MINIMAX_REGION accepts global_en or cn_zh, and cn_zh maps to https://api.minimaxi.com/v1. To set the key, base URL and model explicitly:

export OPENAI_API_KEY=your_minimax_api_key_here
export OPENAI_BASE_URL=https://api.minimax.io/v1
export MINIMAX_MODEL_ID=MiniMax-M3.1-Flash

To choose the region instead, leave OPENAI_BASE_URL unset and set MINIMAX_REGION. ^[1]

Next steps

The client here turns natural language into tool calls. The natural next steps are to send tool results back to the model for a final answer, and to turn the Python main code into an interactive loop. The Java example also shows a Bot interface for handling several natural-language requests in turn. ^[1]

Adapted from lesson 3 of the introductory section of Microsoft's open-source course *mcp-for-beginners*, "Creating a client with LLM"^[1], released under the MIT License.