Routing requests in MCP: content, load and tool routing

Who this is for and what you can do after reading

This is for developers connecting several models, several MCP servers or tools deployed in several regions. After reading, you can separate the three kinds of routing (by content, by load and by tool context), follow the classification and selection logic in the samples, and recognize which parts are placeholders that should not go live as they are.

What routing solves

Routing sends a request to the most suitable model or service. The criteria can include content type, user context and system load^[1]. In MCP, this shows up at three levels: picking a specialized service by content, spreading load across several nodes, and deciding which endpoint a tool call goes to based on context^[1].

Content-based routing: classify first, then forward

The idea is direct. Code-generation requests go to a code model, and creative-writing requests go to a creative model^[1]. The C# sample uses a dictionary whose keys are categories (code, creative, scientific and general) and whose values are the matching MCP clients^[1].

The classifier matches keywords. It lowercases the prompt. If the prompt contains "code", "function", "program" or "algorithm", it is treated as code. If it contains "story", "creative", "imagine" or "design", it is creative. If it contains "science", "research", "analyze" or "study", it is scientific. Anything else is general^[1]. The router method itself is short:

    public async Task<McpResponse> RouteAndProcessAsync(string prompt, IDictionary<string, object> parameters = null)
    {
        // Classify the prompt to determine the best specialized service
        string category = await _classifier.ClassifyPromptAsync(prompt);
        
        // Get the appropriate client or fall back to general
        var client = _specializedClients.ContainsKey(category) 
            ? _specializedClients[category] 
            : _specializedClients["general"];
            
        Console.WriteLine($"Routing request to {category} specialized service");
        
        // Send request to the selected service
        return await client.SendPromptAsync(prompt, parameters);
    }

Keyword classification is coarse. A prompt that contains "design" goes to the creative model even when it is about engineering design. Before production use, keywords work best as a fallback, and the main classifier should be a more reliable model or rule set. The checks also run in a fixed order, so a prompt containing both "program" and "story" matches the code category first.

Load balancing: three strategies and their details

The Java sample implements three strategies: round robin, weighted response time and content-aware^[1].

Round robin is the simplest. The sample first filters for healthy nodes, then uses an atomic counter modulo the number of healthy nodes to assign requests in turn^[1]:

    public static class RoundRobinStrategy implements LoadBalancingStrategy {
        private AtomicInteger counter = new AtomicInteger(0);
        
        @Override
        public McpServerNode selectNode(List<McpServerNode> nodes, McpRequest request) {
            List<McpServerNode> healthyNodes = nodes.stream()
                .filter(McpServerNode::isHealthy)
                .collect(Collectors.toList());
            
            if (healthyNodes.isEmpty()) {
                throw new RuntimeException("No healthy nodes available");
            }
            
            int index = counter.getAndIncrement() % healthyNodes.size();
            return healthyNodes.get(index);
        }
    }

The response-time strategy picks the healthy node with the lowest average response time^[1]. Despite the name "weighted", no weights are computed. The code takes the minimum value. Real weighted distribution would need to be implemented separately.

The content-aware strategy first classifies the request. A prompt containing "code", or allowed tools that include codeInterpreter, counts as a code request. A prompt containing "creative" or "story" counts as a creative request. A match selects a healthy node with the same specialization. Without a match, the strategy picks the healthy node with the lowest current load^[1].

Failure handling is worth reading closely. When a node throws, the balancer calls recordFailure() on it and tries one fallback node from the remaining nodes. If no nodes remain, it throws "All MCP server nodes failed to process the request"^[1]. This is one fallback attempt, not a loop that retries until success.

Health checks run through startHealthChecks. On a ScheduledExecutorService, it checks each node at a fixed interval, and a node whose check throws is marked unhealthy^[1].

Tool routing: by region and by version

Tool calls need routing too. A weather tool may need to switch to a regional endpoint based on where the user is, and a calculator tool may need to stay on a particular API version^[1]. The Python McpToolRouter class keeps three registries for this: base tool endpoints, regional endpoints and versioned endpoints^[1].

The entry method is simple. It selects an endpoint and then sends the request:

    async def route_tool_request(self, tool_name, parameters, user_context=None):
        """Route a tool request to the appropriate endpoint based on context"""
        endpoint = self._select_endpoint(tool_name, parameters, user_context)
        
        if not endpoint:
            raise ValueError(f"No endpoint available for tool: {tool_name}")
        
        # Perform the actual request to the selected endpoint
        return await self._execute_tool_request(endpoint, tool_name, parameters)

Endpoint selection happens in steps. The first step takes the base endpoint. The second reads _version from the parameters and falls back to the tool's default version, which is v2 for the weather tool^[1]. The third checks the user's region. If that region's registry includes the tool, the regional endpoint is used^[1]:

        # Check for regional routing if user region is known
        if user_context and "region" in user_context:
            user_region = user_context["region"]
            
            if user_region in self.regional_endpoints:
                regional_tools = self.regional_endpoints[user_region]
                
                if tool_name in regional_tools:
                    # Use region-specific endpoint
                    return regional_tools[tool_name]

This step has a trap. Once the region matches, the function returns the regional endpoint directly, and the version chosen earlier is ignored^[1]. A request that specifies both a region and a version may therefore get an endpoint other than the one it asked for. Before going live, add a test that defines the expected result for that combination.

The data residency and latency-sensitive branches contain only comments and pass, so they do nothing^[1]. They are placeholders. The names in the code do not mean the behavior exists.

The execution step posts JSON containing toolName and parameters to the selected endpoint with aiohttp, and raises an exception when the status is not 200^[1]. The error path prints the message and re-raises it. A comment there says retry or fallback logic still needs to be written^[1].

Points to check when reading the code

• All three samples are sketches. The endpoints are placeholder domains, classification relies on keywords, and health checks only print results. They show structure and are not ready to deploy.
• The architecture flowchart links the three kinds of routing into one path. The request router and analyzer come first, the load balancer then picks a node from the server pool, and tool calls from the models pass through a tool router to primary or regional tool registries^[1].
• In this material, "sampling" means classifying incoming requests by content. That is not the same as the sampling feature in the MCP specification, so keep the two apart when you read the specification.

Common problems

• Everything lands in the general category: check the keyword lists first, then look for wording outside them. When keyword coverage is incomplete, the general fallback takes most requests.
• All nodes fail: the Java sample makes only one fallback attempt, and when every node fails it throws the exception quoted above. At that point, the problem is in the nodes themselves rather than the routing logic.
• A pinned tool version has no effect: the request most likely also carries a region, which the regional branch overrides^[1].

Next steps

Sampling is the next topic worth reading. Protocol-level rules follow the MCP 2026-07-28 specification, linked in the footnote^[2].

Adapted from lesson material in Microsoft's open-source course *MCP for Beginners*, covering request routing in its advanced topics^[1], released under the MIT License.