Tool Calling Agents in smolagents

Every action an agent takes must first pass through a representation question: in what format does the model's decision get handed to the executor? The two mainstream answers are to let the model write executable code, or to have it emit a structured JSON object. smolagents ships both as ready-made classes: CodeAgent takes the code route, ToolCallingAgent the JSON route ^[1].

The difference is clearest with a concrete task: ask the agent to search for both "Best catering services in Gotham City" and "Party theme ideas for superheroes". A CodeAgent generates and runs a Python loop:

for query in [
    "Best catering services in Gotham City",
    "Party theme ideas for superheroes"
]:
    print(web_search(f"Search for: {query}"))

A ToolCallingAgent instead outputs a JSON array whose elements carry just two fields, the tool name and its arguments:

[
    {"name": "web_search", "arguments": "Best catering services in Gotham City"},
    {"name": "web_search", "arguments": "Party theme ideas for superheroes"}
]

The framework parses this blob and dispatches the calls to the matching tools; no code parser is ever involved ^[1].

Why the framework leans toward code

The maintainers' position is explicit: smolagents primarily focuses on CodeAgents because they perform better overall, a claim backed by their own paper (arXiv 2402.01030). The reasoning is worth unpacking. Code can declare variables, reuse intermediate results, and express loops and conditionals. JSON is a purely declarative structure: every level of nesting costs escaping and brackets, and intermediate results cannot be reused across calls. For the same task, a code version is usually shorter, cheaper in tokens, and capable of control flow that JSON cannot express.

JSON still has a comfort zone. Its structure is strict and trivially parseable, and more importantly every major provider's native API supports the function calling protocol — models have seen plenty of schema-shaped JSON during training, so their output stability in this mode is actually high. Simple systems that don't require variable handling or complex tool calls are exactly where ToolCallingAgent earns its keep ^[1].

What it looks like in use

ToolCallingAgent follows the same multi-step workflow as CodeAgent: think, act, observe, repeat until the final answer. The only difference is how actions are structured — instead of executable code, the model generates JSON objects that specify tool names and arguments, which the system parses and executes ^[1]. A web-connected agent takes a few lines:

from smolagents import ToolCallingAgent, WebSearchTool, InferenceClientModel

agent = ToolCallingAgent(tools=[WebSearchTool()], model=InferenceClientModel())

agent.run("Search for the best music recommendations for a party at the Wayne's mansion.")

In the trace, a CodeAgent prints "Executing parsed code:", while here you get a call announcement: the framework states which tool it is invoking (web_search) and with which query. The agent generates a structured tool call, the system processes it into an output, and no code execution happens anywhere ^[1].

Field notes for engineering

A few practices make JSON tool calling reliable. First, many providers support grammar-restricted, schema-guided decoding that guarantees well-formed JSON; schema violations are usually caught at parse time. Feed the parse error back as an observation instead of crashing — the model typically corrects itself on the next turn.

Second, validate arguments before execution: types, ranges, required fields. On failure, return a readable error message into the conversation so the agent can fix itself in its next Thought step ^[1]. Keep the tool registry clean too: few tools, unambiguous names, specific descriptions. The model picks tools from the descriptions in the system prompt, so overlapping names or vague descriptions directly raise the rate of wrong-tool and wrong-argument mistakes.

Third, follow a rule of thumb for selection: when every step is a single, self-contained tool call with fixed arguments, ToolCallingAgent is enough; once loops, conditionals, arithmetic, or cross-call state enter the picture, switch to CodeAgent ^[1]. Also note that WebSearchTool wraps DuckDuckGo search, whose results are inherently nondeterministic — evaluate agent traces that depend on it against replayable fixtures in tests, not live requests.

A final remark on the running example: Alfred the butler solves the same party-preparation task twice, first with a CodeAgent writing a search loop and then with a ToolCallingAgent emitting two JSON search calls, which neatly isolates the agent-type variable for comparison.

Adapted from the "Writing actions as code snippets or JSON blobs" section of Hugging Face's agents-course^[1], released under the Apache-2.0 License.

^[1]: Writing actions as code snippets or JSON blobs | Hugging Face Agents Course