Agents and tools in LlamaIndex

Who this is for and what you will be able to do

If you already know what an LLM can do and want to build an agent that calls your own functions, searches your documents or works with other agents, this guide is for you. By the end you should be able to create a basic LlamaIndex agent, keep its conversation state, wrap a query engine as a tool, and combine several agents into one workflow.

The three agent types

LlamaIndex supports three main kinds of reasoning agents^[1]:

1. Function Calling Agents, which work with models that can call specific functions^[1].
2. ReAct Agents, which work with any model that offers a chat or text endpoint, and which handle complex reasoning and show the reasoning behind each action^[1].
3. Advanced Custom Agents, which use more complex methods for harder tasks and workflows^[1].

For a first agent, you only need to pass a set of functions or tools. The agent is built from them^[1].

Build a minimal agent

The following example defines a multiply function, sets up a Hugging Face inference LLM, and creates an agent with AgentWorkflow.from_tools_or_functions(). The function's type annotations, name and docstring are all included in the schema the model reads^[1]:

from llama_index.llms.huggingface_api import HuggingFaceInferenceAPI
from llama_index.core.agent.workflow import AgentWorkflow
from llama_index.core.tools import FunctionTool

# define sample Tool -- type annotations, function names, and docstrings, are all included in parsed schemas!
def multiply(a: int, b: int) -> int:
    """Multiplies two integers and returns the resulting integer"""
    return a * b

# initialize llm
llm = HuggingFaceInferenceAPI(model_name="Qwen/Qwen2.5-Coder-32B-Instruct")

# initialize agent
agent = AgentWorkflow.from_tools_or_functions(
    [FunctionTool.from_defaults(multiply)],
    llm=llm
)

Agents are stateless by default. Each run starts with no memory of earlier runs. To keep a conversation going, create a Context object and pass it on every call^[1]:

# stateless
response = await agent.run("What is 2 times 2?")

# remembering state
from llama_index.core.workflow import Context

ctx = Context(agent)

response = await agent.run("My name is Bob.", ctx=ctx)
response = await agent.run("What was my name again?", ctx=ctx)

Notice the await keyword. Agents in LlamaIndex are asynchronous, so the calls must run inside async code^[1].

Tools in LlamaIndex

The tool interface is what the model sees, so a clear set of tools matters for performance^[2]. LlamaIndex has four main kinds of tools: FunctionTool, QueryEngineTool, ToolSpecs and utility tools^[2].

FunctionTool wraps any Python function. It accepts synchronous or asynchronous functions, and the optional name and description parameters help the agent decide when and how to use the tool^[2]. You can test a tool directly by calling it:

from llama_index.core.tools import FunctionTool

def get_weather(location: str) -> str:
    """Useful for getting the weather for a given location."""
    print(f"Getting weather for {location}")
    return f"The weather in {location} is sunny"

tool = FunctionTool.from_defaults(
    get_weather,
    name="my_weather_tool",
    description="Useful for getting the weather for a given location.",
)
tool.call("New York")

Agentic RAG with a QueryEngineTool

Retrieval-augmented generation (RAG) is the usual way to answer questions about your own documents. In an agentic setup, the agent decides whether to use the query engine or some other tool, instead of answering automatically from retrieved text^[1]. To make that possible, wrap the query engine with a name and a description. The model relies on those two fields to choose the tool correctly^[1].

The example below builds a vector index on a persistent Chroma collection, then turns its query engine into a QueryEngineTool^[2]. The embedding model is BAAI/bge-small-en-v1.5^[2]:

from llama_index.core import VectorStoreIndex
from llama_index.core.tools import QueryEngineTool
from llama_index.llms.huggingface_api import HuggingFaceInferenceAPI
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.vector_stores.chroma import ChromaVectorStore

embed_model = HuggingFaceEmbedding("BAAI/bge-small-en-v1.5")

db = chromadb.PersistentClient(path="./alfred_chroma_db")
chroma_collection = db.get_or_create_collection("alfred")
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)

index = VectorStoreIndex.from_vector_store(vector_store, embed_model=embed_model)

llm = HuggingFaceInferenceAPI(model_name="Qwen/Qwen2.5-Coder-32B-Instruct")
query_engine = index.as_query_engine(llm=llm)
tool = QueryEngineTool.from_defaults(query_engine, name="some useful name", description="some useful description")

The snippet uses chromadb without importing it, so add import chromadb yourself before running it.

Several agents in one workflow

AgentWorkflow also runs multi-agent systems. Give each agent a name and a description. The system keeps one active speaker at a time, and an agent can hand off to another agent^[1]. Narrowing each agent's scope usually improves its accuracy on the messages it receives^[1].

Agents can also be used as tools for other agents. The example below defines two plain functions for a calculator agent^[1]:

from llama_index.core.agent.workflow import (
    AgentWorkflow,
    FunctionAgent,
    ReActAgent,
)

# Define some tools
def add(a: int, b: int) -> int:
    """Add two numbers."""
    return a + b


def subtract(a: int, b: int) -> int:
    """Subtract two numbers."""
    return a - b

The calculator is a ReActAgent with its own name, description, system prompt and tools^[1]:

calculator_agent = ReActAgent(
    name="calculator",
    description="Performs basic arithmetic operations",
    system_prompt="You are a calculator assistant. Use your tools for any math operation.",
    tools=[add, subtract],
    llm=llm,
)

The workflow then names calculator as the root agent and runs one user message through it^[1]:

# Create and run the workflow
agent = AgentWorkflow(
    agents=[calculator_agent, query_agent], root_agent="calculator"
)

# Run the system
response = await agent.run(user_msg="Can you add 5 and 3?")

The example mentions query_agent without defining it in the same snippet. It is a second agent that wraps the query engine tool from the RAG section.

Choose the agent class by the model. FunctionAgent suits models with a function-calling API, and ReActAgent works with any LLM.

Utility tools for large results

Querying an API directly can return more data than you need. The data can crowd the context window or push token use up. Utility tools handle that case^[2].

OnDemandToolLoader wraps any LlamaIndex data loader, which is a BaseReader class. In a single tool call it loads the data, indexes it and runs a query against the index^[2]. LoadAndSearchToolSpec takes an existing tool and returns two tools: one that loads and indexes the output, and one that searches that index^[2].

Connect MCP tools

LlamaIndex can use MCP tools through a ToolSpec from LlamaHub, which lists ToolSpecs and utility tools^[2]. Install the MCP ToolSpec first^[2]:

pip install llama-index-tools-mcp

The snippet below connects to an MCP server at a local address and wraps it with McpToolSpec. It then creates an agent context^[2]:

from llama_index.tools.mcp import BasicMCPClient, McpToolSpec

# We consider there is a mcp server running on 127.0.0.1:8000, or you can use the mcp client to connect to your own mcp server.
mcp_client = BasicMCPClient("http://127.0.0.1:8000/sse")
mcp_tool = McpToolSpec(client=mcp_client)

# get the agent
agent = await get_agent(mcp_tool)

# create the agent context
agent_context = Context(agent)

The get_agent helper is not defined in the snippet, so you need to write it yourself before the code will run.

Limits and common problems

• Tool names and descriptions drive tool choice. Vague wording leads to the wrong tool or the wrong arguments^[2].
• Each agent in a multi-agent setup needs a clear name and description, because those fields are how the system decides who speaks^[1].
• Use utility tools when raw results are too large to hand straight back to the model^[2].
• The examples use a Hugging Face inference LLM and a local Chroma store. Both need their own setup before the code runs.

Next steps

For the MCP side, the free MCP course is a good next step^[2]. Toolspecs and utility tools are on LlamaHub at https://llamahub.ai/^[2].

Adapted from the LlamaIndex lessons on agents and tools in Unit 2 of Hugging Face's open-source *Agents Course*^[1][2], released under the Apache-2.0 License.