An LLM can describe what should happen; a tool call can make it happen. That transition is the focus of CS329Z’s fourth lecture. The lecture starts by revisiting RAG’s limits, then asks how an agent can reach live state, private data, exact computation, and external systems through tools.
Key insights #
- Tool use is a sequence of decisions—selection, argument construction, execution, interpretation, and recovery—not a single JSON output.
- Toolformer and Gorilla teach or adapt tool behavior; runtime schemas make tools available without retraining for every API.
- REST, function calling, and MCP solve different interface problems, while authorization and validation remain application responsibilities.
- Multi-step success and consistency across repeated trials matter more than one syntactically valid call.
Why a tool is more than another source of text #
Retrieval can supply documents, but it cannot itself execute a database update, run a calculation, or observe an API’s current state. A tool is a callable operation with arguments and a result. The agent must choose the operation, construct valid arguments, execute it through a harness, interpret the result, and decide whether to continue. Errors can occur at each boundary.
The lecture’s RAG recap also explains why adding more retrieved pages is not a universal fix. Readers can miss evidence in long context, citations can be hallucinated, and retrieval cannot change external state. Tools cover three gaps: exact computation, information unavailable during training, and interfaces the model cannot reach through text prediction alone.
The lecture uses Toolformer to explain one way models learn this behavior. Toolformer samples possible API calls in text, executes them, and keeps calls that improve prediction. The model learns when a tool helps as well as its syntax. Gorilla tackles a different scale problem: generating calls across many APIs, using retrieved documentation to adapt to API changes and reduce invented calls. ToolLLM extends the many-API theme with a large curated tool set. These papers make clear that tool use is learned and evaluated behavior, not a capability granted merely by listing functions in a prompt.
How models learn tool use #
Toolformer’s training recipe has three distinct steps in the lecture: sample candidate calls at positions in text, execute them, then filter for calls whose results reduce next-token loss. The filter teaches the model to make useful calls rather than simply more calls. The slides also introduce ToolkenGPT, which treats a tool as a learned token and switches into argument generation after selecting it. These approaches place tool knowledge partly in training.
ToolLLM and Gorilla focus on scale. ToolLLM uses a retriever to narrow a catalog of thousands of APIs. Gorilla’s retrieval-aware training exposes the model to correct and incorrect documentation so it can use or ignore retrieved context at inference time. Retrieving a relevant tool description and executing that tool correctly are separate capabilities; an agent can succeed at the former and still invent a parameter or misread the result.
Three layers of interface #

Source: CS329Z Lecture 4, PDF page 48.
The slides compare three generations of tool interface:
| Interface | What the model or harness receives | Main burden |
|---|---|---|
| REST | Endpoints and human-oriented documentation | Parse docs, form requests, manage service-specific details |
| Function calling | A named operation with a JSON argument schema | Validate and dispatch calls in the application |
| MCP | Discoverable tools, resources, and prompts from servers | Manage a reusable client/server boundary and authorization |
Function calling makes the expected input shape explicit. It does not make the choice of operation correct. The MCP specification adds a shared protocol for discovering tools (tools/list) and invoking them (tools/call). The lecture distinguishes the host, client, and server, then points to inbound and outbound authorization as separate concerns. Standardizing a wire format does not remove the need to decide which capabilities an agent should receive.
The course LMS example makes the interface progression concrete: REST routes for assignments, submissions, and grading become model-facing MCP tools with names and input schemas. Discovery tells the agent what exists; execution sends a named call and returns content. The inbound token authorizing the agent to reach an MCP server and the outbound token allowing that server to reach the LMS are different security boundaries. Keeping them distinct limits what a model can see and do.
Evaluate the sequence, not the prettiness of the final answer #

Source: CS329Z Lecture 4, PDF page 67.
The lecture contrasts single-call tests such as BFCL with multi-step environments such as ToolBench and τ-bench. A single call can fail through wrong tool selection, missing fields, wrong types, wrong units, or a hallucinated API. In a longer task, the agent can skip a prerequisite, repeat a failing request, misread a result, or act when a policy requires a user decision.
BFCL can judge tool-call syntax and execution, including state changes. ToolBench exercises larger catalogs of real APIs. τ-bench adds an interacting user and policies to follow, so a valid API call is not enough. The lecture contrasts pass@k, the chance that at least one of several attempts succeeds, with pass^k, the chance they all do. A production user experiences an individual run, making consistency central. I would instrument selection, arguments, results, retries, and policy decisions, then score both final state and reliability across runs.
One practical design rule follows: give tools precise names and schemas, limit the available set to what the task needs, validate arguments before dispatch, and make failures legible to the agent.
Further reading #
- Lecture 4 slides and video with captions
- Schick et al., Toolformer
- Patil et al., Gorilla
- Model Context Protocol specification, 2025-06-18