The Tool Interface — Why Agents Need Structured I/O
A language model produces tokens. A program takes actions. The gap between those two is the tool interface: a contract that lets the model request an action and the host execute it. Every 2026 stack — function calling on OpenAI, Anthropic, and Gemini; MCP's `tools/call`; A2A's task parts — is a different encoding of the same four-step loop. This lesson names the loop and shows the minimum machinery to run it.
Prerequisites: Phase 11 (LLM completion APIs)
Video coming soon
An AI-narrated walkthrough for this module is on the way. Everything you need is in the lesson below.
#Learning Objectives
- Explain why an LLM that can only generate text cannot, on its own, take actions against the real world.
- Draw the four-step tool-call loop (describe → decide → execute → observe) and name who owns each step.
- Write a tool description as three parts: name, JSON Schema input, and a deterministic executor function.
- Distinguish pure and side-effecting tools and state why the split matters for safety.
#The Problem
An LLM emits a probability distribution over the next token. That is the entire output surface. If you ask a chat model "what is the weather in Bengaluru right now," it can write a plausible sentence, but it cannot dial into a weather API. The sentence might be right by coincidence or three days stale.
Closing that gap is the purpose of the tool interface. The host program — your agent runtime, Claude Desktop, ChatGPT, Cursor, or a custom script — advertises a list of callable tools to the model. The model, when it decides an action is needed, emits a structured payload naming a tool and its arguments. The host parses that payload, runs the tool for real, and feeds the result back. The loop continues until the model decides no more calls are needed.
The first version of this contract shipped in June 2023 as OpenAI's "functions" parameter. Anthropic followed with tool_use blocks in Claude 2.1. Gemini added functionDeclarations a few months later. Every provider now exposes the same shape: a JSON-Schema-typed tool list in, a JSON-payload tool call out. The Model Context Protocol (November 2024) generalized the contract so one tool registry serves every model. A2A (April 2026, v1.0) layered the same primitive for agent-to-agent delegation.
The four-step loop is the invariant underneath all of these. Everything else in Phase 13 is an elaboration.
#The Concept
#Step one: describe
The host declares each tool with three fields.
- Name. A stable, machine-readable identifier.
get_weather, not "weather thing". - Description. A one-paragraph natural-language brief. "Use when the user asks about current conditions for a specific city. Do not use for historical data."
- Input schema. A JSON Schema object (draft 2020-12) describing the tool's arguments.
The model receives the list. Modern providers serialize these declarations into the system prompt using a provider-specific template, so you as the caller only deal with the structured form.
#Step two: decide
Given the user's message and the available tools, the model chooses one of three behaviors.
- Answer directly in text. No tool call.
- Call one or more tools. Emit structured call objects. Under
parallel_tool_calls: true(default on OpenAI and Gemini, opt-in on Anthropic) the model can emit multiple calls in one turn. - Refuse. Strict-mode structured outputs can produce a typed
refusalblock instead of a call.
A tool call payload has three stable fields: a call id, a tool name, and a JSON arguments object. The id exists so the host can correlate the later result with the specific call, which matters when parallel calls come back out of order.
#Step three: execute
The host receives the call, validates arguments against the declared schema, and runs the executor. Invalid arguments mean the model hallucinated a field or used the wrong type — a very common failure mode on weak models. Production hosts do one of three things on invalid arguments: fail fast and surface the error to the model, repair the JSON with a constrained parser, or retry the model with the validation error included in the prompt.
The executor itself is ordinary code. Python, TypeScript, a shell command, a database query. It produces a result, which is usually a string but can be any JSON value or a structured content block (text, image, or resource reference in MCP). The result must be serializable.
#Step four: observe
The host appends the tool result to the conversation (as a tool role message with matching id) and re-invokes the model. The model now has the tool output in context and can produce a final answer or request more calls. This continues until the model stops emitting calls or the host hits a safety limit on iteration count.
#The trust split
Tools come in two flavors that matter for safety.
- Pure. Read-only, deterministic, no side effects.
get_weather,search_docs,get_current_time. Safe to call speculatively. - Consequential. Mutates state, spends money, touches user data.
send_email,delete_file,execute_trade. Must be gated.
Meta's 2026 "Rule of Two" for agent security says a single turn may combine at most two of: untrusted input, sensitive data, consequential action. The tool interface is where you enforce that rule — by rejecting calls, requiring user confirmation, or escalating scopes. See Phase 13 · 15 for the full security chapter and Phase 14 · 09 for agent-level permission policies.
#Where the loop lives
| Context | Who describes | Who decides | Who executes |
|---|---|---|---|
| Single-turn function calling (OpenAI/Anthropic/Gemini) | App developer | LLM | App developer |
| MCP | MCP server | LLM via MCP client | MCP server |
| A2A | Agent Card publisher | Calling agent | Called agent |
| Web browser (function-calling agent) | Browser extension / WebMCP | LLM | Browser runtime |
Everywhere, the same four steps. The column names change; the structure does not.
#Why not just prompt the model to emit JSON?
"Ask the model to reply in JSON" was the pre-function-calling pattern. It fails ~5 to 15 percent of the time on frontier models and far more on smaller models. Failure modes include missing braces, trailing commas, hallucinated fields, and wrong types. You then need a JSON repair pass, a retry, or a constrained decoder.
Native function calling is better for three reasons. First, the provider trains the model end-to-end on the exact call shape, so valid-JSON rate climbs to 98 to 99 percent on strict mode. Second, the call payload sits in its own protocol slot, not inside free-text — so a tool call never leaks into the user-visible reply. Third, providers enforce schema compliance with constrained decoding (OpenAI's strict mode, Anthropic's tool_use, Gemini's responseSchema). The output is guaranteed to validate.
Phase 13 · 02 walks the three provider APIs side by side. Phase 13 · 04 goes deep on structured outputs.
#Circuit breakers
The loop terminates when the model stops emitting calls or the host hits a maximum turn count. Production hosts set this to between 5 and 20 turns. Beyond that, you are almost certainly in a loop the model cannot exit. Claude Code defaults to 20; OpenAI Assistants to 10; Cursor's agent mode to 25.
The alternative — unbounded loops — shows up every six months as "agent spent $400 in API calls overnight" post-mortems. Do not ship without a bound.
Phase 14 · 12 covers error recovery and self-healing in depth; Phase 17 covers production rate limits.
#Where Phase 13 goes from here
- Lessons 02 through 05 polish the provider-level tool-call surface.
- Lessons 06 through 14 generalize the loop into MCP.
- Lessons 15 through 18 defend the loop against hostile servers, adversarial users, and unauthenticated remote auth surfaces.
- Lessons 19 through 22 extend the pattern to agent-to-agent collaboration, observability, routing, and packaging.
- Lesson 23 ships a complete ecosystem using every primitive.
Every remaining lesson is an elaboration of this four-step loop. Hold it in mind as the invariant.
#Use It
code/main.py runs the four-step loop without an LLM. A fake "decider" function simulates the model by pattern-matching on the user message; the executor, schema validator, and observe-step harness are real. Run it to see the full request/response choreography with printable intermediate state, then replace the fake decider with any real provider in a later lesson.
What to look at:
- The tool registry holds three fields per tool: name, description, schema, and an executor reference.
- The validator is a minimal JSON Schema subset (types, required, enum, min/max) written in stdlib only. Phase 13 · 04 ships a fuller one.
- The loop bounds iteration count at five. Production agents need exactly this kind of circuit breaker.
#Ship It
This lesson produces outputs/skill-tool-interface-reviewer.md. Given a draft tool definition (name + description + schema + executor outline), the skill audits it for loop fitness: is the name machine-stable, is the description a complete usage brief, does the schema use JSON Schema 2020-12 correctly, and is the pure-vs-consequential classification explicit.
#Exercises
Add a fourth tool to
code/main.pycalledget_stock_price(ticker). Write its description as "Use when the user asks for a current stock price by ticker. Do not use for historical prices or market summaries." Run the harness and confirm the fake decider routes queries mentioning tickers to the new tool.Break the schema validator. Pass a call whose
argumentsobject is missing a required field, and confirm the host rejects it before execution. Then pass a call with an extra unknown field. Decide: should the host reject or ignore? Justify your choice with a safety argument.Classify each tool in the harness as pure or consequential. Add a
consequential: trueflag to the registry entries that need it, and change the loop to print a "would confirm with user" line whenever a consequential tool is chosen. This is the shape of the confirmation gate every production host needs.Draw the four-step loop on paper with the provider-column table above filled in for your favorite client (Claude Desktop, Cursor, ChatGPT, or a custom stack). Cross-reference with the MCP-specific variant in Phase 13 · 06.
Read OpenAI's function-calling guide top to bottom. Identify the one field that sits in the request but not in the four-step loop as presented here. Explain what it adds and why it is convenient rather than essential.
#Key Terms
| Term | What people say | What it actually means |
|---|---|---|
| Tool | "A thing the model can call" | A triple of name + JSON-Schema-typed input + executor function |
| Function calling | "Native tool use" | Provider-level API support for emitting structured tool calls instead of prose |
| Tool call | "The model's request to act" | A JSON payload with id, name, arguments emitted by the model |
| Tool result | "What the tool returned" | The executor's output, wrapped in a tool role message with matching id |
| Parallel tool calls | "Many calls at once" | Multiple call objects in one model turn, independent and orderable by id |
| Strict mode | "Guaranteed JSON" | Constrained decoding that forces the model's output to validate against the declared schema |
| Pure tool | "Read-only tool" | No side effects; safe to re-run |
| Consequential tool | "Action tool" | Mutates external state; requires gate, audit, or user confirmation |
| Four-step loop | "The tool-call cycle" | describe → decide → execute → observe |
| Host | "Agent runtime" | The program that holds the tool registry, calls the model, and runs the executor |
#Further Reading
- OpenAI — Function calling guide — canonical reference for OpenAI-style tool declarations and call shapes
- Anthropic — Tool use overview — Claude's
tool_use/tool_resultblock format - Google — Gemini function calling —
functionDeclarationsand parallel-call semantics in Gemini - Model Context Protocol — Specification 2025-11-25 — the provider-agnostic generalization of the tool interface
- JSON Schema — 2020-12 release notes — the schema dialect every modern tool API speaks
Run this lesson in the Code Lab
A real JupyterLab, with this lesson's code files (and a ready-to-run notebook). The Quick Lab opens instantly in your browser. The Cloud Lab adds a terminal, saved files and the full toolchain.
"""Phase 13 Lesson 01 - the tool interface, four-step loop, no LLM.
Implements the describe -> decide -> execute -> observe cycle used by every
2026 tool-calling stack (OpenAI, Anthropic, Gemini, MCP, A2A). The "decide"
step is faked with a keyword router so the loop runs offline; replace it with
any real provider in Lesson 02.
The harness:
- registers three tools (add, get_time, get_weather)
- validates tool-call arguments against a minimal JSON Schema subset
- prints each step so you can read the choreography
- bounds iteration at MAX_TURNS to prevent runaway loops
Run: python code/main.py
"""
from __future__ import annotations
import datetime as dt
import json
import re
import time
import uuid
from dataclasses import dataclass
from typing import Any, Callable
MAX_TURNS = 5
@dataclass
class Tool:
name: str
description: str
input_schema: dict
executor: Callable[[dict], Any]
consequential: bool = False
def tool_add(args: dict) -> dict:
return {"sum": args["a"] + args["b"]}
def tool_get_time(args: dict) -> dict:
tz = args.get("timezone", "UTC")
now = dt.datetime.now(dt.timezone.utc).isoformat(timespec="seconds")
return {"now": now, "timezone": tz}
def tool_get_weather(args: dict) -> dict:
fake = {"Bengaluru": 28, "Tokyo": 12, "Zurich": 4, "Lagos": 31}
city = args["city"]
units = args.get("units", "celsius")
temp = fake.get(city, 20)
return {"city": city, "temp": temp, "units": units}
REGISTRY: list[Tool] = [
Tool(
name="add",
description=(
"Use when the user asks for the sum of two numbers. "
"Do not use for subtraction, product, or symbolic algebra."
),
input_schema={
"type": "object",
"properties": {
"a": {"type": "number"},
"b": {"type": "number"},
},
"required": ["a", "b"],
},
executor=tool_add,
),
Tool(
name="get_time",
description=(
"Use when the user asks what time it is. "
"Do not use for historical dates or future scheduling."
),
input_schema={
"type": "object",
"properties": {
"timezone": {"type": "string"},
},
"required": [],
},
executor=tool_get_time,
),
Tool(
name="get_weather",
description=(
"Use when the user asks about current conditions in a named city. "
"Do not use for forecasts or historical weather data."
),
input_schema={
"type": "object",
"properties": {
"city": {"type": "string"},
"units": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["city"],
},
executor=tool_get_weather,
),
]
def validate(schema: dict, value: Any) -> list[str]:
errors: list[str] = []
t = schema.get("type")
if t == "object":
if not isinstance(value, dict):
return [f"expected object, got {type(value).__name__}"]
for field in schema.get("required", []):
if field not in value:
errors.append(f"missing required field '{field}'")
for key, sub in schema.get("properties", {}).items():
if key in value:
errors.extend(validate(sub, value[key]))
return errors
if t == "number" and not isinstance(value, (int, float)):
errors.append(f"expected number, got {type(value).__name__}")
if t == "string" and not isinstance(value, str):
errors.append(f"expected string, got {type(value).__name__}")
if "enum" in schema and value not in schema["enum"]:
errors.append(f"value {value!r} not in enum {schema['enum']}")
return errors
def fake_decide(user_msg: str, history: list[dict]) -> dict:
"""Stand-in for the model. Routes by keyword so the loop runs offline.
Production substitute: swap this for provider.chat.completions.create with
tools=[t.input_schema for t in REGISTRY]. Same return shape.
"""
last = history[-1] if history else {}
if last.get("role") == "tool":
return {"content": f"Final answer built from tool output: {last.get('content')}"}
msg = user_msg.lower()
if re.search(r"\b(add|sum|plus)\b", msg):
nums = [float(n) for n in re.findall(r"-?\d+\.?\d*", msg)]
if len(nums) >= 2:
return {
"tool_calls": [
{
"id": f"call_{uuid.uuid4().hex[:8]}",
"name": "add",
"arguments": {"a": nums[0], "b": nums[1]},
}
]
}
if "time" in msg:
return {
"tool_calls": [
{
"id": f"call_{uuid.uuid4().hex[:8]}",
"name": "get_time",
"arguments": {"timezone": "UTC"},
}
]
}
match = re.search(r"weather in (\w+)", msg)
if match:
city = match.group(1).title()
return {
"tool_calls": [
{
"id": f"call_{uuid.uuid4().hex[:8]}",
"name": "get_weather",
"arguments": {"city": city, "units": "celsius"},
}
]
}
return {"content": "I cannot route that query to any registered tool."}
def run_loop(user_msg: str) -> None:
print("=" * 72)
print(f"USER : {user_msg}")
print("-" * 72)
tools_by_name = {t.name: t for t in REGISTRY}
history: list[dict] = [{"role": "user", "content": user_msg}]
for turn in range(1, MAX_TURNS + 1):
decision = fake_decide(user_msg, history)
if "content" in decision:
print(f"TURN {turn} DECIDE : final answer")
print(f"MODEL : {decision['content']}")
return
for call in decision["tool_calls"]:
tool = tools_by_name.get(call["name"])
print(f"TURN {turn} DECIDE : call {call['name']} id={call['id']}")
print(f" args = {json.dumps(call['arguments'])}")
if tool is None:
print(f" ERROR : unknown tool {call['name']}")
return
errs = validate(tool.input_schema, call["arguments"])
if errs:
print(f" VALIDATION ERRORS : {errs}")
return
if tool.consequential:
print(" GATE : tool is consequential, would confirm")
start = time.perf_counter()
result = tool.executor(call["arguments"])
ms = (time.perf_counter() - start) * 1000
print(f"TURN {turn} EXECUTE: {tool.name} -> {json.dumps(result)}"
f" [{ms:.2f} ms]")
history.append({
"role": "tool", "id": call["id"],
"name": tool.name, "content": json.dumps(result),
})
print(f"TURN {turn} OBSERVE: history length = {len(history)}")
print("LOOP TERMINATED : hit MAX_TURNS circuit breaker")
def describe_registry() -> None:
print("TOOL REGISTRY")
print("-" * 72)
for t in REGISTRY:
kind = "consequential" if t.consequential else "pure"
print(f" {t.name:14s} [{kind}] - {t.description}")
print()
def main() -> None:
print("=" * 72)
print("PHASE 13 LESSON 01 - THE TOOL INTERFACE")
print("=" * 72)
describe_registry()
for query in (
"please add 7 and 35",
"what time is it?",
"tell me the weather in Bengaluru",
"write me a haiku about tea",
):
run_loop(query)
print()
if __name__ == "__main__":
main()
// Phase 13 Lesson 01 — the tool interface, in TypeScript.
//
// Mirrors code/main.py: describe -> decide -> execute -> observe.
// The "decide" step is faked with a keyword router so the loop runs offline;
// replace with any real provider client and the shape stays the same.
//
// Spec references:
// OpenAI tool calling https://platform.openai.com/docs/guides/function-calling
// Anthropic tool use https://docs.anthropic.com/en/docs/build-with-claude/tool-use
// MCP tool primitive https://modelcontextprotocol.io/specification/2025-11-25
//
// Run: npx tsx code/main.ts
import { randomUUID } from "node:crypto";
const MAX_TURNS = 5;
type JsonSchema = {
type?: "object" | "string" | "number" | "integer" | "boolean" | "array";
properties?: Record<string, JsonSchema>;
required?: string[];
enum?: unknown[];
};
type ToolArgs = Record<string, unknown>;
type ToolResult = Record<string, unknown>;
type Tool = {
name: string;
description: string;
inputSchema: JsonSchema;
executor: (args: ToolArgs) => ToolResult;
consequential?: boolean;
};
type HistoryEntry =
| { role: "user"; content: string }
| { role: "tool"; id: string; name: string; content: string };
type ToolCall = {
id: string;
name: string;
arguments: ToolArgs;
};
type Decision = { content: string } | { toolCalls: ToolCall[] };
function toolAdd(args: ToolArgs): ToolResult {
const a = args.a as number;
const b = args.b as number;
return { sum: a + b };
}
function toolGetTime(args: ToolArgs): ToolResult {
const timezone = (args.timezone as string | undefined) ?? "UTC";
const now = new Date().toISOString().replace(/\.\d{3}Z$/, "Z");
return { now, timezone };
}
function toolGetWeather(args: ToolArgs): ToolResult {
const fake: Record<string, number> = {
Bengaluru: 28,
Tokyo: 12,
Zurich: 4,
Lagos: 31,
};
const city = args.city as string;
const units = (args.units as string | undefined) ?? "celsius";
const temp = fake[city] ?? 20;
return { city, temp, units };
}
const REGISTRY: Tool[] = [
{
name: "add",
description:
"Use when the user asks for the sum of two numbers. " +
"Do not use for subtraction, product, or symbolic algebra.",
inputSchema: {
type: "object",
properties: {
a: { type: "number" },
b: { type: "number" },
},
required: ["a", "b"],
},
executor: toolAdd,
},
{
name: "get_time",
description:
"Use when the user asks what time it is. " +
"Do not use for historical dates or future scheduling.",
inputSchema: {
type: "object",
properties: {
timezone: { type: "string" },
},
required: [],
},
executor: toolGetTime,
},
{
name: "get_weather",
description:
"Use when the user asks about current conditions in a named city. " +
"Do not use for forecasts or historical weather data.",
inputSchema: {
type: "object",
properties: {
city: { type: "string" },
units: { type: "string", enum: ["celsius", "fahrenheit"] },
},
required: ["city"],
},
executor: toolGetWeather,
},
];
function validate(schema: JsonSchema, value: unknown): string[] {
const errors: string[] = [];
const t = schema.type;
if (t === "object") {
if (typeof value !== "object" || value === null || Array.isArray(value)) {
return [`expected object, got ${describeType(value)}`];
}
const obj = value as Record<string, unknown>;
for (const field of schema.required ?? []) {
if (!(field in obj)) errors.push(`missing required field '${field}'`);
}
for (const [key, sub] of Object.entries(schema.properties ?? {})) {
if (key in obj) errors.push(...validate(sub, obj[key]));
}
return errors;
}
if (t === "number" && typeof value !== "number") {
errors.push(`expected number, got ${describeType(value)}`);
}
if (t === "string" && typeof value !== "string") {
errors.push(`expected string, got ${describeType(value)}`);
}
if (schema.enum && !schema.enum.includes(value as never)) {
errors.push(`value ${JSON.stringify(value)} not in enum ${JSON.stringify(schema.enum)}`);
}
return errors;
}
function describeType(value: unknown): string {
if (value === null) return "null";
if (Array.isArray(value)) return "array";
return typeof value;
}
function newCallId(): string {
return `call_${randomUUID().replace(/-/g, "").slice(0, 8)}`;
}
// Stand-in for the model. Routes by keyword so the loop runs offline.
// Production substitute: replace with a provider call returning the same shape.
function fakeDecide(userMsg: string, history: HistoryEntry[]): Decision {
const last = history[history.length - 1];
if (last && last.role === "tool") {
return { content: `Final answer built from tool output: ${last.content}` };
}
const msg = userMsg.toLowerCase();
if (/\b(add|sum|plus)\b/.test(msg)) {
const nums = (msg.match(/-?\d+\.?\d*/g) ?? []).map((n) => Number(n));
if (nums.length >= 2) {
return {
toolCalls: [
{ id: newCallId(), name: "add", arguments: { a: nums[0], b: nums[1] } },
],
};
}
}
if (msg.includes("time")) {
return {
toolCalls: [
{ id: newCallId(), name: "get_time", arguments: { timezone: "UTC" } },
],
};
}
const weatherMatch = msg.match(/weather in (\w+)/);
if (weatherMatch) {
const city = weatherMatch[1][0].toUpperCase() + weatherMatch[1].slice(1);
return {
toolCalls: [
{
id: newCallId(),
name: "get_weather",
arguments: { city, units: "celsius" },
},
],
};
}
return { content: "I cannot route that query to any registered tool." };
}
function runLoop(userMsg: string): void {
console.log("=".repeat(72));
console.log(`USER : ${userMsg}`);
console.log("-".repeat(72));
const toolsByName = new Map(REGISTRY.map((t) => [t.name, t]));
const history: HistoryEntry[] = [{ role: "user", content: userMsg }];
for (let turn = 1; turn <= MAX_TURNS; turn++) {
const decision = fakeDecide(userMsg, history);
if ("content" in decision) {
console.log(`TURN ${turn} DECIDE : final answer`);
console.log(`MODEL : ${decision.content}`);
return;
}
for (const call of decision.toolCalls) {
const tool = toolsByName.get(call.name);
console.log(`TURN ${turn} DECIDE : call ${call.name} id=${call.id}`);
console.log(` args = ${JSON.stringify(call.arguments)}`);
if (!tool) {
console.log(` ERROR : unknown tool ${call.name}`);
return;
}
const errs = validate(tool.inputSchema, call.arguments);
if (errs.length > 0) {
console.log(` VALIDATION ERRORS : ${JSON.stringify(errs)}`);
return;
}
if (tool.consequential) {
console.log(" GATE : tool is consequential, would confirm");
}
const start = performance.now();
const result = tool.executor(call.arguments);
const ms = performance.now() - start;
console.log(
`TURN ${turn} EXECUTE: ${tool.name} -> ${JSON.stringify(result)} [${ms.toFixed(2)} ms]`,
);
history.push({
role: "tool",
id: call.id,
name: tool.name,
content: JSON.stringify(result),
});
}
console.log(`TURN ${turn} OBSERVE: history length = ${history.length}`);
}
console.log("LOOP TERMINATED : hit MAX_TURNS circuit breaker");
}
function describeRegistry(): void {
console.log("TOOL REGISTRY");
console.log("-".repeat(72));
for (const t of REGISTRY) {
const kind = t.consequential ? "consequential" : "pure";
console.log(` ${t.name.padEnd(14)} [${kind}] - ${t.description}`);
}
console.log();
}
function main(): void {
console.log("=".repeat(72));
console.log("PHASE 13 LESSON 01 - THE TOOL INTERFACE (TypeScript port)");
console.log("=".repeat(72));
describeRegistry();
const queries = [
"please add 7 and 35",
"what time is it?",
"tell me the weather in Bengaluru",
"write me a haiku about tea",
];
for (const q of queries) {
runLoop(q);
console.log();
}
}
main();
Reusable outputs from this lesson. Copy them into your own work.
---
name: tool-interface-reviewer
description: Audit a tool definition (name + description + JSON Schema + executor outline) for loop fitness before it ships to an LLM.
version: 1.0.0
phase: 13
lesson: 01
tags: [tool-calling, function-calling, json-schema, tool-design]
---
Given a proposed tool definition, review it against the four-step loop (describe, decide, execute, observe) and flag loop-breaking defects before the tool reaches a model.
Produce:
1. Name audit. Is the name `snake_case`, stable across versions, and unambiguous? Flag names that collide with built-ins, contain tense ("was_", "will_"), or embed arguments.
2. Description audit. Does the description read as a complete usage brief? Require the two-sentence shape: "Use when X. Do not use for Y." Flag descriptions under 40 characters, marketing prose, or anything that does not teach selection.
3. Schema audit. Is the schema valid JSON Schema 2020-12? Every field typed? `required` list explicit? Enums used for closed value sets? Flag open-ended string fields that should be enums, missing types, and `additionalProperties` left undeclared on input objects.
4. Executor audit. Is the executor deterministic given arguments? Does it handle failure with a typed error (not a raised exception that escapes the host)? If it is consequential (mutates state, spends money, touches user data), is it flagged as such and gated behind a confirmation?
5. Classification. State whether the tool is pure or consequential and why. A consequential tool without a gate is an immediate reject.
Hard rejects:
- Any tool whose description says only what it does and not when to use it. The model needs the "when" for step two.
- Any schema with an untyped field. The validator cannot do its job.
- Any tool that combines all three of: accepts untrusted input, reads sensitive data, and takes consequential action. Violates Meta's Rule of Two.
- Any tool whose executor raises unhandled exceptions on bad input. The host should not need a try/except around every call.
Refusal rules:
- If the tool definition is missing a schema, refuse. Route to Phase 13 · 04 first.
- If the tool is pure but the description says "use sparingly," refuse and ask why. Pure tools should be cheap to re-run.
- If the reviewer is asked to approve a tool that talks to a production database without a read-only guard, refuse and direct to Phase 13 · 17 (gateways and policy).
Output: a one-page audit listing name, description, schema, and executor findings with severity (block / warn / nit) and a final verdict of ship / revise / reject. End with a one-line rewrite suggestion for any reject, if feasible.
The text and code of this lesson are based on AI Engineering from Scratch by Rohit Ghumare, MIT licensed.