Capstone 01 — Terminal-Native Coding Agent
By 2026 the shape of a coding agent is settled. A TUI harness, a stateful plan, a sandboxed tool surface, a loop that plans, acts, observes, recovers. Claude Code, Cursor 3, and OpenCode all look the same from 50 feet. This capstone asks you to build one end to end — CLI in, pull request out — and measure it against mini-swe-agent and Live-SWE-agent on SWE-bench Pro. You will learn why the hard part is not the model call but the tool loop, the sandbox, and the cost ceiling on a 50-turn run.
Prerequisites: Phase 11 (LLM engineering), Phase 13 (tools and protocols), Phase 14 (agents), Phase 15 (autonomous systems), Phase 17 (infrastructure)
Video coming soon
An AI-narrated walkthrough for this module is on the way. Everything you need is in the lesson below.
#Problem
Coding agents became the dominant AI application category in 2026. Claude Code (Anthropic), Cursor 3 with Composer 2 and Agent Tabs (Cursor), Amp (Sourcegraph), OpenCode (112k stars), Factory Droids, and Google Jules all ship variations of the same architecture: a terminal harness, a permissioned tool surface, a sandbox, and a plan-act-observe loop built around a frontier model. The frontier is narrow — Live-SWE-agent reached 79.2% on SWE-bench Verified with Opus 4.5 — but the engineering craft is wide. Most failure modes are not model mistakes. They are tool-loop instability, context poisoning, runaway token cost, and destructive filesystem operations.
You cannot reason about these agents from the outside. You have to build one, watch the loop crash on turn 47 when ripgrep returns 8MB of matches, and rebuild the truncation layer. That is the point of this capstone.
#Concept
The harness has four surfaces. Plan maintains a TodoWrite-style state object that the model rewrites each turn. Act dispatches tool calls (read, edit, run, search, git). Observe captures stdout / stderr / exit codes, truncates, and feeds the summary back. Recover handles tool errors without blowing the context window or looping forever. The 2026 shape adds one more thing: hooks. PreToolUse, PostToolUse, SessionStart, SessionEnd, UserPromptSubmit, Notification, Stop, and PreCompact — configurable extension points where the operator injects policy, telemetry, and guardrails.
The sandbox is E2B or Daytona. Each task runs in a fresh devcontainer with a git worktree mounted read-write. The harness never touches the host filesystem. The worktree gets torn down on success or failure. Cost control is enforced at three layers: a per-turn token ceiling, a per-session dollar budget, and a hard turn limit (typically 50). The observability layer is OpenTelemetry spans with GenAI semantic conventions, shipped to a self-hosted Langfuse.
#Architecture
user CLI -> harness (Bun + Ink TUI)
|
v
plan / act / observe loop <---> Claude Sonnet 4.7 / GPT-5.4-Codex / Gemini 3 Pro
| (via OpenRouter, model-agnostic)
v
tool dispatcher (MCP StreamableHTTP client)
|
+------------+------------+----------+
v v v v
read/edit ripgrep tree-sitter git/run
| | | |
+------------+------------+----------+
|
v
E2B / Daytona sandbox (worktree isolated)
|
v
hooks: Pre/Post, Session, Prompt, Compact
|
v
OpenTelemetry -> Langfuse (spans, tokens, $)
|
v
PR via GitHub app#Stack
- Harness runtime: Bun 1.2 + Ink 5 (React-in-terminal)
- Model access: OpenRouter unified API with Claude Sonnet 4.7, GPT-5.4-Codex, Gemini 3 Pro, Opus 4.5 (for hardest tasks)
- Tool transport: Model Context Protocol StreamableHTTP (MCP 2026 revision)
- Sandbox: E2B sandboxes (JS SDK) or Daytona devcontainers
- Code search: ripgrep subprocess, tree-sitter parsers for 17 languages (pre-compiled)
- Isolation:
git worktree addper task, cleanup on success / failure - Eval harness: SWE-bench Pro (verified subset) + Terminal-Bench 2.0 + your own 30-task holdout
- Observability: OpenTelemetry SDK with
gen_ai.*semconv → self-hosted Langfuse - PR posting: GitHub App with fine-grained token, scope limited to the target repo
#Build It
TUI and command loop. Scaffold a Bun project with Ink. Accept
agent run <repo> "<task>". Print a split view: plan pane (top), tool-call stream (middle), token budget (bottom). Add cancel on Ctrl-C that firesSessionEndhook before exit.Plan state. Define a typed TodoWrite schema (pending / in_progress / done items with notes). Model rewrites the full state each turn as a tool call — do not let it mutate incrementally. Persist plan to
.agent/state.jsonso crashes can resume.Tool surface. Define six tools:
read_file,edit_file(with diff preview),ripgrep,tree_sitter_symbols,run_shell(with timeout),git(status / diff / commit / push). Expose over MCP StreamableHTTP so the harness is transport-agnostic. Every tool returns truncated output (cap at 4k tokens per call).Sandbox wrapping. Each task spawns an E2B sandbox.
git worktree add -b agent/$TASK_IDa fresh branch. All tool calls execute inside the sandbox. Host filesystem is unreachable.Hooks. Implement all eight 2026 hook types. Wire at least four user-authored hooks: (a)
PreToolUsedestructive-command guard that blocksrm -rfoutside the worktree, (b)PostToolUsetoken accounting, (c)SessionStartbudget initialization, (d)Stopwrites a final trace bundle.Eval loop. Clone a 30-issue subset of SWE-bench Pro Python. Run your harness against each. Compare to mini-swe-agent (the minimal baseline) on pass@1, turns-per-task, and $-per-task. Write the results to
eval/results.jsonl.Cost control. Hard cutoffs: 50 turns, 200k context, $5 per task.
PreCompacthook summarizes older turns into a prior-state block at the 150k mark, freeing room for new observations without losing the plan.PR posting. On success, the final step is
git push+ a GitHub API call that opens a PR with the plan and the diff summary in the body.
#Use It
$ agent run ./my-repo "Fix the race condition in worker.rs"
[plan] 1 locate worker.rs and enumerate mutex uses
2 identify shared state under contention
3 propose fix, verify tests
[tool] ripgrep mutex.*lock -t rust (44 matches, truncated)
[tool] read_file src/worker.rs 120..180
[tool] edit_file src/worker.rs (+8 -3)
[tool] run_shell cargo test worker:: (passed)
[plan] 1 done · 2 done · 3 done
[done] PR opened: #482 turns=9 tokens=38k cost=$0.41#Ship It
The deliverable skill lives in outputs/skill-terminal-coding-agent.md. Given a repo path and a task description, it runs the full plan-act-observe loop in a sandbox and returns a PR URL plus a trace bundle. The rubric for this capstone:
| Weight | Criterion | How it is measured |
|---|---|---|
| 25 | SWE-bench Pro pass@1 vs baseline | Your harness vs mini-swe-agent on 30 matched Python tasks |
| 20 | Architecture clarity | Plan/act/observe separation, hook surface, tool schema — reviewed against Live-SWE-agent layout |
| 20 | Safety | Sandbox escape tests, permission prompts, destructive-command guard passes red-team |
| 20 | Observability | Trace completeness (100% of tool calls spanned), token accounting per turn |
| 15 | Developer UX | Cold-start < 2s, crash recovery resumes plan, Ctrl-C cancels mid-tool cleanly |
| 100 |
#Exercises
Swap the backing model from Claude Sonnet 4.7 to Qwen3-Coder-30B served on vLLM. Compare pass@1 and $-per-task. Report where the open model underperforms.
Add a
reviewersub-agent that reads the diff before PR posting and can request a revision loop. Measure whether false-positive reviews drop SWE-bench pass rate below the single-agent baseline (hint: usually yes).Stress-test the sandbox: write a task that tries to
curlan external URL and a task that writes outside the worktree. Confirm both are blocked by the PreToolUse hook. Log the attempts.Implement
PreCompactsummarization with a smaller model (Haiku 4.5). Measure how much plan fidelity is lost at 3x compaction.Swap MCP StreamableHTTP transport for stdio. Benchmark cold-start and per-call latency. Pick a winner for local-only use.
#Key Terms
| Term | What people say | What it actually means |
|---|---|---|
| Harness | "The agent loop" | The code surrounding the model that dispatches tools, maintains plan state, and enforces budgets |
| Hook | "Agent event listener" | A user-authored script run on one of eight lifecycle events by the harness |
| Worktree | "Git sandbox" | A linked git checkout at a separate path; disposable without touching the main clone |
| TodoWrite | "Plan state" | A typed list of pending/in-progress/done items the model rewrites each turn |
| StreamableHTTP | "MCP transport" | 2026 MCP revision: long-lived HTTP connection with bidirectional streaming; replaces SSE |
| Token ceiling | "Context budget" | Per-turn or per-session cap on input+output tokens; triggers compaction or termination |
| pass@1 | "Single-attempt pass rate" | Fraction of SWE-bench tasks solved on the first run without retry or test-set peeking |
#Further Reading
- Claude Code documentation — reference harness from Anthropic
- Cursor 3 changelog — Agent Tabs and Composer 2 product notes
- mini-swe-agent — minimal baseline for SWE-bench harness comparison
- Live-SWE-agent — 79.2% SWE-bench Verified with Opus 4.5
- OpenCode — open harness, 112k stars
- SWE-bench Pro leaderboard — the evaluation this capstone targets
- Model Context Protocol 2026 roadmap — StreamableHTTP, capability metadata
- OpenTelemetry GenAI semantic conventions — span schema for tool calls and token usage
Run this lesson in the Code Lab
A real JupyterLab, with this lesson's code files (and a ready-to-run notebook). The Quick Lab opens instantly in your browser. The Cloud Lab adds a terminal, saved files and the full toolchain.
"""Terminal-native coding agent — minimal plan/act/observe loop scaffold.
The hard architectural primitive in a 2026 coding agent is not the model call
or any single tool. It is the plan-act-observe-recover loop with bounded
context, a structured plan state, a sandboxed tool dispatcher, and hook
callbacks at every lifecycle point. This file implements that loop end to end
in stdlib Python. The LLM is stubbed out with a deterministic script so the
loop logic stays observable and testable without network calls.
Run: python main.py
"""
from __future__ import annotations
import json
import os
import subprocess
import sys
import time
from dataclasses import asdict, dataclass, field
from typing import Any, Callable
# ---------------------------------------------------------------------------
# plan state -- TodoWrite shape, rewritten whole each turn
# ---------------------------------------------------------------------------
@dataclass
class TodoItem:
id: int
description: str
status: str # "pending" | "in_progress" | "done" | "failed"
note: str = ""
@dataclass
class PlanState:
goal: str
items: list[TodoItem] = field(default_factory=list)
def summary(self) -> str:
lines = [f"GOAL: {self.goal}"]
for it in self.items:
mark = {"pending": " ", "in_progress": ">", "done": "x", "failed": "!"}[it.status]
lines.append(f" [{mark}] {it.id}. {it.description}")
return "\n".join(lines)
# ---------------------------------------------------------------------------
# budget -- hard ceilings on turns, tokens, dollars
# ---------------------------------------------------------------------------
@dataclass
class Budget:
max_turns: int = 50
max_tokens: int = 200_000
max_dollars: float = 5.00
turns_used: int = 0
tokens_used: int = 0
dollars_used: float = 0.0
def step(self, tokens: int, dollars: float) -> None:
self.turns_used += 1
self.tokens_used += tokens
self.dollars_used += dollars
def exceeded(self) -> str | None:
if self.turns_used >= self.max_turns:
return "turn_limit"
if self.tokens_used >= self.max_tokens:
return "token_limit"
if self.dollars_used >= self.max_dollars:
return "dollar_limit"
return None
# ---------------------------------------------------------------------------
# hooks -- 2026 eight-event surface (Pre/PostToolUse, SessionStart/End, etc)
# ---------------------------------------------------------------------------
HookFn = Callable[[dict[str, Any]], dict[str, Any]]
class HookBus:
EVENTS = ("SessionStart", "SessionEnd", "PreToolUse", "PostToolUse",
"UserPromptSubmit", "Notification", "Stop", "PreCompact")
def __init__(self) -> None:
self._hooks: dict[str, list[HookFn]] = {e: [] for e in self.EVENTS}
def on(self, event: str, fn: HookFn) -> None:
self._hooks[event].append(fn)
def fire(self, event: str, payload: dict[str, Any]) -> dict[str, Any]:
for fn in self._hooks[event]:
payload = fn(payload) or payload
return payload
# ---------------------------------------------------------------------------
# tool surface -- six tools, each sandboxed, each returns truncated text
# ---------------------------------------------------------------------------
TRUNCATE_BYTES = 4096
def tool_read_file(sandbox: str, path: str) -> str:
full = os.path.join(sandbox, path)
if not os.path.realpath(full).startswith(os.path.realpath(sandbox)):
raise RuntimeError("path escapes sandbox")
with open(full, "r", encoding="utf-8", errors="replace") as fh:
return fh.read()[:TRUNCATE_BYTES]
def tool_run_shell(sandbox: str, cmd: str, timeout: int = 30) -> str:
proc = subprocess.run(cmd, cwd=sandbox, shell=True, capture_output=True,
text=True, timeout=timeout)
out = (proc.stdout + proc.stderr)[:TRUNCATE_BYTES]
return f"exit={proc.returncode}\n{out}"
TOOLS: dict[str, Callable[..., str]] = {
"read_file": tool_read_file,
"run_shell": tool_run_shell,
}
# ---------------------------------------------------------------------------
# stub model -- deterministic script so loop is testable without LLM
# ---------------------------------------------------------------------------
SCRIPT = [
{"plan": [("locate target file", "in_progress"),
("read and diagnose", "pending"),
("apply fix and verify", "pending")],
"tool": ("run_shell", {"cmd": "ls"}),
"tokens": 1200, "cost": 0.02},
{"plan": [("locate target file", "done"),
("read and diagnose", "in_progress"),
("apply fix and verify", "pending")],
"tool": ("read_file", {"path": "README.md"}),
"tokens": 900, "cost": 0.02},
{"plan": [("locate target file", "done"),
("read and diagnose", "done"),
("apply fix and verify", "done")],
"tool": None, # terminal turn
"tokens": 600, "cost": 0.01},
]
def model_step(plan: PlanState, turn: int) -> dict[str, Any]:
"""Stubbed model: returns a plan rewrite and (optionally) a tool call."""
if turn >= len(SCRIPT):
return {"plan": plan.items, "tool": None, "tokens": 200, "cost": 0.005}
s = SCRIPT[turn]
items = [TodoItem(i + 1, desc, status) for i, (desc, status) in enumerate(s["plan"])]
return {"plan": items, "tool": s["tool"], "tokens": s["tokens"], "cost": s["cost"]}
# ---------------------------------------------------------------------------
# main loop -- plan / act / observe / recover with full hook integration
# ---------------------------------------------------------------------------
def destructive_guard(payload: dict[str, Any]) -> dict[str, Any]:
cmd = payload.get("args", {}).get("cmd", "")
if "rm -rf" in cmd or "shutdown" in cmd:
payload["blocked"] = True
payload["reason"] = "destructive command blocked by PreToolUse hook"
return payload
def run_agent(task: str, sandbox: str) -> dict[str, Any]:
plan = PlanState(goal=task, items=[])
budget = Budget()
hooks = HookBus()
trace: list[dict[str, Any]] = []
hooks.on("PreToolUse", destructive_guard)
hooks.on("PostToolUse", lambda p: (trace.append({"event": "tool", **p}), p)[1])
hooks.on("SessionStart", lambda p: (trace.append({"event": "start", **p}), p)[1])
hooks.on("SessionEnd", lambda p: (trace.append({"event": "end", **p}), p)[1])
hooks.fire("SessionStart", {"task": task, "sandbox": sandbox,
"started_at": time.time()})
turn = 0
while True:
stop = budget.exceeded()
if stop:
hooks.fire("Stop", {"reason": stop, "turn": turn})
break
step = model_step(plan, turn)
plan.items = step["plan"]
budget.step(step["tokens"], step["cost"])
call = step["tool"]
if call is None:
hooks.fire("Stop", {"reason": "complete", "turn": turn})
break
name, args = call
pre = hooks.fire("PreToolUse", {"tool": name, "args": args})
if pre.get("blocked"):
hooks.fire("PostToolUse", {"tool": name, "blocked": True,
"reason": pre.get("reason", "")})
turn += 1
continue
try:
result = TOOLS[name](sandbox, **args)
hooks.fire("PostToolUse", {"tool": name, "ok": True,
"bytes": len(result)})
except Exception as exc:
hooks.fire("PostToolUse", {"tool": name, "ok": False,
"error": str(exc)})
turn += 1
hooks.fire("SessionEnd", {"turns": budget.turns_used,
"tokens": budget.tokens_used,
"dollars": budget.dollars_used})
return {"plan": plan.summary(), "budget": asdict(budget), "trace": trace}
def main() -> None:
task = "demonstrate the plan-act-observe loop without network calls"
sandbox = os.path.dirname(os.path.abspath(__file__))
result = run_agent(task, sandbox)
print(result["plan"])
print("---")
print(f"turns={result['budget']['turns_used']} "
f"tokens={result['budget']['tokens_used']} "
f"dollars=${result['budget']['dollars_used']:.3f}")
print("---")
print(f"trace events: {len(result['trace'])}")
for ev in result["trace"]:
print(" ", json.dumps(ev, default=str))
if __name__ == "__main__":
main()
{
"name": "phase-19-01-terminal-native-coding-agent",
"version": "0.1.0",
"private": true,
"type": "module",
"scripts": {
"start": "tsx src/index.ts",
"test": "tsx --test tests/*.test.ts",
"typecheck": "tsc --noEmit"
},
"dependencies": {
"zod": "^3.23.8"
},
"devDependencies": {
"@types/node": "^20.17.10",
"tsx": "^4.19.2",
"typescript": "^5.7.3"
}
}
# Capstone 19/01 — Terminal-Native Coding Agent (TypeScript)
Multi-file TypeScript harness for the plan/act/observe loop described in
`../docs/en.md`. Offline, deterministic, zero network calls.
## Layout
```text
src/
index.ts entry point; runs a scripted demo and the eval, then exits 0
repl.ts interactive command parser (run / eval / help / quit)
harness.ts the plan-act-observe loop, wired through the hook bus
hooks.ts eight-event hook bus plus a destructive-command guard
model.ts scripted offline LLM that drives the demo
tools.ts read_file + run_shell with zod-validated args
plan.ts PlanState (todo rewrite) + Budget (turn / token / dollar ceilings)
eval.ts tiny pass/fail counter across three offline tasks
types.ts shared shape definitions
tests/
harness.test.ts
tools.test.ts
```
## Run
```bash
npm install
npm start # runs the scripted demo + offline eval, exits 0
npm start -- --repl # opens the interactive harness REPL
npm test # node --test runner via tsx
npm run typecheck # tsc --noEmit
```
The non-interactive `npm start` path asserts that the eval reports `passed=3
failed=0` and that the scripted run converges to an all-done plan. Any drift
fails the run.
{
"compilerOptions": {
"target": "ES2022",
"module": "nodenext",
"moduleResolution": "nodenext",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"resolveJsonModule": true,
"forceConsistentCasingInFileNames": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noImplicitOverride": true,
"noFallthroughCasesInSwitch": true,
"allowImportingTsExtensions": true,
"noEmit": true,
"isolatedModules": true,
"verbatimModuleSyntax": false
},
"include": ["src/**/*.ts", "tests/**/*.ts"]
}
6 questions on this lesson
Answer each one and see straight away whether you were right and why. Your first answer to a question counts. You are not signed in, so this is practice and nothing is saved.
Reusable outputs from this lesson. Copy them into your own work.
---
name: terminal-coding-agent
description: Build and evaluate a terminal-native coding agent against SWE-bench Pro with bounded cost, sandboxed tools, and full 2026 hook surface.
version: 1.0.0
phase: 19
lesson: 01
tags: [capstone, coding-agent, claude-code, swe-bench, mcp, hooks, sandbox]
---
Given a target repository and a natural-language task, build a harness that plans, executes in a sandbox, and opens a pull request. Match or beat the mini-swe-agent baseline on a 30-task SWE-bench Pro subset while staying under a $5-per-task budget.
Build plan:
1. Stand up a Bun + Ink TUI harness with a plan pane, a tool-call stream, and a live token/dollar budget.
2. Define six tools (read_file, edit_file, ripgrep, tree_sitter_symbols, run_shell, git) over Model Context Protocol StreamableHTTP. Every call returns at most 4k tokens.
3. Run every tool call inside an E2B or Daytona sandbox on a fresh `git worktree add` branch. Never touch the host filesystem.
4. Wire all eight 2026 hook events: SessionStart, SessionEnd, PreToolUse, PostToolUse, UserPromptSubmit, Notification, Stop, PreCompact. Ship at least four user-authored hooks (destructive-command guard, token accounting, OTel span emitter, trace bundle writer).
5. Enforce three budgets: 50 turns, 200k tokens, $5 dollars. PreCompact fires at 150k and summarizes older turns.
6. Emit OpenTelemetry spans with GenAI semantic conventions to a self-hosted Langfuse.
7. On success, push the branch and open a PR with the plan and trace bundle in the body.
8. Evaluate against mini-swe-agent on a 30-issue SWE-bench Pro Python subset and record pass@1, turns, tokens, and dollars per task.
Assessment rubric:
| Weight | Criterion | Measurement |
|:-:|---|---|
| 25 | SWE-bench Pro pass@1 | Matched 30-task subset vs mini-swe-agent baseline |
| 20 | Architecture clarity | Plan/act/observe separation, hook surface, tool schema readability |
| 20 | Safety | Sandbox escape red-team + destructive-command guard audit |
| 20 | Observability | 100% of tool calls spanned, token accounting per turn |
| 15 | Developer UX | Cold-start under 2s, crash recovery, Ctrl-C cancel semantics |
Hard rejects:
- Harness that shells out to git on the host filesystem instead of inside the sandbox.
- Any agent that can write outside the worktree or curl external URLs without an explicit allowlist hook.
- Eval numbers reported without a matched baseline run on the same 30 issues.
- "Pass rate" claims that depend on `git reset --hard` between retries; SWE-bench Pro is pass@1.
Refusal rules:
- Refuse to push directly to main under any configuration. PR branches only.
- Refuse to disable the destructive-command guard. It is a hard requirement of the rubric.
- Refuse to run without a budget ceiling. Open-ended runs contaminate the eval comparison.
Output: a repo containing the harness, a fixed 30-task SWE-bench Pro eval harness with matched mini-swe-agent baseline run, an OpenTelemetry trace archive for at least 5 full runs, and a write-up naming which tasks the harness solves that the baseline does not and vice versa. End with a section on the top three failure modes you observed and the hook change that fixed each.
The text and code of this lesson are based on AI Engineering from Scratch by Rohit Ghumare, MIT licensed.