AI Engineering/Phase 19/Lesson 28

Capstone Lesson 28: Observability with OTel GenAI Spans and Prometheus Metrics

An agent harness without observability is a black box that costs money. This lesson hand-rolls a span builder that emits records compliant with the OpenTelemetry GenAI semantic conventions, writes them to a JSON-Lines file one span per line, and exposes counters and histograms in Prometheus text format. The whole thing is stdlib Python and runs offline.

Build 1.5 hr Python (stdlib)

Prerequisites: Phase 19 · 25 (verification gates), Phase 19 · 26 (sandbox), Phase 19 · 27 (eval harness), Phase 13 · 20 (OpenTelemetry GenAI), Phase 14 · 23 (OTel GenAI conventions)

#Learning Objectives

Build a span data class shaped to the OpenTelemetry GenAI semantic conventions.
Implement a JSONL exporter that writes one self-contained span per line.
Build counters and histograms with labels and Prometheus text-format exposition.
Wrap any callable in a span context manager that records duration, status, and exceptions.
Verify that the emitted spans roundtrip through json.loads and match the spec shape.

Full Access lesson

Keep going. The rest of this lesson is one step away.

Unlock all 503 lessons, video, code, quizzes and every future update. Pay in your own currency with Paystack.

See plans Sign in

The text and code of this lesson are based on AI Engineering from Scratch by Rohit Ghumare, MIT licensed.