↓ Skip to main content
  1. Archive/

CS329Z Lecture 5: Frameworks, Harnesses, and Durable Agents

Author
AI Assistant
Curated for Pradithya Aria Pura · AI-written summary of the source material
Stanford CS329Z: Engineering AI Agents - This article is part of a series.
Part : This Article

“Which agent framework should I use?” is too broad a question. In CS329Z’s fifth lecture, the frameworks solve different parts of the engineering problem. DSPy describes and optimizes model programs. LangGraph makes state and control flow explicit. Agent harnesses wrap a model with tools, prompts, and execution behavior. Temporal addresses durable execution. Tracing and eval tools make the resulting system observable.

Key insights
#

  1. Data, system design, and evaluation must be developed together; a framework supplies only some of that work.
  2. DSPy, LangGraph, harnesses, Temporal, and tracing tools occupy different layers of an agent stack.
  3. A harness can materially alter success, cost, and latency even when the model stays the same.
  4. Long-running agents need durable execution and visible failure paths, not only a prompt and an LLM loop.

Start with data, system, and metric
#

DSPy signatures, modules, and optimizers

Source: CS329Z Lecture 5, PDF page 22.

The lecture returns to three pillars: data, the system or model, and evaluation with a metric. A framework can make the system easier to express, but it cannot supply representative tasks or a good measure of success. This point matters because much of the lecture is a live demo: a program that extracts event details from an email and then updates a calendar. The transcript shows how small specification choices—such as what an output field means when an email contains two dates—become real implementation questions.

The DSPy portion introduces signatures for input/output contracts, modules for composing calls and logic, and optimizers for improving prompts or weights against examples and a metric. The assigned DSPy paper argues for programs whose model calls can be optimized, instead of hand-tuned prompt strings scattered through code. This is most valuable when you can state what better output looks like and have data to test it.

The calendar demo combines extraction of event fields with a second choice, such as selecting an emoji for a calendar event. The speaker builds a signature, composes model calls as modules, then measures performance against examples. A question raised during the demo is revealing: a variable name and docstring may be too vague when an email contains multiple candidate dates. A framework makes a contract easy to express; the developer still has to define it.

A signature says what transformation is expected. A module says how to attempt it—direct prediction, chain of thought, or a composition of calls. An optimizer searches demonstrations or instructions that raise the downstream score. Prompt behavior becomes a parameter of the program rather than an incidental string literal.

Explicit state changes how you reason about a workflow
#

LangGraph agent loop showing shared state, model, tools, start, and end

Source: CS329Z Lecture 5, PDF page 58.

The LangGraph demo builds around shared state, nodes that read and write it, and edges that determine what happens next. Conditional edges provide branching; a loop can route between an LLM and tools until it is done. LangGraph’s overview gives the same core model. The benefit is not a magical increase in model intelligence. It is that control flow and state transitions become inspectable and testable.

For the calendar example, state can hold the request, extracted details, a proposed action, and tool results. A node updates that state; an edge chooses the next node. In a fixed workflow the graph encodes the order. In an agent loop, a conditional edge can return from a tool result to the LLM until it decides the task is finished.

The lecture then distinguishes a graph framework from an agent harness. A harness packages the operational decisions around the core loop: which tools are present, how context is assembled, how actions execute, and how observations return. The tour of mini-SWE-agent, OpenHands, Pi, and OpenClaw shows that these choices differ even when the underlying LLM is the same. Arena’s HarnessTax is assigned to emphasize that harness design can materially change coding-agent results.

The slides give concrete harness contrasts. Mini-SWE-agent is intentionally spare: a bash tool, linear message history, and independent subprocess calls. OpenHands stores events on disk and rebuilds model messages, while a persistent terminal preserves shell state. Pi exposes read, write, edit, and bash tools and stores history as a branching tree. OpenClaw adds scheduled heartbeat turns and persistent identity files. Each design changes what the agent remembers, how it acts, and what it can recover from.

A long-running agent needs a runtime
#

Temporal execution model with activities, workflows, task queues, workers, and storage

Source: CS329Z Lecture 5, PDF page 98.

Once an agent runs long enough to hit timeouts, flaky services, or process restarts, orchestration becomes a separate design problem. The Temporal section models fallible work as activities inside a durable workflow. The demo adds retries and server-based timing rather than assuming one process will stay alive. This is distinct from deciding what the agent should do; it is about keeping already-decided work recoverable.

In the Temporal model, activities are the units that can fail; workflows connect them. A server keeps execution history and task queues, while workers execute tasks. The lecture’s retry demo shows a flaky step being retried without losing the entire calendar request. A side-effecting operation still needs idempotency so a retry does not duplicate the calendar event.

The observability tour includes MLflow, LangSmith, and Langfuse. Their common job is to make trajectories visible so a developer can locate the failing component. The lecture warns that abstractions can hide details: after a framework change, inspect messages, state transitions, and tool payloads before blaming the model.

The closing guidance is refreshingly concrete: trace model calls, tool calls, state changes, and errors; set timeouts, retry limits, and budgets; run evals on representative tasks and edge cases. Start with the simplest system that meets the task. A framework is useful when it makes these concerns easier to handle, and its abstraction should be judged by that result.

Further reading
#

Stanford CS329Z: Engineering AI Agents - This article is part of a series.
Part : This Article

Related