The optimization lecture begins with a prerequisite rather than an algorithm: you need task data and an evaluation that reflects the behavior you want. Without them, prompt edits and training runs are guesses. With them, an agent’s recurring failures become material for systematic improvement.
Key insights #
- Optimization becomes useful when representative data and a credible metric expose recurring, domain-specific errors.
- Context optimization searches instructions and demonstrations; weight optimization changes the model’s learned behavior.
- Rich feedback from full trajectories can guide prompt revision more efficiently than a scalar score alone.
- Supervised fine-tuning, RL, and on-policy distillation use different sources of training signal and have different cost and verifier requirements.
- Agent-driven experimentation can optimize code and harnesses, but it will also exploit a weak metric.
Why optimize a strong model at all? #
General models are trained for broad usefulness. A specific application has its own vocabulary, output contract, cost target, and failure cases. The lecture calls this last-mile adaptation. Examples and feedback can communicate requirements that are hard to express fully in a prompt, especially in a multi-step system. Optimization may improve reliability, let a smaller model do the job, or reduce tool calls and latency.
The video opens with a prompt-choice poll to make a simple point: the best wording or example ordering is difficult to predict by intuition. That motivates an evaluation-driven search rather than endless manual tweaking.
A held-out evaluation is essential because searching over prompts can overfit a small training set.
Context optimization: change what the model sees #

Source: CS329Z Lecture 9, PDF page 26; figure from the original paper.
Few-shot examples have more structure than their count suggests. The order of examples can change performance. For reasoning tasks, labeled inputs and answers may lack useful intermediate steps, so STaR and DSPy-style bootstrapping sample reasoning and keep traces that lead to correct outcomes. The optimizer can then search over which demonstrations to include, rather than accepting the first successful trace.
The first family of methods changes instructions and demonstrations. Few-shot optimization selects useful examples; bootstrapping can generate reasoning traces for examples that originally have only answers. MIPROv2, from the assigned paper, proposes instructions grounded in the program and data, collects successful demonstrations, and searches combinations against a downstream metric. The lecture’s iris-classification walkthrough makes this tangible: the optimizer is tuning a complete module’s prompt behavior, not polishing one sentence in isolation.
MIPROv2’s three stages are worth keeping distinct: collect successful traces, propose instructions using program and data context, then evaluate combinations of instructions and demonstrations. The lecture’s iris example walks through proposal grounding and repeated minibatch evaluation. The point is to optimize the whole program’s downstream behavior; a locally impressive instruction can be poor when paired with another module’s prompt.

Source: CS329Z Lecture 9, PDF page 65; figure from the original paper.
The lecture then introduces reflective methods. TextGrad propagates natural-language feedback about failures through a model program. GEPA inspects trajectories, proposes prompt revisions, tests them, and keeps complementary improvements on a Pareto frontier. The important distinction is the richness of feedback: a trace can say why an attempt failed, whereas a scalar reward only says how it scored.
GEPA takes a more reflective route. It observes a failed trajectory, writes a diagnosis in language, proposes a revision, and measures the revision. Its Pareto frontier preserves candidates that solve different parts of the task rather than collapsing all progress into one prompt too early. TextGrad similarly sends natural-language feedback backward through a multi-stage program. These methods rely on informative traces, not merely final pass/fail labels.
Weight optimization: change what the model has learned #

Source: CS329Z Lecture 9, PDF page 81.
Supervised fine-tuning can train a student on successful agent rollouts from a teacher. This is attractive when a capable teacher is expensive but repeated production work can be handled by a smaller model. Reinforcement learning instead rolls out the student and rewards verified task completion. The lecture walks from PPO to GRPO using DeepSeekMath as an assigned foundation for the group-relative idea. Verifiability is the limiting condition: if success cannot be scored, optimizing the reward can teach the wrong behavior.
On-policy distillation is presented as another route: let the student generate its own trajectories, then use a stronger teacher to provide learning signal on those trajectories. This addresses a mismatch that can arise when a student only imitates teacher-generated examples.
The slides give concrete supervised examples: FireAct and OpenThoughts-Agent train on trajectories from stronger teachers. Diversity across tasks and environments matters so a student does not memorize one narrow path. For RL, the student must try actions itself and receive a reward tied to verified completion. GRPO compares rewards within a group of sampled outputs to form a relative advantage, reducing reliance on a separate value model compared with PPO. A poor reward or exploitable environment can still teach the wrong policy.
The lecture’s chess-coaching analogy explains on-policy distillation: feedback on a game the student actually played is more targeted than copying someone else’s game. A student generates its own trajectories, while a stronger teacher supplies token-level learning signal on those trajectories. The slide results shown are from a particular study; the general lesson is to match training signal to the student’s actual errors.
Agents can also search the design space #
The final section considers agents that modify programs or experiments themselves, including the Darwin Gödel Machine and autoresearch. The core loop is familiar: propose a change, run a measurable test, keep improvements, and preserve a record of what was tried. The difference is that the object being optimized may be an agent’s own code or research workflow.
The lecture’s decision rule is pragmatic. Try context optimization when model weights are inaccessible, changes must be frequent, or rollouts are expensive. Consider weight optimization when prompts become bloated, a small model needs a learned capability, or a teacher can produce valuable training signal. Either way, hold out evaluation tasks and track cost alongside success. An optimizer will exploit the metric you give it; the engineering work is making that metric worth optimizing.
The lecture also mentions sequential improvement—optimize prompts, then weights, then prompts again—when one pass does not capture the whole system.
Further reading #
- Lecture 9 slides and video with captions
- Agrawal et al., GEPA
- Opsahl-Ong et al., Optimizing Instructions and Demonstrations
- Shao et al., DeepSeekMath