Companion code: 14-meta-agent

TL;DR

• A family of systems improves agents automatically with one loop: keep an archive of candidates, pick one, have a model propose a change, score the change automatically, keep the good ones. The candidates can be agent designs (ADAS), an agent’s own code (DGM), prompts (GEPA) or algorithms (AlphaEvolve).
• It works where there’s a fast, trustworthy, automatic evaluator. The evaluator is what all the gains are measured against, and what all the risk depends on.
• It’s noisier than the headlines suggest. A 2026 re-evaluation found results swing with random variation and even with the order of tasks. Use held-out tests, several runs, and skepticism.

The idea: let the model do the design work

Every post in this series so far has been about choices a person makes: the loop, the workflow pattern, the context strategy, the tools, the topology. A line of research since 2024 asks the obvious next question. If models can write good code, and agents are code, why not let a model design the agent?

Four systems, one loop

ADAS: designing agents in code (2024). Automated Design of Agentic Systems (Hu, Lu, Clune) pointed out that agents built from prompts, tools and control flow can all be written as code. Code can express any agent design, so a “meta agent” can search over agents by programming them. Its algorithm, Meta Agent Search, keeps an archive of discovered agents and repeatedly asks a model to write an interesting new one, informed by what’s already in the archive. The discovered agents beat state-of-the-art hand-designed ones across coding, science and maths, and, notably, kept their edge when transferred to other domains and models.

Darwin Gödel Machine: an agent that edits itself (2025). The DGM (Zhang, Hu et al.) made the candidate the coding agent itself, including the code it uses to improve itself. It keeps an archive of agent versions, samples one, has a model propose a modified version, and keeps changes that score well on coding benchmarks. Over many iterations it discovered its own improvements, such as better file-editing tools, context-window management and a peer-review step, and its SWE-bench score rose from 20.0% to 50.0% (Polyglot: 14.2% to 30.7%). Two design choices mattered: an open-ended archive rather than keeping only the current best, and empirical validation of every change instead of trying to prove it helps. The authors ran it with sandboxing and human oversight.

GEPA: evolving prompts by reflecting in words (2025). GEPA (Agrawal et al.) applies the loop to the prompts inside an AI system. Its insight is about the learning signal. Reinforcement learning learns from a single number per attempt, a reward. GEPA has a model read what happened (the reasoning, the tool calls, the errors) and diagnose in plain language what a prompt is missing. It then keeps a Pareto front: every candidate that’s the best on at least one example, so different partial solutions survive and can be combined. Across six tasks it beat the reinforcement-learning method GRPO by 6% on average, and by up to 20%, while using up to 35 times fewer rollouts.

AlphaEvolve: evolving algorithms (2025). Google DeepMind’s AlphaEvolve applies the loop to programs that solve hard problems, with automated evaluators that check correctness and measure quality. Inside Google it found a more efficient data-centre scheduling heuristic, simplified a hardware accelerator circuit, and sped up the training of the model AlphaEvolve itself runs on. In mathematics it found a way to multiply two 4×4 complex-valued matrices with 48 scalar multiplications, the first improvement in that setting in 56 years over the approach that follows from Strassen’s 1969 algorithm.

System	What evolves	What scores it
ADAS (2024)	agent designs, as code	task benchmarks
DGM (2025)	a coding agent’s own code	coding benchmarks
GEPA (2025)	the prompts in a system	task metrics, plus reflection on traces
AlphaEvolve (2025)	algorithms and programs	automated correctness and quality checks

The common ingredient: an evaluator you can trust

Look at the right-hand column. Every system depends on an automatic, fast, trustworthy evaluator. That’s no accident: the loop optimizes whatever the evaluator rewards, thousands of times, without a person in between. It’s the same lesson as the C compiler in post #7 (“the agent solves whatever problem the tests define”) and the evals in post #12. It’s why AlphaEvolve’s successes are in mathematics and infrastructure, where correctness can be checked by machine, and why self-improvement on fuzzy goals like “be a better assistant” is much harder.

It also suggests where the risk is. A loop that maximizes a flawed evaluator will find the flaw. The DGM authors reported instances of their system gaming its own evaluation. Sandboxing, human review of what changed, and held-out tests aren’t optional extras here.

The 2026 reality check

In August 2026, On the Fragility of Self-Improving Agents (Ye, Li et al.) re-ran two published memory-based self-improving methods (the kind from post #9) with a broader evaluation: several independent runs, and the same tasks in shuffled order. Two findings stand out.

• Noise compounds. Agent evaluation is already noisy (post #12), and stacking a self-improvement loop on top amplified the noise. A single run can show an improvement that isn’t there.
• Order matters. The improvement depended heavily on the order tasks were presented in. The default orderings in earlier work had acted as a hidden curriculum.

The authors traced part of the problem to underspecification: tasks and environments that don’t make clear what “good” means. Adding detailed rubrics and environment feedback when building memory partly closed the gap. That’s the evaluator lesson again, from the other side.

The experiment

The companion code (14-meta-agent) is a tiny GEPA-style loop. The “agent” is a single prompt that sorts support tickets into five categories. The labels follow house rules the starting prompt doesn’t mention: a suspicious login is security even if it mentions a charge, and a plan change is account, not billing.

Each round picks a parent from the Pareto front, runs it on twelve training tickets, and asks a model to reflect on the failures:

"Diagnose what general rule the instruction is missing, then write an improved instruction. State rules "
"that generalize to new tickets; do not list or quote these specific tickets. Keep the same output format."

At the end, the best evolved prompt and the original are both scored on twelve held-out tickets the loop never saw, exercising the same rules with different wording. The held-out score is the only number that matters. A prompt that scores perfectly on training by quoting training tickets has learned nothing.

We ran it once against Claude Opus 5.5. The one-line seed prompt scored 10/12 on training and 10/12 on the held-out test. A single round of reflection read its two training failures and diagnosed the problem (“the instruction gives only category names and no definitions”). It then wrote a prompt with definitions and precedence rules, which scored 12/12 on training and 12/12 on the held-out test, so the loop stopped at round 2. The evolved prompt learned the house rules as rules, not examples:

- Classify by the main action the customer wants, not by topics mentioned in passing.
  Money being indirectly involved does not make a ticket billing.
- If the customer wants to change their plan, seats, users, or account configuration, choose account.
- Security takes precedence when the ticket involves possible compromise or unauthorized access.

That’s a nice result, and also a small one: two tickets of difference on a twelve-ticket test is within the noise post #12 warns about. Run several seeds, read the evolved prompt, and check that it stated rules, not examples. In other words, apply the fragility paper’s advice to your own result.

Trade-offs

• The evaluator is the ceiling. These loops are only as good as what they optimize. Invest in the evaluator before the loop.
• Search is expensive. Each candidate needs a full evaluation. GEPA’s sample efficiency is a big part of why it matters, and on a tight budget, reflection on a few rich failures beats scoring many blind variants.
• Self-modification needs a sandbox and a human. Systems that change their own code or prompts should be contained (post #10), and every adopted change should be reviewable and reversible.

Next in the series: Watching the agents: identity, logs and visibility.

Sources & further reading

• Hu, Lu, Clune, Automated Design of Agentic Systems (2024).
• Zhang, Hu et al., Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents (2025).
• Agrawal et al., GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning (2025).
• Novikov et al. (Google DeepMind), AlphaEvolve: A coding agent for scientific and algorithmic discovery (2025).
• Ye, Li et al., On the Fragility of Self-Improving Agents: Variance, Task Order, and Underspecification (2026).

#self-improvement #research #optimization