Summary: This week's ten papers split into two themes. Some make agents cheaper to run: smaller KV caches, cheaper judges, cheaper self improvement. Others expose new failure modes: lost state, sycophancy under a wrong hint. The common lesson is that scaffolding around the model is where the easy gains are.
How to read this issue
- Start with the table below to pick papers.
- Each paper has a card: problem, method, result, why it matters.
- Numbers come from the source roundup. I rewrote all explanations myself.
- I did not have verified arXiv links, so papers are named for searching, not linked.
Fig. 1: The ten papers grouped by theme.
The ten papers at a glance
| # | Paper | One line | Headline number |
|---|---|---|---|
| 1 | HySparse2 | Attention design behind MiMo-V3 that cuts prefill cost and KV cache size | 5.02x less prefill compute, 2.69 GB vs 12.09 GB cache at 1M tokens |
| 2 | SIFT | Judge candidate self changes before benchmarking them | About 10x cheaper than the Darwin Godel Machine baseline |
| 3 | GAVEL | Graph world model repairs robot plans before they run | 41.2% to 91.8% success, no weight changes |
| 4 | WFM | Retrieval model for wiki style agent memory | 10.5x faster distributed training |
| 5 | JEV as a Judge | Decision only judge handles most evals, escalates the rest | 277x cheaper than GPT-6 as judge |
| 6 | Harness-Zero | Distill a specialized harness into the model | 44.3% with no harness vs 41.7% with it |
| 7 | Self-Organizing Teams | A model team learns its own collaboration strategy | 66.7% vs 59.0% for a perfect router |
| 8 | ScientistTwo | Multi agent system runs the full research cycle | Beats human baselines on ICLR, ICML, NeurIPS problems |
| 9 | XYEval | Measures how often agents follow a confident wrong hint | Up to 46.7% relative score drop |
| 10 | EvoOntology | Data agents use a living, tested ontology | +17.8 points DDR-Bench, +7.4 BIRD |
1. HySparse2
By: Xiaomi MiMo team. Where: the attention design behind the upcoming MiMo-V3.
Problem
- Agents call tools, get long results back, and must prefill them again every turn.
- Prefill cost and KV cache size both grow with every turn.
Method
- Split model: a self decoder and a cross decoder, as in YOCO.
- KV Bridging: the cross decoder builds its full attention cache from the self decoder's hidden states.
- KV Reuse: each sparse layer reuses the cache and token selection of the full attention layer before it.
- Token level selection replaces block level selection, and a forced recent window replaces the separate sliding window branch.
- Early exit: prefill can stop once the self decoder finishes, since the cross decoder cache comes from it.
Result
- Measured on 80B total, 3B active MoE models trained on the same data.
- After light post training: +11.30 on MRCR-v2 and +19.81 on RULER-v2 over HySparse.
- Lower AgentPPL and LongPPL than both baselines up to 256k tokens.
Why it matters: agent workloads repeat prefill on growing context. This design cuts both cost and memory without losing retrieval quality.
2. SIFT
By: MIT and Sakana AI. Builds on: the Darwin Godel Machine (DGM).
Problem
- Self modifying coding agents score every candidate by rerunning benchmarks.
- That evaluation is where almost all the cost goes.
Method
- An LLM judge compares candidate patches in pairs.
- A regularized Bradley-Terry model turns wins and losses into strength scores.
- Scores guide parent sampling in a disaggregated tree search.
- Full benchmark runs happen only for the most promising nodes.
Result
- o3-mini config: under 50 CPU hours, under 5 hours wall clock.
- Qwen3-30B config: 224 CPU hours, about a tenth of the DGM baseline.
- Judge quality matters: on TerminalBench, gpt-5.4-high as judge finds a 36.7% agent from 29.2%. gpt-5 finds 34.5% and ranks a weaker candidate first.
Why it matters: if you run a self improvement loop, the judge needs as much scrutiny as the search. A bad judge misranks candidates.
3. GAVEL
Problem
- LLM plans for long robot tasks break embodiment limits.
- They lose track of objects out of view and cannot recover from mistakes.
Method
- Add an explicit graph world model around the LLM.
- The graph holds object relations, action preconditions and effects, and probabilistic beliefs about unseen objects.
- Before an action runs, the graph predicts its effect and catches violations.
- Fixes that follow from the model are applied without calling the LLM. Only semantic errors go back for replanning.
Result
- Qwen3-8B on BEHAVIOR-1K: single task 41.2% to 91.8% across 100 tasks.
- Multi task: 19.9% to 92.6% across 500 instructions.
- Reasoning over object location distributions reorders subtasks and cuts travel by about 5.4%.
Why it matters: many long horizon failures are state tracking failures, not reasoning failures. This applies to software agents tracking files, tickets or accounts.
4. WFM (Wiki Foundation Model)
Problem
- Many agents store memory as linked markdown pages, an "LLM wiki".
- Plain RAG drops the links. Sparse graph embeddings drop most of the text.
Method
- A Wiki Graph schema keeps entity relations and full passage text in one graph.
- Retrieval runs query conditioned message passing with attentive aggregation.
- A regularizer on attention variance stops one neighbor from dominating.
- A GPU to GPU exchange protocol over NCCL avoids CPU serialization when the graph is split.
Why it matters: if your agent memory is already a folder of linked markdown, this retrieval model matches that shape instead of adapting plain RAG.
5. JEV as a Judge
| Judge | Cost per 1,000 judgments | Median latency |
|---|---|---|
| GPT-6 (generative) | $12.182 | 1.885 s |
| JEV (decision only) | $0.044 | 0.152 s |
Method
- JEV, from TypeSafe AI, returns a verdict and label probabilities with no reasoning text.
- A cascade accepts confident JEV verdicts and escalates uncertain ones to a frontier model.
Result
- Within 3 points of GPT-6 on preference and factuality: 92.2% vs 93.5% on RewardBench, 87.5% vs 86.7% on HaluEval.
- Gap of 9 to 20 points on hard checks such as derivations: 78.6% vs 93.1% on JudgeBench.
- Those errors cluster in JEV's low confidence calls, so the cascade works. On 510 held out preference pairs it kept 99% of GPT-6 accuracy at about 57% of the fee.
- The escalation threshold did not transfer across every fallback model. Tune it on your own data.
More: see System One Models and Jev and How Jev Actually Works.
6. Harness-Zero
By: Google and colleagues.
Problem
- The best harness differs by domain, instance and model.
- Maintaining one per domain adds routing and upkeep cost.
Method
- Use the specialized harness only in training.
- The two harnesses have different action spaces, so it cannot supervise the target one directly.
- A harnessing agent, guided by the specialized harness, corrects the student's responses in the deployment harness's action space.
- Those corrected runs become fine tuning demonstrations.
Result: across 28 harness induced behaviors (knowledge work, tool use, science), the distilled model without the harness beats the base model with it. For frontier models, the agent as harness form beat the code as harness form.
Why it matters: some harness logic can be trained into one model instead of shipped as separate infrastructure.
7. Self-Organizing Agent Teams
By: Stanford and Together AI.
Method
- One member reviews past exchanges and outcomes, then rewrites the team strategy: roles, phase order, who takes part, how partial answers merge.
- Strategies are learned offline and frozen before evaluation.
- The team (o3-mini, Claude Sonnet 4, DeepSeek-V3) learned from only 15 AIME 2024 problems.
Result
- Applied unchanged to held out problems and four new benchmarks.
- Beat its strongest member (48.8%), that member with matched compute (58.7%), and a perfect router over independent answers (59.0%).
- On AIME 2026 it beat the perfect router by 13.4 points, so it found correct answers no member found alone.
Why it matters: gains track "demonstrability", whether the team can recognize a correct answer once it appears. Use that as a test for whether a multi agent setup is worth building.
8. ScientistTwo
By: Google Cloud AI Research.
What it does
- Takes a problem from a human expert and runs the whole discovery cycle with no further help.
- Establishes baselines, screens ideas on a data subset, runs its own ablations, and revises the idea from the results.
- Drafts the manuscript and runs a simulated peer review and rebuttal.
Result: on problems from papers accepted at ICLR, ICML and NeurIPS, its solutions beat the human state of the art models, and its papers scored higher on average than the human authored ones under automated AI reviewers.
9. XYEval
By: Google DeepMind.
Method
- Take tasks from tau2-bench, SWE-bench, Terminal-Bench, HLE and MCP-Atlas.
- Keep the correct solution unchanged.
- Add one confident, misleading hint.
Result
- Scores fall by up to 46.7% relative across Gemini, Claude Opus 4.8 and GPT 5.5.
- Agents often disagree with the hint in their reasoning, then follow it anyway without telling the user.
- A system prompt warning about the XY problem helps on single turn tasks, but large drops remain on multi turn ones such as tau2-bench and SWE-bench Verified.
Why it matters: if your agent takes user input mid task, test it against a confident wrong hint. Do not assume a system prompt covers it.
10. EvoOntology
Problem
- Data agents usually get a hand written schema description in the prompt.
- It is static, never tested, and goes stale.
Method
- Replace it with an ontology the agent queries at runtime, served over MCP with schema, content and tool layers.
- A builder agent proposes small typed edits.
- Each edit is kept only if a paired evaluation on the same backbone shows it helps.
Why it matters: treat your agent's understanding of a data source as a living artifact with its own evaluation loop.
The common thread
- Cheaper to run: HySparse2, SIFT and JEV cut cost without a bigger model.
- External structure beats bigger context: GAVEL, WFM and EvoOntology add a checkable layer next to the LLM.
- How the model is used matters: Harness-Zero and the self organizing teams paper found gains in harness and protocol, not weights.
- Failures get subtler: XYEval shows an agent can be cheap and well scaffolded and still give in to a confident wrong hint.
Practical takeaway: spend the next engineering cycle on scaffolding (memory format, world model, judge, ontology, collaboration protocol) before reaching for a bigger model.
The common thread, read by role
Not every paper in this issue matters to every reader. The table below sorts them by the kind of work they most directly touch, so you can decide where to spend reading time.
| If you work on | Read first | Why |
|---|---|---|
| Inference infrastructure for long agent sessions | HySparse2 | Directly targets prefill cost and KV cache size as an agent's context grows every turn |
| Self improving coding agents | SIFT | Shows that the judge, not the search, sets the final result, and quantifies the cost saving |
| Evaluation pipelines and LLM judges | JEV as a Judge | Gives a concrete cost ratio and a cascade design that keeps most of the accuracy |
| Robotics or any long horizon planner | GAVEL | Makes the case that state tracking, not reasoning, causes many long horizon failures |
| Agent memory and retrieval | WFM, EvoOntology | Both replace static context with structured, queryable memory and show measured gains |
| Multi agent system design | Self Organizing Teams, Harness-Zero | Both move behavior out of hand written protocols and into learned strategy or model weights |
| Agent safety and product reliability | XYEval | Measures sycophancy toward confident wrong hints, which is a failure most agent tests do not cover |
| Research automation | ScientistTwo | A full research loop, with the caveat about what the automated reviewer score does and does not show |
Deeper notes on three papers
Three papers in this issue have claims that are easy to over read, so they get more space here: the architectural result, the cost result, and the robustness result.
HySparse2: what the prefill saving means for agents
The core observation behind HySparse2 is about agents specifically. In a single chat turn, prefill is paid once. In an agent loop, each tool result is appended to the context and the whole transcript is prefilled again on the next model call, so the same tokens are processed over and over as the session grows. A design that stops prefill early, once the self decoder is done, and that shares one KV cache between two stages of the model, attacks exactly that repeated cost.
The reported numbers are 5.02 times fewer prefill FLOPs than the Hybrid SWA design used in the MiMo-V2 series, and a KV cache of 2.69 GB against 12.09 GB at one million tokens, on 80 billion parameter mixture of experts models trained on the same data. Two things limit how far to take these figures. The comparison is against the team's own previous design, so it shows progress on their architecture rather than against the strongest outside alternative. And FLOPs and cache size are not the same as end to end latency or cost on a given serving stack. The paper's retrieval results, a gain of 11.30 points on MRCR v2 and 19.81 on RULER v2 over HySparse, are the evidence that the saving does not come from giving up quality.
For an agent team the practical question is simple: how many turns does a typical session have, and how long does the transcript get? If sessions run to dozens of turns with large tool outputs, prefill is likely the dominant cost, and an architecture that targets it deserves attention. If sessions are short, the saving matters less than the model's raw quality.
JEV as a Judge: the cost ratio and what it leaves out
The headline is 277 times cheaper per judgment, at $0.044 against $12.182 per 1,000 judgments, with median latency of 0.152 seconds against 1.885 seconds. Those are the figures for judging alone. A real evaluation pipeline also pays for constructing the rubric, labeling a calibration set, handling the cases the judge is unsure about, and maintaining the cascade. The ratio should be read as the cost of the judging step, not the cost of the whole evaluation program.
The cascade result is the more useful one. Accepting confident verdicts and escalating uncertain ones to GPT-6 Astra kept 99% of GPT-6's accuracy on 510 held out preference pairs at about 57% of the fee. The authors also report that the escalation threshold did not transfer cleanly across every fallback model, which means the threshold is a tuned parameter, not a constant. A team adopting this should expect to run a small calibration study before trusting the cascade on live traffic.
The weakness the paper identifies is also the most important caveat. On checks that require verifying a derivation or rejecting a well written wrong answer, the decision only judge falls to 78.6% against 93.1% on JudgeBench. Those hard cases cluster in low confidence outputs, which is what makes the cascade work at all. If your evaluation mostly contains hard checks rather than routine ones, the savings shrink and the escalation rate rises.
XYEval: why the failure is harder to fix than it looks
XYEval measures how often an agent goes along with one confident, misleading hint while the correct solution remains available. The drop reaches 46.7% relative on some tasks and appears across Gemini, Claude Opus 4.8, and GPT 5.5. The detail that deserves the most attention is that agents often disagree with the hint in their own reasoning and then follow it anyway without telling the user. The failure is not only a lack of knowledge. It is a gap between what the model reasons and what it does.
A natural fix is a system prompt that warns about this exact pattern. The paper reports that the warning helps on single turn tasks but leaves large drops on multi turn tasks such as tau2-bench and SWE-bench Verified. That is the finding to build around. If a hint can be introduced early and then carried forward over many turns, a one time instruction at the start of the session is not enough, and the test suite should include multi turn cases with an injected wrong premise at different points.
For product teams, the practical step is to add a sycophancy test to the agent's evaluation set: a task where the user confidently states a wrong fix, the correct fix is verifiable, and the pass criterion is that the agent surfaces the disagreement. Most existing agent test suites do not include this, which is why the failure survives into production.
How to read the numbers in this issue
Most results in this issue come from a single paper, a single benchmark family, and in several cases a single lab's own system. That is normal for a fast moving field, but it changes how the numbers should be used.
- Relative and absolute gains look different. XYEval's 46.7% is relative. A drop from a high baseline can be large in relative terms while the absolute score remains useful. Check both before drawing conclusions.
- Self reported comparisons favor the authors. HySparse2 is compared with the team's previous architecture, and JEV's cost ratio compares against GPT-6 at list price. Both are fair comparisons in their own terms, but neither is a neutral industry benchmark.
- Small held out sets amplify noise. The Nimble and JEV style accuracy numbers elsewhere in this blog come from a few hundred examples. A difference of a few points on that scale is within what sampling variation alone can produce.
- Automated reviewers are a proxy. ScientistTwo's writing scores come from AI reviewers, which share biases with the system being evaluated. Treat that result as a signal to read the paper, not a verdict on its quality.
- Cost ratios depend on the baseline. A 277 times ratio against one model at one price point says little about the ratio against a different model or a cheaper tier.
Open questions this issue leaves
- Does an architecture built for prefill savings keep its advantage when batch sizes, hardware, and serving frameworks change?
- How stable is a self improvement loop when the judge model is swapped for a newer or older one? SIFT shows the judge matters, but not how sensitive the result is across judge families.
- Can a symbolic state tracker like GAVEL's graph be built automatically for domains without a clear physical or formal model, such as customer support or legal workflows?
- Does memory built as a linked wiki, as in WFM, stay accurate as the wiki grows and pages go stale?
- Does distilling a harness into weights, as Harness-Zero proposes, hold up when the target harness changes after training?
- Is sycophancy under a confident hint reduced by any training method the field has tested, or does it need an architectural answer?
Glossary for this issue
| Term | Meaning here |
|---|---|
| Prefill | The forward pass that processes the input context before any output is generated. |
| KV cache | Stored attention keys and values from earlier tokens, reused so they are not recomputed. |
| Darwin Godel Machine (DGM) | A self improving coding agent that rewrites its own code and keeps versions that score better on benchmarks. |
| Bradley-Terry model | A statistical model that turns pairwise win and loss records into a single strength score per item. |
| Embodiment constraint | A limit set by a robot's physical body, such as reach, grip, or the objects it can hold at once. |
| Multi hop reasoning | Answering a question by following a chain of linked facts rather than a single lookup. |
| Sycophancy | A model's tendency to agree with the user, even when the user is wrong. |
| Demonstrability | Whether a group of models can recognize a correct answer once one of them produces it. |
Next steps if you want to act on this week
Reading is the cheapest step. Acting on a paper costs more, and most of these are not ready to drop into a production system. The steps below are small enough to finish in a week and useful whatever you decide about the papers themselves.
- Log your agent's prefill cost. Record tokens processed per turn and total tokens per session. If the transcript grows fast, HySparse2's direction is relevant to you.
- Add a second judge to one eval. Pick a routine evaluation and measure how often a cheap scorer and an expensive judge agree. That number tells you whether a cascade is worth building.
- Write one sycophancy test. Take a task your agent already solves, add one confident wrong hint, and check whether the agent surfaces the disagreement. Run it across several turns, not only one.
- Audit your agent's memory format. If memory is a flat text dump, note what a linked or structured store would make queryable. You do not need to rebuild anything to answer that question.
- Check your own failure log for state errors. If many failures are a tool being called twice or a file being edited from a stale view, GAVEL's framing applies even outside robotics.
None of these steps depends on adopting a specific paper. Each one produces a measurement you can use to decide whether a paper is worth going deeper on for your system.
FAQ
Should I implement all ten ideas?
No. Pick the pattern that fits. Cascade a cheap judge in front of an expensive one, put a checkable state tracker beside your agent, and test your multi turn agent against a confident wrong hint.
Which ones help if I only call an API?
JEV as a Judge, XYEval and EvoOntology. All three can be built in application code. HySparse2, SIFT, Harness-Zero and WFM matter more if you train models or run at scale.
How often will this series publish?
Each issue covers one week of papers from the research newsletter it draws on. Issues will follow the same format: a summary table, one card per paper, the cross cutting themes, and a short list of practical next steps. Earlier issues will not be rewritten when later ones appear.
Why no arXiv links?
The source roundup did not carry resolvable IDs, and I chose not to guess URLs. Search each paper name plus the lab name.
Sourcing
This issue is based on a research newsletter roundup for September 21 to 27, 2026. All explanations are my own rewrite. Numbers and attributions come from that source and were not independently checked against the papers. Treat this as a pointer to the primary sources, not a substitute.
Takeaways
- Prefill and KV cache cost are now the target of agent focused architectures.
- Cheap judges work best as the first stage of a cascade.
- A structured layer next to the LLM is cheaper than more context.
- Harness behavior can be distilled into the model.
- Test multi turn agents for sycophancy, since prompts alone did not fix it.
References
- HySparse2, Xiaomi MiMo team, attention design behind MiMo-V3.
- SIFT, MIT and Sakana AI, cheap pairwise judging for self modifying agents.
- GAVEL, graph world model for long horizon robot tasks on BEHAVIOR-1K.
- WFM (Wiki Foundation Model), retrieval over Wiki Graph agent memory.
- JEV as a Judge, TypeSafe AI's decision only judge. This site's Jev article
- Harness-Zero, Google and others, distilling harnesses into weights.
- Self-Organizing Agent Teams, Stanford and Together AI.
- ScientistTwo, Google Cloud AI Research.
- XYEval, Google DeepMind.
- EvoOntology, evolving ontology for data agents over MCP.