← All papers
First page of AutoAssumption: Searching over Assumptions Discovers ML-Systems Algorithms

AutoAssumption: Searching over Assumptions Discovers ML-Systems Algorithms

Igor Sadalski, Minghao Guo, Benjamin Jones, Wojciech Matusik

cs.AI Nov 10, 2026 · v1 cs.LG
LLM agents formalize 89 ML-systems algorithms as Lean-checked proof graphs; variants that relax one assumption must re-prove in Lean before benchmarking.
Many advances in ML systems came from breaking an assumption that earlier methods took for granted: speculative decoding dropped the assumption that a model needs one forward pass per token. Yet algorithms rarely state their assumptions, so experts and discovery harnesses search over programs, where one edit changes several design choices and gains are difficult to attribute. Searching over the assumptions themselves remains largely unexplored. Writing algorithms as proofs makes their assumptions explicit and turns algorithm design into a search over a small, structured space that an LLM can explore and a proof checker can audit. We formalize 89 algorithms from KV-cache eviction, weight pruning and speculative decoding as proof graphs with explicit assumptions, checked in Lean. Our harness, AutoAssumption, relaxes one assumption of a published method, re-derives and re-proves it, and generates its code. In KV-cache eviction and weight pruning, this search improved on every published algorithm we ran and produced state-of-the-art algorithms. The eviction algorithm has the highest RULER-4K score of the published methods we ran, and the pruning algorithm the lowest perplexity of thirteen on four LLaMA models. At equal budget, the harness scores within a point of OpenEvolve, an open-source implementation of AlphaEvolve, with a lower standard deviation across repeats. Future work will formalize more fields and combine them through the principal assumptions they share.

Advances in ML-systems algorithms often come from breaking an implicit assumption. Code-level discovery harnesses edit programs, where a single edit changes several design choices at once, so gains are hard to attribute.

Each published algorithm is written as a proof graph of assumptions, definitions and theorems that ends in a formal problem statement (the contract). LLM agents produce the proof from the paper and code, transcribe it into Lean, and check it completely, along with a coverage test mapping code lines to proof statements. The AutoAssumption harness relaxes one pooled assumption of a parent algorithm, re-proves only the dependent statements in Lean, generates code, and benchmarks the variants that pass.

89 algorithms were formalized: 44 in KV-cache eviction, 26 in speculative decoding and 19 in weight pruning. Starting from 26 parents, the search found a higher-scoring variant for every one. RestoreKV GM reached the best RULER-4K score among the published eviction methods run (77.0 vs 73.9 for its parent on Llama-3.1-8B), and Pruner-Zero LG had the lowest perplexity of thirteen pruning methods on four LLaMA models. At equal budget the harness scored within a point of OpenEvolve, with lower variance across repeats.

MethodRULER 4K LlamaRULER 8K LlamaRULER 16K LlamaRULER 4K Qwen3LongBench Llama
RestoreKV73.974.873.372.137.8
RestoreKV GM (ours)77.077.976.574.539.2
RestoreKV vs. RestoreKV GM: RULER and LongBench scores