← All papers
First page of On Compositional Learning Behaviours in Formal Mathematics

On Compositional Learning Behaviours in Formal Mathematics

Kevin Yandoka Denamganaï

cs.CL May 27, 2026 · v2
Evaluates ten Lean 4 theorem provers on miniF2F whole-proof generation and steers DeepSeek-Prover-V2-7B activations during Lean proof generation.
Self-evolving scientific agents capable of conquering the hard tail of formal mathematics require Compositional Learning Behaviours (CLBs) – the capacity to ground and recombine novel symbolic structures in context, beyond mere recombination of prelearned atoms. We propose S2B-LM, an adaptation of the CLB-evaluating Symbolic Behaviour Benchmark that removes numerical processing as a confound and adds chain-of-thought scaffolding to elicit rather than merely probe latent CLB competency. Cross-evaluating ten Lean 4 theorem provers on CLB competency in S2B-LM and miniF2F whole-proof performance, we find correlational and causal evidence of our claim: First, a necessary-condition analysis via quadrant test yields $p=0.004$, with model scale being ruled out as a confound. Second, extracting a CLB-encoding activation direction from DeepSeek-Prover-V2-7B using S2B-LM traces via Contrastive Activation Addition and applying it during miniF2F whole-proof generation on the AIME subset, CLB suppression collapses solve rate from $32.3\%$ to $2.9\%$, without loss of coherence, while suppressing a random activation direction of equal magnitude leaves it at $31.9\%$. Together, these results show that CLB competency is necessary but not sufficient for the hard tail of formal mathematical verification.

Formal theorem provers may need Compositional Learning Behaviours (CLBs), the ability to ground novel symbolic structures from in-context examples and recombine them, to solve hard formal mathematics problems. It was unclear whether this capacity relates to prover performance.

S2B-LM adapts the Symbolic Behaviour Benchmark for language models. It replaces continuous numerical stimuli with categorical word stimuli and adds chain-of-thought scaffolding. Ten Lean 4 provers (DeepSeek-Prover, Goedel-Prover, Kimina-Prover variants) are scored on adjusted ZSCT and compared with their miniF2F Pass@32 accuracy using exact permutation tests. A CLB direction is extracted via Contrastive Activation Addition and suppressed during miniF2F proof generation.

A global ceiling test gives p=0.052, while a tail partition test shows that the five models above 75% on miniF2F are exactly the five with the highest CLB scores (p=0.004). Model scale is not a significant predictor. Suppressing the CLB direction drops the AIME-subset solve rate from 32.3% to 2.9%, versus 31.9% for a random direction of equal magnitude.

ConfigurationZSCT (%)Adj. ZSCT
S2B (SCS, 0-shot)57.9 ± 5.715.8
Categorical + 0-shot40.0 ± 8.50.0
S2B-LM (Categorical + 10-shot)87.0 ± 4.174.0
Effect of domain and shot adaptations on ZSCT