← All papers
First page of Fast Collection Operations from Indexed Stream Fusion

Fast Collection Operations from Indexed Stream Fusion

Scott Kovach, Praneeth Kolichala, Kyle A. Miller, David Broman, Fredrik Kjolstad

cs.PL Jul 8, 2025 · v1
Implements an indexed-stream-fusion collection library in Lean and provides a mechanized Lean proof of its functional correctness.
We present a system of efficient methods for traversing and combining associative collection data structures. A distinguishing feature of the system is that, like traditional sequential iterator libraries, it does not require specialized compiler infrastructure or staged compilation for efficiency and composability. By using a representation based on indexed streams, the library can express complex joins over input collections while using no intermediate allocations. We implement the library for the Lean, Morphic, and Rust programming languages and provide a mechanized proof of functional correctness in Lean.

Iterator libraries for sequential traversal are standard, but efficiently expressing complex joins over associative collections (e.g., sparse intersection, union, triangle queries) without intermediate allocations or specialized compiler infrastructure remains difficult.

The authors implement an indexed stream fusion library for Lean, Morphic, and Rust. Indexed streams generalize iterators by exposing a seek/advance interface keyed by an ordered index, enabling non-sequential traversal and multi-way joins with no intermediate allocations. A mechanized proof of functional correctness is provided in Lean, building on prior indexed stream theory.

The library matches hand-written performance on benchmarks including triangle queries and red-black tree operations. It requires no staged compilation or specialized compiler passes. The Lean correctness proof covers the shallow embedding directly, providing end-to-end guarantees without trusting a custom code generator.