← All papers
First page of FCP: Formal Context Protocol for Verifying LLM-Generated Artifacts

FCP: Formal Context Protocol for Verifying LLM-Generated Artifacts

Shashank

cs.AR Nov 15, 2026 · v1 cs.AI
Lean 4 proves the paper's propositions and corollary (84 theorems) and provides a reference semantics checking the FCP-PL-to-monitor compiler registers.
When a language model writes both an artifact and the tests meant to check it, a passing test is weak evidence: one misreading of the request yields a faulty artifact and a check that accepts it. We present the Formal Context Protocol (FCP), which checks AI-generated artifacts against the request itself. FCP separates conformance (does the artifact satisfy a formal reading of a requirement?), which model checkers and SMT solvers decide, from fidelity (does the reading mean what the prose means?), which has no decision procedure but is refutable by a concrete witness. Eight ways a proof can falsely certify a requirement each become a mechanical check that returns a witness, and what no machine can settle becomes concrete questions for the request's author. Adapters place hardware, text, agents and code on the same engines. On four hardware designs, trusting AI-written assertions certified 23% of violated requirements, and FCP certified none. On ten code tasks, code from three language models passed its own tests and violated the request every time. Trusting the tests certified all 33 violations, and FCP certified three. Agreement among models, readings or tests did not imply correctness.

When an LLM writes both an artifact and the tests that check it, passing tests are weak evidence, because a single misreading of the request produces both the faulty artifact and a check that accepts it. Verification is needed that checks AI-generated artifacts against the request itself.

The Formal Context Protocol (FCP) separates conformance, decided by model checkers and SMT solvers (SymbiYosys PDR/BMC, z3), from fidelity, which is refuted by witnesses that distinguish independent formal readings. Eight ways a proof can falsely certify a requirement each become a mechanical check. Adapters encode hardware, text, agents and code as circuits, and symbolic execution extends proofs for Python code beyond the tested scope. The core propositions are proved in Lean 4, and the compiler's registers are checked against a Lean reference semantics.

On four hardware designs, trusting AI-written assertions certified 23% of violated requirements, while FCP certified none. On ten code tasks, models' own tests certified all 33 violations, while FCP certified three. All 45 disagreements between formalizers were real errors.

MeasureSelf-checkFCP
Violated requirements certified as holding56/244 (23%)0/244
Injected formalization faults certifying buggy design71/1610/161 (all flagged)
Injected RTL defects detected28/2828/28 (all replayed)
Hardware study: self-check vs FCP