
This page shows the whole method: what is measured, on which slice of the corpus, how the routing is chosen, and how the record is frozen. Every claim lives in the source at a path your reviewers can open — and every step reruns on your machine, so the comparison is yours to make.
The corpus is cut into three parts — training, dev, and held‑out — and the published measurement reads only the held‑out slice, forced in code, not by discipline. The middle slice exists because of a real mistake: a generative prompt was once tuned while reading held‑out scores. The leak was caught, repaired, and the split hardened so it cannot recur silently: a test fails the suite if two slices share a document or a phrasing.
One honest limit, stated by the repository itself: the corpus is authored by us. The held‑out slice defends against grading our own copy — it does not defend against unrealism. That is what measuring your own cases is for.
Run the suite. The split test walks the corpus and fails if any document or phrasing appears in two slices.
npm testfails if two slices share a document or a phrasingWhere it livessrc/corpus.ts:164 · src/cascade.test.ts:47
Five fields, seven tiers per field: the optimiser walks all 16 807 assignments exhaustively — a recursive enumeration, no heuristic, no sampling. The count on this page is recalculated from the sources on every check, never typed in.
The asterisk: the seventh tier, human review, is an assumption, not a measurement — and the tool prints exactly that on every pass rather than letting the figure blend in with the measured six.
Run the optimiser. It enumerates from the sealed record and announces the human tier as an assumption, every time.
npm run optimiseprints the optimal routing, human tier flagged as assumedWhere it livessrc/optimise.ts:379 · src/readme.ts:827
The published record carries a content fingerprint — a SHA‑256 taken over the record in canonical form and truncated to sixteen hex characters, and the reader refuses a record whose fingerprint is absent or no longer matches. Refuses, not warns: a hand‑edited measurement fails loudly. The stamp in this site's masthead is that fingerprint.
You do not have to take our word for it, and you should not hash the file by hand — a plain checksum of the file will not match, because the fingerprint is taken over a canonical form of the record, not over its bytes. The tool recomputes it in front of you instead, and prints what it finds.
Two limits the sealing tool states about itself: the fingerprint covers the whole record except fingerprint keys themselves, and it proves nothing about what happened before sealing — the only gesture that produces a measurement is the measurement command. And one repair published rather than hidden: the git commit the reference record was measured at was later rewritten away by a history purge; the repository records the replacement commit and labels the mapping “inferred, not recorded”.
Point the sealing command at the sealed record shipped with the repository. It recomputes the fingerprint and prints it; on an untouched record it changes nothing.
npm run sceller -- profiles-2026-08-20-coeur-rendu.jsonalready sealed, and the seal matches: 1151f5a1cfaae0c0Where it livessrc/measure.ts:298 · src/sceller.ts:12
Every rate ships with a Wilson interval at 95%. The repository admits, in writing, that the level is an inherited default rather than a weighed choice. The interval function throws on impossible input instead of returning a quiet NaN, and for two tiers graded on the same cases the repository prefers a paired McNemar test to comparing Wilson intervals, which it documents as too conservative for that job.
The interval code is thirty lines and its guard is tested; the declared method is in the published record.
npm testcovers the interval guard and the declared methodWhere it livessrc/interval.ts:32 · src/inventory.ts:63
Not everything measured got published. On the constrained‑output bench, every duration moved by 16 to 60 per cent between two passes on a memory‑starved machine — so the durations were withdrawn, and in the data they were renamed so the defect travels with the number: the field itself is called msMedianeNonTransportableFamineMemoire — non-transportable, memory starvation. You cannot cite it without its qualifier.
The token counts from the same bench reproduce to the digit, and those are published. That is the line: what reproduces is shown, what does not is named and withheld.
Rerun the bench. Below the memory floor it marks its own durations non-transportable rather than publishing them.
npm run contrainteregenerates the bench record, durations qualified in the field nameWhere it livesrapports/2026-08-22-contrainte-de-sortie.md:17 · contrainte.json:34
The title is an instruction. The full measurement reruns with one command — it insists on an explicit flag because it downloads 1.3 GB of pinned weights on first run — the measurement itself took 32 minutes on the published pass — and freezes a fresh record you can set beside ours. Your own files run through the same grader and the same intervals. And two sealed passes compare case by case, because a rate that rose may simply have lost cases.
The three commands below are the whole loop: remeasure, measure yours, compare.
MESURE_VOULUE=1 npm run measurereruns the full measurement and freezes a sealed recordnpm run measure:yours -- --cases=your-file.csvyour cases, same grader, same intervals — nothing leaves your machinenpm run diffcompares two sealed passes case by caseWhere it livessrc/measure.ts:960 · src/your-cases.ts:1185