{"path":"research/the-canon-distillation-pass.md","content":"# The canon distillation pass — what it is, why now, and how it is governed\n\n**Opened**: 2026-08-29 · **Status**: in progress, expected to run over multiple sessions\n**Type**: method + open-loop register for a long-running review\n\n---\n\n## 1. What this is, in the founder's words\n\n> *\"What all this is is basically me going through the project's long-term memory, sifting through it\n> and making sure the important tidbits and insights and principles and convictions etc etc are etched\n> into our short-term memory, so you Claude can have a much easier time following along with what my\n> vision is, how I imagine we best build it and why, on every level of concreteness and abstraction and\n> philosophy and linguistics and science and epistemology etc etc etc…*\n>\n> *Because Claude keeps having trouble remembering all key insights details and inferences from all\n> around the corpus, I'll have to do this conscious scan/review/compilation/distillation of the whole\n> corpus into the always-loaded memory so we at least have strongly worded pointers to the most\n> important documents even if not precisely all crucial tidbits are directly included in CLAUDE.md.\"*\n\n**The occasion**: in one session Claude asserted the *opposite* of three findings the corpus already\nheld, with the corpus's own pointers loaded. His response: *\"I bet you'd make some 'surprising'\n'discoveries' if you read more of the corpus. All of this is NOT a surprise to me, I've been over this\na thousand times in different sessions. Every session that I ask you to read up on something from the\ncorpus, you seem to have a revelation or four.\"*\n\n## 2. The structural insight that reframes it — three reviews, one activity\n\n> *\"This work is basically deeply entangled with / the same work as the review of the constellation,\n> convictions, unruled claims etc, which we need to complete arc by arc, because all that obviously\n> deserves to go in CLAUDE.md if it is not already given prime token real estate there.\"*\n\nHe is right, and naming it changes the sequencing. **Three things previously tracked as separate\nreviews are three views of one activity — deciding what the project's canon is and where it lives:**\n\n| Surface | The question it asks | Output |\n|---|---|---|\n| The **ruling ledger** (`session26-open-rulings.md`) | which asserted positions does the founder actually hold? | ruled claims |\n| The **constellation review** (`specs/landing-redesign/constellation-map.md`) | which held positions can be said plainly to a stranger? | tiles |\n| **This pass** | which held positions must be loaded before a session can reason correctly? | CLAUDE.md lines |\n\nA position flows through all three: asserted → ruled → compressed → resident. So they are not competing\nqueues, and the arc-by-arc ordering already agreed for the ledger governs this too.\n\n## 3. The budget, measured 2026-08-29 — and it is the binding constraint\n\n| | tokens |\n|---|---|\n| project `CLAUDE.md` | **100,599** |\n| global `~/.claude/CLAUDE.md` | 80,226 |\n| **combined, loaded every session and again per subagent** | **~180,825** |\n\nThe global file's own record states that unbounded growth *\"crossed the 200k subagent-spawn ceiling\"*\nat ~178k tokens and forced the 2026-07-06 slim. **We are back at that number, split across two files,\nwith roughly 19k of headroom.** So insights cannot be added freely; every addition is paid for.\n\n**And the project file is the larger half**, which is not the shape anyone assumed.\n\n**Where the payment comes from — the timeline is the one-liners in the wrong shape.** The\n`## Status: First Build Deployed` section is 21 per-session bullets totalling **20,660 tokens**\n(sessions 1–8 are 26–490 each; the last eleven are 900–2,600 each, ~15,000 of the total). Reading them,\neach mixes narrative, durable findings, and ⚠ corrections that exist nowhere else. **It grew because a\nfinding had no other resident home.** Converting it is therefore the same job as this pass, not a\nseparate cleanup — and the estimate is 20,660 tokens becoming ~4–5,000 of topic-shaped, denser content.\n\n## 4. The filter, its name, and the founder's correct doubt\n\n**The property, named 2026-08-29: a finding is WRONG BY DEFAULT when a session lacking it will\nconfidently assert its opposite.** Not *important* — most important things are safely looked up. Not\n*surprising* — surprise is about the reader's mood. The distinguishing feature is that **the naive\nprior points the wrong way**, so absence is not ignorance but active error, and the error arrives with\nconfidence.\n\n*(Named per the founder's own naming criterion — self-explanatory without a gloss. Alternative\nconsidered and rejected: \"inverted prior\", precise but requiring a lookup.)*\n\n**The founder's doubt, verbatim, and it lands:**\n\n> *\"I like this, but it'll be a bit hard to assess (I guess you can assess your state before/after\n> having read a document but idk if I always trust your self-assessment on this but we can start with\n> this and see if I feel it breaks and re-assess this test if so…)\"*\n\n**The refinement that mostly dissolves it: prefer DEMONSTRATED inversions to predicted ones.** Asking\nClaude *\"would you have got this wrong?\"* is introspection, and the corpus already holds the finding\nthat introspection of one's own reasoning gets no elevated credence (the bias blind spot is *not*\nmediated by actual susceptibility). But **an inversion that actually happened on the record is\nevidence, not self-assessment** — and this session produced three of them in front of the founder,\nwith the corpus's pointers loaded:\n\n1. *\"No decomposition path looks up an existing claim\"* — `auto_connect` does, 84 edges.\n2. *\"The stranger test is anti-reuse, a tension nobody has named\"* — the literature calls it\n   *normalization* and treats it as the enabling step; the corpus recorded that a month earlier.\n3. *\"Make identity structural — nothing in the corpus has proposed it\"* — it is the scheme-relative\n   hypothesis, held since April with six warnings, one aimed exactly at that move, and its first rung\n   shipped 2026-08-20.\n\n**So the operational rule: a candidate is `demonstrated` when a session has actually asserted its\nopposite on the record, and `predicted` otherwise — and the two are labelled differently.** Predicted\ncandidates are the weaker class and should be admitted more sparingly. The accumulating record of\ndemonstrated inversions is itself the calibration data for whether the test works.\n\n## 4b. Two rulings made after this doc was first written (2026-08-29)\n\n**The grades are a LIFECYCLE, not a verdict.** Claude had over-dismissed the weaker grade; the\nfounder corrected it: *\"Predicted wrong-by-default doesn't seem of NO value, it has some value, but\ndemonstrated is better ofc.\"* He is right. A predicted line is a hypothesis from a reader who has just\nseen where the naive prior points, and **it upgrades to `demonstrated` the first time a session\nactually inverts it** — which makes the register self-improving rather than static, and makes the\naccumulating record its own calibration data.\n\n**Run the filter over the CUT, not only over the keep.** The first size-driven trim in this pass took\na topic entry from 1,636 to 479 tokens, dropping 24 of 25 sentence-units. Audited on the founder's\nchallenge (*\"Did it deserve removal?\"*), **two of the dropped units passed the filter and were\nrestored** — *flag versus reasons* (the variable in durable attitude change is neither dose nor\nduration) and *on sacred values, argument is the wrong instrument*. **Trimming by size is not trimming\nby value**, so every compression pass owes the filter a second run over what it is about to discard.\n\n## 5. Open loops for this pass\n\n- [x] **DECISION — evict-and-convert the timeline? DONE 2026-08-29** (`SESSION-HISTORY.md`, 21 entries, the two ⚠ corrections preserved; marked here 2026-09-09). 20,660 tokens of per-session narrative → a\n  session-history doc, with durable findings extracted as one-liners and every ⚠ correction preserved.\n  Argued against: moving it wholesale, since several corrections exist nowhere else and their\n  superseded versions still stand elsewhere in the corpus.\n- [ ] **DECISION — is the session-history doc web-served or born-private?** It carries founder-challenge\n  quotes and internal error narratives; everything under `docs/` publishes. Asked 2026-08-29, unanswered.\n- [ ] **The eight candidates from this session's reading await founder review**, listed in § 6. Three\n  are `demonstrated`, five are `predicted`.\n- [ ] **Sequencing**: this pass runs arc by arc with the ruling ledger and the constellation review,\n  per § 2. Arc J (the headline tests) went first and is complete as of 2026-09-09; it governs how any compressed line is\n  written.\n- [ ] **The verbosity audit of `CLAUDE.md`**, opened by the founder 2026-08-29: *\"Sounds like we need\n  to audit it for unnecessary/long-winded bits as part of this whole big review.\"* The shape test\n  catches size; it cannot catch long-windedness inside the cap, so this is a reading pass. Its\n  discipline is § 4b: the filter runs over what is cut.\n- [ ] **Read-coverage is 2 research docs read deeply this session** against the whole corpus (`ls docs/research/*.md | wc -l`, growing — cite the command, not a number; it was 180 on 2026-08-29) (plus ~8 partially).\n  The pass is not close to done, and any claim about what the corpus does or does not hold is\n  unreliable until it is.\n\n## 6. Candidate lines from this session's reading\n\nFormat follows the brf-auto quick-reference style: a bold short name, then a dense self-contained\nstatement carrying the number and the consequence.\n\n**Demonstrated** (a session asserted the opposite on the record):\n\n1. **Decontextualisation enables matching, it does not fight it.** The production claim-matching\n   literature calls it *normalization* and treats it as the enabling preprocessing; the winning\n   cross-document architecture is *collaborative* — the model normalises and contextualises each claim,\n   a dedicated matcher decides — and a frontier model asked directly whether two claims match\n   **underperforms by nearly 10 CoNLL F1 points**. The extraction pipeline's stranger-test pass is\n   already that first half. → `sameness-merge-and-split-across-fields.md` §3d, §5\n2. **The lookup exists and fires; the gap is where it runs.** `auto_connect` offers `decomposes_into`\n   among five outcomes with two-direction handling and has produced **84 cross-source decomposition\n   edges**. It is called only from the two extraction paths against *other* sources, so claims the\n   correction UX mints never pass through it. → `the-reuse-flywheel-prior.md` §3\n3. **\"Match on structure not prose\" is a held hypothesis with six warnings, and its first rung is\n   built and weak.** It is the scheme-relative hypothesis (2026-04-19), explicitly *not a proposal* —\n   one warning aims at elegant cross-tradition reconciliations specifically. `structural_kinship_candidates`\n   shipped 2026-08-20 and measures **≈3.5/6 hand precision, the weakest of three generators**, with an\n   information-free-signature failure gated at four atoms. → `claim-sameness-philosophical-readings.md`,\n   `auto-connect-upgrade.md` §2b\n\n**Predicted** (weaker class, admitted more sparingly):\n\n4. **Structure converges, weighting diverges.** Across Schwartz's value circle, moral foundations,\n   Curry's seven rules, reverse mathematics, cuisine, music and colour naming, what converges is the\n   *list* and what diverges is the *ranking*. This is the sharpened wager, and it predicts the horizon\n   map should fill with **fittingness and contextual** residues rather than structural or axiom-choice\n   ones — which is checkable and can go against us. → `fractal-priors-for-convergence.md` §0\n5. **The flywheel and the error corridor are one mechanism.** Identity is transitive, so\n   `owl:sameAs` at web scale ran 3–20% erroneous links into one closure that falsely unified\n   **177,000 names**. Every increment of reuse is an increment of error-propagation; hence no silent\n   transitive chaining. → `sameness-merge-and-split-across-fields.md` §3c\n6. **The compression ceiling is 72.5% and it saturates.** A professional debater, ten minutes per side\n   and **without seeing the data**, wrote seven key points covering 72.5% of 6,515 crowd arguments over\n   28 topics; one point alone covers 41.3%, and five-to-seven buys one percentage point. The head is\n   narrow and predictable; the residual quarter does not compress, and that quarter is where the crux\n   lives. → `the-reuse-flywheel-prior.md` §6\n7. **Every field that hit the identity wall built process, not algorithm.** Taxonomy's `sec.`\n   convention, library authority records, graded predicates after the sameAs catastrophes,\n   lexicography's agreement target with an underspecified escape hatch. And the honest cost: the risk\n   transfers from *no algorithm can do this* to *nobody shows up to judge*. →\n   `sameness-merge-and-split-across-fields.md` §5\n8. **The 22.8% gray zone is a property of the judgment, not weak machinery.** In IBM's key-point\n   corpus, that fraction of argument-to-key-point matches was ambiguous to trained annotators. A\n   borderline band is descriptively correct rather than a concession. → same doc §3e\n9. **Determinism amplifies systematic error rather than cancelling it.** Random errors wash out across\n   a large graph; a wrong choice in the deterministic strength layer repeats identically on every claim\n   it touches — which is why the corpus's worst measured defect lived there. →\n   `the-residual-error-taxonomy.md` §0\n10. **Cosine measures vocabulary, not argument structure — and every edge in the graph was made by\n    it.** Verified 2026-08-29: zero edges carry `r.generator`, so the 2026-08-20 upgrade (concept-route,\n    structural kinship, stance guard) is live in code and has never run. Its generators surfaced\n    **58–76% of candidates cosine never found**. And reading a low cosine as *not related* is explicitly\n    **not a safe inference**. → `auto-connect-upgrade.md`, `retrieval-instruments-beyond-cosine.md` §0\n\n**Cross-references**: `session26-open-rulings.md` (the ruling ledger) ·\n`specs/landing-redesign/constellation-map.md` (the constellation review and its ordering) ·\n`the-residual-error-taxonomy.md` §5 (the pre-launch audit this pass is downstream of)\n\n---\n\n**Direct entry logged 2026-08-30**: the harness conviction entered The Convictions in One Breath by\ndirect founder ratification (with the amendment that the two human-reserved classes be named in the\nline), outside this pass's queue — legitimate, since the pass exists to serve rulings, not to gate\nthem. Grade: demonstrated (a session asserted the opposite twice; the delegation correction's\nlogged regression is the evidence). Home: [the-harness-conviction.md](the-harness-conviction.md).\n\n11. **Nothing is permanently terminal** (the horizon-side twin of *nothing is permanently atomic*; **demonstrated**, 2026-09-14). A horizon is where shared mapping currently ends under the current moves, dated, and every kind names what continues past it — the world (empirical), the word's users (definitional), weighing (the value kinds). A kind is a reading over the claims' current arrangement, never a home for content and never an end; the finality words in the code (*terminus, verdict, resolved, bedrock*) predate the rulings that say so. Without the line a session asserts the opposite: on 2026-09-14 one said of an empirical terminus *\"the map's work is done and evidence takes over\"*, and only the founder caught it. Founder on the principle, hedged: *\"Otherwise I guess I'm game for the naming principle.\"* → `what-belongs-in-the-ontology.md` § 6c.1–6c.2\n   **BLESSED into `CLAUDE.md` 2026-09-15**, reworded to carry the founder's conservatism, verbatim: *\"Blessed for now, can always reword later as part of our big review of everything... We should also be a bit conservative about what we classify as terminal, even when knowing it's not permanently terminal, obviously. Feel free to reword this line as you think is wise based on what I've said.\"* Sits directly after *Nothing is permanently atomic* in the convictions list.\n"}