{"path":"research/the-parts-review.md","content":"# The parts review: putting the parts-versus-evidence judgement to a person\n\n**Date**: 2026-09-22 · **Type**: build record · **State**: built and tested; deployed the same\nday, and again with § 4c's changes on 2026-10-05 (this line said *not deployed* until then);\npage at `/part-review/{name}`, API at `/part-reviews/{name}`\n\n**Why it exists, in one line**: a propose-only pass judged all 197 decomposition edges in the\ngraph, and **nothing scores it** — `MATCH ()-[r:DECOMPOSES_INTO]->() WHERE r.support_semantics IS\nNOT NULL` returns **0**, so no person has ever set one by hand.\n\n---\n\n## 1. The question it answers, and why no measurement could\n\nThe founder asked on 2026-09-22 whether the system distinguishes the types of parts *accurately*.\nTwo things make that unanswerable as posed, and both are already in the corpus:\n\n- **There is no key.** The linked-versus-convergent criterion has close to a dozen rival proposals\n  and no agreed test ([a-cut-through-a-concept-is-an-argument.md](a-cut-through-a-concept-is-an-argument.md)\n  § 5), and Yu & Zenker argue the standard tests are circular\n  ([weakest-link-arithmetic-and-the-merge-hunch.md](weakest-link-arithmetic-and-the-merge-hunch.md)\n  § 1d).\n- **So reviewer disagreement is the expected case rather than a defect** — already the corpus's own\n  conclusion, reached about re-filings before this review existed.\n\n**The instrument is therefore agreement between readers, never a score.** That is not a\nconsolation prize: it is the method that set WordNet's and OntoNotes's sense granularity, and the\nsame method the sameness sitting already runs for claim identity.\n\n## 2. What it is built around: the 18 edges where two readers already disagree\n\nThe judgement pass ran **two** readers over every edge — the model, and a deterministic structural\nsignal (does the child share the parent's subject and fragment its predicate?), which is a\ncriterion of the kind Freeman's 2023 reply defends against the circularity critique. Measured on\nthe live graph:\n\n| Both readers | count |\n|---|---|\n| disagree | **18** |\n| agree | 27 |\n| structural reader silent | 152 |\n\n**The 18 are the sharpest material in the graph**, because a person's answer there breaks a real\ntie rather than rubber-stamping a lone proposal. So `build_sitting` includes **all of them** and\nsamples the rest — sampling the sharpest material to keep a round number would be spending the\ninstrument on decoration.\n\n**The 77% silence is not a defect of the structural reader.** It is the grey zone Yu & Zenker\npredict, and the reason the judgement pass carries a confession value at all.\n\n## 3. Three readings it produces, none of which exists today\n\n1. **Which reader a person sides with**, on the 18.\n2. **The rate at which a machine-written \"part\" is not one** — the 2026-09-22 run found extraction\n   filing evidence as parts **20 times in 30**, which was *predicted in writing* before the pass\n   existed and has never been confirmed by a human eye. Scored per writer, because the run split\n   almost entirely along that line: 0 genuine parts of 84 from `auto_connect`, 30 of 44 from\n   `split`.\n3. **Anchoring**: one edge in five is served with the proposal **withheld by the API**, not hidden\n   in the page, so the gap between answers given with the machine's reading and without it is\n   measured rather than assumed.\n\n## 4. Two design calls worth recording\n\n**The keys are in a reader's words, not the schema's.** `necessary` becomes *a piece — take it\naway and the claim fails*; `miscast_as_evidence` becomes *not a piece — it is evidence ABOUT the\nclaim*. Each maps to exactly one schema value, so answers stay usable. The standing product rule\nis that structure is output rather than demanded as input, and a label naming the enum would ask\nfor the ontology fluency no contribution is allowed to require. A test pins it. *(Since\n2026-10-05 one answer maps to no schema value: see § 4c.)*\n\n**There are no seeded wrongs, and that is stated rather than papered over.** The sameness sitting\ngets them free from `ATTACKS` edges, where *the same* is definitely wrong. Nothing here is\ndefinitely anything — that is the whole reason for the review — so rather than fabricate a seed,\nthe catch-rate question is left to the 18, where a wrong answer at least sides against both\nreaders at once. The cost: rubber-stamping is **less** detectable here than in the sameness\nsitting.\n\n## 4b. The full census, 2026-09-22: three quarters of the decomposition edges are not parts\n\nThe review above was designed around 18 disagreeing edges drawn from a partial run. The\nreason-before-verdict repair forced a re-run over **all 197** decomposition edges in the graph, and\nthe distribution is the finding:\n\n| Kind proposed | Edges | Share |\n|---|---|---|\n| `does_not_fit` | 81 | 41% |\n| `miscast_as_evidence` | 70 | 35% |\n| `necessary` | 35 | 17% |\n| `corroborative` | 11 | 5% |\n\n**197 judged, 0 call failures, 0 unconfident, 59.6 minutes.** Mean relevance 0.33. The two-detector\ncomparison: 152 no-signal (77%), 27 agree (13%), **18 disagree (9%)** — the disagreement figure the\nreview is built on.\n\n**So 151 of 197 — 76% — are not part-of relations at all.**\n\n**⚠ Corrected 2026-10-07: that figure pooled two populations, and one of them could not have come out\notherwise.** 84 of the 197 were written by the automatic linker, which until that day stored every\nsuch link from the premise to the conclusion, the reverse of a part-of link. The judge was asked\nwhether a conclusion is a piece of its premise and answered no 84 times of 84. **Among the 113 links\nwritten the right way round, 67 were judged not parts: 59%.** By writer: the splitter 15 of 44\n(34%), extraction 22 of 30 (73%), people 30 of 39 (77%). The replication below stands for those\n113; for the 84 it replicated the storage. All 261 of the linker's links are retired now and were\nread again as what they were, premise links ([auto-connect-upgrade.md](auto-connect-upgrade.md) § 2c).\n\n**And this is a REPLICATION, which is the more useful fact.** The run in\n[session35-the-verification-night.md](session35-the-verification-night.md) § 4 covered the same 197\nedges *before* the reason-before-verdict repair, and its per-provenance table sums to **45 genuine\nparts, 78 miscast, 74 fitting nothing** against today's **46 / 70 / 81**. **Correcting the field\norder did not move the distribution.**\n\nTwo things follow, and the second matters more than the first. The ordering defect, real as it was\nelsewhere, **cost little on this particular task** — so do not cite this pass as evidence that the\nrepair mattered. And the 76% is now carried by **two independent runs whose prompts differed in the\none respect most likely to change a verdict**, which makes it far harder to dismiss as an artefact\nof how the question was asked than either run alone.\n\n**The 70 `miscast_as_evidence` are the same defect the keeper hit the same day, in a different\npass.** Both are an *evidential* relation filed as a *decomposition* one: there, `overlaps` (which\nthe keeper's prompt defines as \"establishing one would partly advance the other, judged by evidence\nnot topic\") short-circuited before the scheme classifier could name the support; here, 70 edges\nalready in the graph say outright that they are not parts (⚠ corrected 2026-09-27: the census label reliably means only *not a part*; read by hand, about 13 of the 31 machine-written ones were support running child to parent, so the 29 written by the automatic linker were retired rather than re-filed (`session39-what-the-graph-stopped-claiming.md` § 3); ⚠ 2026-10-07: *child* there named the premise by its role, and as stored the premise sat in the parent's place, since the linker wrote these links backwards). Two passes, one week\napart in construction, same confusion between *part of* and *evidence for*.\n\n**⚠ Nothing was written.** These are proposals, `support_semantics` was untouched, and re-filing 70\nedges as support links is a separate ruling — it moves strength on every mother claim above them.\nThat is exactly the shape of decision the review exists to inform, and the census makes the review\nmore load-bearing rather than less: a person is now calibrating a pass whose proposals would rewrite\na third of the decomposition layer.\n\n**What happened to the 70, 2026-09-27.** The founder handed the call over (*\"I trust your judgment on\nimmediate fixes including correcting evidence support links\"*), and the planned re-filing did not\nsurvive a hand reading: of the 31 edges the housekeeping ruling lets a machine touch, about 13 were\nsupport running child to parent (⚠ 2026-10-07: by role; as stored they ran from the whole's place to the part's, premise first, because the linker wrote these links backwards: [auto-connect-upgrade.md](auto-connect-upgrade.md) § 2c), 3 ran the other way, 2 were reported speech, 5 were the\nassisted-dying referent mismatch retired on 2026-09-13, and 8 were not support at all. So the 29\nwritten by `auto_connect` were retired, not re-filed (`scripts/retire_miscast_parts.py`, reversible),\nand the other 41 stay filed as parts: 17 from extraction, held only by an automatic `confirmed` flag\n(ledger Q29); 16 a person wrote; 7 from the splitter, two of which may be genuine presupposition\nparts; and 1 from the automatic linker that the second reader disputes. This review's 10\ndisagreements are spread across those groups. This review still decides those, and its catch-rate question is now\nsharper: the census reliably detects *not a part*, and says little about *what instead*.\n\n## 4c. The sitting of machine-cut parts, and a seventh answer (2026-10-05)\n\nOn 28 September this page was reused for a second purpose: one sitting, `splits-1`, that holds\nevery part the splitter cut, so a person's answers could decide whether the correction panel on\nthe claim page is still needed. Read a week later against what had happened since, it had four\ngaps. The account is in\n[pass-quality-and-the-choice-of-model.md](pass-quality-and-the-choice-of-model.md) § 6b; what\nchanged in this module is recorded here.\n\n- **A sitting of every link one writer made includes the links the judgement pass never read.**\n  The pass last ran on 22 September and every part cut since has no reading: 14 of the 51\n  machine-cut parts on 5 October, among them all six cut by the splitter's current version. Such\n  a link is served with no machine reading and marked `unread`. A sampled review is unchanged: it\n  still draws only from links with a reading, since it exists to measure agreement with one.\n- **A seventh answer: *not a piece — the claim denies it, only grants it, or never says it***\n  (`not_asserted`, key 5; *barely relate* moved to key 6). The splitter's known fault is a part\n  that states as fact what its claim denies or concedes. Such a part is closely related to the\n  claim, is not evidence about it and is not a piece of it, and the six answers had no place for\n  it. The judgement pass cannot give this answer, so agreement with it is read one level up:\n  piece or not a piece.\n- **Results are read per splitter version and per model** (`by_cut`). A measured rate holds for\n  one version of a pass on one model, and on 5 October the 51 parts were 45 from the first\n  version on Claude Sonnet and 6 from the second on a Gemini stand-in.\n- **This sitting is built on each request**; a sampled review is still built once per process, so\n  a refresh returns the same links.\n\n**It does have known wrongs, unlike the review above.** Six of its parts were read as harmful when\na session read the splitter's output item by item on 29 September\n([microtasks-in-broad-daylight.md](microtasks-in-broad-daylight.md) § 3a), and the judgement pass\nhad dissented on all six a week before (§ 3b there). A person's answers on those six are this\nsitting's catch rate, read after the fact from the stored answers. They are a session's reading\nand not a key: where the person disagrees, the person's answer stands.\n\n**Why its test had passed**: the stand-in graph in the test answers a query whatever its filter\nsays, so a sitting described as *every machine-cut part* was never checked against a part with\nno reading. The tests added on 5 October run the real query on an embedded graph, on two real\nsplits copied from the live one.\n\n**On a phone, and on a weak connection** (the same afternoon, for both this page and the sameness\nsitting; `web/src/lib/sitting.ts`). The founder was about to sit both on a bus. Three changes:\n\n- The next item starts at the top of the page. Before, it appeared while the view was still\n  scrolled down at the answers, which on this page meant scrolling back up 51 times.\n- An answer that does not reach the server is sent again (three more tries, about fifteen seconds\n  in all). If it still has not landed it is not counted as answered, a line on the page says how\n  many are waiting, and it comes up again before the page can say the review is done. Before, the\n  page awaited the request and never read the reply's status, so a refusal passed for a save and a\n  dropped answer was never asked again. The answers are the cases later models are measured\n  against, so a hole there is worse than a repeat.\n- The next item is the next one nobody has answered, so going back to change an answer returns\n  to where the reviewer was.\n\n**Trying these pages without a sign-in**: `scripts/mock_review_api.py` stands in for the two\nreview endpoints on the port the web app uses in development. It serves the real items, read\nonce from the live site, as if the visitor were signed in, keeps answers in memory only, and can\nbe told to fail chosen answers (`MOCK_FAIL_TWICE`, `MOCK_REFUSE`, `MOCK_DROP`). Start it and the\nweb app from `.claude/launch.json` (`review-mock-api`, `deliberus-web`). The real pages show\nnothing to a visitor who is not signed in, and an answer given there is written to the live\nanswers table, so this is the way to look at them.\n\n## 5. What it does not do\n\n**Nothing is written to the graph.** A review produces answers; `support_semantics` stays unset\nuntil the numbers say otherwise, exactly as auto-merge stays switched off behind its own gate.\n\n---\n\n## Cross-references\n\n`deliberus/parts_review.py` · `deliberus/decomposition_judgement.py` (the pass being reviewed) ·\n`deliberus/sameness.py` (the sibling sitting, and the shuffle this reuses) ·\n[linked-convergent-and-what-we-are-actually-asking.md](linked-convergent-and-what-we-are-actually-asking.md)\n(the measurement and why accuracy is the wrong target) ·\n[session35-the-verification-night.md](session35-the-verification-night.md) § 4 (the run) ·\n[soft-canonical-clustering-and-reversible-merge-semantics.md](soft-canonical-clustering-and-reversible-merge-semantics.md)\n(the review protocol this copies, founder-revised 2026-08-31) ·\n[pass-quality-and-the-choice-of-model.md](pass-quality-and-the-choice-of-model.md) (2026-10-05: this\nreview and the two others still hold no answer; its labels would be used again for every later model,\n§ 5e)\n"}