{"path":"research/synthetic-stress-suite-and-ontology-reflection.md","content":"# The Synthetic Stress Suite: Four Built-to-Break Examples, and What the Ontology Confessed\n\n**Date**: 2026-08-22 · **Method**: ontology stress-testing by synthesis (CLAUDE.md § 2026-08-18 — sanctioned; the deliverable is named gaps, not quality measurement; nothing here tests the pipeline, which is governed by substance-over-meta) · **Occasion**: founder instruction — *\"Create a bunch of synthetic examples and do recursive decomposition on their subgraphs/trees of claims/evidence on all of them, feed it all into the live argument graph and reflect on the ontology afterwards.\"* · **Artifacts**: `scripts/seed_synthetic_stress_suite.py` (re-runnable, provenance-labeled synthetic throughout), 26 claims + one minted verdict claim across four sources, all live and stored through the production path, all measured through the DEPLOYED public API rather than local calls.\n\n## The four examples, and what each was built to press on\n\n| Source | Register | Machinery it stresses |\n|---|---|---|\n| **The bridge reopening** (`synthetic-bridge-reopening`) | engineering sign-off | three-level decomposition with `necessary` joins at BOTH levels; a planted attack on a great-grandchild (the expired calibration certificate) — does leaf weakness reach the root? |\n| **The acquisition offer** (`synthetic-acquisition-weighing`) | a founders' dispute | the weighing descent incl. the new three-ingredient screen; a state-deficit fact (an exhausted founder) and a satisfier-belief dispute (is selling really the only job-preserving move?) |\n| **The tasting machine** (`synthetic-tasting-machine`) | philosophy of mind in consumer clothing | a descent from an evaluative dismissal to the phenomenal floor — the first live `is_bedrock` terminus, minted with its verdict-claim ground |\n| **The fasting evidence pile** (`synthetic-fasting-evidence`) | health claims | evidence-quality spread (meta-analysis vs mouse study vs testimonial); ONE piece of evidence supporting two siblings — the join-semantics / shared-evidence case, live |\n\nEvery claim carries `(synthetic stress-suite example — asserted by nobody)` in `attributed_to` and `evidence_type='synthetic_example'` — the always-mint labeling rule applied to operator-authored material, same as the thermostat seed.\n\n## What broke — five named gaps\n\n**Status update, same day (founder: \"Fix all five gaps\"): ALL FIVE RESOLVED and live-verified.** The bridge mother now reads UNSUPPORTED 0.459 (the leaf weakness arrives through the recursive walk); denying any required part craters the hinge to 0.0 (hinge 0.5, was 0.022); the fasting mother reads generic SUPPORTED (no more STRONG ANALOGY off the mouse study); the badge note distinguishes attached-from-absent; and every weighing question now carries a real swing (0.2, was 0.0). **One correction owed to this document's own record: gap five as first stated was partly MY measurement error** — the endpoint DID serve the weighing descent all along under the response key `critical_questions`, and my curl parser looked for the wrong key and reported an empty list. The real, narrower gap (confirmed by re-measuring before fixing): every served weighing question carried `hinge_sensitivity: 0.0`, because the ordering signal could not see claim-attached questions. The fix gives claim-level questions oriented entries on their two mirrored polarity edges and clamps all of a question's occurrences at once. A finding-report is itself a surface that can lie; this one did, and the single-surface-correction rule applies to it like any other.\n\n### 1. The strength walk is one level deep — leaf weakness reaches its parent and dies there (the headline)\n\nMeasured through the live badge endpoint: the calibration attack pulled `b2` (load capacity) down to **0.459 UNSUPPORTED** — the `necessary` ceiling working exactly as shipped at that level, the weakest leg deciding. But the mother `b1` (bridge safe to reopen) reads **SUPPORTED, blue, 0.5**: its subclaim channel computes each child's strength from that child's *direct* evidence counts, not from the child's own computed badge. `b2` has no direct evidence, so it enters its parent's ceiling at a clean 0.5, and the refuted calibration three levels down never arrives. **A bridge whose load-test calibration is undercut renders as SUPPORTED.**\n\nThis is the strength-layer audit's unspecified-middle pattern exactly one layer up: the March spec specified edge-level functions, the audit specified the claim-level aggregate, and **nobody has yet specified the recursive composition** — how a child's computed strength (not its raw evidence count) enters its parent. The hinge endpoint already does the recursive DAG walk with memoization, so the shape exists in the codebase. Making the badge recursive changes strengths corpus-wide, so it was filed as a founder decision — **ratified and shipped same day**: an examined child enters at its computed badge strength, unexamined children keep the count-based prior (preserving the one-level responsiveness to unscored evidence), memoized and cycle-guarded (`compute_badge_recursive`).\n\n### 2. The hinge is blind to join semantics — it whispers exactly where it should shout\n\nOn the bridge tree, *denying a jointly-required part outright* moves the mother by **0.009–0.022** (`strength_if_denied` 0.491 on the steel certification). Under `necessary` semantics the honest answer is a crater — the ceiling means a denied required part caps the whole near the floor. The hinge predates `support_semantics` and computes sensitivity through the additive energy only; it never reads `d.support_interpretation`. Dogfood run 2's \"the hinge whispers at light engagement\" therefore has a second, structural cause: on conjunction-shaped trees it whispers *by construction*. **Fixed same day**: the hinge propagation now applies `interpreted_energy` grouping and the necessary ceiling per recorded interpretation — denying the calibration leaf takes the live bridge mother to 0.0.\n\n### 3. \"No evidence bears on this yet\" was false on its face — FIXED same day\n\nA claim with an attached-but-unscored attack (`b6`, the calibration leaf) rendered UNEXAMINED with the note *\"no evidence bears on this yet\"* — while its own details array displayed the attack. Absence of edges and absence of *scoring* are different states, and the note collapsed them: the reader is told nobody has looked when in fact an attack is standing and merely unanswered. This is the badge-boundary defect (a two-step contribution presented as one step) surfacing on the note text itself. Fixed: the note now reads *\"N evidence edge(s) attached, none scored yet — answering their critical questions is what moves this claim\"*, regression-pinned (`test_unexamined_note_distinguishes_attached_from_absent`).\n\n### 4. The badge headline inherits the nearest edge's scheme label — a methodology inversion\n\nThe fasting mother's badge term is **\"STRONG ANALOGY\"** — the scheme label of its one direct edge, the *mouse study* — while the meta-analysis (the strongest methodology in the pile) sits two hops away supporting both children and contributing almost nothing to the display. The headline a reader sees is named after whatever evidence happens to be adjacent, not the best evidence in the subtree. **Fixed same day**: the scheme-flavored headline now appears only when the direct-edge channel carries the verdict (|edge energy| ≥ |subclaim energy|); a parts-carried number gets the generic vocabulary.\n\n### 5. Two question systems, one server — the hinge-ordered endpoint cannot see the weighing descent\n\n`open_weighing` on the acquisition value premise instantiated the full ten-question descent — including the new state-deficit and satisfier-belief screen — as Question nodes `ABOUT` the claim plus twenty polarity claims at `argument_from_weighing`. The claim page renders all of it. But `GET /claims/{id}/cqs` — the endpoint that gained hinge-ordering on 2026-08-21 — returns **an empty list** for the same claim: it reads Questions attached to relationship *edges* (the scheme-CQ system) and is blind to Questions attached to *claims* (the weighing system). See the status update above: the served-questions half of this was my own parse error; the real gap was the all-zero sensitivities, **fixed same day** via oriented mirrored-edge entries + grouped clamping.\n\n## What held — the machinery that behaved exactly as designed\n\n- **The `necessary` ceiling at one level**: `b2` capped at 0.459 by its weakest required part — first live, non-test use of `support_semantics`, working.\n- **`is_bedrock` end-to-end**: typed on the what-is-it-like leaf via the production path, verdict-claim ground minted (`CLASSIFIES_TERMINUS`), and the live residue map now reads `classified 3 · residues 1 · resolved 1 · is_bedrock 1 · fraction 0.5` — the fraction untouched by the is-side floor, exactly the accounting shipped this morning.\n- **The ten-question weighing descent** opened on the lexical marker \"outweighs\", new screen included.\n- **The completeness oracle** classified the bridge tree correctly (`fully_exposed`, all four leaves as terminus candidates). One observation, not a defect: `fully_exposed` co-exists with a live unresolved attack, because exposure certifies decomposition, not contest-resolution — within its stated spec, but the verdict word invites over-reading; noted for the future.\n- **Shared evidence is now reproducible**: the meta-analysis supports both fasting siblings, so the fair-share discount (built 2026-08-19, deliberately unadopted) has its first standing reproduction case in the live graph, and `study_count=12` / `heterogeneity=0.73` sit on the claim, visibly weighing nothing — the documented inert fields, now demonstrable on demand.\n\n## The reflection\n\nThree of the five gaps (1, 2, 5) share one shape: **a mechanism shipped correctly at the layer it was built for, and the layer above it was never specified.** The ceiling works per-node but composition through depth was unspecified; the hinge works per-tree but join-semantics arrived later and the hinge was never revisited; hinge-ordering works per-endpoint but the question ontology has two attachment shapes and the endpoint predates the second. This is the strength-layer audit's core lesson — *the defect lives in the layer nobody wrote a spec for* — showing up as a **pattern of seams between shipped layers**, which is exactly what built-to-break synthesis finds cheaply and real-material dogfooding finds slowly. Two of the five (3, 4) are display-layer honesty failures, the class the confession principle exists for: both were cases of the surface asserting more than the machinery knew.\n\nMethod note: all four examples were designed against *predicted* seams, and two of the five findings (the note lie, the CQ-server split) were **unplanned** — found only because the suite was actually run against the deployed system rather than reasoned about. The synthesis method's scope guard held: nothing here measures extraction quality, and none of these findings could have been found by prompt inspection.\n\n## Wave two (same day, founder: \"Can/should any further atomic decomposition be done? Can we learn more?\") — one principled descent per example, four pre-registered expectations, one new gap\n\nGrounded in the decomposition doctrine (molecular-not-atomic; decompose-along-a-contested-axis; the alignment warning that decomposition WITHOUT per-child evidence degrades): wave two is not wholesale atomization but one descent per example along a **different axis each**, arriving WITH its evidence alignment. Seeder: `scripts/seed_stress_suite_wave2.py` (11 claims, 15 edges; run after wave 1, whose cleanup sweeps these too).\n\n| Example | Axis | The descent |\n|---|---|---|\n| Bridge | predicate-argument + degree | the calibration attack splits into FACT (certificate lapsed) + WARRANT (accuracy unverified, though not necessarily wrong) — the undercut/rebut distinction stated as claims; and the smuggled standard surfaces: *\"the certified requirement is a sufficient threshold for reopening\"* is a **value premise inside an engineering sign-off**, now a fourth necessary part |\n| Acquisition | quantifier/scope | *\"the ONLY way\"* splits into the positive half and the universal negative — and the bridge-round attack retargets the half it always meant |\n| Tasting machine | definitional/frame | the dismissal's unstated crux minted: *\"the only meaning a rating can carry is testimony of experienced quality\"* — the functionalist attack retargets it, and under the prediction reading the dispute dissolves (run 3F's stated-on-one-side-implicit-on-the-other, reproduced synthetically WITH live strength flow: the dismissal mother fell to 0.459 through its attacked crux) |\n| Fasting | evidence-as-subgraph | the meta-analysis decomposes into per-outcome findings routed each to its own sibling (the ALIGNED configuration), and `heterogeneity=0.73` is **minted as a claim** with an attack edge on the bundle |\n\n**Expectations, scored:**\n\n- **E1 REFUTED — and the refutation is the lesson.** The evidence-as-subgraph doc expected a minted heterogeneity claim with an attack edge to press on the meta-analysis \"with no new formula\". Measured: the parent stays UNEXAMINED — an unscored edge contributes zero by design. Minting converts a property into something *contestable*; weight still arrives only through scoring. The doc's line is corrected at source.\n- **E2 CONFIRMED.** Fair-share dry run: before decomposition the metabolic sibling's discount is 0.5 (all its evidence borrowed from the shared bundle) and the weight sibling's 0.75; after per-outcome routing, **every discount is 1.0** — apparent sharing fully dissolved, the founder's 2026-08-15 correction demonstrated numerically. The discount's remaining job is genuine sharing only.\n- **E3 CONFIRMED, with an unplanned twist that became the wave's headline gap** (below). The bridge mother sharpened 0.459 → 0.368 through the deepened chain.\n- **E4 CONFIRMED as a state change.** The first end-to-end weighing answer: evidence on the state-deficit positive polarity claim flipped the question to *answered* and the weighing claim from UNEXAMINED to GROUNDED (weight 2). The *number* barely moved — the mirrored pos/neg pair nearly cancels at one piece of evidence — a calibration observation worth keeping: a single answer registers as engagement before it registers as direction.\n\n**The new named gap: edge supersession does not exist, and it double-counts — three instances in one wave.** Deprecation-over-deletion was honored by KEEPING each original edge when its sharper replacement landed, and the strength layer then counts both: the calibration leaf carries the bundled attack AND the warrant attack (same objection, twice — part of the 0.368 is duplication); the only-way claim keeps the direct attack while the same attacker also hits the universal-negative child; the weight-loss sibling counts the meta-analysis bundle's support AND the decomposed weight finding. **Decomposition-plus-retargeting without supersession inflates whatever it touches.** The ontology needs an edge-level lifecycle — a superseded edge stays visible and stops weighing — the same shape as claim lifecycle and the always-mint non-propagating link. Until then, the correction UX also has no affordance for \"this edge actually targets the part\" at all. Filed as a founder decision — **ratified and SHIPPED same day**: `supersede_edge`/`restore_edge` (successor named as `<src>-><dst>` and required to exist), `POST /edges/supersede`, and the filter in every weighing path (badge inputs, hinge, completeness walk, feed bridging), with drift-guard tests pinning each. Superseded edges stay on every display surface marked *\"no longer weighs\"*. Applied to the wave's four duplicates and live-verified: the bridge mother returned from the double-counted 0.368 to the honest single-attack 0.459, the fasting mother from 0.506 to 0.501, and the superseded calibration attack renders visibly-marked on the public claim page. The correction-UX affordance shipped the same day: the claim page's incoming supports/attacks cards offer *\"this actually bears on a part…\"* wherever decomposition children exist (one action: copy-onto-child + supersede, `retarget_to_claim_id` on the endpoint), and superseded edges render dimmed with a *no longer weighs* pill — DOM-verified live. Conceptual placement of the whole mechanism against Graphiti's bitemporal model: [supersession-and-bitemporal-lifecycles.md](supersession-and-bitemporal-lifecycles.md).\n\n**What legitimately stays undecomposed**: the is-bedrock leaf (terminal by verdict — the stop held), the weighing descent (it deepens by ANSWERING, not splitting), and everything no one contests — per the corpus's own rule, an undecomposed axis is a marked state, not a defect.\n\n## Cross-references\n\n`scripts/seed_synthetic_stress_suite.py` · [strength-layer-audit.md](strength-layer-audit.md) (the unspecified-middle pattern; the ceiling; the audit these findings extend) · [evidence-division-and-the-foundation.md](evidence-division-and-the-foundation.md) (the join-semantics problem the fasting pile reproduces) · [decomposing-value.md](decomposing-value.md) §6 (the screen the weighing example exercises) · [what-belongs-in-the-ontology.md](what-belongs-in-the-ontology.md) (the confession-channel discipline; `is_bedrock`'s ratification) · [dogfood-run-2-orthogonal-experiments.md](dogfood-run-2-orthogonal-experiments.md) (the hinge-whisper finding gap 2 explains structurally)\n"}