{"path":"research/strength-layer-audit.md","content":"# The Strength Layer, Audited\n\n**Date**: 2026-08-17\n**Status**: Findings, measured against the live graph and the shipped code. **Three defects. Two were fixed on 2026-08-17 (polarity and direction); the remaining two are founder decisions because they move strengths across the whole graph.**\n**Prompted by**: the founder, on the observation that Deliberus does not iterate — *\"I want and need more insight into all of this. The paper and theory around it. How it relates to our graph and the architecture around it.\"*\n\n---\n\n## 1. The theory the implementation departs from\n\nGradual argumentation semantics is a **class of evaluation functions**, not a single one. Three levels are routinely conflated and should not be:\n\n**QBAF** is the *structure*. A Quantitative Bipolar Argumentation Framework: arguments carrying numeric base scores, with both attack and support relations. Bipolar contrasts with Dung's original frameworks, which had attacks only.\n\n**Gradual semantics** is the *class of functions* mapping such a structure to a strength per argument on a continuum, as opposed to Dung's extension-based semantics giving a binary in/out verdict.\n\n**QEM** — the Quadratic Energy Model — is *one member* of that class. Others are DF-QuAD, the Euler-based family, h-categorizer, and the aggregative semantics. `deliberus/graph/qbaf.py` records why QEM was chosen over four of them: no saturation, open-mindedness, gravity toward the initial weight, symmetry.\n\n**Potyka's formulation is a continuous dynamical system.** Strengths are the *fixed point* of an update applied to all arguments repeatedly until convergence, and a substantial part of that work concerns when convergence is guaranteed at all — cyclic graphs can oscillate or diverge.\n\n**Deliberus computes a single pass.** QEM once per edge from that edge's critical-question answers, then (until 2026-08-17) an **arithmetic mean** across the claim's edges, now signed QEM energy, then one further QEM update using the already-computed strengths of direct decomposition children. The recursive descent through decomposition children is memoized, so transitive flow does exist **along the decomposition channel**. There is no iteration over the attack/support channel at all.\n\n**Cycles were the plausible justification and the graph does not support it.** Measured: **3 two-cycles and 1 three-cycle** in the support/attack channel, against roughly 3,950 such edges, and **zero** cycles in decomposition. Iteration would converge over almost the entire graph, with a handful of cases needing Potyka's conditions. So the single pass is not buying much safety.\n\n**The averaging step was not from the literature, and it is gone.** Aggregating a claim's edges by arithmetic mean meant **a weak edge diluted a strong one**, cancelling part of the open-mindedness the module was chosen for. It is now signed energy, per the file header's own specification.\n\n---\n\n## 2. The severe one: polarity was ignored, and direction was preferred backwards\n\n*Corrected 2026-08-17, same day. The first version of this section said a claim with 61 supporters displayed `NO DATA`. **That was wrong** — it tested the aggregation function with empty input rather than the endpoint, which had an incoming fallback I had not read. The real defects are below, and one of them is worse than the claim I got wrong.*\n\nThe extraction prompt fixes the convention explicitly: **\"A supports B\" means A is evidence FOR B** (`cross_structure.py`). So a claim's supporters and attackers sit on its **incoming** edges.\n\n**Defect one, and the severe one: polarity was ignored entirely.** `compute_claim_badge` read each edge's scheme and critical-question strengths and never read its *type*. A well-supported attack raised a claim exactly as a well-supported defence did. Measured in the live graph: **1,567 incoming `ATTACKS` edges were counted as though they were supports.**\n\n**Defect two: direction preference was backwards.** The endpoint queried outgoing edges first and fell back to incoming only when there were none. So a claim that both makes inferences and receives them was read from the wrong side — **54 claims** — and a claim with only outgoing edges displayed a confident badge derived from arguments it *makes* rather than any evidence for it. The synthesis path in `truth_graph.py` had the same query with no fallback at all.\n\n**Both are fixed.** The badge now reads incoming edges, excludes `DECOMPOSES_INTO` so parts cannot enter through two channels at once, signs each contribution by polarity, and aggregates as QEM energy rather than an arithmetic mean — which is what the module's own file header specified all along and the claim-level function never implemented. Missing polarity now raises rather than defaulting, because a silent default is how the original survived.\n\n**Corpus impact, measured.** Claims displaying a badge fall from **3,828 to 427**. Of the difference, **3,401 now honestly read `NO DATA`** because no evidence bears on them at all, and **54 are read from the correct side for the first time**. That is a large visible change to the site and it is the right direction: a confident badge with no evidential basis was the graph telling readers something it had no grounds for.\n\n### Where it came from\n\nNot a misunderstanding of QBAF. The March design spec (`.claude/specs/scheme-bounded-decomposition/design.md`) specifies `compute_edge_strength` and `strength_to_badge` — **both edge-level, both correct**. It never specifies a claim-level aggregate. `compute_claim_badge` arrived in the same commit as an undesigned extra, and when the endpoint was wired months later, \"the edges attached to this claim\" was implemented as outgoing.\n\n**The defect lived in the layer nobody wrote a spec for**, which is the pattern this session has now found four times.\n\n**And the generalisable form is not \"we needed more context\".** The convention it violated was documented, and whoever wrote the endpoint plausibly knew it. What was missing was a **question**: an unspecified layer has no moment at which *\"which direction should this read?\"* must be answered, so it never was. The tests could not catch it either, for a structural reason — they pinned the implementation rather than the intent, and **a test authored from the code cannot falsify the code**, which is the fabricated-fixture failure one level up. Cross-project treatment, including the three distinct failure classes only one of which retrieval fixes: `~/dotfiles/docs/context_sufficiency_before_code_changes_2026_08_17.md`.\n\n### One consequence the fix did NOT deliver, contrary to what this document implied\n\nThe original finding bundled a user-facing consequence into the direction defect: *`add-evidence` writes `evidence → claim`, i.e. incoming, so a user can attach evidence and watch the badge not move.* Fixing the direction was expected to resolve it. **Measured against the shipped functions on 2026-08-17: it does not.**\n\nAn edge whose own critical questions are unanswered computes to 0.5, and claim-level contributions are offsets from 0.5, so such an edge contributes **exactly zero** energy. Ten of them leave a claim at 0.500. Attaching evidence therefore still cannot move a badge on its own — no longer because the query reads the wrong side, but because an unexamined supporter is *designed* to count for nothing. That design is right, and `qbaf.py`'s header says why: an edge at 0.5 means \"nobody has examined this\", and summing raw strengths would let three unexamined supporters manufacture a strong claim out of silence. A second gate sits in front of it — a user-created evidence edge carries no scheme until the correction pipeline's LLM pass assigns one, and the badge only reads scheme-bearing edges.\n\nSo the correction-UX promise is a **two-step loop presented as one**: attach evidence, then answer the critical questions it raises. Recorded here rather than adjusted, because the fix is either copy or interaction design and neither is a strength-layer decision. The general lesson is the one this session keeps producing: **a defect and its user-visible symptom are different objects, and fixing the first does not entitle you to strike the second.**\n\n### One consequence logged rather than tuned\n\nMethodology tier raises an *edge's* prior, and the claim now moves off its own prior by the energy that edge supplies rather than inheriting the edge's value. **Methodology therefore reaches a claim in damped form.** Whether that damping is right is a separate question from the direction fix, and it is in TODO rather than adjusted here.\n\n## 2b. The badge words are the founder's, from a hand-drawn sketch — and the sketch specified the fix\n\n*Added 2026-08-17, after the founder asked where the labels came from. Traced rather than recalled.*\n\n**\"VALID\" is his own word, from Sketch 2**, one of the 2009–2013 hand-drawings: a conclusion at the top with three *\"VALID\" badges* connecting down to argument bundles ([sketches.md](../sketches.md) § Sketch 2). The other ten terms are generalisations of that one, entered in `ae8a2f7` (2026-03-30) and specified as a table in `.claude/specs/scheme-bounded-decomposition/requirements.md` AC-5.4 — CREDIBLE for source-based schemes, WELL-EVIDENCED for causal, GROUNDED for values, and so on.\n\n**And the sketch already specified the state that the shipped code lost.** As recorded in [scheme-bounded-decomposition-and-evidence-as-subgraph.md](scheme-bounded-decomposition-and-evidence-as-subgraph.md) line 307:\n\n> \"The **VALID badge** = all CQs for this scheme addressed (green), some addressed (blue/amber), **none (orange)**, one failed (red)\"\n\n**Four states, with \"none addressed\" distinct and warning-coloured.** The specification then compressed it to three numeric bands — AC-5.3: *\">0.8 green, >0.5 blue, >0.2 amber, <0.2 challenged\"* — and **\"none addressed\" ceased to exist as a state**, because a claim with nothing answered computes to exactly 0.5 and 0.5 falls inside the blue band.\n\n**So the grey `UNEXAMINED` state added today is a restoration, not an invention.** The design intention survived thirteen years from a hand-drawing into a research doc and was lost in the translation to acceptance criteria.\n\n### Why it defaulted positive rather than negative, which is the founder's question\n\n*Founder, 2026-08-17: \"Why the heck was this not the case to begin with? I'm not sure I understand how the platform could default to this kind of structural sycophancy.\"* The name is his and it is the right one. The mechanism is arithmetic rather than intentional, and the arithmetic had to land somewhere.\n\n**There is no band that means \"no reading\".** AC-5.3 maps one number onto four colours, and any such scheme forces the *no-measurement* case into one of the *measurement* buckets — it cannot abstain. Unanswered computes to exactly 0.5, the boundary was written `>= 0.5`, and so it landed on the positive side. Had the comparison been strict, the same claim would have read *amber* and been wrong in the opposite direction. **The sycophancy was not chosen; it is the default direction of a coin that had to land somewhere**, and the coin exists because a four-state design was flattened onto a one-dimensional scale.\n\n**Two acceptance criteria, one page apart, and only one of them knew.** AC-5.1 states *\"0.5 for unanswered\"* — the specification knew exactly what that number meant. AC-5.3 then compares it against thresholds meaning *how good is this*. The knowledge existed in the document and did not travel the eight lines to where it was needed.\n\n**And it survived five months because it looked plausible.** A page reading `CREDIBLE` under an expert-opinion claim is unremarkable; the word matches the scheme, nothing appears broken. It is only wrong if you know nobody answered anything — **which the page had no way of saying.** That is a confession-channel failure in its purest form: with no vocabulary for *\"I have no reading here\"*, the system said the nearest thing it could, and the nearest thing was positive.\n\n**The uncomfortable part, and the reason it belongs in this corpus rather than a bug tracker.** This is precisely the failure Deliberus exists to detect — a confident-sounding claim with nothing underneath it — occurring *inside Deliberus*, on the surface that certifies other people's claims. The instrument built to measure unsupported confidence was itself displaying unsupported confidence, for five months, on 115 of the 116 claims that carried any structure at all.\n\n### Why this is the sharpest instance of the day's pattern\n\n**The code implements its specification correctly.** Nothing downstream was wrong. The loss happened *in the specification*, when a four-state semantic became a three-band threshold — and once \"none addressed\" had no representation there, no test, review or type could catch its absence. This is the unspecified-middle finding with the layers inverted: §2's defect lived where no spec existed, and this one lives where the spec **replaced** a richer intention with a poorer one.\n\nTwo details worth keeping. **The word `UNEXAMINED` was already in the spec's own table**, as the *low* term for the values cluster — so the vocabulary existed and was attached to the wrong axis. And **AC-5.1 is correct and honest**: *\"0.5 for unanswered\"*. The defect is entirely in AC-5.3 treating that 0.5 as a *position on a scale* rather than as *the absence of a reading*. A number meaning \"nobody has looked\" was fed to a comparison that assumes every number means \"how good is it\".\n\n**The generalisable rule:** a sentinel value that means *no measurement* must not be routed into a comparison that assumes *a measurement*. It is the same failure as a temperature of zero meaning \"sensor offline\" being averaged into a daily mean.\n\n## 3. Decomposition pays: splitting a claim raises its strength\n\n`subclaim_energy(strengths) = Σ (s − 0.5) × 0.3` is additive over children with no normalisation and no correlation term. Measured directly:\n\n| Children | Each at | Mother's strength |\n|---|---|---|\n| 3 | 0.70 | 0.516 |\n| 6 | 0.70 | **0.557** |\n| 12 | 0.70 | **0.671** |\n| 3 | 0.90 | **0.557** |\n\n**Six mediocre children score exactly what three excellent ones do, and twelve mediocre children beat them outright.** The scoring function pays you to split.\n\nIn a system whose thesis is that decomposition is neutral truth-seeking, that is the Goodhart hazard the corpus already names as a directive: a measure that becomes a target ceases to be a good measure. Nobody is gaming it today because nobody outside is contributing. **It becomes exploitable on the day the platform succeeds**, which is the worst moment to discover it.\n\n---\n\n## 4. What the Dr Smith example actually exposes\n\nThe founder connected this to the evidence question and to the mother-claim pattern, and the connection is exact.\n\n*\"Dr. Smith is credible as an expert in labor economics\"* decomposes into: a publication record, an institutional appointment, publication-implies-expertise, and domain match. **Those parts are jointly required.** Domain match failing does not reduce credibility a little; it guts the claim. That is a **conjunction**.\n\nNow consider three independent studies all finding the same effect. Those are **corroborative** — each raises confidence, and losing one leaves the others intact.\n\n**The energy function treats both identically.** Additive, with each child contributing in proportion to how far above 0.5 it sits. For corroboration that is roughly right. For conjunction it is wrong in the dangerous direction: a mother claim can score well while one *necessary* part is weak, because three strong siblings outvote it. Conjunction wants something closer to a minimum or a product; corroboration wants a sum.\n\n**So `DECOMPOSES_INTO` is carrying two different relations under one name**, and the strength function cannot tell them apart because the ontology does not record which is which. That is the same shape as the taxonomy findings from runs 3F, 5 and 6 — a distinction the register demands that the closed vocabulary cannot express — and it lands on the most load-bearing computation in the system.\n\nIt also composes with the splitting incentive above: **splitting a conjunctive claim into more parts raises its strength while making it strictly more fragile.** That is the exact inverse of what the founding ideal wants decomposition to do.\n\n**Fair-share discounting is a partial and separate fix.** It corrects for shared *evidence* across children. It does nothing about conjunction, which is about the logical relation between the children and their mother.\n\n---\n\n## 4b. Measured corpus-wide, 2026-08-17: the strength layer is correct and inert\n\nThe section above reasons from the code. This is the measurement, taken from the live site by sampling 250 random claims and calling `GET /claims/{id}/badge` on each.\n\n| | |\n|---|---|\n| Claims probed | 238 (12 timed out) |\n| `NO DATA` — no incoming evidence at all | 122 |\n| Carrying evidence | **116** |\n| Of those, computing to exactly **0.500** | **115** |\n| The remaining one | 0.501 |\n\n**Every one of the 116 displays its scheme cluster's positive term, in blue** — `VALID`, `WELL-EVIDENCED`, `STRONG ANALOGY`, `SUBSTANTIATED`, `BEST EXPLANATION`, `GROUNDED`, `CREDIBLE`, `SUPPORTED`, `SOUND`. The spread of *words* is a spread of scheme clusters, not of strength. There is no strength: 115 of 116 sit exactly on the neutral prior.\n\nThe cause is §4's mechanism operating everywhere at once rather than occasionally. **No critical-question polarity claim in the corpus has ever received evidence**, so every edge computes to 0.5, every contribution is an offset of exactly zero, and every claim lands on its own prior. Sampling `/claims/{id}/cqs` across the same 116 found 34 with critical questions attached and every polarity claim still sitting at its default confidence.\n\nTwo consequences worth stating separately.\n\n**The polarity fix was correct and cannot yet express itself.** A claim with four incoming attacks and a claim with four incoming supports currently compute identically, because both sets contribute zero. The fix is not undone by this — it is waiting for the human work the design requires. But it does mean the corpus contains no evidence that the fix changed any *displayed* number, and nobody should claim otherwise until a CQ is answered somewhere.\n\n**The neutral point falls on the positive side of the badge boundary**, and that is a confession-channel gap rather than an arithmetic one. `strength_to_badge` reads `>= 0.5` as the high term in blue, so 0.500 — which means *nobody has examined this* — renders as `WELL-EVIDENCED`. The gray `NO DATA` state exists for a claim with no edges; the have-edges-nothing-answered state has no visual equivalent and currently borrows the confident one. A reader cannot distinguish \"examined and holding up\" from \"never examined\". Not fixed here: it re-labels a shipped surface, so it is a founder call.\n\nA boundary artefact found in the same sample and worth one line: one claim returns `INVALID` in amber while displaying `strength: 0.5`, because the displayed figure is rounded to three places while the term is chosen from the unrounded value, and a decomposition ceiling had pulled it just under. Two claims can therefore show the same number and opposite verdicts.\n\n## 5. Ordered by severity, all of them founder decisions\n\n1. ~~Badge directionality and polarity.~~ **Fixed 2026-08-17.**\n2. **Conjunctive versus corroborative decomposition.** An ontology addition before it is a formula change: the graph cannot currently say which kind a decomposition is.\n3. **The splitting incentive.** Normalising by child count is the obvious candidate and it interacts with both items above, so it should not be decided alone.\n4. **Iteration.** Available — the graph is nearly acyclic — and the smallest of the four in effect. Worth doing after the direction question is settled, since iterating in the wrong direction would only propagate the first defect further.\n5. ~~The mean across edges.~~ **Replaced 2026-08-17** by the signed energy the file header always specified.\n6. **Methodology damping**, new as of the fix: a tier raises an edge's prior, and the claim now moves off its own prior by that edge's energy rather than inheriting the edge's value, so methodology reaches a claim in damped form. Decide whether that is right.\n\n---\n\n## Cross-references\n\n[self-similar-decomposition-and-claim-ontology.md](self-similar-decomposition-and-claim-ontology.md) (the mother-claim pattern this audits) · [decomposition-axes.md](decomposition-axes.md) §2b (fair-share discounting and evidence routing) · [scheme-bounded-decomposition-and-evidence-as-subgraph.md](scheme-bounded-decomposition-and-evidence-as-subgraph.md) (what evidence is) · [qbaf-gradual-semantics-research.md](qbaf-gradual-semantics-research.md) (the original selection of QEM over its alternatives) · [graph-daemons-design-space.md](graph-daemons-design-space.md) (why a daemon layer must wait for these)\n"}