{"path":"research/dogfood-run-2-orthogonal-experiments.md","content":"# Dogfood Run 2: Three Orthogonal Experiments\n\n*July 7-8, 2026. Run unattended while the founder rested (covid). Design brief (founder, verbatim intent): \"Pick some randomly orthogonal topics... one topic that you expect will exercise the hinge score, another topic that you don't know what to make of, one topic that might exercise or challenge some other feature/metadata/knob we've built lately.\" Budget cap $30; actual spend well under $5 (extractions ran mostly on the Gemini fallback chain during a flagship load-shedding storm, which was itself a live test of the capacity-fallback feature shipped in run 1).*\n\n*Genre per run 1: friction log. Findings numbered G1... continuing the F1-F15 series from [dogfood-run-1-friction-log.md](dogfood-run-1-friction-log.md).*\n\n## Experiment design\n\n| Run | Topic | Chosen to stress | Sources |\n|---|---|---|---|\n| 2a | Minimum wage (pro/con pair) | **Hinge score** (the debate has a famous empirical crux: employment effects) | EconLib \"Minimum Wages\" (Neumark lineage, con) + American Prospect \"Reframing the Minimum-Wage Debate\" (net-benefit case, pro) |\n| 2b | Graeber, \"On the Phenomenon of Bullshit Jobs\" | **Genuinely unpredictable**: rhetorically rich, part-empirical part-normative part-conceptual; contested concept (\"bullshit\") at the core | Strike! Magazine original (2013) |\n| 2c | Assisted dying (pro/con pair) | **Sacredness brake + residue typing**: protected-value language on both sides; fittingness vs permissive-zone genuinely live | Not Dead Yet \"Assisted Suicide Talking Points\" (con) + Humanists UK \"The Case for Assisted Dying\" (pro) |\n\n## Experiment 2a: minimum wage (COMPLETE)\n\n**Extractions**: EconLib 57 claims / 85 CQs; Prospect 47 claims (212 nodes with scaffolding). Both stored with forensics rows and `postgres_saved: true` (the run-1 confession fixes held).\n\n**Result 1, auto-connect richness: the debate self-assembled.** 72 cross-source edges between the con and pro texts (20 ATTACKS, 31 QUALIFIES, 21 SUPPORTS) with no hint that they were related. Compare run 1's baseline (a handful of cross edges between the retribution texts): topic-dependent, but the claim-rich/edge-poor baseline is not universal. Empirically dense debates connect much more heavily than value-heavy ones.\n\n**Result 2, G1 (recall gap in weighing detection, FIXED same run).** Zero weighings detected in either text, including the Prospect essay whose entire thesis is an explicit net-benefit weighing (\"The primary criterion... should be the net benefits provided to all low-wage workers\"). The v1 lexicon had outweigh/more-important-than dialects but not the policy-analysis dialect: \"net benefits\", \"on balance\". Fix shipped mid-run: four new marker patterns + two regression tests pinning the exact missed claim text. The general lesson: weighing language is register-specific (moral-philosophy texts say \"outweigh\"; policy texts say \"net benefits\"; more registers will surface with more corpora).\n\n**Result 3, hinge mechanics verified end-to-end; resolution is the finding (G2).** The pro thesis was decomposed into its four pillars (employment, earnings, fiscal, distributional value judgment), which the LLM decomposer expanded to a 17-node tree including the monopsony mechanism chain and Card-Krueger evidence chain. Baseline hinge: perfectly flat (0.022 per child), truthfully reporting an unengaged tree. After uneven engagement (supporting evidence on the Card-Krueger leaf and the fiscal leaf, attacking evidence on the job-loss leaf), the ranking moved correctly: engaged children rose to the top, the attacked child's local strength visibly dropped (0.50 → 0.46). But the contrast is tiny: one evidence item shifts a hinge by ~0.001 (0.022 → 0.021 spread). **Calibration finding**: with QEM's additive subclaim energy over trees this size, hinge differentiation requires heavy engagement. Options (not decided): raise evidence weight in the hinge's local-strength model; propagate hinge through SUPPORTS/ATTACKS as well as DECOMPOSES_INTO; or accept that hinge is a mature-graph instrument and label low-engagement readings as such in the UI. The instrument never lied; it is just quiet.\n\n**Result 4, G3 (decompose-path weighing gap, FIXED same run).** The decomposition minted \"Broad gains outweigh concentrated losses\" (a value premise, a literal weighing) with no detection, because the weighing pass only ran at extraction time. Fix shipped: decompose children now get detection in authored mode (INVITED, never auto-opened, per the provenance split; the decomposing human is present, so the sacredness policy applies). A pre-existing silent `except Exception: pass` on the correction pipeline was upgraded to `logger.exception` in the same commit.\n\n**Result 5, G4 (run-1 meta-claim recurrence).** The decomposer again emitted a claim *about* the decomposition (\"The net-benefits criterion is comprised of four subclaims...\"), run 1's F15. Now confirmed as a pattern, not a one-off: the decompose prompt needs an explicit \"no meta-claims about the decomposition itself\" instruction. (Not fixed this run; prompt changes touch the Apr-1 outage class and deserve their own attended pass.)\n\n**Smaller friction**: G5, the text-input path derives the source slug from the text's first line rather than the provided title (Prospect landed as `this-article-appeared-in-the-summer-2016-issue-of-the-americ`); G6, the platform's URL fetcher gets bot-blocked (403/429) by EPI, Cato, Mercatus, Brookings, and Prospect while a browser UA passes — a fetcher-header hardening task; G7, during flagship load-shedding, gemini-3.5-flash occasionally emits parallel tool calls that instructor rejects (\"use List[Model] instead\"), recovered by retries: harmless but noisy, worth a targeted mode/config look.\n\n## Experiment 2b: Graeber (COMPLETE)\n\n**Extraction**: 72 claims from the Strike! original; quality self-eval 1.0.\n\n**Result 6, G8 (polemic in, attributed sociology out).** 64 of 72 claims typed *empirical* on a flamboyantly polemical essay. Spot-checking the morally-loaded ones showed why this is right rather than alarming: the pipeline converted rhetoric into **attributed, checkable sociological claims** (\"Graeber characterizes the consumer-goods explanation as a simplistic morality tale\"; \"there is a widespread cultural sentiment that...\"). This is epistemic contextualization working on the hardest register yet: the polemic's force survives as claims about beliefs, explanations, and social patterns, each individually assessable, while the moral core concentrates into the 8 normative/value claims. (It also mirrors the communication-act discipline in LawZero's Scientist-AI safety case, independently converged on: see [bengio-safety-from-honesty-and-deliberus.md](bengio-safety-from-honesty-and-deliberus.md).)\n\n**Result 7, G9 (coinage blindness).** Contested-concept detection found the canonical contested terms (social value/utility, efficiency, productive labor, dignity in labor) but not \"bullshit jobs\" itself: the text's own novel coinage, arguably THE contested concept of the piece. Hypothesis: the detector recognizes terms with established contestation patterns and under-weights a text's self-minted vocabulary. Worth a targeted prompt look.\n\n**Result 8.** The undecomposed central thesis reads `bundled` at exposure 0.0, confirming across a third register that the road from thesis to structure does not exist until walked (run 1's load-bearing baseline).\n\n## Experiment 2c: assisted dying (COMPLETE)\n\n**Extractions**: Not Dead Yet 106 claims; Humanists UK 48 claims. **88 cross-source edges, attack-dominant** (43 ATTACKS, 27 QUALIFIES, 18 SUPPORTS), versus the minimum-wage pair's qualify-dominant profile (31/20/21). Edge-type distribution is fingerprinting debate character: empirical-policy debates connect through qualification, rights conflicts through attack.\n\n**Result 9, G10 (the register ceiling of lexical weighing detection, mapped).** Zero weighing detections and zero sacredness-brake triggers on either text, with the extended lexicon live, and ground-truthed against the sources: **neither text contains a single weighing or sacredness marker.** Disability-rights opposition deliberately avoids sanctity language (its case is discrimination and coercion, a strategic register choice by the movement); humanist advocacy asserts autonomy entitlements without weighing phrases. The value conflict is carried entirely implicitly. Three registers are now mapped: moral philosophy says \"outweigh\" (run 1, detected), policy analysis says \"net benefits\" (G1, now detected), rights advocacy says neither (undetectable lexically by construction). This is the empirical case for the planned semantic tier: when a text yields value premises but zero lexical weighings, an LLM pass should ask whether an implicit weighing structures the argument. (Ironic footnote: the pro campaign's page *title* contains \"life is precious\", near-sacred language living in the packaging while the body stays secular.)\n\n**Result 10: the classifier is two for two on blind reads, in opposite directions.** Run 1: the hand-decomposed retribution leaf, proposed `fittingness` at 1.00. Run 2: Not Dead Yet's core trade-off claim (\"the legal right of a small group results in systemic harm and danger for others\"), which extraction had typed `value_premise`, proposed **`empirical` at 0.90**, with textbook reasoning: the claim is a checkable consequence prediction with existing jurisdictional evidence, not value bedrock. The instrument discriminates: it neither over-types toward residues (the convergence-illusion direction) nor flattens everything to empirical. The proposal was confirmed with a grounded verdict-claim (`claim_780586c81f3c`, attributed to the dogfood operator, challengeable and reversible; an unattended judgment call the founder can overturn), moving the **residue map to its first non-trivial reading: 2 classified termini, 1 residue, 1 resolved, residue fraction 0.5.**\n\n**Result 11: the first cross-register flattening dataset.** Disagreement-preservation scores across all five extractions: EconLib encyclopedia 0.942 (86 pairs, 5 flagged), Graeber polemic 0.922 (90 pairs), Humanists UK advocacy 0.909 (33 pairs), Not Dead Yet advocacy 0.897 (29 pairs), Prospect advocacy essay **0.818** (192 pairs, 35 flagged). Reference-style prose preserves best; one-sided advocacy essays flatten most, directionally consistent with the sycophancy hypothesis (reconstructing oppositions from advocacy gives the model the most room to blur). Five points is a pattern to test, not a result, but it is exactly the shape of the Flattening Eval the funding applications propose.\n\n## Meta\n\nThe entire run executed during a Gemini flagship 503 storm, which turned into an unplanned stress test of run 1's capacity-fallback chain: all five extractions completed, mostly on `gemini-3.1-flash-lite`, with zero manual intervention (G7's instructor parallel-tool-call hiccups included). The confession layer reported the one real failure honestly (the Prospect URL fetch 429), and the text-path fallback recovered it. Two recall gaps surfaced by the experiments (G1 lexicon dialect, G3 decompose-path weighing) were fixed and regression-tested inside the run; suite at 847. Total spend: well under $5 against the $30 cap.\n\n**Scoreboard across runs 1-2**: 4 experiments, 7 sources extracted, 2 debates self-assembled across sources, 2 blind classifier reads (both discriminating correctly), 1 calibration finding (hinge contrast), 3 register findings (weighing dialects), 2 same-day fixes with tests, first preservation dataset, residue map at 2/1/1.\n\n**Continued**: run 3F, the frontier extraction experiment (a frontier model executing the full pipeline by hand during the Gemini credit outage; H-series findings continue this G-series): [frontier-extraction-experiment.md](frontier-extraction-experiment.md).\n"}