{"path":"research/flywheels-in-building-deliberus/literature-report-2026-10-22.md","content":"# Flywheels in building Deliberus: what the literature says\n\nResearch report, 2026-10-22. Question: what makes a loop compound rather than repeat, which loops lower the demand of other loops on a scarce constraint, how loops fail, and how a loop's gain is measured. The constraint in every section is the founder's attention (reading machine output, labelling cases, ruling on decisions).\n\nConvention in this report: each item has a mechanism paragraph, a line \"For Fredrik\" on what it implies for a solo founder whose attention is the constraint, and a line \"Definition\" where it contradicts or sharpens the agreed flywheel definition (closes; leaves a durable store; the store lowers next-turn cost or raises yield; part of it moves without a push; the saving is named; plus condition, friction, limit, shadow, error propagation, governor, signed gearing). Section 6 ends with the ranked practices, the measurement methods, the failure modes and a verification note. Sources are inline. Where I only saw a search summary or an abstract, I say so in the verification note.\n\n## 1. First principles and systems\n\n### 1.1 Reinforcing and balancing loops, Forrester, Senge\n\nSystem dynamics (Jay Forrester, *Industrial Dynamics*, 1961) models any system as stocks changed by flows, steered by two loop types. A reinforcing loop amplifies a change (more of X causes more of X). A balancing loop pushes a variable back toward a target. Senge's archetypes in *The Fifth Discipline* (1990) are named combinations of these. \"Limits to Growth\" is a reinforcing growth loop attached to a balancing loop that weakens it as a constraint is approached; \"Success to the Successful\" is two efforts competing for one finite resource, where the one ahead gets a growing share and starves the other (descriptions confirmed in a secondary summary: [ALNAP archetype sheet](https://library.alnap.org/system/files/content/resource/files/main/PG01E-System-Archetypes-at-a-Glance.pdf)). Two points matter beyond the vocabulary. First, every real flywheel has a balancing loop somewhere, so the question is what that limit is, not whether it exists. Second, balancing loops with delay make people overreact when the first push seems not to work, which is a documented cause of abandoning a loop one turn before it pays.\n\nFor Fredrik: your \"gearing with a sign\" is Success to the Successful. If two wheels draw on one stock (his attention), the one that already pays will starve the one that would pay more later. Decide the allocation on purpose.\n\nDefinition: sharpens. \"Limit\" and \"governor\" are two different balancing loops in the literature: the natural limit (a resource runs out) and the designed governor (a control that stops errors). Keep both names. The definition also lacks the delay: a wheel's gain is often invisible for several turns, so \"the next turn costs less\" has to be read over a window, not per turn.\n\n### 1.2 Meadows' leverage points\n\nDonella Meadows' essay \"Leverage Points: Places to Intervene in a System\" (1999; first circulated 1997) ranks twelve places to intervene, weakest to strongest: numbers; buffer sizes; stock-and-flow structure; delays; strength of balancing loops; gain of reinforcing loops; information flows (who sees what); rules; the power to self-organise (change the system's own structure); goals; the paradigm; transcending paradigms. The list is reproduced at the [Dartmouth reprint](https://sites.dartmouth.edu/climateaction/?p=27) (that page's numbering of items 7 to 9 is garbled in the copy I fetched; the order above is Meadows' and matches the [Wikipedia summary](https://en.wikipedia.org/wiki/Twelve_leverage_points)). Meadows' own caveat is that the ordering is a guide, and that leverage points are often counterintuitive, with people pushing the right lever in the wrong direction.\n\nFor Fredrik: most of the practice in this report sits at three levels: information flows (what he sees and when), rules (the canon, ruling ledger, checks) and self-organisation (loops that change the other loops). That is high on Meadows' list, which supports spending attention there rather than on parameters (thresholds, prompt wording).\n\nDefinition: sharpens. The \"gearing\" idea corresponds to a meta-loop at the self-organisation level. Meadows also warns that reinforcing gain is high leverage and high risk: raising gain without raising the governor speeds up errors as much as progress.\n\n### 1.3 Goldratt and the Theory of Constraints\n\nIn *The Goal* (1984) and the later formalisation, Goldratt gives five focusing steps: identify the constraint; exploit it (squeeze more from what it already has); subordinate everything else to it (align non-constraints with the constraint's pace, including policies and measures); elevate it (add capacity) only if it still limits; then repeat and watch for inertia, meaning that old rules built around the former constraint become the new one. The steps are consistent across sources ([summary](https://www.learnsignal.com/blog/theory-of-constraints-five-focusing-steps-explained/); [Management and Accounting Web on Goldratt 1990](https://maaw.info/ArticleSummaries/ArtSumGoldratt90WhatTOC.htm)). The core finding is that improving a non-constraint is wasted effort, and can be harmful, since it builds queues in front of the constraint.\n\nFor Fredrik: attention is the constraint, so \"subordinate\" is the strongest rule here. Every machine pass should be run at the rate his review can absorb, and no faster. A pass that produces more output than he can read does not add capacity; it adds queue. Exploit before elevate: reduce waste in what he reads (ordering, summaries, dedup) before adding any reading capacity.\n\nDefinition: sharpens strongly. Condition (5) \"what it saves is named\" should be the first gate: a loop qualifies as a flywheel only if it lowers demand on the constraint. A loop that speeds up a non-constraint is a treadmill with an optimistic name. Step 5 (inertia) also names a failure the definition lacks: the store of rules itself becomes the constraint when rules accumulate faster than they are retired.\n\n### 1.4 Engelbart: bootstrapping and the A/B/C activities\n\nEngelbart's 1962 report, *Augmenting Human Intellect: A Conceptual Framework* ([Stanford-hosted text](https://web.stanford.edu/class/history34q/readings/Engelbart/Engelbart_AugmentIntellect.html)), argues that the human system (tools, concepts, methods, training) should be improved as a whole. In its section on interdependence and regeneration it points out that gains feed back: better symbol manipulation supports better concept structuring, which enables better symbol manipulation. The explicit A/B/C scheme is from his 1990s work, summarised at the [Doug Engelbart Institute](https://www.dougengelbart.org/content/view/191/): A is the core work, B is any activity that improves how A is done, C is any activity that improves how B improves. The institute traces the model to *Toward High-Performance Organizations* (section 3, \"The ABC Model of Organizational Improvement\") and calls bootstrapping the C activity: use what you build to improve the group that builds it. I could not retrieve a printed year for these 1990s texts from that page; the 1962 report does not itself state A/B/C.\n\nFor Fredrik: Deliberus the product is A. Eval sets, rulings, canon and checks are B. The review of whether B is paying off (how much attention a case costs this month versus last) is C. The literature's central claim is that the highest-payoff work is at C, and that C is the layer organisations skip.\n\nDefinition: sharpens. The definition's \"gearing with a sign\" is the B/C relation: a C loop is a flywheel only if it lowers the demand of B loops on the constraint. Also, Engelbart treats the improvers as part of the system being improved (the builders use their own tools); in your case Claude Code agents are both the work-doers and the objects of improvement, which makes the C level unusually cheap.\n\n## 2. Software engineering\n\n### 2.1 Tests and CI as accumulating stores; DORA\n\nA test suite is the clearest case of a store that lowers later cost: each test, once written, checks forever for near-zero marginal attention. DORA's long-running research programme (Forsgren, Humble, Kim, *Accelerate*, 2018, and annual reports) names the capabilities that statistically predict delivery performance. The [current DORA capability list](https://dora.dev/capabilities/) contains 34, including continuous integration, test automation, trunk-based development, working in small batches, work-in-process limits, documentation quality, visibility of work in the value stream, proactive failure notification, team experimentation, learning culture and generative culture. The 2024 report (secondary summaries: [InfoQ](https://www.infoq.com/news/2024/11/2024-dora-report/), [RedMonk](https://redmonk.com/rstephens/2024/11/26/dora2024/)) found that heavier AI adoption was associated with lower delivery stability (about 7%) and throughput (about 1.5%) despite gains at the task level, from a survey of roughly 39,000 respondents. The [2025 report page](https://dora.dev/research/2025/dora-report/) states that AI mainly amplifies an organisation's existing strengths and weaknesses, and that the return comes from the surrounding system.\n\nFor Fredrik: AI coding agents amplify whatever feedback system exists. If the checks are weak, they produce faster, less stable output; if the checks are strong, they produce faster stable output. The checks are the asset to build first.\n\nDefinition: sharpens. Condition (3) should include \"amplifier risk\": a store that lowers cost for correct work also lowers cost for incorrect work unless a verifier sits in the loop. The DORA numbers are survey correlations and should not be quoted as causal.\n\n### 2.2 Blameless postmortems to automated checks (Google SRE)\n\nThe [SRE book chapter on postmortem culture](https://sre.google/sre-book/postmortem-culture/) sets out the mechanism. Define in advance which events require a postmortem; focus on contributing causes, not individuals, since blame suppresses reporting; include impact, root causes and prioritised action items; review for completeness; and file the result in a shared repository. Culture practices include a \"postmortem of the month\", reading clubs on old postmortems and the Wheel of Misfortune role-play. The loop compounds only if action items change the system (a check, a guard, an alert); the chapter itself does not describe how items are tracked to closure, and that is where practice commonly fails.\n\nFor Fredrik: your rulings are his postmortems. A ruling that ends as prose in a ledger is a postmortem with no action items. A ruling that ends as an executable check or a one-line canon rule is closed.\n\nDefinition: sharpens condition (2): the durable store must be one that acts without being read. A document nobody loads is not a store in the flywheel sense.\n\n### 2.3 Paved road, golden paths\n\nNetflix's \"paved road\" and Spotify's \"golden path\" (from engineering blogs, which I did not fetch; treat as secondary) are the practice of making the preferred way the default way, so that most engineers never make the decision at all. The mechanism is the same as a good default: the cost of the right choice falls to zero and the wrong choice needs effort. DORA lists platform engineering among its capabilities.\n\nFor Fredrik: the agents are the \"engineers\". Defaults in the repo (templates, scaffolds, hooks, canon) decide what they do without being asked, which is the cheapest governor there is.\n\nDefinition: neutral. It adds a name for \"part of it keeps moving without a push\" in design terms: the default.\n\n### 2.4 Toyota: jidoka, andon, poka-yoke, kaizen\n\nJidoka (autonomation) means machines and processes that detect an abnormality and stop themselves; andon is the signal and cord that stops the line; poka-yoke is mistake-proofing in the process; kaizen is continual improvement from the analysis of recurring issues ([Toyota UK on andon](https://mag.toyota.co.uk/andon-toyota-production-system/)). A common implementation failure, reported in practitioner sources, is to restart the line after the quick fix and skip the root-cause step, so the same defect returns. In software the analogues are CI that blocks merge, type systems, linters and pre-commit hooks (mapping from my own knowledge; the sources I fetched were about manufacturing).\n\nFor Fredrik: stop-the-line is the correct governor for machine passes. A pass that fails its own verifier halts and reports; it does not continue and hand him a pile that mixes good and bad.\n\nDefinition: sharpens \"governor\": the strong form is stop-the-line (the loop halts itself), the weak form is a warning (it needs a human to look).\n\n### 2.5 ML data flywheel, active learning, human-in-the-loop labelling\n\nBurr Settles' [*Active Learning Literature Survey*](https://research.cs.wisc.edu/techreports/2009/TR1648.pdf) (University of Wisconsin technical report 1648, 2009) summarises the central idea: a learner that chooses which examples to label can reach a given accuracy with fewer labels. Strategies include uncertainty sampling and query by committee (the strategy names are from my knowledge of the survey; my searches confirmed the report and its central claim, not those sections). Tesla's \"data engine\", as described in Karpathy's talks and secondary write-ups ([IEEE Spectrum](https://spectrum.ieee.org/tesla-autopilot-data-deluge)), is the same loop at industrial scale: deploy, detect disagreement or rare cases with triggers, collect them, label, retrain, redeploy. Labelling cost was a stated constraint, and drivers acted as implicit labelers. Eugene Yan's [Product Evals in Three Simple Steps](https://eugeneyan.com/writing/product-evals/) adds the practical version: once an evaluator is precise enough it can flag likely failures in unlabeled data so that humans label those first.\n\nFor Fredrik: the human labels the cases where machine and verifier disagree, or where machine confidence is low, first. Labelling in sequence or at random wastes the scarce resource on cases the system already handles.\n\nDefinition: sharpens. The flywheel's \"next turn yields more\" has a standard quantitative form here: label-efficiency (accuracy reached per human label). It also adds a risk: the loop selects hard cases, so the store skews toward the tail and its aggregate pass rate understates typical performance. State what population the store represents.\n\n### 2.6 Evals as durable assets\n\nHamel Husain and Shreya Shankar's [evals FAQ](https://hamel.dev/blog/posts/evals-faq/) (hamel.dev) states a consistent position. Error analysis is the most important evals activity and decides which evals to write. Read traces and write open-ended notes first (open coding), then group the notes into a failure taxonomy and count (axial coding). Do not write a rubric before reading data: reviewers working from a rubric miss problems outside it, and criteria change as more traces are read. Annotate at least 30 traces yourself, review at least 100 diverse traces, and stop when new traces stop revealing new failure modes (\"theoretical saturation\"). Prefer binary pass/fail to 1-5 scales. Build one judge per failure mode, validate it against human labels with true positive and true negative rates on a held-out split, and distrust a 100% pass rate. They report 60-80% of development time going to error analysis and evaluation. Anthropic's engineering post [Demystifying evals for AI agents](https://anthropic.com/engineering/demystifying-evals-for-ai-agents) agrees: start with 20-50 tasks drawn from real failures; read transcripts to separate real agent mistakes from grader mistakes; use code graders where possible, model graders with calibration, human graders as the standard; treat an eval as a living artifact with an owner; and when an eval saturates, \"graduate\" it into a regression suite and add harder tasks. It says explicitly that the value of an eval compounds over the agent's lifetime.\n\nFor Fredrik: your human-labelled cases are the product's moat and its test suite at once. The literature's most consistent advice is that a small, real-failure-derived, binary-labelled set beats a large synthetic one, and that reading raw cases yourself, early, is not delegable.\n\nDefinition: sharpens. \"Criteria drift\" (Shankar et al., section 5.1 below) means the store's meaning changes as it is used. Version labels, record the criterion in force when each was labelled, and treat relabelling after a ruling change as a cost of the loop.\n\n### 2.7 Architecture Decision Records\n\nMichael Nygard's 2011 post \"Documenting Architecture Decisions\" proposed a short record per decision with title, context (the forces), decision, status (proposed, accepted, deprecated, superseded, with a pointer to the replacement) and consequences, including negative ones ([arc42 summary](https://faq.arc42.org/questions/C-9-3/); [Fowler](https://www.martinfowler.com/bliki/ArchitectureDecisionRecord.html) adds that records are not edited when a decision changes but linked to the superseding one). The mechanism is that the reason for a decision survives its author's memory, so the next person does not re-argue it or break it unknowingly.\n\nFor Fredrik: your ruling ledger is an ADR log. The two fields that make it work as a flywheel are status with supersession (so stale rulings stop applying) and consequences (so a ruling can be checked against what happened). I did not retrieve Nygard's own text; the field list comes from secondary summaries.\n\nDefinition: sharpens \"shadow\" and \"decay\": the ADR literature's answer to lock-in is supersession, not deletion.\n\n## 3. Academic research and knowledge work\n\n### 3.1 Luhmann's Zettelkasten\n\nJohannes Schmidt's chapter \"Niklas Luhmann's Card Index: Thinking Tool, Communication Partner, Publication Machine\" (in Cevolini (ed.), *Forgetting Machines*, Brill, 2016) and its shorter 2018 version in *Sociologica* ([listing](https://pub.uni-bielefeld.de/record/2942471)) describe a system with fixed numbered locations, no predetermined order, and notes whose meaning comes from their links to other notes. Luhmann's own description, as quoted there, is that he wrote notes in order to surprise himself: the index returned combinations he could not have deduced. I could not read the full text (the PDF I fetched was unreadable), so this rests on abstract-level summaries.\n\nFor Fredrik: the mechanism is link density plus retrieval by structure, so the store \"talks back\". For Deliberus this is close to the claim graph itself. A store you only append to is an archive; a store whose links produce things you did not put in is a partner.\n\nDefinition: sharpens condition (3): \"yields more\" should include surprise (output not in any single input). It also contradicts nothing.\n\n### 3.2 Matuschak's evergreen notes\n\nAndy Matuschak's [evergreen notes](https://notes.andymatuschak.org/Evergreen_notes) are notes written to be revised, linked and built on across projects, in contrast to transient capture notes; principles include atomic notes, concept-oriented organisation, dense linking and writing for oneself. His claim is that the goal is better thinking, and that capture without the effortful step of developing the note does not compound.\n\nFor Fredrik: a raw ruling is capture. A ruling rewritten as an atomic, linked principle with its reason is evergreen. The rewriting step is where the attention goes, and it is the step that makes later turns cheap.\n\nDefinition: sharpens: separate the capture step (cheap, automatic) from the consolidation step (effortful, human or well-verified machine). Most stores fail by having only the first.\n\n### 3.3 Nonaka and Takeuchi's SECI spiral\n\n*The Knowledge-Creating Company* (1995) describes four conversions: socialisation (tacit to tacit), externalisation (tacit to explicit), combination (explicit to explicit) and internalisation (explicit to tacit), cycling upward ([summary](https://concepts.dsebastien.net/concept/seci-model/); date given as 1995 or 1996 depending on source; I did not read the book). The bottleneck is externalisation: getting what the expert knows into a form others (here, agents) can use.\n\nFor Fredrik: his judgment about a claim is tacit; labelling a case with a reason and writing the reason as a rule is externalisation; the agents reading the canon is combination; his later faster judgment is internalisation. Externalisation costs his attention, and the case-by-case protocol is a way of paying that cost in small, bounded instalments.\n\nDefinition: sharpens: a flywheel in knowledge work has two stores, one outside (explicit) and one inside the person (tacit). The saving may show up in internalisation (he decides faster) rather than in the store. Measure both.\n\n### 3.4 Argyris and Schön: single-loop and double-loop learning\n\nArgyris's thermostat is the standard example: a thermostat that turns the heat on below a set temperature is single-loop; one that asks why it is set there and whether another setting would better serve the goal is double-loop ([1977 HBR article and 1991 follow-up; secondary summary](https://en.wikipedia.org/wiki/Double-loop_learning); the 68-degree figure is from Argyris's 1991 text per the search results). Argyris's claim is that most organisational learning is single-loop, correcting errors within existing policy.\n\nFor Fredrik: a loop that tunes parameters or adds rules within the current frame is single-loop. The question \"should this canon rule exist at all, or is the criterion itself wrong\" is double-loop and is a decision only he can make; it is the governor in the definition.\n\nDefinition: sharpens \"governor\": the strongest governor is a double-loop review that can change the criterion, not a threshold that clips errors.\n\n### 3.5 Common law: case, holding, rule\n\nEdward Levi's *An Introduction to Legal Reasoning* (1949) describes reasoning by example: see a similarity between cases, announce the rule from the first, apply it to the second. Levi's key point is that the rule's scope and meaning are reset with each application, because what counts as similar is decided case by case; principles are often discovered through later cases rather than stated at the start ([summary and excerpts](https://www.garretwilson.com/books/reviews/introduction-legal-reasoning)). The doctrine of precedent also has an explicit overruling mechanism, which keeps the store from freezing.\n\nFor Fredrik: this is the exact shape of \"one case at a time, ending in a rule he can bless\". The literature's useful addition is that rules are provisional, narrow at first, and tested on the next case. Write the holding narrowly and widen only when new cases fit.\n\nDefinition: sharpens condition (3): the store gets more valuable because each application refines it, not just because it grows. Also adds \"overruling\" as a designed property.\n\n### 3.6 Mathlib and Lean\n\nThe Lean mathematical library paper ([arXiv 1910.09336](https://arxiv.org/pdf/1910.09336), CPP 2020) reports growth from 15k to 140k lines in two years with 73 contributors and 11 maintainers; by June 2024 Lean's [roadmap](https://lean-lang.org/roadmap) cites over 300 contributors and about 1.56M lines. The Lean FRO's [year-one review](https://lean-lang.org/blog/2024-8-13-lean-fro-year-1-in-review) reports efficiency regressions and a backlog of technical debt, and the roadmap sets a goal of a technical-debt-reduction plan to support growth toward 10M lines. The compiler is the governor: nothing enters unless it checks. The cost is maintenance, which grows with the library.\n\nFor Fredrik: a verified store compounds because later work can rely on earlier items without re-checking. It also shows that the store has a carrying cost that must be planned for, and that a mechanical verifier is what lets contributors (or agents) add without a human reading each line.\n\nDefinition: sharpens \"decay\" and \"friction\": add maintenance burden as a first-class term and note that a mechanical verifier is the strongest governor available.\n\n### 3.7 Kuhn and Lakatos\n\nLakatos's methodology of research programmes distinguishes a hard core (protected commitments) from a protective belt of auxiliary hypotheses; failed predictions are redirected to the belt. A programme is progressive if its modifications predict new facts, and degenerating if they only explain anomalies after the fact ([overview](https://bertie.ccsu.edu/naturesci/PhilSci/Lakatos.html); [stagnant research programmes paper](https://arxiv.org/pdf/2404.18307)). Kuhn's paradigm shifts are the discontinuous alternative; Lakatos emphasises continuity (from secondary summaries; I did not read Kuhn or Lakatos directly).\n\nFor Fredrik: canon rules added after each failure are the protective belt. If each addition only explains the last failure and predicts none, the programme is degenerating. A progressive canon rule prevents a failure that has not occurred yet. This is a usable test.\n\nDefinition: sharpens condition (3): \"yields more\" means new correct predictions, not more patches. A wheel that only patches turns but does not go anywhere (compare Collins' doom loop in 4.3).\n\n## 4. Human collaboration\n\n### 4.1 Dalio's Principles\n\nDalio's *Principles* (2017) describes recording the criteria behind each decision, turning recurring ones into written principles, and, where clear enough, into algorithms, with \"pain plus reflection equals progress\" as the learning mechanism. My sources were secondary summaries ([example](https://jamesbachini.com/?p=7686)); the \"algorithms\" framing is the summarisers' gloss as much as a verbatim passage, so treat it with care. The pattern is the same as ADRs and common law, with the further step of automating.\n\nFor Fredrik: decisions to principles to checks is the ranking of stores by how little attention they need: a decision costs attention each time, a principle costs reading, a check costs nothing.\n\nDefinition: confirms the shape; adds the ordering as a ladder (ruling, canon line, executable check).\n\n### 4.2 Ostrom's commons governance\n\nOstrom's *Governing the Commons* (1990) found that long-lived commons share design features: clear boundaries, rules fitted to local conditions, participants able to change the rules, monitoring by users or those accountable to them, graduated sanctions, cheap conflict resolution, recognition by outside authorities and nested layers ([summary](https://wiki.p2pfoundation.net/Elinor_Ostrom’s_Eight_Commons_Governance_Design_Principles); later work treats them as diagnostics, not a strict test).\n\nFor Fredrik: for a store shared by agents and a human, the transferable features are monitoring (someone sees violations), graduated response (a warning before a block) and the ability for users to change rules (agents can propose canon changes, he approves).\n\nDefinition: adds \"monitoring\" and \"graduated sanction\" as parts of the governor.\n\n### 4.3 Bezos/Amazon flywheel and Collins\n\nCollins' flywheel, from *Good to Great* (2001) and the 2019 monograph *Turning the Flywheel*, says transformation comes from consistent pushing in one direction rather than one big action; the doom loop is the comparison companies' habit of launching new programs after disappointing results and so never building momentum ([Collins' page](https://www.jimcollins.com/concepts/the-flywheel.html)). The Amazon flywheel is Collins' and secondary sources' account of Bezos's growth loop (lower prices, more customers, more sellers, lower costs); I did not verify the Bezos origin story.\n\nFor Fredrik: Collins adds a condition the working definition lacks: direction. A loop that is re-pointed every few weeks is a doom loop however good each turn is. It also fits \"propagation of errors\": a response to a failure that changes direction instead of fixing the wheel feeds the doom loop.\n\nDefinition: sharpens: add \"stable direction\" (a stable criterion for what a good turn is). Mild tension with Shankar's criteria drift: the criterion must be stable enough to compound but is learned by labelling. Resolve by versioning, not by freezing.\n\n### 4.4 Wenger's communities of practice, Wikipedia policy, institutional memory, decision journals\n\nWenger's communities of practice (*Communities of Practice*, 1998, from my knowledge; not fetched) treat practice as shared repertoire that accumulates through participation. Wikipedia's policies and guidelines are described by its own pages as products of consensus that usually need consensus to change, and consensus is treated as revisable ([example: Wikipedia:Sham consensus](https://www.wikipedia.com/wiki/Wikipedia:SHAM)); I did not retrieve the primary policy pages for \"policy follows practice\". \"Institutional memory\" and decision journals are the same mechanism at the scale of an organisation or one person, with the same lesson: memory that is not retrieved at the moment of decision does not count.\n\nFor Fredrik: retrieval at decision time is the part that matters. A principle the agent does not see when it acts has no effect (this is why a canon is loaded into every session).\n\nDefinition: adds \"retrieval at the point of use\" to condition (3).\n\n## 5. Human-AI specifics, 2024 to 2026\n\n### 5.1 LLM-as-judge calibrated against human labels\n\nZheng et al., [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://arxiv.org/html/2306.05685v4) (NeurIPS 2023), report that a strong model as judge agrees with human preferences over 80% of the time, about the level of human-human agreement, and identify position, verbosity and self-enhancement biases. Shankar et al., [Who Validates the Validators?](https://arxiv.org/pdf/2404.12272) (UIST 2024), find criteria drift: people need criteria to grade outputs, but grading outputs is what helps them form criteria, and some criteria depend on the outputs seen. Husain, Yan and Anthropic (section 2.6) give the working protocol: label a held-out set, report true positive and true negative rate or precision, recall and Cohen's kappa, one judge per failure mode, swap order in pairwise comparisons, allow an \"Unknown\" verdict. Angelopoulos et al., [Prediction-Powered Inference](https://arxiv.org/pdf/2301.09633) (2023), give the statistical tool: combine a small trusted labelled set with many machine predictions to get valid confidence intervals, with tighter intervals for more accurate predictions, no assumptions about the predictor.\n\nFor Fredrik: the judge does not replace his labels; it makes them go further. His labelled set both calibrates the judge and sets the confidence interval on the error rate of the whole machine pass.\n\nDefinition: sharpens governor: the governor for a machine pass is a human-labelled calibration set with reported error rates, plus a drift check.\n\n### 5.2 Constitutional and principle-based improvement\n\nBai et al., [Constitutional AI](https://ar5iv.org/abs/2212.08073) (Anthropic, 2022), train a model using only a written list of principles as human oversight: the model critiques and revises its own outputs against the principles, is fine-tuned on the revisions, and a preference model trained on AI judgments gives the reward (RLAIF). So a small durable store of principles can steer a large amount of machine output. But Huang et al., [Large Language Models Cannot Self-Correct Reasoning Yet](https://arxiv.org/abs/2310.01798v2) (ICLR 2024), find that intrinsic self-correction without external feedback often fails to help or lowers accuracy, and that earlier reported gains often used oracle labels to decide when to stop.\n\nFor Fredrik: principles in a canon work as the constitution, but a machine loop that critiques itself against them with no external check is not the same as one anchored in his labels. The principles lower the demand for review only up to the point where the model's reading of them is verified.\n\nDefinition: sharpens condition (3) and governor: the machine's self-judgment is not an independent oracle.\n\n### 5.3 Compounding error in agentic chains\n\nSinha et al., [The Illusion of Diminishing Returns](https://arxiv.org/pdf/2509.09677) (2025, ICLR 2026), show that small gains in per-step accuracy compound into large gains in the task length a model completes, and also that per-step accuracy falls as a task goes on because models are more likely to err when their own earlier mistakes sit in context (\"self-conditioning\"); larger models alone do not fix this, and thinking models mitigate it. Anthropic's evals post gives the arithmetic for repeated trials: a 75% per-trial success rate gives about 42% chance of three passes in a row (pass^k). Shumailov et al., [AI models collapse when trained on recursively generated data](https://www.pure.ed.ac.uk/ws/portalfiles/portal/460496122/ShumailovEtalNature2024AIModelsCollapseWhen.pdf) (Nature, 2024), show that indiscriminately training on model-generated content loses the tails of the distribution (a separate-but-related error mode, relevant if machine output flows back into a store).\n\nFor Fredrik: a chain of passes multiplies error rates, so a pass in the middle of a pipeline needs a check at its output, not only at the end. Machine output going into a store without verification can pollute every later turn.\n\nDefinition: sharpens \"propagation of errors\": two routes, in-context (self-conditioning in a chain) and in-store (machine output saved as if it were truth). Each needs its own governor.\n\n### 5.4 Evals-driven development and error analysis first\n\nThe consistent practitioner position (Husain and Shankar, Yan, Anthropic) is that evaluation is part of development, not a late stage: start from real failures, read raw traces before any metric, build the metric from the taxonomy of what you saw, and run the eval at every change. The shared warning is against generic metrics (helpfulness, similarity scores) that produce false confidence.\n\nFor Fredrik: the first purchase is reading cases himself, 30 to 100 of them, before any dashboard. The attention is real but it is the highest-yield attention in the system, because it decides which checks get written.\n\nDefinition: confirms; it also adds a stopping rule for human reading (theoretical saturation).\n\n### 5.5 Does the loop actually save time? The perception problem\n\nMETR's randomised trial ([blog, July 2025](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/)) of 16 experienced open-source developers on 246 issues found tasks took 19% longer with AI tools allowed, though the developers forecast a 24% speedup and afterwards believed they had been 20% faster. The authors state strict caveats (early-2025 tools, one setting, small sample, possible learning effects; the page notes a later update). The transferable point is not the sign of the effect but the gap between felt and measured speed.\n\nFor Fredrik: a felt sense that a loop is compounding is not evidence of compounding. His reading time per case, logged, is.\n\nDefinition: sharpens condition (5): \"what it saves is named\" should be \"what it saves is measured\", at least once.\n\n## 6. Synthesis\n\n### (i) The 8 to 12 most transferable practices, ranked by how much they lower demand on his attention\n\n1. **Turn each blessed ruling into the cheapest durable form that acts without being read**: an executable check or hook first, then a one-line canon rule, and the ledger entry last (SRE action items; jidoka and poka-yoke; Dalio's ladder; Mathlib's compiler). Recurrence of a ruled issue then costs him nothing.\n2. **Run every machine pass at the rate his review can absorb and stop the line on verifier failure** (Goldratt's subordinate step; andon). This removes the queue and the mixed good-and-bad pile from his desk.\n3. **Calibrate machine judges against a small held-out set of his labels, report true positive and true negative rates, and let the judge screen so he reads only disagreements and low-confidence items** (Zheng; Husain and Shankar; Yan; Anthropic; prediction-powered inference for error bars).\n4. **Choose which cases he labels by information value**: disagreements between passes, low confidence, new clusters, triggers on rare cases first (Settles; Tesla data engine; Yan on active learning).\n5. **Case review one at a time, ending in a candidate rule tested on the remaining cases, bringing him only the residue** (Levi; open and axial coding; SECI externalisation). One reasoned verdict becomes many automatic ones.\n6. **Build the eval set from real failures, binary-labelled, with graduation of saturated items into regression** (Anthropic; Husain). It is both his test suite and his proof.\n7. **Decision records with status, supersession and consequences, so closed questions stay closed** (Nygard; Fowler). This removes re-argument, which is a recurring attention cost.\n8. **Defaults and scaffolds in the repo** (paved road; golden paths) so agents do the right thing without being told. The best decision is one nobody has to make.\n9. **Sample audits instead of full review once a pass is calibrated**: a fixed number of fresh cases and a few outliers per cycle (Hamel's 100 per cycle and 10 to 20 outliers weekly as reference points, from the FAQ).\n10. **Prune the store on a schedule**: retire rules that never fire, merge duplicates, and mark rules whose reason has expired (Lakatos's degenerating-programme test; Mathlib's debt backlog; Goldratt's inertia). A rule that is loaded into every session has a carrying cost in attention and context.\n11. **Run a C-level review on a cadence**: each loop's attention-minutes per accepted item this month versus last, and which loop is now the constraint (Engelbart; Goldratt step 5).\n12. **Keep a double-loop slot**: a periodic question of whether the criterion itself is right (Argyris), because single-loop tuning cannot find a wrong criterion.\n\n### (ii) Measurement methods the literature uses for a loop's gain\n\n- **Cycle-time and flow metrics.** DORA's four keys (lead time, deployment frequency, change failure rate, time to restore; the 2024 report added a rework-rate measure, per [RedMonk](https://redmonk.com/rstephens/2024/11/26/dora2024/)). I did not retrieve the primary definitions. Used for the software pipeline of the product, not for judging claims.\n- **Cost per unit over time (learning curve).** Attention-minutes per accepted case, per ruling or per review, tracked by month. A flywheel shows a falling trend; a treadmill is flat. The learning-curve model (Wright's law) is the classical frame, from my own knowledge and not verified in this research.\n- **Label efficiency.** Accuracy or agreement reached per human label (Settles). A flywheel bends the curve upward over successive rounds.\n- **Judge agreement against human labels.** True positive and true negative rate, precision, recall, Cohen's kappa, on a held-out set (Husain, Yan), with confidence intervals from prediction-powered inference (Angelopoulos).\n- **Reliability under repetition.** pass@k and pass^k (Anthropic) for agent steps; the arithmetic of chained steps (Sinha).\n- **Recurrence rate.** The share of incident or failure classes that return after the fix (the SRE postmortem test, implicit in the chapter).\n- **Coverage and hit rate of the store.** Share of new cases decided by an existing rule without human input; share of canon lines that fire at least once in a period (my formulation; no source I fetched gives this exact metric).\n- **Saturation.** New failure modes per batch of 100 traces (Husain); eval pass rate nearing 100% as a signal to replace the eval (Anthropic).\n- **Progressive versus degenerating.** Whether each new rule predicted something not yet observed (Lakatos, qualitative).\n- **Controlled comparison.** A randomised or holdout comparison with and without the loop (METR), because felt speed is unreliable.\n- **Gain as a ratio.** In system dynamics terms, the ratio of one turn's output to the previous one; the doubling time is a convenient summary. I did not find a software-specific source that applies this directly; treat it as a modelling choice.\n\n### (iii) Failure modes\n\n1. **Errors ride the wheel**: self-conditioning in chains (Sinha), machine output saved as truth (Shumailov), self-critique with no external anchor (Huang). Governor: external ground truth, a stop-the-line verifier and a calibration set.\n2. **Criteria drift**: the meaning of the labels changes as people grade (Shankar). Governor: version the criterion, record which version labelled each case.\n3. **Degenerating patching**: rules added after each failure that predict nothing (Lakatos). Governor: require each canon rule to name what it prevents.\n4. **Doom loop**: changing direction after each disappointment (Collins). Governor: stable target criterion, changes only through double-loop review.\n5. **Lock-in and entanglement**: the store becomes hard to change (Sculley et al., [Hidden Technical Debt in Machine Learning Systems](https://proceedings.neurips.cc/paper/2015/file/86df7dcfd896fcaf2674f757a2463eba-Paper.pdf), 2015: changing anything affects everything; hidden feedback loops; undeclared consumers). Governor: supersession not overwrite; declared consumers.\n6. **Maintenance load of a growing store** (Mathlib's debt backlog). Governor: scheduled pruning and a verifier.\n7. **Success to the Successful**: the visible loop starves the one with more long-term value (Senge). Governor: allocate attention explicitly.\n8. **Inertia** (Goldratt step 5): rules built around yesterday's constraint become today's.\n9. **Selection skew**: active learning and trigger-based collection fill the store with hard cases, so aggregate numbers misdescribe typical performance.\n10. **Amplifier effect**: AI and automation speed up whatever the feedback system already is (DORA 2025); a weak verifier gives faster wrong output.\n11. **Perception gap**: believing a loop saves time without measuring it (METR).\n12. **Capture without consolidation**: stores that only accumulate (Matuschak).\n13. **Retrieval failure**: a rule not present at the moment of action has no effect (institutional-memory literature, and my own reasoning from the other items).\n14. **Blame suppressing reporting** (SRE). In a one-person project this has an analogue: skipping the record of an error because it is embarrassing.\n\n### Net effect on the working definition\n\nThe definition survives, with six sharpenings from the literature:\n\n- Condition (5) should be a gate and a measurement: the saving must be on the constraint (Goldratt) and measured at least once (METR).\n- Condition (3) should say what makes the store pay: refinement on use (Levi), surprise (Luhmann), predictive content (Lakatos) and retrieval at the point of action. Size alone does not count.\n- The loop needs a stable direction (Collins) and a versioned criterion (Shankar); these two pull against each other and are reconciled by versioning.\n- \"Limit\" (resource) and \"governor\" (designed control) are distinct; the strongest governors are stop-the-line and an external verifier; the strongest review is double-loop.\n- The store has a carrying cost and a decay path (Mathlib, Sculley); pruning belongs in the loop.\n- \"Gearing with a sign\" corresponds to Engelbart's C level and Senge's Success to the Successful; the test for a positive sign is that the C loop lowers demand from the B loops on the constraint.\n\n### (iv) What I verified at a primary source and what I did not\n\nOpened and read the primary page (or the author's own site), at least the relevant part: Engelbart 1962 (only the regenerative-interaction passage; the A/B/C text is from the Doug Engelbart Institute page, which is the institute's account and gives no year for the 1990s papers); the Meadows list (Dartmouth reprint of her text; numbering in my fetch was garbled for three items, the order given matches Meadows); the Google SRE postmortem chapter; Hamel Husain's evals FAQ (first 100,000 of about 153,000 characters read); Anthropic's evals post; Eugene Yan's post; Matuschak's evergreen-notes page; Jim Collins' flywheel page; DORA's capabilities list and 2025 report landing page (the latter has little detail); the METR blog post.\n\nConfirmed at abstract or search-snippet level only (the paper exists and says what I attribute, but I did not read the body): Zheng et al. (MT-Bench); Shankar et al. (criteria drift); Bai et al. (Constitutional AI); Huang et al. (self-correction); Sinha et al. (long-horizon execution); Shumailov et al. (model collapse); Angelopoulos et al. (prediction-powered inference); Wang et al. (Voyager, which I did not use in the text above but which supports the skill-library-as-compounding-store pattern for agents); Sculley et al.; the Mathlib paper and Lean roadmap; Settles' survey (section content on specific query strategies is from my own knowledge); Lakatos and Levi (via secondary summaries and excerpts).\n\nTaken from secondary sources or my own knowledge, not checked at the primary: Forrester 1961 and Senge 1990 (the archetype descriptions come from a study-guide PDF, not Senge's book); Goldratt's five steps (consistent across several secondary summaries; I did not open *The Goal* or his 1990 book); Nygard's ADR text (secondary summaries and Fowler); SECI (summary pages; the book date varies by source); Argyris (secondary; the 68-degree detail is from search results quoting the 1991 piece); Dalio (low-quality secondary summaries; the \"algorithms\" framing is partly the summarisers'); Tesla's data engine (talk summaries, not the talks); Toyota's concepts (manufacturing sources only; the software mapping is mine); Luhmann (the Schmidt text could not be read); Ostrom (summary pages); Wenger and Wikipedia policy (not retrieved at a primary; the Wikipedia point rests on essay pages); paved road and golden paths (my knowledge of engineering blogs, not fetched); Wright's law as a learning-curve model; the Amazon flywheel's origin; the DORA effect sizes (secondary summaries of the 2024 report) and the definition of the four key metrics (not on the capabilities page I fetched).\n\nFlags on uncertainty in the text: the DORA figures are survey correlations; the METR figure is for one setting, and its authors note a later update that I did not read; the Hamel figures (30, 100, 100-200 labels, 60-80% of time) are practitioner guidance and not empirical findings from a controlled study.\n"}