{"path":"research/one-inference-per-link.md","content":"# One inference per link: how to make the keeper's chain provably small, without asking twice\n\n*2026-10-24. A pass of the all-angles kind: the corpus first, then what published work adds, then tests on\nrecorded and constructed cases, then recommendations. No keeper call was made (his hold, below). Every\nreading of a step here is Claude's unless it names another reader.*\n\n## 0. The answer\n\n**Yes, it can be made checkable by code, and asking a model a second time is not needed for it.** Write\neach link of the chain in slots: a rule with a placeholder in its condition and its outcome, the term it is\nsaid of, the case, the conclusion, and the rule's kind (strict, normic or statistical). Then code checks\nthat the case is the condition said of the term, that the conclusion is the outcome said of the term, that\nthe rule does not name its own case, and that each case is the first claim, an earlier conclusion or a\npremise declared as new. A leap then cannot pass in the arrow. It can only sit in a rule written in the\nopen, which is a claim anyone can attack, and whose exceptions count by the rulings on rules.\n\n**Tested on the floor trial's 25 recorded steps** (13 October, each labelled gap or no gap by that trial's\nreader, a model asked the refutation question): the form check, read by hand, refuses all 8 steps whose\ngap is that the conclusion says more than the rule's outcome or the condition is not stated, and refuses\none step the reader passed, whose starting claim is idle (the second test of the ruled floor). The reader's\nother 5 gaps are each a doubt about whether a written rule or premise is true, which the floor's own third\nclause gives a place: the rule's claim. **A small NLI model scores the reader's gap steps higher on\n*follows* (0.93) than the 12 without a gap (0.70)**, so a meaning-based check cannot stand in for the form check:\nit reads a near-miss as a match.\n\n## 1. His questions, and what is already decided\n\nHis words, 2026-10-24, on the keeper's chain: *\"Yes but is it provably only making one logical inference\nstep between each link in the chain? That's only for deductions ofc... I'm just thinking it's liable to make\nirresponsibly large leaps, since LLMs and humans seem prone to do that a lot, this has been mentioned in one\nof the recent CC sessions from yesterday, consider it deeply\"*. The session he meant: his instruction to the\ninstitute map, 23 October, *\"think about the smallest-step / link in the chain / inferences that should be\nspelled out in the map. Don't make leaps that are irresponsibly large.\"*\n\nOn a second reading: *\"But a second-reading? Would that be a second model call? Haven't I argued against\nthat? Search in my messages/sessions. We need method, not repetition, without clarification, the answer is\nnot liable to change much. Any other way in which we could ensure/check/enforce this, ideally with as little\neffort as possible, but as stringently as possible.\"* He has: *\"'Asked twice'? I don't think I ever blessed\nthat particular part.\"* (2026-10-10, ledger AD14).\n\nOn rules: *\"So, with all possible exceptions? Should we trust a rule's own words? In a saturated graph, all\nrules would have exceptions/counterexamples/cases, right, but we don't have that yet. So maybe we\ncould/should teach method about this to the model as well, from our rulings and corpus distilled? Seems\nsensible?\"*\n\nAnd his hold: *\"we shouldn't run more keeper tests until I'm happy with updated keeper pass prompting and\nthe theory behind that etc.\"*\n\n| | State |\n|---|---|\n| A machine-written step is at its floor by two tests, no gap and nothing removable; *the way a step is tested is the refutation question* | RULED, AG14 (2026-10-14) |\n| The floor's third clause: one stated rule, applied once, to one stated case; every doubt left is about a written sentence or is a standing question of that kind of step | RULED 2026-10-14 (`the-chain-inside-a-link.md` § 10.3) |\n| The keeper's chain is written to that floor | RULED today, AM11 |\n| A pass is taught the method, the graph's contents are never its lesson | RULED today, AM12 (resident) |\n| A generalization keeps one number, its *usually* made exact in its words; the writer asks whether a failure calls for an explanation; strict, normic and statistical; an exception is a claim of its own | RULED, AT1, AT2, AT12, AT13 |\n| Asking the same thing twice | never blessed (AD14) |\n| How the floor is checked; which rulings on rules go in the keeper's grounding | OPEN, AM14 and AM13 |\n\n**The open half in one sentence**: how to check, as stringently and cheaply as possible, that each link of a\nmachine-written chain is one inference, given that the ruled test is the refutation question and asking a\nmodel twice is not blessed.\n\n## 2. What the corpus already holds\n\n| Angle | Where | What it says |\n|---|---|---|\n| The floor and its tests | `the-chain-inside-a-link.md` § 10–11 | the floor's definition; ten links written to it; the refutation question found a gap in 13 of 25 steps where *is anything missing?* found none; asked the same wording twice the reader changed side on 6 of 25 |\n| Two tests give completeness, not smallness | `claim-parts-and-inferential-reasons-2026-10-14.md` § 9–10, 13 | a two-application chain passes both tests; *smallest* needs the allowed moves named |\n| Step checks in the literature | `writing-out-the-unwritten/outside-reading-explication-with-checks.md` § 3 | Natural Program (only cited premises: 52% to 69%); voting raises acceptance, not rejection (38% to 40%); a four-way rubric κ .85; a solver on a formalisation is deterministic; an NLI verifier transferred poorly (REVEAL) |\n| Verification in the same context against a fresh one | `writing-out-the-unwritten/outside-reading-dependable-small-tasks.md` | Chain-of-Verification: factored (fresh context) beats joint; error rates rise when a model's context holds its own earlier output |\n| Machines naming the missing premise | `what-a-link-rests-on/outside-reading-machines-naming-premises.md` | whether the added premise is the right one has no measured check anywhere |\n| Rules and their exceptions | `normic-rules-normative-reasons-and-trying-rules-out.md`, `what-a-case-counts-for.md` | the AT and AY rulings |\n| The keeper's chain today | `deliberus/keeper.py` | asks *one inference per step*; code refuses only *overlaps* on a chain longer than two |\n\nNot covered, and researched for this note (§ 3): scheme-constrained steps, form checks by code\n(natural logic, templates), defeasible-logic solvers fed by a model, small trained step verifiers, and\nreal cases with known answers.\n\n## 3. What published work adds\n\nTwo research reports, saved verbatim: `one-inference-per-link/report-methods-for-one-inference.md` and\n`one-inference-per-link/report-datasets-and-historical-cases.md`. Read-status tags are the helpers'.\n\n- **Nobody has measured one-inference-per-step for defeasible natural-language steps.** The nearest\n  measurements are on formal logic with a fixed rule set (FineLogic, Findings EMNLP 2025), purely deductive\n  chains (Sadeddine & Suchanek, Findings ACL 2025) and human annotation of argument form (Saadat-Yazdi &\n  Kökciyan, ACL 2024).\n- **Prompting alone rarely produces atomic chains**: 1.6% to 5.7% of chains had every step one rule\n  (GPT-4o, DeepSeek-R1, Llama-3.1-8B, few-shot); fine-tuning on structured symbolic steps raised it to 20% to\n  25%; natural-language steps *\"fuse multiple reasoning rules\"* (FineLogic). So a form check will refuse\n  much of what an unprompted writer produces, and how often a prompted writer gets the slots right is the\n  number this design still lacks.\n- **Checks that parse free text are precise and blind**: a parser plus a natural-deduction solver had\n  precision 0.94–0.97 and recall 0.16–0.26 on real chains (FOLIO); adding an NLI model raised recall to\n  0.60–0.69 at precision 0.74–0.81. The form check here avoids parsing by having the writer fill the slots,\n  which moves the difficulty to the writer.\n- **Conformity is not validity**: a model asked to fill named slots *\"generated nonsensical phrases while\n  conforming to the required output structure\"* (Saadat-Yazdi & Kökciyan). The check guarantees the shape,\n  never the truth of what is in it.\n- **Scheme naming is unreliable on real text**: classifying the argument scheme of natural arguments reached\n  0.5–0.65 F1 over four to seven schemes and about 0.29 over 24 in dialogue; template-written text classifies\n  back at 0.94–1.00, because the reader reads its own template.\n- **Machine judges of completeness over-credit**: judges kept giving middling coverage scores with 70% of\n  the steps deleted, correlation with true coverage 0.13–0.33 (C2-Faith, preprint 2026); zero-shot leap\n  detection about a third precision and recall (CoT-Bridge, NeurIPS 2025).\n- **Rules with exceptions are where models fail**: at best about 0.64 F1 on generic rules with exceptions\n  (DEFREASING, NAACL 2025); reasoning with several defaults fell to 39% on the valid side (LogicBench).\n  This is the evidence for teaching the method on rules (AM13).\n- **Structured output can cost accuracy when the answer key comes before the reasoning key**, and one\n  provider once sorted keys alphabetically (Tam et al. 2024; a practitioner replication). The project's\n  reason-before-verdict test covers the order in our models; whether the provider honours it was not\n  checked here.\n- **Datasets**: EntailmentBank (5,881 steps, each written as one inference, positives only), REVEAL (human\n  step labels on model chains, gated, no derivatives), METGEN's four-way labels (release unconfirmed),\n  Becker et al. (719 argument pairs with the missing premises written by experts, location unconfirmed).\n\n## 4. Theory first\n\n**What one inference is.** In logic an inference applies one rule of inference to premises. In the floor's\nterms the rule of inference is fixed (put the term into the rule, then detach the outcome) and the content\nis a stated rule, a stated case and a conclusion. That is a property of form. It can be checked without\njudging whether anything is true, the way a proof checker checks a proof without believing its axioms.\n\n**Why the form check meets both of the floor's tests by construction.** If the case is the condition said\nof the term and the conclusion is the outcome said of it, then for a strict generalization no situation makes the\nrule and the case true and the conclusion false: that would make the rule false of this term, so not every\nsentence would hold. For a normic generalization the only such situations are the rule's exceptions, which the floor\nallows. For a statistical generalization they are the expected minority. So the no-gap test is met by form. The\nnothing-removable test is met too: without the rule nothing licenses the conclusion, and without the case\nthe rule is applied to nothing, unless the rule has no condition, which the check refuses (the idle start).\n\n**Where the leap goes.** Nowhere hidden. A large leap must now be written as a rule (*whenever a\ncapability's pace accelerates, there is no time to deliberate*), which is a claim: a reader can attack it,\ncases move it, and the writer has to answer the ruled question about it, whether a failure would call for\nan explanation. A rule that is a leap is a rule with many failures nobody explains. That is something the\ngraph can see over time, and the step check need not.\n\n**What the form check cannot see.** Whether the rule is true, whether the case sentence is true of the\nclaims, and whether the writer chose the right rule. The first two are doubts about written sentences, with\na place by the floor. The third is the content choice that no published check measures (§ 2). None of the\nthree is a leap.\n\n**Why not ask a second time.** The literature in the corpus agrees with him: voting does not help a reader\nreject a wrong step, and the same wording asked twice moved 6 of 25 answers in the trial. A different\nquestion in a fresh call is a different instrument, and Chain-of-Verification shows a fresh context checks\nbetter than the writer's own. But once the form is checked by code, the second question's only remaining\njob is whether a rule is true, which is the graph's job, not the step's.\n\n**What this predicts, written before the test**: the trial's form-type gaps (conclusion says more than the\noutcome, a condition not stated) are refused by the form check; its other gaps are doubts about written\nsentences; an NLI model does not find the gaps, since they are near-misses in wording.\n\n## 5. Tests\n\n### 5.1 The exact case\n\n`form_check.py`, run: his squirrel of 2011 at its floor passes; the same arrow concluding *the squirrel died\nof poison* is refused (*conclusion is not the rule's outcome said of the term*); a rule written around the\nlink (*{x} is the squirrel that smells…*) is refused as naming its own case.\n\n### 5.2 The floor trial's 25 recorded steps, read for form\n\nData: `the-chain-inside-a-link/floor_trial.json`, first-written steps, each with the trial reader's verdict.\nClaude's form reading of each, with its reason: `form-reading-of-the-floor-trial-2026-10-24.json`.\n\n| | Reader: gap | Reader: no gap |\n|---|---|---|\n| **Form check refuses** | 8 (steps 2, 8, 10, 11, 21, 22, 24, 25) | 1 (step 18, an idle start) |\n| **Form check passes** | 5 (steps 1, 4, 6, 15, 16) | 11 |\n\n- The 8 refused gaps are each a conclusion that says more than the outcome (*strong* reasons; *are\n  pursuing* where the rule says *will pursue*; *a variety of* words; *appear to show* where the rule says\n  *can be reported as*) or a condition no sentence states. Three of them are the trial's own three examples,\n  and in each the reader's closing sentence is what the form check would demand be written.\n- The 5 passed gaps are each a doubt about whether a written rule or premise is true (the rule says *each\n  kind*, the reader asks *every instance*; is the identity of two differences true), which the floor's\n  third clause places in that sentence's claim.\n- Step 18 is the one the reader passed and the check refuses: its rule is the conclusion said generally, so\n  the claim it starts from is idle, the failure the trial's second test found by hand (§ 11.4).\n\n**What it cannot show.** The form reading is one model reading by hand; the code check runs only on steps\nwritten in slots, and no writer has yet been asked to write in slots, which needs model calls and waits on\nthe hold. What it shows is what the check would decide on these 25 steps if they were written in slots.\n\n### 5.3 A meaning-based check on the same steps\n\n`nli_on_floor_trial.py` (the multilingual NLI model on this machine's CPU, premise = the step's sentences,\nhypothesis = what it arrives at; results in `nli-on-floor-trial-2026-10-24.jsonl`): it flags 3 of the 25\nand none of them is a reader's gap; mean *follows* is 0.93 on the reader's gaps and 0.70 on the rest. After\nthe trial's cutting (29 steps, 3 reader gaps) it flags 4, none a gap. The gaps are near-misses in wording,\nthe overlap these models are known to read as entailment\n([the four papers](https://claude.ai/artifact/UbGX8PDKcewYFvHALNMFWB)). A negative result: no meaning-based\ntolerance should replace the exact match.\n\n### 5.4 A public set of single steps, with one premise taken away\n\n`entailmentbank_test.py` on EntailmentBank's dev split (596 steps of two or more premises, each written by\ntrained annotators as one inference; 1,255 versions with one premise removed). Output:\n`entailmentbank-test-2026-10-24.txt`.\n\n| Check | Flags full steps | Flags cut steps |\n|---|---|---|\n| Coverage by code (every content word of the conclusion appears among the premises) | 222 of 596 | 1,210 of 1,255 |\n| NLI, entailment below 0.5 | 88 of 596 | 842 of 1,255; AUC 0.817 |\n\nOn short science sentences the NLI model separates a full step from a cut one fairly well, and on the\ntrial's argument steps it did not separate them at all (§ 5.3): the cut there is a near-miss in wording,\nhere a whole premise gone. The word-coverage check catches nearly every removed premise and raises false\nalarms on over a third of the complete steps (a complete step renames: *a mirror* for *a reflective object*). On\nthe trial's 25 steps it flags 10 of the 13 reader gaps and 8 of the 12 others. Neither is a gate. Both are\nscreens for free text a writer did not put in slots.\n\n### 5.5 Four documented leaps, in slots\n\n`historical_cases.py`; chains restated from the helper's sources (§ B of its report), output\n`historical-cases-2026-10-24.txt`.\n\n| Case | As argued | What the check does with the hidden premise |\n|---|---|---|\n| Kelvin's age of the Earth (1863) | refused: the case does not state the rule's condition | forces *the Earth cools by conduction alone* to be written: Perry's 1895 point |\n| The Literary Digest poll (1936) | refused: same | forces *its answers represent the electorate* to be written |\n| Farr's elevation law (1852) | passes | the leap is the rule (*where mortality falls with elevation, the disease spreads by air*), written in the open, which Snow's water-company figures attack |\n| The Griess test, Birmingham Six (1975) | passes | the leap is the rule (*a positive test means nitroglycerine was handled*), written in the open, which the later tests attack |\n\n**So there are two kinds of leap**, and the check handles both by refusing to let either stay hidden: a\npremise missing about the case is refused until it is written; a rule too broad to be true passes and\nstands as a claim, attackable, where its cases count against it. The check does not know which premise is\nfalse. Neither did the reasoners, and the helper notes Kelvin's case has a decoy: the folk story blames the\nmissing heat source, while the documented flaw was conduction alone.\n\n## 6. Recommendations\n\nIn order of dependency.\n\n1. **Write each link of a machine-written chain in slots and check its form in code** (AM14, revised): rule\n   with a placeholder in condition and outcome, its kind, the term, the case, the conclusion, any new\n   premise declared. Code refuses a mismatch, a rule naming its case, a rule with no condition, an\n   unconnected case, a chain that does not arrive. Evidence: § 5.1, § 5.2 (8 of 8 form-type gaps and the\n   idle start), § 5.5. What it changes: the keeper's chain field becomes structured; the refutation question\n   is no longer needed for the form, so no second call.\n2. **A refused link goes back to its writer once, with the code's fault in words.** That is repetition with\n   clarification, which his own sentence allows; a second reading of the same question without it is not\n   asked. A link still refused after that is stored as refused, with the fault, and the pair's key is not\n   written.\n3. **No meaning-based model stands in for the form check.** The NLI model ranked the trial's gaps above its\n   inferences without a gap (§ 5.3). *Without a gap* means only that the trial's reader found no situation where every sentence holds and the conclusion fails; it is not *sound* in the logician's sense, which would need the premises true. It may still order pairs for the keeper by opposition (`resonance-db-and-negation.md`).\n4. **The method on rules goes in the keeper's grounding** (AM13): models fail on rules with exceptions\n   (§ 3), and a form-checked chain moves every leap into a rule.\n5. **Before building the retry loop, measure how often a writer fills the slots correctly**, on the\n   twenty-one pairs of the machine-consciousness map (AM2): published prompting rates of 2% to 6% atomic\n   chains predict many refusals. This is a keeper run and waits on his hold.\n6. **A free-text screen for chains not written in slots** (the extraction's links, people's own steps): the\n   word-coverage check, as a flag for a person, never a gate (§ 5.4).\n\n## 7. What only he decides\n\n- Whether the form check replaces the refutation question as the way the floor is tested for a machine's\n  chain (AG14 names the refutation question; recommendation 1 keeps it for people and drops it as a second\n  machine call).\n- Whether a refused link may go back to its writer once (recommendation 2).\n- AM13 and AM14 as revised.\n- When the hold on keeper runs ends, and whether recommendation 5 is the first run after it.\n\n## 8. Every kind of inference, and the method each owes (2026-10-24, second pass)\n\nHis question, verbatim: *\"Just say what's generally hard for modern models to do and what the right thing is to\ndo here for all kinds of inferences that are logically possible? And following the method that we've ruled for\nrules/cases/exceptions, all the induction related stuff, that's probably equally muddled by raw LLMs (and\nhumans, when uneducated). Minds need education and method to know how to do anything stringently and\ncarefully. Otherwise they'll just leap from neuron to neuron and it'll be a mess, not the crystallized lattice\nthat Deliberus should become.\"*\n\n**What the evidence says about his claim** (`one-inference-per-link/report-what-is-hard-per-kind.md`, read\ntags there): strong for statistical and methodological reasoning in people (brief rule-plus-example training\nraised statistical answers and transferred to untrained domains; graduate fields that practise the rules\nimproved, chemistry did not), mixed for deduction (abstract logic training transfers poorly; pragmatic schemas\nand worked examples help), and indirect for the rest. General critical-thinking instruction has a small average\neffect; narrow, rule-specific training a larger one. **For models, the methods that measurably help add an\noutside checker** (a solver, an interpreter, a computed Bayes step): a prose procedure the model follows alone\nhelps much less, and on rules with exceptions more deliberation was measured to hurt. Models propose induced\nrules well and apply their own rules badly; they judge validity by how believable a conclusion is; they\nneglect base rates when a description is representative; they stop at an explanation rather than the simplest\none; their analogies break on unfamiliar variants.\n\n**So the design is method in the prompt and a checker in code, per kind** (`method_per_kind.py`):\n\n| Kind | What the writer must state | What code refuses |\n|---|---|---|\n| every rule | the answers to AT12's two questions about a failed case: does it call for an explanation (no: statistical), could an explanation leave the rule standing (no: strict; yes: normic) | a rule with the questions unanswered, or a kind the answers do not give |\n| normic | the exceptions it knows, each a claim (AT13) | |\n| statistical | the proportion (AT1); the rule's condition is its reference class | a case outside the class as written |\n| induction | each case filed under the rule (AY2); the band (AY3); from one case, the wider rule that says such cases go alike | a case that does not meet the condition; one case with no wider rule |\n| abduction | the rivals that would also explain the observation (Q40's question) | an explanation with no rival named |\n| analogy | the respect as a rule, *things alike in R are alike in Q*, and that the target has R | likeness with no respect written |\n\n**Tested on six documented leaps** (`kinds_cases.py`, output `kinds-cases-2026-10-24.txt`; chains restated\nfrom the helper's sources): the Griess test written as strict is refused with the two questions unanswered, refused again when\nanswered honestly (a failure could be explained and the rule left standing, so it is normic), and passes as\nnormic with its known exception written; Robinson's state-level rate applied to one immigrant is\nrefused as outside its reference class; Farr's air theory is refused until the water is named as a rival; the\n2008 housing rule, induced from periods without a national downturn, is refused when applied to one; the Comet\nanalogy is refused until the respect is written, which is the airframe-matches-the-test premise (weaker\nsourcing). **The Griess case changed the design**: written as strict it passed every form check, so every rule now\nowes the answers that give its kind. ⚠ The first version of this fix (the same hour) asked a strict generalization\nfor *a definition or proof*: that took AT12's examples for its test. A law of nature is strict when a\ncounterexample would refute it, though it is neither; the test is the two questions. Corrected on his\nquestion: *\"What's the thing about \\\"naming the definition\\\"? Sounds off to me, explain it better, suspect it's\nmuddled, but I may be wrong\"*, and his note on the pass as a whole: *\"Seems like you didn't understand deep the\nmethod rabbithole should go, the first time around\"*.\n\n**What it cannot do**: decide that a rule is true, that the rivals named are the right ones, or that the\nrespect named is the one that matters. Those are claims, argued in the graph, and the cases filed under them\nmove them over time.\n\n**Recommendation (ledger AM15)**: every inference a machine writes names which kind it is and meets that\nkind's form, checked in code; the method on rules goes in the pass's grounding (AM13). Unbuilt; the writer's\ncompliance is measured after his hold.\n\n**2026-10-24, later.** What software exists for each kind, and the design it points to (code compiles the filled slots into clingo or ProbLog, so the model never writes logic; refusals go back to the writer a few rounds): `tools-for-checking-inferences.md`, ledger AM16 and AM17, open. One piece is ruled and built the same night: several statistical generalizations for one case never reach an engine together, the narrowest counted reference class answers alone, and otherwise the rules combine by the graph's own rule (AM18, no engine combines two rules for one case; `deliberus/rules_for_one_case.py`; the tools note § 7).\n"}