{"path":"listening/2026-09-26-what-counts-as-evidence.md","content":"# What counts as evidence, read aloud\n\n> **What this is.** A walkthrough written to be listened to, the second piece made for that purpose\n> (the first is the Arc P walkthrough of the same day). The founder asked for it, 2026-09-26: *\"deeply\n> research what in academic context is considered evidence and what different tiers of evidence can\n> consist of. Then review how evidence can be translated optimally to the ontology and processes and\n> rationality idealized in Deliberus. Then produce an audio walkthrough of all your findings and\n> resulting reasoning.\"* And then: *\"Document everything carefully before you spit out the audio\n> walkthrough.\"*\n>\n> **What it covers.** The research synthesis\n> [`what-counts-as-evidence.md`](../research/what-counts-as-evidence.md) and its dossier: what the\n> corpus and the code already say about evidence, what philosophy, statistics, evidence-based\n> medicine and the professions of proof mean by it, and the twenty-six proposals of Arc Q in the\n> ruling ledger ([`session26-open-rulings.md`](../research/session26-open-rulings.md)).\n>\n> **Status of its content.** A snapshot. Every proposal in it is unruled. The ledger is the live\n> surface; when a row is ruled, the ledger changes and this text does not.\n>\n> **How it was written:** [`how-to-write-for-listening.md`](how-to-write-for-listening.md).\n>\n> The listening text starts below the line, unchanged from what was delivered.\n\n---\n\nThis is a walk through one question: what counts as evidence, and how evidence should enter Deliberus. It runs about an hour.\n\nHere's the map. Part one is what Deliberus already says about evidence, in your words and in the code, because that's where the research started. Part two is what academic philosophy and statistics mean by evidence. Part three is tiers: the evidence pyramid, and why the fields that invented it left it. Part four is what the professions of proof teach, meaning law, intelligence analysis, history, journalism and forensic science. Part five is independence, which turns out to be the single variable that matters most. Part six is testimony, lived experience and normative evidence. Part seven answers your own open questions directly. Part eight is the translation, meaning how all of this should enter the graph. Part nine is what none of it changes, and what I found broken in the code along the way. And part ten is the decisions waiting for you, as questions to carry on the walk.\n\nOne thing before starting. Everything in parts seven and eight is a proposal. None of it is ruled. It's all filed as Arc Q in the ruling ledger, twenty-six rows, and the documentation is saved before this was written, so nothing here exists only in this audio.\n\n## Part one. What Deliberus already says about evidence\n\nThe research started from the corpus, not from the literature, so let me start there too.\n\nYour founding statement on this, from March, in your words: \"I don't want any copouts, I want people and AI agents surfing around in Deliberus to see exactly what the best minds on our planet think in detail and what the best evidence shows, evidence that also is decomposed into its constituent parts and arguments.\" And then the sentence that set this whole research pass: \"What actually constitutes un-decomposable, more atomic evidence is an open question to me, scientifically, rationally, philosophically.\" You also said a meta-analysis PDF \"should be decomposed as far as possible, maximum granularity that is helpful to the rest of humanity.\"\n\nNow, a correction I owe you, because it's about my own words. In August, a research document answered your open question with a line that has been quoted since: \"there is no atomic evidence, only claims whose decomposition nobody has demanded yet.\" That line was mine, not yours. It was written as if it settled your question. It didn't, and the research below says it was half right. Your own later statement, from mid-September, points the same way but more carefully: \"evidence breaks down into a subgraph, and all of the subgraph claims can and should be dealt with in depth.\"\n\nSeveral of your rulings already constrain how evidence can be handled. The mother-claim pattern from April. Fact-checking being at least as important as values-checking, with the fact layer shipping first, from August eighteenth. Evidence named among the things that must be right before launch. Fallible self-knowledge. Bare judgments being telemetry, with no ratify act. Confident housekeeping being the machine's job. And horizon kinds being read, never set. Everything proposed later has to fit inside those.\n\nNow the code, because it's the other half of the grounding, and it's less flattering.\n\nIn the code, evidence isn't a type of thing. It's a position. Any claim that sits at the tail of a supports edge or an attacks edge is evidence for, or against, whatever is at the head. That's actually a good design, and the literature agrees with it, as you'll hear.\n\nBut strength only moves through two channels: answers to critical questions on incoming edges, and decomposition children. And the answers are counted. Each answer adds zero point one five on its question, capped at zero point four five. The content of the answer doesn't matter. Whether the answer has been attacked doesn't matter. Whether three answers came from one account doesn't matter. Three one-word answers count the same as three randomised trials. Keep that picture in mind, because counting is the one thing every tradition I read rejects.\n\nSecond, every property a scientist would call evidential quality is stored in the graph, and none of it does anything. Methodology tier, sample size, effect size, confidence interval, study count, heterogeneity, evidence type, epistemic status, confidence, edge strength. All stored, all inert. And evidence a person adds by hand is always labelled anecdotal and observed, whatever it actually is.\n\nThird, the weight readout counts inputs. It doesn't saturate, and it has no idea whether two inputs came from the same place.\n\nAnd what's missing entirely: any notion of a source's reliability separate from the claim; any notion of two sources being independent; any likelihood ratio; any way to say \"we searched and found nothing\"; page numbers, timestamps, or quote checks on evidence; timestamps on edges and answers; a scheme for study evidence as such; any difference between kinds of attack; and any status that says \"this is open\".\n\nWhat's sound: the split between direction and weight from August, with no number shown for anything unexamined. The weakest-part rule for required parts. Human-ratified expiry of stale claims. Reported speech that doesn't propagate strength. And evidence routing with fair-share discounting, which is built but not adopted.\n\nThe research also found three contradictions inside the corpus, and I'll resolve them as we go. One: some documents say impact testimony is never weighed as evidence, others say it is. Two: there are three different vocabularies of evidence kind in the code, and nothing maps them to each other. Three: two older documents call the strength model Bayesian, which it isn't. That last one is already corrected in place, pointing to ledger row P five.\n\n## Part two. What academia means by evidence\n\nPhilosophers give the word evidence four jobs, and they pull apart. Thomas Kelly, who wrote the Stanford Encyclopedia's entry on it, lists them. Evidence is what justifies belief. It's what a rational thinker respects. It's a sign or symptom of how things are. And it's a neutral arbiter between people who disagree.\n\nThat fourth job is the one Deliberus exists for. And it's the one philosophers doubt evidence can fully do, because what counts as evidence is itself contested. There's a real measurement of this. The Forecasting Research Institute ran an adversarial collaboration on AI risk, where good-faith experts spent between twenty-five and a hundred hours each arguing it out. What remained at the end wasn't disagreement about facts. It was disagreement about standards of evidence. The design answer the literature supports is the one Deliberus already leans toward: where evidence can't be neutral, make the disagreement about what counts public, as claims.\n\nNow the most important idea in this whole part, so I'll take it slowly. Evidence is comparative.\n\nThe question \"is this evidence for my hypothesis?\" is incomplete. The complete question is: how much more expected is this evidence if my hypothesis is true than if it's false? That ratio is called the likelihood ratio. The statistician I. J. Good, who worked with Alan Turing at Bletchley Park, called its logarithm the weight of evidence. Turing measured it in units he called decibans. And the property that matters here is that it adds up across items, but only when the items are conditionally independent. That's both the formal licence for summing evidence and the exact condition under which summing is allowed.\n\nThe statistician Richard Royall put this as three questions, in a book from nineteen ninety-seven. What should I believe? That needs a prior. What should I do? That needs costs and values. And what does this evidence say? That needs neither. Just the likelihood ratio. Statistics had been mixing the three up for a century. Hold onto Royall's three questions, because in part eight they turn out to be the cleanest description of what Deliberus is.\n\nA few more ideas from the philosophy that do work later.\n\nRaising a probability isn't the same as making it high. Rudolf Carnap drew that line. Today's badge reads as both at once.\n\nSeverity, from the philosopher of statistics Deborah Mayo. Data count as evidence for a hypothesis only if the test would probably have failed that hypothesis were it false. Publication bias, stopping a study when the numbers look good, and data-dredging all lower severity without changing a single data point. So they attach to the inference, not to the conclusion. In graph terms, they belong on the edge.\n\nWeight versus balance. John Maynard Keynes, the economist, in nineteen twenty-one: how much evidence stands behind a judgment is a different thing from which way it points. The philosopher Brian Skyrms made weight concrete as resilience, meaning how little the judgment would move if more evidence arrived. Deliberus's August split into direction and weight is exactly Keynes's split, and it's on the right side of this line.\n\nTotal evidence. Carnap again, plus Good's theorem that free evidence can never hurt you in expectation. A verdict reached from less than the available evidence is defective even if every step in it is valid. That's the formal name for what the corpus calls coherent absence.\n\nThen aggregation, where the surprises are.\n\nAgreement multiplies only across independent errors. That goes back to Condorcet's jury theorem. If two reports share an origin, their agreement can be worth barely more than one report.\n\nBut the common slogan, diverse evidence beats repeated evidence, is false in general. The philosophers Luc Bovens and Stephan Hartmann showed that two agreeing reports from one instrument can be evidence that the instrument is reliable. Whether variety helps depends on how sources fail. That's why the corpus's own rule from the end of August, that a known shared source becomes a node and never a blind discount, is exactly what this literature recommends. Your weakest-link work landed on the right answer before this research arrived.\n\nCoherence isn't truth-conducive in general either. There are impossibility results, from Bovens and Hartmann and from Erik Olsson, showing that a tightly coherent cluster of claims from one family of sources isn't thereby likely to be true. Remember that one, it comes back as a detector in part eight.\n\nThen causation. Federica Russo and Jon Williamson argue that causal claims normally need two kinds of evidence together: difference-making evidence, like trials and correlations, and mechanistic evidence, meaning how it works. And Nancy Cartwright, the philosopher of science, has the phrase \"evidence for use\": \"it worked there\" isn't \"it will work here\". Relevance is a property of the study, the target and the argument together. In graph terms, relevance belongs to the edge, not to the evidence.\n\nAnd finally the question you asked directly: is any evidence atomic? The philosophers are unanimous on one half. Wilfrid Sellars, attacking what he called the Myth of the Given, and Quine, and every modern modest foundationalist agree: no evidence is immune to challenge. That supports the August line, as anti-foundationalism.\n\nBut two qualifications cut against it. The first is Susan Haack's crossword. She compares justification to a crossword puzzle. The entries support each other, but there are also clues, and the clues are a different kind of thing from the entries. They aren't beliefs, and they aren't infallible, but a crossword filled in coherently while ignoring the clues is just wrong. Evidence has an anchor that other claims don't have. The second is that \"demanded\" should be \"reasonably challenged\". If any bare demand triggers decomposition, decomposition becomes a stalling weapon, and someone arguing in bad faith can demand forever. Robert Brandom's model, where a claim stands by default until someone gives a reason to challenge it, is the fix.\n\nAnd one more idea: hinges, from Wittgenstein. Some commitments make evaluation possible at all, and aren't improved by adding support. That's a principled place where decomposition stops being productive, and it shouldn't be scored as a copout.\n\n## Part three. Tiers, and why the pyramid's inventors left it\n\nYou asked what tiers of evidence can consist of. The familiar answer is the pyramid: systematic reviews and meta-analyses at the top, then randomised trials, then cohort studies, case-control studies, case series, case reports, and expert opinion at the bottom.\n\nHere's the important thing about the pyramid. It was always a proxy for one question: how likely are confounding and selection to distort an estimate of whether a treatment works? It was never a general measure of how much a piece of evidence weighs. And the fields that invented it have left it.\n\nOxford's Centre for Evidence-Based Medicine replaced it in twenty eleven with a table: a different ladder for each kind of question. How common is something, is this diagnosis accurate, what's the prognosis, does the treatment help, what are the common harms, the rare harms. For \"how common is this\", the top rung is a local random survey, and a randomised trial is simply irrelevant. And an observational study with a dramatic effect sits beside the trial.\n\nIn twenty sixteen a group led by Hassan Murad redrew the pyramid with wavy lines between the layers, and took the systematic review off the top. They drew it as a lens held over the pyramid instead. A review is a reading of the studies under it, and it inherits their quality. It isn't a better kind of study.\n\nThe framework that actually runs medicine now is called GRADE. Here's what it does, because it's the best worked model of evidence grading anywhere.\n\nGRADE rates certainty in four levels: high, moderate, low, very low. And certainty is always about one claim: how sure are we that the true effect lies on one side of some threshold? It's never a free-floating property of a study. The unit is the body of evidence for one outcome. The same review can be high certainty for one outcome and very low for another.\n\nStudy design is only the starting point. Randomised evidence starts high, observational evidence starts low. Then five named things move it down: risk of bias, inconsistency between studies, indirectness, imprecision, and publication bias. And three things move it up: a large effect, a dose-response relationship, and confounding that would have made the effect smaller rather than bigger.\n\nAnd certainty is kept strictly apart from the recommendation. Values, costs, equity, acceptability and feasibility are separate rows in a separate framework. One number I verified: in WHO guidelines, about fifty-five percent of strong recommendations rested on low or very low certainty evidence. Decisions routinely outrun evidence. Sometimes that's legitimate, because values justify acting anyway, and sometimes it isn't. Only a structure that holds both evidence and values, visibly apart, makes the difference visible.\n\nLast thing on GRADE: grading is noisy. Trained raters agree reasonably well overall, but on individual judgments like imprecision and risk of bias their agreement can fall almost to chance. So grading is contested ground. The machine should propose grades and people should argue about them.\n\nAcross fields, the same shape repeats. The climate panel, the IPCC. The WHO's cancer agency, IARC. The European food safety authority. England's Education Endowment Foundation, with its padlock ratings. They all do four things. Separate lines of evidence by type. Appraise each on a few named dimensions. Combine them by explicit, mostly ordinal rules, very often weakest-link. And report certainty separately from the direction of the effect, and separately again from the decision. No mature framework produces one cardinal number from a design label.\n\nThree details worth borrowing. The IPCC keeps confidence, meaning how much evidence and how much agreement, completely apart from likelihood, meaning a calibrated probability. The cancer agency IARC separates hazard, can this cause cancer at all, from risk, how much and at what exposure. And IARC has a category called Group Three, \"not classifiable\", which means the evidence doesn't decide this. It's an explicit \"this is open\" status. Deliberus has nothing like it. And the padlock ratings take the lowest of three scores and then mark down from there, weakest-link then downgrade, and they say outright that the rating takes no account of whether the intervention worked. Certainty kept apart from direction, again.\n\nNow meta-analysis, since your March sentence was about a meta-analysis PDF. Meta-analyses mislead in known ways. A twenty twenty-six study of a thousand meta-analyses across ten disciplines found that fewer than half of those with dependent effect sizes adjusted for the dependence, and fewer than half of those that detected publication bias adjusted for it. And re-running a meta-analysis's arithmetic reproduces well, while re-making its choices doesn't. Many defensible choices, different conclusions from the same studies.\n\nI have to own a correction here too. The corpus had that thousand-meta-analyses result attributed to \"Hedges and colleagues\". It's by Wu, Duan, Reed and Tipton, in Research Synthesis Methods. The attribution is corrected in place.\n\nTwo older results matter as well. Vote counting, meaning tallying significant studies against non-significant ones, can converge on the wrong answer as underpowered studies pile up. That's Larry Hedges and Ingram Olkin, nineteen eighty. That's the statistical proof that counting fails. And a p-value of zero point zero five corresponds, under broad conditions, to a Bayes factor of at most about two and a half. That's weak evidence, whatever the word \"significant\" suggests.\n\nAnd one example that stays with me. Twenty-nine research teams were given one dataset and one question: are football referees more likely to give red cards to dark-skinned players? Their answers ranged from slightly less likely to nearly three times as likely. A single study's estimate is a draw from a distribution of defensible analyses.\n\nSo here's the verdict on the methodology ladder in the code, the one that goes from meta-analysis at zero point seven down to expert opinion at zero point three. It fails on six grounds from the literature. It ignores the type of question. It grades the wrong unit, a source rather than a result. It treats design as a verdict rather than a starting point. It ranks meta-analysis as a tier, when a meta-analysis is a reading of its inputs, and double-counts the trials inside it. It puts expert opinion on the ladder, when GRADE's own position is that expert opinion is an interpretation of evidence, not evidence. And it has no place at all for mechanistic or qualitative evidence. There's a seventh, internal to the corpus: the numbers are uncited house choices, and a cardinal gap between zero point four five and zero point five five claims a precision no framework claims.\n\nWhat survives is a labelled default starting point for claims about whether something works, stated as a claim anyone can attack.\n\n## Part four. What the professions of proof teach\n\nFive professions decide under evidence for a living: law, intelligence analysis, history, journalism, and forensic science. None of them has a genre ladder at the centre of its practice. And none of them resolves anything by counting supporting items. The same four moves recur across all five, and law adds a fifth.\n\nThe first move: separate the source from the report. How good a sayer is in general is a different question from whether this statement is true. NATO's intelligence services grade every report on two separate axes, letters A to F for how reliable the source usually is, numbers one to six for how credible this particular piece of information is. The axes are supposed to disagree. A reliable source can be deceived this once. An unreliable source can be independently confirmed. And the cases where they disagree are the most informative ones. But here's a measurement of what goes wrong. In about fourteen hundred US Army field reports from the nineteen sixties, eighty-seven percent of ratings fell on the diagonal: A-one, B-two, C-three. In practice the two axes collapsed into one.\n\nThe second move: break credibility into named parts, each of which can be attacked separately. The law of hearsay names four ways testimony can fail: perception, memory, narration, and sincerity. The evidence scholar David Schum, whose work I'll come back to, names veracity, objectivity, and observational sensitivity. And here's the part I found most useful. The hearsay rule doesn't mean second-hand information is worthless. Historians keep hearsay, and say so. What it really means is this: a claim whose credibility can't be interrogated shouldn't be weighted as if it had been. Cross-examination is the machine for interrogating it. And every exception to the hearsay rule names a structural reason why one of the four failure modes is unlikely. A statement against the speaker's own interest, since people rarely lie against themselves. A routine business record. An excited utterance, made with no time to invent anything. So answering critical questions is Deliberus's cross-examination. And an exception-type fact, like \"this was said against the speaker's own interest\", is itself a claim that argues for a higher starting weight.\n\nThe third move: weigh evidence by how much it discriminates between explanations, not by how much of it there is. Forensic scientists call this the likelihood ratio. Richards Heuer, who wrote the CIA's textbook on the psychology of intelligence analysis, called it diagnosticity. And he had the image that carries it. A high temperature tells a doctor the patient is sick, and almost nothing about which illness. In his words: \"most of the evidence supporting what you believe is the most likely hypothesis really is not very helpful, because that same evidence is also consistent with other hypotheses.\" One honest caution goes with it. Heuer's specific procedure, Analysis of Competing Hypotheses, did no better than untrained analysis in a randomised study, thirty-six percent correct against thirty-three. The concept is settled. That procedure isn't.\n\nThe fourth move: agreement counts only in proportion to independence. The standard manual of historical method, by Langlois and Seignobos in eighteen ninety-eight, has this as a sequence of chapter headings: \"Agreement of statements. Necessity of proving them to be independent. Perfect agreement not so conclusive as occasional coincidence.\" Word-for-word agreement suggests copying. Scholars of old manuscripts turned that into a method: shared errors reveal a common ancestor, so a family of copies counts once.\n\nThe canonical failure is the Iraq weapons case. One informant, codenamed Curveball, never met by any American officer, supplied about a hundred reports on mobile bioweapons labs. The intelligence estimate attributed the judgment to \"multiple sensitive sources\". And the commission that investigated afterwards wrote that the daily briefs \"left an impression of many corroborating reports where in fact there were very few sources.\"\n\nThat record holds four separate failures, and each has a direct analogue in a graph. One: many reports from one source counted as volume. Two: \"multiple sources\" asserted without anyone checking their independence. Three: a source flagged as a fabricator in May two thousand two was still being used in October, because the retraction never propagated. And four: an agency that passed reports along saw itself as \"just a conduit\", so relaying a report laundered its credibility. A preprint from this very month states the machine version: with the evidence held fixed and the number of reporting AI agents raised from one to thirty-two, naive aggregation dropped the reliability of the result from about ninety-four percent to about twenty-six. Their phrase: \"another agent is not another observation.\" Every language-model pass in Deliberus is a relay in exactly that sense.\n\nThe fifth move is law's. The threshold for accepting a claim is set by the question and by the cost of getting it wrong, not by one universal scale. Balance of probabilities in a civil case, beyond reasonable doubt in a criminal one. There's a formal system in the same Walton lineage Deliberus already uses, called Carneades, in which every statement carries its own proof standard, and \"acceptable\" means meeting it.\n\nAnother correction to own here. A figure circulating in my own grounding notes had Carneades defining five standards. The original twenty-oh-seven paper defines three: a scintilla of evidence, meaning at least one defensible argument in favour; best argument; and dialectical validity, meaning no defensible counter-argument survives. The five-standard version, adding preponderance, clear and convincing, and beyond reasonable doubt, is a later paper from twenty-oh-nine. The synthesis says so now.\n\nCarneades gives three things Deliberus lacks. A statement can meet a low standard and its opposite at the same time, which is the honest difference between acceptable and true. Burden is allocated by the type of premise: some critical questions merely ask, and the proponent then has to answer; others must bring evidence before they bite. And critical questions come in kinds: some undercut the inference, some rebut the conclusion.\n\nNow the single most important piece from this whole strand, so again, slowly. David Schum, the evidence scholar, showed that two items of evidence pointing at one claim can be at least five different things, and they combine differently.\n\nIf different sources report the same event, that's corroborative. Corroboration should saturate. In Schum's phrase, a second report \"springs to life to the extent that we find the first source not credible\". If the first source were perfectly credible, the second would add nothing.\n\nIf different events point to the same conclusion, that's convergent. Convergent items add, and they can be synergistic: two items together can rule out an explanation that neither rules out alone.\n\nIf the second item's force is already contained in the first, that's cumulative, and it adds nothing new. Law's own word for that is cumulative.\n\nIf the reports are of events that can't both have happened, they're contradictory. And contradictions are resolved on credibility, never on headcount. There's a formal result from nineteen seventy-three, by Schum and Kelly: the majority wins only if its sources are jointly more reliable than the minority's. Source reliability matters more than the amount of agreement. So a badge must never show three against one as a majority.\n\nAnd if the events can both have happened but point different ways, they're divergent, and they need a joint explanation, not a tally.\n\nDeliberus today has one evidential relation, and its contributions add. So it overcounts corroboration and cumulative evidence, and it undercounts synergy. The corpus's own five-case taxonomy of double-counting, from the end of August, is the structural half of this. Schum supplies the semantic half. And it names the repair for something measured earlier: the seventy decomposition edges that turned out to be evidence misfiled as parts. Three studies of one effect are corroborating witnesses, which saturate. They were never necessary parts, which would be weakest-link.\n\nTwo more lessons from history. First, the argument from silence has stated conditions. Silence proves something only if no document mentioning the event ever existed, and only if the event is of a kind that couldn't fail to be recorded. And Langlois and Seignobos name its typical abuse: silence about \"the secret acts of governments and the complaints of the lower classes.\" An archive that systematically leaves out a class of events can't support negative conclusions about that class. That's a critical-question set for arguments from ignorance, with a political-bias warning attached. Heuer adds the answer menu for any absence: it didn't happen, it wasn't observable, it was concealed, or nobody looked.\n\nSecond, the historian Marc Bloch prized what he called witnesses in spite of themselves. A merchant's ledger over a chronicle, because an unintended trace has no motive to mislead. Veracity is a question only for intended testimony.\n\nFrom journalism, one finding that surprised me. Professional fact-checkers agree on the facts and disagree on the rating. A study comparing PolitiFact and the Washington Post found that where the two differed, the difference came from their judgment of how significant an inaccuracy was, not from whether the statement was true. On one claim by Jeb Bush, both gave identical reasons, and verdicts two notches apart. Another study matched seven hundred and forty-nine claims checked by both Snopes and PolitiFact and, after adjusting for scale and wording, found one genuine conflict. So the disagreement lives in exactly one step: mapping a set of findings onto a single verdict. That's the step Deliberus declines to take. The rule to carry over is that each finding becomes its own claim, with its own significance claim beside it, \"this omission matters because\". And there's a free validation test in it. On claims checked by several organisations, a Deliberus decomposition should reproduce their shared findings wherever their ratings differ.\n\nAnd from forensic science, four lessons.\n\nOne: whether a method is valid is measured, not credentialed. A US presidential science council reviewed forensic methods in twenty sixteen and said a subjective method's validity can only be established by black-box studies, where many examiners judge samples whose true answer is known. Their measured false-positive rates for fingerprint comparison ran between one in three hundred and six and one in eighteen, depending on the study. Bitemark analysis was typically above ten percent. This is the professions' strongest argument against a genre ladder: they grade the specific method on this report by its measured error rate, not by the kind of evidence it is.\n\nTwo: report a likelihood, never a posterior. European forensic guidelines require findings to be stated as \"support for this proposition rather than that one\", on a six-step verbal scale. The wording is there to block two named errors. The prosecutor's fallacy reads the chance of the evidence, given innocence, as the chance of innocence. The defence fallacy dismisses a huge likelihood ratio because the final probability is still modest.\n\nThree, and this one goes straight to the badge: the weak evidence effect. In experiments with about nine hundred people, evidence described as weakly supporting guilt made people lower their belief in guilt. They inverted it. So a label that says \"weak support for X\" can be heard as evidence against X.\n\nFour: the order in which you see things changes your judgment. Fingerprint examiners given irrelevant context, like a confession, have contradicted their own earlier identifications. The fix is called linear sequential unmasking: analyse the trace alone, write it down, and only then see the reference. And the science council found examiners changing the features they'd marked after seeing a matching print. That's the same circularity a claim graph invites when it shows someone the current badge before they answer a question about it.\n\nAnd a fifth forensic idea: the hierarchy of propositions. One finding has different force at each level. \"The DNA is his\" is one level. \"The stain is his\" is the next. \"He did the act\" is the next. \"He's guilty\" is the top. A DNA match can be strong at the first level and nearly worthless at the third, because transfer and persistence sit in between. Outside crime the ladder reads: the study measured X, X happened in the sample, the intervention caused X, the policy is good. An edge that jumps several levels at once skips exactly the steps that carry the contestable assumptions.\n\n## Part five. Independence, and the formal models\n\nIndependence has come up in every part so far. Here's how the formal and computational fields handle it, and a few other things they get right.\n\nFirst, three kinds of attack, from the philosopher John Pollock and the argumentation framework called ASPIC-plus. A rebutter is a reason against the conclusion. An undercutter says the premises don't support the conclusion here. The standard example: the object looks red, but it's lit by red light. And an underminer attacks a premise. The key point is that an undercutter should move a claim toward \"we don't know\", not toward \"false\". A model where every attack subtracts strength can't express that, and Deliberus's current model is that kind of model.\n\nSecond, counting versus quality is a proved trade-off, not a preference. Leila Amgoud and Jonathan Ben-Naim showed you can't have both \"fewer attackers wins\" and \"one strong objection beats many weak ones\". The current strength model is counting-sensitive and fully compensating. The literature's advice is: choose, and state the choice.\n\nThird, subjective logic, from Audun Jøsang. An opinion there has four parts: belief, disbelief, uncertainty, and a base rate. It's a published algebra for exactly the badge's split between direction and weight, with three things the badge lacks: a base rate for each kind of claim, weight that saturates by construction, and different rules for combining independent sources and dependent ones. And in that algebra, a long chain of hearsay drives an opinion toward uncertainty, not toward disbelief. That's the right behaviour.\n\nFourth, copy detection. The computer scientists who work on what's called truth discovery found that you detect copying by shared errors, not shared truths. Independent sources rarely make the same mistake. That transfers directly: identical phrasings, identical citation errors, identical defeated positions.\n\nFifth, and this is the one production system I found that combines very different kinds of evidence using explicit strength tiers: clinical genetics. When a lab decides whether a gene variant causes disease, it uses a scheme recast in twenty eighteen as a simple probability model. Supporting evidence is worth one point, moderate two, strong four, very strong eight, and the points add up to a classification. Three rules in it matter for us. A type of evidence earns its tier only at the strength it's been measured to have, not by declaration. The same data can't be used twice under two different criteria. And \"uncertain significance\" is the default outcome when evidence is thin or conflicting.\n\nAnd sixth, a database of protein interactions called STRING treats all text-mining evidence as one single channel. So a thousand abstracts repeating one finding can't multiply. That's independence as structure.\n\nOne warning from the ontologies. The Gene Ontology's evidence codes were designed to describe what kind of work supported a claim, and their designers said explicitly that the codes don't mean quality. Users turned them into a quality ladder anyway. Any evidence type we display will be read as a weight.\n\n## Part six. Testimony, lived experience, and normative evidence\n\nThis part matters to you personally, I think, because it touches impact testimony and the attunement pole.\n\nThe most defensible version of standpoint epistemology, from the philosopher Alison Wylie, together with Miranda Fricker's work on epistemic injustice, and the psychology of introspection, all support one thing: evidential advantage. Being in a position makes you more likely to know something specific, to a degree, in a way that can be checked. And all of them warn against sliding from that into authority, where a position's say-so closes a question. Olúfẹ́mi Táíwò's critique of what he calls elite capture is, underneath, a statistical point: the testimony a platform receives is a self-selected sample of the positions it claims to represent. The corpus rule, testimony is evidence for a premise and never a verdict, is the advantage-without-authority position in software. It holds up.\n\nNow the strongest evidence for impact testimony that I found. Daniel Gilbert and colleagues, in Science, twenty-oh-nine. People predicted their own reaction to an experience better from one other person's report of going through it than from full information about the experience itself. And both the participants and outside observers expected the opposite. The disability paradox points the same way: people living with serious disability rate their quality of life far above what outsiders expect. So impact testimony's main job is correcting other people's imagined versions of a position. It's less about establishing causes.\n\nThen your ratified conviction on fallible self-knowledge. One half of it is very well supported: people are fallible about the causes of what they feel, about counterfactuals, and about what would satisfy them. The classic study by Nisbett and Wilson, the choice blindness experiments, affective forecasting. All strong.\n\nThe other half, that people can't be wrong about the felt experience itself, is challenged. The philosopher Eric Schwitzgebel argued in twenty-oh-eight that people make gross errors about their current experience, at least about its fine structure. Defenders reply that coarse present states, \"this hurts\", are reliable. So I propose a rewording, unruled, because it touches a ratified line. People are authoritative about their present refusal or endorsement, since that's a speech act that makes its own fact. Presumptive about their present coarse state. Fallible about the fine structure of experience and about generalisations over time. And weak about causes, counterfactuals and satisfiers.\n\nChoice blindness adds a design warning. People confronted with a choice they didn't make will often defend it as their own. So a person confirming a machine's rendering of their position is weak evidence that the rendering is right. That's why their own words should come first.\n\nNow the corpus contradiction from part one: is impact testimony weighed as evidence or not? It dissolves once you see it's three different edges.\n\nFirst, testimony as evidence for an empirical premise. \"This policy costs people in my position in this particular way.\" That moves strength, subject to its critical questions, like any testimony.\n\nSecond, testimony as input to a values premise, or a weighing. \"Can this be justified to people in my position?\" That moves strength only through the weighing scheme's own questions, never as proof of the conclusion.\n\nAnd third, registered testimony. Reported speech, on the edge that doesn't propagate. Heard, available to the first two roles, moving nothing. Forcing every piece of testimony into an evidential role is itself a documented harm. South Africa's Truth and Reconciliation Commission explicitly recognised four kinds of truth for this reason.\n\nOne more amendment to the corpus's testimony tiers: \"positionally corroborated\" should require independent corroboration, strongest from people in the same position who otherwise disagree. That's the Community Notes logic, applied to testimony.\n\nAt scale, truth commissions in Peru, Guatemala and Kosovo treated testimony as indispensable and also systematically biased as a sample. They resolved it neither by discounting nor by counting, but by modelling the process that produced the testimony. They compared overlapping independent lists of victims and estimated how many were on none of them. In Peru, that reversed the picture the raw testimony counts suggested about who was responsible for most of the deaths. That's a working method for coherent absence: the overlap between independent lists is the evidence about what's missing.\n\nNormative premises have reasons, not evidence in the empirical sense. They're assessed by coherence, by stability when the presentation changes, by resistance to debunking arguments, and by reciprocity. And a debunking argument, \"you only believe this because of how you evolved\", is an undercutter in exactly Pollock's sense. The idea that bridges the two is inductive risk, from the philosophers Richard Rudner and Heather Douglas: the stakes of a decision legitimately set how much evidence is enough. Stakes never stand in for the evidence itself.\n\nAnd expert judgment without an expert gate. Weighting experts by their demonstrated calibration on questions with known answers beat equal weighting in twenty-six of thirty-three studies. Status and experience predicted performance poorly. And seeing others' estimates narrows the spread without improving accuracy. But calibration can only be measured where ground truth eventually arrives. So it's a route for the fact layer, never for evaluative claims.\n\n## Part seven. Answers to your open questions\n\nYour March question first. What actually constitutes un-decomposable, more atomic evidence?\n\nHere's the answer the research converges on, proposed and unruled. There is no atomic proposition, but there is an atomic encounter.\n\nEvery evidential claim can be challenged and decomposed. Sellars and Quine are right about that. But decomposition doesn't bottom out in a smaller claim. It bottoms out in an encounter: the moment something in the world was observed, measured, recorded, or said. The graph can only point at that encounter, with a locator, meaning the source, the page, the timestamp, the span of text, and with a record of how it came to us.\n\nAn encounter can't be decomposed into claims. It can only be re-encountered. Re-read, re-measured, replicated, observed again. Claims about the encounter, like \"the thermometer was calibrated\", \"the witness was there\", \"the transcription is faithful\", stay ordinary claims that decompose like any other.\n\nFour separate traditions arrive at this same shape. Haack's crossword clues. Schum's point that the report is not the event. Fact-checking systems, where evidence is always a located span in an identified source. And Yoshua Bengio's safety work, which separates a factual observation from a communication about it.\n\nSo the proposed restatement of the August line is this: no evidence is immune to challenge, and every evidential claim decomposes on reasonable challenge. What evidence finally rests on isn't a smaller claim, but a located encounter with the world, which can only be re-examined or re-encountered.\n\nNext, ledger row P two, your hope from April that the same evidence should give everyone the same probability. It's vindicated where Deliberus operates, and qualified elsewhere. On what the evidence says, agreement is achievable and doesn't depend on who you are. The likelihood ratio doesn't depend on anyone's prior. Two people who start in very different places can agree exactly on how much a finding favours one hypothesis over another. On what to believe, agreement is a long-run ideal, not something finite evidence can require. With thin evidence the rational state can be a range rather than a point, and two people choosing different points inside that range aren't being irrational.\n\nThen P three, your fear of fuzzy, gooey subjectivity. The answer is named slots. Every divergence in conclusion from shared evidence has to be traced to one of four things: a prior, a standard of proof, a model, or a piece of evidence. Each of those becomes a claim with a name on it, which anyone can attack. Subjectivity isn't banned. It's located.\n\nAnd P seven, whether Deliberus should ever output a probability. The research favours no, with a check. The badge reports the evidence term: direction, weight, and certainty. Never a final probability, which would import a prior the platform has no right to choose. Where claims eventually resolve against the world, the oracle bridge can still test whether strong readings turn out true more often. That's my recommendation. P seven stays open.\n\n## Part eight. The translation into Deliberus\n\nEverything in this part is a row in Arc Q, and all of it is open. I'll go from the frame to the concrete.\n\nThe frame, Q one. Royall's three questions become the public statement of the no-verdict design. Deliberus answers \"what does the evidence say?\", maps the inputs to \"what should I believe?\" and \"what should I do?\", and answers neither of those for anyone. The badge is the evidence term, not a posterior. It's the most defensible academic form of \"maps reasoning, does not endorse conclusions\" that I found.\n\nQ two is the atomic encounter from part seven.\n\nQ three: weigh critical-question answers, never count them. An answer moves its edge through its own standing, not by existing. The recursion the badge already uses for decomposition children should reach answers too. An attacked answer should weigh less than an unattacked one, and three one-word answers from one account shouldn't move anything more than one. Each answer would carry a direction and a proposed strength tier, on the clinical-genetics shape: one, two, four, eight. The tier is proposed and is itself attackable. A bare answer defaults to the lowest tier. And a kind of answer earns a higher default only when its strength has actually been measured. That's the one route by which any methodology default could become live without ratifying invented constants. Counting survives, honestly labelled, as coverage: \"four of six questions addressed\".\n\nQ four: independence as structure. Sources become nodes, with their reliability as attackable claims: people, labs, instruments, datasets, funders, publications. Inputs sharing an origin form one channel. Channels combine as independent, and items within a channel don't multiply. Shared origin is proposed, never silently discounted, using the shared-errors test. And agreeing reports from one source raise that source's reliability, not the claim's strength.\n\nQ five: weight becomes Keynesian. Count only independent inputs. Let weight saturate as the scheme's questions are covered. And add a resilience readout: how far the reading would move if its single most influential input were removed. The hinge machinery can already compute that.\n\nQ six: certainty as a third badge axis. High, moderate, low, very low, or not yet assessed, always relative to a threshold claim, read rather than set, with each step linked to the claim that justifies it. So \"why is this moderate?\" is a question the graph can answer. On public surfaces, GRADE's plain verbs: probably, may, we're uncertain whether. Never a number.\n\nQ seven: three claims, not one, for studies. A study-result claim, particular and anchored, where risk-of-bias questions live. A body-of-evidence claim above it, the mother, where inconsistency, imprecision and publication bias live. And the target claim someone is actually arguing, where indirectness lives, on the edge, because the same body of evidence can be direct for one target and indirect for another. Study results are corroborating children of the body, never necessary parts. And that's your meta-analysis PDF, decomposed to maximum granularity, with a known skeleton.\n\nQ eight: the methodology ladder retires as a strength input. At most it survives as a labelled default starting certainty, per question type, or better, built from design features, each an attackable claim. Expert opinion leaves the ladder and becomes a scheme with its own questions.\n\nQ nine: typed attacks and typed questions. Rebut, undercut, undermine. And questions typed as assumptions, where an open question withholds weight, or exceptions, where an open question costs nothing until someone backs it up.\n\nQ ten: absence as data. A search claim, saying what was looked for, where, how, when, and how sensitive the search was. Without the sensitivity, it's shown as nearly worthless. This one also unsticks a scheme that today can never leave neutral.\n\nQ eleven: scope and locators. A particular finding must carry a source and a locator. General claims are supported by particulars through a generalising inference that carries the \"will it work here?\" questions. And provenance in two layers: who found what, by which method, separately from which contributor or which version of the pipeline wrote it into the graph, and when.\n\nQ twelve: source reliability and report credibility kept apart, NATO-style. And for testimony or citations, Schum's chain on demand: did the source say it, did they believe it, were they positioned to observe it, did it happen.\n\nQ thirteen is impact testimony as three edges. Q fourteen is the rewording of the fallible-self-knowledge conviction, which is entirely your call. Q fifteen is the named slots for divergence. Q sixteen: a standard of proof is a claim attached to a question by whoever asks it, and the graph says whether the current reading meets that person's stated standard, never a verdict. Q seventeen: normative premises typed and graded by their own method, debunking arguments filed as undercutters, and inductive risk as an explicit bridge.\n\nQ eighteen: an \"open\" status. The evidence doesn't decide this. Modelled on the cancer agency's Group Three, and distinct from \"no effect\". That's the same gap your friend's AI governance map hit earlier this week, where twenty-four explicit open questions had nowhere faithful to live.\n\nQ nineteen: the import discipline. The corpus rule for borrowed taxonomies applies with force: a borrowed category earns its place by yielding a move. GRADE's domains, Schum's attributes and the clinical tiers all yield moves, so they enter as critical questions and parameters, each with a does-not-fit value. Never as schema cells, and never as a ladder displayed as weight.\n\nQ twenty: two structural facts that are displayed and never added to strength. Consilience, meaning support arriving from regions or methods that fail in different ways. And a coherent-but-unanchored flag: a cluster of claims that support each other strongly while containing no anchored finding at all. That's Haack's crossword filled in without the clues.\n\nQ twenty-one: a machine pass's measured error rate is itself evidence about every output of that pass, and should discount the confidence of all its unexamined outputs, not just the ones already caught.\n\nAnd the last five, from the professions. Q twenty-two: typed combinations, where a machine pass proposes whether edges are corroborative, convergent, cumulative, contradictory or divergent, and anyone can contest it. Q twenty-three: retraction propagates and relays don't launder. The Curveball lessons, with machine passes counted as relays. And any surface that says \"N items support this\" also says from how many independent origins. Q twenty-four: a flag when an evidential edge jumps more than one level of the hierarchy of propositions. Q twenty-five: badge wording rules. \"Support for X rather than Y\", with the rival named. Never \"weak support\" near neutral, say \"not yet discriminating\" instead. And never likelihood and confidence in one phrase, on any surface, including the ones agents read. Since that's public copy, it's also a matter of your voice. And Q twenty-six: answer before seeing the badge, where feasible, with a record of what the person had seen.\n\n## Part nine. What this doesn't change, and what I found broken\n\nNone of these proposals needs a new node type. Everything uses claims, edges and properties, and sources are already nodes. Nothing requires an evidence node type.\n\nPropose, never assert, still holds. Tiers, gradings, shared-origin links and certainty steps are all proposals and attackable claims. The grading-reliability numbers from part three say they have to be.\n\nBare judgments stay telemetry. But the research sharpens the reason, and states the condition under which that could change: judgments that are both independent and measurably calibrated can be evidence. Only the fact layer can supply those two conditions.\n\nNo verdicts. Certainty and standards of proof are readings relative to stated thresholds.\n\nAnd decomposition stays maximal down to separability. \"Reasonably challenged\" governs decomposition that people request. It doesn't limit the machine's own separability floor.\n\nNow the defects. Reading the code for the grounding, I found things that are repairs rather than rulings, and they're listed separately in the to-do file, so they don't get mixed in with Arc Q. None was fixed today, because you asked for research and documentation. The main ones.\n\nThere's a second badge implementation, feeding the query endpoint and the syntheses, that skips every exclusion the real badge applies. So the synthesis prompt reads strengths the claim page wouldn't show.\n\nTwo contribution endpoints, adding evidence and answering critical questions, swallow errors with a silent catch-all. That's exactly the pattern your hard-won gotchas forbid. When it fires, the new edge becomes invisible to the badge, and nobody is told.\n\nAny polarity value other than \"supports\" becomes an attack, unvalidated.\n\nThe filter for unconfirmed machine proposals is partial, so some unconfirmed proposals still move strength.\n\nAnd a scheme with only one critical question can never leave the neutral band on a single edge. At full saturation it reads zero point five oh four, unsettled forever. That's why arguments from lack of evidence are permanently stuck today.\n\nPlus smaller ones: question ordering prices states that can't actually occur, a confession fallback that can never be reached, gaps in timestamps and staleness handling, values outside their allowed sets, and the measurements that are owed. All in the to-do file, under the evidence layer heading.\n\nEarlier today, separately, I did fix one real bug, in commit five-a-b-eight-c-one-d: the query endpoint crashed on claims with no data, and it had also been miscounting claims nobody had weighed as support. That fix is committed on the branch.\n\n## Part ten. The decisions waiting for you\n\nThis is the last part. These are the questions to carry, not tasks.\n\nFirst. Is \"there is no atomic proposition, but there is an atomic encounter\" the right answer to your March question? If it is, the August line gets restated, and locators become mandatory on particular findings.\n\nSecond. Do you want critical-question answers weighed by their own standing, with proposed strength tiers? That's the biggest single change here. It moves every strength in the graph, and it's the one every tradition I read would ask for first.\n\nThird. Should independence become structure, with sources as nodes, channels, and shared origin proposed on the shared-errors test? Or does that feel like too much machinery before the reviews are done?\n\nFourth. Does certainty deserve a third axis on the badge, beside direction and weight? And does \"not yet assessed\" belong there as an honest default?\n\nFifth. Should the methodology ladder be retired as a strength input, as the research says, even though its numbers never did anything?\n\nSixth. Is the fallible-self-knowledge rewording faithful to what you meant by incorrigible? That one touches a ratified line, so only you can move it.\n\nSeventh. Do you want an \"open\" epistemic status, the evidence doesn't decide this, distinct from no effect?\n\nAnd eighth. Do the badge wording rules feel like your voice? \"Support for X rather than Y\". \"Not yet discriminating\" instead of \"weak\".\n\nIn one sentence: evidence is a relation, measured by how much it discriminates between rivals, worth something only in proportion to its independence, and resting finally on located encounters with the world, and Deliberus already has the right skeleton for all of that but still counts where it should weigh.\n\nEverything in this walkthrough is saved in the repository. The synthesis is the research document called \"what counts as evidence\", with a folder of nine supporting reports beside it. The twenty-six proposals are Arc Q in the ruling ledger. The code defects are in the to-do file under the evidence layer. And this text is saved in the listening folder, next to this morning's Arc P walkthrough.\n\nOne last thing, since you bless changes to the always-loaded project file. It got two small edits: the ledger count, now two hundred and forty-seven claims with a hundred and eighty-one open, and one new pointer in its topic index, for what counts as evidence. Everything is committed and pushed.\n"}