{"path":"research/what-counts-as-evidence/formal-and-computational-models-of-evidence.md","content":"# Formal and computational models of evidence — how argumentation theory, knowledge representation, ontologies and machine systems represent and weigh evidence\n\n*Research report for the Deliberus evidence review, 2026-09-26. Written against the briefing\n(`research-briefing.md`), so each section ends by saying what the tradition offers for one of the named gaps.\nVerification status: headline attributions, dates and numbers were checked against publisher pages or\nthe papers' own abstracts where marked; points I could not check in this pass are marked\n**[unverified]**. Nothing here is a ruling; it maps the design space.*\n\n---\n\n## 0. The shape of the field in one page\n\nSeven traditions have each built a formal account of evidence, and they disagree mainly about **what\nthe unit of evidence is** and **what is being added up**:\n\n| Tradition | Unit of evidence | What aggregates | Where \"no evidence\" lives |\n|---|---|---|---|\n| Argumentation (Toulmin, Pollock, ASPIC+, Walton, Carneades) | a premise / an argument | acceptability (in/out) or gradual strength | burden of proof; unanswered critical questions |\n| Bayesian argumentation (Hahn & Oaksford, Zenker) | an observation with a likelihood pair | posterior odds via likelihood ratios | a likelihood ratio of 1 |\n| Belief functions / subjective logic (Dempster–Shafer, Jøsang) | a mass assignment / an opinion (b, d, u, a) | fusion operators (Dempster, cautious, cumulative, averaging) | explicit mass on ignorance, `u` |\n| Bayesian networks (Schum, Fenton–Neil–Lagnado) | a report node E* distinct from the event E | full joint distribution | prior; unobserved nodes |\n| Truth discovery (TruthFinder, Dawid–Skene, Dong) | a (source, object, value) triple | joint estimate of value truth + source reliability + copying | absent triples |\n| Evidence ontologies (ECO, SEPIO, PROV-O, nanopubs, micropubs, discourse graphs) | an evidence *item* typed by method, attached to an assertion | usually nothing — they describe, not compute | a missing annotation |\n| Fact-checking / claim verification (FEVER, SciFact, AVeriTeC, FActScore) | a retrieved sentence / rationale / QA pair | a three- or four-way label | \"Not Enough Info\" |\n\nTwo cross-cutting results matter most for a platform that currently **counts** answers:\n\n1. **Counting versus quality is a formally studied, genuinely incompatible choice** — in gradual\n   semantics, *cardinality precedence* and *quality precedence* cannot both hold (Amgoud & Ben-Naim\n   2013; Bonzon et al. 2016). Any aggregation is a stance on this.\n2. **Every serious computational system that combines evidence states an independence assumption\n   explicitly**, and the ones that work in production either (a) enforce independence by construction\n   (STRING removes a prior and multiplies complements per *channel*; ACMG/ClinGen forbids double-use\n   of the same data across criteria) or (b) estimate dependence from the data (Dong et al.'s copy\n   detection). Nothing combines well while treating source overlap as invisible.\n\n---\n\n## 1. Argumentation-theoretic evidence\n\n### 1.1 Toulmin (1958): data is not backing\n\nToulmin's *The Uses of Argument* (Cambridge UP, 1958) splits an argument into six roles:\n\n- **Claim (C)** — the conclusion.\n- **Data / grounds (D)** — the facts appealed to *for this case* (\"Harry was born in Bermuda\").\n- **Warrant (W)** — the general, inference-licensing step from D to C (\"a man born in Bermuda will\n  generally be a British subject\").\n- **Backing (B)** — what makes the warrant itself credible (\"statutes and legal provisions…\"). Backing\n  is **field-dependent**: statutes in law, taxonomic records in biology, trial results in medicine.\n- **Qualifier (Q)** — the modal force (\"presumably\", \"almost certainly\").\n- **Rebuttal (R)** — the conditions of exception (\"unless both his parents were aliens\").\n\nThe data/backing distinction is the load-bearing one for an evidence model. **Data bears on the\ncase; backing bears on the rule.** A meta-analysis cited to show \"drug X reduces mortality in this\npopulation\" is data. The same meta-analysis cited to show \"RCT-derived effect sizes transfer to\npopulations like this one\" is backing for a warrant. The *same document* can occupy either role, and\nthe role — not the document type — determines what attacking it does: attacking data weakens one\ncase; attacking backing weakens every argument that uses the warrant. Toulmin also noted that\nbacking is often left implicit because it is shared within a field — which is the implicit-crux\nfinding of the Deliberus corpus in its 1958 form.\n\n**Weakness.** Toulmin gives roles, not a calculus; there is no account of how several data combine\nor how a rebuttal quantitatively discounts a qualifier. His \"rebuttal\" also conflates what Pollock\nlater separates (see below): an exception condition is closer to an *undercutter* than to a\ncounter-argument for ¬C.\n\n### 1.2 Pollock: rebutting versus undercutting defeaters\n\nJohn Pollock (\"Defeasible reasoning\", *Cognitive Science* 11, 1987; *Cognitive Carpentry*, MIT Press\n1995) distinguishes two ways a defeasible reason for P can be defeated:\n\n- **Rebutting defeater** — a reason for **¬P** (or for the negation of the conclusion). Two witnesses\n  disagree about whether the car was red.\n- **Undercutting defeater** — a reason to deny that **the premises support the conclusion here**,\n  without giving any reason for ¬P. Pollock's canonical example: \"the object looks red\" is a reason\n  to believe \"the object is red\"; learning that \"the object is illuminated by red light\" undercuts\n  that reason — it tells you nothing about the object's colour, only that the appearance is no longer\n  evidence of it.\n\nThe distinction matters for evidence because **undercutters remove weight without adding counter-\ndirection**. After an undercutter, the right state is *uninformed*, not *probably false*. Systems that\nmodel defeat only as \"attack subtracts strength\" treat every undercutter as a weak rebutter, which\nover-penalises: the red-lit object is pushed toward \"not red\" when it should be pushed back to\n\"unknown\".\n\n### 1.3 ASPIC+: rebut, undercut, undermine, and premise types\n\nASPIC+ (Prakken, \"An abstract framework for argumentation with structured arguments\", *Argument &\nComputation* 1(2), 2010; Modgil & Prakken, \"A general account of argumentation with preferences\",\n*Artificial Intelligence* 195, 2013; Modgil & Prakken tutorial, *Argument & Computation* 5(1), 2014)\ninstantiates Dung's abstract frameworks with structure:\n\n- **Strict rules** (→) and **defeasible rules** (⇒); each defeasible rule r has a name n(r) in the\n  language so that it can be denied.\n- **Knowledge base premise types**: **axioms** Kn (cannot be attacked), **ordinary premises** Kp\n  (attackable; attack succeeds only if not weaker), **assumptions** Ka (attackable; attack *always*\n  succeeds regardless of preference). The 2010 version also had \"issues\", dropped in 2013.\n- **Three attack forms**:\n  - **Rebutting** — attacking the conclusion of a *defeasible* rule application (restricted rebut:\n    conclusions of strict rules on strict/firm grounds cannot be rebutted).\n  - **Undercutting** — deriving ¬n(r): the rule does not apply here. Always succeeds.\n  - **Undermining** — attacking an ordinary premise or assumption.\n- **Preferences** determine which attacks become *defeats*, with **last-link** (compare the last\n  defeasible rules) and **weakest-link** (compare the weakest rule and premise anywhere in the\n  argument) orderings, and elitist/democratic set comparisons.\n\nTwo things transfer directly. (a) ASPIC+'s **weakest-link principle** is the argumentation-theory\ntwin of the min/Fréchet default Deliberus already chose for necessary parts: an argument is only as\nstrong as its weakest defeasible step. (b) The **premise-type split** gives a principled way to say\nwhich claims may be treated as fixed for a given inquiry (axioms, within a context) versus\nopen-but-presumed (ordinary) versus mere assumptions (any challenge wins until defended) — which is\nthe missing \"standard of what is on the table\" the briefing lists.\n\n### 1.4 Walton's evidential schemes and critical questions\n\nWalton, Reed & Macagno, *Argumentation Schemes* (Cambridge UP, 2008) catalogue ~60 schemes; the\nevidence-bearing ones and their characteristic critical questions (CQs) are:\n\n- **Argument from expert opinion** — six CQs: Expertise (how credible is E as an expert?), Field (is\n  E expert in the field of A?), Opinion (what did E assert?), Trustworthiness (is E personally\n  reliable/biased?), Consistency (is A consistent with other experts?), Backup evidence (is E's\n  assertion based on evidence?).\n- **Argument from position to know** — a is in a position to know about domain S; a asserts A; so A.\n  CQs: is a in that position; is a honest; did a assert A.\n- **Argument from witness testimony** — Walton's monograph *Witness Testimony Evidence:\n  Argumentation, Artificial Intelligence, and Law* (Cambridge UP, 2008) treats testimony as a\n  specialisation of position-to-know plus a **truth-telling** premise, with CQs on internal\n  consistency, consistency with known facts and other witnesses, bias, and the plausibility of the\n  statement. The book's central move is that testimony is a *defeasible presumption* whose weight is\n  set dialectically by how the CQs are answered, not by a fixed credibility score, and it connects\n  this explicitly to Wigmore's evidence charts and to AI-and-law systems.\n- **Argument from sign** — a finding B is generally indicative of A; so A. CQs: what is the strength\n  of the correlation; are there other events that would more reliably account for the sign?\n- **Argument from correlation to cause** — CQs: is there really a correlation; is it more than\n  coincidence; could a third factor cause both? (The confounding question is formally the Bayes-net\n  \"common cause\" idiom; see §3.)\n- **Argument from evidence to a hypothesis** (abductive/verification form) — if A then B would be\n  observed; B is observed; so A is more plausible. CQs: could B be explained by something else? Is B\n  observed reliably?\n- **Argument from ignorance / lack of evidence** — if A were true, A would be known (or evidence\n  would have been found); A is not known / no evidence was found; so A is (presumptively) false. The\n  crucial CQ is about **the depth and completeness of the search** (\"epistemic closure\"): how far has\n  the search gone, and would evidence have turned up if it existed?\n\n**How CQs map to Pollock/ASPIC+ attack types.** Verheij (\"Dialectical argumentation with\nargumentation schemes\", *Artificial Intelligence and Law* 11, 2003; venue corrected 2026-09-28) and later Prakken showed that CQs are not\nhomogeneous: some ask whether a **premise** holds (→ undermining), some point to an **exception** that\nmakes the scheme inapplicable (→ undercutting), and some raise a **counter-argument** to the\nconclusion (→ rebutting). A system that renders all CQs as the same kind of polarity claim and\ncounts answers erases this three-way difference. *(Added 2026-09-28: Verheij names four roles, not three. The one this summary dropped is conditions of use, where the scheme applies only if something holds, and it is where relevance questions sit; his example is Walton's ad hominem question \"Is the allegation that a is a bad person relevant to judging a's argument?\" `strength-calculus-literature/computational-argumentation.md` § 2 item 19.)*\n\n### 1.5 Carneades: proof standards, burden of proof, weights, audiences\n\nGordon, Prakken & Walton, \"The Carneades model of argument and burden of proof\", *Artificial\nIntelligence* 171(10–15), 2007, 875–896 (verified); extended in Gordon & Walton, \"Proof burdens and\nstandards\" (in *Argumentation in AI*, Springer 2009) and Gordon & Walton, \"Formalizing balancing\narguments\" (COMMA 2016) [latter unverified in this pass].\n\nKey features:\n\n- **Statements have a status in a stage of dialogue** (stated, questioned, accepted, rejected), and\n  **acceptability depends on a proof standard assigned per statement**, not a global rule.\n- **Proof standards** (2009 formulation, with thresholds α, β, γ): *scintilla of evidence* (at least\n  one applicable pro argument), *preponderance of evidence* (strongest pro weighs more than strongest\n  con), *clear and convincing evidence* (preponderance + pro weight > α + pro exceeds con by > β),\n  *beyond reasonable doubt* (clear and convincing + con weight < γ), and *dialectical validity*.\n  Note that these compare the **strongest** arguments on each side — a quality-precedence choice (§1.9).\n- **Premise kinds carry the burden allocation for CQs.** Carneades splits a scheme's premises into\n  **ordinary premises**, **assumptions** and **exceptions**. An *assumption* holds until questioned —\n  but merely **asking** the CQ shifts the burden back to the proponent. An *exception* does not hold\n  until someone **proves** it — merely asking the CQ does nothing. So the same \"unanswered CQ\" means\n  opposite things depending on its kind. (E.g., for expert opinion, \"is E an expert?\" behaves as an\n  assumption; \"is E biased?\" behaves as an exception.)\n- **Argument weights** and **audiences**: an audience is a set of assumptions plus a weight\n  assignment; the same graph yields different acceptability for different audiences. This is the\n  formal slot for \"worldview lenses\".\n\nCarneades is the closest existing formal model to Deliberus's CQ mechanic, and its central lesson is\nthat **CQs need a typed default**: which ones count against the claim merely by being open, and which\nones count only once someone substantiates them.\n\n### 1.6 Bex's hybrid theory (stories + arguments)\n\nFloris Bex, *Arguments, Stories and Criminal Evidence: A Formal Hybrid Theory* (Springer, Law and\nPhilosophy Library 92, 2011). Evidence is used two ways: **arguments** reason from evidence items to\nconclusions (atomistic); **stories** are causal explanations (scenarios) that explain the evidence\n(holistic, abductive). The hybrid uses arguments to support/attack elements of stories and compares\nstories by **evidential coverage** (how much evidence a story explains), **evidential contradiction**,\nand **story coherence** (fit with story schemes, completeness, consistency, plausibility). A key\nconcept is that a story can be well-supported yet incoherent, or coherent yet explain little — two\nindependent axes.\n\nRelevance: this is the missing account for \"evidence bears on claim-webs\" (Duhem–Quine). A story is a\nbundle of claims evaluated jointly by how much evidence it covers; comparing *rival bundles* by\ncoverage is a different operation from propagating strength edge-by-edge.\n\n### 1.7 Verheij's case models\n\nBart Verheij, \"Proof with and without probabilities: Correct evidential reasoning with presumptive\narguments, coherent hypotheses and degrees of uncertainty\", *Artificial Intelligence and Law* 25(1),\n2017, 127–154 [year/volume from memory, plausible; **unverified** in this pass]; with Di Bello and\nothers on \"arguments, scenarios and probabilities\" (2016–2017). A **case model** is a finite set of\nmutually exclusive, logically consistent cases (full descriptions) with a *total preorder* of\npreference (e.g., by probability). Arguments are evaluated against the case model: an argument from\nP to Q is **coherent** if some case entails P ∧ Q, **presumptively valid** if the most preferred case\nentailing P also entails Q, **conclusive** if all cases entailing P entail Q. It unifies arguments,\nscenarios and (ordinal or numeric) probabilities in one structure and shows that qualitative\ncomparisons suffice for many evidential inferences — **no numbers needed** where only an ordering is\nknown. Useful where people refuse point probabilities but will rank.\n\n### 1.8 Bayesian accounts of argument strength\n\n- **Hahn & Oaksford, \"The rationality of informal argumentation: A Bayesian approach to reasoning\n  fallacies\", *Psychological Review* 114(3), 2007, 704–732.** Classical fallacies are not formally\n  invalid patterns but arguments whose strength is **a function of content via likelihoods**.\n  - *Argument from ignorance*: \"no evidence that drug X is toxic, so it is not toxic\" is strong when\n    the test's sensitivity is high and many tests were run; its strength is the likelihood ratio\n    P(¬e | ¬toxic) / P(¬e | toxic). Negative evidence is typically weaker than positive evidence\n    but not worthless. Participants in their experiments tracked exactly these variables.\n  - *Circularity* (begging the question): self-dependent justification can still be informative in\n    some structures (e.g., \"God exists because the Bible says so, and the Bible is true because it is\n    God's word\" versus benign circular updating in hierarchical models).\n  - *Slippery slope*: strength depends on the conditional probability of each step and on the utility\n    of the endpoint; later work (Corner, Hahn & Oaksford 2011, *J. Memory and Language*) supported\n    the category-boundary account.\n- **Zenker (ed.), *Bayesian Argumentation: The Practical Side of Probability*, Springer Synthese\n  Library 362, 2013.** Collects the programme: argument strength as the probabilistic impact of a\n  reason (Δ in posterior, or the likelihood ratio), convergent/linked support in Bayesian terms,\n  testimony and source reliability (Bovens & Hartmann's *Bayesian Epistemology*, 2003, is the key\n  background: coherence, reliability and independence of witnesses — their result that **coherence\n  of independent partially reliable reports raises confidence, but coherence is not truth-conducive\n  in general** is directly relevant).\n- **Hahn, Harris & Corner (2009, *Argumentation* 23)** — source reliability and argument strength\n  interact multiplicatively; people are sensitive to both.\n\n**Central idea for the platform:** the Bayesian school replaces \"how many supports\" with\n**diagnosticity**: evidence E counts for H in proportion to log[P(E|H)/P(E|¬H)]. Evidence that is\nequally expected under H and ¬H has zero weight *however many times it is cited*.\n\n### 1.9 Probabilistic argumentation (Hunter, Thimm)\n\n- **Thimm, \"A probabilistic semantics for abstract argumentation\", ECAI 2012** — a probability\n  distribution over sets of arguments (the *epistemic* approach), with rationality constraints such\n  as COH (P(A) ≤ 1 − P(B) if A attacks B) and JUS.\n- **Hunter, \"A probabilistic approach to modelling uncertain logical arguments\", *IJAR* 54(1), 2013**;\n  **Hunter & Thimm, \"Probabilistic reasoning with abstract argumentation frameworks\", *JAIR* 59,\n  2017**. Two readings of probabilities: **constellations** (uncertainty about which arguments/attacks\n  *exist*) versus **epistemic** (degree of belief that an argument is acceptable). The epistemic\n  approach allows *partial* specification and consistency-checking of beliefs, with a \"neutral\" 0.5\n  that means *no commitment*, and it studies what constraints are rational when beliefs are\n  incomplete.\n\nRelevance: the epistemic approach supplies what the badge redesign already reaches for — an explicit\ndistinction between 0.5-as-balanced and 0.5-as-unknown. In Hunter–Thimm, the rationality postulates\ntell you which combinations of beliefs are coherent without forcing a number where none is known.\n\n### 1.10 Evidential and bipolar frameworks\n\n- **Oren & Norman, \"Semantics for evidence-based argumentation\", COMMA 2008, IOS Press, 276–284**\n  (verified). The **Evidential Argumentation System (EAS)**: arguments cannot be accepted unless they\n  are supported, ultimately, by a special **evidence** node η (the \"environment\"). Support is set-\n  based (a set of arguments supports one). Extensions are *e-preferred, e-stable, e-grounded*. Refined\n  by Polberg & Oren (\"Revisiting support in abstract argumentation systems\", COMMA 2014). Oren & Norman\n  also wrote \"Subjective logic and arguing with evidence\" (*Artificial Intelligence* 171, 2007),\n  which grounds argument strength in subjective-logic opinions — an early bridge between §1 and §2.\n  **Design point:** EAS formalises *foundationalism* — every accepted argument must trace a support\n  chain to evidence. That is the opposite of \"no atomic evidence\", and it is the opposite on purpose:\n  in EAS, what counts as evidence is an explicit boundary choice. Deliberus can read η as \"the empirical\n  horizon kind\": the place where a chain hands over to observation.\n- **Quantitative bipolar argumentation (QBAF) and gradual semantics.** Baroni, Rago & Toni, \"From\n  fine-grained properties to broad principles for gradual argumentation: A principled spectrum\",\n  *IJAR* 105, 2019: two broad principles, **balance** and **monotonicity**. Potyka, \"Continuous\n  dynamical systems for weighted bipolar argumentation\", KR 2018 (the Quadratic Energy Model, QEM):\n  energy E = Σ strengths(supporters) − Σ strengths(attackers); influence h(E) = max(E,0)²/(1+max(E,0)²).\n  Mossakowski & Neuhaus, \"Modular semantics and characteristics for bipolar weighted argumentation\n  graphs\" (arXiv:1807.06685, 2018) factor semantics into an **aggregation function** (sum, product,\n  **top/max**) and an **influence function** (linear, Euler-based, MLP-based, quadratic energy) — so\n  \"counting versus quality\" becomes a choice of aggregation module.\n- **ArgRAG** — Zhu, Potyka, Hernández et al., \"ArgRAG: Explainable Retrieval Augmented Generation\n  using Quantitative Bipolar Argumentation\", NeSy 2025, PMLR 284 (arXiv:2508.20131; verified). Builds a\n  QBAF from retrieved documents (support/attack between claims and retrieved passages), computes\n  strength under gradual semantics deterministically, and evaluates on PubHealth and RAGuard fact\n  verification. **Retrieved evidence is modelled as arguments** — i.e., \"evidence is claims\" — and the\n  contestability claim is that a user can inspect and challenge a specific passage's support.\n  *(Corrected 2026-09-28: an earlier caveat here said their base scores come from a model's\n  relevance and stance confidence. Their best configuration initialises base scores with no prior;\n  retriever scores were tested and did worse, which the authors read as retriever confidence not\n  matching relevance or trustworthiness. `strength-calculus-literature/computational-argumentation.md` § 2 item 33.)*\n\n### 1.11 Gradual-semantics axioms that bear on counting\n\nThis is the section that most directly concerns \"answers to CQs are COUNTED not weighed\".\n\n- **Amgoud & Ben-Naim, \"Ranking-based semantics for argumentation frameworks\", SUM 2013**, introduced\n  properties including:\n  - **Cardinality Precedence (CP)**: an argument with *fewer* attackers ranks higher.\n  - **Quality Precedence (QP)**: if a has an attacker stronger than all of b's attackers, b ranks\n    higher than a — one strong objection outweighs any number of weak ones.\n  - **Compensation**: several weak attacks can compensate for one strong one.\n  They show **CP and QP are incompatible** (no semantics satisfies both in general); Compensation sits\n  between them.\n- **Bonzon, Delobelle, Konieczny & Maudet, \"A comparative study of ranking-based semantics for\n  abstract argumentation\", AAAI 2016** — tabulate which semantics satisfy which properties\n  (categoriser, discussion-based, burden-based, counting, tuples…). The h-categoriser, for example,\n  satisfies neither CP nor QP but *Compensation*; burden-based semantics lean cardinality-first.\n- **Amgoud & Ben-Naim, \"Weighted bipolar argumentation graphs: Axioms and semantics\", IJCAI 2018**\n  (verified) — extend QP, CP and Compensation to supporters as well as attackers.\n- **Amgoud et al. on similarity-adjusted semantics (KR 2018)** — already in the Deliberus corpus:\n  attackers/supporters that are similar to each other should not count twice.\n\n**What this means concretely.** QEM's sum aggregation makes the platform **cardinality-sensitive and\ncompensating**: six mediocre inputs equal three good ones, and three one-word answers from one account\nequal three RCTs. The literature does not say one principle is right; it says **choose, and state the\nchoice**. Evidence practice in law and science leans toward **quality precedence with a cardinality\ntie-break** (Carneades compares strongest arguments; GRADE rates the *body* of evidence by its best\ndesign, then downgrades; ACMG caps same-data reuse). A *top* aggregator (Mossakowski–Neuhaus) or a\n**saturating, diminishing-returns** aggregator for corroboration (e.g., Jøsang's cumulative fusion,\n§2.3, where the n-th confirming observation moves uncertainty less and less) are the standard\nalternatives.\n\n---\n\n## 2. Belief functions and imprecise probability\n\n### 2.1 Dempster–Shafer theory\n\nDempster (\"Upper and lower probabilities induced by a multivalued mapping\", *Ann. Math. Stat.* 38,\n1967); Shafer (*A Mathematical Theory of Evidence*, Princeton UP, 1976). On a frame Θ, a **mass\nfunction** m assigns belief to *subsets*, including Θ itself: m(Θ) is mass explicitly committed to\n**ignorance**. **Belief** Bel(A) = Σ_{B⊆A} m(B) and **plausibility** Pl(A) = 1 − Bel(¬A); the interval\n[Bel, Pl] separates *support for A* from *lack of evidence against A*.\n\n**Dempster's rule** combines two independent sources by intersecting focal sets and renormalising by\n1 − K, where K is the mass on conflicting (empty) intersections.\n\n**Zadeh's counterexample** (Zadeh, review of Shafer, *AI Magazine* 5(3), 1984; and \"A simple view of the\nDempster–Shafer theory of evidence…\", *AI Magazine* 7(2), 1986): Doctor 1: meningitis 0.99, tumour\n0.01. Doctor 2: concussion 0.99, tumour 0.01. Dempster's rule yields **tumour = 1.0** — a near-total\nconflict (K = 0.9999) is renormalised away and the one hypothesis both considered nearly impossible\nbecomes certain. Lessons drawn: (a) high K should be *reported*, not normalised away; Smets's\n**Transferable Belief Model** keeps conflict as m(∅); Yager's rule assigns conflict to Θ (ignorance);\n(b) Dempster's rule presupposes **distinct (independent) bodies of evidence**.\n\n### 2.2 Denœux's cautious rule for non-distinct evidence\n\nDenœux, \"Conjunctive and disjunctive combination of belief functions induced by nondistinct bodies of\nevidence\", *Artificial Intelligence* 172(2–3), 2008, 234–264. The **cautious rule** is **idempotent**:\ncombining a source with itself returns the source. It is the least-committed combination compatible\nwith both inputs (by the weight-of-evidence decomposition: take the *minimum* of the canonical\nweights). Denœux also defines a family of t-norm-based rules interpolating between cautious (min,\nidempotent, for fully overlapping evidence) and Dempster's rule (product, for fully distinct\nevidence). This is exactly the dial the Deliberus weakest-link doc already found: **min is correct\nwhen inputs may share origin, product/sum when they are known distinct.**\n\n### 2.3 Jøsang's subjective logic\n\nJøsang, \"A logic for uncertain probabilities\", *IJUFKS* 9(3), 2001; *Subjective Logic: A Formalism for\nReasoning Under Uncertainty*, Springer 2016.\n\n- A binomial **opinion** about x held by A: ω = (b, d, u, a) with b + d + u = 1; b belief, d disbelief,\n  u uncertainty (uncommitted mass), a **base rate** (prior probability absent evidence). **Projected\n  probability** P = b + a·u.\n- **Evidence mapping** (via the Beta distribution): with r positive and s negative observations and\n  non-informative prior weight W (default 2): b = r/(r+s+W), d = s/(r+s+W), u = W/(r+s+W). So u falls\n  as evidence accumulates, with **diminishing returns**: 1 observation → u = 0.67; 8 → u = 0.2; 98 → 0.02.\n- **Fusion operators** (named to match dependence assumptions):\n  - **Cumulative (consensus) fusion** — for *independent* sources: evidence counts add (r_A + r_B),\n    so uncertainty shrinks.\n  - **Averaging fusion** — for *dependent* sources (e.g., the same underlying evidence seen by two\n    observers): opinions average, uncertainty does **not** shrink from the mere second report.\n  - **Weighted belief fusion** and **consensus & compromise fusion** — variants for other situations\n    (Jøsang 2016 ch. 12; van der Heijden, Kopp & Kargl, \"Multi-source fusion operations in subjective\n    logic\", FUSION 2018).\n- **Trust discounting** — A's derived opinion about x via advisor B is B's opinion about x discounted\n  by A's trust in B: in the uncertainty-favouring form, b_x^{A:B} = P(trust_B)·b_x^B, d likewise, and\n  the remainder goes to uncertainty. Chains of sources multiply discounts, so **a long hearsay chain\n  converges to uncertainty, not to disbelief** — exactly the Pollock-undercutter behaviour (§1.2).\n\n**Why it fits the Deliberus badge redesign.** The August 2026 badge already split **direction** from\n**weight** after Keynes. Subjective logic is a worked, published algebra for exactly that split, with\nthree extras the badge lacks: (1) a **base rate** a, which says what \"no evidence\" should default to\nper claim kind (a base rate for \"a randomly chosen published social-science effect replicates\" is not\n0.5); (2) **saturation** of weight by construction; (3) **named fusion operators keyed to an\nindependence judgement** — so the system must record whether two inputs are independent to know which\noperator applies.\n\n### 2.4 Imprecise probability and credal sets\n\nWalley, *Statistical Reasoning with Imprecise Probabilities* (Chapman & Hall, 1991): beliefs as\n**lower and upper previsions**, equivalently a closed convex **credal set** of probability\ndistributions. The width of the interval represents **the amount of evidence** (Keynesian weight),\nthe location represents direction. Walley's **imprecise Dirichlet model** (JRSS B 58, 1996) gives, for\nn observations with s a hyperparameter (often 1 or 2), interval widths shrinking like s/(n+s) — the\nsame diminishing-weight behaviour as subjective logic. Key philosophical claim relevant to the founder's\nopen position (\"given the same evidence, probability should not differ between people\"): Walley's\nanswer is that **with little evidence the rational state is an interval, not a point**; people who\npick different points inside the interval are not irrational, and the interval narrows as evidence\naccumulates. Imprecision is the honest representation of *not yet enough evidence to agree*.\n\nCritiques: dilation (an interval can *widen* on learning, Seidenfeld & Wasserman 1993), and decision\nrules under imprecision are not unique (E-admissibility, maximality, Γ-maximin).\n\n### 2.5 Game-theoretic probability and e-values\n\nShafer & Vovk, *Probability and Finance: It's Only a Game!* (Wiley, 2001) and *Game-Theoretic\nFoundations for Probability and Finance* (Wiley, 2019). Probability is defined by betting: a hypothesis\nis discredited to the degree that a bettor, betting at the hypothesis's odds, multiplies their\ncapital. The modern offshoot is **e-values / safe testing** (Vovk & Wang, *Annals of Statistics* 49,\n2021; Grünwald, de Heide & Koolen, \"Safe testing\", *JRSS B* 2024 [journal/year from memory;\n**unverified**]). E-values **multiply across independent studies** and remain valid under **optional\ncontinuation** (deciding to run another study because the first was promising) — the property that\np-values and naive meta-analysis lack. Shafer, \"Testing by betting\" (*JRSS A* 184, 2021) argues that the\nbetting score is the natural *communicable* measure of evidence. Relevance: an accumulating open\nplatform is exactly an optional-continuation setting — people add evidence *because* of what they see.\n\n---\n\n## 3. Bayesian networks for evidence\n\n### 3.1 Schum: the report is not the event\n\nDavid Schum, *The Evidential Foundations of Probabilistic Reasoning* (Wiley, 1994; Northwestern UP\n2001), and Kadane & Schum, *A Probabilistic Analysis of the Sacco and Vanzetti Evidence* (Wiley, 1996).\nSchum's first principle: **evidence of an event E (the report E\\*) is not the event E.** Between E\\*\nand E sits a chain of credibility attributes; for testimony, **veracity** (does the witness believe\nwhat they report?), **objectivity** (did they form the belief from the sensory evidence rather than\nexpectation?), and **observational sensitivity** (were their senses/instruments capable of the\nobservation?). Schum also catalogues *substance-blind* evidence relations: **corroborative redundancy**\n(two reports of the same event), **cumulative redundancy**, **synergy** (two items together worth more\nthan apart), **dissonance** (contradiction or conflict), and shows that the **force** of evidence is\nthe likelihood ratio, which composes through the chain.\n\nThis is precisely Bengio's \"communication act vs factual observation\" split and Deliberus's always-mint\nREPORTS edge, stated in 1994 with a probability calculus attached.\n\n### 3.2 Fenton, Neil & Lagnado's evidence idioms\n\nFenton, Neil & Lagnado, \"A general structure for legal arguments about evidence using Bayesian\nnetworks\", *Cognitive Science* 37(1), 2013, 61–102 (verified); Lagnado, Fenton & Neil, \"Legal idioms: a\nframework for evidential reasoning\", *Argument & Computation* 4(1), 2013, 46–63. Idioms are reusable\nBN fragments:\n\n- **Evidence idiom** — hypothesis → evidence.\n- **Evidence-accuracy idiom** — the evidence node has a second parent, **accuracy** (reliability) of\n  the source/test, so an observation is interpreted through a reliability variable that can itself\n  be attacked (Schum's chain in two nodes).\n- **Cause–consequence idiom** — causal structure between hypotheses.\n- **Motive** and **opportunity idioms** — prior-shaping parents of the hypothesis.\n- **Dependency idiom** — two evidence nodes share a dependency parent (e.g., two witnesses who\n  talked, two tests from one lab), so the second is less informative given the first.\n- **Alibi idiom** — evidence that directly contradicts the prosecution hypothesis, with its own\n  accuracy node (an alibi from a relative has a reliability that depends on the relation).\n- **Explaining-away idiom** — two causes of one effect (a collider); learning one cause reduces the\n  probability of the other.\n- **Mediation** is not, to my knowledge, one of the named idioms in the 2013 papers **[unverified;\n  the task brief listed it — it may appear in later Fenton–Neil work]**.\n\nThey also show BNs guard against classic fallacies (prosecutor's fallacy, ignoring dependency between\nitems) and later develop **Bayesian model comparison** for competing legal arguments (Neil, Fenton,\nLagnado & Gill, *AI and Law* 27, 2019). Fenton et al.'s analyses of the Simonshaven case (Fenton, Neil,\nYet & Lagnado, *Topics in Cognitive Science*, 2020) sit in a comparison special issue with argument\n(Bex/Verheij/Prakken) and scenario analyses of the same case — a rare head-to-head of the traditions.\n\n**Explaining away** matters structurally: it is a **non-monotonic interaction between evidence items\nthat no edge-local aggregation can represent**. If a symptom has two possible causes, evidence for one\ncause is (indirectly) evidence *against* the other. A QBAF with independent edges into each cause will\nnever produce that.\n\n**Weaknesses.** BNs need a full conditional probability table; elicitation is expensive and the numbers\nare often invented; \"the numbers\" become a rhetorical authority. Hence the argument/scenario/BN hybrid\nresearch programme (Verheij et al., \"Arguments, scenarios and probabilities: connections between three\nnormative frameworks for evidential reasoning\", *Law, Probability and Risk* 15, 2016).\n\n---\n\n## 4. Truth discovery and source reliability from conflicting data\n\n### 4.1 Dawid–Skene (1979)\n\nDawid & Skene, \"Maximum likelihood estimation of observer error-rates using the EM algorithm\",\n*JRSS C (Applied Statistics)* 28(1), 1979, 20–28. Each annotator j has a **confusion matrix** π_j\n(probability of reporting label l when the truth is k); the true labels are latent; EM alternates\nbetween estimating truths given annotator accuracies and accuracies given truths. It is the ancestor of\ncrowdsourced label aggregation (Raykar et al., \"Learning from crowds\", *JMLR* 11, 2010; Whitehill et\nal., GLAD, NeurIPS 2009, adding item difficulty). Core insight: **majority vote is optimal only when\nall voters are equally reliable and independent**; otherwise reliability should be *estimated from\nagreement patterns*, and a reliable minority beats an unreliable majority.\n\n### 4.2 TruthFinder\n\nYin, Han & Yu, \"Truth discovery with multiple conflicting information providers on the web\", KDD 2007;\n*IEEE TKDE* 20(6), 2008. Iterative: a **fact's confidence** depends on the trustworthiness of the\nwebsites asserting it; a **website's trustworthiness** is the average confidence of its facts. Adds\n**implication between facts** (similar values support each other; \"Jennifer Widom\" supports \"J. Widom\").\nSurvey: Li et al., \"A survey on truth discovery\", *SIGKDD Explorations* 17(2), 2016.\n\n### 4.3 Copy detection: source dependence\n\nDong, Berti-Équille & Srivastava, \"Integrating conflicting data: the role of source dependence\",\n*PVLDB* 2(1), 2009 (verified), plus \"Truth discovery and copying detection in a dynamic world\" (same\nvolume). Key ideas:\n\n- If two sources share **many false values**, that is strong evidence one copies the other — because\n  independent sources rarely make the *same* mistake. Shared true values are weak evidence of copying\n  (independent accurate sources agree anyway). This is a **Bayesian test of dependence using the\n  asymmetry between shared truths and shared errors**.\n- Once copying is detected, **copied votes are discounted**, so a popular falsehood echoed by copiers\n  does not beat a true value asserted by fewer independent sources.\n- Iteration: detect dependence ↔ estimate truth ↔ estimate accuracy.\n\nLi, Dong et al., \"Truth finding on the deep web: Is the problem solved?\", *PVLDB* 6(2), 2012, examined\nreal domains (stocks, flights) and found substantial copying and inconsistency even among \"reputable\"\nsources, with fusion methods that account for copying performing best **[specific accuracy figures\nunverified]**. Pochampally, Das Sarma, Dong, Meliou & Srivastava, \"Fusing data with correlations\",\nSIGMOD 2014, generalise from copying to arbitrary positive/negative correlations between sources.\n\n### 4.4 Knowledge-Based Trust (Google, 2015)\n\nDong, Gabrilovich, Murphy et al., \"Knowledge-Based Trust: Estimating the trustworthiness of web\nsources\", *PVLDB* 8(9), 2015 (verified). Trust estimated from **endogenous** signals (the correctness of\nfacts a page states) rather than link structure. The crucial modelling move is a **multi-layer model\nthat separates extraction errors from source errors**: a triple can be wrong because the page is wrong\nor because the extractor misread the page. Applied to 2.8B extracted triples, estimating trust for\n119M pages. Direct analogue: in Deliberus, a claim can be wrong because the source is wrong or because\nthe *extraction/paraphrase* mis-rendered the source — and these need separate variables.\n\n### 4.5 Slot-filling validation\n\nYu et al., \"The wisdom of minority: Unsupervised slot filling validation based on multi-dimensional\ntruth-finding\", COLING 2014 [**unverified details**]: in the TAC-KBP slot-filling task, validating\nanswers by modelling system reliability, source credibility and linguistic evidence jointly; its title\nlesson is that a correct answer is often held by a **minority** of systems, so majority voting fails.\n\n### 4.6 Lessons about independence and copying\n\n1. **Agreement is only evidence to the degree the agreers are independent.** Every aggregation that\n   counts agreement needs a dependence model.\n2. **Shared errors reveal dependence better than shared truths.** A detector for Deliberus: two\n   sources/contributors whose claims share *attacked-and-lost* positions or distinctive phrasing are\n   candidate copies.\n3. **Reliability should be learned from track record, not assigned** — but track record needs\n   ground truth, which only exists on the empirical leg (the oracle gradient again).\n4. **Separate the rendering error from the source error** (KBT).\n5. **Estimated reliability is attackable by strategy-class adversaries** (sybils that agree with truth\n   on easy items to build reliability, then lie on the contested one) — a known weakness of all\n   agreement-based truth discovery, noted across the crowdsourcing literature.\n\n---\n\n## 5. Evidence ontologies and scholarly-knowledge representations\n\nBe precise about what each models as **evidence item**, **claim** and **provenance** — they differ.\n\n### 5.1 ECO — Evidence & Conclusion Ontology\n\nChibucos et al., \"Standardized description of scientific evidence using the Evidence Ontology (ECO)\",\n*Database* 2014; Giglio et al., *NAR* 47(D1), 2019; Nadendla et al., \"ECO: the Evidence and Conclusion\nOntology, an update for 2022\", *NAR* 50(D1), D1515–D1521 (verified). **Two root classes: `evidence`\nand `assertion method`.** Evidence classes are *types of method that produced support*: experimental\n(e.g., \"mutant phenotype evidence\"), computational (e.g., \"sequence similarity evidence\"), **author\nstatement**, **curator inference**. Assertion method is *manual* vs *automatic* assertion; leaf terms\nare **cross-products** (e.g., \"sequence similarity evidence used in automatic assertion\"). Maps to GO\nevidence codes. **What ECO models:** the *kind* of evidence behind an annotation (the annotation is the\nclaim), not the evidence content. **Crucially ECO separates \"what kind of evidence\" from \"was a human\nor a machine the one who asserted it\"** — two axes Deliberus currently fuses in `evidence_type` and\nprovenance.\n\n### 5.2 GO evidence codes\n\nGene Ontology evidence codes: experimental (EXP, IDA, IPI, IMP, IGI, IEP, plus high-throughput HTP,\nHDA, HMP, HGI, HEP), phylogenetic (IBA, IBD, IKR, IRD), computational (ISS, ISO, ISA, ISM, IGC, RCA),\nauthor statements (TAS, NAS), curator (IC, ND), and electronic (IEA — no curator review). **GO's own\nguidance states evidence codes describe the type of work, not the strength or quality of an\nannotation** [paraphrase of GO documentation; exact wording **unverified**], yet in practice many users\nfilter out IEA and rank experimental above computational — a live example of a descriptive taxonomy\nbeing used as a de-facto quality ladder against its designers' advice. The same fate awaits any\nDeliberus \"methodology ladder\": if a type is displayed, users will treat it as a weight.\n\n### 5.3 SEPIO — Scientific Evidence and Provenance Information Ontology\n\nBrush, Shefchek, Haendel et al., \"SEPIO: A semantic model for the integration and analysis of\nscientific evidence\", ICBO 2016 (CEUR); ontology at OBO Foundry / Monarch Initiative (verified). Core\npattern: **Assertion** ← *has evidence line* ← **Evidence Line** ← *has evidence item* ← **Evidence\nItem (information content entity)**; plus **provenance**: the *process* (method, tool, agent) that\ncreated each assertion, evidence line and item. An **evidence line is a set of items that together\nconstitute one argument for/against the assertion**, and it carries its own **direction** (supports /\ndisputes) and **strength**. This is the ClinGen data model's backbone (the ClinGen Interpretation\nModel uses SEPIO: a variant pathogenicity interpretation is supported by evidence lines per ACMG\ncriterion, each with items and a strength). **Key distinction**: SEPIO says an item is *not*\nintrinsically supporting — it becomes evidence only as part of a line aimed at an assertion; the same\nitem (a population frequency) can be in a supporting line for one assertion and a refuting line for\nanother. That is \"evidence is a position in the graph\" — already Deliberus's view — but with an\nintermediate node (the line) that bundles items and carries a strength *as a unit*.\n\n### 5.4 W3C PROV-O\n\nLebo, Sahoo, McGuinness et al., *PROV-O: The PROV Ontology*, W3C Recommendation, 30 April 2013. Core:\n**Entity, Activity, Agent**; relations *wasGeneratedBy, used, wasDerivedFrom, wasAttributedTo,\nwasAssociatedWith, actedOnBehalfOf*, plus *wasQuotedFrom, hadPrimarySource, wasRevisionOf*. PROV is\n**not an evidence model** — it records derivation and responsibility — but it provides the\n**chain-of-custody vocabulary** the briefing lists as missing: `prov:wasQuotedFrom` (quote),\n`prov:hadPrimarySource`, `prov:wasDerivedFrom` (paraphrase/extraction), qualified with the Activity\nthat did it and when.\n\n### 5.5 Nanopublications\n\nGroth, Gibson & Velterop, \"The anatomy of a nanopublication\", *Information Services & Use* 30(1–2),\n2010; Kuhn et al., \"Decentralized provenance-aware publishing with nanopublications\", *PeerJ Computer\nScience* 2:e78, 2016; Kuhn & Dumontier, \"Trusty URIs\", ESWC 2014. A nanopublication is four named RDF\ngraphs: **head**, **assertion** (the minimal claim, as triples), **provenance** (how the assertion came\nabout: method, evidence, source — the provenance *of the assertion*), and **publication info**\n(who made and published *this nanopub*, when, licence — the provenance *of the record*). Trusty URIs\nmake each nanopub **immutable and verifiable by hash**; corrections are new nanopubs that supersede.\nKnowledge Pixels' **Nanodash** provides templated authoring. **Distinction to note:** nanopubs split\n*assertion provenance* from *publication provenance* — \"Smith et al. 2019 found X via RCT\" versus\n\"this record was created by extractor v3 on 2026-09-26\". Deliberus currently has one provenance slot.\n\n### 5.6 Micropublications\n\nClark, Ciccarese & Goble, \"Micropublications: a semantic model for claims, evidence, arguments and\nannotations in biomedical communications\", *Journal of Biomedical Semantics* 5:28, 2014. A\nmicropublication has a principal **Claim** with **Attribution**; **Statements**, **Data**, **Methods**,\n**Materials** as supporting elements (grouped as *Representations*); relations **supports** and\n**challenges** forming a **support graph** and a **challenge graph**; the *minimal* micropublication is\njust a statement plus its attribution, the maximal is a full argument with data and methods. It is\nexplicitly aligned to Toulmin and to the SWAN/AIF lineage, and it treats **methods and materials as\nfirst-class evidence** (not just statements) [structure as I recall it; exact class names\n**partly unverified**]. It is the scholarly-communication design closest to Deliberus: claims all the\nway down, but with Data and Method as distinct node kinds that can be supported/challenged.\n\n### 5.7 CiTO — Citation Typing Ontology\n\nShotton, \"CiTO, the Citation Typing Ontology\", *J Biomed Semantics* 1(Suppl 1):S6, 2010; Peroni &\nShotton, \"FaBiO and CiTO\", *J Web Semantics* 17, 2012. Typed citation relations, e.g. *cites as\nevidence, cites as data source, uses method in, supports, confirms, agrees with, obtains support\nfrom, disputes, refutes, disagrees with, critiques, qualifies, corrects, extends, discusses*.\nDistinguishes **factual** citation functions from **rhetorical** ones. Lesson: *citation ≠ support*.\nA reference can be data source, method source, or target of refutation.\n\n### 5.8 scite.ai smart citations\n\nNicholson et al., \"scite: A smart citation index that displays the context of citations and\nclassifies their intent using deep learning\", *Quantitative Science Studies* 2(3), 2021, 882–898\n(verified). Citation statements (the sentence(s) around an in-text citation) are classified as\n**supporting, contrasting, or mentioning**, from 25M+ full-text articles (880M+ classified statements\nat publication). Empirically the large majority are *mentioning*, with contrasting citations a small\nminority [reported ratio ~ tens-to-one **unverified exact numbers**]. Lesson: in the wild, **most\nreferences are not evidential at all**; a platform that counts cited sources as support will be\ncounting mostly mentions.\n\n### 5.9 ORKG\n\nJaradeh et al., \"Open Research Knowledge Graph: Next generation infrastructure for semantic scholarly\nknowledge\", K-CAP 2019; Auer et al. 2020. Papers become **contributions** (research problem, method,\nmaterials, results as property–value pairs), and **comparisons** tabulate contributions across papers\nalong shared properties. ORKG models evidence as **comparable structured results**, not as support/\nattack — the right structure for meta-analytic synthesis (heterogeneity, sample sizes are columns),\nand the wrong one for dispute.\n\n### 5.10 AIF — Argument Interchange Format\n\nChesñevar et al., \"Towards an argument interchange format\", *Knowledge Engineering Review* 21(4),\n2006; Rahwan, Zablith & Reed, \"Laying the foundations for a World Wide Argument Web\", *AIJ* 171, 2007;\nAIF+ (Reed et al. 2008, adding dialogue: L-nodes, YA illocutionary, TA transitions). **I-nodes**\n(information/claims) and **S-nodes** (scheme applications): **RA** (inference), **CA** (conflict),\n**PA** (preference), with **Form nodes** naming schemes (e.g., expert opinion). **Evidence has no\nspecial node kind**: it is an I-node at the premise end of an RA-node. That is exactly \"evidence is\nclaims\". AIF+'s L-node / YA split is the formal version of reported speech (locution vs proposition).\n\n### 5.11 Discourse graphs\n\nChan et al.; notably Chan, \"Steps towards an infrastructure for scholarly synthesis\", arXiv:2407.20666\n(2024; verified); the Discourse Graph protocol (Roam/Obsidian plugins; Protocol Labs discussions).\nBase node types: **Question (QUE), Claim (CLM), Evidence (EVD)**, and often **Source**; relations\n*informs, supports, opposes*. **Evidence is a distinct node type from claim**: an EVD node is a\n*specific, bounded empirical result reported in a specific source*, typically written as a\npast-tense, context-bearing statement (\"In Study X, participants who… showed Y (n=…)\"), whereas a CLM\nis a general, present-tense, reusable assertion. Claims are supported by evidence, and evidence always\npoints to exactly one source.\n\n**Does \"evidence is claims\" lose something?** The discourse-graph choice encodes three things\nDeliberus's single Claim type does not:\n\n1. **Scope and tense.** An evidence node is *particular* (this study, this sample, this time); a claim\n   is *general*. The inferential step from \"in study X, Y\" to \"Y in general\" is exactly where\n   external validity, sample and heterogeneity bite — and in a one-type graph it disappears into a\n   SUPPORTS edge.\n2. **Source binding.** An evidence node has exactly one source; a claim may have many. So evidence\n   nodes are the natural place to hang locators (page/timestamp), independence, and quote checks.\n3. **Different defeaters.** Evidence is attacked by method critique (confounding, measurement) —\n   undermining; claims are attacked by counter-evidence and counter-argument.\n\nNone of this requires a second node type — Deliberus could keep one Claim type and add a **`scope:\nparticular | general`** field (or treat \"reports a specific finding from one source\" as a claim kind),\nplus a *generalises* edge carrying external-validity CQs. But the three distinctions must live\nsomewhere, and today they do not.\n\n---\n\n## 6. Automated fact-checking and claim verification\n\n### 6.1 FEVER\n\nThorne, Vlachos, Christodoulopoulos & Mittal, \"FEVER: a large-scale dataset for Fact Extraction and\nVERification\", NAACL 2018. 185,445 claims written by mutating Wikipedia sentences; labels\n**SUPPORTED / REFUTED / NOTENOUGHINFO**; evidence = sets of Wikipedia sentences; the **FEVER score**\ncredits a label only when the retrieved evidence contains a complete gold evidence set. Weaknesses:\nsynthetic claims enable **claim-only artefacts** (Schuster et al., \"Towards debiasing fact verification\nmodels\", EMNLP 2019 — claim-only models far above chance **[exact accuracy unverified]**), and NEI has\nno gold evidence — it is *defined by the absence of retrieved support*, conflating \"the knowledge\nsource is silent\" with \"the searcher didn't look\".\n\n### 6.2 SciFact\n\nWadden et al., \"Fact or fiction: Verifying scientific claims\", EMNLP 2020. 1,409 expert-written claims\nagainst 5,183 abstracts; labels **SUPPORTS / REFUTES / NOINFO** plus **rationales** (minimal sentence\nsets sufficient to justify the label). Rationales are the unit of evidence: sentence sets, not\ndocuments. Later SciFact-Open (Wadden et al., Findings EMNLP 2022) showed performance drops sharply in\nan open corpus.\n\n### 6.3 AVeriTeC\n\nSchlichtkrull, Guo & Vlachos, \"AVeriTeC: A dataset for real-world claim verification with evidence\nfrom the web\", NeurIPS 2023 Datasets & Benchmarks (verified); shared task, FEVER workshop 2024. 4,568\nreal claims from 50 fact-checking organisations. Each claim has: **four verdicts — Supported, Refuted,\nNot Enough Evidence, Conflicting Evidence/Cherry-picking**; **question–answer pairs** grounded in web\nevidence (the evidence is decomposed into the questions a fact-checker asked, with answer types\nextractive/abstractive/boolean/unanswerable); a textual justification; and claim metadata (speaker,\ndate, location). Evidence retrieval is **temporally restricted** to before the claim date to prevent\nleakage. Scoring requires evidence quality above a threshold before label credit.\n\nTwo lessons. (a) **QA decomposition = critical questions.** AVeriTeC's evidence is literally a set of\nquestions with answers — the fact-checking community converged on the CQ shape independently.\n(b) **\"Conflicting Evidence/Cherry-picking\" is a real fourth state** and the hardest: models detect it\nat ~10% precision even with gold evidence (per the shared task findings); naive stance aggregation\n(\"any refuting evidence → conflicting\") massively over-fires, because context-adding answers get\nmisread as refutations.\n\n### 6.4 Claim decomposition for verification\n\n- **FActScore** — Min et al., EMNLP 2023: split a generation into **atomic facts**, verify each against\n  a knowledge source (Wikipedia), report the % supported (ChatGPT biographies ≈ 58% **[figure from\n  memory]**).\n- **SAFE** — Wei et al., \"Long-form factuality in large language models\" (Google DeepMind, 2024):\n  atomic facts checked via multi-step Google search; agreed with crowd annotators ~72% and was judged\n  correct in ~76% of disagreements **[figures from memory]**.\n- **VeriScore** — Song, Kim & Iyyer, Findings EMNLP 2024: extract only **verifiable** claims (drop\n  opinions, hedges), then verify against search.\n- **DnDScore** — Wanner, Van Durme & Dredze, arXiv:2412.13175, EMNLP 2025 (verified): decomposition\n  isolates atomic facts while decontextualization re-inserts context, so a decontextualized subclaim\n  contains *several* facts; DnDScore verifies the atomic subclaim **in light of** its context, and\n  shows the choice of strategy changes factuality scores materially.\n- **Hu et al. (NAACL 2025) and arXiv:2602.10380** (already in the Deliberus corpus): decomposition helps\n  verification only when evidence is **aligned per subclaim** and hurts when claim-level evidence is\n  repeated across children.\n- **CORE (ACL 2025, cited in the corpus)**: decompose-then-verify scores can be gamed by padding with\n  trivially true or repetitive subclaims.\n\n### 6.5 Attribution: AIS and ALCE\n\n- **AIS** — Rashkin et al., \"Measuring attribution in natural language generation models\", arXiv\n  2112.12870 (2021); *Computational Linguistics* 49(4), 2023. A statement s is **Attributable to\n  Identified Sources** if a generic hearer would affirm \"**According to P, s**\". Two-stage human\n  protocol: first *interpretability* (is s understandable on its own?), then *attributability*. This is\n  exactly the quote-verification test Deliberus lacks for REPORTS edges and source spans.\n- **ALCE** — Gao, Yen, Yu & Chen, \"Enabling large language models to generate text with citations\",\n  EMNLP 2023: **citation recall** (is each statement fully supported by its cited passages?) and\n  **citation precision** (is each citation necessary/relevant?), judged by an NLI model (TRUE).\n  Automated AIS at scale, with the known caveat that NLI judges have their own failure modes.\n\n### 6.6 LLM-as-judge reliability\n\nDocumented issues: position and verbosity bias (Zheng et al., \"Judging LLM-as-a-judge with MT-Bench\",\nNeurIPS 2023), self-preference, and sycophantic reversal under pushback (the Deliberus corpus's\n\"Jagged Judges\", arXiv:2608.12645: 25–71% flips under scripted pressure). For verification specifically,\nNLI/LLM verifiers are sensitive to decontextualization and to evidence phrasing (DnDScore). Treat any\nautomated support/refute label as a **proposal with a confidence**, and keep human-contestable\nevidence alignment.\n\n### 6.7 What these systems treat as the unit of evidence\n\nFEVER: a sentence set. SciFact: a rationale (sentence set) within one abstract. AVeriTeC: a\nquestion–answer pair tied to a URL. FActScore/SAFE: a retrieved passage per atomic fact. AIS/ALCE: a\ncited passage per statement. **Common thread:** evidence is always **a located span in an identified\nsource, aligned to one specific (sub)claim**. None treats \"a source\" or \"a document\" as the unit.\n\n---\n\n## 7. Quantitative weighting of evidence items in production systems\n\n### 7.1 Likelihood ratios and verbal scales\n\nIn forensic science, evidence strength is reported as a likelihood ratio (LR) with a **verbal\nequivalence scale**; the ENFSI *Guideline for Evaluative Reporting in Forensic Science* (2015) uses\nbands roughly: LR 2–10 weak support, 10–100 moderate, 100–1,000 moderately strong, 1,000–10,000\nstrong, 10,000–1,000,000 very strong, >1,000,000 extremely strong **[band boundaries from memory;\nverify before citing]**. The LR is attached to the **evidence given two named hypotheses**, not to the\nhypothesis; the prior belongs to the fact-finder. This separation — *the analyst supplies the LR, the\ndecider supplies the prior* — is a direct template for \"machine proposes, never asserts\".\n\nI.J. Good's **weight of evidence**, W(H:E) = log[P(E|H)/P(E|¬H)] (Good, *Probability and the Weighing of\nEvidence*, 1950; \"Weight of evidence: a brief survey\", *Bayesian Statistics 2*, 1985), is additive\nacross **conditionally independent** items — the formal version of Deliberus's corroborative summed\noffsets, and it inherits the same condition. Keynes's *weight of arguments* (A Treatise on Probability,\n1921, ch. VI) is a *different* quantity: the amount of relevant evidence, which can rise while the\nprobability stays put. The badge's direction/weight split is Good (direction) + Keynes (weight).\n\n### 7.2 STRING — combining evidence channels as independent probabilities\n\nSTRING (Szklarczyk et al., *NAR* database issues, biennial; von Mering et al. 2005 for the scoring\nscheme). Protein–protein associations are scored in separate **evidence channels** (neighbourhood,\ngene fusion, co-occurrence, co-expression, experiments, databases, text-mining, plus homology-transferred\nvariants). Each channel score is **calibrated against a gold standard (KEGG pathways)** so it is a\nprobability. Combination (verified from STRING FAQ): remove the prior p = 0.041 (probability two random\nproteins are associated) from each channel, s′ = (s − p)/(1 − p); combine as **1 − Π(1 − s′_i)**\n(noisy-OR, assuming channels independent); add the prior back, s = s′_tot + p(1 − s′_tot). Two\ndesign choices matter: (a) **independence is asserted at the channel level, not the item level** —\nall text-mining evidence is one channel, so a thousand abstracts repeating one finding cannot multiply;\n(b) **scores are calibrated per channel** against an external benchmark, so \"0.7 from co-expression\"\nand \"0.7 from experiments\" mean the same probability. STRING also notes (FAQ) that channels are not\ntruly independent and that the combination is an approximation.\n\n### 7.3 ACMG/AMP and the Tavtigian Bayesian reformulation\n\n- **Richards et al., \"Standards and guidelines for the interpretation of sequence variants\", *Genetics\n  in Medicine* 17(5), 2015** (ACMG/AMP). 28 criteria, each with a direction and a default strength:\n  pathogenic **Very Strong (PVS1), Strong (PS1–4), Moderate (PM1–6), Supporting (PP1–5)**; benign\n  **Stand-alone (BA1), Strong (BS1–4), Supporting (BP1–7)**. Combining rules (e.g., 1 VS + ≥1 S →\n  Pathogenic; 1 S + 1–2 M → Likely Pathogenic) yield five classes: Pathogenic, Likely Pathogenic,\n  **Uncertain Significance (VUS)**, Likely Benign, Benign. Criteria strengths can be **modified up or\n  down** by expert panels given the evidence quality (e.g., PS3 downgraded to moderate).\n- **Tavtigian et al., \"Modeling the ACMG/AMP variant classification guidelines as a Bayesian\n  classification framework\", *Genetics in Medicine* 20(9), 2018, 1054–1060** (verified). Recasts the\n  rules as a **naive Bayes classifier**: prior probability of pathogenicity 0.10; odds of\n  pathogenicity for \"Very Strong\" O_PVSt = 350; strength levels **scale exponentially**: Strong =\n  350^(1/2) ≈ 18.7, Moderate = 350^(1/4) ≈ 4.33, Supporting = 350^(1/8) ≈ 2.08. Posterior thresholds:\n  Pathogenic > 0.99, Likely Pathogenic 0.90–0.99 (the \"LP\" definition in ACMG as >90% certainty). They\n  showed the rules are **almost entirely internally consistent** with this model, identified two\n  combination rules that were not, and noted the model naturally handles **conflicting evidence**\n  (benign criteria contribute odds < 1) which the original rules handled poorly.\n- **Tavtigian, Harrison, Boucher & Biesecker, \"Fitting a naturally scaled point system to the ACMG/AMP\n  variant classification guidelines\", *Human Mutation* 41(10), 2020** (verified). Points proportional\n  to log-odds: **Supporting = 1, Moderate = 2, Strong = 4, Very Strong = 8**; benign criteria subtract.\n  Classification: **≥10 Pathogenic; 6–9 Likely Pathogenic; 0–5 VUS**; benign side: sources give\n  **Likely Benign −1 to −6, Benign ≤ −7** in the ClinGen SVI usage, while one secondary source lists\n  −1 to −5 and ≤ −6 **[the exact benign cut-points differ between sources I found — verify against the\n  paper's Table before citing]**.\n- **ClinGen SVI calibrations** — computational predictors (Pejaver et al., *AJHG* 109, 2022) and\n  functional assays (Brnich et al., *Genome Medicine* 12, 2020) calibrated so that a tool/assay result\n  earns a strength level *only* at the likelihood ratio it has been shown to achieve (e.g., OddsPath\n  thresholds for PS3/BS3). **Evidence earns a tier by demonstrated diagnosticity, not by its type.**\n- **Double-counting rules**: ClinGen guidance forbids using the same underlying data under two criteria\n  (e.g., PM1 hotspot and PP2 missense constraint, or PS3 functional data counted again as PP3\n  predictors) **[specific pairings from memory; the principle is well established]**.\n\nWhy this is exceptional as a model for Deliberus: it is a **production, clinically consequential\nsystem for combining heterogeneous evidence** that (1) has **explicit strength tiers on an exponential\n(log-odds) scale**, (2) requires **calibration** before a type of evidence earns a tier, (3) handles\n**both directions** additively in log space, (4) has an explicit **\"uncertain\" class** that is the\ndefault outcome of thin or conflicting evidence, (5) is **modifiable per context** by expert panels,\nand (6) has **explicit anti-double-counting rules**. And it is *points*, so humans can compute and audit\nit.\n\n### 7.4 GRADE (added: the per-outcome, not per-study, model)\n\nNot in the brief but directly relevant to \"methodology assigned per source, not per claim\". GRADE\n(Guyatt et al., *BMJ* 336, 2008; *J Clin Epidemiol* series 2011) rates **certainty of evidence per\noutcome** across the *body* of evidence: start high for RCTs, low for observational; **downgrade** for\nrisk of bias, inconsistency (heterogeneity), indirectness, imprecision, publication bias; **upgrade**\nfor large effect, dose–response, and plausible confounding that would reduce the effect. Four levels:\nhigh, moderate, low, very low. The same review can carry \"high\" for one outcome and \"very low\" for\nanother. **This is the literature's answer to per-source methodology**: the design tier is a starting\nprior, then five named, attackable reasons move it — each of which is a natural CQ.\n\n---\n\n## 8. Cross-cutting synthesis: the named gaps against the literature\n\n| Deliberus gap (briefing) | What the literature offers |\n|---|---|\n| CQ answers counted, not weighed | QP vs CP (Amgoud & Ben-Naim); top-aggregation (Mossakowski–Neuhaus); ACMG/Tavtigian log-odds tiers; saturating evidence (Jøsang, Walley) |\n| No source reliability separate from claim | Evidence-accuracy idiom (Fenton et al.); Schum's credibility chain; subjective-logic trust discounting; KBT (source vs extraction error) |\n| No independence / shared origin | Copy detection (Dong et al.); cautious rule (Denœux); averaging vs cumulative fusion (Jøsang); STRING channel-level independence; ACMG no-double-use; dependency idiom |\n| No likelihood ratio / diagnosticity | Good's weight of evidence; Hahn & Oaksford; forensic LR scales; ClinGen OddsPath calibration |\n| No sample-size / precision weighting | Beta/Dirichlet evidence counts (Jøsang, Walley IDM); GRADE imprecision; ORKG comparison columns |\n| \"Searched and found nothing\" not represented | Walton argument from ignorance (depth-of-search CQ); Hahn & Oaksford's sensitivity-based strength; FEVER NEI / AVeriTeC NEE; D-S mass on Θ |\n| No chain of custody / quote verification / locators | PROV-O (wasQuotedFrom, hadPrimarySource); nanopub assertion vs pubinfo provenance; AIS \"According to P, s\"; ALCE citation precision/recall |\n| No edge/answer timestamps | Nanopub immutable time-stamped records; AVeriTeC temporal restriction; PROV Activity times |\n| No scheme for statistical/study evidence | Discourse-graph EVD nodes; micropublication Data/Method; GRADE domains; ORKG contributions |\n| No rebut vs undercut typing | Pollock; ASPIC+ (rebut/undercut/undermine); Verheij/Prakken CQ-to-attack mapping; Carneades assumption vs exception |\n| No \"this is open\" epistemic status | D-S ignorance mass; subjective-logic u; Hunter–Thimm neutral 0.5 vs unknown; ACMG VUS; AVeriTeC NEE |\n| Methodology per source, not per claim | GRADE per outcome; SEPIO evidence lines; ECO method × assertion-method cross-product |\n| No mechanistic-evidence tier | ECO method classes; GRADE indirectness; (and the EBM+ programme — Russo–Williamson thesis, Clarke et al. 2014 — **not researched in this pass**) |\n| Publication bias, replication, funding conflict | GRADE publication-bias domain; Walton expert-opinion Trustworthiness CQ; scite contrasting citations as a replication signal |\n| No standard-of-proof concept | Carneades proof standards; ACMG posterior thresholds; ENFSI verbal scale |\n\n---\n\n## 9. Most transferable to a claim-graph argumentation platform like Deliberus\n\nConcrete, ordered from cheapest to most structural. None is a ruling; each names what it would replace.\n\n**1. Replace counted CQ answers with tiered, log-odds-scaled contributions (ACMG/Tavtigian shape).**\nGive each *answered* CQ a strength tier — supporting / moderate / strong / very strong — worth\n1 / 2 / 4 / 8 points in log-odds (or 2.08 / 4.33 / 18.7 / 350 in odds), signed by direction. The tier\nis not a free choice: it is **proposed by the answer's own content** (an answer citing a preregistered\nreplication earns more than a one-word assertion) and is itself a claim that can be attacked (the ClinGen\n\"modify the strength\" move). Default tier for a bare answer: *supporting* at most. This directly kills\n\"three one-word answers = three RCTs\", keeps arithmetic auditable by hand, and makes the one-CQ scheme\n(lack_of_evidence) able to leave the neutral band when its one answer is strong. It also gives a\ncalibration route: a scheme/answer-type earns a higher default tier only after its diagnosticity is\nmeasured (ClinGen OddsPath), never by fiat — which is how the inert methodology ladder could become\nlive without a ratification of made-up constants.\n\n**2. Carry subjective-logic opinions (b, d, u, a) on claims, not a single strength.** Direction = b vs d;\nweight = 1 − u; **base rate a per claim kind** (e.g., a base rate for \"published effect replicates\" well\nbelow 0.5), which is the missing prior. Evidence counts map via b = r/(r+s+W), giving **saturation** for\nfree: the weight dial the badge already displays stops being an unbounded count. The unexamined state is\nu = 1 exactly — no number shown, consistent with current policy. This is the published algebra for what\nthe August badge split is already doing informally.\n\n**3. Record independence explicitly and choose the fusion operator from it.** Every pair of inputs into\nthe same target is *independent*, *shared-source*, or *unknown*. Independent → cumulative fusion\n(Good/Dempster/sum); shared → averaging/cautious (Denœux's idempotent min); unknown → cautious by\ndefault. This is Deliberus's own \"a known shared source becomes a node, never a blind discount\" with\nthe operator table attached. Add **STRING-style channels**: all inputs of one *kind from one origin*\n(e.g., all answers from one account; all claims extracted from one source) form one channel; channels\ncombine as independent, items within a channel do not multiply.\n\n**4. Build copy detection, keyed on shared errors.** Two contributors or sources that repeatedly share\n*defeated* positions, identical phrasings, or identical citation errors are candidate copies (Dong et\nal.'s asymmetry: shared falsehoods reveal dependence, shared truths do not). Output is a proposal (a\n`SHARES_ORIGIN_WITH` link, attackable), never a silent discount. This is the only sybil-adjacent\ndefence the truth-discovery literature has that works without identity verification.\n\n**5. Split source reliability from claim strength (evidence-accuracy idiom + KBT).** A REPORTS or\ntestimony edge gets two separate attackable nodes: **did the source say it** (quote fidelity — AIS test\n\"According to P, s\", checkable by a separate model and by humans) and **is the source reliable on this**\n(Walton's position-to-know / expertise / trustworthiness CQs). Separating them prevents the KBT\nconfusion — a wrong claim blamed on the source when the extraction mis-rendered it. Hearsay chains use\ntrust discounting, which drives long chains to *uncertainty*, not disbelief.\n\n**6. Type every CQ as premise / assumption / exception and every attack as rebut / undercut /\nundermine (Carneades + ASPIC+).** Two consequences. (a) **Default behaviour of an open CQ differs by\ntype**: an open *assumption* CQ (is E an expert?) withholds weight until answered; an open *exception*\nCQ (is E biased?) costs nothing until someone substantiates it. Today all open CQs sit at 0.5 alike.\n(b) An **undercutter moves an edge toward u (uninformed)**, not toward disbelief — which is what a\nQBAF attack cannot express and a subjective-logic opinion can.\n\n**7. Adopt FEVER/AVeriTeC's evidence-state vocabulary as display states, with one crucial split.**\nSupported / Refuted / **Not Enough Evidence** / **Conflicting**. And split NEE into **unexamined**\n(nobody looked — u = 1) versus **searched, found nothing** (a lack-of-evidence claim with a recorded\nsearch: what was searched, how, when). The second is data and should carry weight scaled by search\nsensitivity (Hahn & Oaksford); the first is not. This is the representation for \"we searched and found\nnothing\" the briefing lists as missing. Beware AVeriTeC's lesson: *conflicting* must not fire whenever\nany answer disagrees — context-adding answers are not refutations.\n\n**8. Keep \"evidence is claims\", but add scope and the evidence line.** The discourse-graph and SEPIO\ncritique does not require a second node type; it requires (a) a **`scope: particular | general`** field\nso \"in study X (n=…), Y\" is distinguishable from \"Y\", joined by a *generalises* inference whose scheme\ncarries external-validity and heterogeneity CQs; (b) a **source binding and locator** (page, timestamp,\nspan, retrieval date) mandatory on particular-scope claims; and (c) SEPIO's **evidence line** as a\ngrouping over several particulars that together make one argument, with its strength assigned to the\nline — which is where the 70 misfiled decomposition edges belong (they are witnesses gathered into a\nline, not parts of a whole). What \"evidence is claims\" loses without these is exactly the step where\nmost scientific disputes actually live.\n\n**9. Rate methodology per claim with GRADE-style moves, not per source with a fixed ladder.** Start\nfrom the design tier as a prior, then five named downgrades (risk of bias, inconsistency, indirectness,\nimprecision, publication bias) and three upgrades (large effect, dose–response, residual confounding\nworking against the effect) — each an ordinary CQ answered with a claim. This turns the inert ladder\ninto structure, localises it to the claim it bears on, and supplies the missing slots for\npublication bias, replication status (scite-style contrasting citations as one input), and funding\nconflict (an expertise/trustworthiness CQ).\n\n**10. Choose and publish the counting principle for gradual semantics.** Current QEM-with-sum is\ncardinality-sensitive with full compensation. Options, stated as the literature states them:\n**quality precedence** (top aggregation — one strong answer dominates), **saturating compensation**\n(diminishing returns per extra input, e.g., log or Jøsang weight), or **cardinality precedence**. For an\nopen platform with brigading exposure, quality precedence within a channel plus saturating compensation\nacross independent channels is the combination most aligned with Carneades (strongest-argument\ncomparisons), ACMG (tiered points with anti-double-use) and STRING (channel independence) — but it is\na design choice, and the axioms make the trade explicit.\n\n**11. Standards of proof as per-claim display thresholds, never as verdicts.** Carneades-style proof\nstandards (scintilla, preponderance, clear-and-convincing, beyond-reasonable-doubt) or ACMG-style\nposterior thresholds can be attached to a **question** by whoever asks it (\"for a policy decision, I\nneed clear-and-convincing\"), and the graph reports whether the current state meets that standard for\nthat audience. That respects \"no verdicts\" while giving the FRI finding — residual disagreement from\n*different standards for what counts as evidence* — an explicit, attributable slot rather than a hidden\none.\n\n**12. Provenance in two layers, immutable (nanopubs + PROV-O).** Assertion provenance (who found what,\nby which method, in which source, with `wasQuotedFrom` / `hadPrimarySource`) separate from record\nprovenance (which contributor or extractor version wrote this graph object, when). Content-addressed,\nsuperseding-never-overwriting records (Trusty URIs) are the formal version of \"contributions are added,\nnever overwritten\", and supply the edge/answer timestamps the briefing lists.\n\n**Where the literature challenges existing Deliberus positions.**\n- *\"There is no atomic evidence\"* survives the argumentation literature (AIF, ArgRAG, micropublications\n  all treat evidence as claims) but meets **foundationalist** counter-designs (Oren & Norman's EAS, where\n  acceptance requires grounding in η) and **the fact-checking consensus that evidence is a located span\n  in an identified source**. The reconciliation: decomposition has no floor, but *locatability* does —\n  every particular-scope claim bottoms out in a span of a source, and that locator is the atom, not\n  the proposition.\n- *\"Probability should not differ between people given the same evidence\"* is contested by the\n  imprecise-probability school (Walley): with thin evidence the rational object is an interval, and\n  disagreement *within* it is not irrationality. The two positions converge as evidence accumulates —\n  which suggests displaying intervals (or u) rather than points is how to honour the founder's intuition\n  without manufacturing false precision.\n- *Min for necessary parts* is supported by ASPIC+'s weakest-link principle and Denœux's cautious rule,\n  but the literature adds the caveat that **explaining-away and other cross-item interactions** (Fenton\n  et al.) are invisible to any edge-local rule; stories-and-coverage (Bex) is the argumentation-side\n  remedy for evaluating claim-bundles jointly.\n\n---\n\n## References\n\n(Verified in this pass = checked against a publisher page, the paper's abstract, or an official\ndocumentation page. Others are standard citations given from knowledge; details flagged in-text where\nuncertain.)\n\n**Argumentation theory**\n- Toulmin, S. (1958). *The Uses of Argument*. Cambridge University Press.\n- Pollock, J. L. (1987). Defeasible reasoning. *Cognitive Science* 11(4), 481–518.\n- Pollock, J. L. (1995). *Cognitive Carpentry*. MIT Press.\n- Prakken, H. (2010). An abstract framework for argumentation with structured arguments. *Argument & Computation* 1(2), 93–124.\n- Modgil, S., & Prakken, H. (2013). A general account of argumentation with preferences. *Artificial Intelligence* 195, 361–397.\n- Modgil, S., & Prakken, H. (2014). The ASPIC+ framework for structured argumentation: a tutorial. *Argument & Computation* 5(1), 31–62.\n- Walton, D., Reed, C., & Macagno, F. (2008). *Argumentation Schemes*. Cambridge University Press.\n- Walton, D. (2008). *Witness Testimony Evidence: Argumentation, Artificial Intelligence, and Law*. Cambridge University Press.\n- Verheij, B. (2003). Dialectical argumentation with argumentation schemes: An approach to legal logic. *Artificial Intelligence and Law* 11, 167–195. [venue verified 2026-09-28]\n- Gordon, T. F., Prakken, H., & Walton, D. (2007). The Carneades model of argument and burden of proof. *Artificial Intelligence* 171(10–15), 875–896. (verified)\n- Gordon, T. F., & Walton, D. (2009). Proof burdens and standards. In Rahwan & Simari (eds.), *Argumentation in Artificial Intelligence*, Springer.\n- Bex, F. J. (2011). *Arguments, Stories and Criminal Evidence: A Formal Hybrid Theory*. Springer (Law and Philosophy Library 92).\n- Verheij, B. (2017). Proof with and without probabilities. *Artificial Intelligence and Law* 25(1), 127–154. [unverified details]\n- Verheij, B., Bex, F., Timmer, S., Vlek, C., Meyer, J.-J., Renooij, S., & Prakken, H. (2016). Arguments, scenarios and probabilities: connections between three normative frameworks for evidential reasoning. *Law, Probability and Risk* 15(1), 35–70.\n- Hahn, U., & Oaksford, M. (2007). The rationality of informal argumentation: A Bayesian approach to reasoning fallacies. *Psychological Review* 114(3), 704–732.\n- Hahn, U., Harris, A. J. L., & Corner, A. (2009). Argument content and argument source: An exploration. *Informal Logic* 29(4). [venue from memory]\n- Zenker, F. (ed.) (2013). *Bayesian Argumentation: The Practical Side of Probability*. Springer (Synthese Library 362).\n- Bovens, L., & Hartmann, S. (2003). *Bayesian Epistemology*. Oxford University Press.\n- Thimm, M. (2012). A probabilistic semantics for abstract argumentation. *ECAI 2012*, 750–755.\n- Hunter, A. (2013). A probabilistic approach to modelling uncertain logical arguments. *International Journal of Approximate Reasoning* 54(1), 47–81.\n- Hunter, A., & Thimm, M. (2017). Probabilistic reasoning with abstract argumentation frameworks. *JAIR* 59, 565–611.\n- Oren, N., & Norman, T. J. (2008). Semantics for evidence-based argumentation. *COMMA 2008*, IOS Press, 276–284. https://ebooks.iospress.nl/publication/4196 (verified)\n- Oren, N., Norman, T. J., & Preece, A. (2007). Subjective logic and arguing with evidence. *Artificial Intelligence* 171(10–15), 838–854.\n- Polberg, S., & Oren, N. (2014). Revisiting support in abstract argumentation systems. *COMMA 2014*.\n- Potyka, N. (2018). Continuous dynamical systems for weighted bipolar argumentation. *KR 2018*.\n- Baroni, P., Rago, A., & Toni, F. (2019). From fine-grained properties to broad principles for gradual argumentation. *IJAR* 105, 252–286.\n- Mossakowski, T., & Neuhaus, F. (2018). Modular semantics and characteristics for bipolar weighted argumentation graphs. arXiv:1807.06685.\n- Amgoud, L., & Ben-Naim, J. (2013). Ranking-based semantics for argumentation frameworks. *SUM 2013*, LNCS 8078. https://www.irit.fr/~Leila.Amgoud/amgoud-bennaim.pdf\n- Amgoud, L., & Ben-Naim, J. (2018). Weighted bipolar argumentation graphs: Axioms and semantics. *IJCAI 2018*. (verified)\n- Bonzon, E., Delobelle, J., Konieczny, S., & Maudet, N. (2016). A comparative study of ranking-based semantics for abstract argumentation. *AAAI 2016*.\n- Amgoud, L., Bonzon, E., Delobelle, J., Doder, D., Konieczny, S., & Maudet, N. (2018). Gradual semantics accounting for similarity between arguments. *KR 2018*.\n- Zhu, Y., Potyka, N., Hernández, D., et al. (2025). ArgRAG: Explainable retrieval augmented generation using quantitative bipolar argumentation. *NeSy 2025*, PMLR 284. https://proceedings.mlr.press/v284/zhu25a.html ; arXiv:2508.20131 (verified)\n\n**Belief functions and imprecise probability**\n- Dempster, A. P. (1967). Upper and lower probabilities induced by a multivalued mapping. *Annals of Mathematical Statistics* 38, 325–339.\n- Shafer, G. (1976). *A Mathematical Theory of Evidence*. Princeton University Press.\n- Zadeh, L. A. (1984). Review of Shafer's *A Mathematical Theory of Evidence*. *AI Magazine* 5(3), 81–83.\n- Zadeh, L. A. (1986). A simple view of the Dempster–Shafer theory of evidence and its implication for the rule of combination. *AI Magazine* 7(2), 85–90.\n- Smets, P., & Kennes, R. (1994). The transferable belief model. *Artificial Intelligence* 66, 191–234.\n- Denœux, T. (2008). Conjunctive and disjunctive combination of belief functions induced by nondistinct bodies of evidence. *Artificial Intelligence* 172(2–3), 234–264.\n- Jøsang, A. (2001). A logic for uncertain probabilities. *IJUFKS* 9(3), 279–311.\n- Jøsang, A. (2016). *Subjective Logic: A Formalism for Reasoning Under Uncertainty*. Springer.\n- van der Heijden, R. W., Kopp, H., & Kargl, F. (2018). Multi-source fusion operations in subjective logic. *FUSION 2018*.\n- Walley, P. (1991). *Statistical Reasoning with Imprecise Probabilities*. Chapman & Hall.\n- Walley, P. (1996). Inferences from multinomial data: learning about a bag of marbles. *JRSS B* 58(1), 3–57.\n- Seidenfeld, T., & Wasserman, L. (1993). Dilation for sets of probabilities. *Annals of Statistics* 21(3).\n- Shafer, G., & Vovk, V. (2001). *Probability and Finance: It's Only a Game!* Wiley.\n- Shafer, G., & Vovk, V. (2019). *Game-Theoretic Foundations for Probability and Finance*. Wiley.\n- Shafer, G. (2021). Testing by betting. *JRSS A* 184(2), 407–431.\n- Vovk, V., & Wang, R. (2021). E-values: Calibration, combination and applications. *Annals of Statistics* 49(3).\n- Grünwald, P., de Heide, R., & Koolen, W. (2024). Safe testing. *JRSS B*. [unverified venue/year]\n\n**Bayesian networks and evidence**\n- Schum, D. A. (1994). *The Evidential Foundations of Probabilistic Reasoning*. Wiley.\n- Kadane, J. B., & Schum, D. A. (1996). *A Probabilistic Analysis of the Sacco and Vanzetti Evidence*. Wiley.\n- Fenton, N., Neil, M., & Lagnado, D. A. (2013). A general structure for legal arguments about evidence using Bayesian networks. *Cognitive Science* 37(1), 61–102. https://onlinelibrary.wiley.com/doi/10.1111/cogs.12004 (verified)\n- Lagnado, D. A., Fenton, N., & Neil, M. (2013). Legal idioms: a framework for evidential reasoning. *Argument & Computation* 4(1), 46–63.\n- Neil, M., Fenton, N., Lagnado, D., & Gill, R. D. (2019). Modelling competing legal arguments using Bayesian model comparison and averaging. *Artificial Intelligence and Law* 27, 403–430.\n- Fenton, N., Neil, M., Yet, B., & Lagnado, D. (2020). Analyzing the Simonshaven case using Bayesian networks. *Topics in Cognitive Science* 12(4).\n- Good, I. J. (1950). *Probability and the Weighing of Evidence*. Griffin.\n- Good, I. J. (1985). Weight of evidence: A brief survey. In *Bayesian Statistics 2*, North-Holland, 249–270.\n- Keynes, J. M. (1921). *A Treatise on Probability*. Macmillan (ch. VI).\n\n**Truth discovery**\n- Dawid, A. P., & Skene, A. M. (1979). Maximum likelihood estimation of observer error-rates using the EM algorithm. *JRSS C* 28(1), 20–28.\n- Raykar, V. C., et al. (2010). Learning from crowds. *JMLR* 11, 1297–1322.\n- Whitehill, J., et al. (2009). Whose vote should count more. *NeurIPS 2009*.\n- Yin, X., Han, J., & Yu, P. S. (2007). Truth discovery with multiple conflicting information providers on the web. *KDD 2007*; *IEEE TKDE* 20(6), 2008.\n- Dong, X. L., Berti-Équille, L., & Srivastava, D. (2009). Integrating conflicting data: the role of source dependence. *PVLDB* 2(1), 550–561. http://www.vldb.org/pvldb/vol2/vldb09-pvldb47.pdf (verified)\n- Dong, X. L., Berti-Équille, L., & Srivastava, D. (2009). Truth discovery and copying detection in a dynamic world. *PVLDB* 2(1), 562–573.\n- Li, X., Dong, X. L., Lyons, K., Meng, W., & Srivastava, D. (2012). Truth finding on the deep web: Is the problem solved? *PVLDB* 6(2).\n- Pochampally, R., Das Sarma, A., Dong, X. L., Meliou, A., & Srivastava, D. (2014). Fusing data with correlations. *SIGMOD 2014*; arXiv:1503.00306.\n- Dong, X. L., Gabrilovich, E., Murphy, K., et al. (2015). Knowledge-Based Trust: Estimating the trustworthiness of web sources. *PVLDB* 8(9), 938–949. https://www.vldb.org/pvldb/vol8/p938-dong.pdf (verified)\n- Li, Y., Gao, J., Meng, C., Li, Q., Su, L., Zhao, B., Fan, W., & Han, J. (2016). A survey on truth discovery. *SIGKDD Explorations* 17(2).\n- Yu, D., et al. (2014). The wisdom of minority: Unsupervised slot filling validation based on multi-dimensional truth-finding. *COLING 2014*. [unverified details]\n\n**Ontologies and scholarly representations**\n- Chibucos, M. C., et al. (2014). Standardized description of scientific evidence using the Evidence Ontology (ECO). *Database* 2014:bau075.\n- Nadendla, S., et al. (2022). ECO: the Evidence and Conclusion Ontology, an update for 2022. *Nucleic Acids Research* 50(D1), D1515–D1521. https://academic.oup.com/nar/article/50/D1/D1515/6431816 (verified)\n- Gene Ontology Consortium. Guide to GO evidence codes. http://geneontology.org/docs/guide-go-evidence-codes/\n- Brush, M. H., Shefchek, K., & Haendel, M. (2016). SEPIO: A semantic model for the integration and analysis of scientific evidence. *ICBO 2016* (CEUR). https://github.com/monarch-initiative/SEPIO-ontology (verified)\n- Lebo, T., Sahoo, S., McGuinness, D., et al. (2013). *PROV-O: The PROV Ontology*. W3C Recommendation. https://www.w3.org/TR/prov-o/\n- Groth, P., Gibson, A., & Velterop, J. (2010). The anatomy of a nanopublication. *Information Services & Use* 30(1–2), 51–56.\n- Kuhn, T., et al. (2016). Decentralized provenance-aware publishing with nanopublications. *PeerJ Computer Science* 2:e78.\n- Kuhn, T., & Dumontier, M. (2014). Trusty URIs: Verifiable, immutable, and permanent digital artifacts for linked data. *ESWC 2014*.\n- Clark, T., Ciccarese, P. N., & Goble, C. A. (2014). Micropublications: a semantic model for claims, evidence, arguments and annotations in biomedical communications. *Journal of Biomedical Semantics* 5:28.\n- Shotton, D. (2010). CiTO, the Citation Typing Ontology. *Journal of Biomedical Semantics* 1(Suppl 1):S6.\n- Peroni, S., & Shotton, D. (2012). FaBiO and CiTO: ontologies for describing bibliographic resources and citations. *Journal of Web Semantics* 17, 33–43.\n- Nicholson, J. M., et al. (2021). scite: A smart citation index that displays the context of citations and classifies their intent using deep learning. *Quantitative Science Studies* 2(3), 882–898. https://direct.mit.edu/qss/article/2/3/882/102990 (verified)\n- Jaradeh, M. Y., et al. (2019). Open Research Knowledge Graph: Next generation infrastructure for semantic scholarly knowledge. *K-CAP 2019*.\n- Chesñevar, C., et al. (2006). Towards an argument interchange format. *Knowledge Engineering Review* 21(4), 293–316.\n- Rahwan, I., Zablith, F., & Reed, C. (2007). Laying the foundations for a World Wide Argument Web. *Artificial Intelligence* 171(10–15), 897–921.\n- Chan, J., et al. (2024). Steps towards an infrastructure for scholarly synthesis. arXiv:2407.20666. (verified)\n\n**Fact-checking and verification**\n- Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2018). FEVER: a large-scale dataset for fact extraction and verification. *NAACL 2018*.\n- Schuster, T., et al. (2019). Towards debiasing fact verification models. *EMNLP 2019*.\n- Wadden, D., et al. (2020). Fact or fiction: Verifying scientific claims. *EMNLP 2020*.\n- Schlichtkrull, M., Guo, Z., & Vlachos, A. (2023). AVeriTeC: A dataset for real-world claim verification with evidence from the web. *NeurIPS 2023 Datasets & Benchmarks*; arXiv:2305.13117. (verified)\n- Schlichtkrull, M., et al. (2024). The Automated Verification of Textual Claims (AVeriTeC) shared task. *FEVER Workshop 2024*; arXiv:2410.23850.\n- Min, S., et al. (2023). FActScore: Fine-grained atomic evaluation of factual precision in long form text generation. *EMNLP 2023*.\n- Wei, J., et al. (2024). Long-form factuality in large language models. arXiv:2403.18802 (NeurIPS 2024).\n- Song, Y., Kim, Y., & Iyyer, M. (2024). VeriScore: Evaluating the factuality of verifiable claims in long-form text generation. *Findings of EMNLP 2024*.\n- Wanner, M., Van Durme, B., & Dredze, M. (2025). DnDScore: Decontextualization and decomposition for factuality verification in long-form text generation. *EMNLP 2025*; arXiv:2412.13175. (verified)\n- Rashkin, H., et al. (2023). Measuring attribution in natural language generation models. *Computational Linguistics* 49(4); arXiv:2112.12870.\n- Gao, T., Yen, H., Yu, J., & Chen, D. (2023). Enabling large language models to generate text with citations. *EMNLP 2023*.\n- Zheng, L., et al. (2023). Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. *NeurIPS 2023 Datasets & Benchmarks*.\n\n**Quantitative evidence weighting in production**\n- Szklarczyk, D., et al. STRING database (NAR database issues, 2019/2021/2023). Scoring: https://string-db.org/help/faq/ (verified: prior 0.041, channel combination).\n- von Mering, C., et al. (2005). STRING: known and predicted protein–protein associations, integrated and transferred across organisms. *NAR* 33, D433–D437.\n- Richards, S., et al. (2015). Standards and guidelines for the interpretation of sequence variants. *Genetics in Medicine* 17(5), 405–424.\n- Tavtigian, S. V., et al. (2018). Modeling the ACMG/AMP variant classification guidelines as a Bayesian classification framework. *Genetics in Medicine* 20(9), 1054–1060. https://www.nature.com/articles/gim2017210 (verified)\n- Tavtigian, S. V., Harrison, S. M., Boucher, K. M., & Biesecker, L. G. (2020). Fitting a naturally scaled point system to the ACMG/AMP variant classification guidelines. *Human Mutation* 41(10), 1734–1737. https://onlinelibrary.wiley.com/doi/abs/10.1002/humu.24088 (verified)\n- Brnich, S. E., et al. (2020). Recommendations for application of the functional evidence PS3/BS3 criterion using the ACMG/AMP sequence variant interpretation framework. *Genome Medicine* 12, 3.\n- Pejaver, V., et al. (2022). Calibration of computational tools for missense variant pathogenicity classification and ClinGen recommendations for PP3/BP4 criteria. *AJHG* 109(12), 2163–2177.\n- ENFSI (2015). *ENFSI Guideline for Evaluative Reporting in Forensic Science*. [verbal scale bands to verify]\n- Guyatt, G. H., et al. (2008). GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. *BMJ* 336, 924–926.\n\n**Not researched in this pass (flagged for follow-up):** the EBM+ / Russo–Williamson mechanistic-evidence\nliterature; Wigmore charts in detail (Anderson, Schum & Twining, *Analysis of Evidence*, 2nd ed. 2005);\nBayesian meta-analytic bias models (e.g., Turner et al. 2009 bias adjustment); exact scite supporting/\ncontrasting ratios; exact ENFSI bands; exact Tavtigian 2020 benign cut-points.\n"}