{"path":"research/testable-bets/hedges-vagueness-and-testability-literature.md","content":"# Hedges, vague probability words and testable claims: what the literature says\n\nPrepared 2026-10-20 for Deliberus. The question: Fredrik has ruled that an author's stance hedges (*may*, *probably*, *I'd guess*) move out of a claim's wording into a separate stance field, while scope words (*tend to*, *most*, *some*), stated probabilities (*a 70% chance*) and possibility-as-content (*this policy could cut emissions*) stay in the wording. This note reviews the research on hedges, verbal probability and testability that bears on that ruling. It complements `prediction-markets-argumentation.md` and does not repeat it.\n\nHow to read it. Every factual statement carries a link. *(secondary)* means I read a summary of the source, not the source. *(unverified)* means I could not confirm it. Section 8 lists everything I could not check. To stay within quotation limits the note contains one direct quotation (section 1.1); the rest is paraphrase. Expressions under study are set in italics.\n\n---\n\n## 0. Summary\n\n**What the literature says for the ruling**\n\n1. Keeping the proposition apart from the speaker's commitment to it is standard practice in several fields. Toulmin gives the qualifier its own slot, separate from claim, data and warrant ([Toulmin, *The Uses of Argument*, excerpt](https://iphils.uj.edu.pl/~a.trybus/classes/ct/The.Uses.of.Argument.Excerpt.pdf)). The US intelligence standard, the UK assessment standard and the IPCC all separate the likelihood of an event from the confidence in the judgment ([ICD 203](https://www.odni.gov/files/documents/ICD/ICD-203.pdf), [UK PHIA](https://www.gov.uk/government/publications/explaining-uncertainty-in-uk-intelligence-assessment/explaining-uncertainty-in-uk-intelligence-assessment), [IPCC AR6 TS](https://www.ipcc.ch/report/ar6/wg1/downloads/report/IPCC_AR6_WGI_TS.pdf)). FactBank scores whether an event is factual separately for each source, with the author as one source among others ([Saurí and Pustejovsky 2012](https://aclanthology.org/J12-2002.pdf)). Nanopublications separate assertion from provenance ([nanopub.net guidelines](https://nanopub.net/guidelines/working_draft/)).\n2. Hedge words carry confidence badly. Kent found the colleagues who signed one estimate meant anything from 20 to 80 percent by *serious possibility* ([Kent 1964](https://www.cia.gov/resources/csi/static/Words-of-Estimative-Probability.pdf)). Readers translated *may* in policy sentences into anything from 0.08 to 0.59 ([Tetlock, Mellers and Scoblic 2017](https://pdf.stafforini.com/tetlock-2017-bringing-probability-judgments.pdf)). In an experiment with the US intelligence table, readers' numbers matched the intended ranges 32% of the time with no guidance, and only putting the numbers inside the sentence raised that substantially (to 66%) ([Wintle et al. 2019](https://pmc.ncbi.nlm.nih.gov/articles/PMC6469752/)). A hedge in the wording is a lossy channel; a stance field that keeps the cue word and a coarse band is a better home.\n3. Hedged claims are hard to refute or bet on. Popper, Tetlock, Hanson and the forecasting platforms agree that a claim has to forbid something specific and be resolvable without asking the author what they meant ([Popper](https://www.stephenhicks.org/wp-content/uploads/2024/02/PopperK-Science-as-Falsification-text-full.pdf), [Tetlock et al. 2017](https://pdf.stafforini.com/tetlock-2017-bringing-probability-judgments.pdf), [Metaculus FAQ](https://www.metaculus.com/faq/)). Leaving *may* in the wording lets the speaker say afterwards that they only said it was possible.\n4. Hedges do several jobs. Hyland separates hedges about the truth of the proposition, hedges about precision, and hedges that protect the writer or defer to the reader ([Hyland 1998 preview](https://api.pageplace.de/preview/DT0400.9789027282583_A24761077/preview-9789027282583_A24761077.pdf)). Only some of them are a stance on the proposition, which is the distinction the ruling needs.\n\n**What the literature says against, or as a warning**\n\n5. Removing a hedge strengthens the claim, and strengthening in transmission is a well-documented failure. Popularisations drop hedges and raise certainty ([Fahnestock 1986](https://journals.sagepub.com/doi/10.1177/0741088386003003001)). News stories repeat press-release exaggeration with odds ratios of 6.5 to 56 ([Sumner et al. 2014](https://pmc.ncbi.nlm.nih.gov/articles/PMC4262123/)). LLM summaries of science were roughly twice as likely to generalise as the source abstracts, and some models overgeneralised in 26 to 73 percent of cases ([Peters and Chin-Yee 2025](https://royalsocietypublishing.org/rsos/article/12/4/241776/235656/Generalization-bias-in-large-language-model)).\n6. The classification is noisy even for humans. Annotators agree well that *might* is epistemic (98% raw agreement) and badly on *could* (kappa 0.20 to 0.26) ([Moon et al. 2016](https://aclanthology.org/2016.lilt-14.6.pdf)). Epistemic and root readings get confused, and scientific prose often merges them ([Ruppenhofer and Rehbein 2012](https://aclanthology.org/L12-1458.pdf), [Huschová 2014](https://doi.org/10.5817/bse2014-1-5)). I found no dataset that labels the exact distinction in the ruling (section 3.6).\n7. Scope words and stated probabilities are themselves ambiguous. Quantifier thresholds differ between people ([Ramotowska et al. 2024](https://link.springer.com/article/10.3758/s13423-024-02502-7)). *A 70% chance* can express the author's credence in one event or a chance process in the world ([Ülkümen, Fox and Malle 2016](https://pubmed.ncbi.nlm.nih.gov/27442037/)). Prediction platforms treat the number as the forecaster's credence about a resolvable event, not as part of the event ([Tetlock et al. 2017](https://pdf.stafforini.com/tetlock-2017-bringing-probability-judgments.pdf)). The ruling's carve-out for stated probabilities cuts against that convention (section 7, item 15).\n8. Possibility claims (*could*, *can*) are existential statements. They can be verified by one case and cannot be refuted by any finite search ([IEP, Popper](https://iep.utm.edu/pop-sci/)), so they need a different testable form from a prediction.\n\n**Net.** The separation is well precedented. The danger is the direction of error: wrongly stripping a content modal creates a stronger claim that a named author never made, while wrongly keeping a stance hedge only leaves a vaguer claim. Default to keeping the cue when unsure. Make the unit that users and summaries see *claim plus holder plus stance*, never the bare firm claim attributed to a person. Give each claim a resolution-condition block and run the clairvoyance test before offering it as a bet.\n\n---\n\n## 1. Words of estimative probability\n\n### 1.1 Kent (1964)\n\n- Sherman Kent's essay appeared in *Studies in Intelligence* 8(4), Fall 1964 ([CIA archive page](https://www.cia.gov/resources/csi/studies-in-intelligence/archives/vol-8-no-4/words-of-estimative-probability); full text read from [the PDF](https://www.cia.gov/resources/csi/static/Words-of-Estimative-Probability.pdf)).\n- The origin story: National Intelligence Estimate 29-51 (March 1951) called an attack on Yugoslavia *a serious possibility*. A few days later the Policy Planning Staff chairman asked what odds that meant. Kent had in mind about 65 to 35 in favour of attack. The policy side had read much lower odds. When Kent asked the Board of National Estimates, each member had different odds in mind, the lowest about 20 to 80 and the highest 80 to 20 ([Kent 1964](https://www.cia.gov/resources/csi/static/Words-of-Estimative-Probability.pdf)).\n- Kent's five-step table, each with a plus-or-minus margin: almost certain 93% (±6), probable 75% (±12), chances about even 50% (±10), probably not 30% (±10), almost certainly not 7% (±5) ([Kent 1964](https://www.cia.gov/resources/csi/static/Words-of-Estimative-Probability.pdf)).\n- Kent reserved *possible* for things that matter but whose odds cannot be estimated, and set a first rule, the one most relevant to Deliberus: \"The word 'possible' (and its cognates) must not be modified.\" He objected to *serious possibility* because it lifts a word from the bottom of the odds scale into the upper half ([Kent 1964](https://www.cia.gov/resources/csi/static/Words-of-Estimative-Probability.pdf)).\n- Kent describes the opposing camp, whom he calls the *poets*, who wanted to keep *may well* and *distinctly possible* for their flavour. He answers that the flavour is odds communicated without quoting them, which is fudging by another name ([Kent 1964](https://www.cia.gov/resources/csi/static/Words-of-Estimative-Probability.pdf)). The ruling puts that flavour into a separate field, which answers the poets' complaint in a different way: the flavour survives, but outside the claim sentence.\n- His scheme was never fully adopted. A 1975 CIA memo says so, as summarised on Wikipedia (secondary) ([Wikipedia](https://en.wikipedia.org/wiki/Words_of_estimative_probability), [CIA reading room](https://www.cia.gov/readingroom/node/845361)).\n\n### 1.2 How readers actually interpret the words\n\n- Wallsten and colleagues (1986) showed that the usual finding is large between-person variability, greatest in the middle of the scale, with heavy overlap between terms. Within-person variability is smaller but not trivial. They proposed measuring the meaning of a term as a membership function over 0 to 1 ([Wallsten et al. 1986](https://faculty.ucr.edu/~ramiz/papers/Vague_meanings_of_probability_terms.pdf)).\n- Mauboussin and Mauboussin surveyed about 1,700 people. For *likely* the middle 90% of answers fell between 55% and 90%. For *real possibility* they fell between 20% and 80%, which the authors call nearly meaningless (numbers read from [a secondary summary](https://blog.jcx.au/posts/how-likely-is-likely); original at [HBR, July 2018](https://hbr.org/2018/07/if-you-say-something-is-likely-how-likely-do-people-think-it-is), paywalled, numbers not checked there).\n- Tetlock, Mellers and Scoblic note that when readers translate *may*, *could* and *might* from policy sentences into probabilities, answers run from 0.08 to 0.59 ([Science 2017](https://pdf.stafforini.com/tetlock-2017-bringing-probability-judgments.pdf)). The source for that range is their reference 1, which I did not identify.\n- Intelligence analysts' own lexicons overlap too. In one study *possible* stood for values from 10% to 60%, and *highly unlikely* for both 10% and 20% ([Dhami 2018](https://repository.mdx.ac.uk/download/c1d4d3f8698a141697c4b24c33ebafabd0fe62c59c9a81d8f228ebe1259ecc2b/506810/Dhami_2018.pdf)).\n- When people are asked for an open-ended probability they over-use 50%, and the blip largely vanishes when explicit response options are offered. A 50 is often used to mean *no idea* ([Fischhoff and Bruine de Bruin 1999](https://research.tue.nl/en/publications/fifty-fifty50/), [Bruine de Bruin and colleagues](https://pmc.ncbi.nlm.nih.gov/articles/PMC4152727/)). This matters because a stance field must have a value for *the author gave no real estimate*.\n\n### 1.3 The official tables\n\n| Scheme | Terms and ranges |\n|---|---|\n| Kent 1964 | almost certain 93; probable 75; about even 50; probably not 30; almost certainly not 7 (each ± a margin) |\n| US ICD 203 (signed 2 Jan 2015, as amended) | almost no chance 1–5; very unlikely 5–20; unlikely 20–45; roughly even chance 45–55; likely 55–80; very likely 80–95; almost certain 95–99 |\n| UK PHIA yardstick | remote chance up to ≈5; highly unlikely ≈10–20; unlikely ≈25–35; realistic possibility ≈40 to under 50; likely or probable ≈55–75; highly likely ≈80–90; almost certain ≈95 to under 100 |\n| NATO (2016, as reproduced by Mandel and Irwin) | highly unlikely under 10; unlikely 10–40; even chance 40–60; likely 60–90; highly likely over 90 |\n| IPCC AR6 | virtually certain 99–100; extremely likely 95–100; very likely 90–100; likely 66–100; more likely than not 50–100; about as likely as not 33–66; unlikely 0–33; very unlikely 0–10; extremely unlikely 0–5; exceptionally unlikely 0–1 |\n\nSources: [Kent](https://www.cia.gov/resources/csi/static/Words-of-Estimative-Probability.pdf), [ICD 203](https://www.odni.gov/files/documents/ICD/ICD-203.pdf), [PHIA](https://www.gov.uk/government/publications/explaining-uncertainty-in-uk-intelligence-assessment/explaining-uncertainty-in-uk-intelligence-assessment), [Mandel and Irwin 2021](https://doi.org/10.1017/s1930297500008603), [IPCC AR6 Technical Summary](https://www.ipcc.ch/report/ar6/wg1/downloads/report/IPCC_AR6_WGI_TS.pdf). I could not fetch the NATO doctrine AJP-2.1 itself (section 8).\n\nObservations from the table:\n\n- The word *likely* has four different ranges across the US, UK, NATO and IPCC schemes (55–80, ≈55–75, 60–90, 66–100). The IPCC terms are nested, so *likely* includes *very likely*, which readers find confusing ([Wintle et al. 2019](https://pmc.ncbi.nlm.nih.gov/articles/PMC6469752/)). The US and UK lexicons disagree for every phrase according to an earlier comparison ([Ho et al. 2015](https://doi.org/10.1177/237946151500100206)).\n- The UK yardstick uses *realistic possibility* as a probability band of about 40 to under 50 percent, which is the opposite of Kent's rule that *possible* should stay unmodified and mean *odds cannot be estimated* ([PHIA](https://www.gov.uk/government/publications/explaining-uncertainty-in-uk-intelligence-assessment/explaining-uncertainty-in-uk-intelligence-assessment)). In practice *possible* is a probability word with a wide band, so a claim containing *possible* may carry Kent's sense, a low-to-moderate-probability sense, or an ability sense.\n- The UK standard itself says that bare conditionals (if X occurred, Y might follow) are an inappropriate use of terms that imply a judgment without saying what it is ([PHIA](https://www.gov.uk/government/publications/explaining-uncertainty-in-uk-intelligence-assessment/explaining-uncertainty-in-uk-intelligence-assessment)). That is the same pattern as *this policy could cut emissions*.\n- The IPCC note says findings backed by overwhelming evidence can be stated as plain fact without uncertainty qualifiers, and warns that *about as likely as not* should not be used to express lack of knowledge ([IPCC AR5 guidance note](https://www.ipcc.ch/site/assets/uploads/2018/05/uncertainty-guidance-note.pdf)).\n\n### 1.4 Do standards fix the problem?\n\n- Not much. In an online experiment with 924 participants, bracketing the ICD 203 numbers inside the sentence raised average correspondence with the intended range to 66%, against 32% in the control. The abstract says correspondence improved substantially only in that bracketed condition, not with a click-through table or a tooltip ([Wintle et al. 2019](https://pmc.ncbi.nlm.nih.gov/articles/PMC6469752/)).\n- With IPCC sentences, the earlier Budescu study found 40% consistency with bracketed numbers against 27% with only an appendix table. Participants clicked through to the guidelines table about one time in eight ([Wintle et al., summarising Budescu et al.](https://pmc.ncbi.nlm.nih.gov/articles/PMC6469752/)).\n- Evidence-based lexicons, built from how people actually read the words, were better understood than those of the IPCC, the US National Intelligence Council and UK Defence Intelligence ([Ho et al. 2015](https://doi.org/10.1177/237946151500100206)).\n- Mandel and Irwin tested 1,202 participants. Numeric and combined word-plus-number formats agreed better with the intended probability than words alone, the combined format added nothing over numbers alone, and the authors conclude that numerically bounded word schemes are an ineffective way to convey probability ([Mandel and Irwin 2021, abstract on RePEc](https://ideas.repec.org/a/cup/judgdm/v16y2021i2p363-393_5.html)).\n- Senders prefer words, receivers prefer numbers (the *communication mode preference paradox*), and Beyth-Marom suggested that the sender preference may partly reflect a wish for judgments to be less verifiable ([Mandel and Irwin 2021, introduction](https://doi.org/10.1017/s1930297500008603)).\n\n### 1.5 What happens to decisions and accountability when words replace numbers\n\n- Anecdotes as retold in a Tetlock-coauthored paper: the Joint Chiefs wrote *fair chance* of success for the Bay of Pigs and President Kennedy read it as optimistic although the author says he meant a lukewarm assessment. In 2011 advisers' estimates that bin Laden was in Abbottabad ranged from 35% to 95%, and unpacking *strong possibility* exposed a 60% versus 95% disagreement ([Friedman et al. NBER draft](https://conference.nber.org/confer/2015/SI2015/PENS/Friedman_Baker_Mellers_Tetlock_Zeckhauser.pdf)). Both are second-hand retellings of historical events.\n- Evidence on information loss. Using 764,448 forecasts by 1,718 people in the Good Judgment Project, the authors round numerical forecasts to coarser categories and find that qualitative expressions systematically sacrifice predictive information ([Friedman et al. NBER draft](https://conference.nber.org/confer/2015/SI2015/PENS/Friedman_Baker_Mellers_Tetlock_Zeckhauser.pdf)). I read the 2015 draft; I did not locate the published version.\n- Vague language also blocks scoring. Among 3,622 Canadian strategic-intelligence forecasts, 73% had an outcome that could be coded as occurred or not, and of those 23% used wording that could not be mapped to a number. The coded forecasts were well calibrated but underconfident, so verbal hedges carried real, usable calibration signal ([Mandel and Barnes 2018](https://doi.org/10.1002/bdm.2055)). The same draft by Friedman et al. reports a 33% too-vague-to-score figure for a sample of 2,897 forecasts from the earlier study ([NBER draft, note 23](https://conference.nber.org/confer/2015/SI2015/PENS/Friedman_Baker_Mellers_Tetlock_Zeckhauser.pdf)); I did not establish why the two figures differ (different samples or coding are the likely reasons).\n- In a review of 379 declassified US estimates from 1964 to 1994, only 4% of key judgments used a numeric probability ([Friedman et al. NBER draft, note 12](https://conference.nber.org/confer/2015/SI2015/PENS/Friedman_Baker_Mellers_Tetlock_Zeckhauser.pdf)).\n- Tetlock's diagnosis: without explicit probabilities, outcomes and time frames, there is no way to assess accuracy, and vague verbiage works as a survival skill for pundits ([Edge Master Class](https://www.edge.org/node/26465)).\n\n### 1.6 Likelihood versus confidence\n\n- ICD 203 asks for the likelihood of an event and the analyst's confidence in the basis of the judgment, and forbids combining a confidence level and a degree of likelihood in one sentence ([ICD 203](https://www.odni.gov/files/documents/ICD/ICD-203.pdf)).\n- The UK standard defines probability as the likelihood that a statement is true and analytical confidence as the soundness and stability of the foundations of that estimate. It uses separate ratings for them ([PHIA](https://www.gov.uk/government/publications/explaining-uncertainty-in-uk-intelligence-assessment/explaining-uncertainty-in-uk-intelligence-assessment)).\n- The IPCC does the same with a qualitative confidence scale and a quantitative likelihood scale ([IPCC AR6 TS](https://www.ipcc.ch/report/ar6/wg1/downloads/report/IPCC_AR6_WGI_TS.pdf)).\n- Analysts confuse the two in practice ([Friedman and Zeckhauser, cited in a NATO report chapter, secondary](https://www.researchgate.net/publication/341523231_UK_and_US_policies_for_communicating_probability_in_intelligence_analysis_A_review)).\n- Ordinary speech has a version of the split. Phrases like *I'm 90% sure* or *I'm fairly confident* mainly express epistemic uncertainty (what the speaker does not know about a single case), while phrases like *there's a 90% chance* mainly express aleatory uncertainty (the tendency of a class of events). Speakers choose between them in a two-year sample of New York Times articles, and listeners infer different reasoning from each ([Ülkümen, Fox and Malle 2016](https://pubmed.ncbi.nlm.nih.gov/27442037/)). This is a tendency and not a rule, but it is the nearest empirical handle I found on *stance probability versus content probability*.\n\n### 1.7 Scope words are vague too\n\n- Many, few and most have fuzzy thresholds that depend on context ([Solt 2011](https://www.leibniz-zas.de/fileadmin/Archiv2019/mitarbeiter/solt/2011_solt_quantity.pdf)). People differ in where they put the threshold for *most* (about 50.5% to 62% across clusters in one study) ([Ramotowska et al. 2024](https://link.springer.com/article/10.3758/s13423-024-02502-7)).\n- Generics such as *tigers are striped* give no information about how many members have the property, and their truth conditions vary with context ([Stanford Encyclopedia, Generics](https://plato.stanford.edu/entries/generics/)).\n- Consequence: keeping scope words in the wording (as the ruling does) is right, since they say how much of a population is covered, but they will not turn into a resolvable test without an operationalisation (section 7, item 18).\n\n---\n\n## 2. Tetlock on vagueness, question precision and the rigor-relevance trade-off\n\nI did not read *Expert Political Judgment* in this session. What follows comes from Tetlock's later articles and talks.\n\n- Political debate, they say, is plagued by vague-verbiage predictions in which a forecaster says raising rates *may* trigger a recession, aid *could* provoke a response, or legislation *might* raise prices. If it happens, the forecaster says it was flagged as a distinct possibility; if it does not, they say they only said it was possible. Tetlock, Mellers and Scoblic argue that vagueness thereby blocks accountability, which blocks learning ([Science 2017](https://pdf.stafforini.com/tetlock-2017-bringing-probability-judgments.pdf)).\n- Tetlock describes high-status pundits as having learned to appear to go out on a limb without actually doing so, by attaching vague quantifiers to dramatic scenarios. He grants it can be a rational survival skill given a stochastic world, but says tournaments differ by requiring explicit probabilities on well-defined outcomes in well-defined time frames ([Edge Master Class](https://www.edge.org/node/26465)).\n- Tetlock's examples of statements that fail the clairvoyance test (defined next): that conflict in Yemen *may* increase in the next two weeks, or that Baghdad will get worse before it gets better. You cannot tell afterwards who was right ([Knowledge at Wharton](https://knowledge.wharton.upenn.edu/article/how-to-improve-the-science-of-forecasting/)).\n- The clairvoyance test: a question passes if a genuine clairvoyant could answer it without coming back for clarification. Their example pair: asking whether one will fall in love fails, while asking whether one will marry a named person within a year passes ([Scoblic and Tetlock, Foreign Affairs 2020](https://www.foreignaffairs.com/articles/united-states/2020-10-13/better-crystal-ball)). The test comes from Ronald Howard's clarity test in decision analysis (1988) ([Wikipedia: Clarity test](https://en.wikipedia.org/wiki/Clarity_test), secondary).\n- How often real forecasts fail it: 27% of the Canadian intelligence forecasts above could not be coded ([Mandel and Barnes 2018](https://doi.org/10.1002/bdm.2055)).\n- Rigor-relevance: rigorous questions are narrow, relevant questions are broad. Tetlock's answer is *Bayesian question clusters*: sets of narrow, resolvable indicator questions that together bear on a broad question. His example: *will increasing automation destabilise white-collar labour markets over 20 years?* is too ill-defined to resolve, so experts generate resolvable micro-questions such as driverless taxis in a major city by a date ([Science 2017](https://pdf.stafforini.com/tetlock-2017-bringing-probability-judgments.pdf)). The same idea is used commercially under the name question clusters ([Good Judgment](https://goodjudgment.com/question_clusters/)).\n- The Forecasting Research Institute formalised cluster building as *conditional trees*. Interviews with 21 AI experts and 3 superforecasters produced 75 resolvable questions, ranked by the value of information each gives about a far-future outcome. Questions resolving by 2030 were on average nine times more informative than the most-traded comparison questions on two platforms (p = .025). The authors note the method is labour-intensive and suggest LLMs might help ([FRI, August 2024](https://forecastingresearch.org/research/ai-conditional-trees)).\n- Tetlock also concedes that the specificity that makes questions resolvable rules out big questions ([Science 2017](https://pdf.stafforini.com/tetlock-2017-bringing-probability-judgments.pdf)).\n\n---\n\n## 3. Linguistics of hedging and epistemic modality\n\n### 3.1 What counts as a hedge\n\n- Lakoff (1972) introduced *hedge* for words that make things fuzzier or less fuzzy. The notion was later narrowed to devices that qualify a speaker's confidence in the truth of a proposition ([Hyland 1998 preview](https://api.pageplace.de/preview/DT0400.9789027282583_A24761077/preview-9789027282583_A24761077.pdf)).\n- Hyland defines a hedge as any means of showing either incomplete commitment to the truth of a proposition or a wish not to state that commitment categorically. He argues the main purpose is to get past the fact that any statement can be denied, and so win the reader's acceptance. He restricts the term to epistemic uses that actually mark uncertainty ([Hyland 1998 preview](https://api.pageplace.de/preview/DT0400.9789027282583_A24761077/preview-9789027282583_A24761077.pdf)).\n- Hyland's functional taxonomy (from the chapter list) divides hedges into content-oriented ones (accuracy-oriented: attribute hedges and reliability hedges; and writer-oriented hedges) and reader-oriented hedges ([Hyland 1998 preview](https://api.pageplace.de/preview/DT0400.9789027282583_A24761077/preview-9789027282583_A24761077.pdf)). In the usual summaries an attribute hedge specifies how precisely a term describes the phenomenon, a reliability hedge assesses how certain the proposition is, a writer-oriented hedge limits personal responsibility, and a reader-oriented hedge shows deference ([Livytska 2019](https://doi.org/10.2478/jolace-2019-0003), secondary).\n- Mapping to the ruling: reliability hedges are the *stance* Fredrik wants out of the wording; attribute hedges (*roughly*, *somewhat*, *about*) change the content's precision and belong in the wording; writer- and reader-oriented hedges are mostly politeness or self-protection and also belong in the stance record, not in the claim.\n- Hyland warns that epistemic and affective functions are often conveyed at once, which prevents clean descriptive categories ([Hyland 1998 preview](https://api.pageplace.de/preview/DT0400.9789027282583_A24761077/preview-9789027282583_A24761077.pdf)). Expect residual ambiguity.\n- Crompton (1997) argues *hedge* should be reserved for expressions of epistemic modality attached to the author's own new claims, since authors can only be held responsible for modalising their own propositions ([Crompton 1997 abstract](https://www.academia.edu/50160307/Hedging_in_academic_writing_Some_theoretical_problems)). His definition is an item the speaker uses to explicitly qualify their lack of commitment to the truth of a proposition, and his test asks: can the proposition be restated, unchanged, with greater commitment by the author? If so it is hedged ([as reported by Marta, secondary](https://www.diacronia.ro/ro/indexing/details/A28729/pdf)). Salager-Meyer criticised the test as adding little that is new ([same source](https://www.diacronia.ro/ro/indexing/details/A28729/pdf)). The test is useful but circular on its own, because deciding whether *the proposition* stayed the same is the very question to be answered.\n- Hyland's later model of stance and engagement treats hedges, boosters, attitude markers and self-mention as stance; stance is how writers comment on accuracy, commitment and attitude ([Hyland 2005 abstract](https://journals.sagepub.com/doi/10.1177/1461445605050365); the four-part list is from a [secondary summary](https://sophia.repo.nii.ac.jp/records/2020552)).\n- Hedges are frequent: about 18% of sentences in biomedical abstracts and 20% in full papers contain one (BioScope corpus, cited in [Tang et al., CoNLL-2010](https://aclanthology.org/W10-3002.pdf) via Vincze et al. 2008, secondary). They are almost three times as common in research articles as in textbooks of the same disciplines ([Hyland 2005, Prudence, precision and politeness](https://roderic.uv.es/rest/api/core/bitstreams/29ab6f89-f1cb-4819-8f08-d5499060efad/content)).\n\n### 3.2 Epistemic versus root modality and the tests linguists use\n\n- Formal semantics treats modals as quantifiers over possible worlds: possibility modals are existential, necessity modals universal. The kind of modality depends on the modal base. An epistemic modal quantifies over worlds compatible with what is known; circumstantial (dynamic) modality over worlds compatible with the circumstances; deontic over worlds satisfying rules ([von Fintel 2006](https://web.mit.edu/fintel/fintel-2006-modality.pdf)). Kratzer argues modals have a skeletal meaning completed by context, so the reading is largely fixed by context and individual words only lean one way ([von Fintel 2006](https://web.mit.edu/fintel/fintel-2006-modality.pdf)).\n- Practical tests:\n  - Paraphrase: epistemic *may* can be paraphrased by *it is possible that* or *it may be that*, root uses by *it is possible for* ([Neliti paper](https://media.neliti.com/media/publications/66988-EN-hedging-through-the-use-of-modal-auxilia.pdf)). The Canadian Department of Justice style guide makes the same split: *may* is epistemic, *can* is general possibility, and in legislation *may* mostly grants permission ([Justice Canada, Legistics](https://canada.justice.gc.ca/eng/rp-pr/csj-sjc/legis-redact/legistics/p1p10.html)).\n  - Scope: for epistemic possibility, negation, tense and hypothetical marking attach to the main predication (*they may not come* means it is possible that they will not come); for root possibility they affect the modal itself ([Huschová 2014](https://doi.org/10.5817/bse2014-1-5)).\n  - Aspect: the epistemic modal expresses a current state of mind while the verb describes an event with its own time and aspect, so perfect forms such as *may have* are a signal of the epistemic reading ([Huschová 2014](https://doi.org/10.5817/bse2014-1-5)).\n  - Subject and conditions: root possibility is tied to external enabling conditions or the subject's capacities ([Huschová, discourse corpus](https://journals.muni.cz/discourse-and-interaction/article/download/7025/8336/16264)).\n  - Source: FactBank's annotation uses a battery of discriminatory tests based on Horn's analysis of scalar modality and records whose view the factuality value represents ([Saurí and Pustejovsky 2012](https://aclanthology.org/J12-2002.pdf)).\n- Word-level tendencies. *May* has both readings, with epistemic the primary sense; *can* is mainly root and rarely epistemic ([Huschová 2014](https://doi.org/10.5817/bse2014-1-5), [Ruppenhofer and Rehbein 2012](https://aclanthology.org/L12-1458.pdf)).\n\n### 3.3 Where the distinction blurs\n\n- In formal and scientific writing, cautious epistemic possibility and root possibility presented as fact tend to merge ([Huschová 2014](https://doi.org/10.5817/bse2014-1-5)). In a corpus of linguistics articles root readings made up 61% of the possibility uses and even *may/might* were root in 31% ([Huschová, discourse corpus](https://journals.muni.cz/discourse-and-interaction/article/download/7025/8336/16264)). In academic text the lexical default is therefore unreliable.\n- Coates's *merger* category covers cases where both readings are compatible and need no resolving, such as *the time required may be considerable* ([REM annotation guide](https://www.degruyterbrill.com/publication/isbn/9783110734157/downloadAsset/9783110734157__03Annotation_Guide_REM_Project.pdf)).\n- Ruppenhofer and Rehbein found the epistemic reading of *could* ambiguous with an existential or dispositional reading. Their example: *a sun storm could cause 20 times more economic damage than Katrina* reports the storm's generic power more than it speculates about an ongoing event. They also report a scope ambiguity between the epistemic operator and an existential quantifier in *coups could be destabilizing* ([Ruppenhofer and Rehbein 2012](https://aclanthology.org/L12-1458.pdf)). Both are close cousins of *this policy could cut emissions*.\n- Agreement numbers: for *may/might* kappa 0.62 (89% raw agreement), for *can* 0.61 (77%) and for *must* 0.85, with a four-way Ability/Possibility/Epistemic/Deontic scheme abandoned because agreement was very poor ([Ruppenhofer and Rehbein 2012](https://aclanthology.org/L12-1458.pdf)). For *could* kappa was 0.20 to 0.26 against 98% raw agreement for *might* ([Moon, Kirvaitis and Madden 2016](https://aclanthology.org/2016.lilt-14.6.pdf)). In the MoVerb dataset (4,540 utterances, three annotators each) Krippendorff's alpha was 0.60 under Quirk's categories and 0.37 under Palmer's, with epistemic and dynamic uses often confused ([Owan et al. 2023](https://aclanthology.org/2023.conll-1.13.pdf)).\n\n### 3.4 Whose hedge is it?\n\n- FactBank records event factuality relative to a source. The author is the default source; reports, beliefs and impersonal constructions introduce others. In 208 documents (9,488 events, 13,506 event/source pairs), inter-annotator kappa was 0.81. Per-value F1 between annotators was 0.93 for certain-positive and 0.88 for unknown, but only 0.46 to 0.57 for probable and possible values ([Saurí and Pustejovsky 2012](https://aclanthology.org/J12-2002.pdf)). Humans separate *certain* from *not certain* far better than they grade the middle.\n- Crompton's point that authors can only be held responsible for hedging their own propositions ([abstract](https://www.academia.edu/50160307/Hedging_in_academic_writing_Some_theoretical_problems)) means a hedge inside a reported claim must attach to the reported speaker.\n\n### 3.5 Automatic detection and reported accuracy\n\n- CoNLL-2010 shared task (hedge detection in biomedical papers and Wikipedia), 23 teams. The best F1 for sentence-level uncertainty on biological text was 86.4 and on Wikipedia 60.2; the best scope-detection F1 was 57.3 ([Farkas et al. 2010](https://aclanthology.org/W10-3001.pdf)). The cue list in that task includes the auxiliaries *may, might, can, would, should, could*, hedging verbs, probability adjectives and adverbs, and even conjunctions such as *or* ([Farkas et al. 2010](https://aclanthology.org/W10-3001.pdf)). So the task treats every modal as a cue for an *uncertain sentence* and does not separate stance from content. Its scores therefore overstate how well the ruling's distinction could be learned from it.\n- A 2024 joint word-and-POS model reports a new top score on the CoNLL-2010 Wikipedia corpus; the HTML version gives 69.74 F1 ([Katerenchuk and Levitan 2024](https://arxiv.org/abs/2405.13319), number from the paper's HTML as shown in search highlights, not re-checked).\n- Certainty is more than hedge counting. Annotators rated 2,167 scientific findings (Krippendorff's alpha 0.67 for sentence-level certainty). The number of hedges correlated with rated certainty at only r = 0.55; a fine-tuned SciBERT reached r = 0.63, and aspect-level certainty (number, extent, probability, framing, condition, suggestion) reached average binary F1 0.66. Only under 10% of *condition* aspects were judged uncertain, contrary to earlier typologies ([Pei and Jurgens 2021](https://aclanthology.org/2021.emnlp-main.784.pdf)).\n- LLM era, spoken narratives. On a corpus of hedges in retold cartoon stories, fine-tuned BERT scored F1 0.91, few-shot GPT-4o 0.71 and zero-shot GPT-4o about 0.51 to 0.54. Models over-predicted hedges (mean precision 0.46, recall 0.65), and GPT-4o labelled disfluencies as hedges ([Paige et al. 2024](https://aclanthology.org/2024.sigdial-1.18/), [arXiv version](https://arxiv.org/html/2408.03319v1)). The domain is conversation, not claims.\n- Modal sense classification: RoBERTa fine-tuned on MoVerb reached F1 82.2 (Quirk scheme) and 78.3 (Palmer scheme) ([Owan et al. 2023](https://aclanthology.org/2023.conll-1.13.pdf)).\n- LLMs as readers of hedge words: seven of ten models mapped uncertainty expressions to numbers in a human-like way, but their numbers shifted depending on whether the statement was actually true or false, i.e. on their own prior knowledge ([Belem et al. 2024, via an abstract aggregator](https://www.emergentmind.com/papers/2407.15814); arXiv 2407.15814). An LLM extractor can therefore leak its own belief about the claim into the stance value it assigns.\n- LLMs as writers of hedges: deployed models were reluctant to express uncertainty, and among confident-sounding answers the average error rate was 47% ([Zhou et al. 2024](https://doi.org/10.18653/v1/2024.acl-long.198)).\n\n### 3.6 Gap\n\nI found no dataset or shared task that labels exactly the ruling's distinction: author-held epistemic stance versus modality that is part of the content. The closest are CoNLL-2010 and BioScope (uncertain sentence versus factual), MoVerb and the Ruppenhofer corpus (modal sense), FactBank (source-relative factuality) and Pei and Jurgens (certainty level and aspect). This is an absence from my search, not proof that none exists.\n\n---\n\n## 4. Falsifiability, bold versus hedged claims, and bets as tests\n\n### 4.1 Popper\n\n- Popper counts a good theory as a prohibition: it forbids certain events, and the more it forbids, the better. A theory no conceivable event could refute is not scientific. Astrologers escaped refutation by making prophecies vague enough to explain away any outcome, which Popper treats as a soothsayer's device for making a prediction unfailable. The Marxist and psychoanalytic cases he treats as theories rescued by reinterpretation ([Popper, *Conjectures and Refutations* ch. 1](https://www.stephenhicks.org/wp-content/uploads/2024/02/PopperK-Science-as-Falsification-text-full.pdf)).\n- Informative content and probability vary inversely, so bold, improbable theories are the more valuable; informative content is in proportion to testability ([SEP, Popper](https://plato.stanford.edu/entries/popper/)). A hedged claim says less, is more probable, and is harder to refute. That is the philosophical core of the ruling: the firm proposition is the testable one, and the confidence belongs elsewhere.\n- Popper's treatment of existential statements matters for possibility-as-content. Pure existential claims (*unicorns exist*) cannot be shown false by failing to find an instance ([IEP, Popper](https://iep.utm.edu/pop-sci/)), while laws are prohibitions that can be refuted by one counter-instance ([Popper, LSD §15 excerpt](http://www.theopensociety.net/2014/12/falsifying-universal-statements/)). A *could/can* claim read as *there is a circumstance in which it works* is existential in this sense.\n\n### 4.2 Lakatos\n\n- Lakatos replaces the single falsifiable theory with a research programme: a hard core protected by a belt of auxiliary hypotheses that absorbs refutations. A programme is progressive if each new version predicts novel facts, some of which are corroborated, and degenerating if not ([SEP, Lakatos](https://plato.stanford.edu/entries/lakatos/)). Relevance to hedges: a hedge can be a protective-belt device, and what matters is whether it is added after a failure (degenerating) or specifies new testable content (progressive). That is a graph-level question over time, which fits Deliberus's idea of strength computed from structure over time.\n\n### 4.3 Bets and betting institutions\n\n- Tabarrok argues that a bet works as a levy on empty talk: it costs the bluffer more than the honest believer, so willingness to bet signals honest belief, and a properly structured bet is the best guarantee of disinterest ([Marginal Revolution, 2012](https://marginalrevolution.com/marginalrevolution/2012/11/a-bet-is-a-tax-on-bullshit.html)).\n- Hanson's idea futures: an author carefully words a set of claims; a judge agrees in advance to rule on them. He says clear-cut claims and judging criteria that leave little room for discretion help keep judges honest, and that conjunctive and conditional assets let traders bet on dependencies ([Hanson, Could gambling save science?](https://mason.gmu.edu/~rhanson/gamble.html), [draft](http://li.mit.edu/Stuff/CNSE/Paper/Hanson90.pdf); the journal version is dated 1995 in [Social Epistemology](https://doi.org/10.1080/02691729508578768)).\n- Long Bets: even odds and a win/lose result; the foundation says forcing a win/lose outcome helps force lucidity in the terms. Bettors may negotiate the judgment criteria, there is a two-year minimum and a mediator reviews wording; the foundation adjudicates disagreements ([Long Bets procedure](https://longbets.org/procedure/), [rules, archived](https://web.archive.org/web/20060207113852/www.longbets.org/rules)).\n- Rootclaim lists each analysis as a conclusion paired with a probability, for example 94% for a lab leak, and offers $100,000 challenges decided by two judges who each rule on which hypothesis is more likely. Both sides put money in escrow ([Rootclaim challenge](https://www.rootclaim.com/rootclaim-challenge)). In the 2023–24 COVID-origins debate both judges independently found for the natural-origin side, and Rootclaim paid ([Rootclaim blog, Feb 2024](https://blog.rootclaim.com/rootclaims-covid-19-origins-debate-results/), [judge Stansifer's decision](https://ermsta.com/r/covid_decision_20240603.pdf)). Rootclaim blames its debate structure, notably a live format that favoured memorised detail and no ongoing judge feedback, and says probabilistic inference is hard to translate into human language ([Rootclaim blog](https://blog.rootclaim.com/rootclaims-covid-19-origins-debate-results/)). The relevant points: the bet is on a comparison of hypotheses judged by humans, not on an observable event, and the format determined who won.\n\n### 4.4 Replication markets as an operationalisation example\n\n- Prediction markets on replications predicted outcomes correctly 73% of the time across 103 findings; surveys 66%; beliefs ran about ten percentage points above the observed replication rate ([Pooled analysis, PMC8046229](https://pmc.ncbi.nlm.nih.gov/articles/PMC8046229/)).\n- In DARPA SCORE the tested claim is operationalised in advance: success means an effect in the same direction as the original and statistically significant at p < 0.05, in a direct or data-analytic replication; claims are extracted by the Center for Open Science first, then forecasters estimate the probability ([Gordon et al. 2020](https://royalsocietypublishing.org/rsos/article/7/7/200566/95673/Are-replication-rates-the-same-across-academic), [Replication Markets paper](https://ar5iv.labs.arxiv.org/html/2005.04543)). About 5% of the claims will eventually be assessed by replication ([Gordon et al. 2020](https://royalsocietypublishing.org/rsos/article/7/7/200566/95673/Are-replication-rates-the-same-across-academic)), so most forecasts are never scored directly, which is why the Replication Markets team added decision markets and surrogate scoring ([Replication Markets paper](https://ar5iv.labs.arxiv.org/html/2005.04543)).\n\n### 4.5 What makes a claim bet-able\n\n- Metaculus: questions should concern tangible, well-defined facts, not opinion; a good question is unambiguously resolvable, meaning a community reading the terms can agree before and after the event whether the outcome meets them. Only the resolution criteria decide resolution, not the background text. If the criteria do not anticipate an outcome, the question may be annulled or resolved ambiguous, and editing a live question in ways that shift meaning can unfairly change scores ([Metaculus FAQ](https://www.metaculus.com/faq/)).\n- The community Forecasting Wiki checklist adds: compare the resolution criteria to the title for cases where they would disagree, name a fallback if the data source disappears, and aim for binary questions whose probability is likely between 10% and 90% ([Forecasting Wiki](https://forecasting.wiki/wiki/How_to_Write_Forecasting_Questions)).\n- Failure cases show how wording becomes the dispute. Polymarket's market on whether Zelenskyy would be photographed wearing a suit relied on a consensus of credible reporting without defining a suit, and the dispute went to the UMA oracle ([Next Event Horizon, reproducing the market rules](https://nexteventhorizon.substack.com/p/the-debacle-that-is-polymarket)). A community guide cites a market on whether Biden was senile that resolved 50/50 for lack of objective criteria ([PolymarketGuide, secondary](https://polymarketguide.gitbook.io/polymarketguide-archive/precedents/polymarket/bad-market-descriptions)). In a 2022 TIME Person of the Year market, a clarifying clause using the word *exclusively* conflicted with the main rule and had to be argued out ([UMA, Medium](https://medium.com/uma-project/2022-time-person-of-the-year-how-uma-handled-a-polymarket-dispute-edge-case-941da7200250)). Reported market volumes and governance accusations for the suit case come from secondary sources and I have not checked them.\n- Existential and possibility claims: a one-sided bet is possible (one documented instance verifies) but a null result never refutes (section 4.1). Kent's original advice, to use *possible* for important contingencies whose odds cannot be estimated, is the same point from the intelligence side ([Kent 1964](https://www.cia.gov/resources/csi/static/Words-of-Estimative-Probability.pdf)).\n\n---\n\n## 5. What is lost when hedges are removed\n\n### 5.1 Calibration information\n\n- Verbal probabilities in intelligence reports, once mapped to numbers, were well calibrated though underconfident, and discrimination was very good ([Mandel and Barnes 2018](https://doi.org/10.1002/bdm.2055)). The hedge word carries a real, scoreable signal about the author.\n- Coarsening numeric forecasts loses accuracy ([Friedman et al. NBER draft](https://conference.nber.org/confer/2015/SI2015/PENS/Friedman_Baker_Mellers_Tetlock_Zeckhauser.pdf)).\n- If stance is dropped when a claim is extracted, no one can later score the author's calibration against the resolved claim.\n\n### 5.2 Social function and politeness\n\n- Hedges show deference and collegiality as well as doubt; Hyland found they work to confirm the writer's professional persona and to gain acceptance, and hedges are about three times as common in articles as in textbooks, where claims are already accepted ([Hyland 1996 abstract](https://www.academia.edu/7567241/Writing_without_conviction_Hedging_in_scientific_research_articles), [Hyland 2005](https://roderic.uv.es/rest/api/core/bitstreams/29ab6f89-f1cb-4819-8f08-d5499060efad/content)). Removing them from the displayed text changes how a quoted author sounds, which is another reason to show the original span next to any claim attributed to a person.\n\n### 5.3 Hedges are dropped in transmission, and exaggeration follows\n\n- Fahnestock paired scientific reports with popular accounts and found that the accounts remove hedges and qualifications and so confer greater certainty ([Fahnestock 1986](https://journals.sagepub.com/doi/10.1177/0741088386003003001)). Latour and Woolgar's five-step scale places a plain statement with no modality at the top, as an accepted fact ([Wikipedia summary of *Laboratory Life*](https://en.wikipedia.org/wiki/Laboratory_Life), secondary).\n- Sumner et al. compared 462 UK university press releases and 668 news stories with the underlying papers. Forty percent of press releases gave more explicit advice than the paper, 33% made stronger causal claims from correlational data and 36% inflated inference from animals to humans. News carried the same exaggeration in 58%, 81% and 86% of cases when the press release did, against 17%, 18% and 10% when it did not (odds ratios 6.5, 20, 56). The study is correlational ([Sumner et al. 2014](https://pmc.ncbi.nlm.nih.gov/articles/PMC4262123/)).\n- Peters and Chin-Yee tested 10 LLMs on 4,900 summaries of science. LLM summaries were about twice as likely to contain generalised conclusions as the originals; DeepSeek, ChatGPT-4o and LLaMA 3.3 70B overgeneralised in 26 to 73% of cases; newer models did worse; the Claude models tested had the lowest rates (−1 to 20%); prompting for accuracy made it about twice as likely (odds ratio 1.9); temperature 0 cut the rate by 76% ([Peters and Chin-Yee 2025](https://royalsocietypublishing.org/rsos/article/12/4/241776/235656/Generalization-bias-in-large-language-model)). The models tested were those of early 2025.\n\n### 5.4 Reader trust\n\n- Experiment with 601 participants: scientists and journalists were seen as more trustworthy when cancer-news coverage was hedged, and most when the hedging was attributed to the scientists responsible ([Jensen 2008](https://doi.org/10.1111/j.1468-2958.2008.00324.x)).\n- Five experiments (n = 5,780): communicating uncertainty made people perceive more uncertainty but cost little trust. Trust in the numbers fell by an effect size of about 0.55 for explicit verbal uncertainty against 0.15 for numerical ranges, and trust in the source fell 0.21 for verbal against 0.03 for numeric. The authors recommend a numerical range with a point estimate ([van der Bles et al. 2020](https://www.pnas.org/doi/10.1073/pnas.1913678117)).\n- Consequence for display: if a hedge is shown at all, a coarse numeric or banded stance is less costly in trust than a verbal softener.\n\n### 5.5 Charity and the straw man\n\n- The principle of charity asks for interpretation that maximises the truth or rationality of what is said ([Routledge Encyclopedia](https://www.rep.routledge.com/articles/thematic/charity-principle-of)).\n- Walton's analysis of the straw man says that to decide what someone is committed to you must look at the thesis they argued for, how they qualified their support for it, and what they said in detail; charge of fallacy is relative to that evidence ([Walton, The straw man fallacy](https://dwc.knaw.nl/DL/publications/PU00010685.pdf)). So dropping the author's qualifier and then attacking the stronger claim is the textbook shape of the fallacy.\n- Argument-mapping practice agrees. Kialo's educator guidance says students often mistake strongly worded arguments for strong ones, and advises qualifiers such as *may, might, sometimes, often* so a claim is not deflated by a single counterexample; it also warns that misrepresenting viewpoints leads to straw-manning ([Kialo Edu blog](https://blog.kialo-edu.com/tips-and-tricks/how-to-hold-a-successful-class-discussion-on-kialo-edu/)). That guidance does not distinguish stance hedges from scope words, which is the distinction the ruling makes.\n\n### 5.6 Loss of qualifiers in claim extraction\n\n- Claimify's principles require every extracted claim to be entailed by the source, understandable alone and not to omit critical context. Its example: dropping a condition on the World Trade Organization's trade-barrier support turns a true claim into a false-looking one. It flags ambiguity it cannot resolve, and puts bracketed text around inferred content, noting that inferred content is less reliable ([Metropolitansky and Larson 2025](https://arxiv.org/html/2502.10855v2), [Microsoft blog](https://www.microsoft.com/en-us/research/blog/claimify-extracting-high-quality-claims-from-language-model-outputs/)).\n- Decomposition errors in fact-checking include omission of context and alteration of meaning ([Hu et al. 2025](https://arxiv.org/html/2411.02400v2)); decontextualisation reintroduces non-atomic claims ([Wanner et al. 2024](https://arxiv.org/html/2412.13175)). A recent preprint argues for attaching qualifiers to a base triple, as Wikidata does, instead of folding them into the claim ([TriQua, preprint, not peer reviewed](https://arxiv.org/html/2608.05228v1)).\n- Under the ruling the extracted proposition is deliberately stronger than the sentence. A source saying X *may* cause Y does not entail the bare claim that X causes Y, so Claimify's entailment test would fail the node on its own; it holds only for the assertion record *this author asserts, with stance S, that X causes Y*. This is why the unit shown to users should be the assertion record (section 7, item 1).\n\n---\n\n## 6. Precedents that separate a proposition from a speaker's confidence\n\n- **Toulmin.** Claim, data, warrant, backing, qualifier and rebuttal; the qualifier (*presumably*, *probably*, *necessarily*) states the force the data confer on the claim under the warrant and is a separate slot ([Uses of Argument excerpt](https://iphils.uj.edu.pl/~a.trybus/classes/ct/The.Uses.of.Argument.Excerpt.pdf)). One commentator stresses that the qualifier should be read as the trace of substantive rebuttals and not a mere softener ([Plantin](https://icar.cnrs.fr/dicoplantin/layout-of-argument-toulmin-e/)). Toulmin scholars also argue that the qualifier *probably* is a guarded commitment that numbers cannot fully capture ([van Eemeren et al., summarising Ennis](https://link.springer.com/rwe/10.1007/978-94-007-6883-3_4-1)).\n- **Nanopublications.** An assertion graph, a provenance graph (how, who, from where) and a publication-info graph; provenance includes attribution ([nanopub.net guidelines](https://nanopub.net/guidelines/working_draft/)). The original design adds an *assertedBy* relation so users can choose which accounts of the same statement to trust ([Groth, Gibson and Velterop 2010](https://doi.org/10.3233/isu-2010-0613)). The guidelines I read do not define a slot for the author's confidence, so confidence would live in provenance or annotation.\n- **Wikidata.** Statements carry qualifiers, references and ranks. Two qualifiers hold weakeners: *sourcing circumstances* (values such as presumably, probably, allegedly, hypothetically, disputed, circa) and *nature of statement* (values such as hypothesis, often, allegedly, rarely, mainly) ([P1480](https://www.wikidata.org/wiki/Property:P1480), [P5102](http://wikidata.org/wiki/Property:P5102)). A 2024 study of weaker-logical-status claims found overlapping terms and arbitrary choice among contributors, and that a qualified statement at normal rank seems to imply it is asserted ([Di Pasquale et al. 2024](https://doi.org/10.3233/sw-243686)). Note that *nature of statement* lumps frequency words (*often, rarely, mainly*) with epistemic ones (*hypothesis, allegedly*), which is the mix-up the ruling wants to avoid.\n- **FactBank.** Factuality is a pair of modality (certain, probable, possible) and polarity, assigned per source, with the author as source zero ([Saurí and Pustejovsky 2012](https://aclanthology.org/J12-2002.pdf)).\n- **Intelligence and climate standards.** Separate likelihood and confidence scales (section 1.6).\n- **Carneades.** A formal argument model whose proof standards (scintilla of evidence, best argument, dialectical validity) are assigned per statement and issue, with burden of proof allocated separately for production and persuasion ([Gordon, Prakken and Walton 2007](https://doi.org/10.1016/j.artint.2007.04.010)). Proof standard belongs to the evaluation context, not the speaker.\n- **Kialo.** Claims should make a single argument; reviewers can mark a claim as Unclear or More than one claim ([Kialo Edu help](https://support.kialo-edu.com/en/hc/creating-a-claim/), [marks](https://support.kialo-edu.com/en/hc/types-of-mark/)). Impact voting is described as separating agreement with a claim from a judgment of how good the argument is ([third-party explainer, not Kialo's own page](https://vectree.io/c/kialo-edu)). A commenter notes the impact vote is too limited to say how relevant a sub-claim is to its grandparent ([LessWrong thread](https://www.greaterwrong.com/posts/g3odvaj8opqCF9egv/kialo-an-online-discussion-platform-that-attempts-to-support)).\n- **Society Library.** Debate maps built by AI agents that split a structured claim into atomic components and evaluate each for clarity, specificity and fact-checkability, then link quoted evidence with stance ([Society Library, June 2025](https://societylibrary.medium.com/weve-just-made-our-maps-smarter-deeper-and-stronger-with-ai-agents-4626ae561833)).\n- **Rootclaim.** Conclusion and probability as separate columns ([Rootclaim challenge](https://www.rootclaim.com/rootclaim-challenge)).\n- **Forecast platforms.** The question text and resolution criteria are one object; forecasters attach probabilities to it separately; and edits to a live question are treated as a fairness problem because they can shift scores ([Metaculus FAQ](https://www.metaculus.com/faq/)).\n- **Claim operationalisation tooling.** FRI turned expert-named indicators into 75 objectively resolvable questions ([FRI](https://forecastingresearch.org/research/ai-conditional-trees)); SCORE fixed what replication success means before forecasting ([Gordon et al. 2020](https://royalsocietypublishing.org/rsos/article/7/7/200566/95673/Are-replication-rates-the-same-across-academic)); Metaculus uses separate description, resolution criteria and fine print fields ([Metaculus question types](https://metaculus-metaculus.mintlify.app/guides/question-types), third-party documentation).\n\n---\n\n## 7. What this suggests for Deliberus\n\nThe numbered items are my inferences from the sources above; they are proposals, not findings. Items are grouped by the three questions in the brief, then validation.\n\n### A. Data model\n\n1. **Make the shown unit *assertion = (claim, holder, stance, source span)*.** The node holds the firm proposition P. A source links to P through an assertion record that stores the holder (author or quoted speaker), the stance type and band, the cue words, and the verbatim span. In every card, summary, export and agent answer, an attribution renders as *holder, with stance, asserts P*, never as *holder says P*. This is the nanopublication split (assertion versus provenance) plus FactBank's per-source factuality, and it resolves the entailment problem noted in 5.6: the source text entails the assertion record, not the bare P. ([nanopub.net](https://nanopub.net/guidelines/working_draft/), [FactBank](https://aclanthology.org/J12-2002.pdf), [Claimify](https://arxiv.org/html/2502.10855v2))\n2. **Type the stance and give it a coarse band.** Suggested types: author's epistemic credence (*probably, I'd guess, I'm fairly sure*), evidential or inferential (*the data suggest, reportedly, according to*), and reader-oriented or politeness hedge (*one might argue*). Suggested band: firm, high, middling, low, plus *unquantified*. Three levels is already near the limit of what human annotators separate (FactBank's probable/possible F1 is only 0.46 to 0.57), so do not convert words to numbers by default; store the original cue words and let a number appear only when the author gave one. ([FactBank](https://aclanthology.org/J12-2002.pdf), [Wintle et al.](https://pmc.ncbi.nlm.nih.gov/articles/PMC6469752/), [Hyland](https://api.pageplace.de/preview/DT0400.9789027282583_A24761077/preview-9789027282583_A24761077.pdf))\n3. **Add a content-modality type to P itself.** When a modal stays in the wording, tag its kind so downstream code knows how to test it: ability or feasibility (*can/could do*), dispositional or generic (*tends to*), deontic (*should, may* as permission), or approximator (*roughly*). ([von Fintel](https://web.mit.edu/fintel/fintel-2006-modality.pdf), [Hyland taxonomy](https://api.pageplace.de/preview/DT0400.9789027282583_A24761077/preview-9789027282583_A24761077.pdf))\n\n### B. Telling stance from content reliably\n\n4. **Decide in steps, not in one classification.** Step 1: find cues (modals, adverbs, verbs like *suggest*, first-person *I think/I'd guess*, approximators, evidentials). Step 2: for each cue, answer explicit questions with evidence from the sentence: (a) whose view is it (author, quoted speaker, generic)? (b) does the paraphrase *it is possible that S* hold, or only *it is possible for [subject] to V*? (c) is a perfect or past reading available (*may have V-ed*), which signals epistemic use? (d) does the modal answer *how sure is the speaker?* (epistemic base) or *what do the circumstances allow?* (circumstantial base)? (e) if the cue is deleted, does the proposition's truth condition change, or only the author's commitment to it (Crompton's test)? ([Justice Canada](https://canada.justice.gc.ca/eng/rp-pr/csj-sjc/legis-redact/legistics/p1p10.html), [Huschová](https://doi.org/10.5817/bse2014-1-5), [von Fintel](https://web.mit.edu/fintel/fintel-2006-modality.pdf), [Crompton via Marta](https://www.diacronia.ro/ro/indexing/details/A28729/pdf)) No single test is reliable; agreement among them is the signal.\n5. **Output three classes: stance, content, ambiguous.** Treat *ambiguous* as a first-class result and keep the cue in the wording for it, flagged *stance cue unresolved*. Reason: the two errors are asymmetric. Wrongly stripping a content modal produces a stronger claim that a named person never made and that a single counterexample can sink; wrongly keeping a stance hedge produces only a vaguer claim. Human disagreement on the underlying distinction is large for some cues (*could* kappa 0.20 to 0.26), so an ambiguous bin will not be rare. ([Moon et al.](https://aclanthology.org/2016.lilt-14.6.pdf), [Walton](https://dwc.knaw.nl/DL/publications/PU00010685.pdf))\n6. **Use cue-specific priors, but never lexical defaults alone.** *Might/may* lean epistemic and *can/could* lean root, but in academic text root readings of *may/might* are common, and in legal or policy text *may* often means permission. Give the classifier the register (legal, scientific, newspaper, forum) as an input. ([Ruppenhofer and Rehbein](https://aclanthology.org/L12-1458.pdf), [Huschová](https://journals.muni.cz/discourse-and-interaction/article/download/7025/8336/16264), [Justice Canada](https://canada.justice.gc.ca/eng/rp-pr/csj-sjc/legis-redact/legistics/p1p10.html))\n7. **Treat conditional-policy *could/might* as content by default.** *If X were done, Y might happen* and *this policy could cut emissions* usually state what the policy makes possible, or are generic claims about its power, not the author's credence in a specific outcome. The UK standard itself discourages the bare form because it hides the judgment. Store it as a feasibility claim and, where the source allows, ask the user to supply the missing condition and strength. ([Ruppenhofer and Rehbein](https://aclanthology.org/L12-1458.pdf), [PHIA](https://www.gov.uk/government/publications/explaining-uncertainty-in-uk-intelligence-assessment/explaining-uncertainty-in-uk-intelligence-assessment))\n8. **Keep degree and frequency words out of the stance field, and keep scope words separate from epistemic words.** Wikidata's lumping of *often/rarely/mainly* with *allegedly/hypothesis* caused overlapping terms and arbitrary labelling; a closed lexicon of scope words (*some, most, often, tend to, roughly*) checked before the stance step avoids the same mistake. ([Di Pasquale et al.](https://doi.org/10.3233/sw-243686), [Hyland](https://api.pageplace.de/preview/DT0400.9789027282583_A24761077/preview-9789027282583_A24761077.pdf))\n9. **Do not let the extractor's belief about the claim set the stance value.** LLMs map hedge words to different numbers depending on whether they themselves think the statement is true. Extract the stance blind to the claim's truth, for instance by masking the content or running the stance step on the sentence with the proposition abstracted. ([Belem et al. 2024, aggregator](https://www.emergentmind.com/papers/2407.15814))\n\n### C. Keeping the author's hedged use visible\n\n10. **Round-trip check against strengthening.** After extracting (P, stance), regenerate the sentence from the stored stance and compare with the original span using a two-way entailment model. Fail the extraction if the regenerated text is stronger than the source, or if P stripped of its stance is entailed by the source while the stance is not firm. This is Claimify's entailment criterion applied to the assertion record. ([Claimify](https://arxiv.org/html/2502.10855v2))\n11. **Run a generalisation detector on every LLM rewrite.** The pipeline step that rewrites each claim so a stranger understands it is exactly where LLM summarisation overgeneralises: tense, genericity and action-guiding force drift. Compare those three features between the source span and the rewritten claim, use temperature 0, and do not rely on telling the model to be accurate. ([Peters and Chin-Yee](https://royalsocietypublishing.org/rsos/article/12/4/241776/235656/Generalization-bias-in-large-language-model))\n12. **Mark editor-supplied restoration.** Where the system adds context to make a claim stand alone, show it in brackets, as Claimify does, so the reader can tell the author's words from the system's. ([Claimify](https://arxiv.org/html/2502.10855v2))\n13. **Attach the stance wherever the author's name travels.** Counterexamples should be scored against the right target: a single counterexample defeats a universal P, but only the author's stance-qualified assertion was ever made. Show a stance chip beside every attributed use, and keep the verbatim span one click away. This also keeps the deference and persona signals that Hyland describes. ([Kialo Edu blog](https://blog.kialo-edu.com/tips-and-tricks/how-to-hold-a-successful-class-discussion-on-kialo-edu/), [Walton](https://dwc.knaw.nl/DL/publications/PU00010685.pdf), [Jensen 2008](https://doi.org/10.1111/j.1468-2958.2008.00324.x))\n14. **Keep the stance records usable for author calibration.** Verbal hedges are informative once mapped (calibrated but underconfident in intelligence forecasts). When a claim with a resolution condition resolves, the stored band lets the platform show how well-calibrated an author's *probably* was, which is lost if stance is discarded. ([Mandel and Barnes](https://doi.org/10.1002/bdm.2055))\n\n### D. A testable form alongside the wording\n\n15. **Revisit the carve-out for stated probabilities.** Forecast platforms store the event as the proposition and the number as the forecaster's credence, because a single probabilistic outcome cannot score a statement that *the probability is 70%*. Under the ruling as worded, *there is a 70% chance that E* stays inside the claim. I suggest two cases: (a) if the probability describes a chance process or rate (*the vaccine is 95% effective*, *a 3% annual failure rate*) it is content; (b) if it is the author's credence about a one-off event (*I'd put 70% on E*, *E is probably going to happen*) the number moves to the stance field and E becomes the resolvable proposition. Also note an asymmetry in the ruling: *probably E* is moved out as stance while *70% chance of E* is kept in, though IC lexicons treat them as the same thing at different precision. Ülkümen et al. give phrasing cues (confidence versus likelihood language) but only as tendencies. Please decide which way you want this. ([Tetlock et al. 2017](https://pdf.stafforini.com/tetlock-2017-bringing-probability-judgments.pdf), [Ülkümen et al.](https://pubmed.ncbi.nlm.nih.gov/27442037/), [ICD 203](https://www.odni.gov/files/documents/ICD/ICD-203.pdf))\n16. **Add a resolution-condition block to each claim.** Fields: testability class (resolvable, resolvable after operationalisation, existential or feasibility, normative, definitional); the resolution condition in one or two sentences; the observation window or deadline; the named resolution source; operationalisations for each vague term; what happens if the source disappears or the outcome falls outside the criteria (ambiguous or annulled); and the question form (binary, number, date). This follows the structure of Metaculus questions and the fine-print and fallback advice in the Forecasting Wiki. ([Metaculus FAQ](https://www.metaculus.com/faq/), [Forecasting Wiki](https://forecasting.wiki/wiki/How_to_Write_Forecasting_Questions))\n17. **Use the clairvoyance test as the gate, and keep its output.** Ask an LLM acting as an omniscient oracle to answer the claim, and list every clarifying question it would need to ask. Pass if the list is empty. If not, the list is the to-do list for operationalisation. About a quarter of real intelligence forecasts failed this test, so expect many extracted claims to fail it. ([Foreign Affairs](https://www.foreignaffairs.com/articles/united-states/2020-10-13/better-crystal-ball), [Mandel and Barnes](https://doi.org/10.1002/bdm.2055))\n18. **Store operationalisations as proposals, separate from the author's words.** For scope words the system can propose thresholds (*most* as above 50%; tested thresholds vary from about 50% to 62% across participant clusters), but mark them as editorial and show the alternatives; never pick silently. Version the resolution condition once bets attach to it, since Metaculus treats edits to live questions as a fairness problem. The Polymarket suit market is the cautionary case. ([Ramotowska et al.](https://link.springer.com/article/10.3758/s13423-024-02502-7), [Metaculus FAQ](https://www.metaculus.com/faq/), [Next Event Horizon](https://nexteventhorizon.substack.com/p/the-debacle-that-is-polymarket))\n19. **Give possibility and feasibility claims their own testable forms.** A claim of the form *X can/could produce Y* is existential: one documented case verifies it, no finite search refutes it. Offer two derived forms: a demonstration form (at least one documented case meeting an explicit threshold by a date) and a rate or conditional form (in at least k of n comparable cases; or the probability of a demonstration by a date). Do not let a null result move the claim's strength symmetrically. Kent's rule that *possible* stays unmodified, for contingencies whose odds cannot be estimated, fits the existential type. ([IEP](https://iep.utm.edu/pop-sci/), [Kent](https://www.cia.gov/resources/csi/static/Words-of-Estimative-Probability.pdf))\n20. **Decompose broad claims into indicator questions with conditional edges.** Tetlock's question clusters and FRI's conditional trees give a pattern: each narrow, resolvable child carries a conditional probability for the parent (P(parent given child yes/no)), and children can be ranked by value of information. In Deliberus terms, support and attack edges between a resolvable child and a broad parent could carry such conditionals. ([Tetlock et al. 2017](https://pdf.stafforini.com/tetlock-2017-bringing-probability-judgments.pdf), [FRI](https://forecastingresearch.org/research/ai-conditional-trees))\n21. **Offer *make this a bet* only after the gate.** Require a pass on the clairvoyance test, a named source, a deadline and a resolution condition. Consider Long Bets' even-odds win/lose rule for contested claims, and Rootclaim's lesson that if the resolver is a person, the format matters. ([Long Bets](https://longbets.org/procedure/), [Rootclaim](https://blog.rootclaim.com/rootclaims-covid-19-origins-debate-results/))\n\n### E. Validation\n\n22. **Build a labelled set from the real corpus, and measure the two errors separately.** Sample sentences with the target cues from Deliberus's own sources, label each with at least three raters using the step questions from item 4, and include an *ambiguous* class. Report agreement per cue (benchmarks above: *might* about 98% raw, *could* kappa 0.2, *may/might* kappa about 0.6), and the false-strip rate (a content modal wrongly moved to stance) apart from the false-keep rate. Set the acceptance bar on false strips. ([Ruppenhofer and Rehbein](https://aclanthology.org/L12-1458.pdf), [Moon et al.](https://aclanthology.org/2016.lilt-14.6.pdf), [Owan et al.](https://aclanthology.org/2023.conll-1.13.pdf))\n23. **Compare a small fine-tuned encoder against LLM prompting.** In one hedge-detection study a fine-tuned BERT clearly beat few-shot GPT-4o, and LLMs over-predicted hedges. Test both, and also test the LLM's step-wise rationale against the encoder's call. ([Paige et al. 2024](https://aclanthology.org/2024.sigdial-1.18/))\n24. **Audit periodically.** Run the round-trip and generalisation checks (items 10 and 11) on a monthly sample of live claims, since extractor models change.\n\n### Questions for Fredrik\n\n- Do you want the carve-out for stated probabilities to stay as ruled, or follow the credence-versus-chance split in item 15?\n- Should the stance field hold a band, or numbers when authors give them, or both?\n- Should feasibility claims (*could*, *can*) be a visible claim type with their own testable forms (item 19)?\n- Do you want authors to be scoreable on their stance bands when claims resolve (item 14)?\n\n---\n\n## 8. What I could not verify or did not read\n\n- Tetlock's *Expert Political Judgment* and *Superforecasting* themselves; I relied on his later papers, talks and interviews.\n- The NATO doctrine AJP-2.1. The NATO ranges come from Mandel and Irwin's reproduction of the 2016 standard.\n- The Mauboussin figures at the primary: the HBR page was paywalled and the numbers come from a blog summary; I did not check probabilitysurvey.com.\n- The published version of Friedman et al.; I read the 2015 NBER draft. The 0.08 to 0.59 range comes from the Science paper's reference 1, which I did not identify. The Bay of Pigs and bin Laden stories are second-hand.\n- Barnes (2015/2016) on Canadian use of numeric probabilities, and Dhami and Mandel (2021), which I only met in citations.\n- Crompton's paper and the criticism of it: both reached me through secondary sources (Marta; abstract only).\n- Hyland: I read the preface, chapter list and opening of the 1998 book, the 1996 abstract and the 2005 paper only as search-tool excerpts of the repository copy, not the full texts.\n- Latour and Woolgar: read through summaries only.\n- Kialo's own help page on impact voting; the description is from a third-party explainer. Kialo's claim guidance and mark types I read at Kialo.\n- Belem et al. 2024 (LLM mapping of uncertainty words): read via an abstract aggregator; Katerenchuk and Levitan's 69.74 F1 comes from search-highlighted text of the HTML version.\n- The Polymarket suit-market volume, vote-concentration and governance claims (secondary sources); only the market rule text and the dispute outline are reasonably documented.\n- Vincze et al. 2008 hedge frequency figures (via Tang et al.).\n- I found no dataset for the ruling's exact distinction, and I did not search exhaustively for 2025–26 LLM benchmarks on stance versus content modality.\n- Links: I checked every URL by script on 2026-10-20. About twenty answered 403, 405 or 203 to the script (publisher and resolver bot-blocking: doi.org, Sage, PNAS, Royal Society, odni.gov, Metaculus, Medium, ResearchGate, Academia, Wharton, PubMed). I did not re-open each of them by another route, so treat those links as unconfirmed rather than broken. The Hyland 2005 DOI redirect returned 404 to the script, so that source is linked at the university repository copy instead.\n"}