{"path":"research/retrieval-instruments-beyond-cosine.md","content":"# The Right Instrument for Finding Related Claims\n\n**Date**: 2026-08-17\n**Status**: Literature findings. One of them bears directly on an open founder decision and may remove the need for it. Nothing decided.\n**Prompted by**: the founder, after the CQ-exclusion trace — *\"What IS the right instrument for that population then, or for search in Deliberus in general, if not cosine over embeddings?\"*\n\n---\n\n## 0. The whole thing in plain terms, with the examples that make it concrete\n\n*Written because the technical sections below are correct and unreadable, and the distinction\nthey draw is one a reader has to hold to use any of it.*\n\n**What the system does now.** Each claim is turned into a long list of numbers, and two claims\nare called related when their lists point in a similar direction. That works — **for a\nnarrower question than the project has been asking it.**\n\n**The question it answers well:** *are these about the same topic?*\n**The question it is being asked:** *are these the same kind of argument?*\n\n**The example that shows the gap.** *\"Harsher sentences deter crime\"* and *\"Higher fines deter\ntax evasion\"* are the **same argument** — raise the cost of an action and fewer people take it.\nThey share almost no vocabulary. The system sees two unrelated sentences, and this is exactly\nwhat dogfood run 6 measured: 402 candidate pairs inside one debate, four across the whole rest\nof the corpus.\n\n**Why the low scores are weaker evidence than they look.** Run 6 read 0.60 and concluded *not\nrelated*. §2 below says that inference is not safe — the meaning lives in combinations the\nmeasurement cannot see, so the pair may be kin with the instrument blind to it. **The corpus\nmay have less evidence of cross-domain sparseness than it believes.**\n\n**The alternative, in one image.** Instead of comparing word-clouds, compare *meaning\nskeletons* — who does what to whom. Both sentences above reduce to `[penalty] → discourages →\n[behaviour]`. Different words, matching shape.\n\n**And its honest cost, because it is not a free win.** Skeletons catch structure and get fooled\nby superficial structural differences; word-clouds catch fine shades of meaning and miss word\norder entirely. **Neither method wins.** What tips it for *this* project is a third property:\nthe skeleton method can **show its work** — here is the part of your argument that lines up\nwith the part of theirs. A score of 0.62 is an assertion nobody can dispute. An overlaid\ndiagram is a claim a reader can contest, and contestability is the product.\n\n**The example behind §4, which is the part that touches an open decision.** *\"Dr. Smith is\ncredible\"* rests on: she has published, a good university hired her, publishing implies\nexpertise, and this is her field. Those are **one** argument move — expert testimony — with\nfour parts, and **all of them are required**. Whereas *\"minimum wage barely costs jobs\"* rests\non Card-Krueger's study, monopsony theory, and low observed job loss: **three separate moves**\nthat happen to agree, where **each adds a bit** and losing one leaves the rest standing.\n\nThat difference is the `necessary` / `corroborative` distinction shipped this session — and\n§4 finds a published criterion for reading it off the structure instead of tagging it by hand.\n\n---\n\n## 1. The corpus measured a documented property, not a local quirk\n\nDogfood run 6 found claim-level cosine returning 402 within-topic candidate pairs and **4** cross-domain, none above threshold, and concluded that cosine measures vocabulary while cross-domain kinship is structural. **The argument-retrieval literature says the same thing in its own words.** Bergmann, Lenz, Ollinger & Pfister, on case-based retrieval of argument graphs, describing exactly the approach Deliberus uses:\n\n> \"With the cosine measure applied to the topic vectors it is also possible to compute a kind of semantic similarity between two whole argument graphs. This similarity is however, **purely based on the textual content, totally neglecting the structure of the argument.**\"\n\nSo the measurement was a rediscovery. That is reassuring about the measurement and unflattering about the reading list.\n\n---\n\n## 2. Cosine is weaker than assumed at the geometric level, not merely imprecise\n\nThe stronger critique is about the metric itself. From *Testing the assumptions about the geometry of sentence embedding spaces*:\n\n> \"the cosine similarity — which **treats dimensions shallowly** — captures (shallow) commonalities or differences between sentence embeddings, **which are not predictive of their performance on specific tasks**. Linguistic information is rather encoded in **weighted combinations of different dimensions, which are not reflected in the geometry** of the sentence embedding space.\"\n\nThe usual assumption is that closeness in the space means relatedness in the language. That paper says the implication does not reliably hold, because the information lives in dimension *combinations* that a single angle cannot see. **This is a reason to distrust a low cosine as evidence of unrelatedness** — which is precisely the inference run 6 made when it read 0.60 as \"not kin\". The claims may well be kin; the instrument cannot see it.\n\n---\n\n## 3. The structural instrument is a meaning graph, and the trade-off is honest in both directions\n\n**Abstract Meaning Representation** — a graph of predicate-argument structure rather than words — is the named alternative, and the ACL comparison of graph-based against vector-based metrics is refreshingly two-sided:\n\n> \"the **AMR-based metrics can better capture meanings dependent on sentence structures, but can also be distracted by structural differences** — whereas the BERT-based metrics represent finer-grained meanings of individual words, but often **fail to capture the ordering effect of words** within sentences and **suffer from interpretability problems**.\"\n\n**Neither dominates. They have distinct use cases**, which is the answer to \"what should replace cosine\": nothing replaces it, and the question is which instrument answers which question.\n\n**But one property tips it for this project specifically.** The AMR similarity work offers *\"the explicit graph alignment between two arguments' meaning graphs that better marks related structures, and that **can help explain argument similarity judgements**.\"* An explainable similarity is worth more here than an accurate opaque one, because inspectability is the product. A cosine of 0.62 is an assertion; an aligned subgraph is an argument a reader can contest.\n\n---\n\n## 4. The finding that touches an open decision: linked versus convergent may be derivable\n\nYu (*Informal Logic* 45:1, 2025) proposes deciding the **linked / convergent** distinction — the same distinction shipped today as `necessary` / `corroborative` — by counting argument scheme instances:\n\n> \"A **linked** argument can instantiate an argument scheme **only once** … In a linked argument, the premises work together, insofar as they are **all part of the same argument scheme instance**. In contrast, the **convergent** premises work separately, since they appear in **different argument scheme instances**.\"\n\n**Deliberus already classifies Walton schemes on every relationship edge.** If that criterion holds, the interpretation a human was going to tag by hand is **computable from data already in the graph**: premises inside one scheme instance are necessary to each other; premises in separate instances are corroborative.\n\n**The first check was run, and the answer is no.** Edges carry `scheme`, `schemes`, `scheme_cluster` and `scheme_confidence` — all **type** names — across 3,515 scheme-bearing edges, and **no instance identifier**. Yu's criterion needs instances, so it is not directly computable today.\n\n**A proxy is available and should be labelled as one.** Premises supporting the *same target claim* under the *same scheme type* are plausibly one instance, which makes the criterion derivable *approximately* — enough to **propose** an interpretation, never enough to assert one. That is exactly the propose-only pattern the corpus already runs for terminus classification and stance conflicts, and it fails in the right direction: a wrong proposal is corrected by a reader, where a wrong assertion silently moves every strength beneath it.\n\n**The second check stands open.** This is a proposal in an informal-logic paper rather than a settled result, and the corpus's rule about load-bearing claims applies.\n\n**If it survives both checks, the decision changes shape**: from *\"who tags each decomposition\"* to *\"propose the interpretation from the scheme structure, and let a human correct it\"* — which is the propose-only pattern the corpus already uses for terminus classification and stance conflicts. It would not settle `alternative`, which is a third case Yu's binary does not cover.\n\n---\n\n## 5. What this suggests for each population\n\n| Population | Why cosine struggles | Better instrument |\n|---|---|---|\n| **CQ polarity claims** (template-generated) | The template dominates the vocabulary, so similarity measures the form and ignores the filled slots | **Structural identity by construction** — same template, same slots. The corpus's own mint-time dedup carve, and no embedding needed |\n| **Cross-domain claims** | Kinship is structural, vocabulary is not shared | **Meaning-graph alignment** (AMR), with the alignment shown to the reader |\n| **Same-debate claims** | Works — run 6 produced 402 candidates within one debate | Cosine is fine here, and this is its real use case |\n| **Concepts** | Already works, 87% reuse | Cosine plus the existing concept layer |\n\n**The honest summary: cosine is not wrong, it is narrow.** It answers \"is this about the same topic\" well and \"is this the same kind of argument\" not at all. Deliberus has been asking it the second question and reading the answer as if it had asked the first.\n\n---\n\n## Sources\n\nBergmann, Lenz, Ollinger & Pfister, *Similarity Measures for Case-Based Retrieval of Natural Language Argument Graphs* · *Explainable Unsupervised Argument Similarity Rating with Abstract Meaning Representation* · *Semantic Similarity as a Window into Vector- and Graph-Based Metrics* (ACL) · *Testing the assumptions about the geometry of sentence embedding spaces: the cosine measure* · Yu, *The Argument Scheme-based Approach to Argument Structure*, Informal Logic 45:1 (2025) · *Leveraging Argumentation Knowledge Graph for Interactive Argument Pair Identification* · *Semantics-Aware Dual Graph Convolutional Networks for Argument Pair Extraction* (ACL).\n\n## Cross-references\n\n[dogfood-run-6-israel-palestine-cross-domain.md](dogfood-run-6-israel-palestine-cross-domain.md) (the measurement this explains) · [support-semantics-and-the-conjunction-problem.md](support-semantics-and-the-conjunction-problem.md) (the linked/convergent distinction Yu's criterion may derive) · [lowering-the-cost.md](lowering-the-cost.md) §6 (concept reuse against claim reuse) · [strength-layer-audit.md](strength-layer-audit.md) (the vector-search work that prompted the CQ trace)\n"}