{"path":"research/the-literature-as-a-distributed-data-structure/report-data-structure-and-ground-truth-2026-11-04.md","content":"# Helper report, verbatim: \"the literature as a distributed data structure\", proposals, mechanics, and measured ground truth (2026-11-04)\n\n*A second research helper on Claude Sonnet, briefed by the session for `the-literature-as-a-distributed-data-structure.md` § 4 and § 5: what \"distributed data structure\" could mean in the hands of those who have proposed or built one, the software analogues, and the cases with published ground truth (one-time review's error catch rate, continuous evaluation's measured yield, gate failures in both directions). Kept as it arrived, with one edit: the two absolute paths to the helper's scratchpad directory are replaced by a bracketed note, since a machine path never goes in a served document. Its saved pages and its retraction-lag analysis were in its scratchpad and are not committed; the URLs at the end are the record. The session recomputed the numbers it uses in `cases.py`.*\n\n---\n\n# Research report: \"the literature as a distributed data structure\", proposals, mechanics, and measured ground truth\n\nDate of research: 2026-11-04. Tags on every claim: [full] full text read; [part] extract or portion read; [abs] abstract only; [sec] secondary description; [own] my computation on primary data. Where I did not find something, I say \"not found\". Saved pages and my analyses are in [the helper's scratchpad directory, not committed] (key files listed at the end).\n\nQuote policy: I paraphrase everywhere except one 14-word quote (Tao, Part C). Exact wording is in the saved files.\n\n## 0. The Roy post\n- Posted 2 Nov 2026, about 10:24, on X (@roydanroy). A second post at 10:27 clarified that \"fully\" means he actually resigned about two years ago and had only been handling resubmissions. [sec: aggregator copies at dailyjournal.news, digg.com/ai/zc2q5xs5, agihunt.info; the tweet URLs x.com/roydanroy/status/2117246521234063414 and …247149868286371 were not opened]\n- One visible reply (@ziv_ravid) asked what the solution is. I found no answer from Roy, and no elaboration of \"distributed data structure\". [sec]\n- Roy's own homepage lists him as Google DeepMind visiting researcher (since 31 Dec 2025) and, at the time of the snapshot, JMLR Action Editor. [part]\n- His post has two separable claims: (1) top-model AI reviews now beat time-pressed human ones; (2) one-time accept/reject gating is the wrong form. Part B1 tests (1) and Parts A–B test (2).\n\n## PART A. \"Distributed data structure\", made concrete\n\n### A1. Proposals and systems (unit, continuity, whether evaluation attaches, size, fate)\n\n**Written proposals**\n- **Nielsen, \"Micropublication and open source research\"** (2007 essay). [part]\n  - Unit: the smallest publishable increment; data, code, simulations and commentary count as first-class outputs.\n  - Gate: no pre-publication peer review; \"publish-then-filter\".\n  - He names three problems himself: recognition of contributions, quality assurance, and organizing and searching the mass of output. He adds a fourth, migration, because researchers won't leave the journal credit system.\n  - Fate: essay; his Polymath work is the cited existence proof.\n- **Lorig, \"Publishing Without Journals\", arXiv 2607.05454** (cs.DL, 17 Jul 2026, v2). [part, about half read]\n  - Unit: the paper, stored as source (LaTeX) and forkable.\n  - Process: open deposit; attributed, up/down-voted commentary by \"approved\" users under real names; author replies; forks carry a permanent link to their parent. There is no accept/reject bit.\n  - He lists his own objections: sparse participation, real names chilling criticism, loss of the certification signal, rich-get-richer attention, brigading, fraud, and who approves the approvers.\n  - Fate: preprint with no implementation.\n- **Kejriwal, \"Semantics for Science\" (\"ClaimRxiv\")**, CEUR Vol-4293 (2026, SEMANTiCS-style blue-sky paper). [part]\n  - Unit: a scoped assertion node linked to evidence, context, provenance and epistemic status; papers remain \"narrative containers\".\n  - Relations: supports, challenges, refines, replicates, supersedes.\n  - Main unsolved problem named: claim identity across papers (string similarity is insufficient).\n  - Fate: proposal.\n- **Zharnikov, \"Research as Repository\"** (working paper v2.2.0, Mar 2026, rev. Jun 2026). [part]\n  - A research program is a version-controlled repository; a paper is a tagged release \"forked to a journal\"; reviewers commit with attribution; commit-reveal hashes establish priority.\n  - Fate: single-author proposal.\n- **Tao, \"Mathematics in the age of AI\"** (arXiv 2608.16753, 24 Aug 2026), plus his posts on Palomar (18 Aug) and Hexagon (6 Oct). [part/full for Palomar]\n  - Model: a proof moves through generation, verification, exposition, publication and canonicalization, and authorship of the stages will decouple. He proposes \"putting a proof up for adoption\" and a priority date equal to the latest of preprint, exposition and (later) formalization.\n  - **Palomar** (open for submissions 18 Aug 2026): a registry of Lean repository snapshots, each pinned to a GitHub commit. Check (a) is mechanical (a Lean tool, Comparator, confirms the solution proves exactly the challenge file); check (b) is an LLM judging whether the informal description matches. There is no human review.\n  - **Hexagon** (public since 28 Sep 2026; Antieau 5 Oct, Tao cross-post 6 Oct): a math/TCS repository that explicitly accepts fully LLM-generated work with no claimed human understanding. It rejects only for \"obvious defects\", not for significance or novelty. Submission rate starts at 1 per day and grows with moderation history. [part]\n- **XScientist** (arXiv 2607.12301, Jul 2026). [abs + part]\n  - Each autonomous-research run exports an \"exploration DAG\" with content hashes, claim-to-evidence anchors, failed branches and re-execution hooks.\n  - Scope: for AI-run research, not for the human literature.\n- **\"Version Control for Scientific Reasoning\"** (OpenReview, Agents4Science 2025, 17 Sep 2025). Observe/hypothesize/execute/analyze as a DAG. [abs]\n- **Seemay Chou, Astera, \"Scientific publishing: enough is enough\"** (mid-2025). [full]\n  - Astera and Arcadia no longer fund journal publication.\n  - She argues for shorter narratives, datasets, code and models released faster, and says the sanitized journal record is poor training material for scientific AI agents.\n  - Cites LK-99 as work that was \"peer tested\" in under a month.\n  - Reader comments on the page object that preprints get about zero reviews against 2–4 for journal papers, and that enforcement of data/code sharing is weaker without journals. (Anonymous commenter, so no name.)\n\n**Running systems**\n- **COS Lifecycle Journal** (Nosek). [part]\n  - Publish-before-review. No final accept/reject; authors may optionally declare a Version of Record. Evaluation services (human, machine, empirical) post assessments, which are all published. Reviews may attach to research plans as well as outcomes.\n  - 3-year pilot opened Jan 2025. COS reports 34 submissions published in 2025. The CODECHECK register says the pilot has since closed to new submissions (date not found).\n- **eLife Publish-Review-Curate.** [part, from eLife pages as search extracts]\n  - 2023–Jan 2025: 12,228 submissions; 27.3% sent for review; 3,054 Reviewed Preprints; 1,601 Versions of Record (VOR). 96% of VORs came after at least one revision round.\n  - Median 95 days from submission to Reviewed Preprint.\n  - In the 2,051-article comparison set, evidence-strength terms improved between Reviewed Preprint and VOR for 43.8%, stayed the same for 53.7% and fell for 2.4%.\n  - 2025: 2,853 submissions (6,396 in 2024), 1,013 sent for review, 35% selected for review (27% in 2024), after Clarivate withdrew the Journal Impact Factor eligibility.\n  - Output fell from 143 to 84 articles per month.\n- **Sciety / Peer Community In / Review Commons / MetaROR / PREreview.** [part]\n  - Sciety listing today: eLife 54,043 evaluations on 8,376 curated articles; Review Commons 6,815; Rapid Reviews Infectious Diseases 2,332; PREreview 2,050.\n  - Sciety end-2024: 36,000 evaluated preprints, 76,000 evaluations, 27 initiatives.\n  - PCI 2025: 521 submissions, 248 recommended preprints, 2,800 recommenders, 21 communities. Cost about €718 per recommended preprint (€198 excluding in-kind).\n  - MetaROR year one: 28 articles published, 59 reviewers.\n  - Evaluations attach to the preprint as separately citable objects rather than being discarded on rejection.\n- **The Unjournal.** [part]\n  - 100+ evaluations of 55+ papers; 57 tracked packages. Evaluators get median 8 hours and about $400 each; evaluations carry structured ratings.\n  - April 2026 tracking: of 22 papers manually assessed, 5 showed clear substantive updating and 7 stated an intention to update; of 57 packages, 19 had an author response.\n  - Of the 22 assessed, 13 (59%) showed no observed public change; 1 (5%) had an explicit public link between an evaluation and a correction.\n  - Two-year reach: about 34,800 unique visits.\n- **ResearchHub.** [part, partly sec]\n  - Pays $150 in its token per accepted review; 1,163 reviews in Q2 2026; $2.2M deployed to 55 funded projects; token down about 91% from its peak by Jan 2026. [sec]\n- **Living evidence.** [abs]\n  - Survey tracking 119 living evidence syntheses for 2 years: 49 (41.2%) produced at least one update; the median was one update report.\n  - A second survey counted 168 living systematic reviews (549 records) and found update frequencies vary widely.\n  - A review of 46 living systematic reviews (Cochrane and non-Cochrane): 10 (22%) were cited by guidelines.\n- **Claim-graph prototype in git, \"Tardigraph\".** [full, self-reported, undated]\n  - 3,038 nodes, 9,591 edges, 274 papers, 1,690 claims with source and locator, 161 recorded contradictions kept rather than resolved, and 20 data-integrity flags raised against published work. About 4 MB as two JSONL files.\n  - It describes itself as 8 days old; no external review.\n- **Discovery layers (not claim stores).** [part]\n  - Semantic Scholar's academic graph: 225M+ papers and 2.8B+ citation edges, with parsed text, summaries and embeddings.\n  - Elicit searches 138M papers and extracts structured columns at query time; Consensus does evidence summaries. Neither publishes a persistent, shared claim layer.\n- **Replication and retraction ledgers.** [part]\n  - FORRT FLoRA: 3,003 entries; 4,631 original findings in its Atlas: 956 replicated, 847 failed, 755 mixed, 2,072 with no replication on record.\n  - Retraction Watch Database (Crossref, CC0): 73,561 records at the 2 Nov 2026 snapshot.\n  - Curate Science is archived; Crossref holds 1,147,778 Peer Review DOIs.\n\n**Not found:** a named \"science as DAG\" proposal by Alex Danco; any Stuart Buck proposal beyond NIH review reform; a primary Marblestone source on publication units. Marblestone and Nielsen genealogy appears only in a forum summary (longevitybase, 5 Oct 2026) that reads as LLM-written, so I did not rely on it. alphaXiv and Protocol Labs/DeSci primary pages were not fetched. Jessica Polka's ReimagineReview is a registry of peer-review experiments (launched 2019 with about 34 listings), not a structural proposal. [part]\n\n### A2. Software analogues: mechanics, and what the sources do or do not address\n- **Git.** [part] A content-addressable store: each object is keyed by the SHA-1 of its content plus header; commits point to parents, giving the DAG. Not addressed: who decides which branch counts.\n- **PyPI yank (PEP 592).** [part] Non-destructive withdrawal with an optional reason string, shown to users; reversible (\"unyank\"); recorded in the journal. Installers ignore a yanked release unless an exact pin selects it; a mechanical re-install of a pinned version continues. Current practice: whole releases only.\n- **npm.** [part] Registry data is immutable. Unpublish is allowed within 72 hours if nothing depends on the package, or later only if under 300 weekly downloads, no dependents and a single owner. Otherwise: deprecate, which prints a warning but keeps builds working.\n- **Cargo yank.** [part] Removes a version from the index for new resolutions but does not delete it; existing lockfiles still work. Guidance: publish a compatible fix first; for vulnerabilities prefer an advisory database over yanking.\n- **Linux kernel review.** [part]\n  - Review is continuous on mailing lists; there is no accept/reject moment, though maintainers merge.\n  - LWN 2016 (Corbet reporting Sang): the share of mainline patches with Acked-by/Reviewed-by/Tested-by tags rose from about 20% (around v3.0) to about 40%, and unprocessed patches were increasing.\n  - Vetter 2018: outside graphics, about 30% of maintainer self-commits showed any sign of oversight.\n  - Duda 2019: of 792k patches (v3.0–v4.20), 18k (2.3%) went unanswered for over 6 months and never merged; mean time to first answer 4 days 5 hours, median 12 hours.\n  - netdev stats, June 2026: 339 emails per day; more than 150 patches per working day; review percentage 50.84%, down from 53.36%; maintainers prioritize patches by the submitter's \"review standing\".\n- **CRDTs.** [abs/part] Strong eventual consistency: replicas that have received the same updates are in the same state, with no coordination. Graph and set types exist (Shapiro et al., INRIA RR-7687). Kleppmann et al.: the one case a CRDT cannot resolve automatically is concurrent updates of the same property of the same object; it keeps both values and leaves resolution to the application or user.\n- **Gaps in the analogy (stated by the sources' scope):** none of these systems defines correctness by external evidence; they converge on state, not on truth. The kernel and registry sources describe review capacity and consumer pinning as the limiting factors, but nothing in them measures error detection.\n\n### A3. 2026 writing reacting to AI-scale volume\n- **arXiv.** [part]\n  - Submissions: 20,569 (Sep 2024) → 26,646 (Sep 2025) → 40,363 (Sep 2026), against 9,869 in Sep 2016 (New Scientist 5 Oct; Times Higher Education 3 Nov).\n  - Rules in 2026: endorsement for first-time posters (21 Jan; moderator rejection rate about 4% → 10–12% per Wijers in Science); 1-year ban plus peer-review requirement for unchecked LLM output (about May); cap of 2 papers per month from 1 Oct (Nature 15 Oct).\n  - Dietterich (chair, editorial advisory council) and Sigurdsson call the cap a stopgap; Sigurdsson also says AI reviewers are coming.\n  - Kevin Buzzard: arXiv \"might not survive\"; he now uses AI to triage it. [part]\n- **arXiv paper 2610.34449 (25 Oct 2026).** [abs + part] Growth break in Feb 2026, 17% above trend outside CS; mathematics submissions doubled in a year; two thirds of the math increase came from more distinct authors rather than a few prolific ones; solo-authored math papers up 70%.\n- **TMLR editors' post, 15 Oct 2026.** [full]\n  - Submissions up 300% in a year; solo-authored up 1,300%.\n  - Desk rejection 6% (2023) → about 53%.\n  - 159 authors suspended from full review for one year; co-authorship-based quotas; deanonymizing major integrity violations.\n  - Every submission gets an AI soundness review from Oct 2026; the human action editor still decides.\n- **Rao & Shah, \"Evaluating AI Reviewers… for TMLR\", 30 Jun 2026.** [part] Four candidate AI reviewers scored against action-editor soundness labels on 50 recent papers per category plus inserted-error papers. On papers with inserted claim-breaking errors the systems differed widely (e.g., 70% identified for one system; 25% and 40% for others); it names no system as a strong detector. Details in B1.\n- **Chronicle of Higher Education op-ed (Jalloh, 3 Nov 2026).** [part] ICLR submissions 490 (2017) → over 62,000 for 2027; Organization Science task force (April 2026): submissions +42% since Nov 2022, review quality dropped as heavily AI-generated reviews rose [sec]. He proposes submission fees; he says increasing capacity is not possible.\n- **Sever (bioRxiv/medRxiv), in THE 3 Nov 2026.** [full] The long-term fix is changing the ratio of data deposits to narrative papers (a thousand genomes: one thousand papers, or a thousand deposits and one paper?). He calls quotas temporary.\n- **Nature 30 Oct 2026:** IEEE retracted 1,417 conference papers at once. [part]\n\n## PART B. Cases with published ground truth\n\n### B1. How well one-time human review catches errors\n- **Baxt et al. 1998**, Annals of Emergency Medicine, a fake migraine/propranolol paper with 10 major and 13 minor planted errors, sent to 262 reviewers (203 returned). [abs]\n  - Recommended acceptance (n=15): found 17.3% of major, 11.8% of minor errors.\n  - Recommended revision (n=67): 29.6% major, 22.0% minor.\n  - Recommended rejection (n=117): 39.1% major, 25.2% minor.\n  - 68% did not notice the conclusions were unsupported; the authors conclude reviewers missed two thirds of major errors.\n  - My weighted average of the three groups: about 34% of major errors [own].\n- **Godlee et al. 1998** (JAMA): a paper accepted by a journal, modified to carry 8 weaknesses; 221 of 420 reviewers returned reports; mean 2 weaknesses (about 25%) were commented on; blinding or signing made no difference. [abs]\n- **Schroter et al. 2008** (JRSM; BMJ trial): 607 reviewers, three test papers with 9 major and 5 minor errors. At baseline reviewers found 2.58 of 9 (about 29%); 2.71 and 3.0 on papers 2 and 3. Biased randomization was found most often (over 60% of reviewers who rejected). Training had small effects. [abs]\n- **Jefferson et al., Cochrane Review 2007:** 28 studies included; no clear evidence for author/reviewer concealment effects; no evidence referee training helps; the authors say little empirical evidence supports editorial peer review as a quality mechanism, and absence of evidence is not evidence of absence. [abs]\n- **Agreement between reviewers.** Rothwell & Martyn 2000: kappa 0.08 (179 papers, Journal A) and 0.28 (116 papers, Journal B); abstract scores: abstract identity explained only 10–20% of variance. [abs] NeurIPS 2021 consistency experiment: two independent committees disagreed on 23% of papers; about half of accepted papers would change on a rerun (consistent with 2014). [abs] Peters & Ceci 1982 (resubmitted published papers; 8 of 9 sent to review were rejected) is known to me only via Lorig's paper. [sec]\n- **Shah's conference experiment (reported in Rao & Shah 2026).** Three versions of a paper, each with one fatal flaw; 79 reviews; one review mentioned the flaw. [sec]\n- **Test of Roy's claim (1), the newest data:**\n  - A medRxiv preprint dated 25–28 Oct 2026 (Brian et al.) re-ran the Schroter test papers through 15 LLMs. Humans (491 reviews): 2.40 of 9 major-error points, about 27% of inserted errors. Models: 4.73 to 8.65 of 9 on the original papers; on 58 randomized trials published in 2026, sensitivity from 45.8% (Haiku 4.5) to 92.9% (Opus 5.5), with open-weight models 55–81%. Models flagged 14–29% of error types in control reports that had none. [abs + part, preprint]\n  - Willner & Yanagizawa-Drott \"Verifying the Verifiers\": best-model recall about 30% at end 2024, one in six errors missed by Feb 2026, a model reaching 99% recall in Jul 2026; recall degrades with multiple errors; confidence poorly calibrated. [part]\n  - Opposite-direction results: Alharbi (31 Jul 2026), two open multimodal models on ICLR 2026 submissions: they detected 12.1% of 145 inserted errors (22.2% with a verification prompt) and scored papers 7.0–8.1 against human 3.4–6.8. [abs] SPOT (2025): no model above 21.1% recall or 6.1% precision on real errata-level errors. [abs] \"Benchmarking Agentic Review Systems\" (2026): best configuration 71.6% of injected errors, union of six models 83.3%. [abs]\n  - All seeded-error tests measure detection of planted flaws, not agreement with long-run truth.\n\n### B4. What continuous or post-publication evaluation has measured\n- **Reproducibility Project: Psychology (2015):** 100 studies; 97% of originals significant versus 36% of replications; 47% of original effect sizes inside the replication interval; 39% rated replicated; mean effect r fell from 0.403 to 0.197. [abs]\n- **Reproducibility Project: Cancer Biology (2021):** planned 193 experiments from 53 papers; 50 experiments from 23 papers completed (26%), producing 158 effects. Median effect size 85% smaller; success 46% (51/112) on a multi-criterion score; positive effects 40% (39/97), null effects 80% (12/15). [abs + part]\n- **Many Labs:** not fetched.\n- **DARPA SCORE (Nature, 1 Apr 2026).** [part]\n  - 3,900 papers (2009–2018) sampled; replications of 274 claims from 164 papers: 55.1% of claims and 49.3% of papers replicated; effect size r 0.25 → 0.10.\n  - Reproduction of 143 papers: 74% approximately, 54% precisely; data available for only 24% of 600 papers.\n  - Robustness: 100 papers, 5+ analysts each; 34% of reanalyses matched within a narrow tolerance, 57% within a wide one.\n  - Human forecasts (repliCATS, Replication Markets): 76% and 78% by the best metric. Three machine methods were not consistently effective.\n- **Earlier forecast studies.** Dreber 2015 (PNAS): markets on 44 psychology studies; 41 replicated-or-not outcomes; market correct for 71% (29/41); 39% of the 41 replicated; mean prior probability of hypothesis true about 9% (median). [part] Camerer 2018: 21 Nature/Science experiments, 62% replicated, markets correct on 18 of 21 [sec: press release].\n- **PubPeer flagged → corrected or retracted.**\n  - Brookes 2014: 497 papers with documented data-integrity problems; 18 months; 23% of publicly discussed papers got corrective action versus 3.1% of privately reported ones; 6.5× more retractions, 7.7× more corrections. [full/abs]\n  - Delgado-Quirós & Ortega: 17,244 PubPeer-commented articles; only 21.5% of those with errors or misconduct got an editorial notice (34% for 2019 papers). [part]\n  - Ortega 2020: 39,449 PubPeer articles; 2,308 errata, 189 expressions of concern, 1,531 retractions; delay between publication and notice often over 3 years. [abs]\n  - Semenza case study: 14 of 15 retracted papers had earlier PubPeer criticism; the first comment identified core issues at least 23 months before retraction; journals issued an editor's note or correction for three. [abs]\n  - Bishop, Feb 2026: of 800 highly cited papers retracted 2021–2025, 58% had earlier PubPeer comments; the 10 most prolific commenters account for 27% of those first comments. Not peer-reviewed; she retitled the post after a correlation-versus-causation objection. [full]\n- **Time to retraction (own computation).** [own] From the Retraction Watch CC0 CSV (downloaded 4 Nov 2026; 73,597 rows; 68,154 retractions with usable dates): median 1.36 years; mean 2.46; 90th percentile 6.03; 39.5% within a year; 64.0% within two; 86.8% within five; 4.1% after ten. Research articles: median 1.78 years. Retractions citing data/image falsification or fabrication (n=2,053): median 4.26 years; 20.7% after ten years. Paper-mill retractions (n=11,716): median 1.60 years. Retractions per year of retraction: 3,102 (2020), 5,592 (2022), 13,371 (2023), 6,502 (2024), 6,006 (2025). Caveat: recent publication cohorts are right-censored. Literature: biomedical 2014–2023 median 390 days (n=7,485) [part]; chemistry 2001–2021 median 1.7 years (n=1,292) [sec]; oncology 2011–2020 median 545 days [abs]; Shuai et al. median two years [sec].\n- **eLife and Unjournal outcome numbers:** see A1 (revision changes the evaluation of 46% of papers; Unjournal feedback produces public change in a minority of assessed cases).\n- **Living reviews:** fewer than half updated in two years (A1).\n\n### B5. Fast catches, slow catches, and gaming\n- **LK-99.** arXiv 2307.12008 and 2307.12037 both submitted 22 Jul 2023 (arXiv API [full]); Nature news \"LK-99 isn't a superconductor\" 16 Aug 2023, 25 days later, reporting copper sulfide impurity (resistivity drop near 104 °C, partial levitation) and dozens of replication attempts. Never passed a journal gate. [part]\n- **Arsenic life.** Science online 2 Dec 2010; print 3 Jun 2011 with eight Technical Comments and a response; two Science papers in July 2012 showed no arsenic incorporation; retracted 24 Jul 2025 (14 years 7 months after online), on a changed retraction standard (key conclusion unsupported, no misconduct alleged). Some authors dissented. [part]\n- **Wakefield.** Lancet 28 Feb 1998; Deer's investigation began 2004; General Medical Council finding 28 Jan 2010; full retraction 2 Feb 2010 (about 12 years). [part]\n- **Hwang.** Science papers March 2004 and June 2005; Seoul National University final report 10 Jan 2006; both retracted 20 Jan 2006 (about 22 and 7 months after publication). [part]\n- **Gaming and failure of continuous or downstream checking.**\n  - IEEE retracted 1,417 conference papers on 23 Oct 2026, traced to a study of 4,407 paper-mill authorship adverts; 2,062 IEEE papers across 366 proceedings were flagged; authorship slots cost $43–400. Hindawi retracted over 9,500 articles by end 2023. [part]\n  - Clarivate suppressed impact factors for 20 journals in the 2025 Journal Citation Reports (17 in 2024, 4 in 2023) for self-citation or citation stacking. [part]\n  - PubPeer: Sarkar's defamation suit; in Dec 2016 the Michigan Court of Appeals held he was not entitled to unmask the anonymous commenters. A Harvard professor alleged cyberstalking by commenters (RW, Dec 2023). [part]\n  - TMLR reports authors creating second OpenReview accounts and dropping over-quota co-authors to evade quotas (above).\n  - Agents4Science (Oct 2025): 315 submissions, three LLM reviewers per paper, 48 accepted after human review of 80; a human reviewer called accepted papers technically correct but neither interesting nor important. [part]\n\n### B6. Wrong at the gate, right later\n- **Akerlof, \"Market for Lemons\".** Submitted June 1967; rejected by the American Economic Review (trivial), the Journal of Political Economy (too general to be true) and the Review of Economic Studies (trivial); accepted by the Quarterly Journal of Economics (1970). Cited by 23,235 works in OpenAlex as of 4 Nov 2026. [part: Gans & Shepherd 1994; own OpenAlex query]\n- **Higgs 1964.** Second paper rejected by Physics Letters as not meriting rapid publication; revised, with the particle prediction added, and accepted by Physical Review Letters (13:508; cited by 4,699 in OpenAlex). Higgs said the rejection is why the prediction is there. The 1964 papers had under 50 citations by about 1970 and about 18,000 by 2022 (CERN Alumni; the combined figure is secondary). [part]\n- **Margulis 1967.** She recounted that the paper was rejected by 15 journals (self-report; the Berkeley Understanding Science page attributes it to subject-area fit) before the Journal of Theoretical Biology; cited by 1,872 in OpenAlex. [part]\n- **Systematic evidence.** Gans & Shepherd 1994 asked over 140 leading economists for rejected papers: more than 60 essays, including 15 Nobel winners; most had been rejected, often repeatedly, and some rejected papers became classics. [abs] Campanario 2009: 24 future Nobel laureates met editor or referee resistance to manuscripts on discoveries that later earned the prize. [abs]\n- **Accepted then retracted after years:** see B4 (RW) and the cases in B5; the long tail reaches decades (maximum 81 years).\n\n## PART C. The other side\n\n**Gatekeeping as load-bearing**\n- **Cosma Shalizi, 21 May 2012:** separates dissemination (arXiv can do it) from certification; argues that without referees many readers redo the same filtering, so division of labour reinvents peer review; notes that insiders' attention filtering and outsiders' reliability filtering may need different mechanisms. [part]\n- **Vanessa Heggie, Notes and Records, 2016:** removing gatekeepers also erases expertise; journals will retain curating and guaranteeing expertise because readers use journal names to sift unintelligible material. [part]\n- **Pascal Boyer, 2018:** wants post-publication quality control, but calls the move a coordination problem: publication in a high-status journal is a quality signal and withholding it reads as low quality. [part]\n- **Hugh Desmond, Synthese, 2024:** gatekeeping should be conserved; information overload means scientists need help judging quality; replacing it with citation counts or search algorithms creates new gatekeepers and runaway prestige. [part]\n- **Dan Elton (Asimov Press, Oct 2024):** defends peer review but concedes error-detection studies give 25, 15, 29 and 34 percent from only two medical journals over 1998–2008; says no one knows what happens without it and asks for experiments. Mastroianni (Dec 2022) says he opposes any universal system and wants diversity of formats. [part]\n- **Nature Neuroscience editorial, Jul 2023:** editorial selection before review is a core journal function; unfiltered servers are like a marketplace; reputation guides trust. (A journal speaking for itself.) [part]\n- **Tao (Aug 2026):** AI filters may triage submissions, but: \"I do not believe that human referees can be removed from the publication process.\" Community acceptance is slow, human and outside the authors' control. [part]\n- **Steven Benner (SciAm, Jul 2025):** reacting to the arsenic retraction, he says Science should not act as a gatekeeper by retracting wrong-but-not-fraudulent papers; they should stay as an object lesson. (A critic of the retraction gate, not the publication gate.) [part]\n- **Lorig's own caveats:** a certification signal is lost until replaced; crowd attention is rich-get-richer; bias does not vanish when a crowd replaces a panel. [part]\n\n**Critics or internal critics of claim-level decomposition**\n- No named author found who rejects claim-level stores outright.\n- **Kozlov & Stuart, 2024:** experimental articles work as modernist-style narratives, as vehicles for reasoning rather than arguments or claim sets. [part]\n- **de Waard (HypER) and Buckingham Shum's SWJ 2010 piece:** research consists of argumentation around facts; flat triples drop epistemic status and the link to evidence. [part]\n- **Al Khatib et al. 2021 (with de Waard):** most processing treats tentative or persuasive statements as factual; a claim and its premise can sit in different sections. [part]\n- **Hyland 2005:** hedges carry the writer's negotiation with the reader. [part]\n- **Kejriwal:** flattening to triples early loses study design, effect sizes, temporal context and negative evidence. [part]\n\n## Key saved files (full paths)\n- [the helper's scratchpad directory]`rw_lag_analysis.md` (retraction-lag analysis; data in `…/helper2/rw/rw.csv`)\n- `…/helper2/tmlr_blog_2026.txt`, `lorig_arxiv_2607_05454.txt`, `seemay_chou.txt`, `kejriwal_claimrxiv.txt`, `tardigraph.txt`, `tao_math_age_ai.txt`, `tao_palomar.txt`, `cos_score_news.txt`, `flora_report.txt`, `brookes_2014_pmc.txt`, `bishop_pubpeer_2026.txt`, `delgado_ortega_pubpeer.txt`, `xie_2025_retraction.txt`, `tmlr_ai_reviewer_evals.txt`, `the_he_slop.txt`, `mastroianni_2022.txt`, `shalizi_bactra.txt`, `gans_shepherd_1994.txt`, `epmc_baxt.json`, `epmc_schroter.json`, `epmc_godlee.json`, `epmc_jefferson.json`, `epmc_rothwell.json` (all in the `helper2` directory).\n- Helper scripts: `…/scratchpad/tools/fetch.sh`, `epmc.py`, `oa.py`.\n\n## Sources\nRoy: https://dailyjournal.news/perfis/ia-tecnologia/roydanroy ; https://digg.com/ai/zc2q5xs5 ; https://danroy.org/\nProposals: https://arxiv.org/html/2607.05454 ; https://ceur-ws.org/Vol-4293/paper69.pdf ; https://github.com/spectralbranding/sbt-papers/blob/main/r14-paper-as-repository/paper.md ; https://arxiv.org/html/2608.16753 ; https://terrytao.wordpress.com/2026/08/18/palomar-a-registry-of-lean-verified-mathematics/ ; https://terrytao.wordpress.com/2026/10/06/hexagon/ ; https://antieau.github.io/2026/10/05/hexagon.html ; https://hexagonmath.org/about ; https://arxiv.org/html/2607.12301v1 ; https://openreview.net/forum?id=KQX70xNaL6 ; https://asterainstitute.substack.com/p/scientific-publishing-enough-is-enough ; https://michaelnielsen.org/blog/micropublication-and-open-source-research/ ; https://tardigraph.meow-meow.io/graphs/\nSystems: https://www.cos.io/blog/lifecycle-journal ; https://lifecyclejournal.org/ ; https://www.cos.io/impact/2025/programmatic-highlights ; https://codecheck.org.uk/register/venues/journals/lifecycle_journal/ ; https://elifesciences.org/inside-elife/7da92a51/the-elife-model-three-year-update ; https://elifesciences.org/inside-elife/8947f033/the-elife-model-two-year-update ; https://sciety.org/groups ; https://blog.sciety.org/sciety-wrapped-2024/ ; https://peercommunityin.org/2026/01/14/pci-and-peer-community-journal-highlights-of-2025/ ; https://peercommunityin.org/2026/01/12/2025-finances-article-costs/ ; https://researchonresearch.org/metaror-turns-one/ ; https://info.unjournal.org/impact.html ; https://unjournal.github.io/unjournaldata/posts/do-authors-respond-2026/ ; https://www.researchhub.foundation/ ; https://heybeluga.com/articles/research-coin-brian-armstrong-desci-ai-funding-tools/ ; https://www.sciencedirect.com/science/article/abs/pii/S0895435626004075 ; https://www.jclinepi.com/article/S0895-4356%2825%2900237-9/fulltext ; https://link.springer.com/article/10.1186/s13643-026-03209-7 ; https://lukasroeseler.github.io/florareport/ ; https://forrt.org/flora-replication-atlas/browse/ ; https://www.retraction-search.com/about ; https://www.crossref.org/documentation/retrieve-metadata/retraction-watch/ ; https://data.crossref.org/reports/statusReport.html ; https://arxiv.org/html/2301.10140v2 ; https://support.elicit.com/en/articles/14758040-elicit-s-source-for-papers ; https://reimaginereview.asapbio.org/about/\nSoftware analogues: https://git-scm.com/book/en/v2/Git-Internals-Git-Objects ; https://peps.python.org/pep-0592/ ; https://docs.pypi.org/project-management/yanking/ ; https://docs.npmjs.com/policies/unpublish/ ; https://doc.rust-lang.org/cargo/commands/cargo-yank.html ; https://lwn.net/Articles/703005/ ; https://osr.cs.fau.de/wp-content/uploads/2020/03/duda_2019.pdf ; https://lists.openwall.net/netdev/2026/06/17/240 ; https://blog.ffwll.ch/2018/04/maintainer-statistics.html ; https://www.lip6.fr/Marc.Shapiro/papers/RR-7687.pdf ; https://martin.kleppmann.com/papers/local-first.pdf\n2026 volume: https://www.newscientist.com/article/2591679-arxiv-might-not-survive-ai-slop-onslaught-warn-mathematicians/ ; https://www.nature.com/articles/d41586-026-03194-w ; https://www.timeshighereducation.com/depth/how-can-research-platforms-deal-slop-deluge ; https://www.science.org/content/article/arxiv-preprint-server-clamps-down-ai-slop ; https://arxiv.org/abs/2610.34449 ; https://blog.tmlr.org/2026/adapting-our-processes-to-the-changing-world-of-scientific-publication/ ; https://www.cs.cmu.edu/~nihars/preprints/TMLR_AI_reviewer_evals.pdf ; https://www.chronicle.com/article/can-academe-survive-the-tidal-wave-of-ai-slop ; https://www.nature.com/articles/d41586-026-03455-8\nError detection: https://www.ncbi.nlm.nih.gov/pubmed/9737492 ; https://pubmed.ncbi.nlm.nih.gov/9676667/ ; https://pubmed.ncbi.nlm.nih.gov/18840867/ ; https://pubmed.ncbi.nlm.nih.gov/17443635/ ; https://pubmed.ncbi.nlm.nih.gov/10960059/ ; https://arxiv.org/abs/2306.03262 ; https://doi.org/10.64898/2026.10.25.26366455 ; https://ape.socialcatalystlab.org/cred/verifying-the-verifiers.pdf ; https://arxiv.org/abs/2608.28626 ; https://arxiv.org/html/2606.19749 ; https://llms-in-science.github.io/spot/ ; https://www.experimental-history.com/p/the-rise-and-fall-of-peer-review ; https://press.asimov.com/articles/peer-review\nReplication and post-publication: https://www.science.org/doi/10.1126/science.aac4716 ; https://elifesciences.org/articles/71601 ; https://www.nature.com/articles/s41586-025-10078-y ; https://www.cos.io/about/news/large-scale-collaboration-releases-new-findings-on-research-credibility ; https://www.nature.com/articles/s41562-018-0399-z ; https://www.pnas.org/doi/10.1073/pnas.1516179112 ; https://pmc.ncbi.nlm.nih.gov/articles/PMC3994617/ ; https://revista.profesionaldelainformacion.com/index.php/EPI/article/download/87096/63326/294978 ; https://doi.org/10.1002/leap.1339 ; https://www.cjstp.cn/EN/10.11946/cjstp.202512181587 ; http://deevybee.blogspot.com/2026/02/the-role-of-pubpeer-in-retractions-of.html ; https://escienceediting.org/journal/view.php?number=377 ; https://retractionwatch.com/2016/03/08/ready-to-geek-out-on-retraction-data-read-this-new-preprint/ ; https://cen.acs.org/research-integrity/misconduct/Flawed-chemistry-papers-take-year/101/web/2023/08 ; https://gitlab.com/crossref/retraction-watch-data\nCases: https://www.nature.com/articles/d41586-023-02585-7 ; https://arstechnica.com/science/2023/09/the-room-temperature-superconductor-that-wasnt/ ; http://export.arxiv.org/api/query?id_list=2307.12008,2307.12037 ; https://www.science.org/doi/10.1126/science.adu5488 ; https://www.science.org/content/blog-post/last-step-long-process-arsenic-life ; https://www.scientificamerican.com/article/arsenic-life-microbe-study-retracted-after-15-years-of-controversy/ ; https://www.bmj.com/content/342/bmj.c5347.full ; https://www.science.org/doi/10.1126/science.1124926 ; https://cen.acs.org/articles/84/web/2006/01/Science-Retracts-Stem-Cell-Papers.html ; https://retractionwatch.com/2025/06/18/clarivate-impact-factor-suppression-list-2025-self-citation-stacking/ ; https://law.justia.com/cases/michigan/court-of-appeals-published/2016/326667.html ; https://retractionwatch.com/2016/12/07/pubpeer-wins-appeal-court-ruling-unmask-commenters/ ; https://retractionwatch.com/2023/12/04/cyberstalking-pits-harvard-professor-against-pubpeer/ ; https://www.science.org/content/article/futuristic-meeting-ais-took-lead-producing-and-reviewing-all-studies\nRejected-then-famous: https://pubs.aeaweb.org/doi/pdfplus/10.1257/jep.8.1.165 ; https://link.springer.com/article/10.1007/s11192-008-2141-5 ; https://www.ph.ed.ac.uk/higgs/brief-history ; https://alumni.cern/news/741463 ; https://www.sciencenews.org/blog/context/higgs-owes-his-nobel-editor-and-biologist ; https://www.science.org/doi/10.1126/science.1218027 ; https://undsci.berkeley.edu/cells-within-cells-an-extraordinary-claim-with-extraordinary-evidence/the-community-reacts/ ; https://api.openalex.org (queried 4 Nov 2026)\nCritics: https://bactra.org/weblog/916.html ; https://royalsocietypublishing.org/rsnr/article/70/4/381/48595/Distrust-and-expertise-Can-scientific-journals ; https://cognitionandculture.net/blogs/pascal-boyer/can-we-please-have-science/index.html ; https://link.springer.com/article/10.1007/s11229-024-04559-2 ; https://www.nature.com/articles/s41593-023-01388-3 ; https://pmc.ncbi.nlm.nih.gov/articles/PMC11252173/ ; https://ceur-ws.org/Vol-523/deWaard.pdf ; https://www.semantic-web-journal.net/sites/default/files/swj127.pdf ; https://aclanthology.org/2021.sdp-1.7.pdf ; https://doi.org/10.7203/qfilologia.10.5083\n"}