{"path":"research/reasons-and-cases/real-cases-dependent-instances.md","content":"# Real cases: independent evidence against one pooled rate (helper report, verbatim)\n\n*A research helper's report for [../reasons-and-cases.md](../reasons-and-cases.md), 2026-10-23, kept as it\narrived. Brief: published situations where several pieces of evidence, or several instances, were combined\nas if independent, with the separate inputs known as numbers and a published corrected figure or ground\ntruth. Read-status tags are the helper's own; \"my arithmetic\" marks the helper's computations, which the\nnote recomputes where it relies on them.*\n\n---\n\n# Report: independent evidence vs. one pooled rate, real cases with numbers\n\nTags: [full text] = I read the passage that holds the numbers. [part] = I read excerpts or the result sections only. [abstract only]. [secondary] = number taken from someone else's account of the source.\n\"My arithmetic\" marks numbers I computed from published inputs. Everything else is as published. I wrote no files.\nQuotes are paraphrased on purpose. Verbatim wording is in the linked sources.\n\n---\n\n## 1. Sally Clark (R v Clark, 1999)\n\n**Inputs**\n- Meadow's single-death figure: 1 in 8,543 SIDS for a family like the Clarks (affluent, non-smoking parents, mother over 26). Source: CESDI study, Fleming et al. 2000, p. 92 [secondary, via Sesardic 2007 p. 302, which I read in full text].\n- Squared: 8,543² = 72,982,849, the \"1 in 73 million\" (my arithmetic; Hill 2004 p. 324 states it) [full text].\n- Meadow also said a double cot death would occur \"once in a hundred years\", using about 700,000 births a year (Hill 2004 p. 325) [full text].\n- General-population SIDS rate: 363 SIDS deaths in 472,823 live births, 1993–96, about 1 in 1,300 (Hill 2004 p. 321) [full text].\n\n**Evidence that a second death is more likely after a first (Hill 2004, p. 322)** [full text]\n- CESDI: 323 SIDS families had 5 previous SIDS deaths. 1,288 control families had 2.\n- Dependency factor = (5/323)/(2/1288) ≈ 10 (my arithmetic: 9.97).\n- Hill's alternative, using the expected number E = 0.914 in controls (E = T/1300, with 0.923 prior siblings per control family): factor ≈ 22 (my arithmetic: 21.8). He keeps 10 as the better estimate.\n- CONI programme (Waite et al., unpublished 2002 draft): 5.7. Norway (Øyen et al. 1996, Am J Epidemiol 144:300–305): 5.8 [abstract only, 95% CI 2.1–13.2].\n- Hill's conclusion: the risk is 5 to 10 times higher after a first SIDS.\n\n**Other recurrence estimates (all [secondary])**\n- Sesardic 2007 p. 313 lists 3.7 (Irgens 1984), 1.9 (Peterson 1986), 4.8 (Guntheroth 1990) and about 6 (Carpenter 2005). He argues the true factor is probably under 4.\n- Folbigg Inquiry, Counsel Assisting Part 2 Ch. 6, paras 110–118, reports:\n  - Carpenter 2005 relative risk ≥5.71 (4.10–7.74), from 6,373 CONI babies.\n  - Beal & Blundell 1988: 10.1 times the expected rate (21.2 per 1,000 siblings), but only 8% of families were high-risk.\n  - Emery 1986: baseline about 1 in 500, second death about 3 times that.\n- Sesardic p. 311 also lists 1980s–2002 SIDS authorities who said recurrence risk was not raised.\n\n**Combined figures**\n\n| Source | Figure |\n|---|---|\n| Meadow (published) | 1 in 73 million |\n| Meadow's own 1/8,543 baseline with factor 5–10 (my arithmetic) | 1 in 14.6 million to 1 in 7.3 million |\n| Sesardic, factor 4 (p. 313) | 1 in 18 million |\n| Hill on the whole-population baseline: (1/1300)·(x/1300), x = 5.7 or 10 (my arithmetic) | 1 in 296,000 to 1 in 169,000 |\n| Hill's reading of that last line | about 4–5 second cot deaths a year in the UK (p. 325) |\n\nThe independence error is about one order of magnitude. A second error mattered more: the unconditional comparison with double homicide.\n\n**Comparison with double homicide (Hill pp. 322–324)** [full text]\n- Single SIDS vs single homicide: 1/1300 against 1/21,700, about 17 to 1.\n- CONI, 33 second sudden unexpected deaths: 27 natural, 6 non-natural. Of the 27, 9 included an explained death, leaving 18 double SIDS. Of the 6, only 2 parents were convicted of both deaths. So double SIDS : double homicide = 18 : 2 = 9 : 1. Hill's range is 4.5 : 1 to 9 : 1.\n- Implied homicide dependency factor y = 176 (my arithmetic reproduces 176.5).\n- Sesardic disputes this (Table 1, pp. 307–308, 316). His single-event prior ratio S/M is 16.4 with no covert homicide among SIDS, falling to 3.1 at 20%. With Joyce's 0.1 recurrence-given-murder figure and 10% misclassified SIDS, his prior favours double murder about 25 to 1. Hill's own recurrence-given-murder figure is 0.0078 (Sesardic p. 315).\n\n**Royal Statistical Society statement, 23 Oct 2001** [full text]. https://rss.org.uk/RSS/media/File-library/Membership/Sections/2020/Sally-Clark-RSS-statement-2001.pdf\n- Squaring is valid only if SIDS cases arise independently within families. No empirical justification was given.\n- There are strong a priori reasons to expect dependence, such as unknown genetic or environmental predisposition.\n- The 1 in 73 million \"has no statistical basis\". The error is likely large and in one direction.\n- It also flags the prosecutor's fallacy.\n- The follow-up RSS letter of 23 Jan 2002 is [secondary].\n\nSesardic criticises the RSS for asserting dependence without data. His own survey of the data still gives a factor of 2–6.\n\nOutcome: conviction quashed in 2003 on newly disclosed microbiological evidence (Sesardic p. 300).\n\nSources\n- Hill R, Paediatr Perinat Epidemiol 2004;18:320–326, doi 10.1111/j.1365-3016.2004.00560.x. Read at https://www.researchgate.net/publication/8348032\n- Sesardic N, Brit J Phil Sci 2007;58:299–329, https://www.stat.berkeley.edu/users/aldous/157/Papers/SIDS.pdf\n- Folbigg Inquiry: https://2019folbigginquiry.dcj.nsw.gov.au/content/dam/dcj/2019-folbigg-inquiry/documents/Counsel_assisting_Part_2_Chapter_6.pdf\n\nWhat it tests: squaring a population rate treats two deaths as independent draws. Pooling them as instances of one family-level risk gives a factor of about 5–10 (range 2–22 across estimates). The pooled counts are tiny (5 vs 2), so the factor is itself uncertain.\n\n---\n\n## 2. People v. Collins, 68 Cal.2d 319 (1968)\n\n**The six probabilities** (court's fn. 10, pp. 325–26; reprinted in Fairley & Mosteller 1974) [full text]\n- Partly yellow car 1/10.\n- Man with mustache 1/4.\n- Girl with ponytail 1/10.\n- Girl with blond hair 1/3.\n- Negro man with beard 1/10.\n- Interracial couple in car 1/1000.\n- Product 1/12,000,000 (10·4·10·3·10·1000, checked). The prosecutor also claimed \"one in a billion\".\n- The Justia and Stanford copies omit the table. The values above come from Fairley & Mosteller and a Temple law blog [secondary]. The opinion's own text confirms yellow car, mustache and ponytail (p. 327) and the 1/4 × 1/10 beard-and-mustache issue (fn. 15).\n\n**Court's critique** [full text, Justia]\n- No evidence for any of the six values (pp. 327–28).\n- No proof of independence (pp. 328–29). Bearded men and mustached men overlap; the court says multiplying 1/4 by 1/10 has no basis (fn. 15).\n- Conceptual error (pp. 330–31): the product is at best the chance a random couple has the traits, not the chance the defendants are guilty.\n- Appendix (pp. 333–35): P(a second such couple | at least one) = [1 − (1−Pr)^N − N·Pr·(1−Pr)^(N−1)] / [1 − (1−Pr)^N]. The court puts this at \"approximately 41 percent\" when N ≈ 12 million.\n- My arithmetic: the exact value at N = 12M is 0.4180. The limit as N grows with Pr = 1/N is (e−2)/(e−1) = 0.41802.\n\n**Later statisticians: Fairley & Mosteller, \"A Conversation about Collins\", Univ. Chicago Law Review 41(2):242 ff. (1974)** [full text]. https://chicagounbound.uchicago.edu/cgi/viewcontent.cgi?article=3802&context=uclrev\n- The dependencies are obvious. The lawyer in their dialogue says the interracial-couple probability, given a blond girl and a bearded Negro man, must be about 1000 times the 1/1000 used.\n- A reading more favourable to the prosecution: the six numbers could be sequential conditional probabilities. Then multiplying them is valid, but they cannot also be the marginal probabilities.\n- Their example: the marginal for \"Negro man with beard\" in 1964 was perhaps 1/50, not 1/10.\n- The court's 41% is correct only if N ≈ 12M. Using the court's own Pr = 1/12M with N = 250,000 gives P(>1 | ≥1) ≈ 0.01 (my exact value 0.0104).\n- If Pr = 1/25,000 with N = 250,000, the odds are about 2,200 to 1 that another such couple exists (my exact value 2,200 : 1).\n- Kingston (1966) built an explicit dependent model that reached a figure near 1/12,000,000 (fn. 6).\n\nMy arithmetic stress test (illustrative, not published): set mustache-and-beard to 1/10 (full overlap) and drop the 1/1000 (implied by blond girl plus bearded Negro man). The product becomes 1/3,000, which is 4,000 times larger than 1/12,000,000.\n\nSources\n- https://law.justia.com/cases/california/supreme-court/2d/68/319.html\n- https://law.temple.edu/aer/?p=2281\n\nWhat it tests: multiplying six marginals treats traits as independent. It is valid only for chain-rule conditionals. Unsupported marginals and the leap from rarity to identity are separate errors.\n\n---\n\n## 3. Efron & Morris: baseball batting averages\n\nSource: JASA 70(350):311–319 (1975), Tables 1 and 2, p. 313 [full text]. https://jhanley.biostat.mcgill.ca/bios602/MultilevelData/EfronMorrisJASA1975.pdf\n- y = average after the first 45 at-bats.\n- p = rest-of-season average (treated as truth).\n- AB = rest-of-season at-bats.\n- X and θ = arcsine-transformed values.\n- JS = Stein estimate, retransformed (Table 2).\n\n| i | Player | y | p | AB | X | θ | JS |\n|---|---|---|---|---|---|---|---|\n| 1 | Clemente | .400 | .346 | 367 | −1.35 | −2.10 | .290 |\n| 2 | F. Robinson | .378 | .298 | 426 | −1.66 | −2.79 | .286 |\n| 3 | F. Howard | .356 | .276 | 521 | −1.97 | −3.11 | .281 |\n| 4 | Johnstone | .333 | .222 | 275 | −2.28 | −3.96 | .277 |\n| 5 | Berry | .311 | .273 | 418 | −2.60 | −3.17 | .273 |\n| 6 | Spencer | .311 | .270 | 466 | −2.60 | −3.20 | .273 |\n| 7 | Kessinger | .289 | .263 | 586 | −2.92 | −3.32 | .268 |\n| 8 | L. Alvarado | .267 | .210 | 138 | −3.26 | −4.15 | .264 |\n| 9 | Santo | .244 | .269 | 510 | −3.60 | −3.23 | .259 |\n| 10 | Swoboda | .244 | .230 | 200 | −3.60 | −3.83 | .259 |\n| 11 | Unser | .222 | .264 | 277 | −3.95 | −3.30 | .254 |\n| 12 | Williams | .222 | .256 | 270 | −3.95 | −3.43 | .254 |\n| 13 | Scott | .222 | .303 | 435 | −3.95 | −2.71 | .254 |\n| 14 | Petrocelli | .222 | .264 | 538 | −3.95 | −3.30 | .254 |\n| 15 | E. Rodriguez | .222 | .226 | 186 | −3.95 | −3.89 | .254 |\n| 16 | Campaneris | .200 | .285 | 558 | −4.32 | −2.98 | .249 |\n| 17 | Munson | .178 | .316 | 408 | −4.70 | −2.53 | .244 |\n| 18 | Alvis | .156 | .200 | 70 | −5.10 | −4.32 | .239 |\n\n**Published results (1975)**\n- Total squared prediction error on the arcsine scale: raw averages 17.56, James–Stein 5.01. Efficiency 3.50.\n- The estimate of 1/(1+τ²) is (k−3)/V = 0.791, so τ = 0.514. The weight on a player's own value is 0.209. X̄ = −3.275. The rule is 0.209·Xi − 2.59 (p. 312).\n- James–Stein is closer than the raw average for 15 of 18 batters. It is worse for Clemente, Swoboda and E. Rodriguez.\n- Limited-translation versions: efficiency 3.91 (s = 0.9) and 3.01 (s = 0.8).\n- Maximum single-player error: 2.17 for raw averages, 1.08 for James–Stein (Munson).\n- Second example (chi-square size simulation, Table 6, p. 317), total loss:\n  - raw estimates 18.9\n  - Stein, one shrinkage for all 17 values 10.2\n  - separate Stein rules for two groups 9.9\n  - \"bigger constant\" variant 8.9\n  - all estimates fixed at .05 (complete pooling) 23.4\n\n**Scientific American 1977 (Efron & Morris, 236(5):119)** [part, text of a copy at https://gwern.net/doc/statistics/bayes/regression-to-mean/1977-efron.pdf]\n- Total squared error on the raw scale: .077 for raw averages, .022 for James–Stein (3.5 times).\n- Grand average .265, shrinking factor c = .212, z = .265 + .212·(y − .265).\n- Closer for 16 of 18 players. Clemente .400 → .294 (truth .346). Munson .178 → .247 (truth .316).\n\n**My recomputation from the table** (reproduces the published numbers)\n- Hits recovered as round(45·y).\n- Arcsine-scale error 17.57 (raw) and 5.00 (James–Stein). (k−3)/V = 0.7910.\n- My X̄ = −3.317, against the paper's −3.275. That is a minor discrepancy in the paper.\n- Raw-scale error 0.0754 (raw averages), 0.0213 (the 1977 formula) and 0.0215 (the 1975 retransformed values).\n- Full pooling, all 18 set to the grand mean .2654: 0.0243. Complete pooling captures most of the gain, and partial pooling is slightly better still.\n\nWhat it tests: each player's rate estimated from his own 45 at-bats versus a start learned from sibling players. Independence of players is assumed (Efron & Morris state it) and used to estimate the common prior.\n\n---\n\n## 4. Serial or repeated diagnostic tests with correlated results\n\n### 4a. Repeat faecal immunochemical test (FIT), Oxford, raw counts\nSource: Fanshawe et al. 2024, \"A review of methods for the analysis of diagnostic tests performed in sequence\", Table 2 and Table 3 [full text]. https://pmc.ncbi.nlm.nih.gov/articles/PMC11370044/ (journal title not verified). Threshold 10 µg Hb/g; reference standard is colorectal cancer.\n\n| First FIT | Second FIT | Cancer | Non-cancer |\n|---|---|---|---|\n| + | + | 8 | 63 |\n| + | − | 1 | 66 |\n| − | + | 0 | 48 |\n| − | − | 1 | 1122 |\n\nTotal 10 cancers, 1,299 non-cancers.\n\n**Published**\n- First-test sensitivity 0.900 (0.555–0.997). Specificity 0.901.\n- Observed correlation between repeat tests: ρ+ = 0.667 in cancers, ρ− = 0.479 in non-cancers. I reproduced both from the table.\n- OR rule (positive on either test), observed: sensitivity 9/10 = 0.900 (0.596–0.982), specificity 1122/1299 = 0.864.\n- Under conditional independence: sensitivity 0.990, specificity 0.811.\n- Their Table 3 for the OR rule, under assumed ρ = 0 / .25 / .50 / .75:\n  - sensitivity .990 / .968 / .945 / .922\n  - specificity .811 / .834 / .856 / .878\n- The paper warns that 10 cancers is very few.\n\n**My arithmetic, using the table**\n- A false positive on the first test is followed by a false positive on the second in 63/129 = 0.488. The marginal rate is 111/1299 = 0.0855, a ratio of 5.7.\n- Likelihood ratio for double positive, observed: (8/10)/(63/1299) = 16.5.\n- Naive product of the two tests' positive LRs: 9.06 × 9.36 = 84.8 (82.1 if the same LR is squared).\n- Likelihood ratio for double negative, observed: 0.116. Naive product: 0.0243. This rests on a single false negative.\n\n### 4b. Serial rapid antigen tests for SARS-CoV-2\nSource: Soni et al., Ann Intern Med 2023, doi 10.7326/M23-0385 [part, results sections]. https://pmc.ncbi.nlm.nih.gov/articles/PMC10321467/\n- Design: 7,361 enrolled, 5,353 eligible. Antigen tests plus RT-PCR every 48 h for 15 days. 154 participants had at least one positive RT-PCR: 97 asymptomatic and 57 symptomatic.\n- Sensitivity of 1, 2 and 3 antigen tests, days 0–6 after first positive RT-PCR, by number of tests (95% CI where given), with independence prediction 1−(1−s)^n (my arithmetic):\n\n| Group | 1 test | 2 tests | 3 tests | Independence for 2 and 3 |\n|---|---|---|---|---|\n| Symptomatic | 82.5% (78.3–86.3) | 93.4% (90.4–95.9) | 94.3% (91.4–97.0) | 96.9%, 99.5% |\n| Asymptomatic | 34.4% (28.8–39.8) | 55.3% (48.2–61.6) | 68.5% (61.0–75.7) | 57.0%, 71.8% |\n| Asymptomatic, singleton PCR positives excluded | 38.8% (32.7–45.2) | 62.7% (57.0–70.5) | 79.0% (70.1–87.4) | 62.5%, 77.1% |\n\nThe PubMed abstract gives 54.7–70.0 for that 62.7% interval, so there is a minor discrepancy between sources.\n\n- Day-0 only (testing starts at the first positive PCR):\n\n| Group | 1 test | 2 tests | 3 tests | Independence for 2 and 3 |\n|---|---|---|---|---|\n| Symptomatic | 59.6% (46.7–71.4) | 92.2% (81.5–96.9) | 93.6% (82.8–97.8) | 83.7%, 93.4% |\n| Asymptomatic | 9.3% (5.0–16.7) | 39.3% (29.8–49.7) | 56.4% (45.4–66.9) | 17.7%, 25.4% |\n\nMy reading, not the authors': pooled over days 0–6, the symptomatic repeat tests add less than independence predicts (positive dependence given infection). On day 0 the second test beats the independence prediction, because sensitivity rises with time since onset. The tests are therefore not exchangeable draws from one rate. The shared factor, viral load over time, acts in both directions.\n\n### 4c. Simulation and methods papers\n- van Walraven et al., J Clin Epidemiol 2009;62(12):1301–5 [abstract only]. Multiplying the odds by the second test's LR overestimates disease probability when tests correlate. With perfect correlation, the true probability after two positives equals that after one. The error exceeded 25 percentage points in some combinations.\n- Gardner et al., Prev Vet Med 2000;45:107–122 [abstract and introduction only]. Positive dependence in sensitivity lowers the combined sensitivity of \"either test positive\". Positive dependence in specificity lowers the specificity of \"both positive\". Moderate-to-high positive dependence in sensitivities in all three diseases studied (swine toxoplasmosis, swine brucellosis, cattle Johne's disease). Covariance values: not found.\n- Vacek, Biometrics 1985;41:959–968 [abstract only]. Assuming conditional independence when it is false can substantially underestimate both tests' error rates. Numbers: not found.\n\nWhat it tests: multiplying likelihood ratios of repeated tests treats each as independent evidence. The data show a shared latent factor (disease burden or measurement artifact) making the second result carry far less than the first. The time-varying case shows the factor can also work the other way.\n\n---\n\n## 5. Forecast aggregation with an extremizing factor (Good Judgment Project)\n\n**Meaning of the factor.** a multiplies the mean log-odds. a = 1 is plain averaging of log-odds, which is right if everyone has the same information. a = N, the number of forecasters, is summing log-odds, which is right if everyone has independent, calibrated evidence on a common prior.\n\n**Fitted values**\n\n| Source | Result |\n|---|---|\n| Baron et al. 2014, Decision Analysis 11(2):133–145, Table 1, p. 140 [full text; I read the page image] | Optimal a, with leave-one-question-out SE, and mean Brier score at optimum vs a = 1 (80 two-option questions, year-1 data) |\n| – high self-rated expertise, mean | 2.43 (0.05); Brier 0.148 vs 0.187 |\n| – high expertise, median | 1.78 (0.03); Brier 0.160 vs 0.176 |\n| – low expertise, mean | 3.08 (0.09); Brier 0.139 vs 0.196 |\n| – low expertise, median | 2.34 (0.06); Brier 0.153 vs 0.184 |\n| Satopää et al. 2014, IJF 30:344–356; read via the thesis chapter, Satopää 2016, https://repository.upenn.edu/edissertations/1991 [part; the IJF paper itself is paywalled] | 69 questions, over 1,300 forecasters; 86–647 forecasters per question (mean 235.1). Brier-optimal a \"around 2.0\". 99% CI for a: [1.161, 3.921]. Mean absolute difference between the arithmetic mean and the extremized forecast: 0.175. |\n| Satopää, Pemantle, Ungar, arXiv 1406.2148v5 [part] | 44 superforecasters, 123 events (23 occurred), 17–34 forecasts per event (mean 24.2). Brier: mean of probabilities 0.132, log-odds average 0.128, partial-information aggregator 0.123. |\n| Atanasov et al., SSRN 4691513 (Wharton PDF) [part] | Season-4 settings from seasons 1–3: a = 1.5 (sub-elite teams), 1.32 (elite teams). Extremizing market prices: Brier 0.173 → 0.161 at a = 1.32, 0.158 at a = 1.75 (hindsight-optimal). |\n| Metaculus test by a forum commenter (~847 questions) [secondary; https://forum.effectivealtruism.org/posts/sMjcjnnpoAQCcedL2/] | a = 2.5 gave no gain (weighted Brier 0.115–0.116 vs 0.116–0.117 unextremized; log score worse). a = 1.5 won on recent questions. |\n\nAtanasov et al. 2017 (Management Science) prints the team parameter with its decimal point lost in my text extraction (\"105\"), and says a = 2 would have improved Brier by under 2%. Value not verified.\n\n**Theory linking a to information overlap**\n- Satopää–Pemantle–Ungar, location of the optimal factor under compound symmetry: x0 = N/(1+(N−1)λ) · √((1−δ)/(1−δ′)), where λ is the overlap fraction. Full overlap gives no extremizing; zero overlap gives a factor of order N.\n- Neyman & Roughgarden 2022 (arXiv 2111.03153) [part]. Independent signals: extremizing by k recovers the truth exactly. Identical signals: a factor of 1. The minimax single factor is 2k/(k+1). With a known prior and \"projective substitutes\" signals, the limit is √3 ≈ 1.73.\n\nMy reading: with N ≈ 235 forecasters per question, independence would imply a ≈ 235 and full redundancy a = 1. The fitted a ≈ 2 (CI 1.16–3.92) is about as if each question had two independent forecasters' worth of information. Underconfidence confounds this reading.\n\nWhat it tests: summing log-odds across forecasters is right only for independent information. The fitted factor sits between averaging and summing, near averaging. It falls to 1.3–1.5 for teams and elite forecasters who share information.\n\n---\n\n## 6. Naive Bayes calibration\n\n**Niculescu-Mizil & Caruana, ICML 2005, pp. 625–632** [part; https://www.cs.cornell.edu/~alexn/papers/calibration.icml05.crc.rev3.pdf]\n- Naive Bayes pushes predicted probabilities toward 0 and 1. The opposite distortion shows up in boosted trees, SVMs and boosted stumps. Neural nets and bagged trees are close to calibrated.\n- Setup: 8 problems; models trained on 4,000 cases and calibrated on 1,000.\n- Squared error and log-loss for naive Bayes improve dramatically after Platt or isotonic calibration. A sigmoid is not the right correction for naive Bayes. Isotonic is better.\n- Numeric values appear only in Figure 8 (bar chart). Not found as numbers.\n\n**Zadrozny & Elkan, ICML 2001, pp. 609–616, Table 1** (KDD'98 data, test set 96,367 records; MSE) [part]. https://cseweb.ucsd.edu/~elkan/calibrated.pdf\n\n| Predictor | Training MSE | Test MSE |\n|---|---|---|\n| Naive Bayes | 0.10089 | 0.10111 |\n| Binned (calibrated) naive Bayes | 0.09546 | 0.09528 |\n| All-base-rate constant | 0.09636 | 0.09602 |\n| All zero | 0.10151 | 0.10113 |\n\nUncalibrated naive Bayes scores worse than predicting the base rate for everyone. Its ranking is still good. The authors attribute the extremeness to positively correlated attributes.\n\n**Bennett, CMU-CS-00-155 (2000)** [part; https://apps.dtic.mil/sti/tr/pdf/ADA385120.pdf]\n- Naive Bayes posteriors go to 0 or 1 exponentially with document length.\n- Reuters-21578: standard train/test split 7,769 / 3,019 documents. Earn is 37.03% of the training set and Corn 2.33%. The classifier almost always outputs a posterior near 0 or 1. Recalibrating with a sigmoid cut MSE (Corn: MSE 0.0003) but not F1.\n- Other numeric results in that report: not read.\n\n**Domingos & Pazzani, Machine Learning 29:103–130 (1997)** [full text]. https://ics.uci.edu/~pazzani/Publications/mlj97-pedro.pdf. Pages are approximate.\n- Section 4 (pp. 111–112) works the duplicated-attribute case: with A = B, naive Bayes effectively counts A twice. The optimal rule picks \"+\" when q > 1−p. Naive Bayes picks \"+\" when q > (1−p)²/(p²+(1−p)²). They disagree only in two narrow regions.\n- Their illustration: true P(+|E) = 0.51 but the naive Bayes estimate is 0.99, still the right class.\n- My arithmetic: p = 0.6, q = 0.35 with equal priors. The true posterior is 0.447 (pick −). Naive Bayes gives 0.548 (pick +).\n- 28 UCI datasets, average accuracy (Table 2, p. 109): Bayes 79.1, Gauss variant 77.8, C4.5 77.2, PEBLS 77.6, CN2 76.2. The R² between average attribute dependence and the accuracy gap to rivals was 0.04 (C4.5), 0.0004 (PEBLS) and 0.002 (CN2).\n\nWhat it tests: summing log-likelihood ratios of correlated features overcounts, giving overconfident probabilities (worse than the base rate in the KDD'98 case). The sign of the sum, and the ranking, often survive.\n\n---\n\n## 7. Same-design studies combined as independent, later found to share a bias\n\n### 7a. Hormone therapy: observational overview vs trial\n- Stampfer & Colditz, Prev Med 1991;20:47–63 [abstract only]. https://pubmed.ncbi.nlm.nih.gov/1826173/\n  - All studies pooled: relative risk of CHD 0.56 (0.50–0.61).\n  - Internally controlled prospective and angiographic studies only: 0.50 (0.43–0.56).\n  - 15 of 16 prospective studies showed a reduction.\n- Women's Health Initiative, JAMA 2002;288:321–333, Table 2 [part]. https://jamanetwork.com/journals/jama/fullarticle/195120\n  - CHD events 164 of 8,506 on estrogen plus progestin (0.37% a year) against 122 of 8,102 on placebo (0.30% a year). HR 1.29 (1.02–1.63).\n- Nelson et al., JAMA 2002;288:872–881 [part]; AHRQ evidence review https://www.ahrq.gov/downloads/pub/prevent/pdfser/hrtcvdser.pdf\n  - Current users, CHD incidence: RR 0.80 (0.68–0.95).\n  - Studies that controlled for socioeconomic status: 0.97 (0.82–1.16).\n  - Studies that did not adjust for socioeconomic status: current-user RR 0.72 (0.61–0.84) in AHRQ.\n- Ann Intern Med 2002;137(4), doi 10.7326/0003-4819-137-4-200208200-00012 [abstract only] (author list not shown in my extract)\n  - Studies not adjusting for socioeconomic status: current-user RR 0.71 (credible interval 0.64–0.78).\n  - Adjusted studies: 0.97–1.11.\n  - Adjusted for socioeconomic status and risk factors: 1.07 (0.79–1.48).\n- My arithmetic: ln(1.29/0.56) = 0.835. The standard error of the 31-study pooled log RR is about 0.051, so the trial result lies about 16 standard errors from the observational pool.\n\nShared factor: healthy-user and socioeconomic confounding across all the observational studies.\n\n### 7b. Magnesium in myocardial infarction\n- Teo et al., BMJ 1991;303:1499–1503 [abstract + excerpts]. Seven small trials: 25 deaths of 657 on magnesium (3.8%) against 53 of 644 on control (8.2%). Typical OR 0.45 (0.28–0.71), p < 0.001.\n- ISIS-4, Lancet 1995;345:669–685 [abstract]. 5-week mortality: 2,216 (7.64%) of 29,011 on magnesium against 2,103 (7.24%) of 29,039 on control.\n- Per-trial counts for 22 trials are in https://wviechtb.github.io/metadat/reference/dat.li2007.html (Li et al., Cochrane 2007). My recomputation from that table:\n  - First seven small trials (Morton to Singh): 16/392 against 47/406, inverse-variance OR 0.355 (0.194–0.650).\n  - 21 trials without ISIS-4: OR 0.819 (0.736–0.912).\n  - All 22 trials: OR 0.993 (0.941–1.048).\n  - ISIS-4 crude OR 1.059.\n- Egger & Davey Smith, BMJ 1995;310:752 [abstract only]. The seven small trials were enough to reach p < 0.001 by 1990. The effect was then called effective and simple by Yusuf et al. (1993).\n- Egger et al., Lancet 1998 letter [part]:\n  - Only 2 of 15 magnesium trials described concealment of allocation, against 17 of 20 streptokinase trials.\n  - All magnesium trials except ISIS-4 were single-centre.\n  - The funnel plot was clearly asymmetric.\n- Sterne et al., BMJ 2011;343:d4002 [part]. ISIS-4 has 90% of the fixed-effect weight. Heterogeneity I² = 68%. MAGIC (2000): OR 1.0 (0.8–1.1).\n- Contrast, the same method working: for streptokinase, 15 trials were enough for p < 0.001 by 1977, later confirmed by the mega-trials GISSI-1 and ISIS-2 [abstract only]. I did not retrieve the table with per-trial numbers.\n\n### 7c. Ego depletion\n- Hagger et al. 2010 meta-analysis: d = 0.62 (0.57–0.67) over 198 tests [via Carter & McCullough 2014, Front Psychol, https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2014.00823/full; part].\n- Carter et al. 2015, J Exp Psychol Gen, doi 10.1037/xge0000083, as reported in the next paper [secondary]:\n  - Uncorrected g = 0.43 (41% unpublished).\n  - Trim-and-fill g = 0.24.\n  - Bias-corrected (PET-PEESE) g = 0.003.\n- Multi-lab preregistered replication, Hagger et al., Perspect Psychol Sci 2016, doi 10.1177/1745691616652873 [part]. 23 labs, N = 2,141. d = 0.04 (−0.07 to 0.15). Heterogeneity I² = 36%. Only 3 of 23 labs had confidence intervals excluding zero (one of those negative).\n- My arithmetic: the 2010 pooled estimate is about 24 of its own standard errors from zero. The replication estimate is about 10 of its own standard errors below it.\n\n### 7d. Bem's precognition experiments\n- Bem et al. 2016 meta-analysis, F1000Research 4:1188 v2 [Table 1 and abstract]. https://pmc.ncbi.nlm.nih.gov/articles/PMC4706048/\n  - 90 experiments, 12,406 participants. g = 0.09 (0.06–0.11), z = 6.33, p = 1.2×10⁻¹⁰, Bayes factor 5.1×10⁹.\n  - 69 independent replications (10,082 participants): g = 0.06 (0.03–0.09), z = 4.16, Bayes factor 3,853.\n  - The earlier preprint reported Bayes factor 1.4×10⁹.\n- Kekecs et al., Royal Society Open Science 2023, 10:191375 [part]. https://royalsocietypublishing.org/rsos/article/10/2/191375/91837/\n  - Preregistered multi-lab replication of Bem's Experiment 1. 2,220 participants (2,115 contributed valid data), 37,836 erotic trials.\n  - 49.89% successful guesses. 99.75% CI 49.11–50.67%. Bem's original: 53.07% of 1,560 trials.\n  - Data 72 times more likely under chance than under the better-than-chance model. All four preregistered tests supported chance.\n- My arithmetic: replication z = −0.43; Bem's original z = +2.43.\n\n### 7e. Antidepressant trials (selective publication)\nTurner et al., NEJM 2008;358:252–260 [part]. 74 FDA-registered trials, 12,564 patients, 12 drugs. 38 of 74 (51%) positive by the FDA. In the journals: 48 of 51 published studies reported positive results (94%). Mean effect size 0.31 (0.27–0.35) from FDA data against 0.41 (0.36–0.45) from journals, a 32% increase (11–69% per drug).\n\n### 7f. Meta-analyses against later large trials\n- LeLorier et al., NEJM 1997;337:536–542 [abstract and excerpts]. 12 large RCTs, 19 meta-analyses, 40 outcomes. Kappa 0.35 (0.06–0.64). Positive predictive value 68%, negative 67%. The meta-analysis failed to predict the trial 35% of the time.\n- Egger et al., BMJ 1997;315:629–634 [abstract only]. Of 8 meta-analysis–large-trial pairs, 4 were discordant. In all 4 the meta-analysis showed the larger effect. Funnel-plot asymmetry was present in 3 of the 4 and in none of the concordant pairs.\n\n### 7g. Extra case: Lucia de Berk, p-values multiplied across wards\n- Elffers' one-ward model: 14 incidents in her 201 shifts, from 27 incidents in 1,704 shifts. Hypergeometric probability for exactly 14 is 1 in 3.4 million; the tail P(≥14) is 1 in 3.1 million (my arithmetic; the paper says about 1 in 3 million).\n- Elffers multiplied the separate ward p-values and reported 1 in 342 million. Gill reports this multiplication as incorrect. Correct stratified analysis (Mantel–Haenszel): about 1 in 100,000 [part; https://pub.math.leidenuniv.nl/~gillrd/elfferscorrected.pdf].\n- Allowing nurse-to-nurse variation (a Poisson–exponential mixture, so a geometric distribution with tμ = 3.18486): P(≥14) = 0.021864, about 1 in 46 (my arithmetic reproduces it). Gill, Groeneboom & de Jong, Chance (c. 2018) [part]. https://diamhomes.ewi.tudelft.nl/~pgroeneboom/Lucia_Chance.pdf\n- Ground truth: the case ended in acquittal.\n\nWhat section 7 tests: whether repeated same-design studies are independent draws or share a filter (publication selection, confounding, small-study quality). In each case the pooled confidence interval is far too narrow against the independent check.\n\n---\n\n## Compact comparison\n\n| Case | Published combined | Corrected or ground truth | Source of correction |\n|---|---|---|---|\n| Clark | 1/73M | about 1/7M to 1/18M with factor 5–10 or 4 | Hill 2004; Sesardic 2007 |\n| Collins | 1/12M | no valid number; P(second couple) 0.01 to 0.42 depending on N | Fairley & Mosteller 1974; court appendix |\n| Repeat FIT | naive LR(++) 84.8 | observed 16.5 (10 cancers) | Fanshawe 2024 data |\n| Serial antigen, symptomatic | independence 96.9% / 99.5% | 93.4% / 94.3% | Soni 2023 |\n| GJP extremizing | sum implies a ≈ N ≈ 235 | fitted a ≈ 2 (1.16–3.92) | Satopää 2014 |\n| Naive Bayes (KDD'98) | MSE 0.1011 | base rate 0.0960; calibrated 0.0953 | Zadrozny & Elkan 2001 |\n| HRT | RR 0.56 (0.50–0.61) | HR 1.29 (1.02–1.63) | WHI 2002 |\n| Magnesium | OR 0.45 (0.28–0.71) | OR 1.06 (ISIS-4) | ISIS-4 1995 |\n| Ego depletion | d = 0.62 | d = 0.04 (−0.07 to 0.15) | Hagger 2016 |\n| Bem | Bayes factor 5.1×10⁹ | 49.89% hits vs 50% chance, 72 times for chance | Kekecs 2023 |\n| Lucia de Berk | 1/342M | about 1/100,000 stratified, about 1/46 with heterogeneity | Gill et al. |\n\n---\n\n## Not found or only partly verified\n- Numeric calibration values for naive Bayes from Niculescu-Mizil & Caruana 2005 (only a bar chart).\n- The Satopää et al. 2014 IJF paper's own text (paywalled). I used the thesis chapter that reproduces it; Sevilla's forum post quotes the same 99% interval.\n- The decimal in Atanasov et al. 2017's team recalibration parameter.\n- The number of forecasters per question in Baron et al. 2014.\n- Gardner 2000 covariance values and Vacek 1985 numbers.\n- Fleming et al. (CESDI 2000) itself, and the Egger 1995 streptokinase table. I relied on Hill, Sesardic and abstracts.\n- The Stampfer & Colditz overview and the Egger 1995 and 1997 papers: abstracts only.\n"}