{"path":"research/asteroid-and-feeling-around-sources.md","content":"# ASTEROID and feeling around sources\n\n*2026-10-27. Fredrik heard a podcast episode on the paper during a run and asked: \"Could it (the framework in an\nabstract sense) help us in any abstract way in terms of how future Deliberus microtasks/sweeps/extraction navigates\nunfamiliar landscapes on the internet or anything like that? Perhaps \"feeling around\" different data/argumentation\nsources? Think broadly and deeply about this and ideate.\" Claude's reading from the abstract and introduction (read\nin part, via the arXiv page); unruled, no ledger rows.*\n\nPaper: Poddar, Bao, Krantz, Chang, Puig, Mottaghi, Jaques, Gupta, *Asymmetric On-Policy Distillation Learns\nIn-Context Exploration* (arXiv:2610.30105, 21 October 2026; University of Washington and Meta).\n\n## 1. Verdict\n\nThe paper's gain does not carry over as it stands: it trains a policy's weights, in environments with a hidden state\nthat a privileged expert can see and a reward that says when the search succeeded. Deliberus trains no models, and an\nargument landscape has no hidden state that anyone can see in full. What carries over is the shape of the teaching,\nand two of its three parts are already in our methods. The third, a search that remembers what it ruled out, is the\nidea worth building when a microtask first goes looking for sources.\n\n## 2. What the paper does\n\nA student acts under partial observability and writes its own history (what it looked at, what it found). A\nprivileged expert, who sees the hidden state and never needs to explore, labels the student's next action given that\nhistory. The student is trained on those labels.\n\nTwo findings make it work:\n\n- **The labels are on the student's own trajectory.** Copying an expert's trajectories (offline imitation) teaches the\n  average expert action across possible hidden states, so the student \"dithers between hypotheses instead of resolving\n  them\". Labelling the histories the student actually reaches teaches it what to do from where it is.\n- **A failed probe is evidence.** In their example the object may be at A, B or C. Searched A, found nothing: every\n  label on that history is search(B) or search(C), so the student learns never to retry what it has ruled out, to\n  search while uncertain and to exploit once the hidden state is found. The paper likens this to posterior sampling.\n\nResults: about twice the returns under a fixed exploration budget across eleven environments, up to a hundredth of\nthe interactions reinforcement learning needs, better exploration than prompted frontier models, and +40% success on\na real robot working without vision.\n\n\n### 2a. At test time, with no teacher\n\nHis follow-up the same day: *\"But how could it then... navigate unknown rooms and landscapes? The model that runs on\nASTEROID in the end?\"* Claude's reading of the paper: across many training tasks the student learns two things, a\nprior over how rooms tend to be laid out, and how each probe rules hypotheses out. Deployed, its own history is its\nworking memory: every probe narrows the set of layouts still consistent with what it has seen, and it samples its\nnext move from what remains (the posterior sampling the paper names). No weights change. It generalises to a room it\nhas never seen when that room is drawn from the same kind of rooms; far outside them, the prior misleads it.\n\n## 3. Where it meets the corpus\n\n| The paper | Deliberus today | State |\n|---|---|---|\n| Teacher labels the student's own outputs, never ideal trajectories | Case review: a person reads what a microtask actually produced, one case at a time, and his reasons become rules (the principle *A person reads what a machine microtask produced…*; ledger Y9) | ruled, practised |\n| Teaching the method of search, not the answer | AM12: a microtask is taught the method; the graph's contents are never its lesson | ruled |\n| Prompted frontier models explore badly | `active-inference-context-acquisition-and-deliberus.md`: frontier models asked 1–3 questions worse than optimal, so the question policy is deterministic (`worth_asking`) | built for the descent |\n| Search while uncertain, stop once found | the ask-or-act stopping rule, display-band crossing | built for questions |\n| A failed probe recorded and never repeated | the record of machine judgments stores a call that concluded nothing; standing work counts a stop answer as read (R8) | built for pair reading; no source search exists |\n| A privileged teacher only at training time | the founder's rulings, a full-context reader, a hand extraction (run 3F) | exists as answer keys |\n\nSo the paper is outside support for two habits we already hold, measured where it can be: label the outputs the\ncheap microtask really produces, and do not trust a prompted model to explore well.\n\n## 4. Ideas for feeling around sources\n\nEach is a proposal, unbuilt.\n\n1. **A search that carries its own history of ruled-out hypotheses.** When a microtask looks for evidence on a claim (a\n   reference class's count, the opposing text, the study behind a statistic), it first writes the places the answer\n   could live, then probes them in turn, and each empty probe is a record that changes where it looks next. The record\n   of machine judgments already holds the history; what is missing is a microtask that reads its own past probes before the\n   next one. This is the paper's core move without any training.\n2. **Hypotheses about where, not only what.** The paper's hidden state is a location. For us the location is a kind of\n   source: a meta-analysis, a regulator's register, the other side's best text, a consultation answer. A microtask that\n   names its candidate kinds before searching can be checked: did it search the kind a person would have tried first?\n3. **Answer keys as the privileged teacher.** Where we know the hidden state by construction, we can check a search\n   policy without a person: a hand-mapped debate whose sources we hide, a claim whose reference-class count is known,\n   run 3F's implicit cruxes. This is the scarce-resource test gate, built once and used for every change to the\n   search instructions.\n4. **Distillation in context, at the method level.** Our version of the student's training data is a few worked\n   searches in the instructions, each showing a failed probe and the reasoning that moved the search on. AM12 permits\n   this because the examples carry method; they must never carry the answers to the claims being searched.\n5. **Training a small searcher, later.** The Z4 can run a small model; a frontier model with the full source could\n   label its next search action on its own histories, which is ASTEROID itself. Premature while no source search\n   exists and the hold on microtasks stands.\n\n6. **Both of the student's lessons, kept outside the model.** The prior over rooms is a record we can count: for\n   claims of each reference class, where their evidence was found last time (a register, a meta-analysis, the other\n   side's text), updated by every finished search. The elimination is the search's own record of empty probes. A\n   frontier model already brings a broad prior over where things live on the web; what the paper measures it lacking\n   is the second lesson, so the record supplies it. A room unlike any counted class falls back to broad search, and the\n   microtask says it is outside what it knows. This is the harness conviction in one case: the structure a trained student\n   carries in its weights is carried here in the record, where it can be read and argued.\n\n## 5. Where it does not reach\n\n- **Coherent absence.** The teacher in the paper sees the hidden state. Nobody sees what a corpus silently omits, so\n  no teacher can label the search that would have found it; the reserved human class stays reserved.\n- **No reward.** A found object ends the paper's episode. A search for evidence has no such moment: finding a source is\n  not finding that the claim holds. A search policy can be scored only on keys built by construction.\n- **Strategy-class adversaries.** A policy learned on fixed environments meets a web where sources are written to be\n  found. The paper's environments do not adapt.\n\n## 6. The offered test\n\nA hidden-source bench: take three hand-mapped debates, hide the sources behind their strongest premises, give a microtask a\nbudget of searches, and compare two instructions, one that keeps and reads its own failed probes and one that does\nnot. Measured by how often each finds the hidden source within the budget. Waits on his word, and on the hold on\nmicrotasks. **To be taken up in early November**, his word 2026-10-27: *\"We'll come back to the asteroid question in early november, before we plan the graph-saturation-with-microtasks bursts\"* (`TODO.md`). His second word the same evening, on the bench: *\"Wouldn't this be a good way to test early dev of opening up microtasks?\"* Logged as a candidate first run in `TODO.md`'s plan for switching microtasks back on.\n\n## Cross-references\n\n- [active-inference-context-acquisition-and-deliberus.md](active-inference-context-acquisition-and-deliberus.md) — ask\n  or act, and the stopping rule\n- [priors-as-standing-work.md](priors-as-standing-work.md) — reference classes, whose counts a source search would fill\n- [machine-work-as-a-standing-property.md](machine-work-as-a-standing-property.md) — the record a search would read\n- [derivability-of-missing-considerations.md](derivability-of-missing-considerations.md) — what a machine can reach\n"}