{"path":"listening/2026-09-29-proving-reliability-and-bend.md","content":"# Proving a pass reliable, and what Bend offers: a listening walkthrough (2026-09-29)\n\n- **What it is**: the second listening piece of 2026-09-29, on the founder's request after the first,\n  while out walking: *\"The devil's in the details. For example, machine splitting and sameness\n  detection and merging (read up on both of these) can probably be trusted since (yet if and only\n  if) they are reliable enough on the microtasks in question. … I think this can be proven? Read up\n  deeply. Also look into the Bend programming language. … Then create a walkthrough explaining\n  carefully all your findings and reasoning. Don't use the turbo but rather the full version of V4\n  btw. Make that our default. Also in Ruby.\"*\n- **Rendered**: the full `eleven_v4`, voice *Lucy*, stability 0.5, similarity 0.8, speed 1.0, mp3 at\n  44.1 kHz and 128 kbps, with `scripts/render_listening.py`. Two requests of 8,563 and 6,690\n  characters, cut at the part boundary (the `---` line below), carrying 51 emotion tags; 14 minutes\n  34 seconds.\n- **What it cost**: 922 quota units for 15,253 characters (518 and 404); the monthly counter moved\n  from 2,180 to 3,102 of 10,000. The same rate of about 0.06 units per character as the morning's\n  render.\n- **Checked**: transcribed locally with the tiny Whisper model. All 34 paragraphs present, no tag\n  read aloud.\n- **Snapshot**: the ledger rows it mentions (T1 to T4, and S2 and S5 marked superseded) were open on\n  this date. Do not retro-edit; mark it superseded in this header if they move.\n- **⚠ Partly superseded the same evening**: T1's core became a canon line (*\"Yes.\"*), and part\n  five's *\"sobering number\"* (14 to 28 per cent on hard text tasks) came from zero-shot labels on\n  contested categories, a poor prior for specified tasks like ours\n  ([microtasks-in-broad-daylight.md](../research/microtasks-in-broad-daylight.md) § 1).\n- **Audio**: a local copy sits in `.private/local-only/listening-audio/` (not in git). The text below\n  re-renders it.\n- **Method**: [how-to-write-for-listening.md](how-to-write-for-listening.md) · **Research**:\n  [proving-a-pass-reliable.md](../research/proving-a-pass-reliable.md),\n  [bend-and-deliberus.md](../research/bend-and-deliberus.md)\n\n---\n\n## The rendered text, as delivered\n\n[warmly] Hi Fredrik. This one answers the two questions from your walk. First, can the machine's splitting, sameness detection and merging be proven reliable enough that review doesn't have to keep up? Second, could the Bend programming language help Deliberus? It runs about a quarter of an hour, in two halves. And it starts with a correction, because you caught me.\n\n[serious] Part one. What I got wrong this morning. In the major-transitions walkthrough I proposed two rules. One said a machine may check what a machine wrote, but may not build on it until a person has engaged with it. The other said the analogy daemon should propose no faster than people can review. Both ration machine work by how much people can review. [thoughtful] You said review doesn't need to keep up if the passes are reliable enough. You were right, and the reason sits inside the very idea I was leaning on.\n\n[curious] The chemist Manfred Eigen's error threshold doesn't limit how many copies get made. It limits the error per copy, multiplied by the length of the copying chain. So picture the splitter writing parts that nothing else builds on. If two in every hundred are wrong, then after a hundred splits, or a hundred thousand, two in every hundred are still wrong. The number of mistakes grows. Their share doesn't. [emphatically] So review doesn't have to keep pace with volume. It has to keep that error rate measured.\n\n[thoughtful] What does compound is depth. A part cut from a part that the machine cut, or a link drawn to one, inherits the error above it. At two per cent per step, two steps are wrong about four per cent of the time, five steps about ten, ten steps about eighteen. And the chain starts earlier than I'd assumed. Extraction is itself a machine step, and nobody has ever measured its error rate against a person's labels.\n\n[gently] So both rules are replaced, and I've marked the old ones superseded in the ledger. The new version of the first says a machine pass may build on machine output only as deep as its measured error rate allows, and with no measured rate, a person engages first. My old rule survives as the special case of a pass nobody has measured. The new version of the second says what has to stay small is the daemon's unexamined error, not its number of proposals. For the analogy daemon that may change little in practice, because kinship between domains has no answer key, so its error rate is hard to measure. But the reason is now the right one, and it governs every other pass.\n\n[short pause] Part two. What proven can mean. [serious] Never a zero. \"Never has to correct anything\" can't be shown. \"Wrong at most two per cent of the time, with ninety-five per cent confidence\" can. The recipe is simple. Draw items at random from what the pass actually wrote. A person labels each one without seeing the machine's answer. Count the wrong ones.\n\n[intrigued] There's a lovely old rule of thumb, from a nineteen eighty-three paper whose title asks, \"If nothing goes wrong, is everything all right?\" If none of n items is wrong, the error rate is below about three divided by n. So about sixty clean items shows it's under five per cent. About a hundred and fifty shows under two per cent. About three hundred shows under one. One wrong answer among them pushes those numbers up by more than half.\n\n[thoughtful] And a proof is a snapshot, so it needs a way to stay true. That's a small, steady trickle of random items, read with a sequential test that stops as soon as the evidence is clear. To tell \"at most two per cent wrong\" from \"at least five\", about ninety-five clean items confirm it, and every error found pushes the finish line back by about thirty. [warmly] That's the trust ladder, the rungs a machine pass climbs from proposing to trusted, with its automatic demotion finally in numbers.\n\n[short pause] Part three. Not every microtask has the same kind of truth. [curious] \"Is the splitter reliable?\" is itself a bundled claim, so I split it. Some questions have an answer key in the text itself. Did a part say something its parent never said? Did the cut drop something? Those can be proven tightly. Some have no key, but careful people agree, like whether a part reads on its own to a stranger. And some are genuinely contested, like exactly where to cut, or whether a part is a piece of the claim or evidence for it. There, the most anyone can show is that the machine agrees with a careful person as often as two careful people agree with each other. Past that, \"accuracy\" means nothing.\n\n[intrigued] Your own prior is testable right here. You said of splitting, in your words: \"I think logic would reign here. I don't expect different cuts to be a matter of taste, this is probably more a science than an art?\" If two people cut the same claims the same way most of the time, you're right, the cut has a key, and the splitter can be proven against it. The friends' round you asked for in August is exactly that measurement. [amused] I couldn't find anyone who has measured how often two people cut a claim the same way. You'd be first.\n\n[short pause] Part four. The devils, since you said that's where they live. [serious] Here are the seven that matter most.\n\nOne. People are the reference, so measure how often two people agree before trusting any machine score. For nearby tasks the answer is lower than you'd think. Annotators matching fact-checked claims across languages agreed only moderately.\n\nTwo. Checking what the pass wrote tells you how often its writes are wrong. It says nothing about what it failed to do, the bundle it never flagged. That needs its own sample, drawn from what it left alone.\n\nThree. The base rate. [intrigued] Suppose one in twenty candidate pairs is truly the same claim, and a detector finds nine tenths of those and wrongly calls two per cent of the rest \"the same\". It's ninety-eight per cent accurate, and nearly a third of the merges it proposes are wrong. What must be proven is how often its \"same\" is right, on the real stream.\n\nFour. Merging chains. A equals B and B equals C puts A and C together. At one per cent false \"same\", a cluster chained through ten links holds a false one about a tenth of the time. The famous cautionary case is a Semantic Web identity chain that falsely merged a hundred and seventy-seven thousand names.\n\nFive. Only blind labels prove anything about the machine. People shown its answer follow it, even when they notice it's wrong.\n\nSix. A proof covers one version. A new model or prompt resets it, and so does a new kind of claim. The splitter's first proposals were mostly long implicit premises. No ordinary extracted claim has ever been split.\n\nSeven. Every item can be right while the whole drifts. Reuse could quietly favour vaguer claims. Only measures of the whole population catch that.\n\n[short pause] Part five. The best tool in the literature. [warmly] Let the machine decline its uncertain items, and prove the bound only on the rest. A paper called Trust or Escalate does exactly this for model judges. Its judges answered between fifty-six and sixty-eight per cent of items while agreeing with people eighty to ninety per cent of the time. [intrigued] And here's the lovely part: the items they declined were the ones the people themselves disagreed on most. The machine's doubt tracked the task's own contestedness. That's the division of labour Deliberus wants. The plainly bundled claim gets split. The hard cut goes to a person. [serious] One warning from the same paper. Thresholds set from the raw agreement rate held the target in only about half to four-fifths of trials. Set by the bound, over ninety per cent. And a sobering number from another group: on hard text tasks, a guaranteed method saved only fourteen to twenty-eight per cent of the expert work. So expect partial automation, earned claim by claim, not total.\n\n[short pause] Part six. Where things stand. [gently] Nothing is proven yet. Every accuracy figure we have is a model reading another model's output. A second model called sixty-two to sixty-eight per cent of the written parts genuine, and the splitter's own check held four of its ten proposals back for invented content. Those numbers aren't proof, and they aren't reassuring either. The good news is that the instruments exist. Three review sittings, prepared batches you judge in one go, are built and waiting, and the splits one only needs a deploy. Your model-read census isn't wasted either. Combined with a small human sample it can, at best, halve the number of labels a proof needs.\n\n\n---\n\n[warmly] Second half. The Bend programming language, and what it could mean for Deliberus.\n\n[curious] Part seven. What Bend is now. There are two languages with the one name. The original Bend, from May twenty twenty-four, ran on a runtime made of interaction combinators: programs as little graphs, rewritten by local rules, so anything that could run in parallel did, on a graphics card, with no extra effort. [intrigued] Then, on the seventeenth of September, twelve days ago, the company launched a new Bend. It's a different language with a proof system, and it dropped interaction nets entirely. Its author said he \"could not make them as fast as lower-order variants on everyday hardware.\" [serious] The independent numbers agree. The original needed a top-end graphics card to beat a single core of an ordinary language. And the company's bigger bet, a symbolic search engine aimed at general intelligence, is proprietary, and its promised benchmark result was never published.\n\n[short pause] Part eight. The verdict. [firmly] Don't adopt it. Two reasons. First, Deliberus spends its time waiting on model calls and the database. A ten-minute recording took about twenty minutes to map, almost all of it in seventy-eight model calls, and when similarity search was slow, the arithmetic turned out to be one per cent of the cost. Bend speeds up arithmetic. Second, the new Bend is twelve days old, its README says the compiler is ninety-nine per cent AI-written and unaudited, and the company is four people. [thoughtful] Our own research on why projects like this succeed or fail names maintenance as the killer. A brand-new language is the biggest maintenance risk there is.\n\n[warmly] But four ideas from its family are worth taking, as habits rather than dependencies.\n\n[short pause] Part nine. Confluence. [curious] The deepest idea under the original Bend is a theorem by Yves Lafont: those graph rewrites reach the same result whatever order they happen in, because each one only touches its own little patch. Deliberus's graph is also built by local steps, extracting, linking, splitting, applied in whatever order sources arrive. So I asked: would the same sources, arriving in a different order, build the same graph?\n\n[intrigued] I measured it this morning, read-only, on the live graph. When a new source arrives, the linker compares each of its claims with the five most similar claims already there. So a pair only gets judged if it made the top five from the later source's side. Of about seventeen hundred candidate pairs, fifty-nine per cent are reachable from one side only. For those, whether they're ever judged depends on which source happened to come second. [gently] The good news: none of them are near-duplicates, so sameness is safe. What's exposed is supports and attacks in the middle range. The fix is cheap: judge a pair when either side ranks it. I'd estimate that means about forty per cent more candidates.\n\n[thoughtful] And you already ruled this property, for people. \"Contributions are added, never overwritten\" is confluence. Additions don't depend on order. Overwrites do, which is exactly why edit wars are won by whoever acts last.\n\n[short pause] Part ten. Laws for code, samples for judgments. [intrigued] The new Bend lets you write down laws a program must obey, and prove them. In one outside test, ninety-three lines of laws caught a deliberate off-by-one bug that sample outputs missed. [thoughtful] That splits Deliberus's proof question neatly in two. The strength calculation is code, so it gets laws: adding parts that follow from existing ones never raises a parent, a stronger supporter never lowers its target, the order links were added in doesn't matter. Model judgments, like \"is this a faithful split?\", get samples, from the first half. And the checkable part of a split can get a law too. A part may not contain a number, a date, or a name that its parent never mentioned. That's a check a few lines of code can run on every split.\n\n[warmly] The new Bend also trusts only a small kernel, and treats its large, AI-written compiler as untrusted. The corpus already had that exact picture from Lean: machine passes propose, a small set of rules decides what may be written. And Bend's critics found the same limit the corpus names. A kernel can check form. It can't check whether the laws say what you meant.\n\n[short pause] Part eleven. Evidence that must not count twice. [curious] The new Bend is \"affine\": a value can be used at most once. Deliberus has a resource like that. One piece of evidence can reach a claim along several paths, and counting it every time manufactures strength from nothing. The fair-share routing we built splits it into shares that sum to one. [intrigued] That turns out to be a known idea in the logic of programs, called fractional permissions: a resource split into fractions that never add up to more than the whole. It doesn't decide anything new, but it puts a mature theory behind a design that's still waiting for your call.\n\n[short pause] Part twelve. The vision. [thoughtful] The company is betting that general intelligence comes from symbolic search. But search needs something to search against: examples, laws, a checker. [emphatically] In deliberation, that specification is the very thing in dispute. So even if their bet wins, it wins where a specification exists, and the two kinds of error we reserve for people, what the corpus silently leaves out and the frame it grew inside, are exactly where none exists. The bet changes the tools on the machine's side. It doesn't move the line.\n\n[short pause] What I filed. Two research documents, one on proving reliability and one on Bend, with the literature report and the Bend report kept word for word. In the ledger, a new arc of four rows, with the two rules I proposed this morning marked superseded. And the to-do list has the proof steps, the symmetric linking fix, the invention law and the property tests. Ruby and the walkthrough script now use the full v4 model.\n\n[warmly] The questions to carry. First, and the one everything waits on: when do you want to sit the first proof, about an hour with the machine's answers hidden? Second, how much wrongness can you live with: is five per cent right for splits, and two per cent for merges, checked per cluster? Third, do the two replacement rules capture what you meant, that the machine may build on its own work only as deep as its measured error allows? And fourth, does a pair get judged when either side ranks it?\n\n[warmly] In one line: you were right, reliability can be proven, one microtask and one version at a time, and review only has to keep measuring, not keep up.\n"}