{"path":"research/storing-the-record-of-machine-judgments.md","content":"# Storing the record of machine judgments\n\n**Status (2026-10-06)**: **step one is built and live** (§ 7a), on the founder's word the same\nday: *\"Yes build it.\"*, then *\"Go ahead and deploy\"*. The two tables are in the production\ndatabase, the write-once rule is proven there, and every model call is recorded, on the live site\nsince 19:10 that evening and from the keeper's machine since 17:23. Steps two to six (§ 7) are not\nbuilt. The design below is as proposed; where the building changed it, § 7a says so.\n\nIt began as his question, after he had said yes to building the record: *\"Yes but how can we store\nit best to not swamp the graph/DB with this metadata? Can we make it as efficient as possible?\"*\nThen: *\"gimme your proposal for storage design first and persist the proposal\"*. Ledger rows Z1,\nZ1b and Z11 to Z13 ([session26-open-rulings.md](session26-open-rulings.md)).\n\n**What the record is**: one row for every call to a model, saying which model answered, under which\ninstructions, on which inputs, and what it replied. Why it is wanted, and why first:\n[what-a-record-linkage-project-teaches-deliberus.md](what-a-record-linkage-project-teaches-deliberus.md) § 3.\nIts first design: [machine-work-as-a-standing-property.md](machine-work-as-a-standing-property.md) § 3b.\n\n---\n\n## In plain language\n\n**It goes on disk, never into the graph.** The graph lives in memory, and memory is the scarce\nstore. The record goes in Postgres. The graph gets one whole number on each thing a machine wrote.\n\n**Every text is stored once.** A pass's instructions are the same on every call, so they are stored\nonce and each call points at them. A claim compared in forty pairs is stored once. A source read by\ntwenty calls is stored once. The prompt itself is never stored: it can be put together again from\nits parts, exactly.\n\n**So a judgment costs about a third of a kilobyte, plus its reply.** Everything the project has\never asked a model would take about 10 to 15 MB on disk. The graph takes 84 MB of memory today.\n\n**The larger saving is in calls, not bytes.** With every question fingerprinted, asking the same\nquestion twice can become a lookup. Storage here is megabytes. Calls are dollars and minutes.\n\n**And the graph's real weight is somewhere else.** About 64 of its 84 MB are embeddings kept in a\nwasteful form. Stored packed they would take 16.\n\n---\n\n## 1. What was measured (2026-10-06, read-only)\n\n| What | Measured |\n|---|---|\n| The graph, in memory | 84 MB, of which 83 are the properties on claims |\n| Text on claims (wording, reasons, quoted passages, titles, asserters) | about 1.7 million characters, under 2 MB |\n| Embeddings | on 3,895 claims, 1,024 numbers each, stored as plain number lists |\n| The same embeddings on a throwaway engine, 400 claims | 6 MB as lists, 1 MB packed. Search works on the packed form |\n| The production database, on disk | 10.5 MB in all |\n| Machine output of all 33 extractions, as text | 3.16 MB (0.93 MB as stored, compressed) |\n| Source texts | 0.39 MB in all, the largest 90,000 characters |\n| Calls the Claude engine has recorded since 23 August | 2,050, in 21 kinds, $104 at list price |\n| Calls the paid router engine has recorded | 64 |\n| Calls on the free route | **no record at all**: nobody can say how many |\n| A sameness reply | 700 to 1,000 characters, read from the 147 reasons the check has stored on links |\n| An extraction reply | 400 to 2,000 tokens, so 2 to 8 KB |\n| An extraction prompt | 1,500 to 9,000 tokens, so 6 to 36 KB |\n\nOne thing was measured for the design itself. The sameness check's prompts for the 147 live pairs\nare 300 KB as sent, 2,040 bytes each. Take the two claims out of each, and **one** instruction text\nof 1,746 bytes is left, the same on every call. The claims are 138 distinct texts of 156 bytes\neach. Stored once each, that is 23 KB where the prompts were 300: thirteen times smaller.\n\nThe estimates below are built from these pieces. They are estimates until the record exists and\ncan be asked.\n\n## 2. The design\n\n### 2a. Where\n\nIn Postgres. A compartment's process writes to its own database, by the switch that already\nselects it.\n\n**Never in the graph, for two reasons.** The graph is held in memory, so every property is paid for\nin memory. And everything on a claim is served publicly, while the record can hold words nobody\npublished (a person's draft is an input like any other).\n\n### 2b. Two tables\n\n**Texts.** Every distinct text, once.\n\n| Column | Holds |\n|---|---|\n| a small whole number | its name everywhere else |\n| a fingerprint (SHA-256) | unique, so the same text can never be stored twice |\n| the text | instructions, a claim's wording, a source, a reply, a model's name, an error message |\n| when it was emptied | set only when a person's words are erased (§ 2f) |\n\n**Judgments.** One narrow row per call. Every reference is a number pointing into the texts.\n\n| Column | Holds |\n|---|---|\n| when | the time of the call |\n| the pass | which step asked |\n| asked for, and answered | the model named in the request, and the model that answered, read from the reply |\n| instructions | the prompt with the inputs taken out (§ 2c) |\n| inputs | the texts put into the prompt, in order |\n| reply | the reply as received. Empty when the call failed |\n| outcome | found something · examined and found nothing · could not decide · failed |\n| about | a claim, a pair, a link or a source, by its identifier |\n| what it took | milliseconds, tokens in and out, cost, tries |\n| question key | a fingerprint of pass, instructions and inputs together (§ 2e) |\n\n*As built (§ 7a), three more: the form the reply had to fit, the whole prompt for a call whose\ninputs nobody named, and how many named inputs were not in the prompt as written.*\n\nAbout 175 bytes a row and about 100 more in indexes (estimated from the column sizes). Postgres\ncompresses a text longer than about 2 KB by itself.\n\n### 2c. Telling instructions from inputs\n\nThe caller hands the shared model-call function the texts it put into the prompt: the two claims,\nthe passage, the source. The function swaps each for a marker. What is left is the instructions,\nand **the fingerprint of the instructions is the pass's version**. Nobody types a version label, so\nnone can go stale.\n\nPutting the inputs back at their markers gives the prompt as sent, character for character. That is\ntested as a round trip on every write. If an input is not found word for word (it was escaped on\nthe way in, say), that call's prompt is stored whole and the row says so.\n\nA call site that passes no inputs is still recorded, with its prompt stored whole. A count of such\nrows shows which passes are not yet converted. Identical whole prompts are still stored once.\n\n### 2d. What the graph carries\n\nOne whole number on each machine-written claim and link: the judgment that wrote it. For today's\nten thousand items that is under a megabyte even at a generous hundred bytes each.\n\nIt replaces what three passes write today, each in its own shape: the splitter puts the model's\nname inside a record on the link, the title writer puts it on the claim, the relevance judge puts\nit on the link. Each is a string repeated on every item. In the record a model's name is stored\nonce.\n\nEverything else about an item is asked of the record: *what has ever been asked about this pair*\nis one indexed query.\n\n### 2e. The same question, asked twice\n\nThe same pass, the same instructions and the same inputs give the same question key. Before paying\na model, a pass can ask the record whether this exact question has an answer, and take it.\n\nThis is the larger saving. A run that is repeated, or resumed after a crash, pays nothing for what\nit already asked. A sweep after new claims arrive asks only the new questions. And the same\nquestion gets the same answer, by memory.\n\n**Off unless a pass opts in.** Asking again on purpose is how steadiness is measured, and an answer\nthat was wrong should not be frozen in. The standing-work design already proposes reuse when\nnothing a judgment depends on has changed\n([machine-work § 3b](machine-work-as-a-standing-property.md)); this is its key.\n\n### 2f. Write-once, with one exception\n\nThe database itself refuses a change or a deletion on a judgment. That is a rule in the database,\nsix lines, so it holds for every program that connects.\n\nA text can be emptied. When a person asks for their words to be erased, the text is found by its\nfingerprint and its content is removed. The rows that read it stay, and so does the fact that it\nwas read.\n\n### 2g. When the database cannot be reached\n\nThe row goes to a local file and is loaded at the next write that succeeds. A call is never lost to\nthe record and never held up by it. The log says so each time.\n\n### 2h. What is not stored\n\n- The prompt as sent. It is put together again from its parts.\n- The reply's parsed form. It can be parsed again.\n- Embeddings. They can be computed again.\n\n## 3. What it costs\n\n| What | On disk |\n|---|---|\n| One sameness judgment | about 1.2 KB: the row, its indexes, a reply of about 900 bytes |\n| The sameness check over all 1,729 candidate pairs | about 2 MB |\n| Everything the Claude engine has recorded since August, 2,050 calls | about 3 MB |\n| One extracted source, about 80 calls | 0.15 to 0.3 MB |\n| All 33 sources | 5 to 10 MB |\n| **Everything to date** | **about 10 to 15 MB. The production database is 10.5 MB today** |\n| A thousand sources | about 0.3 GB |\n| Ten thousand sources | about 3 GB |\n\nNone of it is in memory.\n\n**Against storing each call's prompt and reply whole**: thirteen times smaller on the sameness\ncheck's prompts (measured), and an estimated four to six times smaller for an extraction, where\none source text would otherwise be stored again by each call that reads it.\n\n## 4. What it replaces\n\nThree partial records exist today, in three shapes:\n\n- the Claude engine's file of calls on the keeper's machine (cost and time; no inputs, no reply);\n- the router engine's file (the same, with tokens);\n- the model names three passes write onto the graph (§ 2d).\n\nThe free route keeps nothing. One record takes the place of all of these.\n\n## 5. Considered and rejected\n\n| Alternative | Why not |\n|---|---|\n| Put the model, the version and the reason on each claim and link | It is paid for in memory, it repeats the same strings on every item, and what sits on a claim is public |\n| Judgments as nodes in the graph | The same memory cost, for something nobody travels through |\n| One row per call holding the whole prompt and reply | Simple. It stores a source once per call that reads it, and a pass's instructions thousands of times |\n| Keep a fingerprint of the reply and drop the reply | The smallest. The reply is the one thing that cannot be worked out again |\n| Local files, one per engine | That is today. They sit on one machine, cannot be asked together with anything else, and say nothing of what was written |\n| A separate store built for logs | Not needed at this size. Postgres is already there |\n\n## 6. Beside the record: where the graph's memory goes\n\nThe question was about swamping the graph. Measured, the graph's weight today is its embeddings:\n3,895 claims with 1,024 numbers each, stored as plain number lists at about 16 bytes a number. That\nis about 64 of the 84 MB.\n\nThe engine has a packed form at 4 bytes a number. On a throwaway engine 400 such claims took 6 MB\nas lists and 1 MB packed, and similarity search ran on the packed form with no conversion. Stored\npacked, the live graph would be about 36 MB.\n\nThe search code already notes that the embeddings were stored as lists and that changing the form\nwould also allow a real index (`deliberus/graph/vector_search.py`). This is a separate change from\nthe record. It needs one test before it is made: the same searches return the same claims in the\nsame order.\n\nA second, smaller duplication for later: the reasons on links (520,000 characters on 3,160 links)\nand on claims (290,000) are parts of replies, kept in memory. Once the record holds the replies,\na page could read a reason from the record when somebody opens it.\n\n## 7. Order of building\n\n1. The two tables, and the shared function writes a row for every call, prompt stored whole. This\n   is the part that cannot be made up later. **Built 2026-10-06 (§ 7a).**\n2. Each call site passes its inputs. There are 25, plus three calls that go around the shared\n   function (two that read a PDF, one that transcribes).\n3. The number on every machine-written claim and link, and the test that fails where a path writes\n   without one.\n4. The three model stamps on the graph are retired.\n5. Reuse by question key, one pass at a time.\n6. Packed embeddings, with their own test.\n\nStep 1 adds two tables to the production database. The live service creates them the next time it\nstarts, so for the live site to record, that step needs a deploy. Runs from the keeper's machine\ncan record before that.\n\n## 7a. Step one, as built (2026-10-06)\n\nThe founder, asked whether to build step one: *\"Yes build it.\"*\n\n**What exists.**\n\n- `deliberus/judgments.py`: the two tables, and the function `llm_call` now calls for every call,\n  failed ones included. A call site says what its call is about inside a `judgments.about(...)`\n  block: its inputs, what is being judged, the pass's name, and how to read the reply.\n- `scripts/judgment_facts.py`: what the record holds, counted pass by pass; `--prove`; `--show N`.\n- **The two tables are in the production database**, made from the keeper's machine at 17:23.\n- **One call site names its inputs**: the sameness check, as the pass `keeper.pair`. The other 24\n  are recorded with their prompt stored whole, and the counts say which.\n- 36 tests, on a store of their own, through the same code that writes to Postgres.\n- **Live on the site since 19:10 the same evening** (founder: *\"Go ahead and deploy\"*). Checked\n  there by one real sameness call made inside the running container: the record was on, its\n  folder usable, and the call wrote its row.\n\n**The write-once rule, fired on the live database.** `--prove` writes a made-up row inside one\ntransaction, tries each change, and undoes everything:\n\n| Tried | The database |\n|---|---|\n| Change a judgment | refused: *judgments are write-once: UPDATE refused* |\n| Delete a judgment | refused: *judgments are write-once: DELETE refused* |\n| Empty the judgments table | refused: *judgments are write-once: TRUNCATE refused* |\n| Rewrite a text | refused: *a judgment text can only be emptied* |\n| Give a text another fingerprint | refused: *a judgment text keeps its number, its fingerprint and its size* |\n| Delete a text | refused: *a judgment text is never removed, only emptied* |\n| Empty a text and say when | went through, as designed |\n| The store afterwards | unchanged: 0 judgments and 0 texts before, 0 and 0 after |\n\nOn a store without the rule (the tests' own) the same proof reports every forbidden change as\nhaving gone through and does not call itself proved, so the prover can fail.\n\n**The first real row showed something nobody could see before.** One sameness call from the\nkeeper's machine asked for `gemini-3-flash-preview`. That model was out of quota, the second in\nline was overloaded, and `gemini-3.1-flash-lite` answered. The row says both names. On the free\nroute there was no record at all until now (§ 1).\n\n**What the building changed in the design.**\n\n1. **The reply is stored as the validated form the pass was handed, not as the characters the\n   model sent.** All three engines hand the shared function a parsed object. § 2b said *the reply\n   as received*; storing that would need each engine changed, and is not done.\n2. **The form a reply must fit is part of the question.** It is stored as a text of its own and is\n   part of the question key. The same prompt with another form is another question, and a form\n   that changes is a new version of the pass that the instructions' fingerprint alone would miss.\n3. **A pass says how its reply reads when the row is written.** § 2b lists four outcomes (found\n   something, examined and found nothing, could not decide, failed). The shared function cannot\n   tell the first three apart, and a write-once row cannot be refined afterwards. So the call site\n   hands over a reading (`about(outcome=...)`), and a site that hands over none is recorded as\n   *answered*.\n4. **A named input that is not in the prompt as written is counted on the row.** Its prompt is\n   stored whole, as § 2c said; the count is how § 8's second test is run, pass by pass.\n5. **An input under 12 characters is kept and not swapped for a marker.** A number or a label\n   also occurs in the fixed instructions, and swapping it there would give the same pass a new\n   version on every call.\n6. **Blocks nest.** A caller further out can name what is being judged (the pair's two\n   identifiers), and the call site its inputs.\n7. **A new dependency: `psycopg`, a Postgres driver for ordinary code.** The shared function is\n   ordinary code, called from whichever thread asked, and the driver the project already had works\n   only inside async code. Rejected: a private event loop in a thread of its own around the\n   existing driver. That needs no new dependency and has more that can go wrong.\n8. **The module makes its own tables and rules**, in one transaction, so a script that never runs\n   the service's start-up has them too. A trigger is made only where it is missing.\n9. **In production the local file has a named folder, and a missing one costs nothing per call.**\n   Found by tracing the deploy before running it: the container's user is made with no home\n   directory, and the local file lived under the home directory by default. After every row it\n   writes, the record looks there for rows kept during an outage, and in the container that look\n   would have raised: an error with a traceback in the log for every model call on the live site,\n   and the row's number lost, though the row itself was written. No test had the case, since the\n   tests' folder was always usable. Now the look cannot raise, the service says once at its start\n   when the folder is unusable, and the deploy names `/app/data/judgments`, the one folder that\n   user owns.\n\n**What two real calls showed.** The same pair of claims was judged once from the keeper's machine\nand once from inside the live container. Both rows carry the same question key, so the record\ncounts one question asked twice. The second call added one text, its reply: the instructions, the\ntwo claims, the reply form and both model names were already stored. And both calls asked for\n`gemini-3-flash-preview` and were answered by `gemini-3.1-flash-lite`.\n\n**What step one does not do.**\n\n- 24 call sites have not named their inputs (step 2), and three calls go around the shared\n  function and are not recorded at all (two that read a PDF, one that transcribes).\n- No claim or link carries a judgment's number yet (step 3), and the three model stamps on the\n  graph stand (step 4).\n- Tokens and cost are filled in for the router engine only; the other two do not report them to\n  the shared function.\n- The two engines' own files of calls are unchanged.\n- A row kept in the local file inside a container is lost if the container is replaced before the\n  database comes back.\n- Nothing reads the record yet except the counting script.\n\n## 8. What would show this wrong\n\n- **Replies far larger than estimated.** One query a week after step 1 gives the real sizes.\n- **Inputs that often fail to be found word for word.** The count of whole-prompt rows shows it,\n  pass by pass.\n- **A pass whose inputs cannot be listed.** A pass that builds its prompt from a search result has\n  the search as an input too. The standing-work design names this: the search needs its own\n  version and a record of what it searched.\n\n## 9. Proposals (ledger Arc Z)\n\n| Ledger | Proposal |\n|---|---|\n| Z11 | **The record is stored in Postgres as two tables: every text once, by fingerprint, and one narrow row per call that points at texts by number.** The graph carries one number per machine-written item. A pass's version is the fingerprint of its instructions, computed by taking the inputs out of the prompt. Judgments are write-once in the database; a text can be emptied on a person's request |\n| Z12 | **A pass may answer a repeated question from the record**, by a key made of pass, instructions and inputs. Off unless the pass opts in |\n| Z13 | **Embeddings are stored packed**, about 16 MB where they take about 64, after a test that searches return the same claims in the same order |\n\n## Cross-references\n\n- [what-a-record-linkage-project-teaches-deliberus.md](what-a-record-linkage-project-teaches-deliberus.md)\n  § 3: why the record comes first.\n- [machine-work-as-a-standing-property.md](machine-work-as-a-standing-property.md) § 3b and § 5b:\n  the record's first design, and reuse when nothing a judgment depends on has changed.\n- [pass-quality-and-the-choice-of-model.md](pass-quality-and-the-choice-of-model.md) § 4c: three of\n  the 25 places that ask a model record which model answered.\n- [retrieval-instruments-beyond-cosine.md](retrieval-instruments-beyond-cosine.md): similarity\n  search inside the graph engine.\n- [store-the-uses-read-the-meaning.md](store-the-uses-read-the-meaning.md): why this record is the ground every computed reading stands on, word senses and a\n  graph of real things included.\n"}