{"path":"research/antithesis-simulation-pilot.md","content":"# Deliberus: a local simulation pilot before real users\n\nResearch and pilot proposal, 2026-10-01. Not an implementation or an architecture ruling. The cross-project report with sources, pricing, and a detailed podcast-chapter paraphrase is at `~/dotfiles/docs/research/antithesis_simulation_testing_2026-10-01.md`. Antithesis is principally a later distributed fault-exploration platform, **not** a substitute for Deliberus ontology rulings, source-based semantic evaluations or real-person UX dogfooding. [Vendor prerequisites](https://antithesis.com/docs/faq/prerequisites/), [podcast publisher transcript](https://www.complexsystemspodcast.com/episodes/software-testing-with-will-wilson/).\n\n## Why this project is a plausible candidate\n\n`docs/architecture-reference.md` describes a live FastAPI/SvelteKit app whose extraction is orchestrated by Temporal and stores data in both Postgres and FalkorDB. Retries, timing and partial failures across these services can create state divergence despite passing unit tests. `tests/conftest.py` correctly sets test database and graph names before imports and blocks live model calls. That is essential, but **not whole-system isolation**: normal local development and production share service infrastructure per the architecture reference. `tests/test_graph_store.py` uses a named test graph on an existing server; the newer `cypher_graph` fixture uses an embedded engine. A stress/fault pilot must not target shared servers, even under a different database name. It needs dedicated, disposable instances and explicit guards against production endpoints.\n\n`tests/test_temporal_workflow.py`, the API/graph tests and `web/src/lib/*.test.ts` show an existing example/behavior-test base. Extend it with *sequences*: a request retried while a worker restarts; a model returning schema-valid but surprising content; a graph write succeeding while metadata storage fails; two users editing or correcting related claims concurrently. Antithesis can explore ordering and failures; it cannot decide whether a particular argument classification is philosophically or socially correct.\n\n## Pilot boundary and candidate oracles\n\nStart with FastAPI and **separate** throwaway Postgres, FalkorDB and Temporal instances, a small synthetic corpus, and local doubles for Gemini, auth and external HTTP. A driver makes API requests to create, correct and read data while injecting timeouts, 429/500 responses, duplicated delivery and restarts. Confirm that no call can leave the test environment or send real ntfy notifications. Run this locally first: [Hypothesis stateful testing](https://hypothesis.readthedocs.io/en/latest/stateful.html) and the [Temporal Python test environment](https://github.com/temporalio/sdk-python#testing) can provide sequence and retry tests without a commercial platform.\n\n**Candidate properties to ratify against current code and product rules, not assertions to paste in unexamined:**\n\n- A visible \"complete\" state entails the graph objects and Postgres metadata it claims to have stored; a failed store is visible rather than silently green.\n- Repeated delivery of one logical extraction does not create unintended duplicate public claims or links. If two attempts are intentionally retained for forensics, the product must distinguish them.\n- Provenance remains attached to the source text, contributor and input that produced a claim; a correction cannot silently change authorship.\n- A refused edge does not reappear as accepted through a separate read/write path.\n- A private draft or community-scoped contribution does not become public merely because another worker processes the same topic; scope survives retries and restarts.\n\nFor each property specify the authoritative stored field/API response, *when* it may be evaluated (during a workflow or only after recovery/convergence), and an independently implemented reference check. Otherwise the oracle may repeat the same product bug. Use explicit time and failure boundaries instead of requiring instantaneous cross-store consistency without a product promise to that effect.\n\n## What the pilot will not validate\n\nA fake Gemini can test schema handling, retry, timeout and storage consequences. It does not test recall of implicit premises, incorrect argument polarity, worldview attunement or whether a newcomer understands the map. Keep specimen-based LLM evaluations and the two-person from-scratch UX rehearsal; `CLAUDE.md` gives these high priority. Browser behavior can be probed separately with [Playwright mocks](https://playwright.dev/docs/mock) or [Bombadil](https://github.com/antithesishq/bombadil) against an isolated test origin. Replace Google OAuth with test identity. Do not run destructive exploration against public deliberus.com.\n\n## An open-source engine between the local tests and a commercial trial\n\n*Added 2026-10-03.* [ldgr](https://github.com/ldgr-rs/ldgr) describes itself as *\"the open, self-hostable deterministic simulation testing (DST) engine\"*. Read from its README, not run: it controls time, randomness, scheduling, network and storage, records each run in a journal, and can replay and shrink a failing run. Its stated limit is that code under test must run behind its own effects boundary (Rust written against its runtime, or a guest compiled to `wasm32-wasip1`); arbitrary external processes and OS threads are outside its model. This Python service stack therefore cannot be placed under it as it stands. A small model of one property from the list above could be written for it. The local pilot comes first either way, since it tests the real code. The engine is AGPL-licensed, a single-contributor project created in August 2026.\n\n## Decision gate for the commercial platform\n\nIf the local pilot finds real defects and the **whole relevant topology** can be packaged as offline Linux x86-64 containers, Antithesis may be worth a PoC for deterministic cross-service exploration. Its [currently published Quickstart order](https://antithesis.com/legal/poc_terms_2026_09_04/) states USD 4,000 for 30 days without assured single-tenant isolation. Before any such decision require synthetic data, no secrets in images/logs, reviewed properties, a measured local baseline and approved privacy/cost terms. No vendor contact, signup or purchase has been made.\n"}