{"path":"architecture-reference.md","content":"# Architecture reference — the built system, in detail\n\n**Extracted from `CLAUDE.md` on 2026-08-29** under the rule that always-loaded memory preloads what\nmust fire *before you would think to look*, and references what fires *reactively*. Everything here\nis the second kind: you come looking for it when you touch that part of the system.\n\n**The gotchas that must fire preemptively stayed resident** in `CLAUDE.md` § Hard-won gotchas — the\nEnum→Literal rule that caused a 15-day silent outage, the no-silent-except rule, Jinja escaping, the\nTemporal registration requirement, the async-engine loop binding, the Kamal secrets pattern, and test\nDB isolation. If you are reading this file for a rule rather than for a description, check there first.\n\n**Contents**: the 8-pass extraction pipeline · shared data architecture and the Darwin port map ·\nmonitoring and notification tiers · deployment · security · authentication · the web app · graph\nvisualisation · the FastAPI route inventory · the FB scraper · Claimify · the graph database · voice\nmemos.\n\n---\n\n## Extraction Pipeline (built Mar 28, scheme detection + CQs added Mar 29-30)\n\n**Plain-language overview**: `docs/extraction-pipeline.md` is the concise pedagogical explanation of the live 8-step extraction path. Use it when orienting newcomers or describing the system to non-programmers. Use the research docs below for design reasoning and tradeoffs.\n\n**8-pass pipeline** with instructor + Pydantic + Gemini 3 Flash, orchestrated by Temporal:\n1. **Holistic scout** — verbatim thesis quotes, attribution, argument landscape\n2. **Focused extraction (DnDScore split)** — 2a: DECOMPOSE into raw atoms in author's words, 2b: DECONTEXTUALIZE + CLASSIFY (stranger test, four types, epistemic modality, `short_title` for graph display). **Note (2026-08-27): only 2b's output is stored.** `AtomicClaim.text` is the decontextualized form and the graph has no field for the raw atom, so DnDScore's own resolution to the decomposition/decontextualization tension — keep the pair, read the atom in the context of the augmented form — is not expressible here. Consequence: when a user attacks a claim, whether they are attacking the author's assertion or the context extraction inserted is undecidable from the record\n3. **Relationship detection + scheme classification** — supports/attacks/qualifies/reframes/decomposes_into + Walton's 96 argument schemes } parallelized\n4. **Contested concept detection** — semantic + criterion contestedness                                                                      } via Temporal parallel activities\n5. **CQ generation** — critical questions parameterized from detected schemes (CQs ARE premises). Stranger test applied\n6. **Self-evaluation** — per-claim and per-relationship checks with an `issue` and a `suggested_fix` on each (stranger test, entailment, atomicity, type, epistemic status; direction and type for relationships). **No score: `overall_quality_score` was removed 2026-08-18** because the prompt requested a number with no rubric, so the quantity was undefined and uncalibratable; the issue counts are now derived in code from the booleans in the same response\n7. **Store** — FalkorDB (graph) + Postgres (JSONB)\n8. **Embed + auto-link + auto-connect** — Darwin's Qwen3-Embedding-4B generates embeddings, SIMILAR_TO edges at 0.80 cosine threshold, cross-extraction SUPPORTS/ATTACKS discovery. **Upgraded 2026-08-20 beyond cosine** (`docs/research/auto-connect-upgrade.md`): candidates also arrive via shared concepts (polysemy sense-gate) and deterministic structural-kinship signatures — measured 58% new pairs on the run-6 source; agreement edges get a write-time stance caution (same-fact-opposite-use); and `scripts/propose_cross_source_premises.py` proposes the unwritten cross-source crux premises (propose-only, operator-gated)\n\n**Post-store weighing pass** (stream path, Jul 2026): after claims store, `process_extraction_weighings` runs the provenance split — source-text weighings auto-open the CQ1–8 descent, authored/sacred input gets an invitation (`deliberus/weighing.py`). Deterministic lexical detection v1; its register ceiling is empirically mapped (philosophy says \"outweigh\", policy says \"net benefits\", rights-advocacy says neither — the semantic tier is the documented next step; `docs/research/dogfood-run-2-orthogonal-experiments.md` G10). Decompose-created children get the same detection in authored mode. NOT yet on the Temporal/PDF path.\n\n**PDF ingestion** (Mar 31, 2026): Gemini reads PDFs natively via `inline_data` (`mime_type: application/pdf`, base64-encoded) — sees tables, figures, formatting. No `pymupdf`/`pdfplumber` dependency. Scout pass sends PDF directly; text extracted via separate Gemini call for subsequent passes. `POST /extract/pdf` endpoint with drag-and-drop frontend. PDF saved to temp file (avoids Temporal's 2MB payload limit). Instructor doesn't support multimodal — `call_genai_with_pdf()` in `client.py` bypasses instructor, uses genai SDK directly with `response_schema` for constrained JSON + Pydantic validation. Tier 1/2 PDFs (<30 pages) work now; Tier 3 (book-length) needs RLM pattern. See `docs/research/extraction-pipeline-design.md` §RLM.\n\n**Evidence-as-subgraph** (exploratory): uploaded PDFs extracted by same pipeline into linked claim subgraphs. See `docs/research/scheme-bounded-decomposition-and-evidence-as-subgraph.md`.\n\n**Performance**: Pass 2 runs all arguments concurrently (ThreadPoolExecutor, max 8 workers). Source text in system message for Gemini implicit caching (~85% input token cost reduction). Combined: ~10 min → ~2-3 min per extraction. See `docs/research/llm-speed-optimization.md`.\n\n**Tech stack**: Gemini 3 Flash via `instructor.from_provider(\"google/gemini-3-flash-preview\")` (v2 API, GENAI_TOOLS mode). Response models use `Literal` types, NOT `Enum` — GENAI_TOOLS strict validation rejects enum instances (returns strings). System messages work natively with `from_provider`. Direct genai SDK for PDF multimodal passes. Direct FalkorDB via falkordb-py (scoped to `deliberus_extraction` — NEVER write to other graphs). Python/uv, FastAPI REST API, SvelteKit web app.\n\n**⛔ RÄTTELSE 2026-08-13 (mätt, inte antaget):** allt ovan om separata nycklar och isolerade fakturakonton är HISTORIA. Samtliga varianter — `GEMINI_API_KEY`, `DIM_GEMINI_API_KEY`, `DELIBERUS_GEMINI_API_KEY`, `LILLEN_LLM_API_KEY`, `GEMINI_API_KEY_BRFAUTO` — är **samma nyckel**, verifierat genom strängjämförelse. Prepay-kontot `0127AC` avslutades 21 juli; huvudkontot `01DCA2` **spärrades 8 augusti** för obetalda 3 202,19 kr (juni + juli). Gemini svarar därför gratisnivå, **20 anrop per dygn och modell, delat över alla projekt**. Trettio dagars frist till ~7 september innan kontot och dess projekt avslutas. Full bild + belopp + åtgärder: din-mamma `docs/user-context.md` § Gemini / Google Cloud billing (RÄTTELSE 2026-08-13).\n\n**Gemini capacity fallback (Jul 6, 2026)**: Google pools shed load under demand spikes (503 UNAVAILABLE, *load-dependent* — tiny pings succeed while 24K-char extraction calls get rejected, so health checks lie). `llm_call` in `deliberus/extraction/client.py` falls through `gemini-3.5-flash` → `gemini-3.1-flash-lite` on capacity errors ONLY (validation/auth/schema errors raise immediately — never masked by a model swap); per-model config strips temperature for 3.5 per Google's guidance. `DELIBERUS_GEMINI_MODEL` env var repoints the primary without a code change (operational lever for capacity incidents). All extraction passes MUST route through `llm_call` — never `client.chat.completions.create` directly. **⚠ Gemini key routing (Jul 10–12, 2026): Deliberus reads `DELIBERUS_GEMINI_API_KEY` first, falling back to `GEMINI_API_KEY` — both currently resolve to the shared `brf-auto` billing key, NOT a dedicated Deliberus key.** The personal *prepay* key depleted Jul 9 (429 RESOURCE_EXHAUSTED, all extraction down prod + dev); `.env` + `.kamal/secrets` were reverted to the `brf-auto` *postpay* key and redeployed so extraction runs during scarcity without pre-loading credits (fabric moved with it; run-3F unblocked). **This is deliberate and temporary — do NOT \"fix\" it as a misconfiguration, and do NOT assume Deliberus owns an isolated key.** The `DELIBERUS_GEMINI_API_KEY` name was added Jul 12 to end a silent-shadowing bug: a fleet-wide shell `GEMINI_API_KEY` (from `tier-all.env`) was overriding the project's local `.env` key, because `python-dotenv` `load_dotenv()` defaults to `override=False` (an already-set shell var beats `.env`). That broke LOCAL dev extraction while prod stayed fine (containers have no shell shadowing). Per-project var names make the collision structurally impossible: the client reads its own name; prod reaches the value via the fallback. Repoint Deliberus's key independently by setting `DELIBERUS_GEMINI_API_KEY` in `tier-all.env`. ~~Billing 429s stay un-masked by the capacity-fallback chain (that only catches load-shedding 503s).~~ **FEL, rättat 2026-08-13 genom att läsa koden:** `_CAPACITY_MARKERS` i `client.py:178` innehåller både `\"429\"` och `\"RESOURCE_EXHAUSTED\"`, så kedjan faller över ÄVEN på kvotslut. Det är tur, för sedan 8 augusti ligger nyckeln på gratisnivå med 20 anrop per dygn och modell, och kedjan ger därmed extraktionen ungefär tre gånger så många anrop som primärmodellen ensam. Kvaliteten sjunker i motsvarande grad. The account / prepay-vs-postpay / key-routing rationale is private → `~/Projects/din-mamma/docs/user-context.md` § Gemini / Google Cloud billing.\n\n**Async-engine loop binding (Jul 6, 2026 — recurrence-risk gotcha)**: the pooled asyncpg engine binds connections to the event loop that created them. Sync code running in the threadpool (the `/extract/stream` SSE generator) must NEVER call async DB functions via `asyncio.run()` — fresh loop per call, pooled connection reuse dies with \"attached to a different loop\", and before Jul 6 this silently ate stream-path Postgres saves (claims in the graph, extraction row missing, page 404s, stream still reporting success). Correct pattern: `asyncio.run_coroutine_threadsafe(coro, app.state.main_loop)` — the main loop is captured at lifespan. The stream result event now carries `postgres_saved` and a store-error SSE event fires on failure.\n\n**Instructor migration** (Mar 31, 2026): `from_genai()` → `from_provider()`. `GENAI_STRUCTURED_OUTPUTS` → `GENAI_TOOLS` (for `dict` fields). `Enum` → `Literal` in all Pydantic response models (GENAI_TOOLS returns strings that strict validation rejects as non-enum-instances). System messages (`{\"role\": \"system\"}`) work with `from_provider` but NOT with `from_genai` GENAI_TOOLS mode.\n\n**CRITICAL: Enum → Literal rule is not optional** (reforged Apr 15, 2026 after the 15-day Apr 1 outage): **Every field annotation on a Pydantic response model used by `instructor.from_provider` + `GENAI_TOOLS` mode MUST be a `Literal[...]` alias, never an `Enum` class.** Gemini returns plain strings; Pydantic v2's `is_instance_of` check on the enum class returns False for raw strings even when the enum inherits from `str`. The validation error is 1 per field per claim — a typical response produces 5–30 errors and instructor gives up. If the failure is then swallowed by an `except Exception: return empty` block, the entire pipeline silently degrades to zero claims with no diagnostic signal. **After any response-model refactor**, `grep \"class.*: (Claim|Trust|Edge|Contested|Claim|Methodology|Evidence|Epistemic|Relationship)[A-Z]\" deliberus/extraction/models.py` and verify every field annotation is `*Lit` (Literal), not the bare enum class. Define a `*Lit = Literal[...]` alias next to the enum class so both exist — enum members for code value constants, Literal for response model fields. The project has a regression test (`tests/test_models.py::test_claim_kind_accepts_plain_string`) that enforces this round-trip for `ClaimKindLit`; add analogous tests for any new Literal alias. Full incident: [session13-notification-snr-and-extraction-outage-rca.md](research/session13-notification-snr-and-extraction-outage-rca.md) §Problem 3.\n\n**CRITICAL: No silent `except Exception: return empty`** (reforged Apr 15, 2026): The `focused._extract_one` silent-except was the proximate cause of making the Apr 1 regression invisible for 15 days. The rule from global CLAUDE.md (\"Silent capability degradation is the deadliest bug class\") applies to every extraction-pipeline helper: **if a try-block wraps an LLM call and falls back to an empty result on any exception, the except block MUST call `logger.exception(...)` with enough context (argument label, input preview, model, etc.) to reconstruct what failed.** A `print` is not a logger and is invisible in production. A bare `except Exception: return []` is a hidden capability-degradation bomb. Every future fallback in the pipeline must follow `_extract_one`'s updated pattern: log first, fall back second.\n\n**Extraction attempts forensics** (Apr 15, 2026): new table `extraction_attempts` persists every TEMPORAL-path submission. **Known gap (Jul 6, 2026, dogfood F5, OPEN): `/extract/stream` (the inline path) writes NO forensics row — its failures are invisible to this table.** Port pending; see `docs/research/dogfood-run-1-friction-log.md` F5/F7 (the stream path also lacks cleanup-at-start, so re-extracting a source via stream duplicates claims). Row is written by `_start_temporal_extraction` BEFORE `client.start_workflow`, updated via `workflow_lifecycle_activity` on terminal status (`completed` | `empty` | `failed`). Columns: `workflow_id` (PK), `user_email`, `input_type`, `url`, `title`, `pdf_filename`, `text_preview` (first 500 chars), `status`, `source_id` (resolved during fetch), `n_arguments`, `n_claims`, `error`, `created_at`, `updated_at`. Indexed on `(user_email)`, `(status)`, `(created_at DESC)`. This is the forensics layer: every failure — fetch error, scout empty, focused validation, storage crash, timeout — leaves a row regardless of which branch the workflow took. **Debug starting query**: `SELECT workflow_id, user_email, status, url, text_preview, n_arguments, error, created_at FROM extraction_attempts WHERE status IN ('failed','empty') ORDER BY created_at DESC LIMIT 20;`. See `docs/research/session13-notification-snr-and-extraction-outage-rca.md` §Problem 2.\n\n**Jinja template escaping**: instructor's `from_provider` parses `{{` as Jinja template syntax. Wikipedia references (`{{cite web}}`), LaTeX, and other content with double braces crash the template parser. All source text MUST be escaped (`{{ → { {`) before passing to instructor. See `_escape_jinja()` in `scout.py` and `focused.py`.\n\n**URL text extraction**: `trafilatura` (v2.0) extracts clean article body from HTML — strips navigation, sidebars, footers, ads. Markdown output preserves `##` headings for section-based chunking. Replaces regex HTML stripping.\n\n**Section-based chunking**: Texts >30K chars split by `##` headings, scouted in parallel (up to 6 concurrent workers), results merged. Handles Wikipedia-scale articles. See `deliberus/extraction/scout.py` `_split_into_sections()`.\n\n**Batch embeddings**: `embed_all_claims()` sends all claim texts in one batch HTTP call to Darwin's embedder (was 99 sequential calls). ~100x faster. See `deliberus/graph/linker.py` `get_embeddings()`.\n\n**Voice transcription**: Audio sent as `inline_data` to Gemini (not Files API which has async ACTIVE state wait). Same pattern as PDF ingestion.\n\n**Results on UBI Wikipedia (DnDScore)**: 14 arguments → 74 atomic claims → 26 relationships → 4 contested concepts. Previous pipeline: 46 claims, 4 definitional. DnDScore: 74 claims, 20 definitional — 5x better at detecting definitions as separate assertions. See `docs/research/first-pipeline-run-analysis.md`.\n\n**Claim linking**: Darwin's Qwen3-Embedding-4B (darwin.home:8080/v1, ~41ms, 1024 dims) embeds every claim. `SIMILAR_TO` edges auto-created above 0.80 cosine threshold. API: `/claims/{id}/similar`, `/claims/search`, `/embeddings/generate`. Backend for P10 flywheel + P13 composing=retrieval. See `deliberus/graph/linker.py`.\n\n**Storage**: FalkorDB (graph: claims, relationships, concepts, CQs, embeddings) + Postgres (extraction metadata + JSONB combined output, user accounts). No JSON files on disk — Postgres is the sole source of truth for extraction data. See `deliberus/database.py`.\n\n\n## Shared Data Architecture (Mar 31, 2026)\n\n**One graph, one database, two workers.** Local dev (Mac Mini) and production (Darwin container) share ALL data stores — FalkorDB, Postgres, embedder. Extracting from either machine writes to the same graph and the same Postgres. \"One graph, not separate canvases\" applied to infrastructure.\n\n- **FalkorDB**: `darwin.home:6380` — shared, same graph (`deliberus_extraction`)\n- **Postgres**: `darwin.home:5432` — shared, LAN-exposed. Tables: `extractions` (JSONB + `input_type` + `pdf_data` BYTEA + `pdf_filename` + `owner_id`), `extraction_attempts` (forensic row per submission, see Extraction Pipeline §forensics), `users` (identity). No single reputation score — see `docs/research/epistemic-gamification.md` for the 4-signal architecture\n- **Embedder**: `darwin.home:8080` — shared, Qwen3-Embedding-4B\n- **Temporal**: `darwin.home:7233` — shared server, **separate task queues**: production uses `deliberus-extraction` (set via `TEMPORAL_TASK_QUEUE` in deploy.yml), dev defaults to `deliberus-extraction-dev`. This prevents local code changes from affecting production workflows while sharing the same data stores\n- **Why separate task queues**: Temporal round-robins activities between all workers on the same queue. If local code differs from deployed code (mid-development), a production extraction could be processed by untested local code. Separate queues ensure you always know which code version processed which extraction\n- **Postgres CLI on Darwin**: `psql -h localhost -U deliberus -d deliberus_production` (set `PGPASSWORD` or use `.pgpass`). LAN-exposed on port 5432 for Mac access: `psql -h darwin.home -U deliberus -d deliberus_production`\n- **Local dev dotenv**: `database.py` loads `.env` via `python-dotenv` at import time so `DATABASE_HOST=darwin.home` is picked up before engine construction. All modules that read env vars at import time must load dotenv first\n- **Test isolation** (Mar 31, 2026): `tests/conftest.py` sets `DATABASE_NAME=deliberus_test` before any module import. `load_dotenv()` doesn't override existing env vars, so tests never hit production. Without this, `save_extraction()` in tests wrote to `deliberus_production` — a \"Test\" extraction appeared on deliberus.com. Shared data is right for dev↔prod (same data, different task queues), but tests are a third category requiring explicit isolation\n\n**No chunking**: Full text in context for small texts. RLM pattern for large texts. See `docs/research/extraction-pipeline-design.md`.\n\n**Cost tracking**: NOT yet implemented. Estimated ~$0.27-0.37 per extraction (Gemini 3 Flash). Auto-connect costs are capped at `new_claims × top_k(5)` LLM calls regardless of graph size — linear in new claims, not in graph density. Needs: token count logging per activity, cumulative cost display per extraction. See `docs/research/scheme-bounded-decomposition-and-evidence-as-subgraph.md` §Extraction Cost Analysis for detailed breakdown.\n\n**CQ generation fix** (Mar 31, 2026): Two cascading issues. (1) `dict[str, str]` generates `additionalProperties` rejected by `GENAI_STRUCTURED_OUTPUTS` — fixed by switching to `GENAI_TOOLS`. (2) GENAI_TOOLS returns enum values as strings, strict Pydantic validation rejects them — fixed by switching Enum fields to `Literal` types + migrating to `from_provider()`. All CQs backfilled. The correction UX is fully functional.\n\n\n### Blobs: separate contents, same code (2026-09-19)\n\nA **blob** is a separate FalkorDB graph plus a separate Postgres database on the same servers, both\nnamed `deliberus_blob_<name>`, selected by the single variable `DELIBERUS_BLOB=<name>`\n(`deliberus/blob.py`, read by `deliberus/graph/store.py` and `deliberus/database.py` at import). The\npublic app is connected to a different graph and a different database, so nothing in a blob can reach\na public surface. **Never implement community-private material as a hidden flag inside the public\ngraph**: a flag holds only while every reader and the linking pass remember the filter. The switch\nderives both store names, wins over the environment file, and refuses a blob-shaped store name set by\nhand, so a blob cannot be half-configured. Operations: `scripts/blob.py` (`create` needs the blob's\nwritten membrane on file first; `list`; `dissolve --confirm`). Hand-made extractions go in through\n`scripts/store_hand_extraction.py`, whose data files name their destination and are refused anywhere\nelse. Tables are created by `init_db()` on first use; the graph is created on first write. Design\nand reasoning: `docs/research/blobs-community-sandboxes-and-the-advice-process.md` § 3. Not built:\nhosting a blob for anyone but its keeper.\n\n## Monitoring (Session 8 — Mar 30, 2026)\n\n- **Prometheus**: Auto-instrumented request metrics at `/metrics` (request count, latency, error rate by endpoint). Custom counters: `deliberus_extractions_total`, `deliberus_client_errors_total`. Darwin's Prometheus (`monitoring-prometheus-1`) could scrape this but does NOT — no deliberus job in its scrape config (verified 2026-08-19; adding one is a proposed fleet change)\n- **ntfy**: Push notifications on extraction success/failure, system errors, frontend errors. Topic: `deliberus-alerts`. User `deliberus` on self-hosted ntfy server. Token in `.env` as `NTFY_TOKEN`\n- **ntfy signal discipline** (Apr 8, 2026): known scanner probes (`/wp-admin`, `/wordpress/wp-admin`, `.env`, `.git`, etc.) are blocked before normal routing and digested only after a threshold instead of creating per-route activity pushes. Real doc reads notify from `/api/docs/{path}` with logged-in user identity when present. Extraction submissions, key Temporal stages, and truth-graph queries now emit explicit ntfy notifications.\n- **ntfy SNR rebuild** (Apr 15, 2026, session 13): rebuilt around **interrupts only**. Three tiers: (1) **interrupt-worthy** (`new_user`, `user_contribution`, `extraction_failed`, `system_error`, `client_error` deduped, `extraction_started`/`extraction_completed` external-user-only) → ntfy at `default`/`high`/`urgent`; (2) **aggregated daily digest** (page views, doc reads, query volume) → one `low`-priority push per day from an asyncio task in `lifespan`, idempotent flush once 24h elapses; (3) **silent** (pipeline intermediate stages, scanner probes, owner events, bot UA events) → logs and Prometheus only. **Owner identification** is layered: session email matches `DELIBERUS_OWNER_EMAIL` → HMAC-signed `deliberus_owner` cookie (set automatically on owner's Google OAuth callback, 365d max-age, covers cross-device browsing, survives session expiry) → narrow `DELIBERUS_OWNER_IPS` infrastructure allowlist (empty by default, **never the home WAN**, because housemates share it). Bot UA filter catches Googlebot / GPTBot / ClaudeBot / PerplexityBot / Applebot / AhrefsBot / facebookexternalhit / curl / wget / python-requests / empty UAs. Temporal migration via `workflow.patched(\"v2-notifications-lifecycle-only\")`: new workflows fire a single lifecycle activity at completion/empty/failure, legacy in-flight replays take the log-only no-op path. **Failure notifications are self-contained forensics**: `extraction_failed` body now carries `user`, `url`, and a 200-char `preview` so the phone alert alone is enough to diagnose. Full rationale: `docs/research/session13-notification-snr-and-extraction-outage-rca.md` §Problem 1. **RULE (Jul 6, 2026): every notification call site MUST pass `user_email`** — owner-suppression depends on it, and all four `user_contribution` sites + the stream-completion site omitted it for months (every contribution fired a high-priority push, including the owner's own) until the dogfood run caught it.\n- **Health check**: `GET /health` probes FalkorDB connectivity. Returns `{\"status\": \"ok\"}` or `{\"status\": \"degraded\", \"falkordb\": \"error\"}`\n- **Frontend error reporting**: Global `unhandledrejection` + `error` handlers in `+layout.svelte` → `POST /api/client-error` → Prometheus counter + ntfy push. Rate-limited 10/min\n- **Darwin aliases**: `dlogs`, `dtail`, `dexec`, `dstatus` in `~/.zshenv` for quick log access. `ssh darwin dlogs` for live logs\n- **Docker DNS**: `/etc/docker/daemon.json` on Darwin has `\"dns\": [\"192.168.4.1\", \"1.1.1.1\"]` — all containers resolve AdGuard DNS rewrites (`darwin.home`, `sarpetorp.home`). Changed Mar 31, 2026. Benefits all Kamal projects\n\n\n## Deployment\n\n**deliberus.com** — LIVE (paid Mar 30, 2026). `kamal deploy` works end-to-end (~55s from MERIAN, ~25s from Mac Mini).\n- DNS: A record → 94.254.88.116 (TTL 300). www CNAME → deliberus.com. Managed via Loopia XML-RPC API.\n- Docker container on Darwin, kamal-proxy routing with TLS (Let's Encrypt), `--forward-headers` for HTTPS.\n- FalkorDB connection: darwin.home:6380 (Darwin LAN, verified reachable from Docker containers on `kamal` network).\n- **Temporal**: Server 1.24.2 on Darwin at `temporal-temporal-1:7233` (Docker `kamal` network). Namespace: `deliberus`. UI: `darwin.home:8233`. CLI installed on Mac + Darwin (v1.6.2). Python SDK: `temporalio` via uv. Worker co-located with FastAPI in the same container (Maxim Fateev recommended pattern). Extraction runs as a Temporal workflow with per-pass activities, heartbeats for deploy survival (~30s recovery), and cleanup activity at workflow start for code-change safety. Full architectural reasoning: `docs/research/temporal-extraction-architecture.md`.\n- **Temporal registration gotcha** (Apr 8, 2026): every activity used by a workflow must be registered in the worker's `ALL_ACTIVITIES`, and every workflow class must be registered in the worker's `workflows=[...]`. A production extraction failed after `implicit_premises_activity` was added to `ExtractionWorkflow` but not registered in `deliberus/temporal/worker.py`; Temporal then scheduled an activity no worker knew how to run. When adding/changing workflow passes, update `deliberus/temporal/worker.py` and the Temporal config tests in the same commit.\n- Deploy: `kamal deploy` from project root. Config: `config/deploy.yml`. Secrets: `.kamal/secrets` (simple grep from `.env` — **NEVER use `${VAR:-default}` pattern**, see incident below).\n- Container runs as non-root `appuser`. `DELIBERUS_ENV=production` set in Dockerfile.\n- **deploy.yml** includes `healthcheck: path: /health`, `stop-timeout: 60`, `cache: options: mode=max`, `boot: limit: 1`.\n- **Kamal deploy root causes fixed** (Mar 30, 2026): (1) Missing `healthcheck.path` — kamal-proxy defaults to `/up` which FastAPI doesn't serve; (2) stale `KAMAL_REGISTRY_USERNAME` in shell env (`fredrik-807e5d517504` from Docker Hub default) overriding `.env`'s `admin` — `Dotenv.load` doesn't override existing env vars. Fix: hardcoded `username: admin` in deploy.yml.\n- **Secrets mangling incident** (Apr 4, 2026): Codex session \"improved\" `.kamal/secrets` from simple grep to `${VAR:-...}` env-first fallback — the exact pattern the global CLAUDE.md warns against. Kamal stripped `${`, injecting corrupted values (`deliberus:-deliberus}` as DB password). Site showed zero extractions for ~12 hours. Data was safe (Postgres container has its own env). Fix: reverted to simple grep pattern. **The only safe `.kamal/secrets` pattern is `VAR=$(grep '^VAR=' .env | cut -d'=' -f2- | tr -d \"'\")`**. See `docs/research/session12-production-recovery-and-secrets-fix.md`.\n- **MERIAN deploy verified** (Apr 4, 2026): Ruby 3.3.6 (rbenv), Kamal 2.11.0, Docker CLI 24.0.9, buildx v0.11.2. `.env` has production secrets (DB password, registry password, Google OAuth, Gemini key). Remote builder on Darwin. Deploy lock cleanup needs `rmdir` (Codex destructive guard blocks `rm -rf`).\n- **Orphan deploy lock** (met 2026-09-19): a deploy that is cut off after the new container boots leaves Kamal's lock held, and the next `kamal deploy` stops with *\"Deploy lock found\"* naming the old version. Before releasing it, confirm it is an orphan rather than somebody's live deploy: `kamal lock status` shows who took it and when, `~/.kamal/deliberus-audit.log` on Darwin shows whether that deploy ever logged a release, and no `kamal` process should be running on either machine. Then `kamal lock release`. That day's lock was 22 days old (the previous deploy, 2026-08-28, image `90c4ae0`).\n- **What is live right now**: `kamal app version` prints the running image's git SHA. Trust it over any SHA written in a doc, which is true only on the day it was written (2026-09-19 alone saw six deploys, counted in `~/.kamal/deliberus-audit.log` on the production host).\n- **Run the Darwin preflight before every deploy** (memory, load, `docker ps` answering within a timeout, disk): the builder is the production host and also the router, so a build on a starved machine takes the whole house offline.\n- **deliberus.se** registered (Mar 31, 2026 via Loopia API). DNS not yet configured.\n\n\n## Security (Session 8 — Mar 30, 2026)\n\n**Full security audit + hardening before public launch.** See `docs/research/auth-patterns-deliberation-platforms.md` for auth research.\n\n- **SSRF protection**: URL validation with DNS resolution + private IP range rejection + redirect-following with per-hop validation. Blocks access to LAN services (FalkorDB 6380, embedder 8080, PostgreSQL 5433, Redis 6379). See `_validate_url()` in `api.py`.\n- **Cypher injection whitelist**: `VALID_RELATIONSHIP_TYPES` set in `store.py` — only SUPPORTS/ATTACKS/QUALIFIES/REFRAMES/DECOMPOSES_INTO accepted. Blocks prompt injection → Cypher injection chain.\n- **XSS fix**: `escapeHtml()` on graph tooltip innerHTML in `ClaimGraph.svelte`.\n- **CORS**: Locked to `deliberus.com` in production, `localhost` in dev. Controlled by `DELIBERUS_ENV`.\n- **Rate limiting**: slowapi on all LLM-invoking POST endpoints — measured 2026-08-22 at 3–10/minute depending on endpoint (the old \"1/minute\" note was stale).\n- **Error sanitization**: Generic messages to clients, `logger.exception` server-side. No `str(e)` leakage.\n- **Input size limits**: `max_length` on ExtractRequest fields, 25MB audio cap, query `limit` caps with `le=` constraints.\n- **Path traversal**: `_validate_source_id()` regex on GET endpoints, `_DOC_PATH_RE` + `resolve()` containment on doc endpoints.\n- **800+ tests (Jul 2026; count drifts upward — trust `pytest` over this line). Coverage: SSRF (all private IP ranges), Cypher injection, XSS, CORS, rate limits, path traversal, input validation, auth flow, session behavior, protected vs public endpoints, Prometheus metrics, ntfy notifications, client error reporting, docs routing, URL cleaning, PDF ingestion + source attribution, auto-connect pipeline, embeddings/linker edge cases, feed algorithm (all 8 modes), CQ generation, database CRUD, extraction pipeline (section splitting, confidence computation), Temporal workflows.\n\n\n## Authentication (Session 8 — Mar 30, 2026)\n\n**Google OAuth via Authlib + HTTP-only session cookies.** Decision: sessions over JWTs (same-origin deployment eliminates JWT's only advantage; revocation is instant; no XSS exposure). See `docs/research/auth-patterns-deliberation-platforms.md` for the full JWT vs session analysis and platform auth research (Kialo, Pol.is, Discourse, Wikipedia, LessWrong).\n\n- **Module**: `deliberus/auth.py` — Authlib OAuth with Google + GitHub (GitHub optional, needs client ID/secret).\n- **Session**: Starlette `SessionMiddleware`, signed HTTP-only cookie (`deliberus_session`), 14-day expiry, `SameSite=lax`, `https_only` in production.\n- **Protected endpoints**: `POST /extract/start`, `POST /extract/stream`, `POST /extract/pdf`, `/transcribe`, `/claims/generate-titles`, `/embeddings/generate` — all require `Depends(require_auth)`.\n- **Public endpoints**: All GET routes (claims, extractions, concepts, graph stats, docs). `POST /claims/search` also public (read-only).\n- **Frontend**: `AuthButton.svelte` in layout header (avatar + sign out when logged in, Google login button when not). Extract button shows \"Sign in with Google to extract\" when unauthenticated. Voice error shows red fade on mic button instead of error text.\n- **Google OAuth setup**: Console at `console.cloud.google.com/apis/credentials`, project `fredrikbranstrom`. Scopes: `openid`, `email`, `profile` (non-sensitive, no verification needed). App published (not in testing mode).\n- **Env vars**: `GOOGLE_CLIENT_ID`, `GOOGLE_CLIENT_SECRET`, `SESSION_SECRET` in `.env` + `.kamal/secrets` + `deploy.yml`.\n- **Dev vs prod redirect**: `_frontend_url` redirects to `localhost:5173` in dev, `/` in production. `.env` loaded via `python-dotenv` for local dev.\n- **Owner identification** (June 4, 2026): **Default path** — sign in with Google on each device; the OAuth callback auto-sets a 365-day `deliberus_owner` cookie, no `/me` visit needed. **Side door for edge cases** (incognito, guest browser, devices you'd rather not Google-sign-in on): owner-only `/me` settings page → `GET /auth/claim-link` (401 anon, 403 non-owner) mints a 15-min HMAC-signed token → opening `/auth/claim?token=X` on the target browser sets the same cookie there without re-OAuth. Cookie scope unchanged (notification-suppression only, NOT an auth gate — confirmed via security audit). `is_owner` flag exposed in `/auth/me` so the frontend can show owner-only affordances. **SPA gotcha**: any new top-level route must be added to the `spa_fallback` decorator list in `deliberus/api.py` — SvelteKit's static adapter emits `<route>.html` at the build root (not `<route>/index.html`), so `StaticFiles(html=True)` 404s without explicit fallback. See `deliberus/auth.py` for `_mint_claim_token`/`_verify_claim_token` and `web/src/routes/me/+page.svelte` for the QR + countdown UI.\n- **Future**: Tiered pseudonymous model planned — anonymous reading, pseudonymous voting, reputation-gated claim submission. See auth research doc for platform patterns.\n\n\n## Web App\n\n**SvelteKit** frontend at `web/` (adapter-static, built into Docker image).\n- **MANDATORY: Dynamic route data loading** — any page under `[id]`, `[slug]`, or `[...path]` MUST use `$effect(() => { data.param; ... })` to reload data when the route param changes. NEVER use bare `if (typeof window !== 'undefined') loadData()` or `onMount(() => loadData())` — these only fire once, so client-side navigation between `/claim/A` → `/claim/B` shows stale data from A until the user reloads. The docs page (`[...path]`) is the reference pattern. (Mar 31, 2026 incident: claim page showed wrong claim after client-side navigation; only correct after browser reload.)\n- **Design**: Manrope font, dark blue-gray (#0d1117). Composable CSS classes named by vibe: obsidian, breathe, aurora, ember, crystal, shimmer, stance, facet, pulse, trail, beacon, undercurrent. Glow-on-hover (never raise/lift). Same-colored outer glow for all interactive elements (100ms fade). Purple radial ambient glow behind logo area. **Full guide**: [docs/design-guide.md](design-guide.md) — tokens, color system, composable classes, hover effects, typography, component reference, key principles.\n- **Logo**: \"deliberus ∴\" — three gradient dots (indigo→violet→purple) in therefore pattern, matching 2013 original but in current palette. Shared `BackLogo.svelte` component for back-navigation on all sub-pages.\n- **Footer**: \"Built by Fredrik Bränström\" (linked to fredrikbranstrom.se) on every page via `Footer.svelte` in layout.\n- **Landing (REBUILT 2026-08-19 — the derivation page)**: checking-vs-persuasion hero → conviction ladder (thermostat war stepped untangling with the crux panel; pineapple/vibe residue rung; dog claim-kinds rung; confession line) → missing-layer + unlock beats → **three doors** (walk the seeded thermostat map `/claim/claim_thermo_t9` · vision.md in full · frontier/research garden) → input form retitled \"Analyze your own thinking\" (paused note while the LLM key is unfinanced) → 7 concept keywords → Feed banner → previous-extractions grid. Copy is Claude's draft pending the founder-voice pass. Structure + all Socratic decisions: `specs/landing-redesign/` + `docs/research/site-declutter-directions.md`. Thermostat map seeding: `scripts/seed_thermostat_example.py` (re-runnable, synthetic-labeled). Ambient purple radial glow retained.\n- **What a signed-in person can do on a claim page today** (inventoried 2026-09-19 from `web/src/routes/claim/[id]/+page.svelte`, because the landing's example map may only hint at actions that exist): agree or disagree (`castVote`) · add evidence, as support or as a challenge (`submitEvidence` with `evidencePolarity`) · answer an open critical question, yes-confirms or no-denies with a reason (`submitCQAnswer`) · split a claim into smaller ones, which reads *Decompose this premise* under *What's underneath?* on a value premise and *Add another layer* on a claim that already has parts (`submitDecompose`) · say which meaning of a contested word is meant, or refine one in your own words (`submitConceptClarification`) · re-point a link at the part of its target it really bears on, the old link staying visible but no longer counting (`retargetEdge`, *this actually bears on a part…*). **Not on the page**: marking a claim out of date and arguing that a part is or is not required both exist as endpoints only.\n- **The example map (2026-09-19)**: `web/src/lib/components/ExampleMap.svelte` opens the landing, above the constellation: a hand-laid-out SVG in the claim graph's visual language (not the D3 component, which reads the API and is ego-centric), two layouts from one data set, its action prompts deliberately inert. Since the third round (2026-09-19) it is a mirror: Alex's side down the left, Sam's down the right, each ending in its own weighing of the same two things, with the unspoken premise and a shared fact between them. The phone layout is leaner by the founder's word (no drawn prompts, one fact fewer), and there the readout rides the bottom of the screen while something is selected, because the map is taller than a phone. ⚠ An earlier version of this line said the map is *present without JavaScript*. It is not: the whole site is a client-rendered app (`ssr = false` in `web/src/routes/+layout.ts`), so with scripts off a visitor sees nothing at all.\n- **The constellation (built and deployed 2026-09-19)**: **it follows the example map on the landing** (founder 2026-09-19, titled *The constellation of convictions*; the 2026-08-19 hero and conviction ladder are hidden behind `showDerivation` in `+page.svelte`, never deleted) — the root conviction plus 44 one-sentence stars in four branches. `web/src/lib/components/Constellation.svelte` renders it; **the wording lives in `web/src/lib/constellation.ts`, the display copy of `specs/landing-redesign/constellation-map.md`** (`ruled: true` = founder-ruled, shown bright and openable; `ruled: false` = a draft from the map, shown faint and labelled a draft). The reasoning under a star is always in the DOM and opens with a CSS grid-rows slide, so it is readable by an agent that renders the page and in a static copy of the rendered page. (⚠ Not *without JavaScript*, which an earlier version of this line claimed: the site is client-rendered, `ssr = false`, so nothing at all renders with scripts off.) `scripts/snapshot_landing.py` renders the running dev page into one self-contained HTML file for showing without a deploy (`--artifact` for the body-only form). `.claude/launch.json` starts the frontend alone for a visual check.\n- **Shared wall components (2026-09-19)**: `web/src/lib/components/Constellation.svelte` renders any *wall* (one root sentence, branches of one-sentence tiles that open onto their reasoning) and shows the landing's constellation when given no props; shapes in `web/src/lib/wall.ts`. **What bright and faint mean belongs to each wall, so each wall passes its own legend** (on the landing: the founder has ruled the wording). `WorkedExample.svelte` is the landing ladder's step-by-step idiom as a component. Pages that are meant to stay unlisted are served with an `X-Robots-Tag: noindex` header from `deliberus/api.py`, because a robots meta tag only exists after the client renders; they are never added to `robots.txt`, where a Disallow line would advertise the path, and never named in this directory, which is publicly served.\n- **Input**: Accepts text, URLs (auto-detected), or **documents** (drag-and-drop: PDF, DOC, DOCX, TXT, MD, HTML, RTF, EPUB). Document shows styled SVG icon + human-readable title in input area. Voice input (Gemini transcription).\n- **Source attribution**: Extraction detail page shows source type (URL link / PDF download / text/voice icon), user avatar + name, date. PDFs stored in Postgres BYTEA, downloadable via `GET /extractions/{id}/pdf`. `input_type` field on extractions table.\n- **Clarification surfaces**: claim pages support explicit contested-term clarification with reusable definition claims, concept pages expose accumulated shared sense memory plus emergent lifecycle states, the new `/concepts` index makes that layer browseable across the graph, extraction pages include an **Interpretation Checkpoint** for source-specific ambiguous usages, and the landing page now runs a lightweight clarification/interpretation preflight for text, URL, and document input before full extraction begins. Preflight sense chips use shared graph memory when available and pipeline-detected fallback senses for first-time sources. Uploaded documents default to external-source reading but expose a contextual \"I wrote this\" correction, switching prompts to the user's own intended meanings. Remaining gap: the voice-first loop still needs direct sense-selection continuity.\n- **Decomposition guidance**: the backend now scores whether a claim still looks bundled rather than atomic-for-now, returns a structured decomposition guide, claim pages can explain why decomposition is being invited and seed a non-blank starting structure, and extraction pages surface a \"Deepen Next\" list whose links open directly into decomposition focus mode. Parent claims now also expose a decomposition state, so a mother claim can say whether its direct substructure is not yet decomposed, partially deepened, still bundled downstream, or coherent-for-now; that state now also notices when the direct layer looks fine but the downstream frontier is still bundled. The same readiness layer now reaches CQ polarity claims too: critical-question yes/no answers can carry decomposition metadata and surface direct deepening links instead of remaining static sorry placeholders. Template-only scaffold submissions are rejected so the guided path does not pollute the graph with empty decomposition noise. This is still a guidance/state layer, not yet full transitive mother-claim strength handling.\n- **Feed**: `needs-help` is no longer only CQ-density. It now also surfaces claims that still look bundled and likely want decomposition, can route directly into focused deepening when decomposition is the clearest next intervention, and can surface bundled CQ polarity claims when the work queue should descend into that layer. The feed is starting to act more like an epistemic work queue for clarification and deepening.\n- **Docs route**: `/docs/[path]` renders any `docs/*.md` file with dark-themed markdown styling. Links between docs rewritten to SvelteKit routes. API: `/api/readme`, `/api/docs/{path}`, `/api/docs-index`. **Section anchors need help from us**: `marked` dropped built-in heading ids in v6 (we pin ^17), so every `](#some-heading)` link written in the corpus silently resolved to nothing until Aug 2026 — two in `convergence.md` had been dead for months. The renderer now injects GitHub-compatible slugs onto bare `<h2>`–`<h4>`, the `.md` link-rewrite regex tolerates a trailing `#fragment`, and scroll honors the hash after the async render instead of always jumping to top. When bumping `marked`, re-check that headings still arrive without ids (the injector only matches attribute-less `<hN>`, so it is safe either way, but a future version emitting its own ids would change the slug spelling).\n- **Extraction**: Real-time SSE progress — spinner, pass-by-pass status from actual backend events, progress bar, elapsed timer, pass indicator dots.\n- **Results**: Contested concepts (the \"aha\" section), claims grouped by argument structure with type/status/confidence badges, expandable relationships with color-coded pills.\n- **Previous extractions**: Grid of past analyses, click to load instantly without re-extraction.\n- Dev: use `scripts/dev` from the repo root for local backend + frontend servers. It uses absolute paths, the correct reload directory, port cleanup, PID management, and `.dev-logs/`; avoid relative `uvicorn --reload-dir deliberus` commands because CWD drift can leave localhost serving stale code.\n- Production: served by FastAPI as static files from `/app/static`.\n- Brand assets: `docs/brand/` — 2013 logo + wallpaper recovered from FERMI/Meteor prototype.\n- Stitch design proposals: `docs/design/` (gitignored).\n\n\n## Graph Visualization (Session 6 — Mar 28-29, 2026)\n\n**Interactive ego-centric D3 force-directed graph** on every claim page (`/claim/[id]`). First realization of UX Principle P1 (The Graph Is a Map). Inspired by Fredrik's hand-drawn sketches from ~2009-2013 (contention diagrams, argument bundles, scoring axes).\n\n- **Component**: `web/src/lib/components/ClaimGraph.svelte` — D3.js force simulation\n- **Center node**: focused claim with gradient glow (indigo→purple→pink), fixed position\n- **Satellite nodes**: 1-hop neighbors (max 5 incoming + 5 outgoing + 3 similar), colored by claim type\n- **Edges**: colored by relationship type (green=supports, red=attacks, amber=qualifies, purple=reframes, blue=decomposes_into, indigo=similar)\n- **Labels**: LLM-generated `short_title` (3-5 word headline per claim). Fallback: keyword extraction with attribution stripping + stop-word removal\n- **Interaction**: Click node → `goto('/claim/{id}')` → graph refocuses. Drag nodes. Zoom/pan. Hover highlights connected edges + dims unrelated nodes. Tooltip shows full claim text\n- **API**: `GET /claims/{id}/neighborhood` returns graph-ready `{nodes, edges, center_id}` including inter-neighbor edges\n\n**Key insight**: Algorithmic keyword extraction fails when claims share a topic — all nodes show identical labels (\"universal basic income\" × 4). LLM-generated short titles differentiate them (\"UBI redistributes via taxation\" vs \"Adequate UBI funding impossible\" vs \"Poorer citizens net gain\"). The `short_title` field is now part of the AtomicClaim model (Pass 2b), essentially free since the LLM already processes each claim. Batch backfill: `POST /claims/generate-titles`.\n\n\n## FastAPI REST API\n\n**`deliberus/api.py`** — wraps the extraction pipeline.\n- `POST /extract/stream` — same pipeline with real-time SSE progress events (parallel Pass 3a+3b)\n- `POST /extract/start` — start Temporal-backed extraction workflow\n- `POST /extract/pdf` — upload PDF/document for Temporal-backed extraction\n- `GET /extract/status/{workflow_id}` — poll workflow status\n- `GET /extract/status/{workflow_id}/stream` — SSE workflow progress stream\n- `GET /extractions` — list previous extractions with friendly titles + summary stats\n- `GET /extractions/{id}` — load full extraction data (backfills title from arg labels)\n- `GET /extractions/{source_id}/pdf` — download stored source PDF\n- `GET /extractions/{source_id}/clarification-opportunities` — source-specific contested-term usages + reusable senses/definitions for the extraction-level interpretation checkpoint\n- `GET /claims` — list claims from graph\n- `GET /claims/{id}` — claim + its relationships\n- `GET /claims/{claim_id}/cqs` — critical questions around a claim's relationships. **Hinge-ordered since 2026-08-20** (NeSy-RAG yield): open questions first, sorted by how far a decisive answer would move THIS claim's badge (`hinge_sensitivity` + `strength_if_yes/no` in the response; `deliberus/question_ordering.py`; fails open to storage order). **Worth-asking priced since 2026-08-23**: each question carries `worth_asking` — offered iff a decisive answer could cross a display band (`qbaf.display_band`; λ IS the band structure, no tunable constant) — plus a `worth_asking` summary that confesses when the surface goes quiet; the claim page collapses below-the-line questions behind a state line\n- `GET /claims/{claim_id}/badge` — QBAF badge / gradual-semantics strength summary\n- `POST /claims/{claim_id}/decompose` — user-driven recursive decomposition\n- `POST /claims/{claim_id}/add-evidence` — attach supporting/attacking evidence claims\n- `POST /claims/{claim_id}/answer-cq` — answer critical questions with polarity claims\n- `POST /edges/supersede` — the edge-level lifecycle (2026-08-22): mark an edge superseded by a sharper successor (`restore: true` undoes; `retarget_to_claim_id` does copy-onto-part + supersede in one action). Superseded edges stay visible everywhere, marked, and stop weighing in every strength computation — without this, decomposition-plus-retargeting double-counts (stress-suite wave two)\n- `POST /claims/{claim_id}/validity` — human ratification that a claim's validity LAPSED (mints a challengeable verdict-claim ground; reason required; `restore: true` undoes). Only ratified `invalid_at` reaches the strength layer; the staleness daemon's flags never do\n- `GET /graph/staleness` + `POST /graph/staleness/sweep` — the temporal rung's instrument and its deterministic flag-tier daemon (no LLM; datable_fraction is the honest coverage stat)\n- `GET /claims/{claim_id}/concepts` — contested concepts for one claim, with candidate senses and reusable definitions\n- `POST /claims/{claim_id}/clarify-concept` — create definitional claim and bind selected sense to the target claim\n- `GET /claims/{claim_id}/decomposition-readiness` — does this claim still look bundled rather than atomic-for-now, plus a structured guide and a non-blank starting structure\n- `GET /claims/{claim_id}/decomposition-state` — the parent-side view: not-yet-decomposed / partially-deepened / still-bundled-downstream / coherent-for-now\n- `GET /extractions/{source_id}/decomposition-opportunities` — the extraction page's \"Deepen Next\" list\n- `POST /extract/preflight` + `POST /extract/pdf-preflight` — the landing-page clarification/interpretation checkpoint before a full extraction (text/URL, and uploaded documents with the \"I wrote this\" provenance correction)\n- `GET /claims/{id}/neighborhood` — ego-network for graph visualization (nodes + edges + inter-neighbor edges)\n- `GET /claims/{id}/similar` — find similar claims across extractions\n- `POST /claims/{claim_id}/vote` — agree / disagree vote\n- `GET /claims/{claim_id}/votes` — aggregate vote counts\n- `GET /claims/{claim_id}/my-vote` — current user's vote\n- `POST /claims/search` — search claims by text (embedding-based)\n- `POST /claims/generate-titles` — batch-generate LLM short titles for claims missing them\n- URL fetching: extracts `<title>` tag for slug/title, strips nav/header/footer before text extraction\n- `GET /concepts` — contested concepts\n- `GET /concepts/{term}` — accumulated graph state for one contested concept\n- **Synthesis artifacts — pre-generated readings agents get for free** (built 2026-08-13, nothing published yet). `deliberus/synthesis_artifacts.py` (pure: eligibility, subgraph fingerprint, batch planning) + `ArtifactSynthesis` / `build_source_context` / `build_concept_context` / `synthesize_artifact` in `truth_graph.py` + the `synthesis_artifacts` table + rendering into `/extraction/{id}.md|.json` and `/concept/{term}.md|.json`. Operator surface: `scripts/generate_syntheses.py {plan,generate,list,publish}`. **Four invariants**: (1) `published` defaults to false and generation NEVER sets it — publication is a separate act because a published synthesis is crawlable prose under Fredrik's name and publication is one-way; (2) **no deterministic fallback** — `synthesize_artifact` returns `None` when the LLM is down, because a template is acceptable as a live answer to a waiting human and unacceptable as a permanent page; (3) **staleness is split across two checks and neither covers the other** — the request path runs the cheap `cheap_staleness_signal` (claim-count comparison, no extra query, catches claims appearing/disappearing and NOT strengths moving), while the full **fingerprint** recheck (claim ids + rounded strengths + depth) lives in `generate_syntheses.py check` because recomputing it per page view would rebuild the subgraph on every read and defeat the cache; do NOT \"fix\" this by moving the fingerprint check onto the read path; (4) `llms.txt` advertises the layer ONLY when `published_readings > 0` — announcing a surface before it has content is how this file sent agents to three 404s. An artifact carries the project's own **`lifecycle`** vocabulary (`draft` on generation → `canonical` on publish → `superseded` when the sweep finds drift) plus `revision` and `previous_fingerprint`, because a published reading must not be the one publishable object in the system with no lifecycle — and the reader is told in plain words that it is a snapshot, that it does not update itself, and that **where the reading and the live claim list disagree, the claims win**. **Known limit worth remembering: seed truncation is invisible to both omission classes** (`expand_subgraph` caps seeds before the prompt exists; 20 live / 60 artifacts) — `seeds_available` vs `seeds_used` on the context is what makes it visible. Design + the permanence objection: `docs/research/synthesis-build-plan.md` § 9\n- **`POST /query` — the truth-graph read layer, and the project's SHIPPED SYNTHESIS ENGINE** (auth-gated, which is why a cloud session cannot exercise it). `deliberus/truth_graph.py` retrieves relevant claims, expands the subgraph, computes QBAF badges, detects gaps / contested concepts / bridging, then has an LLM synthesize prose at `casual | curious | expert` depth and returns a `QuerySynthesis` (graph-vs-background provenance + `cited_claim_ids` + answer + provisional draft structure). **Do NOT conclude Deliberus \"declines to synthesize\" — it synthesizes here.** Since 2026-08-13 it also returns **`synthesis_ledger`** (`deliberus/synthesis_ledger.py`, pure stdlib so it is unit-testable anywhere): the depth budget applied, claims truncated before the model saw them (Class A) and shown-but-uncited (Class B), `conflict_coverage`, `citation_balance`, dropped (unshown) citations, `mode` = `llm` | `fallback`, and `retrieval_failed` (a failed graph lookup arrives as `has_data: False` exactly like an empty graph — the flag is what stops the answer from telling the reader the corpus is thin when the lookup simply broke). **Four rules when touching this path**: (1) the depth display limits live ONLY in `synthesis_ledger.DEPTH_DISPLAY_LIMITS` — never re-inline them, or the prompt and the ledger will disagree about what was shown; (2) every reported citation is validated against the shown set, and that gate is not optional (an unvalidated citation field manufactures auditability the reader cannot check); (3) the accounting stays OFF the LLM response schema — an LLM must never be asked to grade its own omissions; (4) **input breadth and answer length are two separate caps and must never be re-merged** — `DEPTH_DISPLAY_LIMITS` governs how many claims the model READS (3+2 / 8+5 / 16+12), `truth_graph.DEPTH_ANSWER_TARGET` governs how long the answer may be (420 / 1200 / 2400 chars, with `QUERY_SYNTHESIS_ANSWER_MAX = 2400` as a schema backstop only). They were merged until 2026-08-13, when a flat 520-char cap at every depth made \"curious means short structured paragraphs\" a promise the schema could not keep; showing the model more claims never lengthened the reply, it only made the reply better informed. Every named `*_MAX` is referenced by its field rather than duplicated as a literal. Build record + where the build corrected its own plan: `docs/research/synthesis-build-plan.md` § 7\n- `GET /api/feed` — epistemic feed modes\n- `GET /graph/stats` — node/edge counts\n- `POST /embeddings/generate` — generate embeddings for all/specific claims\n- `POST /transcribe` — audio transcription via Gemini\n- `POST /api/client-error` — frontend error reporting hook\n- `GET /api/readme`, `GET /api/docs/{path}`, `GET /api/docs-index` — documentation endpoints\n- `GET /claims/{id}/completeness` — derivational-completeness oracle (exposure, verdict, sorry-frontier, terminus candidates, unsupported value premises)\n- `GET /claims/{id}/horizon` — the claim's horizon reading, computed fresh from the arrangement beneath it (a kind is **read, never set** — ruled 2026-09-15, built 2026-09-16): state (unexamined / examined), kind or none, `read_at`, the cross-attack record, slicing stability, no gate; proposals (deterministic + LLM) beside the reading, a pre-ruling hand-set stamp shown as legacy provenance\n- `POST /claims/{id}/terminus` — retired 2026-09-16, answers 410 pointing at the reading (it used to confirm a kind and mint a `CLASSIFIES_TERMINUS` verdict-claim; the three it minted stay as ordinary claims)\n- `POST /claims/{id}/terminus/classify` — propose-only LLM horizon classifier (frontier-grade via the capacity-fallback chain, first-class `undecided`); stores `terminus_llm_*` proposal fields, shown beside the reading and never confirmed into one\n- `POST /claims/{id}/parts/{part}/requirement` — argue that a claim REQUIRES one of its parts: mints an ordinary attackable claim (\"W requires X: …\", `ARGUES_REQUIREMENT`, pointer on the decomposition edge, never overwritten); the part's weakest-part cap on the whole then follows that claim's own strength (presumed 0.5 on confirmed parts, 0 on unconfirmed proposals, full at an explicit `necessary` tag) — the designed completion of the 2026-08-31 padding ruling, built 2026-09-17\n- `POST /claims/{id}/weighing/open` — accept a weighing invitation / open any claim as a weighing: instantiates the weighing-question descent claim-level (14 questions as of 2026-08-25; idempotent). The extraction stream runs the detection pass automatically (source→open, authored/sacred→invite)\n- `GET /claims/{id}/discursive-dilemma` — premise-votes vs conclusion-votes divergence flag (List-Pettit, surfaced as a finding)\n- `GET /claims/{id}/hinge` — QBAF sensitivity: how far the claim's strength moves if each descendant were fully granted vs denied; high-hinge nodes are crux CANDIDATES (a typed residue with a high hinge = a load-bearing pebble). Decomposition-channel only, mirroring shipped QEM. QEM property worth knowing (updated by the 2026-08-31 padding ruling): untagged decomposition children aggregate by their WEAKEST part, so the hinge concentrates on the part the whole hangs on; the old additive behaviour — neutral siblings never diluting a pebble — survives only for children explicitly tagged corroborative (`weakest-link-arithmetic-and-the-merge-hunch.md`)\n- `GET /extractions/{id}/disagreement-preservation` — honesty instrument: did LLM mediation flatten opposing claims toward the semantic center?\n- `GET /graph/stance-conflicts` — honesty instrument for the **fourth** flattening mechanism (stance loss at the relationship layer, named by dogfood run 6 and built 2026-08-14; `deliberus/stance.py`, two Cypher queries, **no model call**). Finds agreement edges spanning sources that attack each other — an agreement between two authors in documented opposition is a checkable inconsistency candidate. Two signals reported SEPARATELY, never combined into one score: `near_identical` (cosine ≥ 0.80) and `reported_asymmetry` (exactly one side is reported speech, reusing `weighing._REPORTED_RE`). **Propose-only: candidates never verdicts, and it never touches an edge** — opposed authors share background facts constantly, and telling shared ground from same-fact-opposite-use is a judgment about AUTHORS, which is exactly what the graph cannot see. Measured live at ship time: **8 adversarial pairs, 84 candidates, 1 near-identical, 4 reported-asymmetric**, and it independently surfaced the run-6 phenomenon in the minimum-wage and capital-punishment debates. Two limitations LOGGED not tuned: the 0.80 band straddles the phenomenon (run-6 pair 0.891, best minimum-wage instance 0.791 — retuning would fit the instrument to its own motivating example, so the calibration is a founder question); and the reported-speech signal misses the run-6 case itself (\"took the contrary position\" is authorial distance no pattern covers — the axis wants a semantic tier, not more strings).\n- `GET /graph/residue-map` — corpus falsification metric for the convergence wager (typed-residue fraction of classified termini)\n- **Agent-readable surface**: `Accept`-header content negotiation on `/claim/{id}`, `/extraction/{id}`, `/concept/{term}` (+ explicit `.md`/`.json` twins), the **plural index twins** `/claims.json`, `/extractions.json`, `/concepts.json` (all three 404'd until dogfood run 4 — `llms.txt` advertised them and they had never been implemented), `/llms.txt`, `/robots.txt`, `/sitemap.xml`, `Link` headers, `Vary: Accept`, app-wide HEAD support. **`llms.txt` also carries an Instruments section** (added 2026-08-13 after an audit found every honesty instrument live and none discoverable — completeness / hinge / discursive-dilemma / badge / cqs / disagreement-preservation / residue-map, each with what it certifies and what it does not) **and a Synthesis section** stating that `POST /query` is deliberately unreachable by agents because it spends model tokens per call, so its absence reads as a decision rather than a gap. Per-claim endpoints are advertised as code-span path templates, never markdown links — a link containing `{id}` is a broken link, which is how this surface produced 404s the first time. Every markdown surface opens with the reader preamble (maps-not-endorses) and claim pages carry a Scrutiny state section (completeness verdict + terminus). See `deliberus/agent_readable.py` module docstring for the design reasoning (markdown-primary, Deliberus-native JSON, deliberately NOT AIF-conformed).\n- `GET /health` — health check\n\n\n## FB Scraper Technical Notes\n\nTwo-phase archival scraper at `scripts/scrape_fb_group.rb` (v3). Validated against live DOM via Chrome DevTools MCP.\n\n**Status (Mar 28, 2026)**: Complete. 406 posts, 1,956 comments (1.3MB). Phantom content fixed. 65 posts at 10-comment Facebook cap (old content limitation). 35 posts with 0 content are genuine link/image shares.\n\n**Login**: `LOGIN_ONLY=1 SHOW_BROWSER=1 ruby scripts/scrape_fb_group.rb` — session cookies persist across days. Re-login only if FB expires session.\n\n**Reharvest**: `PHASE2_ONLY=1 REHARVEST_CAPPED=1 SHOW_BROWSER=1 ruby scripts/scrape_fb_group.rb` — re-visits only posts with >=10 comments (cap) or phantom content. Carries forward clean posts from largest existing archive.\n\n**For detached runs**: `nohup ~/.rbenv/versions/3.3.6/bin/ruby scripts/scrape_fb_group.rb > data/fb_group_archive/scraper.log 2>&1 &` (system Ruby 2.6 lacks ferrum; CC Bash times out after 10 min).\n\n**DOM findings** (validated Mar 27, 2026 via Chrome DevTools MCP):\n- Feed posts: `h3` headings in `[role='feed']` children (NOT `[role='article']`)\n- Feed needs 3s wait between scrolls for Facebook lazy-loading\n- Permalink pages: post opens as `[role='dialog']` overlay. **4 dialogs on each page** — use h2 walk-up to `[role='dialog']` ancestor, NOT `document.querySelector(\"[role='dialog']\")` (gets wrong one)\n- Post content: `div[data-ad-rendering-role=\"story_message\"]` within the post dialog — Facebook's internal marker\n- Posts with no text content (link/image shares) have 0 `div[dir='auto']` — correct, not a bug\n- Comments: `div[role='article']` with `aria-label=\"Comment by ...\"` — scope to post dialog\n- Comments lazy-load: scroll the LARGEST dialog (not first), 2s between attempts, 5 stale rounds tolerance\n- Ferrum Chrome and CDT Chrome work side by side (different profiles, no conflict)\n\n\n## Claimify Integration\n\nLocal clone at `~/Projects/claimify/` (includes paper PDF). LLM-agnostic library — accepts any `llm(prompt, temperature) -> str` function. User may build custom extraction pipeline instead of using library directly. Key divergence documented in `docs/technical-direction.md`: Deliberus needs wider extraction (opinions + their premises), not just verifiable factual claims.\n\n\n## Graph Database\n\nFalkorDB running on Darwin (darwin.home:6380) via Graphiti MCP. Temporal features and episodic memory are natural fits for tracking how claims, evidence, and community understanding evolve over time.\n\n\n## Voice Memos (BOHR Drive)\n\nTwo voice memos transcribed from `/Volumes/BOHR/Voice Memos/`:\n- **Emanuel + Sofia** (33 min, Mar 2013): Swedish philosophical conversation. Key: discourse layers above semantics, semantic agreement masking real disagreement, consensus≠truth distinction. Analysis: `docs/research/voice-memo-emanuel-sofia.md`\n- **Robin + Martin** (~5.5 hours, Mar 2013): Transcribed (168KB). Raw output at `data/voice_memos/Robin + Martin; Deliberus.txt`. Analysis pending — massive primary source\n\n**BOHR Google Drive Backup**: 23 files from Nils Janse collaboration era. Mostly `.gdoc` stubs (need `branstrom@gmail.com` auth). Fredrik notes: \"these may not reflect my own vision\" — Nils era was pragmatic/commercial, not epistemological. Attribute authorship carefully.\n\n"}