Technical Reference
Architecture, data model, AI engine, generative pipeline, and API reference for the InterroGate world-simulation and narrative engine. Reflects the codebase as of 3 August 2026 (multi-user beta).
Architecture Overview
InterroGate is a single-host web application designed for local or self-hosted VPS deployment. It is a Python backend, a static single-page frontend, and a flat-file data store. There is no database — all persistent state is JSON (and JSONL) on disk, scoped per user and per workspace.
Stack
- Backend: FastAPI (Python 3.10+) on a single-worker Uvicorn process. Blocking LLM calls run in FastAPI's threadpool (defined with
def, notasync def, so they never block the event loop). - Frontend: Vanilla JS / HTML / CSS — no framework, no build step. Static assets are cache-busted with
?v=Nstamps inindex.html. - AI provider: Anthropic Claude, reached through a provider abstraction (see Provider & Models). Interactive modes stream via Server-Sent Events (SSE).
- Persistence: Flat JSON / JSONL files under
data/users/<username>/. No database engine. - Theming: a single tokenised CSS contract (
style.css) with[data-theme]blocks; user overrides persist in settings.
Directory Layout
InterroGate/
├── app.py # FastAPI application — all routes
├── auth.py # UserStore, InviteStore, persisted sessions, quota, rate limiters
├── machine_worlds.py # Machine Worlds API — World Pack import, readiness, consult
├── orchestrator/
│ ├── orchestrator.py # Core engine — prompt construction, simulate/interrogate/observe
│ ├── world_loader.py # Loads/merges world_bible + characters; builds context string
│ ├── pov.py # POV-wrapper primitives + per-type migration configs
│ ├── provider.py # Provider protocol + AnthropicProvider + normalized errors
│ ├── director.py # Autonomous scene director (interior solo-POV path)
│ ├── ensemble.py # Multi-agent dialogue engine (casting + dramatic director + Beat Cop)
│ ├── render.py # Render chain — distill → POV prose → Beat Cop validate/regen
│ ├── plan.py # SceneSpec/Beat/ChapterPlan, confirm gate, plan composer
│ ├── chapter.py # Chapter loop — scene-at-a-time, gated valid+pointful+confirmed
│ ├── voice.py # AuthorVoice — analyse Library prose → reusable style block
│ ├── usage.py # Per-call $ cost instrumentation (by stage / model / cache)
│ ├── replay.py # Dev record/replay cache over the Provider (free build iteration)
│ ├── sweep.py, snapshot.py # Quill Sweep scan/apply engine + workspace snapshots
│ ├── world_pack.py # World Pack exporter (portable interchange JSON)
│ ├── textnorm.py # NFKD ASCII transliteration (accent-safe slugs/identity keys)
│ ├── spend.py # Per-user spend ledger (beta economics)
│ └── scene_spec.py # SceneSpec dataclass (handoff currency between stages)
├── data/ # Per-user everything (gitignored, never rsync'd)
│ └── users/<username>/
│ ├── settings.json # {theme, active_workspace, api_key}
│ ├── activity.jsonl # Per-request activity log
│ └── workspaces/
│ └── <slug>/
│ ├── meta.json # {display_name, template?, kind?, universe?}
│ ├── characters/*.json # Character sheets
│ ├── world_bible/ # timeline / geography / politics / concepts
│ ├── library.json # Prose vault entries
│ ├── sessions/ # Saved session snapshots
│ ├── canon/ # Pending queue + approved_log.jsonl
│ ├── live_session.json # Durable autosave of the working conversation
│ ├── create_chapter.json, create_sessions.json, voices.json
│ └── source_text.txt # Optional raw prose → auto-import on switch
├── data/users.json # {username: {password_hash, is_admin, api_key_override}}
├── data/invites.json # Invite codes
├── static/ # index.html, app.js, style.css, icons.svg, docs/
│ ├── knowmap-graph.js # Continuity Map (egocentric node graph)
│ ├── knowmap-bands.js # Knowledge bands (parametric SVG)
│ ├── knowmap-pedigree.js # Event Map (narrative-order timeline lens)
│ ├── story-board.js # Library Stage board (kanban)
│ ├── welcome-chips.js # Archive-driven welcome suggestion chips
│ └── theme-editor.js # Live theme editor
└── deploy/ # nginx config, update.sh
workspaces/, sessions/, etc.) is migrated into data/users/<DEFAULT_USER>/ automatically on first access. In single-user mode (MULTI_USER=false) the default account owns everything.
Request Lifecycle (streaming modes)
- The frontend's
apiFetch()wrapper POSTs JSON to an/api/*endpoint (cookie auth carried automatically). _get_user(request)resolves the user (cookie, orDEFAULT_USERin single-user mode) and records activity._require_orch(user_id)returns the per-userOrchestratorfromapp.state.orch_pool(lazy-created), and_check_quotaenforces rate limit + daily quota.- The Orchestrator assembles the system prompt from the context stack and calls the
Provider, which streams text chunks. - The backend wraps chunks as SSE events; the frontend appends tokens to the live message bubble. A terminal event closes the stream.
Modes
The sidebar exposes eleven modes, ordered by user journey. Mode state lives in the frontend (body.mode-<name>); the backend routes by endpoint. Create and Admin are conditionally shown (see below).
| Mode | Icon | Purpose | Primary endpoints |
|---|---|---|---|
| Library | ⊞ | Prose vault. Import / browse / search written prose; List ⇄ Stage board (kanban) toggle; AI scene breakdown; extract world data; compile to TXT / Markdown / DOCX. | /api/library/* |
| Archive | ◫ | The world surface. Card grids (with images) for every category + search + approval log, unified with the Continuity Map, Event-Map Timeline, and Atlas views behind one top-bar segment control. | /api/archive* · /api/atlas/* |
| Canon Review | ⚖ | Top-level panel to review flagged items, conflicts, and Beat-Cop consequences; approve, edit, or apply per-field. | /api/canon/* |
| World Builder | ❖ | Manual brainstorm entry (no AI generation). Drafts inject into prompts; promoting sends to Canon Review. | /api/brainstorm* |
| Simulate | ◈ | Causal world reasoning — what would happen if. AI reasons from world state; it analyses, it does not narrate. | /api/simulate |
| Interrogate | ◉ | Character voice. Question a character in-character; "YOU ARE" lets you play a persona the character knows. | /api/interrogate |
| Observe | ◎ | Scene rendering in the author's voice, with multi-turn memory and the active Author Voice. | /api/observe |
| Create premium · unlock-gated | ✶ | Generative world→prose. Autonomous director / Ensemble dialogue → render chain → Library draft. See Create Pipeline. | /api/create/* |
| Chat | ⬡ | Shared beta-tester message board with screenshot uploads, unread indicator, and a public progress Board tab (per-tester readiness bands). | /api/chatboard* |
| Settings | ⚙ | Tabbed (API · Appearance · Workspace · Data · Beta · Admin): keys, themes, workspace + universe management, backup/restore, resets, beta status. | /api/settings |
| Admin admin-gated | ⛊ | Tabbed (Users / Beta / 🔓Unlocked / Ops): users, invites, activity, unlock examination + grants, budget caps, deploy, maintenance, restart, BYOK mode, bug queue. | /api/admin/* |
is_admin || !multi_user. Create appears for admins, the local owner, or any user whose account has advanced_unlocked (granted through the in-app unlock request → admin examine → grant pipeline). All gates are enforced server-side (_require_owner_or_admin / require_admin / the Create access gate); hiding a button is cosmetic.
/api/quill/* (Ask rides /api/jeeves). The same modal flips to the Stage Manager — an out-of-character helper that answers how the app works, grounded only on the shipped docs + patch notes (never world canon; Haiku; /api/stage-manager), with allowlisted deep-link buttons that navigate to the relevant panel. Quill and the Stage Manager mutually redirect canon vs app-mechanics questions to each other.
GET /api/landing/summary): resume cards for recent sessions (pinned first), needs-attention chips (canon queue, drafts, unread patch notes, tool findings), and the readiness progress bar with clickable unmet checks. A content-less account still gets the first-run welcome paths.
Multi-User & Authentication
InterroGate runs single-user (local owner) or multi-user (invite-gated alpha), controlled by the MULTI_USER env var. auth.py holds the account system.
- Accounts —
UserStore(bcrypt password hashes indata/users.json). Registration requires a valid invite code (InviteStore, atomic consumption under lock; a failed registration refunds the code). - Sessions — opaque tokens in HttpOnly cookies (
secureflag on in multi-user/HTTPS), persisted todata/sessions.jsonso a deploy/restart no longer logs everyone out.GET /api/auth/mereturns the current user (or 401 → the frontend shows the auth screen). - Password change —
POST /api/auth/change-password(Settings): requires the current password even with a valid session, and on success revokes every other session for the account (a change is a leak-revocation, not just a hash rotation). Its own failure-only rate limiter (5 / 15 min) is deliberately separate from the login limiter. Registration enforces a minimum 8-character password server-side; the floor applies only to setting a password — legacy accounts keep logging in. - Hardening — login rate limiter (10 attempts / 15 min); a dummy bcrypt check on unknown usernames defeats timing-based enumeration; per-user API rate limit (20 calls / 60 s).
- Isolation — every path helper is keyed by
user_id. Workspace slugs are validated (^[a-z0-9_]+$) and containment-checked to prevent cross-user path traversal. Slug/ID generation is accent-safe (NFKD transliteration), so accented and plain spellings of a name resolve to the same identity key.
Quotas & BYOK
Non-admin users have a daily quota of 25 calls (DailyQuota, persisted to data/daily_quota.json so a deploy/restart no longer resets counts). Admins are exempt.
Bring-your-own-key (BYOK): a user who saves their own key spends on it and bypasses the daily quota. The key system is open-provider: Anthropic (first-class), OpenAI, or a local endpoint (Ollama / LM Studio) plug into the same provider abstraction. Key resolution order: per-request override → per-user settings key → shared ANTHROPIC_API_KEY. A founder-controlled Strict-BYOK toggle (data/byok_mode.json) removes the shared-key fallback for non-admins. Admins/owner always keep the shared key.
Beta economics layer: every house-key call is written to a per-user spend ledger; a readiness scorecard aggregates it (bands surface on the chatboard Board tab). Enforcement is a master flag (data/beta_enforce.json, currently ON) with runtime budget knobs in data/beta_caps.json — a once-off first-spend-day onboarding allowance ($5), a recurring daily cap ($3), and a lifetime bootstrap pool ($15) per account, all adjustable from the Admin 💳 card with no deploy. When a non-admin exhausts a budget they hit a friendly wall (_WALL_COPY) pointing to BYOK. The unlock pipeline (user requests → admin 🔬 examines viability → grant → mark credits allocated) gates the Create tier per-account (advanced_unlocked).
Admin
The Admin panel (admin-gated; bypassed for the owner in single-user mode) provides: invite creation/listing, the user list with activity LEDs, per-user activity logs, the deploy console (streaming rsync + remote restart over SSE), site-wide maintenance mode, a scheduled restart with countdown, the Strict-BYOK toggle, and the in-app bug-report queue.
Workspaces
A workspace is an isolated world context — its own characters, world bible, library, sessions, and canon. The active workspace is stored per user in settings.json (active_workspace). Switching reloads the orchestrator with the new world.
POST /api/workspaces/create— new blank workspace (slug +meta.json), switches to it.POST /api/workspaces/switch— ifsource_text.txtexists and the library is empty, auto-detects chapters and bulk-imports the prose.- New users start on an empty default workspace with a create / import choice (the old Silver Blaze auto-seed was retired).
World Context System
orchestrator/world_loader.py reads and merges all world data for the active workspace and builds the context string injected into prompts. It is key-tolerant (reads both legacy and native key conventions) and POV-aware — every read funnels through pov.resolve(), so wrapped and plain values are handled transparently.
The header is per-workspace (the loader reads meta.json → story title), so prompts are no longer hardcoded to any one project. build_context_string() composes a world heading + body sections (timeline, geography, politics/entities, concepts) and, when the workspace is attached to a universe, prepends the Universal canon block.
Four-Tier Context Stack
Every prompt is assembled from layered context, broad → narrow:
| Tier | Name | Source | Scope |
|---|---|---|---|
| 0 (broadest) | Universal | Parent universe workspace (optional) | Shared canon spanning multiple worlds. Most stable layer — sits first in the prompt for cache stability. |
| 1 (base) | Canon | World bible + character JSON + approved log | The settled truth of this world. Always present. |
| 2 (staging) | Brainstorm / Session | World Builder draft items; durable working conversation | Tentative additions visible to the AI but distinct from canon. |
| 3 (immediate) | Scene | Timeline position, scene setup, characters present | The immediate narrative circumstance for this query. |
Precedence is broad → narrow: Universal > Canon > Brainstorm/Session > Scene. Scene framing can focus, but it cannot rewrite canon facts. The heavy world prefix uses prompt caching where the same prefix recurs within the cache TTL.
Universal Layer
A universe is a normal workspace flagged meta.kind="universe"; a world points up via meta.universe="<slug>". A world attached to a universe inherits a # UNIVERSAL CANON block (the parent's body sections) ahead of its own bible — across all four modes, with zero extra API calls and a cache-stable prefix. Inheritance is single-level by construction.
- Authoring — a ✦ Universal action on Canon Review cards routes a reviewed item to the universe's store, then reloads so the world re-inherits immediately.
- Conflict surfacing —
find_universal_conflicts()flags structural collisions (same id / normalized label across world and universe); Universal takes precedence. - Emergent capability — because attach is a one-field flip + reload with no canon mutation, the same world can be pointed at universe A vs B to render the same scene under two macro-truths (a reversible alternate-reality switch).
POV-Aware Schema
Opinionated leaf fields (biography, psychology, voice, arc, relationships, and the equivalent world-bible fields) can be stored as a POV wrapper:
"summary": {
"canonical": "The author's default value...",
"perspectives": [
{"pov": "_narrator", "value": "...", "source": null, "extracted_at": "..."},
{"pov": "char_jane", "value": "...", "source": "lib_abc123", "extracted_at": "..."}
]
}
canonical is the author default; perspectives[] hold POV-tagged variants. Legacy plain strings/lists are accepted unchanged (treated as a single _narrator perspective). pov.resolve(node, pov_id) unwraps for reads; hard-canon scalars (id, name, dates, tags) are never wrapped. POST /api/archive/upgrade-schema is an idempotent migration that wraps existing leaves.
Sessions
A session is a JSON snapshot of conversation state (meta + per-mode histories). Sessions live under the workspace's sessions/ directory.
- Durable autosave — the working conversation is persisted to
live_session.jsonafter every turn (atomic write) and rehydrated when the orchestrator is rebuilt, so a deploy/restart/workspace-switch no longer drops in-flight turns. - Pinning —
POST /api/sessions/{name}/pin. Pinned sessions survive "Clear History" (reset-all). - To Library —
POST /api/sessions/{name}/to-librarysaves a transcript as a Library entry. - Pre-wipe preview —
GET /api/reset-full/previewreturns{total, pinned}.
Canon Review Pipeline
The pipeline promotes generative output into the permanent record. The AI never writes canon directly.
- Flag — the writer flags an AI bubble; it enters the pending queue (persisted to
canon/pending_queue.json, survives restart). - Review — Canon Review (a top-level mode) lists items, conflicts, and consequences. Conflict cards render an honest field diff: green = addition, red = real conflict, grey = no-op. A per-field apply (
POST /api/canon/apply-fields) — available for every entry type, characters and world-bible alike — overwrites only ticked paths into the canonical slot, preserving perspectives and never blanking existing canon from a partial extraction. World-bible cards keep a one-click Use New / Keep Old fast path that swaps to Resolve (n changed) the moment any row is touched. - Fact-level list reconciliation — list-valued leaves (knowledge buckets, constraints, any list) diff per item, never whole-list either-or: disjoint additions render as tick-to-add rows (existing facts always retained — absence is not retraction), a cheap semantic pass flags reworded duplicates and genuine contradictions, and a contradiction offers Use new / Keep old / Keep both (belief update) —
sincedating lets both be true. - Approve — approving writes the structured entry into the world bible / character files and appends to
approved_log.jsonl(with timestamps; dismissals are logged recoverably too). POV-tagged values merge as perspectives rather than conflicts. - Beat Cop consequence pass — after a structured approval, a background Claude pass analyses affected entities and proposes
knowledge_state/ relationship patches as gold "consequence" cards for per-patch review. - ⟲ Reopen — every approval logs the full pre-image, so any Approval Log row can be reopened: the prior canon is restored and the item re-queued for a fresh decision (
POST /api/canon/reopen, newest-first guarded).
Items can also be promoted to a parent universe (POST /api/canon/universal) instead of the world.
World Builder (Brainstorm Layer)
World Builder is manual entry — no AI generation. Draft items (character, event, location, organisation, concept, entity, plot_note) are injected into prompts while they are drafts; promoting an item sends it to Canon Review and removes it from injection until approved (the "ghost state"). The UI labels each item's visibility (green "Active — the AI sees this now" vs dim "In review — invisible until approved"). Promotion normalises brainstorm fields into the canonical schema.
Provider & Models
All LLM traffic flows through a single Provider interface (orchestrator/provider.py) with chat() / stream_chat(), a normalized error hierarchy (auth, rate-limit, overloaded, transient, bad-request, request-too-large, status), and a RETRYABLE_PROVIDER_ERRORS set. Concrete implementations: Anthropic (first-class), OpenAI, and local OpenAI-compatible endpoints (Ollama, LM Studio) — users pick a provider + key in Settings. Streaming captures final token usage for cost accounting.
Model routing
| Tier | Model (current) | Used for |
|---|---|---|
| Premium | claude-opus-4-6 | Interactive Simulate/Interrogate/Observe; Create published prose (always Opus). |
| Workhorse | claude-sonnet-4-6 | World/scene extraction, scene breakdown, planning/Beat-Cop validators, judges, and the Create spine default. |
| Cheap | claude-haiku-4-5 | Ensemble casting, knowledge filtering, self-contained scene titles, knowledge dating. |
Provider* hierarchy. Two hard guards: a per-workspace extraction lock (a concurrent run 409s immediately instead of double-spending, and the UI reports it honestly as blocked, never as "done"), and a truncation guard — the provider's stop_reason is checked before JSON parsing, so an output-ceiling hit (32k tokens) fails fast with one clear 413 ("select fewer entries; completed phases are never re-billed") instead of paying for retries that can't succeed.
Timeline-Aware Knowledge Filtering (Fix B)
Characters do not leak post-canonical knowledge into earlier scenes. A knowledge_state item is either a plain string (always-known) or {"fact", "since"} where since is the story-time it was acquired (null = baseline). When a scene has a timeline position, a cheap pass removes facts dated after the scene and relocates them to "does not know". It is a pure no-op (zero API calls) when no scene date is set or no fact is dated. POST /api/archive/date-knowledge backfills since values against the timeline; extraction dates new facts automatically.
Create Pipeline premium · unlock-gated
Create turns a world + a scene spec into prose, grounded in canon. The differentiator is world-consistency, not raw generation: knowledge walls, POV filtering, and a validating Beat Cop sit in the loop. Drafts land in the Library as drafts (extracted:false) — never auto-canon.
Stages
- SceneSpec (
scene_spec.py) — the handoff currency: participants, pov, location, timeline_point, the turn the scene must land, feeling (open/turn), opening, title, and apov_unnamedflag (a reveal-order naming wall that forbids the POV's own name in narration). - Spine — either the interior Director (
director.py: a PLAYER advances one beat at a time, a DIRECTOR judges turn-landed and steers) for solo-POV scenes, or Ensemble (ensemble.py) for multi-party dialogue. The spine default is Sonnet; Opus is a per-scene max-fidelity toggle. - Render chain (
render.py) — distil dialogue to voice → render POV-bound prose (filtered through the POV character's knowledge walls, in the author's Voice) → Beat Cop validate against canon. On apov_leak | canon | timeline | character | communication_wall | namingviolation it regenerates with the fix as a constraint. - Planning + chapter loop (
plan.py/chapter.py) — a confirm gate sits between plan and prose; the chapter loop renders one beat at a time, gated valid + pointful + confirmed, with continuity re-synced from edited drafts. Scene positions auto-increment (act.chapter.scene) so drafts land sorted in the Library.
Ensemble (multi-agent dialogue)
EnsembleEngine drives genuine multi-party scenes: a casting director picks the next speaker, a dramatic director steers toward the scene's turn, and an in-flow Beat Cop validates each turn as it is produced. Each participant is an independent grounded character agent (built from build_participant_sheet + Fix-B knowledge filtering), so the scene is canon-clean by construction — the grounding prevents the canon-creep a generic multi-agent chat would accumulate.
Author Voice
voice.py analyses a selection of the writer's own Library prose into a reusable VoiceProfile (sentence shape, register, signature moves) and renders a style block that supersedes the few-shot in the render chain. Voices are per-workspace (voices.json, with an active_id); the render path is byte-identical to the legacy few-shot when no voice is selected. Observe also renders in the active voice, and Voice is reachable as a zoom-out module.
Cost Instrumentation & Dev Replay
usage.py records every LLM call (by stage, model, cache-hit ratio, and dollar cost) and streams a per-run cost summary to the admin-only Create cost panel. Prompt caching is applied where ≥2 calls share a prefix within the TTL (intra-scene regens, interactive multi-turn). replay.py is a dev-only cassette cache over the Provider (IG_REPLAY, default off): an unchanged prompt replays a recorded real response for free, so process/UI iteration costs nothing; any prompt change re-records one live call.
Library & Extraction
The Library is a per-workspace prose vault. Entry shape: {id, position, title, content, word_count, created, updated, extracted, phase_state, kind?}. Entries sort by the dotted position; entries flagged kind:"reference" (world-archive / extraction-source material rather than manuscript prose) collapse into a single reference-appendix group below the manuscript, so a large imported world bible never buries chapter one.
- Import — paste,
.txt/.docxupload (chapter detection via headings or Word heading styles), or save a session._cleanup_prose()normalises on every path (line-mode-aware paragraphs, OCR/Gutenberg artefacts). - Stage board — a List ⇄ Board toggle projects the same entries as a kanban: sticky act bands, chapter columns, scene cards. Drag-to-reorder cascades positions atomically (
POST /api/library/positions, snapshot on big reorgs), title-only stubs reserve slots (Create can fill a stub in place), a Book ⇄ Story time lens surfaces flashback markers, and per-card menus launch Interrogate-at-this-moment, Observe, dating, and Create. - Scene breakdown — a Sonnet line-number pass returns break markers; chapters >3000 words auto-chunk at paragraph boundaries.
- Extract World — phased Claude passes (world bible → major characters → minor characters) with per-phase tri-state tracking, selective re-run, and retry. New facts merge; conflicts route to Canon Review; freshly-extracted characters are dated for Fix B. Duplicate prevention is built into the write path: the live archive roster is injected into every extraction prompt (reuse existing ids verbatim), incoming names are identity-normalised (honorific-stripping, accent-safe) and re-targeted onto the existing keeper as a reviewable conflict rather than a fork, surface-form variants accumulate as aliases, and a fuzzy creation gate stops near-miss new files. Events carry extraction provenance (source entry + in-scene sequence), which powers the Event Map's narrative ordering.
- Polish — a reviewable, minimal-touch copy-edit pass (Sonnet) that fixes mechanics and dialogue paragraphing only, explicitly forbidden from touching deliberate style. Preview-then-apply.
- Scene Dating — an AI pass that infers each scene's
story_date(+ confidence and optional end/frame dates) from the workspace timeline-as-vocabulary, with sync/async (flashback) displacement detection — flashbacks are dated by the events depicted, not the narration frame. Metadata-only; enables story-order sorting and the knowledge-state chain. - Pronoun Consistency lint — one Sonnet coreference pass per scene against each character's canonical pronouns; returns findings
{character, expected, found, quote}. Report-only — prose is never edited. - ReScribe — inline single-paragraph rewrite to an instruction, in the active Author Voice, single-pass Beat-Copped, presented as a reviewable proposal (Sonnet default, Opus toggle).
- Braindump — bulk-ingest flipside of Brainstorm (World Builder tab): paste any unstructured content and the extract-world engine turns it into reviewable Archive entries.
- Act.Chapter.Scene import — the upload/paste chapter detector reads
x.y.z-numbered manuscripts and preserves their positions. - Compile —
GET /api/library/compileconcatenates selected entries into a downloadable TXT, Markdown, or Word (.docx) manuscript in sort order (DOCX uses Heading 1 per chapter, so it round-trips through the upload detector). - Append flow — the character card's ADD zone opens a composer: free text + type chips + date → preview → brainstorm → Canon Review → placement-aware merge. Corrections are sovereign; additions are reviewed.
Archive hygiene toolbar
Batch tools run from the Archive head-bar, each metered and recorded to a per-workspace tool-run history with persistent findings reports: Deep duplicate scan (one semantic Sonnet pass per category groups duplicates by meaning, date-agnostic; resolve = snapshot → keep one → delete the rest; a scoped scan also fires automatically after every extraction/Braindump), reviewable character merge (field-level plan — collections union, prose conflicts default to an additive "keep both", identity scalars are a true either-or — then apply repoints every reference to the survivor across characters and world bible), Date All (batch knowledge dating over the whole cast, knows + suspects, prefix-cached, idempotent), Gender & Pronouns (first-class fields the renderer and the pronoun lint both read), ⚖ Stances (backfills a first-class 13-stage ally→enemy stance onto relationship prose — one cheap pass per character, reviewable before apply; stances colour the Continuity Map and ground the engine's disposition lines), ⛓ Link causes (one AI pass proposes cause→effect links across the whole timeline, reviewed row-by-row, snapshot-first, idempotent), and the POV schema migration.
World Surfaces — Continuity Map · Event Map · Atlas
Archive, Map, Timeline, and Atlas are one surface: while in Archive mode the top bar shows a [◫ Archive | ◉ Map | ⏱ Timeline | 🧭 Atlas] segment control. Switching views is non-destructive (focal, lens, and drill state survive round-trips); card MAP zones route by category — a character card opens the Continuity Map focused on them, an event card opens the Event Map drilled at that event, a location card flies to its Atlas pin.
Continuity Map (◉)
An egocentric node graph (DOM nodes + SVG edges, no physics library): the focal character centred, their known world radiating out — people, entities, places, plus concentric knowledge bands (Suspects / Knows / Timeline, and Blind Spots under the God lens). Relationship edges colour by stance (green↔red by ordinal; no stance = neutral chrome), with an (i) marker opening the full relationship prose. Chip clicks drill into a relationship view ("Y as X sees them" — X's prose for Y plus only the facts of X's that mention Y); double-click re-centres. A Single / All scope renders the entire cast as one graph (shared entities dedup to central hubs, mutual edges carry both directions). Node drags persist per workspace + focal; sidebar cards launch Interrogate / Simulate / Observe scoped to the node.
Event Map (⏱)
The timeline lens is an event lattice ordered by narrative position, not just dates: order derives mechanically from extraction provenance (source scene position + in-scene sequence), arc membership is authored (select cards → new m-arc container / parallel p-arc track; entry/exit anchors sync rows across arcs; ▽ gaps show where a sparse arc elapses against a dense one), and causal leads_to links are annotations that never move cards — a date that contradicts the lattice flags amber, never silently reorders. A segmented year → month → day drill lens is the second view; event cards edit causes/leads-to inline, and the ⛓ Link-causes tool backfills causal links with one reviewed AI pass. Band summaries, + Event creation, and per-band flatten round it out.
World Atlas (🧭)
Uploadable map images per workspace (atlas.json sidecar + /api/atlas/*) with pan/zoom and pinnable locations: a pin mode + location tray places, moves, and removes pins (normalised 0–1 coordinates, whole-list PUT, debounced). Location cards and Read sheets route into the Atlas focused on their pin; deleting a location cleans its pins.
World Pack & Machine Worlds API
The canon engine is reachable by other tools, in both directions:
- World Pack export —
GET /api/world-packserialises the active workspace to a portable, tool-neutral interchange JSON (world_pack: "1.0"): bible sections rendered by the loader's own formatters, approved facts distilled from the canon log, character sheets (relationships resolved to display names, POV/stance aware), and the inherited universal tier when attached. Read-only, no AI cost; one button in Settings → Data. Spec:docs/architecture/world-pack.md. - Machine Worlds API —
/api/machine/*, a token-authed (IG_MACHINE_TOKENbearer; constant-time compare; disabled entirely when unset) headless surface for story-generation tools:POST /api/machine/worldsimports a World Pack as a workspace under an isolated_machineservice namespace (never visible to human accounts),GET …/readinessreturns a coverage verdict (cast / setting / rules / facts checks → band + human-readable gaps),POST …/consultruns a Beat Cop canon audit on draft prose ({verdict: pass|revise, findings: [{severity, fact, quote, note}]}), andDELETE …/{slug}reclaims the world (traversal-proof, re-validated before removal). Consults spend the machine world's own key — pack-supplied or the server'sIG_MACHINE_*env provider — never the human credit ledger, and model choice is server-controlled (IG_MACHINE_MODELSoverrides any pack-supplied mapping). An admin Settings tab lists and deletes machine worlds without exposing the token. Contract:docs/architecture/machine-worlds.md. First consumer: StoryForge.
Settings & Themes
Settings persist per user in data/users/<user>/settings.json (theme, active_workspace, api_key). Saving a key re-resolves and rebuilds the orchestrator in place — no restart.
Theme families
Two invertible default families select via a Spectrum/Binary toggle above the Dark/Light toggle:
- Spectrum — an accent-driven theme with a violet→red nav-rail spectrum; each mode's accent equals its nav slot, so a module's colour is consistent from idle icon to open panel. Maps to
data-theme="tech-noir"/"light". - Binary — high-contrast greyscale, no hue; every accent collapses to neutral. Maps to
"binary"/"binary-light". Exception: genuine status indicators keep their semantic colour via a dedicated--signal-ok/-warn/-badset.
Stored as theme.family × theme.scheme → one of four [data-theme] blocks. The theme editor (theme-editor.js) allows live token overrides with an APCA "constrain" mode; overrides persist in theme.custom. Appearance toggles include desaturate-logo (Binary), borderless / minimal top bar, and the per-browser sidebar collapse. The login screen is pinned to Spectrum-dark.
Data Export & Import
GET /api/export assembles a ZIP of the active workspace (characters, world bible, library, sessions, canon log). POST /api/import restores a ZIP into the active workspace and returns counts; a page reload re-applies world data to the orchestrator. Import overwrites the active workspace — restore into a dedicated one. Template workspaces are import-guarded.
Reset Operations
| Operation | Scope | Preserved | Confirm |
|---|---|---|---|
| Session Reset | Clears in-memory conversation histories (incl. live_session.json). | All world data, saved sessions, canon, library | Sidebar — none |
Clear History (reset-all) | Deletes all unpinned sessions. | Pinned sessions, world data, library | Modal |
Full Wipe (reset-full) | Deletes the active workspace's sessions, canon, characters, and world bible. | API key, settings, library | Type DELETE |
API Endpoints Reference
The surface is ~197 routes. Grouped families below (auth carried via HttpOnly cookie; streaming endpoints emit SSE; /api/machine/* is bearer-token authed instead).
Auth & account
| Method | Path | Description |
|---|---|---|
| POST | /api/auth/register | Register with an invite code. |
| POST | /api/auth/login | Login; sets HttpOnly session cookie. |
| POST | /api/auth/logout | Clear session. |
| POST | /api/auth/change-password | Self-service password change (current password required; revokes other sessions). |
| GET | /api/auth/me | Current user, quota, multi_user flag, restart status. |
| GET | /api/me/status · /api/landing/summary · /api/board | Beta status + readiness / landing Status Hub aggregate / public progress board. |
Conversation
| Method | Path | Description |
|---|---|---|
| POST | /api/simulate | SSE. Causal world reasoning. |
| POST | /api/interrogate | SSE. Character voice; accepts persona_* (YOU ARE) + scene fields. |
| POST | /api/observe | SSE. Author-voice scene rendering with multi-turn memory. |
| POST | /api/reset · /api/reload | Reset histories / reload world from disk. |
Workspaces, universes, characters, archive
| Method | Path | Description |
|---|---|---|
| GET | /api/workspaces | List (includes kind for universes). |
| POST | /api/workspaces/switch · /create | Switch (auto-import prose) / create blank. |
| GET | /api/universes | List universes + current attachment + conflicts. |
| POST | /api/universes/create · /api/workspaces/universe | Create a universe / set-clear attachment. |
| GET | /api/characters · /api/archive · /api/archive/search | Character list / full catalog / full-text search. |
| PUT DEL | /api/archive/{type}/{id} | Edit / delete an archive entry (triggers reload_world). |
| POST | /api/archive/upgrade-schema · /api/archive/date-knowledge | POV migration / backfill knowledge dates. |
| POST | /api/archive/event · /api/archive/append/preview|apply | Create a timeline event (snapshot-first) / character append composer. |
| various | /api/archive/image/{type}/{id} · /api/portraits/* | Per-card images for every archive category (POST/GET/DELETE) / legacy character portraits. |
| various | /api/archive/timeline-arcs · /timeline-bands · /timeline/slot | Event Map: authored arcs sidecar / user band summaries / server-computed manual slot moves. |
| various | /api/atlas/maps* | World Atlas: map CRUD, image upload/serve, whole-list pin replace. |
Canon & brainstorm
| Method | Path | Description |
|---|---|---|
| GET | /api/canon | Pending queue (conflicts + consequences). |
| POST | /api/canon/flag · /approve · /dismiss | Flag / approve (writes structured canon) / dismiss. |
| POST | /api/canon/apply-fields · /api/canon/universal | Per-field apply / promote to the universe. |
| GET POST DEL | /api/brainstorm* | List / add / discard / promote brainstorm items. |
Library
| Method | Path | Description |
|---|---|---|
| GET | /api/library · /{id} · /search · /compile | List / full entry / search / compile to manuscript (txt · md · docx). |
| various | /api/library/positions · /api/library/board · /{id}/time-mode | Stage board: atomic position cascade / act-title sidecar / sync-async scene mode. |
| POST | /api/library/upload · /bulk · /breakdown | Upload / bulk add / AI scene breakdown. |
| POST | /api/library/extract-world · /polish · /reclean | Phased extraction (per-workspace concurrency-locked, 409 on double-run) / reviewable polish / re-clean. |
| POST | /api/library/date-scenes · /pronoun-lint · /rewrite/preview|apply · /api/braindump/extract | Scene Dating / pronoun lint (report-only) / ReScribe paragraph rewrite / bulk brain-dump ingest. Dating & lint have /estimate twins. |
| PUT DEL | /api/library/{id} | Update / delete an entry. |
Quill & archive tools
| Method | Path | Description |
|---|---|---|
| POST | /api/jeeves | Quill Ask — read-only canon Q&A (grounded, gap-honest). |
| POST | /api/quill/preview · /apply | Quill Sweep — plain-English edit of existing entries: semantic scan → preview diff → snapshot → apply. |
| POST | /api/quill/establish/preview · /apply | Quill Establish — add-new canon: draft → preview → Brainstorm draft (default) or Canon Review direct. |
| various | /api/quill/snapshots · /restore | Write history (snapshots) + restore. |
| POST | /api/archive/duplicates/scan · /resolve | Semantic duplicate scan (all categories) / snapshot + keep-one resolve. (GET /api/archive/duplicates = the free structural character pass.) |
| POST | /api/archive/merge/preview · /apply | Reviewable character merge (field plan → union/combine/choose → repoint references). |
| various | /api/archive/date-knowledge* · /date-knowledge-all · /gender/scan|apply · /tool-runs | Knowledge dating (single + batch, with estimates) / gender & pronouns tools / tool-run history. |
| various | /api/archive/stance/estimate|scan|apply · /api/archive/link-causes/estimate|scan|apply | Relationship-stance backfill / AI causal-link backfill (both estimate → scan → review → snapshot-first apply). |
| POST | /api/canon/reopen · /api/stage-manager | Reopen a logged canon decision (restore + re-queue) / out-of-character app help. |
Create (unlock-gated)
| Method | Path | Description |
|---|---|---|
| POST | /api/create/generate | SSE. Scene spec → director/ensemble → render → Library draft. |
| POST | /api/create/chapter/beat | SSE. Render one chapter beat (gated). |
| GET POST DEL | /api/create/plan · /api/create/sessions* | Persisted chapter plan / named chapter sessions. |
| various | /api/create/voices* · /api/create/guest/promote | Author Voice CRUD + analyse/activate; promote a guest to cast. |
Sessions, settings, data, system
| Method | Path | Description |
|---|---|---|
| various | /api/session/save · /api/sessions · /api/session/load · /api/sessions/{name}/pin · /to-library | Session save/list/load/pin/promote. |
| GET POST | /api/settings | Get (key masked) / update settings. |
| GET POST | /api/export · /api/import | ZIP backup / restore. |
| POST | /api/reset-all · /api/reset-full · /api/reset-full/preview | Clear history / full wipe / pre-wipe counts. |
| various | /api/patch-notes* · /api/bug-report · /api/chatboard* · /api/maintenance · /api/waitlist · /api/music/playlist | What's-New notes / bug report / chat board / maintenance status / public waitlist / Aura playlist. |
| POST | /api/admin/* · /api/deploy | Admin: users, invites, activity, cohorts, unlock requests + examine, budget caps, BYOK mode, machine worlds, restart, deploy (SSE). |
Machine (bearer-token authed)
| Method | Path | Description |
|---|---|---|
| GET | /api/world-pack | Export the active workspace as a portable World Pack JSON (session-authed, read-only). |
| POST | /api/machine/worlds · /{slug}/pack | Import / idempotently re-upload a World Pack as a machine world. |
| GET | /api/machine/worlds/{slug}/readiness | Coverage-readiness verdict (band + missing checks + counts). |
| POST | /api/machine/worlds/{slug}/consult | Beat Cop canon audit of draft prose → {verdict, findings}. |
| DEL | /api/machine/worlds/{slug} | Delete a machine world (traversal-guarded). |
Character JSON Schema
One JSON file per character in the workspace's characters/. Sections may be POV-wrapped (see POV-Aware Schema); hard-canon scalars are not.
{
"id": "evelyn_morse", // matches filename stem (hard canon)
"name": "Evelyn Morse",
"real_name": "...", "role": "...", "archetype": "...",
"importance": "major", // major | minor | extra
"aliases": ["..."],
"biography": { "summary": ..., "neuralink_origin": {...} }, // POV-wrappable
"psychology": { "core_drive": ..., ... }, // POV-wrappable
"voice": { "register": ..., "verbal_tells": [...] }, // POV-wrappable
"arc": { "start_state": ..., "pressure_points": [...] },// POV-wrappable
"relationships": { // POV-wrappable
"char_id": "plain prose", // legacy shape
"other_id": { "text": "prose", // stance-carrying shape
"stance": "adversary", // 13-stage ally→enemy enum
"stance_qualifier": "business" } // partner/adversary only
},
"knowledge_state": {
"knows": [ {"fact": "...", "since": "2037"}, "always-known string" ],
"suspects": [ ... ],
"does_not_know": [ ... ]
},
"gender": "...", "pronouns": "she/her",
"agent_instructions": { "communicative": false, ... }, // meta, not POV-wrapped
"portrait": "images/portraits/evelyn_morse.png"
}
Only id and name are strictly required. Every present section is grounded into the character sheet when that character is active. knowledge_state is inherently POV (it is the character's perspective) and drives Fix-B filtering and the Continuity Map's knowledge bands. Relationship stances are directional (X→Y lives on X's sheet), colour the Continuity Map deterministically, and render as conditional Relationships (disposition) lines in both the participant sheets and the world context — with zero stances the prompts are byte-identical (cache-stable).
World Bible JSON Structure
World bible files are read key-tolerantly. Conventional shapes:
timeline.json
{ "timeline": { "summary": "...", "events": [ {
"id": "evt_breach", "label": "The Breach",
"date": "2036-12-16", "date_precision": "day", // day|month|year|relative
"anticipation": "AOG", // CAL (calendar) | AOG (act-of-god)
"characters_affected": ["evelyn_morse", "..."],
"summary": "...", "consequences": "...", "tags": ["..."]
} ] } }
politics.json / concepts.json / geography.json
{ "politics": { "entities": [ {"id","label","type","status","members":[...], ...} ] } }
{ "concepts": { "concepts": [ {"id","label","category","summary", ...} ] } }
{ "geography": { "locations": [ {"id","label","status","role","key_sites":[...], ...} ] } }
POV-wrappable leaves inside these (summary, description, role_in_story, etc.) follow the wrapper shape; hard-canon scalars (id, label, dates, tags, members) do not.
Deployment & Operations
- Topology — Ubuntu VPS, nginx reverse proxy (
client_max_body_size 20M,proxy_read_timeout 900s— sized so a legitimate long extraction streams to completion), the app as a systemd service running as a dedicatedinterrogateuser, Let's Encrypt TLS. - Deploy — preferred path is the in-app Admin → Deploy button (
/api/deploy, streams rsync + remote restart over SSE). Fallback is a manual rsync with a strict exclude list (.env,venv/,data/,settings.json, sessions/canon,__pycache__). Static-only changes are an rsync ofstatic/+ chown, no restart. - Never rsync'd —
data/(all per-user worlds, sessions, canon, quotas, invites, chat). It only lives on the server. - Env —
ANTHROPIC_API_KEY,MULTI_USER,DEFAULT_USER; Machine Worlds:IG_MACHINE_TOKEN(unset = API disabled),IG_MACHINE_PROVIDER/IG_MACHINE_BASE_URL/IG_MACHINE_API_KEY/IG_MACHINE_MODELS(server-controlled model map); optional integrations like a Discord bug-report webhook;IG_REPLAY(dev cassette cache).
Roadmap
Shipped since the first spec: multi-user auth + quotas + multi-provider BYOK + self-service password change, Canon Review with per-field apply (all entry types) + fact-level knowledge reconciliation + ⟲ Reopen + Beat-Cop consequences, POV schema (both phases, including perspective cards, Keep-Both split-view, and simulator POV resolution), relationship stances, the unified world header with the Continuity Map / Event Map / World Atlas, Fix-B knowledge filtering (knows + suspects), the Universal Layer + Universal Extractor, the full Create / Ensemble / Author-Voice generative engine with the unlock pipeline, Quill (Ask / Sweep / Establish + snapshots) + the Stage Manager, the Archive hygiene toolbar (semantic dedup + prevention-at-extraction, character merge, Date All, Gender & Pronouns, Stances, Link causes) with tool-run history, the Library prose passes (Scene Dating, pronoun lint, ReScribe, Braindump, x.y.z import) + the Stage board + the reference-appendix group, manuscript compile (TXT/MD/DOCX), the beta economics layer (spend ledger, readiness board, runtime budget caps, onboarding allowance), the landing Status Hub, patch notes, Spectrum/Binary themes, extraction truncation/concurrency guards, accent-safe slugs, and the World Pack + Machine Worlds integration surface.
In progress / planned
- in dev Magistrate — the Editors' Suite — the post-draft editorial chain. Shipped links: Polish, ReScribe, Scene Dating, pronoun-lint worklist. Ahead: the knowledge-state run (scenes dated × knowledge dated → find prose where a character acts on knowledge they don't yet have), the pronoun-run rewrite half, draft-vs-final modes, and the tier gate.
- scoped Account recovery — in-app reset request (Settings + login screen), admin one-shot temp password, forced change on next login (
docs/scopes/account-recovery-scope.md). - scoped Workspace profiles — a
meta.profileconfig layer (label map, chunker, prompt pack, mode gating) so non-fiction workspaces stop wearing fiction labels; first profile scoped: legal casework (transcript chunking with page:line cites, testimony-bucket knowledge, a contradiction ledger).docs/scopes/workspace-profiles-scope.md. - planned Model refresh (Opus 4.6 → Opus 5 for the prose tier) and mixed per-stage provider routing (cheap judge/extraction tiers on an alternate provider while prose stays premium — the Beat Cop eval groundwork is done); Scene Setup / Create wizards; scheduled canon auto-backup (distinct from tool snapshots); Rigorous Mode (knowledge-gap detection).
- planned Launch infrastructure — local engine + thin license-relay (the launch privacy architecture), licensing/billing. The provider abstraction is the first load-bearing piece.