Technical Reference

Architecture Overview

InterroGate is a single-host web application designed for local or self-hosted VPS deployment. It consists of a Python backend, a static single-page frontend, and a flat-file data store. There is no database — all persistent state is stored as JSON files on disk.

Stack

• Backend: FastAPI (Python 3.10+), served via Uvicorn
• Frontend: Single-page application — vanilla JS, HTML, CSS (no framework dependency)
• AI provider: Anthropic Claude API (claude-3-5-sonnet or later), streaming via Server-Sent Events (SSE)
• Persistence: Flat JSON files under sessions/, canon/, characters/, world_bible/
• Configuration: settings.json in the project root; optional .env for API key

Directory Layout

InterroGate/
├── app.py                      # FastAPI application — all routes
├── settings.json               # Active workspace, API key, theme (excluded from rsync)
├── .env                        # ANTHROPIC_API_KEY (excluded from rsync)
├── orchestrator/
│   ├── orchestrator.py         # Core simulation logic, prompt construction
│   └── world_loader.py         # Loads and merges world_bible/ JSON files
├── tools/
│   └── gutenberg_import.py     # Bootstrap workspace from Project Gutenberg
├── workspaces/                 # Per-workspace data (excluded from rsync)
│   └── <name>/
│       ├── meta.json           # {"display_name": "..."}
│       ├── source_text.txt     # Raw prose (triggers auto-import on workspace switch)
│       ├── library.json        # Prose vault entries
│       ├── characters/         # Character JSON files
│       └── world_bible/        # timeline, geography, politics, concepts, entities
├── sessions/                   # Saved session snapshots (excluded from rsync)
├── canon/
│   └── approved_log.jsonl      # Append-only log of approved canon items
├── static/
│   ├── index.html
│   ├── app.js
│   ├── style.css
│   └── docs/                   # This documentation
└── agents/                     # System prompt templates

Request Lifecycle

All generative calls (simulate, interrogate, observe) follow the same pattern:

1. Frontend POSTs a JSON request to the relevant /api/* endpoint.
2. The Orchestrator assembles a prompt from the three-tier context stack.
3. The Anthropic SDK streams the response as text chunks.
4. The backend wraps each chunk as a JSON-encoded SSE event (data: {"t": "..."}).
5. The frontend reads the EventSource stream and appends tokens to the current message bubble in real time.
6. A terminal data: [DONE] event closes the stream.

Modes

InterroGate operates in seven modes (six shipped, one in active development), each presenting a different interface and prompting posture. Mode state is held in the frontend only; the backend is stateless with respect to mode.

World Context System

WorldLoader

orchestrator/world_loader.py is responsible for reading and merging all world data from disk at startup (and on explicit reload). It loads two categories of data:

• World bible files — thematic JSON files in world_bible/. Each file covers a domain (timeline, factions, geography, concepts, entities, etc.) and is merged into a single world context dictionary.
• Character files — one JSON file per character in characters/. Each file conforms to the character schema (see below).

The loader caches world context in memory. The POST /api/reload endpoint clears the cache and re-reads all files from disk, allowing the writer to update JSON files externally and pull changes in without restarting the server.

World Bible Directory

The world_bible/ directory is open-ended — add any JSON file and it will be picked up on the next load. Conventional filenames by domain:

Three-Tier Context Stack

Every prompt sent to the AI is assembled from three layers of context, applied in order of precedence:

The Orchestrator merges these tiers in ascending priority order — scene context can override brainstorm framing, which can override canon defaults — but canon facts are always anchored in the base layer. The system prompt is re-assembled on each request from the current merged state.

Session System

A session captures the complete conversation state at a point in time. Sessions are stored as JSON files in sessions/ and can be loaded to restore any prior working state.

SessionState fields

Save / Load

Sessions are saved via POST /api/session/save with an optional name. If no name is supplied the current timestamp is used. The filename is sanitised to safe characters and stored as sessions/{name}.json. Sessions are listed via GET /api/sessions sorted by modification time, most recent first, with a pinned boolean field on each entry.

Loading a session via POST /api/session/load fully restores the Orchestrator's in-memory state — all conversation histories, scene context, and characters present. The frontend then reflects the restored mode and state.

Session Pinning & Protection

Sessions can be pinned via POST /api/sessions/{name}/pin (toggle). Pinned sessions survive the "Clear History" wipe (POST /api/reset-all) — only unpinned sessions are deleted. Before any wipe, GET /api/reset-full/preview returns {"total": N, "pinned": M} so the UI can warn the user about how many sessions would be destroyed.

A session can be promoted directly to the Library via POST /api/sessions/{name}/to-library. This converts the transcript into a Library entry (with auto-title from the first exchange), preserving the conversation as readable prose.

Clear History vs New Story

Canon Review Pipeline

The canon pipeline is the mechanism by which generative output is selectively promoted into the permanent world record. It enforces writer authority — the AI never writes directly to canon.

Pipeline stages

1. Flag — The writer highlights any AI response bubble and clicks "Flag for Canon". The content is added to the in-memory review queue via POST /api/canon/flag. A badge counter on the sidebar Canon Review panel updates immediately.
2. Review — The Canon Review panel lists all flagged items with their origin mode and character (if applicable). The writer can edit the content in-place via PUT /api/canon/{index}.
3. Approve or Dismiss — Approving an item (POST /api/canon/approve) appends it to canon/approved_log.jsonl as a permanent record and includes it in subsequent Canon context. Dismissing (POST /api/canon/dismiss) removes it from the queue with no side effects.
4. Materialise — Approved items accumulate in the log. The writer is responsible for periodically reviewing the log and manually updating the corresponding world bible or character JSON files, completing the cycle. This step is intentionally manual — it keeps the writer in control of how approved observations become structured canon.

Brainstorm Layer

The brainstorm layer is a session-scoped staging area for world elements the writer is actively developing but has not yet committed to canon. Items are added manually via the World Builder mode — no AI generation is involved. The writer supplies all content.

Item types

Lifetime

Brainstorm items exist in memory for the lifetime of the current session. They are included in the session snapshot when saved and restored on session load. Promoting an item (POST /api/brainstorm/{item_id}/promote) sends it to the canon review queue, making it available for approval. Items can also be discarded (DELETE /api/brainstorm/{item_id}).

Settings

All settings are stored in settings.json in the project root. The file is read on startup and written on any settings POST. Structure:

{
  "api_key": "sk-ant-...",
  "theme": {
    "scheme": "dark",     // "dark" | "light"
    "text_size": 13       // integer, 11–18 (px)
  },
  "workspace_name": "Silver Blaze",
  "active_workspace": "silver_blaze"
}

If a new API key is saved, the Orchestrator is reinitialised with the new key in-place — no server restart required. All world data and session history are preserved.

Data Export & Import

InterroGate supports full workspace backup as a ZIP archive. The export endpoint (GET /api/export) assembles a ZIP containing:

• sessions/ — all saved session JSON files
• characters/ — all character JSON files
• world_bible/ — all world bible JSON files
• canon/approved_log.jsonl — the full approved canon log

The import endpoint (POST /api/import) accepts a ZIP file, extracts each component into the correct directory, and returns a count of restored items. A full page reload is required after import to apply world data changes to the in-memory Orchestrator.

Reset Operations

API Endpoints Reference

Character JSON Schema

Each character is stored as a single JSON file in characters/. The filename becomes the character's ID (minus .json). All fields are strings unless noted.

{
  "id":           "evelyn_morse",        // matches filename stem
  "name":         "Evelyn Morse",
  "role":         "Senior CCARB analyst",
  "age":          "34",
  "background":   "Narrative biography — upbringing, formative events, ...",
  "personality":  "How they think, what drives them, core tensions",
  "speech":       "How they speak — register, vocabulary, verbal tics",
  "knowledge":    "What they know and don't know at story-present",
  "relationships": {
    "character_id": "Description of relationship",
    ...
  },
  "arc":          "Where they start, where they're going, what changes",
  "secrets":      "What they conceal and from whom",
  "portrait":     "images/evelyn_morse.png",   // optional, relative to /static/
  "notes":        "Anything not covered by other fields"
}

Only id and name are strictly required. All other fields are included in the context sent to the AI when that character is active in Interrogate mode. Missing fields are gracefully omitted.

World Bible JSON Structure

World bible files have no enforced schema — the WorldLoader reads them as-is and passes them to the Orchestrator's prompt construction logic. Conventional structures by type:

timeline.json

{
  "timeline": {
    "summary": "One-paragraph overview of the story's historical context",
    "events": [
      {
        "date": "March 2036",
        "title": "The Incident",
        "description": "Full description of the event and its consequences"
      }
    ]
  }
}

factions.json

{
  "politics": {
    "factions": [
      {
        "name": "CCARB",
        "type": "government_agency",
        "purpose": "What this organisation does and why it exists",
        "power": "How much influence it holds and over what",
        "internal_tensions": "Divisions, factions within the faction",
        "key_members": ["evelyn_morse", "..."]
      }
    ]
  }
}

geography.json

{
  "geography": {
    "locations": [
      {
        "name": "The Percolator",
        "type": "café",
        "description": "Physical description and atmosphere",
        "significance": "Why this place matters to the story"
      }
    ]
  }
}

Planned Development

The following features are on the development roadmap. None are currently implemented.

World Builder Domain Framework

planned A guided world-building interface with completeness indicators. Rather than free-form entry, the writer is walked through structured domains (political, geographical, sociological, technological, historical) with prompts for each sub-element. A completeness bar shows how much of each domain is populated, helping identify gaps in the world model before they become consistency problems during drafting.

Session Capture

planned A one-click mechanism to save any AI-generated output directly as a persistent brainstorm item, bypassing the manual canon flag workflow. Intended for generative sessions where the writer wants to capture a high volume of output quickly without reviewing each item in the canon queue immediately. Captured items persist across sessions and can be reviewed and promoted at any time.

Audit Mode

planned An AI-powered consistency validator. The writer selects a scope (a character, a timeline period, a faction's behaviour) and Audit mode cross-references the entire canon against it, identifying contradictions, implausibilities, or gaps. Audit does not make creative suggestions — it flags logical inconsistencies and leaves resolution to the writer. Intended for use before major drafting phases.

In-Panel Editing

planned Direct editing of character cards and world bible entries from within the Archive panel. Currently these files must be edited externally in a text editor. In-panel editing would allow the writer to update canon data without leaving the application, with immediate in-memory reload on save.

Remove / Retire Characters

planned Soft-delete (retire) and hard-delete (remove) operations for character files. Retiring would flag a character as inactive — excluded from Interrogate mode selection and context injection but preserved on disk. Hard delete would remove the file entirely with a confirmation step. Useful for characters who die, merge, or are restructured during development.

Portrait Upload via UI

planned A file upload interface for character portrait images directly in the Archive or Settings panel, rather than requiring manual file placement in static/images/ and JSON editing. The uploaded image would be saved to the correct path and the character's portrait field updated automatically.

Prose Library shipped

A seventh mode providing persistent per-workspace storage for actual written prose. Entries have a title, a timeline position, a prose body, and auto-computed word count. Full-text search across titles and prose. Add via paste, .txt, or .docx upload. Gutenberg-formatted texts (double blank lines between paragraphs) handled correctly. Sessions can be promoted directly to Library entries. Isolated from the AI workflow — excluded from the Full Wipe ("New Story") operation.

Workspace System shipped

Multiple isolated world contexts in workspaces/<name>/. Each workspace has its own characters, world bible, library, and session history. Switching workspaces reloads the orchestrator with the new world context. source_text.txt in a workspace triggers automatic chapter detection and library import on first switch.

Gutenberg Import Tool shipped

Command-line tool (tools/gutenberg_import.py) to bootstrap a workspace from any Project Gutenberg text. Supports --prose-only mode (no Claude, no token cost — downloads prose only) and full extraction mode (three Claude passes: world bible, characters, entities). Automatically bounds extraction to a specific story within a collection via --next-story.

Example / Demo Dataset shipped

The Silver Blaze workspace (Arthur Conan Doyle, 1892 — The Memoirs of Sherlock Holmes) ships as the reference test environment. 11 characters, 19 timeline events, 11 locations, extracted automatically via the Gutenberg import tool. Demonstrates the full workspace pipeline from raw text to interactive simulation.

Sessions Per Workspace

planned Currently sessions are stored globally in sessions/. Isolating sessions per workspace (to workspaces/<name>/sessions/) would prevent sessions from one world context appearing in another's session panel. Agreed as the correct architecture; not yet implemented.

World Builder Domain Framework

planned A guided world-building interface with completeness indicators. Rather than free-form entry, the writer is walked through structured domains (political, geographical, sociological, technological, historical) with prompts for each sub-element. A completeness bar shows how much of each domain is populated, helping identify gaps in the world model before they become consistency problems during drafting.

Audit Mode

planned An AI-powered consistency validator. The writer selects a scope (a character, a timeline period, a faction's behaviour) and Audit mode cross-references the entire canon against it, identifying contradictions, implausibilities, or gaps. Audit does not make creative suggestions — it flags logical inconsistencies and leaves resolution to the writer. Intended for use before major drafting phases.

In-Panel Editing

planned Direct editing of character cards and world bible entries from within the Archive panel. Currently these files must be edited externally in a text editor. In-panel editing would allow the writer to update canon data without leaving the application, with immediate in-memory reload on save.

Online Deployment & Subscription Model

planned InterroGate supports local and self-hosted VPS deployment. A managed cloud option is under consideration for writers who prefer not to manage their own server. Key design constraints: data sovereignty (writers own their world data), API key management (bring-your-own vs. metered subscription), and multi-workspace support per user account.