Promethean · Technical Architecture Overview

Document type Technical architecture · CTO · staff engineer · platform architect Length ~9 pages Last revised 2026-05-18 (v1.3 — post-launch hardening) Audience precondition Comfortable with cryptographic primitives (SHA-256, Ed25519, hash chains), Node.js / Python toolchains, and regulated-software audit requirements. Canonical theory PROMETHEAN_THEORY_AND_FINDINGS — five primitives, threat model, adversarial findings, verifier benchmarks. This document is the engineering companion to that paper. Companion docs PRODUCT_DATASHEET · SECURITY_BRIEF

1. The architectural commitment

The substrate's defining commitment since v0.1: AI as building material, not arbiter. The build-time pipeline uses an LLM in exactly one step (intent → typed ProductSpec) and 49+ deterministic generators take over downstream. Emitted products contain no LLM at runtime — until Phase R.

Phase R (May 2026) extends the same pattern to LLM calls inside emitted products. Same closed-enum schema enforcement. Same reviewer-queue gating. Same cryptographic receipt chain. Same Ed25519 trust anchor. Same audit primitives, applied at request-time instead of build-time.

This is the substrate's central technical bet: the cryptographic + closed-enum machinery that works for compiler-style code emission also works for runtime AI calls, because both are decision events with bounded surface area and verifiable provenance.

2. The L1–L12 commitment stack

Twelve independent hash-chained logs, all rooted in one Ed25519 trust anchor, all committed under L1 state.json which is itself Bitcoin-anchored via OpenTimestamps (L8).

Why twelve, not one combined chain. Different layers attest to different facts under different operational constraints. L12 entries are written per request (microseconds matter); L7 entries are written per nightly state regeneration (latency tolerant). Separating the chains lets each be optimised for its workload while keeping all heads committed under the same L1 + Bitcoin anchor.

3. The runtime-AI primitive

A RuntimeAISpec declares the contract for one type of LLM call. The full interface (from contrib/jarvis/src/runtime-ai-spec.ts):

interface RuntimeAISpec<TInput, TOutput> {
  readonly specId: string;                       // kebab-case, ≤ 128 chars
  readonly displayName: string;
  readonly description: string;
  readonly category: RuntimeAICategory;          // closed enum (5 values)
  readonly schemaVersion: 'promethean-runtime-ai-spec-1.0';
  readonly canonicalForm: 'v1';
  readonly inputSchema: RuntimeAISchema;         // substrate-defined schema type
  readonly outputSchema: RuntimeAISchema;        // closed-enum vocabularies enforced
  readonly promptTemplate: { system: string; user: string };
  readonly modelIdentity: RuntimeAIModelIdentity;
  readonly reviewerGate: RuntimeAIReviewerGate;  // closed enum: 'always' | 'on-low-confidence' | 'never'
  readonly lowConfidenceThreshold?: number;      // used when reviewerGate = 'on-low-confidence'
  readonly maxLatencyMs: number;
  readonly fallbackBehavior: RuntimeAIFallbackBehavior;
  readonly deterministicDefault?: TOutput;       // used when fallbackBehavior = 'deterministic-default'
}

const RUNTIME_AI_CATEGORIES = [
  'classifier', 'extractor', 'drafter', 'router', 'summariser',
] as const;

const RUNTIME_AI_REVIEWER_VERDICTS = [
  'approved', 'rejected', 'amended',
] as const;

const RUNTIME_AI_PROVIDERS = [
  'anthropic', 'openai', 'azure-openai', 'local-llama', 'mock',
] as const;

The SDK enforces the spec at request time. The runConstrainedAI call takes three arguments — spec, input, and an options bag (private key handle / reviewer queue / tenant ID / nowMs):

const decision = await runConstrainedAI(spec, input, options);
// At this line, ONE L12 entry has been:
//   - written to <log>.jsonl (atomic append)
//   - committed in the sidecar head.json (O(1) read for next call)
//   - signed under the operator's Ed25519 key
//   - hash-chained to the previous entry
// `decision.output` is guaranteed to conform to spec.outputSchema (or be the deterministicDefault).

The function call is the entire integration surface. The spec is the contract; the SDK does the rest.

4. The L12 entry shape

One entry per LLM call. Canonical form is stable across substrate versions (canonical-form v1 since ADR (tt)):

{
  "id": 142,
  "recordedAtMs": 1747500000000,
  "recordedAtIso": "2026-05-17T18:40:00.000Z",
  "productId": "paysafe",
  "specId": "fraud-classifier",
  "specHash": "0a1b2c3d…",
  "inputHash": "1a2b3c4d…",
  "outputCanonicalHash": "2a3b4c5d…",
  "category": "classifier",
  "modelIdentity": {
    "provider": "anthropic",
    "model": "claude-sonnet-4-5",
    "version": "2026-05-01"
  },
  "latencyMs": 142,
  "schemaValid": true,
  "reviewerVerdict": null,
  "fallbackTriggered": false,
  "prevHash": "13e4274c…",
  "hash": "f8a91c2b…",
  "attestation": { algorithm, publicKey, signature, signedAtIso, canonicalForm }
}

Key technical decisions:

• Per-entry signing, not chain-tail signing. A runtime AI decision is the artifact — there's no underlying signed object the entry could delegate trust to. The signature lives on the entry itself. (Same logic as L10 and L11.)
• Input hashed before recording (PII-safe). Raw input is never written. inputHash is SHA-256 over the canonical input. Replay against redacted source.
• Integer-only numerics (Issue 31). id / recordedAtMs / latencyMs enforced as integers. Closes the cross-runtime float-serialisation drift class (JS's JSON.stringify(1.0) is "1"; Python's json.dumps(1.0) is "1.0").
• Optional tenantId (R4). Multi-tenant deployments append tenantId at end of canonical form when present, omitted when absent. Single-tenant chains hash byte-identically post-R4.

5. The verification surface

The regulator-grade verifier is verify.mjs — plain JavaScript, Node 18+ stdlib only, zero npm dependencies. ~500 lines; exact LoC + sha256 of the deployed file are served live at /api/facts/verifier-loc. Eight checks, all falsifiable, all offline-runnable:

$ node verify.mjs paysafe-runtime-ai-receipts.jsonl

Verifying paysafe-runtime-ai-receipts.jsonl ...
  entries:           142
  hash chain:        OK
  signatures:        142/142 verified
  trust anchor:      y6F3rt10CEcSakCrnJIYkGymA66x3VXm0zCWbPjRxP8=
  head hash:         f8a91c2b3d4e5f6a…
  earliest entry:    2026-04-12T08:14:33.000Z
  latest entry:      2026-05-13T09:31:07.000Z
  OTS anchor age:    3m 24s (Bitcoin block 893,221)
  malformed lines:   0
  ids contiguous:    yes
  timestamps:        monotonic

PASS (exit 0)

Verifier independence properties:

• Reads the chain file only (no network access required for chain integrity + signature checks)
• Bitcoin verification uses any full node the regulator chooses (default: blockchain.info / mempool.space; fully air-gappable against a local node)
• Cross-language byte parity: same verify.mjs accepts JS-emitted and Python-emitted chains identically
• Closed-field schema check (post-launch): entries containing any unknown top-level field are rejected. An attacker who appends an out-of-allowlist field (e.g. "extraField":"…") to the JSONL line cannot have it silently stripped during canonical-form rehash. Paper §2.1.2 / §5 round 3.
• Trust-anchor fingerprint banner: the verifier prints the sha256 fingerprint of every distinct public key it encountered, and a count of entries under each. If --trusted-key is supplied, the count of entries signed under that key is reported separately.

5.1 Verifier performance (benchmarks)

Measured on commodity hardware (Apple M2, Node 20.10.0):

Linear beyond ~1K. At 10M entries (Enterprise monthly allowance), a single verification pass completes in ~15 minutes on the same hardware. Streaming variant (auto-selected above 50 MB chain size) processes entries one at a time without holding the chain in memory; PASS/FAIL output is byte-identical to the in-memory variant. Source: paper §5.2.

6. Cross-language byte parity (v1.2)

The substrate emits products in TypeScript+Next, Python+FastAPI, and Rust+Axum. A Python-emitted product writing its own L12 entries MUST produce byte-identical canonical bytes + Ed25519 signatures to JS — otherwise the regulator-grade verifier would reject Python-written entries.

What ships in each language today:

Parity invariants for the canonical-form layer (load-bearing):

Verification: 8 fixture-driven differential tests at runtime-ai-py/tests/test_cross_language.py. JS generates a known signed entry → fixture written → Python re-derives canonical bytes, SHA-256 hash, Ed25519 signature → asserted byte-equal. The 8 tests cover canonical bytes match · SHA-256 match · Ed25519 signature byte-identity · Python signature verifies under JS pubkey · JS signature verifies with Python verifier · tamper detection · integer-invariant rejection · tenantId optional placement.

7. Performance characteristics

SDK overhead per call (production deployment, in-memory key):

Concurrency: per-log-path in-process serialisation queue (writeQueues map). For multi-process / multi-host deployments see SECURITY_BRIEF §5 (single-writer architecture, sharded chains, or operator-added proper-lockfile).

8. Deployment topologies

Single-host, single-product (pilot default)

┌─────────────────────────────────────────┐
│ Product process                         │
│  ├── runConstrainedAI()  (one process)  │
│  └── L12 chain on local disk            │
│         ↓                               │
│   OTS anchor cron (5-min)               │
│         ↓                               │
│   Bitcoin testnet/mainnet               │
└─────────────────────────────────────────┘

Multi-host, single chain (production)

┌──────────────┐  gRPC   ┌─────────────────┐
│ Product A    │────────→│ L12 writer pod  │
└──────────────┘         │ (one-writer-per-│
┌──────────────┐  gRPC   │ chain pattern)  │
│ Product B    │────────→│                 │
└──────────────┘         │ Chain on shared │
┌──────────────┐  gRPC   │ encrypted vol   │
│ Product C    │────────→└─────────────────┘
└──────────────┘                ↓
                          OTS anchor
                          + watch daemon

Multi-tenant (R4, hosted Author roadmap)

                ┌────────────────────────────────┐
                │ Promethean-operated infra      │
Tenant A ──tls─→│  ├── multi-tenant SDK gateway  │
Tenant B ──tls─→│  ├── per-tenant L12 chains     │
Tenant C ──tls─→│  ├── shared trust anchor +     │
                │  │   tenant-scoped verification │
                │  ├── HSM-backed signing        │
                │  └── continuous-watch + alerts │
                └────────────────────────────────┘

Federated (R5)

Cross-substrate federation commitments: each substrate signs a RuntimeAIFederationClaim carrying (substrateId, headHash, entryCount, signedAt). Industry consortia share rate posture without exchanging chain entries.

9. Reliability mechanisms (eight pillars)

Detail in ADR (ww) reliability stack + ADR (xx) v1.2 closeout:

10. Failure modes (closed-enum exit states)

The SDK returns one of seven closed-enum states per call. No "the model did something weird":

11. SDK ergonomics

// Author the spec (one-time, ~15–25 lines for a typical classifier):
import { defineRuntimeAISpec } from "@/contrib/jarvis/src/runtime-ai-spec";

const fraudSpec = defineRuntimeAISpec({
  specId: "fraud-classifier",
  displayName: "Fraud risk classifier",
  description: "Classifies a transaction into a four-tier risk band.",
  category: "classifier",
  inputSchema: { /* substrate-defined schema for transaction shape */ },
  outputSchema: { /* closed-enum: risk ∈ {low,medium,high,block} */ },
  promptTemplate: {
    system: "You score transaction fraud risk into four bands…",
    user: "Transaction: {{transaction}}",
  },
  modelIdentity: { provider: "anthropic", model: "claude-sonnet-4-5", version: "2026-05-01" },
  reviewerGate: "on-low-confidence",
  lowConfidenceThreshold: 0.7,
  maxLatencyMs: 5_000,
  fallbackBehavior: "deterministic-default",
  deterministicDefault: { risk: "medium", reason: "model-failed-schema" },
});

// Invoke at request time:
const decision = await runConstrainedAI(fraudSpec, transactionInput, {
  privateKeyBase64: process.env.SUBSTRATE_KEY,
  logPath: "/var/log/promethean/runtime-ai-receipt-log.jsonl",
  tenantId: "acme-bank",        // R4 multi-tenant; optional
  client: anthropicRuntimeAIClient, // or your own RuntimeAIClient
});

// decision.output is the typed, schema-valid result (or the deterministic default).
// decision.reviewerVerdict is 'approved' | 'rejected' | 'amended' | null.
// L12 entry is already written + signed.

That is the whole integration. Everything from canonical form to Bitcoin anchoring is downstream of this function call.

12. Test suite (what you inherit)

The substrate's own tests run against every reference deployment:

• 243 Phase R tests across 16 suites (0 failed, 2 platform-skipped)
• 7,400+ total tests across 223 files
• Mutation-tested via Stryker (kill-rate ≥ 75% break threshold)
• Cross-language differential tests assert byte-identity across runtimes
• Adversarial autoplay runs 8 closed-enum attacker strategies against every spec in CI
• Golden chains catch canonical-form drift between substrate versions
• tsc --noEmit clean across kernel + contrib pipeline

12a. Post-launch hardening surface (2026-05-17 → 2026-05-18)

Six iterative adversarial stress-test rounds closed 31 findings between v1.2 launch and 2026-05-18. The fixes that materially change the production architecture surface, beyond the v1.2 documentation:

• withWorkspaceLock generalised. Per-workspace Redis lock with atomic Lua-EVAL release + exponential backoff. Covers ingest, anchor cron, tier change, delete, session refresh, and key rotation. Closes the round-2 finding where 10 concurrent ingests dropped 4 entries while each returning accepted:1. Paper §5.1.
• Atomic Lua email-claim. Closes the round-2 audit-C residual on createWorkspace email TOCTOU. (The legacy-scan window remains a known residual — see SECURITY_BRIEF §10.)
• Email verification gate. workspaces.verifiedAtIso is null until the magic-link click at /verify (token verification handler at /api/auth/verify). SDK ingest + paid-tier upgrade are refused while null; downgrade to Dev is still permitted (de-escalation).
• Strict-fields check in verify.mjs. Closed-field schema enforced — unknown top-level keys cause verification failure. Closes the "tampered field gets silently stripped" class of attack.
• Verifier trust-anchor banner. verify.mjs prints the sha256 fingerprint of every distinct public key encountered + entry counts, so a regulator can sanity-check the chain was signed under the expected operator key without copying base64 strings.
• /api/facts/* layer. Six endpoints (verifier-loc, trust-anchor, ots-health, anchor-cadence, tier-allowances, regulatory-coverage) return machine-readable provenance for every quantitative claim the site makes. The verifier LoC + sha256 recompute live from the deployed file.
• Webhook-secret containment. toPublicView strips webhookSigningSecret from any client-readable workspace shape; reveal-only via authenticated /api/workspaces/.../reveal-webhook-secret route (round-5 critical fix).
• Stripe webhook event-id dedup. SET NX on event.id with 30-day TTL; replayed webhooks ack as duplicate and skip the handler. No double-billing events.
• CSRF / SSRF tightening. Origin-check middleware on anonymous form endpoints; IPv6 SSRF allowlist; chunked-transfer body cap; Cache-Control: private, no-store on auth-gated endpoints.

The detailed round-by-round trajectory (31 findings, severity curve, what each round tested in the previous round's fixes) lives in PROMETHEAN_THEORY_AND_FINDINGS §5. Per-commit references are on /release-notes.

13. Build artifacts

Every release produces:

• state.json (L1-signed nightly) — currently published at promethean.software/state.json
• One L4 build receipt per emitted product
• L11 substrate-component log (meta-generator emissions)
• L9 self-evaluation entry (substrate's own quality check)
• Runnable audit bundle (buildRunnableAuditBundle) — single JSON with embedded verify.mjs; runs on Node 18+ stdlib only

14. References

• PROMETHEAN_THEORY_AND_FINDINGS — canonical theory paper · adversarial findings · verifier benchmarks
• ADR (tt) runtime-ai-containment
• ADR (uu) runtime-ai-r4-multi-tenant
• ADR (vv) runtime-ai-r5-federation
• ADR (ww) runtime-ai-reliability-stack
• ADR (xx) runtime-ai-v1.2-closeout
• OPERATIONS_RUNTIME_AI — key ceremony · HSM · FS hardening · DR
• /api/facts — live machine-readable provenance for every quantitative claim
• /release-notes — buyer-facing roadmap with commit references for every post-launch fix

15. Open questions for your team

These are deliberately open, intended for the technical-fit interview:

1. Which LLM provider(s) will the substrate sit between? Single-provider integrations land in weeks; multi-provider routing needs spec design discussion.
2. Do you require HSM-backed signing in v1.2 or can you tolerate in-memory keys until v1.3? (We can patch signCanonical locally; documented in OPERATIONS §3.)
3. What's the read pattern on the L12 chain? Continuous verifier vs nightly audit changes infrastructure sizing.
4. Multi-tenant or single-tenant deployment? R4 primitives ship today; R4 ops tooling (per-tenant key rotation, per-tenant HSM partitions) is roadmap.
5. Is anyone in your team comfortable owning the Ed25519 key custody, or do you want Promethean-operated hosted Author (roadmap Q4)?

Authored by Promethean substrate engineering. Falsifiable claims throughout — every architectural decision links to its governing ADR with predictions. Last revised 2026-05-18.