Evidence kinds/Schema validation result
Schema validation result
structural evidence
Per-entry record of whether the model output satisfied the spec's closed-enum schema.
What it is
Each RuntimeAISpec declares an outputSchema with closed-enum constraints (specific allowed values for each field). At runtime, the substrate validates the model's output against this schema using a Zod-based validator.
The result — schemaValid: true or false — is recorded on the L12 entry. When false, the fallbackBehavior fires (reject, deterministic-default, or queue-for-review).
Schema validation is the substrate's structural answer to LLM hallucination. The regulator can audit the closed envelope the model was forced to operate within — not just what it did say, but what it COULD have said.
What it guarantees
- Per-decision validation result recorded.
- Closed-enum enforcement at runtime — invalid output never reaches the caller without fallback intervention.
- Schema-validity rate is a directly regulatable system-reliability metric.
What it does NOT guarantee
Honest limits. Every primitive has them.
- That the schema itself is correct or comprehensive — operator's design.
- That valid output is semantically correct — the substrate checks structure, not meaning.
Regulations that engage this kind
Accuracy + robustness
Logging events including failures
Sectors that rely on this kind
Related evidence kinds
Fallback decision log
Records when the substrate fell back to a safe default after model failure or schema violation.
Spec hash commitment
Every L12 entry includes a SHA-256 of the active RuntimeAISpec, making silent spec drift detectable.
L12 receipt chain
Hash-chained, Ed25519-signed log of every LLM-in-the-loop decision a Promethean-built product makes.
FAQ
What kinds of schemas can we declare?
Closed enums (a fixed list of allowed values), strings with length limits, numbers with min/max, booleans, structured objects with required + optional fields. The Zod-style declaration gives flexibility without leaking back into free-text LLM output.