Why traditional logging (Datadog, Splunk, CloudWatch) is not an audit log

Most engineering teams already log everything. Datadog, Splunk, CloudWatch, Loki, Grafana — whichever stack the company uses for SRE telemetry. When someone first hears about "audit logs" for an AI compliance regime, the natural reaction is: we already do that. We log every request, every response, every decision. Done.

Application logs and regulator-grade audit logs have a lot of surface overlap. They both contain timestamps. They both record events. They both end up in some kind of database. The functional difference is subtle until a regulator asks the specific question: how do you know this log entry was created at the time it says, rather than backdated by someone with write access?

That question has exactly two acceptable answers: cryptographic chain or trusted third-party witness. "Because we trust our operations team" is not one of them.

The question that distinguishes them

Imagine a regulator inspecting your AI-decisioning system after a complaint. They have:

• The complaint, dated 2026-05-10.
• Your claim that the decision in question (made on 2026-04-15) followed your standard reviewer process.
• Your log export showing the reviewer engagement on 2026-04-15.

The regulator's specific concern:

• Can the company prove the log entry was created on 2026-04-15, rather than written between 2026-05-10 (the complaint) and now to fit the company's story?

For a Datadog log: the company can show internal access controls, SOC 2 evidence, the operations runbook. The regulator's pushback: "All of those are organisational controls. We're asking about technical evidence."

For a hash-chained signed log that's been anchored to Bitcoin every 15 minutes: the company can show that the entry's recordedAtIso field is 2026-04-15, the entry's hash matches the SHA-256 of its canonical form, the chain that includes this entry has an OpenTimestamps proof tying it to Bitcoin block height X mined on 2026-04-15, and Bitcoin is not retroactively rewritable. There's no organisational control to argue about; the math is the argument.

Three properties an audit log needs

For regulator-grade evidence, the log has to be:

• Tamper-evident. Editing any entry must leave a detectable trace. Editing the storage layer (the database file, the S3 object, the index) must not allow undetected entry-level modification.
• Time-bound. The chain head (or equivalent commitment) must be published externally at regular cadence, so that "this entry existed before time T" is independently verifiable.
• Independently verifiable. The audit procedure must be runnable on infrastructure the regulator controls, with no dependency on the operator's systems at verification time.

Standard application logging stacks rarely satisfy any of the three by default. They can be augmented to satisfy them — but the augmentations are precisely what makes the difference between an application log and an audit log.

How standard stacks score against each

Datadog, Splunk, Loki, similar SIEMs

Tamper-evident: No, by default. Entries are stored in indexes that admins with appropriate roles can modify. Splunk Enterprise Security has a Cryptographic Auditing add-on that signs index buckets; useful but bucket-level, not entry-level.

Time-bound: No external commitment. Datadog timestamps are wall-clock from the agent, which the agent's host clock determines.

Independently verifiable: No. The auditor depends on the company's Datadog access + the company's signing infrastructure (if any).

Verdict: excellent for SRE and security telemetry; not an audit log for regulated AI decisioning.

AWS CloudTrail / GCP Cloud Audit Logs / Azure Monitor

Tamper-evident: Yes, at the API-call level. CloudTrail file-integrity validation hashes hourly log files + chains those hashes. GCP and Azure have similar.

Time-bound: Partial. The cloud provider publishes the hash digest; this is a trusted-third-party anchor where the cloud provider is the third party. Stronger than nothing; weaker than a public-blockchain anchor.

Independently verifiable: Yes, the verification utility is open and runnable.

Verdict: solid for what they cover — infrastructure-level API calls. Does not cover application-level audit events (the actual AI decision content). Complements an L12 chain.

Object-lock S3 / write-once storage

Tamper-evident: Custody-level only. Object-lock prevents deletion or modification of files after their write time. Doesn't prevent the file content from being misleading at write time.

Time-bound: Partial — AWS publishes the object's creation time, but you're trusting AWS's clock.

Independently verifiable: Partial. Anyone with read access can confirm the file exists and is unmodified since its lock; can't confirm it was created when it claims to be.

Verdict: a useful complement to a cryptographic chain (the chain on object-locked storage is harder to delete than the chain on regular storage), not a replacement.

L12-style cryptographic chain

Tamper-evident: Yes. Every entry's hash field commits to its content; the next entry's prevHash commits to the previous entry's hash. Editing any entry requires re-signing every subsequent entry — which requires the operator's private Ed25519 key. Without the key, the forgery is detectable cryptographically.

Time-bound: Yes, when anchored. The chain head is published to OpenTimestamps periodically; the OTS proof ties the head to a Bitcoin block. To forge a chain past the anchor, the operator would have to rewrite Bitcoin's history past the anchored block — practically impossible.

Independently verifiable: Yes. The verifier is ~500 lines of Apache-2.0 Node code. Anyone can run it; anyone can audit the source; anyone can fork it into their own audit environment.

Anatomy of an L12 entry

{
  "id": 84291,
  "recordedAtMs": 1747300231000,
  "recordedAtIso": "2026-05-15T14:32:11.000Z",
  "productId": "acme-fintech",
  "specId": "fraud-classifier-v3",
  "specHash": "5b05721a...",         // which spec was active
  "inputHash": "8d277be5...",        // SHA-256 of input
  "outputCanonicalHash": "6a3103c3...",
  "category": "classifier",
  "modelIdentity": {
    "provider": "anthropic",
    "model": "claude-sonnet-4-5",
    "version": "2026-05-01"
  },
  "latencyMs": 162,
  "schemaValid": true,
  "reviewerVerdict": null,
  "fallbackTriggered": false,
  "prevHash": "9c34c863...",          // chain linkage
  "hash": "f8a91c2b...",              // SHA-256 of canonical form
  "attestation": {
    "algorithm": "ed25519",
    "publicKey": "...",
    "signature": "...",               // signed-this-entry proof
    "signedAtIso": "2026-05-15T14:32:11.001Z",
    "canonicalForm": "v1"
  }
}

Six properties of this structure:

• Bounded. Every field is structurally typed; the closed-enum category and the declared outputSchema mean the entry shape is predictable.
• PII-safe. inputHash and outputCanonicalHash are SHA-256; raw input never enters the chain. GDPR Article 5(1)(c) data minimisation works with the chain rather than against it.
• Chained. prevHash commits to the previous entry. Editing any entry breaks every subsequent prevHash.
• Signed. The Ed25519 signature is over the entry's canonical form. Forgery requires the operator's private key.
• Reproducible. The canonical-form serialization is deterministic across implementations (the substrate ships parallel JS and Python emitters that produce byte-identical canonical forms).
• Verifiable. The verifier needs no additional information beyond the chain itself plus an optional --trusted-key.

When to use which

Practical posture for an AI-decisioning system in a regulated EU product:

• SRE telemetry: keep using Datadog / Splunk / Loki. Latency, errors, traces, alerting — none of this needs cryptographic integrity. The operational tooling is correct for the operational concerns.
• Cloud infrastructure: enable CloudTrail (or GCP / Azure equivalent). Cover the "who deployed what" questions with the cloud provider's existing audit stream.
• Per-AI-decision audit log: cryptographic chain (Promethean's L12 or equivalent). Each decision gets a signed, hash-chained, externally-anchored receipt. This is the layer that satisfies Article 12 AI Act, GDPR Article 22 evidence, proposed PSR Arts. 85–89 reproducibility, HIPAA §164.312(b) audit controls.

What it costs

Cryptographic chains are cheap. Ed25519 signature generation is sub-millisecond on modern hardware; verification is roughly the same speed. The on-disk size per entry is a few hundred bytes including the signature. For a system doing 10,000 decisions per month, the chain size is a few MB per month.

The operational cost is in two places: (a) key custody — the signing key needs reliable storage, ideally HSM-backed in production; (b) anchoring — submitting the chain head to OpenTimestamps periodically costs nothing in fees but requires a working internet connection at the anchor cadence.

For Promethean operators, both are handled by the hosted substrate: per-workspace keys generated at signup, operator-managed; anchoring cadence determined by tier (daily on Team, hourly on Production, 30-min on Scale, 15-min on Enterprise). The Dev tier is self-managed for both.

Summary

• Application logging tools answer different questions than audit logs. Keep both.
• An audit log must be tamper-evident, time-bound, and independently verifiable. Most stacks satisfy zero of the three by default.
• A hash chain with per-entry Ed25519 signatures and periodic external anchoring satisfies all three with mathematical rather than organisational guarantees.
• The cost is small. The evidentiary upgrade is large. A regulator's pointed question about backdating becomes a one-command answer.

Promethean's L12 chain is one implementation of this pattern, specifically targeted at LLM-in-the-loop features in regulated products. See the quickstart for an integration walkthrough; the SDK is BSL-1.1 source-available and the verifier is Apache-2.0.

Sector-specific guidance