Recorder Docs

Proof recording and tamper-evident evidence documentation.

Other product docs

Recorder Threat Model

Overview

Recorder is the deterministic evidence data plane for governance systems. It ingests unsealed boundary events, seals them into hash-chained envelopes, persists them in append-only segments, and exports verifiable bundles for offline checks.

Recorder is composed of:

  • Core contracts (recorder-core): identity, hash/signature abstractions, filter/cursor types, error algebra.
  • Envelope model (recorder-envelope): canonical encoding, envelope/segment/bundle types, verification result algebra.
  • Decision Gate integration adapter (recorder-decision-gate-adapter): deterministic MCP/runpack transcript mapping, runpack integrity policy, and transcript redaction/bounds policy enforcement.
  • OpenTelemetry integration adapter (recorder-otel-adapter): deterministic OTel JSON export mapping, strict record validation, and redaction/bounds policy enforcement.
  • Recorder runtime (recorder): ingest validation, segment lifecycle, bundle builder, offline verifier, provider/adapter implementations.
  • Rust SDK facade (recorder-sdk): deterministic builder and unified in-process facade over recorder, adapter, provider, verifier, and store handles.
  • Python sidecar SDK (recorder-client): contract-driven sync/async sidecar HTTP client with strict response/error typing, retry-safe idempotency behavior, and generated-contract drift checks.
  • Storage backends (recorder-store): SQLite and in-memory implementations of envelope, attachment, and segment stores.
  • Sidecar config authority (recorder-sidecar-config): canonical sidecar config model, fail-closed validation, and deterministic config projection artifacts (schema/compat/example/docs) consumed by contract generation and CI gates.
  • Sidecar service surface (recorder-sidecar): HTTP/JSON runtime exposing recorder/query/bundle/config workflows over Unix/TCP with auth/bounds/ idempotency controls, strict media-type policy enforcement, and first-party OTel JSON ingest compatibility route.
  • CLI surface (recorder-cli): local operator commands for record/query/bundle/ config/doctor workflows plus Decision Gate and OTel fixture ingest.

Recorder now supports both local-process workflows and a built-in sidecar HTTP service boundary. External wrappers (for example MCP bridges or API gateways) remain in-scope as hostile boundaries.

Security Goals

  • Deterministic, reproducible envelope and bundle integrity outcomes.
  • Tamper-evident append-only history with chain continuity enforcement.
  • Fail-closed behavior on malformed, missing, ambiguous, or unverifiable data.
  • Offline-verifiable bundle portability without trusted live dependencies.
  • Clear trust boundaries between adapters, recorder runtime, stores, and consumers.
  • Bounded input/resource exposure at operator entry points.
  • Strong adversary posture: designs must remain defensible against compromised insiders and nation-state-capable operators.

Non-Goals / Out of Scope

  • Confidentiality guarantees after evidence is disclosed to downstream systems.
  • Defense against full host/kernel compromise without external platform controls.
  • Key custody/HSM/remote attestation infrastructure (beyond current local key usage hooks).
  • Built-in transport security for remote deployment wrappers (mTLS/authn/authz are deployment responsibilities today).
  • Non-repudiation signing of bundle archives at rest (external signing/WORM is required).

Assets

  • Unsealed and sealed envelope data (event type, actor, environment, payload, time, correlation IDs).
  • Envelope integrity material (content_hash, chain_hash, sequence position, segment linkage).
  • Segment metadata (SegmentGenesis, predecessor linkage, seal records).
  • Attachment bytes and content hashes.
  • Bundle contents (segments, envelopes, attachments, proof anchors).
  • Verification manifests, verdict artifacts, and verifier transcript artifacts.
  • Recorder runtime configuration (recorder_id, schema version, auto-seal, startup verification depth).
  • SQLite store file and in-memory state.
  • Signing key material and derived key IDs when signers are configured.
  • Persisted signer-rotation state (store_meta keys for current key, epoch, and last rotation event marker).
  • Generated contract artifacts under Docs/generated/recorder/** and manifest authority Docs/generated/recorder/index.json, including sidecar API artifacts (Docs/generated/recorder/apis/sidecar.*), sidecar config artifacts (Docs/generated/recorder/config/sidecar.*), and SDK projection artifacts (Docs/generated/recorder/sdk/*).
  • Performance policy artifacts (target/perf-matrix/latest.json, target/perf/v2/*.json, target/perf/triage/latest.md, system-tests/perf_baselines/*.json) and runner fingerprint metadata.
  • CLI artifacts (selector JSON, payload JSON, attachment file/inline payloads, Decision Gate fixture JSON, OTel fixture JSON, config JSON, bundle JSON, verifier transcript JSON, and assurance check JSON outputs).
  • Container deployment artifacts and runtime mounts (Dockerfile image outputs, compose/env definitions, mounted sidecar config/data/run paths, Docker secret token source paths, and tmpfs token target paths).

Adversary Model

  • Nation-state adversaries with full code visibility and long-term persistence.
  • Untrusted adapters/clients submitting malicious envelopes or attachments.
  • Compromised insiders with write access to config, binaries, or storage.
  • Attackers tampering with on-disk SQLite data, bundle files, or exported artifacts.
  • Attackers attempting replay, chain forking, payload mutation, or attachment substitution.
  • Attackers inducing resource exhaustion via oversized or expensive inputs.
  • Malicious integrators exposing Recorder via remote transports without sufficient access controls.

Trust Boundaries

  • Adapter ingest boundary: UnsealedEnvelope and attachment data are untrusted.
  • CLI/operator boundary: all command-line values and JSON files are untrusted.
  • Runtime config boundary: config file contents and store path are untrusted.
  • Sidecar HTTP boundary: request line, headers, auth token, JSON body, and transport metadata are untrusted.
  • Sidecar enterprise bridge boundary: enterprise control-plane admission and hook calls are untrusted network dependencies and must fail closed.
  • Storage boundary: persisted rows/blobs are untrusted on read.
  • Bundle boundary: imported bundle JSON is untrusted before verification.
  • Evidence provider boundary: consumer calls to list/fetch/verify/query are untrusted requests.
  • Rust SDK facade boundary: builder/runtime inputs (recorder_id, adapter metadata, storage path, trust root, signer/trust policy) are untrusted and must preserve recorder/provider fail-closed behavior.
  • Python SDK boundary: sidecar endpoint/base URL, auth token, idempotency headers, retry/backoff policy inputs, request models, query parameters, and HTTP response payloads are untrusted and must fail closed on contract drift, unknown error codes, or schema mismatch.
  • Decision Gate adapter boundary: Decision Gate transcript/runpack fixture payloads are untrusted and must pass mapping/integrity checks before ingest.
  • OTel adapter boundary: OpenTelemetry JSON export payloads are untrusted and must pass strict record-shape, identifier, timestamp, and bounded/redacted mapping checks before ingest.
  • Signing boundary: signer/verifier integrations and key management are external trust dependencies.
  • Contract generation boundary: generated projection artifacts are untrusted until hash-verified against canonical index manifest.
  • Performance harness boundary: runner-class labels, workload knobs, and perf artifacts are untrusted until lane identity (regression/capacity), workload metadata, and pinned-runner policy checks are validated.
  • External wrapper boundary: any network-facing bridge is outside this repo and must enforce authn/authz and transport protections.
  • Container-orchestration boundary: compose/env/runtime mount definitions, Docker secret source paths, tmpfs token-target paths, and host filesystem ownership/permission models are untrusted deployment inputs and must satisfy sidecar token/config hardening constraints.

Entry Points and Attack Surfaces

  • CLI commands: recorder segment open|seal|status|list, recorder signer rotate, recorder envelope record|record-with-attachments, query envelopes, bundle build|verify|inspect, assurance startup|trust-root|gates, contract generate|check, decision-gate ingest-fixture, otel ingest-fixture, config validate, doctor.
  • Sidecar commands: recorder-sidecar --config <path>, recorder-sidecar healthcheck --url <url>, recorder-sidecar healthcheck --unix-socket <path> --path <path>.
  • Sidecar HTTP routes: GET /health, GET /startup, GET /ready, GET /v1/segments, GET /v1/segments/active, POST /v1/envelopes, POST /v1/envelopes/with-attachments, POST /v1/query/envelopes, POST /v1/bundles/build|verify|inspect, POST /v1/ingest/otel, GET /v1/config.
  • File inputs: runtime config JSON/TOML, payload JSON/file, attachment file/inline base64, Decision Gate fixture JSON, selector JSON, bundle JSON.
  • Container deployment inputs: docker/sidecar/Dockerfile, docker/sidecar/docker-compose.yml, docker/sidecar/.env, mounted sidecar.toml, Docker secret token source file, tmpfs token target directory, and bind-mounted data/run directories.
  • Library APIs: RecorderBuilder::build, Recorder::record|record_with_attachments, Recorder::build_bundle|verify_bundle, Recorder::list_bundles|fetch_bundle|query_envelopes, RecorderAdapter::emit_envelope, RecorderAdapter::emit_with_attachments, DecisionGateAdapter::ingest_fixture, RecorderEngine::open_segment|record_envelope|seal_segment, EvidenceProvider::list_bundles|fetch_bundle|verify_bundle|query_envelopes.
  • Storage entry points: SQLite DB creation/open, SQL row materialization, attachment fetch, genesis decode.
  • Bundle verification pipeline: manifest validation, attachment hash checks, content/chain checks, cross-segment checks, signature phase.

Security Controls and Invariants

  • Canonical JCS encoding for deterministic hash inputs (serde_jcs).
  • Envelope content hash and chain hash formulas are explicit and stable.
  • Segment genesis anchors first-envelope chain computation.
  • Hash equality uses constant-time comparison in core/verifier/store paths.
  • Adapter/validation layer enforces fail-closed checks before recording.
  • Envelope builder enforces required fields, UTC times, deterministic attachment ref order.
  • Envelope attachment and segment-genesis constructors reject empty, blank, and control-character-bearing content_type and recorder_id inputs.
  • Recorder config validation fail-closes malformed recorder_id values at engine startup (ActorId-compatible identity shape required).
  • Core identity constructors reject blank/whitespace-only and control-character string inputs; key_id is enforced as exact lowercase hex(SHA-256(public_key_bytes)) shape (64 chars).
  • Core identity constructors enforce explicit max lengths: trace_id/session_id/actor_id/environment_id <= 256 chars, event_type/adapter_id <= 128 chars.
  • String-backed identity serde paths are constructor-validated; hostile JSON cannot bypass identifier invariants at deserialize boundaries.
  • Store append enforces open-segment and chain-continuity invariants.
  • Store segment creation enforces single-open-segment and duplicate-segment-ID fail-closed checks.
  • Store active-segment lookup fail-closes on corrupted persisted state with multiple open segments instead of selecting an arbitrary row.
  • Store query/existence paths fail closed on storage-layer query errors instead of defaulting to permissive existence outcomes.
  • SQLite append path is transaction-scoped for atomic metadata/data updates.
  • SQLite enables WAL and foreign keys during schema initialization.
  • Bundle builder uses deterministic ordering (BTreeMap/BTreeSet) and tracks omitted parents/proof anchors.
  • Verifier executes 7 phases with trust-root-aware signature verification; it marks signature-skip only when no trust root is configured.
  • Verifier phase 1 fail-closes on manifest structural tampering (segment entry mismatches and attachment-manifest mismatches), not only version/algorithm mismatches.
  • Verifier enforces bounded-work policy before phase execution: max segments 4096, max envelopes per segment 100000, max envelopes total 200000, max attachments total 200000, max attachment refs per envelope 4096.
  • Recorder startup verification re-checks active segment read-back integrity for configured depth windows (startup_verification_depth).
  • Evidence provider enforces stable bundle IDs and true fetch-by-ID semantics for sealed segments.
  • Evidence provider enforces bounded query/materialization policy: max list_bundles and query_envelopes limit 10000, max per-segment envelope materialization 200000.
  • CLI enforces bounded file-read limits: config 1 MiB, payload 4 MiB, selector 1 MiB, bundle 32 MiB, trust root 1 MiB, attachment file/inline payload 16 MiB, Decision Gate fixture 8 MiB, signer key 64 KiB, signer passphrase material 1 KiB.
  • CLI query enforces bounded page policy: default page size 1000, max page size 10000, and deterministic cursor-based continuation contract in JSON mode.
  • CLI i18n catalogs are build-validated fail closed: locale tag uniqueness, required en default locale, mandatory locale provenance (machine or human_reviewed), exact message-key parity against English, and exact placeholder-set parity for every key across enabled locales.
  • CLI localization rendering is strict and fail-closed: generated typed key/macro access prevents unknown message keys at compile time, and runtime rendering rejects placeholder-set mismatch and unresolved placeholders without silent fallback to key IDs or English.
  • CLI locale provenance policy is explicit: machine-provenance locales emit deterministic operator notice, while JSON output payloads remain locale-neutral across locale selection.
  • CLI runtime resolution validates recorder_id using ActorId constructor semantics before command execution, including config validate parity with recorder config validation behavior.
  • CLI signer loading is profile-aware (secure default) and fail-closed on encrypted-key requirements, malformed key material, insecure opt-down misuse, and key-state mismatch in secure mode.
  • Explicit key-rotation workflow emits deterministic lifecycle events (system.adapter.key_rotated / break-glass variant) and persists idempotence markers in SQLite metadata.
  • Contract generation emits deterministic projection artifacts with manifest-hash authority and strict drift-check mode.
  • Contract generation/check fail closed on unsafe output-root filesystem entries, including symlinked roots, hard-linked artifact files, or symlink traversal beneath the output tree.
  • Contract schema generation applies explicit runtime constraint overlays for identifier/text/hash/signature fields (including full BundleSelector variant algebra) and fails closed if expected schema pointer paths are missing.
  • Contract generation now emits sidecar API artifacts (sidecar.openapi/errors/enums/examples/compat) and generated manifest authority includes those paths.
  • Contract generation now emits SDK projection artifacts (sdk/types.json, sdk/methods.json, sdk/test_vectors.json, sdk/manifest.json) and drift-check mode fail-closes on SDK artifact mismatch.
  • Rust SDK builder/facade wiring fail-closes invalid identity/config/store inputs while preserving recorder/provider determinism and bounded-work limits.
  • Python SDK request/response wiring fail-closes invalid input/query parameter values, unknown problem codes, unexpected content types, and response schema drift against generated SDK projection artifacts while preserving sidecar idempotency semantics across retry attempts.
  • Sidecar compatibility gates fail closed in CI by validating current pre-release-break /v1 policy, enum-wire stability, error-code registry stability, and mutating-endpoint idempotency contract enforcement from generated artifacts.
  • Sidecar perf policy gates split regression (sidecar_perf) and capacity (sidecar_capacity) lanes with explicit lane identity/workload metadata in artifacts to prevent fixed-workload regression signals from being misread as max-throughput results.
  • Pinned-runner capacity enforcement fails closed when required sidecar_capacity artifacts are missing, and capacity floors (max_sustainable_rps) are anti-sandbagging protected by threshold policy checks.
  • Sidecar config validation fail-closes invalid protocol/security settings (schema/API version, transport host/port, non-loopback token requirement, signer key parseability, hard header-size cap, and numeric limit bounds) before binding listeners.
  • Sidecar config file and secret-material loading enforce bounded hostile-input file sizes and fail closed on oversize input: config <= 1 MiB, token/control-plane-token files <= 4096 bytes, signer key file <= 65536 bytes.
  • Sidecar token/control-plane-token file opening uses fail-closed descriptor-bound semantics:
    • Unix: no-follow open (O_NOFOLLOW) plus regular-file and mode checks.
    • Windows: reparse-point-safe open (FILE_FLAG_OPEN_REPARSE_POINT) plus opened-handle reparse-point rejection.
    • Other targets: fail-closed unsupported-platform rejection for secret-token file loading.
  • Sidecar middleware stack enforces server/client correlation ID handling (x-server-correlation-id, x-correlation-id, legacy x-request-id), strict fail-closed media-type parsing (exact application/json with optional charset=utf-8) and strict JSON-only Accept parsing, aggregate header byte bounds, declared content-length bounds, bounded queue admission and in-flight concurrency, and per-request timeout fail-closed behavior.
  • Sidecar request media-type policy is centralized in recorder-sidecar-config and projected into generated sidecar API compatibility artifacts so runtime and contract media-type allowlists cannot drift.
  • Sidecar token auth compares bearer tokens in constant time and bypasses auth only for explicit probe routes (/health, /startup, /ready) or open mode.
  • Sidecar probe/readiness controls are explicit and fail closed: /health remains lightweight process liveness, /startup reports startup completion state, and /ready can enforce storage-only or storage+enterprise dependency readiness (probes.readiness_mode) with optional capacity-exhaustion fail-closed behavior (probes.ready_fail_on_admission_saturation).
  • Sidecar bounds middleware explicitly bypasses probe routes so orchestrator liveness/startup/readiness signals are observable even under admission saturation and active-request exhaustion.
  • Sidecar auth failures emit RFC 6750 WWW-Authenticate challenge metadata and sidecar errors use RFC 9457 Problem Details (application/problem+json).
  • Sidecar container packaging defaults to distroless nonroot runtime and compose profiles that require explicit UID/GID mapping, Docker secret token source inputs, and bootstrap materialization of hardened token files into tmpfs runtime paths before sidecar startup.
  • Sidecar mutating operations enforce persistent idempotency replay/conflict semantics with stream-scoped persistent retention (idempotency_entries) and deterministic request-hash scoping.
  • Sidecar mutating envelope operations enforce explicit stream identity inputs (tenant_id, recorder_id) and reject recorder mismatches fail-closed.
  • SQLite operation dispatch uses bounded queueing (sync_channel) with deterministic fail-closed rejection when queue capacity (4096) is saturated.
  • Verifier transcript emission is deterministic and schema-versioned; bundle verification can export transcript JSON for machine-auditable evidence packets.
  • CLI assurance checks provide explicit fail-closed deployment assertions for startup-verification depth, trust-root mode parsing, and contract gate execution.
  • System tests validate tamper detection and fail-closed oversized-input behavior.
  • System tests validate external adapter-ingest coupling using OpenClaw mock event streams, including signed/unsigned verification and sequence-gap strict fail-closed vs anomaly-recording policies.
  • OpenClaw integration mapping applies deterministic sensitive-field redaction and bounded payload handling (string and JSON size limits) before immutable evidence recording.
  • System tests validate external control-plane coupling using Decision Gate MCP runpack flow fixtures, including signed/unsigned verification and runpack integrity strict fail-closed vs anomaly-recording policies, including manifest self-integrity recomputation from declared file hashes.
  • System tests validate sidecar subprocess lifecycle over real HTTP and fail-closed idempotency replay/conflict persistence across restart boundaries.
  • Decision Gate integration mapping applies deterministic transcript-field redaction and bounded payload handling (string and JSON size limits) before immutable evidence recording.
  • Production recorder-decision-gate-adapter enforces mapping-contract version checks, deterministic event IDs/timestamps, transcript redaction, bounded payload metadata, and strict-vs-anomaly runpack integrity handling.
  • Production recorder-otel-adapter enforces deterministic OTel span/log mapping (otel.span, otel.log.<severity>), strict identifier/timestamp parsing, deterministic ID derivation, and bounded/redacted payload projection.

Known current security-relevant gaps:

  • Sidecar does not include built-in TLS/mTLS; non-loopback deployments should terminate TLS and enforce client identity in a hardened front proxy/gateway.
  • Sidecar provides bounded local admission controls but not distributed/global rate limiting; deployment platforms still need caller quotas and abuse controls.

Threats and Mitigations

Authentication, Authorization, and Access Control

  • Threat: unauthorized local or remote invocation of recording/query/bundle paths.
  • Mitigation: sidecar supports explicit token mode with constant-time bearer token checks, RFC 6750 WWW-Authenticate challenge responses, and fail-closed auth misconfiguration handling; CLI remains local process invocation.
  • Residual risk: open sidecar mode is deployment-sensitive and should only be used in trusted local IPC environments; non-loopback network deployments still require gateway-layer identity and transport controls.

Input Validation and Parsing

  • Threat: malformed IDs, non-UTC times, invalid event types, malformed JSON, or ambiguous payloads.
  • Mitigation: typed ID constructors, UTC enforcement, strict event-type validation, strict key-id hex shape validation, fail-closed envelope validation, strict attachment/segment text-field validation (content_type, recorder_id), strict recorder config recorder_id validation, typed JSON parsing.
  • Mitigation: bounded file reads prevent oversized JSON ingestion at CLI boundary, including trust-root and signer-key inputs.

Evidence Integrity and Authenticity

  • Threat: payload mutation, chain forking, attachment substitution, or replayed envelopes.
  • Mitigation: canonical-hash content addressing + chain continuity checks at append and verify time.
  • Mitigation: verifier phase 1 enforces structural manifest integrity against bundle segment and attachment materialization.
  • Mitigation: strict Ed25519 verification and trust-root policy enforcement are wired in verifier phase 6.
  • Mitigation: secure-profile signer loading requires encrypted PKCS#8 key material by default and explicit opt-down for less secure dev ergonomics.
  • Mitigation: explicit key-rotation events and persisted idempotence markers preserve signing lineage continuity and crash-recovery reconciliation.
  • Residual risk: authenticity remains configuration-dependent when operators choose to verify without a trust root (explicit warning emitted).

Disclosure and Data Exposure

  • Threat: accidental over-disclosure through bundle exports and query results.
  • Mitigation: explicit operator-driven export/query commands and typed filters.
  • Residual risk: no built-in field-level redaction policy in Recorder core; wrapper layers must enforce disclosure policy.

Storage and Bundle Integrity

  • Threat: DB/file tampering, corruption, or partial writes.
  • Mitigation: SQLite WAL, transactional append, chain validation, corruption detection when parsing invalid stored rows.
  • Mitigation: bundle verification recomputes attachment/content/chain links and flags integrity failures.
  • Residual risk: bundle archives are not signed by default; use external signature and immutable storage controls for non-repudiation.

Availability and Resource Exhaustion

  • Threat: oversized config/payload/selector/bundle inputs and expensive verification workloads.
  • Mitigation: CLI hard limits on file reads; fail-closed rejection before parse.
  • Mitigation: typed errors and early rejection on invalid selectors/filters.
  • Mitigation: recorder verifier/provider APIs enforce bounded-work limits and fail closed on oversized in-memory workloads.
  • Mitigation: sidecar middleware enforces bounded header/body/content-length limits, bounded queue + active concurrency, and timeout fail-closed behavior.
  • Mitigation: SQLite operation dispatch is bounded; enqueue saturation fails closed instead of allowing unbounded memory growth.
  • Residual risk: no distributed/global rate limiter exists for sidecar or wrapper-exposed deployments; operators still need caller quotas and abuse controls.

Supply Chain and Execution Environment

  • Threat: compromised binaries/config/runtime host or malicious dependency behavior.
  • Mitigation: strict lint/no-panic posture, typed errors, deterministic contracts, and explicit architecture boundaries.
  • Residual risk: host compromise and dependency compromise require external controls (SBOM/audit/signing/hardening).

Multi-Tenant and Isolation

  • Threat: cross-tenant data exposure in shared deployments.
  • Mitigation: Recorder IDs are labels; no implicit tenant isolation claims.
  • Residual risk: shared deployments must enforce tenant-aware storage, encryption, and access control at wrapper/deployment layers.

Auditability and Observability

  • Threat: inability to prove what happened after suspected tampering.
  • Mitigation: append-only segment model, deterministic hashes, bundle manifests, offline verifier, and artifact-first system tests.
  • Residual risk: comprehensive security audit logging for remote auth decisions is wrapper-dependent.

Implementation References (Controls and Protections)

Core and Envelope Integrity

  • Canonical encoding and hash formulas: crates/recorder-envelope/src/encoding.rs.
  • Hash types, algorithm registry, constant-time equality: crates/recorder-core/src/hash.rs.
  • Identity and input-shape validation contracts: crates/recorder-core/src/identity.rs, crates/recorder-core/src/filter.rs.
  • Signature contracts and key identity model: crates/recorder-core/src/signature.rs, crates/recorder-envelope/src/signer.rs, crates/recorder-envelope/src/signature.rs.
  • Envelope/segment invariants: crates/recorder-envelope/src/envelope.rs, crates/recorder-envelope/src/segment.rs, crates/recorder-envelope/src/attachment.rs.

Recorder Runtime and Verification

  • Recorder lifecycle, chain progression, seal behavior: crates/recorder/src/engine.rs.
  • Auto-seal policy validation: crates/recorder/src/config.rs.
  • Adapter boundary validation: crates/recorder/src/validation.rs, crates/recorder/src/local_adapter.rs.
  • Deterministic bundle assembly and proof anchors: crates/recorder/src/bundle_builder.rs.
  • 7-phase verification pipeline and warning model: crates/recorder/src/verifier.rs.
  • Evidence provider boundary behavior: crates/recorder/src/evidence_provider.rs.

SDK Facade Surface

  • Rust SDK facade composition and deterministic builder defaults: crates/recorder-sdk/src/builder.rs, crates/recorder-sdk/src/lib.rs.
  • SDK prelude and stable consumer import boundary: crates/recorder-sdk/src/prelude.rs.
  • SDK facade conformance tests: crates/recorder-sdk/tests/test_builder.rs, crates/recorder-sdk/tests/test_facade.rs.
  • Python SDK contract projection generator and typed client surface: sdk/python/scripts/generate_contract.py, sdk/python/src/recorder_client/generated_contract.py, sdk/python/src/recorder_client/client.py, sdk/python/src/recorder_client/client_common.py, sdk/python/src/recorder_client/sync_client.py, sdk/python/src/recorder_client/async_client.py, sdk/python/src/recorder_client/models.py, sdk/python/src/recorder_client/errors.py.
  • Python SDK unit/system conformance tests: sdk/python/tests/test_contract_generated.py, sdk/python/tests/test_models.py, sdk/python/tests/test_errors.py, sdk/python/tests/test_client_unit.py, sdk/python/tests/test_client_system.py.

Integration Adapters

  • Decision Gate adapter ingestion, deterministic mapping, and fail-closed integrity-policy enforcement: crates/recorder-decision-gate-adapter/src/adapter.rs.
  • Decision Gate mapping and redaction/bounds policy defaults: crates/recorder-decision-gate-adapter/src/policy.rs.
  • Decision Gate fixture input contract: crates/recorder-decision-gate-adapter/src/model.rs.
  • Decision Gate adapter typed error algebra: crates/recorder-decision-gate-adapter/src/error.rs.
  • OTel adapter ingestion, deterministic mapping, and strict-fail policy: crates/recorder-otel-adapter/src/adapter.rs.
  • OTel mapping and redaction/bounds policy defaults: crates/recorder-otel-adapter/src/policy.rs.
  • OTel JSON export input contract and adapter error algebra: crates/recorder-otel-adapter/src/model.rs, crates/recorder-otel-adapter/src/error.rs.

Storage

  • Storage contracts and invariants: crates/recorder-store/src/traits.rs, crates/recorder-store/src/error.rs.
  • In-memory backend behavior: crates/recorder-store/src/memory.rs.
  • SQLite initialization, WAL, FK, connection thread model: crates/recorder-store/src/sqlite/schema.rs, crates/recorder-store/src/sqlite/connection.rs, crates/recorder-store/src/sqlite/mod.rs.
  • SQLite envelope/segment/attachment operations: crates/recorder-store/src/sqlite/envelope_ops.rs, crates/recorder-store/src/sqlite/segment_ops.rs, crates/recorder-store/src/sqlite/attachment_ops.rs.

CLI and Validation Surface

  • Command routing and runtime resolution: crates/recorder-cli/src/main.rs.
  • Input hard limits and bounded file reads: crates/recorder-cli/src/main.rs.
  • Localization and deterministic output behavior: crates/recorder-cli/build.rs, crates/recorder-cli/i18n/locales/en.toml, crates/recorder-cli/i18n/locales/ca.toml, crates/recorder-cli/src/i18n.rs.
  • Contract artifact generation and drift checks: crates/recorder-contract/src/lib.rs, crates/recorder-contract/src/sidecar_api.rs, crates/recorder-contract/src/sidecar_api/openapi.rs, crates/recorder-contract/src/sidecar_api/artifacts.rs, crates/recorder-contract/src/sidecar_api/specs/mod.rs, scripts/ci/generate_all.sh.
  • Sidecar contract compatibility and perf policy gates: scripts/ci/sidecar_contract_gates.py, scripts/ci/perf_matrix.py, scripts/ci/perf_gate.py, scripts/ci/perf_threshold_policy.py, scripts/perf/run.py, scripts/perf/update_baseline.py.

Sidecar Service Boundary

  • Sidecar bootstrap/config and transport lifecycle: crates/recorder-sidecar-config/src/config.rs, crates/recorder-sidecar-config/src/validation.rs, crates/recorder-sidecar-config/src/validation/enterprise.rs, crates/recorder-sidecar/src/server.rs, crates/recorder-sidecar/src/shutdown.rs.
  • Sidecar auth and request-bound controls: crates/recorder-sidecar/src/middleware/auth.rs, crates/recorder-sidecar/src/middleware/bounds.rs, crates/recorder-sidecar/src/middleware/request_id.rs.
  • Sidecar single-writer ingest/idempotency and error contract: crates/recorder-sidecar/src/ingest.rs, crates/recorder-sidecar/src/idempotency.rs, crates/recorder-sidecar/src/error.rs, crates/recorder-sidecar/src/contract.rs.
  • Sidecar contract conformance integration tests: crates/recorder-sidecar/tests/error_registry_contract.rs, crates/recorder-sidecar/tests/idempotency_restart.rs.
  • Sidecar container packaging and operator profile: docker/sidecar/Dockerfile, docker/sidecar/docker-compose.yml, docker/sidecar/config/sidecar.toml, crates/recorder-sidecar/src/bin/docker_bootstrap.rs.

System Validation

  • Security/system tests: system-tests/tests/suites/security.rs, system-tests/tests/suites/bundle.rs, system-tests/tests/suites/persistence.rs, system-tests/tests/suites/recorder.rs, system-tests/tests/suites/sidecar.rs.
  • Docker packaging and compose-lifecycle validation: system-tests/tests/suites/sidecar_docker.rs, system-tests/tests/helpers/docker.rs.
  • Registry-driven validation contract: system-tests/AGENTS.md, system-tests/test_registry.toml.

Operational Requirements

  • Treat all deployments as hostile environments; assume compromised insiders and nation-state-capable adversaries.
  • Restrict filesystem access to store/config/bundle paths using least-privilege OS ACLs.
  • Use signed releases, dependency auditing, and host hardening for production nodes.
  • If exposing Recorder sidecar or wrapper APIs over non-loopback networks, enforce strong authn/authz, mTLS/TLS termination, and command allowlists.
  • For containerized sidecar deployments, run non-root, provide token source via Docker secret input, materialize token files into tmpfs with owner-only mode (0600 on Unix) before sidecar startup, and map host UID/GID explicitly so writable data/run paths remain deterministic.
  • Keep CLI input limits enabled; tighten wrapper-level limits where possible.
  • Do not treat unsigned bundles as non-repudiable artifacts; apply external signing and immutable retention controls.
  • Segment tenants into isolated stores/processes unless explicit shared-tenancy controls are implemented.
  • Monitor verifier warnings (especially signature-skip and partial-segment warnings) as policy-significant outcomes.

Failure Posture

  • Fail closed on invalid or ambiguous inputs.
  • Reject appends that violate segment state or chain continuity.
  • Reject malformed config/filter/time-range definitions.
  • Mark tampered bundles as Invalid or fail verification with typed errors on manifest/infrastructure mismatches.
  • Never assume trust in persisted state without integrity re-checks.

Threat Model Delta (2026-02-08, OSS Baseline + Docker)

  • Initial Recorder repository threat model baseline added under Docs/security/threat_model.md.
  • Mapped trust boundaries, controls, and residual risks to concrete current implementation files.
  • Closed signer/runtime wiring gap with profile-aware secure defaults, bounded signer input limits, and fail-closed key-material handling.
  • Added explicit signer-rotation workflow with deterministic lifecycle events and persisted idempotence metadata.
  • Added deterministic contract-generation boundary (recorder-contract) with manifest-hash authority and drift-check controls.
  • Hardened recorder-contract output-root handling to reject symlinked roots, hard-linked artifact files, and symlink traversal under generated artifact trees in both generate and check modes.
  • Added first-party sidecar container deployment boundary: Dockerfile/Compose/config artifacts are now explicit attack surfaces and deployment trust boundaries.
  • Added system-test Docker coverage for sidecar packaging hardening and Docker Compose lifecycle open/record/query validation.
  • Closed tracked P1 system-test security-relevant coverage gaps for selector dedup, proof-anchor semantics, cross-segment mismatch fail-closed behavior, attachment closure fail-closed behavior, and SQLite/in-memory parity.
  • Hardened recorder-core identifier validation to reject whitespace-only and control-character string identifiers and require strict 64-char lowercase hex key_id values.
  • Hardened deserialize boundaries so constrained identifier and envelope text fields cannot be instantiated from serde payloads without constructor checks.
  • Hardened recorder config and verifier boundaries: recorder startup rejects malformed recorder_id values during config validation, phase-1 verification fail-closes on manifest structural mismatches, and verifier now enforces explicit bounded-work ceilings.
  • Hardened recorder evidence-provider library boundaries with explicit fail-closed limits for list/query/materialization workloads.
  • Hardened recorder-store SQLite connection dispatch with bounded queue capacity and deterministic saturation rejection semantics.
  • Added system-test coverage for malformed identifiers, bounded CLI file input surfaces, secure signer-file policy, signer-rotation recovery/corruption behavior, contract CLI symlink safety, manifest structural tamper cases, SQLite corruption materialization failures, and single-open-segment CLI enforcement.
  • Closed all findings tracked in Docs/security/audits/OSS_launch_0.md.
  • Implemented recorder-sidecar as a built-in HTTP/JSON boundary with fail-closed config validation, request auth/bounds controls, bounded queue/concurrency controls, structured error contract, and persistent idempotency semantics; updated trust boundaries, attack surfaces, mitigations, and residual risks to reflect this runtime surface.
  • Implemented generated sidecar API artifact coverage (sidecar.openapi/errors/enums/examples/compat) with fail-closed CI compatibility gates enforcing current pre-release-break /v1, enum-wire stability, error-code registry stability, and mutating-endpoint idempotency contracts.
  • Consolidated sidecar runtime config semantics into a dedicated recorder-sidecar-config crate and wired sidecar runtime startup to consume its typed fail-closed config validation boundary.
  • Expanded generated contract scope to include sidecar config artifacts (config/sidecar.schema.json, config/sidecar.compat.json, config/sidecar.example.toml, config/sidecar.md) emitted by recorder-contract from recorder-sidecar-config projection helpers.
  • Extended sidecar compatibility gates to fail closed on sidecar config artifact drift and cross-artifact inconsistency (schema/compat/example/docs).
  • Added sidecar integration/system validation for generated error-registry compatibility and restart-boundary idempotency replay/conflict persistence.

Threat Model Delta (2026-02-08)

  • Added enterprise sidecar boundary mode with explicit fail-closed config validation (enterprise.mode, control-plane URL/token requirements, and token-mode security requirement).
  • Added enterprise identity extraction boundary (x-tenant-id, x-principal-id, x-authn-method, optional namespace) and fail-closed admission behavior on malformed/missing identity context.
  • Added pre-handler enterprise policy gate middleware that denies protected operations when control-plane authz/quota checks are unavailable or deny.
  • Added managed-cloud writer-lease assertion gating for mutating operations (x-recorder-writer-lease=true required when configured).
  • Added explicit sidecar enterprise boundary hook emission to control-plane (/v1/enterprise/sidecar/hooks) at admission and result phases with fail-closed behavior for successful protected operations when hook export fails.
  • Result: sidecar ingress now has an additional external dependency on enterprise control-plane availability/integrity; outage or invalid responses are handled as fail-closed admission denials and successful-hook export failures are treated as fail-closed request errors.

Threat Model Delta (2026-02-08, Probe and Readiness Hardening)

  • Added explicit startup probe route (GET /startup) to sidecar entry-point inventory and probe-auth bypass policy.
  • Added fail-closed readiness dependency modes (storage_only, storage_and_enterprise) with config validation and control-plane readiness dependency checks in enterprise-enabled modes.
  • Added optional readiness fail-closed behavior when request admission/execution capacity is exhausted (request_capacity_exhausted readiness reason).
  • Added explicit probe-priority middleware behavior: /health, /startup, and /ready bypass normal auth/policy/bounds request gates to preserve orchestration observability during overload.
  • Added Docker system-test probe coverage validating containerized /startup and /ready behavior before and after segment-open lifecycle transitions.

Threat Model Delta (2026-02-08, continued)

  • Renamed enterprise boundary adapter implementation from enterprise.rs to control_plane_bridge.rs to make the trust-boundary intent explicit (control-plane bridge only; no evidence-math authority).
  • Centralized sidecar enterprise ingress/header/hook vocabulary in crates/recorder-sidecar/src/contract.rs and wired middleware/bridge paths to those constants to reduce drift-induced policy bypass risk.
  • Added conformance tests to ensure sidecar operation labels, identity headers, and hook boundary/outcome labels remain synchronized with control-plane contract vocabulary.
  • Added cross-repo end-to-end enterprise lane coverage proving control-plane deny decisions fail closed at sidecar mutation boundaries while successful admitted paths preserve deterministic evidence verification outcomes.

Threat Model Delta (2026-02-08, CLI OSS Expansion)

  • Added CLI operator command recorder envelope record-with-attachments with fail-closed attachment boundary validation for file and inline base64 inputs.
  • Added CLI operator command decision-gate ingest-fixture that routes fixture inputs through production recorder-decision-gate-adapter integrity and mapping policies.
  • Added CLI runtime config parity checks so malformed recorder_id values fail closed during config validate using the same identity-shape constraints as recorder runtime.
  • Expanded CLI auto-seal boundary to include duration and combined policies, with fail-closed zero-value rejection for both count and duration thresholds.
  • Added CLI query page bounds (default 1000, max 10000) and deterministic JSON pagination contract (envelopes + next_cursor) to prevent unbounded materialization at operator boundary.

Threat Model Delta (2026-02-10, SDK Facade + SDK Artifacts)

  • Added first-party Rust SDK facade crate (recorder-sdk) that composes existing recorder/adapter/provider/store/verifier boundaries without introducing new protocol semantics.
  • Added SDK builder trust boundary inputs (storage path, recorder and adapter identifiers, signer/trust-root injection, startup policy knobs) and required fail-closed validation behavior inherited from recorder/provider layers.
  • Expanded generated-contract asset inventory and controls to include SDK projection artifacts (Docs/generated/recorder/sdk/types.json, Docs/generated/recorder/sdk/methods.json, Docs/generated/recorder/sdk/test_vectors.json, Docs/generated/recorder/sdk/manifest.json) with manifest-hash authority and drift-check enforcement.
  • Added SDK facade integration tests covering builder defaults, storage selection, record/build/verify flows, and provider query/fetch/verify parity.

Threat Model Delta (2026-02-14, Perf Lane Split and Pinned Capacity Gating)

  • Added explicit perf-harness trust boundary semantics: fixed-workload regression artifacts and saturation-capacity artifacts are distinct evidence classes with different operational decisions.
  • Added lane identity and workload metadata requirements to sidecar perf artifacts (lane, workload.*) so downstream gates can fail closed on missing/ambiguous perf context.
  • Added pinned-runner conditional gate requirement for capacity artifacts: when pinned release jobs are configured for capacity enforcement, missing sidecar_capacity reports are treated as a fail-closed condition.
  • Added anti-sandbagging policy coverage for capacity floors to prevent threshold downgrades of max_sustainable_rps without explicit governance.
  • Added resource-exhaustion saturation test notes: capacity sweeps are allowed to intentionally drive bounded 429 and elevated latency to identify sustainable operating region and knee point without relaxing fail-closed runtime behavior.

Threat Model Delta (2026-02-15, Perf Workflow Governance and Schema v2)

  • Added canonical perf orchestration boundary (scripts/perf/run.py) that normalizes matrix artifacts into schema-v2 per-suite reports with explicit runner fingerprint and lane identity metadata.
  • Added baseline-governance update surface (scripts/perf/update_baseline.py) requiring explicit sample-count/window/author/reason metadata on baseline refresh.
  • Added per-suite macrobench artifact paths (target/performance/stress/latest.json, target/performance/bundle/latest.json) to reduce ambiguity and prevent evidence overwrite confusion in performance triage.

Threat Model Delta (2026-02-11, Python SDK Surface)

  • Added first-party Python sidecar SDK (sdk/python) as a committed projection consumer of Docs/generated/recorder/sdk/{types,methods,test_vectors,manifest}.
  • Added Python SDK trust boundary for endpoint/token/idempotency/retry inputs and response decoding behavior with fail-closed validation posture.
  • Added deterministic Python contract-constants generator that derives generated_contract.py directly from canonical generated SDK artifacts.
  • Added strict Python client controls for unknown error-code rejection, content-type validation, operation-local error-contract validation, and stable idempotency key reuse across mutating-operation retries.
  • Added Python SDK unit and system tests and wired repository orchestration (scripts/ci/generate_all.sh) to run Python SDK generate/check/test gates.

Threat Model Delta (2026-02-11, OSS Assurance WS1-WS3)

  • Added deterministic verifier transcript artifact model (VerificationTranscript) and wired CLI export surface (bundle verify --verdict-transcript-output) for machine-verifiable verification packet evidence.
  • Added explicit OSS assurance command boundary (assurance startup, assurance trust-root, assurance gates) to provide fail-closed deployment assertions for startup verification depth, trust-root mode, and contract gate execution.
  • Added system-test coverage proving SQLite operation queue saturation is rejected fail closed with deterministic summary artifact output.

Threat Model Delta (2026-02-13, Docker Token Bootstrap Parity Hardening)

  • Eliminated host bind-mounted token-file dependence for sidecar Docker profile.
  • Added dedicated container bootstrap binary (recorder-sidecar-docker-bootstrap) that ingests token source material from Docker secret file/env input and writes runtime token file to configured target path with fail-closed validation and owner-only permissions.
  • Updated Docker Compose profile to use a Docker secret source and tmpfs token target mount, preserving explicit non-root UID/GID mapping for writable data and run directories.
  • Expanded system-test Docker suite hardening checks to enforce secret-source-to-tmpfs token bootstrap behavior.
  • Result: Windows host ACL variance no longer determines runtime token-file mode parity inside Linux containers because sidecar token file is now created in-container with deterministic permission semantics.

Threat Model Delta (2026-02-13, Phase 1 Strict Media-Type Boundary Hardening)

  • Replaced permissive sidecar media-type checks with strict fail-closed parser behavior for Content-Type and Accept at the HTTP boundary.
  • Sidecar request-body endpoints now enforce exact application/json with optional charset=utf-8; unknown parameters, malformed values, and non-UTF-8 charset values are rejected with stable unsupported_media_type.
  • Sidecar response negotiation now enforces strict JSON-only Accept ranges (application/json or */*) with malformed/non-JSON ranges rejected via stable not_acceptable.
  • Centralized route media-type policy in recorder-sidecar-config and projected that policy into generated sidecar OpenAPI/compat artifacts and CI contract gates, reducing runtime/contract drift risk on protocol boundary widening.

Threat Model Delta (2026-02-13, Phase 3 OTel Compatibility Surface)

  • Added production OTel compatibility adapter crate (crates/recorder-otel-adapter) and sidecar ingest route (POST /v1/ingest/otel) as a new hostile network boundary.
  • New OTel ingest boundary enforces strict JSON-only media policy, bearer auth, enterprise policy middleware, bounded admission/execution controls, timeout controls, and idempotency-key requirements consistent with other mutating sidecar routes.
  • OTel adapter mapping is deterministic and fail-closed: strict identifier/timestamp parsing, deterministic SHA-256-to-UUIDv7-shaped IDs, deterministic event-type mapping (otel.span, otel.log.<severity>), and bounded/redacted payload projection.
  • Added generated integration mapping contract artifacts for Decision Gate and OTel under Docs/generated/recorder/contracts/integrations/ with runtime/contract drift checks in generator test coverage.
  • Added CLI operator parity command recorder otel ingest-fixture, expanding operator input attack surface while preserving bounded fixture reads and adapter fail-closed behavior.

Threat Model Delta (2026-02-13, Enterprise Durable Control-Plane Dependency)

  • Expanded enterprise dependency threat posture to include durable control-plane state backends (Postgres-backed fleet/governance/cloud-ops) hosted in Asset-Core and consumed by sidecar enterprise admission paths.
  • Added explicit mutation-path outage fail-closed test coverage when control- plane dependency becomes unavailable: crates/recorder-sidecar/tests/enterprise_control_plane_e2e.rs and system-tests/tests/suites/sidecar_enterprise.rs.
  • Security effect: enterprise mode write paths deny on control-plane outage; no best-effort/fail-open mutation behavior is permitted.

Threat Model Delta (2026-02-14, Sidecar Single-Writer Ingest Refactor)

  • Replaced direct mutating envelope handler writes with a bounded ingest queue plus dedicated single-writer runtime (IngestGateway / IngestWriterRuntime) that owns durable commit and stream-position assignment.
  • Added stream-scoped identity and ordering boundary to sidecar mutating and query routes: tenant_id + recorder_id now define authoritative stream scope for ordering and idempotency.
  • Added persistent stream-scoped idempotency table (idempotency_entries) with deterministic request-hash replay/conflict semantics and bounded TTL/cap retention per stream.
  • Added writer/projection persistence surfaces in SQLite schema v2: streams, idempotency_entries, and query_projection_envelopes; startup now fails closed on non-v2 databases and requires a fresh DB for this pre-release break.
  • Removed mutating segment HTTP routes (POST /v1/segments, POST /v1/segments/seal) from sidecar runtime, reducing external mutation attack surface while keeping read-only segment observability routes.

Threat Model Delta (2026-02-14, Single-Writer Performance Completion)

  • Completed hot-path migration to writer-native sealing and single-transaction SQLite micro-batch commits; mutating sidecar ingest no longer depends on per-envelope RecorderEngine::record_envelope commits.
  • Added explicit SQLite durability/checkpoint tuning controls at sidecar config boundary (storage.sqlite_synchronous, storage.sqlite_wal_autocheckpoint_pages, storage.sqlite_busy_timeout_ms) with fail-closed validation and synchronous=FULL secure default.
  • Updated segment observability contract: GET /v1/segments/active now returns paginated active-segment lists with stream identity (tenant_id, recorder_id), reducing ambiguity in per-stream roll/debug workflows.

Threat Model Delta (2026-02-15, Sidecar Config File-Read Bounds Hardening)

  • Hardened recorder-sidecar-config hostile file-input boundaries to fail closed on oversized config/secret/key files before parse/use.
  • Added explicit bounded file-read ceilings at sidecar config boundary: sidecar config file <= 1 MiB, token/control-plane-token files <= 4096 bytes, signer key file <= 65536 bytes.
  • Added unit coverage proving oversize config/token/control-plane-token/signer key inputs are rejected with stable typed error codes.

Threat Model Delta (2026-02-15, Sidecar Secret-Path TOCTOU Hardening)

  • Closed Unix and Windows token/control-plane-token path-check/use race windows in recorder-sidecar-config by opening secret files with no-follow/reparse-safe flags and applying validation to opened descriptors/handles before bounded reads.
  • Added enterprise-path parity tests proving symlink and world-readable control-plane token files are rejected fail closed.
  • Added fail-closed unsupported-platform handling for non-Unix/non-Windows targets to avoid permissive fallback behavior.

Threat Model Delta (2026-02-15, recorder-store Active-Segment Corruption Guard)

  • Hardened recorder-store active-segment readback to fail closed when hostile persisted state contains multiple open segments.
  • Added deterministic tie-break ordering (created_at, then segment UUID bytes) for segment listings across in-memory and SQLite backends to remove equal-timestamp nondeterminism.
  • Hardened SQLite filter handling for explicit empty segment_ids and envelope IN-list filter values so empty vectors deterministically return no matches instead of surfacing SQL syntax errors.
  • Added regression tests in recorder-store covering multi-open-segment corruption detection, equal-timestamp ordering stability, and empty-list filter handling.

Threat Model Delta (2026-02-15, CLI I18n Catalog Integrity Hardening)

  • Replaced permissive CLI localization behavior with build-validated, fail-closed catalog generation (crates/recorder-cli/build.rs) sourced from repo-tracked locale catalogs (crates/recorder-cli/i18n/locales/*.toml).
  • Added strict controls that block build/test on localization drift: required default locale en, locale-tag uniqueness, mandatory provenance metadata (machine or human_reviewed), exact key parity across locales, and exact placeholder-set parity for every message key.
  • Added compile-fail misuse gates for typed i18n macro invocation (crates/recorder-cli/tests/i18n_compile_fail.rs and trybuild fixtures), preventing unknown keys and placeholder misuse from compiling.
  • Enforced no-silent-fallback rendering semantics: runtime now fail-closes i18n integrity violations (unknown key in compatibility path, placeholder mismatch, unresolved placeholder token) instead of permissive fallback to key IDs or English.
  • Added explicit locale provenance runtime behavior: machine-provenance locales emit a deterministic operator notice, while JSON machine-mode outputs remain locale-neutral and stable across locale selection.