Recorder Threat Model
Overview
Recorder is the deterministic evidence data plane for governance systems. It ingests unsealed boundary events, seals them into hash-chained envelopes, persists them in append-only segments, and exports verifiable bundles for offline checks.
Recorder is composed of:
- Core contracts (
recorder-core): identity, hash/signature abstractions, filter/cursor types, error algebra. - Envelope model (
recorder-envelope): canonical encoding, envelope/segment/bundle types, verification result algebra. - Decision Gate integration adapter (
recorder-decision-gate-adapter): deterministic MCP/runpack transcript mapping, runpack integrity policy, and transcript redaction/bounds policy enforcement. - OpenTelemetry integration adapter (
recorder-otel-adapter): deterministic OTel JSON export mapping, strict record validation, and redaction/bounds policy enforcement. - Recorder runtime (
recorder): ingest validation, segment lifecycle, bundle builder, offline verifier, provider/adapter implementations. - Rust SDK facade (
recorder-sdk): deterministic builder and unified in-process facade over recorder, adapter, provider, verifier, and store handles. - Python sidecar SDK (
recorder-client): contract-driven sync/async sidecar HTTP client with strict response/error typing, retry-safe idempotency behavior, and generated-contract drift checks. - Storage backends (
recorder-store): SQLite and in-memory implementations of envelope, attachment, and segment stores. - Sidecar config authority (
recorder-sidecar-config): canonical sidecar config model, fail-closed validation, and deterministic config projection artifacts (schema/compat/example/docs) consumed by contract generation and CI gates. - Sidecar service surface (
recorder-sidecar): HTTP/JSON runtime exposing recorder/query/bundle/config workflows over Unix/TCP with auth/bounds/ idempotency controls, strict media-type policy enforcement, and first-party OTel JSON ingest compatibility route. - CLI surface (
recorder-cli): local operator commands for record/query/bundle/ config/doctor workflows plus Decision Gate and OTel fixture ingest.
Recorder now supports both local-process workflows and a built-in sidecar HTTP service boundary. External wrappers (for example MCP bridges or API gateways) remain in-scope as hostile boundaries.
Related Documentation
- Repository overview:
README.md. - Security posture and requirements:
Docs/Standards/codebase_engineering_standards.md. - Architecture references: Docs/architecture/recorder_system_architecture.md, Docs/architecture/recorder_evidence_model_and_integrity_architecture.md, Docs/architecture/recorder_runtime_and_bundle_architecture.md, Docs/architecture/recorder_storage_architecture.md, Docs/architecture/recorder_sidecar_architecture.md, Docs/architecture/recorder_cli_and_operator_workflows_architecture.md, Docs/architecture/recorder_system_test_and_validation_architecture.md.
- Design references:
Docs/design/recorder_core_math_evidence_observability_and_verification.md,Docs/design/dg_proof_recorder_and_mobile_edge_interaction_model.md.
Security Goals
- Deterministic, reproducible envelope and bundle integrity outcomes.
- Tamper-evident append-only history with chain continuity enforcement.
- Fail-closed behavior on malformed, missing, ambiguous, or unverifiable data.
- Offline-verifiable bundle portability without trusted live dependencies.
- Clear trust boundaries between adapters, recorder runtime, stores, and consumers.
- Bounded input/resource exposure at operator entry points.
- Strong adversary posture: designs must remain defensible against compromised insiders and nation-state-capable operators.
Non-Goals / Out of Scope
- Confidentiality guarantees after evidence is disclosed to downstream systems.
- Defense against full host/kernel compromise without external platform controls.
- Key custody/HSM/remote attestation infrastructure (beyond current local key usage hooks).
- Built-in transport security for remote deployment wrappers (mTLS/authn/authz are deployment responsibilities today).
- Non-repudiation signing of bundle archives at rest (external signing/WORM is required).
Assets
- Unsealed and sealed envelope data (event type, actor, environment, payload, time, correlation IDs).
- Envelope integrity material (
content_hash,chain_hash, sequence position, segment linkage). - Segment metadata (
SegmentGenesis, predecessor linkage, seal records). - Attachment bytes and content hashes.
- Bundle contents (segments, envelopes, attachments, proof anchors).
- Verification manifests, verdict artifacts, and verifier transcript artifacts.
- Recorder runtime configuration (
recorder_id, schema version, auto-seal, startup verification depth). - SQLite store file and in-memory state.
- Signing key material and derived key IDs when signers are configured.
- Persisted signer-rotation state (
store_metakeys for current key, epoch, and last rotation event marker). - Generated contract artifacts under Docs/generated/recorder/** and manifest authority Docs/generated/recorder/index.json, including sidecar API artifacts (Docs/generated/recorder/apis/sidecar.*), sidecar config artifacts (Docs/generated/recorder/config/sidecar.*), and SDK projection artifacts (Docs/generated/recorder/sdk/*).
- Performance policy artifacts (
target/perf-matrix/latest.json,target/perf/v2/*.json,target/perf/triage/latest.md,system-tests/perf_baselines/*.json) and runner fingerprint metadata. - CLI artifacts (selector JSON, payload JSON, attachment file/inline payloads, Decision Gate fixture JSON, OTel fixture JSON, config JSON, bundle JSON, verifier transcript JSON, and assurance check JSON outputs).
- Container deployment artifacts and runtime mounts (Dockerfile image outputs, compose/env definitions, mounted sidecar config/data/run paths, Docker secret token source paths, and tmpfs token target paths).
Adversary Model
- Nation-state adversaries with full code visibility and long-term persistence.
- Untrusted adapters/clients submitting malicious envelopes or attachments.
- Compromised insiders with write access to config, binaries, or storage.
- Attackers tampering with on-disk SQLite data, bundle files, or exported artifacts.
- Attackers attempting replay, chain forking, payload mutation, or attachment substitution.
- Attackers inducing resource exhaustion via oversized or expensive inputs.
- Malicious integrators exposing Recorder via remote transports without sufficient access controls.
Trust Boundaries
- Adapter ingest boundary:
UnsealedEnvelopeand attachment data are untrusted. - CLI/operator boundary: all command-line values and JSON files are untrusted.
- Runtime config boundary: config file contents and store path are untrusted.
- Sidecar HTTP boundary: request line, headers, auth token, JSON body, and transport metadata are untrusted.
- Sidecar enterprise bridge boundary: enterprise control-plane admission and hook calls are untrusted network dependencies and must fail closed.
- Storage boundary: persisted rows/blobs are untrusted on read.
- Bundle boundary: imported bundle JSON is untrusted before verification.
- Evidence provider boundary: consumer calls to list/fetch/verify/query are untrusted requests.
- Rust SDK facade boundary: builder/runtime inputs (
recorder_id, adapter metadata, storage path, trust root, signer/trust policy) are untrusted and must preserve recorder/provider fail-closed behavior. - Python SDK boundary: sidecar endpoint/base URL, auth token, idempotency headers, retry/backoff policy inputs, request models, query parameters, and HTTP response payloads are untrusted and must fail closed on contract drift, unknown error codes, or schema mismatch.
- Decision Gate adapter boundary: Decision Gate transcript/runpack fixture payloads are untrusted and must pass mapping/integrity checks before ingest.
- OTel adapter boundary: OpenTelemetry JSON export payloads are untrusted and must pass strict record-shape, identifier, timestamp, and bounded/redacted mapping checks before ingest.
- Signing boundary: signer/verifier integrations and key management are external trust dependencies.
- Contract generation boundary: generated projection artifacts are untrusted until hash-verified against canonical index manifest.
- Performance harness boundary: runner-class labels, workload knobs, and perf
artifacts are untrusted until lane identity (
regression/capacity), workload metadata, and pinned-runner policy checks are validated. - External wrapper boundary: any network-facing bridge is outside this repo and must enforce authn/authz and transport protections.
- Container-orchestration boundary: compose/env/runtime mount definitions, Docker secret source paths, tmpfs token-target paths, and host filesystem ownership/permission models are untrusted deployment inputs and must satisfy sidecar token/config hardening constraints.
Entry Points and Attack Surfaces
- CLI commands:
recorder segment open|seal|status|list,recorder signer rotate,recorder envelope record|record-with-attachments,query envelopes,bundle build|verify|inspect,assurance startup|trust-root|gates,contract generate|check,decision-gate ingest-fixture,otel ingest-fixture,config validate,doctor. - Sidecar commands:
recorder-sidecar --config <path>,recorder-sidecar healthcheck --url <url>,recorder-sidecar healthcheck --unix-socket <path> --path <path>. - Sidecar HTTP routes:
GET /health,GET /startup,GET /ready,GET /v1/segments,GET /v1/segments/active,POST /v1/envelopes,POST /v1/envelopes/with-attachments,POST /v1/query/envelopes,POST /v1/bundles/build|verify|inspect,POST /v1/ingest/otel,GET /v1/config. - File inputs: runtime config JSON/TOML, payload JSON/file, attachment file/inline base64, Decision Gate fixture JSON, selector JSON, bundle JSON.
- Container deployment inputs:
docker/sidecar/Dockerfile,docker/sidecar/docker-compose.yml,docker/sidecar/.env, mountedsidecar.toml, Docker secret token source file, tmpfs token target directory, and bind-mounted data/run directories. - Library APIs:
RecorderBuilder::build,Recorder::record|record_with_attachments,Recorder::build_bundle|verify_bundle,Recorder::list_bundles|fetch_bundle|query_envelopes,RecorderAdapter::emit_envelope,RecorderAdapter::emit_with_attachments,DecisionGateAdapter::ingest_fixture,RecorderEngine::open_segment|record_envelope|seal_segment,EvidenceProvider::list_bundles|fetch_bundle|verify_bundle|query_envelopes. - Storage entry points: SQLite DB creation/open, SQL row materialization, attachment fetch, genesis decode.
- Bundle verification pipeline: manifest validation, attachment hash checks, content/chain checks, cross-segment checks, signature phase.
Security Controls and Invariants
- Canonical JCS encoding for deterministic hash inputs (
serde_jcs). - Envelope content hash and chain hash formulas are explicit and stable.
- Segment genesis anchors first-envelope chain computation.
- Hash equality uses constant-time comparison in core/verifier/store paths.
- Adapter/validation layer enforces fail-closed checks before recording.
- Envelope builder enforces required fields, UTC times, deterministic attachment ref order.
- Envelope attachment and segment-genesis constructors reject empty, blank, and
control-character-bearing
content_typeandrecorder_idinputs. - Recorder config validation fail-closes malformed
recorder_idvalues at engine startup (ActorId-compatible identity shape required). - Core identity constructors reject blank/whitespace-only and control-character
string inputs;
key_idis enforced as exact lowercasehex(SHA-256(public_key_bytes))shape (64 chars). - Core identity constructors enforce explicit max lengths:
trace_id/session_id/actor_id/environment_id <= 256 chars,event_type/adapter_id <= 128 chars. - String-backed identity serde paths are constructor-validated; hostile JSON cannot bypass identifier invariants at deserialize boundaries.
- Store append enforces open-segment and chain-continuity invariants.
- Store segment creation enforces single-open-segment and duplicate-segment-ID fail-closed checks.
- Store active-segment lookup fail-closes on corrupted persisted state with multiple open segments instead of selecting an arbitrary row.
- Store query/existence paths fail closed on storage-layer query errors instead of defaulting to permissive existence outcomes.
- SQLite append path is transaction-scoped for atomic metadata/data updates.
- SQLite enables WAL and foreign keys during schema initialization.
- Bundle builder uses deterministic ordering (
BTreeMap/BTreeSet) and tracks omitted parents/proof anchors. - Verifier executes 7 phases with trust-root-aware signature verification; it marks signature-skip only when no trust root is configured.
- Verifier phase 1 fail-closes on manifest structural tampering (segment entry mismatches and attachment-manifest mismatches), not only version/algorithm mismatches.
- Verifier enforces bounded-work policy before phase execution:
max segments
4096, max envelopes per segment100000, max envelopes total200000, max attachments total200000, max attachment refs per envelope4096. - Recorder startup verification re-checks active segment read-back integrity for
configured depth windows (
startup_verification_depth). - Evidence provider enforces stable bundle IDs and true fetch-by-ID semantics for sealed segments.
- Evidence provider enforces bounded query/materialization policy:
max
list_bundlesandquery_envelopeslimit10000, max per-segment envelope materialization200000. - CLI enforces bounded file-read limits: config 1 MiB, payload 4 MiB, selector 1 MiB, bundle 32 MiB, trust root 1 MiB, attachment file/inline payload 16 MiB, Decision Gate fixture 8 MiB, signer key 64 KiB, signer passphrase material 1 KiB.
- CLI query enforces bounded page policy:
default page size
1000, max page size10000, and deterministic cursor-based continuation contract in JSON mode. - CLI i18n catalogs are build-validated fail closed:
locale tag uniqueness, required
endefault locale, mandatory locale provenance (machineorhuman_reviewed), exact message-key parity against English, and exact placeholder-set parity for every key across enabled locales. - CLI localization rendering is strict and fail-closed: generated typed key/macro access prevents unknown message keys at compile time, and runtime rendering rejects placeholder-set mismatch and unresolved placeholders without silent fallback to key IDs or English.
- CLI locale provenance policy is explicit: machine-provenance locales emit deterministic operator notice, while JSON output payloads remain locale-neutral across locale selection.
- CLI runtime resolution validates
recorder_idusingActorIdconstructor semantics before command execution, includingconfig validateparity with recorder config validation behavior. - CLI signer loading is profile-aware (
securedefault) and fail-closed on encrypted-key requirements, malformed key material, insecure opt-down misuse, and key-state mismatch in secure mode. - Explicit key-rotation workflow emits deterministic lifecycle events
(
system.adapter.key_rotated/ break-glass variant) and persists idempotence markers in SQLite metadata. - Contract generation emits deterministic projection artifacts with manifest-hash authority and strict drift-check mode.
- Contract generation/check fail closed on unsafe output-root filesystem entries, including symlinked roots, hard-linked artifact files, or symlink traversal beneath the output tree.
- Contract schema generation applies explicit runtime constraint overlays for
identifier/text/hash/signature fields (including full
BundleSelectorvariant algebra) and fails closed if expected schema pointer paths are missing. - Contract generation now emits sidecar API artifacts
(
sidecar.openapi/errors/enums/examples/compat) and generated manifest authority includes those paths. - Contract generation now emits SDK projection artifacts
(
sdk/types.json,sdk/methods.json,sdk/test_vectors.json,sdk/manifest.json) and drift-check mode fail-closes on SDK artifact mismatch. - Rust SDK builder/facade wiring fail-closes invalid identity/config/store inputs while preserving recorder/provider determinism and bounded-work limits.
- Python SDK request/response wiring fail-closes invalid input/query parameter values, unknown problem codes, unexpected content types, and response schema drift against generated SDK projection artifacts while preserving sidecar idempotency semantics across retry attempts.
- Sidecar compatibility gates fail closed in CI by validating current
pre-release-break
/v1policy, enum-wire stability, error-code registry stability, and mutating-endpoint idempotency contract enforcement from generated artifacts. - Sidecar perf policy gates split regression (
sidecar_perf) and capacity (sidecar_capacity) lanes with explicit lane identity/workload metadata in artifacts to prevent fixed-workload regression signals from being misread as max-throughput results. - Pinned-runner capacity enforcement fails closed when required
sidecar_capacityartifacts are missing, and capacity floors (max_sustainable_rps) are anti-sandbagging protected by threshold policy checks. - Sidecar config validation fail-closes invalid protocol/security settings (schema/API version, transport host/port, non-loopback token requirement, signer key parseability, hard header-size cap, and numeric limit bounds) before binding listeners.
- Sidecar config file and secret-material loading enforce bounded hostile-input
file sizes and fail closed on oversize input:
config
<= 1 MiB, token/control-plane-token files<= 4096 bytes, signer key file<= 65536 bytes. - Sidecar token/control-plane-token file opening uses fail-closed
descriptor-bound semantics:
- Unix: no-follow open (
O_NOFOLLOW) plus regular-file and mode checks. - Windows: reparse-point-safe open (
FILE_FLAG_OPEN_REPARSE_POINT) plus opened-handle reparse-point rejection. - Other targets: fail-closed unsupported-platform rejection for secret-token file loading.
- Unix: no-follow open (
- Sidecar middleware stack enforces server/client correlation ID handling
(
x-server-correlation-id,x-correlation-id, legacyx-request-id), strict fail-closed media-type parsing (exactapplication/jsonwith optionalcharset=utf-8) and strict JSON-onlyAcceptparsing, aggregate header byte bounds, declared content-length bounds, bounded queue admission and in-flight concurrency, and per-request timeout fail-closed behavior. - Sidecar request media-type policy is centralized in
recorder-sidecar-configand projected into generated sidecar API compatibility artifacts so runtime and contract media-type allowlists cannot drift. - Sidecar token auth compares bearer tokens in constant time and bypasses auth
only for explicit probe routes (
/health,/startup,/ready) oropenmode. - Sidecar probe/readiness controls are explicit and fail closed:
/healthremains lightweight process liveness,/startupreports startup completion state, and/readycan enforce storage-only or storage+enterprise dependency readiness (probes.readiness_mode) with optional capacity-exhaustion fail-closed behavior (probes.ready_fail_on_admission_saturation). - Sidecar bounds middleware explicitly bypasses probe routes so orchestrator liveness/startup/readiness signals are observable even under admission saturation and active-request exhaustion.
- Sidecar auth failures emit RFC 6750
WWW-Authenticatechallenge metadata and sidecar errors use RFC 9457 Problem Details (application/problem+json). - Sidecar container packaging defaults to distroless nonroot runtime and compose profiles that require explicit UID/GID mapping, Docker secret token source inputs, and bootstrap materialization of hardened token files into tmpfs runtime paths before sidecar startup.
- Sidecar mutating operations enforce persistent idempotency replay/conflict
semantics with stream-scoped persistent retention
(
idempotency_entries) and deterministic request-hash scoping. - Sidecar mutating envelope operations enforce explicit stream identity inputs
(
tenant_id,recorder_id) and reject recorder mismatches fail-closed. - SQLite operation dispatch uses bounded queueing (
sync_channel) with deterministic fail-closed rejection when queue capacity (4096) is saturated. - Verifier transcript emission is deterministic and schema-versioned; bundle verification can export transcript JSON for machine-auditable evidence packets.
- CLI assurance checks provide explicit fail-closed deployment assertions for startup-verification depth, trust-root mode parsing, and contract gate execution.
- System tests validate tamper detection and fail-closed oversized-input behavior.
- System tests validate external adapter-ingest coupling using OpenClaw mock event streams, including signed/unsigned verification and sequence-gap strict fail-closed vs anomaly-recording policies.
- OpenClaw integration mapping applies deterministic sensitive-field redaction and bounded payload handling (string and JSON size limits) before immutable evidence recording.
- System tests validate external control-plane coupling using Decision Gate MCP runpack flow fixtures, including signed/unsigned verification and runpack integrity strict fail-closed vs anomaly-recording policies, including manifest self-integrity recomputation from declared file hashes.
- System tests validate sidecar subprocess lifecycle over real HTTP and fail-closed idempotency replay/conflict persistence across restart boundaries.
- Decision Gate integration mapping applies deterministic transcript-field redaction and bounded payload handling (string and JSON size limits) before immutable evidence recording.
- Production
recorder-decision-gate-adapterenforces mapping-contract version checks, deterministic event IDs/timestamps, transcript redaction, bounded payload metadata, and strict-vs-anomaly runpack integrity handling. - Production
recorder-otel-adapterenforces deterministic OTel span/log mapping (otel.span,otel.log.<severity>), strict identifier/timestamp parsing, deterministic ID derivation, and bounded/redacted payload projection.
Known current security-relevant gaps:
- Sidecar does not include built-in TLS/mTLS; non-loopback deployments should terminate TLS and enforce client identity in a hardened front proxy/gateway.
- Sidecar provides bounded local admission controls but not distributed/global rate limiting; deployment platforms still need caller quotas and abuse controls.
Threats and Mitigations
Authentication, Authorization, and Access Control
- Threat: unauthorized local or remote invocation of recording/query/bundle paths.
- Mitigation: sidecar supports explicit token mode with constant-time bearer
token checks, RFC 6750
WWW-Authenticatechallenge responses, and fail-closed auth misconfiguration handling; CLI remains local process invocation. - Residual risk:
opensidecar mode is deployment-sensitive and should only be used in trusted local IPC environments; non-loopback network deployments still require gateway-layer identity and transport controls.
Input Validation and Parsing
- Threat: malformed IDs, non-UTC times, invalid event types, malformed JSON, or ambiguous payloads.
- Mitigation: typed ID constructors, UTC enforcement, strict event-type
validation, strict key-id hex shape validation, fail-closed envelope
validation, strict attachment/segment text-field validation (
content_type,recorder_id), strict recorder configrecorder_idvalidation, typed JSON parsing. - Mitigation: bounded file reads prevent oversized JSON ingestion at CLI boundary, including trust-root and signer-key inputs.
Evidence Integrity and Authenticity
- Threat: payload mutation, chain forking, attachment substitution, or replayed envelopes.
- Mitigation: canonical-hash content addressing + chain continuity checks at append and verify time.
- Mitigation: verifier phase 1 enforces structural manifest integrity against bundle segment and attachment materialization.
- Mitigation: strict Ed25519 verification and trust-root policy enforcement are wired in verifier phase 6.
- Mitigation: secure-profile signer loading requires encrypted PKCS#8 key material by default and explicit opt-down for less secure dev ergonomics.
- Mitigation: explicit key-rotation events and persisted idempotence markers preserve signing lineage continuity and crash-recovery reconciliation.
- Residual risk: authenticity remains configuration-dependent when operators choose to verify without a trust root (explicit warning emitted).
Disclosure and Data Exposure
- Threat: accidental over-disclosure through bundle exports and query results.
- Mitigation: explicit operator-driven export/query commands and typed filters.
- Residual risk: no built-in field-level redaction policy in Recorder core; wrapper layers must enforce disclosure policy.
Storage and Bundle Integrity
- Threat: DB/file tampering, corruption, or partial writes.
- Mitigation: SQLite WAL, transactional append, chain validation, corruption detection when parsing invalid stored rows.
- Mitigation: bundle verification recomputes attachment/content/chain links and flags integrity failures.
- Residual risk: bundle archives are not signed by default; use external signature and immutable storage controls for non-repudiation.
Availability and Resource Exhaustion
- Threat: oversized config/payload/selector/bundle inputs and expensive verification workloads.
- Mitigation: CLI hard limits on file reads; fail-closed rejection before parse.
- Mitigation: typed errors and early rejection on invalid selectors/filters.
- Mitigation: recorder verifier/provider APIs enforce bounded-work limits and fail closed on oversized in-memory workloads.
- Mitigation: sidecar middleware enforces bounded header/body/content-length limits, bounded queue + active concurrency, and timeout fail-closed behavior.
- Mitigation: SQLite operation dispatch is bounded; enqueue saturation fails closed instead of allowing unbounded memory growth.
- Residual risk: no distributed/global rate limiter exists for sidecar or wrapper-exposed deployments; operators still need caller quotas and abuse controls.
Supply Chain and Execution Environment
- Threat: compromised binaries/config/runtime host or malicious dependency behavior.
- Mitigation: strict lint/no-panic posture, typed errors, deterministic contracts, and explicit architecture boundaries.
- Residual risk: host compromise and dependency compromise require external controls (SBOM/audit/signing/hardening).
Multi-Tenant and Isolation
- Threat: cross-tenant data exposure in shared deployments.
- Mitigation: Recorder IDs are labels; no implicit tenant isolation claims.
- Residual risk: shared deployments must enforce tenant-aware storage, encryption, and access control at wrapper/deployment layers.
Auditability and Observability
- Threat: inability to prove what happened after suspected tampering.
- Mitigation: append-only segment model, deterministic hashes, bundle manifests, offline verifier, and artifact-first system tests.
- Residual risk: comprehensive security audit logging for remote auth decisions is wrapper-dependent.
Implementation References (Controls and Protections)
Core and Envelope Integrity
- Canonical encoding and hash formulas:
crates/recorder-envelope/src/encoding.rs. - Hash types, algorithm registry, constant-time equality:
crates/recorder-core/src/hash.rs. - Identity and input-shape validation contracts:
crates/recorder-core/src/identity.rs,crates/recorder-core/src/filter.rs. - Signature contracts and key identity model:
crates/recorder-core/src/signature.rs,crates/recorder-envelope/src/signer.rs,crates/recorder-envelope/src/signature.rs. - Envelope/segment invariants:
crates/recorder-envelope/src/envelope.rs,crates/recorder-envelope/src/segment.rs,crates/recorder-envelope/src/attachment.rs.
Recorder Runtime and Verification
- Recorder lifecycle, chain progression, seal behavior:
crates/recorder/src/engine.rs. - Auto-seal policy validation:
crates/recorder/src/config.rs. - Adapter boundary validation:
crates/recorder/src/validation.rs,crates/recorder/src/local_adapter.rs. - Deterministic bundle assembly and proof anchors:
crates/recorder/src/bundle_builder.rs. - 7-phase verification pipeline and warning model:
crates/recorder/src/verifier.rs. - Evidence provider boundary behavior:
crates/recorder/src/evidence_provider.rs.
SDK Facade Surface
- Rust SDK facade composition and deterministic builder defaults:
crates/recorder-sdk/src/builder.rs,crates/recorder-sdk/src/lib.rs. - SDK prelude and stable consumer import boundary:
crates/recorder-sdk/src/prelude.rs. - SDK facade conformance tests:
crates/recorder-sdk/tests/test_builder.rs,crates/recorder-sdk/tests/test_facade.rs. - Python SDK contract projection generator and typed client surface:
sdk/python/scripts/generate_contract.py,sdk/python/src/recorder_client/generated_contract.py,sdk/python/src/recorder_client/client.py,sdk/python/src/recorder_client/client_common.py,sdk/python/src/recorder_client/sync_client.py,sdk/python/src/recorder_client/async_client.py,sdk/python/src/recorder_client/models.py,sdk/python/src/recorder_client/errors.py. - Python SDK unit/system conformance tests:
sdk/python/tests/test_contract_generated.py,sdk/python/tests/test_models.py,sdk/python/tests/test_errors.py,sdk/python/tests/test_client_unit.py,sdk/python/tests/test_client_system.py.
Integration Adapters
- Decision Gate adapter ingestion, deterministic mapping, and fail-closed
integrity-policy enforcement:
crates/recorder-decision-gate-adapter/src/adapter.rs. - Decision Gate mapping and redaction/bounds policy defaults:
crates/recorder-decision-gate-adapter/src/policy.rs. - Decision Gate fixture input contract:
crates/recorder-decision-gate-adapter/src/model.rs. - Decision Gate adapter typed error algebra:
crates/recorder-decision-gate-adapter/src/error.rs. - OTel adapter ingestion, deterministic mapping, and strict-fail policy:
crates/recorder-otel-adapter/src/adapter.rs. - OTel mapping and redaction/bounds policy defaults:
crates/recorder-otel-adapter/src/policy.rs. - OTel JSON export input contract and adapter error algebra:
crates/recorder-otel-adapter/src/model.rs,crates/recorder-otel-adapter/src/error.rs.
Storage
- Storage contracts and invariants:
crates/recorder-store/src/traits.rs,crates/recorder-store/src/error.rs. - In-memory backend behavior:
crates/recorder-store/src/memory.rs. - SQLite initialization, WAL, FK, connection thread model:
crates/recorder-store/src/sqlite/schema.rs,crates/recorder-store/src/sqlite/connection.rs,crates/recorder-store/src/sqlite/mod.rs. - SQLite envelope/segment/attachment operations:
crates/recorder-store/src/sqlite/envelope_ops.rs,crates/recorder-store/src/sqlite/segment_ops.rs,crates/recorder-store/src/sqlite/attachment_ops.rs.
CLI and Validation Surface
- Command routing and runtime resolution:
crates/recorder-cli/src/main.rs. - Input hard limits and bounded file reads:
crates/recorder-cli/src/main.rs. - Localization and deterministic output behavior:
crates/recorder-cli/build.rs,crates/recorder-cli/i18n/locales/en.toml,crates/recorder-cli/i18n/locales/ca.toml,crates/recorder-cli/src/i18n.rs. - Contract artifact generation and drift checks:
crates/recorder-contract/src/lib.rs,crates/recorder-contract/src/sidecar_api.rs,crates/recorder-contract/src/sidecar_api/openapi.rs,crates/recorder-contract/src/sidecar_api/artifacts.rs,crates/recorder-contract/src/sidecar_api/specs/mod.rs,scripts/ci/generate_all.sh. - Sidecar contract compatibility and perf policy gates:
scripts/ci/sidecar_contract_gates.py,scripts/ci/perf_matrix.py,scripts/ci/perf_gate.py,scripts/ci/perf_threshold_policy.py,scripts/perf/run.py,scripts/perf/update_baseline.py.
Sidecar Service Boundary
- Sidecar bootstrap/config and transport lifecycle:
crates/recorder-sidecar-config/src/config.rs,crates/recorder-sidecar-config/src/validation.rs,crates/recorder-sidecar-config/src/validation/enterprise.rs,crates/recorder-sidecar/src/server.rs,crates/recorder-sidecar/src/shutdown.rs. - Sidecar auth and request-bound controls:
crates/recorder-sidecar/src/middleware/auth.rs,crates/recorder-sidecar/src/middleware/bounds.rs,crates/recorder-sidecar/src/middleware/request_id.rs. - Sidecar single-writer ingest/idempotency and error contract:
crates/recorder-sidecar/src/ingest.rs,crates/recorder-sidecar/src/idempotency.rs,crates/recorder-sidecar/src/error.rs,crates/recorder-sidecar/src/contract.rs. - Sidecar contract conformance integration tests:
crates/recorder-sidecar/tests/error_registry_contract.rs,crates/recorder-sidecar/tests/idempotency_restart.rs. - Sidecar container packaging and operator profile:
docker/sidecar/Dockerfile,docker/sidecar/docker-compose.yml,docker/sidecar/config/sidecar.toml,crates/recorder-sidecar/src/bin/docker_bootstrap.rs.
System Validation
- Security/system tests:
system-tests/tests/suites/security.rs,system-tests/tests/suites/bundle.rs,system-tests/tests/suites/persistence.rs,system-tests/tests/suites/recorder.rs,system-tests/tests/suites/sidecar.rs. - Docker packaging and compose-lifecycle validation:
system-tests/tests/suites/sidecar_docker.rs,system-tests/tests/helpers/docker.rs. - Registry-driven validation contract:
system-tests/AGENTS.md,system-tests/test_registry.toml.
Operational Requirements
- Treat all deployments as hostile environments; assume compromised insiders and nation-state-capable adversaries.
- Restrict filesystem access to store/config/bundle paths using least-privilege OS ACLs.
- Use signed releases, dependency auditing, and host hardening for production nodes.
- If exposing Recorder sidecar or wrapper APIs over non-loopback networks, enforce strong authn/authz, mTLS/TLS termination, and command allowlists.
- For containerized sidecar deployments, run non-root, provide token source via
Docker secret input, materialize token files into tmpfs with owner-only mode
(
0600on Unix) before sidecar startup, and map host UID/GID explicitly so writable data/run paths remain deterministic. - Keep CLI input limits enabled; tighten wrapper-level limits where possible.
- Do not treat unsigned bundles as non-repudiable artifacts; apply external signing and immutable retention controls.
- Segment tenants into isolated stores/processes unless explicit shared-tenancy controls are implemented.
- Monitor verifier warnings (especially signature-skip and partial-segment warnings) as policy-significant outcomes.
Failure Posture
- Fail closed on invalid or ambiguous inputs.
- Reject appends that violate segment state or chain continuity.
- Reject malformed config/filter/time-range definitions.
- Mark tampered bundles as
Invalidor fail verification with typed errors on manifest/infrastructure mismatches. - Never assume trust in persisted state without integrity re-checks.
Threat Model Delta (2026-02-08, OSS Baseline + Docker)
- Initial Recorder repository threat model baseline added under Docs/security/threat_model.md.
- Mapped trust boundaries, controls, and residual risks to concrete current implementation files.
- Closed signer/runtime wiring gap with profile-aware secure defaults, bounded signer input limits, and fail-closed key-material handling.
- Added explicit signer-rotation workflow with deterministic lifecycle events and persisted idempotence metadata.
- Added deterministic contract-generation boundary (
recorder-contract) with manifest-hash authority and drift-check controls. - Hardened
recorder-contractoutput-root handling to reject symlinked roots, hard-linked artifact files, and symlink traversal under generated artifact trees in both generate and check modes. - Added first-party sidecar container deployment boundary: Dockerfile/Compose/config artifacts are now explicit attack surfaces and deployment trust boundaries.
- Added system-test Docker coverage for sidecar packaging hardening and Docker Compose lifecycle open/record/query validation.
- Closed tracked P1 system-test security-relevant coverage gaps for selector dedup, proof-anchor semantics, cross-segment mismatch fail-closed behavior, attachment closure fail-closed behavior, and SQLite/in-memory parity.
- Hardened
recorder-coreidentifier validation to reject whitespace-only and control-character string identifiers and require strict 64-char lowercase hexkey_idvalues. - Hardened deserialize boundaries so constrained identifier and envelope text fields cannot be instantiated from serde payloads without constructor checks.
- Hardened
recorderconfig and verifier boundaries: recorder startup rejects malformedrecorder_idvalues during config validation, phase-1 verification fail-closes on manifest structural mismatches, and verifier now enforces explicit bounded-work ceilings. - Hardened
recorderevidence-provider library boundaries with explicit fail-closed limits for list/query/materialization workloads. - Hardened
recorder-storeSQLite connection dispatch with bounded queue capacity and deterministic saturation rejection semantics. - Added system-test coverage for malformed identifiers, bounded CLI file input surfaces, secure signer-file policy, signer-rotation recovery/corruption behavior, contract CLI symlink safety, manifest structural tamper cases, SQLite corruption materialization failures, and single-open-segment CLI enforcement.
- Closed all findings tracked in Docs/security/audits/OSS_launch_0.md.
- Implemented
recorder-sidecaras a built-in HTTP/JSON boundary with fail-closed config validation, request auth/bounds controls, bounded queue/concurrency controls, structured error contract, and persistent idempotency semantics; updated trust boundaries, attack surfaces, mitigations, and residual risks to reflect this runtime surface. - Implemented generated sidecar API artifact coverage
(
sidecar.openapi/errors/enums/examples/compat) with fail-closed CI compatibility gates enforcing current pre-release-break/v1, enum-wire stability, error-code registry stability, and mutating-endpoint idempotency contracts. - Consolidated sidecar runtime config semantics into a dedicated
recorder-sidecar-configcrate and wired sidecar runtime startup to consume its typed fail-closed config validation boundary. - Expanded generated contract scope to include sidecar config artifacts
(
config/sidecar.schema.json,config/sidecar.compat.json,config/sidecar.example.toml,config/sidecar.md) emitted byrecorder-contractfromrecorder-sidecar-configprojection helpers. - Extended sidecar compatibility gates to fail closed on sidecar config artifact drift and cross-artifact inconsistency (schema/compat/example/docs).
- Added sidecar integration/system validation for generated error-registry compatibility and restart-boundary idempotency replay/conflict persistence.
Threat Model Delta (2026-02-08)
- Added enterprise sidecar boundary mode with explicit fail-closed config
validation (
enterprise.mode, control-plane URL/token requirements, and token-mode security requirement). - Added enterprise identity extraction boundary (
x-tenant-id,x-principal-id,x-authn-method, optional namespace) and fail-closed admission behavior on malformed/missing identity context. - Added pre-handler enterprise policy gate middleware that denies protected operations when control-plane authz/quota checks are unavailable or deny.
- Added managed-cloud writer-lease assertion gating for mutating operations
(
x-recorder-writer-lease=truerequired when configured). - Added explicit sidecar enterprise boundary hook emission to control-plane
(
/v1/enterprise/sidecar/hooks) atadmissionandresultphases with fail-closed behavior for successful protected operations when hook export fails. - Result: sidecar ingress now has an additional external dependency on enterprise control-plane availability/integrity; outage or invalid responses are handled as fail-closed admission denials and successful-hook export failures are treated as fail-closed request errors.
Threat Model Delta (2026-02-08, Probe and Readiness Hardening)
- Added explicit startup probe route (
GET /startup) to sidecar entry-point inventory and probe-auth bypass policy. - Added fail-closed readiness dependency modes (
storage_only,storage_and_enterprise) with config validation and control-plane readiness dependency checks in enterprise-enabled modes. - Added optional readiness fail-closed behavior when request admission/execution
capacity is exhausted (
request_capacity_exhaustedreadiness reason). - Added explicit probe-priority middleware behavior:
/health,/startup, and/readybypass normal auth/policy/bounds request gates to preserve orchestration observability during overload. - Added Docker system-test probe coverage validating containerized
/startupand/readybehavior before and after segment-open lifecycle transitions.
Threat Model Delta (2026-02-08, continued)
- Renamed enterprise boundary adapter implementation from
enterprise.rstocontrol_plane_bridge.rsto make the trust-boundary intent explicit (control-plane bridge only; no evidence-math authority). - Centralized sidecar enterprise ingress/header/hook vocabulary in
crates/recorder-sidecar/src/contract.rsand wired middleware/bridge paths to those constants to reduce drift-induced policy bypass risk. - Added conformance tests to ensure sidecar operation labels, identity headers, and hook boundary/outcome labels remain synchronized with control-plane contract vocabulary.
- Added cross-repo end-to-end enterprise lane coverage proving control-plane deny decisions fail closed at sidecar mutation boundaries while successful admitted paths preserve deterministic evidence verification outcomes.
Threat Model Delta (2026-02-08, CLI OSS Expansion)
- Added CLI operator command
recorder envelope record-with-attachmentswith fail-closed attachment boundary validation for file and inline base64 inputs. - Added CLI operator command
decision-gate ingest-fixturethat routes fixture inputs through productionrecorder-decision-gate-adapterintegrity and mapping policies. - Added CLI runtime config parity checks so malformed
recorder_idvalues fail closed duringconfig validateusing the same identity-shape constraints as recorder runtime. - Expanded CLI auto-seal boundary to include duration and combined policies, with fail-closed zero-value rejection for both count and duration thresholds.
- Added CLI query page bounds (default
1000, max10000) and deterministic JSON pagination contract (envelopes+next_cursor) to prevent unbounded materialization at operator boundary.
Threat Model Delta (2026-02-10, SDK Facade + SDK Artifacts)
- Added first-party Rust SDK facade crate (
recorder-sdk) that composes existing recorder/adapter/provider/store/verifier boundaries without introducing new protocol semantics. - Added SDK builder trust boundary inputs (storage path, recorder and adapter identifiers, signer/trust-root injection, startup policy knobs) and required fail-closed validation behavior inherited from recorder/provider layers.
- Expanded generated-contract asset inventory and controls to include SDK projection artifacts (Docs/generated/recorder/sdk/types.json, Docs/generated/recorder/sdk/methods.json, Docs/generated/recorder/sdk/test_vectors.json, Docs/generated/recorder/sdk/manifest.json) with manifest-hash authority and drift-check enforcement.
- Added SDK facade integration tests covering builder defaults, storage selection, record/build/verify flows, and provider query/fetch/verify parity.
Threat Model Delta (2026-02-14, Perf Lane Split and Pinned Capacity Gating)
- Added explicit perf-harness trust boundary semantics: fixed-workload regression artifacts and saturation-capacity artifacts are distinct evidence classes with different operational decisions.
- Added lane identity and workload metadata requirements to sidecar perf
artifacts (
lane,workload.*) so downstream gates can fail closed on missing/ambiguous perf context. - Added pinned-runner conditional gate requirement for capacity artifacts:
when pinned release jobs are configured for capacity enforcement, missing
sidecar_capacityreports are treated as a fail-closed condition. - Added anti-sandbagging policy coverage for capacity floors to prevent
threshold downgrades of
max_sustainable_rpswithout explicit governance. - Added resource-exhaustion saturation test notes: capacity sweeps are allowed
to intentionally drive bounded
429and elevated latency to identify sustainable operating region and knee point without relaxing fail-closed runtime behavior.
Threat Model Delta (2026-02-15, Perf Workflow Governance and Schema v2)
- Added canonical perf orchestration boundary (
scripts/perf/run.py) that normalizes matrix artifacts into schema-v2 per-suite reports with explicit runner fingerprint and lane identity metadata. - Added baseline-governance update surface (
scripts/perf/update_baseline.py) requiring explicit sample-count/window/author/reason metadata on baseline refresh. - Added per-suite macrobench artifact paths
(
target/performance/stress/latest.json,target/performance/bundle/latest.json) to reduce ambiguity and prevent evidence overwrite confusion in performance triage.
Threat Model Delta (2026-02-11, Python SDK Surface)
- Added first-party Python sidecar SDK (
sdk/python) as a committed projection consumer of Docs/generated/recorder/sdk/{types,methods,test_vectors,manifest}. - Added Python SDK trust boundary for endpoint/token/idempotency/retry inputs and response decoding behavior with fail-closed validation posture.
- Added deterministic Python contract-constants generator that derives
generated_contract.pydirectly from canonical generated SDK artifacts. - Added strict Python client controls for unknown error-code rejection, content-type validation, operation-local error-contract validation, and stable idempotency key reuse across mutating-operation retries.
- Added Python SDK unit and system tests and wired repository orchestration
(
scripts/ci/generate_all.sh) to run Python SDK generate/check/test gates.
Threat Model Delta (2026-02-11, OSS Assurance WS1-WS3)
- Added deterministic verifier transcript artifact model
(
VerificationTranscript) and wired CLI export surface (bundle verify --verdict-transcript-output) for machine-verifiable verification packet evidence. - Added explicit OSS assurance command boundary
(
assurance startup,assurance trust-root,assurance gates) to provide fail-closed deployment assertions for startup verification depth, trust-root mode, and contract gate execution. - Added system-test coverage proving SQLite operation queue saturation is rejected fail closed with deterministic summary artifact output.
Threat Model Delta (2026-02-13, Docker Token Bootstrap Parity Hardening)
- Eliminated host bind-mounted token-file dependence for sidecar Docker profile.
- Added dedicated container bootstrap binary
(
recorder-sidecar-docker-bootstrap) that ingests token source material from Docker secret file/env input and writes runtime token file to configured target path with fail-closed validation and owner-only permissions. - Updated Docker Compose profile to use a Docker secret source and tmpfs token target mount, preserving explicit non-root UID/GID mapping for writable data and run directories.
- Expanded system-test Docker suite hardening checks to enforce secret-source-to-tmpfs token bootstrap behavior.
- Result: Windows host ACL variance no longer determines runtime token-file mode parity inside Linux containers because sidecar token file is now created in-container with deterministic permission semantics.
Threat Model Delta (2026-02-13, Phase 1 Strict Media-Type Boundary Hardening)
- Replaced permissive sidecar media-type checks with strict fail-closed parser
behavior for
Content-TypeandAcceptat the HTTP boundary. - Sidecar request-body endpoints now enforce exact
application/jsonwith optionalcharset=utf-8; unknown parameters, malformed values, and non-UTF-8 charset values are rejected with stableunsupported_media_type. - Sidecar response negotiation now enforces strict JSON-only
Acceptranges (application/jsonor*/*) with malformed/non-JSON ranges rejected via stablenot_acceptable. - Centralized route media-type policy in
recorder-sidecar-configand projected that policy into generated sidecar OpenAPI/compat artifacts and CI contract gates, reducing runtime/contract drift risk on protocol boundary widening.
Threat Model Delta (2026-02-13, Phase 3 OTel Compatibility Surface)
- Added production OTel compatibility adapter crate
(
crates/recorder-otel-adapter) and sidecar ingest route (POST /v1/ingest/otel) as a new hostile network boundary. - New OTel ingest boundary enforces strict JSON-only media policy, bearer auth, enterprise policy middleware, bounded admission/execution controls, timeout controls, and idempotency-key requirements consistent with other mutating sidecar routes.
- OTel adapter mapping is deterministic and fail-closed:
strict identifier/timestamp parsing, deterministic SHA-256-to-UUIDv7-shaped
IDs, deterministic event-type mapping (
otel.span,otel.log.<severity>), and bounded/redacted payload projection. - Added generated integration mapping contract artifacts for Decision Gate and OTel under Docs/generated/recorder/contracts/integrations/ with runtime/contract drift checks in generator test coverage.
- Added CLI operator parity command
recorder otel ingest-fixture, expanding operator input attack surface while preserving bounded fixture reads and adapter fail-closed behavior.
Threat Model Delta (2026-02-13, Enterprise Durable Control-Plane Dependency)
- Expanded enterprise dependency threat posture to include durable control-plane state backends (Postgres-backed fleet/governance/cloud-ops) hosted in Asset-Core and consumed by sidecar enterprise admission paths.
- Added explicit mutation-path outage fail-closed test coverage when control-
plane dependency becomes unavailable:
crates/recorder-sidecar/tests/enterprise_control_plane_e2e.rsandsystem-tests/tests/suites/sidecar_enterprise.rs. - Security effect: enterprise mode write paths deny on control-plane outage; no best-effort/fail-open mutation behavior is permitted.
Threat Model Delta (2026-02-14, Sidecar Single-Writer Ingest Refactor)
- Replaced direct mutating envelope handler writes with a bounded ingest queue
plus dedicated single-writer runtime (
IngestGateway/IngestWriterRuntime) that owns durable commit and stream-position assignment. - Added stream-scoped identity and ordering boundary to sidecar mutating and
query routes:
tenant_id + recorder_idnow define authoritative stream scope for ordering and idempotency. - Added persistent stream-scoped idempotency table
(
idempotency_entries) with deterministic request-hash replay/conflict semantics and bounded TTL/cap retention per stream. - Added writer/projection persistence surfaces in SQLite schema v2:
streams,idempotency_entries, andquery_projection_envelopes; startup now fails closed on non-v2 databases and requires a fresh DB for this pre-release break. - Removed mutating segment HTTP routes (
POST /v1/segments,POST /v1/segments/seal) from sidecar runtime, reducing external mutation attack surface while keeping read-only segment observability routes.
Threat Model Delta (2026-02-14, Single-Writer Performance Completion)
- Completed hot-path migration to writer-native sealing and single-transaction
SQLite micro-batch commits; mutating sidecar ingest no longer depends on
per-envelope
RecorderEngine::record_envelopecommits. - Added explicit SQLite durability/checkpoint tuning controls at sidecar config
boundary (
storage.sqlite_synchronous,storage.sqlite_wal_autocheckpoint_pages,storage.sqlite_busy_timeout_ms) with fail-closed validation andsynchronous=FULLsecure default. - Updated segment observability contract:
GET /v1/segments/activenow returns paginated active-segment lists with stream identity (tenant_id,recorder_id), reducing ambiguity in per-stream roll/debug workflows.
Threat Model Delta (2026-02-15, Sidecar Config File-Read Bounds Hardening)
- Hardened
recorder-sidecar-confighostile file-input boundaries to fail closed on oversized config/secret/key files before parse/use. - Added explicit bounded file-read ceilings at sidecar config boundary:
sidecar config file
<= 1 MiB, token/control-plane-token files<= 4096bytes, signer key file<= 65536bytes. - Added unit coverage proving oversize config/token/control-plane-token/signer key inputs are rejected with stable typed error codes.
Threat Model Delta (2026-02-15, Sidecar Secret-Path TOCTOU Hardening)
- Closed Unix and Windows token/control-plane-token path-check/use race windows
in
recorder-sidecar-configby opening secret files with no-follow/reparse-safe flags and applying validation to opened descriptors/handles before bounded reads. - Added enterprise-path parity tests proving symlink and world-readable control-plane token files are rejected fail closed.
- Added fail-closed unsupported-platform handling for non-Unix/non-Windows targets to avoid permissive fallback behavior.
Threat Model Delta (2026-02-15, recorder-store Active-Segment Corruption Guard)
- Hardened
recorder-storeactive-segment readback to fail closed when hostile persisted state contains multiple open segments. - Added deterministic tie-break ordering (
created_at, then segment UUID bytes) for segment listings across in-memory and SQLite backends to remove equal-timestamp nondeterminism. - Hardened SQLite filter handling for explicit empty
segment_idsand envelopeIN-list filter values so empty vectors deterministically return no matches instead of surfacing SQL syntax errors. - Added regression tests in
recorder-storecovering multi-open-segment corruption detection, equal-timestamp ordering stability, and empty-list filter handling.
Threat Model Delta (2026-02-15, CLI I18n Catalog Integrity Hardening)
- Replaced permissive CLI localization behavior with build-validated,
fail-closed catalog generation (
crates/recorder-cli/build.rs) sourced from repo-tracked locale catalogs (crates/recorder-cli/i18n/locales/*.toml). - Added strict controls that block build/test on localization drift:
required default locale
en, locale-tag uniqueness, mandatory provenance metadata (machineorhuman_reviewed), exact key parity across locales, and exact placeholder-set parity for every message key. - Added compile-fail misuse gates for typed i18n macro invocation
(
crates/recorder-cli/tests/i18n_compile_fail.rsand trybuild fixtures), preventing unknown keys and placeholder misuse from compiling. - Enforced no-silent-fallback rendering semantics: runtime now fail-closes i18n integrity violations (unknown key in compatibility path, placeholder mismatch, unresolved placeholder token) instead of permissive fallback to key IDs or English.
- Added explicit locale provenance runtime behavior: machine-provenance locales emit a deterministic operator notice, while JSON machine-mode outputs remain locale-neutral and stable across locale selection.