Recorder Decision Gate Asset Core
Docs
Recorder Docs

Proof recording and tamper-evident evidence documentation.

Other product docs

Recorder System Architecture

Audience: Engineers working across Recorder crates or integrating Recorder with control-plane systems such as Decision Gate.

Table of Contents

  1. Executive Overview
  2. Layered Architecture
  3. End-to-End Data Flow
  4. Trust Boundaries and Security Posture
  5. Determinism Model
  6. Current Implementation Scope
  7. File-by-File Cross Reference

Executive Overview

Recorder is a data plane for deterministic evidence recording. It captures unsealed events at adapter boundaries, seals them into hash-chained envelopes, persists them in append-only segment histories, and exports portable bundles for offline verification.

The system is organized as a Rust workspace with strict crate boundaries: core contracts, envelope model, recorder orchestration, storage backends, an ergonomic Rust SDK facade, a first-party generated Python sidecar SDK, canonical sidecar config semantics, CLI operations, and a sidecar HTTP service runtime. F:Cargo.toml L9-L26

Layered Architecture

  1. Core contracts (recorder-core)
  2. Evidence model (recorder-envelope)
  3. Integration adapters (recorder-decision-gate-adapter, recorder-otel-adapter)
  4. Recording runtime (recorder)
  5. Persistence (recorder-store)
  6. Sidecar config authority (recorder-sidecar-config)
  7. Rust SDK facade (recorder-sdk)
  8. Generated sidecar SDK (recorder-client for Python)
  9. Operational surface (recorder-cli)
  10. Network service surface (recorder-sidecar)
  1. Projection contract generation (recorder-contract)

End-to-End Data Flow

Adapter / CLI input
    -> Sidecar mutating route validation (tenant_id + recorder_id + idempotency key)
    -> IngestGateway bounded queue admission
    -> IngestWriterRuntime writer-native sealing + single-tx micro-batch commit
    -> Stream/global commit index assignment + idempotency + projection update
    -> BundleBuilder resolves selector + attachment closure + manifest
    -> Verifier runs 7 phases offline
    -> EvidenceProvider serves bundles/envelope queries to control-plane bridge

Key flow entry points:

Trust Boundaries and Security Posture

Determinism Model

Determinism is achieved by combining:

  • canonical JCS encoding for hashable data,
  • deterministic ordering structures (BTreeMap, sorted vectors),
  • stable hash and selector logic,
  • replayable offline verification.

F:crates/recorder-envelope/src/encoding.rs L13-L31 F:crates/recorder/src/bundle_builder.rs L27-L30 F:crates/recorder/src/bundle_builder.rs L466-L477

Current Implementation Scope

Implemented now:

  • Full local recording lifecycle and bundle verification.
  • SQLite/in-memory store backends.
  • recorder-sdk single-crate facade for Rust consumers, including deterministic builder defaults and direct adapter/provider/store accessors.
  • recorder-client Python SDK with generated contract constants, typed sync/async sidecar clients, and unit/system tests.
  • CLI-driven operations and localized output.
  • CLI attachment-aware recording (record-with-attachments) with bounded file and inline attachment inputs.
  • CLI Decision Gate fixture ingest command path wired to production adapter implementation.
  • CLI OTel fixture ingest command path wired to production adapter implementation.
  • System-test harness with registry-driven coverage, including OpenClaw gateway/CLI mock-flow and Decision Gate runpack-flow adapter-ingest integration tests.
  • CLI expansion system-tests for recorder-id shape validation parity, attachment-recording fail-closed boundaries, auto-seal duration/combined lifecycle behavior, query JSON pagination/limit guardrails, and Decision Gate CLI ingest-fixture success/strict-fail paths.
  • Production Decision Gate adapter crate (crates/recorder-decision-gate-adapter) with deterministic MCP/runpack mapping, runpack-integrity strict/anomaly policy handling, and transcript redaction/bounds controls.
  • Production OTel adapter crate (crates/recorder-otel-adapter) with deterministic span/log mapping, strict identifier/time parsing, and bounded/redacted payload controls.
  • OpenClaw integration harness mapping policy with deterministic sensitive-field redaction and bounded payload handling for fixture-driven coupling tests.
  • Decision Gate integration system-tests wired to the production adapter crate, covering signed/unsigned lanes, root-hash and manifest-integrity mismatch fail-closed behavior, and deterministic replay/hash stability.
  • Sidecar Docker operator profile (docker/sidecar) and system-test coverage for Dockerfile/Compose hardening plus containerized probe/record/query flow.
  • Sidecar readiness semantics now support fail-closed dependency modes: storage-only readiness and storage+enterprise-control readiness.

Current performance/stress posture:

  • P2 stress/performance system-test expansion is now implemented with deterministic artifact-first lanes: performance::stress_concurrent_append_query_sqlite, performance::performance_bundle_materialization_smoke, and sidecar_perf::sidecar_perf_smoke_generates_gate_report (regression lane), plus sidecar_capacity::sidecar_capacity_sweep_generates_report (capacity lane).
  • CI now runs a profile matrix (test informational + release gated) on the pinned perf runner via scripts/ci/perf_matrix.py and enforces release thresholds/regression policy via scripts/ci/perf_gate.py and system-tests/perf_baselines/linux_x86_64_ci.json.
  • Sidecar ingest 10k target is defined only on the capacity lane metric max_sustainable_rps and is enforced only for pinned runner policy class (linux_x86_64_pinned); the regression lane remains a deterministic fixed workload anti-regression signal.
  • Sidecar perf throughput/latency lanes now run against persistent keep-alive HTTP client pools to avoid connection-churn distortion in write-path measurements.
  • Bundle-path metric bundle_build_envelopes_per_sec remains a separate core bundle materialization metric and is not an ingest-capacity proxy.

Recently closed system-level hardening items:

File-by-File Cross Reference

AreaFileNotes
Workspace boundariesCargo.tomlCrate membership and lint posture.
Core contractscrates/recorder-core/src/lib.rsShared domain primitives and traits.
Envelope modelcrates/recorder-envelope/src/lib.rsEvidence data model and contracts.
Recorder runtimecrates/recorder/src/engine.rsIngest, chain progression, segment lifecycle.
Rust SDK facadecrates/recorder-sdk/src/lib.rs, crates/recorder-sdk/src/builder.rsErgonomic builder and unified recorder/provider facade for Rust consumers.
Python SDK facadesdk/python/src/recorder_client/client.py, sdk/python/src/recorder_client/client_common.py, sdk/python/src/recorder_client/sync_client.py, sdk/python/src/recorder_client/async_client.py, sdk/python/src/recorder_client/models.py, sdk/python/src/recorder_client/errors.pyGenerated-contract-driven sync/async sidecar client surface with strict fail-closed validation and typed error mapping.
Bundle + verifiercrates/recorder/src/bundle_builder.rsSelector resolution and bundle materialization.
Bundle verificationcrates/recorder/src/verifier.rs7-phase offline verification.
Storage traitscrates/recorder-store/src/traits.rsPersistence contract invariants.
SQLite queue boundarycrates/recorder-store/src/sqlite/connection.rsBounded async-to-sync dispatch and saturation fail-closed behavior.
CLI surfacecrates/recorder-cli/src/main.rs, crates/recorder-cli/src/commands.rs, crates/recorder-cli/src/support.rsOperational command architecture split across dispatch, command handlers, and runtime/utility helpers.
Sidecar servicecrates/recorder-sidecar/src/lib.rs, crates/recorder-sidecar/src/server.rsHTTP runtime, middleware stack, and transport lifecycle.
Sidecar config/securitycrates/recorder-sidecar-config/src/config.rs, crates/recorder-sidecar-config/src/validation.rs, crates/recorder-sidecar/src/middleware/auth.rs, crates/recorder-sidecar/src/middleware/bounds.rs, crates/recorder-sidecar/src/ingest.rsFail-closed config validation and projection authority plus sidecar auth/bounds controls and single-writer ingest idempotency behavior.
Sidecar container packagingdocker/sidecar/Dockerfile, docker/sidecar/docker-compose.yml, docker/sidecar/config/sidecar.toml, crates/recorder-sidecar/src/bin/docker_bootstrap.rsFirst-party container build/runtime profile aligned to sidecar validation/security invariants, including secret-source token bootstrap into hardened tmpfs runtime path.
Contract generatorcrates/recorder-contract/src/lib.rs, crates/recorder-contract/src/sidecar_api.rs, crates/recorder-contract/src/sidecar_api/openapi.rs, crates/recorder-contract/src/sidecar_api/artifacts.rs, crates/recorder-contract/src/sidecar_api/specs/mod.rsDeterministic generated artifact authority + drift checks.
System-testssystem-tests/README.mdEnd-to-end validation contract.
Decision Gate adaptercrates/recorder-decision-gate-adapter/src/adapter.rsProduction mapping/redaction/integrity policy implementation for Decision Gate ingest.
OTel adaptercrates/recorder-otel-adapter/src/adapter.rsProduction mapping/redaction/bounds policy implementation for OTel JSON ingest.
OpenClaw integration testssystem-tests/tests/suites/integration_openclaw.rsFixture-driven adapter ingest verification for external workflow coupling.
OpenClaw integration architectureDocs/architecture/recorder_openclaw_integration_architecture.mdVersioned mapping and redaction/bounded payload policy contract.
Decision Gate integration testssystem-tests/tests/suites/integration_decision_gate.rsFixture-driven MCP runpack flow ingest verification for control-plane coupling.
Decision Gate integration architectureDocs/architecture/recorder_decision_gate_integration_architecture.mdVersioned MCP flow mapping, runpack integrity policy, and transcript redaction/bounds contract.
OTel integration architectureDocs/architecture/recorder_otel_integration_architecture.mdVersioned OTel JSON mapping contract, deterministic IDs/event types, and sidecar/CLI ingest wiring.