Arxi Storage Architecture
Audience: Engineers changing persistence logic, query semantics, schema, or storage backends.
Table of Contents
- Executive Overview
- Store Trait Contracts
- In-Memory Backend
- SQLite Backend
- SQLite Schema and Indexing
- Chain Continuity Enforcement
- Query and Pagination Semantics
- Corruption and Error Handling
- File-by-File Cross Reference
Executive Overview
arxi-store provides the persistence abstraction and two concrete backends:
InMemoryStorefor deterministic testing and local execution,SqliteStorefor durable operation with WAL and transactional checks.
Both implement shared trait contracts for envelope, attachment, and segment operations. F:crates/arxi-store/src/lib.rs L11-L37 F:crates/arxi-store/src/lib.rs L49-L71
Store Trait Contracts
Storage contract is expressed as three async traits:
EnvelopeStore: append/get/query/chain-head/segment-envelopes,AttachmentStore: put/get/exists for content-addressed blobs,SegmentStore: create/seal/list/get metadata and genesis.
Core invariants include append-only behavior, chain continuity checks, and sealed-segment immutability. Segment creation is fail-closed: stores reject a new segment when another open segment already exists or when the segment ID already exists. F:crates/arxi-store/src/traits.rs L43-L101 F:crates/arxi-store/src/traits.rs L107-L138 F:crates/arxi-store/src/traits.rs L144-L206
In-Memory Backend
InMemoryStore uses lock-protected maps and vectors, preserving invariant checks
but without crash consistency guarantees.
F:crates/arxi-store/src/memory.rs L12-L16
Behavior highlights:
- validates segment state and chain continuity on append,
- maintains first/last envelope IDs and chain head updates,
- enforces single-open-segment and no-duplicate-segment-ID creation rules,
- supports deterministic segment listing by
created_at, - enforces deterministic envelope query ordering by
segment_idthen sequence.
F:crates/arxi-store/src/memory.rs L124-L180 F:crates/arxi-store/src/memory.rs L317-L407
SQLite Backend
SqliteStore routes all DB operations through a dedicated connection thread
(SqliteConnection) to preserve single-writer semantics safely across async
callers.
F:crates/arxi-store/src/sqlite/mod.rs L12-L22
F:crates/arxi-store/src/sqlite/connection.rs L12-L21
SqliteConnection::execute uses bounded sync_channel + oneshot channels to
bridge async code with sync rusqlite operations. Queue admission is explicit:
enqueue saturation fails closed with StoreError::ConcurrencyConflict rather
than allowing unbounded in-memory work growth.
F:crates/arxi-store/src/sqlite/connection.rs L42-L136
F:crates/arxi-store/src/sqlite/connection.rs L167-L222
SqliteStore additionally exposes typed metadata helpers over store_meta
(get_meta, set_meta, set_meta_batch) for deterministic runtime state such
as signer-key rotation idempotence markers.
F:crates/arxi-store/src/sqlite/mod.rs
SQLite Schema and Indexing
Initialization enables:
- WAL mode,
- foreign keys,
- schema metadata table,
- normalized
segments,envelopes,attachments, and junction tables, - indexes for deterministic envelope/segment query patterns.
F:crates/arxi-store/src/sqlite/schema.rs L111-L144 F:crates/arxi-store/src/sqlite/schema.rs L43-L105
Chain Continuity Enforcement
Both backends enforce append-time chain validation using expected chain-head recomputation and constant-time comparison.
In-memory append validation: F:crates/arxi-store/src/memory.rs L140-L158
SQLite append validation is transaction-scoped:
- verify segment exists and is open,
- recompute expected chain hash,
- reject mismatch,
- insert envelope + envelope-attachment rows,
- update segment metadata atomically.
F:crates/arxi-store/src/sqlite/envelope_ops.rs L42-L141
Query and Pagination Semantics
Envelope queries support multi-field filtering plus cursor continuation and limit.
SQLite query order is ORDER BY segment_id, sequence; segment-envelope queries
order by sequence ASC.
F:crates/arxi-store/src/sqlite/envelope_ops.rs L167-L280 F:crates/arxi-store/src/sqlite/envelope_ops.rs L306-L347
Segment listing supports status/time/id filters and orders by created_at ASC.
F:crates/arxi-store/src/sqlite/segment_ops.rs L148-L245
Corruption and Error Handling
StoreError explicitly differentiates integrity breaks, not-found cases,
concurrency conflicts, backend failures, and corruption detection.
F:crates/arxi-store/src/error.rs L36-L103
SQLite segment parsing treats invalid DB values as corruption (invalid UUID, unknown status, unknown hash algorithm, invalid timestamps). F:crates/arxi-store/src/sqlite/segment_ops.rs L336-L416
System tests now validate fail-closed behavior when persisted SQLite rows are tampered with invalid segment UUID/genesis data between CLI operations. F:system-tests/tests/suites/persistence.rs L375-L468
File-by-File Cross Reference
| Area | File | Notes |
|---|---|---|
| Storage contracts | crates/arxi-store/src/traits.rs | Canonical storage trait invariants. |
| Segment types | crates/arxi-store/src/types.rs | Segment metadata and filter shapes. |
| In-memory backend | crates/arxi-store/src/memory.rs | Deterministic map-based implementation. |
| SQLite facade | crates/arxi-store/src/sqlite/mod.rs | Store construction and backend composition. |
| SQLite metadata helpers | crates/arxi-store/src/sqlite/mod.rs | store_meta read/write helpers for runtime state persistence. |
| SQLite bridge | crates/arxi-store/src/sqlite/connection.rs | Dedicated thread async bridge. |
| SQLite schema | crates/arxi-store/src/sqlite/schema.rs | DDL, indexes, schema versioning. |
| Envelope SQL ops | crates/arxi-store/src/sqlite/envelope_ops.rs | Transactional append and filter query logic. |
| Segment SQL ops | crates/arxi-store/src/sqlite/segment_ops.rs | Segment lifecycle and metadata decoding. |
| Attachment SQL ops | crates/arxi-store/src/sqlite/attachment_ops.rs | Content-addressed put/get/exists semantics. |
| Error taxonomy | crates/arxi-store/src/error.rs | Typed storage failure model. |