Specification · 002-trade-ledger

Append-only trade ledger

The single source of truth: an immutable, gap-free log of every trade, unlimited in history, fast to append and complete to replay.

Status

Accepted

Spec ID 002-trade-ledger Parent

001-portfolio-analytics

Owner Brian Corbin · @briancorbinxyz Reviewers Brian Corbin · @briancorbinxyz Created 21 Jul 2026 Principlesv0.1.0 Smallest demoableAppend a trade, get a monotonic sequence number back; replay the log and get the exact same events in order. Read time

The ledger is the foundation the whole platform folds over (001). It appends immutable TradeEvents under a monotonic, gap-free sequence, never mutating or deleting history — corrections are compensating events. History is unlimited: no retention cap, cold events tiered to secondary storage while replay stays complete. It offers full and ranged replay, and periodic fold snapshots so consumers bound cold-start. The hot path is a single-writer append measured in microseconds; durability, gap-freedom, and bounded hot memory are the reliability envelope.

I Independent: Depends only on the umbrella's TradeEvent model. Knows nothing about positions, weights, or lots.
N Negotiable: Outcome (immutable, gap-free, unlimited, replayable, snapshottable), not a named storage engine or log library.
V Valuable: Unlocks SC-001 — a durable, replayable source of truth every analytic can trust.
E Estimable: Append + sequence + replay + snapshot are well-understood log primitives; the perf targets are measurable up front.
S Small: One entity, one writer, five functional requirements; tiering-engine choice is deferred.
T Testable: Gap-freedom, replay equality, and append latency are all mechanically checkable.

1 · Context

Per the umbrella, every analytic is a deterministic fold over one ledger. That makes the ledger's guarantees the platform's guarantees: if the log can lose an event, reorder one, or let a stale read tear across a snapshot boundary, every downstream number inherits the flaw. HFT ingest rates and unlimited retention put those guarantees under pressure that a naive append-to-a-list would not survive.

This slice owns the log and nothing else: sequencing, replay, snapshotting, unlimited retention with cold-tiering, and idempotent append. The math that reads the log lives in the sibling slices.

2 · User scenarios & testing

Each story MUST be independently testable.

US1 · Append and sequence P1

As an order-management producer, I want each accepted fill appended and stamped with the next sequence number, so downstream folds have a total order to replay.

Acceptance: appending N events yields sequence numbers 1..N with no gaps and no reordering; a prior event is never mutated or removed.

US2 · Complete replay under unlimited history P2

As an analytic bootstrapping from zero, I want to replay the entire ledger — including events long since tiered to cold storage — in append order, so a from-zero fold is always possible.

Acceptance: a full replay returns every event ever appended, in sequence order, whether hot or cold-tiered; a ranged replay [a, b] returns exactly that window.

US3 · Snapshot to bound cold-start P3

As an operator, I want to checkpoint a fold and restore from it, so restart time does not grow with lifetime trade count.

Acceptance: restoring from a snapshot at sequence N then replaying N+1.. equals a full replay from zero.

Edge cases

Duplicate delivery. An at-least-once producer re-sends an event with the same client key; the ledger appends it once (FR-005), not twice.
Cold read during hot append. A replay reaching into tiered storage must not block or tear against concurrent appends to the hot tail.
Crash between append and ack. An event acknowledged as durable is never lost; an un-acked append may be absent, but never leaves a sequence gap.

3 · Requirements

Conformance keywords (MUST, SHOULD, MAY) follow RFC 2119.

Functional

The ledger MUST be append-only and immutable: each TradeEvent receives the next value of a monotonic, gap-free sequence, and a persisted event MUST NOT be mutated or deleted. A correction MUST be expressed as a new compensating event.

History MUST be unlimited: the ledger MUST NOT impose a fixed retention cap. Cold events MAY be tiered to secondary storage, but replay MUST remain complete across hot and cold tiers.

The ledger MUST support deterministic replay — full and ranged [seqA, seqB] — returning events in append (sequence) order.

The ledger MUST support fold snapshots and restore at a sequence number, such that restore-then-replay equals full replay from zero (the umbrella's FR-003, realized here).

Append SHOULD be idempotent by an optional client event key, so an at-least-once producer that re-delivers does not double-book.

Non-functional

Single-writer append on the hot path MUST meet SLO-append.

in-process wall-clock to assign a sequence and append to the hot tail, pre-durability-ack (good ÷ valid appends)

Sustained ingest throughput MUST meet SLO-ingest so the ledger keeps up with a full HFT fill stream.

sustained single-writer append rate as a fraction of the 1e6/s floor (achieved ÷ floor), sampled per minute

Durability and integrity MUST meet SLO-integrity: no acknowledged event is lost and no sequence gap is introduced.

count of acknowledged-append losses or detected sequence gaps (defects ÷ 28d window; target zero)

Under unlimited history, hot-tier memory MUST stay bounded per SLO-hotset, independent of lifetime trade count.

resident hot-tier bytes as a fraction of the 2 GB ceiling, sampled at peak (hot bytes ÷ ceiling)

Out of scope (deferred)

Deriving net position or cost basis from the log — the ledger stores events, it does not fold them.
Lot construction and FIFO/LIFO relief over the event stream.
The concrete durable-storage and cold-tiering engine (mmap segment files, an embedded LSM, or a log service) — an implementation choice for the design.
Geo-replication and multi-writer sequencing — a later reliability slice; this spec assumes a single writer.

4 · Data model

The ledger persists the umbrella's TradeEvent and adds only the machinery to sequence, tier, and checkpoint it.

TradeEvent: As defined by 001; persisted verbatim, immutable, under an assigned seq.
Snapshot: { seq, foldId, opaqueState } — a checkpoint of one analytic's fold at a sequence, restorable to a state bit-identical to replaying [0, seq].
Tier boundary: The hot-tail / cold-store split. Transparent to replay: a ranged replay spanning the boundary returns a single ordered stream.

Invariant: sequence numbers are dense and monotonic. A gap is not an anomaly to log; it is a correctness fault under NFR-003.

5 · Success criteria

Appending N events then replaying returns exactly those N events, in order, with sequence 1..N and no gaps — including after some events have been cold-tiered.

Restoring a fold from a snapshot at sequence N and replaying N+1..M yields state bit-identical to a full replay of [0, M].

6 · Assumptions

A single logical writer owns the append path; multi-writer sequencing is deferred to TBD-cross-region-replication.

Secondary (cold) storage is durable and available; the ledger owns tiering policy, not the storage medium's own durability.

7 · Open questions

None outstanding — the append-only, unlimited-retention, snapshot-and-replay contract is fixed here; the concrete storage engine is explicitly deferred to the design (TBD-storage-engine) rather than left open.

8 · Conformance index

Auto-built from every <spec-requirement> in this document.

9 · Change log

21 Jul 2026Initial draft. Child of 001-portfolio-analytics: the append-only, immutable, gap-free ledger (FR-001) with unlimited history and cold-tiering (FR-002), deterministic full/ranged replay (FR-003), fold snapshot/restore (FR-004), and idempotent append (FR-005). Reliability envelope: append latency (NFR-001), ingest throughput (NFR-002), durability/gap-freedom (NFR-003), bounded hot memory (NFR-004).