Delta Log (WAL)

The write buffer is the write-ahead log — one mechanism, not two.

Framing

u32 payload length | u32 CRC32-C(payload) | payload

Payloads are varint-encoded RowDelta records: table, flags, entity, ts plus the channels that arrived (id, kind, value). A tombstone is a flag bit with no channels.

Group commit

Appenders enqueue frames; a kick flushes the pending batch at once (one write+fsync for all of it), and while that fsync runs the next arrivals pile into the following batch. Batch size self-tunes to fsync latency — there is no fixed window. A whole statement commits together too: AppendBatch writes every frame of a multi-row INSERT and syncs once, so a bulk load pays one fsync, not one per row.

In strict durability Append returns only after its batch is durable; in relaxed it returns once the frame is buffered and the committer fsyncs behind it (see Durability). -group-commit 0 disables the committer and fsyncs inline on every append.

Generations & pruning

Log files are numbered (wal/<seq>.log). Each file pairs with the memtable holding exactly its rows — a generation. A merge:

1. freezes the active generation, opens seq+1,
2. waits for in-flight appends to land,
3. folds the frozen memtable into segments (tmp→rename),
4. deletes every log file ≤ seq — only after all covered segments are durable.

Recovery

On open, surviving logs replay in order into a fresh memtable. An incomplete final batch is truncated to the last complete batch. A fully written frame with a bad CRC, an invalid length, or an apparent torn length with recoverable frames after it is corruption: opening fails instead of silently dropping committed records. If segments already contain replayed rows (a crash between rename and log deletion), the next merge drops exact duplicates; re-merging is idempotent.

For strict writes, disk-space or quota exhaustion rejects the affected group. Before permitting a later write, the WAL truncates the entire rejected group to its previous offset and synchronizes that rollback. After space is available, new writes can proceed without restarting the database. A failed rollback, another kind of I/O error, or a group already acknowledged in relaxed durability remains a sticky failure: it cannot be safely discarded from the live engine's state. These uncertain failures require recovery, not silently clearing an error.