Skip to main content

Durability & Backup

The delta log is the WAL​

There is no separate journal: the row-delta buffer itself is an on-disk append-only log. A write is framed (length | CRC32-C | payload), appended and fsync'ed — then acked.

Batch commit. A whole statement commits as one unit: a multi-row INSERT writes all its rows in a single write and a single fsync, so bulk loading costs one sync, not one per row. Independently, concurrent writers arriving together share a commit too — the committer flushes the pending batch immediately and, while that fsync runs, the next arrivals pile into the following batch. Batch size self-tunes to fsync latency instead of a fixed time window.

Durability modes​

Durability is a per-database setting, changed live and persisted:

ALTER DATABASE SET DURABILITY STRICT;   -- the default
ALTER DATABASE SET DURABILITY RELAXED;
SHOW DURABILITY;

Strict (default) is synchronous: a write is fsync'ed before its ack, so an acknowledged write is durable, full stop. This is the safe default — nothing acknowledged is ever lost, even to a kill -9.

Relaxed acknowledges a write once it is in the WAL buffer and memtable, and the committer fsyncs behind it — MongoDB's j:false with a journal commit interval. Because the committer keeps flushing continuously, the exposure window is about one fsync, not the full interval (a 100 ms backstop bounds it if the log goes idle). Relaxed trades that small window for a large throughput gain on single-row writes, where strict is fsync-bound. Choose it per database: strict for the tables you can't lose, relaxed for high-rate streams you can replay.

Crash recovery​

On open, the engine replays every surviving log file into a fresh memtable. A torn or corrupt tail — the write in flight when power died — is caught by CRC and truncated away.

  • Strict: everything acknowledged is intact by construction. A real kill -9 mid-load recovers every acked row.
  • Relaxed: the recovered log is a clean, in-order prefix — never corrupt, never a fabricated value, never more rows than were written. Only the un-synced tail (that ~one-fsync window) may be missing.

If the crash landed between a merge finishing its segments and deleting its logs, the replayed rows already live in a segment. The next merge deduplicates exact duplicates, so re-merging is idempotent.

CHECKPOINT and backups​

Copying a live database directory mid-write is unsupported. The contract is:

CHECKPOINT;

which forces the delta into columnar, seals the segments and drops a CHECKPOINT marker file. After it returns, a plain file copy is a consistent snapshot:

psql "..." -c 'CHECKPOINT;' && rsync -a /var/lib/xcon-db/dbs/acme/ backup:/acme/

Even without a checkpoint, a restored copy self-heals through log replay — the checkpoint just guarantees the copy is minimal and merge-complete.

Replication is deliberately outside the core: backup + re-route the database to another node.