The Timeseries Sync Feed
Rows tables are born with a change feed — x_rev
rides every write. A timeseries table can opt in to the same
contract: every ingested fragment (SQL INSERT and ILP alike) is
stamped with a hidden monotone sequence, x_seq, and a client can
drain exactly what arrived since it last looked — resumable after a
crash, never skipping a concurrent write. This is the door the
rule engine's TableTrigger consumes;
any client that wants a replica, a mirror, or an audit trail can speak
it directly.
CREATE TABLE sensor (p00 DOUBLE, p01 DOUBLE) WITH (SYNC);
-- or later, on a live table:
ALTER TABLE sensor SET SYNC; -- and SET SYNC OFF to stop stamping
Off by default — a table that never feeds a runner pays nothing.
The pull
SELECT * FROM sensor WHERE x_seq > 41250 LIMIT 500;
-- resumable form, guarded against rebuilds:
SELECT * FROM sensor WHERE x_epoch = 3 AND x_seq > 41250 LIMIT 500;
- Pages come back in ascending
x_seq— arrival order, not time order: late data lands in the feed where it arrived, so a downstream consumer never misses a backfilled reading. LIMITis required — the feed pages, it does not stream unbounded.SELECT *in a drain gainsx_seqas the last column (the puller needs its cursor); name columns explicitly to choose your own shape.- Each row is the coalesced row at the fragment's
(entity, ts), delivered at least once per fragment — re-applying a page is idempotent by construction, so "apply, advance the cursor to the page's maxx_seq, repeat" is the whole client. - A concurrent slow writer is never skipped: the feed stops below the oldest in-flight write and the next pull picks it up (the same watermark rule as the rows feed).
The handshake
SHOW SYNC FROM sensor;
-- enabled | epoch | seq | watermark
-- true | 3 | 41250| 41250
seq is the write-sequence high-water; watermark is the highest seq
a pull may answer right now. A resuming client compares its stored
(epoch, cursor) and drains — or resyncs.
Epochs: rebuilds are loud
TRUNCATE (and re-enabling sync) bumps the table's epoch. A cursor
that names a stale epoch is refused with resync in full — a rebuilt
table can never silently swallow the gap between an old cursor and its
new life. The sequence itself never runs backwards, ever: a crash burns
a small gap in x_seq and re-mints nothing (gaps are fine — the cursor
is an order, not a count).
Two more refusals, same spirit: a cursor beyond the table's write sequence (a dropped-and-rebuilt table) and a drain on a table with sync off. Errors, never empty silence.
Bootstrap
Fragments written before sync was enabled carry no x_seq and are
invisible to the feed. Starting a mirror on a table with history means:
copy once by time range, then drain from (epoch, 0). Creating the
table WITH (SYNC) from day one makes drain-from-zero the complete
story.
x_seq and x_epoch are engine-owned (the x_ prefix is reserved):
never writable, absent from SELECT * outside a drain and from
SHOW COLUMNS.