Skip to main content

Data Model

There are three table families, all riding the same storage engine: timeseries (this page — the observation streams), rows (current-state metadata, one live row per engine-minted id) and kv (key→value state, one value per user key). The families exist so the time-axis machinery below never leaks into metadata, and metadata verbs never leak into streams.

Tables, entities, channels​

A timeseries table is a set of channels (columns) observed per entity (device) over time:

  • entity — string device/row key, implicit on every timeseries table
  • ts — Unix milliseconds, implicit on every timeseries table
  • every other column is a channel with a declared type and fill policy

The table key is (entity, ts): it gives ordering, fragment coalescing and scan locality in one stroke.

Because every channel is an independent columnar stream, schema evolution is a catalog edit, not a table rewrite — see ALTER TABLE.

The frozen type system​

Channels are f64 or string. Nothing else, ever. Booleans are 0/1 over f64 (the catalog can label a channel BOOL so the interfaces validate 0/1 and render true/false), integers are whole f64s (the INT label validates whole numbers and renders real integers — exact up to 2⁵³), enums are strings (the engine dictionary-compresses them), binary blobs live in external object storage as reference strings. This is a load-bearing constraint: the core stays small because the answer to every new-type request is code, not engine.

Sparse by construction​

Devices send fragments. Whatever channels a fragment carries get appended to those channels' streams; the others receive nothing. There is no "row waiting to be completed" — a row is something reads reconstruct, not something storage keeps. See Forward-Fill.

Mutability classes​

KindDeclared withBehavior
IoT data (default)—immutable, append-only; duplicates allowed; retention applies
MetadataWITH (MUTABLE)last-write-wins per entity; DELETE supported

Mutable tables never mutate in place: an update is a versioned append (same entity, newer ts), a delete is a tombstone. The merge worker resolves both — the newest version survives, tombstoned entities vanish. The API feels like a document store; the files behave like an LSM tree.

CREATE TABLE devices (name TEXT, fw DOUBLE) WITH (MUTABLE);
INSERT INTO devices (entity, ts, name) VALUES ('d1', 1, 'first');
INSERT INTO devices (entity, ts, name) VALUES ('d1', 2, 'renamed');  -- update
DELETE FROM devices WHERE entity = 'd1';                             -- tombstone