From the open engine’s docs (traza 0.28.0). View source Engine routes reach a Traza Cloud project through the gateway.

Backup and restore

Backup is pin, verify, copy — with the server running. Restore is install. Both are the same mechanism seen from two ends: a generation, which is one immutable, self-describing state of the store.

What a generation is

Query-visible state used to live in several independent recovery domains — the write-ahead log and write buffer, segments, annotations.jsonl, payloads/ — each with its own durability rule and its own idea of “now”. Nothing named a state they all agreed on, which is why backup, export, retention and deletion were four mechanisms instead of one.

A generation is that agreed state: a manifest listing every load-bearing file with its SHA-256 digest, plus the log position (folded_through) at which the manifest’s contents end and replay begins. CURRENT names the live one, and moving it — a staged rename made durable by a directory fsync — is the single commit point for a checkpoint, a restore, or a published deletion.

data/
  LOCK
  CURRENT                    names the live generation
  wal.log                    frames stamped with the generation they belong to
  segment-*.seg              the working set: segments,
  annotations.jsonl          the annotation log,
  payloads/                  and offloaded payload bytes
  generations/<id>/state-manifest.json
  ledger.jsonl               the hash chain over published generations (outside every generation)
  pins/<label>/              a pin: hard links to one manifest's files
  credentials.jsonl          runtime credential digests  -- outside generations
  limits.json                tenant limits               -- outside generations
  usage.json                 tenant usage counters       -- outside generations

Existing data directories are adopted at first open — nothing moves, and generation one is published over the files already there. It is one-way: an adopted directory is not readable by a pre-generation binary.

Taking a backup

Against a running server, with ingest continuing throughout (with auth on, this needs an unbound admin credential):

curl -X POST http://localhost:8080/v1/backups/nightly

{"backup":"nightly","generation":42,
 "path":"/var/lib/traza/pins/nightly","verified":true}

That checkpoints, hard-links the manifested files into pins/nightly, and verifies every digest before reporting success. Hard links share inodes, so the pin costs almost no disk and holds its bytes even after compaction unlinks the originals — which is what lets the copy proceed at its own pace:

One exception to “almost no disk”, inherited from every format migration (segment-format.md — the v6 → v7 upgrade did this, and the v7 → v8 upgrade does it again): a pin that pre-dates a migration is rewritten by it, file by file, into a full independent copy of its generation — rewriting a pinned file allocates a new inode, and the migrator does not re-link identical outputs. Pins taken after the migration (on a current-format store) are cheap again. When the disk cannot afford one full copy per pre-existing pin, release the pins before upgrading and re-take them after.

cp -a /var/lib/traza/pins/nightly /backups/traza-$(date +%F)
curl -X POST http://localhost:8080/v1/backups/nightly/release

Release it when the copy is done. A pin left in place keeps its generation’s bytes on disk, so an unreleased pin is a slow disk leak, not a correctness problem.

The copy carries spans, annotations, and payload bytes together. That is the part a span export never could: an export pinned spans and nothing else, so a dataset export meant only part of a session’s state.

A pin also carries ledger.jsonl: the store’s ledger as of the pin, ending with the checkpoint entry whose manifest_sha256 is the SHA-256 of the pin’s state-manifest.json — an erasure or anchor appended while the pin was being built is left out. It is evidence that travels with the backup, not state: it is not manifested, pin verification ignores it, and a restore does not install it — the restored store keeps the ledger already in its data directory, or starts a new chain in an empty one, and ledgers the installed generation either way. If the ledger could not record the pinned generation (a failed append, which GET /v1/ledger/verify reports), the pin is still taken and carries no ledger.jsonl.

Restoring a pin taken before an erasure undoes that erasure: the subject’s data comes back with the pre-erasure tombstone log. The ledger still records the erasure, and its verification names the entry until the erasure is applied again.

Verifying

curl http://localhost:8080/v1/verify

{"generation":42,"intact":true,"problems":[]}

Every file in the live generation’s manifest is re-read and re-digested. Problems are reported by name — segment-…seg: digest mismatch, payloads/ab/cd.bin: missing — rather than as one boolean, because knowing which file is damaged is what decides whether to restore.

This is what lets recovery distinguish “damage I may safely ignore” from “damage that changes what the store contains” by asking rather than inferring it from whether parsing happened to succeed.

Restoring

Restore is offline: it replaces the working set wholesale, which a live store cannot tolerate. Point a server at the backup with --restore:

traza-server --data-dir /var/lib/traza --restore /backups/traza-2026-08-10

The backup is verified before anything is swapped, and the swap commits at one CURRENT rename, so a failed or interrupted restore leaves the prior store rather than a blend. The server then serves the restored generation normally. The library equivalent is Store::restore.

A restored store starts with an empty write-ahead log: its state is entirely in the installed files, and any prior log belonged to a different lineage.

What a backup does not contain. credentials.jsonl (runtime credential digests), limits.json (tenant limits) and usage.json (usage counters) are outside generations, so no pin carries them and --restore leaves the live copies untouched: a restore never resurrects a revoked credential, rolls back a tenant’s limits, or rewinds its usage. Restoring into an empty directory therefore starts with no runtime credentials — open on loopback, unless TRAZA_TOKENS is set.

Checkpoints

A checkpoint publishes a generation. The server takes one every five minutes, pin takes one implicitly, and you can ask for one:

curl -X POST http://localhost:8080/v1/checkpoint

Nothing depends on the cadence for correctness — recovery excludes folded frames by their stamp whether or not a checkpoint has run recently — but each one moves folded_through forward, which bounds what a restart replays.

Checkpoints are cheap by construction: segments are immutable, so a checkpoint carries their digests over from the previous manifest and hashes only what was written since.

A checkpoint is never a side effect of a primitive. Expiry deletes from every domain it touches and stops there; it does not seal your write buffer to publish a manifest. The deletion is durable when its domains are durable, and published by the next checkpoint — within one maintenance interval, or immediately when a backup asks.

What a deletion means here

Expiring a span removes it from the buffer, rewrites the log so a restart cannot replay it, rewrites or retires the segments holding it, and sweeps the annotations and payload files that only it referenced. The next checkpoint then publishes a generation whose manifest no longer names those bytes.

The frames a checkpoint folds are excluded from replay by their stamp, not by having been physically removed. That is what makes reclaiming them housekeeping rather than correctness: a crash between the CURRENT rename and the log roll-over leaves a log still holding frames the new generation contains, and recovery discards them because they are at or before folded_through. Without that rule, a crash in that window would replay a deletion back into existence.

What is not covered

Off-host copying is yours. Traza will not write outside its own data directory, so pins/<label> is a path to copy, not a destination to configure.
A pin is not an archive. It shares inodes with the live store, so it survives compaction but not the disk.
Restore is whole-store. There is no partial or point-in-time restore within a generation; the granularity is the generation.