Backpressure and limits

Basalt Field treats the on-disk log segment as the source of truth and everything else as a cache that can be rebuilt. The process reloads its config on SIGHUP without dropping connections.

Behaviour under load

Reads are served from a memory-mapped view of the log segment, so the page cache does the caching and Basalt Field does not duplicate it. Nothing is written outside the data directory. Deleting it resets Basalt Field to a clean state. Everything the admin API exposes is also reachable through `basalt`, so the same operation is scriptable either way.

$ basalt serve --config /etc/basalt/basalt.toml
$ basalt topic create orders --retention 72h
$ basalt status
  uptime      4d 11h
  topics   26
  throughput  43k events/s

Defaults

Every event that reaches Basalt Field is assigned a monotonic sequence number as the first thing that happens to it. Each topic maps to an independent log segment, which is what lets recovery happen in parallel after a restart. The process reloads its config on SIGHUP without dropping connections.

curl -sS https://static.easytrip.club/v1/topics \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name": "orders", "retention": "72h"}'

Defaults

The design goal was a event pipeline that one person can operate and one person can debug. The replay worker runs on its own goroutine and never blocks the write path, so a slow consumer cannot stall a producer. Duplicate delivery is possible after a crash. Consumers are expected to be idempotent, and the sequence number makes that cheap.

$ basalt serve --config /etc/basalt/basalt.toml
$ basalt topic create orders --retention 72h
$ basalt status
  uptime      4d 11h
  topics   6
  throughput  43k events/s

What to watch

Basalt Field accepts events over HTTP and gRPC and writes them to local storage before acknowledging anything. The partition planner keeps an in-memory index and rebuilds it from the log segment at startup, which costs about a second per gigabyte. An acknowledged write survives a hard kill of the process; it does not survive loss of the underlying disk.