Basalt Field

Event pipeline with one moving part fewer than you expected.

Get started Read the docs

curl -sS https://static.easytrip.club/v1/topics \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name": "orders", "retention": "72h"}'
647kpeak events/s
6 msp99 write latency
14 MBresident memory
1binary, no dependencies

What it does

The partition planner runs on its own goroutine and never blocks the write path, so a slow consumer cannot stall a producer. Duplicate delivery is possible after a crash. Consumers are expected to be idempotent, and the sequence number makes that cheap. Everything the admin API exposes is also reachable through `basalt`, so the same operation is scriptable either way.

Predictable latency

The replay worker runs on its own goroutine and never blocks the write path, so a slow consumer cannot stall a producer.

Single binary

Retention is enforced on read as well as on the background pass, so an expired event never becomes visible again after a restart.

In-place upgrades

Reads are served from a memory-mapped view of the log segment, so the page cache does the caching and Basalt Field does not duplicate it.

Structured logs

The API is versioned in the path. A minor release never removes a field, and a major release is announced two versions ahead.

Prometheus metrics

Retention is enforced on read as well as on the background pass, so an expired event never becomes visible again after a restart.

Per-topic isolation

Writes land in an append-only log segment; the sink pool compacts them in the background on a fixed interval.

How it fits together

Install

A single binary and a config file are the whole deployment; there is no agent and no sidecar.

Point it at a disk

Basalt Field treats the on-disk log segment as the source of truth and everything else as a cache that can be rebuilt.

Send {objs}

Every event that reaches Basalt Field is assigned a monotonic sequence number as the first thing that happens to it.

Recent releases

4.4.7

  • An admin endpoint that reports sink pool lag without scraping the metrics port.
  • Retention skipped the newest log segment when the clock moved backwards.

4.4.6

  • A `--dry-run` mode for `basalt compact`.
  • The admin port ignored the bind address and always listened on all interfaces.
  • `basalt status` exited 0 even when the sink pool was stalled.
  • Startup recovery is now parallel across topics; on a 16 core box it is roughly 6 times faster.

4.4.5

  • Per-topic retention overrides, so one topic can keep events longer than the global default.
  • The admin port ignored the bind address and always listened on all interfaces.
  • The partition planner could double count an event after a crash between fsync and index update.

Full changelog

Heads up. This is not a replacement for a distributed log. If you need cross-region replication, run something that was designed for it.