atproto utils for zig zat.dev

Add optional FirehoseClient checkpoint and batching adapters #9

Open zzstoatzz.io
Open opened by zzstoatzz.io

Remaining work #

Provide small, optional adapters for durable checkpoints and bounded event batching around FirehoseClient, driven by a concrete consumer. Keep the existing handler API usable without either adapter.

  • Let the application supply checkpoint storage and its durability policy; do not choose a database or persistence backend for it.
  • Advance a durable checkpoint only after the consumer accepts the corresponding work. Define replay behavior after processing or checkpoint failures.
  • If batching is needed, support bounded batch size and a maximum wait, with explicit cancellation and shutdown behavior.
  • Verify restart/replay, partial-batch failures, checkpoint failures, and buffer bounds without promising exactly-once delivery.

Already available #

The client accepts a starting cursor, tracks progress in memory, resumes on reconnect, and exposes onConnect / onReconnect. Its fallible onRawFrame hook advances the cursor only after downstream acceptance. These mechanisms do not provide durable storage or batching by themselves.

Scope #

Jetstream-specific checkpoint and batching work belongs in the separate Jetstream SDK. This issue covers the remaining raw-firehose adapters in zat; it does not request a new subscription framework.

Sign up or Sign in to add to the discussion.