A diet Tap option for handling atproto sync

Add deletes for records that disappeared before a resync #21

Merged pds.dad
targeting tab from deleted fork:feature/resync-deletes
Merged opened by and assigned to pds.dad
targeting tab from deleted fork:feature/resync-deletes

A resync only emits creates and updates. When a record is deleted while its repo is not being followed live, for example while the repo waits in desynchronized or pending, the resync finds it missing from the repo but never says so, and consumers keep a record that no longer exists.

This makes the resync emit a delete for each stored record that is missing from the repo it just walked.

How #

  • While walking the MST, the resync now notes which stored records it found, including those it skips because their CID is unchanged.
  • After the walk, staleRecordEvents builds a delete event for every stored RepoRecord that was not found, at the repo's current rev, with live: false. They go through AddRecordEvents like any delete, which also removes the RepoRecord rows.
  • Records in collections outside the current filters are left alone. They are not walked, so their absence says nothing, and reporting them as deleted after a filter is narrowed would be wrong.
  • Events are sorted by collection and rkey, so the order is deterministic.

Testing #

  • TestStaleRecordEvents covers missing records, records still present, and records outside the filters.
  • go test ./cmd/tab/ and go vet ./cmd/tab/ pass.
  • Run against a real database: a stored record was planted for a tracked repo that does not have it, along with a matching row in a consumer's index. After a resync, tab emitted one delete at the repo's current rev, the consumer tombstoned that row, the repo's stored record count went from 18 to 17, and the repo's 17 real records were neither re-sent nor touched.

This pairs well with the /resync/:did route, since requesting a resync becomes a way to bring a repo fully up to date, deletes included.

Commits 0