Clone this repository
For self-hosted knots, clone URLs may differ based on your setup.
Download tar.gz
Set DuckDB memory_limit=512MB and threads=2 to prevent FTS shadow
table rebuilds from consuming all container RAM alongside V8.
Remove backfillChildTables() — was a one-time migration, child rows
are already populated during normal indexing and backfill.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace buffered res.arrayBuffer() with incremental stream parsing.
Each block is .slice()d into its own Uint8Array, eliminating the single
large external ArrayBuffer that V8 can't GC (213MB → 64MB external).
Also adds diff-based backfill via `since` parameter with fallback to
full import when the PDS has compacted history.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Retry timers in triggerAutoBackfill bypassed the concurrency check in
processMessage, allowing unbounded concurrent CAR downloads (23 observed
in production, 2GB RSS). Move the concurrency gate into
triggerAutoBackfill itself and use config.backfill.parallelism instead of
a hardcoded constant. Add dedup set to prevent re-schedule timer
accumulation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>