Identities for entities did.bot

pds: a record that names many DIDs costs seconds of CPU before any limit, and stalls every route #232

Closed permadeath.com
Closed opened by permadeath.com

What happens. Before every write, in every collection, the server collects the DIDs a record names. It removes repeats by comparing each new DID against every DID kept so far, so the cost grows with the square of the count. The size check on that list comes only after the list is fully built. A 2 MiB record holding about 60,000 distinct DIDs takes about 8 seconds of CPU per pass. The first pass runs on an async worker, and the write path runs a second pass later.

How to reproduce.

  • The floor's declared-AI-preference built-in applies to every collection (crates/didbot-policy-records/src/builtin.rs:107-114).
  • write_record calls state.lookups.prefetch(...) on the request task, before spawn_blocking (crates/didbot-serve/src/routes/repo.rs:198).
  • wanted collects every named DID (crates/didbot-serve/src/lookups.rs:105-110) and only then refuses when named.len() > checked (lookups.rs:111-116).
  • The dedupe is if !found.iter().any(|held| held == did) (crates/didbot-policy-records/src/named.rs:58).
  • A local release-mode test ran named_accounts on a 2,097,134-byte record holding 59,917 distinct did:plc strings. It took 8.3 s.

Why it matters. Any token that can create one record is enough, including a third-party app scoped to one collection. Two such requests at once freeze a 2-vCPU host's async workers. Every route stalls, /health and the firehose included. The 30 s request timeout cannot interrupt CPU-bound work, and nothing charges the caller first.

Fixed in #1245. The walk keeps a HashSet of DIDs seen and stops at the first account past the check limit, and the mention-handle walk only runs once the count passes. A record naming 50,000 accounts is refused in milliseconds; with the old code, that write plus one other took 15.6 s.

Sign up or Sign in to add to the discussion.