very fast at protocol indexer with flexible filtering, xrpc queries, cursor-backed event stream, and more, built on fjall
rust
fjall
at-protocol
atproto
indexer
[docs] document pds ratelimits / tiers management
+44
-1
+44
-1
README.md
+44
-1
README.md
···
3
3
-> [hydrant](#hydrant)</br>
4
4
-> [vs tap](#vs-tap) | [stream](#stream-behavior) | [multi-relay](#multiple-relay-support) | [crawler sources](#crawler-sources)</br>
5
5
-> [configuration](#configuration) | [build features](#build-features)</br>
6
-
-> [rest api](#rest-api) | [filter](#filter-management) | [ingestion](#ingestion-control) | [crawler](#crawler-management) | [firehose](#firehose-management) | [repos](#repository-management)</br>
6
+
-> [rest api](#rest-api) | [filter](#filter-management) | [ingestion](#ingestion-control) | [crawler](#crawler-management) | [firehose](#firehose-management) | [pds](#pds-management) | [repos](#repository-management)</br>
7
7
-> [xrpc api](#data-access-xrpc) | [backlinks](#bluemicrocosmlinks) | [identity](#bluemicrocosmidentity) | [atproto](#comatproto) | [custom](#systemsgazehydrant)
8
8
9
9
# hydrant
···
141
141
| `ENABLE_CRAWLER` | `true` if full network or crawler sources are configured, `false` otherwise | whether to actively query the network for unknown repositories. |
142
142
| `CRAWLER_MAX_PENDING_REPOS` | `2000` | max pending repos for crawler. |
143
143
| `CRAWLER_RESUME_PENDING_REPOS` | `1000` | resume threshold for crawler pending repos. |
144
+
| `TRUSTED_HOSTS` | | comma-separated list of PDS hostnames to pre-assign to the `trusted` rate tier at startup. hosts not listed here use the `default` tier unless assigned via the API. |
145
+
| `RATE_TIERS` | | comma-separated list of named rate tier definitions in `name:base/mul/hourly/daily` format (e.g. `trusted:5000/10.0/18000000/432000000`). built-in tiers (`default`, `trusted`) are always present and can be overridden. |
144
146
145
147
## build features
146
148
···
284
286
- returns `200 OK` if the relay was found and removed, `404 Not Found` otherwise.
285
287
- `DELETE /firehose/cursors`: reset the stored cursor for a given firehose relay URL. body: `{ "key": "..." }`
286
288
where key is a URL. causes the next firehose connection to restart from the beginning.
289
+
290
+
### PDS management
291
+
292
+
<small>[<- back to toc](#table-of-contents)</small>
293
+
294
+
hydrant rate-limits firehose events per PDS. each PDS is assigned to a named
295
+
rate tier that controls how aggressively hydrant limits events from it. two
296
+
built-in tiers are always present: `default` (conservative limits for unknown
297
+
operators) and `trusted` (higher limits for well-behaved operators). additional
298
+
tiers can be defined via `RATE_TIERS`.
299
+
300
+
the per-second limit scales with the number of active accounts on the PDS:
301
+
`max(per_second_base, accounts × per_second_account_mul)`.
302
+
303
+
- `GET /pds/tiers`: list all current tier assignments alongside the available
304
+
tier definitions.
305
+
- returns `{ "assignments": [{ "host": string, "tier": string }], "rate_tiers": { <name>: { "per_second_base": int, "per_second_account_mul": float, "per_hour": int, "per_day": int } } }`.
306
+
- `assignments` only contains PDSes with an explicit assignment; any PDS not
307
+
listed uses the `default` tier.
308
+
- `PUT /pds/tiers`: assign a PDS to a named rate tier.
309
+
- body: `{ "host": string, "tier": string }`.
310
+
- `host` is the PDS hostname (e.g. `pds.example.com`).
311
+
- `tier` must be one of the configured tier names. returns `400` if unknown.
312
+
- assignments are persisted to the database and survive restarts.
313
+
- re-assigning the same host updates the tier in place without creating a duplicate.
314
+
- `DELETE /pds/tiers`: remove an explicit tier assignment for a PDS, reverting
315
+
it to the `default` tier.
316
+
- body: `{ "host": string }`.
317
+
- returns `200` even if no assignment existed.
318
+
- `GET /pds/rate-tiers`: list the available rate tier definitions.
319
+
- returns a map of tier name to `{ "per_second_base", "per_second_account_mul", "per_hour", "per_day" }`.
320
+
321
+
hosts listed in `TRUSTED_HOSTS` are seeded as `trusted` at startup, but only
322
+
when no database assignment already exists for that host — DB entries always win.
323
+
the seed is not written to the database, so it is re-applied on every restart.
324
+
consequences: if you remove a host from `TRUSTED_HOSTS` and it has no DB entry,
325
+
it will revert to `default` on the next restart. if you explicitly assign a host
326
+
via the API (which writes to the DB), that assignment persists regardless of
327
+
`TRUSTED_HOSTS`. if you delete a host's DB assignment via the API while it is
328
+
still listed in `TRUSTED_HOSTS`, it will be re-seeded as `trusted` on the next
329
+
restart.
287
330
288
331
### repository management
289
332