very fast at protocol indexer with flexible filtering, xrpc queries, cursor-backed event stream, and more, built on fjall
rust
fjall
at-protocol
atproto
indexer
[docs] move title headers into title frontmatter
+80
-40
+26
-22
docs/README.md
+26
-22
docs/README.md
···
1
-
`hydrant` is an AT Protocol indexer built on the `fjall` database. it's built to be flexible, supporting both full-network indexing and filtered indexing (e.g., by DID), allowing querying with XRPCs (not only `com.atproto.*`!), providing an ordered event stream, etc. oh and it can also act as a relay!
2
-
3
-
you can see [random.wisp.place](https://tangled.org/did:plc:dfl62fgb7wtjj3fcbb72naae/random.wisp.place) (standalone binary using http API) or the [statusphere example](https://tangled.org/did:plc:dfl62fgb7wtjj3fcbb72naae/hydrant/blob/main/examples/statusphere.rs) (hydrant-as-library) for examples. for rust docs look at https://hydrant.klbr.net/ for now.
4
-
5
-
**WARNING: *the db format is only partially stable.*** we provide migrations in hydrant itself, so nothing should go wrong! you should still probably keep backups just in case!
6
-
7
-
## what's here
8
-
9
-
- [getting started](getting-started.md): building, running, reverse proxying
10
-
- [configuration](configuration.md): all environment variables
11
-
- [build features](build-features.md): optional cargo features (`relay`, `backlinks`, etc.)
12
-
- [concepts](concepts/README.md): how the stream works, relay comparison, multi-relay support
13
-
- [rest api](api/README.md): management API reference
14
-
- [xrpc](xrpc/README.md): data access via XRPC
15
-
16
-
## quick start
17
-
18
-
```bash
19
-
cargo build --release
20
-
export HYDRANT_DATABASE_PATH=./hydrant.db
21
-
./target/release/hydrant
22
-
```
1
+
---
2
+
title: hydrant
3
+
---
4
+
5
+
`hydrant` is an AT Protocol indexer built on the `fjall` database. it's built to be flexible, supporting both full-network indexing and filtered indexing (e.g., by DID), allowing querying with XRPCs (not only `com.atproto.*`!), providing an ordered event stream, etc. oh and it can also act as a relay!
6
+
7
+
you can see [random.wisp.place](https://tangled.org/did:plc:dfl62fgb7wtjj3fcbb72naae/random.wisp.place) (standalone binary using http API) or the [statusphere example](https://tangled.org/did:plc:dfl62fgb7wtjj3fcbb72naae/hydrant/blob/main/examples/statusphere.rs) (hydrant-as-library) for examples. for rust docs look at https://hydrant.klbr.net/ for now.
8
+
9
+
**WARNING: *the db format is only partially stable.*** we provide migrations in hydrant itself, so nothing should go wrong! you should still probably keep backups just in case!
10
+
11
+
## what's here
12
+
13
+
- [getting started](getting-started.md): building, running, reverse proxying
14
+
- [configuration](configuration.md): all environment variables
15
+
- [build features](build-features.md): optional cargo features (`relay`, `backlinks`, etc.)
16
+
- [concepts](concepts/README.md): how the stream works, relay comparison, multi-relay support
17
+
- [rest api](api/README.md): management API reference
18
+
- [xrpc](xrpc/README.md): data access via XRPC
19
+
20
+
## quick start
21
+
22
+
```bash
23
+
cargo build --release
24
+
export HYDRANT_DATABASE_PATH=./hydrant.db
25
+
./target/release/hydrant
26
+
```
+3
-1
docs/api/README.md
+3
-1
docs/api/README.md
+3
-1
docs/api/crawler.md
+3
-1
docs/api/crawler.md
+3
-1
docs/api/database.md
+3
-1
docs/api/database.md
···
1
-
# database operations
1
+
---
2
+
title: database operations
3
+
---
2
4
3
5
- `POST /db/train`: train zstd compression dictionaries for the `repos`, `blocks`, and `events` keyspaces. dictionaries are written to disk; a restart is required to apply them. the crawler, firehose, and backfill worker are paused for the duration and restored on completion.
4
6
- `POST /db/compact`: trigger a full major compaction of all database keyspaces in parallel. the crawler, firehose, and backfill worker are paused for the duration and restored on completion.
+3
-1
docs/api/filter.md
+3
-1
docs/api/filter.md
+3
-1
docs/api/firehose.md
+3
-1
docs/api/firehose.md
+3
-1
docs/api/ingestion.md
+3
-1
docs/api/ingestion.md
+3
-1
docs/api/pds.md
+3
-1
docs/api/pds.md
···
1
-
# PDS management
1
+
---
2
+
title: PDS management
3
+
---
2
4
3
5
hydrant rate-limits firehose events per PDS. each PDS is assigned to a named rate tier that controls how aggressively hydrant limits events from it. two built-in tiers are always present: `default` (conservative limits for unknown operators) and `trusted` (higher limits for well-behaved operators). additional tiers can be defined via `RATE_TIERS`.
4
6
+3
-1
docs/api/repos.md
+3
-1
docs/api/repos.md
+3
-1
docs/build-features.md
+3
-1
docs/build-features.md
+3
-1
docs/concepts/README.md
+3
-1
docs/concepts/README.md
+3
-1
docs/concepts/relay.md
+3
-1
docs/concepts/relay.md
+3
-1
docs/concepts/vs-tap.md
+3
-1
docs/concepts/vs-tap.md
···
1
-
# hydrant vs tap
1
+
---
2
+
title: hydrant vs tap
3
+
---
2
4
3
5
while [`tap`](https://github.com/bluesky-social/indigo/tree/main/cmd/tap) is designed as a firehose consumer and simply just propagates events while handling sync, `hydrant` is flexible, it allows you to directly query the database for records, and it also provides an ordered view of events, allowing the use of a cursor to fetch events from a specific point. it can act as both an indexer or an ephemeral view of some window of events.
4
6
+3
-1
docs/configuration.md
+3
-1
docs/configuration.md
+3
-1
docs/getting-started.md
+3
-1
docs/getting-started.md
+3
-1
docs/xrpc/README.md
+3
-1
docs/xrpc/README.md
+3
-1
docs/xrpc/atproto.md
+3
-1
docs/xrpc/atproto.md
+3
-1
docs/xrpc/backlinks.md
+3
-1
docs/xrpc/backlinks.md