Webhooks for the AT Protocol
airglow.run
atproto
atprotocol
automation
webhook
docs: updates round
+31
-15
+20
-4
README.md
+20
-4
README.md
···
11
11
### For end users
12
12
13
13
1. Sign in to Airglow using [AT Protocol OAuth](https://atproto.com/specs/oauth).
14
-
2. Create an automation by choosing a lexicon to listen to (e.g. `sh.tangled.feed.star`, `site.standard.document`).
15
-
3. Add conditions to filter events — simple equality checks on record fields like the record owner or specific values in the data. The schema is known from the lexicon, so Airglow can present the available fields.
16
-
4. Add actions — deliver a webhook to a callback URL, or create a record on your PDS using a template.
14
+
2. Create an automation by choosing a lexicon to listen to (e.g. `sh.tangled.feed.star`, `site.standard.document`) and which operations to watch (`create`, `update`, `delete`).
15
+
3. Add conditions to filter events — match record fields using operators like `eq`, `startsWith`, `endsWith`, or `contains`. The `{{self}}` placeholder resolves to the automation owner's DID. The schema is known from the lexicon, so Airglow can present the available fields.
16
+
4. Optionally add fetch steps to retrieve records from a PDS at event time. Fetched data can then be referenced in action templates using `{{fetchName.record.field}}` placeholders.
17
+
5. Add actions — deliver a webhook to a callback URL, or create a record on your PDS using a template.
18
+
19
+
Automations can be created in **dry-run mode** — all logic (condition matching, fetches, template rendering) runs, but no side effects occur. Results are logged so you can verify behavior before going live.
17
20
18
21
Airglow verifies that webhook callback URLs actually support the selected lexicon before activating the automation (see [Callback endpoints](#callback-endpoints) below).
22
+
23
+
Each user has a **public profile** at `/u/<handle>` showing their automations and maintained lexicons. Individual automations can be viewed and duplicated by other users.
19
24
20
25
#### Data ownership
21
26
···
107
112
108
113
## Self-hosting
109
114
110
-
Airglow is designed to be easy to self-host. Instance operators can configure allowlists and blocklists on NSID domains to control which lexicons their instance handles. For example, a typical instance may want to block `app.bsky.*` or `app.bsky.feed.*` since those collections are very active and could overwhelm a small instance.
115
+
Airglow is designed to be easy to self-host. Configuration is done via environment variables (see `.env.example`):
116
+
117
+
| Variable | Purpose |
118
+
| --- | --- |
119
+
| `PUBLIC_URL` | Public-facing base URL of the instance |
120
+
| `DATABASE_PATH` | Path to the SQLite database file |
121
+
| `JETSTREAM_URL` | Jetstream WebSocket endpoint |
122
+
| `COOKIE_SECRET` | Secret for session cookies (min 32 chars) |
123
+
| `NSID_ALLOWLIST` | Comma-separated NSIDs to allow (empty = allow all) |
124
+
| `NSID_BLOCKLIST` | Comma-separated NSIDs to block (empty = block none) |
125
+
126
+
Instance operators can configure `NSID_ALLOWLIST` and `NSID_BLOCKLIST` to control which lexicons their instance handles. For example, a typical instance may want to block `app.bsky.*` or `app.bsky.feed.*` since those collections are very active and could overwhelm a small instance.
111
127
112
128
## References
113
129
+9
-9
docs/performance.md
+9
-9
docs/performance.md
···
2
2
3
3
## Current Architecture
4
4
5
-
Airglow maintains a **single WebSocket connection** to Jetstream, regardless of how many user subscriptions exist.
5
+
Airglow maintains a **single WebSocket connection** to Jetstream, regardless of how many user automations exist.
6
6
7
7
### How it works
8
8
9
-
1. **One WebSocket, deduplicated collections** — `JetstreamConsumer` is a singleton. On startup (and whenever subscriptions change), it loads all active subscriptions from the database and groups them by lexicon (collection). Only the **unique collection names** are sent as `wantedCollections` params to Jetstream. If 100 users subscribe to `app.bsky.feed.post`, Jetstream sends events for that collection once.
9
+
1. **One WebSocket, deduplicated collections** — `JetstreamConsumer` is a singleton. On startup (and whenever automations change), it loads all active automations from the database and groups them by collection + operation (e.g. `app.bsky.feed.post` + `create`). Only the **unique collection names** are sent as `wantedCollections` params to Jetstream. If 100 users watch `app.bsky.feed.post`, Jetstream sends events for that collection once.
10
10
11
-
2. **In-memory fan-out** — When an event arrives, the consumer looks up all subscriptions for that collection in a `Map<string, Subscription[]>` and iterates through them, evaluating each subscription's conditions. Only matching subscriptions trigger their actions (webhook delivery or record creation).
11
+
2. **In-memory fan-out** — When an event arrives, the consumer looks up all automations for that collection + operation in a `Map<string, Automation[]>` and iterates through them, evaluating each automation's conditions. Only matching automations trigger their actions (webhook delivery or record creation).
12
12
13
-
3. **WebSocket reconnection on collection changes** — When a subscription is created or deleted, if the set of watched collections changes, the consumer closes and reopens the WebSocket with updated `wantedCollections` params. If only the subscriptions within an existing collection change, no reconnection is needed — the in-memory map is simply updated.
13
+
3. **WebSocket reconnection on collection changes** — When an automation is created or deleted, if the set of watched collections changes, the consumer closes and reopens the WebSocket with updated `wantedCollections` params. If only the automations within an existing collection change, no reconnection is needed — the in-memory map is simply updated.
14
14
15
15
### Why this works well at current scale
16
16
17
-
- The `Map` lookup by collection is O(1).
18
-
- Condition matching is a simple linear scan per subscription — fast for dozens or even hundreds of subscriptions per collection.
17
+
- The `Map` lookup by collection + operation is O(1).
18
+
- Condition matching is a simple linear scan per automation — fast for dozens or even hundreds of automations per collection.
19
19
- All fan-out happens in-process with no network overhead.
20
20
- A single WebSocket minimizes Jetstream resource usage.
21
21
22
22
## Potential Future Improvements
23
23
24
-
As the number of subscriptions per collection grows (thousands+), the linear scan through conditions on every event could become a bottleneck. Some options:
24
+
As the number of automations per collection grows (thousands+), the linear scan through conditions on every event could become a bottleneck. Some options:
25
25
26
-
- **Condition indexing** — Build inverted indexes on condition fields/values so only potentially matching subscriptions are evaluated, rather than scanning all of them.
27
-
- **Batch/parallel condition evaluation** — Evaluate conditions for multiple subscriptions concurrently rather than sequentially.
26
+
- **Condition indexing** — Build inverted indexes on condition fields/values so only potentially matching automations are evaluated, rather than scanning all of them.
27
+
- **Batch/parallel condition evaluation** — Evaluate conditions for multiple automations concurrently rather than sequentially.
28
28
- **Sharded consumers** — Run multiple consumer instances, each responsible for a subset of collections or users, to distribute the fan-out load.
29
29
- **Pre-filtering with Jetstream features** — If Jetstream adds more granular filtering (e.g. by DID or record fields), leverage that to reduce the volume of events the consumer needs to process.
30
30
+1
-1
docs/sqlite-bun-compat.md
+1
-1
docs/sqlite-bun-compat.md
···
79
79
The `.env` file is auto-loaded by Bun but not by Node (Vite SSR). `lib/config.ts` includes a minimal `.env` loader for the Node context:
80
80
81
81
```ts
82
-
if (typeof globalThis.Bun === "undefined" && existsSync(".env")) {
82
+
if (!("Bun" in globalThis) && existsSync(".env")) {
83
83
for (const line of readFileSync(".env", "utf-8").split("\n")) {
84
84
const m = line.match(/^([^#=\s]+)=(.*)$/);
85
85
if (m?.[1] && !(m[1] in process.env)) process.env[m[1]] = m[2];
+1
-1
lib/jetstream/matcher.ts
+1
-1
lib/jetstream/matcher.ts
···
72
72
/**
73
73
* Check if all conditions match the event.
74
74
* Empty conditions = match all events for that collection.
75
-
* `ownerDid` is the subscription owner's DID, used to resolve {{self}} in condition values.
75
+
* `ownerDid` is the automation owner's DID, used to resolve {{self}} in condition values.
76
76
*/
77
77
export function matchConditions(
78
78
event: JetstreamEvent,