AT Feeds#

A monorepo for indexing and displaying Standard.site documents from the AT Protocol, powered by Cloudflare Workers, D1, and Queues.

Architecture#

┌─────────────────────────────────────────────────────────────┐
│                     Cloudflare                               │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  ┌──────────────┐     ┌──────────────┐     ┌─────────────┐  │
│  │    Pages     │────▶│   Worker     │────▶│     D1      │  │
│  │   (Client)   │     │   (API)      │     │  (Database) │  │
│  └──────────────┘     └──────────────┘     └─────────────┘  │
│                              ▲                    ▲          │
│                              │                    │          │
│                       ┌──────┴───────┐    ┌──────┴───────┐  │
│                       │    Queue     │    │     Cron     │  │
│                       │  (Resolver)  │    │  (Refresh)   │  │
│                       └──────┬───────┘    └──────────────┘  │
│                              │                              │
└──────────────────────────────┼──────────────────────────────┘
                               │ POST /webhook/tap
                    ┌──────────┴───────────┐
                    │   Tap Instance       │
                    │   (External VPS)     │
                    └──────────────────────┘

Components:

Tap Indexer (External) - Subscribes to the AT Protocol firehose and sends webhook events
Server (packages/server) - Cloudflare Worker with Hono API, D1 database, and Queue consumer
Client (packages/client) - Vite + React app deployed to Cloudflare Pages

Quick Start#

Prerequisites#

Bun installed
Wrangler CLI installed and authenticated
A tap instance running somewhere (VPS, Fly.io, etc.)

Setup#

Install dependencies:

bun install

Create the D1 database:

bun run db:create

Copy the database ID and update packages/server/wrangler.toml.

Create the queue:

wrangler queues create document-resolution

Run database migrations:

# Local development
bun run db:migrate

# Production
bun run db:migrate:prod

(Optional) Set webhook secret:

bun run secret:set

Deploy the worker:

bun run deploy

Configure your tap instance:

TAP_WEBHOOK_URL=https://your-worker.workers.dev/webhook/tap
TAP_SIGNAL_COLLECTION=site.standard.document
TAP_COLLECTION_FILTERS=site.standard.document

Trigger initial resolution of existing records:

curl -X POST https://your-worker.workers.dev/admin/resolve-all

Local Development#

Start the worker locally:

bun run dev:server

The API will run on http://localhost:8787.

Start the client (in a separate terminal):

bun run dev:client

The client will run on http://localhost:5173.

API Endpoints#

Health & Stats#

Endpoint	Method	Description
`/health`	GET	Health check
`/stats`	GET	Database statistics

Feed Endpoints#

Endpoint	Method	Description
`/feed`	GET	Pre-resolved documents (fast)
`/feed-raw`	GET	Raw record references (for client-side resolution)
`/records/:did`	GET	Records by DID

Webhook#

Endpoint	Method	Description
`/webhook/tap`	POST	Receives events from tap
`/webhook/tap/debug`	POST	Debug endpoint (echoes payload)

Admin#

Endpoint	Method	Description
`/admin/resolve-all`	POST	Queue unresolved records for processing

How It Works#

Tap subscribes to the AT Protocol firehose and filters for site.standard.document records
Webhook receives events and stores record references in D1, then pushes to the resolution queue
Queue consumer resolves each document (PDS lookup → record fetch → publication URL) and stores in resolved_documents
Cron job (every 15 min) refreshes stale documents and processes any missed records
/feed endpoint reads directly from resolved_documents for instant responses

Project Structure#

.
├── package.json            # Root workspace config
└── packages/
    ├── server/             # Cloudflare Worker
    │   ├── wrangler.toml   # Worker configuration
    │   ├── schema.sql      # D1 database schema
    │   ├── package.json
    │   └── src/
    │       └── index.ts    # API + Queue consumer + Cron handler
    └── client/             # Vite + React app
        ├── package.json
        ├── vite.config.ts
        └── src/
            ├── main.tsx
            └── App.tsx

Scripts#

# Development
bun run dev              # Run all packages in dev mode
bun run dev:server       # Run worker locally
bun run dev:client       # Run client locally

# Deployment
bun run deploy           # Deploy worker to Cloudflare
bun run deploy:client    # Deploy client to Cloudflare Pages

# Database
bun run db:create        # Create D1 database
bun run db:migrate       # Run migrations (local)
bun run db:migrate:prod  # Run migrations (production)

# Secrets
bun run secret:set       # Set TAP_WEBHOOK_SECRET

Resources#

License#

MIT

Configure Feed