Spark feed generator template
readme
+75
-39
+75
-39
README.md
+75
-39
README.md
···
1
1
# Spark Feed Generator
2
2
3
-
This repository is a feed generator for the Spark atproto app. It is a simple
4
-
feed generator that fetches posts from a database and returns them in a format
5
-
that can be used by the Spark client.
3
+
A template for creating custom feed generators for [sprk.so](https://sprk.so).
6
4
7
-
The whole structure is heavily based on the Bluesky
8
-
[feed-generator](https://github.com/bluesky-social/feed-generator) repository,
9
-
most of the things said there apply here as well.
5
+
## Architecture
10
6
11
-
## Setup
7
+
The feed generator has three main components:
12
8
13
-
### Option 1: Docker Compose (Recommended)
9
+
1. **Ingester** (`ingester/`) - Consumes the Spark firehose in real-time and
10
+
indexes records to MongoDB. By default, it indexes `so.sprk.feed.post`
11
+
records. Handlers for `follow`, `like`, and `repost` are included but
12
+
disabled - enable them by uncommenting in `ingester/index.ts`.
14
13
15
-
The easiest way to get started is using Docker Compose, which will automatically
16
-
build the application and set up the MongoDB database.
14
+
2. **Algorithms** (`algos/`) - Define how posts are selected and sorted for
15
+
your feed. Each algorithm exports:
16
+
- `handler` - Query function that returns posts from the database
17
+
- `needsAuth` - Whether the feed requires user authentication
18
+
- `publisherDid` - The DID of the feed publisher
19
+
- `rkey` - Unique identifier for this algorithm
17
20
18
-
1. Start the services (this will automatically build the image):
21
+
3. **API Server** - Exposes XRPC endpoints that Spark clients call to fetch
22
+
feed content (`so.sprk.feed.getFeedSkeleton`, `so.sprk.feed.describeFeedGenerator`).
23
+
You won't need to modify these when creating a feed.
19
24
20
-
```sh
21
-
docker-compose up -d
22
-
```
25
+
## Creating Custom Feeds
23
26
24
-
2. The application will be available at http://localhost:3000
27
+
**For topic/community feeds:** Filter posts at the ingester level. Modify
28
+
the handler in `ingester/handlers/post.ts` to only index posts matching your
29
+
criteria (hashtags, keywords, specific authors, etc.).
25
30
26
-
3. To stop the services:
31
+
**For personalized/sorted feeds:** Create a new algorithm in `algos/`. Copy
32
+
`simple-desc.ts` as a starting point, then modify the query logic. Register
33
+
your algorithm in `algos/index.ts`.
27
34
28
-
```sh
29
-
docker-compose down
35
+
Example algorithm structure:
36
+
```ts
37
+
// algos/my-feed.ts
38
+
export const info = {
39
+
handler: async (ctx, params, did) => {
40
+
// Query posts from ctx.db.models.Post
41
+
// Return { cursor, feed: [{ post: "at://..." }] }
42
+
},
43
+
needsAuth: false, // Set true if feed needs user's DID
44
+
publisherDid: "did:plc:your-did",
45
+
rkey: "my-feed", // at://your-did/so.sprk.feed.generator/my-feed
46
+
} as Algorithm;
30
47
```
31
48
32
-
### Option 2: Manual Setup
33
-
34
-
If you prefer to run the application manually or use an existing MongoDB
35
-
instance:
49
+
## Running
36
50
37
-
#### Prerequisites
51
+
### Prerequisites
38
52
39
-
- [Deno](https://deno.com/) runtime
40
-
- MongoDB instance (local or remote)
53
+
Create a `.env` file in the project root:
41
54
42
-
#### Steps
43
-
44
-
1. Create a `.env` file in the root of the repository and add the following
45
-
variables:
46
-
47
-
```
48
-
SPRK_DB_NAME=feed-gen
55
+
```sh
56
+
# Required
57
+
FEEDGEN_DOMAIN=feeds.example.com # Your feed generator's domain
58
+
SPRK_DB_NAME=feed-gen # MongoDB database name
49
59
SPRK_DB_URI=mongodb://localhost:27017
50
60
SPRK_DB_USER=username
51
-
SPRK_DB_PASSWORD=password
52
-
SPRK_FEEDGEN_DOMAIN=feeds.example.com
61
+
SPRK_DB_PASS=password
62
+
63
+
# Optional
64
+
SPRK_HOST=0.0.0.0 # Server bind address
65
+
SPRK_PORT=3000 # Server port
53
66
NODE_ENV=production
54
67
```
55
68
56
-
2. Install dependencies:
69
+
### Docker Compose (Recommended)
70
+
71
+
Starts both the feed generator and a MongoDB instance:
57
72
58
73
```sh
59
-
deno install
74
+
# Start services
75
+
docker-compose up -d
76
+
77
+
# View logs
78
+
docker-compose logs -f feed-gen
79
+
80
+
# Stop services
81
+
docker-compose down
60
82
```
61
83
62
-
3. Make sure MongoDB is running.
84
+
### Manual Setup
63
85
64
-
4. Start the development server:
86
+
Requires [Deno](https://deno.com/) and a MongoDB instance.
65
87
66
88
```sh
89
+
# Install dependencies
90
+
deno install
91
+
92
+
# Start server
67
93
deno task start
68
94
```
69
95
70
-
5. The application will be available at http://localhost:3000
96
+
The server will be available at `http://localhost:3000`.
97
+
98
+
## Endpoints
99
+
100
+
| Endpoint | Description |
101
+
|----------|-------------|
102
+
| `GET /` | Service info |
103
+
| `GET /health` | Health check |
104
+
| `GET /.well-known/did.json` | DID document for `did:web` resolution |
105
+
| `GET /xrpc/so.sprk.feed.describeFeedGenerator` | List available feeds |
106
+
| `GET /xrpc/so.sprk.feed.getFeedSkeleton` | Fetch feed posts |