Coffee journaling on ATProto (alpha)
alpha.arabica.social
coffee
docs: ideas and quickslice plan (for later)
authored by
Patrick Dewey
and committed by
tangled.org
1d9df7e1
e71c8d38
+535
+142
docs/ideas.md
+142
docs/ideas.md
···
1
+
# Arabica Feature Ideas
2
+
3
+
## 1. Recipes
4
+
5
+
A **recipe** is a reusable, shareable brew procedure — distinct from a brew log, which records a
6
+
specific cup. Recipes are the social object people want to discover and re-use.
7
+
8
+
### New Lexicon: `social.arabica.alpha.recipe`
9
+
10
+
| Field | Type | Description |
11
+
| -------------- | ----------------- | ------------------------------------------------ |
12
+
| `title` | string (required) | "My Hario Switch Recipe for Light Roasts" |
13
+
| `description` | string | Longer notes, tips, rationale |
14
+
| `method` | string | V60, Aeropress, etc. |
15
+
| `temperature` | int (tenths °C) | Same encoding as brew |
16
+
| `coffeeAmount` | int (grams) | Dose |
17
+
| `waterAmount` | int (grams) | Total water |
18
+
| `grindSize` | string | "medium-fine" |
19
+
| `timeSeconds` | int | Total brew time |
20
+
| `pours` | array | `[{waterAmount, timeSeconds}]` — pour schedule |
21
+
| `tags` | []string | Optional: ["fruity", "light roast", "comp-prep"] |
22
+
| `beanNotes` | string | Optional: "works well with washed Ethiopians" |
23
+
| `createdAt` | datetime | |
24
+
25
+
### Interaction Flows
26
+
27
+
- **Likes / Comments** — already works; just point the existing `like` and `comment` records at the
28
+
recipe AT-URI.
29
+
- **"Try This"** — button on a recipe opens the brew form pre-populated with the recipe's
30
+
parameters. The resulting brew record optionally stores `basedOn: {uri, cid}` referencing the
31
+
recipe (strong ref).
32
+
- **"Save as Recipe"** — button on a completed brew detail page creates a recipe record from that
33
+
brew's parameters (minus the specific bean).
34
+
- **"X people tried this"** — count brews across the index whose `basedOn` field references the
35
+
recipe's AT-URI.
36
+
37
+
### Social Flywheel
38
+
39
+
Recipe → community tries it → brews reference it → author sees who tried it → ratings provide
40
+
feedback → better recipes emerge.
41
+
42
+
### Feed Integration
43
+
44
+
Recipes should appear in the community feed as a new record type. The feed already supports
45
+
filtering by type, so recipes slot in naturally.
46
+
47
+
---
48
+
49
+
## 2. Roaster Analytics
50
+
51
+
A **public analytics page** for each roaster, aggregated from community brews via the AT Protocol
52
+
ref chain: `brew.beanRef → bean.roasterRef → roaster`.
53
+
54
+
No schema changes required — the firehose index already stores all records and their refs. SQLite's
55
+
`json_extract()` lets us follow the ref chain in a single JOIN query.
56
+
57
+
### URL
58
+
59
+
`/roasters/{did}/{rkey}` → constructs `at://{did}/social.arabica.alpha.roaster/{rkey}`
60
+
61
+
### Analytics Data
62
+
63
+
| Metric | Query approach |
64
+
| ------------------- | ------------------------------------------------------- |
65
+
| Total brews | COUNT brews whose bean references this roaster |
66
+
| Total beans indexed | COUNT distinct beans referencing this roaster |
67
+
| Active brewers | COUNT DISTINCT brew.did |
68
+
| Avg rating | AVG(json_extract(brew.record, '$.rating')) |
69
+
| Median rating | Fetch all ratings, sort in Go, pick middle |
70
+
| Top beans | GROUP BY bean, AVG rating DESC |
71
+
| Brew method mix | GROUP BY json_extract(brew.record, '$.method'), COUNT |
72
+
| Rating by month | GROUP BY strftime('%Y-%m', created_at), AVG rating |
73
+
74
+
### Core SQL Pattern
75
+
76
+
```sql
77
+
SELECT
78
+
bean.uri,
79
+
json_extract(bean.record, '$.name') AS bean_name,
80
+
COUNT(brew.uri) AS brew_count,
81
+
AVG(json_extract(brew.record, '$.rating')) AS avg_rating,
82
+
COUNT(DISTINCT brew.did) AS brewer_count
83
+
FROM records brew
84
+
JOIN records bean ON bean.uri = json_extract(brew.record, '$.beanRef')
85
+
WHERE brew.collection = 'social.arabica.alpha.brew'
86
+
AND bean.collection = 'social.arabica.alpha.bean'
87
+
AND json_extract(bean.record, '$.roasterRef') = ? -- roaster AT-URI
88
+
GROUP BY bean.uri
89
+
ORDER BY avg_rating DESC
90
+
```
91
+
92
+
### Page Sections
93
+
94
+
1. **Header** — Roaster name, location, website, total brews / beans / brewers
95
+
2. **Rating summary** — Avg ★, median ★, total rated brews
96
+
3. **Top beans** — Table: bean name, brew count, avg rating
97
+
4. **Brew method breakdown** — Bar/list showing V60, Aeropress, etc.
98
+
5. **Rating trend** — Month-by-month avg rating (simple list or sparkline)
99
+
100
+
### Linking
101
+
102
+
Anywhere a roaster name appears in the feed or on a brew/bean detail page, link to
103
+
`/roasters/{did}/{rkey}`.
104
+
105
+
### Public Access
106
+
107
+
The analytics page is fully public (no auth required) since all data is already public in the
108
+
firehose index.
109
+
110
+
---
111
+
112
+
## 3. Personal Analytics Dashboard
113
+
114
+
A private `/me/stats` page showing the authenticated user's own brewing trends, computed from
115
+
their PDS records (no cross-user aggregation needed).
116
+
117
+
### Metrics
118
+
119
+
| Metric | Source |
120
+
| -------------------- | ----------------------------------------- |
121
+
| Brews per week/month | Count brews by created_at bucket |
122
+
| Avg rating over time | Avg rating by month |
123
+
| Favourite bean | Most brewed bean (+ highest avg rating) |
124
+
| Favourite method | Most used brew method |
125
+
| Equipment usage | Most used grinder / brewer |
126
+
| Taste evolution | Rating trend over time |
127
+
| Bags opened/closed | Count beans by `closed` flag |
128
+
129
+
### Implementation Notes
130
+
131
+
- Query user's own PDS via `store.ListBrews()` — no firehose needed.
132
+
- Aggregate in Go (small data set per user, no need for SQL aggregation).
133
+
- Cache in `SessionCache` to avoid repeated PDS fetches.
134
+
- No new lexicons required.
135
+
136
+
---
137
+
138
+
## Priority
139
+
140
+
1. **Roaster analytics** — immediate value, no schema changes, pure SQL over existing indexed refs
141
+
2. **Recipes** — high social value, new lexicon + feed integration required
142
+
3. **Personal stats** — lower complexity, pure client-side aggregation, quality-of-life feature
+393
docs/quickslice-implementation-plan.md
+393
docs/quickslice-implementation-plan.md
···
1
+
# Quickslice Integration Plan
2
+
3
+
Quickslice is a self-hostable AppView framework for AT Protocol. It connects to Jetstream,
4
+
indexes records matching your lexicons into SQLite/Postgres, and exposes a GraphQL API with
5
+
built-in joins across record types.
6
+
7
+
**Goal:** Reduce PDS fetches for reads by querying a local quickslice instance, keeping the
8
+
existing PDS path as a fallback.
9
+
10
+
---
11
+
12
+
## Value Proposition
13
+
14
+
### The Problem With PDS-First Architecture
15
+
16
+
Arabica's current architecture makes reads directly against users' Personal Data Servers.
17
+
This is correct for writes (AT Protocol mandates it), but for reads it has significant costs:
18
+
19
+
**Sequential, unbatched fetches.** When you view a brew, the app makes individual `getRecord`
20
+
calls — one for the brew, then one for the bean, then one for the roaster nested inside that
21
+
bean, then one for the grinder, then one for the brewer. These are sequential because each
22
+
reference is resolved after the previous record arrives. That's 4-5 round trips per brew
23
+
view, each one a network call to an external PDS.
24
+
25
+
**N+1 on the community feed.** The feed currently polls registered users' PDS instances
26
+
for their brews. Resolving each brew's references for display (bean name, roaster, grinder)
27
+
requires additional PDS calls per brew card. As the user base grows, this becomes
28
+
expensive fast.
29
+
30
+
**Cross-user reads have no cache.** The 5-minute session cache covers own-user data well.
31
+
But when viewing another user's public brew, there's no caching — every page load hits
32
+
their PDS for 4-6 calls.
33
+
34
+
**PDS polling for the feed.** The feed service periodically polls each registered user's
35
+
PDS. This doesn't scale well and is the reason the backfill system exists. Jetstream
36
+
already has all this data in a push model.
37
+
38
+
### What Quickslice Provides
39
+
40
+
Quickslice acts as a **local read index** — it subscribes to Jetstream and maintains a
41
+
local SQLite/Postgres copy of all arabica records from across the network. Because the data
42
+
is local, reads are fast and joined queries are cheap.
43
+
44
+
The key feature is **automatic join generation from lexicons**. Because arabica's lexicons
45
+
define typed references between records (`beanRef`, `grinderRef`, `brewerRef`), quickslice
46
+
generates GraphQL joins that follow those references in a single query. What currently
47
+
takes 4-5 sequential PDS calls becomes one local query.
48
+
49
+
```
50
+
Current: brew → PDS → bean → PDS → roaster → PDS → grinder → PDS → brewer → PDS
51
+
(5 sequential network calls, each blocked on the previous)
52
+
53
+
With QS: ──────────────────► quickslice (1 local query, all joins in one response) ◄──────
54
+
```
55
+
56
+
### Where This Matters Most
57
+
58
+
**Public brew views (highest value).** Viewing another user's brew currently requires 4-6
59
+
calls to their PDS via `public_client`. There's no caching. Quickslice replaces this with
60
+
a single local query, and the result could be cached locally. This is the biggest latency
61
+
improvement for the most visible user-facing operation.
62
+
63
+
**Community feed enrichment.** Each brew card in the feed shows bean name, roaster, and
64
+
equipment. Currently this requires fetching that data per-brew. A single quickslice query
65
+
can return feed brews with all references resolved, replacing N×5 PDS calls with 1 query.
66
+
67
+
**Reference resolution after brew creation.** When you create a brew, the app immediately
68
+
resolves its references to populate the full model (for rendering the response). That's
69
+
2-4 extra PDS calls. Quickslice can't help immediately after a write (Jetstream lag), but
70
+
the references themselves (bean, grinder, brewer) are already indexed — so a targeted
71
+
query for those specific records is instant.
72
+
73
+
**Own-user reads on cold cache.** The session cache (5-minute TTL) covers most own-user
74
+
list operations after the first load. But on first load or after cache expiry, `ListBrews`
75
+
makes 5 PDS calls. Quickslice serves this from local storage instead.
76
+
77
+
**Future social features.** Likes and comments will need cross-user aggregation — "how many
78
+
people liked this brew?" requires querying many users' PDS instances. With quickslice, this
79
+
becomes a reverse-join query (find all `social.arabica.alpha.like` records pointing at a
80
+
given brew URI), served locally.
81
+
82
+
### What Quickslice Does Not Fix
83
+
84
+
- **Writes** always go to the user's PDS. OAuth, DPOP, and mutations are unaffected.
85
+
- **Freshness after writes**: Jetstream has seconds of lag. A record you just created
86
+
won't be in quickslice immediately. The existing PDS path must remain for post-write reads.
87
+
- **Auth-gated data**: Quickslice indexes public records only (AT Protocol records are
88
+
public by default, so this is fine for arabica).
89
+
- **Scale ceiling**: Quickslice is early-stage (v0.20.x). At very high scale, you'd
90
+
eventually need a more mature indexing pipeline. But for arabica's current scale, SQLite
91
+
is more than sufficient.
92
+
93
+
---
94
+
95
+
## Current Pain Points
96
+
97
+
| Operation | Current PDS calls |
98
+
|-----------|:-:|
99
+
| `GetBrewByRKey` | 1 (brew) + 1 (bean) + 1 (roaster) + 1 (grinder) + 1 (brewer) = **up to 5** |
100
+
| `ListBrews` (cold cache) | 1 (brews) + 1 (beans) + 1 (roasters) + 1 (grinders) + 1 (brewers) = **5** |
101
+
| Public brew view (cross-user) | 1 (brew) + 1 (bean) + 1 (roaster) + 1 (grinder) + 1 (brewer) = **up to 5** |
102
+
| Community feed brew cards | 5 calls × N brews (N+1 problem on ref resolution) |
103
+
104
+
With quickslice, each of the above becomes **1 GraphQL query** with forward joins.
105
+
106
+
---
107
+
108
+
## Architecture
109
+
110
+
```
111
+
arabica server
112
+
├─ Writes ──────────────────────────────► User's PDS (unchanged)
113
+
└─ Reads ──► quickslice GraphQL client ──► quickslice HTTP API
114
+
│ (on error/miss)
115
+
└──────────────────────► existing PDS path (fallback)
116
+
117
+
quickslice service
118
+
├─ Subscribes to Jetstream (firehose)
119
+
├─ Indexes arabica lexicon records → SQLite or Postgres
120
+
└─ Exposes GraphQL API at :8080/graphql (no auth required for reads)
121
+
```
122
+
123
+
Quickslice is a **read-only index**. All writes (create/update/delete) continue to go to the
124
+
user's PDS. Quickslice self-populates via Jetstream, so no backfill code changes are needed.
125
+
126
+
---
127
+
128
+
## Constraints and Caveats
129
+
130
+
- **Eventual consistency**: Jetstream has ~seconds of lag. A record just written to the PDS
131
+
may not appear in quickslice immediately. After any write operation, skip quickslice and
132
+
read directly from PDS for that request.
133
+
- **Early-stage software**: Quickslice is v0.20.x, "APIs may change without notice". Wrap it
134
+
behind an interface so swapping it out is painless.
135
+
- **Gleam runtime**: Quickslice is written in Gleam. Debugging upstream issues requires
136
+
learning a new language — treat it as a black box.
137
+
- **Writes stay on PDS**: OAuth sessions, DPOP tokens, and all mutations are unaffected.
138
+
139
+
---
140
+
141
+
## Phase 1: Infrastructure
142
+
143
+
**Goal:** Get quickslice running and indexing arabica records.
144
+
145
+
### 1a. Add quickslice to docker-compose
146
+
147
+
Add a `quickslice` service to the project's docker-compose (or nix deployment config):
148
+
149
+
```yaml
150
+
quickslice:
151
+
image: ghcr.io/slices-network/quickslice:latest
152
+
environment:
153
+
DATABASE_URL: "" # empty = use SQLite
154
+
LEXICON_DIR: /lexicons
155
+
JETSTREAM_URL: wss://jetstream2.us-east.bsky.network/subscribe
156
+
volumes:
157
+
- ./lexicons:/lexicons:ro
158
+
- quickslice-data:/data
159
+
ports:
160
+
- "8080:8080"
161
+
```
162
+
163
+
Point `LEXICON_DIR` at the repo's existing `lexicons/` directory. Quickslice reads lexicon
164
+
JSON files and auto-generates its GraphQL schema from them.
165
+
166
+
### 1b. Verify indexing
167
+
168
+
After startup, check the GraphQL playground at `http://localhost:8080` and run a sample
169
+
query for `social.arabica.alpha.brew` records. Confirm that records from the firehose appear
170
+
with the expected fields.
171
+
172
+
### 1c. Add environment variable to arabica
173
+
174
+
```
175
+
QUICKSLICE_URL=http://localhost:8080/graphql
176
+
```
177
+
178
+
When unset or empty, quickslice integration is disabled and arabica falls back to PDS-only
179
+
mode. This makes it safe to deploy without quickslice running.
180
+
181
+
---
182
+
183
+
## Phase 2: Go GraphQL Client
184
+
185
+
**Goal:** Write a minimal quickslice client in Go. No codegen — just HTTP + JSON.
186
+
187
+
### File: `internal/quickslice/client.go`
188
+
189
+
```go
190
+
type Client struct {
191
+
endpoint string
192
+
httpClient *http.Client
193
+
}
194
+
195
+
func New(endpoint string) *Client { ... }
196
+
197
+
// Returns nil, ErrNotFound if the record isn't indexed yet (freshness gap after writes).
198
+
func (c *Client) GetBrew(ctx context.Context, did, rkey string) (*models.Brew, error) { ... }
199
+
func (c *Client) ListBrews(ctx context.Context, did string) ([]*models.Brew, error) { ... }
200
+
func (c *Client) GetBeanWithRoaster(ctx context.Context, did, rkey string) (*models.Bean, error) { ... }
201
+
202
+
// helpers
203
+
func (c *Client) query(ctx context.Context, q string, vars map[string]any, out any) error { ... }
204
+
```
205
+
206
+
GraphQL queries use forward joins to resolve references in a single round-trip. Example for
207
+
`GetBrew`:
208
+
209
+
```graphql
210
+
query GetBrew($did: String!, $rkey: String!) {
211
+
socialArabicaAlphaBrew(where: { did: { eq: $did }, rkey: { eq: $rkey } }, first: 1) {
212
+
edges {
213
+
node {
214
+
uri cid did rkey createdAt
215
+
# forward join: beanRef → bean record
216
+
beanRefResolved {
217
+
... on SocialArabicaAlphaBeanRecord {
218
+
uri rkey name origin process roastLevel
219
+
# nested: roasterRef → roaster record
220
+
roasterRefResolved {
221
+
... on SocialArabicaAlphaRoasterRecord { uri rkey name location }
222
+
}
223
+
}
224
+
}
225
+
grinderRefResolved {
226
+
... on SocialArabicaAlphaGrinderRecord { uri rkey brand model burrType }
227
+
}
228
+
brewerRefResolved {
229
+
... on SocialArabicaAlphaBrewerRecord { uri rkey brand model type }
230
+
}
231
+
}
232
+
}
233
+
}
234
+
}
235
+
```
236
+
237
+
**Note:** The exact GraphQL field names are generated from lexicon IDs. Verify them against
238
+
the quickslice playground after Phase 1 is complete.
239
+
240
+
---
241
+
242
+
## Phase 3: Wire Into AtprotoStore
243
+
244
+
**Goal:** Add quickslice as an optional fast path in `AtprotoStore`, falling back to existing
245
+
PDS code on any error.
246
+
247
+
### 3a. Add field to AtprotoStore
248
+
249
+
```go
250
+
// internal/atproto/store.go
251
+
type AtprotoStore struct {
252
+
client *Client
253
+
quickslice *quickslice.Client // nil if QUICKSLICE_URL is unset
254
+
did string
255
+
sessionID string
256
+
cache *SessionCache
257
+
}
258
+
```
259
+
260
+
Wire `quickslice.Client` in during store construction (in `handlers.go` or wherever
261
+
`AtprotoStore` is instantiated).
262
+
263
+
### 3b. Wrap Get* methods
264
+
265
+
Pattern for all Get* methods:
266
+
267
+
```go
268
+
func (s *AtprotoStore) GetBrewByRKey(ctx context.Context, rkey string) (*models.Brew, error) {
269
+
if s.quickslice != nil {
270
+
brew, err := s.quickslice.GetBrew(ctx, s.did, rkey)
271
+
if err == nil {
272
+
return brew, nil
273
+
}
274
+
// log at debug level, fall through
275
+
}
276
+
// existing implementation unchanged below
277
+
...
278
+
}
279
+
```
280
+
281
+
Do **not** wrap immediately after a write. The handler already has the created record in
282
+
hand at that point — no PDS read needed.
283
+
284
+
### 3c. Wrap ListBrews (cold-cache path only)
285
+
286
+
`ListBrews` already checks `SessionCache` first. The quickslice path sits between cache miss
287
+
and PDS:
288
+
289
+
```go
290
+
func (s *AtprotoStore) ListBrews(ctx context.Context, userID string) ([]*models.Brew, error) {
291
+
if brews := s.cache.GetBrews(s.sessionID); brews != nil {
292
+
return brews, nil
293
+
}
294
+
if s.quickslice != nil {
295
+
brews, err := s.quickslice.ListBrews(ctx, s.did)
296
+
if err == nil {
297
+
s.cache.SetBrews(s.sessionID, brews)
298
+
return brews, nil
299
+
}
300
+
}
301
+
// existing PDS implementation unchanged
302
+
...
303
+
}
304
+
```
305
+
306
+
Apply the same pattern to `ListBeans`, `ListRoasters`, `ListGrinders`, `ListBrewers`.
307
+
308
+
---
309
+
310
+
## Phase 4: Cross-User Reads (Highest Value)
311
+
312
+
**Goal:** Replace the `resolveBrewReferences` function in `internal/handlers/brew.go` that
313
+
handles public brew view (other users' brews). This is the highest-value target because
314
+
there is no session cache here — every page view currently makes 4-6 PDS calls to
315
+
`public_client`.
316
+
317
+
### Current flow (brew.go ~line 313)
318
+
319
+
```go
320
+
func resolveBrewReferences(ctx context.Context, client PublicClient, brew *models.Brew) error {
321
+
// Individual GetRecord calls for bean, roaster, grinder, brewer
322
+
}
323
+
```
324
+
325
+
### New flow
326
+
327
+
```go
328
+
func (h *Handler) resolveBrewReferences(ctx context.Context, brew *models.Brew) error {
329
+
if h.quickslice != nil {
330
+
resolved, err := h.quickslice.GetBrew(ctx, brew.AuthorDID, brew.RKey)
331
+
if err == nil {
332
+
*brew = *resolved // copy resolved refs onto brew
333
+
return nil
334
+
}
335
+
}
336
+
// existing public_client fallback
337
+
}
338
+
```
339
+
340
+
This also benefits the community feed — brew cards in the feed can be enriched with bean
341
+
names and roaster info in a single batch query instead of N×5 PDS calls.
342
+
343
+
---
344
+
345
+
## Phase 5: Community Feed (Optional, Later)
346
+
347
+
The feed service (`internal/feed/service.go`) polls registered users' PDS for brews.
348
+
Quickslice already indexes all arabica records from the firehose network-wide.
349
+
350
+
Long-term, the feed query could become a single GraphQL query with sorting and pagination:
351
+
352
+
```graphql
353
+
query CommunityFeed($limit: Int!, $after: String) {
354
+
socialArabicaAlphaBrew(
355
+
first: $limit, after: $after
356
+
sortBy: [{ field: "createdAt", direction: DESC }]
357
+
) {
358
+
edges {
359
+
node {
360
+
uri did actorHandle createdAt
361
+
beanRefResolved { ... }
362
+
}
363
+
cursor
364
+
}
365
+
pageInfo { hasNextPage endCursor }
366
+
}
367
+
}
368
+
```
369
+
370
+
This replaces the polling-based aggregation entirely. Hold off until the quickslice API
371
+
stabilizes (it's still early-stage).
372
+
373
+
---
374
+
375
+
## Implementation Order
376
+
377
+
1. **Phase 1** — Infrastructure (quickslice running, lexicons loaded, env var wired)
378
+
2. **Phase 2** — Go client with `GetBrew` and verify joins work against the live instance
379
+
3. **Phase 4** — Cross-user reads (public brew view) — highest value, no auth complexity
380
+
4. **Phase 3** — Own-user Get* reads with fallback
381
+
5. **Phase 3c** — List* reads (lower priority — session cache already covers most of this)
382
+
6. **Phase 5** — Feed service replacement (deferred until API stabilizes)
383
+
384
+
---
385
+
386
+
## Testing Strategy
387
+
388
+
- Unit test the quickslice client with a mock HTTP server returning known GraphQL responses
389
+
- Integration test: start quickslice in docker, write a record to a test PDS, wait for
390
+
indexing, query via client
391
+
- Smoke test fallback: with `QUICKSLICE_URL` unset, all existing behavior must be unchanged
392
+
- After any write, assert the next read uses the PDS path (not quickslice) to avoid
393
+
stale-data bugs