Auto-indexing service and GraphQL API for AT Protocol Records
docs: add rkey notification sorting implementation plan
+396
+396
dev-docs/plans/2025-12-26-rkey-notification-sorting.md
+396
dev-docs/plans/2025-12-26-rkey-notification-sorting.md
···
1
+
# Rkey-Based Notification Sorting Implementation Plan
2
+
3
+
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
4
+
5
+
**Goal:** Sort notifications by TID (timestamp identifier) extracted from record rkey for chronological ordering across collections.
6
+
7
+
**Architecture:** Add a generated `rkey` column to the record table that extracts the last path segment from the URI. Use this for sorting notifications instead of `indexed_at`, since TIDs are lexicographically sortable and encode creation timestamps.
8
+
9
+
**Tech Stack:** Gleam, SQLite, GraphQL (swell)
10
+
11
+
---
12
+
13
+
### Task 1: Add Migration for rkey Generated Column
14
+
15
+
**Files:**
16
+
- Create: `server/db/migrations/20241227000001_add_rkey_column.sql`
17
+
18
+
**Step 1: Create the migration file**
19
+
20
+
```sql
21
+
-- migrate:up
22
+
23
+
-- Add rkey as a generated column extracted from uri
24
+
-- URI format: at://did/collection/rkey
25
+
-- We extract everything after the last '/'
26
+
ALTER TABLE record ADD COLUMN rkey TEXT
27
+
GENERATED ALWAYS AS (
28
+
substr(uri, instr(substr(uri, instr(substr(uri, 6), '/') + 6), '/') + instr(substr(uri, 6), '/') + 6)
29
+
) STORED;
30
+
31
+
-- Index for efficient sorting by rkey (TID-based chronological order)
32
+
CREATE INDEX IF NOT EXISTS idx_record_rkey ON record(rkey DESC);
33
+
34
+
-- migrate:down
35
+
36
+
DROP INDEX IF EXISTS idx_record_rkey;
37
+
-- Note: SQLite doesn't support DROP COLUMN, would need table rebuild
38
+
```
39
+
40
+
**Step 2: Test the SQL extraction logic manually**
41
+
42
+
Run in SQLite to verify the extraction works:
43
+
```sql
44
+
SELECT
45
+
uri,
46
+
substr(uri, instr(substr(uri, instr(substr(uri, 6), '/') + 6), '/') + instr(substr(uri, 6), '/') + 6) as rkey
47
+
FROM (
48
+
SELECT 'at://did:plc:abc123/social.grain.photo/3maungnepas2t' as uri
49
+
);
50
+
```
51
+
52
+
Expected: `3maungnepas2t`
53
+
54
+
**Step 3: Commit**
55
+
56
+
```bash
57
+
git add server/db/migrations/20241227000001_add_rkey_column.sql
58
+
git commit -m "feat(db): add rkey generated column for TID-based sorting"
59
+
```
60
+
61
+
---
62
+
63
+
### Task 2: Update Record Type to Include rkey
64
+
65
+
**Files:**
66
+
- Modify: `server/src/database/types.gleam`
67
+
68
+
**Step 1: Add rkey field to Record type**
69
+
70
+
In `server/src/database/types.gleam`, update the Record type:
71
+
72
+
```gleam
73
+
pub type Record {
74
+
Record(
75
+
uri: String,
76
+
cid: String,
77
+
did: String,
78
+
collection: String,
79
+
json: String,
80
+
indexed_at: String,
81
+
rkey: String,
82
+
)
83
+
}
84
+
```
85
+
86
+
**Step 2: Run build to find all places that need updating**
87
+
88
+
```bash
89
+
cd /Users/chadmiller/code/quickslice/server && gleam build 2>&1
90
+
```
91
+
92
+
Expected: Compile errors showing all places constructing Record
93
+
94
+
**Step 3: Commit**
95
+
96
+
```bash
97
+
git add server/src/database/types.gleam
98
+
git commit -m "feat(types): add rkey field to Record type"
99
+
```
100
+
101
+
---
102
+
103
+
### Task 3: Update Record Decoder and Columns
104
+
105
+
**Files:**
106
+
- Modify: `server/src/database/repositories/records.gleam`
107
+
108
+
**Step 1: Update record_columns function**
109
+
110
+
Find `record_columns` and add `rkey`:
111
+
112
+
```gleam
113
+
fn record_columns(exec: Executor) -> String {
114
+
"uri, cid, did, collection, json, indexed_at, rkey"
115
+
}
116
+
```
117
+
118
+
**Step 2: Update record_decoder function**
119
+
120
+
Find `record_decoder` and add rkey decoding:
121
+
122
+
```gleam
123
+
fn record_decoder() -> decode.Decoder(Record) {
124
+
use uri <- decode.field(0, decode.string)
125
+
use cid <- decode.field(1, decode.string)
126
+
use did <- decode.field(2, decode.string)
127
+
use collection <- decode.field(3, decode.string)
128
+
use json <- decode.field(4, decode.string)
129
+
use indexed_at <- decode.field(5, decode.string)
130
+
use rkey <- decode.field(6, decode.string)
131
+
decode.success(Record(uri:, cid:, did:, collection:, json:, indexed_at:, rkey:))
132
+
}
133
+
```
134
+
135
+
**Step 3: Fix any remaining Record constructor calls**
136
+
137
+
Search for `Record(` and update any direct constructions to include rkey. For test fixtures, extract rkey from uri:
138
+
139
+
```gleam
140
+
// Helper to extract rkey from uri for tests
141
+
fn extract_rkey(uri: String) -> String {
142
+
uri
143
+
|> string.split("/")
144
+
|> list.last
145
+
|> result.unwrap("")
146
+
}
147
+
```
148
+
149
+
**Step 4: Run tests to verify**
150
+
151
+
```bash
152
+
cd /Users/chadmiller/code/quickslice/server && gleam test 2>&1 | tail -20
153
+
```
154
+
155
+
**Step 5: Commit**
156
+
157
+
```bash
158
+
git add server/src/database/repositories/records.gleam
159
+
git commit -m "feat(records): update decoder to include rkey column"
160
+
```
161
+
162
+
---
163
+
164
+
### Task 4: Update get_notifications to Sort by rkey
165
+
166
+
**Files:**
167
+
- Modify: `server/src/database/repositories/records.gleam`
168
+
169
+
**Step 1: Update ORDER BY clause**
170
+
171
+
In `get_notifications` function (around line 1143), change:
172
+
173
+
```gleam
174
+
// Before:
175
+
<> " ORDER BY indexed_at DESC, uri DESC LIMIT "
176
+
177
+
// After:
178
+
<> " ORDER BY rkey DESC, uri DESC LIMIT "
179
+
```
180
+
181
+
**Step 2: Update cursor WHERE clause**
182
+
183
+
In the cursor clause building (around line 1118), change:
184
+
185
+
```gleam
186
+
// Before:
187
+
#(" AND (indexed_at, uri) < (" <> p1 <> ", " <> p2 <> ")", [
188
+
Text(indexed_at_value),
189
+
Text(cid_value),
190
+
])
191
+
192
+
// After:
193
+
#(" AND (rkey, uri) < (" <> p1 <> ", " <> p2 <> ")", [
194
+
Text(rkey_value),
195
+
Text(uri_value),
196
+
])
197
+
```
198
+
199
+
Note: The cursor now uses `(rkey, uri)` tuple instead of `(indexed_at, cid)`.
200
+
201
+
**Step 3: Update cursor decoding**
202
+
203
+
Update the cursor value extraction:
204
+
205
+
```gleam
206
+
// The cursor format is now: rkey|uri (encoded as base64)
207
+
let rkey_value =
208
+
decoded.field_values |> list.first |> result.unwrap("")
209
+
let uri_value = decoded.cid // cid field now holds uri for this cursor
210
+
```
211
+
212
+
**Step 4: Run tests**
213
+
214
+
```bash
215
+
cd /Users/chadmiller/code/quickslice/server && gleam test 2>&1 | tail -20
216
+
```
217
+
218
+
**Step 5: Commit**
219
+
220
+
```bash
221
+
git add server/src/database/repositories/records.gleam
222
+
git commit -m "feat(notifications): sort by rkey for TID-based chronological order"
223
+
```
224
+
225
+
---
226
+
227
+
### Task 5: Update Notification Cursor Generation
228
+
229
+
**Files:**
230
+
- Modify: `server/src/database/repositories/records.gleam`
231
+
232
+
**Step 1: Update end_cursor generation**
233
+
234
+
In `get_notifications` (around line 1156-1162), update cursor generation:
235
+
236
+
```gleam
237
+
let end_cursor = case list.last(trimmed) {
238
+
Ok(record) -> {
239
+
// Encode cursor as rkey|uri for notification pagination
240
+
let cursor_content = record.rkey <> "|" <> record.uri
241
+
Some(pagination.encode_base64(cursor_content))
242
+
}
243
+
Error(_) -> None
244
+
}
245
+
```
246
+
247
+
**Step 2: Run tests**
248
+
249
+
```bash
250
+
cd /Users/chadmiller/code/quickslice/server && gleam test 2>&1 | tail -20
251
+
```
252
+
253
+
**Step 3: Commit**
254
+
255
+
```bash
256
+
git add server/src/database/repositories/records.gleam
257
+
git commit -m "feat(notifications): update cursor to use rkey|uri format"
258
+
```
259
+
260
+
---
261
+
262
+
### Task 6: Add rkey to Pagination Field Extraction
263
+
264
+
**Files:**
265
+
- Modify: `server/src/database/queries/pagination.gleam`
266
+
267
+
**Step 1: Add rkey to extract_field_value**
268
+
269
+
Update the `extract_field_value` function:
270
+
271
+
```gleam
272
+
pub fn extract_field_value(record: Record, field: String) -> String {
273
+
case field {
274
+
"uri" -> record.uri
275
+
"cid" -> record.cid
276
+
"did" -> record.did
277
+
"collection" -> record.collection
278
+
"indexed_at" -> record.indexed_at
279
+
"rkey" -> record.rkey
280
+
_ -> extract_json_field(record.json, field)
281
+
}
282
+
}
283
+
```
284
+
285
+
**Step 2: Add rkey to build_cursor_field_reference**
286
+
287
+
```gleam
288
+
fn build_cursor_field_reference(exec: Executor, field: String) -> String {
289
+
case field {
290
+
"uri" | "cid" | "did" | "collection" | "indexed_at" | "rkey" -> field
291
+
_ -> executor.json_extract(exec, "json", field)
292
+
}
293
+
}
294
+
```
295
+
296
+
**Step 3: Add rkey to build_order_by**
297
+
298
+
```gleam
299
+
// In build_order_by, update the field_ref case:
300
+
let field_ref = case field_name {
301
+
"uri" | "cid" | "did" | "collection" | "indexed_at" | "rkey" ->
302
+
table_prefix <> field_name
303
+
// ... rest unchanged
304
+
}
305
+
```
306
+
307
+
**Step 4: Run tests**
308
+
309
+
```bash
310
+
cd /Users/chadmiller/code/quickslice/server && gleam test 2>&1 | tail -20
311
+
```
312
+
313
+
**Step 5: Commit**
314
+
315
+
```bash
316
+
git add server/src/database/queries/pagination.gleam
317
+
git commit -m "feat(pagination): add rkey field support"
318
+
```
319
+
320
+
---
321
+
322
+
### Task 7: Update Notification E2E Tests
323
+
324
+
**Files:**
325
+
- Modify: `server/test/graphql/notifications_e2e_test.gleam`
326
+
327
+
**Step 1: Update test fixtures to use TID-like rkeys**
328
+
329
+
Ensure test records use TID-format rkeys so sorting is predictable:
330
+
331
+
```gleam
332
+
// Use TIDs that sort in a known order
333
+
// Earlier TID (should appear second when sorted DESC)
334
+
let earlier_rkey = "3kaungnepas2t"
335
+
// Later TID (should appear first when sorted DESC)
336
+
let later_rkey = "3maungnepas2t"
337
+
```
338
+
339
+
**Step 2: Update assertions to verify rkey-based ordering**
340
+
341
+
```gleam
342
+
// Records should be ordered by rkey DESC
343
+
// So later_rkey record should come before earlier_rkey record
344
+
```
345
+
346
+
**Step 3: Run notification tests**
347
+
348
+
```bash
349
+
cd /Users/chadmiller/code/quickslice/server && gleam test 2>&1 | grep -A5 "notification"
350
+
```
351
+
352
+
**Step 4: Commit**
353
+
354
+
```bash
355
+
git add server/test/graphql/notifications_e2e_test.gleam
356
+
git commit -m "test(notifications): update e2e tests for rkey-based sorting"
357
+
```
358
+
359
+
---
360
+
361
+
### Task 8: Run Full Test Suite and Verify
362
+
363
+
**Step 1: Run all tests**
364
+
365
+
```bash
366
+
cd /Users/chadmiller/code/quickslice/server && gleam test 2>&1
367
+
```
368
+
369
+
Expected: All tests pass
370
+
371
+
**Step 2: Verify migration works on fresh database**
372
+
373
+
```bash
374
+
cd /Users/chadmiller/code/quickslice/server && rm -f quickslice.db && gleam run -- migrate
375
+
```
376
+
377
+
**Step 3: Final commit if any fixes needed**
378
+
379
+
```bash
380
+
git add -A
381
+
git commit -m "fix: address test failures from rkey sorting migration"
382
+
```
383
+
384
+
---
385
+
386
+
## Summary
387
+
388
+
After completing these tasks:
389
+
390
+
1. **Records table** has a generated `rkey` column that extracts the last URI segment
391
+
2. **Notifications** are sorted by `rkey DESC` instead of `indexed_at DESC`
392
+
3. **Cursor pagination** uses `(rkey, uri)` tuple for stable pagination
393
+
4. **TID-based records** (majority) sort chronologically by creation time
394
+
5. **Non-TID rkeys** sort lexicographically (deterministic, just not chronological)
395
+
396
+
This provides chronological notification ordering across collections without relying on arbitrary `indexed_at` timestamps.