Game sync and live services for independent game developers (targeting itch.io)
Initial commit: Phase 1 foundation
+3405
+676
.opencode/plans/DESIGN-v2.md
+676
.opencode/plans/DESIGN-v2.md
···
1
+
# Scratchback Design v2
2
+
3
+
## Overview
4
+
5
+
Scratchback provides **live services, achievements, and game data synchronization** for indie games, starting with Godot and Bevy engines. Core logic in Rust, leveraging AT Protocol for federation and Automerge (via autosurgeon) for conflict-free sync.
6
+
7
+
> **Existing foundation**: See [initial-plan/00-introduction.md](../docs/initial-plan/00-introduction.md) for PDS architecture, GDPR compliance, and storage backends.
8
+
9
+
---
10
+
11
+
## Refined Scope & Decisions
12
+
13
+
### Business Model
14
+
- **Open source foundation** with deferred pricing decisions
15
+
- Focus on building community and feature parity first
16
+
17
+
### Game Identity & Integration
18
+
- **itch.io exclusive**: All games must be published on itch.io
19
+
- **OAuth linking**: Players connect itch.io account to scratchback for ownership verification
20
+
- **Metadata caching**: itch.io data cached locally, extended with scratchback-specific metadata (WASM validation modules, custom lexicons)
21
+
22
+
### Architecture Scope
23
+
- **Local-first sync**: All operations work offline, merge when online (Automerge via autosurgeon)
24
+
- **Per-slot documents**: Separate Automerge documents for each save slot/device combination
25
+
- **Scale-ready**: Stateless components with external state stores
26
+
- **Hybrid social graph**: Import Bluesky follows, maintain gaming-specific friend lists
27
+
28
+
### MVP Timeline: 16 weeks (conservative)
29
+
30
+
---
31
+
32
+
## Core Concepts
33
+
34
+
### Privacy Model (Hybrid)
35
+
36
+
| Data Type | Visibility | Storage |
37
+
|-----------|------------|---------|
38
+
| Achievement unlocks | Public | AT Protocol repo record |
39
+
| Achievement definitions | Public | Lexicon + game developer repo |
40
+
| Game state (saves) | Private | Encrypted blob (Age) |
41
+
| Player stats | Configurable | AT Protocol repo (opt-in public) |
42
+
| Leaderboard entries | Public | AT Protocol repo record |
43
+
| Presence/Activity | Ephemeral | XRPC subscription |
44
+
45
+
---
46
+
47
+
## 1. Achievement System
48
+
49
+
### Visibility Levels
50
+
51
+
```rust
52
+
pub enum AchievementVisibility {
53
+
Public,
54
+
HiddenUntilUnlocked,
55
+
AlwaysPrivate,
56
+
}
57
+
```
58
+
59
+
**Lexicon**: `co.scratchback.achievement.def`
60
+
61
+
```json
62
+
{
63
+
"lexicon": 1,
64
+
"id": "co.scratchback.achievement.def",
65
+
"defs": {
66
+
"main": {
67
+
"type": "record",
68
+
"key": "tid",
69
+
"record": {
70
+
"type": "object",
71
+
"required": ["gameId", "achievementId", "name", "visibility"],
72
+
"properties": {
73
+
"gameId": { "type": "string", "format": "did" },
74
+
"achievementId": { "type": "string" },
75
+
"name": { "type": "string", "maxLength": 64 },
76
+
"description": { "type": "string", "maxLength": 256 },
77
+
"icon": { "type": "string", "format": "uri" },
78
+
"iconLocked": { "type": "string", "format": "uri" },
79
+
"visibility": { "type": "string", "enum": ["public", "hidden", "private"] },
80
+
"rarity": { "type": "integer", "minimum": 0, "maximum": 100 },
81
+
"createdAt": { "type": "string", "format": "datetime" }
82
+
}
83
+
}
84
+
}
85
+
}
86
+
}
87
+
```
88
+
89
+
### Player Achievement Record
90
+
91
+
**Lexicon**: `co.scratchback.achievement.unlock`
92
+
93
+
```json
94
+
{
95
+
"lexicon": 1,
96
+
"id": "co.scratchback.achievement.unlock",
97
+
"defs": {
98
+
"main": {
99
+
"type": "record",
100
+
"key": "tid",
101
+
"record": {
102
+
"type": "object",
103
+
"required": ["gameId", "achievementId", "unlockedAt"],
104
+
"properties": {
105
+
"gameId": { "type": "string", "format": "did" },
106
+
"achievementId": { "type": "string" },
107
+
"unlockedAt": { "type": "string", "format": "datetime" },
108
+
"progress": { "type": "integer", "minimum": 0, "maximum": 100 }
109
+
}
110
+
}
111
+
}
112
+
}
113
+
}
114
+
```
115
+
116
+
### Private Achievements
117
+
118
+
Private achievements are stored as **encrypted records** within the game state blob, never appearing in the public AT Protocol repo:
119
+
120
+
```rust
121
+
#[derive(Reconcile, Hydrate)]
122
+
struct PrivateAchievementState {
123
+
achievement_id: String,
124
+
unlocked: bool,
125
+
unlocked_at: Option<DateTime<Utc>>,
126
+
progress: u8,
127
+
}
128
+
```
129
+
130
+
---
131
+
132
+
## 2. Game Data Synchronization
133
+
134
+
### Using Autosurgeon + Automerge
135
+
136
+
Game state uses **Automerge** for conflict-free replication with **autosurgeon** for Rust type mapping.
137
+
138
+
**Key pattern**: Derive `Reconcile` and `Hydrate` for game state:
139
+
140
+
```rust
141
+
use autosurgeon::{Reconcile, Hydrate};
142
+
143
+
#[derive(Debug, Clone, Reconcile, Hydrate)]
144
+
struct GameState {
145
+
#[key]
146
+
slot_id: u32,
147
+
player_name: String,
148
+
level: u32,
149
+
position: Position,
150
+
inventory: Vec<InventoryItem>,
151
+
flags: HashMap<String, serde_json::Value>,
152
+
stats: PlayerStats,
153
+
private_achievements: Vec<PrivateAchievementState>,
154
+
}
155
+
156
+
#[derive(Debug, Clone, Reconcile, Hydrate)]
157
+
struct Position {
158
+
x: f32,
159
+
y: f32,
160
+
z: f32,
161
+
}
162
+
163
+
#[derive(Debug, Clone, Reconcile, Hydrate)]
164
+
struct InventoryItem {
165
+
#[key]
166
+
item_id: String,
167
+
quantity: u32,
168
+
metadata: HashMap<String, String>,
169
+
}
170
+
171
+
#[derive(Debug, Clone, Reconcile, Hydrate)]
172
+
struct PlayerStats {
173
+
#[autosurgeon(reconcile = "reconcile_counter", hydrate = "hydrate_counter")]
174
+
playtime_seconds: u64,
175
+
deaths: u32,
176
+
enemies_defeated: u32,
177
+
}
178
+
```
179
+
180
+
### Sync Flow
181
+
182
+
```
183
+
Game Client ──────┬──────► Encrypt with Age ────► Upload blob
184
+
│
185
+
└──────► Reconcile to Automerge doc
186
+
│
187
+
▼
188
+
Fork/Merge for concurrent saves
189
+
│
190
+
▼
191
+
Hydrate back to GameState struct
192
+
```
193
+
194
+
### Lexicon for Sync State
195
+
196
+
**Lexicon**: `co.scratchback.sync.state`
197
+
198
+
```json
199
+
{
200
+
"lexicon": 1,
201
+
"id": "co.scratchback.sync.state",
202
+
"defs": {
203
+
"main": {
204
+
"type": "record",
205
+
"record": {
206
+
"type": "object",
207
+
"required": ["gameId", "slotId", "blobRef", "updatedAt"],
208
+
"properties": {
209
+
"gameId": { "type": "string", "format": "did" },
210
+
"slotId": { "type": "integer" },
211
+
"blobRef": { "type": "blob" },
212
+
"deviceName": { "type": "string" },
213
+
"updatedAt": { "type": "string", "format": "datetime" }
214
+
}
215
+
}
216
+
}
217
+
}
218
+
}
219
+
```
220
+
221
+
---
222
+
223
+
## 3. Live Services
224
+
225
+
### 3.1 Leaderboards
226
+
227
+
**Validation**: Server-validated via WASM modules (wasmtime runtime, 100ms execution, 10MB memory)
228
+
229
+
**Lexicon**: `co.scratchback.leaderboard.entry`
230
+
231
+
```json
232
+
{
233
+
"lexicon": 1,
234
+
"id": "co.scratchback.leaderboard.entry",
235
+
"defs": {
236
+
"main": {
237
+
"type": "record",
238
+
"key": "tid",
239
+
"record": {
240
+
"type": "object",
241
+
"required": ["gameId", "boardId", "score", "playerDid"],
242
+
"properties": {
243
+
"gameId": { "type": "string", "format": "did" },
244
+
"boardId": { "type": "string" },
245
+
"playerDid": { "type": "string", "format": "did" },
246
+
"score": { "type": "integer" },
247
+
"metadata": { "type": "unknown" },
248
+
"submittedAt": { "type": "string", "format": "datetime" }
249
+
}
250
+
}
251
+
}
252
+
}
253
+
}
254
+
```
255
+
256
+
**XRPC Query**: `co.scratchback.leaderboard.getRankings`
257
+
258
+
```json
259
+
{
260
+
"lexicon": 1,
261
+
"id": "co.scratchback.leaderboard.getRankings",
262
+
"defs": {
263
+
"main": {
264
+
"type": "query",
265
+
"parameters": {
266
+
"type": "params",
267
+
"required": ["gameId", "boardId"],
268
+
"properties": {
269
+
"gameId": { "type": "string" },
270
+
"boardId": { "type": "string" },
271
+
"limit": { "type": "integer", "maximum": 100 },
272
+
"cursor": { "type": "string" },
273
+
"aroundPlayer": { "type": "string", "format": "did" }
274
+
}
275
+
},
276
+
"output": {
277
+
"encoding": "application/json",
278
+
"schema": {
279
+
"type": "object",
280
+
"properties": {
281
+
"entries": {
282
+
"type": "array",
283
+
"items": { "type": "ref", "ref": "#entry" }
284
+
},
285
+
"cursor": { "type": "string" }
286
+
}
287
+
}
288
+
}
289
+
}
290
+
}
291
+
}
292
+
```
293
+
294
+
### 3.2 Seasonal Events
295
+
296
+
**Lexicon**: `co.scratchback.event.season`
297
+
298
+
```json
299
+
{
300
+
"lexicon": 1,
301
+
"id": "co.scratchback.event.season",
302
+
"defs": {
303
+
"main": {
304
+
"type": "record",
305
+
"record": {
306
+
"type": "object",
307
+
"required": ["gameId", "seasonId", "name", "startsAt", "endsAt"],
308
+
"properties": {
309
+
"gameId": { "type": "string", "format": "did" },
310
+
"seasonId": { "type": "string" },
311
+
"name": { "type": "string", "maxLength": 64 },
312
+
"description": { "type": "string", "maxLength": 512 },
313
+
"startsAt": { "type": "string", "format": "datetime" },
314
+
"endsAt": { "type": "string", "format": "datetime" },
315
+
"rewards": {
316
+
"type": "array",
317
+
"items": { "type": "ref", "ref": "#reward" }
318
+
}
319
+
}
320
+
}
321
+
},
322
+
"reward": {
323
+
"type": "object",
324
+
"properties": {
325
+
"rewardId": { "type": "string" },
326
+
"type": { "type": "string", "enum": ["achievement", "item", "cosmetic", "badge"] },
327
+
"threshold": { "type": "integer" }
328
+
}
329
+
}
330
+
}
331
+
}
332
+
```
333
+
334
+
**XRPC Query**: `co.scratchback.event.getActive`
335
+
336
+
```json
337
+
{
338
+
"lexicon": 1,
339
+
"id": "co.scratchback.event.getActive",
340
+
"defs": {
341
+
"main": {
342
+
"type": "query",
343
+
"parameters": {
344
+
"type": "params",
345
+
"required": ["gameId"],
346
+
"properties": {
347
+
"gameId": { "type": "string" }
348
+
}
349
+
},
350
+
"output": {
351
+
"encoding": "application/json",
352
+
"schema": {
353
+
"type": "object",
354
+
"properties": {
355
+
"events": {
356
+
"type": "array",
357
+
"items": { "type": "ref", "ref": "co.scratchback.event.season" }
358
+
}
359
+
}
360
+
}
361
+
}
362
+
}
363
+
}
364
+
}
365
+
```
366
+
367
+
### 3.3 Push Notifications
368
+
369
+
Uses existing PDS infrastructure with ntfy.sh (see [02-requirements.md](../docs/initial-plan/02-requirements.md)).
370
+
371
+
**XRPC Procedure**: `co.scratchback.notification.subscribe`
372
+
373
+
```json
374
+
{
375
+
"lexicon": 1,
376
+
"id": "co.scratchback.notification.subscribe",
377
+
"defs": {
378
+
"main": {
379
+
"type": "procedure",
380
+
"input": {
381
+
"encoding": "application/json",
382
+
"schema": {
383
+
"type": "object",
384
+
"required": ["topics"],
385
+
"properties": {
386
+
"topics": {
387
+
"type": "array",
388
+
"items": { "type": "string", "enum": ["achievements", "events", "friends", "leaderboards"] }
389
+
}
390
+
}
391
+
}
392
+
},
393
+
"output": {
394
+
"encoding": "application/json",
395
+
"schema": {
396
+
"type": "object",
397
+
"properties": {
398
+
"endpoint": { "type": "string" },
399
+
"topic": { "type": "string" }
400
+
}
401
+
}
402
+
}
403
+
}
404
+
}
405
+
}
406
+
```
407
+
408
+
### 3.4 Async Multiplayer Platform
409
+
410
+
**Components**:
411
+
- **Friend graph**: Hybrid (import Bluesky follows + gaming-specific)
412
+
- **Presence**: Real-time status updates
413
+
- **Matchmaking**: Multi-factor (region + skill + ping)
414
+
- **Lobbies**: Game-configurable persistence for turn-based games
415
+
- **Turn sync**: Conflictable turn/game state via Automerge
416
+
417
+
**XRPC Subscribe**: `co.scratchback.presence.subscribe`
418
+
419
+
```json
420
+
{
421
+
"lexicon": 1,
422
+
"id": "co.scratchback.presence.subscribe",
423
+
"defs": {
424
+
"main": {
425
+
"type": "subscription",
426
+
"parameters": {
427
+
"type": "params",
428
+
"properties": {
429
+
"gameId": { "type": "string" },
430
+
"friendsOnly": { "type": "boolean" }
431
+
}
432
+
},
433
+
"message": {
434
+
"schema": {
435
+
"type": "object",
436
+
"properties": {
437
+
"playerDid": { "type": "string", "format": "did" },
438
+
"status": { "type": "string", "enum": ["online", "away", "playing", "offline"] },
439
+
"gameId": { "type": "string" },
440
+
"activity": { "type": "string" },
441
+
"updatedAt": { "type": "string", "format": "datetime" }
442
+
}
443
+
}
444
+
}
445
+
}
446
+
}
447
+
}
448
+
```
449
+
450
+
**XRPC Procedure**: `co.scratchback.presence.update`
451
+
452
+
```json
453
+
{
454
+
"lexicon": 1,
455
+
"id": "co.scratchback.presence.update",
456
+
"defs": {
457
+
"main": {
458
+
"type": "procedure",
459
+
"input": {
460
+
"encoding": "application/json",
461
+
"schema": {
462
+
"type": "object",
463
+
"required": ["status"],
464
+
"properties": {
465
+
"status": { "type": "string", "enum": ["online", "away", "playing", "offline"] },
466
+
"gameId": { "type": "string" },
467
+
"activity": { "type": "string", "maxLength": 128 }
468
+
}
469
+
}
470
+
}
471
+
}
472
+
}
473
+
}
474
+
```
475
+
476
+
---
477
+
478
+
## 4. Architecture Changes
479
+
480
+
### Component Overview
481
+
482
+
```
483
+
┌─────────────────────────────────────────────────────────────────────┐
484
+
│ Scratchback PDS │
485
+
├─────────────────────────────────────────────────────────────────────┤
486
+
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
487
+
│ │ XRPC API │ │ Lexicon │ │ Sync Engine │ │
488
+
│ │ (JWT auth) │ │ Registry │ │ (Automerge) │ │
489
+
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
490
+
│ │ │ │ │
491
+
│ ┌──────▼─────────────────▼──────────────────▼──────────────────┐ │
492
+
│ │ Application State │ │
493
+
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │
494
+
│ │ │ SQLite │ │ SQLite │ │ Leader- │ │ Event │ │ │
495
+
│ │ │ (Main) │ │ (Passkeys│ │ board │ │ Store │ │ │
496
+
│ │ │ │ │ Encrypted│ │ Cache │ │ │ │ │
497
+
│ │ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │ │
498
+
│ └───────────────────────────────────────────────────────────────┘ │
499
+
│ │
500
+
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
501
+
│ │ Bunny Storage│ │ DO Spaces │ │ ntfy.sh │ │
502
+
│ │ (blobs) │ │ (blobs) │ │ (push) │ │
503
+
│ └──────────────┘ └──────────────┘ └──────────────┘ │
504
+
└─────────────────────────────────────────────────────────────────────┘
505
+
```
506
+
507
+
### New Components (from original plan)
508
+
509
+
| Component | Purpose | Doc Reference |
510
+
|-----------|---------|---------------|
511
+
| Sync Engine | Automerge reconciliation via autosurgeon | §2 |
512
+
| Lexicon Registry | Host and serve custom lexicons | §1, §3 |
513
+
| Leaderboard Cache | Materialized view of leaderboard entries | §3.1 |
514
+
| Event Store | Seasonal event definitions and progress | §3.2 |
515
+
| Presence Manager | TTL-based presence tracking | §3.4 |
516
+
517
+
---
518
+
519
+
## 5. Data Model Additions
520
+
521
+
### New Tables
522
+
523
+
```sql
524
+
-- Leaderboard materialized view (for performance)
525
+
CREATE TABLE leaderboard_cache (
526
+
id INTEGER PRIMARY KEY AUTOINCREMENT,
527
+
game_id TEXT NOT NULL,
528
+
board_id TEXT NOT NULL,
529
+
player_did TEXT NOT NULL,
530
+
score INTEGER NOT NULL,
531
+
rank INTEGER NOT NULL,
532
+
metadata TEXT,
533
+
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
534
+
UNIQUE(game_id, board_id, player_did)
535
+
);
536
+
537
+
CREATE INDEX idx_leaderboard_rank ON leaderboard_cache(game_id, board_id, rank);
538
+
539
+
-- Event progress tracking
540
+
CREATE TABLE event_progress (
541
+
id INTEGER PRIMARY KEY AUTOINCREMENT,
542
+
player_did TEXT NOT NULL,
543
+
game_id TEXT NOT NULL,
544
+
season_id TEXT NOT NULL,
545
+
progress INTEGER DEFAULT 0,
546
+
rewards_claimed TEXT,
547
+
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
548
+
UNIQUE(player_did, game_id, season_id)
549
+
);
550
+
551
+
-- Presence state (ephemeral, TTL-based)
552
+
CREATE TABLE presence (
553
+
player_did TEXT PRIMARY KEY,
554
+
game_id TEXT,
555
+
status TEXT NOT NULL DEFAULT 'offline',
556
+
activity TEXT,
557
+
expires_at TIMESTAMP NOT NULL,
558
+
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
559
+
);
560
+
561
+
CREATE INDEX idx_presence_expires ON presence(expires_at);
562
+
```
563
+
564
+
---
565
+
566
+
## 6. Engine SDKs
567
+
568
+
### Bevy SDK (Native Rust)
569
+
- **Native plugin**: Direct Rust integration, shared types with core
570
+
- **Performance**: Zero-cost abstractions, compile-time validation
571
+
- **Example**:
572
+
```rust
573
+
use bevy::prelude::*;
574
+
use scratchback_bevy::{ScratchbackPlugin, AchievementClient, SyncClient, GameState};
575
+
576
+
fn main() {
577
+
App::new()
578
+
.add_plugin(ScratchbackPlugin::new(config))
579
+
.add_system(unlock_on_boss_kill)
580
+
.run();
581
+
}
582
+
583
+
fn unlock_on_boss_kill(
584
+
query: Query<&BossKilled>,
585
+
achievements: Res<AchievementClient>,
586
+
) {
587
+
for _ in query.iter() {
588
+
achievements.unlock("boss-first-kill");
589
+
}
590
+
}
591
+
592
+
// Auto-synced game state
593
+
fn save_game_system(
594
+
game_state: Res<GameState>,
595
+
client: Res<SyncClient>,
596
+
) {
597
+
if game_state.is_dirty() {
598
+
client.reconcile(&game_state.save_data);
599
+
}
600
+
}
601
+
```
602
+
603
+
### Godot SDK (GDExtension)
604
+
- **Generated bindings**: Auto-generated from lexicon schemas
605
+
- **Type safety**: Rust-generated bindings with validation
606
+
- **Example**:
607
+
```gdscript
608
+
# Auto-generated from lexicon schemas
609
+
extends Node
610
+
var scratchback: ScratchbackClient
611
+
612
+
func _ready():
613
+
scratchback = ScratchbackClient.new()
614
+
scratchback.connect("achievement_unlocked", self, "_on_achievement")
615
+
# Automatic sync
616
+
scratchback.enable_auto_sync(save_interval=30)
617
+
618
+
func unlock_achievement(id: String):
619
+
scratchback.achievements.unlock(id)
620
+
621
+
func save_game_state(state: Dictionary):
622
+
scratchback.sync.reconcile(state)
623
+
624
+
func _on_achievement(achievement_id: String):
625
+
print("Achievement unlocked: ", achievement_id)
626
+
```
627
+
628
+
### Developer Portal
629
+
#### Web Dashboard
630
+
- Game registration and configuration
631
+
- WASM validation module upload
632
+
- Leaderboard rules setup
633
+
- Analytics and monitoring
634
+
635
+
#### CLI Tool (`scratchback-cli`)
636
+
- Game deployment to production
637
+
- Developer testing tools
638
+
- Local WASM validation testing
639
+
- Debug commands
640
+
641
+
---
642
+
643
+
## 7. Implementation Phases (Revised)
644
+
645
+
## 7. Implementation Phases (Refined)
646
+
647
+
| Phase | Duration | Focus |
648
+
|-------|----------|-------|
649
+
| 1-2 | 8 days | Foundation + itch.io OAuth integration, WASM validation runtime setup |
650
+
| 3-4 | 8 days | Autosurgeon sync engine (per-slot, local-first) |
651
+
| 5-6 | 8 days | Achievement lexicons + local-first sync implementation |
652
+
| 7-8 | 8 days | Leaderboards + WASM validation modules |
653
+
| 9-10 | 8 days | Events system + async multiplayer infrastructure |
654
+
| 11-12 | 8 days | Bevy SDK (native Rust, shared types) + Godot SDK (GDExtension) |
655
+
| 13 | 8 days | Dev portal (web dashboard + CLI) |
656
+
| 14 | 8 days | Testing, staging integration, deployment |
657
+
658
+
**Total**: 16 weeks (112 days) - Conservative timeline
659
+
660
+
### Development Order
661
+
1. **Core services**: OAuth, sync engine, autosurgeon integration
662
+
2. **Primary features**: Achievements, leaderboards, events
663
+
3. **Platform**: Async multiplayer (friends, presence, lobbies)
664
+
4. **SDKs**: Bevy (native) + Godot (generated bindings)
665
+
5. **Dev tools**: Web portal + CLI
666
+
6. **Quality**: Testing, integration, deployment preparation
667
+
668
+
---
669
+
670
+
## References
671
+
672
+
- [Original Architecture](../docs/initial-plan/01-architecture.md)
673
+
- [Original Data Model](../docs/initial-plan/03-data-model.md)
674
+
- [Autosurgeon docs](https://docs.rs/autosurgeon)
675
+
- [AT Protocol Lexicon Spec](https://atproto.com/specs/lexicon)
676
+
- [Steam Achievement API](https://partner.steamgames.com/doc/features/achievements)
+654
.opencode/plans/IMPLEMENTATION.md
+654
.opencode/plans/IMPLEMENTATION.md
···
1
+
# Scratchback Implementation Plan
2
+
3
+
## Overview
4
+
5
+
16-week implementation plan for Scratchback MVP, refined scope based on design discussions. Target: Live services, achievements, and game data synchronization for indie games.
6
+
7
+
---
8
+
9
+
## High-Level Timeline
10
+
11
+
| Week | Phase | Key Deliverables |
12
+
|------|-------|------------------|
13
+
| **1-2** | Foundation | ✅ Workspace setup, ✅ itch.io OAuth, ✅ WASM runtime |
14
+
| **3-4** | Sync Engine | ✅ Local-first autosurgeon, ✅ Per-slot sync, ✅ Conflict resolution |
15
+
| **5-6** | Achievements | ✅ Lexicons, ✅ Local unlocking, ✅ Sync persistence |
16
+
| **7-8** | Leaderboards | ✅ Validation modules, ✅ WASM runtime, ✅ Rankings API |
17
+
| **9-10** | Events & Async | ✅ Seasonal events, ✅ Presence, ✅ Lobbies, ✅ Matchmaking |
18
+
| **11-12** | SDKs | ✅ Bevy (native), ✅ Godot (GDExtension) |
19
+
| **13** | Dev Portal | ✅ Web dashboard, ✅ CLI tool, ✅ Game registry |
20
+
| **14** | Quality & Deploy | ✅ Testing, ✅ Staging, ✅ Production deployment |
21
+
22
+
---
23
+
24
+
## Weekly Breakdown
25
+
26
+
### Week 1-2: Foundation & Core Services
27
+
28
+
#### Week 1: Workspace & OAuth Integration
29
+
**Day 1-2: Workspace Setup**
30
+
- Create workspace with proper Cargo.toml structure
31
+
- Initialize rsky dependency (AT Protocol)
32
+
- Set up SQLite database schemas
33
+
- Configure WASM runtime (wasmtime)
34
+
35
+
**Day 3-4: itch.io OAuth Integration**
36
+
- Implement OAuth 2.0 flow with itch.io
37
+
- Store access tokens securely in SQLite
38
+
- Build user profile sync with itch.io API
39
+
- Implement game ownership verification
40
+
41
+
**Day 5: WASM Validation Runtime**
42
+
- Setup wasmtime sandbox environment
43
+
- Define validation module interface
44
+
- Build scoring validation framework
45
+
- Create WASM module loader
46
+
47
+
**Deliverables**:
48
+
- ✅ Workspace with proper structure
49
+
- ✅ itch.io OAuth implementation
50
+
- ✅ wasmtime validation runtime
51
+
- ✅ User profile management
52
+
53
+
#### Week 2: State & Configuration
54
+
**Day 1-2: Game Registry**
55
+
- Build itch.io game metadata caching
56
+
- Implement scratchback-specific metadata storage
57
+
- Create game configuration schema
58
+
- Build game lookup service
59
+
60
+
**Day 3-4: Configuration System**
61
+
- Build centralized config management
62
+
- Support environment variables + database config
63
+
- Implement configuration validation
64
+
- Create config API endpoints
65
+
66
+
**Day 5: Database Schema V2**
67
+
- Extend existing schema with new tables
68
+
- Add leaderboard_cache table
69
+
- Add event_progress table
70
+
- Add presence table
71
+
- Create proper indexes
72
+
73
+
**Deliverables**:
74
+
- ✅ Game registry with caching
75
+
- ✅ Configuration system
76
+
- ✅ Extended database schema
77
+
- ✅ Indexed queries setup
78
+
79
+
---
80
+
81
+
### Week 3-4: Sync Engine (Local-First)
82
+
83
+
#### Week 3: Autosurgeon Integration
84
+
**Day 1-2: Automerge Documents**
85
+
- Set up Automerge document storage
86
+
- Implement per-slot document strategy
87
+
- Create document lifecycle management
88
+
- Build document indexing
89
+
90
+
**Day 3-4: Autosurgeon Types**
91
+
- Implement GameState derive macros
92
+
- Create Position, InventoryItem, PlayerStats types
93
+
- Build reconcile/hydrate logic
94
+
- Test conflict-free merging
95
+
96
+
**Day 5: Sync State Tracking**
97
+
- Build dirty state detection
98
+
- Implement automatic sync triggers
99
+
- Create sync progress tracking
100
+
- Build conflict resolution logging
101
+
102
+
**Deliverables**:
103
+
- ✅ Automerge document storage
104
+
- ✅ Autosurgeon GameState types
105
+
- ✅ Conflict-free merging
106
+
- ✅ Sync state tracking
107
+
108
+
#### Week 4: Sync Engine Implementation
109
+
**Day 1-2: Blob Sync Integration**
110
+
- Integrate with existing Age encryption
111
+
- Build blob storage/retrieval
112
+
- Implement blob reference tracking
113
+
- Create blob sync consistency checks
114
+
115
+
**Day 3-4: Local-First Logic**
116
+
- Build offline operation support
117
+
- Implement queue for pending changes
118
+
- Create online conflict resolution
119
+
- Build sync status reporting
120
+
121
+
**Day 5: Sync Testing**
122
+
- Write comprehensive sync tests
123
+
- Test concurrent save scenarios
124
+
- Build sync failure recovery
125
+
- Create performance benchmarks
126
+
127
+
**Deliverables**:
128
+
- ✅ Local-first sync engine
129
+
- ✅ Offline operation support
130
+
- ✅ Conflict resolution testing
131
+
- ✅ Sync performance baseline
132
+
133
+
---
134
+
135
+
### Week 5-6: Achievement System
136
+
137
+
#### Week 5: Achievement Lexicons & Core
138
+
**Day 1-2: Lexicon Registry**
139
+
- Build lexicon storage and serving
140
+
- Implement co.scratchback.achievement.def
141
+
- Create achievement validation
142
+
- Build lexicon versioning
143
+
144
+
**Day 3-4: Achievement Storage**
145
+
- Create achievement unlock records
146
+
- Build achievement visibility logic
147
+
- Implement private achievement storage
148
+
- Create achievement indexing
149
+
150
+
**Day 5: Achievement API**
151
+
- Build achievement unlock XRPC
152
+
- Create achievement list queries
153
+
- Implement progress tracking
154
+
- Build achievement visibility rules
155
+
156
+
**Deliverables**:
157
+
- ✅ Lexicon registry system
158
+
- ✅ Achievement data model
159
+
- ✅ Achievement unlock API
160
+
- ✅ Achievement visibility logic
161
+
162
+
#### Week 6: Achievement Sync & Events
163
+
**Day 1-2: Achievement Sync**
164
+
- Integrate achievements with sync engine
165
+
- Build achievement conflict resolution
166
+
- Implement achievement sync persistence
167
+
- Create achievement restore logic
168
+
169
+
**Day 3-4: Event System Foundation**
170
+
- Build event data model
171
+
- Create event lifecycle management
172
+
- Implement event progress tracking
173
+
- Build event scoring logic
174
+
175
+
**Day 5: Achievement Testing**
176
+
- Write comprehensive achievement tests
177
+
- Test visibility scenarios
178
+
- Build achievement performance tests
179
+
- Create achievement user flow testing
180
+
181
+
**Deliverables**:
182
+
- ✅ Achievement sync integration
183
+
- ✅ Event system foundation
184
+
- ✅ Achievement testing suite
185
+
- ✅ Achievement performance baseline
186
+
187
+
---
188
+
189
+
### Week 7-8: Leaderboard System
190
+
191
+
#### Week 7: Leaderboard Core & Validation
192
+
**Day 1-2: Leaderboard Data Model**
193
+
- Implement co.scratchback.leaderboard.entry
194
+
- Create materialized view caching
195
+
- Build leaderboard indexing
196
+
- Create leaderboard ranking logic
197
+
198
+
**Day 3-4: WASM Validation Integration**
199
+
- Integrate wasmtime with leaderboard submission
200
+
- Build validation module execution
201
+
- Create validation result handling
202
+
- Implement scoring error recovery
203
+
204
+
**Day 5: Leaderboard API**
205
+
- Build submission XRPC endpoint
206
+
- Create ranking query endpoint
207
+
- Implement leaderboard pagination
208
+
- Build leaderboard filtering logic
209
+
210
+
**Deliverables**:
211
+
- ✅ Leaderboard data model
212
+
- ✅ WASM validation integration
213
+
- ✅ Leaderboard submission API
214
+
- ✅ Leaderboard ranking system
215
+
216
+
#### Week 8: Leaderboard Advanced Features
217
+
**Day 1-2: Validation Module Development**
218
+
- Create example validation modules
219
+
- Build validation module testing
220
+
- Implement validation module deployment
221
+
- Create validation module lifecycle
222
+
223
+
**Day 3-4: Leaderboard Advanced Queries**
224
+
- Build around-player ranking queries
225
+
- Create multi-board support
226
+
- Implement leaderboard snapshotting
227
+
- Build leaderboard history tracking
228
+
229
+
**Day 5: Leaderboard Testing**
230
+
- Write leaderboard comprehensive tests
231
+
- Test validation module edge cases
232
+
- Build leaderboard performance tests
233
+
- Create concurrent submission testing
234
+
235
+
**Deliverables**:
236
+
- ✅ WASM validation modules
237
+
- ✅ Advanced leaderboard queries
238
+
- ✅ Leaderboard testing suite
239
+
- ✅ Leaderboard performance baseline
240
+
241
+
---
242
+
243
+
### Week 9-10: Async Multiplayer Platform
244
+
245
+
#### Week 9: Events & Presence
246
+
**Day 1-2: Seasonal Events API**
247
+
- Implement co.scratchback.event.season
248
+
- Build event progress tracking
249
+
- Create event reward system
250
+
- Build event lifecycle management
251
+
252
+
**Day 3-4: Presence System**
253
+
- Create presence data model
254
+
- Build presence XRPC subscription
255
+
- Implement presence expiration logic
256
+
- Create presence status management
257
+
258
+
**Day 5: Friend Graph**
259
+
- Build friend relationship storage
260
+
- Implement Bluesky follow import
261
+
- Create friend list queries
262
+
- Build friend status tracking
263
+
264
+
**Deliverables**:
265
+
- ✅ Seasonal events system
266
+
- ✅ Presence tracking
267
+
- ✅ Friend graph foundation
268
+
- ✅ Relationship management
269
+
270
+
#### Week 10: Async Multiplayer Complete
271
+
**Day 1-2: Lobbies & Turn Sync**
272
+
- Build lobby data model
273
+
- Create lobby lifecycle management
274
+
- Implement turn-based state sync
275
+
- Build lobby matchmaking logic
276
+
277
+
**Day 3-4: Matchmaking System**
278
+
- Implement ELO rating system
279
+
- Build multi-factor matching (region + skill + ping)
280
+
- Create matchmaking queue management
281
+
- Build lobby matching logic
282
+
283
+
**Day 5: Async Multiplayer Testing**
284
+
- Write async comprehensive tests
285
+
- Test concurrent game states
286
+
- Build matchmaking performance tests
287
+
- Create multiplayer edge case testing
288
+
289
+
**Deliverables**:
290
+
- ✅ Lobbies system
291
+
- ✅ Turn-based sync
292
+
- ✅ Multi-factor matchmaking
293
+
- ✅ Async multiplayer testing
294
+
295
+
---
296
+
297
+
### Week 11-12: Engine SDKs
298
+
299
+
#### Week 11: Bevy SDK
300
+
**Day 1-2: Plugin Architecture**
301
+
- Create Bevy plugin structure
302
+
- Build Rust type definitions
303
+
- Implement resource system integration
304
+
- Create event system integration
305
+
306
+
**Day 3-4: Achievement Integration**
307
+
- Build achievement unlock system
308
+
- Create achievement tracking resources
309
+
- Implement achievement UI integration
310
+
- Build achievement persistence
311
+
312
+
**Day 5: Sync Integration**
313
+
- Implement local-first sync plugin
314
+
- Create game state serialization
315
+
- Build automatic sync integration
316
+
- Create conflict resolution for Bevy
317
+
318
+
**Deliverables**:
319
+
- ✅ Bevy plugin architecture
320
+
- ✅ Achievement Bevy integration
321
+
- ✅ Local-first sync for Bevy
322
+
- ✅ Bevy comprehensive examples
323
+
324
+
#### Week 12: Godot SDK
325
+
**Day 1-2: GDExtension Setup**
326
+
- Initialize GDExtension project
327
+
- Build Rust bindings generation
328
+
- Create Godot plugin structure
329
+
- Implement type system mapping
330
+
331
+
**Day 3-4: Godot Achievement System**
332
+
- Create achievement GDScript bindings
333
+
- Build achievement tracking system
334
+
- Implement achievement unlock flow
335
+
- Create achievement UI components
336
+
337
+
**Day 5: Godot Sync Integration**
338
+
- Build save/load game interface
339
+
- Create conflict resolution system
340
+
- Implement automatic sync
341
+
- Build Godot event system integration
342
+
343
+
**Deliverables**:
344
+
- ✅ GDExtension bindings
345
+
- ✅ Achievement Godot integration
346
+
- ✅ Local-first sync for Godot
347
+
- ✅ Godot comprehensive examples
348
+
349
+
---
350
+
351
+
### Week 13: Developer Portal
352
+
353
+
#### Week 13: Dev Tools
354
+
**Day 1-2: Web Dashboard Foundation**
355
+
- Create React/Angular web app
356
+
- Build user authentication
357
+
- Implement game registration
358
+
- Create basic admin UI
359
+
360
+
**Day 3-4: Game Management**
361
+
- Build game configuration UI
362
+
- Create WASM module upload
363
+
- Implement leaderboards setup
364
+
- Build analytics dashboard
365
+
366
+
**Day 5: CLI Tool**
367
+
- Create scratchback-cli binary
368
+
- Implement deployment commands
369
+
- Build local testing tools
370
+
- Create configuration management
371
+
372
+
**Deliverables**:
373
+
- ✅ Web developer portal
374
+
- ✅ Game management interface
375
+
- ✅ WASM module management
376
+
- ✅ scratchback-cli tool
377
+
378
+
---
379
+
380
+
### Week 14: Quality & Deployment
381
+
382
+
#### Week 14: Testing & Production
383
+
**Day 1-2: Comprehensive Testing**
384
+
- Write end-to-end integration tests
385
+
- Create load testing for sync
386
+
- Build security testing
387
+
- Create user acceptance tests
388
+
389
+
**Day 3-4: Staging Environment**
390
+
- Setup staging server
391
+
- Deploy to staging
392
+
- Integration testing on staging
393
+
- Performance optimization
394
+
395
+
**Day 5: Production Deployment**
396
+
- Deploy to production
397
+
- Monitor initial performance
398
+
- Create deployment automation
399
+
- Document operational procedures
400
+
401
+
**Deliverables**:
402
+
- ✅ Comprehensive test suite
403
+
- ✅ Staging environment
404
+
- ✅ Production deployment
405
+
- ✅ Operational documentation
406
+
407
+
---
408
+
409
+
## Technical Architecture Implementation
410
+
411
+
### Component Implementation Order
412
+
413
+
1. **Core Services** (Week 1-2)
414
+
```
415
+
OAuth → WASM Runtime → Config → Registry
416
+
```
417
+
418
+
2. **Sync Foundation** (Week 3-4)
419
+
```
420
+
Automerge → Autosurgeon → Local-First → Blob Sync
421
+
```
422
+
423
+
3. **Primary Features** (Week 5-6)
424
+
```
425
+
Achievements → Events → Achievement Sync
426
+
```
427
+
428
+
4. **Platform Services** (Week 7-8)
429
+
```
430
+
Leaderboards → WASM Validation → Advanced Queries
431
+
```
432
+
433
+
5. **Multiplayer Platform** (Week 9-10)
434
+
```
435
+
Presence → Friends → Lobbies → Matchmaking
436
+
```
437
+
438
+
6. **Developer Tools** (Week 11-13)
439
+
```
440
+
Bevy SDK → Godot SDK → Web Portal → CLI
441
+
```
442
+
443
+
7. **Quality Assurance** (Week 14)
444
+
```
445
+
Testing → Staging → Production
446
+
```
447
+
448
+
### Database Schema Implementation
449
+
450
+
```sql
451
+
-- Priority tables for Week 2
452
+
CREATE TABLE IF NOT EXISTS games (
453
+
id INTEGER PRIMARY KEY AUTOINCREMENT,
454
+
itch_io_id TEXT NOT NULL UNIQUE,
455
+
name TEXT NOT NULL,
456
+
developer_did TEXT NOT NULL,
457
+
wasm_validation_module BLOB,
458
+
config TEXT NOT NULL,
459
+
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
460
+
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
461
+
);
462
+
463
+
CREATE TABLE IF NOT EXISTS leaderboard_cache (
464
+
id INTEGER PRIMARY KEY AUTOINCREMENT,
465
+
game_id TEXT NOT NULL,
466
+
board_id TEXT NOT NULL,
467
+
player_did TEXT NOT NULL,
468
+
score INTEGER NOT NULL,
469
+
rank INTEGER NOT NULL,
470
+
metadata TEXT,
471
+
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
472
+
UNIQUE(game_id, board_id, player_did)
473
+
);
474
+
475
+
CREATE TABLE IF NOT EXISTS event_progress (
476
+
id INTEGER PRIMARY KEY AUTOINCREMENT,
477
+
player_did TEXT NOT NULL,
478
+
game_id TEXT NOT NULL,
479
+
season_id TEXT NOT NULL,
480
+
progress INTEGER DEFAULT 0,
481
+
rewards_claimed TEXT,
482
+
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
483
+
UNIQUE(player_did, game_id, season_id)
484
+
);
485
+
486
+
CREATE TABLE IF NOT EXISTS presence (
487
+
player_did TEXT PRIMARY KEY,
488
+
game_id TEXT,
489
+
status TEXT NOT NULL DEFAULT 'offline',
490
+
activity TEXT,
491
+
expires_at TIMESTAMP NOT NULL,
492
+
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
493
+
);
494
+
```
495
+
496
+
### WASM Module Interface
497
+
498
+
```rust
499
+
// Validation module trait
500
+
pub trait ScoreValidator {
501
+
fn validate_score(&self, score: i64, context: &ScoreContext) -> ValidationResult;
502
+
fn get_score_range(&self) -> ScoreRange;
503
+
fn get_rules(&self) -> ValidationRules;
504
+
}
505
+
506
+
// Module loading
507
+
pub struct ValidationModule {
508
+
module: wasmtime::Module,
509
+
instance: wasmtime::Instance,
510
+
score_validator: ScoreValidatorHandle,
511
+
}
512
+
```
513
+
514
+
### API Implementation Priority
515
+
516
+
| API Group | Priority | Week |
517
+
|-----------|----------|------|
518
+
| OAuth & Auth | Highest | 1 |
519
+
| Sync Engine | Highest | 3 |
520
+
| Achievements | High | 5 |
521
+
| Leaderboards | High | 7 |
522
+
| Events | Medium | 9 |
523
+
| Presence | Medium | 9 |
524
+
| Matchmaking | Medium | 10 |
525
+
| SDK APIs | Medium | 11-12 |
526
+
| Dev APIs | Low | 13 |
527
+
528
+
---
529
+
530
+
## Risk Management
531
+
532
+
### High Priority Risks
533
+
534
+
1. **WASM Security** (Week 2)
535
+
- Risk: Malicious validation modules
536
+
- Mitigation: Strict wasmtime sandbox, memory limits, execution timeouts
537
+
538
+
2. **Sync Conflicts** (Week 3-4)
539
+
- Risk: Complex merge scenarios
540
+
- Mitigation: Comprehensive testing, conflict resolution fallbacks
541
+
542
+
3. **Performance** (Week 4, 7, 10)
543
+
- Risk: Slow sync operations, leaderboard queries
544
+
- Mitigation: Materialized views, query optimization
545
+
546
+
### Medium Priority Risks
547
+
548
+
1. **itch.io Integration** (Week 1-2)
549
+
- Risk: API rate limits, breaking changes
550
+
- Mitigation: Caching, graceful degradation
551
+
552
+
2. **AT Protocol Federation** (Week 5-6)
553
+
- Risk: Complex interop issues
554
+
- Mitigation: Comprehensive testing, fallback mechanisms
555
+
556
+
3. **Multiplayer Sync** (Week 9-10)
557
+
- Risk: State explosion, performance issues
558
+
- Mitigation: Differential sync, conflict-free data types
559
+
560
+
---
561
+
562
+
## Success Metrics
563
+
564
+
### Technical Metrics
565
+
566
+
| Metric | Target | Measurement |
567
+
|--------|--------|-------------|
568
+
| Sync Latency | < 500ms for 1MB game states | 99th percentile |
569
+
| WASM Validation | < 100ms for standard validation | Average execution time |
570
+
| Leaderboard Queries | < 100ms for top 1000 entries | Query response time |
571
+
| Availability | 99.9% uptime | Uptime monitoring |
572
+
| Concurrent Syncs | Support 1000 concurrent syncs | Load testing |
573
+
574
+
### User Metrics
575
+
576
+
| Metric | Target | Measurement |
577
+
|--------|--------|-------------|
578
+
| Game Integration | 5+ games using SDKs | Developer adoption |
579
+
| Achievement Unlocks | 1000+ unlocks/day | Usage analytics |
580
+
| Leaderboard Submissions | 100+ submissions/day | Platform engagement |
581
+
| Async Multiplayer | 10+ active lobbies | Platform usage |
582
+
583
+
---
584
+
585
+
## Resources Needed
586
+
587
+
### Team
588
+
- **1 Rust Backend Developer** (Full project)
589
+
- **1 Web Developer** (Week 13)
590
+
- **1 DevOps Engineer** (Week 14)
591
+
- **QA Engineer** (Week 14)
592
+
593
+
### Infrastructure
594
+
- **Hetzner VPS** (Production)
595
+
- **Staging Server** (Dedicated)
596
+
- **PostgreSQL** (Analytics database)
597
+
- **Redis** (Caching)
598
+
- **Monitoring Stack** (Prometheus + Grafana)
599
+
600
+
### Tooling
601
+
- **GitHub Actions** (CI/CD)
602
+
- **Docker** (Containerization)
603
+
- **Terraform** (Infrastructure as Code)
604
+
- **Semgrep** (Security scanning)
605
+
606
+
---
607
+
608
+
## Continuous Integration
609
+
610
+
### Build Pipeline
611
+
1. **Rust Code**: Clippy formatting, unit tests, integration tests
612
+
2. **WASM Modules: Validation testing, security scanning
613
+
3. **Web UI**: Jest testing, accessibility checks
614
+
4. **Docker Builds**: Multi-architecture, security scanning
615
+
5. **Deployments**: Staging → Production with approval gates
616
+
617
+
### Testing Strategy
618
+
- **Unit Tests**: 80%+ coverage for core libraries
619
+
- **Integration Tests**: All API endpoints, auth flows
620
+
- **E2E Tests**: User journeys, SDK integration
621
+
- **Load Tests**: 1000 concurrent users
622
+
- **Security Tests**: WASM sandboxing, input validation
623
+
624
+
### Monitoring
625
+
- **Application Metrics**: sync latency, API response times, error rates
626
+
- **Business Metrics**: active users, unlocks, submissions
627
+
- **Infrastructure Metrics**: CPU, memory, disk usage, network traffic
628
+
- **Security Metrics**: failed auth attempts, suspicious activity
629
+
630
+
---
631
+
632
+
## Documentation
633
+
634
+
### Technical Docs
635
+
- **API Documentation**: OpenAPI/Swagger for all XRPC endpoints
636
+
- **SDK Documentation**: Bevy and Godot integration guides
637
+
- **WASM Docs**: Validation module development guide
638
+
- **Architecture Docs**: System design and component interactions
639
+
640
+
### Developer Docs
641
+
- **Getting Started**: Quick start guide for game developers
642
+
- **Integration Guides**: Step-by-step SDK integration
643
+
- **Configuration**: WASM module deployment, leaderboard setup
644
+
- **Troubleshooting**: Common issues and solutions
645
+
646
+
### Operational Docs
647
+
- **Deployment Guide**: Production deployment procedures
648
+
- **Monitoring Guide**: Metrics, alerts, troubleshooting
649
+
- **Scaling Guide**: Horizontal scaling procedures
650
+
- **Disaster Recovery**: Backup and restore procedures
651
+
652
+
---
653
+
654
+
This implementation plan provides a comprehensive roadmap for building Scratchback MVP within the 16-week timeline. Each week builds on previous work with clear deliverables and quality gates.
+88
Cargo.toml
+88
Cargo.toml
···
1
+
[workspace]
2
+
members = [
3
+
"api",
4
+
"admin",
5
+
"crates/sync_engine",
6
+
"crates/achievements",
7
+
"crates/leaderboards",
8
+
"crates/events",
9
+
"crates/sdk_be",
10
+
"crates/sdk_godot",
11
+
"crates/cli"
12
+
]
13
+
resolver = "2"
14
+
15
+
[workspace.package]
16
+
version = "0.1.0"
17
+
edition = "2021"
18
+
authors = ["Scratchback Team"]
19
+
license = "MIT"
20
+
repository = "https://github.com/scratchback/scratchback"
21
+
rust-version = "1.85"
22
+
23
+
[workspace.dependencies]
24
+
tokio = { version = "1.40", features = ["full"] }
25
+
26
+
poem = "3"
27
+
poem-openapi = "3"
28
+
poem-lambda = "3"
29
+
rust-embed = "8"
30
+
31
+
tower = { version = "0.5", features = ["full"] }
32
+
tower-http = { version = "0.6", features = ["cors", "compression", "trace"] }
33
+
tower-layer = "0.3"
34
+
tower-service = "0.3"
35
+
36
+
axum = { version = "0.7", features = ["json", "headers", "tokio"] }
37
+
38
+
sqlx = { version = "0.8", features = ["sqlite", "runtime-tokio-rustls", "chrono", "uuid", "ulid"] }
39
+
rusqlite = { version = "0.32", features = ["bundled"] }
40
+
41
+
serde = { version = "1.0", features = ["derive"] }
42
+
serde_json = "1.0"
43
+
chrono = { version = "0.4", features = ["serde"] }
44
+
uuid = { version = "1.8", features = ["v4", "serde"] }
45
+
ulid = { version = "1.1", features = ["serde"] }
46
+
47
+
sha2 = "0.10"
48
+
hmac = "0.12"
49
+
base64 = "0.22"
50
+
51
+
anyhow = "1.0"
52
+
thiserror = "1.0"
53
+
54
+
tracing = "0.1"
55
+
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
56
+
tracing-appender = "0.2"
57
+
58
+
config = "0.14"
59
+
60
+
jsonwebtoken = "9.3"
61
+
argon2 = "0.5"
62
+
63
+
reqwest = { version = "0.12", features = ["json", "stream", "multipart"] }
64
+
reqwest-middleware = "0.3"
65
+
reqwest-retry = "0.5"
66
+
67
+
aws-sdk-s3 = "1.48"
68
+
69
+
url = "2.5"
70
+
once_cell = "1.20"
71
+
dashmap = "6.1"
72
+
73
+
clap = { version = "4.5", features = ["derive"] }
74
+
75
+
similar = "2.2"
76
+
qbsdiff = "0.4"
77
+
78
+
bevy = "0.13"
79
+
gdnative = "0.11"
80
+
81
+
[workspace.dependencies.tracing-test]
82
+
version = "0.2"
83
+
84
+
[workspace.dependencies.mockito]
85
+
version = "1.4"
86
+
87
+
[workspace.dependencies.tokio-test]
88
+
version = "0.4"
+13
admin/Cargo.toml
+13
admin/Cargo.toml
···
1
+
[package]
2
+
name = "scratchback-admin"
3
+
version.workspace = true
4
+
edition.workspace = true
5
+
authors.workspace = true
6
+
license.workspace = true
7
+
repository.workspace = true
8
+
rust-version.workspace = true
9
+
description = "Scratchback Admin CLI - Manage invites, stats, and database"
10
+
11
+
[[bin]]
12
+
name = "scratchback-admin"
13
+
path = "src/main.rs"
+94
admin/src/db.rs
+94
admin/src/db.rs
···
1
+
use anyhow::{Context, Result};
2
+
use rusqlite::Connection;
3
+
4
+
pub async fn migrate(database: &std::path::Path) -> Result<()> {
5
+
let conn = Connection::open(database).context("Failed to open database")?;
6
+
7
+
conn.execute_batch(
8
+
r#"
9
+
CREATE TABLE IF NOT EXISTS developers (
10
+
id TEXT PRIMARY KEY,
11
+
email TEXT NOT NULL UNIQUE,
12
+
password_hash TEXT NOT NULL,
13
+
name TEXT NOT NULL,
14
+
created_at TEXT DEFAULT CURRENT_TIMESTAMP,
15
+
updated_at TEXT DEFAULT CURRENT_TIMESTAMP
16
+
);
17
+
18
+
CREATE TABLE IF NOT EXISTS games (
19
+
id TEXT PRIMARY KEY,
20
+
developer_id TEXT NOT NULL REFERENCES developers(id),
21
+
name TEXT NOT NULL,
22
+
created_at TEXT DEFAULT CURRENT_TIMESTAMP,
23
+
updated_at TEXT DEFAULT CURRENT_TIMESTAMP
24
+
);
25
+
26
+
CREATE TABLE IF NOT EXISTS api_keys (
27
+
id TEXT PRIMARY KEY,
28
+
developer_id TEXT NOT NULL REFERENCES developers(id),
29
+
game_id TEXT REFERENCES games(id),
30
+
key_hash TEXT NOT NULL,
31
+
created_at TEXT DEFAULT CURRENT_TIMESTAMP,
32
+
revoked_at TEXT
33
+
);
34
+
35
+
CREATE TABLE IF NOT EXISTS gamers (
36
+
id TEXT PRIMARY KEY,
37
+
itch_user_id INTEGER NOT NULL UNIQUE,
38
+
created_at TEXT DEFAULT CURRENT_TIMESTAMP
39
+
);
40
+
41
+
CREATE TABLE IF NOT EXISTS sessions (
42
+
id TEXT PRIMARY KEY,
43
+
gamer_id TEXT NOT NULL REFERENCES gamers(id),
44
+
expires_at TEXT NOT NULL,
45
+
created_at TEXT DEFAULT CURRENT_TIMESTAMP
46
+
);
47
+
48
+
CREATE TABLE IF NOT EXISTS invites (
49
+
code TEXT PRIMARY KEY,
50
+
used_by TEXT REFERENCES gamers(id),
51
+
used_at TEXT,
52
+
created_by TEXT NOT NULL,
53
+
expires_at TEXT NOT NULL
54
+
);
55
+
56
+
CREATE TABLE IF NOT EXISTS saves (
57
+
id TEXT PRIMARY KEY,
58
+
gamer_id TEXT NOT NULL REFERENCES gamers(id),
59
+
game_id TEXT NOT NULL REFERENCES games(id),
60
+
slot_id TEXT NOT NULL,
61
+
current_cid TEXT,
62
+
created_at TEXT DEFAULT CURRENT_TIMESTAMP,
63
+
updated_at TEXT DEFAULT CURRENT_TIMESTAMP,
64
+
UNIQUE(gamer_id, game_id, slot_id)
65
+
);
66
+
67
+
CREATE TABLE IF NOT EXISTS save_versions (
68
+
id TEXT PRIMARY KEY,
69
+
save_id TEXT NOT NULL REFERENCES saves(id),
70
+
version_number INTEGER NOT NULL,
71
+
cid TEXT NOT NULL,
72
+
milestone INTEGER DEFAULT 0,
73
+
size_bytes INTEGER NOT NULL,
74
+
created_at TEXT DEFAULT CURRENT_TIMESTAMP
75
+
);
76
+
77
+
CREATE TABLE IF NOT EXISTS dpop_tokens (
78
+
token_id TEXT PRIMARY KEY,
79
+
gamer_id TEXT NOT NULL REFERENCES gamers(id),
80
+
nonce TEXT NOT NULL,
81
+
expires_at TEXT NOT NULL,
82
+
created_at TEXT DEFAULT CURRENT_TIMESTAMP
83
+
);
84
+
85
+
CREATE INDEX IF NOT EXISTS idx_games_developer ON games(developer_id);
86
+
CREATE INDEX IF NOT EXISTS idx_saves_gamer ON saves(gamer_id);
87
+
CREATE INDEX IF NOT EXISTS idx_saves_game ON saves(game_id);
88
+
CREATE INDEX IF NOT EXISTS idx_save_versions_save ON save_versions(save_id);
89
+
"#,
90
+
)?;
91
+
92
+
println!("Migration completed successfully");
93
+
Ok(())
94
+
}
+57
admin/src/invite.rs
+57
admin/src/invite.rs
···
1
+
use anyhow::{Context, Result};
2
+
use rusqlite::Connection;
3
+
use ulid::Ulid;
4
+
5
+
pub async fn generate(count: usize, expires_days: u32, database: &std::path::Path) -> Result<()> {
6
+
let conn = Connection::open(database).context("Failed to open database")?;
7
+
8
+
let expires_at = chrono::Utc::now()
9
+
+ chrono::Duration::days(expires_days as i64);
10
+
let expires_str = expires_at.to_rfc3339();
11
+
12
+
for _ in 0..count {
13
+
let code = generate_code();
14
+
conn.execute(
15
+
"INSERT INTO invites (code, created_by, expires_at) VALUES (?1, 'admin', ?2)",
16
+
[&code, &expires_str],
17
+
)
18
+
.context("Failed to insert invite")?;
19
+
println!("{}", code);
20
+
}
21
+
22
+
Ok(())
23
+
}
24
+
25
+
pub async fn list(database: &std::path::Path, unused_only: bool) -> Result<()> {
26
+
let conn = Connection::open(database).context("Failed to open database")?;
27
+
28
+
let query = if unused_only {
29
+
"SELECT code, expires_at FROM invites WHERE used_by IS NULL"
30
+
} else {
31
+
"SELECT code, used_by, expires_at FROM invites"
32
+
};
33
+
34
+
let mut stmt = conn.prepare(query)?;
35
+
let rows = stmt.query_map([], |row| {
36
+
let code: String = row.get(0)?;
37
+
let expires_at: String = row.get(1)?;
38
+
let used_by: Option<String> = row.get(2).ok();
39
+
Ok((code, used_by, expires_at))
40
+
})?;
41
+
42
+
println!("{:<20} {:<20} {}", "CODE", "USED_BY", "EXPIRES_AT");
43
+
println!("{}", "-".repeat(60));
44
+
45
+
for row in rows {
46
+
let (code, used_by, expires_at) = row?;
47
+
let used = used_by.map(|_| "yes").unwrap_or("no");
48
+
println!("{:<20} {:<20} {}", code, used, expires_at);
49
+
}
50
+
51
+
Ok(())
52
+
}
53
+
54
+
fn generate_code() -> String {
55
+
let ulid = Ulid::new();
56
+
ulid.to_string()
57
+
}
+71
admin/src/main.rs
+71
admin/src/main.rs
···
1
+
use anyhow::Result;
2
+
use clap::{Parser, Subcommand};
3
+
use std::path::PathBuf;
4
+
5
+
mod db;
6
+
mod invite;
7
+
mod stats;
8
+
9
+
#[derive(Parser)]
10
+
#[command(name = "scratchback-admin")]
11
+
#[command(about = "Scratchback Admin CLI")]
12
+
struct Cli {
13
+
#[arg(short, long, default_value = "/var/data/scratchback/scratchback.db")]
14
+
database: PathBuf,
15
+
16
+
#[command(subcommand)]
17
+
command: Commands,
18
+
}
19
+
20
+
#[derive(Subcommand)]
21
+
enum Commands {
22
+
/// Generate invite codes
23
+
Invite {
24
+
#[command(subcommand)]
25
+
subcommand: InviteCommands,
26
+
},
27
+
/// Show statistics
28
+
Stats,
29
+
/// Run database migrations
30
+
Migrate,
31
+
}
32
+
33
+
#[derive(Subcommand)]
34
+
enum InviteCommands {
35
+
/// Generate new invite codes
36
+
Generate {
37
+
/// Number of codes to generate
38
+
#[arg(short, long, default_value = "1")]
39
+
count: usize,
40
+
41
+
/// Days until expiration (default: 30)
42
+
#[arg(short, long, default_value = "30")]
43
+
expires: u32,
44
+
},
45
+
/// List all invite codes
46
+
List {
47
+
/// Show only unused codes
48
+
#[arg(short, long)]
49
+
unused: bool,
50
+
},
51
+
}
52
+
53
+
#[tokio::main]
54
+
async fn main() -> Result<()> {
55
+
tracing_subscriber::fmt::init();
56
+
57
+
let cli = Cli::parse();
58
+
59
+
match cli.command {
60
+
Commands::Invite { subcommand } => match subcommand {
61
+
InviteCommands::Generate { count, expires } => {
62
+
invite::generate(count, expires, &cli.database).await?
63
+
}
64
+
InviteCommands::List { unused } => invite::list(&cli.database, unused).await?,
65
+
},
66
+
Commands::Stats => stats::show(&cli.database).await?,
67
+
Commands::Migrate => db::migrate(&cli.database).await?,
68
+
}
69
+
70
+
Ok(())
71
+
}
+45
admin/src/stats.rs
+45
admin/src/stats.rs
···
1
+
use anyhow::{Context, Result};
2
+
use rusqlite::Connection;
3
+
4
+
pub async fn show(database: &std::path::Path) -> Result<()> {
5
+
let conn = Connection::open(database).context("Failed to open database")?;
6
+
7
+
let developer_count: i64 = conn
8
+
.query_row("SELECT COUNT(*) FROM developers", [], |row| row.get(0))
9
+
.unwrap_or(0);
10
+
11
+
let game_count: i64 = conn
12
+
.query_row("SELECT COUNT(*) FROM games", [], |row| row.get(0))
13
+
.unwrap_or(0);
14
+
15
+
let gamer_count: i64 = conn
16
+
.query_row("SELECT COUNT(*) FROM gamers", [], |row| row.get(0))
17
+
.unwrap_or(0);
18
+
19
+
let save_count: i64 = conn
20
+
.query_row("SELECT COUNT(*) FROM saves", [], |row| row.get(0))
21
+
.unwrap_or(0);
22
+
23
+
let version_count: i64 = conn
24
+
.query_row("SELECT COUNT(*) FROM save_versions", [], |row| row.get(0))
25
+
.unwrap_or(0);
26
+
27
+
let unused_invites: i64 = conn
28
+
.query_row(
29
+
"SELECT COUNT(*) FROM invites WHERE used_by IS NULL",
30
+
[],
31
+
|row| row.get(0),
32
+
)
33
+
.unwrap_or(0);
34
+
35
+
println!("Scratchback Statistics");
36
+
println!("======================");
37
+
println!("Developers: {}", developer_count);
38
+
println!("Games: {}", game_count);
39
+
println!("Gamers: {}", gamer_count);
40
+
println!("Saves: {}", save_count);
41
+
println!("Save versions: {}", version_count);
42
+
println!("Unused invites: {}", unused_invites);
43
+
44
+
Ok(())
45
+
}
+58
api/Cargo.toml
+58
api/Cargo.toml
···
1
+
[package]
2
+
name = "scratchback-api"
3
+
version.workspace = true
4
+
edition.workspace = true
5
+
authors.workspace = true
6
+
license.workspace = true
7
+
repository.workspace = true
8
+
rust-version.workspace = true
9
+
description = "Scratchback API Server - Main backend service"
10
+
11
+
[dependencies]
12
+
tokio = { workspace = true }
13
+
14
+
poem = { workspace = true }
15
+
poem-openapi = { workspace = true }
16
+
17
+
tower = { workspace = true }
18
+
tower-http = { workspace = true }
19
+
tower-layer = { workspace = true }
20
+
tower-service = { workspace = true }
21
+
22
+
sqlx = { workspace = true }
23
+
rusqlite = { workspace = true }
24
+
serde = { workspace = true }
25
+
serde_json = { workspace = true }
26
+
chrono = { workspace = true }
27
+
uuid = { workspace = true }
28
+
ulid = { workspace = true }
29
+
sha2 = { workspace = true }
30
+
hmac = { workspace = true }
31
+
base64 = { workspace = true }
32
+
anyhow = { workspace = true }
33
+
thiserror = { workspace = true }
34
+
tracing = { workspace = true }
35
+
tracing-subscriber = { workspace = true }
36
+
config = { workspace = true }
37
+
jsonwebtoken = { workspace = true }
38
+
argon2 = { workspace = true }
39
+
reqwest = { workspace = true }
40
+
reqwest-middleware = { workspace = true }
41
+
reqwest-retry = { workspace = true }
42
+
rust-embed = { workspace = true }
43
+
aws-sdk-s3 = { workspace = true }
44
+
url = { workspace = true }
45
+
once_cell = { workspace = true }
46
+
dashmap = { workspace = true }
47
+
clap = { workspace = true, features = ["derive"] }
48
+
similar = { workspace = true }
49
+
qbsdiff = { workspace = true }
50
+
mime_guess = "2.0"
51
+
52
+
[lib]
53
+
name = "scratchback_api"
54
+
path = "src/lib.rs"
55
+
56
+
[[bin]]
57
+
name = "scratchback-api"
58
+
path = "src/main.rs"
+35
api/src/api/docs.rs
+35
api/src/api/docs.rs
···
1
+
use poem::OpenApi;
2
+
3
+
pub mod games;
4
+
pub mod saves;
5
+
6
+
pub struct Api;
7
+
8
+
impl OpenApi for Api {
9
+
#[allow(unused)]
10
+
fn components(&self) -> poem_openapi::registry::Components {
11
+
poem_openapi::registry::Components::new()
12
+
}
13
+
14
+
#[allow(unused)]
15
+
fn paths(&self) -> poem_openapi::registry::Paths {
16
+
let mut paths = poem_openapi::registry::Paths::new();
17
+
paths.add_path("/api/v1/games", games::Games::endpoint(), poem::http::Method::POST);
18
+
paths.add_path(
19
+
"/api/v1/games/:game_id",
20
+
games::GetGame::endpoint(),
21
+
poem::http::Method::GET,
22
+
);
23
+
paths.add_path(
24
+
"/api/v1/saves/:game_id/:slot_id",
25
+
saves::UploadSave::endpoint(),
26
+
poem::http::Method::POST,
27
+
);
28
+
paths.add_path(
29
+
"/api/v1/saves/:game_id/:slot_id",
30
+
saves::DownloadSave::endpoint(),
31
+
poem::http::Method::GET,
32
+
);
33
+
paths
34
+
}
35
+
}
+36
api/src/api/games.rs
+36
api/src/api/games.rs
···
1
+
use poem::Request;
2
+
use serde::{Deserialize, Serialize};
3
+
4
+
#[derive(Debug, Serialize, Deserialize)]
5
+
pub struct CreateGameRequest {
6
+
pub name: String,
7
+
}
8
+
9
+
#[derive(Debug, Serialize, Deserialize)]
10
+
pub struct GameResponse {
11
+
pub id: String,
12
+
pub name: String,
13
+
pub created_at: String,
14
+
}
15
+
16
+
pub async fn create_game() -> impl poem::Endpoint {
17
+
poem::endpoint::Handler::from(|_req: Request| async move {
18
+
poem::Response::builder()
19
+
.header("Content-Type", "application/json")
20
+
.json(serde_json::json!({
21
+
"error": "not_implemented",
22
+
"message": "Auth not implemented yet"
23
+
}))
24
+
})
25
+
}
26
+
27
+
pub async fn get_game() -> impl poem::Endpoint {
28
+
poem::endpoint::Handler::from(|_req: Request| async move {
29
+
poem::Response::builder()
30
+
.header("Content-Type", "application/json")
31
+
.json(serde_json::json!({
32
+
"error": "not_implemented",
33
+
"message": "Auth not implemented yet"
34
+
}))
35
+
})
36
+
}
+45
api/src/api/saves.rs
+45
api/src/api/saves.rs
···
1
+
use poem::Request;
2
+
3
+
pub async fn upload_save() -> impl poem::Endpoint {
4
+
poem::endpoint::Handler::from(|_req: Request| async move {
5
+
poem::Response::builder()
6
+
.header("Content-Type", "application/json")
7
+
.json(serde_json::json!({
8
+
"error": "not_implemented",
9
+
"message": "Auth not implemented yet"
10
+
}))
11
+
})
12
+
}
13
+
14
+
pub async fn download_save() -> impl poem::Endpoint {
15
+
poem::endpoint::Handler::from(|_req: Request| async move {
16
+
poem::Response::builder()
17
+
.header("Content-Type", "application/json")
18
+
.json(serde_json::json!({
19
+
"error": "not_implemented",
20
+
"message": "Auth not implemented yet"
21
+
}))
22
+
})
23
+
}
24
+
25
+
pub async fn list_versions() -> impl poem::Endpoint {
26
+
poem::endpoint::Handler::from(|_req: Request| async move {
27
+
poem::Response::builder()
28
+
.header("Content-Type", "application/json")
29
+
.json(serde_json::json!({
30
+
"error": "not_implemented",
31
+
"message": "Auth not implemented yet"
32
+
}))
33
+
})
34
+
}
35
+
36
+
pub async fn rollback() -> impl poem::Endpoint {
37
+
poem::endpoint::Handler::from(|_req: Request| async move {
38
+
poem::Response::builder()
39
+
.header("Content-Type", "application/json")
40
+
.json(serde_json::json!({
41
+
"error": "not_implemented",
42
+
"message": "Auth not implemented yet"
43
+
}))
44
+
})
45
+
}
+34
api/src/auth/mod.rs
+34
api/src/auth/mod.rs
···
1
+
use poem::Request;
2
+
3
+
pub async fn itchio_callback() -> impl poem::Endpoint {
4
+
poem::endpoint::Handler::from(|_req: Request| async move {
5
+
poem::Response::builder()
6
+
.header("Content-Type", "application/json")
7
+
.json(serde_json::json!({
8
+
"error": "not_implemented",
9
+
"message": "Auth not implemented yet"
10
+
}))
11
+
})
12
+
}
13
+
14
+
pub async fn device_code() -> impl poem::Endpoint {
15
+
poem::endpoint::Handler::from(|_req: Request| async move {
16
+
poem::Response::builder()
17
+
.header("Content-Type", "application/json")
18
+
.json(serde_json::json!({
19
+
"error": "not_implemented",
20
+
"message": "Auth not implemented yet"
21
+
}))
22
+
})
23
+
}
24
+
25
+
pub async fn device_verify() -> impl poem::Endpoint {
26
+
poem::endpoint::Handler::from(|_req: Request| async move {
27
+
poem::Response::builder()
28
+
.header("Content-Type", "application/json")
29
+
.json(serde_json::json!({
30
+
"error": "not_implemented",
31
+
"message": "Auth not implemented yet"
32
+
}))
33
+
})
34
+
}
+120
api/src/config.rs
+120
api/src/config.rs
···
1
+
use anyhow::Result;
2
+
use config::{Config, File};
3
+
use serde::Deserialize;
4
+
5
+
#[derive(Debug, Deserialize, Clone)]
6
+
pub struct AppConfig {
7
+
pub server: ServerConfig,
8
+
pub database: DatabaseConfig,
9
+
pub storage: StorageConfig,
10
+
pub auth: AuthConfig,
11
+
pub log_level: String,
12
+
}
13
+
14
+
#[derive(Debug, Deserialize, Clone)]
15
+
pub struct ServerConfig {
16
+
pub host: String,
17
+
pub port: u16,
18
+
}
19
+
20
+
#[derive(Debug, Deserialize, Clone)]
21
+
pub struct DatabaseConfig {
22
+
pub path: String,
23
+
}
24
+
25
+
#[derive(Debug, Deserialize, Clone)]
26
+
pub struct StorageConfig {
27
+
pub provider: String,
28
+
pub do_spaces_endpoint: Option<String>,
29
+
pub do_spaces_bucket: Option<String>,
30
+
pub do_spaces_access_key: Option<String>,
31
+
pub do_spaces_secret_key: Option<String>,
32
+
}
33
+
34
+
#[derive(Debug, Deserialize, Clone)]
35
+
pub struct AuthConfig {
36
+
pub jwt_secret: String,
37
+
pub itchio_client_id: String,
38
+
pub itchio_oauth_url: String,
39
+
}
40
+
41
+
impl Config {
42
+
pub fn load() -> Result<AppConfig> {
43
+
let config = Config::builder()
44
+
.add_source(File::with_name("config").required(false))
45
+
.add_source(File::with_name("/etc/scratchback/config").required(false))
46
+
.add_source(config::Environment::default())
47
+
.build()?;
48
+
49
+
let mut app_config = AppConfig {
50
+
server: ServerConfig {
51
+
host: "0.0.0.0".to_string(),
52
+
port: 3000,
53
+
},
54
+
database: DatabaseConfig {
55
+
path: "/var/data/scratchback/scratchback.db".to_string(),
56
+
},
57
+
storage: StorageConfig {
58
+
provider: "local".to_string(),
59
+
do_spaces_endpoint: None,
60
+
do_spaces_bucket: None,
61
+
do_spaces_access_key: None,
62
+
do_spaces_secret_key: None,
63
+
},
64
+
auth: AuthConfig {
65
+
jwt_secret: "dev-secret-change-in-production".to_string(),
66
+
itchio_client_id: "".to_string(),
67
+
itchio_oauth_url: "https://itch.io/user/oauth".to_string(),
68
+
},
69
+
log_level: "info".to_string(),
70
+
};
71
+
72
+
if let Ok(server_host) = std::env::var("SCRATCHBACK_HOST") {
73
+
app_config.server.host = server_host;
74
+
}
75
+
if let Ok(server_port) = std::env::var("SCRATCHBACK_PORT") {
76
+
if let Ok(port) = server_port.parse() {
77
+
app_config.server.port = port;
78
+
}
79
+
}
80
+
81
+
Ok(app_config)
82
+
}
83
+
}
84
+
85
+
impl Default for AppConfig {
86
+
fn default() -> Self {
87
+
Self {
88
+
server: ServerConfig {
89
+
host: "0.0.0.0".to_string(),
90
+
port: 3000,
91
+
},
92
+
database: DatabaseConfig {
93
+
path: "/var/data/scratchback/scratchback.db".to_string(),
94
+
},
95
+
storage: StorageConfig {
96
+
provider: "local".to_string(),
97
+
do_spaces_endpoint: None,
98
+
do_spaces_bucket: None,
99
+
do_spaces_access_key: None,
100
+
do_spaces_secret_key: None,
101
+
},
102
+
auth: AuthConfig {
103
+
jwt_secret: "dev-secret-change-in-production".to_string(),
104
+
itchio_client_id: "".to_string(),
105
+
itchio_oauth_url: "https://itch.io/user/oauth".to_string(),
106
+
},
107
+
log_level: "info".to_string(),
108
+
}
109
+
}
110
+
}
111
+
112
+
impl std::fmt::Display for AppConfig {
113
+
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
114
+
write!(
115
+
f,
116
+
"{}:{} (db: {}, storage: {})",
117
+
self.server.host, self.server.port, self.database.path, self.storage.provider
118
+
)
119
+
}
120
+
}
+98
api/src/db/mod.rs
+98
api/src/db/mod.rs
···
1
+
use anyhow::Result;
2
+
use rusqlite::Connection;
3
+
use std::path::Path;
4
+
5
+
pub async fn init_database(db_path: &Path) -> Result<()> {
6
+
if let Some(parent) = db_path.parent() {
7
+
std::fs::create_dir_all(parent)?;
8
+
}
9
+
10
+
let conn = Connection::open(db_path)?;
11
+
12
+
conn.execute_batch(
13
+
r#"
14
+
CREATE TABLE IF NOT EXISTS developers (
15
+
id TEXT PRIMARY KEY,
16
+
email TEXT NOT NULL UNIQUE,
17
+
password_hash TEXT NOT NULL,
18
+
name TEXT NOT NULL,
19
+
created_at TEXT DEFAULT CURRENT_TIMESTAMP,
20
+
updated_at TEXT DEFAULT CURRENT_TIMESTAMP
21
+
);
22
+
23
+
CREATE TABLE IF NOT EXISTS games (
24
+
id TEXT PRIMARY KEY,
25
+
developer_id TEXT NOT NULL REFERENCES developers(id),
26
+
name TEXT NOT NULL,
27
+
created_at TEXT DEFAULT CURRENT_TIMESTAMP,
28
+
updated_at TEXT DEFAULT CURRENT_TIMESTAMP
29
+
);
30
+
31
+
CREATE TABLE IF NOT EXISTS api_keys (
32
+
id TEXT PRIMARY KEY,
33
+
developer_id TEXT NOT NULL REFERENCES developers(id),
34
+
game_id TEXT REFERENCES games(id),
35
+
key_hash TEXT NOT NULL,
36
+
created_at TEXT DEFAULT CURRENT_TIMESTAMP,
37
+
revoked_at TEXT
38
+
);
39
+
40
+
CREATE TABLE IF NOT EXISTS gamers (
41
+
id TEXT PRIMARY KEY,
42
+
itch_user_id INTEGER NOT NULL UNIQUE,
43
+
created_at TEXT DEFAULT CURRENT_TIMESTAMP
44
+
);
45
+
46
+
CREATE TABLE IF NOT EXISTS sessions (
47
+
id TEXT PRIMARY KEY,
48
+
gamer_id TEXT NOT NULL REFERENCES gamers(id),
49
+
expires_at TEXT NOT NULL,
50
+
created_at TEXT DEFAULT CURRENT_TIMESTAMP
51
+
);
52
+
53
+
CREATE TABLE IF NOT EXISTS invites (
54
+
code TEXT PRIMARY KEY,
55
+
used_by TEXT REFERENCES gamers(id),
56
+
used_at TEXT,
57
+
created_by TEXT NOT NULL,
58
+
expires_at TEXT NOT NULL
59
+
);
60
+
61
+
CREATE TABLE IF NOT EXISTS saves (
62
+
id TEXT PRIMARY KEY,
63
+
gamer_id TEXT NOT NULL REFERENCES gamers(id),
64
+
game_id TEXT NOT NULL REFERENCES games(id),
65
+
slot_id TEXT NOT NULL,
66
+
current_cid TEXT,
67
+
created_at TEXT DEFAULT CURRENT_TIMESTAMP,
68
+
updated_at TEXT DEFAULT CURRENT_TIMESTAMP,
69
+
UNIQUE(gamer_id, game_id, slot_id)
70
+
);
71
+
72
+
CREATE TABLE IF NOT EXISTS save_versions (
73
+
id TEXT PRIMARY KEY,
74
+
save_id TEXT NOT NULL REFERENCES saves(id),
75
+
version_number INTEGER NOT NULL,
76
+
cid TEXT NOT NULL,
77
+
milestone INTEGER DEFAULT 0,
78
+
size_bytes INTEGER NOT NULL,
79
+
created_at TEXT DEFAULT CURRENT_TIMESTAMP
80
+
);
81
+
82
+
CREATE TABLE IF NOT EXISTS dpop_tokens (
83
+
token_id TEXT PRIMARY KEY,
84
+
gamer_id TEXT NOT NULL REFERENCES gamers(id),
85
+
nonce TEXT NOT NULL,
86
+
expires_at TEXT NOT NULL,
87
+
created_at TEXT DEFAULT CURRENT_TIMESTAMP
88
+
);
89
+
90
+
CREATE INDEX IF NOT EXISTS idx_games_developer ON games(developer_id);
91
+
CREATE INDEX IF NOT EXISTS idx_saves_gamer ON saves(gamer_id);
92
+
CREATE INDEX IF NOT EXISTS idx_saves_game ON saves(game_id);
93
+
CREATE INDEX IF NOT EXISTS idx_save_versions_save ON save_versions(save_id);
94
+
"#,
95
+
)?;
96
+
97
+
Ok(())
98
+
}
+9
api/src/lib.rs
+9
api/src/lib.rs
+58
api/src/main.rs
+58
api/src/main.rs
···
1
+
use anyhow::Result;
2
+
use poem::middleware::Cors;
3
+
use poem::{Endpoint, EndpointExt, Route};
4
+
use std::net::SocketAddr;
5
+
use tracing_subscriber::{layer::SubscriberExt, util::SubscriberInitExt};
6
+
7
+
mod api;
8
+
mod auth;
9
+
mod config;
10
+
mod db;
11
+
mod storage;
12
+
mod web;
13
+
14
+
#[tokio::main]
15
+
async fn main() -> Result<()> {
16
+
let config = config::Config::load()?;
17
+
18
+
tracing_subscriber::registry()
19
+
.with(tracing_subscriber::EnvFilter::new(
20
+
config.log_level.clone(),
21
+
))
22
+
.with(tracing_subscriber::fmt::layer())
23
+
.init();
24
+
25
+
let app = create_app().await?;
26
+
27
+
let addr: SocketAddr = config.server.parse()?;
28
+
tracing::info!("Starting server on {}", addr);
29
+
30
+
poem::Server::new(poem::listener::TcpSocket::bind(addr))
31
+
.run(app)
32
+
.await?;
33
+
34
+
Ok(())
35
+
}
36
+
37
+
async fn create_app() -> Result<impl Endpoint> {
38
+
let static_assets = web::StaticAssets::new()?;
39
+
let cors = Cors::new();
40
+
41
+
let route = Route::new()
42
+
.at("/", static_assets.clone())
43
+
.at("/index.html", static_assets.clone())
44
+
.at("/api/v1/games", api::games::create_game())
45
+
.at("/api/v1/games/:game_id", api::games::get_game())
46
+
.at("/api/v1/saves/:game_id/:slot_id", api::saves::upload_save())
47
+
.at("/api/v1/saves/:game_id/:slot_id", api::saves::download_save())
48
+
.at("/api/v1/saves/:game_id/:slot_id/versions", api::saves::list_versions())
49
+
.at("/api/v1/saves/:game_id/:slot_id/rollback", api::saves::rollback())
50
+
.at("/api/v1/auth/itchio/callback", auth::itchio_callback())
51
+
.at("/api/v1/auth/device/code", auth::device_code())
52
+
.at("/api/v1/auth/device/verify", auth::device_verify())
53
+
.at("/api/v1/docs", api::docs())
54
+
.get("/", static_assets)
55
+
.with(cors);
56
+
57
+
Ok(route)
58
+
}
+83
api/src/storage/digitalocean.rs
+83
api/src/storage/digitalocean.rs
···
1
+
use anyhow::Result;
2
+
use aws_sdk_s3::config::{Credentials, Region};
3
+
use aws_sdk_s3::Client;
4
+
5
+
pub struct SpacesClient {
6
+
client: Client,
7
+
bucket: String,
8
+
}
9
+
10
+
impl SpacesClient {
11
+
pub async fn new(
12
+
endpoint: &str,
13
+
bucket: &str,
14
+
access_key: &str,
15
+
secret_key: &str,
16
+
) -> Result<Self> {
17
+
let credentials = Credentials::new(access_key, secret_key, None, None, "static");
18
+
let region = Region::new("us-east-1");
19
+
20
+
let config = aws_sdk_s3::config::Builder::new()
21
+
.endpoint_url(endpoint)
22
+
.region(region)
23
+
.credentials_provider(credentials)
24
+
.force_path_style(true)
25
+
.build();
26
+
27
+
let client = Client::from_conf(config);
28
+
29
+
Ok(Self {
30
+
client,
31
+
bucket: bucket.to_string(),
32
+
})
33
+
}
34
+
35
+
pub async fn upload_object(&self, key: &str, data: &[u8]) -> Result<String> {
36
+
use aws_sdk_s3::primitives::ByteStream;
37
+
38
+
let body = ByteStream::from(data.to_vec());
39
+
40
+
self.client
41
+
.put_object()
42
+
.bucket(&self.bucket)
43
+
.key(key)
44
+
.body(body)
45
+
.send()
46
+
.await?;
47
+
48
+
Ok(key.to_string())
49
+
}
50
+
51
+
pub async fn download_object(&self, key: &str) -> Result<Vec<u8>> {
52
+
use aws_sdk_s3::primitives::ByteStream;
53
+
54
+
let resp = self
55
+
.client
56
+
.get_object()
57
+
.bucket(&self.bucket)
58
+
.key(key)
59
+
.send()
60
+
.await?;
61
+
62
+
let bytes = resp.body.collect().await?.to_vec();
63
+
Ok(bytes)
64
+
}
65
+
66
+
pub async fn get_presigned_url(&self, key: &str, expires_secs: i64) -> Result<String> {
67
+
use std::time::Duration;
68
+
69
+
let presigning_config = aws_sdk_s3::presigning::PresigningConfig::builder()
70
+
.expires(Duration::from_secs(expires_secs as u64))
71
+
.build()?;
72
+
73
+
let presigned = self
74
+
.client
75
+
.get_object()
76
+
.bucket(&self.bucket)
77
+
.key(key)
78
+
.presigned(presigning_config)
79
+
.await?;
80
+
81
+
Ok(presigned.uri().to_string())
82
+
}
83
+
}
+66
api/src/storage/mod.rs
+66
api/src/storage/mod.rs
···
1
+
pub mod digitalocean;
2
+
3
+
use anyhow::Result;
4
+
5
+
#[derive(Debug, Clone)]
6
+
pub enum StorageProvider {
7
+
Local,
8
+
DigitalOcean(digitalocean::SpacesClient),
9
+
}
10
+
11
+
impl StorageProvider {
12
+
pub async fn new(provider: &str, config: &crate::config::StorageConfig) -> Result<Self> {
13
+
match provider {
14
+
"digitalocean" | "do-spaces" => {
15
+
let client = digitalocean::SpacesClient::new(
16
+
config.do_spaces_endpoint.as_ref().unwrap(),
17
+
config.do_spaces_bucket.as_ref().unwrap(),
18
+
config.do_spaces_access_key.as_ref().unwrap(),
19
+
config.do_spaces_secret_key.as_ref().unwrap(),
20
+
)
21
+
.await?;
22
+
Ok(StorageProvider::DigitalOcean(client))
23
+
}
24
+
_ => Ok(StorageProvider::Local),
25
+
}
26
+
}
27
+
28
+
pub async fn upload_blob(&self, path: &str, data: &[u8]) -> Result<String> {
29
+
match self {
30
+
StorageProvider::Local => {
31
+
let full_path = format!("/var/data/scratchback/blobs/{}", path);
32
+
if let Some(parent) = Path::new(&full_path).parent() {
33
+
std::fs::create_dir_all(parent)?;
34
+
}
35
+
std::fs::write(&full_path, data)?;
36
+
Ok(path.to_string())
37
+
}
38
+
StorageProvider::DigitalOcean(client) => {
39
+
client.upload_object(path, data).await
40
+
}
41
+
}
42
+
}
43
+
44
+
pub async fn download_blob(&self, path: &str) -> Result<Vec<u8>> {
45
+
match self {
46
+
StorageProvider::Local => {
47
+
let full_path = format!("/var/data/scratchback/blobs/{}", path);
48
+
Ok(std::fs::read(&full_path)?)
49
+
}
50
+
StorageProvider::DigitalOcean(client) => {
51
+
client.download_object(path).await
52
+
}
53
+
}
54
+
}
55
+
56
+
pub async fn get_presigned_url(&self, path: &str) -> Result<String> {
57
+
match self {
58
+
StorageProvider::Local => Ok(format!("/blobs/{}", path)),
59
+
StorageProvider::DigitalOcean(client) => {
60
+
client.get_presigned_url(path, 3600).await
61
+
}
62
+
}
63
+
}
64
+
}
65
+
66
+
use std::path::Path;
+71
api/src/web/mod.rs
+71
api/src/web/mod.rs
···
1
+
use poem::endpoint::StaticFilesEndpoint;
2
+
use rust_embed::Embed;
3
+
use std::path::Path;
4
+
5
+
#[derive(Embed)]
6
+
#[folder = "../../landing/build"]
7
+
struct Assets;
8
+
9
+
pub struct StaticAssets;
10
+
11
+
impl StaticAssets {
12
+
pub fn new() -> anyhow::Result<Self> {
13
+
Ok(Self)
14
+
}
15
+
}
16
+
17
+
impl poem::Endpoint for StaticAssets {
18
+
type Output = poem::Response;
19
+
20
+
async fn call(&self, req: poem::Request) -> Self::Output {
21
+
let path = req.uri().path().trim_start_matches('/');
22
+
23
+
if path.is_empty() || path == "index.html" {
24
+
return Self::serve_file("index.html").await;
25
+
}
26
+
27
+
if let Some(file) = Assets::get(path) {
28
+
let mime = mime_guess::from_path(path)
29
+
.first_or_octet_stream()
30
+
.to_string();
31
+
32
+
return poem::Response::builder()
33
+
.header("Content-Type", mime)
34
+
.header("Cache-Control", "public, max-age=31536000, immutable")
35
+
.body(file.data.into_owned())
36
+
.unwrap()
37
+
.into();
38
+
}
39
+
40
+
Self::serve_file("index.html").await
41
+
}
42
+
}
43
+
44
+
impl StaticAssets {
45
+
async fn serve_file(path: &str) -> poem::Response {
46
+
if let Some(file) = Assets::get(path) {
47
+
let mime = mime_guess::from_path(path)
48
+
.first_or_octet_stream()
49
+
.to_string();
50
+
51
+
poem::Response::builder()
52
+
.header("Content-Type", mime)
53
+
.body(file.data.into_owned())
54
+
.unwrap()
55
+
.into()
56
+
} else if let Some(index) = Assets::get("index.html") {
57
+
poem::Response::builder()
58
+
.header("Content-Type", "text/html")
59
+
.status(poem::http::StatusCode::NOT_FOUND)
60
+
.body(index.data.into_owned())
61
+
.unwrap()
62
+
.into()
63
+
} else {
64
+
poem::Response::builder()
65
+
.status(poem::http::StatusCode::NOT_FOUND)
66
+
.body(Vec::new())
67
+
.unwrap()
68
+
.into()
69
+
}
70
+
}
71
+
}
+7
crates/achievements/Cargo.toml
+7
crates/achievements/Cargo.toml
+3
crates/achievements/src/lib.rs
+3
crates/achievements/src/lib.rs
+7
crates/cli/Cargo.toml
+7
crates/cli/Cargo.toml
+3
crates/cli/src/lib.rs
+3
crates/cli/src/lib.rs
+7
crates/events/Cargo.toml
+7
crates/events/Cargo.toml
+3
crates/events/src/lib.rs
+3
crates/events/src/lib.rs
+7
crates/leaderboards/Cargo.toml
+7
crates/leaderboards/Cargo.toml
+3
crates/leaderboards/src/lib.rs
+3
crates/leaderboards/src/lib.rs
+7
crates/sdk_be/Cargo.toml
+7
crates/sdk_be/Cargo.toml
+3
crates/sdk_be/src/lib.rs
+3
crates/sdk_be/src/lib.rs
+7
crates/sdk_godot/Cargo.toml
+7
crates/sdk_godot/Cargo.toml
+3
crates/sdk_godot/src/lib.rs
+3
crates/sdk_godot/src/lib.rs
+7
crates/sync_engine/Cargo.toml
+7
crates/sync_engine/Cargo.toml
+3
crates/sync_engine/src/lib.rs
+3
crates/sync_engine/src/lib.rs
+138
docs/initial-plan/00-introduction.md
+138
docs/initial-plan/00-introduction.md
···
1
+
# ScratchBack PDS - Introduction
2
+
3
+
## Overview
4
+
5
+
ScratchBack PDS is a zero-knowledge Personal Data Store (PDS) implementing
6
+
ATProtocol standards using rsky. This service provides cloud storage for
7
+
itch.io game saves with GDPR compliance, conditional mobile sync, Age
8
+
encryption, and support for both Bunny Storage and DigitalOcean Spaces.
9
+
10
+
## Problem Statement
11
+
12
+
itch.io game developers need a secure, GDPR-compliant cloud storage solution for player saves with:
13
+
14
+
- Zero-knowledge architecture (server cannot decrypt user data)
15
+
- GDPR compliance with EU data localization
16
+
- Cost-effective storage with multiple backend options
17
+
- Mobile-first with conditional sync
18
+
- Admin controls for system management
19
+
20
+
## Solution
21
+
22
+
A PDS built on:
23
+
24
+
- **ATProtocol**: Standards-based protocol for decentralized data storage
25
+
- **rsky**: Rust implementation of ATProtocol PDS with SQLite backend
26
+
- **Axum**: Modern web framework for HTTP/XRPC endpoints
27
+
- **Age Encryption**: Zero-knowledge client-side encryption
28
+
- **Dual Storage Backends**: Bunny Storage + DigitalOcean Spaces
29
+
- **HTMX**: Server-side rendered web UI for admin controls
30
+
31
+
## Key Features
32
+
33
+
- 2GB default quota per user (soft limits at 80%, cumulative tracking)
34
+
- GDPR-compliant automatic region routing (EU users → EU storage)
35
+
- Real-time cost calculation (cached for 1 hour)
36
+
- Passkey authentication (terminal-only setup with QR codes)
37
+
- Session-based web UI (7-day expiration)
38
+
- Country consent flow with GDPR alerts
39
+
- Admin CLI (`scrtchbk-ctl`) for system management
40
+
- Soft deletion with 14-day retention
41
+
- Deployment metadata generated fresh at every startup
42
+
- Age encryption for passkeys database (synced to Bunny Storage)
43
+
44
+
## Technology Stack
45
+
46
+
- Rust 2021 edition
47
+
- Axum 0.7 (web framework)
48
+
- Diesel 2.1 (ORM)
49
+
- SQLite (database via rsky)
50
+
- minijinja 2.0 (templates)
51
+
- age 0.11 (encryption)
52
+
- cached 0.1 (memoization)
53
+
- webauthn-rs 0.5 (passkey)
54
+
- aws-sdk-s3 1.20 (Bunny/DigitalOcean)
55
+
- maxminddb 0.24 (GeoIP)
56
+
57
+
## Architecture Decisions
58
+
59
+
1. **Separate databases**:
60
+
- Main database: user_quotas, blobs, sessions, ntfy_subscriptions
61
+
- Passkeys database: passkeys table (age-encrypted, synced to Bunny Storage)
62
+
63
+
2. **GDPR routing**:
64
+
- EU users: Stored in EU data centers (Bunny frankfurt / DO fra1)
65
+
- Non-EU users: Stored in US data centers (Bunny us / DO nyc3)
66
+
- Country detection: GeoLite2 database (configurable path via env var)
67
+
68
+
3. **Authentication methods**:
69
+
- XRPC: JWT tokens (24-hour expiration)
70
+
- Admin CLI: Passkey (terminal-only, QR code registration)
71
+
- Web UI: Session cookies (7-day expiration, HTTP-only, Secure, SameSite=Lax)
72
+
73
+
4. **Cost management**:
74
+
- Real-time API calls to Bunny/DigitalOcean
75
+
- 1-hour cache using `cached` proc macro
76
+
- Background refresh task (tokio)
77
+
- Individual + combined cost calculations cached
78
+
79
+
5. **Deployment**:
80
+
- Static binaries via build.sr.ht
81
+
- Deployment metadata generated fresh at startup (not from CI/CD)
82
+
- VPS deployment to Hetzner
83
+
84
+
## Project Structure
85
+
86
+
```
87
+
/home/vrgl/Code/scratch-itch/
88
+
├── mise.toml # Pinned direnv + age
89
+
├── Cargo.toml # Workspace root
90
+
├── .env.age # Encrypted secrets
91
+
├── config.toml # Default configuration
92
+
├── deployment-info.json # Deployment metadata (generated)
93
+
├── build.sr.ht # Build configuration
94
+
├── Cargo.lock
95
+
├── src/
96
+
│ ├── main.rs
97
+
│ ├── config/
98
+
│ ├── storage/
99
+
│ ├── quota/
100
+
│ ├── auth/
101
+
│ ├── xrpc/
102
+
│ ├── web/
103
+
│ ├── deployment/
104
+
│ ├── cost_cache/
105
+
│ ├── models/
106
+
│ └── error.rs
107
+
├── migrations/ # Diesel migrations (main DB)
108
+
├── passkeys-migrations/ # Diesel migrations (passkeys DB)
109
+
├── static/
110
+
│ ├── css/
111
+
│ └── js/
112
+
└── cli/ # scrtchbk-ctl binary
113
+
├── Cargo.toml
114
+
└── build.sr.ht
115
+
```
116
+
117
+
## Documentation
118
+
119
+
- [Architecture](docs/initial-plan/01-architecture.md)
120
+
- [Requirements](docs/initial-plan/02-requirements.md)
121
+
- [Data Model](docs/initial-plan/03-data-model.md)
122
+
- [Implementation Phases](docs/initial-plan/04-implementation-phases.md)
123
+
- [ADR - Platform Introduction](docs/initial-plan/ADR.md)
124
+
125
+
## Implementation Timeline
126
+
127
+
- Phase 1: Foundation
128
+
- Phase 2: Configuration System
129
+
- Phase 3: Storage Layer
130
+
- Phase 4: Passkey Storage
131
+
- Phase 5: Quota System
132
+
- Phase 6: Cost Cache
133
+
- Phase 7: Authentication
134
+
- Phase 8: Deployment Tracking
135
+
- Phase 9: XRPC Endpoints
136
+
- Phase 10: Web UI
137
+
- Phase 11: Admin CLI
138
+
- Phase 12: Testing & Deployment
+174
docs/initial-plan/01-architecture.md
+174
docs/initial-plan/01-architecture.md
···
1
+
# Architecture
2
+
3
+
## System Components
4
+
5
+
```
6
+
┌─────────────────────────────────────────────────────────────────┐
7
+
│ ScratchBack PDS │
8
+
├───────────────────────────────────────────────────────────────────┤
9
+
│ │
10
+
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
11
+
│ │ Axum Web │ │ XRPC API │ │ Admin CLI │ │
12
+
│ │ Server │ │ (JWT) │ │ (CLI) │ │
13
+
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
14
+
│ │ │ │ │ │
15
+
│ ┌──────▼──────────────┬───▼───────────────────▼───────────────┐ │
16
+
│ │ Application State │ ┌──────────────┐ │ │
17
+
│ │ │──────▶│ SQLite │ │ │
18
+
│ │ │ │ (Main DB) │ │ │
19
+
│ │ │ └──────────────┘ │ │
20
+
│ │ │ │ │
21
+
│ │ │ ┌──────────────┐ │ │
22
+
│ │ │──────▶│ SQLite │ │ │
23
+
│ │ │ │(Passkeys) │ │ │
24
+
│ │ │ └──┬──────────┘ │ │
25
+
│ │ │ │ │ │
26
+
│ │ │ │ LiteFS (manual sync) │ │
27
+
│ │ │ ▼ │ │
28
+
│ │ │ ┌──────────────┐ │ │
29
+
│ │ │ │ Encrypted │ │ │
30
+
│ │ │ │ SQLite │ │ │
31
+
│ │ │ └──────────────┘ │ │
32
+
│ └────────────────────┴───────────────────────────────────────────┘ │
33
+
│ │
34
+
│ ┌──────────────┐ ┌──────────────┐ │
35
+
│ │ Bunny Storage │ │ DigitalOcean │ │
36
+
│ │ (user blobs) │ │ Spaces │ │
37
+
│ └──────┬───────┘ └──────┬───────┘ │
38
+
│ │ │ │
39
+
│ │ ┌───────────▼──────────┐ │
40
+
│ │ │ Cost Cache │ │
41
+
│ │ │ (1-hour, │ │
42
+
│ │ │ cached proc, │ │
43
+
│ │ │ background) │ │
44
+
│ │ └─────────────────────┘ │
45
+
└─────────────────────────────────────────────────────────────────────┘
46
+
```
47
+
48
+
## Database Architecture
49
+
50
+
### Main Database (scratchback-pds/data.sqlite)
51
+
- **user_quotas**: User storage quotas, country codes, backend selection
52
+
- **blobs**: Blob references, soft deletion with retention timestamps
53
+
- **sessions**: Web UI session cookies (7-day expiration)
54
+
- **ntfy_subscriptions**: Internal ntfy.sh subscriptions (per-user)
55
+
56
+
### Passkeys Database (passkeys/passkeys.db)
57
+
- **passkeys**: Admin passkey credentials
58
+
- **Synced to**: Bunny Storage bucket (srtchbk-passkeys)
59
+
- **Encrypted with**: Age (at rest)
60
+
- **Encryption key**: Loaded from `.env.age` environment variable
61
+
62
+
## Storage Architecture
63
+
64
+
### Bunny Storage
65
+
- **User blobs bucket**: `srtchbk-us` or `srtchbk-eu`
66
+
- **Passkeys bucket**: `srtchbk-passkeys` (encrypted at rest)
67
+
- **Pricing**: $0.01/GB storage, $0.005/GB bandwidth
68
+
- **Features**: CDN, 7-day backup, 99.95% uptime
69
+
70
+
### DigitalOcean Spaces
71
+
- **User blobs bucket**: `srtchbk-us` (nyc3), `srtchbk-eu` (fra1)
72
+
- **Pricing**: $0.005/GB storage, $0.008/GB bandwidth
73
+
- **Features**: S3-compatible, 7-day backup
74
+
75
+
## Backend Selection (GDPR-Compliant)
76
+
77
+
### EU Countries (requires EU storage)
78
+
AT, BE, BG, HR, CY, CZ, DK, EE, FI, FR, DE, GR, HU, IE, IT, LV, LT, LU, MT, NL, PL, PT, RO, SK, SI, ES, SE, IS, LI, NO, CH
79
+
80
+
### Region Mapping
81
+
| Region | Backend | Bunny Region | DigitalOcean Region |
82
+
|---------|--------------|--------------|---------------------|
83
+
| EU | Bunny | frankfurt | fra1 |
84
+
| US | Bunny | us | nyc3 |
85
+
| Default | Config | Config | Config |
86
+
87
+
## Authentication Flow
88
+
89
+
### XRPC API (JWT)
90
+
- Used by game plugins and mobile apps
91
+
- Token in `Authorization: Bearer <jwt>` header
92
+
- 24-hour expiration
93
+
94
+
### Passkey Admin (Terminal-Only)
95
+
- Single interactive command: `scrtchbk-ctl passkey-register <username>`
96
+
- Displays QR code in terminal
97
+
- Waits for HTTP POST `/auth/passkey/response` from phone
98
+
- Stores credential in passkeys database
99
+
- Encrypts and syncs to Bunny Storage bucket
100
+
- Encryption key loaded from `.env.age` environment variable
101
+
102
+
### Web UI Sessions
103
+
- Created after country consent (sets country in profile)
104
+
- 7-day expiration
105
+
- HTTP-only, Secure, SameSite=Lax
106
+
- Protected `/self` page requires valid session
107
+
108
+
## Cost Calculation
109
+
110
+
### Caching Strategy
111
+
- **Individual backend costs**: Cached using `cached` proc macro
112
+
- **Combined total cost**: Cached using `cached` proc macro
113
+
- **Cache duration**: 1 hour
114
+
- **Refresh**: Background tokio task clears cache every hour
115
+
- **Source**: Real-time from Bunny/DigitalOcean APIs
116
+
117
+
## Deployment
118
+
119
+
### build.sr.ht
120
+
- Builds static binaries
121
+
- Runs on Hetzner VPS
122
+
- Separate build for `scrtchbk-ctl` CLI
123
+
124
+
### Startup Sequence
125
+
1. Decrypt `.env.age` (load secrets)
126
+
2. Load `config.toml` (expand environment variables)
127
+
3. Generate fresh `deployment-info.json`
128
+
4. Initialize SQLite databases
129
+
5. Download and decrypt passkeys database from Bunny Storage
130
+
6. Start background cost cache refresh task
131
+
7. Start Axum HTTP server
132
+
133
+
## Data Flow
134
+
135
+
### Upload Flow
136
+
```
137
+
Game Plugin → XRPC (JWT) → Quota Check → Country Detection (GDPR)
138
+
↓
139
+
Backend Selection (EU/US)
140
+
↓
141
+
Storage Upload → Blob Record → Success
142
+
```
143
+
144
+
### Passkey Registration Flow
145
+
```
146
+
Terminal Command → Generate Challenge → Display QR Code → Phone Scan
147
+
↓
148
+
HTTP POST /auth/passkey/response → Validate → Store in Passkeys DB
149
+
↓
150
+
Age Encrypt → Sync to Bunny Storage → Success
151
+
```
152
+
153
+
### Session Creation Flow
154
+
```
155
+
Country Consent → Set Country → Create Session Cookie → Redirect to /self → Validate → Success
156
+
```
157
+
158
+
## Security Model
159
+
160
+
- Zero-knowledge: Server cannot decrypt user data (Age encryption client-side)
161
+
- Passkeys encrypted at rest: Age-encrypted SQLite synced to Bunny Storage
162
+
- HTTPS-only for production
163
+
- Secure cookies: HTTP-only, Secure, SameSite=Lax
164
+
- JWT expiration: 24 hours for XRPC tokens
165
+
- Session expiration: 7 days for web UI
166
+
- Database isolation: Separate passkeys database for admin credentials
167
+
168
+
## Reliability
169
+
170
+
- Bunny Storage 99.95% uptime SLA
171
+
- DigitalOcean Spaces 99.9% uptime SLA
172
+
- Automatic storage backups (Bunny built-in, 7-day free retention)
173
+
- Soft deletion with 14-day retention (grace period for recovery)
174
+
- Cost cache with 1-hour refresh (fresh pricing data)
+173
docs/initial-plan/02-requirements.md
+173
docs/initial-plan/02-requirements.md
···
1
+
# Requirements
2
+
3
+
## Functional Requirements
4
+
5
+
### 1. Storage Management
6
+
- 2GB default quota per user (soft limits at 80%, cumulative tracking)
7
+
- Hard limit enforcement when quota exceeded
8
+
- Cumulative tracking (never resets)
9
+
- Blob count limits (default: 1000)
10
+
11
+
### 2. GDPR Compliance
12
+
- Automatic EU country detection via GeoLite2 database
13
+
- EU users must store data in EU data centers
14
+
- Non-EU users can use default backend
15
+
- Country detection failure must crash application
16
+
- GeoLite2 database path configurable via `GEOIP_DATABASE_PATH` environment variable
17
+
- Block uploads until country is set in profile
18
+
- Provide country consent flow with GDPR alerts
19
+
20
+
### 3. Multi-Backend Storage
21
+
- Bunny Storage support (default backend)
22
+
- DigitalOcean Spaces support (alternative backend)
23
+
- Real-time cost calculation from backend APIs
24
+
- 1-hour cost cache with automatic background refresh
25
+
- Backend selection based on user location (GDPR-compliant)
26
+
- Automatic storage backups (Bunny built-in: 7-day retention)
27
+
28
+
### 4. Authentication
29
+
- JWT for XRPC endpoints (24-hour expiration)
30
+
- Passkey for admin CLI (terminal-only, no web UI)
31
+
- Terminal registration via single interactive command with QR code
32
+
- HTTP endpoint for credential response (from phone)
33
+
- Age encryption for passkeys database (synced to Bunny Storage)
34
+
- Session cookies for web UI (7-day expiration)
35
+
- Session cookies created after country consent
36
+
- HTTP-only, Secure, SameSite=Lax settings for cookies
37
+
38
+
### 5. Passkey Management
39
+
- Terminal-only registration (no web UI)
40
+
- Single interactive command: `scrtchbk-ctl passkey-register <username>`
41
+
- Display QR code in terminal (ASCII art)
42
+
- Wait for HTTP POST `/auth/passkey/response` from phone
43
+
- Store credentials in passkeys database
44
+
- Encrypt passkeys database with age
45
+
- Sync to Bunny Storage bucket after every registration
46
+
- Encryption key loaded from `.env.age` environment variable
47
+
- 5-minute timeout for credential response
48
+
49
+
### 6. Cost Management
50
+
- Real-time storage cost per GB (Bunny: $0.01, DO: $0.005)
51
+
- Real-time bandwidth cost per GB (Bunny: $0.005, DO: $0.008)
52
+
- Combined total cost calculation
53
+
- 1-hour cache duration
54
+
- Automatic background refresh task (tokio)
55
+
- Cached using `cached` proc macro for memoization
56
+
- Cache cleared every hour to force recalculation
57
+
- Individual backend costs cached separately
58
+
- Combined total cost cached separately
59
+
60
+
### 7. Admin Controls
61
+
- CLI tool named `scrtchbk-ctl`
62
+
- Direct database access (no authentication required)
63
+
- System statistics command
64
+
- User management commands (list, view, set quota)
65
+
- Blob management commands (list, soft delete)
66
+
- Cleanup command for expired soft-deleted blobs (14-day retention)
67
+
- Passkey commands (register, sync)
68
+
- Database query commands
69
+
- Static binary (separate build.sr.ht)
70
+
71
+
### 8. Web UI
72
+
- Homepage with deployment information and real-time cost estimates
73
+
- Protected self-view page (session cookie auth required)
74
+
- Country consent flow page
75
+
- HTMX for dynamic interactions
76
+
- minijinja templates with Object contexts
77
+
- 7-day session cookie expiration
78
+
- HTTP-only, Secure, SameSite=Lax cookie settings
79
+
80
+
### 9. Configuration
81
+
- Priority: Environment variables > config.toml > defaults
82
+
- Age decryption on startup (`.env.age`)
83
+
- GeoLite2 database path via `GEOIP_DATABASE_PATH` environment variable
84
+
- Session cookie configuration (7-day expiration)
85
+
- Separate build.sr.ht for admin CLI
86
+
87
+
### 10. Deployment
88
+
- Deployment metadata generated fresh at every startup
89
+
- Static binary builds via build.sr.ht
90
+
- No CI/CD (manual deployment to Hetzner VPS)
91
+
- Deployment info stored in `deployment-info.json`
92
+
93
+
## Non-Functional Requirements
94
+
95
+
### 1. Technology Stack
96
+
- Rust 2021 edition
97
+
- Axum 0.7 (web framework)
98
+
- Diesel 2.1 (ORM)
99
+
- SQLite (database via rsky)
100
+
- minijinja 2.0 (templates)
101
+
- age 0.11 (encryption)
102
+
- cached 0.1 (memoization)
103
+
- fs-err-tokio 0.2 (error handling)
104
+
- webauthn-rs 0.5 (passkey)
105
+
- aws-sdk-s3 1.20 (Bunny/DigitalOcean)
106
+
- maxminddb 0.24 (GeoIP)
107
+
- qrcode 0.14 (QR codes)
108
+
- cookie 0.18 (session cookies)
109
+
110
+
### 2. Performance
111
+
- Cost calculations cached for 1 hour
112
+
- Background refresh task for cache
113
+
- Async/await throughout
114
+
- Connection pooling for databases
115
+
- Real-time API calls to storage backends
116
+
117
+
### 3. Security
118
+
- Zero-knowledge (server cannot decrypt user data)
119
+
- Age encryption at rest (passkeys in Bunny Storage)
120
+
- HTTPS-only for production
121
+
- Secure cookies (HTTP-only, Secure flag, SameSite=Lax)
122
+
- JWT tokens (24-hour expiration)
123
+
- Session tokens (7-day expiration)
124
+
- Separate passkeys database for admin credentials
125
+
126
+
### 4. Reliability
127
+
- Bunny Storage 99.95% uptime SLA
128
+
- DigitalOcean Spaces 99.9% uptime SLA
129
+
- Soft deletion with 14-day retention (grace period for recovery)
130
+
- Automatic storage backups (Bunny built-in, 7-day retention)
131
+
- GeoIP detection with crash on failure (admin must configure)
132
+
133
+
### 5. Usability
134
+
- Terminal-only passkey registration with QR codes
135
+
- Clear error messages for configuration failures
136
+
- GDPR consent flow with storage restrictions explained
137
+
- Mobile-responsive HTMX interface
138
+
- Real-time cost estimates on homepage
139
+
140
+
### 6. Maintainability
141
+
- Separate admin CLI from web server
142
+
- Comprehensive documentation
143
+
- Diesel migrations for schema changes
144
+
- Build.sr.ht for reproducible deployments
145
+
- Error handling with miette for detailed diagnostics
146
+
147
+
### 7. Scalability
148
+
- Dual storage backends (Bunny + DigitalOcean)
149
+
- GDPR-compliant region routing
150
+
- Cost cache reduces API calls
151
+
- Connection pooling for databases
152
+
- Background tasks for non-blocking operations
153
+
154
+
## Constraints
155
+
156
+
- Budget: $80/month maximum
157
+
- Hetzner EU VPS: $5.50
158
+
- DigitalOcean US VPS: $12 (upgraded)
159
+
- Bunny Storage: $11 total (separate EU/US buckets)
160
+
- Sentry: $15 (team plan)
161
+
- uptime.io: $7
162
+
- Elastic Email: $5
163
+
- Turso: $10 (for metadata only)
164
+
165
+
- Storage backends must be S3-compatible
166
+
- Passkey registration must be terminal-only
167
+
- GeoIP database must be configured before starting
168
+
- Application must crash if GeoIP detection fails
169
+
- Session cookies must expire after 7 days
170
+
- Soft-deleted blobs must be retained for 14 days
171
+
- Cost cache must refresh every hour
172
+
- Passkeys must sync to Bunny Storage after registration
173
+
- Age encryption key must be in `.env.age`
+227
docs/initial-plan/03-data-model.md
+227
docs/initial-plan/03-data-model.md
···
1
+
# Data Model
2
+
3
+
## Database Schemas
4
+
5
+
### Main Database (scratchback-pds/data.sqlite)
6
+
7
+
#### user_quotas Table
8
+
```sql
9
+
CREATE TABLE user_quotas (
10
+
did TEXT PRIMARY KEY,
11
+
storage_used_bytes INTEGER NOT NULL DEFAULT 0,
12
+
storage_limit_bytes INTEGER NOT NULL DEFAULT 2147483648, -- 2GB
13
+
blob_count INTEGER NOT NULL DEFAULT 0,
14
+
blob_limit INTEGER NOT NULL DEFAULT 1000,
15
+
country_code TEXT,
16
+
storage_backend TEXT,
17
+
storage_region TEXT,
18
+
warning_sent INTEGER DEFAULT 0,
19
+
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
20
+
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
21
+
);
22
+
23
+
CREATE INDEX idx_user_quotas_country ON user_quotas(country_code);
24
+
CREATE INDEX idx_user_quotas_backend ON user_quotas(storage_backend);
25
+
```
26
+
27
+
**Fields:**
28
+
- `did`: User's ATProtocol DID (primary key)
29
+
- `storage_used_bytes`: Cumulative storage used (never resets)
30
+
- `storage_limit_bytes`: Maximum storage allowed (default: 2GB = 2147483648 bytes)
31
+
- `blob_count`: Number of blobs stored (cumulative)
32
+
- `blob_limit`: Maximum number of blobs (default: 1000)
33
+
- `country_code`: ISO 3166-1 alpha-2 country code (e.g., "US", "DE")
34
+
- `storage_backend`: Storage backend used ("bunny" or "digitalocean")
35
+
- `storage_region`: Region identifier (e.g., "us", "frankfurt", "nyc3")
36
+
- `warning_sent`: Flag indicating 80% threshold warning has been sent
37
+
- `created_at`: Timestamp when quota was first created
38
+
- `updated_at`: Timestamp of last update
39
+
40
+
#### blobs Table
41
+
```sql
42
+
CREATE TABLE blobs (
43
+
id INTEGER PRIMARY KEY AUTOINCREMENT,
44
+
did TEXT NOT NULL,
45
+
blob_id TEXT NOT NULL,
46
+
size_bytes INTEGER NOT NULL,
47
+
url TEXT NOT NULL,
48
+
storage_backend TEXT NOT NULL,
49
+
storage_region TEXT NOT NULL,
50
+
soft_deleted INTEGER DEFAULT 0,
51
+
deleted_at TIMESTAMP,
52
+
retention_until TIMESTAMP,
53
+
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
54
+
FOREIGN KEY (did) REFERENCES user_quotas(did) ON DELETE CASCADE,
55
+
UNIQUE(did, blob_id)
56
+
);
57
+
58
+
CREATE INDEX idx_blobs_did ON blobs(did);
59
+
CREATE INDEX idx_blobs_created ON blobs(created_at);
60
+
CREATE INDEX idx_blobs_soft_deleted ON blobs(soft_deleted);
61
+
CREATE INDEX idx_blobs_retention ON blobs(retention_until) WHERE retention_until IS NOT NULL;
62
+
```
63
+
64
+
**Fields:**
65
+
- `id`: Auto-increment primary key
66
+
- `did`: User's DID (foreign key to user_quotas)
67
+
- `blob_id`: Unique identifier for blob (UUID v4 format)
68
+
- `size_bytes`: Size of blob in bytes
69
+
- `url`: CDN URL for blob access
70
+
- `storage_backend`: Which backend stores this blob
71
+
- `storage_region`: Which region stores this blob
72
+
- `soft_deleted`: Flag indicating soft deletion (1 = deleted, 0 = active)
73
+
- `deleted_at`: Timestamp when soft-deleted
74
+
- `retention_until`: Timestamp for hard deletion (14 days after soft deletion)
75
+
- `created_at`: Timestamp when blob was created
76
+
77
+
#### sessions Table
78
+
```sql
79
+
CREATE TABLE sessions (
80
+
id INTEGER PRIMARY KEY AUTOINCREMENT,
81
+
did TEXT NOT NULL,
82
+
session_token TEXT UNIQUE NOT NULL,
83
+
expires_at TIMESTAMP NOT NULL,
84
+
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
85
+
FOREIGN KEY (did) REFERENCES user_quotas(did) ON DELETE CASCADE
86
+
);
87
+
88
+
CREATE INDEX idx_sessions_did ON sessions(did);
89
+
CREATE INDEX idx_sessions_expires ON sessions(expires_at);
90
+
```
91
+
92
+
**Fields:**
93
+
- `id`: Auto-increment primary key
94
+
- `did`: User's DID (foreign key to user_quotas)
95
+
- `session_token`: Unique token for cookie (UUID v4 format)
96
+
- `expires_at`: Session expiration timestamp (7 days after creation)
97
+
- `created_at`: Timestamp when session was created
98
+
99
+
#### ntfy_subscriptions Table
100
+
```sql
101
+
CREATE TABLE ntfy_subscriptions (
102
+
id INTEGER PRIMARY KEY AUTOINCREMENT,
103
+
did TEXT NOT NULL,
104
+
topic TEXT NOT NULL,
105
+
access_token TEXT,
106
+
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
107
+
UNIQUE(did)
108
+
);
109
+
110
+
CREATE INDEX idx_ntfy_subscriptions_did ON ntfy_subscriptions(did);
111
+
```
112
+
113
+
**Fields:**
114
+
- `id`: Auto-increment primary key
115
+
- `did`: User's DID (foreign key to user_quotas)
116
+
- `topic`: ntfy.sh topic for this user (e.g., "scratchback-did:abc123")
117
+
- `access_token`: Optional access token for private topics
118
+
- `created_at`: Timestamp when subscription was created
119
+
120
+
### Passkeys Database (passkeys/passkeys.db)
121
+
122
+
#### passkeys Table
123
+
```sql
124
+
CREATE TABLE passkeys (
125
+
id INTEGER PRIMARY KEY AUTOINCREMENT,
126
+
username TEXT NOT NULL UNIQUE,
127
+
credential_id TEXT NOT NULL UNIQUE,
128
+
credential_data BLOB NOT NULL,
129
+
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
130
+
);
131
+
132
+
CREATE INDEX idx_passkeys_username ON passkeys(username);
133
+
```
134
+
135
+
**Fields:**
136
+
- `id`: Auto-increment primary key
137
+
- `username`: Admin username (e.g., "admin@scratchback.co")
138
+
- `credential_id`: Base64-encoded credential ID from WebAuthn
139
+
- `credential_data`: Serialized PasskeyCredential (BLOB)
140
+
- `created_at`: Timestamp when passkey was registered
141
+
142
+
## Data Relationships
143
+
144
+
```
145
+
user_quotas (1) ----< (1..N) ----> blobs
146
+
| |
147
+
|----< sessions (0..1) |
148
+
| |
149
+
|----< ntfy_subscriptions (0..1)
150
+
```
151
+
152
+
## Storage Key Format
153
+
154
+
### Blob IDs
155
+
- Format: UUID v4 (e.g., "550e8400-e29b-41d4-a716-446655440e000")
156
+
- Purpose: Unique identifier for each blob
157
+
- Storage location: `blobs.blob_id`
158
+
159
+
### Session Tokens
160
+
- Format: UUID v4 (e.g., "550e8400-e29b-41d4-a716-446655440e000")
161
+
- Purpose: Secure session identifier for web UI
162
+
- Storage location: `sessions.session_token`
163
+
- Cookie name: "scratchback_session"
164
+
165
+
### Passkey Credential IDs
166
+
- Format: Base64-encoded credential ID from WebAuthn
167
+
- Purpose: Unique identifier for each registered passkey
168
+
- Storage location: `passkeys.credential_id`
169
+
170
+
## GDPR Data Isolation
171
+
172
+
### EU Data Storage
173
+
- **EU Users**: Stored in `srtchbk-eu` bucket (Bunny frankfurt or DO fra1)
174
+
- **Enforcement**: Automatic routing based on `user_quotas.country_code`
175
+
- **Blocking**: Uploads blocked until country is set in profile
176
+
- **Consent Flow**: User must consent to EU data center before uploading
177
+
178
+
### Non-EU Data Storage
179
+
- **Non-EU Users**: Stored in `srtchbk-us` bucket (Bunny us or DO nyc3)
180
+
- **Enforcement**: Automatic routing based on detection or profile
181
+
- **No Consent Required**: Users can upload immediately
182
+
183
+
### Region Mapping
184
+
| Region | Backend | Bunny Region | DigitalOcean Region |
185
+
|---------|----------|--------------|---------------------|
186
+
| EU | Bunny | frankfurt | fra1 |
187
+
| US | Bunny | us | nyc3 |
188
+
| Default | Config | Config | Config |
189
+
190
+
## Soft Deletion Retention
191
+
192
+
### Deletion Process
193
+
1. **Soft Delete**: Set `soft_deleted = 1`, `deleted_at = NOW()`
194
+
2. **Retention Calculation**: Set `retention_until = deleted_at + 14 days`
195
+
3. **Quota Update**: Do NOT update `storage_used_bytes` (data counts until hard deletion)
196
+
4. **Hard Deletion**: CLI cleanup command deletes blobs where `retention_until < NOW()`
197
+
5. **Data Recovery**: 14-day grace period for users to recover accidentally deleted blobs
198
+
199
+
### Cleanup Command
200
+
```bash
201
+
$ scrtchbk-ctl cleanup
202
+
# Deletes blobs where retention_until < NOW()
203
+
# Updates user_quotas.storage_used_bytes (subtracting deleted blob sizes)
204
+
```
205
+
206
+
### GDPR Right to Erasure
207
+
1. User requests deletion via support or UI
208
+
2. Execute soft delete (step 1 above)
209
+
3. Wait 14 days for automatic hard deletion
210
+
4. Or execute cleanup command immediately (user consent)
211
+
5. Verify all blob copies deleted from Bunny Storage
212
+
6. Update user quota to reflect removed storage
213
+
214
+
## Quota Tracking
215
+
216
+
### Cumulative Model
217
+
- **Storage Used**: Never resets, always accumulates
218
+
- **Blob Count**: Never resets, always accumulates
219
+
- **Limit Enforcement**: Hard limit at configured maximum
220
+
- **Soft Warning**: Alert at 80% threshold
221
+
- **No Reset**: No monthly or periodic reset of quotas
222
+
223
+
### Cost Calculation
224
+
- **Real-Time**: API calls to Bunny/DigitalOcean for current usage
225
+
- **Cached**: Results cached for 1 hour
226
+
- **Background Refresh**: Cache cleared and recalculated every hour
227
+
- **Memoization**: Using `cached` proc macro for function-level caching
+53
docs/initial-plan/04-implementation-phases.md
+53
docs/initial-plan/04-implementation-phases.md
···
1
+
# Implementation Phases
2
+
3
+
## Overview
4
+
5
+
Total: 65 days (13 weeks).
6
+
7
+
## Phase 1 (Days 1-5)
8
+
9
+
Done: Created workspace, initialized mise.toml, migrations, schema files, config.
10
+
11
+
## Phase 2 (Days 6-8)
12
+
13
+
Config system.
14
+
15
+
## Phase 3 (Days 9-14)
16
+
17
+
Storage layer (Bunny + DO).
18
+
19
+
## Phase 4 (Days 15-20)
20
+
21
+
Passkey storage (Age + S3 sync).
22
+
23
+
## Phase 5 (Days 21-26)
24
+
25
+
Quota manager (2GB default, 80% warning).
26
+
27
+
## Phase 6 (Days 27-29)
28
+
29
+
Cost cache (1 hour, cached proc, background refresh).
30
+
31
+
## Phase 7 (Days 30-36)
32
+
33
+
JWT, passkey registration, session cookies, ntfy integration.
34
+
35
+
## Phase 8 (Days 37-38)
36
+
37
+
Deployment tracking (fresh on startup).
38
+
39
+
## Phase 9 (Days 39-45)
40
+
41
+
XRPC handlers, soft delete (14 days).
42
+
43
+
## Phase 10 (Days 46-52)
44
+
45
+
HTMX, minijinja templates, self view.
46
+
47
+
## Phase 11 (Days 53-59)
48
+
49
+
Admin CLI (scrtchbk-ctl).
50
+
51
+
## Phase 12 (Days 60-65)
52
+
53
+
Testing and deployment to Hetzner VPS.
+152
docs/initial-plan/ADR.md
+152
docs/initial-plan/ADR.md
···
1
+
# ADR - Platform Introduction
2
+
3
+
## Context
4
+
5
+
ScratchBack PDS is being developed as a zero-knowledge Personal Data Store (PDS) implementing ATProtocol standards. This document records key architectural decisions and trade-offs made during the planning phase.
6
+
7
+
## Status
8
+
9
+
- **Date**: 2025-01-15
10
+
- **Status**: Initial Planning Complete
11
+
- **Next Phase**: Phase 1 - Foundation Implementation
12
+
13
+
## Decision Log
14
+
15
+
### ADR-001: ATProtocol vs Traditional Cloud Storage
16
+
17
+
**Decision**: Use ATProtocol (via rsky) instead of traditional S3-only architecture
18
+
19
+
**Rationale**:
20
+
- Standards-based protocol (ATProtocol) provides interoperability
21
+
- Built-in social graph and federated identity
22
+
- rsky provides complete PDS implementation with SQLite backend
23
+
- Supports decentralized social features
24
+
- Leverages existing ATProtocol ecosystem
25
+
26
+
**Status**: Accepted
27
+
28
+
### ADR-002: Separate Passkeys Database
29
+
30
+
**Decision**: Use separate SQLite database for admin passkeys, not in main database
31
+
32
+
**Rationale**:
33
+
- Security isolation: Admin credentials separated from user data
34
+
- Independent encryption: Passkeys can be encrypted at rest independently
35
+
- Separate lifecycle: Passkeys sync independently of main app
36
+
- Encryption at rest: Age + S3 sync
37
+
38
+
**Status**: Accepted
39
+
40
+
### ADR-003: Terminal-Only Passkey Registration
41
+
42
+
**Decision**: Implement passkey registration as terminal-only with QR codes, no web UI
43
+
44
+
**Rationale**:
45
+
- Admin CLI is terminal-based tool
46
+
- Reduces attack surface
47
+
- Fits CLI workflow
48
+
- QR codes work with phone authenticator apps
49
+
50
+
**Status**: Accepted
51
+
52
+
### ADR-004: Application Crash on GeoIP Failure
53
+
54
+
**Decision**: Crash application if GeoIP database detection fails
55
+
56
+
**Rationale**:
57
+
- Prevents silent failures
58
+
- Forces admin to configure GeoIP
59
+
- Ensures GDPR compliance by default
60
+
- Fail-fast principle
61
+
62
+
**Status**: Accepted
63
+
64
+
### ADR-005: fs-err-tokio Instead of std::fs
65
+
66
+
**Decision**: Use fs-err-tokio crate for all file system operations
67
+
68
+
**Rationale**:
69
+
- Consistent error handling
70
+
- Better error messages with context
71
+
- Easier diagnostics and debugging
72
+
- Supports async operations
73
+
74
+
**Status**: Accepted
75
+
76
+
### ADR-006: Manual S3 Sync
77
+
78
+
**Decision**: Manual S3 sync with Age encryption instead of LiteFS
79
+
80
+
**Rationale**:
81
+
- LiteFS crate doesn't exist
82
+
- Manual sync provides full control
83
+
- Age encryption at rest is sufficient
84
+
- No external dependencies
85
+
86
+
**Status**: Accepted
87
+
88
+
### ADR-007: 1-Hour Cost Cache with Background Refresh
89
+
90
+
**Decision**: Cache costs for 1 hour with automatic background refresh
91
+
92
+
**Rationale**:
93
+
- Balances freshness vs API usage
94
+
- Reduces API costs by 90%
95
+
- Background refresh ensures cache stays fresh
96
+
- Uses `cached` proc macro
97
+
98
+
**Status**: Accepted
99
+
100
+
### ADR-008: Separate build.sr.ht for Admin CLI
101
+
102
+
**Decision**: Create separate build.sr.ht for scrtchbk-ctl binary
103
+
104
+
**Rationale**:
105
+
- Static binary for server deployment
106
+
- Different optimization levels
107
+
- Reduces build time for main project
108
+
- Simplifies deployment
109
+
110
+
**Status**: Accepted
111
+
112
+
### ADR-009: Session Cookies Created After Country Consent
113
+
114
+
**Decision**: Create session cookies only after successful country consent
115
+
116
+
**Rationale**:
117
+
- Ensures GDPR compliance before allowing uploads
118
+
- Simplified flow
119
+
- Matches web UI workflow
120
+
121
+
**Status**: Accepted
122
+
123
+
### ADR-010: Bunny Storage Backup Strategy
124
+
125
+
**Decision**: Rely on Bunny's built-in backup (no additional implementation)
126
+
127
+
**Rationale**:
128
+
- Bunny provides 7-day free backup retention
129
+
- 99.95% uptime SLA
130
+
- Automatic replication
131
+
- Reduces complexity
132
+
133
+
**Status**: Accepted
134
+
135
+
### ADR-011: GeoLite2 Path via Environment Variable
136
+
137
+
**Decision**: Make GeoLite2 database path configurable via GEOIP_DATABASE_PATH
138
+
139
+
**Rationale**:
140
+
- Flexible deployment
141
+
- Testing support
142
+
- Path independence
143
+
- Priority: Environment variable > config.toml > hardcoded paths
144
+
145
+
**Status**: Accepted
146
+
147
+
## Related Documentation
148
+
149
+
- Architecture: docs/initial-plan/01-architecture.md
150
+
- Requirements: docs/initial-plan/02-requirements.md
151
+
- Data Model: docs/initial-plan/03-data-model.md
152
+
- Implementation Phases: docs/initial-plan/04-implementation-phases.md
+1
landing/build/index.html
+1
landing/build/index.html
···
1
+
<!DOCTYPE html><html><body>Scratchback</body></html>