Unofficial Paperbnd/Popfeed plugin for KOReader
initial auto-sync implementation
+203
-98
+90
-80
CLAUDE.md
+90
-80
CLAUDE.md
···
1
-
# Paperbnd KOReader Plugin
1
+
# Paperbnd KOReader Plugin - Developer Documentation
2
+
3
+
> **Document Purpose**: This is technical documentation for developers and maintainers. For user-facing installation and usage instructions, see [README.md](README.md).
2
4
3
5
A KOReader plugin for syncing reading progress to an AT Protocol PDS using the Popfeed List Item lexicon.
4
6
···
67
69
68
70
## Technical Details
69
71
72
+
### External Resources
73
+
74
+
- **KOReader Plugin Development**: https://github.com/koreader/koreader/wiki/Developer-documentation
75
+
- **AT Protocol Specifications**: https://atproto.com/specs/atp
76
+
- **Popfeed Platform**: https://popfeed.social/
77
+
- **Paperbnd Website**: https://paperbnd.club/
78
+
70
79
### Lexicons Used
71
80
72
81
- `social.popfeed.feed.listItem` - Individual book entries with progress tracking
···
113
122
- Leverages existing Popfeed records as the source of truth
114
123
115
124
### Advanced Session Management
116
-
The plugin implements a sophisticated multi-level session management system:
125
+
126
+
The plugin implements a sophisticated multi-level session management system that is the core reliability feature.
127
+
128
+
**Architecture:**
129
+
130
+
The session management system has three levels of components:
131
+
132
+
1. **`call()` method in xrpc.lua**: Generic XRPC request handler that detects auth errors (401 or 400 "expired"), calls `renewSession()` on failure, rebuilds requests with new tokens, and retries automatically. This is the single point of entry for all XRPC operations.
133
+
134
+
2. **`renewSession()` method**: Multi-level fallback coordinator that first attempts `refreshSession()`, then falls back to `createSession()` with app password if refresh token expired. Returns boolean success status.
135
+
136
+
3. **`validateSession()` method**: Makes lightweight test request at plugin startup to verify token validity and proactively renew expired tokens before user operations.
137
+
138
+
**Integration with Main Plugin:**
139
+
140
+
- **Initialization**: Loads credentials from persistent storage, configures XRPC client with tokens and app password, and sets callback for automatic token persistence
141
+
- **Token Callback**: Receives new tokens from XRPC client, saves to plugin settings immediately, ensuring credentials never get out of sync
142
+
- **Startup Validation**: Runs `validateSession()` silently when plugin loads to ensure tokens are fresh before first use
117
143
118
144
**Automatic Renewal Flow:**
145
+
119
146
1. Authentication error detected (401 or 400 "expired")
120
147
2. Attempt to refresh using refresh token
121
148
3. If refresh token expired, create new session with stored app password
122
149
4. If renewal successful, retry the original request automatically
123
150
5. Save new tokens to persistent storage via callback
124
151
125
-
**Session Validation:**
126
-
- Session checked at plugin startup
127
-
- Proactively renews expired tokens before user operations
128
-
- Prevents mid-operation authentication failures
152
+
**Example Flow - User syncs progress with expired access token:**
153
+
154
+
1. `syncProgress()` calls `xrpc:putRecord()`
155
+
2. `putRecord()` calls `call()` with POST body
156
+
3. Server returns 401 Unauthorized
157
+
4. `call()` detects auth error, calls `renewSession()`
158
+
5. `renewSession()` tries `refreshSession()`
159
+
6. If refresh token valid: new tokens received, callback fired
160
+
7. If refresh token expired: `createSession()` with app password
161
+
8. `call()` rebuilds request with new access token
162
+
9. `call()` retries POST request
163
+
10. Request succeeds, progress synced
164
+
11. User sees success message, unaware of token renewal
129
165
130
166
**Benefits:**
167
+
131
168
- No manual "Refresh Session" button needed
132
169
- Tokens can expire for weeks without user intervention
133
170
- Original requests automatically retried after renewal
134
171
- Seamless user experience without interruptions
135
172
- Callback mechanism keeps plugin and XRPC client in sync
173
+
- Proactive validation prevents mid-operation failures
136
174
137
175
### Authentication Strategy
138
176
- Uses Slingshot service to resolve PDS from handle (no manual PDS entry)
···
165
203
166
204
## Code Quality Assessment
167
205
168
-
**Current State (January 2025):**
206
+
**Current State (December 2024):**
169
207
170
208
The codebase is well-structured and production-ready with the following characteristics:
171
209
···
179
217
- Clear function naming and logical organization
180
218
181
219
**Architecture:**
182
-
- **XRPC Client (299 lines)**: Self-contained networking layer with 11 public methods
183
-
- **Main Plugin (447 lines)**: UI integration with 17 public methods
184
-
- **Metadata (5 lines)**: KOReader plugin manifest
185
-
- **Total**: ~750 lines of well-documented Lua code
220
+
- **XRPC Client (353 lines)**: Self-contained networking layer with 15 public methods
221
+
- **Main Plugin (446 lines)**: UI integration with 17 public methods
222
+
- **Metadata (6 lines)**: KOReader plugin manifest
223
+
- **Total**: 805 lines of well-documented Lua code
186
224
187
225
**Known Limitations:**
188
-
- One TODO at `main.lua:415` for updating `listUri` when changing reading status
226
+
- One TODO in `main.lua` for updating `listUri` when changing reading status (currently commented out at lines 413-416). Implementation would require fetching the user's "currently_reading_books" list URI and updating the `listUri` field when moving a book to that list.
189
227
- Session validation at startup runs synchronously (could be async but startup is fast enough)
190
228
- No retry logic for network failures (only authentication failures)
191
229
- No offline queuing (by design, for simplicity)
···
198
236
## Future Enhancements (Not Implemented)
199
237
200
238
Potential features intentionally left out for simplicity:
201
-
- Automatic book detection via ISBN
202
-
- Batch syncing multiple books
203
-
- Conflict resolution for concurrent edits
204
-
- Offline queue for syncing when network unavailable
205
-
- Automatic list URI updates when changing reading status (see TODO in code)
206
-
- Retry logic for transient network failures
207
-
- Progress indicators for long-running operations
208
239
209
-
## Files
210
-
211
-
- `main.lua` - Main plugin with UI and KOReader integration
212
-
- `xrpc.lua` - XRPC client for AT Protocol communication
213
-
- `CLAUDE.md` - This documentation file
214
-
215
-
## Session Management Implementation Details
216
-
217
-
The advanced session management system is the core reliability feature of this plugin:
240
+
- **Automatic book detection via ISBN**: Query ISBN databases to automatically match books instead of manual linking
241
+
- **Batch syncing multiple books**: Sync progress for all linked books at once
242
+
- **Conflict resolution for concurrent edits**: Handle cases where book data is modified from multiple clients
243
+
- **Offline queue for syncing**: Queue progress updates when network unavailable and sync when connection restored
244
+
- **Automatic list URI updates**: When changing a book's reading status (e.g., to "currently_reading_books"), automatically fetch and update the `listUri` field to point to the correct list. Currently commented out in code - would require additional `listRecords` call to fetch the user's lists and find the matching URI.
245
+
- **Retry logic for transient network failures**: Automatically retry failed requests due to temporary network issues
246
+
- **Progress indicators for long-running operations**: Show loading spinners or progress bars for network requests
218
247
219
-
### XRPCClient Methods (xrpc.lua)
248
+
## Files
220
249
221
-
**`validateSession()`** (`xrpc.lua:229-242`)
222
-
- Makes lightweight test request to verify token validity
223
-
- Returns boolean success status
224
-
- Uses `skip_renewal` flag to prevent recursion
225
-
- Called at plugin startup to proactively renew tokens
250
+
- `main.lua` - Main plugin with UI and KOReader integration (446 lines)
251
+
- `xrpc.lua` - XRPC client for AT Protocol communication (353 lines)
252
+
- `_meta.lua` - KOReader plugin manifest (6 lines)
253
+
- `README.md` - User-facing documentation with installation and usage instructions
254
+
- `CLAUDE.md` - This technical documentation for developers and maintainers
226
255
227
-
**`refreshSession()`** (`xrpc.lua:245-277`)
228
-
- Swaps access token with refresh token temporarily
229
-
- Calls `com.atproto.server.refreshSession` endpoint
230
-
- Restores old token on failure
231
-
- Notifies plugin of new tokens via callback
232
-
- Returns new session or error
256
+
## Troubleshooting
233
257
234
-
**`renewSession()`** (`xrpc.lua:280-303`)
235
-
- Multi-level fallback coordinator
236
-
- Level 1: Attempts `refreshSession()`
237
-
- Level 2: Falls back to `createSession()` with app password
238
-
- Returns boolean success and error message
239
-
- Central method called by `call()` on auth errors
258
+
### Common Issues
240
259
241
-
**`call()`** (`xrpc.lua:52-176`)
242
-
- Generic XRPC request handler
243
-
- Detects auth errors (401 or 400 "expired")
244
-
- Calls `renewSession()` on auth failure
245
-
- Rebuilds request with new authorization header
246
-
- Retries original request after successful renewal
247
-
- Single point of entry for all XRPC operations
260
+
**"Failed to resolve PDS" error**
261
+
- Verify handle is correct (e.g., "user.bsky.social")
262
+
- Check internet connection
263
+
- Confirm Slingshot service (slingshot.microcosm.blue) is accessible
248
264
249
-
### Main Plugin Integration (main.lua)
265
+
**"Authentication failed" error**
266
+
- Ensure app password is correct (not your main account password)
267
+
- Create a new app password if needed from your account settings
268
+
- Verify handle format matches your AT Protocol identifier
250
269
251
-
**Initialization** (`main.lua:17-51`)
252
-
- Loads credentials from persistent storage
253
-
- Configures XRPC client with tokens and app password
254
-
- Sets callback for automatic token persistence
255
-
- Triggers startup session validation
270
+
**"Failed to fetch books: No books found"**
271
+
- Books must be added to Popfeed/Paperbnd first before linking
272
+
- Books must be in a "currently_reading_books" list
273
+
- Use the Popfeed website or app to add books to your account
256
274
257
-
**Token Callback** (`main.lua:64-68`)
258
-
- Receives new tokens from XRPC client
259
-
- Saves to plugin settings immediately
260
-
- Ensures credentials never get out of sync
275
+
**"Failed to sync progress" error**
276
+
- Check that the book is still linked (may have been unlinked)
277
+
- Verify internet connection
278
+
- Token renewal should happen automatically, but if persisting, try re-entering credentials
261
279
262
-
**Startup Validation** (`main.lua:70-80`)
263
-
- Runs `validateSession()` when plugin loads
264
-
- Silent operation (no user-visible errors)
265
-
- Ensures tokens are fresh before first use
280
+
**Book not appearing in link list**
281
+
- Ensure book has `creativeWorkType` set to "book"
282
+
- Verify book is in "currently_reading_books" list type
283
+
- Try unlinking and re-linking the book
266
284
267
-
### Flow Example
285
+
### Debug Tips
268
286
269
-
User syncs progress with expired access token:
270
-
1. `syncProgress()` calls `xrpc:putRecord()`
271
-
2. `putRecord()` calls `call()` with POST body
272
-
3. Server returns 401 Unauthorized
273
-
4. `call()` detects auth error, calls `renewSession()`
274
-
5. `renewSession()` tries `refreshSession()`
275
-
6. If refresh token valid: new tokens received, callback fired
276
-
7. If refresh token expired: `createSession()` with app password
277
-
8. `call()` rebuilds request with new access token
278
-
9. `call()` retries POST request
279
-
10. Request succeeds, progress synced
280
-
11. User sees success message, unaware of token renewal
287
+
- Check KOReader logs for detailed error messages
288
+
- Verify credentials are saved by checking settings file at `<KOReader settings dir>/paperbnd.lua`
289
+
- Test network connectivity with other KOReader plugins
290
+
- Ensure document has page count metadata (required for progress percentage)
281
291
282
292
## Development
283
293
+2
flake.nix
+2
flake.nix
+110
-17
main.lua
+110
-17
main.lua
···
26
26
self.refresh_token = self.settings:readSetting("refresh_token")
27
27
self.did = self.settings:readSetting("did")
28
28
self.document_mappings = self.settings:readSetting("document_mappings") or {}
29
+
self.auto_sync_enabled = self.settings:readSetting("auto_sync_enabled")
30
+
if self.auto_sync_enabled == nil then
31
+
self.auto_sync_enabled = false -- Default OFF for safety
32
+
end
33
+
34
+
-- Auto-sync state tracking
35
+
self.auto_sync_task = nil -- Scheduled sync task reference
36
+
self.last_synced_page = nil -- Prevent duplicate syncs
37
+
self.auto_sync_delay = 10.0 -- 10 second debounce delay
29
38
30
39
-- Initialize XRPC client
31
40
self.xrpc = XRPCClient:new()
···
58
67
self.settings:saveSetting("refresh_token", self.refresh_token)
59
68
self.settings:saveSetting("did", self.did)
60
69
self.settings:saveSetting("document_mappings", self.document_mappings)
70
+
self.settings:saveSetting("auto_sync_enabled", self.auto_sync_enabled)
61
71
self.settings:flush()
62
72
end
63
73
···
70
80
function Paperbnd:validateSessionAtStartup()
71
81
-- Validate session asynchronously to avoid blocking plugin startup
72
82
-- This will trigger automatic renewal if the session is expired
73
-
local valid, err = self.xrpc:validateSession()
74
-
75
-
if not valid then
76
-
-- Session validation failed, but renewSession will be called
77
-
-- automatically on the next authenticated request
78
-
-- No need to show error to user at startup
79
-
end
83
+
-- Session validation errors are intentionally ignored at startup
84
+
-- renewSession will be called automatically on the next authenticated request
85
+
self.xrpc:validateSession()
80
86
end
81
87
82
88
function Paperbnd:addToMainMenu(menu_items)
···
109
115
end,
110
116
},
111
117
{
118
+
text = _("Auto-sync on page turn"),
119
+
checked_func = function()
120
+
return self.auto_sync_enabled
121
+
end,
122
+
callback = function()
123
+
self.auto_sync_enabled = not self.auto_sync_enabled
124
+
self:saveSettings()
125
+
126
+
-- Cancel pending sync when disabling
127
+
if not self.auto_sync_enabled and self.auto_sync_task then
128
+
UIManager:unschedule(self.auto_sync_task)
129
+
self.auto_sync_task = nil
130
+
self.last_synced_page = nil
131
+
end
132
+
133
+
local status = self.auto_sync_enabled and _("enabled") or _("disabled")
134
+
UIManager:show(InfoMessage:new {
135
+
text = T(_("Auto-sync %1"), status),
136
+
timeout = 2,
137
+
})
138
+
end,
139
+
},
140
+
{
112
141
text = _("Unlink current book"),
113
142
enabled_func = function()
114
143
return self:isCurrentBookLinked()
···
209
238
})
210
239
211
240
-- Resolve PDS
212
-
local pds_url, err = self.xrpc:resolvePDS(handle)
213
-
if err then
241
+
local pds_url, pds_err = self.xrpc:resolvePDS(handle)
242
+
if pds_err then
214
243
UIManager:show(InfoMessage:new {
215
-
text = T(_("Failed to resolve PDS: %1"), err),
244
+
text = T(_("Failed to resolve PDS: %1"), pds_err),
216
245
})
217
246
return
218
247
end
···
221
250
self.xrpc:setPDS(pds_url)
222
251
223
252
-- Create session
224
-
local session, err = self.xrpc:createSession(handle, password)
225
-
if err then
253
+
local session, session_err = self.xrpc:createSession(handle, password)
254
+
if session_err then
226
255
UIManager:show(InfoMessage:new {
227
-
text = T(_("Authentication failed: %1"), err),
256
+
text = T(_("Authentication failed: %1"), session_err),
228
257
})
229
258
return
230
259
end
···
284
313
-- Filter for books only
285
314
local books = {}
286
315
for _, record in ipairs(response.records) do
287
-
if record.value and record.value.creativeWorkType == "book" and record.value.listType == "currently_reading_books" then
316
+
if record.value and record.value.creativeWorkType == "book"
317
+
and record.value.listType == "currently_reading_books" then
288
318
table.insert(books, {
289
319
title = record.value.title or "Unknown",
290
320
author = record.value.mainCredit or "Unknown",
···
326
356
height = Screen:getHeight() - Screen:scaleBySize(100),
327
357
fullscreen = true,
328
358
single_line = false,
329
-
onMenuSelect = function(menu, item)
359
+
onMenuSelect = function(_, item)
330
360
self:confirmLinkBook(item.book)
331
361
end,
332
362
}
···
379
409
local mapping = self.document_mappings[doc_path]
380
410
381
411
-- Get document statistics
382
-
local stats = self.ui.doc_settings:readSetting("stats") or {}
383
412
local pages = self.ui.document:getPageCount()
384
413
local current_page = self.ui.paging and self.ui.paging.current_page or 1
385
414
local percent = math.floor((current_page / pages) * 100)
···
416
445
-- end
417
446
418
447
-- Put updated record
419
-
local put_response, put_err = self.xrpc:putRecord(
448
+
local _, put_err = self.xrpc:putRecord(
420
449
self.did,
421
450
"social.popfeed.feed.listItem",
422
451
mapping.rkey,
···
436
465
})
437
466
end
438
467
468
+
function Paperbnd:scheduleDebouncedSync(pageno)
469
+
-- Only schedule if auto-sync enabled and authenticated/linked
470
+
if not self.auto_sync_enabled then
471
+
return
472
+
end
473
+
474
+
if not self:isAuthenticated() or not self:isCurrentBookLinked() then
475
+
return
476
+
end
477
+
478
+
-- Cancel any existing scheduled sync
479
+
if self.auto_sync_task then
480
+
UIManager:unschedule(self.auto_sync_task)
481
+
self.auto_sync_task = nil
482
+
end
483
+
484
+
-- Create closure-based task for this scheduling
485
+
self.auto_sync_task = function()
486
+
self:performAutoSync(pageno)
487
+
end
488
+
489
+
-- Schedule sync after debounce delay
490
+
UIManager:scheduleIn(self.auto_sync_delay, self.auto_sync_task)
491
+
end
492
+
493
+
function Paperbnd:performAutoSync(pageno)
494
+
-- Clear task reference
495
+
self.auto_sync_task = nil
496
+
497
+
-- Check if page actually changed since last sync
498
+
if self.last_synced_page == pageno then
499
+
return
500
+
end
501
+
502
+
-- Double-check auth and linking
503
+
if not self:isAuthenticated() or not self:isCurrentBookLinked() then
504
+
return
505
+
end
506
+
507
+
-- Perform sync (reuses existing logic)
508
+
self:syncProgress()
509
+
510
+
-- Track synced page
511
+
self.last_synced_page = pageno
512
+
end
513
+
514
+
function Paperbnd:onPageUpdate(pageno)
515
+
-- Fires on every page turn - debounce to avoid excessive syncing
516
+
self:scheduleDebouncedSync(pageno)
517
+
518
+
-- Return false to allow event propagation
519
+
return false
520
+
end
521
+
439
522
-- Hook into document close to sync progress
440
523
function Paperbnd:onCloseDocument()
524
+
-- Cancel any pending auto-sync
525
+
if self.auto_sync_task then
526
+
UIManager:unschedule(self.auto_sync_task)
527
+
self.auto_sync_task = nil
528
+
end
529
+
530
+
-- Clear page tracking for next document
531
+
self.last_synced_page = nil
532
+
533
+
-- Always sync on close (original behavior)
441
534
if self:isAuthenticated() and self:isCurrentBookLinked() then
442
535
self:syncProgress()
443
536
end
+1
-1
xrpc.lua
+1
-1
xrpc.lua
···
157
157
if code ~= 200 then
158
158
local error_msg
159
159
if response_body and response_body ~= "" then
160
-
local error_data, err = rapidjson.decode(response_body)
160
+
local error_data = rapidjson.decode(response_body)
161
161
if error_data and error_data.message then
162
162
error_msg = error_data.message
163
163
else