this repo has no description
feat: enhance update command and auto-updater with detailed status messages and error handling
+222
-40
+115
-9
README.md
+115
-9
README.md
···
1
1
# Stagehand - Telegram Image Queue Bot
2
2
3
-
A Telegram bot that takes links from supported websites, extracts images, and queues them for posting to a Telegram channel at scheduled intervals.
3
+
A Telegram bot that takes links from supported websites, extracts images and videos, and queues them for posting to a Telegram channel at scheduled intervals.
4
4
5
5
## Features
6
6
7
-
- Extract images from various supported websites
7
+
- Extract images and videos from various supported websites
8
8
- Process forwarded messages containing links (including button links)
9
-
- Queue images for scheduled posting
9
+
- Queue media for scheduled posting
10
10
- Customizable posting schedule using cron syntax
11
11
- Access control to limit who can use the bot
12
-
- Post images with source attribution and link back to original
12
+
- Post media with source attribution and link back to original
13
13
- Modular design for easy addition of new website scrapers
14
14
- Interactive visual queue management with inline buttons
15
+
- Intelligent queue monitoring with automatic alerts
16
+
- Perceptual image hashing for duplicate detection
17
+
- Automatic media cache management with recaching support
18
+
- Scheduled announcements with individual cron schedules
19
+
- Auto-updater for seamless updates from Git repository
20
+
- Discord webhook integration (optional)
15
21
16
22
## Supported Websites
17
23
···
46
52
- Extracts direct download URLs, titles, and artist information
47
53
- Handles both image and video content
48
54
- Preserves proper attribution and metadata
55
+
56
+
### SoFurry
57
+
- Uses direct access to the SoFurry API with OAuth authentication
58
+
- Extracts submission IDs from URLs
59
+
- Requires manual setup:
60
+
1. Create an application on the [SoFurry Developer Portal](https://developer.sofurry.com)
61
+
2. Generate an OAuth access token manually
62
+
3. Add the access token to your `.env` file as `SOFURRY_ACCESS_TOKEN`
63
+
- Fetches submission data, including display URLs and metadata
64
+
- Supports both image and video content
65
+
- Handles proper attribution with author and title information
66
+
67
+
### Weasyl (Currently Broken)
68
+
- Implements Weasyl's API for submission data
69
+
- Requires an API key from [Weasyl](https://www.weasyl.com/)
70
+
- Add your API key to `.env` as `WEASYL_API_KEY`
71
+
- Extracts submission information and media URLs
49
72
50
73
## Media Caching and Transcoding
51
74
···
189
212
- `OWNER_ID`: (Optional) User ID of the bot owner for admin-level commands
190
213
191
214
### Integration Options
192
-
- `WEASYL_API_KEY`: (Optional) API key for Weasyl integration
215
+
- `WEASYL_API_KEY`: (Optional) API key for Weasyl integration - Get yours from [Weasyl](https://www.weasyl.com/)
216
+
- `SOFURRY_ACCESS_TOKEN`: (Optional) OAuth access token for SoFurry API - Create an app and generate a token at the [SoFurry Developer Portal](https://www.sofurry.com/oauth/applications)
193
217
- `DISCORD_WEBHOOK_URL`: (Optional) For Discord integration
194
218
- `DISCORD_ENABLED`: Set to 'true' to enable Discord posting
195
219
···
266
290
- `/shuffle` - Toggle shuffle mode (randomizes queue after each post, persists between restarts)
267
291
- `/clear` - Clear the entire queue
268
292
- `/cleancache` - Clean expired items from media cache
269
-
- `/announce` - Create a new announcement
270
-
- `/announcements` - Manage existing announcements
293
+
- `/recache` - Recache missing files and remove items that fail after 3 attempts
294
+
- `/announce` - Create a new announcement (with custom text and schedule)
295
+
- `/announcements` - Manage existing announcements (view, edit, delete)
296
+
- `/update` - Check for updates and apply them (owner only)
271
297
272
298
### Adding Images to Queue
273
299
···
337
363
338
364
This project includes comprehensive documentation in the `docs/` directory:
339
365
366
+
- **[Bot Architecture](docs/bot-architecture.md)** - Detailed architecture documentation for the modular bot system
367
+
- **[Bot Architecture Migration](docs/bot-architecture-migration.md)** - Migration guide from monolithic to modular architecture
340
368
- **[Queue Monitoring Implementation](docs/queue-monitoring-implementation.md)** - Complete implementation details for the queue monitoring and alert system
341
369
- **[Queue Monitoring Configuration](docs/queue-monitoring-configuration.md)** - Configuration guide for queue alerts and thresholds
370
+
- **[Image Hashing](docs/image-hashing.md)** - Documentation on perceptual image hashing for duplicate detection
371
+
372
+
## Advanced Features
373
+
374
+
### Perceptual Image Hashing
375
+
376
+
Stagehand includes a sophisticated image hashing system to detect duplicate and similar images:
377
+
378
+
- **Automatic Processing**: Images are automatically hashed when cached
379
+
- **SQLite Database**: Stores perceptual hashes with URLs and metadata
380
+
- **Similarity Detection**: Find visually similar images using Hamming distance
381
+
- **Duplicate Prevention**: Helps identify duplicate content from different sources
382
+
- **Cleanup Management**: Automatically removes database entries for deleted files
383
+
384
+
The image hashing system uses the `imghash` library to generate perceptual hashes (pHash) that can detect similar images even after resizing, compression, or minor modifications.
385
+
386
+
For detailed information, see [Image Hashing Documentation](docs/image-hashing.md).
387
+
388
+
### Announcement System
389
+
390
+
Create and manage scheduled text announcements that are posted to your channel:
391
+
392
+
- **Multiple Announcements**: Support for multiple independent announcements
393
+
- **Individual Schedules**: Each announcement can have its own cron schedule
394
+
- **Interactive Management**: Create, edit, and delete announcements through the bot
395
+
- **Persistent Storage**: Announcements are saved and survive bot restarts
396
+
- **Test Functionality**: Test announcements before scheduling them
397
+
398
+
Use `/announce` to create new announcements and `/announcements` to manage existing ones.
399
+
400
+
### Auto-Updater
401
+
402
+
Stagehand includes an automatic update system that keeps your bot up to date:
403
+
404
+
- **Periodic Checks**: Automatically checks for updates every 12 hours
405
+
- **Git Integration**: Pulls updates from your Git repository
406
+
- **PM2 Restart**: Automatically restarts the bot using PM2 after updating
407
+
- **Manual Updates**: Use `/update` command to check and apply updates immediately
408
+
- **Owner-Only**: Update command is restricted to the bot owner
409
+
- **Dev Mode Exclusion**: Auto-updater is disabled in development mode
410
+
411
+
The updater runs in the background and will notify you when updates are applied.
412
+
413
+
### Media Recaching
414
+
415
+
Automatically handle missing or corrupted cache files:
416
+
417
+
- **Automatic Detection**: Identifies missing cache files in the queue
418
+
- **Redownload Support**: Automatically redownloads missing media files
419
+
- **Failure Tracking**: Tracks failed attempts and removes items after 3 failures
420
+
- **Manual Trigger**: Use `/recache` command to manually trigger recaching
421
+
- **Scheduled Execution**: Can be scheduled to run automatically via cron
422
+
- **Progress Reporting**: Reports how many items were processed and removed
423
+
424
+
This ensures your queue remains healthy and all media files are available when needed.
425
+
426
+
### Discord Integration
427
+
428
+
Optional Discord webhook support for cross-platform posting:
429
+
430
+
- **Webhook Support**: Post media to Discord channels via webhooks
431
+
- **Parallel Posting**: Post to both Telegram and Discord simultaneously
432
+
- **Independent Configuration**: Enable/disable Discord without affecting Telegram
433
+
- **Media Compatibility**: Handles both images and videos
434
+
435
+
Configure using the `DISCORD_WEBHOOK_URL` and `DISCORD_ENABLED` environment variables.
436
+
437
+
## Documentation
438
+
439
+
This project includes comprehensive documentation in the `docs/` directory:
440
+
342
441
- **[Bot Architecture](docs/bot-architecture.md)** - Detailed architecture documentation for the modular bot system
343
442
- **[Bot Architecture Migration](docs/bot-architecture-migration.md)** - Migration guide from monolithic to modular architecture
443
+
- **[Queue Monitoring Implementation](docs/queue-monitoring-implementation.md)** - Complete implementation details for the queue monitoring and alert system
444
+
- **[Queue Monitoring Configuration](docs/queue-monitoring-configuration.md)** - Configuration guide for queue alerts and thresholds
445
+
- **[Image Hashing](docs/image-hashing.md)** - Documentation on perceptual image hashing for duplicate detection
344
446
345
447
## License
346
448
···
355
457
- [x] Weasyl Scraper
356
458
- [x] Interactive Graphical Queue Manager
357
459
- [x] Add shuffle mode for queue
358
-
- [ ] Add perceptual hashing
359
-
- [ ] Redo Queue Manager
460
+
- [x] Add perceptual hashing
360
461
- [x] Redo Bluesky Module
361
462
- [x] Redo Telegram Module
463
+
- [x] Queue monitoring and alerts
464
+
- [x] Announcement system
465
+
- [x] Auto-updater
466
+
- [x] Media recaching system
467
+
- [ ] Redo Queue Manager
362
468
- [ ] Redo Discord Module
+9
-4
bot/telegrambot/commands/update.js
+9
-4
bot/telegrambot/commands/update.js
···
32
32
return;
33
33
}
34
34
35
-
const statusMessage = await this.bot.sendMessage(chatId, 'Updates found! Downloading and applying updates...');
35
+
const statusMessage = await this.bot.sendMessage(chatId, 'Updates found! Starting update process...');
36
36
37
37
const updateResult = await updater.manualUpdate();
38
38
39
-
if (updateResult) {
39
+
if (updateResult.success) {
40
40
await this.bot.editMessageText('Update successful! Bot will restart to apply changes.', {
41
41
chat_id: chatId,
42
42
message_id: statusMessage.message_id
···
49
49
await execAsync('pm2 restart --update-env stagehand');
50
50
} catch (restartError) {
51
51
console.error('Error restarting bot:', restartError);
52
-
this.bot.sendMessage(chatId, `Error during restart: ${restartError.message}`);
52
+
this.bot.sendMessage(chatId, `❌ Failed to restart bot: ${restartError.message}\n\nPlease restart the bot manually.`);
53
53
}
54
54
}, 2000);
55
+
} else if (updateResult.error) {
56
+
await this.bot.editMessageText(`❌ Update failed: ${updateResult.error}`, {
57
+
chat_id: chatId,
58
+
message_id: statusMessage.message_id
59
+
});
55
60
} else {
56
61
this.bot.sendMessage(chatId, 'Update process completed, but no changes were applied.');
57
62
}
58
63
} catch (error) {
59
64
console.error('Error during manual update:', error);
60
-
this.bot.sendMessage(chatId, `Error during update: ${error.message}`);
65
+
this.bot.sendMessage(chatId, `❌ Update failed: ${error.message}`);
61
66
}
62
67
});
63
68
}
+98
-27
utils/updater.js
+98
-27
utils/updater.js
···
193
193
194
194
/**
195
195
* Manually trigger an update check
196
-
* @returns {Promise<boolean>} True if updates were applied
196
+
* @returns {Promise<{success: boolean, error?: string}>} Object with success status and error message if failed
197
197
*/
198
198
async manualUpdate() {
199
199
try {
200
-
// Fetch changes from remote
201
-
const fetchResult = await execAsync('git fetch origin main');
202
-
if (fetchResult.stdout) {
203
-
console.log(fetchResult.stdout);
200
+
// Step 1: Check for local changes
201
+
console.log('Checking for local changes...');
202
+
let hasLocalChanges = false;
203
+
try {
204
+
const statusCheck = await execAsync('git status --porcelain');
205
+
hasLocalChanges = statusCheck.stdout.trim().length > 0;
206
+
} catch (error) {
207
+
return {
208
+
success: false,
209
+
error: `Failed to check git status: ${error.message}`
210
+
};
211
+
}
212
+
213
+
// Step 2: Stash local changes if any exist
214
+
if (hasLocalChanges) {
215
+
console.log('Local changes detected, stashing...');
216
+
try {
217
+
const stashResult = await execAsync('git stash save "Auto-stash before update"');
218
+
console.log(stashResult.stdout);
219
+
} catch (error) {
220
+
return {
221
+
success: false,
222
+
error: `Failed to stash local changes: ${error.message}`
223
+
};
224
+
}
225
+
}
226
+
227
+
// Step 3: Fetch changes from remote
228
+
console.log('Fetching changes from remote...');
229
+
try {
230
+
const fetchResult = await execAsync('git fetch origin main');
231
+
if (fetchResult.stdout) {
232
+
console.log(fetchResult.stdout);
233
+
}
234
+
} catch (error) {
235
+
return {
236
+
success: false,
237
+
error: `Failed to fetch from remote: ${error.message}`
238
+
};
239
+
}
240
+
241
+
// Step 4: Check if there are updates to pull
242
+
let changeCount = 0;
243
+
try {
244
+
const statusResult = await execAsync('git rev-list HEAD..origin/main --count');
245
+
changeCount = parseInt(statusResult.stdout.trim(), 10);
246
+
} catch (error) {
247
+
return {
248
+
success: false,
249
+
error: `Failed to check for remote changes: ${error.message}`
250
+
};
204
251
}
205
252
206
-
const statusResult = await execAsync('git rev-list HEAD..origin/main --count');
207
-
const changeCount = parseInt(statusResult.stdout.trim(), 10);
253
+
if (changeCount === 0) {
254
+
console.log('No updates to pull');
255
+
return { success: false };
256
+
}
208
257
209
-
if (changeCount > 0) {
210
-
// Save current HEAD for comparison
258
+
// Step 5: Save current HEAD for comparison
259
+
let oldHeadHash = '';
260
+
try {
211
261
const oldHead = await execAsync('git rev-parse HEAD');
212
-
const oldHeadHash = oldHead.stdout.trim();
213
-
214
-
// Pull the changes
262
+
oldHeadHash = oldHead.stdout.trim();
263
+
} catch (error) {
264
+
return {
265
+
success: false,
266
+
error: `Failed to get current commit: ${error.message}`
267
+
};
268
+
}
269
+
270
+
// Step 6: Pull the changes
271
+
console.log('Pulling updates...');
272
+
try {
215
273
const pullResult = await execAsync('git pull origin main');
216
274
console.log(pullResult.stdout);
217
-
218
-
// Get the new HEAD
275
+
} catch (error) {
276
+
return {
277
+
success: false,
278
+
error: `Failed to pull changes: ${error.message}`
279
+
};
280
+
}
281
+
282
+
// Step 7: Get the new HEAD
283
+
let newHeadHash = '';
284
+
try {
219
285
const newHead = await execAsync('git rev-parse HEAD');
220
-
const newHeadHash = newHead.stdout.trim();
221
-
222
-
// Show change statistics
223
-
if (oldHeadHash !== newHeadHash) {
224
-
const changeStats = await this.getChangeStats(oldHeadHash, newHeadHash);
225
-
if (changeStats) {
226
-
console.log(changeStats);
227
-
}
286
+
newHeadHash = newHead.stdout.trim();
287
+
} catch (error) {
288
+
console.error('Warning: Could not verify new commit:', error);
289
+
}
290
+
291
+
// Step 8: Show change statistics
292
+
if (oldHeadHash && newHeadHash && oldHeadHash !== newHeadHash) {
293
+
const changeStats = await this.getChangeStats(oldHeadHash, newHeadHash);
294
+
if (changeStats) {
295
+
console.log(changeStats);
228
296
}
229
-
230
-
return true;
231
297
}
232
-
return false;
298
+
299
+
console.log('Update completed successfully');
300
+
return { success: true };
233
301
} catch (error) {
234
-
console.error('Error during manual update:', error);
235
-
return false;
302
+
console.error('Unexpected error during manual update:', error);
303
+
return {
304
+
success: false,
305
+
error: `Unexpected error: ${error.message}`
306
+
};
236
307
}
237
308
}
238
309