Merge pull request #3 from malpercio-dev/claude/research-phpbb-C0Fil

+1091

2 changed files

expand all

docs

research

phpbb-research.md

realtime-architecture.md

+468

docs/research/phpbb-research.md

··· 1 + # phpBB Research: Features, Complaints & Lessons for atBB 2 + 3 + *Research conducted 2026-02-07* 4 + 5 + --- 6 + 7 + ## Table of Contents 8 + 9 + 1. [phpBB Overview & History](#phpbb-overview--history) 10 + 2. [Complete Feature Inventory](#complete-feature-inventory) 11 + 3. [Common Complaints & Pain Points](#common-complaints--pain-points) 12 + 4. [Why Traditional Forums Declined](#why-traditional-forums-declined) 13 + 5. [Lessons from Modern Competitors](#lessons-from-modern-competitors) 14 + 6. [Gap Analysis: atBB vs phpBB](#gap-analysis-atbb-vs-phpbb) 15 + 7. [Recommendations for atBB](#recommendations-for-atbb) 16 + 17 + --- 18 + 19 + ## phpBB Overview & History 20 + 21 + phpBB (PHP Bulletin Board) is the most widely deployed open-source forum software in history, first released in 2000. It's written in PHP and available under the GPL. 22 + 23 + ### Version History 24 + 25 + | Version | Codename | Released | Key Changes | 26 + |---------|----------|----------|-------------| 27 + | 1.x | — | ~2000 | Original release | 28 + | 2.x | — | ~2002–2008 | Made phpBB mainstream | 29 + | 3.0 | Olympus | Dec 2007 | Complete rewrite, prosilver theme, security audit | 30 + | 3.1 | Ascraeus | Oct 2014 | Symfony adoption, extension system, AJAX | 31 + | 3.2 | Rhea | Jan 2017 | PHP 7 support, emoji, Font Awesome, new installer | 32 + | 3.3 | Proteus | Jan 2020 | PHP 7.4/8.x, extended emoji, MySQL 8 (current stable) | 33 + | 4.0 | Triton | Alpha Sep 2025 | Symfony 6.4, PHP 8.1+, @mentions, Cloudflare Turnstile | 34 + 35 + ### Architecture 36 + 37 + - **Backend:** PHP on Symfony (since 3.1), custom template engine 38 + - **Database:** Supports MySQL/MariaDB, PostgreSQL, SQLite, MSSQL, Oracle 39 + - **Frontend:** Server-rendered HTML with prosilver theme (HTML5/CSS3), minimal JS 40 + - **Extension system:** Self-contained extensions installed via Admin Control Panel 41 + - **Search:** Multiple backends — native full-text index, MySQL FULLTEXT, PostgreSQL FULLTEXT, Sphinx 42 + 43 + --- 44 + 45 + ## Complete Feature Inventory 46 + 47 + ### 1. Forum Structure & Organization 48 + 49 + | Feature | Details | atBB Status | 50 + |---------|---------|-------------| 51 + | **Hierarchical categories** | Unlimited nesting of categories containing forums | Partial — `forum.category` is flat (single level) | 52 + | **Unlimited subforums** | Forums can contain subforums to any depth | Missing — no subforum concept | 53 + | **Password-protected forums** | Individual forums can require a password | Missing | 54 + | **Forum-specific styles** | Different visual themes per forum | Missing | 55 + | **Forum links** | A "forum" entry that links to an external URL | Missing | 56 + | **Forum rules display** | Per-forum or global rules shown at top | Missing | 57 + | **Topic pruning** | Auto-delete old/inactive topics by configurable criteria | Missing | 58 + | **Active topics display** | Most active topics shown at top of forum | Missing | 59 + 60 + ### 2. Topic & Post Features 61 + 62 + | Feature | Details | atBB Status | 63 + |---------|---------|-------------| 64 + | **Topic types: Normal** | Standard topic | Implemented (post without reply ref) | 65 + | **Topic types: Sticky** | Pinned to top of first page | Partial — `pin` mod action exists but no display logic | 66 + | **Topic types: Announcement** | Pinned above stickies, shown on all pages | Missing | 67 + | **Topic types: Global** | Shown at top of every forum on the board | Missing | 68 + | **Time-limited sticky/announcement** | Auto-revert to normal after N days | Missing | 69 + | **Topic subscriptions** | Email/notification on new replies | Missing | 70 + | **Forum subscriptions** | Email/notification on new topics in forum | Missing | 71 + | **Topic bookmarks** | Users save topics for quick access | Missing | 72 + | **Post ordering** | Sort by date, alphabetical, reply count | Missing | 73 + | **"My posts" indicator** | Mark topics where user has posted | Missing | 74 + | **Printer-friendly view** | Clean print layout for topics | Missing | 75 + | **Email a topic** | Send topic link to friend | Missing | 76 + | **Unread tracking** | Track which topics/posts user has read, even across sessions | Missing | 77 + | **Topic split** | Moderator splits a topic into two separate topics | Missing | 78 + | **Topic merge** | Moderator merges two topics into one | Missing | 79 + | **Topic move** | Move topic to different forum (with shadow link) | Missing | 80 + | **Topic lock** | Prevent further replies | Partial — `lock` mod action exists | 81 + | **Post lock** | Lock individual posts from editing | Missing | 82 + | **Topic copy** | Duplicate a topic to another forum | Missing | 83 + 84 + ### 3. Post Composition 85 + 86 + | Feature | Details | atBB Status | 87 + |---------|---------|-------------| 88 + | **BBCode formatting** | Bold, italic, lists, links, images, etc. | Missing — plain text only (300 grapheme limit) | 89 + | **Rich text / Markdown** | Formatted text in posts | Missing (noted as future work) | 90 + | **Smilies / Emoticons** | Inline emoji/emoticon insertion | Missing | 91 + | **Post preview** | Preview post before submitting | Missing | 92 + | **File attachments** | Upload files attached to posts | Missing (noted as future work) | 93 + | **Inline attachments** | Place attachments within post body with BBCode | Missing | 94 + | **Polls** | Create polls with multiple options, voting, time limits | Missing | 95 + | **Topic icons** | Visual icon next to topic title | Missing | 96 + | **Drafts** | Save unfinished posts for later | Missing | 97 + | **Post editing** | Edit your own posts after submission | Architecturally feasible (AT Proto supports updates via `putRecord` — CID changes, firehose emits update event) | 98 + | **Post edit history** | Track edit reasons and history | Missing | 99 + | **Word censoring** | Auto-replace banned words | Missing | 100 + | **Custom BBCodes** | Admin-defined formatting codes | Missing | 101 + | **@Mentions** | Mention and notify other users (phpBB 4.0 feature) | Missing | 102 + | **Direct media playback** | Play video/audio in post body (phpBB 4.0) | Missing | 103 + 104 + ### 4. User Profiles & Identity 105 + 106 + | Feature | Details | atBB Status | 107 + |---------|---------|-------------| 108 + | **User registration** | Account creation flow | Planned — AT Proto OAuth | 109 + | **Social login** | Google, Facebook, etc. | N/A — AT Proto identity replaces this | 110 + | **Avatars** | Gallery, uploaded, remote URL | Missing — could pull from AT Proto profile | 111 + | **Signatures** | Text/images appended to every post | Missing | 112 + | **Custom profile fields** | Admin-defined extra fields (location, website, etc.) | Missing — AT Proto profiles have some | 113 + | **User ranks** | Title based on post count or special assignment | Missing | 114 + | **User badges** | Visual indicators (verified, special roles) | Missing | 115 + | **Post count display** | Show post count next to username | Missing | 116 + | **Join date display** | Show when user joined | Partial — `membership.joinedAt` exists | 117 + | **User online status** | Show if user is currently online | Missing | 118 + | **User profile page** | Dedicated page showing user info, recent posts | Missing (noted as future work) | 119 + | **Friends & Foes list** | Add users to friends/ignore list | Missing | 120 + | **User post history** | View all posts by a user | Missing (noted as future work) | 121 + 122 + ### 5. Private Messaging 123 + 124 + | Feature | Details | atBB Status | 125 + |---------|---------|-------------| 126 + | **Direct messages** | Private messages between users | Missing entirely | 127 + | **Group messaging** | PM multiple users or groups | Missing | 128 + | **BCC support** | Blind carbon copy on PMs | Missing | 129 + | **PM attachments** | Attach files to PMs | Missing | 130 + | **PM drafts** | Save PM drafts | Missing | 131 + | **PM export** | Export/archive PMs | Missing | 132 + | **PM reply tracking** | Chronological PM thread history | Missing | 133 + | **PM notifications** | Notify on new PMs | Missing | 134 + 135 + **Note:** Private messaging may not be in scope for atBB since AT Proto has its own direct messaging infrastructure (Bluesky DMs via `chat.bsky.*` lexicons). This could be an intentional omission — users already have a DM channel through their AT Proto identity. 136 + 137 + ### 6. Moderation Tools 138 + 139 + | Feature | Details | atBB Status | 140 + |---------|---------|-------------| 141 + | **Ban by username** | Ban specific user | Partial — `ban` mod action exists | 142 + | **Ban by IP** | Block IP address | Missing — not applicable (AT Proto is DID-based) | 143 + | **Ban by email** | Block email address | Missing — not applicable | 144 + | **Time-limited bans** | Temporary bans with auto-expiry | Partial — `expiresAt` field exists on modAction | 145 + | **Warning system** | Issue warnings before bans | Missing | 146 + | **Warning auto-escalation** | Auto-ban after N warnings | Missing | 147 + | **Moderation queue** | Pre-approval for new users' posts | Missing | 148 + | **Post approval workflow** | Require mod approval before publish | Missing | 149 + | **Post reporting** | Users flag posts for mod review | Missing | 150 + | **Report reasons** | Configurable list of report reasons | Missing | 151 + | **Moderator notes** | Private notes on users visible to mods | Missing | 152 + | **Topic split/merge/move** | Reorganize discussions | Missing | 153 + | **Shadow topics** | Leave redirect link when moving topics | Missing | 154 + | **Edit/delete any post** | Moderator override on post editing | Partial — `delete` mod action hides from index | 155 + | **Change post author** | Reassign post ownership | Missing — not possible with AT Proto | 156 + | **Moderator log** | Audit trail of all mod actions | Missing | 157 + | **Admin log** | Audit trail of admin actions | Missing | 158 + | **User action log** | Track user behavior (IP logs, etc.) | Missing | 159 + 160 + ### 7. Permission System 161 + 162 + | Feature | Details | atBB Status | 163 + |---------|---------|-------------| 164 + | **Granular ACL** | YES/NO/NEVER per-permission per-user/group | Simplified — role hierarchy only | 165 + | **Permission types** | f_* (forum), m_* (mod), u_* (user), a_* (admin) | Simplified — `forum.role` with permission tokens | 166 + | **Role-based permissions** | Pre-defined permission bundles | Partial — role definitions exist | 167 + | **Per-forum permissions** | Different permissions per forum | Missing | 168 + | **Per-user overrides** | Override group permissions for individuals | Missing | 169 + | **Group-based permissions** | Assign permissions to groups, users inherit | Simplified — single role per membership | 170 + | **"Newly registered" group** | Restricted permissions for new users | Missing | 171 + | **Global vs local permissions** | Board-wide vs per-forum scoping | Missing | 172 + 173 + **Note:** phpBB's permission system is arguably over-engineered and a frequent source of admin confusion. atBB's simpler model (role hierarchy: Owner → Admin → Moderator → Member → Guest) is actually a feature, not a bug. The key missing piece is **per-category permissions** — the ability to restrict certain categories to certain roles. 174 + 175 + ### 8. Search 176 + 177 + | Feature | Details | atBB Status | 178 + |---------|---------|-------------| 179 + | **Full-text search** | Search post content | Missing (noted as future work — Meilisearch) | 180 + | **Multiple search backends** | Native, MySQL FULLTEXT, PostgreSQL, Sphinx | Missing | 181 + | **Search by author** | Find posts by specific user | Missing | 182 + | **Search by forum** | Scope search to specific forum/category | Missing | 183 + | **Search by date range** | Filter results by post date | Missing | 184 + | **Unanswered topics** | Find topics with no replies | Missing | 185 + | **Active topics** | Find topics with recent activity | Missing | 186 + | **"New posts since last visit"** | Show unread content | Missing | 187 + 188 + ### 9. Notifications & Feeds 189 + 190 + | Feature | Details | atBB Status | 191 + |---------|---------|-------------| 192 + | **Email notifications** | Notify on replies, PMs, etc. | Missing (noted as future work) | 193 + | **Jabber/XMPP notifications** | IM-based alerts | Missing | 194 + | **ATOM feeds** | Board-wide, per-forum, per-topic feeds | Missing (noted as future work — RSS) | 195 + | **In-app notifications** | On-site notification system | Missing | 196 + | **Digest emails** | Periodic summary emails | Missing | 197 + 198 + ### 10. Administration 199 + 200 + | Feature | Details | atBB Status | 201 + |---------|---------|-------------| 202 + | **Admin Control Panel** | Comprehensive web-based admin UI | Missing — admin panel is Phase 4 | 203 + | **Board statistics** | User count, post count, activity graphs | Missing | 204 + | **Database backup/restore** | Built-in DB management | Missing | 205 + | **Extension management** | Install/enable/disable extensions from UI | Missing | 206 + | **Style/theme management** | Upload and switch themes | Missing | 207 + | **Language packs** | Multi-language support | Missing | 208 + | **User management** | Search, edit, ban, delete users | Missing | 209 + | **Group management** | Create/edit groups, assign members | Partial — roles exist | 210 + | **Mass email** | Send email to all users or groups | Missing | 211 + | **Forum pruning** | Auto-cleanup old content | Missing | 212 + | **Cron/scheduled tasks** | Background maintenance jobs | Missing | 213 + | **Board configuration** | Extensive settings (registration, posting, etc.) | Missing | 214 + 215 + ### 11. Anti-Spam & Security 216 + 217 + | Feature | Details | atBB Status | 218 + |---------|---------|-------------| 219 + | **CAPTCHA** | Multiple CAPTCHA options (visual, reCAPTCHA) | N/A — AT Proto OAuth handles registration | 220 + | **Cloudflare Turnstile** | Modern anti-bot (phpBB 4.0) | N/A | 221 + | **Two-factor authentication** | 2FA for admins/mods | N/A — delegated to AT Proto identity | 222 + | **IP logging** | Track IPs per post | N/A — AT Proto is DID-based | 223 + | **Flood control** | Rate limiting on posts | Missing but important | 224 + | **New user restrictions** | Limited permissions for new accounts | Missing — could use Discourse-style trust levels | 225 + | **Word censoring** | Auto-filter content | Missing | 226 + | **Post approval queue** | Pre-moderate new user content | Missing | 227 + 228 + --- 229 + 230 + ## Common Complaints & Pain Points 231 + 232 + These are the most frequent criticisms of phpBB that atBB should learn from and avoid: 233 + 234 + ### 1. Spam Bots (Critical) 235 + phpBB is notorious for spam. Within days of installing a phpBB forum, admins report hundreds of bot registrations. The default anti-spam measures are insufficient. **atBB advantage:** AT Proto OAuth means users must have a real AT Proto identity (a PDS account), which is a significant barrier to spam bots. This is one of atBB's biggest inherent strengths. 236 + 237 + ### 2. Outdated User Interface 238 + phpBB's default prosilver theme looks dated. The Admin Control Panel is considered user-unfriendly. Modern users expect mobile-first, clean design. **atBB lesson:** The HTMX + server-rendered approach is good, but invest in modern CSS and responsive design from the start. Don't let the UI become an afterthought. 239 + 240 + ### 3. Difficult Extension/Theme Customization 241 + phpBB's old MOD system required editing core files, making upgrades painful. Even the newer extension system is considered harder to work with than competitors. **atBB lesson:** If an extension/plugin system is ever built, design it as a first-class API from day one. The AT Proto lexicon system already provides some natural extensibility. 242 + 243 + ### 4. Overly Complex Permission System 244 + phpBB's YES/NO/NEVER ACL system with four permission types (forum, mod, user, admin), per-group and per-user overrides, global vs local scopes — is powerful but confusing. Admins frequently misconfigure permissions. **atBB advantage:** The simpler role hierarchy is actually better UX. Don't over-complicate it. Per-category permissions (Phase 2+) should be the ceiling of complexity. 245 + 246 + ### 5. Slow Development & Modernization 247 + phpBB is developed entirely by volunteers. Major versions take years. phpBB 4.0 has been in development since ~2016 and is still in alpha as of 2025. **atBB lesson:** Ship MVP fast, iterate. The AT Proto foundation means the core protocol handles hard problems (identity, data ownership) and atBB can focus on forum UX. 248 + 249 + ### 6. Poor Mobile Experience 250 + phpBB's mobile experience, while improved with prosilver's responsive updates, still feels like a desktop forum crammed into a phone. **atBB lesson:** HTMX enables progressive enhancement. Design mobile-first from the beginning. 251 + 252 + ### 7. Search is Weak 253 + phpBB's native search indexes words, not phrases. Full-text search is basic. Large forums struggle with search performance. **atBB lesson:** Plan for Meilisearch or similar from the start (already noted as future work). Make search a core feature, not an afterthought. 254 + 255 + ### 8. No Real-Time Features 256 + phpBB is entirely request-response. No live updates, no real-time notifications, no "someone is typing" indicators. **atBB lesson:** The AT Proto firehose subscription already provides a real-time data stream. Use this to power live topic updates via HTMX SSE or WebSocket integration. 257 + 258 + --- 259 + 260 + ## Why Traditional Forums Declined 261 + 262 + Understanding why phpBB-style forums lost market share is critical for atBB's positioning: 263 + 264 + ### The Migration Pattern 265 + 1. **~2005–2010:** Social media (Facebook, Twitter) drew casual discussion away from forums 266 + 2. **~2010–2015:** Reddit centralized niche communities under one roof with a single account 267 + 3. **~2015–present:** Discord replaced forums for real-time community interaction 268 + 4. **~2020–present:** Growing backlash against centralized platforms, renewed interest in self-hosted/decentralized alternatives 269 + 270 + ### Why People Left Forums 271 + - **Friction:** Creating new accounts per forum, managing credentials, email notifications 272 + - **Discovery:** No way to find new communities from within the forum 273 + - **Single login:** Reddit/Discord offer one account for all communities 274 + - **Real-time:** Discord offers instant chat; forums feel slow 275 + - **Mobile:** Traditional forums were desktop-first 276 + 277 + ### Why People Want Forums Back 278 + - **Searchability:** Discord conversations are ephemeral and unsearchable by outsiders 279 + - **Ownership:** Reddit's API changes (2023) and platform enshittification 280 + - **Structure:** Forums organize knowledge better than chat or feeds 281 + - **Permanence:** Forum threads remain useful for years; Discord messages disappear 282 + - **Decentralization:** Growing desire to own your community and data 283 + 284 + ### atBB's Unique Position 285 + atBB directly addresses the core reasons forums declined: 286 + - **Single identity:** AT Proto DID = one identity across all forums (like Reddit/Discord) 287 + - **Data ownership:** Posts live on your PDS, not the forum's database 288 + - **Discovery:** Federation enables "common forums" features (like Discord's mutual servers) 289 + - **Portability:** Leave a forum, keep your content 290 + - **Decentralization:** Self-hosted, no central authority 291 + 292 + This is a genuinely compelling value proposition that phpBB never had. 293 + 294 + --- 295 + 296 + ## Lessons from Modern Competitors 297 + 298 + ### Discourse: What They Got Right 299 + - **Trust levels:** Automated progressive permissions (0=New → 4=Leader) reduce mod burden and scale community self-governance. New users are sandboxed; proven users earn privileges automatically. 300 + - **Gamification:** Built-in badges, points, leaderboards drive engagement 301 + - **Real-time:** Live updates, notifications, typing indicators 302 + - **Infinite scroll:** Modern UX pattern vs. pagination 303 + - **Mobile-first:** Responsive by default 304 + - **Markdown:** Modern text formatting instead of BBCode 305 + - **One-click setup:** Docker-based deployment 306 + 307 + ### Flarum: What They Got Right 308 + - **Two-pane interface:** Browse discussions without leaving current thread 309 + - **Lightweight:** Fast, minimal resource usage 310 + - **Modern stack:** Laravel + Mithril.js 311 + - **Tags instead of categories:** More flexible organization 312 + - **Infinite scroll with memory:** Remembers scroll position 313 + 314 + ### XenForo: What They Got Right 315 + - **Social features:** User status updates, activity feed 316 + - **Trophy/achievement system:** Gamification built into core 317 + - **Resource manager:** File/download sharing per forum 318 + - **Built-in media gallery:** Image/video sharing 319 + - **Robust search:** Elasticsearch integration 320 + 321 + --- 322 + 323 + ## Gap Analysis: atBB vs phpBB 324 + 325 + ### Features atBB Has That phpBB Doesn't 326 + 327 + | Feature | Description | 328 + |---------|-------------| 329 + | **Decentralized identity** | AT Proto DID — portable, user-owned identity | 330 + | **Data ownership** | Posts live on user's PDS, not forum's database | 331 + | **Portability** | Content persists even if you leave a forum | 332 + | **Federation** | Multiple forums can share identity namespace | 333 + | **Cross-forum discovery** | "Common forums" feature possible via membership records | 334 + | **Spam resistance** | AT Proto identity is a natural spam barrier | 335 + | **Protocol-level reactions** | Reactions are first-class AT Proto records | 336 + | **Real-time data stream** | Firehose subscription for live updates | 337 + 338 + ### Critical Features atBB is Missing (Priority Order) 339 + 340 + #### P0 — Essential for MVP Parity 341 + These are features without which the forum would feel fundamentally broken: 342 + 343 + 1. **Rich text / Markdown in posts** — Plain text only is not viable. Users expect at minimum bold, italic, links, code blocks, and lists. Markdown is the modern standard. Note: The current `maxGraphemes: 300` limit in `post.yaml` is closer to a tweet than a forum post — this constraint may need revisiting before implementing rich text (phpBB posts typically support thousands of characters). 344 + 345 + 2. **Unread tracking** — phpBB tracks read/unread state per-user across sessions. Without this, users can't efficiently follow conversations. This is the #1 quality-of-life feature for forum users. 346 + 347 + 3. **Topic subscription / notifications** — Users must be notified when someone replies to their topic or a topic they're watching. Without this, the forum is "post and forget." 348 + 349 + 4. **User profile pages** — Display name, avatar (from AT Proto profile), post count, join date, recent posts. The building block for community identity. 350 + 351 + 5. **Search** — Meilisearch integration. Forums are fundamentally knowledge bases; without search, old content is lost. 352 + 353 + #### P1 — Important for Competitive Parity 354 + These make the forum feel complete and usable for serious communities: 355 + 356 + 6. **Topic types: Sticky/Announcement** — Admins need to pin important topics. The `pin` mod action exists but needs display logic and proper announcement/global support. 357 + 358 + 7. **Post reporting** — Users need to flag rule-breaking content for mod review. This is the feedback loop that makes moderation scale. 359 + 360 + 8. **Moderation queue / post approval** — New users' posts can require mod approval. Critical for preventing spam/abuse in growing communities. 361 + 362 + 9. **Warning system** — Graduated enforcement before banning. Warnings → temporary ban → permanent ban. 363 + 364 + 10. **Moderation log / audit trail** — Record of all mod actions with timestamps and reasons. Essential for accountability and mod team coordination. 365 + 366 + 11. **Polls** — Simple voting mechanism within topics. High engagement, low implementation cost. 367 + 368 + 12. **Drafts** — Save unfinished posts. Prevents lost work, improves UX. 369 + 370 + 13. **Topic split/merge/move** — Core moderator workflow for organizing discussions. Without these, moderators can only lock or delete. 371 + 372 + #### P2 — Nice to Have / Differentiators 373 + These would make atBB stand out: 374 + 375 + 14. **Trust levels (Discourse-style)** — Automated progressive permissions based on participation. Reduces mod burden, encourages good behavior. 376 + 377 + 15. **ATOM/RSS feeds** — Per-forum, per-topic feeds. Low cost, high value for power users and integration. 378 + 379 + 16. **Gamification** — Post count ranks, badges, achievements. Drives engagement. 380 + 381 + 17. **File attachments / images** — Upload and embed media in posts. Already noted as future work. 382 + 383 + 18. **@Mentions** — Notify specific users in posts. phpBB 4.0 is adding this — it's becoming table stakes. 384 + 385 + 19. **Real-time updates** — Use the firehose to push new replies to open topic pages via HTMX SSE. Would be a significant differentiator over traditional forums. 386 + 387 + 20. **Subforum / nested categories** — Currently categories are flat. Some communities need hierarchy. 388 + 389 + #### P3 — Future Considerations 390 + 391 + 21. **Private messaging** — May be handled by AT Proto's native DM system instead 392 + 22. **Nested/threaded replies** — Already noted as future work 393 + 23. **Custom themes/branding** — Already noted as future work 394 + 24. **Plugin/extension system** — Already noted as future work 395 + 25. **Per-category permissions** — Restrict categories to certain roles 396 + 397 + --- 398 + 399 + ## Recommendations for atBB 400 + 401 + ### 1. Lean Into Your Strengths 402 + atBB's decentralized identity and data ownership model solves the fundamental problems that killed traditional forums (friction, lock-in, centralization). Don't bury this — make it the primary selling point. The "bring your identity, own your posts" story is compelling. 403 + 404 + ### 2. Avoid phpBB's Complexity Trap 405 + phpBB's permission system is a cautionary tale. The YES/NO/NEVER ACL with four permission types is powerful but confusing. atBB's role hierarchy (Owner → Admin → Moderator → Member → Guest) is the right level of simplicity for MVP. Add per-category permissions later if needed, but resist the urge to build a full ACL system. 406 + 407 + ### 3. Learn from Discourse's Trust Levels 408 + Discourse's automated trust system (where users earn permissions through participation) is one of the best innovations in forum software. Consider implementing a simplified version: 409 + - **Level 0 (New):** Just joined, posts may require approval 410 + - **Level 1 (Basic):** Has read some topics, can post freely 411 + - **Level 2 (Member):** Active participant, can flag posts 412 + - **Level 3 (Regular):** Trusted member, can recategorize topics, edits are instant 413 + - This could integrate with the AT Proto reputation model — a user's reputation across the ATmosphere could bootstrap their trust level. 414 + 415 + ### 4. Markdown, Not BBCode 416 + BBCode is a relic. Markdown is the universal formatting language of the modern web (GitHub, Reddit, Discourse all use it). Implement Markdown with a live preview. Consider a common Markdown library that works both server-side (for rendering) and client-side (for preview). 417 + 418 + ### 5. Real-Time is Your Secret Weapon 419 + You already have the AT Proto firehose. Use it to push real-time updates to open pages via HTMX Server-Sent Events (SSE). When someone replies to a topic you're viewing, the reply should appear without a page refresh. This would be a genuine differentiator over every traditional forum including phpBB. 420 + 421 + ### 6. Don't Neglect Search 422 + Implement Meilisearch (already planned) early, not late. Forums are knowledge bases. The value of a forum compounds over time as content accumulates — but only if that content is findable. 423 + 424 + ### 7. Mobile-First Design 425 + Design the HTMX web UI mobile-first. The majority of web traffic is mobile. phpBB's biggest UX failure is feeling like a desktop application on phones. 426 + 427 + ### 8. Unread Tracking is Non-Negotiable 428 + Implement per-user read state tracking from the beginning. This is the single most important UX feature for forums. Without it, users have no way to efficiently follow conversations across multiple topics and categories. phpBB's read tracking system (storing last-read timestamps per topic per user) has been refined over 20+ years — study it. 429 + 430 + ### 9. Moderation Must Scale 431 + Build moderation features that can scale with the community: 432 + - Post reporting from day one 433 + - Mod action audit log from day one 434 + - Warning system before permanent bans 435 + - Post approval queue for new users 436 + - Consider how AT Proto's decentralized nature affects moderation — users can always post to their PDS even if banned, but the AppView simply won't index their content. This is actually a cleaner model than phpBB's. 437 + 438 + ### 10. The "Forum Revival" Opportunity 439 + There is genuine renewed interest in forum-style communities driven by: 440 + - Discord's poor searchability and content ephemerality 441 + - Reddit's hostile API changes and platform enshittification 442 + - Growing desire for data ownership and decentralization 443 + 444 + atBB is uniquely positioned at the intersection of "forums done right" and "decentralized social." The combination of structured long-form discussion (forums), user-owned data (AT Proto), and portable identity (DIDs) is a compelling story that neither phpBB nor Discourse can tell. 445 + 446 + --- 447 + 448 + ## Sources 449 + 450 + - [phpBB Official Site](https://www.phpbb.com/) 451 + - [phpBB Features](https://www.phpbb.com/about/features/) 452 + - [phpBB History](https://www.phpbb.com/about/history/) 453 + - [phpBB Wikipedia](https://en.wikipedia.org/wiki/PhpBB) 454 + - [phpBB Permission System Docs](https://area51.phpbb.com/docs/dev/3.3.x/extensions/permission_system.html) 455 + - [phpBB Admin Guide](https://www.phpbb.com/support/docs/en/3.2/ug/adminguide/) 456 + - [phpBB 4.0 Alpha Announcement](https://blog.phpbb.com/2025/09/27/phpbb-4-0-0-a1-the-first-alpha-release-is-here/) 457 + - [phpBB Community: "phpBB is a very bad choice"](https://www.phpbb.com/community/viewtopic.php?t=2155257) 458 + - [phpBB Community: "What is the Future of phpBB?"](https://www.phpbb.com/community/viewtopic.php?t=2607651) 459 + - [HN: Why did phpBB forums go out of fashion?](https://news.ycombinator.com/item?id=33726098) 460 + - [HN: Are forum platforms dead?](https://news.ycombinator.com/item?id=32171283) 461 + - [Discourse vs Flarum vs phpBB Comparison](https://www.accuwebhosting.com/blog/discourse-vs-flarum-vs-phpbb/) 462 + - [Discourse Trust Levels](https://blog.discourse.org/2018/06/understanding-discourse-trust-levels/) 463 + - [Discourse Gamification Plugin](https://meta.discourse.org/t/discourse-gamification/225916) 464 + - [phpBB Moderation Guide](https://www.phpbb.com/support/docs/en/3.0/ug/moderatorguide/moderator_modtools/) 465 + - [phpBB Search Indexing](https://www.phpbb.com/support/docs/en/3.0/ug/adminguide/maintenance_search/) 466 + - [phpBB ATOM Feeds FAQ](https://www.phpbb.com/support/docs/en/3.0/kb/article/faq-phpbb-atom-feeds/) 467 + - [LowEndTalk: How did phpBB go from biggest to obscurity?](https://lowendtalk.com/discussion/208407/how-did-phpbb-go-from-the-biggest-forum-platform-on-the-block-to-relative-obscurity) 468 + - [Quora: Pros and cons of phpBB](https://www.quora.com/What-are-the-pros-and-cons-of-using-phpBB-for-web-forums)

+623

docs/research/realtime-architecture.md

··· 1 + # Real-Time Architecture: Firehose → SSE → HTMX 2 + 3 + *Deep-dive research conducted 2026-02-07* 4 + 5 + --- 6 + 7 + ## The Opportunity 8 + 9 + Every traditional forum (phpBB, Discourse, Flarum) works on a request-response model. You load a topic page. It's a snapshot. If someone replies while you're reading, you don't see it until you manually refresh. Discourse has real-time features via WebSockets, but it's a bolt-on — a separate infrastructure layer they built on top of their standard Rails app. 10 + 11 + atBB is different. The AT Protocol *already has* a real-time event stream (the firehose/Jetstream). The AppView *already needs* to subscribe to it for indexing. The web UI *already uses* HTMX, which has declarative SSE support. The entire pipeline exists — it just needs to be connected. 12 + 13 + **The data flow:** 14 + ``` 15 + User writes post AT Proto Jetstream atBB Browser 16 + to their PDS ───▶ Relay ───▶ (JSON WS) ───▶ AppView ───▶ (HTMX SSE) 17 + indexes + 18 + broadcasts 19 + ``` 20 + 21 + End-to-end latency: ~1–2 seconds from PDS write to browser update. That's fast enough to feel "live" without feeling like chat. 22 + 23 + --- 24 + 25 + ## Architecture Overview 26 + 27 + ### Three Layers 28 + 29 + ``` 30 + ┌─────────────────────────────────────────────────────┐ 31 + │ Browser (HTMX) │ 32 + │ │ 33 + │ ┌────────────┐ ┌──────────┐ ┌─────────────────┐ │ 34 + │ │ Topic View │ │ Category │ │ Notification │ │ 35 + │ │ sse-swap= │ │ View │ │ Badge │ │ 36 + │ │ "newReply" │ │ sse-swap=│ │ sse-swap= │ │ 37 + │ │ │ │ "newTopic│ │ "notification" │ │ 38 + │ └─────┬──────┘ └────┬─────┘ └───────┬─────────┘ │ 39 + │ │ │ │ │ 40 + │ └──────────────┴────────────────┘ │ 41 + │ │ SSE │ 42 + └───────────────────────┼──────────────────────────────┘ 43 + │ 44 + ┌───────────────────────┼──────────────────────────────┐ 45 + │ @atbb/web (Hono) │ 46 + │ │ │ 47 + │ GET /sse/thread/:id ──────┐ │ 48 + │ GET /sse/category/:id ────┤ SSE │ 49 + │ GET /sse/global ──────────┤ Endpoints │ 50 + │ │ │ 51 + │ ┌─────────────────────────┘ │ 52 + │ │ Subscribe to AppView event bus │ 53 + │ │ Render HTML fragments │ 54 + │ │ Stream as SSE events │ 55 + └────────────┼─────────────────────────────────────────┘ 56 + │ 57 + ┌────────────┼─────────────────────────────────────────┐ 58 + │ │ @atbb/appview (Hono) │ 59 + │ │ │ 60 + │ ┌─────────▼──────────┐ ┌────────────────────────┐ │ 61 + │ │ Event Bus │ │ Jetstream Consumer │ │ 62 + │ │ (in-process) │◀──│ │ │ 63 + │ │ │ │ space.atbb.* │ │ 64 + │ │ topic:abc → │ │ filter + index │ │ 65 + │ │ [subscriber1] │ │ │ │ 66 + │ │ [subscriber2] │ │ Cursor persistence │ │ 67 + │ │ category:xyz → │ │ Reconnection logic │ │ 68 + │ │ [subscriber3] │ └────────────┬───────────┘ │ 69 + │ └────────────────────┘ │ │ 70 + │ │ │ 71 + │ ┌─────────────────────────────────────▼───────────┐ │ 72 + │ │ PostgreSQL │ │ 73 + │ │ posts | categories | users | memberships │ │ 74 + │ └─────────────────────────────────────────────────┘ │ 75 + └──────────────────────────────────────────────────────┘ 76 + │ 77 + │ WebSocket (JSON) 78 + ▼ 79 + ┌──────────────────────────────────────────────────────┐ 80 + │ Jetstream │ 81 + │ wss://jetstream2.us-east.bsky.network/subscribe │ 82 + │ ?wantedCollections=space.atbb.* │ 83 + └──────────────────────────────────────────────────────┘ 84 + ``` 85 + 86 + ### Why SSE Over WebSocket for the Browser Connection 87 + 88 + | Factor | SSE | WebSocket | 89 + |--------|-----|-----------| 90 + | Direction | Server → Client only | Bidirectional | 91 + | Forum fit | Perfect — users read far more than they write | Overkill | 92 + | User writes | Standard HTMX POST/PUT (already works) | Would need `ws-send` | 93 + | Infrastructure | Works through all proxies, CDNs, load balancers | Needs sticky sessions, special proxy config | 94 + | Reconnection | Browser `EventSource` auto-reconnects natively | Extension handles it | 95 + | Graceful degradation | If SSE breaks, forum still works as normal HTTP | Same | 96 + | HTMX integration | `sse-swap` maps events to DOM targets declaratively | OOB swap by ID only | 97 + | HTTP/2 concern | Uses one connection per stream (H/2 multiplexes) | Separate TCP connection | 98 + 99 + **Forum interactions are fundamentally asymmetric.** Users spend 95% of their time reading. SSE handles the high-volume server→client push (new replies, presence, typing indicators). Standard HTMX POST handles the low-volume client→server actions (submitting replies, reacting). WebSocket's bidirectionality is wasted here. 100 + 101 + --- 102 + 103 + ## Layer 1: Jetstream Consumer (AppView) 104 + 105 + ### Connection Setup 106 + 107 + Use `@skyware/jetstream` for MVP — it provides typed JSON events with cursor management. The full `@atproto/sync` firehose (CBOR + signatures) is available for post-MVP hardening. 108 + 109 + ```typescript 110 + // packages/appview/src/firehose/consumer.ts 111 + 112 + import { Jetstream } from "@skyware/jetstream"; 113 + import type { EventEmitter } from "node:events"; 114 + 115 + interface FirehoseConsumer { 116 + start(): void; 117 + stop(): void; 118 + events: EventEmitter; // broadcast bus 119 + } 120 + 121 + function createFirehoseConsumer(db: Database): FirehoseConsumer { 122 + const events = new EventEmitter(); 123 + 124 + const jetstream = new Jetstream({ 125 + endpoint: "wss://jetstream2.us-east.bsky.network/subscribe", 126 + wantedCollections: ["space.atbb.*"], 127 + cursor: loadCursorFromDb(db), // microsecond timestamp 128 + }); 129 + 130 + // Index new posts and broadcast for SSE 131 + jetstream.onCreate("space.atbb.post", async (event) => { 132 + const { did, commit, time_us } = event; 133 + const post = await indexPost(db, did, commit.rkey, commit.cid, commit.record); 134 + saveCursor(db, time_us); 135 + 136 + // Determine broadcast channel 137 + if (post.rootPostId) { 138 + // It's a reply — broadcast to the thread channel 139 + events.emit(`topic:${post.rootPostId}`, { type: "newReply", post }); 140 + } else { 141 + // It's a new topic — broadcast to the category channel 142 + // NOTE: Current schema has `forumUri` not `categoryId`. To route 143 + // "new topic in category X" events, need to either: (a) resolve 144 + // category from forum metadata, or (b) add categoryUri to posts table. 145 + // For now, broadcast to forum-level channel: 146 + events.emit(`forum:${post.forumUri}`, { type: "newTopic", post }); 147 + } 148 + 149 + // Always broadcast to global (for notification badges, etc.) 150 + events.emit("global", { type: "newPost", post }); 151 + }); 152 + 153 + jetstream.onDelete("space.atbb.post", async (event) => { 154 + const { did, commit, time_us } = event; 155 + const post = await softDeletePost(db, did, commit.rkey); 156 + saveCursor(db, time_us); 157 + 158 + if (post) { 159 + events.emit(`topic:${post.rootPostId}`, { type: "postDeleted", post }); 160 + } 161 + }); 162 + 163 + // Index other record types similarly... 164 + jetstream.onCreate("space.atbb.forum.category", async (event) => { 165 + await indexCategory(db, event.did, event.commit.rkey, event.commit.record); 166 + saveCursor(db, event.time_us); 167 + events.emit("global", { type: "categoryUpdate" }); 168 + }); 169 + 170 + jetstream.onCreate("space.atbb.reaction", async (event) => { 171 + const reaction = await indexReaction(db, event.did, event.commit); 172 + saveCursor(db, event.time_us); 173 + events.emit(`topic:${reaction.topicId}`, { type: "newReaction", reaction }); 174 + }); 175 + 176 + return { 177 + start: () => jetstream.start(), 178 + stop: () => jetstream.close(), 179 + events, 180 + }; 181 + } 182 + ``` 183 + 184 + ### Jetstream Filtering 185 + 186 + Jetstream supports NSID prefix wildcards: 187 + 188 + ``` 189 + ?wantedCollections=space.atbb.* 190 + ``` 191 + 192 + This catches `space.atbb.post`, `space.atbb.forum.forum`, `space.atbb.forum.category`, `space.atbb.membership`, `space.atbb.reaction`, `space.atbb.modAction` — everything in the atBB namespace. 193 + 194 + **Note:** Wildcard syntax is supported as long as the prefix (`space.atbb`) passes NSID validation, which it does. Jetstream allows up to 100 collection filters per connection. If wildcard filtering proves problematic in practice, the seven collections can be enumerated explicitly. 195 + 196 + ### Cursor Management 197 + 198 + Jetstream events have a `time_us` field (Unix microseconds). Persist this as a cursor: 199 + 200 + ```typescript 201 + // Save every 100 events (not every event — that would hammer the DB) 202 + let eventsSinceSave = 0; 203 + function saveCursor(db: Database, cursor: number) { 204 + eventsSinceSave++; 205 + if (eventsSinceSave >= 100) { 206 + db.execute("UPDATE firehose_state SET cursor = $1", [cursor]); 207 + eventsSinceSave = 0; 208 + } 209 + } 210 + 211 + // On reconnect, rewind 5 seconds for safety (process events idempotently) 212 + function loadCursorFromDb(db: Database): number | undefined { 213 + const row = db.queryOne("SELECT cursor FROM firehose_state"); 214 + if (!row?.cursor) return undefined; 215 + return row.cursor - 5_000_000; // 5 seconds in microseconds 216 + } 217 + ``` 218 + 219 + **Backfill window:** Jetstream retains ~24 hours of events. If the AppView is offline longer, fall back to `com.atproto.sync.getRepo` for known DIDs. 220 + 221 + --- 222 + 223 + ## Layer 2: Event Bus → SSE Endpoints (Web Package) 224 + 225 + The web package subscribes to the AppView's event bus and renders HTML fragments streamed to the browser. 226 + 227 + ### Option A: Internal Event Bus (Single-Process) 228 + 229 + If appview and web run in the same process (or web calls an appview SSE endpoint): 230 + 231 + ```typescript 232 + // packages/appview/src/routes/events.ts 233 + 234 + import { Hono } from "hono"; 235 + import { streamSSE } from "hono/streaming"; 236 + 237 + const app = new Hono(); 238 + 239 + // SSE endpoint for a specific thread 240 + app.get("/api/events/topic/:id", async (c) => { 241 + const topicId = c.req.param("id"); 242 + 243 + return streamSSE(c, async (stream) => { 244 + // Send initial connection confirmation 245 + await stream.writeSSE({ event: "connected", data: "ok" }); 246 + 247 + // Heartbeat to prevent proxy timeouts 248 + const heartbeat = setInterval(async () => { 249 + await stream.writeSSE({ event: "heartbeat", data: "" }); 250 + }, 30_000); 251 + 252 + // Subscribe to topic events 253 + const handler = async (event: ForumEvent) => { 254 + // Hono's streamSSE accepts JSX directly in the data field 255 + await stream.writeSSE({ 256 + event: event.type, // "newReply", "newReaction", "postDeleted" 257 + data: renderEventComponent(event), // Returns JSX element 258 + }); 259 + }; 260 + 261 + firehoseConsumer.events.on(`topic:${topicId}`, handler); 262 + 263 + // Cleanup on disconnect 264 + stream.onAbort(() => { 265 + clearInterval(heartbeat); 266 + firehoseConsumer.events.off(`topic:${topicId}`, handler); 267 + }); 268 + }); 269 + }); 270 + 271 + // SSE endpoint for a category (new topics) 272 + app.get("/api/events/category/:id", async (c) => { 273 + const categoryId = c.req.param("id"); 274 + 275 + return streamSSE(c, async (stream) => { 276 + const heartbeat = setInterval(async () => { 277 + await stream.writeSSE({ event: "heartbeat", data: "" }); 278 + }, 30_000); 279 + 280 + const handler = async (event: ForumEvent) => { 281 + await stream.writeSSE({ 282 + event: event.type, 283 + data: renderEventComponent(event) 284 + }); 285 + }; 286 + 287 + firehoseConsumer.events.on(`category:${categoryId}`, handler); 288 + 289 + stream.onAbort(() => { 290 + clearInterval(heartbeat); 291 + firehoseConsumer.events.off(`category:${categoryId}`, handler); 292 + }); 293 + }); 294 + }); 295 + ``` 296 + 297 + ### Option B: AppView Exposes SSE, Web Proxies It 298 + 299 + If appview and web are separate processes, the web package can either: 300 + 1. Proxy the SSE stream from appview directly 301 + 2. Consume appview SSE internally and re-emit with HTML rendering 302 + 303 + Option 1 is simpler — the appview SSE endpoint returns HTML fragments, and the web package's JSX templates include `sse-connect` pointing at the appview. 304 + 305 + ### HTML Fragment Rendering 306 + 307 + The key insight: SSE events carry **pre-rendered HTML fragments**, not JSON. This is what makes HTMX SSE zero custom client-side JS. 308 + 309 + ```typescript 310 + // packages/web/src/components/ReplyCard.tsx 311 + 312 + import type { FC } from "hono/jsx"; 313 + 314 + interface ReplyCardProps { 315 + author: string; 316 + authorDid: string; 317 + text: string; 318 + createdAt: string; 319 + replyCount?: number; 320 + } 321 + 322 + export const ReplyCard: FC<ReplyCardProps> = (props) => ( 323 + <article class="reply" id={`reply-${props.authorDid}-${props.createdAt}`}> 324 + <header class="reply-meta"> 325 + <a href={`/user/${props.authorDid}`} class="reply-author"> 326 + {props.author} 327 + </a> 328 + <time datetime={props.createdAt}> 329 + {new Date(props.createdAt).toLocaleString()} 330 + </time> 331 + </header> 332 + <div class="reply-body"> 333 + {props.text} 334 + </div> 335 + </article> 336 + ); 337 + 338 + // Render JSX component for SSE 339 + function renderReplyComponent(post: IndexedPost) { 340 + // Hono's streamSSE accepts JSX directly — no manual string conversion needed 341 + return <ReplyCard 342 + author={post.authorHandle} 343 + authorDid={post.authorDid} 344 + text={post.text} 345 + createdAt={post.createdAt} 346 + />; 347 + // Alternative if string needed: component.toString() 348 + } 349 + ``` 350 + 351 + --- 352 + 353 + ## Layer 3: HTMX SSE in the Browser 354 + 355 + ### Topic View (Thread Page) 356 + 357 + ```tsx 358 + // packages/web/src/routes/topic.tsx 359 + 360 + export const TopicView: FC<TopicViewProps> = ({ topic, replies }) => ( 361 + <BaseLayout title={topic.title}> 362 + {/* SSE connection scoped to this thread */} 363 + <div hx-ext="sse" sse-connect={`/api/events/topic/${topic.id}`}> 364 + 365 + {/* Thread header */} 366 + <article class="topic-op"> 367 + <h1>{topic.title}</h1> 368 + <div class="post-body">{topic.text}</div> 369 + <div class="post-meta"> 370 + by <a href={`/user/${topic.authorDid}`}>{topic.author}</a> 371 + {" · "} 372 + <time datetime={topic.createdAt}>{topic.createdAt}</time> 373 + </div> 374 + </article> 375 + 376 + {/* Reply list — new replies streamed in at the bottom */} 377 + <section id="replies"> 378 + {replies.map((reply) => ( 379 + <ReplyCard {...reply} /> 380 + ))} 381 + 382 + {/* This target receives new replies via SSE */} 383 + <div sse-swap="newReply" hx-swap="beforebegin"></div> 384 + </section> 385 + 386 + {/* Reaction updates swap into specific post elements via OOB */} 387 + {/* (The SSE "newReaction" event sends OOB-targeted HTML) */} 388 + 389 + {/* Reply count badge — updated in real-time */} 390 + <span id="reply-count" sse-swap="replyCount"> 391 + {replies.length} replies 392 + </span> 393 + 394 + {/* Typing indicator */} 395 + <div id="typing-indicator" sse-swap="typing"></div> 396 + 397 + </div> 398 + 399 + {/* Reply form — standard HTMX POST (not SSE) */} 400 + <form hx-post={`/api/topics/${topic.id}/reply`} 401 + hx-swap="none" 402 + hx-on::after-request="this.reset()"> 403 + <textarea name="text" placeholder="Write a reply..." 404 + required minlength="1" maxlength="3000"></textarea> 405 + <button type="submit">Reply</button> 406 + </form> 407 + </BaseLayout> 408 + ); 409 + ``` 410 + 411 + ### Category View (Topic List) 412 + 413 + ```tsx 414 + export const CategoryView: FC<CategoryViewProps> = ({ category, topics }) => ( 415 + <BaseLayout title={category.name}> 416 + <div hx-ext="sse" sse-connect={`/api/events/category/${category.id}`}> 417 + 418 + <h1>{category.name}</h1> 419 + <p>{category.description}</p> 420 + 421 + <table class="topic-list"> 422 + <thead> 423 + <tr> 424 + <th>Topic</th> 425 + <th>Author</th> 426 + <th>Replies</th> 427 + <th>Last Post</th> 428 + </tr> 429 + </thead> 430 + <tbody id="topic-list-body" 431 + sse-swap="newTopic" 432 + hx-swap="afterbegin"> 433 + {topics.map((topic) => <TopicRow {...topic} />)} 434 + </tbody> 435 + </table> 436 + 437 + </div> 438 + </BaseLayout> 439 + ); 440 + ``` 441 + 442 + ### Global Notification Badge 443 + 444 + This could be placed in the base layout so it works on every page: 445 + 446 + ```tsx 447 + // packages/web/src/layouts/base.tsx 448 + 449 + export const BaseLayout: FC<BaseLayoutProps> = (props) => ( 450 + <html> 451 + <head> 452 + <script src="https://unpkg.com/htmx.org@2.0.4" /> 453 + <script src="https://unpkg.com/htmx-ext-sse@2.2.3/sse.js" /> 454 + </head> 455 + <body hx-boost="true"> 456 + <header hx-ext="sse" sse-connect="/api/events/global"> 457 + <nav> 458 + <a href="/">atBB Forum</a> 459 + {/* Notification badge — updated live */} 460 + <span id="notification-badge" sse-swap="notification"></span> 461 + </nav> 462 + </header> 463 + <main> 464 + {props.children} 465 + </main> 466 + </body> 467 + </html> 468 + ); 469 + ``` 470 + 471 + --- 472 + 473 + ## What This Enables (Concrete Features) 474 + 475 + ### Tier 1: Easy Wins (MVP-compatible) 476 + 477 + | Feature | SSE Event | Behavior | 478 + |---------|-----------|----------| 479 + | **Live replies** | `newReply` | New reply appears at bottom of thread without refresh | 480 + | **Live new topics** | `newTopic` | New topic appears at top of category view | 481 + | **Reply count** | `replyCount` | Reply count badge updates in real-time | 482 + | **Post deletion** | `postDeleted` | Deleted post fades out or shows "[deleted]" | 483 + | **Mod actions** | `modAction` | Locked/pinned status updates live | 484 + 485 + ### Tier 2: Enhanced UX (Post-MVP) 486 + 487 + | Feature | SSE Event | Behavior | 488 + |---------|-----------|----------| 489 + | **Typing indicator** | `typing` | "User X is typing..." shown below reply list | 490 + | **Online presence** | `presence` | "12 users viewing this topic" | 491 + | **Reaction animations** | `newReaction` | Like/upvote count increments with animation | 492 + | **Unread badges** | `notification` | Global nav shows unread count for subscribed topics | 493 + | **Topic bumping** | `topicBumped` | Topic list reorders when a topic gets a new reply | 494 + 495 + ### Tier 3: Differentiators (Future) 496 + 497 + | Feature | Description | 498 + |---------|-------------| 499 + | **Cross-forum activity feed** | Since AT Proto identities span forums, show activity from all forums a user participates in | 500 + | **Live moderation dashboard** | Stream mod queue events in real-time | 501 + | **"Someone replied to your post" toasts** | Non-intrusive notification popups | 502 + 503 + --- 504 + 505 + ## Scaling Considerations 506 + 507 + ### Single Instance (MVP) 508 + 509 + For MVP, an in-process `EventEmitter` is sufficient: 510 + 511 + ``` 512 + Jetstream → AppView process (index + EventEmitter) → SSE streams to browsers 513 + ``` 514 + 515 + No external infrastructure needed. The EventEmitter holds subscriber lists in memory. 516 + 517 + **Capacity:** A single Node.js process can comfortably hold 1,000+ SSE connections. For a self-hosted forum, this is more than enough. 518 + 519 + ### Multi-Instance (Production) 520 + 521 + If atBB needs horizontal scaling: 522 + 523 + ``` 524 + Jetstream → AppView Instance 1 ──┐ 525 + ├── Redis Pub/Sub ──┬── Web Instance 1 → SSE 526 + Jetstream → AppView Instance 2 ──┘ └── Web Instance 2 → SSE 527 + ``` 528 + 529 + - Only one AppView instance should consume Jetstream (use leader election or a single indexer process) 530 + - That instance publishes events to Redis Pub/Sub 531 + - All web instances subscribe to Redis and stream to their connected clients 532 + - **Alternative:** Use PostgreSQL `LISTEN/NOTIFY` instead of Redis — one fewer dependency 533 + 534 + ### HTTP/2 Requirement 535 + 536 + SSE connections hold an HTTP connection open. Under HTTP/1.1, browsers limit to 6 connections per domain — a user with multiple tabs could exhaust this. **HTTP/2 multiplexes all streams over a single TCP connection**, eliminating this issue. 537 + 538 + **Action:** Ensure the deployment setup (Docker Compose with nginx/Caddy) uses HTTP/2. Caddy enables HTTP/2 by default. 539 + 540 + ### Proxy Configuration 541 + 542 + SSE requires long-lived connections. Configure reverse proxy timeouts: 543 + 544 + ```nginx 545 + # nginx — for SSE routes only 546 + location /api/events/ { 547 + proxy_pass http://appview:3000; 548 + proxy_http_version 1.1; 549 + proxy_set_header Connection ""; 550 + proxy_buffering off; 551 + proxy_cache off; 552 + proxy_read_timeout 86400s; # 24 hours 553 + } 554 + ``` 555 + 556 + Or with Caddy (no special config needed — it handles streaming correctly by default). 557 + 558 + --- 559 + 560 + ## Implementation Roadmap 561 + 562 + ### Phase 1 (Part of AppView Core milestone) 563 + 1. **Add Jetstream consumer** — `@skyware/jetstream` with `space.atbb.*` filter 564 + 2. **Add in-process EventEmitter** — broadcast indexed events by channel 565 + 3. **Add SSE endpoint** — `/api/events/topic/:id` using Hono's `streamSSE` 566 + 4. **Persist cursor** — store `time_us` in PostgreSQL, reload on restart 567 + 568 + ### Phase 2 (Part of Web UI milestone) 569 + 5. **Add `htmx-ext-sse`** — include SSE extension alongside HTMX in base layout 570 + 6. **Add `sse-connect` to topic view** — connect to thread-specific SSE stream 571 + 7. **Render reply HTML fragments** — reuse existing JSX components for SSE payloads 572 + 8. **Add `sse-connect` to category view** — connect to category-specific SSE stream 573 + 574 + ### Phase 3 (Post-MVP polish) 575 + 9. **Typing indicators** — debounced POST from textarea keyup, broadcast via SSE 576 + 10. **Presence tracking** — track SSE connections per topic, broadcast count 577 + 11. **Notification badges** — global SSE stream for subscribed topic updates 578 + 12. **Graceful degradation** — ensure forum works perfectly without SSE (progressive enhancement) 579 + 580 + ### Dependencies 581 + 582 + | Step | Depends On | New Packages | 583 + |------|-----------|--------------| 584 + | Jetstream consumer | Phase 1 AppView Core | `@skyware/jetstream`, `ws` | 585 + | EventEmitter | Jetstream consumer | None (Node.js built-in) | 586 + | SSE endpoint | EventEmitter + Hono | None (`hono/streaming` built-in) | 587 + | HTMX SSE extension | Web UI + SSE endpoint | `htmx-ext-sse` (CDN) | 588 + 589 + --- 590 + 591 + ## Why This Matters 592 + 593 + No other forum software has this architecture. Here's the comparison: 594 + 595 + | Forum | Real-Time Approach | 596 + |-------|-------------------| 597 + | **phpBB** | None. Pure request-response. Must refresh to see new replies. | 598 + | **Discourse** | Custom WebSocket implementation via MessageBus gem. Bolt-on architecture, requires Redis, sticky sessions. | 599 + | **Flarum** | Pusher.com integration (third-party SaaS). Adds external dependency and cost. | 600 + | **NodeBB** | Socket.io (WebSocket). Heavy client-side JS framework. | 601 + | **atBB** | Protocol-native firehose → in-process event bus → SSE → HTMX declarative swap. Zero *custom* client-side JS (HTMX itself is ~14KB). The real-time stream is architecturally intrinsic, not bolted on. | 602 + 603 + The AT Protocol firehose means atBB doesn't *add* real-time — it *is* real-time. The firehose consumer that indexes posts is the same component that powers live updates. There's no separate infrastructure, no Redis, no WebSocket server, no client-side framework. Just HTML attributes. 604 + 605 + This is atBB's strongest architectural differentiator and should be a first-class feature from day one. 606 + 607 + --- 608 + 609 + ## Sources 610 + 611 + - [HTMX SSE Extension Docs](https://htmx.org/extensions/sse/) 612 + - [htmx-ext-sse npm package](https://www.npmjs.com/package/htmx-ext-sse) 613 + - [HTMX WebSocket vs SSE comparison](https://htmx.org/extensions/ws/) 614 + - [Hono SSE Streaming API](https://hono.dev/docs/helpers/streaming) 615 + - [Bluesky Jetstream GitHub](https://github.com/bluesky-social/jetstream) 616 + - [Jetstream: Shrinking the Firehose by >99%](https://jazco.dev/2024/09/24/jetstream/) 617 + - [@skyware/jetstream](https://www.npmjs.com/package/@skyware/jetstream) 618 + - [@atproto/sync](https://www.npmjs.com/package/@atproto/sync) 619 + - [@atproto/tap (backfill tool)](https://docs.bsky.app/blog/introducing-tap) 620 + - [Bluesky Firehose Guide](https://docs.bsky.app/docs/advanced-guides/firehose) 621 + - [benc-uk/htmx-go-chat (SSE chat example)](https://github.com/benc-uk/htmx-go-chat) 622 + - [Live Website Updates with Go, SSE, and HTMX](https://threedots.tech/post/live-website-updates-go-sse-htmx/) 623 + - [SSE vs WebSockets](https://www.smashingmagazine.com/2018/02/sse-websockets-data-flow-http2/)

Configure Feed

Configure Feed