Full document, spreadsheet, slideshow, and diagram tooling
0
fork

Configure Feed

Select the types of activity you want to include in your feed.

Merge pull request 'docs: add PRODUCT.md, README.md, CONTRIBUTING.md, update CHANGELOG' (#48) from docs/product-and-contributing into main

scott 9a248f5b 570aa0e5

+1658 -36
+100 -36
CHANGELOG.md
··· 6 6 7 7 ## [Unreleased] 8 8 9 - ### Added 10 - - Both: accessibility, mobile responsive, sharing/permissions (#39) 11 - - Both: right-click context menus, print layout (#38) 12 - - Docs: outline sidebar, table improvements, link previews, zen mode (#37) 13 - - Sheets: drag-to-fill, paste special, format painter, virtual scrolling (#36) 14 - - Sheets: cross-sheet refs, named ranges, formula tracer (#35) 15 - - Sheets: status bar, formula auto-complete, cell notes (#34) 16 - - Collaboration: version history, suggesting mode, offline support (#33) 17 - - Landing: sort, star/favorite, soft-delete/trash, folders, search, username prompt (#32) 18 - - Sheets: charts via Chart.js, multi-column filter/sort (#31) 19 - - Sheets: conditional formatting, data validation, cell borders, wrap text, striped rows (#30) 20 - - Import/Export: PDF export, .docx import, .xlsx import (#29) 21 - - Docs: line spacing, paragraph spacing, page breaks, spell check (#28) 22 - - Add markdown autoformatting input rules (#27) 23 - - Toolbar polish: make more similar to Google Docs (#26) 24 - - Add dark mode support to docs and sheets (#23) 25 - - Sheets: column auto-fit width based on content (#19) 26 - - Sheets: improved drag-to-select range visual feedback (#18) 27 - - Docs+Sheets: autosave indicator with last saved timestamp (#17) 28 - - Docs: word and character count in footer (#16) 29 - - Docs: keyboard shortcut cheatsheet modal (#15) 30 - - Docs: indent/outdent for lists and paragraphs (#14) 31 - - Sheets: auto-detect CSV headers on import (#13) 32 - - Sheets: implement VLOOKUP and HLOOKUP formulas (#12) 33 - - Sheets: cell merging support (#11) 34 - - Docs: font size control beyond heading levels (#10) 35 - - Docs+Sheets: inline comments and annotations (#8) 36 - - Sheets: frozen panes (lock header rows/columns) (#7) 37 - - Sheets: resizable column widths (#6) 38 - - Docs: find and replace (Cmd+F / Cmd+H) (#5) 39 - - Toolbar: responsive collapse on narrow windows (#4) 40 - - Toolbar: add hover tooltips to all buttons (#3) 41 - - Toolbar: grouped dropdowns for alignment and list buttons (#2) 42 - - Toolbar: collapsible overflow menu for less-used items (#1) 9 + ### Documents 10 + - Outline sidebar with navigable H1/H2/H3 heading tree (#37) 11 + - Floating table toolbar: row/column manipulation, cell merge/split, header toggle, cell color (#37) 12 + - Link preview tooltips on hover: URL display, Open/Edit/Remove actions (#37) 13 + - Zen mode (Cmd+Shift+F): distraction-free editing with hidden toolbar (#37) 14 + - Slash commands (`/`): Notion-style command palette with 15 block types (#37) 15 + - Block handles: Notion-style drag handle with context menu (Turn into, Delete, Duplicate, Move) (#37) 16 + - Markdown source toggle (Cmd+Shift+M): switch between WYSIWYG and raw markdown editing 17 + - Markdown export via Turndown with GFM support (tables, task lists, strikethrough, code languages) 18 + - Markdown import via markdown-it with GFM extensions (tables, task lists, strikethrough) 19 + - Markdown autoformatting input rules: `#`, `##`, `-`, `1.`, `>`, `[]`, `---`, backticks, `**`, `~~`, `[text](url)` (#27) 20 + - Line spacing presets (1, 1.15, 1.5, 2, 2.5, 3) and paragraph spacing controls (#28) 21 + - Page breaks with print-friendly rendering (#28) 22 + - Inline comments with author, timestamp, and text (#8) 23 + - Suggesting mode (track changes): insert/delete marks with session grouping, accept/reject per suggestion or bulk (#33) 24 + - Find and replace (Cmd+F / Cmd+H) with match highlighting and active match indicator (#5) 25 + - Font size control beyond heading levels (#10) 26 + - Font family selection, text color, highlight color 27 + - Text alignment (left, center, right, justify) 28 + - Subscript, superscript support 29 + - Task lists with checkboxes (nestable) 30 + - Indent/outdent for paragraphs, headings, and lists (Cmd+]/Cmd+[) (#14) 31 + - PDF export via html2pdf.js with light-mode rendering (#29) 32 + - .docx import via mammoth.js with heading style mapping (#29) 33 + - Word and character count in footer (#16) 34 + - Keyboard shortcut cheatsheet modal (#15) 35 + 36 + ### Spreadsheets 37 + - Custom grid engine with 100x26 default grid, cell editing, formula bar 38 + - Formula engine: recursive descent parser with 50+ functions across math, text, date, lookup, conditional categories 39 + - Functions: SUM, AVERAGE, COUNT, COUNTA, MIN, MAX, MEDIAN, STDEV, ABS, ROUND, ROUNDUP, ROUNDDOWN, INT, MOD, POWER, SQRT, LOG, LN, EXP, PI, RAND, IF, AND, OR, NOT, IFERROR, CONCATENATE, LEN, LEFT, RIGHT, MID, UPPER, LOWER, TRIM, SUBSTITUTE, FIND, SEARCH, TEXT, VALUE, NOW, TODAY, DATE, YEAR, MONTH, DAY, VLOOKUP, HLOOKUP, INDEX, MATCH, SUMIF, COUNTIF, AVERAGEIF (#12) 40 + - Cross-sheet references: `Sheet2!A1`, `'Sheet Name'!A1:B5` (#35) 41 + - Named ranges: human-friendly aliases for cell ranges in formulas (#35) 42 + - Formula autocomplete dropdown with function signatures (#34) 43 + - Formula tracer: trace precedents (inputs) and dependents (outputs) of any cell (#35) 44 + - Charts via Chart.js: bar, line, pie, scatter with auto-detected headers and axis labels (#31) 45 + - Multi-column filter: per-column value checkboxes (#31) 46 + - Multi-column sort: up to 3 sort levels, stable sort (#31) 47 + - Conditional formatting: 7 rule types (greaterThan, lessThan, equalTo, between, textContains, isEmpty, isNotEmpty) with custom colors (#30) 48 + - Data validation: list (dropdown), numberBetween, textLength with visual indicators (#30) 49 + - Cell borders: per-side and preset (all, outline, none) (#30) 50 + - Wrap text toggle per cell (#30) 51 + - Striped rows (alternating background) (#30) 52 + - Cell notes: plain text annotations with triangle indicator and hover tooltips (#34) 53 + - Status bar: SUM, AVERAGE, COUNT, MIN, MAX for multi-cell selections (#34) 54 + - Resizable column widths with drag handles (#6) 55 + - Column auto-fit on double-click (measure content width) (#19) 56 + - Frozen panes: lock header rows and columns (#7) 57 + - Cell merging and unmerging with colspan/rowspan (#11) 58 + - Drag-to-fill: pattern detection (number sequences, date sequences, formula adjustment, text repeat) (#36) 59 + - Paste special: values only, formulas only, formatting only, transpose (#36) 60 + - Format painter: copy cell formatting and apply to other cells (#36) 61 + - Multi-sheet tabs with add/rename/switch (#36) 62 + - .xlsx import via SheetJS with values, formulas, bold, number formats (#29) 63 + - CSV paste with auto-detect headers (#13) 64 + - Improved drag-to-select range visual feedback with border indicators (#18) 65 + - Virtual scrolling module for large sheet performance (pure logic ready) (#36) 66 + 67 + ### Collaboration 68 + - Real-time editing via Yjs CRDT with encrypted WebSocket provider 69 + - Awareness protocol: colored cursors with usernames 70 + - Encrypted sync: all Yjs messages AES-256-GCM encrypted before transmission 71 + - Automatic encrypted snapshot persistence (debounced saves + periodic saves) 72 + - Reconnection with exponential backoff and jitter 73 + - Version history: automatic capture (edit threshold or time threshold), FIFO pruning (max 50), word count deltas, restore (#33) 74 + - Suggesting mode: track changes with insert/delete marks, session grouping, accept/reject (#33) 75 + - Offline support: online/offline detection, change queue, cache strategy (#33) 76 + 77 + ### Sharing & Permissions 78 + - Share dialog with URL builder, mode selector (edit/view), copy to clipboard (#39) 79 + - View-only mode: disables toolbar and editing, shows badge (#39) 80 + - Link expiry: 1h, 1d, 7d, 30d options with server-side enforcement (HTTP 410) (#39) 81 + - E2EE key in URL fragment: encryption key never sent to server 82 + 83 + ### Landing Page 84 + - Document list with type icons, decrypted names, last updated timestamps 85 + - Sort: by last updated, created, name, or type; starred items sort first (#32) 86 + - Search: filter by decrypted document name (#32) 87 + - Folders: create, rename, delete; move documents between folders; breadcrumbs (#32) 88 + - Favorites/Stars: star/unstar with priority sorting (#32) 89 + - Trash: soft delete with 30-day auto-purge; restore or permanently delete (#32) 90 + - Username prompt on first visit; click badge to change (#32) 91 + 92 + ### Platform & UX 93 + - Dark mode: automatic (prefers-color-scheme), manual toggle, persisted (#23) 94 + - Right-click context menus for docs (text, link, image, table) and sheets (cell, column header, row header) with keyboard navigation (#38) 95 + - Print layout: page size presets, margin presets, headers/footers, pagination for docs; grid lines, header repeat, scaling for sheets (#38) 96 + - Responsive toolbar: collapse overflow items on narrow viewports (#4) 97 + - Hover tooltips on all toolbar buttons (#3) 98 + - Grouped dropdowns for alignment and list buttons (#2) 99 + - Collapsible overflow menu for less-used toolbar items (#1) 100 + - Autosave indicator with last saved timestamp (#17) 101 + - Skip links and ARIA roles for accessibility (#39) 102 + - Mobile-responsive layout with touch-friendly targets (#39) 103 + - System fonts only (Charter, system-ui, ui-monospace) -- no external font dependencies 104 + - OkLCH color system for perceptual uniformity across themes 105 + - Legacy localStorage key migration (crypt-* to tools-*) 43 106 44 107 ### Fixed 45 108 - Fix Docker image reference in homelab-nix crypt-tools service (#24) 46 109 - Docs+Sheets: proper Tab key support (indent in docs, cell navigation in sheets) (#20) 47 - - Sheets: wire up Ctrl+Z/Cmd+Z undo keyboard binding (#9) 110 + - Sheets: wire up Ctrl+Z/Cmd+Z undo keyboard binding via Yjs UndoManager (#9) 48 111 49 112 ### Changed 50 - - Add comprehensive tests for all new features (#25) 51 113 - Rename all crypt references to tools across the codebase (#22) 114 + - Add comprehensive test suite: 60+ test files covering all pure logic modules (#25) 115 + - Toolbar redesign to match Google Docs visual style (#26)
+460
CONTRIBUTING.md
··· 1 + # Contributing to Tools 2 + 3 + ## Development Environment Setup 4 + 5 + ### Prerequisites 6 + 7 + - Node.js 22+ (LTS recommended) 8 + - npm 10+ 9 + 10 + ### Getting Started 11 + 12 + ```bash 13 + git clone <repo-url> tools 14 + cd tools 15 + npm install 16 + npm run dev 17 + ``` 18 + 19 + This starts two servers concurrently: 20 + - **Express** on `http://localhost:3000` -- API, WebSocket relay, and production static files 21 + - **Vite** on `http://localhost:5173` -- Frontend with hot module replacement 22 + 23 + Vite proxies `/api` and `/ws` requests to Express (configured in `vite.config.js`). 24 + 25 + Open `http://localhost:5173` to start developing. 26 + 27 + ### HTTPS for Local Development 28 + 29 + The Web Crypto API (`crypto.subtle`) requires a secure context. `localhost` qualifies, so HTTPS is not needed for local development. If you need HTTPS (e.g., testing on a LAN IP), generate a self-signed certificate: 30 + 31 + ```bash 32 + openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 365 -nodes 33 + ``` 34 + 35 + Place `cert.pem` and `key.pem` in the project root (or `DATA_DIR`). The server will detect them and start HTTPS on port 3443. 36 + 37 + ## Project Structure 38 + 39 + ``` 40 + tools/ 41 + server.js # Express server, WebSocket relay, SQLite, REST API 42 + vite.config.js # Vite build configuration (multi-page app) 43 + package.json # Dependencies and scripts 44 + CLAUDE.md # AI assistant context (architecture, key files) 45 + PRODUCT.md # Product vision, features, roadmap 46 + CHANGELOG.md # Release notes 47 + Dockerfile # Multi-stage Docker build 48 + 49 + src/ # Frontend source code 50 + index.html # Landing page 51 + landing.js # Landing page DOM logic 52 + landing-utils.js # Pure utility functions (sort, filter, folders, trash, search) 53 + 54 + docs/ # Document editor 55 + index.html # Docs editor page 56 + main.js # TipTap editor setup, toolbar, all feature integration 57 + extensions/ # Custom TipTap extensions 58 + font-size.js # Font size attribute on TextStyle marks 59 + indent.js # Paragraph indentation (Cmd+]/[) 60 + comment.js # Inline comment marks 61 + line-spacing.js # Line height attribute on paragraphs/headings 62 + paragraph-spacing.js # Margin-bottom attribute on paragraphs/headings 63 + page-break.js # Page break node 64 + suggestion-insert.js # Suggesting mode: inserted text mark 65 + suggestion-delete.js # Suggesting mode: deleted text mark 66 + markdown-autoformat.js # Markdown input rules ([text](url)) 67 + slash-commands.js # Notion-style "/" command palette (TipTap integration) 68 + slash-menu.js # Slash command items, categories, filtering (pure logic) 69 + block-handle.js # Block drag handle and context menu (pure logic) 70 + outline.js # Document outline extraction and tree building 71 + zen-mode.js # Distraction-free editing state 72 + link-preview.js # Link hover tooltip positioning 73 + table-toolbar.js # Floating table manipulation toolbar 74 + search-replace.js # Find & replace TipTap extension 75 + search-state.js # Find & replace state management 76 + tab-support.js # Tab key behavior in docs 77 + tab-handler.js # Tab key handler logic 78 + autoformat-rules.js # Markdown autoformat rule inventory 79 + pdf-export.js # PDF generation via html2pdf.js 80 + docx-import.js # .docx import via mammoth.js 81 + markdown-export.js # HTML-to-Markdown via Turndown 82 + markdown-parser.js # Markdown-to-HTML via markdown-it 83 + markdown-toggle.js # WYSIWYG/Markdown source toggle 84 + 85 + sheets/ # Spreadsheet editor 86 + index.html # Sheets editor page 87 + main.js # Grid engine, cell editing, toolbar, all feature integration 88 + formulas.js # Formula tokenizer, parser, evaluator (50+ functions) 89 + charts.js # Chart.js integration (config, data extraction, transforms) 90 + filter.js # Multi-column filter logic 91 + sort.js # Multi-column sort logic 92 + conditional-format.js # Conditional formatting rule evaluation 93 + data-validation.js # Cell validation (list, numberBetween, textLength) 94 + cell-styles.js # Border, wrap text, and striped row CSS generation 95 + cell-notes.js # Cell annotation CRUD 96 + status-bar.js # Selection stats (SUM, AVERAGE, COUNT, MIN, MAX) 97 + formula-autocomplete.js # Function name suggestions and keyboard navigation 98 + formula-tracer.js # Precedent/dependent cell tracing 99 + cross-sheet.js # Cross-sheet reference parsing and resolution 100 + named-ranges.js # Named range validation, CRUD, and resolution 101 + drag-fill.js # Drag-to-fill pattern detection and value generation 102 + format-painter.js # Format extraction and application 103 + paste-special.js # Values-only, formulas-only, formatting-only, transpose 104 + virtual-scroll.js # Visible row range calculation for large sheets 105 + xlsx-import.js # .xlsx import via SheetJS 106 + tab-handler.js # Tab key handler for cell navigation 107 + 108 + lib/ # Shared modules 109 + crypto.js # AES-256-GCM: generateKey, importKey, exportKey, encrypt, decrypt 110 + provider.js # Encrypted Yjs WebSocket provider with persistence 111 + version-history.js # Version capture triggers, FIFO pruning, word count deltas 112 + offline.js # Online/offline detection, change queue, cache strategy 113 + suggesting.js # Suggesting mode: sessions, accept/reject logic 114 + share-dialog.js # Share URL building, view-only mode, expiry computation 115 + print-layout.js # Print HTML generation (docs pages, sheets tables) 116 + context-menu.js # Right-click context menu component with keyboard nav 117 + 118 + css/ 119 + app.css # All styles (~4000 lines): tokens, components, responsive, dark mode 120 + 121 + tests/ # Test suite (Vitest) 122 + *.test.js # 60+ test files covering all pure logic modules 123 + ``` 124 + 125 + ## Architecture Principles 126 + 127 + ### Pure Logic Modules 128 + 129 + Every non-trivial feature is split into two layers: 130 + 131 + 1. **Pure logic module** -- no DOM, no browser APIs, fully testable. Lives in a `.js` file that exports pure functions and/or classes. 132 + 2. **DOM integration** -- lives in `main.js` (or the HTML file). Imports the pure module and wires it to the DOM. 133 + 134 + This pattern is used throughout: `formulas.js`, `filter.js`, `sort.js`, `conditional-format.js`, `suggesting.js`, `version-history.js`, `offline.js`, `slash-menu.js`, `block-handle.js`, etc. 135 + 136 + **When adding a new feature, always start with the pure logic module and its tests.** 137 + 138 + ### Yjs Integration 139 + 140 + All collaborative data flows through Yjs shared types: 141 + 142 + - **Docs:** `ydoc` is passed to TipTap via `@tiptap/extension-collaboration`. The editor reads/writes ProseMirror state via the Yjs XML fragment. 143 + - **Sheets:** `ydoc.getMap('sheets')` contains per-sheet Y.Maps, each with `cells` (Y.Map of cell data), `colWidths`, `freezeRows`, `freezeCols`, `cfRules`, `validations`, `merges`, `notes`, etc. 144 + 145 + When adding a new synced feature to sheets: 146 + 1. Add a Yjs shared type (usually a Y.Map or Y.Array) inside the sheet map 147 + 2. Add getter/setter helper functions 148 + 3. Read the Yjs data in `renderGrid()` or `refreshVisibleCells()` 149 + 4. Write to Yjs in toolbar handlers or cell edit commits 150 + 5. Changes automatically sync to all connected peers 151 + 152 + ### Encryption 153 + 154 + All data that leaves the browser is encrypted: 155 + - **Snapshots:** `Y.encodeStateAsUpdate(doc)` -> `encrypt(bytes, key)` -> PUT to server 156 + - **WebSocket messages:** Yjs sync/update messages -> `encrypt(plain, key)` -> `ws.send(encrypted)` 157 + - **Document names:** `encryptString(name, key)` -> base64 -> stored in DB 158 + 159 + When adding features that store data on the server, always encrypt first. 160 + 161 + ## How to Add a New Formula Function 162 + 163 + Formula functions are defined in `src/sheets/formulas.js` in the `callFunction()` switch statement. 164 + 165 + ### Step 1: Add the implementation 166 + 167 + ```javascript 168 + // In callFunction() switch statement: 169 + case 'MYFUNCTION': { 170 + const n = nums(args); 171 + // Your implementation here 172 + return result; 173 + } 174 + ``` 175 + 176 + The `args` parameter is an array where each element is either: 177 + - A scalar value (number, string, boolean) 178 + - A flat array (from a range like `A1:A10`) with `_rangeRows` and `_rangeCols` metadata 179 + 180 + Helper functions available: 181 + - `flat(args)` -- flatten all range arrays, filter out empty values 182 + - `nums(args)` -- flatten, convert to numbers, filter out NaN 183 + - `toNum(v)` -- convert a single value to a number (empty/null -> 0) 184 + 185 + ### Step 2: Add to the autocomplete list 186 + 187 + In `src/sheets/formula-autocomplete.js`, add an entry to `FORMULA_FUNCTIONS`: 188 + 189 + ```javascript 190 + { name: 'MYFUNCTION', signature: 'MYFUNCTION(arg1, [arg2])' }, 191 + ``` 192 + 193 + ### Step 3: Write tests 194 + 195 + In `tests/formulas.test.js` or `tests/formulas-extended.test.js`: 196 + 197 + ```javascript 198 + import { evaluate } from '../src/sheets/formulas.js'; 199 + 200 + describe('MYFUNCTION', () => { 201 + const get = (ref) => { 202 + const cells = { A1: 10, A2: 20, A3: 30 }; 203 + return cells[ref] ?? ''; 204 + }; 205 + 206 + it('returns expected result', () => { 207 + expect(evaluate('MYFUNCTION(A1, A2)', get)).toBe(expectedValue); 208 + }); 209 + 210 + it('handles empty arguments', () => { 211 + expect(evaluate('MYFUNCTION()', get)).toBe(expectedDefault); 212 + }); 213 + }); 214 + ``` 215 + 216 + ### Step 4: Run tests 217 + 218 + ```bash 219 + npm test 220 + ``` 221 + 222 + ## How to Add a New TipTap Extension 223 + 224 + TipTap extensions live in `src/docs/extensions/`. Each extension is a self-contained module. 225 + 226 + ### Step 1: Create the extension file 227 + 228 + ```javascript 229 + // src/docs/extensions/my-feature.js 230 + import { Extension } from '@tiptap/core'; 231 + // Or: import { Mark } from '@tiptap/core'; for inline marks 232 + // Or: import { Node } from '@tiptap/core'; for block nodes 233 + 234 + export const MyFeature = Extension.create({ 235 + name: 'myFeature', 236 + 237 + addOptions() { 238 + return { /* default options */ }; 239 + }, 240 + 241 + // For marks/nodes: addAttributes(), parseHTML(), renderHTML() 242 + 243 + addCommands() { 244 + return { 245 + doMyThing: (param) => ({ commands }) => { 246 + // Editor command implementation 247 + }, 248 + }; 249 + }, 250 + 251 + addKeyboardShortcuts() { 252 + return { 253 + 'Mod-Shift-X': () => this.editor.commands.doMyThing(), 254 + }; 255 + }, 256 + }); 257 + ``` 258 + 259 + ### Step 2: Register the extension 260 + 261 + In `src/docs/main.js`, import and add to the editor's extensions array: 262 + 263 + ```javascript 264 + import { MyFeature } from './extensions/my-feature.js'; 265 + 266 + const editor = new Editor({ 267 + extensions: [ 268 + // ... existing extensions 269 + MyFeature, 270 + ], 271 + }); 272 + ``` 273 + 274 + ### Step 3: Add toolbar UI (if needed) 275 + 276 + In `src/docs/main.js`, add a toolbar button handler: 277 + 278 + ```javascript 279 + document.getElementById('btn-my-feature').addEventListener('click', () => { 280 + editor.chain().focus().doMyThing().run(); 281 + }); 282 + ``` 283 + 284 + And in `src/docs/index.html`, add the toolbar button. 285 + 286 + ### Step 4: Add to slash commands (if applicable) 287 + 288 + In `src/docs/slash-menu.js`, add to `SLASH_COMMAND_ITEMS`: 289 + 290 + ```javascript 291 + { 292 + id: 'myFeature', 293 + name: 'My Feature', 294 + description: 'Description for the command palette', 295 + category: 'advanced', 296 + icon: 'icon', 297 + shortcut: 'Mod+Shift+X', 298 + }, 299 + ``` 300 + 301 + And in `src/docs/extensions/slash-commands.js`, add to `getCommandExecutor()`. 302 + 303 + ## How to Add a New Sheet Feature 304 + 305 + ### Step 1: Create the pure logic module 306 + 307 + ```javascript 308 + // src/sheets/my-feature.js 309 + 310 + /** 311 + * My Feature - brief description. 312 + * Pure functions for [what it does]. 313 + */ 314 + 315 + export function computeMyThing(input) { 316 + // Pure logic, no DOM 317 + return result; 318 + } 319 + ``` 320 + 321 + ### Step 2: Write tests 322 + 323 + ```javascript 324 + // tests/my-feature.test.js 325 + import { describe, it, expect } from 'vitest'; 326 + import { computeMyThing } from '../src/sheets/my-feature.js'; 327 + 328 + describe('computeMyThing', () => { 329 + it('handles basic case', () => { 330 + expect(computeMyThing(input)).toBe(expected); 331 + }); 332 + }); 333 + ``` 334 + 335 + ### Step 3: Add Yjs storage (if the feature has persistent state) 336 + 337 + In `src/sheets/main.js`, add getter/setter functions near the top of the file: 338 + 339 + ```javascript 340 + function getMyFeatureState() { 341 + const sheet = getActiveSheet(); 342 + if (!sheet.has('myFeature')) sheet.set('myFeature', new Y.Map()); 343 + return sheet.get('myFeature'); 344 + } 345 + ``` 346 + 347 + ### Step 4: Integrate with rendering 348 + 349 + If the feature affects cell display, modify `renderGrid()` and/or `refreshVisibleCells()`. 350 + 351 + ### Step 5: Add toolbar / keyboard handlers 352 + 353 + Wire up buttons or keyboard shortcuts in the toolbar section of `main.js`. 354 + 355 + ## Testing Conventions 356 + 357 + ### Framework: Vitest 358 + 359 + Tests live in `tests/` and use the `.test.js` extension. Run with: 360 + 361 + ```bash 362 + npm test # Run all tests once 363 + npx vitest watch # Watch mode 364 + npx vitest run tests/formulas.test.js # Run a specific file 365 + ``` 366 + 367 + ### What to Test 368 + 369 + - **All pure logic modules** -- formulas, filter, sort, conditional-format, data-validation, etc. 370 + - **State management classes** -- VersionManager, SuggestionManager, OfflineManager, etc. 371 + - **Import/export functions** -- docx-import, xlsx-import, pdf-export, markdown conversion 372 + - **Edge cases** -- empty inputs, large numbers, special characters, Unicode 373 + - **Error handling** -- invalid formulas, corrupt files, missing data 374 + 375 + ### What NOT to Test (in unit tests) 376 + 377 + - DOM manipulation (covered by integration tests or manual testing) 378 + - TipTap editor behavior (tested via TipTap's own test suite) 379 + - Network requests (mocked in integration tests) 380 + 381 + ### Test Style 382 + 383 + ```javascript 384 + import { describe, it, expect } from 'vitest'; 385 + import { myFunction } from '../src/path/to/module.js'; 386 + 387 + describe('myFunction', () => { 388 + it('returns correct result for basic input', () => { 389 + expect(myFunction('input')).toBe('expected'); 390 + }); 391 + 392 + it('handles edge case: empty input', () => { 393 + expect(myFunction('')).toBe(defaultValue); 394 + }); 395 + 396 + it('handles edge case: null', () => { 397 + expect(myFunction(null)).toBe(defaultValue); 398 + }); 399 + }); 400 + ``` 401 + 402 + ## CSS Conventions 403 + 404 + ### Design Tokens 405 + 406 + All colors, spacing, shadows, and fonts are defined as CSS custom properties in `:root`: 407 + 408 + ```css 409 + :root { 410 + --color-bg: oklch(0.965 0.005 75); 411 + --color-text: oklch(0.22 0.02 55); 412 + --color-accent: oklch(0.52 0.14 25); 413 + /* ... */ 414 + } 415 + ``` 416 + 417 + ### OkLCH Color System 418 + 419 + Colors use the OkLCH color space (`oklch(lightness chroma hue)`) for perceptual uniformity: 420 + - **Lightness**: 0 (black) to 1 (white) 421 + - **Chroma**: 0 (gray) to ~0.4 (saturated) 422 + - **Hue**: 0-360 degrees (25 = terracotta/accent, 75 = warm neutral, 155 = success green, 195 = teal/encrypted) 423 + 424 + ### Dark Mode 425 + 426 + Dark mode overrides are in `[data-theme="dark"]` and `@media (prefers-color-scheme: dark)`: 427 + 428 + ```css 429 + [data-theme="dark"] { 430 + --color-bg: oklch(0.16 0.005 75); 431 + --color-text: oklch(0.88 0.01 75); 432 + /* ... */ 433 + } 434 + ``` 435 + 436 + When adding new UI, always define colors using custom properties so dark mode works automatically. 437 + 438 + ### Fonts 439 + 440 + Three font stacks, no external font dependencies: 441 + - `--font-display`: Charter (serif, for headings) 442 + - `--font-body`: system-ui (sans-serif, for UI and body text) 443 + - `--font-mono`: ui-monospace (for code and sheet cells) 444 + 445 + ### Responsive Breakpoints 446 + 447 + - `768px`: Hide non-essential toolbar items (`.toolbar-mobile-hide`), show mobile "More" button 448 + - `640px`: Stack layouts, larger touch targets 449 + - `480px`: Minimal layout, even larger touch targets 450 + 451 + ## PR Process 452 + 453 + 1. **Branch** from `main`: `feat/<description>`, `fix/<description>`, `chore/<description>` 454 + 2. **Write tests first** (red), then implementation (green), then refactor 455 + 3. **Run tests**: `npm test` 456 + 4. **Build check**: `npm run build` (ensure no build errors) 457 + 5. **Commit** with conventional commit messages: `feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:` 458 + 6. **Push** to feature branch, create PR with clear description 459 + 7. **CI must pass** before merge 460 + 8. **Squash merge** for multi-commit PRs, regular merge for single-commit
+855
PRODUCT.md
··· 1 + # Tools — Product Document 2 + 3 + **Tools** is a privacy-first, end-to-end encrypted collaborative office suite built for individuals and small teams who refuse to trade privacy for productivity. 4 + 5 + --- 6 + 7 + ## 1. Vision 8 + 9 + ### What is Tools? 10 + 11 + Tools is a self-hosted, E2EE office suite that provides Google Docs-caliber collaborative editing without exposing a single byte of document content to the server. It currently ships two surfaces — **Documents** and **Spreadsheets** — with forms, slides, and diagrams on the roadmap. 12 + 13 + ### Who is it for? 14 + 15 + - Privacy-conscious individuals who want a real office suite, not just a notes app 16 + - Small teams and families on a shared Tailscale network who need collaborative editing 17 + - Power users who want keyboard-driven workflows with zero vendor lock-in 18 + - Developers who value self-hosting and minimal dependencies over SaaS subscriptions 19 + 20 + ### What makes it different? 21 + 22 + | | Google Docs / Notion | Cryptee / Standard Notes | Tools | 23 + |---|---|---|---| 24 + | **Full office suite** (docs, sheets, charts) | Yes | No (notes only) | Yes | 25 + | **Real-time collaboration** | Yes | No | Yes | 26 + | **End-to-end encrypted** | No | Yes | Yes | 27 + | **Self-hosted** | No | Partial | Yes | 28 + | **Zero dependencies on cloud services** | No | No | Yes | 29 + | **Offline-capable** | Partial | Yes | Yes (with queued sync) | 30 + 31 + The key insight: encryption and collaboration are not mutually exclusive. Tools uses Yjs CRDTs for conflict-free merging and AES-256-GCM encryption on every byte that leaves the browser. The server is a dumb relay — it never sees plaintext. 32 + 33 + --- 34 + 35 + ## 2. Architecture Overview 36 + 37 + ### Stack 38 + 39 + ``` 40 + Browser (vanilla JS) 41 + ├─ TipTap (ProseMirror) → rich text docs 42 + ├─ Custom grid engine → spreadsheets 43 + ├─ Chart.js → data visualization 44 + ├─ Yjs CRDT → conflict-free collaboration 45 + └─ Web Crypto API → AES-256-GCM E2EE 46 + 47 + Server (Node.js) 48 + ├─ Express → REST API + static serving 49 + ├─ WebSocket relay → encrypted message relay 50 + ├─ SQLite (WAL mode) → encrypted document storage 51 + └─ Optional HTTPS → self-signed for crypto.subtle secure context 52 + ``` 53 + 54 + ### Key Architectural Decisions 55 + 56 + **Yjs CRDT for collaboration.** Every document is a Yjs Doc. Edits merge deterministically without a central authority. The server never resolves conflicts — it relays encrypted binary messages between connected peers. 57 + 58 + **Encryption key in the URL fragment.** The `#` fragment is never sent to the server per the HTTP spec. When a user opens `tools.example.com/docs/abc123#keybase64`, the browser extracts the key from the hash, imports it via `crypto.subtle.importKey()`, and uses it for all encrypt/decrypt operations. The server only knows the document ID. 59 + 60 + **Encrypted snapshots.** The full Yjs document state is periodically encrypted and stored server-side. This allows new clients to bootstrap without waiting for a peer — the server sends the encrypted snapshot, the client decrypts it locally, then connects to the WebSocket room for live updates. 61 + 62 + **Room-based WebSocket relay.** The server groups connections by room ID (document ID). Binary messages are relayed to all other peers in the room unchanged. JSON control messages (peer-count, peer-joined, peer-left) are the only unencrypted traffic and contain no sensitive data. 63 + 64 + **Vanilla JS — no framework.** No React, Vue, or Svelte. The docs editor uses TipTap (ProseMirror wrapper). The sheets editor is a custom `<table>` renderer. This keeps the bundle small, eliminates framework churn, and gives full control over the DOM. 65 + 66 + **Pure logic modules.** Every feature is split into a pure logic module (no DOM, fully testable) and a thin DOM integration layer. `formulas.js`, `filter.js`, `sort.js`, `suggesting.js`, `version-history.js`, `offline.js`, etc. are all DOM-free. 67 + 68 + --- 69 + 70 + ## 3. Current Features 71 + 72 + ### 3.1 Documents 73 + 74 + #### Rich Text Editing 75 + - Full TipTap/ProseMirror editor with headings (H1-H6), paragraphs, blockquotes, horizontal rules 76 + - **Bold**, *italic*, ~~strikethrough~~, underline, subscript, superscript 77 + - Font family selection, font size control (beyond heading levels) 78 + - Text color and highlight color 79 + - Text alignment (left, center, right, justify) 80 + - Line spacing presets (1, 1.15, 1.5, 2, 2.5, 3) and paragraph spacing controls 81 + - Indent/outdent for paragraphs and headings (Cmd+]/Cmd+[) 82 + 83 + #### Block Types 84 + - Bullet lists, numbered lists, task lists (with checkboxes) 85 + - Code blocks (fenced, with language hints) 86 + - Inline code 87 + - Blockquotes and callout boxes 88 + - Tables (with header rows, cell merge/split, column/row add/delete, cell background color) 89 + - Images (URL-based embedding) 90 + - Page breaks 91 + 92 + #### Notion-Style Features 93 + - **Slash commands** (`/` to open command palette) with categorized menu: Text, Lists, Media, Code, Quote, Advanced 94 + - **Block handles** — Notion-style drag handle on hover with context menu: Turn into, Delete, Duplicate, Move up/down 95 + - **Markdown autoformatting** — type `# `, `## `, `- `, `1. `, `>`, `[]`, `---`, `` ` ``, `**`, `~~`, `[text](url)` and the editor auto-converts 96 + 97 + #### Document Navigation 98 + - **Outline sidebar** — auto-extracted H1/H2/H3 headings in a navigable tree, toggleable panel 99 + - **Find and replace** (Cmd+F / Cmd+H) with match highlighting and active match indicator 100 + - **Zen mode** (Cmd+Shift+F) — hides toolbar and topbar for distraction-free writing, persists preference 101 + 102 + #### Markdown Support 103 + - **Markdown source toggle** (Cmd+Shift+M) — switch between WYSIWYG and raw markdown editing 104 + - HTML-to-Markdown export via Turndown with GFM support (tables, task lists, strikethrough, code block languages) 105 + - Markdown-to-HTML import via markdown-it with GFM extensions 106 + 107 + #### Collaboration Features (Docs) 108 + - **Inline comments** — mark text ranges with author, timestamp, and comment text; rendered as highlighted spans 109 + - **Suggesting mode** — track-changes-style editing with insert/delete marks, session grouping (consecutive keystrokes share one suggestion ID), accept/reject per suggestion or bulk 110 + - **Link preview tooltips** — hover over links to see URL, Open, Edit, Remove actions 111 + - **Floating table toolbar** — context-sensitive toolbar when cursor is in a table 112 + 113 + #### Import / Export 114 + - **.docx import** via mammoth.js with heading style mapping 115 + - **PDF export** via html2pdf.js with light-mode rendering 116 + - **Markdown import/export** (toggle between WYSIWYG and source view) 117 + 118 + #### Footer & Metadata 119 + - Word and character count in footer 120 + - Autosave indicator with last saved timestamp 121 + - Keyboard shortcut cheatsheet modal 122 + 123 + ### 3.2 Spreadsheets 124 + 125 + #### Grid Engine 126 + - Custom `<table>`-based grid with 100 rows x 26 columns default (extensible) 127 + - **Cell selection** — click, Shift+click for range, drag-to-select with visual feedback 128 + - **Cell editing** — double-click or type to enter edit mode, formula bar for complex input 129 + - **Multi-sheet tabs** — multiple sheets per workbook with tab navigation 130 + - **Virtual scrolling** — only visible rows + buffer are rendered for performance 131 + 132 + #### Formula Engine 133 + - **50+ functions** across categories: 134 + - **Math**: SUM, AVERAGE, COUNT, COUNTA, MIN, MAX, MEDIAN, STDEV, ABS, ROUND, ROUNDUP, ROUNDDOWN, INT, MOD, POWER, SQRT, LOG, LN, EXP, PI, RAND 135 + - **Logical**: IF, AND, OR, NOT, IFERROR 136 + - **Text**: CONCATENATE, LEN, LEFT, RIGHT, MID, UPPER, LOWER, TRIM, SUBSTITUTE, FIND, SEARCH, TEXT, VALUE 137 + - **Date**: NOW, TODAY, DATE, YEAR, MONTH, DAY 138 + - **Lookup**: VLOOKUP, HLOOKUP, INDEX, MATCH 139 + - **Conditional**: SUMIF, COUNTIF, AVERAGEIF 140 + - **Recursive descent parser** with full operator precedence (comparison, concat, add/sub, mul/div, power, unary) 141 + - **Cell references**: A1, $A$1 (absolute), ranges (A1:B5) 142 + - **Cross-sheet references**: `Sheet2!A1`, `'Sheet Name'!A1:B5` 143 + - **Named ranges**: human-friendly aliases (e.g., `=SUM(Sales)` instead of `=SUM(A2:A100)`) 144 + - **Formula autocomplete** — dropdown suggestions when typing function names, with signatures 145 + - **Formula tracer** — trace precedents (inputs) and dependents (outputs) of any cell 146 + - **Number formatting**: auto, number, currency ($), percent, date, text 147 + - **Formula evaluation cache** for performance 148 + 149 + #### Data Features 150 + - **Charts** via Chart.js — bar, line, pie, scatter with auto-detected headers, multi-series, axis labels, title 151 + - **Multi-column filter** — per-column value checkboxes to show/hide rows 152 + - **Multi-column sort** — up to 3 sort levels with ascending/descending, stable sort, numbers before strings 153 + - **Conditional formatting** — rules engine: greaterThan, lessThan, equalTo, between, textContains, isEmpty, isNotEmpty; custom bgColor/textColor per rule; first-match-wins 154 + - **Data validation** — list (dropdown), numberBetween, textLength; visual indicators for invalid cells; dropdown arrows for list validation 155 + 156 + #### Grid Features 157 + - **Resizable columns** — drag column borders, double-click to auto-fit width based on content 158 + - **Frozen panes** — lock header rows and columns (synced via Yjs) 159 + - **Cell merging** — merge/unmerge selected ranges with colspan/rowspan 160 + - **Cell borders** — per-side (top/bottom/left/right) or presets (all, outline, none) 161 + - **Wrap text** — toggle word wrapping per cell 162 + - **Striped rows** — alternating row background for readability 163 + - **Cell notes** — plain text annotations with triangle indicator, hover tooltips 164 + - **Cell styles** — bold, italic, text color, background color, text alignment 165 + 166 + #### Advanced Operations 167 + - **Drag-to-fill** — pattern detection (number sequences, date sequences, formula adjustment, text repeat) with smart auto-fill 168 + - **Paste special** — values only, formulas only, formatting only, transpose 169 + - **Format painter** — copy cell formatting and apply to other cells 170 + - **Status bar** — SUM, AVERAGE, COUNT, MIN, MAX aggregates for multi-cell selections 171 + - **Undo/redo** via Yjs UndoManager (Cmd+Z / Cmd+Shift+Z) 172 + 173 + #### Import / Export 174 + - **.xlsx import** via SheetJS — reads values, formulas, bold formatting, number formats 175 + - **CSV import** with auto-detected headers 176 + 177 + ### 3.3 Collaboration 178 + 179 + #### Real-Time Editing 180 + - **Yjs CRDT** — conflict-free collaborative editing, no central server needed for conflict resolution 181 + - **Encrypted WebSocket provider** — all Yjs sync messages (state vectors, updates, awareness) are AES-256-GCM encrypted before transmission 182 + - **Awareness protocol** — see other users' cursors, selections, and names with colored labels 183 + - **Peer management** — automatic sync when peers join/leave, reconnection with exponential backoff + jitter 184 + 185 + #### Version History 186 + - **Automatic version capture** — triggers after N edits or M minutes (whichever first) 187 + - **FIFO pruning** — max 50 versions per document, oldest pruned automatically 188 + - **Word count delta** — each version shows +/- word count compared to previous 189 + - **Server-side storage** — versions stored as encrypted snapshots, retrievable via API 190 + - **Version restore** — load any previous version's state 191 + 192 + #### Suggesting Mode 193 + - **Track changes** — insert and delete marks with author, timestamp, session grouping 194 + - **Accept/reject** — per-suggestion or bulk accept/reject all 195 + - **Session grouping** — consecutive keystrokes by the same author at adjacent positions share one suggestion ID (like Google Docs) 196 + 197 + #### Offline Support 198 + - **Online/offline detection** with status indicator 199 + - **Change queue** — offline edits queued and synced when reconnecting 200 + - **Cache strategy** — static assets (cache-first), HTML (network-first), API/WS (network-only) 201 + - **Debounced snapshot saves** — 500ms debounce after each edit, 10s periodic saves 202 + 203 + ### 3.4 Sharing & Permissions 204 + 205 + - **Share dialog** — modal with share link, mode selector (edit/view), expiry options 206 + - **View-only mode** — `?mode=view` disables toolbar and editing; view-only badge shown 207 + - **Link expiry** — 1 hour, 1 day, 7 days, 30 days, or no expiry; server rejects access after expiry (HTTP 410) 208 + - **E2EE key in URL** — the encryption key is part of the share link; anyone with the link can decrypt 209 + - **Server-side share settings** — share_mode and expires_at stored in SQLite 210 + 211 + ### 3.5 Landing Page (Document Browser) 212 + 213 + - **Document list** with type icons, decrypted names, last updated timestamps 214 + - **Sort** — by last updated, created, name, or type; starred documents float to top 215 + - **Search** — filter by decrypted document name (client-side) 216 + - **Folders** — create, rename, delete folders; move documents between folders; breadcrumb navigation 217 + - **Favorites/Stars** — star/unstar documents; starred sort first 218 + - **Trash** — soft delete with 30-day auto-purge; restore or permanently delete; collapsible trash section 219 + - **Username prompt** — first-visit modal for display name; click badge to change; random fallback (User NNNN) 220 + - **Create actions** — large cards for New Document and New Spreadsheet 221 + - **Legacy key migration** — auto-migrates `crypt-*` localStorage keys to `tools-*` 222 + 223 + ### 3.6 Security Model 224 + 225 + #### Encryption 226 + - **AES-256-GCM** via Web Crypto API (`crypto.subtle`) 227 + - **Random IV** (96-bit / 12 bytes) per encryption; prepended to ciphertext 228 + - **Key format**: raw AES key exported as URL-safe base64 (no padding) 229 + - **Key lifecycle**: generated on document creation, stored in localStorage keyed by document ID, shared via URL fragment 230 + 231 + #### What the server sees 232 + - Document IDs (UUIDs) 233 + - Encrypted document names (AES-256-GCM ciphertext in base64) 234 + - Encrypted Yjs snapshots (binary blobs) 235 + - Encrypted WebSocket messages (relayed blindly) 236 + - Share mode (edit/view) and expiry timestamps 237 + - Created/updated timestamps 238 + 239 + #### What the server CANNOT see 240 + - Document content (any form — text, formulas, cell data, formatting) 241 + - Document names (plaintext) 242 + - User identities beyond what they choose to share (display name in awareness) 243 + - Encryption keys (never sent — URL fragment is excluded from HTTP requests) 244 + 245 + #### HTTPS 246 + - Optional self-signed HTTPS for `crypto.subtle` secure context requirement 247 + - Tailscale already encrypts transport; HTTPS is defense-in-depth for the Web Crypto API restriction 248 + 249 + ### 3.7 UI & Accessibility 250 + 251 + - **Dark mode** — automatic via `prefers-color-scheme`, manual toggle, persisted in localStorage 252 + - **Skip links** — "Skip to content" link for screen readers 253 + - **Keyboard focus management** — focus rings only on Tab navigation (hidden on mouse click) 254 + - **ARIA roles** — menus, dialogs, and modals use proper ARIA attributes 255 + - **Context menus** — keyboard navigable (arrow keys, Enter, Escape) 256 + - **Responsive toolbar** — collapses overflow items on narrow viewports 257 + - **Mobile-responsive layout** — touch-friendly create cards, responsive grid 258 + 259 + --- 260 + 261 + ## 4. Planned Features 262 + 263 + ### Wave 1: Formula Engine Hardening + UX (Critical/High) 264 + 265 + **Goal**: Make the formula engine production-grade. 266 + 267 + - **Topological recalculation engine** — evaluate formulas in dependency order instead of ad-hoc (#89) 268 + - **Circular reference detection** with clear error messages (#90) 269 + - **Array formula support and spill behavior** (#91) 270 + - **Volatile function handling** (NOW, TODAY, RAND recalculate appropriately) (#92) 271 + - **Rich formula tooltips** with parameter highlighting (#93) 272 + - **Formula bar syntax highlighting** (#94) 273 + - **Range highlighting** while editing formulas (#95) 274 + - **Contextual error tooltips** for #REF!, #VALUE!, #NAME? etc. (#96) 275 + - **Function help panel** with examples (#97) 276 + 277 + ### Wave 2: Power Functions (Medium) 278 + 279 + **Goal**: Match Google Sheets function library for power users. 280 + 281 + - **XLOOKUP** — modern replacement for VLOOKUP/HLOOKUP (#98) 282 + - **SUMIFS/COUNTIFS/AVERAGEIFS** — multi-criteria aggregation (#99) 283 + - **LET()** — named sub-expressions within formulas (#100) 284 + - **QUERY()** — SQL-like data querying (#85) 285 + - **Dynamic array functions** — FILTER, SORT, UNIQUE, SEQUENCE (#86) 286 + - **SPARKLINE()** — inline mini-charts in cells (#87) 287 + - **LAMBDA()** — user-defined functions (#88) 288 + 289 + ### Wave 3: Forms (High/Medium) 290 + 291 + **Goal**: E2EE form builder that feeds responses into Sheets. 292 + 293 + - **Form builder** with question types: text, number, date, select, multi-select, file upload, rating, matrix (#77) 294 + - **Conditional logic** — show/hide questions based on previous answers (#78) 295 + - **Responses pipeline** — form submissions encrypt and write to a linked Sheet (#79) 296 + 297 + ### Wave 4: AI Integration (Medium) 298 + 299 + **Goal**: Client-side AI assistance that respects E2EE. 300 + 301 + - **AI formula assistant** via Aperture — explain formulas, suggest corrections, generate from natural language (#101) 302 + - **AI assistant for docs** — summarize, rewrite, translate, continue writing (#63) 303 + 304 + ### Wave 5: Platform Infrastructure (High/Medium) 305 + 306 + **Goal**: The foundational capabilities that unlock many surface-level features. 307 + 308 + - **E2EE image and file storage** — encrypted blob upload/download API (#80) 309 + - **Image cells in docs** — drag-drop, paste, and insert images stored encrypted (#81) 310 + - **Image cells in sheets** (#82) 311 + - **Service worker and asset caching** for PWA (#83) 312 + - **IndexedDB document cache** for true offline-first editing (#84) 313 + - **Full-text search** across all documents (client-side index) (#55) 314 + - **Document templates** — pre-built starting points for common use cases (#56) 315 + - **Command palette** (Cmd+K) — global search and action launcher (#52) 316 + 317 + ### Wave 6: Advanced Sheets (Medium) 318 + 319 + **Goal**: Enterprise-grade sheet features. 320 + 321 + - **Pivot tables** (#44) 322 + - **Database views** — kanban, gallery, calendar (#45, #73, #74, #75) 323 + - **Timeline/Gantt view** (#76) 324 + - **Rich cell types** — checkboxes, ratings, progress bars, links, tags (#46) 325 + 326 + ### Wave 7: More Surfaces (Medium/Low) 327 + 328 + **Goal**: Complete the office suite. 329 + 330 + - **Slides** — canvas rendering engine, master layouts/themes, presenter mode, transitions, PPTX import/PDF export (#40, #65, #66, #67, #68, #69) 331 + - **Diagrams/Whiteboard** — E2EE freeform canvas for diagrams, flowcharts, mind maps (#42) 332 + 333 + ### Wave 8: Deep Collaboration (Medium) 334 + 335 + **Goal**: Team-grade collaboration features. 336 + 337 + - **Threaded comments** — reply chains on comment marks, resolve/unresolve (#48) 338 + - **Named versions and version labels** — bookmark specific versions with names (#49) 339 + - **Follow mode** — click a collaborator's avatar to follow their cursor/viewport (#50) 340 + - **Granular permissions** — per-section or per-sheet access control (#51) 341 + 342 + ### Wave 9: Integration (Medium/Low) 343 + 344 + **Goal**: Connect Tools to the broader ecosystem. 345 + 346 + - **Cross-tool integration** — embed live Sheet ranges in Docs, embed Charts from Sheets in Docs, wiki-style cross-document links (#70, #71, #72, #43) 347 + - **Matrix/Owl integration** — share notifications, link previews in chat (#64) 348 + - **REST API** for automation and scripting (#57) 349 + 350 + ### Wave 10: UX Polish (Medium/Low) 351 + 352 + **Goal**: Refinement and delight. 353 + 354 + - **Landing page overhaul** — grid/list toggle, recent files, pinned folders (#58) 355 + - **Split view** — side-by-side editing of two documents (#59) 356 + - **Minimap** for long documents (#60) 357 + - **Focus mode enhancements** — typewriter scrolling, paragraph highlighting (#61) 358 + - **Expanded theming** — custom accent colors, font preferences (#62) 359 + 360 + --- 361 + 362 + ## 5. Design Principles 363 + 364 + ### Privacy by Default 365 + Encryption is not an opt-in feature. Every document is encrypted from the moment of creation. The server architecture makes it impossible for the operator to read document content, even with full database access. 366 + 367 + ### System Fonts 368 + No external font dependencies. The font stack uses Charter for display headings (a beautiful serif with wide availability), system-ui for body text, and ui-monospace for code and spreadsheet cells. This eliminates font-loading latency and external requests. 369 + 370 + ### OkLCH Color System 371 + All colors are defined in OkLCH (a perceptually uniform color space). This ensures consistent perceived contrast in both light and dark themes without manual per-color tuning. The palette is warm-neutral with a terracotta accent and teal for collaborative/encrypted indicators. 372 + 373 + ### Minimal Dependencies 374 + The entire backend is Express + ws + better-sqlite3 + compression. The frontend adds TipTap, Yjs, Chart.js, and a handful of import/export libraries. No ORM, no database server, no build-time CSS framework. Total `dependencies` in package.json: 20 packages. 375 + 376 + ### Keyboard First 377 + Every feature is accessible via keyboard. Slash commands, Cmd+K shortcuts, keyboard-navigable context menus, Tab/Shift+Tab cell navigation in sheets, Cmd+]/[ for indentation, Cmd+Shift+F for zen mode, Cmd+Shift+M for markdown toggle. 378 + 379 + ### Pure Logic Modules 380 + Every non-trivial feature separates pure logic (testable, no DOM) from DOM integration. This makes the codebase easier to test, reason about, and refactor. Examples: `formulas.js`, `filter.js`, `sort.js`, `conditional-format.js`, `data-validation.js`, `suggesting.js`, `version-history.js`, `offline.js`, `print-layout.js`, `context-menu.js`. 381 + 382 + ### Self-Hostable 383 + Single binary (Node.js) with SQLite. No external services required. Deployable as a single Nomad job, Docker container, or systemd service. Data directory is configurable via `DATA_DIR` env var. 384 + 385 + --- 386 + 387 + ## 6. Technical Decisions 388 + 389 + ### Why Vanilla JS (No React/Vue/Svelte)? 390 + 391 + TipTap already provides a reactive document model via ProseMirror. The sheets grid is a straightforward `<table>` render that benefits from direct DOM manipulation over virtual DOM diffing. Adding a framework would increase bundle size, add a layer of abstraction over already-abstracted libraries, and introduce framework-specific patterns for marginal benefit. The pure logic module pattern gives us testability without a framework's component model. 392 + 393 + ### Why TipTap? 394 + 395 + TipTap wraps ProseMirror (the gold standard for collaborative rich text editing) with a clean extension API. It has first-class Yjs integration via `@tiptap/extension-collaboration` and `@tiptap/extension-collaboration-cursor`. The extension system made it easy to add custom marks (comments, suggestions), nodes (page breaks), and behaviors (indent, line spacing, slash commands) without forking the core. 396 + 397 + ### Why Yjs? 398 + 399 + Yjs is the most mature CRDT library for JavaScript. It handles: 400 + - Conflict-free merging of concurrent edits 401 + - Awareness protocol (cursors, user presence) 402 + - Binary encoding (compact wire format) 403 + - Undo/redo with `Y.UndoManager` 404 + - Snapshot encoding for persistence 405 + 406 + The alternative (OT via ShareDB/Firepad) requires a central server to resolve conflicts, which conflicts with our E2EE relay architecture. 407 + 408 + ### Why SQLite? 409 + 410 + Single-file database, zero configuration, WAL mode for concurrent reads during writes. Perfect for a self-hosted app that stores encrypted blobs. The data model is simple (documents table + versions table), so there is no need for a full RDBMS. 411 + 412 + ### Why Self-Signed HTTPS? 413 + 414 + The Web Crypto API (`crypto.subtle`) is only available in secure contexts (HTTPS or localhost). Since Tools is deployed on Tailscale (which already provides WireGuard-level encryption), the self-signed cert exists solely to satisfy the browser's secure context requirement. Tailscale certs are used when available. 415 + 416 + ### Why AES-256-GCM? 417 + 418 + GCM provides authenticated encryption (confidentiality + integrity). The Web Crypto API supports it natively with excellent performance. 256-bit key length provides a comfortable security margin. The 96-bit random IV is prepended to the ciphertext for self-describing messages. 419 + 420 + ### Why Vite? 421 + 422 + Fast dev server with HMR, zero-config for vanilla JS, handles multi-page apps (landing, docs, sheets) cleanly. The production build is straightforward. 423 + 424 + --- 425 + 426 + ## 7. Competitive Landscape & Inspiration 427 + 428 + ### Google Sheets — What Power Users Love 429 + 430 + **What to steal:** 431 + 432 + - **Explore / Data insights panel.** Google Sheets auto-suggests charts, pivot summaries, and trends based on selected data. We could do this client-side with heuristics (detect date columns, numeric ranges, categorical data) without sending data anywhere. The E2EE angle: "AI-powered insights that never leave your browser." 433 + - **Apps Script / Macros.** The ability to write custom functions and automate workflows is Google Sheets' biggest power-user lock-in. We should plan for a lightweight scripting layer — possibly a formula-level LAMBDA() first (#88), then a proper JavaScript sandbox that can read/write cell data. E2EE-safe because scripts run client-side. 434 + - **Data connectors and IMPORTDATA/IMPORTRANGE.** Google Sheets can pull live data from URLs, other sheets, and external databases. For Tools, IMPORTDATA could fetch from URLs client-side (respecting CORS). Cross-document ranges are already partly addressed by cross-sheet refs; extending this to cross-document would be powerful. 435 + - **Pivot table UX.** Google's pivot table builder is drag-and-drop with instant preview. Our planned pivot tables (#44) should prioritize this same discoverability — a sidebar where you drag field pills into Rows, Columns, Values, and Filters. 436 + - **Conditional formatting presets.** Google offers one-click color scales, data bars, and icon sets. Our conditional formatting (#120) should match this ease of use — not just rule-based but visual preset palettes. 437 + - **Named ranges manager.** A sidebar panel listing all named ranges with edit/delete/navigate. We have the backend (#35) but the management UI is minimal. 438 + - **Go-to range (Cmd+G or name box click).** Click the cell address box, type "A100" or a named range, and jump there. This exists (cellAddressInput) but navigation-on-Enter should be verified. 439 + 440 + **What to skip:** 441 + 442 + - Google's server-side compute (ArrayFormula spilling across 10M rows). We are client-side and should optimize for thousands of rows, not millions. Focus on UX rather than scale. 443 + 444 + ### Notion — What Makes It Sticky 445 + 446 + **What to steal:** 447 + 448 + - **Databases with views.** Notion's killer feature: one data source, multiple views (table, board, calendar, gallery, timeline). Our database views (#45, #73-#76) should follow this model exactly — one Sheet, multiple view tabs that share the underlying Yjs data. 449 + - **Relations and rollups.** Link rows in one database to rows in another, then compute summaries (count, sum, average) across the relation. This is basically a JOIN + GROUP BY. For Tools, this could mean linking two Sheets and having formulas that reference the linked rows. Extremely powerful for project management and CRM use cases. 450 + - **Synced blocks.** A block in one document that mirrors content from another document. Changes propagate bidirectionally. For E2EE, this is tricky (both docs need the same key, or a key-sharing mechanism), but the concept of shared/embedded content blocks is compelling. 451 + - **Toggles (collapsible sections).** Simple but beloved. A heading or paragraph that can expand/collapse its children. TipTap can support this with a custom node — the data model is a wrapper node with an `open` boolean attribute. 452 + - **Templates with prefilled structure.** Notion's template buttons create new pages with predefined content. Our templates (#56) should include both document templates (meeting notes, project brief, 1-on-1) and sheet templates (budget, inventory, CRM). 453 + - **Breadcrumb navigation.** Notion shows the page hierarchy at the top. Our landing page has breadcrumbs for folders; extending this to show the document path when editing would aid navigation in multi-folder setups. 454 + - **Inline databases.** Embed a mini-spreadsheet/table inside a document. Our planned "embed live Sheet ranges in Docs" (#70) is the right approach — keep sheets as the data layer, docs as the presentation layer. 455 + 456 + **What to skip:** 457 + 458 + - Notion's workspace/team model (too complex for a self-hosted tool). Multi-tenant is a non-goal. 459 + 460 + ### Airtable — The Spreadsheet-Database Hybrid 461 + 462 + **What to steal:** 463 + 464 + - **Field types as first-class citizens.** Airtable columns have explicit types: single select, multi-select, date, checkbox, URL, email, phone, attachment, linked record, lookup, rollup, formula, count, barcode. Our "rich cell types" (#46) should start with the highest-value types: checkbox, single-select (dropdown), date picker, URL (clickable), rating (stars). 465 + - **Views as filtered/sorted/grouped perspectives.** Airtable views don't copy data; they are saved filter+sort+group configurations on the same table. We should implement this — each view stores a filter state, sort keys, grouping column, and hidden columns. Switching views is instant because the data is shared. 466 + - **Grouping.** Collapse rows by a column value (e.g., group tasks by Status). This is a natural extension of our existing sort/filter system. 467 + - **Record detail expansion.** Click a row to see all fields in a modal/sidebar card view. For sheets with many columns, this is much easier to read than horizontal scrolling. Implement as a "Row detail" panel. 468 + - **Automations (triggers + actions).** "When a record matches conditions, do X." For Tools, the E2EE-safe version is client-side automations that run in the browser: "When column Status changes to Done, set column Completed Date to TODAY()." Store automation rules in the Yjs document. 469 + - **Form view.** Airtable forms feed directly into tables. This is exactly our Forms plan (#41) — the form is a view of a Sheet, and responses become rows. 470 + 471 + ### Obsidian — Knowledge Management Stickiness 472 + 473 + **What to steal:** 474 + 475 + - **Backlinks panel.** Show all documents that link to the current document. Our wiki-style links (#72) should include a backlinks sidebar. Since we decrypt document content client-side, building a backlinks index is feasible (scan all doc content for `[[docName]]` patterns). 476 + - **Graph view.** Visualize the relationship between documents as an interactive node graph. Stunning for discovering clusters and orphans. Could use a lightweight library like d3-force or vis.js. The graph data is derived from cross-document links. 477 + - **Daily notes / journal.** Quick-create a document for today's date. A small feature with big daily-driver appeal. One button on the landing page: "Today's Note" that creates (or navigates to) a doc named with today's date. 478 + - **Local-first as a feature.** Obsidian's biggest selling point is that your data is local Markdown files. Tools' E2EE + offline support tells a similar story: "Your data is encrypted at rest, in transit, and even we can't read it." We should lean into this messaging harder. 479 + - **Command palette.** Obsidian's Cmd+P is the gateway to everything. Our Cmd+K command palette (#52) should be as comprehensive: navigate to any document, run any action, search content, switch themes, open settings. 480 + 481 + **What to skip:** 482 + 483 + - Plugin ecosystem. We should make the core great rather than building an extension API prematurely. 484 + 485 + ### Coda — Packs and Cross-Doc 486 + 487 + **What to steal:** 488 + 489 + - **Buttons that trigger actions.** A button in a doc or sheet that runs a formula or automation. "Click to mark all tasks complete," "Click to calculate totals." This is powerful for building interactive dashboards. For Tools, buttons could execute a predefined sequence of cell mutations. 490 + - **Cross-doc references.** Reference data from one doc in another. For Tools with E2EE, this requires shared keys. The simplest version: if you have the key for both documents, you can reference data across them. 491 + - **Canvas columns (rich content in cells).** Coda allows rich text, images, and embeds inside table cells. Our rich cell types (#46) could include a "rich text" type that opens a mini-editor. 492 + 493 + ### Linear — UX Lessons 494 + 495 + **What to steal:** 496 + 497 + - **Speed above all.** Linear feels instant because of aggressive optimistic updates, local-first data, and minimal re-renders. Our sheets grid should aim for <16ms render on cell edit (requestAnimationFrame batching is already in place). 498 + - **Keyboard-first everything.** Linear can be used entirely without a mouse. Every action has a shortcut, and the command palette (Cmd+K) is the universal entry point. Our command palette should support fuzzy matching and show shortcuts inline. 499 + - **Opinionated defaults.** Linear doesn't have 50 settings; it has great defaults. Tools should resist adding settings and instead pick the right defaults (e.g., auto-save always on, dark mode follows OS). 500 + - **Contextual actions.** Linear shows exactly the actions relevant to your current context. Our context menus and toolbar should adapt similarly — show chart options when in a data range, show table tools when in a table. 501 + - **Cycles and project scoping.** For project management use cases in sheets, having a built-in concept of "sprint" or "cycle" (date-bounded grouping) would differentiate from plain Airtable-style tables. 502 + 503 + --- 504 + 505 + ## 8. User Journeys 506 + 507 + ### 8.1 Personal Budget Tracking 508 + 509 + **Scenario:** Scott creates a monthly budget spreadsheet to track income, expenses, and savings goals. 510 + 511 + **What works great today:** 512 + - Create a new sheet, set up columns (Category, Description, Amount, Date, Type) 513 + - Formulas: `=SUM(A2:A50)`, `=SUMIF(E2:E50,"expense",C2:C50)`, `=AVERAGEIF(E2:E50,"expense",C2:C50)` 514 + - Conditional formatting: highlight expenses > $100 in red, savings contributions in green 515 + - Charts: pie chart of spending by category, bar chart of monthly trends 516 + - Dark mode for evening budget reviews 517 + 518 + **Friction / missing:** 519 + - No date picker — dates must be typed manually, error-prone 520 + - No dropdown for Category column (data validation list exists but discoverability is low) 521 + - No currency formatting that auto-applies to new rows 522 + - No "freeze row 1" button in the toolbar (feature exists but UI is buried) 523 + - Cannot duplicate the sheet as a template for next month 524 + - No row grouping to collapse categories 525 + 526 + **Features that would make it smooth:** 527 + - Cell date picker widget (#123) 528 + - Document duplication (#106) 529 + - Column type presets (set "Amount" column to currency format for all new rows) 530 + - Monthly template with pre-built formulas 531 + - Grouping by Category with subtotals 532 + 533 + ### 8.2 Meeting Notes with Action Items 534 + 535 + **Scenario:** A team lead takes notes during a weekly sync, tags action items, and shares with the team. 536 + 537 + **What works great today:** 538 + - Rich text editor with headings, bullet lists, task lists (checkboxes) 539 + - Slash commands to quickly insert headings, task lists, horizontal rules 540 + - Markdown autoformatting: type `- [ ]` for a task item 541 + - Share via link with E2EE key — recipients can view or edit 542 + - Suggesting mode for team members to propose changes 543 + - View-only mode for read-only sharing 544 + 545 + **Friction / missing:** 546 + - No way to assign task items to people (no @mentions or assignee field) 547 + - No due dates on task items 548 + - No way to filter/view only incomplete tasks across multiple meeting docs 549 + - Cannot link to a previous meeting's notes (no cross-document linking) 550 + - Comments exist but are not threaded — no reply chains 551 + 552 + **Features that would make it smooth:** 553 + - @mention support in docs (references a username from the collaboration awareness) 554 + - Task item metadata (assignee, due date) as inline properties 555 + - Cross-document wiki links (#72) to link meeting series 556 + - Threaded comments (#48) 557 + - Template: "Meeting Notes" with pre-built sections (Attendees, Agenda, Notes, Action Items, Next Steps) 558 + 559 + ### 8.3 Project Planning with Timeline 560 + 561 + **Scenario:** A freelancer plans a client project with tasks, dependencies, milestones, and deadlines. 562 + 563 + **What works great today:** 564 + - Sheet with columns: Task, Owner, Start Date, End Date, Status, Priority, Notes 565 + - Data validation: Status dropdown (Not Started, In Progress, Done, Blocked) 566 + - Conditional formatting: overdue tasks highlighted red, completed in green 567 + - Multi-column sort by Priority then Due Date 568 + - Filter to show only "In Progress" tasks 569 + 570 + **Friction / missing:** 571 + - No Gantt/timeline visualization — must mentally map dates to durations 572 + - No dependency tracking between tasks (Task B starts after Task A) 573 + - No progress bar or % complete column type 574 + - Cannot group tasks by project phase 575 + - Status dropdown works but requires manual configuration per sheet 576 + 577 + **Features that would make it smooth:** 578 + - Timeline/Gantt view (#76) computed from Start Date and End Date columns 579 + - Rich cell types: progress bar, checkbox for done/not-done (#46) 580 + - Database views (#45): table view for editing, Gantt for planning, kanban for execution 581 + - Row grouping by Phase column with collapsible sections 582 + 583 + ### 8.4 Collecting RSVPs or Feedback 584 + 585 + **Scenario:** Organizing an event and collecting RSVPs with dietary preferences and plus-one info. 586 + 587 + **What works great today:** 588 + - (Not much — no forms exist yet) 589 + - Could manually share a sheet in edit mode, but that exposes all other responses 590 + 591 + **Friction / missing:** 592 + - No form builder — must share the raw sheet or use an external tool 593 + - No way to restrict a collaborator to append-only (they can edit others' responses) 594 + - No confirmation/thank-you page after submission 595 + 596 + **Features that would make it smooth:** 597 + - Forms (#41): build a form with fields (Name, Email, Attending?, Dietary Needs, Plus One) 598 + - Form responses pipeline (#79): each submission becomes an encrypted row in a Sheet 599 + - Form share link: separate from the sheet link — respondents see only the form, not the data 600 + - View-only summary: share chart of "Attending: Yes/No/Maybe" breakdown 601 + 602 + ### 8.5 Writing a Blog Post or Long Document 603 + 604 + **Scenario:** An author writes a 5,000-word article with headings, images, code blocks, and footnotes. 605 + 606 + **What works great today:** 607 + - TipTap editor with full formatting: headings, blockquotes, code blocks, images, horizontal rules 608 + - Outline sidebar for navigating between sections 609 + - Zen mode for distraction-free writing 610 + - Word count in footer for tracking progress 611 + - Markdown toggle for source editing 612 + - PDF export for sharing final version 613 + - Version history for tracking drafts 614 + 615 + **Friction / missing:** 616 + - No table of contents that can be inserted into the document itself 617 + - No footnotes/endnotes for citations 618 + - No syntax highlighting in code blocks (just monospace) 619 + - Images can only be inserted via URL — no drag-and-drop upload 620 + - No reading time estimate 621 + - No split view to reference source material while writing 622 + - Export to .docx would be useful for publisher submission 623 + 624 + **Features that would make it smooth:** 625 + - Table of contents auto-generation (#104) 626 + - Footnotes and endnotes (#122) 627 + - Syntax-highlighted code blocks (#110) 628 + - E2EE image upload (#80, #81) 629 + - .docx export (#103) 630 + - Split view (#59) 631 + - Reading time estimate (based on word count / 250 wpm) 632 + 633 + ### 8.6 Team Knowledge Base 634 + 635 + **Scenario:** A small team maintains a shared knowledge base: onboarding docs, process guides, decision logs, meeting archives. 636 + 637 + **What works great today:** 638 + - Folders for organizational structure (Onboarding, Processes, Decisions, Meetings) 639 + - Search by document name 640 + - Share links with E2EE for controlled access 641 + - Favorites for frequently referenced docs 642 + - Dark mode for comfortable reading 643 + 644 + **Friction / missing:** 645 + - No cross-document linking (cannot link from "Deployment Process" to "Server Architecture") 646 + - No backlinks (no way to see what other docs reference the current one) 647 + - Search only matches document titles, not content 648 + - No tags or labels on documents 649 + - No "recently viewed" list for quick access 650 + - No way to embed a sheet (e.g., team contact list) in a doc 651 + 652 + **Features that would make it smooth:** 653 + - Wiki-style cross-document links (#72) with `[[Document Name]]` syntax 654 + - Backlinks panel showing all documents that reference the current one 655 + - Full-text content search (#55, #119) 656 + - Document tags/labels for cross-cutting organization 657 + - Recent documents list (#116) 658 + - Embed live sheet ranges in docs (#70) 659 + 660 + ### 8.7 Data Analysis Workflow 661 + 662 + **Scenario:** A data analyst imports a CSV dataset, cleans it, runs calculations, builds charts, and writes up findings. 663 + 664 + **What works great today:** 665 + - Import .xlsx with formulas and formatting preserved 666 + - CSV paste (tab-separated values paste into grid) 667 + - 50+ formula functions for calculations 668 + - Filter and multi-column sort for exploration 669 + - Charts (bar, line, pie, scatter) for visualization 670 + - Named ranges for cleaner formulas 671 + 672 + **Friction / missing:** 673 + - No CSV file import (only .xlsx and manual paste) 674 + - Cannot export results as CSV for use in other tools 675 + - No QUERY() function for SQL-like filtering/aggregation 676 + - Charts cannot be embedded in a doc alongside written analysis 677 + - No pivot table for quick summarization 678 + - No dynamic arrays (FILTER, SORT, UNIQUE) for intermediate transformations 679 + - Formula errors (#VALUE!, #REF!) lack explanatory tooltips 680 + 681 + **Features that would make it smooth:** 682 + - CSV/TSV import and export (#102) 683 + - QUERY() function (#85) and dynamic arrays (#86) 684 + - Pivot tables (#44) 685 + - Embed charts in docs (#71) 686 + - Contextual error tooltips (#96) 687 + - .xlsx export (#109) for sharing results externally 688 + 689 + ### 8.8 Inventory or CRM Tracking 690 + 691 + **Scenario:** A small business tracks inventory items or customer contacts in a sheet used as a lightweight database. 692 + 693 + **What works great today:** 694 + - Sheet columns: Name, Category, Quantity, Price, Last Updated, Notes 695 + - Data validation: Category dropdown, quantity range validation 696 + - Conditional formatting: low stock highlighted yellow, out-of-stock red 697 + - Cell notes for additional context 698 + - Sort and filter by any column 699 + 700 + **Friction / missing:** 701 + - No rich cell types (checkbox for "active", URL for website, email for contact) 702 + - No kanban/gallery view for visual browsing 703 + - No record detail panel (hard to see all fields for wide tables) 704 + - No row insert/delete via UI (must add data at the end) 705 + - No way to attach images to records (product photos, headshots) 706 + - No form for external data entry (suppliers submitting inventory updates) 707 + 708 + **Features that would make it smooth:** 709 + - Rich cell types: checkbox, URL, email, image (#46) 710 + - Database views: table + gallery (#74) + kanban (#73) 711 + - Row insert/delete operations (#113) 712 + - Image cells (#82) with encrypted storage (#80) 713 + - Forms (#41) for external data entry 714 + - Record detail sidebar (click a row to expand) 715 + 716 + ### 8.9 Collaborative Editing of a Proposal 717 + 718 + **Scenario:** Two co-founders collaborate on a client proposal, one writing content and the other reviewing. 719 + 720 + **What works great today:** 721 + - Real-time collaboration with colored cursors and usernames 722 + - Suggesting mode: reviewer's changes shown as tracked insertions/deletions 723 + - Accept/reject individual suggestions or bulk 724 + - Version history to see evolution of the document 725 + - Share link with E2EE — only people with the link can access 726 + - Inline comments for discussion 727 + - Save indicator confirms both collaborators' changes are persisted 728 + 729 + **Friction / missing:** 730 + - Comments are inline marks, not threaded conversations — no reply/resolve flow 731 + - No way to see who wrote what (no author attribution per paragraph) 732 + - No notification when collaborator leaves a comment or suggestion 733 + - Cannot restrict one collaborator to view-only within specific sections 734 + - No way to export the final version as .docx for the client 735 + 736 + **Features that would make it smooth:** 737 + - Threaded comments with resolve (#48) 738 + - Activity log showing who edited when (#124) 739 + - .docx export (#103) 740 + - Granular permissions (#51) — section-level or comment-only access 741 + - Follow mode (#50) to track collaborator's cursor 742 + 743 + ### 8.10 Encrypted Financial Document Sharing 744 + 745 + **Scenario:** An accountant shares tax documents with a client, requiring confidentiality. 746 + 747 + **What works great today:** 748 + - E2EE by default — the server operator cannot read the documents 749 + - Share link includes encryption key in URL fragment (never sent to server) 750 + - View-only mode prevents accidental edits 751 + - Link expiry (1h, 1d, 7d, 30d) for time-limited access 752 + - Offline access works after initial load (no server needed to read cached content) 753 + 754 + **Friction / missing:** 755 + - No way to revoke access after sharing (key is in the URL — anyone who saved it can still decrypt) 756 + - No audit trail of who accessed the document 757 + - No password protection on top of the encryption key (defense-in-depth) 758 + - No way to watermark view-only documents 759 + - Cannot prove to a regulator that the server never had access to content 760 + 761 + **Features that would make it smooth:** 762 + - Key rotation / re-encryption (generate new key, re-encrypt, invalidate old links) 763 + - Access audit log (encrypted, only visible to document owner) 764 + - Optional password layer on share links (key derivation from password + URL key) 765 + - Compliance documentation / zero-knowledge proof architecture doc 766 + - Encrypted document expiry (document self-destructs after expiry, not just the link) 767 + 768 + --- 769 + 770 + ## 9. E2EE as Differentiator — Features Only We Can Build 771 + 772 + The E2EE architecture is not just a security feature — it enables a category of features that cloud-hosted tools fundamentally cannot offer. 773 + 774 + ### 9.1 Zero-Knowledge Compliance 775 + 776 + **Concept:** Tools can provide a technical guarantee that the server operator cannot access document content, even under legal compulsion (subpoena, government request). 777 + 778 + **Implementation:** 779 + - Architecture document proving zero-knowledge properties (server code is open, encryption is client-side, keys never traverse the network) 780 + - Compliance mode that logs server-side access attempts and proves no plaintext was ever stored 781 + - HIPAA, GDPR, SOC 2 alignment documentation ("we literally cannot be a data breach because we never have the data") 782 + 783 + **Why only E2EE can do this:** Google/Notion/Airtable hold your data in plaintext (or with keys they control). They must comply with data requests. Tools physically cannot. 784 + 785 + ### 9.2 Key Rotation and Re-Encryption 786 + 787 + **Concept:** When a team member leaves or a share link is compromised, rotate the document key: generate a new AES key, decrypt with the old key, re-encrypt with the new key, update the stored snapshot, and invalidate all old share links. 788 + 789 + **Implementation:** 790 + - Client-side operation: decrypt snapshot with old key, re-encrypt with new key, PUT new snapshot 791 + - Update localStorage key for the document 792 + - Generate new share URLs with the new key 793 + - Old URLs with the old key will fail to decrypt (graceful "This link is no longer valid" error) 794 + 795 + **Why only E2EE can do this:** In centralized systems, "revoking access" means changing a permission bit. In E2EE, revoking access means making the ciphertext undecryptable — a much stronger guarantee. 796 + 797 + ### 9.3 Encrypted Backup and Portability 798 + 799 + **Concept:** Export your documents as encrypted archives that can be imported into any Tools instance. Your data is portable without ever being decrypted on a server. 800 + 801 + **Implementation:** 802 + - Export: bundle Yjs snapshot + metadata + key into an encrypted .zip 803 + - Import: drag-and-drop the archive, extract, create new document with the data 804 + - Migration: move from one Tools instance to another without the servers exchanging any data 805 + - The archive is useless without the key (which is in the URL / localStorage, not in the archive) 806 + 807 + **Why only E2EE can do this:** With centralized tools, "export" means the server decrypts, formats, and sends you plaintext. With Tools, the export can remain encrypted end-to-end. 808 + 809 + ### 9.4 Offline-First as a Feature (Not a Workaround) 810 + 811 + **Concept:** Position offline support not as "it works when the internet is down" but as "your documents are always local, always yours, and optionally sync to a server." 812 + 813 + **Implementation:** 814 + - IndexedDB stores encrypted Yjs snapshots locally 815 + - Documents load instantly from local cache, then sync changes in the background 816 + - Full editing capability offline — changes queue and merge when reconnecting 817 + - "Airplane mode" indicator that makes offline feel intentional, not broken 818 + 819 + **Why only E2EE makes this better:** When you work offline with Google Docs, your unencrypted content sits in browser storage. With Tools, even the local cache is encrypted — a stolen laptop doesn't expose your documents. 820 + 821 + ### 9.5 Self-Hosting with One Command 822 + 823 + **Concept:** Make self-hosting as easy as `docker run -p 3000:3000 tools`. No database server, no external services, no configuration. Data lives in a single SQLite file that can be backed up with `cp`. 824 + 825 + **Implementation:** 826 + - Publish Docker image to a public registry 827 + - Single container with all dependencies (Node.js + SQLite) 828 + - `DATA_DIR` env var for persistent storage 829 + - Optional HTTPS with auto-generated self-signed cert 830 + - docker-compose.yml with volume mount for one-command deployment 831 + 832 + **Why self-hosting matters for E2EE:** Self-hosting means the user controls the relay server. Even though the server is zero-knowledge, self-hosting eliminates the need to trust anyone. The user is both the operator and the user. 833 + 834 + ### 9.6 Secure Embeds 835 + 836 + **Concept:** Embed a Tools spreadsheet or document in an external webpage (blog, internal wiki, documentation site) while maintaining E2EE. The embed loads in an iframe, the key is passed via postMessage from the parent frame. 837 + 838 + **Implementation:** 839 + - `/embed/sheets/{docId}` route that renders a read-only view 840 + - Parent page passes the encryption key via `postMessage` to the iframe 841 + - Iframe decrypts and renders — the hosting page never has the key in its DOM 842 + - CSP headers to prevent key leakage 843 + 844 + **Why only E2EE can do this:** Regular embeds expose content to the embedding server. Secure embeds keep the content encrypted — only the user's browser has the key. 845 + 846 + ### 9.7 Verifiable Encryption 847 + 848 + **Concept:** Let users verify that their documents are actually encrypted, not just claimed to be. Open the browser dev tools, inspect the WebSocket traffic, and see nothing but encrypted binary blobs. 849 + 850 + **Implementation:** 851 + - "Verify Encryption" panel in settings that shows: the encryption algorithm, the key fingerprint, a live view of encrypted vs decrypted traffic 852 + - Server health endpoint that shows it only has encrypted blobs 853 + - Browser extension or bookmarklet that verifies the encryption is real 854 + 855 + **Why this matters:** Trust-but-verify. Users should not have to take our word for it. The architecture should be auditable by anyone with dev tools open.
+243
README.md
··· 1 + # Tools 2 + 3 + **End-to-end encrypted collaborative office suite.** Documents and spreadsheets with real-time collaboration, where the server never sees your data. 4 + 5 + <!-- screenshot: landing page with document list, dark mode --> 6 + 7 + ## Features 8 + 9 + **Documents** 10 + - Rich text editor (TipTap/ProseMirror) with full formatting, tables, images, task lists 11 + - Notion-style slash commands (`/`) and block handles 12 + - Markdown source toggle, markdown autoformatting, .docx import, PDF export 13 + - Outline sidebar, find & replace, zen mode, suggesting mode (track changes) 14 + - Inline comments, link previews, page breaks, line/paragraph spacing 15 + 16 + **Spreadsheets** 17 + - Custom grid engine with 50+ formula functions (SUM, VLOOKUP, IF, COUNTIF, etc.) 18 + - Charts (bar, line, pie, scatter) via Chart.js 19 + - Conditional formatting, data validation, multi-column filter & sort 20 + - Cross-sheet references, named ranges, formula autocomplete with signatures 21 + - Cell merging, frozen panes, drag-to-fill, paste special, format painter 22 + 23 + **Collaboration** 24 + - Real-time editing via Yjs CRDT with encrypted WebSocket sync 25 + - Colored cursors with usernames (awareness protocol) 26 + - Version history with word count deltas and restore 27 + - Suggesting mode with accept/reject workflow 28 + - Offline support with queued sync on reconnect 29 + 30 + **Security** 31 + - AES-256-GCM encryption via Web Crypto API 32 + - Encryption key lives in the URL fragment (never sent to server) 33 + - Server is a zero-knowledge relay -- it stores and forwards encrypted blobs 34 + - Share links with view-only mode and configurable expiry 35 + 36 + **Organization** 37 + - Landing page with folders, search, sort, favorites, and trash 38 + - Dark mode (auto or manual), responsive mobile layout 39 + - Right-click context menus, keyboard shortcuts, print layout 40 + 41 + <!-- screenshot: docs editor with collaboration cursors --> 42 + <!-- screenshot: sheets editor with chart and conditional formatting --> 43 + 44 + ## Quick Start 45 + 46 + ```bash 47 + git clone <repo-url> tools 48 + cd tools 49 + npm install 50 + npm run dev # Starts Express server + Vite dev server 51 + ``` 52 + 53 + Open `http://localhost:5173` in your browser. 54 + 55 + ### Scripts 56 + 57 + | Command | Description | 58 + |---------|-------------| 59 + | `npm run dev` | Development mode (Express + Vite HMR) | 60 + | `npm run build` | Production build (outputs to `dist/`) | 61 + | `npm start` | Production server (serves built files) | 62 + | `npm run preview` | Build + start in one step | 63 + | `npm test` | Run test suite (Vitest) | 64 + 65 + ## Architecture 66 + 67 + ``` 68 + Browser Server 69 + +----------------------------------+ +-------------------------+ 70 + | Landing Page (vanilla JS) | | Express + compression | 71 + | Docs Editor (TipTap/ProseMirror)| | REST API (CRUD) | 72 + | Sheets Editor (custom grid) | | WebSocket relay | 73 + | | | SQLite (WAL mode) | 74 + | +----------------------------+ | +----+----+----+----------+ 75 + | | Web Crypto API | | | | | 76 + | | AES-256-GCM encrypt/decrypt| | | | | 77 + | +----------------------------+ | | | | 78 + | | | | | 79 + | +----------------------------+ | | | | 80 + | | Yjs CRDT | | encrypted | | | 81 + | | Encrypted WebSocket Provider+--+--bytes--->+ WS relay (room-based) 82 + | | Awareness (cursors/presence)| | | | | 83 + | +----------------------------+ | | | | 84 + | | encrypted| | | 85 + | encrypt(snapshot) +-----------------blobs--->+ SQLite 86 + | decrypt(snapshot) <-----------------blobs----+ | 87 + +----------------------------------+ +-------------------------+ 88 + ``` 89 + 90 + **Key principle:** All content is encrypted in the browser before it touches the network. The server relays encrypted WebSocket messages between peers and stores encrypted snapshots. It never has access to plaintext. 91 + 92 + ### How E2EE Works 93 + 94 + 1. **Document creation:** Browser generates an AES-256-GCM key, creates a document ID on the server, and stores the key in the URL fragment (`#base64key`). 95 + 96 + 2. **Editing:** The TipTap editor (docs) or custom grid (sheets) writes to a Yjs CRDT document. The `EncryptedProvider` intercepts all Yjs sync messages, encrypts them with the document key, and sends encrypted binary over WebSocket. 97 + 98 + 3. **Relay:** The server receives opaque binary blobs and forwards them to all other clients in the same room. It cannot read or modify the content. 99 + 100 + 4. **Persistence:** Periodically, the browser encrypts the full Yjs document state and PUTs it to the server as an encrypted snapshot. New clients load this snapshot, decrypt it locally, and then connect to the WebSocket for live updates. 101 + 102 + 5. **Sharing:** The share link contains the encryption key in the URL fragment. Anyone with the link can decrypt. The `#` fragment is never sent to the server per HTTP spec. 103 + 104 + ### What the Server Sees 105 + 106 + | Data | Visible to Server? | 107 + |------|-------------------| 108 + | Document content | No (encrypted) | 109 + | Document names | No (encrypted) | 110 + | Encryption keys | No (URL fragment) | 111 + | Document IDs | Yes (UUIDs) | 112 + | Created/updated timestamps | Yes | 113 + | Share mode (edit/view) | Yes | 114 + | Link expiry | Yes | 115 + | Connected peer count | Yes | 116 + 117 + ## Self-Hosting 118 + 119 + ### Docker 120 + 121 + ```bash 122 + docker run -d \ 123 + --name tools \ 124 + -p 3000:3000 \ 125 + -v tools-data:/data \ 126 + tools:latest 127 + ``` 128 + 129 + Open `https://localhost:3000` (or behind your reverse proxy). 130 + 131 + ### Docker Compose 132 + 133 + ```yaml 134 + version: '3.8' 135 + services: 136 + tools: 137 + build: . 138 + ports: 139 + - "3000:3000" 140 + volumes: 141 + - tools-data:/data 142 + environment: 143 + - DATA_DIR=/data 144 + - PORT=3000 145 + restart: unless-stopped 146 + 147 + volumes: 148 + tools-data: 149 + ``` 150 + 151 + ### Environment Variables 152 + 153 + | Variable | Default | Description | 154 + |----------|---------|-------------| 155 + | `PORT` | `3000` | HTTP server port | 156 + | `HTTPS_PORT` | `3443` | HTTPS server port (if TLS certs available) | 157 + | `DATA_DIR` | `.` (project root) | Directory for SQLite database and TLS certs | 158 + | `TLS_CERT` | Auto-detected | Path to TLS certificate (PEM) | 159 + | `TLS_KEY` | Auto-detected | Path to TLS private key (PEM) | 160 + 161 + ### HTTPS 162 + 163 + Tools requires a secure context for `crypto.subtle` (the Web Crypto API). Options: 164 + 165 + 1. **localhost** -- works without HTTPS for local development 166 + 2. **Self-signed cert** -- place `cert.pem` and `key.pem` in `DATA_DIR` 167 + 3. **Tailscale certs** -- auto-detected from `/var/lib/tailscale/certs/` 168 + 4. **Reverse proxy** -- terminate TLS at nginx/Caddy/Traefik 169 + 170 + ## Project Structure 171 + 172 + ``` 173 + tools/ 174 + server.js Express + WebSocket relay + SQLite 175 + src/ 176 + index.html Landing page HTML 177 + landing.js Landing page logic (create, list, search, folders, trash) 178 + landing-utils.js Pure functions for landing page (sort, filter, folders, trash) 179 + docs/ 180 + index.html Docs editor HTML 181 + main.js TipTap editor setup, toolbar, collaboration 182 + extensions/ Custom TipTap extensions (font-size, indent, comments, etc.) 183 + *.js Feature modules (outline, zen-mode, link-preview, etc.) 184 + sheets/ 185 + index.html Sheets editor HTML 186 + main.js Grid engine, cell editing, toolbar, collaboration 187 + formulas.js Formula tokenizer, parser, evaluator (50+ functions) 188 + *.js Feature modules (charts, filter, sort, conditional-format, etc.) 189 + lib/ 190 + crypto.js AES-256-GCM encrypt/decrypt via Web Crypto API 191 + provider.js Encrypted Yjs WebSocket provider 192 + version-history.js Version capture and management 193 + offline.js Online/offline detection and change queuing 194 + suggesting.js Track changes (suggesting mode) logic 195 + share-dialog.js Share URL building and dialog helpers 196 + print-layout.js Print HTML generation for docs and sheets 197 + context-menu.js Right-click context menu component 198 + css/ 199 + app.css All styles (OkLCH tokens, dark mode, responsive) 200 + tests/ Vitest test suite (60+ test files) 201 + dist/ Production build output (generated) 202 + ``` 203 + 204 + ## Contributing 205 + 206 + See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, coding conventions, and how to add new features. 207 + 208 + ### Quick Development Guide 209 + 210 + ```bash 211 + # Install dependencies 212 + npm install 213 + 214 + # Start development server (Express + Vite with HMR) 215 + npm run dev 216 + 217 + # Run tests 218 + npm test 219 + 220 + # Build for production 221 + npm run build 222 + ``` 223 + 224 + The development server runs Express on port 3000 (API + WebSocket) and Vite on port 5173 (frontend with HMR). Vite proxies `/api` and `/ws` to Express. 225 + 226 + ## Tech Stack 227 + 228 + | Layer | Technology | Why | 229 + |-------|-----------|-----| 230 + | Rich text editor | TipTap (ProseMirror) | Best-in-class collaborative editing with extension API | 231 + | Spreadsheet grid | Custom `<table>` renderer | Full control, no framework overhead | 232 + | CRDT | Yjs | Mature, conflict-free collaboration without central authority | 233 + | Encryption | Web Crypto API (AES-256-GCM) | Native browser crypto, no JS dependencies | 234 + | Charts | Chart.js | Lightweight, good defaults, canvas rendering | 235 + | Server | Express + ws | Minimal, fast, WebSocket-native | 236 + | Database | SQLite (better-sqlite3) | Zero config, single file, WAL mode | 237 + | Build | Vite | Fast HMR, zero config for vanilla JS | 238 + | Tests | Vitest | Fast, ESM-native, compatible with Vite config | 239 + | Import/Export | mammoth (.docx), SheetJS (.xlsx), html2pdf.js (PDF), Turndown + markdown-it (Markdown) | Focused libraries for each format | 240 + 241 + ## License 242 + 243 + Private. All rights reserved.