feat(journal-template): commit permanent journal/AGENTS.md + skill-link symlinks

+10 -1

.gitignore

··· 19 19 *.port 20 20 /config*.py 21 21 .installed 22 - /journal 22 + /journal/* 23 23 .agents/ 24 24 .claude/ 25 25 .sandbox.pid ··· 35 35 counts_all.txt 36 36 detections_7days.txt 37 37 search_ping_identity.py 38 + !/journal/AGENTS.md 39 + !/journal/CLAUDE.md 40 + !/journal/GEMINI.md 41 + !/journal/.claude/ 42 + !/journal/.claude/skills/ 43 + !/journal/.claude/skills/** 44 + !/journal/.agents/ 45 + !/journal/.agents/skills/ 46 + !/journal/.agents/skills/**

+1

journal/.agents/skills/journal

··· 1 + ../../../talent/journal

+1

journal/.claude/skills/journal

··· 1 + ../../../talent/journal

+5

journal/AGENTS.md

··· 1 + # Sol Journal 2 + 3 + > **First rule for AI agents in a journal**: before doing anything else, run `sol call identity` to hydrate Sol's self, partner, agency, and awareness. The output of that command tells you who you are, who you're working with, and what's currently on your plate. 4 + 5 + For the full journal layout and `sol call journal` CLI reference, read either `./.claude/skills/journal/` or `./.agents/skills/journal/`. Start with `SKILL.md`, then use `references/cli.md`, `references/config.md`, `references/facets.md`, `references/captures.md`, `references/logs.md`, and `references/storage.md`.

+1

journal/CLAUDE.md

··· 1 + AGENTS.md

+1

journal/GEMINI.md

··· 1 + AGENTS.md

+1

tests/fixtures/journal/.agents/skills/journal

··· 1 + ../../../../../talent/journal

+1

tests/fixtures/journal/.claude/skills/journal

··· 1 + ../../../../../talent/journal

+2 -1081

tests/fixtures/journal/AGENTS.md

··· 1 1 # Sol Journal 2 2 3 - > **First rule for AI agents in a journal**: before doing anything else, run `sol call identity` to hydrate Sol's self, partner, agency, and awareness. The output of that command tells you who you are, who you're working with, and what's currently on your plate. Everything below describes the journal's *layout* — the dynamic identity context comes from the CLI. 4 - 5 - # Journal Layout 6 - 7 - This document describes the layout of a **journal** directory where all captures, extracts, and insights are stored. Each dated `YYYYMMDD` folder is referred to as a **day**, and within each day captured content is organized into **segments** (timestamped duration folders). Each segment folder uses the format `HHMMSS_LEN/` where `HHMMSS` is the start time and `LEN` is the duration in seconds. This folder name serves as the **segment key**, uniquely identifying the segment within a given day. 8 - 9 - ## The Three-Layer Architecture 10 - 11 - solstone transforms raw recordings into actionable understanding through a three-layer pipeline: 12 - 13 - ``` 14 - ┌─────────────────────────────────────┐ 15 - │ LAYER 3: AGENT OUTPUTS │ Narrative summaries 16 - │ (Markdown files) │ "What it means" 17 - │ - talents/*.md (daily outputs) │ 18 - │ - *.md (segment outputs) │ 19 - └─────────────────────────────────────┘ 20 - ↑ synthesized from 21 - ┌─────────────────────────────────────┐ 22 - │ LAYER 2: EXTRACTS │ Structured data 23 - │ (JSON/JSONL files) │ "What happened" 24 - │ - audio.jsonl, *_audio.jsonl │ 25 - │ - screen.jsonl, *_screen.jsonl │ 26 - │ - events/*.jsonl (per-facet) │ 27 - └─────────────────────────────────────┘ 28 - ↑ derived from 29 - ┌─────────────────────────────────────┐ 30 - │ LAYER 1: CAPTURES │ Raw recordings 31 - │ (Binary media files) │ "What was recorded" 32 - │ - *.flac, *.ogg, *.opus, *.wav (audio) │ 33 - │ - *.webm (video) │ 34 - └─────────────────────────────────────┘ 35 - ``` 36 - 37 - ### Vocabulary Quick Reference 38 - 39 - **Pipeline Layers** 40 - 41 - | Term | Definition | Examples | 42 - |------|------------|----------| 43 - | **Capture** | Raw audio/video recording | `*.flac`, `*.ogg`, `*.opus`, `*.wav`, `*.webm` | 44 - | **Extract** | Structured data from captures | `*.jsonl` | 45 - | **Agent Output** | AI-generated narrative summary | `talents/*.md`, `HHMMSS_LEN/*.md` | 46 - 47 - **Organization** 48 - 49 - | Term | Definition | Examples | 50 - |------|------------|----------| 51 - | **Day** | 24-hour activity directory | `20250119/` | 52 - | **Segment** | 5-minute time window | `143022_300/` (14:30:22, 5 min) | 53 - | **Span** | Sequential segment group | Import creating 3 segments | 54 - | **Facet** | Project/context scope | `#work`, `#personal` | 55 - 56 - **Extracted Data** 57 - 58 - | Term | Definition | Examples | 59 - |------|------------|----------| 60 - | **Entity** | Tracked person/project/concept | People, companies, tools | 61 - | **Occurrence** | Time-based event | Meetings, messages, files | 62 - 63 - ## Top-Level Directory Structure 64 - 65 - | Directory/File | Purpose | 66 - |----------------|---------| 67 - | `chronicle/` | Container for daily capture folders (`YYYYMMDD/`) containing segments, extracts, and agent outputs | 68 - | `entities/` | Journal-level entity identity records (`<id>/entity.json`) | 69 - | `facets/` | Facet-specific data: entity relationships, todos, events, news, action logs | 70 - | `talents/` | Talent run logs in per-talent subdirectories (`<name>/<id>.jsonl`), day indexes (`<day>.jsonl`), and latest-run symlinks (`<name>.log`) | 71 - | `apps/` | App-specific storage (distinct from codebase `apps/`) | 72 - | `streams/` | Per-stream state files (`<name>.json`) tracking segment chains and sequence numbers | 73 - | `imports/` | Imported audio files and processing artifacts | 74 - | `tokens/` | Token usage logs from AI model calls, organized by day | 75 - | `indexer/` | Search index (`journal.sqlite` FTS5 database) | 76 - | `health/` | Service health logs (`<service>.log` files) | 77 - | `config/` | Configuration files and journal-level action logs | 78 - | `task_log.txt` | Optional log of utility runs in `[epoch]\tmessage` format | 79 - | `summary.md` | Journal-wide statistics summary (generated by `sol journal-stats`) | 80 - | `stats.json` | Detailed journal statistics in JSON format (generated by `sol journal-stats`) | 81 - 82 - ### Config directory 83 - 84 - - `config/journal.json` – owner configuration for the journal (optional, see [Owner configuration](#owner-configuration)). 85 - - `config/convey.json` – Convey UI preferences (facet/app ordering, selected facet). 86 - - `config/actions/` – journal-level action logs (see [Action Logs](#action-logs)). 87 - 88 - ## Owner configuration 89 - 90 - The optional `config/journal.json` file allows customization of journal processing and presentation based on owner preferences. This file should be created at the journal root and contains personal settings that affect how the system processes and interprets journal data. 91 - 92 - ### Identity configuration 93 - 94 - The `identity` block contains information about the journal owner that helps tools correctly identify the owner in transcripts, meetings, and other captured content: 95 - 96 - ```json 97 - { 98 - "identity": { 99 - "name": "Jeremie Miller", 100 - "preferred": "Jer", 101 - "pronouns": { 102 - "subject": "he", 103 - "object": "him", 104 - "possessive": "his", 105 - "reflexive": "himself" 106 - }, 107 - "aliases": ["Jer", "jeremie"], 108 - "email_addresses": ["jer@example.com"], 109 - "timezone": "America/Los_Angeles" 110 - } 111 - } 112 - ``` 113 - 114 - Fields: 115 - - `name` (string) – Full legal or formal name of the journal owner 116 - - `preferred` (string) – Preferred name or nickname to be used when addressing the owner 117 - - `pronouns` (object) – Structured pronoun set for template usage with fields: 118 - - `subject` – Subject pronoun (e.g., "he", "she", "they") 119 - - `object` – Object pronoun (e.g., "him", "her", "them") 120 - - `possessive` – Possessive adjective (e.g., "his", "her", "their") 121 - - `reflexive` – Reflexive pronoun (e.g., "himself", "herself", "themselves") 122 - - `aliases` (array of strings) – Alternative names, nicknames, or usernames that may appear in transcripts 123 - - `email_addresses` (array of strings) – Email addresses associated with the owner for participant detection 124 - - `timezone` (string) – IANA timezone identifier (e.g., "America/New_York", "Europe/London") for timestamp interpretation 125 - 126 - This configuration helps meeting extraction identify the owner as a participant, enables personalized agent interactions, and ensures timestamps are interpreted correctly across the journal. 127 - 128 - ### Convey configuration 129 - 130 - The `convey` block contains settings for the web application: 131 - 132 - ```json 133 - { 134 - "convey": { 135 - "password_hash": "<set via sol password set>" 136 - } 137 - } 138 - ``` 139 - 140 - Fields: 141 - - `password_hash` (string) – Hashed password for accessing the convey web application. Set via `sol password set`. 142 - 143 - **UI Preferences:** The separate `config/convey.json` file stores UI/UX personalization (facet/app ordering, selected facet). All fields optional: 144 - 145 - ```json 146 - { 147 - "facets": {"order": ["work", "personal"], "selected": "work"}, 148 - "apps": {"order": ["home", "calendar", "todos"], "starred": ["home", "todos"]} 149 - } 150 - ``` 151 - 152 - - `facets.order` – Custom facet ordering. `facets.selected` – Currently selected facet (auto-synced with browser). 153 - - `apps.order` – Custom app ordering in menu bar. 154 - - `apps.starred` – Apps to show in the quick-access starred section. 155 - 156 - ### Retention configuration 157 - 158 - The `retention` block controls automatic cleanup of layer 1 raw media (audio recordings, video captures, screen diffs) while preserving all layer 2 extracts and layer 3 agent outputs. Three modes control when raw media is deleted: 159 - 160 - - `"keep"` – retain raw media indefinitely 161 - - `"days"` – delete raw media after `raw_media_days` days, once the segment has finished processing (default: 7 days) 162 - - `"processed"` – delete raw media as soon as the segment has finished processing 163 - 164 - ```json 165 - { 166 - "retention": { 167 - "raw_media": "days", 168 - "raw_media_days": 30, 169 - "per_stream": { 170 - "plaud": { 171 - "raw_media": "days", 172 - "raw_media_days": 7 173 - }, 174 - "archon": { 175 - "raw_media": "processed" 176 - } 177 - } 178 - } 179 - } 180 - ``` 181 - 182 - Fields: 183 - - `raw_media` (string) – Retention mode: `"keep"`, `"days"`, or `"processed"`. Default: `"days"`. 184 - - `raw_media_days` (integer or null) – Number of days to retain raw media when mode is `"days"`. Default: `7`. Required when `raw_media` is `"days"`, ignored otherwise. 185 - - `per_stream` (object) – Per-stream overrides keyed by stream name. Each entry supports `raw_media` and `raw_media_days`. Omitted fields inherit from the global retention settings. 186 - 187 - "Raw media" means layer 1 capture files only: audio files (`.flac`, `.opus`, `.ogg`, `.m4a`, `.wav`), video files (`.webm`, `.mov`, `.mp4`), and screen diffs (`monitor_*_diff.png`). 188 - 189 - All layer 2 and layer 3 content is always preserved regardless of retention policy: transcripts (`audio.jsonl`, `screen.jsonl`), talent outputs (`talents/*.md`), speaker labels (`talents/speaker_labels.json`), facet events (`events/*.jsonl`), entity data, segment metadata (`stream.json`), and search index entries. 190 - 191 - Raw media is never deleted from segments that haven't finished processing. A segment is considered complete only when all four checks pass: 192 - 193 - - No `_active.jsonl` files in `talents/` (no running talents) 194 - - `audio.jsonl` (or `*_audio.jsonl`) exists if audio raw media was captured 195 - - `screen.jsonl` (or `*_screen.jsonl`) exists if video raw media was captured 196 - - `talents/speaker_labels.json` exists if voice embeddings (`.npz`) are present 197 - 198 - Purged segments remain fully navigable in convey. Transcripts, entities, speaker labels, and summaries are all intact. The only difference is that audio/video playback is unavailable. 199 - 200 - ### Environment variables 201 - 202 - The `env` block provides fallback values for environment variables. These are loaded at CLI startup and used when the corresponding variable is not set in the shell or `.env` file: 203 - 204 - ```json 205 - { 206 - "env": { 207 - "GOOGLE_API_KEY": "your-google-api-key", 208 - "ANTHROPIC_API_KEY": "your-anthropic-api-key", 209 - "OPENAI_API_KEY": "your-openai-api-key", 210 - "REVAI_ACCESS_TOKEN": "your-revai-token", 211 - "PLAUD_ACCESS_TOKEN": "your-plaud-token" 212 - } 213 - } 214 - ``` 215 - 216 - **Precedence order** (highest to lowest): 217 - 1. Shell environment variables 218 - 2. `.env` file in project root 219 - 3. Journal config `env` section 220 - 221 - This allows storing API keys in the journal config as an alternative to `.env`, which can be useful when the journal is synced across machines or when you want to keep all configuration in one place. 222 - 223 - #### Template usage examples 224 - 225 - The structured pronoun format enables proper pronoun usage in generated text and agent responses: 226 - 227 - ```python 228 - # In templates or generated text: 229 - f"{identity.pronouns.subject} joined the meeting" # "he joined the meeting" 230 - f"I spoke with {identity.pronouns.object}" # "I spoke with him" 231 - f"That is {identity.pronouns.possessive} desk" # "That is his desk" 232 - f"{identity.pronouns.subject} did it {identity.pronouns.reflexive}" # "he did it himself" 233 - ``` 234 - 235 - For complete documentation of the prompt template system including all variable categories, composition patterns, and how to add new variables, see [PROMPT_TEMPLATES.md](PROMPT_TEMPLATES.md). 236 - 237 - ### Transcribe configuration 238 - 239 - The `transcribe` block configures audio transcription settings for `sol transcribe`: 240 - 241 - ```json 242 - { 243 - "transcribe": { 244 - "backend": "whisper", 245 - "enrich": true, 246 - "preserve_all": false, 247 - "noise_upgrade_min_speech_ratio": 0.3, 248 - "whisper": { 249 - "device": "auto", 250 - "model": "medium.en", 251 - "compute_type": "default" 252 - }, 253 - "revai": { 254 - "model": "fusion" 255 - } 256 - } 257 - } 258 - ``` 259 - 260 - **Top-level fields:** 261 - - `backend` (string) – STT backend to use: `"whisper"` (local processing) or `"revai"` (cloud with speaker diarization). Default: `"whisper"`. 262 - - `enrich` (boolean) – Enable LLM enrichment for topic extraction and transcript correction. Default: `true`. 263 - - `preserve_all` (boolean) – Keep audio files even when no speech is detected. When `false`, silent recordings are deleted to save disk space. Default: `false`. 264 - - `noise_upgrade_min_speech_ratio` (number) – Min speech/loud ratio required for noisy upgrade (default: `0.3`). Filters out music and other non-speech noise. 265 - 266 - **Whisper backend settings** (`transcribe.whisper`): 267 - - `device` (string) – Device for inference: `"auto"` (detect GPU, fall back to CPU), `"cpu"`, or `"cuda"`. Default: `"auto"`. 268 - - `model` (string) – Whisper model to use (e.g., `"tiny.en"`, `"base.en"`, `"small.en"`, `"medium.en"`, `"large-v3-turbo"`, `"distil-large-v3"`). Default: `"medium.en"`. 269 - - `compute_type` (string) – Compute precision: `"default"` (auto-select optimal for platform), `"float32"` (most compatible), `"float16"` (faster on CUDA GPUs), `"int8"` (fastest on CPU). Default: `"default"`. 270 - 271 - **Rev.ai backend settings** (`transcribe.revai`): 272 - - `model` (string) – Rev.ai transcriber model: `"fusion"` (best quality), `"machine"` (fast automated), or `"low_cost"`. Default: `"fusion"`. 273 - 274 - **Platform auto-detection** (Whisper): When `compute_type` is `"default"`, optimal settings are automatically selected: 275 - - **CUDA GPU**: Uses `float16` for GPU-optimized inference 276 - - **CPU (including Apple Silicon)**: Uses `int8` for ~2x faster inference and significantly faster model loading 277 - 278 - Voice embeddings (resemblyzer) also auto-detect the best device: MPS on Apple Silicon (~16x faster), CUDA when available, or CPU fallback. 279 - 280 - CLI flags can override settings: `--backend` selects the backend, `--cpu` forces CPU mode with int8 (Whisper only), `--model MODEL` overrides the Whisper model. 281 - 282 - ### Describe configuration 283 - 284 - The `describe` block configures screen analysis settings for `sol describe`: 285 - 286 - ```json 287 - { 288 - "describe": { 289 - "max_extractions": 20, 290 - "categories": { 291 - "code": { 292 - "importance": "high", 293 - "extraction": "Extract when viewing different repositories or files" 294 - }, 295 - "gaming": { 296 - "importance": "ignore" 297 - } 298 - } 299 - } 300 - } 301 - ``` 302 - 303 - **Fields:** 304 - - `max_extractions` (integer) – Maximum number of frames to run detailed content extraction on per video. The first qualified frame is always extracted regardless of this limit. When more frames are eligible, selection uses AI-based prioritization (falling back to random selection). Default: `20`. 305 - - `categories` (object) – Per-category overrides for importance and extraction guidance. 306 - 307 - #### Category overrides 308 - 309 - Each category (e.g., `code`, `meeting`, `browsing`) can have: 310 - 311 - | Field | Values | Description | 312 - |-------|--------|-------------| 313 - | `importance` | `high`, `normal`, `low`, `ignore` | Advisory priority hint for AI frame selection. `high` prioritizes these frames, `low` deprioritizes unless unique, `ignore` suggests skipping unless categorization seems wrong. Default: `normal`. | 314 - | `extraction` | string | Custom guidance for when to extract content from this category. Overrides the default from the category's `.json` file. | 315 - 316 - Importance levels are advisory hints passed to the AI selection process, not hard filters. The AI may still select frames from `ignore` categories if it determines the content is valuable or the categorization may be incorrect. 317 - 318 - ### Providers configuration 319 - 320 - The `providers` block enables fine-grained control over which LLM provider and model is used for different contexts. This supports a tier-based system where you can specify capability levels (pro/flash/lite) rather than specific model names. 321 - 322 - ```json 323 - { 324 - "providers": { 325 - "default": { 326 - "provider": "google", 327 - "tier": 2 328 - }, 329 - "contexts": { 330 - "observe.*": {"provider": "google", "tier": 3}, 331 - "talent.system.*": {"tier": 1}, 332 - "talent.system.meetings": {"provider": "anthropic", "disabled": true}, 333 - "talent.entities.observer": {"tier": 2, "extract": false} 334 - }, 335 - "models": { 336 - "google": { 337 - "1": "gemini-3-pro-preview", 338 - "2": "gemini-3-flash-preview", 339 - "3": "gemini-2.5-flash-lite" 340 - } 341 - } 342 - } 343 - } 344 - ``` 345 - 346 - #### Tier system 347 - 348 - Tiers provide a provider-agnostic way to specify model capability levels: 349 - 350 - | Tier | Name | Description | 351 - |------|-------|-------------| 352 - | 1 | pro | Highest capability, best for complex reasoning | 353 - | 2 | flash | Balanced performance and cost (default) | 354 - | 3 | lite | Fastest and cheapest, for simple tasks | 355 - 356 - System defaults map tiers to models for each provider. See `think/models.py` for current tier-to-model mappings (`PROVIDER_DEFAULTS` constant). 357 - 358 - If a requested tier is unavailable for a provider, the system falls back to more capable tiers (e.g., tier 3 → tier 2 → tier 1). 359 - 360 - #### Context matching 361 - 362 - Contexts are matched in order of specificity: 363 - 1. **Exact match** – `"talent.system.meetings"` matches only that exact context 364 - 2. **Glob pattern** – `"observe.*"` matches any context starting with `observe.` 365 - 3. **Default** – Falls back to the `default` configuration 366 - 367 - #### Context naming convention 368 - 369 - Talent configs (agents and generators) use the pattern `talent.{source}.{name}`: 370 - - System configs: `talent.system.{name}` (e.g., `talent.system.meetings`, `talent.system.default`) 371 - - App configs: `talent.{app}.{name}` (e.g., `talent.entities.observer`, `talent.support.support`) 372 - 373 - Other contexts follow the pattern `{module}.{feature}[.{operation}]`: 374 - - Observe pipeline: `observe.describe.frame`, `observe.enrich`, `observe.transcribe.gemini` 375 - 376 - #### Configuration options 377 - 378 - **default** – Global defaults applied when no context matches: 379 - - `provider` (string) – Provider name: `"google"`, `"openai"`, or `"anthropic"`. Default: `"google"`. 380 - - `tier` (integer) – Tier number (1-3). Default: `2` (flash). 381 - - `model` (string) – Explicit model name (overrides tier if specified). 382 - 383 - **contexts** – Context-specific overrides. Each key is a context pattern, value is: 384 - - `provider` (string) – Override provider (optional, inherits from default). 385 - - `tier` (integer) – Tier number (optional). 386 - - `model` (string) – Explicit model name (optional, overrides tier). 387 - - `disabled` (boolean) – Disable this talent config (optional, talent contexts only). 388 - - `extract` (boolean) – Enable/disable event extraction for generators with occurrence/anticipation hooks (optional). 389 - 390 - **models** – Per-provider tier overrides. Maps provider name to tier-model mappings: 391 - ```json 392 - { 393 - "google": {"1": "gemini-3-pro-preview", "2": "gemini-3-flash-preview"}, 394 - "openai": {"2": "gpt-5-mini-custom"} 395 - } 396 - ``` 3 + > **First rule for AI agents in a journal**: before doing anything else, run `sol call identity` to hydrate Sol's self, partner, agency, and awareness. The output of that command tells you who you are, who you're working with, and what's currently on your plate. 397 4 398 - Note: Tier keys in JSON must be strings (`"1"`, `"2"`, `"3"`) since JSON doesn't support integer keys. 399 - 400 - ## Facet folders 401 - 402 - The `facets/` directory provides a way to organize journal content by scope or focus area. Each facet represents a cohesive grouping of related activities, projects, or areas of interest. 403 - 404 - ### Facet structure 405 - 406 - Each facet is organized as `facets/<facet>/` where `<facet>` is a descriptive short unique name. When referencing facets in the system, use hashtags (e.g., `#personal` for the "Personal Life" facet, `#ml_research` for "Machine Learning Research"). Each facet folder contains: 407 - 408 - - `facet.json` – metadata file with facet title and description. 409 - - `activities/` – configured activities and completed activity records (see [Activity Records](#activity-records)). 410 - - `entities/` – entity relationships and detected entities (see [Facet Entities](#facet-entities)). 411 - - `todos/` – daily todo lists (see [Facet-Scoped Todos](#facet-scoped-todos)). 412 - - `events/` – extracted events per day (see [Event extracts](#event-extracts)). 413 - - `news/` – daily news and updates relevant to the facet (optional). 414 - - `logs/` – action audit logs for tool calls (optional, see [Action Logs](#action-logs)). 415 - 416 - ### Facet metadata 417 - 418 - The `facet.json` file contains basic information about the facet: 419 - 420 - ```json 421 - { 422 - "title": "Machine Learning Research", 423 - "description": "AI/ML research projects, experiments, and related activities", 424 - "color": "#4f46e5", 425 - "emoji": "🧠" 426 - } 427 - ``` 428 - 429 - Optional fields: 430 - - `color` – hex color code for the facet card background in the web UI 431 - - `emoji` – emoji icon displayed in the top-left of the facet card 432 - - `muted` – boolean flag to mute/hide the facet from views (default: false) 433 - 434 - ### Facet Entities 435 - 436 - Entities in solstone use a two-tier architecture with **journal-level entities** (canonical identity) and **facet relationships** (per-facet context). There are also **detected entities** (daily discoveries) that can be promoted to attached status. 437 - 438 - #### Entity Storage Structure 439 - 440 - ``` 441 - entities/ 442 - └── {entity_id}/ 443 - └── entity.json # Journal-level entity (canonical identity) 444 - 445 - facets/{facet}/ 446 - └── entities/ 447 - ├── YYYYMMDD.jsonl # Daily detected entities 448 - └── {entity_id}/ 449 - ├── entity.json # Facet relationship 450 - ├── observations.jsonl # Durable facts (optional) 451 - └── voiceprints.npz # Voice recognition data (optional) 452 - ``` 453 - 454 - **Journal-level entities** (`entities/<id>/entity.json`) store the canonical identity: name, type, aliases (aka), and principal flag. These are shared across all facets. 455 - 456 - **Facet relationships** (`facets/<facet>/entities/<id>/entity.json`) store per-facet context: description, timestamps, and custom fields specific to that facet. 457 - 458 - **Entity memory** (observations, voiceprints) is stored alongside facet relationships. 459 - 460 - #### Journal-Level Entities 461 - 462 - Journal entities represent the canonical identity record: 463 - 464 - ```json 465 - { 466 - "id": "alice_johnson", 467 - "name": "Alice Johnson", 468 - "type": "Person", 469 - "aka": ["Ali", "AJ"], 470 - "is_principal": false, 471 - "created_at": 1704067200000 472 - } 473 - ``` 474 - 475 - **Standard fields:** 476 - - `id` (string) – Stable slug identifier derived from name via `entity_slug()` in `think/entities/` (lowercase, underscores, e.g., "Alice Johnson" → "alice_johnson"). Used for folder paths, URLs, and tool references. 477 - - `name` (string) – Display name for the entity. 478 - - `type` (string) – Entity type (e.g., "Person", "Company", "Project", "Tool"). Types are flexible and owner-defined; must be alphanumeric with spaces, minimum 3 characters. 479 - - `aka` (array of strings) – Alternative names, nicknames, or acronyms. Used in audio transcription and fuzzy matching. 480 - - `is_principal` (boolean) – When `true`, identifies this entity as the journal owner. Auto-flagged when name/aka matches identity config. 481 - - `blocked` (boolean) – When `true`, entity is hidden from all facets and excluded from agent context. 482 - - `created_at` (integer) – Unix timestamp in milliseconds when entity was created. 483 - 484 - #### Facet Relationships 485 - 486 - Facet relationships link journal entities to specific facets with context: 487 - 488 - ```json 489 - { 490 - "entity_id": "alice_johnson", 491 - "description": "Lead engineer on the API project", 492 - "attached_at": 1704067200000, 493 - "updated_at": 1704153600000, 494 - "last_seen": "20260115" 495 - } 496 - ``` 497 - 498 - **Relationship fields:** 499 - - `entity_id` (string) – Links to the journal entity. 500 - - `description` (string) – Facet-specific description. 501 - - `attached_at` (integer) – Unix timestamp when attached to this facet. 502 - - `updated_at` (integer) – Unix timestamp of last modification. 503 - - `last_seen` (string) – Day (YYYYMMDD) when last mentioned in journal content. 504 - - `detached` (boolean) – When `true`, soft-deleted from this facet but data preserved. 505 - - Custom fields (any) – Additional facet-specific metadata (e.g., `tier`, `status`, `priority`). 506 - 507 - #### Detected Entities 508 - 509 - Daily detection files (`facets/<facet>/entities/YYYYMMDD.jsonl`) contain entities automatically discovered by agents from journal content: 510 - 511 - ```jsonl 512 - {"type": "Person", "name": "Charlie Brown", "description": "Mentioned in standup meeting"} 513 - {"type": "Tool", "name": "React", "description": "Used in UI development work"} 514 - ``` 515 - 516 - #### Entity Lifecycle 517 - 518 - 1. **Detection**: Daily agents scan journal content and record entities in `facets/<facet>/entities/YYYYMMDD.jsonl` 519 - 2. **Aggregation**: Review agent tracks detection frequency across recent days 520 - 3. **Promotion**: Entities with 3+ detections are auto-promoted to attached, or owners manually promote via UI 521 - 4. **Persistence**: Creates journal entity + facet relationship; remains active until detached 522 - 5. **Detachment**: Sets `detached: true` on facet relationship, preserving all data 523 - 6. **Re-attachment**: Clears detached flag, restoring the entity with preserved history 524 - 7. **Blocking**: Sets `blocked: true` on journal entity and detaches from all facets 525 - 526 - #### Cross-Facet Behavior 527 - 528 - The same entity can be attached to multiple facets with independent descriptions and timestamps. When loading entities across all facets, the alphabetically-first facet wins for duplicates during aggregation. 529 - 530 - ### Facet News 531 - 532 - The `news/` directory provides a chronological record of news, updates, and external developments relevant to the facet. This allows tracking of industry news, research updates, regulatory changes, or any external information that impacts the facet's focus area. 533 - 534 - #### News organization 535 - 536 - News files are organized by date as `news/YYYYMMDD.md` where each file contains the day's relevant news items. Only create files for days that have news to record—sparse population is expected. 537 - 538 - #### News file format 539 - 540 - Each `YYYYMMDD.md` file is a markdown document with a consistent structure: 541 - 542 - ```markdown 543 - # 2025-01-18 News - Machine Learning Research 544 - 545 - ## OpenAI Announces New Model Architecture 546 - **Source:** techcrunch.com | **Time:** 09:15 547 - Summary of the announcement and its relevance to current research projects... 548 - 549 - ## Paper: "Efficient Attention Mechanisms in Transformers" 550 - **Source:** arxiv.org | **Time:** 14:30 551 - Key findings from the paper and potential applications... 552 - 553 - ## Google Research Updates Dataset License Terms 554 - **Source:** blog.google | **Time:** 16:45 555 - Changes to dataset licensing that may affect ongoing experiments... 556 - ``` 557 - 558 - #### News entry structure 559 - 560 - Each news entry should include: 561 - - **Title** – concise headline as a level 2 heading 562 - - **Source** – origin of the news (website, journal, etc.) 563 - - **Time** – optional time of publication or discovery (HH:MM format) 564 - - **Summary** – brief description focusing on relevance to the facet 565 - - **Impact** – optional notes on how this affects facet work 566 - 567 - #### News metadata 568 - 569 - Optionally, a `news.json` file can be maintained at the root of the news directory to track metadata: 570 - 571 - ```json 572 - { 573 - "last_updated": "2025-01-18", 574 - "sources": ["arxiv.org", "techcrunch.com", "nature.com"], 575 - "auto_fetch": false, 576 - "keywords": ["transformer", "attention", "llm", "research"] 577 - } 578 - ``` 579 - 580 - This allows for future automation of news gathering while maintaining manual curation quality. 581 - 582 - ### Activity Records 583 - 584 - The `activities/` directory within each facet stores both the configured activity types (`activities.jsonl`) and completed activity records organized by day (`{day}.jsonl`). Activity records represent completed spans of activity — periods where a specific activity type was continuously tracked across one or more recording segments. 585 - 586 - **File path pattern:** 587 - ``` 588 - facets/personal/activities/activities.jsonl # Configured activity types 589 - facets/personal/activities/20260209.jsonl # Completed records for the day 590 - facets/work/activities/20260209.jsonl 591 - facets/work/activities/20260209/coding_095809_303/session_review.md # Generated output 592 - ``` 593 - 594 - Each day file contains one JSON object per line, where each record represents a completed activity span: 595 - 596 - ```jsonl 597 - {"id": "coding_095809_303", "activity": "coding", "segments": ["095809_303", "100313_303", "100816_303", "101320_302"], "level_avg": 0.88, "description": "Developed extraction prompts using Claude Code and VS Code", "active_entities": ["Claude Code", "VS Code", "sunstone"], "created_at": 1770435619415} 598 - {"id": "meeting_090953_303", "activity": "meeting", "segments": ["090953_303", "091457_303", "092001_304", "092506_304", "093010_304"], "level_avg": 1.0, "description": "Sprint planning meeting with the engineering team", "active_entities": ["Alice", "Bob"], "created_at": 1770435619420} 599 - ``` 600 - 601 - #### Record ID scheme 602 - 603 - Activity record IDs follow the format `{activity_type}_{segment_key}` where `segment_key` is the segment in which the activity started. This is unique within a facet+day because only one activity of a given type can start in a given segment for one facet. 604 - 605 - #### Record fields 606 - 607 - - `id` (string) – Unique identifier: `{activity}_{start_segment_key}` (e.g., `coding_095809_303`) 608 - - `activity` (string) – Activity type ID from the facet's configured activities 609 - - `segments` (array of strings) – Ordered list of segment keys where this activity was active 610 - - `level_avg` (float) – Average engagement level across all segments (high=1.0, medium=0.5, low=0.25) 611 - - `description` (string) – AI-synthesized description of the full activity span 612 - - `active_entities` (array of strings) – Merged and deduplicated entity names from all segments 613 - - `created_at` (integer) – Unix timestamp in milliseconds when the record was created 614 - 615 - #### Lifecycle 616 - 617 - Activity records are created by the `activities` segment agent when it detects that an activity has ended: 618 - 619 - 1. The `activity_state` agent tracks per-segment, per-facet activity states with continuity via `since` fields. Each entry includes an `id` field (`{activity}_{since}`) that uniquely identifies the activity span, and `activity.live` events are emitted for active entries. 620 - 2. The `activities` agent runs after `activity_state` and compares previous vs. current segment states 621 - 3. When an activity ends (explicitly, implicitly, or via timeout), the agent walks the segment chain to collect all data 622 - 4. A record is written to the facet's day file with preliminary description 623 - 5. An LLM synthesizes all per-segment descriptions into a unified narrative 624 - 6. The record description is updated with the synthesized version 625 - 626 - **Segment flush:** If no new segments arrive for an extended period (1 hour), the supervisor triggers `sol dream --flush` on the last segment. Agents that declare `hook.flush: true` (like `activities`) run with `flush=True` in their context, treating all remaining active activities as ended. This ensures activities are recorded promptly even when the owner stops working, and prevents cross-day data loss. 627 - 628 - Records are written idempotently — duplicate IDs are skipped on re-runs. 629 - 630 - #### Generated output 631 - 632 - Activity-scheduled agents (`schedule: "activity"`) produce output that is stored alongside the activity records, organized by day and record ID: 633 - 634 - ``` 635 - facets/{facet}/activities/{day}/{activity_id}/{agent}.{ext} 636 - ``` 637 - 638 - For example, a `session_review` agent processing a coding activity would write to: 639 - ``` 640 - facets/work/activities/20260209/coding_095809_303/session_review.md 641 - ``` 642 - 643 - These output directories are only created when activity-scheduled agents run. The path is computed by `get_activity_output_path()` in `think/activities.py` and passed as `output_path` in the agent request. Output files are indexed for search via the `facets/*/activities/*/*/*.md` formatter pattern. 644 - 645 - ## Facet-Scoped Todos 646 - 647 - Todos are organized by facet in `facets/{facet}/todos/{day}.jsonl` where each file stores todo items as JSON Lines. Todos belong to a specific facet (e.g., "personal", "work", "research") and are completely separated by scope. 648 - 649 - **File path pattern:** 650 - ``` 651 - facets/personal/todos/20250110.jsonl 652 - facets/work/todos/20250110.jsonl 653 - facets/research/todos/20250112.jsonl 654 - ``` 655 - 656 - Each file contains one JSON object per line, with the line number (1-indexed) serving as the stable todo ID. 657 - 658 - ```jsonl 659 - {"text": "Draft standup update"} 660 - {"text": "Review PR #1234 for indexing tweaks", "time": "14:30"} 661 - {"text": "Morning planning session notes", "completed": true} 662 - {"text": "Cancel meeting with vendor", "cancelled": true} 663 - ``` 664 - 665 - ### Format Specification 666 - 667 - **JSONL structure:** 668 - 669 - Each line is a JSON object with the following fields: 670 - - `text` (required) – Task description 671 - - `time` (optional) – Scheduled time in `HH:MM` format (e.g., `"14:30"`) 672 - - `completed` (optional) – Set to `true` when task is done 673 - - `cancelled` (optional) – Set to `true` for soft-deleted tasks 674 - - `created_at` (optional) – Unix timestamp in milliseconds when todo was created 675 - - `updated_at` (optional) – Unix timestamp in milliseconds of last modification 676 - 677 - **Facet context:** 678 - - Facet is determined by the file location, not inline tags 679 - - Each facet has its own independent todo list for each day 680 - - Work todos (`facets/work/todos/`) are completely separate from personal todos (`facets/personal/todos/`) 681 - 682 - **Rules:** 683 - - Line number is the stable todo ID (1-indexed); todos are never removed, only cancelled 684 - - Append new todos at the end of the file to maintain stable line numbering 685 - - Mark completed items with `"completed": true` 686 - - Cancel items with `"cancelled": true` (soft delete preserves line numbers) 687 - 688 - **Tool Access:** 689 - All todo operations require both `day` and `facet` parameters: 690 - - `todo_list(day, facet)` – view numbered checklist for a specific facet 691 - - `todo_add(day, facet, text)` – append new todo 692 - - `todo_done(day, facet, line_number)` – mark complete 693 - - `todo_cancel(day, facet, line_number)` – cancel entry (soft delete) 694 - - `todo_upcoming(limit, facet=None)` – view upcoming todos (optionally filtered by facet) 695 - 696 - This facet-scoped structure provides true separation of concerns while enabling automated tools to manage tasks deterministically. 697 - 698 - ## Action Logs 699 - 700 - Action logs record an audit trail of owner-initiated actions and agent tool calls. There are two types: 701 - 702 - - **Journal-level logs** (`config/actions/`) – actions not tied to a specific facet (settings changes, observer management) 703 - - **Facet-scoped logs** (`facets/{facet}/logs/`) – actions within a specific facet (todos, entities) 704 - 705 - ### Journal Action Logs 706 - 707 - The `config/actions/` directory records journal-level actions. Logs are organized by day as `config/actions/YYYYMMDD.jsonl`. 708 - 709 - ```json 710 - { 711 - "timestamp": "2025-12-16T07:33:05.135587+00:00", 712 - "source": "app", 713 - "actor": "settings", 714 - "action": "identity_update", 715 - "params": { 716 - "changed_fields": {"name": {"old": "John", "new": "John Doe"}} 717 - } 718 - } 719 - ``` 720 - 721 - ### Facet Action Logs 722 - 723 - The `logs/` directory within each facet records facet-scoped actions. Logs are organized by day as `facets/{facet}/logs/YYYYMMDD.jsonl`. 724 - 725 - ```json 726 - { 727 - "timestamp": "2025-12-16T07:33:05.135587+00:00", 728 - "source": "tool", 729 - "actor": "todos:todo", 730 - "action": "todo_add", 731 - "params": { 732 - "text": "Review project proposal" 733 - }, 734 - "facet": "work", 735 - "use_id": "1765870373972" 736 - } 737 - ``` 738 - 739 - ### Log Entry Fields 740 - 741 - Both log types share the same structure: 742 - 743 - - `timestamp` – ISO 8601 timestamp of the action 744 - - `source` – Origin type: "app" for web UI, "tool" for agent tools 745 - - `actor` – App or tool name that performed the action 746 - - `action` – Action name (e.g., "todo_add", "identity_update") 747 - - `params` – Action-specific parameters 748 - - `facet` – Facet name (only present in facet-scoped logs) 749 - - `use_id` – Agent ID (only present for agent tool actions) 750 - 751 - These logs enable auditing, debugging, and potential rollback of automated actions. 752 - 753 - ## Token Usage 754 - 755 - The `tokens/` directory tracks token usage from all AI model calls across the system. Usage data is organized by day as `tokens/YYYYMMDD.jsonl` where each file contains JSON Lines entries for that day's API calls. 756 - 757 - ### Token log format 758 - 759 - Each line in a token log file is a JSON object with the following structure: 760 - 761 - ```json 762 - { 763 - "timestamp": 1736812345000, 764 - "model": "gemini-2.5-flash", 765 - "context": "agent.default.20250113_143022", 766 - "segment": "143022_300", 767 - "usage": { 768 - "input_tokens": 1500, 769 - "output_tokens": 500, 770 - "total_tokens": 2000, 771 - "cached_tokens": 800, 772 - "reasoning_tokens": 200 773 - } 774 - } 775 - ``` 776 - 777 - Required fields: 778 - - `timestamp` – Unix timestamp in milliseconds (13 digits) 779 - - `model` – Model identifier (e.g., "gemini-2.5-flash", "gpt-5", "claude-sonnet-4-5") 780 - - `context` – Calling context (e.g., "agent.name.use_id" or "module.function:line") 781 - - `usage` – Token counts dictionary with normalized field names 782 - 783 - Optional fields: 784 - - `segment` – Recording segment key (e.g., "143022_300") when token usage is attributable to a specific observation window 785 - 786 - Usage fields (all optional depending on model capabilities): 787 - - `input_tokens` – Tokens in the prompt/input 788 - - `output_tokens` – Tokens in the response/output 789 - - `total_tokens` – Total tokens consumed 790 - - `cached_tokens` – Tokens served from cache (reduces cost) 791 - - `reasoning_tokens` – Tokens used for extended thinking/reasoning 792 - - `requests` – Number of API requests made (for batch operations) 793 - 794 - The logging system normalizes provider-specific formats (OpenAI, Gemini, Anthropic) into this unified schema for consistent cost tracking and analysis across all models. 795 - 796 - ## Agent Event Logs 797 - 798 - The `talents/` directory stores event logs for all AI talent sessions managed by Cortex. Each talent session produces a JSONL file containing the complete event history. 799 - 800 - **Directory layout:** 801 - - `<name>/` – per-agent subdirectory (e.g., `default/`, `entities--observer/`) 802 - - `<name>/<use_id>_active.jsonl` – currently running agent (renamed when complete) 803 - - `<name>/<use_id>.jsonl` – completed agent session 804 - - `<name>.log` – symlink to the latest completed run for each agent name 805 - - `<day>.jsonl` – day index with one summary line per agent that completed on that day 806 - 807 - The `use_id` is a Unix timestamp in milliseconds that uniquely identifies the session. 808 - 809 - **Event format (JSONL):** 810 - 811 - Each line is a JSON object with an `event` field indicating the event type: 812 - 813 - ```jsonl 814 - {"event": "start", "ts": 1755450767962, "name": "helper", "prompt": "Help me with...", "facet": "work"} 815 - {"event": "text", "ts": 1755450768000, "content": "I'll help you with that."} 816 - {"event": "tool_call", "ts": 1755450769000, "tool": "search", "params": {"query": "example"}} 817 - {"event": "tool_result", "ts": 1755450770000, "tool": "search", "result": "..."} 818 - {"event": "finish", "ts": 1755450771000, "result": "Here's what I found..."} 819 - ``` 820 - 821 - **Common event types:** 822 - - `start` – agent session started, includes name, prompt, and facet 823 - - `text` – streaming text output from the agent 824 - - `tool_call` – agent invoked a tool 825 - - `tool_result` – result returned from tool execution 826 - - `error` – error occurred during execution 827 - - `finish` – agent session completed, includes final result 828 - 829 - See [CORTEX.md](CORTEX.md) for agent architecture and spawning details. 830 - 831 - ## App Storage 832 - 833 - The `apps/` directory provides storage space for Convey apps to persist configuration, data, and artifacts specific to this journal. Each app has its own directory at `apps/<app_name>/` where it can maintain app-specific state independent of the application codebase. 834 - 835 - Apps typically use `config.json` for journal-specific settings and create subdirectories for data storage (e.g., `cache/`, `data/`, `logs/`). This is distinct from the app metadata file (`apps/<app>/app.json` in the codebase) which defines icon, label, and facet support across all journals. See [APPS.md](APPS.md) for storage utilities (`get_app_storage_path`, `load_app_config`, `save_app_config`). 836 - 837 - ## Search Index 838 - 839 - The `indexer/` directory contains the full-text search index built from journal content. 840 - 841 - **Files:** 842 - - `indexer/journal.sqlite` – FTS5 SQLite database containing indexed chunks from agent outputs, events, entities, todos, and action logs 843 - 844 - The indexer converts content to markdown chunks via the formatters framework, then indexes with metadata fields (day, facet, agent) for filtering. Raw audio/screen transcripts are formattable but not indexed — agent outputs provide more useful search results. Use `get_journal_index()` from `think/indexer/journal.py` to access the database programmatically. 845 - 846 - Which content gets indexed is controlled by the `FORMATTERS` registry in `think/formatters.py`. Each entry maps a glob pattern to a formatter function and an `indexed` flag. The registry patterns must be specific enough to use as `Path.glob()` arguments from the journal root — adding a new content location requires a new entry. 847 - 848 - Run `sol indexer` to rebuild the index from current journal content. 849 - 850 - ## Service Health 851 - 852 - The `health/` directory contains log files for long-running services. 853 - 854 - **Files:** 855 - - `health/<service>.log` – log output for each service (e.g., `observe.log`, `cortex.log`, `convey.log`) 856 - - `health/retention.log` – JSONL log of retention purge operations with timestamps, files deleted, bytes freed, and per-segment details 857 - 858 - These logs are useful for debugging service issues. See [DOCTOR.md](DOCTOR.md) for diagnostics and troubleshooting guidance. 859 - 860 - ## Imported Audio 861 - 862 - The `imports/` directory stores audio files imported via the import app, along with their processing artifacts. Each import is organized by detected timestamp: 863 - 864 - ``` 865 - imports/ 866 - └── YYYYMMDD_HHMMSS/ # Import directory (detected or owner-specified timestamp) 867 - ├── import.json # Import metadata and processing status 868 - ├── {original_filename} # Original uploaded audio file 869 - ├── imported.json # Processed transcript in standard format 870 - └── segments.json # List of segment keys created for this import 871 - ``` 872 - 873 - ### Import metadata 874 - 875 - The `import.json` file tracks the import process: 876 - 877 - ```json 878 - { 879 - "original_filename": "meeting_recording.m4a", 880 - "upload_timestamp": 1755034698276, 881 - "upload_datetime": "2025-08-12T15:38:18.276000", 882 - "detection_result": { 883 - "day": "20250630", 884 - "time": "143256", 885 - "confidence": "high", 886 - "source": "Date/Time Original" 887 - }, 888 - "detected_timestamp": "20250630_143256", 889 - "user_timestamp": "20250630_143256", 890 - "file_size": 13950943, 891 - "mime_type": "audio/x-m4a", 892 - "facet": "work", 893 - "processing_completed": "2025-08-12T15:41:42.970189" 894 - } 895 - ``` 896 - 897 - Once processed, imports are linked into the appropriate day's segment via `imported_audio.jsonl` files that reference the original import location. 898 - 899 - ## Day folder contents 900 - 901 - Within each day, captured content is organized into **segments** (timestamped duration folders). The folder name is the **segment key**, which uniquely identifies the segment within the day and follows this format: 902 - 903 - - `HHMMSS_LEN/` – Start time and duration in seconds (e.g., `143022_300/` for a 5-minute segment starting at 14:30:22) 904 - 905 - Each segment progresses through the three-layer pipeline: captures are recorded, extracts are generated, and agent outputs are synthesized. 906 - 907 - #### Stream identity 908 - 909 - Every segment belongs to a **stream** — a named series of segments from a single source. Streams provide navigable chains linking each segment to its predecessor. 910 - 911 - - `stream.json` – Per-segment stream marker containing: 912 - - `stream` – stream name (e.g., `"archon"`, `"import.apple"`) 913 - - `prev_day` – day of the previous segment in this stream (null for first) 914 - - `prev_segment` – segment key of the predecessor (null for first) 915 - - `seq` – sequence number within the stream 916 - 917 - Stream names follow the convention: `{hostname}` for local observers, `{observer_name}` for observers, `import.{type}` for imports (e.g., `import.apple`, `import.text`). Global stream state is tracked in the top-level `streams/` directory as `{name}.json` files. 918 - 919 - Pre-stream segments (created before stream identity was added) have no `stream.json` and are handled gracefully as `None` throughout the pipeline. 920 - 921 - ### Layer 1: Captures 922 - 923 - Captures are the original binary media files recorded by observation tools. 924 - 925 - #### Audio captures 926 - 927 - Audio files are initially written to the day root with the segment key prefix (Linux) or directly to segment folders (macOS): 928 - 929 - - **Linux**: `HHMMSS_LEN_*.flac` – audio files in day root (e.g., `143022_300_audio.flac`) 930 - - **macOS**: `HHMMSS_LEN/audio.m4a` – audio files written directly to segment folder 931 - 932 - After transcription, audio files are moved into their segment folder: 933 - 934 - - `HHMMSS_LEN/*.flac`, `*.m4a`, `*.ogg`, `*.opus`, or `*.wav` – audio files moved here after processing, preserving descriptive suffix (e.g., `audio.flac`, `audio.m4a`, `imported_audio.opus`) 935 - 936 - Note: The descriptive portion after the segment key (e.g., `_audio`, `_recording`) is preserved when files are moved into segment directories. Processing tools match files by extension only, ignoring the descriptive suffix. 937 - 938 - #### Screen captures 939 - 940 - Screen recordings use per-monitor files with position and connector/displayID in the filename: 941 - 942 - - **Linux**: `HHMMSS_LEN_<position>_<connector>_screen.webm` – screencast video files in day root (e.g., `143022_300_center_DP-3_screen.webm`) 943 - - **macOS**: `HHMMSS_LEN/<position>_<displayID>_screen.mov` – video files written directly to segment folder (e.g., `center_1_screen.mov`) 944 - 945 - After analysis, files are in their segment folder: 946 - 947 - - `HHMMSS_LEN/<position>_<connector>_screen.webm` or `*.mov` – video files (e.g., `center_DP-3_screen.webm`, `center_1_screen.mov`) 948 - 949 - For multi-monitor setups, each monitor produces a separate file. Position labels include: `center`, `left`, `right`, `top`, `bottom`, and combinations like `left-top`. 950 - 951 - ### Layer 2: Extracts 952 - 953 - Extracts are structured data files (JSON/JSONL) derived from captures through AI analysis. 954 - 955 - #### Audio transcript extracts 956 - 957 - The transcript file (`audio.jsonl`) contains a metadata line followed by one JSON object per transcript segment. 958 - 959 - Example transcript file: 960 - 961 - ```jsonl 962 - {"raw": "audio.flac"} 963 - {"start": "00:00:01", "source": "mic", "text": "So we need to finalize the authentication module today."} 964 - {"start": "00:00:15", "source": "sys", "text": "I agree. Let's make sure we have proper unit tests."} 965 - ``` 966 - 967 - **Metadata line (first line):** 968 - - `raw` – path to processed audio file (required) 969 - - `backend` – STT backend used (e.g., "whisper", "revai") 970 - - `model` – model used for transcription (e.g., "medium.en", "revai-fusion") 971 - - `device` – device used for inference (e.g., "cuda", "cpu", "cloud") 972 - - `compute_type` – compute precision used (e.g., "float16", "int8", "api") 973 - - `observer` – observer name if transcribed from an observer source (optional) 974 - - `imported` – object with import metadata for external files (optional): 975 - - `id` – unique import identifier 976 - - `facet` – facet name for entity extraction 977 - - `setting` – contextual setting description 978 - 979 - **Transcript statements (subsequent lines):** 980 - - `start` – timestamp in HH:MM:SS format (required) 981 - - `text` – transcribed text (required) 982 - - `source` – audio source: "mic" or "sys" (optional) 983 - - `speaker` – speaker identifier, numeric or string (optional, not currently populated) 984 - - `corrected` – LLM-corrected version of text (optional, added during enrichment) 985 - - `description` – tone or delivery description, e.g., "enthusiastic", "questioning" (optional, added during enrichment) 986 - 987 - #### Screen frame extracts 988 - 989 - Screen analysis files use per-monitor naming: `<position>_<connector>_screen.jsonl` (e.g., `center_DP-3_screen.jsonl`, `left_HDMI-1_screen.jsonl`). For single-monitor setups, the file is simply `screen.jsonl`. Each file contains one JSON object per qualified frame. Frames qualify when they show significant visual change (≥5% RMS difference) compared to the previous qualified frame. 990 - 991 - Example frame record: 992 - 993 - ```json 994 - { 995 - "frame_id": 123, 996 - "timestamp": 45.67, 997 - "requests": [ 998 - {"type": "describe", "model": "gemini-2.5-flash-lite", "duration": 0.5}, 999 - {"type": "category", "category": "reading", "model": "gemini-3-flash", "duration": 1.2} 1000 - ], 1001 - "analysis": { 1002 - "visual_description": "Documentation page showing API reference.", 1003 - "primary": "reading", 1004 - "secondary": "none", 1005 - "overlap": true 1006 - }, 1007 - "content": { 1008 - "reading": "# API Reference\n\n## Authentication\n\nUse Bearer tokens..." 1009 - } 1010 - } 1011 - ``` 1012 - 1013 - **Common fields:** 1014 - - `frame_id` – sequential frame number in the video 1015 - - `timestamp` – time in seconds from video start 1016 - - `requests` – list of vision API requests made for this frame (type: "describe" for initial, "category" for follow-ups) 1017 - - `analysis` – categorization result with `primary`, `secondary`, `overlap`, and `visual_description` 1018 - - `content` – object containing category-specific extracted content (see below) 1019 - - `error` – present when processing failed after retries 1020 - 1021 - **Category-specific content (inside `content` object):** 1022 - - `messaging` – markdown content when frame contains chat/email apps 1023 - - `browsing` – markdown content when frame contains web browsing 1024 - - `reading` – markdown content when frame contains documents/articles 1025 - - `productivity` – markdown content when frame contains spreadsheets/slides/calendars 1026 - - `meeting` – JSON object when frame contains video conferencing, includes participant detection and bounding boxes 1027 - 1028 - The vision analysis uses multi-stage conditional processing: 1029 - 1. Initial categorization determines content type (e.g., `code`, `meeting`, `browsing`, `reading`). See `observe/categories/` for the full list of categories. 1030 - 2. Category-specific follow-up prompts are discovered from `observe/categories/*.md` files 1031 - 3. Follow-ups are triggered for categories that have extraction content in their `.md` file (currently: messaging, browsing, reading, productivity output markdown; meeting outputs JSON) 1032 - 1033 - #### Event extracts 1034 - 1035 - Generator output processing extracts time-based events from the day's transcripts—meetings, messages, follow-ups, file activity and more. Events are stored per-facet in JSONL files at `facets/{facet}/events/{day}.jsonl`. 1036 - 1037 - There are two types of events: 1038 - - **Occurrences** – events that happened on the capture day (`occurred: true`) 1039 - - **Anticipations** – future scheduled events extracted from calendar views (`occurred: false`) 1040 - 1041 - ```jsonl 1042 - {"type": "meeting", "start": "09:00:00", "end": "09:30:00", "title": "Team stand-up", "summary": "Status update with the engineering team", "work": true, "participants": ["Jeremie Miller", "Alice", "Bob"], "facet": "work", "agent": "meetings", "occurred": true, "source": "20250101/talents/meetings.md", "details": "Sprint planning discussion"} 1043 - {"type": "deadline", "date": "2025-01-15", "start": null, "end": null, "title": "Project milestone", "summary": "Q1 deliverable due", "work": true, "participants": [], "facet": "work", "agent": "schedule", "occurred": false, "source": "20250101/talents/schedule.md", "details": "Final review before release"} 1044 - ``` 1045 - 1046 - **Common fields:** 1047 - - **type** – event kind: `meeting`, `message`, `file`, `followup`, `documentation`, `research`, `media`, `deadline`, `appointment`, etc. 1048 - - **start** and **end** – HH:MM:SS timestamps (or `null` for anticipations without specific times) 1049 - - **date** – ISO date YYYY-MM-DD (anticipations only, indicates scheduled date) 1050 - - **title** and **summary** – short text for display and search 1051 - - **facet** – facet name the event belongs to (required) 1052 - - **agent** – source generator type (e.g., "meetings", "schedule", "flow") 1053 - - **occurred** – `true` for occurrences, `false` for anticipations 1054 - - **source** – path to the output file that generated this event 1055 - - **work** – boolean, work vs. personal classification 1056 - - **participants** – optional list of people or entities involved 1057 - - **details** – free-form string with additional context 1058 - 1059 - This structure allows the indexer to collect and search events across all facets and days. 1060 - 1061 - ### Layer 3: Agent Outputs 1062 - 1063 - Agent outputs are AI-generated markdown files that provide human-readable narratives synthesized from captures and extracts. 1064 - 1065 - #### Segment outputs 1066 - 1067 - After captures are processed, segment-level outputs are generated within each segment folder as `HHMMSS_LEN/*.md` files. Available segment output types are defined by templates in `talent/` with `"schedule": "segment"` in their metadata JSON. 1068 - 1069 - #### Daily outputs 1070 - 1071 - Post-processing generates day-level outputs in the `talents/` directory that synthesize all segments. 1072 - 1073 - **Generator discovery:** Available generator types are discovered at runtime from: 1074 - - `talent/*.md` – system generator templates (files with `schedule` field but no `tools` field) 1075 - - `apps/{app}/talent/*.md` – app-specific generator templates 1076 - 1077 - Each template is a `.md` file with JSON frontmatter containing metadata (title, description, schedule, output format). The `schedule` field is required and must be `"segment"` or `"daily"` - generators with missing or invalid schedule are skipped. Use `get_talent_configs(has_tools=False)` from `think/talent.py` to retrieve all available generators, or `get_talent_configs(has_tools=False, schedule="daily")` to get generators filtered by schedule. 1078 - 1079 - **Output naming:** 1080 - - System outputs: `talents/{agent}.md` (e.g., `talents/flow.md`, `talents/meetings.md`) 1081 - - App outputs: `talents/_{app}_{agent}.md` (e.g., `talents/_entities_observer.md`) 1082 - - JSON output: `talents/{agent}.json` when metadata specifies `"output": "json"` 1083 - 1084 - Each generator type has a corresponding template file (`{name}.md`) that defines how the AI synthesizes extracts into narrative form. 5 + For the full journal layout and `sol call journal` CLI reference, read either `./.claude/skills/journal/` or `./.agents/skills/journal/`. Start with `SKILL.md`, then use `references/cli.md`, `references/config.md`, `references/facets.md`, `references/captures.md`, `references/logs.md`, and `references/storage.md`.

Configure Feed

Configure Feed