personal memory agent
0
fork

Configure Feed

Select the types of activity you want to include in your feed.

remove built-in macOS observer (migrated to solstone-macos native app)

macOS screen/audio capture has moved to the solstone-macos native
companion app which uploads via the remote ingest HTTP API. This
removes the dead Python-based macOS observer subsystem:

- Delete observe/macos/ directory (5 files)
- Delete tests/test_macos_screencapture.py
- Remove observe-macos from CLI registry (sol.py)
- Remove darwin branch from observe/observer.py platform dispatcher
- Remove PyObjC dependencies from pyproject.toml and regenerate uv.lock
- Update docs (OBSERVE.md, CALLOSUM.md, INSTALL.md, README.md, DOCTOR.md)

+16 -1596
+1 -1
README.md
··· 59 59 +-------------+ 60 60 ``` 61 61 62 - - **observe** — captures audio (PipeWire on Linux, sck-cli on macOS) and screen activity. produces FLAC audio, WebM screen recordings, and timestamped metadata. 62 + - **observe** — captures audio (PipeWire on Linux, solstone-macos native app on macOS) and screen activity. produces FLAC audio, WebM screen recordings, and timestamped metadata. 63 63 - **think** — transcribes audio (faster-whisper), analyzes screen captures, extracts entities, detects meetings, and indexes everything into SQLite. runs 30 configurable agent/generator templates from `muse/`. 64 64 - **cortex** — orchestrates agent execution. receives events, dispatches agents, writes results back to the journal. 65 65 - **callosum** — async message bus connecting all services. enables event-driven coordination between observe, think, cortex, and convey.
+1 -1
docs/CALLOSUM.md
··· 70 70 71 71 ### `observe` - Multimodal capture and processing 72 72 **Sources:** 73 - - Capture: `observe/observer.py` → platform-specific (`observe/linux/observer.py`, `observe/macos/observer.py`) 73 + - Capture: `observe/observer.py` → `observe/linux/observer.py` (macOS capture comes from remote/native observers) 74 74 - Processing: `observe/sense.py`, `observe/describe.py`, `observe/transcribe/` 75 75 76 76 **Events:**
+2 -2
docs/DOCTOR.md
··· 200 200 201 201 **Symptoms:** `sol describe` fails with `av.error.InvalidDataError: Invalid data found when processing input`. Sense logs show `describe failed ... exit code 1` and `Segment observed with errors ... ['describe exit 1']`. 202 202 203 - **Diagnosis:** The `.mov` file has `ftyp` + `wide` + `mdat` atoms but is missing the `moov` atom. The `mdat` size is 0 (extends-to-EOF). This means the screen recorder (sck-cli) never finalized the file — it wrote video frames but crashed or was interrupted before writing the metadata index. 203 + **Diagnosis:** The `.mov` file has `ftyp` + `wide` + `mdat` atoms but is missing the `moov` atom. The `mdat` size is 0 (extends-to-EOF). This means the screen recorder (solstone-macos native app) never finalized the file — it wrote video frames but crashed or was interrupted before writing the metadata index. 204 204 205 - Known trigger: screen sharing active during sck-cli capture causes AVAssetWriter finalization to be skipped (missing `endSession()` call in `VideoWriter.swift`). 205 + Known trigger: screen sharing active during solstone-macos native app capture causes AVAssetWriter finalization to be skipped (missing `endSession()` call in `VideoWriter.swift`). 206 206 207 207 ```bash 208 208 # Confirm the issue — should report "moov atom not found"
-9
docs/INSTALL.md
··· 37 37 brew install python git ffmpeg uv 38 38 ``` 39 39 40 - **Screen/audio capture** requires [sck-cli](https://github.com/quartzjer/sck-cli): 41 - 42 - ```bash 43 - git clone https://github.com/quartzjer/sck-cli.git 44 - cd sck-cli 45 - make 46 - sudo make install 47 - ``` 48 - 49 40 --- 50 41 51 42 ## Installation
+3 -2
docs/OBSERVE.md
··· 2 2 3 3 Multimodal capture and AI-powered analysis of desktop activity. 4 4 5 + On macOS, capture is handled by the `solstone-macos` native companion app rather than a built-in Python observer. 6 + 5 7 ## Commands 6 8 7 9 | Command | Purpose | 8 10 |---------|---------| 9 11 | `sol observer` | Screen and audio capture (auto-detects platform) | 10 12 | `sol observe-linux` | Screen and audio capture on Linux (direct) | 11 - | `sol observe-macos` | Screen and audio capture on macOS (direct) | 12 13 | `sol transcribe` | Audio transcription with faster-whisper | 13 14 | `sol describe` | Visual analysis of screen recordings | 14 15 | `sol sense` | Unified observation coordination | ··· 56 57 ## Key Components 57 58 58 59 - **observer.py** - Unified entry point with platform detection 59 - - **linux/observer.py**, **macos/observer.py** - Platform-specific capture using native APIs 60 + - **linux/observer.py** - Linux capture using native APIs 60 61 - **linux/screencast.py** - XDG Portal screencast with PipeWire + GStreamer 61 62 - **gnome/activity.py** - GNOME-specific activity detection (idle, lock, power save) 62 63 - **tmux/capture.py** - Tmux capture library (integrated into Linux observer for fallback capture)
-81
observe/macos/TODO.md
··· 1 - # macOS Observer TODO 2 - 3 - Tracks remaining work for the macOS observer integration. 4 - 5 - ## Completed 6 - 7 - - **Phase 1: Activity Detection** (`activity.py`) - All done 8 - - **Phase 2: ScreenCaptureKit Manager** (`screencapture.py`) - All done 9 - - **Phase 3: Main Observer** (`observer.py`) - All done 10 - - **Phase 5: sck-cli** - All requirements met 11 - 12 - ## Phase 4: Testing & Integration 13 - 14 - ### 4.1 Manual Testing 15 - - [x] Install PyObjC dependencies (now automatic via `make install`) 16 - - [ ] Build and install sck-cli to PATH 17 - - [ ] Run observer: `observe-macos --interval 60` (use 1 min for faster testing) 18 - - [ ] Verify files created in journal directory 19 - - [ ] Test activity detection (go idle, lock screen, etc.) 20 - - [ ] Test window boundaries and file naming 21 - - [ ] Test graceful shutdown (Ctrl-C) 22 - - [ ] Verify Callosum events emitted 23 - 24 - ### 4.2 Multi-Monitor Testing 25 - - [ ] Test with single monitor (position should be "center") 26 - - [ ] Test with dual monitors (side-by-side) 27 - - [ ] Test with three monitors 28 - - [ ] Verify per-display files with position labels 29 - - [ ] Test monitor arrangement changes during capture 30 - 31 - ### 4.3 Edge Cases 32 - - [ ] Test rapid screen lock/unlock 33 - - [ ] Test system sleep/wake 34 - - [ ] Test display disconnect/reconnect 35 - - [ ] Test sck-cli crash/failure during capture 36 - - [ ] Test disk full scenario 37 - - [ ] Test very short capture durations 38 - - [ ] Test very long capture durations (>5 min) 39 - 40 - ### 4.4 Integration with Downstream Tools 41 - - [ ] Verify observe-describe works with .mov files 42 - - [ ] Verify observe-sense dispatches .mov to describe and .m4a to transcribe 43 - - [ ] Test parse_screen_filename() with new displayID format 44 - - [ ] Verify think-indexer handles new file formats 45 - 46 - ## Phase 6: Documentation & Polish 47 - 48 - ### 6.1 Documentation 49 - - [ ] Create observe/macos/README.md with installation and usage 50 - - [ ] Update main README.md to mention macOS support 51 - - [ ] Document differences from Linux observer 52 - 53 - ### 6.2 Code Quality 54 - - [x] Run `make format` and `make lint` 55 - - [x] Add type hints to function signatures 56 - - [x] Proper logging at appropriate levels 57 - 58 - --- 59 - 60 - ## Reference 61 - 62 - ### File Naming Convention 63 - 64 - Files are stored in segment directories: `YYYYMMDD/HHMMSS_LEN/` 65 - 66 - - **Video**: `position_displayID_screen.mov` (e.g., `center_1_screen.mov`) 67 - - **Audio**: `audio.m4a` 68 - - **Draft folder**: `HHMMSS_draft/` during capture, renamed to `HHMMSS_LEN/` on completion 69 - 70 - ### Differences from Linux Observer 71 - - **Audio threshold**: macOS checks at boundary (post-capture), Linux checks real-time 72 - - **Format**: .mov video instead of .webm, .m4a audio instead of .flac 73 - - **Activity APIs**: PyObjC/Quartz instead of DBus 74 - - **Capture**: External sck-cli process instead of GStreamer/PipeWire 75 - - **Connector ID**: Numeric displayID instead of connector names like "DP-3" 76 - - **No tmux mode**: macOS observer only has screencast/idle modes 77 - 78 - ### Dependencies 79 - - sck-cli must be built and available in PATH (or specified via --sck-cli-path) 80 - - PyObjC frameworks (core, Cocoa, Quartz) - installed automatically on macOS via `make install` 81 - - observe.utils.assign_monitor_positions for position label computation
-4
observe/macos/__init__.py
··· 1 - # SPDX-License-Identifier: AGPL-3.0-only 2 - # Copyright (c) 2026 sol pbc 3 - 4 - """macOS-specific observation: activity detection (PyObjC/Quartz) and capture (sck-cli)."""
-136
observe/macos/activity.py
··· 1 - # SPDX-License-Identifier: AGPL-3.0-only 2 - # Copyright (c) 2026 sol pbc 3 - 4 - """macOS system activity detection using PyObjC. 5 - 6 - This module mirrors the observe/gnome/activity.py structure, providing activity 7 - detection primitives using native macOS APIs via PyObjC. 8 - """ 9 - 10 - import logging 11 - import subprocess 12 - 13 - from Quartz import ( 14 - CGDisplayIsAsleep, 15 - CGEventSourceSecondsSinceLastEventType, 16 - CGMainDisplayID, 17 - CGSessionCopyCurrentDictionary, 18 - kCGAnyInputEventType, 19 - ) 20 - 21 - logger = logging.getLogger(__name__) 22 - 23 - 24 - def get_idle_time_ms() -> int: 25 - """ 26 - Get the current system idle time in milliseconds. 27 - 28 - Uses Quartz CGEventSourceSecondsSinceLastEventType to detect time since last 29 - user input event (keyboard, mouse, etc.). 30 - 31 - Returns: 32 - Idle time in milliseconds 33 - 34 - Example: 35 - >>> idle_ms = get_idle_time_ms() 36 - >>> print(f"User idle for {idle_ms / 1000:.1f} seconds") 37 - """ 38 - try: 39 - # kCGEventSourceStateHIDSystemState = 1 (hardware input events) 40 - seconds = CGEventSourceSecondsSinceLastEventType(1, kCGAnyInputEventType) 41 - return int(seconds * 1000) 42 - except Exception as e: 43 - logger.warning(f"Failed to get idle time: {e}") 44 - return 0 45 - 46 - 47 - def is_screen_locked() -> bool: 48 - """ 49 - Check if the screen is currently locked. 50 - 51 - Queries the macOS session state via CGSessionCopyCurrentDictionary. 52 - When the screen is locked, kCGSSessionOnConsoleKey becomes False. 53 - 54 - Returns: 55 - True if screen is locked, False otherwise 56 - 57 - Example: 58 - >>> if is_screen_locked(): 59 - ... print("Screen is locked, skipping capture") 60 - """ 61 - try: 62 - session_dict = CGSessionCopyCurrentDictionary() 63 - if session_dict is None: 64 - logger.warning("CGSessionCopyCurrentDictionary returned None") 65 - return False 66 - 67 - # kCGSSessionOnConsoleKey is True when user is on console (not locked) 68 - # When screen is locked, this becomes False 69 - on_console = session_dict.get("kCGSSessionOnConsoleKey", True) 70 - return not on_console 71 - except Exception as e: 72 - logger.warning(f"Failed to check screen lock status: {e}") 73 - return False 74 - 75 - 76 - def is_power_save_active() -> bool: 77 - """ 78 - Check if display power save mode is active (screen blanked/sleep). 79 - 80 - Uses CGDisplayIsAsleep to detect if the main display is sleeping, 81 - similar to GNOME's DisplayConfig PowerSaveMode check. 82 - 83 - Returns: 84 - True if power save is active (displays off), False otherwise 85 - 86 - Example: 87 - >>> if is_power_save_active(): 88 - ... print("Displays are sleeping") 89 - """ 90 - try: 91 - main_display = CGMainDisplayID() 92 - is_asleep = CGDisplayIsAsleep(main_display) 93 - return bool(is_asleep) 94 - except Exception as e: 95 - logger.warning(f"Failed to check display sleep status: {e}") 96 - return False 97 - 98 - 99 - def is_output_muted() -> bool: 100 - """ 101 - Check if the system audio output is muted. 102 - 103 - Uses osascript to query macOS volume settings, similar to how GNOME 104 - uses pactl for PulseAudio mute status. 105 - 106 - Returns: 107 - True if muted, False otherwise (including on error). 108 - 109 - Example: 110 - >>> if is_output_muted(): 111 - ... print("Audio is muted") 112 - """ 113 - try: 114 - result = subprocess.run( 115 - ["osascript", "-e", "output muted of (get volume settings)"], 116 - capture_output=True, 117 - text=True, 118 - timeout=5, 119 - ) 120 - 121 - if result.returncode != 0: 122 - logger.warning( 123 - f"osascript failed (rc={result.returncode}): {result.stderr}" 124 - ) 125 - return False 126 - 127 - return result.stdout.strip().lower() == "true" 128 - except subprocess.TimeoutExpired: 129 - logger.warning("osascript timed out checking mute status") 130 - return False 131 - except FileNotFoundError: 132 - logger.warning("osascript not found") 133 - return False 134 - except Exception as e: 135 - logger.warning(f"Error checking output mute status: {e}") 136 - return False
-713
observe/macos/observer.py
··· 1 - #!/usr/bin/env python3 2 - # SPDX-License-Identifier: AGPL-3.0-only 3 - # Copyright (c) 2026 sol pbc 4 - 5 - """ 6 - macOS observer for audio and screencast capture using ScreenCaptureKit. 7 - 8 - Continuously captures audio and video using sck-cli based on activity detection. 9 - Creates 5-minute windows, saving both video and audio when voice activity is detected. 10 - """ 11 - 12 - import argparse 13 - import asyncio 14 - import logging 15 - import os 16 - import platform 17 - import shutil 18 - import signal 19 - import socket 20 - import sys 21 - import time 22 - from pathlib import Path 23 - 24 - import av 25 - import numpy as np 26 - 27 - from observe.macos.activity import ( 28 - get_idle_time_ms, 29 - is_output_muted, 30 - is_power_save_active, 31 - is_screen_locked, 32 - ) 33 - from observe.macos.screencapture import AudioInfo, DisplayInfo, ScreenCaptureKitManager 34 - from observe.utils import create_draft_folder, get_timestamp_parts 35 - from think.callosum import CallosumConnection 36 - from think.streams import stream_name, update_stream, write_segment_stream 37 - from think.utils import day_path, get_journal, get_rev, setup_cli 38 - 39 - logger = logging.getLogger(__name__) 40 - 41 - # Host identification 42 - HOST = socket.gethostname() 43 - PLATFORM = platform.system().lower() 44 - 45 - # Constants 46 - IDLE_THRESHOLD_MS = 5 * 60 * 1000 # 5 minutes 47 - CHUNK_DURATION = 5 # seconds 48 - RMS_THRESHOLD = 0.01 49 - MIN_HITS_FOR_SAVE = 3 50 - SAMPLE_RATE = 48000 # Standard audio sample rate 51 - 52 - 53 - class MacOSObserver: 54 - """macOS audio and screencast observer using ScreenCaptureKit.""" 55 - 56 - def __init__( 57 - self, 58 - interval: int = 300, 59 - sck_cli_path: str = "sck-cli", 60 - ): 61 - """ 62 - Initialize the macOS observer. 63 - 64 - Args: 65 - interval: Window duration in seconds (default: 300 = 5 minutes) 66 - sck_cli_path: Path to sck-cli executable 67 - """ 68 - self.interval = interval 69 - self.screencapture = ScreenCaptureKitManager(sck_cli_path=sck_cli_path) 70 - self.running = True 71 - self.stream = stream_name(host=HOST) 72 - 73 - # Callosum connection for events 74 - self._callosum: CallosumConnection | None = None 75 - self._journal_path: Path | None = None 76 - 77 - # State tracking 78 - self.start_at = time.time() # Wall-clock for filenames 79 - self.start_at_mono = time.monotonic() # Monotonic for elapsed calculations 80 - self.capture_running = False 81 - 82 - # Multi-display tracking (similar to Linux observer) 83 - self.current_displays: list[DisplayInfo] = [] 84 - self.current_audio: AudioInfo | None = None 85 - 86 - # Draft folder for current segment (HHMMSS_draft/) 87 - self.draft_dir: str | None = None 88 - 89 - # Activity status cache (updated each loop) 90 - self.cached_is_active = False 91 - self.cached_idle_time_ms = 0 92 - self.cached_screen_locked = False 93 - self.cached_is_muted = False 94 - self.cached_power_save = False 95 - 96 - # Mute state at segment start 97 - self.segment_is_muted = False 98 - 99 - # Lock/power-save transition tracking (for boundary detection) 100 - self.was_locked_or_sleeping = False 101 - 102 - async def setup(self): 103 - """Initialize ScreenCaptureKit and Callosum connection.""" 104 - # Verify sck-cli is available 105 - sck_path = shutil.which(self.screencapture.sck_cli_path) 106 - if not sck_path: 107 - logger.error(f"sck-cli not found: {self.screencapture.sck_cli_path}") 108 - return False 109 - logger.info(f"Found sck-cli at: {sck_path}") 110 - 111 - # Start Callosum connection for events 112 - self._callosum = CallosumConnection(defaults={"rev": get_rev()}) 113 - self._callosum.start() 114 - self._journal_path = Path(get_journal()) 115 - logger.info("Callosum connection started") 116 - 117 - return True 118 - 119 - def check_activity_status(self) -> bool: 120 - """ 121 - Check system activity status and cache values. 122 - 123 - Returns: 124 - True if user is active (not idle and screen unlocked) 125 - """ 126 - idle_time = get_idle_time_ms() 127 - screen_locked = is_screen_locked() 128 - power_save = is_power_save_active() 129 - output_muted = is_output_muted() 130 - 131 - # Cache values for status events 132 - self.cached_idle_time_ms = idle_time 133 - self.cached_screen_locked = screen_locked 134 - self.cached_power_save = power_save 135 - self.cached_is_muted = output_muted 136 - 137 - is_idle = (idle_time > IDLE_THRESHOLD_MS) or screen_locked or power_save 138 - is_active = not is_idle 139 - 140 - # Cache result 141 - self.cached_is_active = is_active 142 - 143 - return is_active 144 - 145 - def _check_audio_threshold(self, audio_path: str) -> bool: 146 - """ 147 - Check if audio file has enough voice activity to save. 148 - 149 - Decodes the m4a file and applies the same 3-chunk RMS threshold 150 - logic as Linux observer uses for real-time audio. 151 - 152 - Args: 153 - audio_path: Path to the m4a audio file 154 - 155 - Returns: 156 - True if audio should be saved (enough voice activity), False otherwise 157 - """ 158 - if not os.path.exists(audio_path): 159 - logger.warning(f"Audio file not found for threshold check: {audio_path}") 160 - return False 161 - 162 - container = None 163 - try: 164 - container = av.open(audio_path) 165 - audio_streams = list(container.streams.audio) 166 - 167 - if not audio_streams: 168 - logger.warning(f"No audio streams in {audio_path}") 169 - return False 170 - 171 - # Check ALL audio streams - pass if ANY has enough voice activity 172 - # Stream 0 is typically system audio, stream 1 is microphone 173 - for stream_idx, stream in enumerate(audio_streams): 174 - sample_rate = stream.rate or SAMPLE_RATE 175 - 176 - # Decode audio and collect samples for this stream 177 - samples = [] 178 - container.seek(0) # Reset to start for each stream 179 - for packet in container.demux(stream): 180 - for frame in packet.decode(): 181 - arr = frame.to_ndarray() 182 - # Convert to mono if stereo (average channels) 183 - if arr.ndim > 1: 184 - arr = arr.mean(axis=0) 185 - samples.append(arr.flatten()) 186 - 187 - if not samples: 188 - continue 189 - 190 - # Concatenate all samples 191 - all_samples = np.concatenate(samples) 192 - 193 - # Split into CHUNK_DURATION (5 second) chunks and count threshold hits 194 - chunk_samples = int(sample_rate * CHUNK_DURATION) 195 - threshold_hits = 0 196 - 197 - for i in range(0, len(all_samples), chunk_samples): 198 - chunk = all_samples[i : i + chunk_samples] 199 - if len(chunk) == 0: 200 - continue 201 - 202 - # Compute RMS for this chunk 203 - rms = float(np.sqrt(np.mean(chunk**2))) 204 - if rms > RMS_THRESHOLD: 205 - threshold_hits += 1 206 - 207 - logger.debug( 208 - f"Audio threshold check stream {stream_idx}: " 209 - f"{threshold_hits}/{MIN_HITS_FOR_SAVE} hits" 210 - ) 211 - 212 - if threshold_hits >= MIN_HITS_FOR_SAVE: 213 - return True 214 - 215 - # No stream passed threshold 216 - return False 217 - 218 - except Exception as e: 219 - logger.warning(f"Error checking audio threshold for {audio_path}: {e}") 220 - # On error, keep the file (safer default) 221 - return True 222 - finally: 223 - if container is not None: 224 - container.close() 225 - 226 - def _build_segment_meta(self, audio_saved: bool) -> dict: 227 - """ 228 - Build metadata dict for the current segment. 229 - 230 - Collects information from displays, audio, and stop event to create 231 - a meta dict that flows through the observe.observing event. 232 - 233 - Args: 234 - audio_saved: Whether the audio file passed threshold and was saved 235 - 236 - Returns: 237 - Dict with segment metadata (audio_devices, stop_reason, etc.) 238 - """ 239 - meta: dict = { 240 - "display_count": len(self.current_displays), 241 - "muted": self.segment_is_muted, 242 - } 243 - 244 - # Audio device info 245 - if self.current_audio: 246 - meta["audio_sample_rate"] = self.current_audio.sample_rate 247 - meta["audio_channels"] = self.current_audio.channels 248 - meta["audio_saved"] = audio_saved 249 - # Extract device names from tracks 250 - devices = [ 251 - t.get("deviceName", "unknown") for t in self.current_audio.tracks 252 - ] 253 - if devices: 254 - meta["audio_devices"] = devices 255 - 256 - # Stop event info 257 - stop_event = self.screencapture.get_stop_event(timeout=0.5) 258 - if stop_event: 259 - meta["stop_reason"] = stop_event.get("reason") 260 - # Include error details if present 261 - if stop_event.get("errorCode"): 262 - meta["stop_error_code"] = stop_event["errorCode"] 263 - if stop_event.get("errorDomain"): 264 - meta["stop_error_domain"] = stop_event["errorDomain"] 265 - # Include device change flags if present 266 - if stop_event.get("inputDeviceChanged"): 267 - meta["input_device_changed"] = True 268 - if stop_event.get("outputDeviceChanged"): 269 - meta["output_device_changed"] = True 270 - 271 - return meta 272 - 273 - def _finalize_segment(self) -> tuple[str, str, list[str], dict]: 274 - """ 275 - Finalize current segment by renaming files and folder. 276 - 277 - Renames video files to simple names, checks audio threshold and 278 - renames/deletes accordingly, then renames draft folder to final name. 279 - 280 - Returns: 281 - Tuple of (date_part, segment_key, saved_files, meta) 282 - """ 283 - # Get timestamp parts for this window and calculate duration 284 - date_part, time_part = get_timestamp_parts(self.start_at) 285 - duration = int(time.time() - self.start_at) 286 - day_dir = day_path(date_part) 287 - segment_key = f"{time_part}_{duration}" 288 - 289 - saved_files: list[str] = [] 290 - 291 - # Rename video files to simple names in draft folder 292 - for display in self.current_displays: 293 - if os.path.exists(display.file_path): 294 - # Simple name: position_displayID_screen.mov 295 - simple_name = f"{display.position}_{display.display_id}_screen.mov" 296 - simple_path = Path(self.draft_dir) / simple_name 297 - try: 298 - os.rename(display.file_path, simple_path) 299 - saved_files.append(simple_name) 300 - except OSError as e: 301 - logger.error(f"Failed to rename {display.file_path}: {e}") 302 - 303 - # Check audio threshold and rename if passing 304 - audio_saved = False 305 - if self.current_audio and os.path.exists(self.current_audio.file_path): 306 - if self._check_audio_threshold(self.current_audio.file_path): 307 - simple_name = "audio.m4a" 308 - simple_path = Path(self.draft_dir) / simple_name 309 - try: 310 - os.rename(self.current_audio.file_path, simple_path) 311 - saved_files.append(simple_name) 312 - audio_saved = True 313 - logger.info(f"Audio passed threshold check, saving: {simple_name}") 314 - except OSError as e: 315 - logger.error(f"Failed to rename audio: {e}") 316 - else: 317 - # Delete the audio file 318 - try: 319 - os.remove(self.current_audio.file_path) 320 - logger.info("Audio below threshold, discarded") 321 - except OSError as e: 322 - logger.warning(f"Failed to remove audio file: {e}") 323 - 324 - # Build metadata before clearing state 325 - meta = self._build_segment_meta(audio_saved) 326 - 327 - # Clear capture state 328 - self.current_displays = [] 329 - self.current_audio = None 330 - 331 - # Rename draft folder to final segment name (atomic handoff) 332 - if self.draft_dir and saved_files: 333 - final_segment_dir = str(day_dir / self.stream / segment_key) 334 - try: 335 - os.rename(self.draft_dir, final_segment_dir) 336 - logger.info(f"Segment finalized: {segment_key}") 337 - except OSError as e: 338 - logger.error(f"Failed to rename draft folder: {e}") 339 - saved_files = [] # Don't emit event if rename failed 340 - 341 - # Write stream identity for this segment 342 - if saved_files: 343 - try: 344 - result = update_stream( 345 - self.stream, 346 - date_part, 347 - segment_key, 348 - type="observer", 349 - host=HOST, 350 - platform=PLATFORM, 351 - ) 352 - write_segment_stream( 353 - final_segment_dir, 354 - self.stream, 355 - result["prev_day"], 356 - result["prev_segment"], 357 - result["seq"], 358 - ) 359 - except Exception as e: 360 - logger.warning(f"Failed to write stream identity: {e}") 361 - elif self.draft_dir: 362 - # No files to save, remove empty draft folder 363 - try: 364 - os.rmdir(self.draft_dir) 365 - logger.debug(f"Removed empty draft folder: {self.draft_dir}") 366 - except OSError: 367 - pass # May have other files, ignore 368 - 369 - self.draft_dir = None 370 - 371 - return date_part, segment_key, saved_files, meta 372 - 373 - def handle_boundary(self): 374 - """ 375 - Handle window boundary rollover. 376 - 377 - Closes the current draft folder, renames files to simple names, 378 - renames folder to final segment name, and emits the observing event. 379 - """ 380 - if self.capture_running: 381 - logger.info("Stopping previous capture") 382 - self.screencapture.stop() 383 - self.capture_running = False 384 - 385 - # Finalize segment (rename files, folder, and build metadata) 386 - date_part, segment_key, saved_files, meta = self._finalize_segment() 387 - 388 - # Emit observing event with saved files and metadata 389 - if saved_files and self._callosum: 390 - self._callosum.emit( 391 - "observe", 392 - "observing", 393 - day=date_part, 394 - segment=segment_key, 395 - files=saved_files, 396 - host=HOST, 397 - platform=PLATFORM, 398 - meta=meta, 399 - stream=self.stream, 400 - ) 401 - logger.info( 402 - f"Segment observing: {segment_key} ({len(saved_files)} files)" 403 - ) 404 - 405 - # Reset timing for new window 406 - self.start_at = time.time() 407 - self.start_at_mono = time.monotonic() 408 - 409 - # Update segment mute state 410 - self.segment_is_muted = self.cached_is_muted 411 - 412 - # Start new capture unless screen is locked or display is sleeping 413 - if not self.cached_screen_locked and not self.cached_power_save: 414 - self.initialize_capture() 415 - 416 - def _create_draft_folder(self) -> str: 417 - """Create a draft folder for the current segment.""" 418 - self.draft_dir = create_draft_folder(self.start_at, self.stream) 419 - logger.debug(f"Created draft folder: {self.draft_dir}") 420 - return self.draft_dir 421 - 422 - def initialize_capture(self) -> bool: 423 - """ 424 - Start a new screencast and audio recording. 425 - 426 - Creates a draft folder and starts sck-cli recording. 427 - 428 - Returns: 429 - True if capture started successfully, False otherwise 430 - """ 431 - 432 - # Create draft folder for this segment 433 - draft_path = self._create_draft_folder() 434 - 435 - # Build output base for sck-cli (inside draft folder) 436 - # sck-cli will create files like: draft/capture_1.mov, draft/capture.m4a 437 - output_base = Path(draft_path) / "capture" 438 - 439 - try: 440 - displays, audio = self.screencapture.start( 441 - output_base, self.interval, frame_rate=1.0 442 - ) 443 - except RuntimeError as e: 444 - logger.error(f"Failed to start capture: {e}") 445 - return False 446 - 447 - self.current_displays = displays 448 - self.current_audio = audio 449 - self.capture_running = True 450 - 451 - logger.info(f"Started capture with {len(displays)} display(s)") 452 - for display in displays: 453 - logger.info( 454 - f" Display {display.display_id}: {display.position} -> {display.file_path}" 455 - ) 456 - if audio: 457 - logger.info(f" Audio: {audio.file_path}") 458 - 459 - return True 460 - 461 - def emit_status(self): 462 - """Emit observe.status event with current state. 463 - 464 - Event structure matches Linux observer for compatibility: 465 - - mode: "screencast" or "idle" (macOS doesn't have tmux mode) 466 - - screencast: recording status and display info 467 - - tmux: always empty (not supported on macOS) 468 - - audio: always empty (macOS checks threshold at boundary, not real-time) 469 - - activity: system activity status 470 - """ 471 - if not self._callosum: 472 - return 473 - 474 - journal_path = str(self._journal_path) if self._journal_path else "" 475 - 476 - # Determine mode (macOS is binary: screencast or idle) 477 - mode = "screencast" if self.capture_running else "idle" 478 - 479 - # Build screencast info (matches Linux observer structure) 480 - if self.capture_running and self.current_displays: 481 - elapsed = int(time.monotonic() - self.start_at_mono) 482 - streams_info = [] 483 - for display in self.current_displays: 484 - try: 485 - rel_file = os.path.relpath(display.file_path, journal_path) 486 - except ValueError: 487 - rel_file = display.file_path 488 - 489 - streams_info.append( 490 - { 491 - "position": display.position, 492 - "connector": str(display.display_id), 493 - "file": rel_file, 494 - } 495 - ) 496 - 497 - screencast_info = { 498 - "recording": True, 499 - "streams": streams_info, 500 - "window_elapsed_seconds": elapsed, 501 - } 502 - else: 503 - screencast_info = {"recording": False} 504 - 505 - # Tmux info (not supported on macOS) 506 - tmux_info = {"capturing": False} 507 - 508 - # Audio info (macOS checks threshold at boundary, not real-time) 509 - audio_info = { 510 - "threshold_hits": 0, 511 - "will_save": False, 512 - } 513 - 514 - # Activity info 515 - activity_info = { 516 - "active": self.cached_is_active, 517 - "idle_time_ms": self.cached_idle_time_ms, 518 - "screen_locked": self.cached_screen_locked, 519 - "power_save": self.cached_power_save, 520 - "sink_muted": self.cached_is_muted, 521 - } 522 - 523 - # Emit status 524 - self._callosum.emit( 525 - "observe", 526 - "status", 527 - mode=mode, 528 - screencast=screencast_info, 529 - tmux=tmux_info, 530 - audio=audio_info, 531 - activity=activity_info, 532 - host=HOST, 533 - platform=PLATFORM, 534 - stream=self.stream, 535 - ) 536 - 537 - async def main_loop(self): 538 - """Run the main observer loop.""" 539 - logger.info(f"Starting observer loop (interval={self.interval}s)") 540 - 541 - # Check initial activity and start capture unless locked/sleeping 542 - self.check_activity_status() 543 - self.segment_is_muted = self.cached_is_muted 544 - self.was_locked_or_sleeping = ( 545 - self.cached_screen_locked or self.cached_power_save 546 - ) 547 - 548 - if not self.cached_screen_locked and not self.cached_power_save: 549 - if not self.initialize_capture(): 550 - logger.error("Failed to start initial capture") 551 - self.running = False 552 - return 553 - 554 - while self.running: 555 - # Sleep for chunk duration 556 - await asyncio.sleep(CHUNK_DURATION) 557 - 558 - # Check activity status (caches idle, lock, mute, power-save) 559 - self.check_activity_status() 560 - 561 - # Check if sck-cli process died unexpectedly 562 - if self.capture_running and not self.screencapture.is_running(): 563 - logger.warning("Capture process died, handling boundary") 564 - self.handle_boundary() 565 - continue 566 - 567 - # Detect lock/power-save transition (either direction) 568 - locked_or_sleeping = self.cached_screen_locked or self.cached_power_save 569 - lock_transition = locked_or_sleeping != self.was_locked_or_sleeping 570 - if lock_transition: 571 - logger.info( 572 - f"Lock/sleep state changed: " 573 - f"{'locked' if self.was_locked_or_sleeping else 'unlocked'} -> " 574 - f"{'locked' if locked_or_sleeping else 'unlocked'}" 575 - ) 576 - self.was_locked_or_sleeping = locked_or_sleeping 577 - 578 - # Detect mute state transition 579 - mute_transition = self.cached_is_muted != self.segment_is_muted 580 - if mute_transition: 581 - logger.info( 582 - f"Mute state changed: " 583 - f"{'muted' if self.segment_is_muted else 'unmuted'} -> " 584 - f"{'muted' if self.cached_is_muted else 'unmuted'}" 585 - ) 586 - 587 - # Check for window boundary (use monotonic to avoid DST/clock jumps) 588 - now_mono = time.monotonic() 589 - elapsed = now_mono - self.start_at_mono 590 - is_boundary = ( 591 - (elapsed >= self.interval) or lock_transition or mute_transition 592 - ) 593 - 594 - if is_boundary: 595 - logger.info( 596 - f"Boundary: elapsed={elapsed:.1f}s lock_change={lock_transition} " 597 - f"mute_change={mute_transition}" 598 - ) 599 - self.handle_boundary() 600 - 601 - # Emit status event 602 - self.emit_status() 603 - 604 - # Cleanup on exit 605 - logger.info("Observer loop stopped, cleaning up...") 606 - await self.shutdown() 607 - 608 - async def shutdown(self): 609 - """Clean shutdown of observer.""" 610 - 611 - # Stop capture if running 612 - if self.capture_running: 613 - logger.info("Stopping capture for shutdown") 614 - self.screencapture.stop() 615 - self.capture_running = False 616 - 617 - # Brief delay for files to be written 618 - await asyncio.sleep(0.5) 619 - 620 - # Finalize segment (rename files and folder) 621 - date_part, segment_key, saved_files, meta = self._finalize_segment() 622 - 623 - # Emit observing event for final segment 624 - if saved_files and self._callosum: 625 - self._callosum.emit( 626 - "observe", 627 - "observing", 628 - day=date_part, 629 - segment=segment_key, 630 - files=saved_files, 631 - host=HOST, 632 - platform=PLATFORM, 633 - meta=meta, 634 - stream=self.stream, 635 - ) 636 - logger.info( 637 - f"Segment observing: {segment_key} ({len(saved_files)} files)" 638 - ) 639 - 640 - # Stop Callosum connection 641 - if self._callosum: 642 - self._callosum.stop() 643 - self._callosum = None 644 - logger.info("Callosum connection stopped") 645 - 646 - logger.info("Shutdown complete") 647 - 648 - 649 - async def async_main(args): 650 - """Async entry point.""" 651 - observer = MacOSObserver( 652 - interval=args.interval, 653 - sck_cli_path=args.sck_cli_path, 654 - ) 655 - 656 - # Setup signal handlers 657 - loop = asyncio.get_running_loop() 658 - 659 - def signal_handler(): 660 - logger.info("Received shutdown signal") 661 - observer.running = False 662 - 663 - for sig in (signal.SIGINT, signal.SIGTERM): 664 - loop.add_signal_handler(sig, signal_handler) 665 - 666 - # Initialize 667 - if not await observer.setup(): 668 - logger.error("Observer setup failed") 669 - return 1 670 - 671 - # Run main loop 672 - try: 673 - await observer.main_loop() 674 - except Exception as e: 675 - logger.error(f"Observer error: {e}", exc_info=True) 676 - return 1 677 - 678 - return 0 679 - 680 - 681 - def main(): 682 - """CLI entry point.""" 683 - parser = argparse.ArgumentParser( 684 - description="macOS audio and screencast observer for journaling." 685 - ) 686 - parser.add_argument( 687 - "--interval", 688 - type=int, 689 - default=300, 690 - help="Duration per capture window in seconds (default: 300 = 5 minutes).", 691 - ) 692 - parser.add_argument( 693 - "--sck-cli-path", 694 - type=str, 695 - default="sck-cli", 696 - help="Path to sck-cli executable (default: sck-cli from PATH).", 697 - ) 698 - args = setup_cli(parser) 699 - 700 - # Run async main 701 - try: 702 - rc = asyncio.run(async_main(args)) 703 - sys.exit(rc) 704 - except KeyboardInterrupt: 705 - logger.info("Interrupted by user") 706 - sys.exit(0) 707 - except Exception as e: 708 - logger.error(f"Fatal error: {e}", exc_info=True) 709 - sys.exit(1) 710 - 711 - 712 - if __name__ == "__main__": 713 - main()
-367
observe/macos/screencapture.py
··· 1 - # SPDX-License-Identifier: AGPL-3.0-only 2 - # Copyright (c) 2026 sol pbc 3 - 4 - """ScreenCaptureKit integration via sck-cli subprocess. 5 - 6 - This module manages the sck-cli subprocess lifecycle for video and audio capture 7 - on macOS using ScreenCaptureKit. sck-cli captures all displays simultaneously 8 - and outputs JSONL metadata to stdout with display geometry, audio device info, 9 - and stop events (reason for capture ending). 10 - """ 11 - 12 - import json 13 - import logging 14 - import select 15 - import signal 16 - import subprocess 17 - import threading 18 - import time 19 - from dataclasses import dataclass 20 - from pathlib import Path 21 - from typing import Optional 22 - 23 - from observe.utils import assign_monitor_positions 24 - 25 - # Timeout for reading metadata from sck-cli (seconds) 26 - METADATA_TIMEOUT = 5.0 27 - 28 - logger = logging.getLogger(__name__) 29 - 30 - 31 - @dataclass 32 - class DisplayInfo: 33 - """Information about a single display's recording.""" 34 - 35 - display_id: int 36 - position: str 37 - x: int 38 - y: int 39 - width: int 40 - height: int 41 - file_path: str # Path where sck-cli writes the file 42 - 43 - 44 - @dataclass 45 - class AudioInfo: 46 - """Information about the audio recording.""" 47 - 48 - file_path: str # Path where sck-cli writes the file 49 - sample_rate: int 50 - channels: int 51 - tracks: list[dict] # Full track dicts from sck-cli (name, deviceName, etc.) 52 - 53 - 54 - class ScreenCaptureKitManager: 55 - """ 56 - Manages sck-cli subprocess for synchronized video and audio capture. 57 - 58 - Wraps the sck-cli tool to provide lifecycle management, handles process 59 - monitoring, and parses JSONL output for display geometry. 60 - """ 61 - 62 - def __init__(self, sck_cli_path: str = "sck-cli"): 63 - """ 64 - Initialize the ScreenCaptureKit manager. 65 - 66 - Args: 67 - sck_cli_path: Path to sck-cli executable (default: "sck-cli" from PATH) 68 - """ 69 - self.sck_cli_path = sck_cli_path 70 - self.process: Optional[subprocess.Popen] = None 71 - self.displays: list[DisplayInfo] = [] 72 - self.audio: Optional[AudioInfo] = None 73 - self._output_threads: list[threading.Thread] = [] 74 - self._exit_logged: bool = False 75 - self._stop_event: Optional[dict] = None 76 - self._stop_received = threading.Event() 77 - 78 - def start( 79 - self, 80 - output_base: Path, 81 - duration: int, 82 - frame_rate: float = 1.0, 83 - ) -> tuple[list[DisplayInfo], Optional[AudioInfo]]: 84 - """ 85 - Start video and audio capture. 86 - 87 - Launches sck-cli as a subprocess with the specified parameters. 88 - Parses JSONL output from stdout to get display geometry information. 89 - Files are written to output_base_<displayID>.mov and output_base.m4a. 90 - 91 - Args: 92 - output_base: Base path for output files (without extension) 93 - duration: Capture duration in seconds 94 - frame_rate: Frame rate in Hz (default: 1.0) 95 - 96 - Returns: 97 - Tuple of (list of DisplayInfo, AudioInfo or None) 98 - 99 - Raises: 100 - RuntimeError: If sck-cli fails to start or returns no displays 101 - 102 - Example: 103 - >>> manager = ScreenCaptureKitManager() 104 - >>> draft_dir = Path("journal/20250101/120000_draft") 105 - >>> output_base = draft_dir / "capture" 106 - >>> displays, audio = manager.start(output_base, duration=300) 107 - """ 108 - # Build command 109 - cmd = [ 110 - self.sck_cli_path, 111 - str(output_base), 112 - "-r", 113 - str(frame_rate), 114 - "-l", 115 - str(duration), 116 - ] 117 - 118 - logger.info(f"Starting sck-cli: {' '.join(cmd)}") 119 - self._exit_logged = False 120 - self._stop_event = None 121 - self._stop_received.clear() 122 - 123 - try: 124 - self.process = subprocess.Popen( 125 - cmd, 126 - stdout=subprocess.PIPE, 127 - stderr=subprocess.PIPE, 128 - text=True, 129 - bufsize=1, # Line buffering for real-time output 130 - ) 131 - except FileNotFoundError: 132 - raise RuntimeError(f"sck-cli not found at: {self.sck_cli_path}") 133 - except Exception as e: 134 - raise RuntimeError(f"Failed to start sck-cli: {e}") 135 - 136 - # Read JSONL metadata from stdout (sck-cli outputs this immediately) 137 - # Each line is either a display info or audio info 138 - displays_raw = [] 139 - audio_info = None 140 - 141 - # Read lines until we get both display and audio metadata. 142 - # Use select() with timeout to avoid blocking forever - the process 143 - # keeps running for the capture duration but outputs metadata upfront. 144 - # Note: "for line in file:" uses block buffering which can hang. 145 - deadline = time.monotonic() + METADATA_TIMEOUT 146 - stdout_fd = self.process.stdout.fileno() 147 - 148 - try: 149 - while time.monotonic() < deadline: 150 - # Wait for data with remaining timeout 151 - remaining = deadline - time.monotonic() 152 - if remaining <= 0: 153 - break 154 - 155 - readable, _, _ = select.select([stdout_fd], [], [], min(remaining, 1.0)) 156 - if not readable: 157 - # No data yet, keep polling until deadline 158 - continue 159 - 160 - line = self.process.stdout.readline() 161 - if not line: 162 - # EOF - process closed stdout 163 - break 164 - 165 - line = line.strip() 166 - if not line: 167 - continue 168 - 169 - try: 170 - data = json.loads(line) 171 - if data.get("type") == "display": 172 - displays_raw.append(data) 173 - elif data.get("type") == "audio": 174 - audio_info = data 175 - except json.JSONDecodeError: 176 - # Not JSON, just a log message (already logged above) 177 - pass 178 - 179 - # Break once we have both display and audio info 180 - if displays_raw and audio_info is not None: 181 - break 182 - except Exception as e: 183 - logger.warning(f"Error reading sck-cli stdout: {e}") 184 - 185 - if not displays_raw: 186 - self.stop() 187 - raise RuntimeError("sck-cli returned no display information") 188 - 189 - # Convert raw display data to monitor format for position assignment 190 - monitors = [] 191 - for d in displays_raw: 192 - x = int(d.get("x", 0)) 193 - y = int(d.get("y", 0)) 194 - width = int(d.get("width", 0)) 195 - height = int(d.get("height", 0)) 196 - monitors.append( 197 - { 198 - "id": str(d["displayID"]), 199 - "box": [x, y, x + width, y + height], 200 - "_raw": d, 201 - } 202 - ) 203 - 204 - # Assign position labels based on geometry 205 - monitors = assign_monitor_positions(monitors) 206 - 207 - # Build DisplayInfo objects 208 - self.displays = [] 209 - for mon in monitors: 210 - raw = mon["_raw"] 211 - self.displays.append( 212 - DisplayInfo( 213 - display_id=raw["displayID"], 214 - position=mon["position"], 215 - x=mon["box"][0], 216 - y=mon["box"][1], 217 - width=mon["box"][2] - mon["box"][0], 218 - height=mon["box"][3] - mon["box"][1], 219 - file_path=raw["filename"], 220 - ) 221 - ) 222 - 223 - # Build AudioInfo if present 224 - if audio_info: 225 - self.audio = AudioInfo( 226 - file_path=audio_info["filename"], 227 - sample_rate=audio_info.get("sampleRate", 48000), 228 - channels=audio_info.get("channels", 1), 229 - tracks=audio_info.get("tracks", []), 230 - ) 231 - else: 232 - self.audio = None 233 - 234 - logger.info(f"sck-cli started with {len(self.displays)} display(s)") 235 - for display in self.displays: 236 - logger.info( 237 - f" Display {display.display_id}: {display.position} " 238 - f"({display.width}x{display.height}) -> {display.file_path}" 239 - ) 240 - if self.audio: 241 - track_names = [t.get("name", "unknown") for t in self.audio.tracks] 242 - logger.info(f" Audio: {self.audio.file_path} ({track_names})") 243 - 244 - # Start background threads to log remaining stdout/stderr in real-time 245 - self._output_threads = [ 246 - threading.Thread( 247 - target=self._stream_stdout, 248 - daemon=True, 249 - name="sck-cli-stdout", 250 - ), 251 - threading.Thread( 252 - target=self._stream_stderr, 253 - daemon=True, 254 - name="sck-cli-stderr", 255 - ), 256 - ] 257 - for thread in self._output_threads: 258 - thread.start() 259 - 260 - return self.displays, self.audio 261 - 262 - def stop(self) -> None: 263 - """ 264 - Stop the running capture gracefully. 265 - 266 - Sends SIGTERM to the sck-cli process and waits for it to finish writing 267 - files properly. This ensures video and audio files are finalized correctly. 268 - """ 269 - if self.process is None: 270 - return 271 - 272 - if self.process.poll() is None: 273 - logger.info("Stopping sck-cli...") 274 - try: 275 - self.process.send_signal(signal.SIGTERM) 276 - try: 277 - exit_code = self.process.wait(timeout=5) 278 - logger.info(f"sck-cli stopped with exit code {exit_code}") 279 - except subprocess.TimeoutExpired: 280 - logger.warning("sck-cli did not exit cleanly, killing") 281 - self.process.kill() 282 - exit_code = self.process.wait() 283 - logger.info(f"sck-cli killed with exit code {exit_code}") 284 - except Exception as e: 285 - logger.warning(f"Error stopping sck-cli: {e}") 286 - 287 - # Wait for output threads to finish (they exit when pipes close) 288 - for thread in self._output_threads: 289 - thread.join(timeout=1) 290 - self._output_threads = [] 291 - 292 - self.process = None 293 - 294 - def _stream_stdout(self) -> None: 295 - """Background thread: stream remaining stdout lines and capture stop events.""" 296 - if self.process is None or self.process.stdout is None: 297 - return 298 - 299 - try: 300 - for line in self.process.stdout: 301 - line = line.strip() 302 - if not line: 303 - continue 304 - 305 - # Try to parse as JSON to capture stop events 306 - try: 307 - data = json.loads(line) 308 - if data.get("type") == "stop": 309 - self._stop_event = data 310 - self._stop_received.set() 311 - logger.info(f"sck-cli stop: {data.get('reason', 'unknown')}") 312 - continue 313 - except json.JSONDecodeError: 314 - pass 315 - 316 - logger.info(f"sck-cli: {line}") 317 - except Exception as e: 318 - logger.debug(f"Error reading sck-cli stdout: {e}") 319 - 320 - def _stream_stderr(self) -> None: 321 - """Background thread: stream stderr lines to logger.""" 322 - if self.process is None or self.process.stderr is None: 323 - return 324 - 325 - try: 326 - for line in self.process.stderr: 327 - line = line.strip() 328 - if line: 329 - logger.info(f"sck-cli stderr: {line}") 330 - except Exception as e: 331 - logger.debug(f"Error reading sck-cli stderr: {e}") 332 - 333 - def is_running(self) -> bool: 334 - """ 335 - Check if the capture subprocess is currently running. 336 - 337 - Returns: 338 - True if subprocess is active, False otherwise 339 - """ 340 - if self.process is None: 341 - return False 342 - exit_code = self.process.poll() 343 - if exit_code is not None: 344 - if not self._exit_logged: 345 - logger.info(f"sck-cli exited with code {exit_code}") 346 - self._exit_logged = True 347 - return False 348 - return True 349 - 350 - def get_stop_event(self, timeout: float = 1.0) -> Optional[dict]: 351 - """ 352 - Get the stop event from sck-cli, waiting briefly if needed. 353 - 354 - Call this after stop() to retrieve the reason the capture ended. 355 - The stop event contains fields like: 356 - - reason: "completed", "device-change", "error", "signal" 357 - - errorCode, errorDomain: present if reason is "error" 358 - - inputDeviceChanged, outputDeviceChanged: present if reason is "device-change" 359 - 360 - Args: 361 - timeout: Maximum seconds to wait for stop event (default: 1.0) 362 - 363 - Returns: 364 - Stop event dict or None if not received 365 - """ 366 - self._stop_received.wait(timeout=timeout) 367 - return self._stop_event
+9 -9
observe/observer.py
··· 4 4 """Unified observer entry point with platform detection. 5 5 6 6 Detects the current platform and delegates to the appropriate 7 - platform-specific observer implementation. 7 + platform-specific observer implementation. Currently supports Linux only; 8 + macOS capture is handled by the solstone-macos native companion app. 8 9 """ 9 10 10 11 import sys ··· 14 15 """Platform-aware observer entry point. 15 16 16 17 Detects the current platform and calls the appropriate observer: 17 - - macOS (darwin): observe.macos.observer 18 18 - Linux: observe.linux.observer 19 - 20 - All command-line arguments are passed through to the platform-specific 21 - implementation via its main() function. 19 + - macOS: handled by solstone-macos native companion app (not this command) 22 20 """ 23 21 platform = sys.platform 24 22 25 - if platform == "darwin": 26 - from observe.macos.observer import main as platform_main 27 - elif platform == "linux": 23 + if platform == "linux": 28 24 from observe.linux.observer import main as platform_main 29 25 else: 30 26 print( 31 27 f"Error: Observer not available for platform '{platform}'", file=sys.stderr 32 28 ) 33 - print("Supported platforms: macOS (darwin), Linux", file=sys.stderr) 29 + print( 30 + "Supported platform: Linux. macOS capture is handled by the" 31 + " solstone-macos native companion app.", 32 + file=sys.stderr, 33 + ) 34 34 sys.exit(1) 35 35 36 36 platform_main()
-4
pyproject.toml
··· 46 46 # Linux-only: GNOME/GTK integration 47 47 "PyGObject; sys_platform == 'linux'", 48 48 "dbus-next; sys_platform == 'linux'", 49 - # macOS-only: PyObjC for activity detection 50 - "pyobjc-core; sys_platform == 'darwin'", 51 - "pyobjc-framework-Cocoa; sys_platform == 'darwin'", 52 - "pyobjc-framework-Quartz; sys_platform == 'darwin'", 53 49 "desktop-notifier", 54 50 "setproctitle", 55 51 "av",
-2
sol.py
··· 62 62 "observer": "observe.observer", 63 63 "remote": "observe.remote_cli", 64 64 "observe-linux": "observe.linux.observer", 65 - "observe-macos": "observe.macos.observer", 66 65 "tmux-observer": "observe.tmux.observer", 67 66 # AI agents (formerly muse package) 68 67 "agents": "think.agents", ··· 141 140 "formatter", 142 141 "detect-created", 143 142 "observe-linux", 144 - "observe-macos", 145 143 ], 146 144 "Help": ["help", "chat"], 147 145 }
-207
tests/test_macos_screencapture.py
··· 1 - # SPDX-License-Identifier: AGPL-3.0-only 2 - # Copyright (c) 2026 sol pbc 3 - 4 - """Tests for observe/macos/screencapture.py.""" 5 - 6 - from unittest.mock import MagicMock 7 - 8 - from observe.macos.screencapture import ( 9 - AudioInfo, 10 - DisplayInfo, 11 - ScreenCaptureKitManager, 12 - ) 13 - 14 - 15 - class TestAudioInfo: 16 - """Test AudioInfo dataclass.""" 17 - 18 - def test_create_with_full_tracks(self): 19 - """AudioInfo stores full track dicts.""" 20 - tracks = [ 21 - { 22 - "name": "system", 23 - "deviceName": "MacBook Pro Speakers", 24 - "deviceUID": "BuiltInSpeakerDevice", 25 - "manufacturer": "Apple Inc.", 26 - "transportType": "built-in", 27 - }, 28 - { 29 - "name": "microphone", 30 - "deviceName": "MacBook Pro Microphone", 31 - "deviceUID": "BuiltInMicrophoneDevice", 32 - "manufacturer": "Apple Inc.", 33 - "transportType": "built-in", 34 - }, 35 - ] 36 - audio = AudioInfo( 37 - file_path="/tmp/test.m4a", 38 - sample_rate=48000, 39 - channels=1, 40 - tracks=tracks, 41 - ) 42 - 43 - assert audio.file_path == "/tmp/test.m4a" 44 - assert audio.sample_rate == 48000 45 - assert audio.channels == 1 46 - assert len(audio.tracks) == 2 47 - assert audio.tracks[0]["deviceName"] == "MacBook Pro Speakers" 48 - assert audio.tracks[1]["name"] == "microphone" 49 - 50 - 51 - class TestDisplayInfo: 52 - """Test DisplayInfo dataclass.""" 53 - 54 - def test_create(self): 55 - """DisplayInfo stores display metadata.""" 56 - display = DisplayInfo( 57 - display_id=1, 58 - position="center", 59 - x=0, 60 - y=0, 61 - width=1920, 62 - height=1080, 63 - file_path="/tmp/test.mov", 64 - ) 65 - 66 - assert display.display_id == 1 67 - assert display.position == "center" 68 - assert display.width == 1920 69 - assert display.height == 1080 70 - 71 - 72 - class TestScreenCaptureKitManagerStopEvent: 73 - """Test stop event capture in ScreenCaptureKitManager.""" 74 - 75 - def test_stop_event_initial_state(self): 76 - """Manager starts with no stop event.""" 77 - manager = ScreenCaptureKitManager() 78 - assert manager._stop_event is None 79 - assert not manager._stop_received.is_set() 80 - 81 - def test_get_stop_event_timeout(self): 82 - """get_stop_event returns None on timeout.""" 83 - manager = ScreenCaptureKitManager() 84 - result = manager.get_stop_event(timeout=0.01) 85 - assert result is None 86 - 87 - def test_get_stop_event_returns_captured(self): 88 - """get_stop_event returns captured stop event.""" 89 - manager = ScreenCaptureKitManager() 90 - 91 - # Simulate stop event being captured 92 - stop_data = {"type": "stop", "reason": "completed"} 93 - manager._stop_event = stop_data 94 - manager._stop_received.set() 95 - 96 - result = manager.get_stop_event(timeout=0.1) 97 - assert result == stop_data 98 - assert result["reason"] == "completed" 99 - 100 - def test_stop_event_with_error(self): 101 - """Stop event with error details is captured.""" 102 - manager = ScreenCaptureKitManager() 103 - 104 - stop_data = { 105 - "type": "stop", 106 - "reason": "error", 107 - "errorCode": -1234, 108 - "errorDomain": "com.apple.ScreenCaptureKit", 109 - } 110 - manager._stop_event = stop_data 111 - manager._stop_received.set() 112 - 113 - result = manager.get_stop_event(timeout=0.1) 114 - assert result["reason"] == "error" 115 - assert result["errorCode"] == -1234 116 - assert result["errorDomain"] == "com.apple.ScreenCaptureKit" 117 - 118 - def test_stop_event_device_change(self): 119 - """Stop event with device change flags is captured.""" 120 - manager = ScreenCaptureKitManager() 121 - 122 - stop_data = { 123 - "type": "stop", 124 - "reason": "device-change", 125 - "inputDeviceChanged": True, 126 - "outputDeviceChanged": False, 127 - } 128 - manager._stop_event = stop_data 129 - manager._stop_received.set() 130 - 131 - result = manager.get_stop_event(timeout=0.1) 132 - assert result["reason"] == "device-change" 133 - assert result["inputDeviceChanged"] is True 134 - assert result["outputDeviceChanged"] is False 135 - 136 - 137 - class TestStreamStdoutParsing: 138 - """Test _stream_stdout JSON parsing for stop events.""" 139 - 140 - def test_parse_stop_event_from_stdout(self): 141 - """_stream_stdout parses stop events from JSONL output.""" 142 - manager = ScreenCaptureKitManager() 143 - 144 - # Create mock process with stdout that yields stop event 145 - mock_stdout = MagicMock() 146 - mock_stdout.__iter__ = MagicMock( 147 - return_value=iter(['{"type": "stop", "reason": "completed"}\n']) 148 - ) 149 - 150 - manager.process = MagicMock() 151 - manager.process.stdout = mock_stdout 152 - 153 - # Run the stream function directly 154 - manager._stream_stdout() 155 - 156 - # Verify stop event was captured 157 - assert manager._stop_event is not None 158 - assert manager._stop_event["type"] == "stop" 159 - assert manager._stop_event["reason"] == "completed" 160 - assert manager._stop_received.is_set() 161 - 162 - def test_parse_non_json_lines(self): 163 - """_stream_stdout handles non-JSON log lines gracefully.""" 164 - manager = ScreenCaptureKitManager() 165 - 166 - mock_stdout = MagicMock() 167 - mock_stdout.__iter__ = MagicMock( 168 - return_value=iter( 169 - [ 170 - "Some log message\n", 171 - "Another message\n", 172 - '{"type": "stop", "reason": "signal"}\n', 173 - ] 174 - ) 175 - ) 176 - 177 - manager.process = MagicMock() 178 - manager.process.stdout = mock_stdout 179 - 180 - manager._stream_stdout() 181 - 182 - # Only stop event should be captured 183 - assert manager._stop_event is not None 184 - assert manager._stop_event["reason"] == "signal" 185 - 186 - def test_parse_empty_lines(self): 187 - """_stream_stdout handles empty lines.""" 188 - manager = ScreenCaptureKitManager() 189 - 190 - mock_stdout = MagicMock() 191 - mock_stdout.__iter__ = MagicMock( 192 - return_value=iter( 193 - [ 194 - "\n", 195 - " \n", 196 - '{"type": "stop", "reason": "completed"}\n', 197 - ] 198 - ) 199 - ) 200 - 201 - manager.process = MagicMock() 202 - manager.process.stdout = mock_stdout 203 - 204 - manager._stream_stdout() 205 - 206 - assert manager._stop_event is not None 207 - assert manager._stop_event["reason"] == "completed"
-58
uv.lock
··· 2604 2604 ] 2605 2605 2606 2606 [[package]] 2607 - name = "pyobjc-core" 2608 - version = "12.1" 2609 - source = { registry = "https://pypi.org/simple" } 2610 - sdist = { url = "https://files.pythonhosted.org/packages/b8/b6/d5612eb40be4fd5ef88c259339e6313f46ba67577a95d86c3470b951fce0/pyobjc_core-12.1.tar.gz", hash = "sha256:2bb3903f5387f72422145e1466b3ac3f7f0ef2e9960afa9bcd8961c5cbf8bd21", size = 1000532, upload-time = "2025-11-14T10:08:28.292Z" } 2611 - wheels = [ 2612 - { url = "https://files.pythonhosted.org/packages/63/bf/3dbb1783388da54e650f8a6b88bde03c101d9ba93dfe8ab1b1873f1cd999/pyobjc_core-12.1-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:93418e79c1655f66b4352168f8c85c942707cb1d3ea13a1da3e6f6a143bacda7", size = 676748, upload-time = "2025-11-14T09:30:50.023Z" }, 2613 - { url = "https://files.pythonhosted.org/packages/95/df/d2b290708e9da86d6e7a9a2a2022b91915cf2e712a5a82e306cb6ee99792/pyobjc_core-12.1-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:c918ebca280925e7fcb14c5c43ce12dcb9574a33cccb889be7c8c17f3bcce8b6", size = 671263, upload-time = "2025-11-14T09:31:35.231Z" }, 2614 - { url = "https://files.pythonhosted.org/packages/64/5a/6b15e499de73050f4a2c88fff664ae154307d25dc04da8fb38998a428358/pyobjc_core-12.1-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:818bcc6723561f207e5b5453efe9703f34bc8781d11ce9b8be286bb415eb4962", size = 678335, upload-time = "2025-11-14T09:32:20.107Z" }, 2615 - { url = "https://files.pythonhosted.org/packages/f4/d2/29e5e536adc07bc3d33dd09f3f7cf844bf7b4981820dc2a91dd810f3c782/pyobjc_core-12.1-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:01c0cf500596f03e21c23aef9b5f326b9fb1f8f118cf0d8b66749b6cf4cbb37a", size = 677370, upload-time = "2025-11-14T09:33:05.273Z" }, 2616 - { url = "https://files.pythonhosted.org/packages/1b/f0/4b4ed8924cd04e425f2a07269943018d43949afad1c348c3ed4d9d032787/pyobjc_core-12.1-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:177aaca84bb369a483e4961186704f64b2697708046745f8167e818d968c88fc", size = 719586, upload-time = "2025-11-14T09:33:53.302Z" }, 2617 - { url = "https://files.pythonhosted.org/packages/25/98/9f4ed07162de69603144ff480be35cd021808faa7f730d082b92f7ebf2b5/pyobjc_core-12.1-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:844515f5d86395b979d02152576e7dee9cc679acc0b32dc626ef5bda315eaa43", size = 670164, upload-time = "2025-11-14T09:34:37.458Z" }, 2618 - { url = "https://files.pythonhosted.org/packages/62/50/dc076965c96c7f0de25c0a32b7f8aa98133ed244deaeeacfc758783f1f30/pyobjc_core-12.1-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:453b191df1a4b80e756445b935491b974714456ae2cbae816840bd96f86db882", size = 712204, upload-time = "2025-11-14T09:35:24.148Z" }, 2619 - ] 2620 - 2621 - [[package]] 2622 - name = "pyobjc-framework-cocoa" 2623 - version = "12.1" 2624 - source = { registry = "https://pypi.org/simple" } 2625 - dependencies = [ 2626 - { name = "pyobjc-core" }, 2627 - ] 2628 - sdist = { url = "https://files.pythonhosted.org/packages/02/a3/16ca9a15e77c061a9250afbae2eae26f2e1579eb8ca9462ae2d2c71e1169/pyobjc_framework_cocoa-12.1.tar.gz", hash = "sha256:5556c87db95711b985d5efdaaf01c917ddd41d148b1e52a0c66b1a2e2c5c1640", size = 2772191, upload-time = "2025-11-14T10:13:02.069Z" } 2629 - wheels = [ 2630 - { url = "https://files.pythonhosted.org/packages/b2/aa/2b2d7ec3ac4b112a605e9bd5c5e5e4fd31d60a8a4b610ab19cc4838aa92a/pyobjc_framework_cocoa-12.1-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:9b880d3bdcd102809d704b6d8e14e31611443aa892d9f60e8491e457182fdd48", size = 383825, upload-time = "2025-11-14T09:40:28.354Z" }, 2631 - { url = "https://files.pythonhosted.org/packages/3f/07/5760735c0fffc65107e648eaf7e0991f46da442ac4493501be5380e6d9d4/pyobjc_framework_cocoa-12.1-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:f52228bcf38da64b77328787967d464e28b981492b33a7675585141e1b0a01e6", size = 383812, upload-time = "2025-11-14T09:40:53.169Z" }, 2632 - { url = "https://files.pythonhosted.org/packages/95/bf/ee4f27ec3920d5c6fc63c63e797c5b2cc4e20fe439217085d01ea5b63856/pyobjc_framework_cocoa-12.1-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:547c182837214b7ec4796dac5aee3aa25abc665757b75d7f44f83c994bcb0858", size = 384590, upload-time = "2025-11-14T09:41:17.336Z" }, 2633 - { url = "https://files.pythonhosted.org/packages/ad/31/0c2e734165abb46215797bd830c4bdcb780b699854b15f2b6240515edcc6/pyobjc_framework_cocoa-12.1-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:5a3dcd491cacc2f5a197142b3c556d8aafa3963011110102a093349017705118", size = 384689, upload-time = "2025-11-14T09:41:41.478Z" }, 2634 - { url = "https://files.pythonhosted.org/packages/23/3b/b9f61be7b9f9b4e0a6db18b3c35c4c4d589f2d04e963e2174d38c6555a92/pyobjc_framework_cocoa-12.1-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:914b74328c22d8ca261d78c23ef2befc29776e0b85555973927b338c5734ca44", size = 388843, upload-time = "2025-11-14T09:42:05.719Z" }, 2635 - { url = "https://files.pythonhosted.org/packages/59/bb/f777cc9e775fc7dae77b569254570fe46eb842516b3e4fe383ab49eab598/pyobjc_framework_cocoa-12.1-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:03342a60fc0015bcdf9b93ac0b4f457d3938e9ef761b28df9564c91a14f0129a", size = 384932, upload-time = "2025-11-14T09:42:29.771Z" }, 2636 - { url = "https://files.pythonhosted.org/packages/58/27/b457b7b37089cad692c8aada90119162dfb4c4a16f513b79a8b2b022b33b/pyobjc_framework_cocoa-12.1-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:6ba1dc1bfa4da42d04e93d2363491275fb2e2be5c20790e561c8a9e09b8cf2cc", size = 388970, upload-time = "2025-11-14T09:42:53.964Z" }, 2637 - ] 2638 - 2639 - [[package]] 2640 - name = "pyobjc-framework-quartz" 2641 - version = "12.1" 2642 - source = { registry = "https://pypi.org/simple" } 2643 - dependencies = [ 2644 - { name = "pyobjc-core" }, 2645 - { name = "pyobjc-framework-cocoa" }, 2646 - ] 2647 - sdist = { url = "https://files.pythonhosted.org/packages/94/18/cc59f3d4355c9456fc945eae7fe8797003c4da99212dd531ad1b0de8a0c6/pyobjc_framework_quartz-12.1.tar.gz", hash = "sha256:27f782f3513ac88ec9b6c82d9767eef95a5cf4175ce88a1e5a65875fee799608", size = 3159099, upload-time = "2025-11-14T10:21:24.31Z" } 2648 - wheels = [ 2649 - { url = "https://files.pythonhosted.org/packages/17/f4/50c42c84796886e4d360407fb629000bb68d843b2502c88318375441676f/pyobjc_framework_quartz-12.1-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:c6f312ae79ef8b3019dcf4b3374c52035c7c7bc4a09a1748b61b041bb685a0ed", size = 217799, upload-time = "2025-11-14T09:59:32.62Z" }, 2650 - { url = "https://files.pythonhosted.org/packages/b7/ef/dcd22b743e38b3c430fce4788176c2c5afa8bfb01085b8143b02d1e75201/pyobjc_framework_quartz-12.1-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:19f99ac49a0b15dd892e155644fe80242d741411a9ed9c119b18b7466048625a", size = 217795, upload-time = "2025-11-14T09:59:46.922Z" }, 2651 - { url = "https://files.pythonhosted.org/packages/e9/9b/780f057e5962f690f23fdff1083a4cfda5a96d5b4d3bb49505cac4f624f2/pyobjc_framework_quartz-12.1-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:7730cdce46c7e985535b5a42c31381af4aa6556e5642dc55b5e6597595e57a16", size = 218798, upload-time = "2025-11-14T10:00:01.236Z" }, 2652 - { url = "https://files.pythonhosted.org/packages/ba/2d/e8f495328101898c16c32ac10e7b14b08ff2c443a756a76fd1271915f097/pyobjc_framework_quartz-12.1-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:629b7971b1b43a11617f1460cd218bd308dfea247cd4ee3842eb40ca6f588860", size = 219206, upload-time = "2025-11-14T10:00:15.623Z" }, 2653 - { url = "https://files.pythonhosted.org/packages/67/43/b1f0ad3b842ab150a7e6b7d97f6257eab6af241b4c7d14cb8e7fde9214b8/pyobjc_framework_quartz-12.1-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:53b84e880c358ba1ddcd7e8d5ea0407d760eca58b96f0d344829162cda5f37b3", size = 224317, upload-time = "2025-11-14T10:00:30.703Z" }, 2654 - { url = "https://files.pythonhosted.org/packages/4a/00/96249c5c7e5aaca5f688ca18b8d8ad05cd7886ebd639b3c71a6a4cadbe75/pyobjc_framework_quartz-12.1-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:42d306b07f05ae7d155984503e0fb1b701fecd31dcc5c79fe8ab9790ff7e0de0", size = 219558, upload-time = "2025-11-14T10:00:45.476Z" }, 2655 - { url = "https://files.pythonhosted.org/packages/4d/a6/708a55f3ff7a18c403b30a29a11dccfed0410485a7548c60a4b6d4cc0676/pyobjc_framework_quartz-12.1-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:0cc08fddb339b2760df60dea1057453557588908e42bdc62184b6396ce2d6e9a", size = 224580, upload-time = "2025-11-14T10:01:00.091Z" }, 2656 - ] 2657 - 2658 - [[package]] 2659 2607 name = "pypdf" 2660 2608 version = "6.7.0" 2661 2609 source = { registry = "https://pypi.org/simple" } ··· 3603 3551 { name = "playwright" }, 3604 3552 { name = "psutil" }, 3605 3553 { name = "pygobject", marker = "sys_platform == 'linux'" }, 3606 - { name = "pyobjc-core", marker = "sys_platform == 'darwin'" }, 3607 - { name = "pyobjc-framework-cocoa", marker = "sys_platform == 'darwin'" }, 3608 - { name = "pyobjc-framework-quartz", marker = "sys_platform == 'darwin'" }, 3609 3554 { name = "pypdf" }, 3610 3555 { name = "pytesseract" }, 3611 3556 { name = "pytest" }, ··· 3653 3598 { name = "playwright", specifier = ">=1.40.0" }, 3654 3599 { name = "psutil" }, 3655 3600 { name = "pygobject", marker = "sys_platform == 'linux'" }, 3656 - { name = "pyobjc-core", marker = "sys_platform == 'darwin'" }, 3657 - { name = "pyobjc-framework-cocoa", marker = "sys_platform == 'darwin'" }, 3658 - { name = "pyobjc-framework-quartz", marker = "sys_platform == 'darwin'" }, 3659 3601 { name = "pypdf" }, 3660 3602 { name = "pytesseract" }, 3661 3603 { name = "pytest" },