small gleam coding and persistent agent daemon with a detachable cli and a canvas ui albedo.sh

Evaluate zstd compression for session payloads #10

Open aly.codes
Open opened by aly.codes

Investigate compression of older, text-heavy session payloads; benchmark size, read latency, and migration/recovery on a copy of a real database. Keep image BLOBs and indexed columns uncompressed.

Research:

  • sqlite-zstd offers column-level transparent compression via views and incremental maintenance. It is LGPL-3.0; requires loading a native extension on each SQLite connection, has view/DDL/ATTACH limitations, and lacks a verified macOS arm64 release. Check whether sqlight exposes a safe load-extension path and whether we can ship/test native binaries on macOS/Linux.
  • sqlite_zstd_vfs compresses the whole file but does not support WAL and requires exclusive locking; not suitable for our live DB.
  • Compare app-level versioned zstd BLOBs (no SQLite plugin dependency) with column-level sqlite-zstd. Only add compression if measurements justify operational cost. VACUUM is required to reclaim pages after conversion.
Sign up or Sign in to add to the discussion.