Ionosphere.tv
3
fork

Configure Feed

Select the types of activity you want to include in your feed.

docs: migrate wholesale to layers.pub, drop tv.ionosphere.annotation

No coexistence period — the pipeline rebuilds from raw, so we replace
tv.ionosphere.annotation with layers.pub annotation layers in Phase 1.
Old enrichment code (enrich.ts, overlayAnnotations) removed up front.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

+8 -5
+8 -5
docs/superpowers/specs/2026-04-12-transcript-formatting-design.md
··· 58 58 59 59 A separate annotation layer with `sourceMethod: "manual-native"` and higher `rank`. The merge step prefers higher-ranked layers. Example: correcting "Blue Sky" to link to the Bluesky concept record is an annotation in this layer that supersedes the auto-detected concept. Published as first-class AT Protocol records — auditable, attributable, and preservable across pipeline re-runs. 60 60 61 - ### Relationship to existing `tv.ionosphere.annotation` 61 + ### Replacing `tv.ionosphere.annotation` 62 62 63 - The existing `tv.ionosphere.annotation` record type (concept mentions anchored to byte ranges) continues to work as-is for Phase 1. The NLP pipeline produces layers.pub annotation layers for structural annotations (sentences, paragraphs) — these are a different concern and do not conflict. 63 + The existing `tv.ionosphere.annotation` record type (concept mentions anchored to byte ranges) is replaced wholesale by layers.pub annotation layers. Since the entire pipeline rebuilds from raw transcripts, there is no migration burden — the next pipeline run produces layers.pub records instead of `tv.ionosphere.annotation` records, and the old records are deleted. 64 64 65 - In Phase 2, when we add NLP-based concept/entity detection, the layers.pub annotation system becomes the canonical source for all enrichment annotations. At that point, existing `tv.ionosphere.annotation` records are migrated to layers.pub annotation layers via a panproto migration. The appview indexer reads both formats during the transition period. 65 + The existing concept data is re-derived by the NLP pipeline as a concept annotation layer (Phase 2), which will produce better results than the current approach. The `tv.ionosphere.annotation` lexicon and related code (`enrich.ts`, `enrich-all.ts`, `overlayAnnotations` in the appview) are removed in Phase 1. 66 66 67 67 ### Panproto Integration 68 68 ··· 252 252 253 253 ## Phase Roadmap 254 254 255 - ### Phase 1 — Structural formatting (this work) 255 + ### Phase 1 — Structural formatting + layers.pub migration (this work) 256 256 257 + - Vendor layers.pub lexicon definitions into `lexicons/pub/layers/` 257 258 - Python NLP pipeline: sentence detection (spaCy) + paragraph segmentation (pause + sentence alignment) 258 259 - layers.pub expression + segmentation records for each transcript 259 260 - Sentence and paragraph annotation layers 260 261 - Panproto lenses: compact transcript <-> layers.pub expression + segmentation 261 262 - Document assembly reads annotation layers, emits structural facets 262 263 - Renderer: sentences as inline spans, paragraphs as block elements 263 - - **Goal:** Transcripts read as paragraphed prose 264 + - Remove `tv.ionosphere.annotation` records, `enrich.ts`/`enrich-all.ts`, `overlayAnnotations` — fully replaced by layers.pub 265 + - **Goal:** Transcripts read as paragraphed prose; all enrichment flows through layers.pub 264 266 265 267 ### Phase 2 — Entity recognition + record linking 266 268 267 269 - spaCy NER pass in the pipeline 268 270 - AT Protocol record resolver: people -> Bluesky profiles (DID resolution via handle/display name lookup), projects -> `tv.ionosphere.concept` records 271 + - Concept annotation layer replaces the old `tv.ionosphere.annotation`-based concept system with richer, NLP-derived results 269 272 - Entity annotation layer with `knowledgeRefs` to resolved records 270 273 - Renderer: entity spans as links/tooltips to profiles and concept pages 271 274 - **Goal:** People and projects mentioned in talks are clickable, linked to real AT Protocol identities