Ionosphere.tv
docs: migrate wholesale to layers.pub, drop tv.ionosphere.annotation
No coexistence period — the pipeline rebuilds from raw, so we replace
tv.ionosphere.annotation with layers.pub annotation layers in Phase 1.
Old enrichment code (enrich.ts, overlayAnnotations) removed up front.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
+8
-5
+8
-5
docs/superpowers/specs/2026-04-12-transcript-formatting-design.md
+8
-5
docs/superpowers/specs/2026-04-12-transcript-formatting-design.md
···
58
58
59
59
A separate annotation layer with `sourceMethod: "manual-native"` and higher `rank`. The merge step prefers higher-ranked layers. Example: correcting "Blue Sky" to link to the Bluesky concept record is an annotation in this layer that supersedes the auto-detected concept. Published as first-class AT Protocol records — auditable, attributable, and preservable across pipeline re-runs.
60
60
61
-
### Relationship to existing `tv.ionosphere.annotation`
61
+
### Replacing `tv.ionosphere.annotation`
62
62
63
-
The existing `tv.ionosphere.annotation` record type (concept mentions anchored to byte ranges) continues to work as-is for Phase 1. The NLP pipeline produces layers.pub annotation layers for structural annotations (sentences, paragraphs) — these are a different concern and do not conflict.
63
+
The existing `tv.ionosphere.annotation` record type (concept mentions anchored to byte ranges) is replaced wholesale by layers.pub annotation layers. Since the entire pipeline rebuilds from raw transcripts, there is no migration burden — the next pipeline run produces layers.pub records instead of `tv.ionosphere.annotation` records, and the old records are deleted.
64
64
65
-
In Phase 2, when we add NLP-based concept/entity detection, the layers.pub annotation system becomes the canonical source for all enrichment annotations. At that point, existing `tv.ionosphere.annotation` records are migrated to layers.pub annotation layers via a panproto migration. The appview indexer reads both formats during the transition period.
65
+
The existing concept data is re-derived by the NLP pipeline as a concept annotation layer (Phase 2), which will produce better results than the current approach. The `tv.ionosphere.annotation` lexicon and related code (`enrich.ts`, `enrich-all.ts`, `overlayAnnotations` in the appview) are removed in Phase 1.
66
66
67
67
### Panproto Integration
68
68
···
252
252
253
253
## Phase Roadmap
254
254
255
-
### Phase 1 — Structural formatting (this work)
255
+
### Phase 1 — Structural formatting + layers.pub migration (this work)
256
256
257
+
- Vendor layers.pub lexicon definitions into `lexicons/pub/layers/`
257
258
- Python NLP pipeline: sentence detection (spaCy) + paragraph segmentation (pause + sentence alignment)
258
259
- layers.pub expression + segmentation records for each transcript
259
260
- Sentence and paragraph annotation layers
260
261
- Panproto lenses: compact transcript <-> layers.pub expression + segmentation
261
262
- Document assembly reads annotation layers, emits structural facets
262
263
- Renderer: sentences as inline spans, paragraphs as block elements
263
-
- **Goal:** Transcripts read as paragraphed prose
264
+
- Remove `tv.ionosphere.annotation` records, `enrich.ts`/`enrich-all.ts`, `overlayAnnotations` — fully replaced by layers.pub
265
+
- **Goal:** Transcripts read as paragraphed prose; all enrichment flows through layers.pub
264
266
265
267
### Phase 2 — Entity recognition + record linking
266
268
267
269
- spaCy NER pass in the pipeline
268
270
- AT Protocol record resolver: people -> Bluesky profiles (DID resolution via handle/display name lookup), projects -> `tv.ionosphere.concept` records
271
+
- Concept annotation layer replaces the old `tv.ionosphere.annotation`-based concept system with richer, NLP-derived results
269
272
- Entity annotation layer with `knowledgeRefs` to resolved records
270
273
- Renderer: entity spans as links/tooltips to profiles and concept pages
271
274
- **Goal:** People and projects mentioned in talks are clickable, linked to real AT Protocol identities