my prefect server setup prefect-metrics.waow.tech
python orchestration
0
fork

Configure Feed

Select the types of activity you want to include in your feed.

docs: clean up pipeline diagrams

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

+46 -45
+23 -22
README.md
··· 3 3 [hub](https://hub.waow.tech) · [grafana](https://prefect-metrics.waow.tech/d/executive-overview/executive-overview?orgId=1&from=now-6h&to=now&timezone=browser) 4 4 5 5 ``` 6 - github API ──┐ 7 - ├──► ingest ──► raw_github_issues ──┐ 8 - tangled PDS ─┤ (hourly) raw_tangled_items ─┤ 9 - bluesky PDS ─┤ raw_likes + raw_liked_posts │ 10 - phi (tpuf) ─┘ raw_phi_observations ─┘ 11 - 12 - transform (dbt) ◄──────┘ 13 - [on ingest ✓] 14 - 15 - ┌───────────────┼───────────────┐ 16 - ▼ ▼ ▼ 17 - brief compact hub UI 18 - [on transform ✓] [on transform ✓] 19 - │ │ 20 - ▼ ▼ 21 - briefing.json TurboPuffer 22 - (phi-users-*) 6 + data sources 7 + ───────────────────────────────────────────── 8 + github API ──┐ 9 + tangled PDS ──┤ 10 + bluesky likes ──┼──► ingest (hourly) ──► DuckDB 11 + phi memory (tpuf) ──┘ 12 + 13 + 14 + transform (dbt) 15 + [on ingest ✓] 16 + 17 + ┌─────────────────────────┼──────────┐ 18 + ▼ ▼ ▼ 19 + brief compact hub UI 20 + [on transform ✓] [on transform ✓] 21 + │ │ 22 + ▼ ▼ 23 + briefing.json TurboPuffer 24 + (phi-users-*) 23 25 24 - morning ──► TurboPuffer + semble 25 - (daily 8am CT) 26 - 27 - rebuild-atlas ──► Cloudflare Pages 28 - (every 6h) 26 + standalone flows 27 + ───────────────────────────────────────────── 28 + morning (daily 8am CT) ──► TurboPuffer + semble 29 + rebuild-atlas (every 6h) ──► Cloudflare Pages 29 30 ``` 30 31 31 32 see [docs/hub.md](docs/hub.md) for the full pipeline breakdown.
+23 -23
docs/hub.md
··· 17 17 ## pipeline 18 18 19 19 ``` 20 - github API ──┐ 21 - ├──► ingest ──► raw_github_issues ──┐ 22 - tangled PDS ─┤ (hourly) raw_tangled_items ─┤ 23 - bluesky PDS ─┤ raw_likes + raw_liked_posts │ 24 - phi (tpuf) ─┘ raw_phi_observations ─┘ 25 - 26 - transform (dbt) ◄──────┘ 27 - [on ingest ✓] 28 - 29 - ┌───────────────┼───────────────┐ 30 - ▼ ▼ ▼ 31 - brief compact hub UI 32 - [on transform ✓] [on transform ✓] /api/cards.json 33 - │ │ +page.svelte 34 - ▼ ▼ 35 - briefing.json TurboPuffer 36 - /api/briefing (phi-users-*) 37 - ``` 38 - 39 - two additional flows run independently: 20 + data sources 21 + ───────────────────────────────────────────── 22 + github API ──┐ 23 + tangled PDS ──┤ 24 + bluesky likes ──┼──► ingest (hourly) ──► DuckDB 25 + phi memory (tpuf) ──┘ 26 + 27 + 28 + transform (dbt) 29 + [on ingest ✓] 30 + 31 + ┌─────────────────────────┼──────────┐ 32 + ▼ ▼ ▼ 33 + brief compact hub UI 34 + [on transform ✓] [on transform ✓] 35 + │ │ 36 + ▼ ▼ 37 + briefing.json TurboPuffer 38 + /api/briefing (phi-users-*) 40 39 41 - ``` 42 - morning (daily 8am CT) ──► TurboPuffer (tag maintenance) + semble (curation) 43 - rebuild-atlas (every 6h) ──► Cloudflare Pages (leaflet-search atlas) 40 + standalone flows 41 + ───────────────────────────────────────────── 42 + morning (daily 8am CT) ──► TurboPuffer + semble 43 + rebuild-atlas (every 6h) ──► Cloudflare Pages 44 44 ``` 45 45 46 46 ## flows