zzstoatzz.io / my-prefect-server
my prefect server setup prefect-metrics.waow.tech
python orchestration
0
fork

Configure Feed

Select the types of activity you want to include in your feed.

Python 74.6%
Svelte 11.7%
TypeScript 7.2%
Just 4.2%
HCL 1.4%
JavaScript 0.3%
CSS 0.2%
HTML 0.2%
Dockerfile 0.1%
136 1 0

Clone this repository

https://tangled.org/zzstoatzz.io/my-prefect-server https://tangled.org/did:plc:xbtmt2zjwlrfegqvch7fboei/my-prefect-server
git@tangled.org:zzstoatzz.io/my-prefect-server git@tangled.org:did:plc:xbtmt2zjwlrfegqvch7fboei/my-prefect-server

For self-hosted knots, clone URLs may differ based on your setup.

Download tar.gz
README.md

personal data pipeline and intelligence layer. digests github, tangled.org, and bluesky activity, scores items, generates LLM-curated briefings, and maintains phi's long-term memory. self-hosted on a single hetzner VM (k3s) running prefect OSS.

hub · grafana

                        data sources
  ─────────────────────────────────────────────
  github API        ──┐
  tangled PDS       ──┤
  bluesky likes     ──┼──► ingest (hourly) ──► DuckDB
  phi memory (tpuf) ──┘
                                                  │
                                                  ▼
                                          transform (dbt)
                                          [on ingest ✓]
                                                  │
                        ┌─────────────────────────┼──────────┐
                        ▼                         ▼          ▼
                      brief                    compact    hub UI
                  [on transform ✓]         [on transform ✓]
                        │                         │
                        ▼                         ▼
                  briefing.json              TurboPuffer
                                             (phi-users-*)

                        standalone flows
  ─────────────────────────────────────────────
  morning (daily 8am CT) ──► TurboPuffer + semble
  rebuild-atlas (every 6h) ──► Cloudflare Pages

see docs/hub.md for the full pipeline breakdown.

deployment

prerequisites#

  • terraform
  • just
  • uv (Python 3.14+)
  • a hetzner cloud API token
  • a domain with DNS you control

setup#

cp .env.example .env
# fill in: HCLOUD_TOKEN, POSTGRES_PASSWORD, AUTH_STRING, DOMAIN, LETSENCRYPT_EMAIL
uv sync                # install workspace (mps + root)

deploy#

just init              # terraform init
just infra             # create the VM
just kubeconfig        # wait for k3s, fetch kubeconfig
just deploy            # cert-manager, prefect server, monitoring, dashboards
just worker            # deploy the kubernetes worker
just storage           # create analytics hostPath + results PVC, patch work pool

after deploy, point your DNS:

  • $DOMAIN → server IP (just server-ip)
  • $GRAFANA_DOMAIN → same IP (default: prefect-metrics.waow.tech)
  • hub.waow.tech → same IP

verify#

just health               # curl the /api/health endpoint
just status               # node + pod resource usage
just prefect work-pool ls

operations#

just logs                 # tail prefect-server logs (default)
just logs worker          # tail worker logs
just prefect flow-run ls  # run any prefect CLI command remotely
just dashboards           # reload grafana dashboards from deploy/dashboards/
just ssh                  # ssh into the server

flow deployments are registered automatically on every push to main via .tangled/workflows/deploy.yml.

hub (sveltekit frontend)#

just web    # build + push + deploy hub.waow.tech

analytics (dbt + duckdb)#

just init-analytics   # first-time: dbt deps, seed, compile