my prefect server setup
prefect-metrics.waow.tech
python
orchestration
clean up repo: add license, rewrite readme, remove dead terraform
- add MIT license
- rewrite README to lead with the pipeline, not the infrastructure
- remove unused prefect/ terraform directory (superseded by scripts/patch_work_pool.py)
- note ANTHROPIC_API_KEY in .env.example (it's a prefect secret block)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
+44
-39
+1
.env.example
+1
.env.example
+20
LICENSE
+20
LICENSE
···
1
+
MIT License
2
+
3
+
Copyright 2025 Nate Wilkins
4
+
5
+
Permission is hereby granted, free of charge, to any person obtaining a copy of
6
+
this software and associated documentation files (the "Software"), to deal in
7
+
the Software without restriction, including without limitation the rights to
8
+
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
9
+
the Software, and to permit persons to whom the Software is furnished to do so,
10
+
subject to the following conditions:
11
+
12
+
The above copyright notice and this permission notice shall be included in all
13
+
copies or substantial portions of the Software.
14
+
15
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
17
+
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
18
+
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
19
+
AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
20
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+23
-3
README.md
+23
-3
README.md
···
1
-
self-hosted prefect OSS on a single hetzner VM (k3s), with monitoring.
1
+
personal data pipeline that digests my github and [tangled.org](https://tangled.org) activity, scores items by importance, and generates an LLM-curated briefing. self-hosted on a single hetzner VM (k3s) running prefect OSS.
2
2
3
-
[executive dashboard](https://prefect-metrics.waow.tech/d/executive-overview/executive-overview?orgId=1&from=now-6h&to=now&timezone=browser) · [hub](https://hub.waow.tech)
3
+
[hub](https://hub.waow.tech) · [grafana](https://prefect-metrics.waow.tech/d/executive-overview/executive-overview?orgId=1&from=now-6h&to=now&timezone=browser)
4
4
5
-
see [docs/hub.md](docs/hub.md) for how the data pipeline and hub frontend work.
5
+
```
6
+
github API ──► gh-notifications ──► raw_github_issues ──┐
7
+
(hourly :00) │
8
+
▼
9
+
enrich (dbt, :05)
10
+
│
11
+
tangled PDS ──► tangled-items ───► raw_tangled_items ────┘
12
+
(hourly :02) │
13
+
▼
14
+
hub_action_items
15
+
(top 200)
16
+
│
17
+
┌──────────────┼──────────┐
18
+
▼ ▼ ▼
19
+
curate (:10) /api/cards hub UI
20
+
│
21
+
▼
22
+
briefing.json
23
+
```
24
+
25
+
see [docs/hub.md](docs/hub.md) for the full pipeline breakdown.
6
26
7
27
<details>
8
28
<summary>deployment</summary>
-16
prefect/main.tf
-16
prefect/main.tf
···
1
-
provider "prefect" {
2
-
endpoint = "https://${var.domain}/api"
3
-
basic_auth_key = var.auth_string
4
-
}
5
-
6
-
# kubernetes work pool — each flow run gets its own pod
7
-
resource "prefect_work_pool" "k8s" {
8
-
name = "kubernetes-pool"
9
-
type = "kubernetes"
10
-
}
11
-
12
-
# environment identifier
13
-
resource "prefect_variable" "environment" {
14
-
name = "environment"
15
-
value = "dogfood"
16
-
}
-10
prefect/variables.tf
-10
prefect/variables.tf