tranquil.farm / tranquil-pds
Our Personal Data Server from scratch! tranquil.farm
pds rust database fun oauth atproto
238
fork

Configure Feed

Select the types of activity you want to include in your feed.

adding k8s docs and a more generalized doc #12

open opened by
dsx.sh targeting main from dsx.sh/tranquil-pds: k8s-doc-updates

I recently went through some pain in getting Tranquil all set up on my k8s cluster. Luckily some of my 'biggest' pain was in the mystery of the email server, which has been resolved in the meantime.

Regardless, I added a bunch more detail on a somewhat generic k8s deployment, with some context for secrets, secret generation, etc.

I also set up a "general configuration" doc that took some of the details and context from all the install-x docs and compiled it together into a more generalized information source with all the env vars I could find centralized into one place.

Sorry if this kind of work is already being done elsewhere, I wanted to get this somewhere so I can point friends to so they do their own PDS.

Labels

None yet.

assignee

None yet.

Participants 1
AT URI
at://did:plc:6so2rykrnjmzebbr2zbwbqbx/sh.tangled.repo.pull/3mlcn4ezr6p22
+239 -11
Diff #2
+211 -7
docs/install-kubernetes.md
··· 1 - # Tranquil PDS on kubernetes 1 + # Tranquil PDS on Kubernetes 2 2 3 - If you're reaching for kubernetes for this app, you're experienced enough to know how to spin up: 3 + If you're reaching for Kubernetes for this app, you're experienced enough to know how to spin up: 4 4 5 5 - cloudnativepg (or your preferred postgres operator) 6 6 - a PersistentVolume for blob storage 7 7 - the app itself (it's just a container with some env vars) 8 8 9 - You'll need a wildcard TLS certificate for `*.your-pds-hostname.example.com`. User handles are served as subdomains. 9 + See `example.toml` for what each config option does and why the secret ones matter. This guide covers the Kubernetes-specific wiring. 10 10 11 - The container image expects: 11 + Minimally, the container image expects: 12 12 - A TOML config file mounted at `/etc/tranquil-pds/config.toml` (or passed via `--config`) 13 13 - `DATABASE_URL` - postgres connection string 14 14 - `BLOB_STORAGE_PATH` - path to blob storage (mount a PV here) ··· 16 16 - `JWT_SECRET`, `DPOP_SECRET`, `MASTER_KEY` - generate with `openssl rand -base64 48` 17 17 - `CRAWLERS` - typically `https://bsky.network` 18 18 19 - and more, check the example.toml for all options. Environment variables can override any TOML value. 20 - You can also point to a config file via the `TRANQUIL_PDS_CONFIG` env var. 19 + --- 21 20 22 - Health check: `GET /xrpc/_health` 21 + ## TLS and DNS 22 + 23 + You need a wildcard TLS cert covering `*.your-pds-hostname.example.com`. User handles are subdomains, so every handle needs a matching SAN. 24 + 25 + An approach using Cert Manager would look something like this: 26 + 27 + ```yaml 28 + apiVersion: cert-manager.io/v1 29 + kind: Certificate 30 + metadata: 31 + name: pds-cert 32 + namespace: pds 33 + spec: 34 + secretName: pds-tls 35 + dnsNames: 36 + - pds.example.com 37 + - "*.pds.example.com" 38 + ``` 39 + 40 + If you're using the apex domain for handles, include it in `dnsNames` alongside the wildcard. 41 + 42 + --- 43 + 44 + ## Secrets 45 + 46 + The three primary key secrets (`JWT_SECRET`, `DPOP_SECRET`, `MASTER_KEY`) must never appear in a manifest or config file. Inject them as a Kubernetes Secret, sourced from wherever you manage secrets. 47 + 48 + To create the Secret directly and manage rotation manually: 49 + 50 + ```bash 51 + kubectl create secret generic pds-secrets -n pds \ 52 + --from-literal=DATABASE_URL='postgres://...' \ 53 + --from-literal=JWT_SECRET="$(openssl rand -base64 48)" \ 54 + --from-literal=DPOP_SECRET="$(openssl rand -base64 48)" \ 55 + --from-literal=MASTER_KEY="$(openssl rand -base64 48)" 56 + ``` 57 + 58 + Then reference it in your Deployment: 59 + 60 + ```yaml 61 + envFrom: 62 + - secretRef: 63 + name: pds-secrets 64 + ``` 65 + 66 + --- 67 + 68 + ## PostgreSQL 69 + 70 + CloudNativePG works well here. Example cluster config: 71 + 72 + ```yaml 73 + apiVersion: postgresql.cnpg.io/v1 74 + kind: Cluster 75 + metadata: 76 + name: postgres 77 + namespace: pds 78 + spec: 79 + instances: 1 80 + bootstrap: 81 + initdb: 82 + database: pds 83 + owner: tranquil_pds 84 + secret: 85 + name: postgres-user-secret # k8s Secret with username/password 86 + storage: 87 + storageClass: your-storage-class 88 + size: 10Gi 89 + ``` 90 + 91 + The `postgres-user-secret` Secret needs `username` and `password` keys. The password you put here is what goes into `DATABASE_URL`. 92 + 93 + Any standard Postgres setup works fine here. 94 + 95 + --- 96 + 97 + ## Storage PVC (if using local-path, nfs, or similar) 98 + 99 + ```yaml 100 + apiVersion: v1 101 + kind: PersistentVolumeClaim 102 + metadata: 103 + name: pds-blobs 104 + namespace: pds 105 + spec: 106 + storageClassName: your-storage-class 107 + accessModes: 108 + - ReadWriteOnce 109 + resources: 110 + requests: 111 + storage: 50Gi 112 + ``` 113 + 114 + Full Deployment example: 115 + 116 + ```yaml 117 + apiVersion: apps/v1 118 + kind: Deployment 119 + metadata: 120 + name: pds 121 + namespace: pds 122 + spec: 123 + replicas: 1 124 + selector: 125 + matchLabels: 126 + app: pds 127 + strategy: 128 + type: Recreate 129 + template: 130 + metadata: 131 + labels: 132 + app: pds 133 + spec: 134 + containers: 135 + - name: pds 136 + image: atcr.io/tranquil.farm/tranquil-pds:latest 137 + ports: 138 + - name: http 139 + containerPort: 3000 140 + env: 141 + - name: SERVER_HOST 142 + value: "0.0.0.0" 143 + - name: SERVER_PORT 144 + value: "3000" 145 + - name: PDS_HOSTNAME 146 + value: "pds.example.com" 147 + - name: PDS_USER_HANDLE_DOMAINS 148 + value: "example.com" 149 + - name: BLOB_STORAGE_BACKEND 150 + value: filesystem 151 + - name: BLOB_STORAGE_PATH 152 + value: /var/lib/tranquil/blobs 153 + - name: CRAWLERS 154 + value: "https://bsky.network" 155 + - name: INVITE_CODE_REQUIRED 156 + value: "true" 157 + envFrom: 158 + - secretRef: 159 + name: pds-secrets 160 + volumeMounts: 161 + - mountPath: /var/lib/tranquil/blobs 162 + name: blobs 163 + readinessProbe: 164 + httpGet: 165 + path: /xrpc/_health 166 + port: http 167 + initialDelaySeconds: 15 168 + periodSeconds: 10 169 + failureThreshold: 6 170 + resources: 171 + requests: 172 + memory: 256Mi 173 + cpu: 100m 174 + limits: 175 + memory: 1Gi 176 + volumes: 177 + - name: blobs 178 + persistentVolumeClaim: 179 + claimName: pds-blobs 180 + ``` 181 + 182 + `SERVER_HOST: "0.0.0.0"` is required. The default `127.0.0.1` won't be reachable by the Kubelet or your ingress controller. 183 + 184 + --- 185 + 186 + ## Ingress 187 + 188 + The ingress needs rules for both the PDS hostname and the wildcard for user handles. 189 + 190 + ```yaml 191 + apiVersion: networking.k8s.io/v1 192 + kind: Ingress 193 + metadata: 194 + name: pds 195 + namespace: pds 196 + spec: 197 + tls: 198 + - hosts: 199 + - pds.example.com 200 + - "*.example.com" 201 + secretName: pds-tls 202 + rules: 203 + - host: pds.example.com 204 + http: 205 + paths: 206 + - path: / 207 + pathType: Prefix 208 + backend: 209 + service: 210 + name: pds 211 + port: 212 + number: 3000 213 + - host: "*.example.com" 214 + http: 215 + paths: 216 + - path: / 217 + pathType: Prefix 218 + backend: 219 + service: 220 + name: pds 221 + port: 222 + number: 3000 223 + ``` 224 + 225 + --- 23 226 24 227 ## Custom homepage 25 228 26 229 Mount a ConfigMap with your `homepage.html` into the container's frontend directory and it becomes your landing page. Go nuts with it. Account dashboard is at `/app/` so you won't break anything. 27 230 231 + 28 232 ```yaml 29 233 apiVersion: v1
+28 -4
example.toml
··· 1 + # Configuration is loaded in this order: 2 + # 3 + # 1. Environment variables (highest priority, always win) 4 + # 2. A custom config file, either passed with --config or the file 5 + # referenced in the TRANQUIL_PDS_CONFIG environment variable 6 + # 3. /etc/tranquil-pds/config.toml (always loaded as a base, even when 7 + # a custom config path is specified) 8 + # 9 + # Useful commands: 10 + # tranquil-pds validate - validate your config without starting 11 + # tranquil-pds config-template - generate a commented example.toml 12 + 1 13 [server] 2 14 # Public hostname of the PDS, such as `pds.example.com`. 3 15 # ··· 142 154 #acquire_timeout_secs = 10 143 155 144 156 [secrets] 145 - # Secret used for signing JWTs. Must be at least 32 characters in 157 + # Secret used for signing session JWTs. Must be at least 32 characters in 146 158 # production. 147 159 # 148 160 # Can also be specified via environment variable `JWT_SECRET`. ··· 355 367 # Default value: 4 356 368 #max_concurrent_repo_exports = 4 357 369 358 - # List of relay / crawler notification URLs. 370 + # List of relay / crawler notification URLs. Notified when new events are 371 + # committed to an account's repo. 372 + # 373 + # Defaults to [ "https://bsky.network" ] when unset. 359 374 # 360 375 # Can also be specified via environment variable `CRAWLERS`. 361 376 #crawlers = 362 377 363 378 [email] 364 - # Sender email address. When unset, email sending is disabled. 379 + # Sender email address. When unset, email sending is disabled entirely. 380 + # 381 + # Email is optional, but at least one comms method (email, Discord, 382 + # Telegram, or Signal) must be set up for account verification, 383 + # password resets, and 2FA backup codes to work. 365 384 # 366 385 # Can also be specified via environment variable `MAIL_FROM_ADDRESS`. 367 386 #from_address = ··· 469 488 #require_tls = false 470 489 471 490 [email.dkim] 472 - # DKIM selector. When unset, outgoing mail is not signed. 491 + # DKIM signing configuration. 492 + # 493 + # The corresponding DNS TXT record at <selector>._domainkey.<domain> must 494 + # be published before mail is sent. 495 + # 496 + # DKIM selector. When unset, outgoing mail is not DKIM-signed. 473 497 # 474 498 # Can also be specified via environment variable `MAIL_DKIM_SELECTOR`. 475 499 #selector =

History

4 rounds 0 comments
sign up or login to add to the discussion
dsx.sh submitted #3
diff interdiff
6 commits
expand
863d8480
adding k8s docs and a more generalized doc
d9c99c04
oops didnt actually mean to take that out
44c7255f
removing gen config altogether, agree with nel
697cf8c2
expanding example.toml a tiny bit and reworking k8s a little
541abaf4
rewording, and some sentence simplification
30e09e90
more rewording
merge conflicts detected
expand
expand 0 comments
dsx.sh submitted #2
interdiff
5 commits
expand
863d8480
adding k8s docs and a more generalized doc
d9c99c04
oops didnt actually mean to take that out
44c7255f
removing gen config altogether, agree with nel
697cf8c2
expanding example.toml a tiny bit and reworking k8s a little
541abaf4
rewording, and some sentence simplification
expand 0 comments
dsx.sh submitted #1
diff interdiff
2 commits
expand
863d8480
adding k8s docs and a more generalized doc
d9c99c04
oops didnt actually mean to take that out
expand 0 comments
dsx.sh submitted #0
diff
1 commit
expand
863d8480
adding k8s docs and a more generalized doc
expand 0 comments