Constellation, Spacedust, Slingshot, UFOs: atproto crates and services for microcosm
cozy gateway: replace cady with nginx
+68
-2
+60
-2
cozy-setup (move to another repo).md
+60
-2
cozy-setup (move to another repo).md
···
167
167
168
168
todo: get raspi vcgencmd outputs into metrics
169
169
170
+
- nginx on gateway
171
+
172
+
```nginx
173
+
# in http
174
+
175
+
##
176
+
# cozy cache
177
+
##
178
+
proxy_cache_path /var/cache/nginx keys_zone=cozy_zone:10m;
179
+
180
+
##
181
+
# cozy limit
182
+
##
183
+
limit_req_zone $binary_remote_addr zone=cozy_ip_limit:10m rate=50r/s;
184
+
limit_req_zone $server_name zone=cozy_global_limit:10m rate=1000r/s;
185
+
186
+
# in sites-available/linsks.bsky.bad-example.com
187
+
188
+
upstream cozy_link_aggregator {
189
+
server link-aggregator:6789;
190
+
keepalive 16;
191
+
}
192
+
193
+
server {
194
+
listen 8080;
195
+
listen [::]:8080;
196
+
197
+
server_name links.bsky.bad-example.com;
198
+
199
+
proxy_cache cozy_zone;
200
+
proxy_cache_background_update on;
201
+
proxy_cache_key "$scheme$proxy_host$uri$is_args$args$http_accept";
202
+
proxy_cache_lock on; # make simlutaneous requests for the same uri wait for it to appear in cache instead of hitting origin
203
+
proxy_cache_lock_age 1s;
204
+
proxy_cache_lock_timeout 2s;
205
+
proxy_cache_valid 10s; # default -- should be explicitly set in the response headers
206
+
proxy_cache_valid any 15s; # non-200s default
207
+
proxy_read_timeout 5s;
208
+
proxy_send_timeout 15s;
209
+
proxy_socket_keepalive on;
210
+
211
+
limit_req zone=cozy_ip_limit nodelay burst=100;
212
+
limit_req zone=cozy_global_limit;
213
+
limit_req_status 429;
214
+
215
+
location / {
216
+
proxy_pass http://cozy_link_aggregator;
217
+
include proxy_params;
218
+
proxy_http_version 1.1;
219
+
proxy_set_header Connection ""; # for keepalive
220
+
}
221
+
}
222
+
```
223
+
170
224
---
171
225
172
226
some todos
···
179
233
- [x] configure systemd to keep it alive
180
234
- [x] configure caddy cache
181
235
- [x] configure caddy rate-limit
182
-
- [ ] configure caddy to use a health check (once it's added)
183
-
- [ ] configure caddy to only expose cache metrics to tailnet :/
236
+
- [ ] configure ~caddy~ nginx to use a health check (once it's added)
237
+
- [ ] ~configure caddy to only expose cache metrics to tailnet :/~
184
238
- [x] make some grafana dashboards
185
239
- [ ] raspi: mount /dev/sda on boot
186
240
- [ ] raspi: run link_aggregator via systemd so it starts on startup (and restarts?)
187
241
242
+
- [x] use nginx instead of caddy
243
+
- [x] nginx: enable cache
244
+
- [x] nginx: rate-limit
245
+
- [ ] nginx: get metrics
+8
link_aggregator/readme.md
+8
link_aggregator/readme.md
···
118
118
- [ ] jetstream: connect retry: only reset counter after some *time* has passed.
119
119
- [x] either count or estimate the total number of links added (distinct from link targets)
120
120
121
+
cache
122
+
- [ ] set api response headers
123
+
- [ ] put "stale-while-revalidate" in Cache-Control w/ num seconds
124
+
- [ ] put "stale-if-error" in Cache-Control w/ num seconds
125
+
- [ ] set Expires or Cache-Control expires
126
+
- [ ] add Accept to vary response
127
+
- [ ] cache vary: might need to take bsky account privacy setting into account (unless this ends up being in query)
128
+
121
129
data fixes
122
130
- [x] add rkey to linkers 🤦♀️
123
131
- [x] don't remove deleted links from the reverse records -- null them out. this will keep things stable for paging.