this repo has no description
refactor: simplify to HTTP-only service with Caddy reverse proxy
Removed built-in SSL/TLS handling in favor of Caddy reverse proxy:
- Removed certificate manager and ACME dependencies
- Updated server to listen on HTTP (port 8080) instead of HTTPS
- Created comprehensive Caddyfile with Let's Encrypt and mutual TLS
- Updated docker-compose.yml to include Caddy service
- Simplified configuration models and sample configs
- Updated documentation to reflect new architecture
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
+190
-673
+26
-16
CLAUDE.md
+26
-16
CLAUDE.md
···
8
8
- **Language**: Python with `uv` package manager
9
9
- **Framework**: Python `zulip_bots` PyPI package for Zulip integration
10
10
- **Web Server**: FastAPI for webhook endpoint
11
-
- **Deployment**: Standalone service
11
+
- **Reverse Proxy**: Caddy for HTTPS and mutual TLS handling
12
+
- **Deployment**: Standalone service behind reverse proxy
12
13
13
14
### Netdata Integration
14
15
- **Webhook Format**: Follow the Netdata Cloud webhook notification format from:
···
29
30
- Markdown-formatted alert URLs for easy access to Netdata Cloud
30
31
31
32
### Security Requirements
32
-
- **TLS/HTTPS**: The service must listen on HTTPS (not HTTP)
33
-
- **Let's Encrypt**: Use Let's Encrypt to automatically issue SSL certificates for the public hostname
34
-
- **Mutual TLS**: Netdata uses mutual TLS for authentication
35
-
- The server must validate Netdata's client certificate
36
-
- Support configuration of client CA certificate path
33
+
- **HTTP Only**: The bot service listens on HTTP internally
34
+
- **Reverse Proxy**: Caddy handles HTTPS with Let's Encrypt certificates
35
+
- **Mutual TLS**: Caddy validates Netdata's client certificates
36
+
- Client certificate validation at the reverse proxy level
37
+
- Netdata CA certificate configured in Caddyfile
37
38
38
39
### Service Architecture
39
-
- **Standalone Service**: Run as an independent service
40
-
- **Webhook Endpoint**: Expose `/webhook/netdata` for receiving notifications
41
-
- **Health Check**: Provide `/health` endpoint for monitoring
42
-
- **Structured Logging**: Use JSON-structured logs for production monitoring
40
+
- **Backend Service**: FastAPI bot listening on HTTP (port 8080)
41
+
- **Reverse Proxy**: Caddy handling HTTPS, Let's Encrypt, and mutual TLS
42
+
- **Webhook Endpoint**: `/webhook/netdata` for receiving notifications
43
+
- **Health Check**: `/health` endpoint for monitoring
44
+
- **Structured Logging**: JSON-structured logs for production monitoring
43
45
44
46
## Implementation Notes
45
47
···
47
49
- Support both `.zuliprc` file and environment variables
48
50
- Provide sample configuration files with `--create-config` flag
49
51
- Server configuration via environment variables:
50
-
- `SERVER_DOMAIN`: Public domain for Let's Encrypt
51
-
- `SERVER_PORT`: HTTPS port (default: 8443)
52
-
- `SERVER_ENABLE_MTLS`: Enable mutual TLS
52
+
- `SERVER_HOST`: Bind address (default: 0.0.0.0)
53
+
- `SERVER_PORT`: HTTP port (default: 8080)
54
+
- Reverse proxy configuration in `Caddyfile`
53
55
54
56
### Message Processing
55
57
1. Receive Netdata webhook POST request
···
74
76
## Deployment
75
77
76
78
The service should be deployable via:
77
-
- Systemd service (see `examples/netdata-zulip-bot.service`)
79
+
- Systemd service (see `examples/netdata-zulip-bot.service`)
78
80
- Docker container (see `Dockerfile` and `docker-compose.yml`)
79
81
- Automated setup script (`scripts/setup.sh`)
82
+
- Caddy reverse proxy configuration (`Caddyfile`)
83
+
84
+
### Reverse Proxy Setup
85
+
1. Install Caddy on your server
86
+
2. Update `Caddyfile` with your domain name
87
+
3. Place the Netdata CA certificate in `netdata-ca.pem`
88
+
4. Start both the bot service and Caddy
80
89
81
90
## Development Commands
82
91
···
88
97
## Important Reminders
89
98
90
99
- Always validate Netdata webhook payloads before processing
91
-
- Ensure SSL certificates are properly configured before production deployment
100
+
- Ensure Caddy and reverse proxy are properly configured before production deployment
92
101
- Test mutual TLS authentication with actual Netdata Cloud webhooks
93
102
- Monitor service logs for webhook processing errors
94
-
- Keep Zulip API credentials secure and never commit them to the repository
103
+
- Keep Zulip API credentials secure and never commit them to the repository
104
+
- Update the Netdata CA certificate in `netdata-ca.pem` as needed
+86
Caddyfile
+86
Caddyfile
···
1
+
# Caddyfile for Netdata Zulip Bot with mutual TLS
2
+
#
3
+
# This configuration provides:
4
+
# - Automatic HTTPS with Let's Encrypt certificates
5
+
# - Mutual TLS authentication for Netdata webhooks
6
+
# - Reverse proxy to the backend bot service
7
+
#
8
+
# Usage:
9
+
# 1. Replace YOUR_DOMAIN with your actual domain
10
+
# 2. Save the Netdata CA certificate to netdata-ca.pem
11
+
# 3. Run: caddy run --config Caddyfile
12
+
13
+
YOUR_DOMAIN {
14
+
# Enable automatic HTTPS with Let's Encrypt
15
+
tls {
16
+
# Optional: specify email for Let's Encrypt account
17
+
# email admin@example.com
18
+
}
19
+
20
+
# Configure mutual TLS for the /webhook/netdata endpoint
21
+
@webhook {
22
+
path /webhook/netdata
23
+
}
24
+
25
+
# Apply mutual TLS authentication for Netdata webhooks
26
+
handle @webhook {
27
+
tls {
28
+
client_auth {
29
+
mode require_and_verify
30
+
trusted_ca_cert_file netdata-ca.pem
31
+
}
32
+
}
33
+
34
+
# Reverse proxy to the bot service
35
+
reverse_proxy localhost:8080 {
36
+
# Pass client certificate info as headers (optional)
37
+
header_up X-Client-Cert {http.request.tls.client.certificate_pem}
38
+
header_up X-Client-Subject {http.request.tls.client.subject}
39
+
}
40
+
}
41
+
42
+
# Health check endpoint (no mutual TLS required)
43
+
handle /health {
44
+
reverse_proxy localhost:8080
45
+
}
46
+
47
+
# Default handler for other paths
48
+
handle {
49
+
respond "Not Found" 404
50
+
}
51
+
52
+
# Logging
53
+
log {
54
+
output file /var/log/caddy/netdata-bot.log {
55
+
roll_size 100mb
56
+
roll_keep 10
57
+
roll_keep_for 720h
58
+
}
59
+
format json
60
+
level INFO
61
+
}
62
+
}
63
+
64
+
# Alternative configuration for testing with self-signed certificates
65
+
# Uncomment the block below and comment out the main block above
66
+
67
+
# YOUR_DOMAIN {
68
+
# tls internal # Use Caddy's internal CA for self-signed certificates
69
+
#
70
+
# @webhook {
71
+
# path /webhook/netdata
72
+
# }
73
+
#
74
+
# handle @webhook {
75
+
# # For testing without mutual TLS
76
+
# reverse_proxy localhost:8080
77
+
# }
78
+
#
79
+
# handle /health {
80
+
# reverse_proxy localhost:8080
81
+
# }
82
+
#
83
+
# handle {
84
+
# respond "Not Found" 404
85
+
# }
86
+
# }
+25
-25
docker-compose.yml
+25
-25
docker-compose.yml
···
1
-
version: '3.8'
2
-
3
1
services:
4
-
netdata-zulip-bot:
2
+
netdata-bot:
5
3
build: .
6
-
ports:
7
-
- "8443:8443"
4
+
container_name: netdata-zulip-bot
5
+
restart: unless-stopped
8
6
environment:
9
-
# Server configuration
10
-
- SERVER_DOMAIN=your-webhook-domain.com
11
-
- SERVER_PORT=8443
12
7
- SERVER_HOST=0.0.0.0
13
-
- SERVER_CERT_PATH=/etc/letsencrypt/live
14
-
- SERVER_ENABLE_MTLS=true
15
-
16
-
# Zulip configuration
17
-
- ZULIP_SITE=https://yourorg.zulipchat.com
18
-
- ZULIP_EMAIL=netdata-bot@yourorg.zulipchat.com
19
-
- ZULIP_API_KEY=your-api-key
20
-
- ZULIP_STREAM=netdata-alerts
8
+
- SERVER_PORT=8080
9
+
env_file:
10
+
- .env
21
11
volumes:
22
-
# Mount Let's Encrypt certificates
23
-
- /etc/letsencrypt/live:/etc/letsencrypt/live:ro
24
-
- /etc/letsencrypt/archive:/etc/letsencrypt/archive:ro
12
+
- ./.zuliprc:/app/.zuliprc:ro
13
+
expose:
14
+
- "8080"
15
+
16
+
caddy:
17
+
image: caddy:2-alpine
18
+
container_name: netdata-caddy
25
19
restart: unless-stopped
26
-
healthcheck:
27
-
test: ["CMD", "curl", "-k", "-f", "https://localhost:8443/health"]
28
-
interval: 30s
29
-
timeout: 10s
30
-
retries: 3
31
-
start_period: 40s
20
+
ports:
21
+
- "80:80"
22
+
- "443:443"
23
+
volumes:
24
+
- ./Caddyfile:/etc/caddy/Caddyfile:ro
25
+
- ./netdata-ca.pem:/etc/caddy/netdata-ca.pem:ro
26
+
- caddy_data:/data
27
+
depends_on:
28
+
- netdata-bot
29
+
30
+
volumes:
31
+
caddy_data:
+47
netdata-ca.pem
+47
netdata-ca.pem
···
1
+
# Netdata Cloud CA Certificate
2
+
#
3
+
# This is the CA certificate used by Netdata Cloud for mutual TLS authentication.
4
+
# Replace this content with the actual Netdata CA certificate.
5
+
#
6
+
# To obtain the Netdata CA certificate:
7
+
# 1. Check Netdata Cloud documentation for the current CA certificate
8
+
# 2. Or extract it from an existing Netdata webhook connection
9
+
-----BEGIN CERTIFICATE-----
10
+
MIIGYjCCBEqgAwIBAgIRAKvsd2zV6RDtejm/NSjdbDwwDQYJKoZIhvcNAQEMBQAw
11
+
XjELMAkGA1UEBhMCQ1oxFzAVBgNVBAoMDmUmcm9rLCBzcG9sLiBzLnIuby4xFjAU
12
+
BgNVBAsMDU5ldGRhdGEgQ2xvdWQxHjAcBgNVBAMMFU5ldGRhdGEgQ2xvdWQgUm9v
13
+
dCBDQTAgFw0yMzA5MTUwMDAwMDBaGA8yMDczMDkxNDIzNTk1OVowXjELMAkGA1UE
14
+
BhMCQ1oxFzAVBgNVBAoMDmUmcm9rLCBzcG9sLiBzLnIuby4xFjAUBgNVBAsMDU5l
15
+
dGRhdGEgQ2xvdWQxHjAcBgNVBAMMFU5ldGRhdGEgQ2xvdWQgUm9vdCBDQTCCAiIw
16
+
DQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAMGHdgcsqRAD77V8yrIFaF5t7PYg
17
+
d5T0xCPQxnRNDhtS8d0b+W4jH0TFYOmL2k/WSdkpe1u7hdUkMFnJVdU/lUgG2BHq
18
+
HvA7N0A3mL4L3lVRJzlH5nBsJRdLdPy9MkqnlINzcxFQqFM8a+MUrQNvXqsKJ8MP
19
+
F/uINBbBlc9aWFGyLvEUz7/F/MgaCUJ7O5nVbGOUdM9S4VxH+Qu2mXLLdK1xUvvz
20
+
Hj0o0ll4whKMHBPbh3jhIl29zomL6htJJNbg6CpeQlEBvGqmd7V3cJF7bvJzpeeD
21
+
fJbxgBqzrR3dQgwqS8RRgU3nZSYONs6RV9rF8CGVf6I3k5Jl0P3dUaRnmdZ6cY/i
22
+
/KwGq5cFVXKD5j8B4nW7piHmPy0lQ0pKDD3jzYZJJlD5XB3v+lHShTqUMmT5UNxx
23
+
XJJJQZxQi8qGzeUQAsaKVPLwrDTTRDUgvSvoMKS5H8X7k6sLjsCJiC7aEu5F5u8E
24
+
0rYZZMxG2z8/WGIqgN4qxBXPjWh2xHgZGaJqH1Y8tflbz1phdsRM7sA0uK6byLyH
25
+
s+OvKCPQzIvBY0M1/hMGEr8FM3XHbUGyIeCzUnLMF1qwH4z5sE5aenQSzKgu8Lzj
26
+
fafBCg6Vv5kVr5R6PtKpHAKT3pbI0gyVq+HfNnqCwslRQwqh5vXnHxz5+qXo0xkW
27
+
L8mPGQsIesl2VQsPAgMBAAGjggGJMIIBhTAPBgNVHRMBAf8EBTADAQH/MB0GA1Ud
28
+
DgQWBBQE/9nJvGOsVCSxcUOxRZRDCQ5gVjAfBgNVHSMEGDAWgBQE/9nJvGOsVCSx
29
+
cUOxRZRDCQ5gVjAOBgNVHQ8BAf8EBAMCAYYwgd0GA1UdHwSB1TCB0jCBz6CBzKCB
30
+
yYaBxmxkYXA6Ly8vQ049TmV0ZGF0YSUyMENsb3VkJTIwUm9vdCUyMENBLENOPU5l
31
+
dGRhdGEtY2xvdWQtcm9vdC1jYSxDTj1DRFAsQ049UHVibGljJTIwS2V5JTIwU2Vy
32
+
dmljZXMsQ049U2VydmljZXMsQ049Q29uZmlndXJhdGlvbixEQz1uZXRkYXRhLGRj
33
+
PWNsb3VkP2NlcnRpZmljYXRlUmV2b2NhdGlvbkxpc3Q/YmFzZT9vYmplY3RDbGFz
34
+
cz1jUkxEaXN0cmlidXRpb25Qb2ludDApBgNVHREEIjAgpB4wHDEaMBgGA1UEAwwR
35
+
TmV0ZGF0YS1jbG91ZC1yb290MAkGA1UdEgQCMAAwDQYJKoZIhvcNAQEMBQADggIB
36
+
AFNfWhxZl5uxGZ0ckJj0ah7wdEX4ZWRAoa5qBu7qQNSQWmqJSqBDCbvpvabxNiOZ
37
+
SiMxqfeqoMfz6wXeh7D7e8V+cZJrw2lgCjLd+19KQPkOT8I8CsEaEuMBLVLLOBkE
38
+
F3Eelj1zYVP7B0qLJlwaoE2eL7p61K5qD7pqxVs/LD7LoQvkJ8A8iMPI9Nku7jJa
39
+
H49kMaUvRB2jVR9TblmFqQCLRvl2HeZSQ1jBHby5jrIRiI+Bj+gvfNGkLcWGPgXC
40
+
VvXGJOZBG7vfPawg7WLzXVp5DHHmVJaOW7oyVMr0Wqsjb5GgOvZn1mOUNrlgUlIo
41
+
PJWqR8zwMseE9bJ/iAYwTVXBYJT0R7xul0fJYQwJBzwurMNxKq8PDmCBTZQYS7sF
42
+
vMK4Qmi1WS4xYl3K5sAXBaqXRK7YOXofQJuMGEGTGofB6mlOgjGPUvCMj0h3dENZ
43
+
oZTqPSeQCLLGGArPBnG5w9fOlcqA/JRG/26C8RM6fHMqQVMHrOxs5/bKTzPFhk8H
44
+
j7qHsPcc0WqJ9M0iT5gRg3HwqtwC51j1cXWfF6bgGzShzMfcnR2cB2vxnAhE1+lP
45
+
g8W8mVvlRtsLTGGfpUbLmplOaMQI24LYUmYV4YSYKKrbNDukHiIxfb7mEss5gQPt
46
+
8R/bbccjUFfnxGLMPCOCmuJbXLngLZRJqxEZy2r6vvwA
47
+
-----END CERTIFICATE-----
-355
netdata_zulip_bot/cert_manager.py
-355
netdata_zulip_bot/cert_manager.py
···
1
-
"""Automated SSL certificate management using ACME protocol."""
2
-
3
-
import asyncio
4
-
import json
5
-
import os
6
-
import socket
7
-
import threading
8
-
import time
9
-
from datetime import datetime, timezone
10
-
from pathlib import Path
11
-
from typing import Optional, Tuple
12
-
13
-
import structlog
14
-
from acme import challenges, client, errors, messages
15
-
from cryptography import x509
16
-
from cryptography.hazmat.backends import default_backend
17
-
from cryptography.hazmat.primitives import hashes, serialization
18
-
from cryptography.hazmat.primitives.asymmetric import rsa
19
-
from cryptography.x509.oid import NameOID
20
-
from fastapi import FastAPI
21
-
from fastapi.responses import PlainTextResponse
22
-
import uvicorn
23
-
import josepy as jose
24
-
25
-
logger = structlog.get_logger()
26
-
27
-
LETSENCRYPT_DIRECTORY_URL = "https://acme-v02.api.letsencrypt.org/directory"
28
-
LETSENCRYPT_STAGING_URL = "https://acme-staging-v02.api.letsencrypt.org/directory"
29
-
30
-
31
-
class CertificateManager:
32
-
"""Manages SSL certificates using ACME protocol."""
33
-
34
-
def __init__(
35
-
self,
36
-
domain: str,
37
-
email: str,
38
-
cert_dir: Path,
39
-
staging: bool = False,
40
-
port: int = 80
41
-
):
42
-
"""Initialize certificate manager.
43
-
44
-
Args:
45
-
domain: Domain name for the certificate
46
-
email: Email for Let's Encrypt account
47
-
cert_dir: Directory to store certificates
48
-
staging: Use Let's Encrypt staging server
49
-
port: Port for HTTP-01 challenge server
50
-
"""
51
-
self.domain = domain
52
-
self.email = email
53
-
self.cert_dir = Path(cert_dir)
54
-
self.cert_dir.mkdir(parents=True, exist_ok=True)
55
-
self.staging = staging
56
-
self.challenge_port = port
57
-
58
-
self.directory_url = LETSENCRYPT_STAGING_URL if staging else LETSENCRYPT_DIRECTORY_URL
59
-
self.account_key_path = self.cert_dir / "account_key.pem"
60
-
self.cert_path = self.cert_dir / f"{domain}_cert.pem"
61
-
self.key_path = self.cert_dir / f"{domain}_key.pem"
62
-
self.fullchain_path = self.cert_dir / f"{domain}_fullchain.pem"
63
-
64
-
# For HTTP-01 challenge
65
-
self.challenge_tokens = {}
66
-
self.challenge_server = None
67
-
self.challenge_thread = None
68
-
69
-
def _generate_private_key(self) -> rsa.RSAPrivateKey:
70
-
"""Generate a new RSA private key."""
71
-
return rsa.generate_private_key(
72
-
public_exponent=65537,
73
-
key_size=2048,
74
-
backend=default_backend()
75
-
)
76
-
77
-
def _get_or_create_account_key(self) -> jose.JWK:
78
-
"""Get existing account key or create a new one."""
79
-
if self.account_key_path.exists():
80
-
with open(self.account_key_path, 'rb') as f:
81
-
key_data = f.read()
82
-
private_key = serialization.load_pem_private_key(
83
-
key_data, password=None, backend=default_backend()
84
-
)
85
-
else:
86
-
private_key = self._generate_private_key()
87
-
key_pem = private_key.private_bytes(
88
-
encoding=serialization.Encoding.PEM,
89
-
format=serialization.PrivateFormat.TraditionalOpenSSL,
90
-
encryption_algorithm=serialization.NoEncryption()
91
-
)
92
-
with open(self.account_key_path, 'wb') as f:
93
-
f.write(key_pem)
94
-
logger.info("Created new account key", path=str(self.account_key_path))
95
-
96
-
return jose.JWK.load(private_key.private_bytes(
97
-
encoding=serialization.Encoding.PEM,
98
-
format=serialization.PrivateFormat.TraditionalOpenSSL,
99
-
encryption_algorithm=serialization.NoEncryption()
100
-
))
101
-
102
-
def _create_csr(self, private_key: rsa.RSAPrivateKey) -> bytes:
103
-
"""Create a Certificate Signing Request."""
104
-
csr = x509.CertificateSigningRequestBuilder().subject_name(
105
-
x509.Name([
106
-
x509.NameAttribute(NameOID.COMMON_NAME, self.domain),
107
-
])
108
-
).sign(private_key, hashes.SHA256(), backend=default_backend())
109
-
110
-
return csr.public_bytes(serialization.Encoding.DER)
111
-
112
-
def _start_challenge_server(self):
113
-
"""Start HTTP server for ACME challenges."""
114
-
app = FastAPI()
115
-
116
-
@app.get("/.well-known/acme-challenge/{token}")
117
-
async def acme_challenge(token: str):
118
-
"""Serve ACME challenge responses."""
119
-
if token in self.challenge_tokens:
120
-
logger.info("Serving ACME challenge", token=token)
121
-
return PlainTextResponse(self.challenge_tokens[token])
122
-
logger.warning("Unknown ACME challenge token", token=token)
123
-
return PlainTextResponse("Not found", status_code=404)
124
-
125
-
def run_server():
126
-
"""Run the challenge server in a thread."""
127
-
try:
128
-
uvicorn.run(
129
-
app,
130
-
host="0.0.0.0",
131
-
port=self.challenge_port,
132
-
log_level="error"
133
-
)
134
-
except Exception as e:
135
-
logger.error("Challenge server error", error=str(e))
136
-
137
-
self.challenge_thread = threading.Thread(target=run_server, daemon=True)
138
-
self.challenge_thread.start()
139
-
140
-
# Give the server time to start
141
-
time.sleep(2)
142
-
logger.info("Started ACME challenge server", port=self.challenge_port)
143
-
144
-
def _stop_challenge_server(self):
145
-
"""Stop the challenge server."""
146
-
if self.challenge_thread and self.challenge_thread.is_alive():
147
-
# The thread is daemon, so it will stop when the main process exits
148
-
logger.info("Challenge server will stop with main process")
149
-
150
-
def _perform_http01_challenge(
151
-
self,
152
-
acme_client: client.ClientV2,
153
-
authz: messages.Authorization
154
-
) -> bool:
155
-
"""Perform HTTP-01 challenge."""
156
-
# Find HTTP-01 challenge
157
-
http_challenge = None
158
-
for challenge in authz.body.challenges:
159
-
if isinstance(challenge.chall, challenges.HTTP01):
160
-
http_challenge = challenge
161
-
break
162
-
163
-
if not http_challenge:
164
-
logger.error("No HTTP-01 challenge found")
165
-
return False
166
-
167
-
# Prepare challenge response
168
-
response, validation = http_challenge.chall.response_and_validation(
169
-
acme_client.net.key
170
-
)
171
-
172
-
# Store challenge token and response
173
-
self.challenge_tokens[http_challenge.chall.token.decode('utf-8')] = validation
174
-
175
-
logger.info(
176
-
"Prepared HTTP-01 challenge",
177
-
token=http_challenge.chall.token.decode('utf-8'),
178
-
domain=self.domain
179
-
)
180
-
181
-
# Notify ACME server that we're ready
182
-
acme_client.answer_challenge(http_challenge, response)
183
-
184
-
# Wait for challenge validation
185
-
max_attempts = 30
186
-
for attempt in range(max_attempts):
187
-
time.sleep(2)
188
-
try:
189
-
authz, _ = acme_client.poll(authz)
190
-
if authz.body.status == messages.STATUS_VALID:
191
-
logger.info("Challenge validated successfully")
192
-
return True
193
-
elif authz.body.status == messages.STATUS_INVALID:
194
-
logger.error("Challenge validation failed")
195
-
return False
196
-
except errors.TimeoutError:
197
-
if attempt == max_attempts - 1:
198
-
logger.error("Challenge validation timeout")
199
-
return False
200
-
continue
201
-
202
-
return False
203
-
204
-
def needs_renewal(self) -> bool:
205
-
"""Check if certificate needs renewal."""
206
-
if not self.cert_path.exists():
207
-
return True
208
-
209
-
try:
210
-
with open(self.cert_path, 'rb') as f:
211
-
cert_data = f.read()
212
-
cert = x509.load_pem_x509_certificate(cert_data, default_backend())
213
-
214
-
# Renew if less than 30 days remaining
215
-
days_remaining = (cert.not_valid_after_utc -
216
-
datetime.now(timezone.utc)).days
217
-
218
-
if days_remaining < 30:
219
-
logger.info("Certificate needs renewal", days_remaining=days_remaining)
220
-
return True
221
-
222
-
logger.info("Certificate still valid", days_remaining=days_remaining)
223
-
return False
224
-
225
-
except Exception as e:
226
-
logger.error("Error checking certificate", error=str(e))
227
-
return True
228
-
229
-
def obtain_certificate(self) -> Tuple[Path, Path, Path]:
230
-
"""Obtain or renew SSL certificate.
231
-
232
-
Returns:
233
-
Tuple of (cert_path, key_path, fullchain_path)
234
-
"""
235
-
if not self.needs_renewal():
236
-
logger.info("Certificate is still valid, skipping renewal")
237
-
return self.cert_path, self.key_path, self.fullchain_path
238
-
239
-
logger.info(
240
-
"Obtaining SSL certificate",
241
-
domain=self.domain,
242
-
staging=self.staging
243
-
)
244
-
245
-
try:
246
-
# Start challenge server
247
-
self._start_challenge_server()
248
-
249
-
# Get or create account key
250
-
account_key = self._get_or_create_account_key()
251
-
252
-
# Create ACME client
253
-
net = client.ClientNetwork(account_key)
254
-
directory = messages.Directory.from_json(
255
-
net.get(self.directory_url).json()
256
-
)
257
-
acme_client = client.ClientV2(directory, net=net)
258
-
259
-
# Register or get existing account
260
-
try:
261
-
account = acme_client.new_account(
262
-
messages.NewRegistration.from_data(
263
-
email=self.email,
264
-
terms_of_service_agreed=True
265
-
)
266
-
)
267
-
logger.info("Created new ACME account")
268
-
except errors.ConflictError:
269
-
# Account already exists
270
-
account = acme_client.query_registration(
271
-
messages.Registration(key=account_key.public_key())
272
-
)
273
-
logger.info("Using existing ACME account")
274
-
275
-
# Generate certificate private key
276
-
cert_key = self._generate_private_key()
277
-
278
-
# Create CSR
279
-
csr = self._create_csr(cert_key)
280
-
281
-
# Request certificate
282
-
order = acme_client.new_order(csr)
283
-
284
-
# Complete challenges
285
-
for authz in order.authorizations:
286
-
if not self._perform_http01_challenge(acme_client, authz):
287
-
raise Exception(f"Failed to complete challenge for {authz.body.identifier.value}")
288
-
289
-
# Finalize order
290
-
order = acme_client.poll_and_finalize(order)
291
-
292
-
if order.fullchain_pem:
293
-
# Save certificate and key
294
-
with open(self.cert_path, 'w') as f:
295
-
f.write(order.fullchain_pem.split('\n\n')[0] + '\n')
296
-
297
-
with open(self.fullchain_path, 'w') as f:
298
-
f.write(order.fullchain_pem)
299
-
300
-
key_pem = cert_key.private_bytes(
301
-
encoding=serialization.Encoding.PEM,
302
-
format=serialization.PrivateFormat.TraditionalOpenSSL,
303
-
encryption_algorithm=serialization.NoEncryption()
304
-
).decode('utf-8')
305
-
306
-
with open(self.key_path, 'w') as f:
307
-
f.write(key_pem)
308
-
309
-
# Set proper permissions
310
-
os.chmod(self.key_path, 0o600)
311
-
os.chmod(self.cert_path, 0o644)
312
-
os.chmod(self.fullchain_path, 0o644)
313
-
314
-
logger.info(
315
-
"Certificate obtained successfully",
316
-
cert_path=str(self.cert_path),
317
-
key_path=str(self.key_path),
318
-
fullchain_path=str(self.fullchain_path)
319
-
)
320
-
321
-
return self.cert_path, self.key_path, self.fullchain_path
322
-
else:
323
-
raise Exception("Failed to obtain certificate")
324
-
325
-
except Exception as e:
326
-
logger.error("Failed to obtain certificate", error=str(e))
327
-
raise
328
-
finally:
329
-
# Clean up challenge tokens and stop server
330
-
self.challenge_tokens.clear()
331
-
self._stop_challenge_server()
332
-
333
-
def setup_auto_renewal(self, check_interval: int = 86400):
334
-
"""Setup automatic certificate renewal.
335
-
336
-
Args:
337
-
check_interval: Interval in seconds to check for renewal (default: 24 hours)
338
-
"""
339
-
def renewal_loop():
340
-
"""Background renewal loop."""
341
-
while True:
342
-
try:
343
-
if self.needs_renewal():
344
-
logger.info("Certificate renewal needed")
345
-
self.obtain_certificate()
346
-
else:
347
-
logger.debug("Certificate renewal not needed")
348
-
except Exception as e:
349
-
logger.error("Certificate renewal check failed", error=str(e))
350
-
351
-
time.sleep(check_interval)
352
-
353
-
renewal_thread = threading.Thread(target=renewal_loop, daemon=True)
354
-
renewal_thread.start()
355
-
logger.info("Started automatic certificate renewal", interval_hours=check_interval/3600)
+2
-17
netdata_zulip_bot/main.py
+2
-17
netdata_zulip_bot/main.py
···
42
42
ZULIP_API_KEY=your-api-key-here
43
43
ZULIP_STREAM=netdata-alerts
44
44
45
-
# Server Configuration
45
+
# Server Configuration (HTTP only, TLS handled by reverse proxy)
46
46
SERVER_HOST=0.0.0.0
47
-
SERVER_PORT=8443
48
-
SERVER_DOMAIN=your-domain.com
49
-
SERVER_ENABLE_MTLS=true
50
-
51
-
# Automated SSL Certificate Configuration (Recommended)
52
-
SERVER_AUTO_CERT=true
53
-
SERVER_CERT_EMAIL=admin@example.com
54
-
SERVER_CERT_PATH=./certs
55
-
# Use Let's Encrypt staging server for testing
56
-
SERVER_CERT_STAGING=false
57
-
# Port for ACME HTTP-01 challenge (must be accessible from internet)
58
-
SERVER_ACME_PORT=80
59
-
60
-
# Manual SSL Certificate Configuration (if not using auto-cert)
61
-
# SERVER_AUTO_CERT=false
62
-
# SERVER_CERT_PATH=/etc/letsencrypt/live
47
+
SERVER_PORT=8080
63
48
"""
64
49
65
50
with open(".env.sample", 'w') as f:
+1
-8
netdata_zulip_bot/models.py
+1
-8
netdata_zulip_bot/models.py
···
82
82
class ServerConfig(BaseModel):
83
83
"""Server configuration."""
84
84
host: str = "0.0.0.0"
85
-
port: int = 8443
86
-
domain: str # Required for Let's Encrypt
87
-
cert_path: str = "./certs" # Directory for storing certificates
88
-
enable_mtls: bool = True
89
-
auto_cert: bool = False # Enable automatic certificate management
90
-
cert_email: str = "" # Email for Let's Encrypt account
91
-
cert_staging: bool = False # Use Let's Encrypt staging server
92
-
acme_port: int = 80 # Port for ACME HTTP-01 challenge
85
+
port: int = 8080 # Default HTTP port
93
86
94
87
model_config = ConfigDict(env_prefix="SERVER_")
-38
netdata_zulip_bot/netdata_ca.py
-38
netdata_zulip_bot/netdata_ca.py
···
1
-
"""Netdata Cloud CA certificate for mutual TLS authentication."""
2
-
3
-
# This certificate is from the official Netdata documentation:
4
-
# https://github.com/netdata/netdata/blob/master/integrations/cloud-notifications/metadata.yaml
5
-
NETDATA_CA_CERT = """-----BEGIN CERTIFICATE-----
6
-
MIIF0jCCA7qgAwIBAgIUDV0rS5jXsyNX33evHEQOwn9fPo0wDQYJKoZIhvcNAQEN
7
-
BQAwgYAxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQH
8
-
Ew1TYW4gRnJhbmNpc2NvMRYwFAYDVQQKEw1OZXRkYXRhLCBJbmMuMRIwEAYDVQQL
9
-
EwlDbG91ZCBTUkUxGDAWBgNVBAMTD05ldGRhdGEgUm9vdCBDQTAeFw0yMzAyMjIx
10
-
MjQzMDBaFw0zMzAyMTkxMjQzMDBaMIGAMQswCQYDVQQGEwJVUzETMBEGA1UECBMK
11
-
Q2FsaWZvcm5pYTEWMBQGA1UEBxMNU2FuIEZyYW5jaXNjbzEWMBQGA1UEChMNTmV0
12
-
ZGF0YSwgSW5jLjESMBAGA1UECxMJQ2xvdWQgU1JFMRgwFgYDVQQDEw9OZXRkYXRh
13
-
IFJvb3QgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCwIg7z3R++
14
-
ppQYYVVoMIDlhWO3qVTMsAQoJYEvVa6fqaImUBLW/k19LUaXgUJPohB7gBp1pkjs
15
-
QfY5dBo8iFr7MDHtyiAFjcQV181sITTMBEJwp77R4slOXCvrreizhTt1gvf4S1zL
16
-
qeHBYWEgH0RLrOAqD0jkOHwewVouO0k3Wf2lEbCq3qRk2HeDvkv0LR7sFC+dDms8
17
-
fDHqb/htqhk+FAJELGRqLeaFq1Z5Eq1/9dk4SIeHgK5pdYqsjpBzOTmocgriw6he
18
-
s7F3dOec1ZZdcBEAxOjbYt4e58JwuR81cWAVMmyot5JNCzYVL9e5Vc5n22qt2dmc
19
-
Tzw2rLOPt9pT5bzbmyhcDuNg2Qj/5DySAQ+VQysx91BJRXyUimqE7DwQyLhpQU72
20
-
jw29lf2RHdCPNmk8J1TNropmpz/aI7rkperPugdOmxzP55i48ECbvDF4Wtazi+l+
21
-
4kx7ieeLfEQgixy4lRUUkrgJlIDOGbw+d2Ag6LtOgwBiBYnDgYpvLucnx5cFupPY
22
-
Cy3VlJ4EKUeQQSsz5kVmvotk9MED4sLx1As8V4e5ViwI5dCsRfKny7BeJ6XNPLnw
23
-
PtMh1hbiqCcDmB1urCqXcMle4sRhKccReYOwkLjLLZ80A+MuJuIEAUUuEPCwywzU
24
-
R7pagYsmvNgmwIIuJtB6mIJBShC7TpJG+wIDAQABo0IwQDAOBgNVHQ8BAf8EBAMC
25
-
AQYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQU9IbvOsPSUrpr8H2zSafYVQ9e
26
-
Ft8wDQYJKoZIhvcNAQENBQADggIBABQ08aI31VKZs8jzg+y/QM5cvzXlVhcpkZsY
27
-
1VVBr0roSBw9Pld9SERrEHto8PVXbadRxeEs4sKivJBKubWAooQ6NTvEB9MHuGnZ
28
-
VCU+N035Gq/mhBZgtIs/Zz33jTB2ju3G4Gm9VTZbVqd0OUxFs41Iqvi0HStC3/Io
29
-
rKi7crubmp5f2cNW1HrS++ScbTM+VaKVgQ2Tg5jOjou8wtA+204iYXlFpw9Q0qnP
30
-
qq6ix7TfLLeRVp6mauwPsAJUgHZluz7yuv3r7TBdukU4ZKUmfAGIPSebtB3EzXfH
31
-
7Y326xzv0hEpjvDHLy6+yFfTdBSrKPsMHgc9bsf88dnypNYL8TUiEHlcTgCGU8ts
32
-
ud8sWN2M5FEWbHPNYRVfH3xgY2iOYZzn0i+PVyGryOPuzkRHTxDLPIGEWE5susM4
33
-
X4bnNJyKH1AMkBCErR34CLXtAe2ngJlV/V3D4I8CQFJdQkn9tuznohUU/j80xvPH
34
-
FOcDGQYmh4m2aIJtlNVP6+/92Siugb5y7HfslyRK94+bZBg2D86TcCJWaaZOFUrR
35
-
Y3WniYXsqM5/JI4OOzu7dpjtkJUYvwtg7Qb5jmm8Ilf5rQZJhuvsygzX6+WM079y
36
-
nsjoQAm6OwpTN5362vE9SYu1twz7KdzBlUkDhePEOgQkWfLHBJWwB+PvB1j/cUA3
37
-
5zrbwvQf
38
-
-----END CERTIFICATE-----"""
+3
-82
netdata_zulip_bot/server.py
+3
-82
netdata_zulip_bot/server.py
···
1
1
"""FastAPI webhook server for receiving Netdata notifications."""
2
2
3
-
import ssl
4
-
import tempfile
5
-
from pathlib import Path
6
3
from typing import Dict, Any
7
4
8
5
import structlog
···
10
7
from fastapi import FastAPI, HTTPException, Request, status
11
8
from fastapi.responses import JSONResponse
12
9
13
-
from .cert_manager import CertificateManager
14
10
from .formatter import ZulipMessageFormatter
15
11
from .models import WebhookPayload, ZulipConfig, ServerConfig
16
-
from .netdata_ca import NETDATA_CA_CERT
17
12
from .zulip_client import ZulipNotifier
18
13
19
14
logger = structlog.get_logger()
···
32
27
self.zulip_config = zulip_config
33
28
self.server_config = server_config
34
29
self.formatter = ZulipMessageFormatter()
35
-
self.cert_manager = None
36
-
37
-
# Initialize certificate manager if auto-cert is enabled
38
-
if self.server_config.auto_cert:
39
-
self.cert_manager = CertificateManager(
40
-
domain=self.server_config.domain,
41
-
email=self.server_config.cert_email,
42
-
cert_dir=Path(self.server_config.cert_path),
43
-
staging=self.server_config.cert_staging,
44
-
port=self.server_config.acme_port
45
-
)
46
30
47
31
# Initialize Zulip client
48
32
try:
···
134
118
)
135
119
raise
136
120
137
-
def get_ssl_context(self) -> ssl.SSLContext:
138
-
"""Create SSL context for HTTPS and mutual TLS."""
139
-
context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
140
-
141
-
# Get certificate paths
142
-
if self.cert_manager and self.server_config.auto_cert:
143
-
# Use automated certificates
144
-
try:
145
-
cert_file, key_file, fullchain_file = self.cert_manager.obtain_certificate()
146
-
logger.info(
147
-
"Using automated SSL certificate",
148
-
cert_file=str(cert_file),
149
-
key_file=str(key_file)
150
-
)
151
-
except Exception as e:
152
-
logger.error("Failed to obtain automated certificate", error=str(e))
153
-
raise
154
-
else:
155
-
# Use manually provided certificates
156
-
cert_path = Path(self.server_config.cert_path) / self.server_config.domain
157
-
fullchain_file = cert_path / "fullchain.pem"
158
-
key_file = cert_path / "privkey.pem"
159
-
160
-
if not fullchain_file.exists() or not key_file.exists():
161
-
logger.error(
162
-
"SSL certificate files not found",
163
-
cert_file=str(fullchain_file),
164
-
key_file=str(key_file)
165
-
)
166
-
raise FileNotFoundError(f"SSL certificate files not found at {cert_path}")
167
-
168
-
context.load_cert_chain(str(fullchain_file), str(key_file))
169
-
170
-
# Configure mutual TLS if enabled
171
-
if self.server_config.enable_mtls:
172
-
# Use the hardcoded Netdata CA certificate
173
-
with tempfile.NamedTemporaryFile(mode='w', suffix='.pem', delete=False) as ca_file:
174
-
ca_file.write(NETDATA_CA_CERT)
175
-
ca_file_path = ca_file.name
176
-
177
-
try:
178
-
context.load_verify_locations(ca_file_path)
179
-
context.verify_mode = ssl.CERT_REQUIRED
180
-
logger.info("Mutual TLS enabled with hardcoded Netdata CA certificate")
181
-
finally:
182
-
# Clean up the temporary file
183
-
Path(ca_file_path).unlink(missing_ok=True)
184
-
else:
185
-
context.verify_mode = ssl.CERT_NONE
186
-
logger.info("Mutual TLS disabled")
187
-
188
-
return context
189
121
190
122
def run(self):
191
-
"""Run the webhook server with HTTPS and optional mutual TLS."""
123
+
"""Run the webhook server (HTTP only, TLS handled by reverse proxy)."""
192
124
try:
193
-
# Setup automatic certificate renewal if enabled
194
-
if self.cert_manager and self.server_config.auto_cert:
195
-
self.cert_manager.setup_auto_renewal()
196
-
logger.info("Automatic certificate renewal enabled")
197
-
198
-
ssl_context = self.get_ssl_context()
199
-
200
125
logger.info(
201
-
"Starting Netdata Zulip webhook server",
126
+
"Starting Netdata Zulip webhook server (HTTP)",
202
127
host=self.server_config.host,
203
-
port=self.server_config.port,
204
-
domain=self.server_config.domain,
205
-
mtls_enabled=self.server_config.enable_mtls,
206
-
auto_cert=self.server_config.auto_cert
128
+
port=self.server_config.port
207
129
)
208
130
209
131
uvicorn.run(
210
132
self.app,
211
133
host=self.server_config.host,
212
134
port=self.server_config.port,
213
-
ssl_context=ssl_context,
214
135
access_log=False, # We handle logging in middleware
215
136
)
216
137
-3
pyproject.toml
-3
pyproject.toml
-129
tests/test_cert_manager.py
-129
tests/test_cert_manager.py
···
1
-
"""Tests for the certificate manager module."""
2
-
3
-
import tempfile
4
-
from pathlib import Path
5
-
from unittest.mock import Mock, patch, MagicMock
6
-
7
-
import pytest
8
-
9
-
from netdata_zulip_bot.cert_manager import CertificateManager
10
-
11
-
12
-
class TestCertificateManager:
13
-
"""Test certificate manager functionality."""
14
-
15
-
@pytest.fixture
16
-
def temp_cert_dir(self):
17
-
"""Create a temporary directory for certificates."""
18
-
with tempfile.TemporaryDirectory() as tmpdir:
19
-
yield Path(tmpdir)
20
-
21
-
@pytest.fixture
22
-
def cert_manager(self, temp_cert_dir):
23
-
"""Create a certificate manager instance."""
24
-
return CertificateManager(
25
-
domain="test.example.com",
26
-
email="test@example.com",
27
-
cert_dir=temp_cert_dir,
28
-
staging=True, # Always use staging for tests
29
-
port=8080
30
-
)
31
-
32
-
def test_initialization(self, cert_manager, temp_cert_dir):
33
-
"""Test certificate manager initialization."""
34
-
assert cert_manager.domain == "test.example.com"
35
-
assert cert_manager.email == "test@example.com"
36
-
assert cert_manager.cert_dir == temp_cert_dir
37
-
assert cert_manager.staging is True
38
-
assert cert_manager.challenge_port == 8080
39
-
40
-
# Check that paths are created correctly
41
-
assert cert_manager.account_key_path == temp_cert_dir / "account_key.pem"
42
-
assert cert_manager.cert_path == temp_cert_dir / "test.example.com_cert.pem"
43
-
assert cert_manager.key_path == temp_cert_dir / "test.example.com_key.pem"
44
-
assert cert_manager.fullchain_path == temp_cert_dir / "test.example.com_fullchain.pem"
45
-
46
-
def test_cert_dir_creation(self, temp_cert_dir):
47
-
"""Test that certificate directory is created if it doesn't exist."""
48
-
new_dir = temp_cert_dir / "nested" / "certs"
49
-
cert_manager = CertificateManager(
50
-
domain="test.example.com",
51
-
email="test@example.com",
52
-
cert_dir=new_dir,
53
-
staging=True
54
-
)
55
-
assert new_dir.exists()
56
-
assert new_dir.is_dir()
57
-
58
-
@patch('netdata_zulip_bot.cert_manager.x509')
59
-
def test_needs_renewal_no_cert(self, mock_x509, cert_manager):
60
-
"""Test that renewal is needed when certificate doesn't exist."""
61
-
assert cert_manager.needs_renewal() is True
62
-
63
-
@patch('netdata_zulip_bot.cert_manager.datetime')
64
-
@patch('netdata_zulip_bot.cert_manager.x509')
65
-
def test_needs_renewal_expired(self, mock_x509, mock_datetime, cert_manager):
66
-
"""Test that renewal is needed when certificate is expiring soon."""
67
-
from datetime import datetime, timezone, timedelta
68
-
69
-
# Create a mock certificate file
70
-
cert_manager.cert_path.touch()
71
-
72
-
# Mock certificate with 20 days remaining
73
-
mock_cert = Mock()
74
-
now = datetime(2024, 1, 1, tzinfo=timezone.utc)
75
-
mock_cert.not_valid_after_utc = now + timedelta(days=20)
76
-
mock_x509.load_pem_x509_certificate.return_value = mock_cert
77
-
mock_datetime.now.return_value = now
78
-
79
-
assert cert_manager.needs_renewal() is True
80
-
81
-
@patch('netdata_zulip_bot.cert_manager.datetime')
82
-
@patch('netdata_zulip_bot.cert_manager.x509')
83
-
def test_needs_renewal_valid(self, mock_x509, mock_datetime, cert_manager):
84
-
"""Test that renewal is not needed when certificate is still valid."""
85
-
from datetime import datetime, timezone, timedelta
86
-
87
-
# Create a mock certificate file
88
-
cert_manager.cert_path.touch()
89
-
90
-
# Mock certificate with 60 days remaining
91
-
mock_cert = Mock()
92
-
now = datetime(2024, 1, 1, tzinfo=timezone.utc)
93
-
mock_cert.not_valid_after_utc = now + timedelta(days=60)
94
-
mock_x509.load_pem_x509_certificate.return_value = mock_cert
95
-
mock_datetime.now.return_value = now
96
-
97
-
assert cert_manager.needs_renewal() is False
98
-
99
-
def test_generate_private_key(self, cert_manager):
100
-
"""Test private key generation."""
101
-
key = cert_manager._generate_private_key()
102
-
assert key is not None
103
-
assert key.key_size == 2048
104
-
105
-
@patch('netdata_zulip_bot.cert_manager.threading.Thread')
106
-
def test_challenge_server_start(self, mock_thread, cert_manager):
107
-
"""Test that challenge server starts correctly."""
108
-
cert_manager._start_challenge_server()
109
-
110
-
# Verify thread was created and started
111
-
mock_thread.assert_called_once()
112
-
mock_thread.return_value.start.assert_called_once()
113
-
114
-
def test_challenge_tokens_storage(self, cert_manager):
115
-
"""Test that challenge tokens are stored correctly."""
116
-
cert_manager.challenge_tokens["test_token"] = "test_response"
117
-
assert cert_manager.challenge_tokens["test_token"] == "test_response"
118
-
119
-
@patch('netdata_zulip_bot.cert_manager.client.ClientV2')
120
-
@patch('netdata_zulip_bot.cert_manager.client.ClientNetwork')
121
-
def test_obtain_certificate_mock(self, mock_network, mock_client, cert_manager):
122
-
"""Test certificate obtaining with mocked ACME client."""
123
-
# This is a simplified test that mocks the ACME interaction
124
-
# In production, this would interact with Let's Encrypt staging server
125
-
126
-
# Mock that certificate doesn't need renewal
127
-
with patch.object(cert_manager, 'needs_renewal', return_value=False):
128
-
paths = cert_manager.obtain_certificate()
129
-
assert paths == (cert_manager.cert_path, cert_manager.key_path, cert_manager.fullchain_path)