update for release · anil.recoil.org/zulip-netdata-bot@6dac20d

+1 -1

CLAUDE.md

··· 1 1 # Netdata Zulip Bot - Development Instructions 2 2 3 - This repository implements a Zulip bot that receives incoming webhook notifications from Netdata Cloud and posts the resulting notifications to a Zulip topic. 3 + This repository implements a production-ready Zulip bot that receives incoming webhook notifications from Netdata Cloud and posts the resulting notifications to a Zulip topic. 4 4 5 5 ## Core Requirements 6 6

+21

LICENSE.md

··· 1 + MIT License 2 + 3 + Copyright (c) 2025 Anil Madhavapeddy 4 + 5 + Permission is hereby granted, free of charge, to any person obtaining a copy 6 + of this software and associated documentation files (the "Software"), to deal 7 + in the Software without restriction, including without limitation the rights 8 + to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 + copies of the Software, and to permit persons to whom the Software is 10 + furnished to do so, subject to the following conditions: 11 + 12 + The above copyright notice and this permission notice shall be included in all 13 + copies or substantial portions of the Software. 14 + 15 + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 + AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 + OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 + SOFTWARE.

+104 -103

README.md

··· 1 1 # Netdata Zulip Bot 2 2 3 - *100% vibe coded, use at your peril* 4 - 5 - A webhook service that receives notifications from Netdata Cloud and forwards them to Zulip channels. Features HTTPS with Let's Encrypt certificates and mutual TLS authentication for secure communication with Netdata Cloud. 3 + A production-ready webhook service that receives notifications from Netdata Cloud and forwards them to Zulip channels. Designed to run behind a reverse proxy (like Caddy) that handles HTTPS and mutual TLS authentication. 6 4 7 5 ## Features 8 6 9 - - 🔐 **Automated SSL Certificates**: Built-in Let's Encrypt integration with automatic renewal 10 - - 🤝 **Mutual TLS**: Secure authentication with Netdata Cloud 7 + - 🔗 **Reverse Proxy Ready**: HTTP service designed to run behind Caddy/nginx 8 + - 🤝 **Mutual TLS Support**: When configured with reverse proxy 11 9 - 📊 **Rich Formatting**: Beautiful Zulip messages with emojis and markdown 12 10 - 🏷️ **Topic Organization**: Automatic topic routing by severity level 13 11 - 📝 **Structured Logging**: JSON-structured logs for monitoring 14 12 - ⚡ **High Performance**: FastAPI-based webhook endpoint 15 - - 🚀 **Standalone**: No external dependencies like certbot required 13 + - 🔧 **Flexible Configuration**: Support for .zuliprc files or environment variables 14 + - ✅ **Webhook Verification**: Built-in Netdata challenge/response handling 16 15 17 16 ## Quick Start 18 17 ··· 21 20 ```bash 22 21 # Using uv (recommended) 23 22 uv sync 24 - 25 - # Or using pip 26 - pip install -e . 27 23 ``` 28 24 29 25 ### 2. Create Configuration 30 26 31 27 ```bash 32 28 # Generate sample configuration files 33 - netdata-zulip-bot --create-config 29 + uv run netdata-zulip-bot --create-config 34 30 35 31 # Copy and customize 36 32 cp .zuliprc.sample ~/.zuliprc 33 + cp .env.sample .env 37 34 ``` 38 35 39 36 ### 3. Configure Zulip Settings ··· 48 45 stream=netdata-alerts 49 46 ``` 50 47 51 - ### 4. Set Server Environment Variables 48 + ### 4. Configure Environment Variables 49 + 50 + Edit `.env` file or set environment variables: 52 51 53 52 ```bash 54 - export SERVER_DOMAIN=your-webhook-domain.com 55 - export SERVER_PORT=8443 56 - export SERVER_ENABLE_MTLS=true 53 + # Server configuration (HTTP only) 54 + export SERVER_HOST=0.0.0.0 55 + export SERVER_PORT=8080 57 56 58 - # For automated SSL certificates (recommended) 59 - export SERVER_AUTO_CERT=true 60 - export SERVER_CERT_EMAIL=admin@example.com 61 - # Use staging for testing (optional) 62 - export SERVER_CERT_STAGING=false 57 + # Required: Netdata webhook challenge secret 58 + export SERVER_CHALLENGE_SECRET=your-challenge-secret-here 59 + 60 + # Optional: Override Zulip stream 61 + export ZULIP_STREAM=netdata-alerts 63 62 ``` 64 63 65 64 ### 5. Run the Service 66 65 67 66 ```bash 68 - # With automated SSL certificates 69 - netdata-zulip-bot 67 + # Start the HTTP service 68 + uv run netdata-zulip-bot 70 69 71 - # The bot will automatically: 72 - # 1. Obtain SSL certificates from Let's Encrypt 73 - # 2. Start the HTTPS server 74 - # 3. Renew certificates before expiration 70 + # Or with custom configuration 71 + uv run netdata-zulip-bot --zuliprc /path/to/.zuliprc 72 + 73 + # The service runs on HTTP (default: localhost:8080) 74 + # Use a reverse proxy like Caddy for HTTPS and mutual TLS 75 75 ``` 76 76 77 77 ## Configuration ··· 101 101 export ZULIP_STREAM=netdata-alerts 102 102 ``` 103 103 104 - Use `--env-config` flag to use environment variables instead of zuliprc. 104 + Use the `--env-config` flag to use environment variables instead of zuliprc: 105 + 106 + ```bash 107 + uv run netdata-zulip-bot --env-config 108 + ``` 105 109 106 110 ### Server Configuration 107 111 108 112 Set these environment variables: 109 113 110 - - `SERVER_DOMAIN`: Your public domain (required) 111 114 - `SERVER_HOST`: Bind address (default: `0.0.0.0`) 112 - - `SERVER_PORT`: HTTPS port (default: `8443`) 113 - - `SERVER_ENABLE_MTLS`: Enable mutual TLS (default: `true`) 115 + - `SERVER_PORT`: HTTP port (default: `8080`) 116 + - `SERVER_CHALLENGE_SECRET`: Netdata webhook challenge secret (required) 114 117 115 - #### Automated SSL Configuration (Recommended) 118 + ### Reverse Proxy Setup 116 119 117 - - `SERVER_AUTO_CERT`: Enable automatic certificate management (default: `false`) 118 - - `SERVER_CERT_EMAIL`: Email for Let's Encrypt account (required when auto_cert is true) 119 - - `SERVER_CERT_PATH`: Directory for storing certificates (default: `./certs`) 120 - - `SERVER_CERT_STAGING`: Use Let's Encrypt staging server for testing (default: `false`) 121 - - `SERVER_ACME_PORT`: Port for ACME HTTP-01 challenge (default: `80`) 120 + The bot is designed to run behind a reverse proxy that handles HTTPS and mutual TLS: 122 121 123 - #### Manual SSL Configuration 122 + #### Using Caddy (Recommended) 124 123 125 - If not using automated certificates: 126 - - `SERVER_CERT_PATH`: Path to certificate directory 127 - - Place `fullchain.pem` and `privkey.pem` in `{SERVER_CERT_PATH}/{SERVER_DOMAIN}/` 124 + 1. Update `Caddyfile` with your domain name 125 + 2. Place Netdata CA certificate in `netdata-ca.pem` 126 + 3. Run both services: 127 + 128 + ```bash 129 + # Start the bot 130 + uv run netdata-zulip-bot & 131 + 132 + # Start Caddy 133 + caddy run --config Caddyfile 134 + ``` 135 + 136 + #### Using Docker Compose 137 + 138 + ```bash 139 + docker-compose up -d 140 + ``` 128 141 129 142 ## Message Format 130 143 ··· 146 159 **Time:** 2024-01-15 14:30:00 UTC 147 160 148 161 **Details:** CPU usage has exceeded 90% for 5 minutes 162 + 149 163 **Summary:** Critical alert: High CPU usage detected 150 164 151 165 [View Alert](https://app.netdata.cloud/spaces/...) ··· 171 185 172 186 ### Systemd Service 173 187 174 - Create `/etc/systemd/system/netdata-zulip-bot.service`: 188 + See `examples/netdata-zulip-bot.service` for a complete systemd service configuration. 175 189 176 - ```ini 177 - [Unit] 178 - Description=Netdata Zulip Bot 179 - After=network.target 190 + ### Automated Setup 180 191 181 - [Service] 182 - Type=simple 183 - User=netdata-bot 184 - WorkingDirectory=/opt/netdata-zulip-bot 185 - Environment=SERVER_DOMAIN=your-domain.com 186 - ExecStart=/opt/netdata-zulip-bot/venv/bin/netdata-zulip-bot 187 - Restart=always 188 - RestartSec=5 192 + Use the provided setup script: 189 193 190 - [Install] 191 - WantedBy=multi-user.target 192 - ``` 193 - 194 - Enable and start: 195 194 ```bash 196 - sudo systemctl enable netdata-zulip-bot 197 - sudo systemctl start netdata-zulip-bot 195 + sudo ./scripts/setup.sh --domain your-domain.com --email admin@example.com 198 196 ``` 199 197 200 198 ### Docker 201 199 202 - ```dockerfile 203 - FROM python:3.11-slim 200 + The included `Dockerfile` and `docker-compose.yml` provide a complete setup with Caddy reverse proxy: 204 201 205 - WORKDIR /app 206 - COPY . . 207 - RUN pip install -e . 208 - 209 - EXPOSE 8443 210 - 211 - CMD ["netdata-zulip-bot"] 202 + ```bash 203 + docker-compose up -d 212 204 ``` 213 205 214 206 ## Security 215 207 216 - ### SSL Certificate Management 208 + ### Architecture 217 209 218 - The bot includes fully automated SSL certificate management: 210 + The bot uses a security-focused architecture: 219 211 220 - 1. **Automatic Provisioning**: Obtains certificates from Let's Encrypt on first run 221 - 2. **Automatic Renewal**: Checks daily and renews certificates 30 days before expiration 222 - 3. **Zero Downtime**: Certificate renewal happens in the background 223 - 4. **ACME HTTP-01 Challenge**: Built-in challenge server (requires port 80 access) 212 + 1. **HTTP Backend**: Simple HTTP service with no direct internet exposure 213 + 2. **Reverse Proxy**: Caddy handles HTTPS, certificates, and client authentication 214 + 3. **Mutual TLS**: Client certificate validation at the reverse proxy level 224 215 225 - ### Mutual TLS Authentication 216 + ### Webhook Security 226 217 227 - The service supports mutual TLS to authenticate Netdata Cloud webhooks: 218 + - **Challenge/Response**: Built-in Netdata webhook verification using HMAC-SHA256 219 + - **Payload Validation**: Strict payload parsing and validation 220 + - **Request Logging**: Comprehensive logging of all webhook requests 221 + - **Error Handling**: Secure error responses without information disclosure 228 222 229 - 1. **Server Certificate**: Automatically managed via built-in ACME client 230 - 2. **Client Verification**: Validates Netdata's client certificate 231 - 3. **CA Certificate**: Built-in Netdata CA certificate for client validation 223 + ### SSL Certificate Management 232 224 233 - ### Webhook Endpoint Security 225 + SSL certificates are managed by the reverse proxy (Caddy): 234 226 235 - - HTTPS-only communication 236 - - Request logging and monitoring 237 - - Payload validation and sanitization 238 - - Error handling without information disclosure 227 + 1. **Automatic Provisioning**: Caddy obtains Let's Encrypt certificates 228 + 2. **Automatic Renewal**: Built-in certificate renewal 229 + 3. **Mutual TLS**: Client certificate validation using Netdata CA certificate 239 230 240 231 ## Monitoring 241 232 ··· 255 246 ### Health Check 256 247 257 248 ```bash 258 - curl -k https://your-domain.com:8443/health 249 + # Direct HTTP check (backend service) 250 + curl http://localhost:8080/health 251 + 252 + # Through reverse proxy 253 + curl https://your-domain.com/health 259 254 ``` 260 255 261 256 Response: ··· 271 266 ### Running Tests 272 267 273 268 ```bash 274 - pytest 269 + uv run python -m pytest tests/ -v 275 270 ``` 276 271 277 272 ### Code Formatting 278 273 279 274 ```bash 280 - black . 281 - ruff check . 275 + uv run black . 276 + uv run ruff check . 282 277 ``` 283 278 284 279 ### Local Development 285 280 286 - For development, you can disable HTTPS and mTLS: 281 + For development, you can run the HTTP service directly: 287 282 288 283 ```bash 289 - export SERVER_ENABLE_MTLS=false 290 - # Use HTTP for testing (not recommended for production) 284 + # Set required environment variables 285 + export SERVER_CHALLENGE_SECRET=test-secret 286 + 287 + # Run the service 288 + uv run netdata-zulip-bot 289 + 290 + # Test webhook endpoint 291 + curl -X POST http://localhost:8080/webhook/netdata?crc_token=test123 291 292 ``` 292 293 293 294 ## Troubleshooting 294 295 295 296 ### Common Issues 296 297 297 - 1. **Certificate Issues** 298 - - For automated certs: Ensure port 80 is accessible for ACME challenges 299 - - Domain must point to your server's IP address 300 - - Check `SERVER_CERT_EMAIL` is set for auto-cert mode 301 - - Use `SERVER_CERT_STAGING=true` for testing to avoid rate limits 298 + 1. **Configuration Issues** 299 + - Ensure `SERVER_CHALLENGE_SECRET` is set (required for Netdata webhook verification) 300 + - Verify `.zuliprc` file contains all required fields 301 + - Check that Zulip bot has permission to post to the configured stream 302 302 303 - 2. **Zulip Connection Failed** 304 - - Verify API credentials in zuliprc 305 - - Test connection with Zulip's API 303 + 2. **Reverse Proxy Issues** 304 + - Ensure Caddy configuration uses correct domain name 305 + - Verify Netdata CA certificate is properly configured 306 + - Check that port 80 is accessible for Let's Encrypt challenges 306 307 307 308 3. **Webhook Not Receiving Data** 308 - - Check firewall settings for port 8443 309 - - Verify domain DNS resolution 310 - - Check Netdata Cloud webhook configuration 309 + - Verify Netdata Cloud webhook URL points to your reverse proxy 310 + - Check webhook challenge secret matches configuration 311 + - Review service logs for error messages 311 312 312 313 ### Logs 313 314 ··· 318 319 319 320 ## License 320 321 321 - MIT License - see LICENSE file for details. 322 + MIT License - see LICENSE file for details.

+1 -1

pyproject.toml

··· 3 3 version = "0.1.0" 4 4 description = "Zulip bot for receiving Netdata Cloud webhook notifications" 5 5 authors = [ 6 - {name = "Your Name", email = "your.email@example.com"} 6 + {name = "Anil Madhavapeddy", email = "anil@recoil.org"} 7 7 ] 8 8 readme = "README.md" 9 9 requires-python = ">=3.11"

Configure Feed

Configure Feed