search for standard sites pub-search.waow.tech
search zig blog atproto
11
fork

Configure Feed

Select the types of activity you want to include in your feed.

docs: add zig logfire client design notes

practitioner notes for a potential official logfire client for zig,
referencing find-bufo rust implementation and OTLP protocol details.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

zzstoatzz 1e29cd8c 1f07b37f

+259
+259
docs/scratch/zig-logfire-client.md
··· 1 + # zig logfire client - design notes 2 + 3 + a potential official logfire client for zig. this document captures what would be useful from a practitioner's perspective, referencing existing implementations and protocol details. 4 + 5 + ## motivation 6 + 7 + logfire has official clients for python, rust, and typescript. a zig client would enable observability for zig applications (like this one - leaflet-search) that want to participate in the pydantic/logfire ecosystem. 8 + 9 + zig's growing use in performance-critical systems (databases, network services, game engines) makes it a natural fit. these are exactly the applications where observability matters most. 10 + 11 + ## what logfire actually is 12 + 13 + logfire is an opinionated wrapper around opentelemetry. it's not a replacement - it's a distribution that: 14 + 15 + 1. simplifies configuration (single `configure()` call vs. wiring up providers/exporters manually) 16 + 2. provides ergonomic macros for structured logging and spans 17 + 3. sends data to logfire's platform (or any OTLP-compatible backend) 18 + 4. follows opentelemetry standards (OTLP protocol, semantic conventions, W3C trace context) 19 + 20 + the key insight: you don't need to implement opentelemetry from scratch. you need to implement a thin wrapper that configures OTLP export and provides nice ergonomics. 21 + 22 + ## protocol details 23 + 24 + from the [logfire alternative clients documentation](https://logfire.pydantic.dev/docs/how-to-guides/alternative-clients/): 25 + 26 + **endpoints:** 27 + - US: `https://logfire-us.pydantic.dev` 28 + - EU: `https://logfire-eu.pydantic.dev` 29 + - signals: `/v1/traces`, `/v1/metrics`, `/v1/logs` 30 + 31 + **authentication:** 32 + - header: `Authorization: {write-token}` 33 + - token obtained from logfire project settings 34 + 35 + **protocol:** 36 + - HTTP with protobuf encoding (`http/protobuf`) - preferred 37 + - HTTP with JSON encoding (`http/json`) - also supported 38 + - no gRPC requirement 39 + 40 + ## reference: rust client architecture 41 + 42 + the [logfire-rust](https://github.com/pydantic/logfire-rust) client (used in find-bufo) demonstrates the pattern: 43 + 44 + ``` 45 + src/ 46 + ├── lib.rs # public API, re-exports 47 + ├── config.rs # LogfireConfigBuilder 48 + ├── exporters.rs # OTLP exporter setup 49 + ├── logfire.rs # core Logfire struct 50 + ├── macros/ # span!(), info!(), debug!(), etc. 51 + ├── metrics.rs # counter, gauge, histogram 52 + ├── bridges/ # integration with tracing/log crates 53 + └── internal/ # implementation details 54 + ``` 55 + 56 + key design decisions from rust client: 57 + 58 + 1. **builder pattern for configuration** 59 + ```rust 60 + let logfire = logfire::configure() 61 + .with_default_level_filter(LevelFilter::INFO) 62 + .finish()?; 63 + ``` 64 + 65 + 2. **shutdown guard for clean exit** 66 + ```rust 67 + let _guard = logfire.shutdown_guard(); 68 + // spans/logs flushed when guard drops 69 + ``` 70 + 71 + 3. **structured logging macros** 72 + ```rust 73 + logfire::info!("search completed", 74 + query = &query_text, 75 + results_count = count as i64 76 + ); 77 + ``` 78 + 79 + 4. **span creation with attributes** 80 + ```rust 81 + let _span = logfire::span!( 82 + "turbopuffer.vector_search", 83 + query = &query, 84 + top_k = k as i64 85 + ).entered(); 86 + ``` 87 + 88 + 5. **wraps opentelemetry, doesn't replace it** 89 + - uses `opentelemetry-otlp` crate for export 90 + - uses `tracing` crate for span/event capture 91 + - provides bridge to integrate with existing tracing code 92 + 93 + ## what would be useful for zig 94 + 95 + ### configuration 96 + 97 + ```zig 98 + const logfire = @import("logfire"); 99 + 100 + pub fn main() !void { 101 + const lf = try logfire.configure(.{ 102 + .service_name = "leaflet-search", 103 + .default_level = .info, 104 + }); 105 + defer lf.shutdown(); 106 + 107 + // ... 108 + } 109 + ``` 110 + 111 + environment variables should work automatically: 112 + - `LOGFIRE_TOKEN` - required for sending to logfire 113 + - `LOGFIRE_SERVICE_NAME` - optional override 114 + - `OTEL_EXPORTER_OTLP_ENDPOINT` - for custom backends 115 + 116 + ### spans 117 + 118 + ```zig 119 + const span = logfire.span("sync.full_sync", .{ 120 + .doc_count = doc_count, 121 + .pub_count = pub_count, 122 + }); 123 + defer span.end(); 124 + 125 + // work happens here 126 + ``` 127 + 128 + or with a callback pattern: 129 + 130 + ```zig 131 + try logfire.withSpan("db.query", .{ .sql = sql }, struct { 132 + fn execute(ctx: *Context) !void { 133 + // work 134 + } 135 + }.execute, &ctx); 136 + ``` 137 + 138 + ### structured logging 139 + 140 + ```zig 141 + logfire.info("search completed", .{ 142 + .query = query, 143 + .results_count = @intCast(results.len), 144 + .latency_ms = timer.read() / std.time.ns_per_ms, 145 + }); 146 + 147 + logfire.err("sync failed", .{ 148 + .error = @errorName(err), 149 + .offset = offset, 150 + }); 151 + ``` 152 + 153 + the key is compile-time type safety on the structured fields while producing OTLP-compatible attribute encoding. 154 + 155 + ### metrics 156 + 157 + ```zig 158 + const search_latency = logfire.histogram("search.latency_ms", .{ 159 + .unit = "ms", 160 + .description = "search request latency", 161 + }); 162 + 163 + // later 164 + search_latency.record(elapsed_ms, .{ .endpoint = "/search" }); 165 + ``` 166 + 167 + ### what i'd actually use day-to-day 168 + 169 + from working on leaflet-search, the most common patterns are: 170 + 171 + 1. **timing operations** 172 + ```zig 173 + const span = logfire.span("turso.query", .{ .sql = sql[0..@min(sql.len, 100)] }); 174 + defer span.end(); 175 + ``` 176 + 177 + 2. **logging with context** 178 + ```zig 179 + logfire.info("sync complete", .{ 180 + .docs = doc_count, 181 + .pubs = pub_count, 182 + .duration_ms = duration, 183 + }); 184 + ``` 185 + 186 + 3. **error tracking** 187 + ```zig 188 + logfire.err("query failed", .{ 189 + .error = @errorName(err), 190 + .query = truncated_query, 191 + }); 192 + ``` 193 + 194 + 4. **request tracing** (if building HTTP services) 195 + - trace ID propagation via headers 196 + - automatic span creation for requests 197 + 198 + ## implementation considerations 199 + 200 + ### OTLP encoding 201 + 202 + protobuf is preferred but requires either: 203 + - generated zig code from `.proto` files 204 + - hand-rolled protobuf encoder (OTLP schema is well-documented) 205 + 206 + JSON encoding is simpler to implement and logfire supports it. could start with JSON and add protobuf later. 207 + 208 + ### batching and export 209 + 210 + spans/logs should be batched and exported asynchronously to avoid blocking application code. this is where zig's async/threading model matters: 211 + 212 + - batch buffer with configurable size/timeout 213 + - background thread or async task for export 214 + - graceful shutdown (flush on exit) 215 + 216 + ### allocator design 217 + 218 + zig clients need to be explicit about allocation. options: 219 + 220 + 1. require allocator passed to `configure()` 221 + 2. use arena per batch, free on export 222 + 3. fixed-size buffers with overflow handling 223 + 224 + ### error handling 225 + 226 + zig's explicit error handling is a strength here. export failures shouldn't crash the application: 227 + 228 + ```zig 229 + lf.flush() catch |err| { 230 + std.log.warn("logfire export failed: {}", .{err}); 231 + }; 232 + ``` 233 + 234 + ## prior art 235 + 236 + - [logfire-rust](https://github.com/pydantic/logfire-rust) - official rust client 237 + - [opentelemetry-zig](https://github.com/open-telemetry/opentelemetry-zig) - community otel implementation (incomplete) 238 + - [zig-opentelemetry](https://github.com/baaalad/zig-opentelemetry) - another community attempt 239 + 240 + the opentelemetry-zig implementations are incomplete, which is actually an opportunity - a logfire-zig client could become the de facto OTLP implementation for zig. 241 + 242 + ## references 243 + 244 + - [logfire docs](https://logfire.pydantic.dev/docs/) 245 + - [alternative clients guide](https://logfire.pydantic.dev/docs/how-to-guides/alternative-clients/) 246 + - [OTLP specification](https://opentelemetry.io/docs/specs/otlp/) 247 + - [find-bufo](https://github.com/zzstoatzz/find-bufo) - rust app using logfire (src/main.rs, src/search.rs) 248 + - [plyr.fm logfire docs](../scratch/../../../plyr.fm/docs/tools/logfire.md) - querying patterns 249 + 250 + ## open questions 251 + 252 + 1. should this be `logfire-zig` (pydantic-branded) or `zig-logfire` (community)? 253 + 2. JSON-first or protobuf-first? 254 + 3. how to handle trace context propagation for HTTP frameworks that don't exist yet in zig? 255 + 4. should it integrate with zig's `std.log` or provide its own logging? 256 + 257 + --- 258 + 259 + *these are practitioner notes, not a specification. the goal is to inform implementation decisions while leaving room for the implementer to make the right choices.*