we (web engine): Experimental web browser project to understand the limits of Claude
2
fork

Configure Feed

Select the types of activity you want to include in your feed.

Implement built-in transform streams: TextDecoderStream, TextEncoderStream, CompressionStream, DecompressionStream #184

open opened by pierrelf.com

Goal#

Implement the four standard built-in transform streams that web pages most commonly use through the Streams API.

Depends on: "Implement Streams API foundation" and "Implement Streams API: pipeTo, pipeThrough, tee, and byte streams".

Scope#

  • TextDecoderStream
    • Constructor: new TextDecoderStream(label = 'utf-8', options?) with { fatal, ignoreBOM }
    • Wraps a streaming TextDecoder (reuse we-encoding)
    • readable produces string chunks; writable accepts Uint8Array / ArrayBuffer / ArrayBufferView
    • Handles boundary splits across multi-byte sequences (UTF-8, UTF-16)
    • encoding, fatal, ignoreBOM getters
  • TextEncoderStream
    • Constructor: new TextEncoderStream()
    • writable accepts string chunks, readable produces Uint8Array chunks of UTF-8 bytes
    • Handles high-surrogate-then-low-surrogate splits across chunk boundaries
    • encoding getter (always 'utf-8')
  • CompressionStream
    • Constructor: new CompressionStream(format) where format is 'gzip' | 'deflate' | 'deflate-raw'
    • Streaming DEFLATE compressor — reuse and, if necessary, extend the Phase 7 DEFLATE implementation in we-image to support streaming compression (move shared code into a small we-deflate module under an existing crate if cleaner; do not introduce a new crate dependency)
    • Emits zlib (deflate) or gzip wrappers as appropriate
  • DecompressionStream
    • Constructor: new DecompressionStream(format)
    • Streaming DEFLATE decompressor reading the Phase 7 wrapper formats
  • All four streams must operate on a per-chunk basis with no full-buffering of input

Acceptance Criteria#

  • response.body.pipeThrough(new TextDecoderStream()).pipeTo(...) produces a stream of strings
  • Split-UTF-8 chunks (e.g. half of a 3-byte sequence per chunk) decode correctly without emitting replacement characters when not at end-of-stream
  • new Blob([...]).stream().pipeThrough(new CompressionStream('gzip')) round-trips through DecompressionStream('gzip') to the original bytes
  • 'deflate' and 'deflate-raw' formats round-trip
  • Invalid format strings throw TypeError from the constructor
  • Unit tests: encoder/decoder boundary splits, all three compression formats with multi-chunk inputs, error on malformed compressed input
  • Conventions
    • cargo fmt --all --check
    • cargo clippy --workspace -- -D warnings
    • cargo test --workspace
    • Zero external dependencies; streaming compressor implemented in pure Rust
sign up or login to add to the discussion
Labels

None yet.

assignee

None yet.

Participants 1
AT URI
at://did:plc:meotu43t6usg4qdwzenk4s2t/sh.tangled.repo.issue/3mly7pv7ysb2n