we (web engine): Experimental web browser project to understand the limits of Claude
2
fork

Configure Feed

Select the types of activity you want to include in your feed.

Integrate Streams API with fetch: Response.body and streaming Request bodies #185

open opened by pierrelf.com

Goal#

Expose HTTP response bodies as ReadableStream instances and accept ReadableStream request bodies in fetch(), replacing the current buffer-everything model in we-js/src/fetch.rs.

Depends on: "Implement Streams API foundation" and "Implement Streams API: pipeTo, pipeThrough, tee, and byte streams".

Scope#

  • Response.body
    • Returns a ReadableStream of Uint8Array chunks instead of being absent
    • Response.body is locked once a reader is acquired or once a body-consuming method (text(), json(), arrayBuffer(), blob(), formData()) is called
    • Response.bodyUsed getter reflects lock/consumption state
    • Response clone produces a tee() of the body
  • HTTP layer
    • Plumb Transfer-Encoding: chunked and Content-Length framing in we-net so chunks are surfaced to the JS side as they arrive, rather than buffered
    • A new internal BodyChunkChannel (or similar) that the HTTP client uses to push chunks; the JS side wraps it in a ReadableStream underlying source
    • Cancellation: cancelling the ReadableStream aborts the underlying HTTP request and closes the connection (or returns it to the pool if appropriate)
  • Request bodies from streams
    • new Request(url, { body: readableStream, duplex: 'half' })
    • fetch(url, { body: readableStream })
    • HTTP client reads the stream and writes via Transfer-Encoding: chunked
    • Errors on the stream abort the request
  • Response.body.tee() works for caching one branch while parsing the other
  • Backpressure: if the JS reader is slow, the network read loop pauses (i.e. stops reading from the socket) until the stream consumer pulls more

Acceptance Criteria#

  • const reader = (await fetch(url)).body.getReader(); yields chunks as the HTTP response arrives, not buffered to the end
  • A 10 MB response can be streamed and processed chunk-by-chunk with only modest memory use
  • Cancelling the response body stream closes the underlying TCP connection (or returns it to the pool with the connection drained, never both)
  • fetch(url, { body: stream, method: 'POST' }) uploads chunks as they are produced
  • response.text() after response.body.getReader() throws TypeError (body locked)
  • Same-origin and CORS checks already in fetch.rs continue to apply
  • Unit and integration tests for: streaming download, streaming upload, body locking, tee on response body, cancellation propagating to the network layer
  • Conventions
    • cargo fmt --all --check
    • cargo clippy --workspace -- -D warnings
    • cargo test --workspace
sign up or login to add to the discussion
Labels

None yet.

assignee

None yet.

Participants 1
AT URI
at://did:plc:meotu43t6usg4qdwzenk4s2t/sh.tangled.repo.issue/3mly7qguqf724