Automations and webhooks for the AT Protocol airglow.run

test(runtime): deflake the startWeb keep-alive stop test #54

Closed tyb.there-hugo.dev
Closed opened by tyb.there-hugo.dev

lib/runtime/web.test.ts > startWeb stop > "closes keep-alive connections that keep sending requests" fails intermittently with AssertionError: expected 0 to be greater than 0 on expect(closedResponses.length).toBeGreaterThan(0).

How I know: running pnpm exec vp test lib/runtime/web.test.ts -t "keep-alive connections that keep sending" ten times in a row gave 3 failures and 7 passes. The test builds a plain Hono app and does not touch mountBuiltAssets, so the flake does not come from the /static/* cache-header change. It also failed once in a full-file run.

Where to start: the test only counts responses that come back with Connection: close once stopping is set. If the request in flight when stop() begins is the last one before the server closes the socket, or if the loop's next request hits the closed socket first and errors, no Connection: close response is recorded. Either guarantee that a request is in flight when stop() runs (for example, start stop() from inside the /busy handler), or assert on the connection closing rather than on a counted response.

Found by the loop while working on an issue, rkey 3myla7csycu22.

Queued: the loop will work on this version of the issue.

loop:queued cid=bafyreih2j2r4mwm2hqu6s44dybs4pfdiqwphqcatxklfjktlaboy3og3zi

Working on this, attempt 1.

loop:claim run=contribute-20261024-144625-3mylcd5pa3w2y branch=loop/3mylcd5pa3w2y attempt=1 issue=54

test(runtime): deflake the startWeb keep-alive stop test

The "closes keep-alive connections that keep sending requests" test started stop() from outside the server. When stop() landed between two requests, the pooled connection was idle, server.close() dropped it, and no response ever carried Connection: close. The test now starts stop() from inside a /busy handler, so a request is always in flight when it runs, and it asserts that this exact response comes back with Connection: close and ends the traffic.

Only lib/runtime/web.test.ts changes; startWeb is untouched.

Verification #

  • Before the change: 2 of 15 runs of pnpm exec vp test lib/runtime/web.test.ts -t "keep-alive connections that keep sending" failed with the error from the issue.
  • After the change: that command passed in 30 of 30 runs, the whole file passed in 10 of 10 runs one after another, and it passed in every run of 4 at once. When 8 ran at once, the container killed half of them for memory (exit 137), so those runs gave no result.
  • Mutation check: I removed the res.setHeader("Connection", "close") that stop() applies to in-flight responses, and the new test fails (expected { body: 'done', connection: 'close' } to deeply equal { body: 'began stop', ... }). An earlier draft only checked that the last response closed. It still passed under this mutation, because the next request on the socket picks up Connection: close from the stopping branch. So the test now tags the response that started stop() and checks that one.
  • pnpm run check: passes. pnpm run test: 161 files, 2538 tests, all passing.

Acceptance criteria #

Decisions the owner can reverse #

  • The test now checks the specific in-flight response, not "any Connection: close response after stopping". This is stricter: it fails if stop() stops marking in-flight responses, even when the stopping branch for new requests still works.

Review: https://tangled.org/exosphere.site/airglow/compare/main...loop/3mylcd5pa3w2y

loop:pushed run=contribute-20261024-144625-3mylcd5pa3w2y sha=c476afc0ae605540357a91ec4ff4c1055c3247bc

Verdict: merge. The branch fixes the flaky "closes keep-alive connections that keep sending requests" test the way the issue suggested. stop() now starts from inside a /busy handler, so a request is always in flight when it runs. The test then asserts that this exact response comes back with Connection: close and is the last response of the traffic. Only lib/runtime/web.test.ts changes, and startWeb is untouched.

What I checked:

  • I read the new test against startWeb.stop() in lib/runtime/web.ts. The request is in inFlight and its headers are not sent yet when stop() runs from the handler, so stop() sets Connection: close on it. The agent then drops the socket. server.close() has already stopped the listener, so the loop's next request fails and the traffic ends. The race the issue describes (stop landing between two requests on an idle socket) can no longer happen. The assertion is also stricter than before, because it checks the specific in-flight response.
  • pnpm run check passes. pnpm run test passes: 161 files, 2538 tests.
  • The target test (vp test lib/runtime/web.test.ts -t "keep-alive connections that keep sending") passed 20 of 20 runs in a row. The whole file passed 9 of 9 runs, done 3 at a time to add load.
  • Scope: the diff touches one test file and nothing else. There is no followup.md.

loop:verdict run=review-20261024-150220-3mylcd5pa3w2y sha=c476afc0ae605540357a91ec4ff4c1055c3247bc verdict=merge

Merged main into the branch: 8404a4829edb494f15344bc59beaf318d6ef0b9b.

loop:carry from=c476afc0ae605540357a91ec4ff4c1055c3247bc sha=8404a4829edb494f15344bc59beaf318d6ef0b9b

Merged main into the branch: 408b7af0808a2d3591250297f3577aef136faf84.

loop:carry from=8404a4829edb494f15344bc59beaf318d6ef0b9b sha=408b7af0808a2d3591250297f3577aef136faf84

Merged onto main as 9c92802400f8ff08328c265c75a4df535037f6ba.

loop:merged sha=9c92802400f8ff08328c265c75a4df535037f6ba head=408b7af0808a2d3591250297f3577aef136faf84

Sign up or Sign in to add to the discussion.