Flushing subnormals costs the compiled tier little, and a zero nothing #3

Playing a patch made of the analog built-ins, the Rust engine's compiled tier spends a quarter to a third of its time on the flush rule (a subnormal becomes a zero of its sign). A patch with nothing played spends more than half of its time there, because an exact zero takes the rule's rare path every time one is made: the reference patch might-as-well costs more silent than with eight notes held.

Two parts, and the first can land without the second:

1. A zero stays on the common path, and the rare half is a branch. Small. The rule is kept exactly, the compiled tier's tests pass, and it is measured below.
2. The rest of what flushing costs the compiled tier. Larger. One way of doing it needs the maintainer's decision first.

Everything here was measured on engine 764c6e7, release build, on one core of an Apple M5 (macOS 27.2, rustc 1.99.1, wasmtime 50.0.1 with Cranelift 0.137.1). The patch every variant below was built from is in the first comment, so nothing depends on anyone's machine.

What was measured #

Whole patches #

atmo-render --tier compiled, 48 kHz, 512-frame blocks, four of the reference patches in assets/default-patches/. No note, one note (60), or eight (48, 52, 55, 60, 64, 67, 72, 76), struck at the start and held to a second before the end. Each figure is the cost of one rendered second: a 23 s render less a 3 s one, over 20, so that starting up is not in it. The first number is instructions retired, in millions, which do not depend on what else the machine does. The second is CPU time as a share of one core, the best of five runs with the variants taken in turn.

patch	notes held	as it is	a zero kept out	subnormals kept	kept, signals on
keys	none	750 M, 2.80%	536 M, 1.95%	296 M, 1.00%	250 M, 0.90%
keys	one	785 M, 3.10%	826 M, 3.10%	474 M, 1.85%	413 M, 1.65%
keys	eight	2,642 M, 10.80%	2,855 M, 10.85%	1,728 M, 7.45%	1,558 M, 7.30%
bass	none	119 M, 0.55%	118 M, 0.55%	118 M, 0.60%	118 M, 0.55%
bass	one	429 M, 2.10%	457 M, 2.05%	340 M, 1.70%	323 M, 1.70%
bass	eight	2,321 M, 11.85%	2,545 M, 11.95%	1,613 M, 9.00%	1,479 M, 8.95%
strings	none	521 M, 1.95%	404 M, 1.40%	240 M, 0.85%	203 M, 0.80%
strings	one	692 M, 2.60%	736 M, 2.60%	474 M, 1.75%	412 M, 1.65%
strings	eight	2,915 M, 11.05%	3,060 M, 11.00%	2,109 M, 8.40%	1,879 M, 7.70%
might-as-well	none	2,430 M, 8.45%	1,751 M, 4.80%	1,044 M, 3.00%	723 M, 2.20%
might-as-well	one	1,762 M, 5.50%	1,786 M, 5.10%	1,079 M, 3.20%	757 M, 2.45%
might-as-well	eight	1,996 M, 6.95%	2,021 M, 6.50%	1,313 M, 4.60%	991 M, 3.85%

The columns:

• as it is: Floats::Flush with wasmtime's signal handlers off, which is what crates/engine/src/dsp/tier.rs compiles and what a plugin host runs.
• a zero kept out: the same, with the four inline tests of the compiled tier changed so that a zero never calls a helper (see part 1). The renders are the same bytes.
• subnormals kept: every module lowered and compiled with Floats::Ieee instead. This is not a fix, since it is not what a browser's audio thread computes. It is the floor: what the patch costs when the rule costs nothing.
• kept, signals on: the same with Tuning { signals: true }, so wasmtime uses guard pages and not a bounds check in code. Only a host that owns its process can run so.

The dsp nodes each patch runs: keys has osc-analog, contour, filter-ota and amp-contour per note, and chorus-bbd and delay-analog shared. Bass has osc-analog, contour, filter-ladder and amp-contour per note and none shared. Strings has three osc-analog, filter-sem and amp-contour per note, and chorus-bbd shared. Might-as-well has one shared reverb and no other dsp node.

What the table says:

• With eight notes held the rule costs 1.3 to 1.5 times: keys 10.80% against 7.45%, bass 11.85% against 9.00%, strings 11.05% against 8.40%, might-as-well 6.95% against 4.60%.
• With nothing played it costs 2.3 to 2.8 times: keys 2.80% against 1.00%, strings 1.95% against 0.85%, might-as-well 8.45% against 3.00%. Bass has no shared dsp node, so nothing of it runs and nothing changes.
• Might-as-well costs more silent (8.45%) than with eight notes held (6.95%).
• Keeping a zero out takes 30% off silent keys, 28% off silent strings and 43% off silent might-as-well. Played, it takes between nothing and 7%.
• Signals on are worth little beside the rule: nothing to 16% with eight notes held, and up to 27% with nothing played.

One module at a time #

crates/dsp-compiled/examples/speed.rs already measures a module in both Floats modes, with signals off and on, against the interpreter. With the built-ins these patches are made of added to it (in the first comment), in multiples of real time at 48 kHz: one instance, its parameters at their defaults, its inputs fed a sine. Two runs of 20 s of audio each; a range where they differed.

built-in	interpreter, flushing	compiled, flushing (as it is)	compiled, kept	compiled, kept, signals on	kept over flushing
reverb, version 4	3.5	19.8	37 to 38	52	1.9
chorus-bbd, version 3	21	85	152	176	1.8
delay-analog, version 1	36	138 to 144	515 to 521	557 to 568	3.7
filter-ladder, version 6	37	126	200 to 208	189 to 193	1.6
filter-ota, version 1	55	264 to 297	442 to 444	449 to 451	1.5 to 1.7
osc-analog, version 1	100	692	1,153 to 1,201	1,340 to 1,391	1.7
contour, version 1	326	1,943 to 1,979	4,071	4,814 to 4,836	2.1

The interpreter is within a few percent in either mode, as crates/dsp/src/run/mod.rs:47-51 says.

The test itself is nearly free. What generated code does around it is not #

filter-ota's process() was ported by hand to Rust, operation for operation, once with native::fz, native::mul and native::round32 after every operation as the tiers place them, and once with none. After 2,000 calls its output is the compiled tier's bit for bit, in both modes. It runs at 512 and 529 times real time with the tests (the two runs) and at 510 and 531 without.

So the same tests, compiled by rustc into native code, cost nothing that can be measured. In the compiled tier they cost that module 1.5 to 1.7 times.

It also says what is not worth doing. With subnormals kept, the hand-written native port is only 15 to 20% faster than the compiled tier (510 to 531 times real time against 442 to 444). Compiling the built-ins to native code ahead of time would buy little once the rule is cheap.

Why, from the code #

The rule. fixtures/dsp/README.md, Subnormals (lines 207 to 253), and the doc of Floats in crates/dsp/src/run/mod.rs:360-408. A host plays in Floats::Flush, defined to compute what Chrome on arm64 computes with FPCR.FZ set: every float an operator, a Math function or an intrinsic makes is a zero of its sign when its exact value is below the smallest normal; every rounding to f32 likewise; everything entering from outside is flushed as it enters. It is "done in software, never by setting the processor's flush modes" (mod.rs:367), and so the same on every machine.

The interpreter's tests. crates/dsp/src/run/native.rs:

• fz (line 22) tests x.to_bits() & 0x7ff0_0000_0000_0000 == 0, the exponent field alone. Its own comment says "zeros pass it too". The rare half, flush (line 33, #[cold] and #[inline(never)]), then returns a zero unchanged and counts nothing.
• mul (line 59) tests p.to_bits() & 0x7fe0_0000_0000_0000 == 0, an exponent field of 0 or 1, and calls mul_tiny (line 71), which for a zero calls flush in turn.
• round32 (line 113) tests r.to_bits() & 0x7f00_0000 == 0 and calls round32_tiny (line 123), which for a zero calls fz32.
• fz32 (line 136) already keeps a zero out: bits & 0x7f80_0000 == 0 && bits << 1 != 0.

The compiled tier's tests. crates/dsp-compiled/src/codegen.rs emits the same tests inline after each operation, "and the rare flush itself out of line" (its header, lines 40 to 45): fz at line 843, mul at 863, round32 at 896, fz32 at 924. Each is an if with a result and an else, and the rare half of each is a call to a WebAssembly function the module carries (helper_body, line 404), written line for line after native.rs. So in generated code a result of exactly zero is a call: one for fz, and two for a product (Helper::Mul calls Helper::Fz, line 448) or an f32 rounding (Helper::Round32 calls Helper::Fz32, line 481). Unlike native::fz32, the generated fz32 tests the exponent alone (line 932), so a zero calls there too.

Zeros are not rare. A silent input is all zeros, and so is everything computed from it until something adds a number: a filter's integrators, a delay line, an envelope at rest.

Where the mode is chosen. crates/engine/src/dsp/tier.rs:143 lowers every module with run::Module::new (which is Floats::Flush), and line 208 compiles it with run::Floats::Flush. No host path plays in Floats::Ieee.

The host's own arithmetic. crates/engine/src/dsp/schedule.rs smooths every parameter with the same operations (Smoother, lines 128 to 185, through Floats::add, mul, sub and fround at mod.rs:410-455), once a frame for an a-rate parameter. A profile of keys with eight notes has 8% of its time in Schedule::run and 3.5% in Floats::fround. That is not where a zero costs (see part 1), but it is the rest of what the rule costs a host outside generated code.

What holds the rule. cargo test -p atmosynth-dsp --test conformance holds the interpreter, flushing, to each fixture's expectFlushed or expect (fixtures/dsp/flush-rule/ is about nothing else). crates/dsp-compiled/tests/conformance.rs holds the compiled tier to the interpreter: the same output bits, the same units, the same stop, and the same count of flushes (crates/dsp-compiled/tests/common/mod.rs:519). crates/dsp-check reports that count to a page (crates/dsp-check/src/lib.rs:516).

1. A zero stays on the common path, and the rare half is a branch #

Every rare half returns a zero unchanged and counts nothing for it, so keeping a zero out of the inline test changes no result and no count. One comparison does it. With bits the float's bits as an unsigned integer, the subtraction wrapping and the comparison unsigned:

test	as it is	with a zero kept out
fz, an f64	bits & 0x7ff0_0000_0000_0000 == 0	(bits << 1) - 1 < 2^53 - 1
mul, an f64 product	bits & 0x7fe0_0000_0000_0000 == 0	(bits << 1) - 1 < 2^54 - 1
round32, an f32	bits & 0x7f00_0000 == 0	(bits << 1) - 1 < 2^25 - 1
fz32, an f32	bits & 0x7f80_0000 == 0	(bits << 1) - 1 < 2^24 - 1

Why it is the same test without the zeros, for fz (the others are the same argument at their own widths): bits << 1 drops the sign. A zero gives 0, and 0 - 1 wraps to the largest integer, which is below nothing. A subnormal has an exponent field of 0 and a significand from 1 to 2^52 - 1, so bits << 1 is from 2 to 2^53 - 2, and one less is below 2^53 - 1. A normal has an exponent field of at least 1, so bits << 1 is at least 2^53, and one less is not below 2^53 - 1. For mul and round32 the test still has to pass an exponent field of 1, as it does now, so their bounds are one bit wider.

Three forms of the generated tests were built, each behind a switch. In millions of cycles for each rendered second (best of five, the variants in turn; CPU time is printed to a hundredth of a second, too coarse for some of these differences):

patch	notes held	as it is	a zero kept out	rare half in a branch, helpers called	no call at all	subnormals kept
keys	none	125	89	77	64	47
keys	eight	470	470	427	394	329
bass	eight	519	521	506	407	383
strings	none	84	60	54	49	36
strings	eight	473	472	454	420	353
might-as-well	none	365	210	203	178	127
might-as-well	eight	295	278	269	245	193

• a zero kept out (ATMO_EXP_ZERO): only the comparison changes; the if with its else and every helper call stay.
• rare half in a branch, helpers called (ATMO_EXP_BRANCH=call): the comparison above, then an if with no else that only the rare case enters, the value staying in its local. fz and fz32 are written out inline in that branch (the count, then the zero of its sign) and call nothing. mul and round32 still call their helpers there. This is the rule exactly. With it on, cargo test -p atmosynth-dsp-compiled --test conformance --test edges --test adapter passes, so the bits, the units, the stops and the count of flushes are the interpreter's over every fixture, and bass, keys, lead, might-as-well and strings (four notes held for 4 s of 8 s) render the same WAV bytes as without it. Against as it is, it takes 3 to 9% off a played patch and 36 to 44% off a silent one, and a silent might-as-well then costs less than a played one.
• no call at all (ATMO_EXP_BRANCH=nocall) is not part 1. It is in part 2.

What was also tried and is not worth doing: the same comparison in native.rs (fz, mul, round32; the three lines are in the first comment). Both tiers render the same bytes with it. A compiled render retires the same instructions to the million, so the host's smoothing is not where a zero costs. The interpreter comes out even (best of three, millions of cycles for a rendered second): 2 to 5% fewer for a silent patch (keys 338 to 330, might-as-well 1,298 to 1,239) and the same or slightly more for a played one (keys 1,874 to 1,904, might-as-well 1,304 to 1,296). So part 1 is codegen.rs alone, and the generated tests then differ in form from native.rs's while computing the same thing; the header of codegen.rs (lines 40 to 45) should say so.

For anyone who does touch native.rs or schedule.rs: check mode is built from them. src/dsp/check-wasm.js holds that WebAssembly module built, and test/dsp-check.test.js fails until node tools/dsp-check-wasm.mjs has written it again.

2. The rest #

With part 1 in, a played patch still costs the compiled tier 1.3 to 1.4 times what it costs with subnormals kept (the third column from the right against the last). What the experiments say about where that is:

• A call in the rare half costs the common path, though it is never taken. With mul and round32 calling nothing either (ATMO_EXP_BRANCH=nocall, the column no call at all), a played patch drops a further 7 to 20%: keys 427 to 394, bass 506 to 407, strings 454 to 420, might-as-well 269 to 245. That form is not the rule: it makes mul the product through fz and round32 the rounding through fz32, so it keeps a product or an f32 rounding that rounds up to the smallest normal from below, which the rule flushes, and flush-rule fails with it, as it should. The five patches still render the same bytes, since none meets that case. It is there to measure what the calls cost, nothing else.
• The test and its branch after every float operation cost the rest. With no call at all a played patch is still 6 to 27% over subnormals kept (keys 394 against 329, bass 407 against 383, strings 420 against 353, might-as-well 245 against 193), and retires 40 to 50% more instructions.
• The test must stay off the value's own path. Written as arithmetic alone (ATMO_EXP_PURE: a select between the value and the zero of its sign, no branch, no call, no count) a played patch is slower than as it is: keys 595 million cycles against 467, bass 781 against 523, might-as-well 324 against 295. A silent one is a little faster (keys 114 against 126, might-as-well 253 against 367).

What was not tried, and is not known:

• Whether the whole rule can be written with no call in generated code, and whether it then costs what no call at all costs. round32_tiny's second half is two comparisons and needs no call. mul_tiny's needs the exact product of two 53-bit significands (product_below_normal, native.rs:84), which is the import MulFlush today.
• Why a call that is never taken costs the common path. The machine code Cranelift makes was not read. A guess, and only that: a call clobbers every float register, so whatever is live across the if is kept in memory on the common path too.
• Whether more tests can be left out. codegen.rs already leaves the test out after something that cannot be subnormal (Math.abs, floor, min; its header, lines 42 to 45).
• Any machine but an Apple M5. A slower or narrower core may weigh the extra instructions differently, and x86-64 was not measured at all.
• 2b below was not built. Nothing here sets FPCR or MXCSR.

Two ways to close the rest, and what each runs into #

Neither is chosen here. The second reverses a stated design and needs the maintainer's word before anyone builds it.

a. Keep the rule in software and make generated code carry it more cheaply. Nothing computed changes: every fixture, every count and the two tiers' agreement stay as they are, on every machine. The work is in crates/dsp-compiled/src/codegen.rs, and the measure is the table in part 1: the column no call at all is what is known to be reachable if the two boundary cases can be had without a call, and what is left beyond it is the price of testing every operation.

b. Let a host that can set the processor's flush mode play the compiled tier with it set, and emit no test. On arm64 that mode (FPCR.FZ) is what the rule is defined as. What it runs into, all read off the code and none of it settled:

• It reverses a stated design. Floats says the rule is "done in software, never by setting the processor's flush modes, so a compiled tier can do exactly the same" (crates/dsp/src/run/mod.rs:366-369), and crates/dsp-compiled/src/lib.rs:26-30 says an audio thread's calls "must run with the processor's flush-to-zero modes off".
• x86-64 is not the rule. There the processor tests tininess after rounding, so a product or an f32 rounding whose exact value lies just below the smallest normal is kept where the rule gives a zero (fixtures/dsp/README.md, x86-64, and mod.rs:400-405). A host playing so on x86-64 would render what that README says Chrome on x86-64 renders, not what arm64 and the interpreter render. Whether that is acceptable is the decision.
• The count goes. A processor flushes without counting. Instance::flushes is compared between the tiers by the conformance test and reported by check mode, so the software rule stays for both, and the two would have to be held to each other some other way (the same output bits over the fixtures and the reference patches, say).
• The interpreter keeps the software rule whatever is decided. Check mode is the interpreter compiled to WebAssembly (crates/dsp-check), which has no flush mode to set, and a host where the compiled tier is unavailable plays the interpreter.
• What is imported runs in the mode too. The transcendental Math functions and the heavier intrinsics are the host's own native code (crates/dsp-compiled/src/imports.rs, host.rs). Today each computes with subnormals kept and its result is tested as it comes. With the processor flushing, their own arithmetic would flush as well, so a result near the smallest normal could differ from the interpreter's unless the mode is cleared around each import. Not tested.
• Whatever the compiler folds. The module's constants are folded by codegen.rs as the rule says. Whether Cranelift folds any float arithmetic of its own, with subnormals kept, was not looked at.
• What enters still has to be flushed. FPCR.FZ flushes what an operation makes, not what it is handed. The instance flushes inputs, parameter values, hook arguments and samples as it copies them in, and would go on doing so.
• The mode is the thread's. A plugin shares its audio thread with its host, so the mode would be set on the way into every entry (process, note_on, note_off, reconstruct) and put back on the way out.

What should be true #

• With the rule kept exactly, a patch with nothing played never costs the compiled tier more than the same patch played, and the four patches above cost it no more than the column rare half in a branch, helpers called.
• For part 2: a played reference patch costs the compiled tier as near to what it costs with subnormals kept as the way chosen allows. The ticket's closing comment gives both tables again, measured the same way, and says what is left and why.
• Nothing heard changes: every fixture in fixtures/dsp/ and fixtures/engine/, and atmo-render's bytes for every reference patch, stay as they are in both tiers. If 2b is decided for x86-64, this is the line it changes, and the ticket says so.
• The two tiers still compute the same bits, the same units and the same stops, as crates/dsp-compiled/tests/conformance.rs holds them to, and the same count of flushes wherever the rule stays in software.
• Nothing is allocated while rendering, as now (crates/dsp-compiled/tests/no_alloc.rs, crates/dsp/tests/no_alloc.rs).
• What is written about the rule is still true: fixtures/dsp/README.md (Subnormals), Floats in crates/dsp/src/run/mod.rs, the headers of crates/dsp-compiled/src/lib.rs and codegen.rs, and the speeds quoted at crates/dsp/src/run/mod.rs:47-51.
• crates/dsp-compiled/examples/speed.rs measures the built-ins the reference patches are made of, so the next person has the second table by running it.

Measuring it #

cargo build --release -p atmosynth-render --features compiled
B=target/release/atmo-render
# one figure: a long render less a short one, over the seconds between them
/usr/bin/time -l $B assets/default-patches/keys.json --seconds 3  --hold 2  --notes 48,52,55,60,64,67,72,76 --tier compiled --out /tmp/x.wav
/usr/bin/time -l $B assets/default-patches/keys.json --seconds 23 --hold 22 --notes 48,52,55,60,64,67,72,76 --tier compiled --out /tmp/x.wav
# nothing played: --notes ''
SECONDS_OF_AUDIO=20 cargo run --release -p atmosynth-dsp-compiled --example speed
cargo test -p atmosynth-dsp-compiled --test conformance --test edges --test adapter

• On macOS /usr/bin/time -l prints instructions retired and cycles elapsed beside the times. Instructions are the same on every run and say whether two builds do the same work. They do not say which is faster: a zero kept out retires more instructions when played and takes no longer, and the flush as arithmetic retires about as many as now and takes much longer.
• Time on a quiet machine, or do not trust it. A run pushed off an Apple machine's fast cores by other work reads about three and a half times slow, in CPU time as much as on the clock, and nothing in the output says so. Take the best of several runs with the variants in turn, and look at the load average first. A first round of these measurements was wrong by that factor for that reason.
• cmp two renders to know that a change moved no byte. atmo-render prints how many node copies were compiled and how many interpreted; any interpreted means the compiled tier was not what was measured.
• A profile of a long render (sample <pid> 8 1 on macOS) shows generated code as frames in no binary. Keys with eight notes, as it is: 62% generated code, 12% the waveshaper's resamplers, 8% Schedule::run, 3.5% Floats::fround, 4% Graph::render itself, 2% each for an instance's copies in and out and for memmove, 2.5% the imported Math.

Beside this ticket #

Seen on the way and not part of it:

• Nothing puts a shared dsp node to sleep while its input is silent, so chorus-bbd, delay-analog and the reverb cost what the none rows say with nothing played. tail on a dsp node is a yes or a no about scope, not a length.
• wasmtime's bounds checks in code (signals off) cost little here: the last two columns of the first table.
• The waveshaper's resamplers are the next largest thing after generated code. dot in crates/engine/src/nodes/waveshaper.rs:198 already vectorises, and the filter is Blink's.