mozilla
neqo
BlogDocsChangelog

chore: Remove randomization from benches

#3202
Comparing
larseggert:chore-no-rand-bench
(
8c9002a
) with
main
(
1c9cbd4
)
CodSpeed Performance Gauge
-33%
Improvements
4
Regressions
6
Untouched
13

Benchmarks

Failed

client
neqo-bin/benches/main.rs::benches::transfer::1-conn/1-100mb-resp (aka. Download)
Regression
CodSpeed Performance Gauge
-11%
768 ms861.2 ms
wallclock-time
neqo-http3/benches/streams.rs::benches::criterion_benchmark::1000-streams/each-1-bytes
Regression
CodSpeed Performance Gauge
-3%
33.1 ms34.3 ms
decode 4096 bytes, mask 7f
neqo-common/benches/decoder.rs::benches::benchmark_decoder
Regression
CodSpeed Performance Gauge
-9%
35.3 µs38.6 µs
decode 4096 bytes, mask ff
neqo-common/benches/decoder.rs::benches::benchmark_decoder
Regression
CodSpeed Performance Gauge
-33%
18 µs27 µs
decode 1048576 bytes, mask ff
neqo-common/benches/decoder.rs::benches::benchmark_decoder
Regression
CodSpeed Performance Gauge
-33%
4.5 ms6.8 ms
decode 1048576 bytes, mask 7f
neqo-common/benches/decoder.rs::benches::benchmark_decoder
Regression
CodSpeed Performance Gauge
-9%
8.9 ms9.8 ms

Improved

client
neqo-bin/benches/main.rs::benches::transfer::1-conn/1-100mb-req (aka. Upload)
CodSpeed Performance Gauge
+11%
918.2 ms829.9 ms
run
neqo-transport/benches/transfer.rs::transfer::benchmark_transfer_fixed::transfer/pacing-true/same-seed/wallclock-time
CodSpeed Performance Gauge
+7%
163 ms152.1 ms
run
neqo-transport/benches/transfer.rs::transfer::benchmark_transfer_variable::transfer/pacing-false/varying-seeds/wallclock-time
CodSpeed Performance Gauge
+6%
168.2 ms158.2 ms
run
neqo-transport/benches/transfer.rs::transfer::benchmark_transfer_variable::transfer/pacing-true/varying-seeds/wallclock-time
CodSpeed Performance Gauge
+5%
162.5 ms154 ms

Passed

run
neqo-transport/benches/transfer.rs::transfer::benchmark_transfer_fixed::transfer/pacing-false/same-seed/wallclock-time
CodSpeed Performance Gauge
+2%
161.4 ms157.4 ms
client
neqo-bin/benches/main.rs::benches::transfer::1-conn/10_000-parallel-1b-resp (aka. RPS)
CodSpeed Performance Gauge
+1%
370.5 ms367.6 ms
wallclock-time
neqo-http3/benches/streams.rs::benches::criterion_benchmark::1000-streams/each-1000-bytes
CodSpeed Performance Gauge
0%
121.3 ms120.8 ms
take_ranges
neqo-transport/benches/sent_packets.rs::benches::take_ranges::sent::Packets
CodSpeed Performance Gauge
0%
30.1 µs30.1 µs
coalesce_acked_from_zero 3+1 entries
neqo-transport/benches/range_tracker.rs::benches::benchmark_coalesce
CodSpeed Performance Gauge
0%
3 µs3 µs
coalesce_acked_from_zero 1+1 entries
neqo-transport/benches/range_tracker.rs::benches::benchmark_coalesce
CodSpeed Performance Gauge
0%
2.8 µs2.8 µs
coalesce_acked_from_zero 10+1 entries
neqo-transport/benches/range_tracker.rs::benches::benchmark_coalesce
CodSpeed Performance Gauge
0%
3 µs3 µs
coalesce_acked_from_zero 1000+1 entries
neqo-transport/benches/range_tracker.rs::benches::benchmark_coalesce
CodSpeed Performance Gauge
0%
2.6 µs2.6 µs
inbound_frame()
neqo-transport/benches/rx_stream_orderer.rs::benches::criterion_benchmark::RxStreamOrderer
CodSpeed Performance Gauge
0%
125.8 ms125.8 ms
wallclock-time
neqo-http3/benches/streams.rs::benches::criterion_benchmark::1-streams/each-1000-bytes
CodSpeed Performance Gauge
-1%
1.1 ms1.1 ms
client
neqo-bin/benches/main.rs::benches::transfer::1-conn/1-1b-resp (aka. HPS)
CodSpeed Performance Gauge
-1%
15.1 ms15.3 ms
decode 1048576 bytes, mask 3f
neqo-common/benches/decoder.rs::benches::benchmark_decoder
CodSpeed Performance Gauge
-2%
11.5 ms11.8 ms
decode 4096 bytes, mask 3f
neqo-common/benches/decoder.rs::benches::benchmark_decoder
CodSpeed Performance Gauge
-3%
45.4 µs46.5 µs

Commits

Click on a commit to change the comparison range
Base
main
1c9cbd4
-33.35%
chore: Remove randomization from benches In the hopes of stabilizing the results some.
8c9002a
3 hours ago
by larseggert
© 2025 CodSpeed Technology
Home Terms Privacy Docs