cod-1740-codspeed-go-invalid-minimum-benchmark-time-with-bloop - Branch - CodSpeedHQ/codspeed-go

fix: invalid minimum benchmark time with b.loop

#41Merged

Comparing

cod-1740-codspeed-go-invalid-minimum-benchmark-time-with-bloop

(

a6c2549

) with

main

(

0f118b3

)

-97%

Improvements: 3

Regressions: 1

Untouched: 38

New: 6

Benchmarks

Failed

BenchmarkLargeSetupInLoop

example/setup_test.go

Regression

-97%

261.8 µs9,740 µs

Improved

bench_go_runner

go-runner/benches/go_runner.rs

+37%

36.1 s26.4 s

bench_collect_results[10000000, 5]

go-runner/benches/go_runner.rs

+15%

2.3 s2 s

bench_collect_results[10000000, 10]

go-runner/benches/go_runner.rs

+12%

2.6 s2.3 s

New

BenchmarkWithoutStartupTraditional

example/setup_test.go

N/A

N/A< 1 ns

BenchmarkWithoutStartupModern

example/setup_test.go

N/A

N/A1.1 ms

BenchmarkWithStopModern

example/setup_test.go

N/A

N/A1.1 ms

BenchmarkWithStopTraditional

example/setup_test.go

N/A

N/A1.1 ms

BenchmarkWithOutlierMeasurementTraditional

example/setup_test.go

N/A

N/A1.1 ms

BenchmarkWithOutlierMeasurementModern

example/setup_test.go

N/A

N/A1.1 ms

Passed

BenchmarkLargeSetup

example/setup_test.go

N/A

9.8 ms< 1 ns

BenchmarkFibonacci10Parallel

example/fib_test.go

N/A

41 ns< 1 ns

fibonacci(10)

example/fib_test.go::BenchmarkFibonacci10::fibonacci(10)

N/A

580 ns< 1 ns

BenchmarkFibonacci20_Loop

example/fib_test.go

68.6 µs68.6 µs

BenchmarkFibonacci20_bN

example/fib_test.go

N/A

73.3 µs< 1 ns

BenchmarkFibonacci20Parallel

example/fib_test.go

N/A

5.3 µs< 1 ns

BenchmarkSleep1ms

example/timing/sleep_test.go

N/A

1 ms< 1 ns

BenchmarkSleep10ms

example/timing/sleep_test.go

N/A

10 ms< 1 ns

BenchmarkSleep10us

example/timing/sleep_test.go

N/A

10.2 µs< 1 ns

BenchmarkSleep100us

example/timing/sleep_test.go

N/A

100.2 µs< 1 ns

bench_collect_results[5000000, 5]

go-runner/benches/go_runner.rs

+2%

1 s1 s

bench_collect_results[500000, 10]

go-runner/benches/go_runner.rs

+2%

127.8 ms125.8 ms

bench_collect_results[10000000, 25]

go-runner/benches/go_runner.rs

+1%

5.1 s5.1 s

bench_collect_results[100000, 5]

go-runner/benches/go_runner.rs

18.6 ms18.6 ms

BenchmarkSleep10ms_Loop

example/timing/sleep_test.go

10 ms10 ms

BenchmarkSleep50ms

example/timing/sleep_test.go

N/A

50 ms< 1 ns

BenchmarkSleep1us_Loop

example/timing/sleep_test.go

1.2 µs1.2 µs

BenchmarkSleep1us

example/timing/sleep_test.go

N/A

1.1 µs< 1 ns

bench_collect_results[5000000, 25]

go-runner/benches/go_runner.rs

+3%

2.7 s2.6 s

BenchmarkTestifyWithT

example/compat/testify_test.go

+2%

3.6 µs3.6 µs

bench_collect_results[500000, 25]

go-runner/benches/go_runner.rs

257.7 ms257.1 ms

BenchmarkSleep100ns_Loop

example/timing/sleep_test.go

240 ns240 ns

BenchmarkSleep50ms_Loop

example/timing/sleep_test.go

50 ms50 ms

BenchmarkSleep1ms_Loop

example/timing/sleep_test.go

1 ms1 ms

BenchmarkSleep100us_Loop

example/timing/sleep_test.go

100.2 µs100.2 µs

BenchmarkSleep10us_Loop

example/timing/sleep_test.go

10.2 µs10.2 µs

BenchmarkSleep100ns

example/timing/sleep_test.go

N/A

231 ns< 1 ns

BenchmarkQuicktest

example/compat/quicktest_test.go

+1%

6.8 µs6.7 µs

BenchmarkWithSlogAssert

example/compat/slogassert_test.go

52.5 µs52.6 µs

bench_collect_results[100000, 10]

go-runner/benches/go_runner.rs

19.9 ms20 ms

bench_collect_results[1000000, 5]

go-runner/benches/go_runner.rs

213.7 ms214.3 ms

bench_collect_results[1000000, 25]

go-runner/benches/go_runner.rs

521.5 ms523.6 ms

bench_collect_results[100000, 25]

go-runner/benches/go_runner.rs

-1%

40.8 ms41.1 ms

BenchmarkTestifyWithNew

example/compat/testify_test.go

-1%

7.5 µs7.6 µs

bench_collect_results[1000000, 10]

go-runner/benches/go_runner.rs

-1%

250.8 ms254.5 ms

bench_collect_results[500000, 5]

go-runner/benches/go_runner.rs

-2%

102.2 ms104.1 ms

bench_collect_results[5000000, 10]

go-runner/benches/go_runner.rs

-3%

1.3 s1.3 s

BenchmarkExample

example/very/nested/module/example_test.go

N/A

< 1 ns< 1 ns

Commits

Click on a commit to change the comparison range

Base

main

0f118b3

-97.31%

fix(testing): save measurements only in the benchmark framework

a6c2549

3 days ago

by not-matthias

Home Terms Privacy Docs