feat(scripts): run all benchmarks even if one fails(#18558)

Merged
Merging
aqrln/bench-run-all
into
main
-27%
IMPROVEMENTS
0
REGRESSIONS
2
UNTOUCHED
1
NEW
0
DROPPED
0

Benchmarks

Failing

client generation ~50 Models
Regression
packages/client/src/__tests__/benchmarks/huge-schema/huge-schema.bench.ts::typescript::client generation ~50 Models
-16%
5.4 s
6.4 s
typescript compilation ~50 Models
Regression
packages/client/src/__tests__/benchmarks/huge-schema/huge-schema.bench.ts::typescript::typescript compilation ~50 Models
-27%
44.5 ms
60.7 ms

Passing

client generation 100 models with relations
packages/client/src/__tests__/benchmarks/lots-of-relations/lots-of-relations.bench.ts::typescript::client generation 100 models with relations
+1%
61.4 s
61 s

Commits

Click on a commit to change the comparison range
chore(bench): fix benchmark GH action on main (#18463) Currently the "Store benchmark result" step of the benchmark pipeline fails with `Error: No benchmark result was found in /home/runner/work/prisma/prisma/output.txt.`. The reason is the second benchmark run overwrites the `output.txt` file from the first run but doesn't contain the raw Benchmark.js output. This commit fixes that.
base
b990cf0
8 months ago by aqrln
-27%
feat(scripts): run all benchmarks even if one fails Run all bechmarks and only print failures rather than exiting as soon as one benchmark fails, and only then throw an error if there were failures. This makes the benchmark runner behave more similarly to a test runner, and it will prevent CodSpeed from marking other benchmarks as "dropped" when one of them crashes.
99a338c
8 months ago by aqrln
ResourcesHomePricingDocsBlogGitHub
Copyright © 2023 CodSpeed Technology SAS. All rights reserved.