Comparing
fix/generator-helper-flaky-test
(7267e01
) withmain
(cec59d3
)-78%
- Improvements
- 0
- Regressions
- 1
- Untouched
- 2
- New
- 0
- Dropped
- 0
- Ignored
- 0
Benchmarks
Failed
typescript compilation ~50 ModelsRegression
packages/client/src/__tests__/benchmarks/huge-schema/huge-schema.bench.ts::typescript::typescript compilation ~50 Models-78%
13.2 ms
60.3 ms
Passed
client generation ~50 Models
packages/client/src/__tests__/benchmarks/huge-schema/huge-schema.bench.ts::typescript::client generation ~50 Models+2%
1.2 s
1.2 s
client generation 100 models with relations
packages/client/src/__tests__/benchmarks/lots-of-relations/lots-of-relations.bench.ts::typescript::client generation 100 models with relations-4%
10.9 s
11.3 s
Commits
Click on a commit to change the comparison rangeBase
main
cec59d3
+2%
fix(tests): fix flaky "generatorHandler > nonexistent executable"
The test sometimes fails on CI with:
```
Error: expect(received).rejects.toThrow()
Received promise resolved instead of rejected
Resolved to value: undefined
```
There are two sources of flakiness in this test:
1. The `setTimeout` timer that waits for `initWaitTime` after the
process spawns to see if it will exit with an error shortly after
starting must only be set after the process actually
starts (`child.on('spawn')` event), otherwise there's a chance that
the timer fires before the process even finishes spawning.
2. We don't spawn the executable directly, but wrapped in a shell. On
macOS CI (or locally under big load with `detect_flake -c 'pnpm
--filter @prisma/generator-helper test generatorHandler.test.ts --
-t nonexistent' -t 100 -r 2 -p -e`), sometimes it takes more than
200 ms for the shell to initialize, parse the command and try
executing the non-existing binary, so the command doesn't fail in
`initWaitTime` and `GeneratorProcess` considers it successfully
started. This commit changes `initWaitTime` in this test to 2 seconds.
With these changes, there are no errors for me locally when spawning
200 parallel instances of the test with 5 runs per thread, but without
them at least 3% of the runs fail with just 100 threads. It is still
possible that it would fail sometimes as any tests that rely on timers
are inherently flaky under load, but the probability of this should be
now much lower.
This commit also fixes some TypeScript errors along the way (we don't
type-check the old tests on CI, so they accumulate type errors
eventually).
47c0d03
2 years ago
by aqrln-80%
Simplify control flow by moving the logic to exit handler
7267e01
2 years ago
by aqrln