Latest Results
ENG-12608: add reflex-bench harness core (CLI, benchmark model, statistics, result schema, reporting)
Add packages/reflex-bench, a never-published workspace member that runs macro
benchmarks through one CLI: list, run, show, compare, doctor and export. CI will
call the same commands and only add output targets.
- registry: @benchmark with setup_cache/setup/prepare/sample/conclude/cleanup
hooks, metrics with SI units, direction and exact assumption, parameter grids
plus hidden parameters, source-hash versions and suite discovery. Instance
exposes every hook as a zero-argument method for callers outside the
scheduler.
- scheduler: warmup, hyperfine's run-count rule, per-hook deadlines on one
worker thread per instance (stuck threads are abandoned, teardown runs on a
fresh one), failure and timeout capture, min_version checks, smoke mode.
- stats (stdlib only): exact order-statistic median CI, Mann-Whitney U (exact
and normal approximation), minimum achievable p, percentile bootstrap of the
median ratio, Holm, two-gate verdicts, Tukey outliers, pyperf/hyperfine
stability warnings, geomeans and a runs-needed estimate.
- schema reflex-bench/1: TypedDicts from which the validator is derived; load
and dump refuse invalid documents; every raw sample is stored.
- store: autosave per machine profile, named baselines, series keys.
- machine: profile id and doctor checks (governor, turbo, load, power, VM,
container, cgroup v2, tools).
- reports: terminal tables, pull request markdown and Bencher Metric Format.
- suites/selftest: sleep, noise, exact, fail and timeout self-tests.
The harness never imports reflex; it only reads its package metadata.farhan/fervent-cray-2fa3k9 Latest Branches
0%
claude/reflex-hosting-build-sdk-603ca2 0%
farhan/fervent-cray-2fa3k9 0%
farhan/sleepy-hawking-bhua3z © 2026 CodSpeed Technology