With your projects becoming larger, you might end up with long-running benchmark workflows, degrading the performance feedback loop and using a lot of resources in your CI. As a solution to these problems, when doing a CI run, you can run only a subset of the benchmarks that are defined in your codebase. For example:
  • only run benchmarks relevant to the code changes in a pull request
  • run a subset of long-running benchmarks on a schedule
Partial runs allows you to run incomplete benchmark suites while still receiving performance reports containing all the benchmarks of your repository. This allow you to reduce CI execution time by only running benchmarks relevant to your code changes, while still maintaining a complete performance history.
Performance Improvement with Partial Runs

Only Suite B was run since it was the only one affected by the code changes

When benchmarks are missing from a run, CodSpeed automatically uses baseline results from previous runs to fill the gaps.

​
How to make partial runs?

There are several ways to implement partial runs in your CI workflow:
  • in addition to using sharded benchmarks:
    • use your benchmarking framework’s “affected” or “changed” feature if it exists
    • detect which package/library of your monorepo was affected by the code changes and run only its benchmarks
  • run a long-running benchmark suite on a schedule (e.g. nightly) in addition to running a smaller suite on each pull request

​
Skipped Benchmarks

When CodSpeed detects missing benchmarks in a run, it automatically:
  1. Identifies missing benchmarks by comparing the current run with its baseline run
  2. Retrieves baseline results from the baseline run
  3. Displays them as “skipped” with in reports
You will thus be able to track the complete performance history of your benchmarks and compare any two runs, even if they ran different subsets of benchmarks.

​
Managing Removed Benchmarks

When removing benchmarks from your codebase, they will first appear as “skipped” in reports. To remove them from future reports, use the benchmark archival feature.