​
Benchmark performance impact

The performance impact denotes an improvement or regression in performance of a benchmark. It is calculated by comparing the benchmark time measurement of the head commit with the time measurement of the base commit. impact=speed−baseSpeedbaseSpeedimpact = \frac{speed - baseSpeed}{baseSpeed} A negative performance metric means that the benchmark is slower than the previous commit. The closer its value is to -1, the slower it is. −1<impact<0-1 \lt impact \lt 0 A positive performance metric means that the benchmark is faster than the previous commit. Its value can go up to +∞+\infty to denote massive speed improvements. 0<impact<+∞0 \lt impact \lt +\infty Naturally, when the benchmark is as fast as the previous commit, the performance metric is 00.

​
Regression threshold

On the settings page of a project, you can set a threshold for a regression to be considered a regression. By default, this value is set to 10% (which is equivalent to 0.1). The value can be set from 0% to 50% by an admin of the project. More information about setting the threshold can be found in the customization documentation.

​
Commit performance impact

To get the overall performance impact of a commit, we aggregate all the benchmark impacts. n being the number of benchmarks.

​
Regression threshold exceeded

If there is a regression above the threshold, the overall commit impact will be the biggest regression impact. commitImpact=min⁡0≤i<nimpact[i]commitImpact = \min_{0\leq i\lt n} impact[i]
For example, with impacts: [0.1, 0, -0.3] and a threshold of 0.25, the overall commit impact will be -0.3.

​
Improvement threshold exceeded

If there is an improvement above the threshold, the overall commit impact will be the maximum improvement impact. commitImpact=max⁡0≤i<nimpact[i]commitImpact = \max_{0\leq i\lt n} impact[i]
For example, with impacts: [0.1, 0.3, -0.2] and a threshold of 0.25, the overall commit impact will be 0.3.

​
No threshold exceeded

Finally, in the remaining cases, a geometric mean is calculated from all the benchmark performance impacts. commitImpact=(∏i=0n−11+impact[i])1n−1commitImpact = \left(\prod_{i=0}^{n-1} 1+impact[i]\right)^{\frac{1}{n}} - 1
For example, with impacts: [0.1, 0.3, -0.2] and a threshold of 0.5, the overall commit impact will be approximately 0.0459.
A geometric mean will give more relevant results than an arithmetic mean for this kind of measure as it will be less sensitive to outliers.

​
Baseline report selection

To create a performance impact, we need to compare the execution speed of the benchmarks against a baseline of those benchmarks’ execution speed. Depending on the context of the run, the baseline report can be different.

​
Pull Request

When triggering a CodSpeed run on a pull request between a head branch and a base branch, the baseline report will be the report of the latest commit of the base branch with a CodSpeed run.
Checked-out commit of a Pull Request in GitHub ActionsBy default, when using the action/checkout in GitHub Actions, the checked-out commit of a pull request will be the merge commit of the pull request. This means that GitHub will create a merge commit M between the base branch and the head branch, and use it as the checked-out commit.
Pull Request Base In this case, the report will be between the HEAD commit C of the base branch and the merge commit M. The report will thus only include the changes of commits B1 and B2.
Every time a new commit is pushed on the feat-branch, the baseline report will be updated to the latest commit of the base branch with a CodSpeed run.
This pull request base selection algorithm will only work on pull_request event for GitHub Actions and merge_request_event for GitLab CI. If you run CodSpeed on a different event, the branch base selection algorithm defined below will be used instead.

​
Branch

When triggering a CodSpeed run following a push on a branch, the baseline report will be the report of the closest commit of the branch. Branch Base
In this example, a report already exists for the commit B of the main branch. A new commit C is pushed on the main branch. The baseline report for C will be the report of the commit B.

​
Performance impact Gauge

The performance impact gauge is a visual representation of the performance impact, displayed in multiple places of the CodSpeed UI. Some examples of the gauge with their corresponding impact values:
Gauge -0.75
Gauge -0.2
Gauge 0
Gauge 0.3
Gauge 1.5
To make it easier to spot regressions and improvements, the mapping between the actual performance impact value and the gauge is not linear.