Skip to main content
Macro runners are bare-metal machines managed by CodSpeed, for walltime measurements with low variance. A GitHub-hosted runner shares its host with other tenants, so the walltime of a benchmark moves from one run to the next for reasons unrelated to your code. A macro runner is dedicated to your job for its whole duration. On GitHub Actions, CodSpeed registers a macro runner in your organization for each queued benchmark job and removes it once the job is done. Your side of it is the runs-on label of the benchmark job.

​
Prerequisites

  • CodSpeedHQ/action >= 3.1.0.
  • A GitHub organization. Registering a self-hosted runner on a repository rather than on an organization needs the Repository: Administration permission, which is too broad, so macro runners are not available on personal accounts. Transfer the repository to an organization to use them.
  • CodSpeed enabled on the repository.

​
1. Point the benchmark job at a macro runner

Replace runs-on: ubuntu-latest with the label of the runner you want, and set the mode of the action to walltime:
Everything else about the workflow is unchanged, see the GitHub Actions guide.
CachingA macro runner may not share the architecture of the GitHub-hosted runners your other jobs use, so include ${{ runner.arch }} in cache keys, or a job restores artifacts built for another architecture:

​
2. Open a pull request

Open a pull request from the branch carrying the change: the benchmark job runs on a macro runner and the pull request receives its performance report. The walltime measurements of a run are comparable to the baseline recorded on your default branch only if both ran on macro runners, so merge the change to record a first baseline on a macro runner.

​
Repository access

​
Private repositories

Macro runners work with the private repositories of your organization without further setup.

​
Public repositories

GitHub keeps self-hosted runners away from public repositories by default, since a pull request from a fork could run code on them. To use macro runners on a public repository, go to Organization Settings → Actions → Runner groups → Default in GitHub and allow public repositories:
Enabling macro runners for public repositories

​
Next Steps

Macro Runners

The runners, and what they are for

Walltime Instrument

Measure and interpret walltime benchmarks

Configure GitHub Actions

Authentication, parallel jobs and advanced options

Performance Checks

Catch regressions on every pull request