Running benchmarks in CI environments presents unique challenges due to the inherent noise and variability of shared cloud infrastructure. Standard GitHub-hosted runners can exhibit significant performance variance.
Read our detailed post on how CI noise affects benchmark consistency.
CodSpeed instruments are designed to mitigate these challenges and gather accurate performance data even in noisy environments. The easiest way to get started running benchmarks in GitHub Actions is to use the CodSpeed GitHub Action.

​
Setup

​
1. Allow the Action to upload on CodSpeed

First, you need to get your CodSpeed Token. There are multiple ways to retrieve it:
  • Once you enable a repository on CodSpeed, you’ll be prompted to copy the token
  • You can also find it on the repository settings page
Upload Token from the settings page
Token Scope: Be mindful that a token is scoped to a specific repository. Make sure that you are on the correct repository settings page when copying the token.
Public repositoriesCodSpeed allows tokenless uploads for public repositories, allowing runs to be triggered from public forks directly.To enable this, you don’t need to store the token and you can simply omit the token input when creating the benchmarks workflow.
Then, create a new encrypted secret in your repository with the name CODSPEED_TOKEN and the value of your token.

​
2. Create the benchmarks workflow

The next step is to create a new workflow to run the benchmarks for your repository. You can do this by creating the codspeed.yml file in the .github/workflows directory with the following content:
.github/workflows/codspeed.yml
name: CodSpeed Benchmarks

on:
  push:
    branches:
      - "main" # or "master"
  pull_request:
  # `workflow_dispatch` allows CodSpeed to trigger backtest
  # performance analysis in order to generate initial data.
  workflow_dispatch:

jobs:
  benchmarks:
    name: Run benchmarks
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      # ...
      # Setup your environment here:
      #  - Configure your Python/Rust/Node version
      #  - Install your dependencies
      #  - Build your benchmarks (if using a compiled language)
      # ...
      - name: Run the benchmarks
        uses: CodSpeedHQ/action@v4
        with:
          mode: instrumentation
          run: <Insert your benchmark command here>
          token: ${{ secrets.CODSPEED_TOKEN }} # optional for public repos
The most important step of this workflow is the usage of CodSpeedHQ/action. This action will configure the CodSpeed environment and upload the benchmarks results.
Make sure to include pull_request in the on section of the workflow. This is required to have reports on pull requests correctly working.Learn more about baseline report selection.

​
Sample configurations

​
3. Check the results

Once the workflow is created, your pull requests will receive a performance report comment and will also receive some additional checks: Pull Request Result Pull Request Result

​
4. Next Steps

Now that everything is up and running (and hopefully green 🎉), you can start enhancing your workflow to get the most out of CodSpeed.

Explore the Performance Metrics

Understand the performance metrics generated by CodSpeed

Enforce Performance Checks

Make sure you or team members never merge unexpected performance regressions

Dive in Trace Generation

Get detailed performance traces for your benchmarks

Shard the execution of your benchmarks

Run your benchmarks in parallel to speed up your CI

​
Advanced

​
Defining environment variables

You can define environment variables for your benchmarks by using the env key in the section of the action: 
- name: Run benchmarks
  uses: CodSpeedHQ/action@v4
  env:
    MY_ENV_VAR: "my-value"
  with:
    token: ${{ secrets.CODSPEED_TOKEN }}

​
Running benchmarks in parallel CI jobs

With Github Actions, you can leverage matrix jobs to improve the performance of your benchmarks. For example, using pytest:
.github/workflows/codspeed.yml
name: CodSpeed Benchmarks

on:
  push:
    branches:
      - "main" # or "master"
  pull_request:
  # `workflow_dispatch` allows CodSpeed to trigger backtest
  # performance analysis in order to generate initial data.
  workflow_dispatch:

jobs:
  benchmarks:
    name: Run benchmarks (Shard #\${{ matrix.shard }})
    runs-on: ubuntu-latest
    strategy:
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4
      - name: Install required-version defined in uv.toml
        uses: astral-sh/setup-uv@v5
      - uses: actions/setup-python@v2
        with:
          python-version: 3.12.8
      - name: Run the benchmarks
        uses: CodSpeedHQ/action@v4
        with:
          mode: instrumentation
          run: uv run pytest tests/benchmarks/ --codspeed --test-group=${{ matrix.shard }} --test-group-count=4
          token: ${{ secrets.CODSPEED_TOKEN }} # optional for public repos
CodSpeed will only be able to aggregate results from your benchmarks if you split them within a single CI workflow.
CodSpeed does not yet support running the same benchmarks multiple times. If your matrix has several dimensions (e.g. a runtime version), please ensure that each benchmark will only run once.If the same benchmark is run multiple times, you will receive the following comment on your pull request:Multiple Benchmark Variations Error Message
Learn more about benchmark sharding and how to integrate with your CI provider.

​
Compatibility

For now, only the following OS and versions are supported on the runners:
  • Ubuntu 22.04 and later
  • Debian 12 and later