Quickstart

CodSpeed is a performance testing tool helping you to consistently measure the performance of your codebase locally and in your CI/CD pipeline. More details on how CodSpeed works here: What is CodSpeed?

This guide is a quick start to get a sample Python repository on GitHub up and running with CodSpeed:

1. Create Performance Tests
2. Connect your Repository
3. Run the Tests in your CI
4. Introduce a Performance Regression

​

Create Performance Tests

1. Install the CodSpeed plugin for pytest:

   Copy

   pip install pytest-codspeed
   

2. Write a performance test using the @pytest.mark.benchmark marker:

   tests/test_sum_squares.py

   Copy

   import pytest
   
   def sum_squares(arr):
       """Sum the squares of the numbers in an array."""
       total = 0
       for x in arr:
           total += x * x
       return total
   
   # Your tests can also be benchmarks
   @pytest.mark.benchmark
   def test_sum_squares():
       assert sum_squares(range(1000)) == 332833500
   

3. Run your performance tests locally:

   Copy

   $ pytest tests/ --codspeed
   ============================= test session starts ====================
   platform darwin -- Python 3.13.0, pytest-7.4.4, pluggy-1.5.0
   codspeed: 3.0.0 (enabled, mode: walltime, timer_resolution: 41.7ns)
   rootdir: /home/user/codspeed-test, configfile: pytest.ini
   plugins: codspeed-3.0.0
   collected 1 items
   
   tests/test_sum_squares.py .                                    [ 100%]
   
                            Benchmark Results
   ┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━┓
   ┃      Benchmark ┃ Time (best) ┃ Rel. StdDev ┃ Run time ┃ Iters  ┃
   ┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━┩
   │test_sum_squares│     1,873ns │        4.8% │    3.00s │ 66,930 │
   └────────────────┴─────────────┴─────────────┴──────────┴────────┘
   =============================== 1 benchmarked ========================
   =============================== 1 passed in 4.12s ====================
   

   Great! We set up our first performance test. Now, let’s track the performance in the CI with CodSpeed!

​

Connect your Repository

1. Login on CodSpeed

2. Go to settings and install the CodSpeed GitHub App by clicking on the “Import” button.

3. Select the organization or the user and add the repositories you want to use with CodSpeed:

4. Back in CodSpeed settings, enable your repositories by clicking on the “Enable” button:

5. Copy your CODSPEED_TOKEN and add it to your GitHub repository secrets

​

Run the Tests in your CI

1. Create a new GitHub Actions workflow file to run your benchmarks:

   .github/workflows/codspeed.yml

   Copy

   name: CodSpeed
   
   on:
     push:
       branches:
         - main # Run on pushes to the main branch
     pull_request: # Run on all pull requests
   
   jobs:
     codspeed:
       name: Run benchmarks
       runs-on: ubuntu-latest
       steps:
         - uses: actions/checkout@v4
         - name: Setup Python
           uses: actions/setup-python@v5
           with:
             python-version: "3.13"
         - name: Install dependencies
           run: pip install pytest pytest-codspeed
         - uses: CodSpeedHQ/action@v3
           with:
             run: pytest tests/ --codspeed
             token: ${{ secrets.CODSPEED_TOKEN }} # Optional for public repositories
   

2. Create a Pull Request installing the workflow to the repository and wait for the report in the comments:

3. Merge it and congrats 🎉, CodSpeed is installed!

​

Introduce a Performance Regression

1. Let’s change the implementation of the sum_squares with a more concise and elegant one:

   tests/test_sum_squares.py

   Copy

   def sum_squares(arr):
       """Sum the squares of the numbers in an array."""
       return sum(map(lambda x: x**2, arr))
   

2. Open a Pull Request and wait for the CodSpeed report:

   Before merging, we can see that even if it’s more concise, this new implementation is slower. Merging it would have introduced a performance regression but we caught it before shipping it!

​

Next Steps