Running benchmarks can be slow and significantly impact your CI time. You can
now split your benchmark suite across multiple jobs in the same CI workflow,
reducing execution time by running them in parallel.
When running heavy benchmark suites, this can divide the total runtime by the
number of jobs, dramatically speeding up your CI pipeline. CodSpeed then
aggregates the results into a single report, ensuring accurate performance
tracking.
For example, using pytest:
# .github/workflows/codspeed.yml
jobs:
benchmarks:
strategy:
matrix:
shard: [1, 2, 3, 4]
name: "Run benchmarks (Shard #${{ matrix.shard }})"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v2
with:
python-version: 3.12.8
- run: pip install -r requirements.txt
- name: Run benchmarks
uses: CodSpeedHQ/action@v3
with:
run:
pytest tests/ --test-group=${{ matrix.shard }} --test-group-count=4
token: ${{ secrets.CODSPEED_TOKEN }}
Learn more about
benchmark sharding and how to integrate it in your CI.
Multi-language Support
With benchmarks written in several languages, it can be difficult to get a
unified performance overview of your project.
You can now run benchmarks written in multiple languages. When run in the same
CI workflow, CodSpeed will aggregate the results of these benchmarks into a
single report.
For example, using pytest and vitest:
# .github/workflows/codspeed.yml
jobs:
python-benchmarks:
name: "Run Python benchmarks"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install required-version defined in uv.toml
uses: astral-sh/setup-uv@v5
- uses: actions/setup-python@v2
with:
python-version: 3.12.8
- name: Run benchmarks
uses: CodSpeedHQ/action@v3
with:
run: uv run pytest tests/benchmarks/ --codspeed
token: ${{ secrets.CODSPEED_TOKEN }}
nodejs-benchmarks:
name: "Run NodeJS benchmarks"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: "actions/setup-node@v3"
- name: Install dependencies
run: npm install
- name: Run benchmarks
uses: CodSpeedHQ/action@v3
with:
run: npm exec vitest bench
token: ${{ secrets.CODSPEED_TOKEN }}
Learn more about
multi-language support.