Changelog

Faster Workflows with Sharded Benchmarks

Running benchmarks can be slow and significantly impact your CI time. You can now split your benchmark suite across multiple jobs in the same CI workflow, reducing execution time by running them in parallel.

When running heavy benchmark suites, this can divide the total runtime by the number of jobs, dramatically speeding up your CI pipeline. CodSpeed then aggregates the results into a single report, ensuring accurate performance tracking.

---

For example, using pytest:

# .github/workflows/codspeed.yml

jobs:
  benchmarks:
    strategy:
      matrix:
        shard: [1, 2, 3, 4] 

    name: "Run benchmarks (Shard #${{ matrix.shard }})"
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v2
        with:
          python-version: 3.12.8
      - run: pip install -r requirements.txt
      - name: Run benchmarks
        uses: CodSpeedHQ/action@v3
        with:
          run:
            pytest tests/ --test-group=${{ matrix.shard }} --test-group-count=4
          token: ${{ secrets.CODSPEED_TOKEN }}

Learn more about benchmark sharding and how to integrate it in your CI.

Multi-language Support

With benchmarks written in several languages, it can be difficult to get a unified performance overview of your project.

You can now run benchmarks written in multiple languages. When run in the same CI workflow, CodSpeed will aggregate the results of these benchmarks into a single report.

For example, using pytest and vitest:

# .github/workflows/codspeed.yml

jobs:
  python-benchmarks:
    name: "Run Python benchmarks"
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install required-version defined in uv.toml
        uses: astral-sh/setup-uv@v5
      - uses: actions/setup-python@v2
        with:
          python-version: 3.12.8
      - name: Run benchmarks
        uses: CodSpeedHQ/action@v3
        with:
          run: uv run pytest tests/benchmarks/ --codspeed
          token: ${{ secrets.CODSPEED_TOKEN }}

  nodejs-benchmarks:
    name: "Run NodeJS benchmarks"
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: "actions/setup-node@v3"
      - name: Install dependencies
        run: npm install
      - name: Run benchmarks
        uses: CodSpeedHQ/action@v3
        with:
          run: npm exec vitest bench
          token: ${{ secrets.CODSPEED_TOKEN }}

Learn more about multi-language support.