Changelog

Walltime Instrument and Macro Runners

We're thrilled to unveil Walltime, a groundbreaking addition to CodSpeed's suite of instruments! 🎉 This new tool measures the wall time of your benchmarks—the total time elapsed from start to finish, capturing not just execution but also waiting on resources like I/O, network, or other processes.

At first glance, this might seem like a shift from our core philosophy of making benchmarks as deterministic as possible. But here's the thing: real-world performance often depends on the messy, noisy details of resource contention and latency. Walltime offers a lens into these real-world scenarios, making it an invaluable tool for system-level insights—without losing sight of the precision you trust CodSpeed to deliver.

Even as we introduce this new instrument, we remain committed to bringing consistent and reproducible results. That's why Walltime is exclusively available on CodSpeed Macro Runners. These hosted bare-metal runners are purpose-built for macro benchmarks, running in isolation to minimize environmental noise. This means you get realistic performance data without unnecessary interference.

Getting Started

Running Walltime measurements is as simple as changing the execution runner to codspeed-macro in GitHub Actions. Here's an example of how to do it:

jobs:
  benchmarks:
    name: Run benchmarks
    runs-on: ubuntu-latest
    runs-on: codspeed-macro
    steps:
      - uses: actions/checkout@v4
      # ...
      - name: Run benchmarks
        uses: CodSpeedHQ/action@v3
        with:
          token: ${{ secrets.CODSPEED_TOKEN }}
          run: "<Insert your benchmark command here>"

Once set up, your Walltime results will shine in the CodSpeed dashboard:

For a deeper dive into the Walltime instrument and Macro Runners, check out our documentation.

Join the Beta

This feature is still in closed beta, but if you're interested in trying it out, please reach out to us on Discord or by email at support@codspeed.io.