Nov 27, 2024
Walltime Instrument and Macro Runners
We're thrilled to unveil Walltime, a groundbreaking addition to CodSpeed's suite of instruments! 🎉 This new tool measures the wall time of your benchmarks—the total time elapsed from start to finish, capturing not just execution but also waiting on resources like I/O, network, or other processes.
At first glance, this might seem like a shift from our core philosophy of making benchmarks as deterministic as possible. But here's the thing: real-world performance often depends on the messy, noisy details of resource contention and latency. Walltime offers a lens into these real-world scenarios, making it an invaluable tool for system-level insights—without losing sight of the precision you trust CodSpeed to deliver.
Even as we introduce this new instrument, we remain committed to bringing consistent and reproducible results. That's why Walltime is exclusively available on CodSpeed Macro Runners. These hosted bare-metal runners are purpose-built for macro benchmarks, running in isolation to minimize environmental noise. This means you get realistic performance data without unnecessary interference.
Getting Started
Running Walltime measurements is as simple as changing the execution runner to
codspeed-macro in GitHub Actions. Here's an example of how to do it:
jobs:
benchmarks:
name: Run benchmarks
runs-on: ubuntu-latest
runs-on: codspeed-macro
steps:
- uses: actions/checkout@v4
# ...
- name: Run benchmarks
uses: CodSpeedHQ/action@v3
with:
token: ${{ secrets.CODSPEED_TOKEN }}
run: "<Insert your benchmark command here>"Once set up, your Walltime results will shine in the CodSpeed dashboard:
An example measuring the time taken to resolve the google.com hostname.
For a deeper dive into the Walltime instrument and Macro Runners, check out our documentation.
Join the Beta
This feature is still in closed beta, but if you're interested in trying it out, please reach out to us on Discord or by email at support@codspeed.io.
This page also come
with individual run pages, allowing for example to dive into the runs made on
push on a default branch: 
