Then you can run the benchmarks with :
The problem that we now face is that the results are inconsistent between runs. Running the command multiple times in a CI/CD runner or on your machine will yield different results. Thus it will be hard to spot performance regressions with confidence.
This is where using CodSpeed comes in handy. CodSpeed provides a way to run benchmarks in a consistent manner, and thus detect performance regressions with confidence.
We are excited to announce that CodSpeed now has first-class support for Vitest!
Install the package and as dev dependencies:
The CodSpeed plugin is only compatible with firstname.lastname@example.org and above.
Create or update the file at the root of your project:
If you are on a project using and , you can also add the plugin to the file.
Let's create a fibonacci function and create some benchmarks on it. Create a new file :
Create a new file :
Now we can run the benchmarks with :
We can see that CodSpeed has been detected and that the benchmarks have been run. For the moment we can see the default output of Vitest's API.
To generate performance reports, you need to run the benchmarks in your CI. This allows CodSpeed to detect the CI environment and properly configure the instrumented environment.
The first step is to log in to CodSpeed and import the repository you want to track. You can find more information about this in the CodSpeed documentation.
After installing the GitHub app and importing the repository in CodSpeed, you will be prompted a secret to copy. This token is used to authenticate the CodSpeed GitHub Actions workflow.
Open a Pull Request on your repo with the previous changes, and the following GitHub Actions workflow . It will run the benchmarks and report the results to CodSpeed on every push to the branch and every pull request:
If everything went well, the workflow should run and after it finishes you should see a comment from the CodSpeed GitHub app on the Pull Request with the run report.
By clicking on the CodSpeed comment in the Pull Request, you will be redirected to the CodSpeed report of the PR. You will see a list of the benchmarks, and when clicking on the benchmark its flame graph will appear:
And that's it for the installation of CodSpeed with Vitest, now we can explore a bit more what CodSpeed offers.
One of the main features of CodSpeed is to detect and visualize the performance changes of a pull request.
Let's now say that we are on a more complex project where we have a function that parses a GitHub pull request. We created a pull request with some performance improvements, and the following is in the CodSpeed report:
We observe a 25% improvement in the overall benchmark, driven by enhancements in the and functions. Although the function has experienced regression, its impact on the overall benchmark is less pronounced compared to the observed improvements. Additionally, a new function, , is introduced and denoted by the distinctive blue color.
Thanks to the differential flame graph, it is now really easy to understand at a glance what code changes impact the performance.
Get ready to supercharge your CI with performance reports, flame graphs, and more!
If you enjoyed this article, you can follow us on Twitter to get notified when we publish new articles and product updates.