> ## Documentation Index
> Fetch the complete documentation index at: https://codspeed.io/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Writing benchmarks with vitest-bench

export const CIWorkflow = ({minimal = false, enableWorkflowDispatch = true, runsOn = "ubuntu-latest", highlight = [], mode, modes, submodules = false, preSteps = [], buildSteps = ["# ...", "# Setup your environment here:", "#  - Configure your Python/Rust/Node version", "#  - Install your dependencies", "#  - Build your benchmarks (if using a compiled language)", "# ..."], benchmarkCommand = ["<Insert your benchmark command here>"], jobName = "Run benchmarks", env = {}}) => {
  const modeList = modes || (mode ? [mode] : undefined);
  if (!modeList || modeList.length === 0) {
    throw new Error("mode or modes is required");
  }
  const indent = (lines, depth) => {
    const reindentedLines = lines.map(l => l.length === 0 ? l : (" ").repeat(depth) + l);
    return reindentedLines.join("\n");
  };
  const workflowDispatchSection = enableWorkflowDispatch ? "  # `workflow_dispatch` allows CodSpeed to trigger backtest\n" + "  # performance analysis in order to generate initial data.\n" + "  workflow_dispatch:\n" : "";
  let yaml = "";
  if (!minimal) {
    yaml += `
name: CodSpeed Benchmarks

on:
  push:
    branches:
      - "main" # or "master"
  pull_request:
`;
    yaml += workflowDispatchSection;
  }
  yaml += `
jobs:
  benchmarks:
    name: ${jobName}
    runs-on: ${runsOn}`;
  if (!minimal) {
    yaml += `
    permissions: # optional for public repositories
      contents: read # required for actions/checkout
      id-token: write # required for OIDC authentication with CodSpeed`;
  }
  if (preSteps.length > 0) yaml += "\n" + indent(preSteps, 4);
  yaml += `
    steps:
      - uses: actions/checkout@v5`;
  if (submodules) {
    const value = typeof submodules === "string" ? submodules : "true";
    yaml += `\n        with:\n          submodules: ${value}`;
  }
  yaml += "\n" + indent(buildSteps, 6);
  const modeValue = modeList.join(",");
  yaml += `
      - name: Run the benchmarks
        uses: CodSpeedHQ/action@v4
        with:
          mode: ${modeValue}`;
  if (benchmarkCommand.length > 0) {
    const indentedBenchCommand = benchmarkCommand.length > 1 ? benchmarkCommand[0] + "\n" + indent(benchmarkCommand.slice(1), 12) : benchmarkCommand;
    const runLine = indent(["run: "], 10) + indentedBenchCommand;
    yaml += `\n${runLine}`;
  }
  const envEntries = Object.entries(env);
  if (envEntries.length > 0) {
    const envLines = ["env:", ...envEntries.map(([k, v]) => `  ${k}: ${v}`)];
    yaml += "\n" + indent(envLines, 8);
  }
  return <CodeBlock language="yaml" highlight={JSON.stringify(highlight)} {...minimal || ({
    filename: ".github/workflows/codspeed.yml",
    icon: "github"
  })}>
      {yaml}
    </CodeBlock>;
};

export const NpmInstall = ({packages}) => {
  return <CodeGroup>
      <code className="language-sh" filename="npm">
      npm install --save-dev {packages}
      </code>

      <code className="language-sh" filename="yarn">
      yarn add --dev {packages}
      </code>

      <code className="language-sh" filename="pnpm">
      pnpm add -D {packages}
      </code>
    </CodeGroup>;
};

## Installation

First install the plugin
[`@codspeed/vitest-plugin`](https://www.npmjs.com/package/@codspeed/vitest-plugin)
and `vitest` (if not already installed):

<Info>
  The CodSpeed plugin is only compatible with `vitest`
  [v3.2 and above](https://www.npmjs.com/package/vitest/v/3.2.4).
</Info>

<NpmInstall packages="@codspeed/vitest-plugin vitest" />

## Usage

### Creating benchmarks

Let's create a fibonacci function and benchmark it with vitest and the CodSpeed
plugin:

<Tabs groupId="language">
  <Tab title="TypeScript">
    ```typescript src/fibo.bench.ts theme={null}
    import { bench, describe } from "vitest";

    function fibonacci(n: number): number {
      if (n < 2) {
        return n;
      }
      return fibonacci(n - 1) + fibonacci(n - 2);
    }

    describe("fibo", () => {
      bench("fibo 10", () => {
        fibonacci(10);
      });
      bench("fibo 15", () => {
        fibonacci(15);
      });
    });
    ```
  </Tab>

  <Tab title="JavaScript">
    ```js src/fibo.bench.js theme={null}
    import { bench, describe } from "vitest";

    function fibonacci(n) {
      if (n < 2) {
        return n;
      }
      return fibonacci(n - 1) + fibonacci(n - 2);
    }

    describe("fibo", () => {
      bench("fibo 10", () => {
        fibonacci(10);
      });
      bench("fibo 15", () => {
        fibonacci(15);
      });
    });
    ```
  </Tab>
</Tabs>

Here, a few things are happening:

* We create a simple recursive fibonacci function.
* We create a new `vitest` suite `"fibo"` with two benchmarks, benching our
  `fibonacci` function for 10 and 15.

Create or update the `vitest.config.ts` file to use the CodSpeed runner:

```ts vitest.config.ts theme={null}
import codspeedPlugin from "@codspeed/vitest-plugin";
import { defineConfig } from "vitest/config";

export default defineConfig({
  plugins: [codspeedPlugin()],
  // ...
});
```

### Testing the benchmarks locally

Now, we can run our benchmarks locally to make sure everything is working as
expected:

<CodeGroup>
  ```sh npm theme={null}
  npx vitest bench --run
  ```

  ```sh yarn theme={null}
  yarn vitest bench --run
  ```

  ```sh pnpm theme={null}
  pnpm vitest bench --run
  ```
</CodeGroup>

This will run the benchmarks and output the results in the terminal (for
example, with `pnpm`):

```shellsession title=terminal icon="square-terminal" theme={null}
$ pnpm vitest bench --run
 RUN  v3.2.4

[CodSpeed] @codspeed/vitest-plugin v5.0.1 - setup

✓ benches/flat.bench.ts (2) 1521ms
✓ fibo (2) 1520ms
name min max mean rme samples
· fibo 10 0.0005 1.0279 0.0006 ±0.50% 880344 fastest
· fibo 15 0.0052 0.0594 0.0057 ±0.08% 86967

BENCH Summary

fibo 10 - benches/flat.bench.ts > fibo 10.12x faster than fibo 15
```

<Note>
  **Troubleshooting errors**

  If you encounter an error like
  `Failed to resolve "@codspeed/vitest-plugin" from "vitest.config.ts"`, you might
  need to rename the `vitest.config.ts` file to `vitest.config.mts`.

  You can find more information in the
  [Vite documentation on importing ESM package](https://vitejs.dev/guide/troubleshooting.html#this-package-is-esm-only).
</Note>

And... Congrats🎉, CodSpeed is installed in your benchmarking suite! When not
used in the CI environment or with the
[CLI](https://github.com/CodSpeedHQ/codspeed), CodSpeed will fallback to using
the default `vitest`.

You can now
[run those benchmarks in your CI](#running-the-benchmarks-in-your-ci) to get
consistent performance measurements.

### Running the benchmarks in your CI

To generate performance reports, you need to run the benchmarks in your CI. This
allows CodSpeed to automatically run benchmarks and warn you about regressions
during development.

<Tip>
  If you want more details on how to configure the CodSpeed action, you can check
  out the [Continuous Reporting section](/integrations/ci).
</Tip>

Here is an example of a GitHub Actions workflow that runs the benchmarks and
reports the results to CodSpeed on every push to the `main` branch and every
pull request:

<CIWorkflow
  buildSteps={[
"- uses: actions/setup-node@v6",
"- name: Install dependencies",
"  run: npm install",
]}
  mode="simulation"
  benchmarkCommand={["npm exec vitest bench"]}
/>

## Recipes

### Running benchmarks in parallel CI jobs

To parallelize your benchmarks, you can use the
[`shard`](https://vitest.dev/guide/cli.html#shard) options from vitest.

For example with `pnpm` on Github Actions:

<CIWorkflow
  minimal
  mode="simulation"
  highlight={[5, 6, 7, 17]}
  preSteps={[
"strategy:",
"  matrix:",
'    shard: ["1/5","2/5","3/5","4/5","5/5"]',
]}
  buildSteps={[
"- uses: actions/setup-node@v6",
"- name: Install dependencies",
"  run: npm install",
]}
  benchmarkCommand={["pnpm vitest bench --run --shard=${{ matrix.shard }}"]}
/>

<Danger>
  **Same benchmark with different variations**

  For now, you cannot run the same benchmarks several times within the same run.
  If the same benchmark is run multiple times, you will receive the following
  comment on your pull request:

  <img src="https://mintcdn.com/codspeed/jKaxX6yy-Kzw1C-0/assets/parallel-benchmarks-variations-warning.png?fit=max&auto=format&n=jKaxX6yy-Kzw1C-0&q=85&s=f5e87d3fecc76f2fe3ce7f302e7f9a93" className="rounded-xl w-full max-w-lg mx-auto" alt="Multiple Benchmark Variations Error Message" width="1832" height="568" data-path="assets/parallel-benchmarks-variations-warning.png" />
</Danger>

Learn more about
[benchmark sharding and how to integrate with your CI provider](/features/sharded-benchmarks).

<Info>
  To combine measurement modes like simulation and memory, check out the
  documentation on [running multiple instruments
  serially](/integrations/ci/github-actions/configuration#running-multiple-instruments-serially).
</Info>

## Limitations

### Async code execution profiling

[Execution profiles](/features/profiling) for async code can sometimes be
unreliable, as profiling tools may lose stack trace information due to the event
loop.