> ## Documentation Index
> Fetch the complete documentation index at: https://codspeed.io/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Writing Benchmarks in Kotlin

> Create benchmarks for your Kotlin codebase using Java Microbenchmark Harness

export const codspeedJvmVersion = "0.2.1";

<Note>
  The JVM integration is still in early development, and **only the [walltime
  instrument](/docs/instruments/walltime) is currently supported**.

  If you have any feedback, please reach out to us via
  [Discord](https://discord.gg/MxpaCfKSqF) or
  [email our support](mailto:contact@codspeed.io).
</Note>

Integrating CodSpeed into your Kotlin project works through a fork of
[JMH](https://github.com/openjdk/jmh) (Java Microbenchmark Harness). You write
standard JMH benchmarks in Kotlin and swap in the CodSpeed JMH fork as a
dependency. When running in CI with CodSpeed, the results are automatically
collected and reported.

JMH generates its measurement harness from the compiled `@Benchmark` classes, so
the setup is close to the Java one. Kotlin only changes how benchmark classes
have to be declared, and which generator a Maven build has to run.

## Installation

CodSpeed provides a fork of JMH that collects walltime results and sends them to
CodSpeed.

<Tabs>
  <Tab title="Gradle">
    Add the CodSpeed JMH fork as a Git submodule:

    ```bash theme={null}
    git submodule add https://github.com/CodSpeedHQ/codspeed-jvm.git third-party/codspeed-jvm
    ```

    Then include it as a composite build in your `settings.gradle.kts`, with
    dependency substitution to redirect JMH dependencies to the CodSpeed fork:

    ```kotlin settings.gradle.kts theme={null}
    includeBuild("third-party/codspeed-jvm/jmh-fork") {              // [!code ++]
        dependencySubstitution {                          // [!code ++]
            substitute(module("org.openjdk.jmh:jmh-core"))           // [!code ++]
                .using(project(":jmh-core"))              // [!code ++]
            substitute(module("org.openjdk.jmh:jmh-generator-annprocess")) // [!code ++]
                .using(project(":jmh-generator-annprocess")) // [!code ++]
        }                                                 // [!code ++]
    }                                                     // [!code ++]
    ```

    We recommend using the
    [JMH Gradle Plugin](https://github.com/melix/jmh-gradle-plugin) as it handles
    benchmark compilation and provides the `jmh` task. It generates the harness from
    the compiled Kotlin classes in `src/jmh/kotlin`, so no annotation processing
    setup is needed. Add it to your `build.gradle.kts`:

    ```kotlin build.gradle.kts theme={null}
    plugins {
        kotlin("jvm") version "2.1.20"
        id("me.champeau.jmh") version "0.7.3"// [!code ++]
    }

    kotlin {
        jvmToolchain(21)// [!code ++]
    }
    ```
  </Tab>

  <Tab title="Maven">
    Add the CodSpeed JMH fork as a Git submodule and publish it to your local Maven
    repository:

    ```bash theme={null}
    git submodule add https://github.com/CodSpeedHQ/codspeed-jvm.git third-party/codspeed-jvm
    cd third-party/codspeed-jvm
    ./gradlew -p jmh-fork publishToMavenLocal
    ```

    We're planning to publish to Maven Central in the future. If you need this,
    please reach out via [Discord](https://discord.gg/MxpaCfKSqF) or
    [email](mailto:contact@codspeed.io).

    Then replace the JMH dependencies in your `pom.xml` with the CodSpeed fork.
    Maven resolves from `~/.m2/repository` by default. Kotlin projects use the JMH
    bytecode generator rather than the annotation processor, so the fork has to be
    substituted in two places: the `jmh-core` dependency and the generator used by
    `exec-maven-plugin`.

    ```xml pom.xml theme={null}
    <properties>
        <jmh.version>1.37</jmh.version>// [!code --]
        <jmh.version>0.2.1</jmh.version>// [!code ++]
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.openjdk.jmh</groupId>// [!code --]
            <groupId>io.codspeed.jmh</groupId>// [!code ++]
            <artifactId>jmh-core</artifactId>
            <version>${jmh.version}</version>
        </dependency>
    </dependencies>

    <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <dependencies>
            <dependency>
                <groupId>org.openjdk.jmh</groupId>// [!code --]
                <groupId>io.codspeed.jmh</groupId>// [!code ++]
                <artifactId>jmh-generator-bytecode</artifactId>
                <version>${jmh.version}</version>
            </dependency>
        </dependencies>
    </plugin>
    ```

    <Note>
      Your Maven project must be configured to compile Kotlin sources, run the JMH
      generator over the compiled classes, and package an executable benchmark JAR.
      The `jmh-kotlin-benchmark-archetype` from JMH generates a project with all of
      this wired up:

      ```bash theme={null}
      mvn archetype:generate -DinteractiveMode=false \
        -DarchetypeGroupId=org.openjdk.jmh \
        -DarchetypeArtifactId=jmh-kotlin-benchmark-archetype \
        -DarchetypeVersion=1.37 \
        -DgroupId=com.example -DartifactId=kotlin-benchmarks -Dversion=1.0
      ```
    </Note>
  </Tab>
</Tabs>

## Creating benchmarks

Write your benchmarks using standard JMH annotations:

```kotlin FibBenchmark.kt theme={null}
package bench

import org.openjdk.jmh.annotations.Benchmark

open class FibBenchmark {

    @Benchmark
    fun fib(): Long = fib(30)

    private fun fib(n: Int): Long =
        if (n <= 1) n.toLong() else fib(n - 1) + fib(n - 2)
}
```

<Info>
  JMH benchmarks must return their result or use `Blackhole.consume()` to
  prevent the JVM from eliminating dead code. All examples on this page return
  the computed value.
</Info>

## Testing the benchmarks locally

To run the benchmarks with CodSpeed locally, first install the `codspeed`
runner:

```bash theme={null}
curl -fsSL https://codspeed.io/install.sh | sh
```

Then run your benchmarks with CodSpeed:

<Tabs>
  <Tab title="Gradle">
    <CodeBlock language="shellsession" filename="terminal" icon="square-terminal">
      {[
              "$ codspeed run --mode walltime -- ./gradlew jmh",
              "►►► Running the benchmarks",
              `# JMH version: ${codspeedJvmVersion}`,
              "# VM version: JDK 21.0.9, OpenJDK 64-Bit Server VM, 21.0.9+10",
              "# Warmup: 3 iterations, 1 s each",
              "# Measurement: 3 iterations, 1 s each",
              "# Benchmark: bench.FibBenchmark.fib",
              "",
              "# Run progress: 0.00% complete, ETA 00:00:18",
              "# Warmup Iteration   1: 284.114 ops/s",
              "...",
              "",
              "Benchmark          Mode  Cnt    Score   Error  Units",
              "FibBenchmark.fib  thrpt    9  286.986 ± 1.660  ops/s",
            ].join("\n")}
    </CodeBlock>
  </Tab>

  <Tab title="Maven">
    <CodeBlock language="shellsession" filename="terminal" icon="square-terminal">
      {[
              "$ mvn package -q",
              "$ codspeed run --mode walltime -- java -jar target/benchmarks.jar",
              "►►► Running the benchmarks",
              `# JMH version: ${codspeedJvmVersion}`,
              "# VM version: JDK 21.0.9, OpenJDK 64-Bit Server VM, 21.0.9+10",
              "# Warmup: 3 iterations, 1 s each",
              "# Measurement: 3 iterations, 1 s each",
              "# Benchmark: com.example.MyBenchmark.fib",
              "",
              "# Run progress: 0.00% complete, ETA 00:00:18",
              "# Warmup Iteration   1: 286.603 ops/s",
              "...",
              "",
              "Benchmark         Mode  Cnt    Score   Error  Units",
              "MyBenchmark.fib  thrpt    9  286.967 ± 2.042  ops/s",
            ].join("\n")}
    </CodeBlock>
  </Tab>
</Tabs>

## Running the benchmarks in your CI

To generate performance reports, you need to run the benchmarks in your CI. This
allows CodSpeed to automatically run benchmarks and warn you about regressions
during development.

<Tip>
  If you want more details on how to configure the CodSpeed action, you can check
  out the [Continuous Reporting section](/docs/integrations/ci).
</Tip>

Here is an example of a GitHub Actions workflow that runs the benchmarks and
reports the results to CodSpeed on every push to the `main` branch and every
pull request:

<Tabs>
  <Tab title="Gradle">
    ```yaml .github/workflows/codspeed.yml icon="github" theme={null}
    name: CodSpeed Benchmarks

    on:
      push:
        branches:
          - "main" # or "master"
      pull_request:
      # `workflow_dispatch` allows CodSpeed to trigger backtest
      # performance analysis in order to generate initial data.
      workflow_dispatch:

    jobs:
      benchmarks:
        name: Run benchmarks
        runs-on: codspeed-macro-arm64-graviton-ubuntu-22-04
        permissions: # optional for public repositories
          contents: read # required for actions/checkout
          id-token: write # required for OIDC authentication with CodSpeed
        steps:
          - uses: actions/checkout@v5
            with:
              submodules: recursive
          - uses: actions/setup-java@v4
            with:
              distribution: temurin
              java-version: 21
          - name: Run the benchmarks
            uses: CodSpeedHQ/action@v5
            with:
              mode: walltime
              run: ./gradlew jmh
    ```
  </Tab>

  <Tab title="Maven">
    ```yaml .github/workflows/codspeed.yml icon="github" theme={null}
    name: CodSpeed Benchmarks

    on:
      push:
        branches:
          - "main" # or "master"
      pull_request:
      # `workflow_dispatch` allows CodSpeed to trigger backtest
      # performance analysis in order to generate initial data.
      workflow_dispatch:

    jobs:
      benchmarks:
        name: Run benchmarks
        runs-on: codspeed-macro-arm64-graviton-ubuntu-22-04
        permissions: # optional for public repositories
          contents: read # required for actions/checkout
          id-token: write # required for OIDC authentication with CodSpeed
        steps:
          - uses: actions/checkout@v5
            with:
              submodules: recursive
          - uses: actions/setup-java@v4
            with:
              distribution: temurin
              java-version: 21
          - name: Install CodSpeed JMH fork
            run: cd third-party/codspeed-jvm && ./gradlew -p jmh-fork publishToMavenLocal
          - name: Build benchmark JAR
            run: mvn package -q
          - name: Run the benchmarks
            uses: CodSpeedHQ/action@v5
            with:
              mode: walltime
              run: java -jar target/benchmarks.jar
    ```
  </Tab>
</Tabs>

## Advanced usage

JMH provides many features for writing expressive benchmarks. Below is a
selection that can be useful in CodSpeed benchmarks.

### Parameterized benchmarks

Use `@Param` to run the same benchmark with different input values. The
annotated property has to be a `var`, since JMH assigns it before the benchmark
runs and rejects final fields:

```kotlin ParamBenchmark.kt theme={null}
package bench

import org.openjdk.jmh.annotations.Benchmark
import org.openjdk.jmh.annotations.Param
import org.openjdk.jmh.annotations.Scope
import org.openjdk.jmh.annotations.State

@State(Scope.Benchmark)
open class ParamBenchmark {

    @Param("10", "20", "30")
    var n: Int = 0

    @Benchmark
    fun fib(): Long = fib(n)

    private fun fib(n: Int): Long =
        if (n <= 1) n.toLong() else fib(n - 1) + fib(n - 2)
}
```

### Shared state

Use `@State` to share setup logic across benchmarks and control the scope of the
state object:

```kotlin StateBenchmark.kt theme={null}
package bench

import org.openjdk.jmh.annotations.Benchmark
import org.openjdk.jmh.annotations.Scope
import org.openjdk.jmh.annotations.Setup
import org.openjdk.jmh.annotations.State

@State(Scope.Benchmark)
open class StateBenchmark {

    private lateinit var numbers: List<Int>

    @Setup
    fun setup() {
        numbers = (0 until 1000).toList()
    }

    @Benchmark
    fun sum(): Int = numbers.sum()
}
```

JMH annotations behave the same in Kotlin as in Java. For a deeper dive, see the
dedicated guide:

<Card title="How to Benchmark Java with JMH" icon="java" href="/docs/guides/how-to-benchmark-java-with-jmh">
  An in-depth guide to writing JMH benchmarks: project setup, annotations,
  parameters, common pitfalls, and CodSpeed CI integration.
</Card>

## Compatibility

* **JDK 21 or later** is required.
* Benchmark classes must not be final, since JMH subclasses them to generate the
  measurement harness.
* All standard JMH annotations are supported.
* Only the [Walltime instrument](/docs/instruments/walltime) is supported.
* CodSpeed uses a custom mode for all benchmarks to collect statistically
  significant results. Any `@BenchmarkMode` annotations in your code are
  ignored.

If you run into issues or require certain features, please
[open an issue](https://github.com/CodSpeedHQ/codspeed-jvm/issues) or
[join our Discord](https://discord.com/invite/MxpaCfKSqF) to get help.

## Next steps

<CardGroup>
  <Card title="Example repository" icon="github" href="https://github.com/CodSpeedHQ/codspeed-jvm">
    The CodSpeed JVM repository with example JMH benchmarks.
  </Card>

  <Card title="How to Benchmark Java with JMH" icon="java" href="/docs/guides/how-to-benchmark-java-with-jmh">
    An in-depth guide to writing JMH benchmarks with CodSpeed.
  </Card>

  <Card title="Walltime instrument" icon="stopwatch" href="/docs/instruments/walltime">
    Learn more about the Walltime instrument and how to use it.
  </Card>

  <Card title="Dive into performance changes" icon="bars-sort" href="/docs/features/profiling">
    Learn more about profiling and how to read flame graphs.
  </Card>
</CardGroup>