Skip to main content
The Java integration is still in early development, and only the walltime instrument is currently supported.If you have any feedback, please reach out to us via Discord or email our support.
Integrating CodSpeed into your Java project works through a fork of JMH (Java Microbenchmark Harness). You write standard JMH benchmarks and swap in the CodSpeed JMH fork as a dependency. When running in CI with CodSpeed, the results are automatically collected and reported.

​
Installation

CodSpeed provides a fork of JMH that collects walltime results and sends them to CodSpeed.
Add the CodSpeed JMH fork as a Git submodule:
git submodule add https://github.com/CodSpeedHQ/codspeed-jvm.git third-party/codspeed-jvm
Then include it as a composite build in your settings.gradle.kts, with dependency substitution to redirect JMH dependencies to the CodSpeed fork:
settings.gradle.kts
includeBuild("third-party/codspeed-jvm/jmh-fork") {              
    dependencySubstitution {                          
        substitute(module("org.openjdk.jmh:jmh-core"))           
            .using(project(":jmh-core"))              
        substitute(module("org.openjdk.jmh:jmh-generator-annprocess")) 
            .using(project(":jmh-generator-annprocess")) 
    }                                                 
}
We recommend using the JMH Gradle Plugin as it handles benchmark compilation and provides the jmh task. Add it to your build.gradle.kts:
build.gradle.kts
plugins {
    java
    id("me.champeau.jmh") version "0.7.2"
}

​
Creating benchmarks

Write your benchmarks using standard JMH annotations:
FibBenchmark.java
package bench;

import org.openjdk.jmh.annotations.*;

public class FibBenchmark {

  @Benchmark
  public long fib() {
    return fib(30);
  }

  private static long fib(int n) {
    if (n <= 1) return n;
    return fib(n - 1) + fib(n - 2);
  }
}
JMH benchmarks must return their result or use Blackhole.consume() to prevent the JVM from eliminating dead code. All examples on this page return the computed value.
For more information on how to write benchmarks with JMH, check out the official JMH samples.

​
Testing the benchmarks locally

To run the benchmarks with CodSpeed locally, first install the codspeed runner: 
curl -fsSL https://codspeed.io/install.sh | sh
Then run your benchmarks with CodSpeed:
terminal
$ codspeed run --mode walltime -- ./gradlew jmh
►►► Running the benchmarks
# JMH version: 0.1.0
# VM version: JDK 21.0.6, OpenJDK 64-Bit Server VM, 21.0.6+7
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Benchmark: bench.FibBenchmark.fib

# Run progress: 0.00% complete, ETA 00:01:40
# Warmup Iteration   1: 112.541 ns/op
...

Benchmark              Mode  Cnt      Score      Error  Units
FibBenchmark.fib       avgt    5    110.342 ±    2.153  ns/op

​
Running the benchmarks in your CI

To generate performance reports, you need to run the benchmarks in your CI. This allows CodSpeed to automatically run benchmarks and warn you about regressions during development.
If you want more details on how to configure the CodSpeed action, you can check out the Continuous Reporting section.
Here is an example of a GitHub Actions workflow that runs the benchmarks and reports the results to CodSpeed on every push to the main branch and every pull request:

​
Advanced usage

JMH provides many features for writing expressive benchmarks. Below is a selection that can be useful in CodSpeed benchmarks, but check out the official JMH samples for an exhaustive list of features.

​
Parameterized benchmarks

Use @Param to run the same benchmark with different input values:
ParamBenchmark.java
package bench;

import org.openjdk.jmh.annotations.*;

@State(Scope.Benchmark)
public class ParamBenchmark {

  @Param({"10", "20", "30"})
  int n;

  @Benchmark
  public long fib() {
    return fib(n);
  }

  private static long fib(int n) {
    if (n <= 1) return n;
    return fib(n - 1) + fib(n - 2);
  }
}

​
Shared state

Use @State to share setup logic across benchmarks and control the scope of the state object:
StateBenchmark.java
package bench;

import org.openjdk.jmh.annotations.*;
import java.util.ArrayList;
import java.util.List;

@State(Scope.Benchmark)
public class StateBenchmark {

  List<Integer> list;

  @Setup
  public void setup() {
    list = new ArrayList<>();
    for (int i = 0; i < 1000; i++) {
      list.add(i);
    }
  }

  @Benchmark
  public int sum() {
    return list.stream().mapToInt(Integer::intValue).sum();
  }
}

​
Compatibility

  • JDK 17 or later is required.
  • All standard JMH annotations are supported.
  • Only the Walltime instrument is supported
  • CodSpeed uses a custom mode for all benchmarks to collect statistically significant results. Any @BenchmarkMode annotations in your code are ignored.
If you run into issues or require certain features, please open an issue or join our Discord to get help.

​
Next steps

Example repository

The CodSpeed JVM repository with example JMH benchmarks.

Walltime instrument

Learn more about the Walltime instrument and how to use it.

Dive into performance changes

Learn more about profiling and how to read flame graphs.