Reproducing a Tricky Bug in Minutes With a Custom Linux Scheduler Written in Java

Ever had a tricky bug caused by a race condition or rare concurrency condition that was really hard to reproduce? It’s great when you have a fix that should work in theory, but without a reproducer, only time will tell whether your fix really worked. In this blog post, we’ll revisit my old blog post Hello eBPF: Concurrency Testing using Custom Linux Schedulers (19), and try to use the concurrency-fuzz-scheduler to reproduce a bug I fixed a while ago in the OpenJDK.

The scheduler aims to be as chaotic as possible; hence, Jake Hillion’s Rust version is called scx_chaos. But we’ll focus on the Java version, the concurrency-fuzz-scheduler, because it’s not only implemented in Java on top of my hello-ebpf library, but it’s also optimized for fuzzing Java applications, inserting random sleeps at the scheduler level with a focus on non-VM threads.

TL;DR: The concurrency scheduler is a nice tool to provoke rare parallelism conditions and create reproducers.

The bug in question is JDK-8366486, reported by David Holmes in August 2025: A test case that checks that we can run multiple recordings with the CPU-time sampler in direct succession does work. The only problem: The test should not work, but it still worked most of the time. If you’re only interested in the actual bug, skip ahead to the end of the blog post for an explanation.

You’ll find the fixed version here and the broken version here (because the old JDK with the actual bug had compilation issues on my current system, I had to reintroduce the bug in a separate branch).

Let’s start with running the test case with the standard Linux scheduler on a large machine, so that everything can run nicely in parallel:

Continue reading

Reading and Writing JFR Files Programmatically

Last week, I showed you the Fastest Way to get the Version of a Java Installation. This week, I’ll show you something completely different: how to interact with JFR data programmatically, showcasing a new library called basic-jfr-processor in the process.

While JFR is a great tool for profiling your application and gaining insights, the file format is, on purpose, not well documented or specified. One of the best sources of information is Gunnar Morling’s blog post on the topic, and of course, the OpenJDK source code.

But of course, there are ready-made APIs for reading JFR files and OpenJDK-adjacent libraries to write them. In this overview blog post, I’ll showcase the built-in Java JFR API, Jaroslav Bachorik’s jafar API, and the JMC JFR writer API, as well as my own basic-jfr-processor library based on the latter.

We start with the built-in API:

Continue reading

The Fastest Way to get the Version of a Java Installation

Last week, I demonstrated that OpenJDK is faster than GraalVM Java, at least for obtaining the Java version. This even prompted the mighty Thomas Wuerthinger (creator of GraalVM) to react. But the measured ~20ms for the OpenJDK is still too slow for applications like execjar, where it could significantly increase the runtime of short-running CLI tools. In this week’s brief blog post, I’ll show you the fastest way to access the Java version.

The main performance issue is that calling java -version creates a process with a fairly large (around 38MB) maximum resident set size, and using a proper command line parser. But do we actually need to call the java binary to get the version?

TL;DR: I created the java-version tool, which can obtain the Java version in under a millisecond.

Basic Idea

No, we can just realize that most Java installations have a release file that contains the relevant information in a machine-readable format. You can find this file in the main folder of the installation (./release when java is in ./bin).

Continue reading

OpenJDK is faster than GraalVM Java*

Well, we all know that the most crucial feature of the JVM runtime is the -version output. So how does the OpenJDK (in the form of SapMachine) compare with GraalVM? It’s significantly faster. Using hyperfine, we can see that GraalVM 25 CE takes almost twice as long to emit the version number as a regular SapMachine 25 on my MacBook Pro M5:

The slowness of java -version was actually one of the performance issues of the tool I showcased in How to Build an Executable from a JAR using ExecJAR, as it originally used java -version a lot to check the Java version constraint.

Is this relevant? Not really. However, so are most microbenchmarks and benchmarks in general that are taken out of context. You should not generalize small benchmarks, and modern systems are complex.

You can find some bigger, non-version-related benchmarks comparing the different JVMs, for example, at https://ionutbalosin.com/2024/02/jvm-performance-comparison-for-jdk-21/.

Join me next week for a blog post on something different and learn how to check the version of a Java installation even faster in under one millisecond:

P.S.: I just ran some more benchmarks: OpenJDK 25 is 18% faster than OpenJDK 17 and 21 and a whopping 84% faster than OpenJDK 11. Upgrade now!

P.P.S.: As many people (Thomas Wuerthinger, Fabio Niebhaus, Volker Simonis, and multiple of my SapMachine colleagues) pointed out, the differences between OpenJDK and GraalVM are due to the GraalVM initializing the JVM Compiler Interface (JVMCI). The difference between the two becomes negligible when running OpenJDK with enabled JVMCI (initialize the JIT at the beginning):

How to Build an Executable from a JAR using ExecJAR

In my last blog post, I covered a new tool called jstall, which enables you to quickly check on a Java application. Because it was tiresome to always call the tool via java -jar jstall, I looked for a way to create executables directly from JARs, inspired by async-profiler’s build system. And I, of course, went down a rabbit hole. In this blog post, I’ll show you how use execjar to easily create your own executable JARs that you can execute directly on the command line while still being valid JARs.

TL;DR: execjar is a CLI and Maven plugin that enables you to create executables from JARs by just adding a few lines to your Maven file:

<plugin>
  <groupId>me.bechberger</groupId>
  <artifactId>execjar</artifactId>
  <version>0.1.1</version>
  <executions>
    <execution>
      <goals>
        <goal>execjar</goal>
      </goals>
    </execution>
  </executions>
</plugin>

When your project is called jstall, this creates an executable with the same name that you can execute directly via ./jstall.

Important: The resulting executable is compatible only with UNIX (Linux and macOS) environments.

However, before I delve into the in-depth configuration options of my new tool, I’d like to provide some background on its implementation.

Continue reading