How to Build an Executable from a JAR using ExecJAR

Posted on by

In my last blog post, I covered a new tool called jstall, which enables you to quickly check on a Java application. Because it was tiresome to always call the tool via java -jar jstall, I looked for a way to create executables directly from JARs, inspired by async-profiler’s build system. And I, of course, went down a rabbit hole. In this blog post, I’ll show you how use execjar to easily create your own executable JARs that you can execute directly on the command line while still being valid JARs.

TL;DR: execjar is a CLI and Maven plugin that enables you to create executables from JARs by just adding a few lines to your Maven file:

<plugin>
  <groupId>me.bechberger</groupId>
  <artifactId>execjar</artifactId>
  <version>0.1.1</version>
  <executions>
    <execution>
      <goals>
        <goal>execjar</goal>
      </goals>
    </execution>
  </executions>
</plugin>

When your project is called jstall, this creates an executable with the same name that you can execute directly via ./jstall.

Important: The resulting executable is compatible only with UNIX (Linux and macOS) environments.

However, before I delve into the in-depth configuration options of my new tool, I’d like to provide some background on its implementation.

Continue reading →

Quickly Inspect your Java Application with JStall

Posted on by

Welcome to the last blog post of the year. Last week, I discussed the limitations of custom JFR events. This week, I’ll also be covering a profiling-related topic and showcasing a tiny tool called JStall.

I hope I’m not the only one who sometimes wonders: “What is my Java application doing right now?” When you don’t see any output. Yes, you could perform a simple thread dump via jstack, but it is hard to understand which threads are actually consuming CPU and making any sort of progress. This is where my tiny tool called JStall comes in:

JStall is a small command-line tool for one-shot inspection of running JVMs using thread dumps and short, on-demand profiling. The tool essentially takes multiple thread dumps of your application and uses the per-thread cpu-time information to find the most CPU-time-consuming Java threads.

First, download the JStall executable from the GitHub releases page. Let us then start by finding the currently running JVMs:

> ./jstall
Usage: jstall <command> <pid|files> [options]

Available commands:
  status     - Show overall status (deadlocks + most active threads)
  deadlock   - Check for deadlocks
  most-work  - Show threads doing the most work
  flame      - Generate flame graph
  threads    - List all threads

Available JVMs:
  7153 ./jstall
  1223 <unknown>
  8136 ./renaissance-gpl-0.16.0.jar
  6138 org.jetbrains.idea.maven.server.RemoteMavenServer36
  5597 DeadlockDemo
  49294 com.intellij.idea.Main

This provides us with a list of options for the main status command, as well as a list of JVM processes and their corresponding main classes. Let’s start checking for deadlocking:

Continue reading →

Don’t use Arrays or other Complex Types in Custom JFR Events

Posted on by

JDK Flight Recorder (JFR) provides support for custom events as a profiler. Around two years ago, I wrote a blog post on this very topic: Custom JFR Events: A Short Introduction. These custom events are beneficial because they enable us to record additional project-specific information alongside the standard JFR events, all in the same file. We can then view and process this information with the JFR tools. You can freely specify these events in Java.

There is only one tiny problem nobody talks about: Array support (and, in more general, the support of complex types).

Take the following event:

class ArrayEvent extends Event {
    
    @Label("String Array")
    String[] stringArray;
    
    @Label("Int Array")
    int[] intArray;
    
    @Label("Long Array")
    long[] longArray;
    
    @Label("Non array field")
    String nonArrayField = "default";
}

What would you expect to happen when we create the event using the following code?

ArrayEvent event = new ArrayEvent();
event.stringArray = new String[]{"one", "two", "three"};
event.intArray = new int[]{1, 2, 3, 4, 5};
event.longArray = new long[]{100L, 200L, 300L};
event.commit();
Continue reading →

Creating a Gridfinity Chocolate Advent Calendar

Posted on by

This week is a bit different as I’m working on a fun year-end blog and doing my regular work, so I’ll share with you how to create an advent calendar:

Normal Advent calendars are boring: so let’s make our own! We’ll combine all our favorite technologies: Gridfinity (for the grid system), 3D printing (for the grid), vacuum molding (for the chocolate), laser cutting (for the frame), and automated paper cutting to create advent calendars that are both beautiful and functional.

Along the way, we’ll cover practical food safety considerations and show how these techniques come together to produce something tasty, nerdy, and gift-worthy:

You can mind the positive for the chocolate mold on MakerWorld:

I also created a 2×2 Gridfinity grid that fits RitterSport:

You can learn how to create QR codes out of chocolate in another video of mine:

I usually post computer/Java-related blog posts, but I hope you also liked this blog post, in which I shared my hobbies.

Who instruments the native instrumenters?

Posted on by

Hot-patching the JVM to hook native Java agents

Over a year ago, I wrote a blog post called Who instruments the instrumenters? together with Mikaël Francoeur on how we debugged the Java instrumentation code. In the meantime, I gave a more detailed talk on this topic at VoxxedDays Amsterdam. The meta-agent that I developed for this worked well for Java agents/instrumenters, but what about native agents? Marco Sussitz found my agent and asked exactly this question. Native agents are agents that utilize the JVMTI API to, for example, modify class bytecode; however, they are not written in Java. With this blog post, I’m proud to announce that the meta-agent now supports instrumenting native agents.

TL;DR: Meta-agent allows you to see how an agent, native or Java, transforms bytecode.

There are many examples of native agents, like DynaTrace‘s monitoring agent or async-profiler‘s method tracer. I’m using the latter in my example here, as it’s open-source and readily available. The method tracer instruments the Java bytecode to trace the execution time of specific methods. You can find more about it in the async-profiler forum.

As a sample program, we use Loop.java:

public class Loop {
    public static void main(String[] args) 
      throws InterruptedException {
        while (true) Thread.sleep(1000);
    }
}

Let’s trace the Thrread.sleep method and use the meta-agent to see what async-profiler does with the bytecode:

java -agentpath:native/libnative_agent.dylib \
     -javaagent:target/meta-agent.jar=server \
     -agentpath:libasyncProfiler.dylib=start,trace=java.lang.Thread.sleep,file=duration.html \
     Loop.java

This opens a server at localhost:7071 and we check how async-profiler modified the Thread class:

Code transformation by the native Java method tracer of async-profiler

So we can now instrument native agents like any other Java agent. And the part: As all Java agents are built on top of the libinstrument native agent, we can also see what any Java agent is doing. For example, we can see that the Java instrumentation agent instruments itself:

So I finally built an instrumenter that can essentially instrument my instrumentation agent, which in turn instruments other instrumentation agents. Another benefit is that the instrumenter can find every modification of any Java agent.

Continue reading →

Running an LLM on an Android Phone

Posted on by

In my last blog post, I showed you how to work with JFR files using DuckDB, which started a blog series that I surely will continue. Just not this week. Instead, I want to showcase a tiny app to run AI models using the MediaPipe API directly on your phone. I created the app for another purpose (perhaps described in a future blog post) earlier this year, but never wrote anything about it. So here we are.

TL;DR: I built an Android app that offers AI models via a server

The app is open-source and available on GitHub; it’s experimental, but maybe it can help you build your own apps. You can download the releases page of the repo and install it.

The LLM API endpoint, writing a poem on a backyard scene
Continue reading →

Making JFR Quack: Importing JFR files into DuckDB

Posted on by

In my previous post, I showed you how tricky it is to compare objects from the JFR Java API. You probably wondered why I wrote about this topic. Here is the reason: In this blog post, I’ll cover how to load JFR files into a DuckDB database to allow querying profiling data with simple SQL queries, all JFR views included.

This blog post will start a small series on making JFR quack.

TL;DR

You can now use a query tool (via GitHub) to transform JFR files into similarly sized DuckDB files:

> java -jar target/query.jar duckdb import jfr_files/recording.jfr duckdb.db
> duckdb duckdb.db "SELECT * FROM Events";
┌───────────────────────────────┬───────┐
│             name              │ count │
│            varchar            │ int32 │
├───────────────────────────────┼───────┤
│ GCPhaseParallel               │ 69426 │
│ ObjectAllocationSample        │  6273 │

Or run the queries directly, with the database file being cached (if you don’t pass --no-cache), directly supporting all built-in JFR views:

> java -jar target/query.jar query jfr_files/metal.jfr "hot-methods" 
Method                                                                                                   Samples Percent
-------------------------------------------------------------------------------------------------------- ------- -------
java.util.concurrent.ForkJoinPool.deactivate(ForkJoinPool.WorkQueue, int)                                   1066   8.09%
scala.collection.immutable.RedBlackTree$.lookup(RedBlackTree.Tree, Object, Ordering)                         695   5.27%
akka.actor.dungeon.Children.initChild(ActorRef)                                                              678   5.14%

This view is implemented as:

CREATE VIEW "hot-methods" AS
SELECT
  (c.javaName || '.' || m.name || m.descriptor) AS "Method",
  COUNT(*) AS "Samples",
  format_percentage(COUNT(*) / (SELECT COUNT(*) FROM ExecutionSample)) AS "Percent"
FROM ExecutionSample es
JOIN Method m ON es.stackTrace$topMethod = m._id
JOIN Class c ON m.type = c._id
GROUP BY es.stackTrace$topApplicationMethod, c.javaName, m.name, m.descriptor
ORDER BY COUNT(*) DESC
LIMIT 25
Continue reading →

JFR and Equality: A tale of many objects

Posted on by

In the last blog post, I showed you how to silence JFR’s startup messages. This week’s blog post is also related to JFR, and no, it’s not about the JFR Events website, which got a simple search bar. It’s a short blog post on comparing objects from JFR recordings in Java and why this is slightly trickier than you might have expected.

Example

Getting a JFR recording is simple; just use the RecordingStream API. We do this in the following to record an execution trace of a tight loop using JFR and store it in a list:

List<RecordedEvent> events = new ArrayList<>();
// Know when to stop the loop
AtomicBoolean running = new AtomicBoolean(true);
// We obtain one hundred execution samples 
// that have all the same stack trace
final long currentThreadId = Thread.currentThread().threadId();
try (RecordingStream rs = new RecordingStream()) {
    rs.enable("jdk.ExecutionSample").with("period", "1ms");
    rs.onEvent("jdk.ExecutionSample", event -> {
        if (event.getThread("sampledThread")
                 .getJavaThreadId() != currentThreadId) {
            return; // don't record other threads
        }
        events.add(event);
        if (events.size() >= 100) {
            // we can signal to stop
            running.set(false);
        }
    });
    rs.startAsync();
    int i = 0;
    while (running.get()) { // some busy loop to produce sample
        for (int j = 0; j < 100000; j++) {
            i += j;
        }
    }
    rs.stop();
}
Continue reading →

Silencing JFR’s Startup Message

Posted on by

TD;DR: -Xlog:jfr+startup=error is your friend.

Ever wondered why JFR emits something like

[0.172s][info][jfr,startup] Started recording 1. No limit specified, using maxsize=250MB as default.
[0.172s][info][jfr,startup] 
[0.172s][info][jfr,startup] Use jcmd 29448 JFR.dump name=1 to copy recording data to file.

when starting the Flight Recorder with -XX:StartFlightRecorder? Even though the default logging level is warning, not info?

This is what this week’s blog post is all about. After I showed you last week how to waste CPU like a Professional, this week I’ll show you how to silence JFR. Back to the problem:

Continue reading →

How to waste CPU like a Professional

Posted on by

Or: Hey, keeping the CPU busy for a given amount of time should be easy?

Welcome back to my blog. Last week, I showed you how to profile your Cloudfoundry application, and the week before, how I made the CPU-time profiler a tiny bit better by removing redundant synchronization. This week’s blog post will be closer to the latter, trying to properly waste CPU.

As a short backstory, my profiler needed a test to check that the queue size of the sampler really increased dynamically (see Java 25’s new CPU-Time Profiler: Queue Sizing (3)), so I needed a way to let a thread spend a pre-defined number of seconds running natively on the CPU. You can find the test case in its hopefully final form here, but be aware that writing such cases is more complicated than it looks.

So here we are: In need to essentially properly waste CPU-time, preferably in user-land, for a fixed amount of time. The problem: There are only a few scant resources online, so I decided to create my own. I’ll show you seven different ways to implement a simple

void my_wait(int seconds);

method, and you’ll learn far more about this topic than you ever wanted to. That works both on Mac OS and Linux. All the code is MIT licensed; you can find it on GitHub in my waste-cpu-experiments, alongside some profiling results.

As another tangent: Apparently, my Java 25’s new CPU-Time Profiler (1) blog post blew up on Hacker News. Fun times.

Continue reading →

Profiling with the Cloud Foundry CLI Java plugin

Posted on by

Welcome back to my blog, this time for a blog post on profiling your Java applications in Cloud Foundry and the tool I helped to develop to make it easier.

Cloud Foundry “is an open source, multi-cloud application platform as a service (PaaS) governed by the Cloud Foundry Foundation, a 501(c)(6) organization” (Wikipedia). It allows you to run your workloads easily in the cloud, including your Java applications. You just need to define a manifest.yml, like for example:

---
applications:
- name: sapmachine21
  random-route: true
  path: test.jar
  memory: 512M
  buildpacks: 
  - sap_java_buildpack
  env:
    TARGET_RUNTIME: tomcat
    JBP_CONFIG_COMPONENTS: "jres: ['com.sap.xs.java.buildpack.jdk.SAPMachineJDK']"
    JBP_CONFIG_SAP_MACHINE_JDK : "{ version: 21.+ }"
    JBP_CONFIG_JAVA_OPTS: "[java_opts: '-XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints']"

But how would you profile this application? This and more is the topic of this blog post.

I will not discuss why you might want to use Cloud Foundry or how you can deploy your own applications. I assume you came this far in the blog post because you already have basic Cloud Foundry knowledge and want to learn how to profile your applications easily.

The Java Plugin

Cloud Foundry has a cf CLI with a proper plugin system with lots of plugins. A team at SAP, which included Tim Gerrlach, started to develop the Java plugin many years ago at SAP. It’s a plugin offering utilities to gain insights into JVMs running in your Cloud Foundry app.

Continue reading →

Java 25’s new CPU-Time Profiler: Removing Redundant Synchronization (4)

Posted on by

The changes I described in this blog post led to segfaults in tests, so I backtracked on them for now. Maybe I made a mistake implementing the changes, or my reasoning in the blog post is incorrect. I don’t know yet.

In the last blog post, I wrote about how to size the request queue properly and proposed the sampler queue’s dynamic sizing. But what I didn’t talk about in this or the previous blog post are two topics; one rather funny and one rather serious:

  1. Is the sampler queue really a queue?
  2. Should the queue implementation use Atomics and acquire-release semantics?

This is what we cover in this short blog post. First, to the rather fun topic:

Is it a Queue?

I always called the primary data structure a queue, but recently, I wondered whether this term is correct. But what is a queue?

Definition: A collection of items in which only the earliest added item may be accessed. Basic operations are add (to the tail) or enqueue and delete (from the head) or dequeue. Delete returns the item removed. Also known as “first-in, first-out” or FIFO.

Dictionary of Algorithms and Data Structures by Paul E. Black

But how does my sampler use the sampler queue?

Continue reading →

Java 25’s new CPU-Time Profiler: Queue Sizing (3)

Posted on by

Welcome back to my series on the new CPU-time profiler in Java 25. In the previous blog post, I covered the implementation of the new profiler. In this week’s blog post, I’ll dive deep into the central request queue, focusing on deciding its proper size.

The JfrCPUTimeTraceQueue allows the signal handler to record sample requests that the out-of-thread sampler and the safepoint handler process. So it’s the central data structure of the profiler:

This queue is thread-local and pre-allocated, as it’s used in the signal handler, so the correct sizing is critical:

  • If the size is too small, you’ll lose many samples because the signal handler can’t record sample requests.
  • If you size it too large, you waste lots of memory. A sampling request is 48 bytes, so a queue with 500 elements (currently the default) requires 24kB. This adds up fast if you have more than a few threads.

So, in this blog post, we’re mainly concerned about setting the correct default size and discussing a potential solution to the whole problem.

Continue reading →

Java 25’s new CPU-Time Profiler: The Implementation (2)

Posted on by

I developed, together with others, the new CPU-time profiler for Java, which is now included in JDK 25. A few weeks ago, I covered the profiler’s user-facing aspects, including the event types, configuration, and rationale, alongside the foundations of safepoint-based stack walking in JFR (see Taming the Bias: Unbiased Safepoint-Based Stack Walking). If you haven’t read those yet, I recommend starting there. In this week’s blog post, I’ll dive into the implementation of the new CPU-time profiler.

It was a remarkable coincidence that safepoint-based stack walking made it into JDK 25. Thanks to that, I could build on top of it without needing to re-implement:

  • The actual stack walking given a sampling request
  • Integration with the safepoint handler

Of course, I worked on this before, as described in Taming the Bias: Unbiased Safepoint-Based Stack Walking. But Erik’s solution for JDK 25 was much more complete and profited from his decades of experience with JFR. In March 2025, whether the new stack walker would get into JDK 25 was still unclear. So I came up with other ideas (which I’m glad I didn’t need). You can find that early brain-dump in Profiling idea (unsorted from March 2025).

In this post, I’ll focus on the core components of the new profiler, excluding the stack walking and safepoint handler. Hopefully, this won’t be the last article in the series; I’m already researching the next one.

Main Components

There are a few main components of the implementation that come together to form the profiler:

Continue reading →

Profiling idea (unsorted from March 2025)

Posted on by

This is my actual collection of ideas from March 2025, when it was unclear whether the updated JFR sampling at safepoints made it into JDK 25. It eventually did, so I scrapped the ideas. But it offers the reader an interesting, unfiltered look into my ideas and thoughts at the time, probably only useful for people who are really into profiling and the OpenJDK. Just be aware that it is therefore a document of its time (March 2025) and doesn’t reflect the actual current implementation. Also, don’t expect any deeper explanations.

Well, I warned you…

An Experimental Front-End for JFR Queries

Posted on by

Ever wondered how the views of the jfr tool are implemented? There are views like hot-methods which gives the most used methods, or cpu-load-samples that gives you the system load over time that you can directly use on the command line:

> jfr view cpu-load-samples recording.jfr

                                     CPU Load

Time                         JVM User           JVM System           Machine Total
------------------ ------------------ -------------------- -----------------------
14:33:29                        8,25%                0,08%                  29,65%
14:33:30                        8,25%                0,00%                  29,69%
14:33:31                        8,33%                0,08%                  25,42%
14:33:32                        8,25%                0,08%                  27,71%
14:33:33                        8,25%                0,08%                  24,64%
14:33:34                        8,33%                0,00%                  30,67%
...

This is helpful when glancing at JFR files and trying to roughly understand their contents, without loading the files directly into more powerful, but also more resource-hungry, JFR viewers.

In this short blog post, I’ll show you how the views work under the hood using JFR queries and how to use the queries with my new experimental JFR query tool.

I didn’t forget the promised blog post on implementing the new CPU-time profiler in JDK 25; it’ll come soon.

Under the hood, JFR views use a built-in query language to define all views in the view.ini file. The above is, for example, defined as:

[environment.cpu-load-samples]
label = "CPU Load"
table = "SELECT startTime, jvmUser, jvmSystem, machineTotal FROM CPULoad"

With my new query tool (GitHub), we can plot this as:

Continue reading →

Java 25’s new CPU-Time Profiler (1)

Posted on by

This is the first part of my series; the other parts are

Back to the blog post:

More than three years in the making, with a concerted effort starting last year, my CPU-time profiler landed in Java with OpenJDK 25. It’s an experimental new profiler/method sampler that helps you find performance issues in your code, having distinct advantages over the current sampler. This is what this week’s and next week’s blog posts are all about. This week, I will cover why we need a new profiler and what information it provides; next week, I’ll cover the technical internals that go beyond what’s written in the JEP. I will quote the JEP 509 quite a lot, thanks to Ron Pressler; it reads like a well-written blog post in and of itself.

Before I show you its details, I want to focus on what the current default method profiler in JFR does:

Continue reading →

Running Garden Linux on a Phone

Posted on by

A month ago, I showed you in my article CAP in the pocket: Developing Java Applications on your Phone (a more concise version), how to run a CAP development environment on your phone. In this week’s short blog post, I use a different angle. I wanted to see whether I could run SAP’s Garden Linux on my phone.

If you’re waiting for my CPU time profiler blog post, it’ll come soon (hopefully on Monday).

Garden Linux is a Debian GNU/Linux derivate that aims to provide small, auditable Linux images for most cloud providers (e.g. AWS, Azure, GCP etc.) and bare-metal machines. Garden Linux is the best Linux for Gardener nodes. Garden Linux provides great possibilities for customizing that is made by a highly customizable feature set to fit your needs.

gardenlinux.io

TL;DR: Yes, you can install it. And it works somewhat.

Continue reading →

Taming the Bias: Unbiased* Safepoint-Based Stack Walking in JFR

Posted on by

Two years ago, I still planned to implement a new version of AsyncGetCallTrace in Java. This plan didn’t materialize, but Erik Österlund had the idea to fully walk the stack at safepoints during the discussions. Walking stacks only at safepoints normally would incur a safepoint-bias (see The Inner Workings of Safepoints), but when you record some program state in signal handlers, you can prevent this. I wrote about this idea and its basic implementation in Taming the Bias: Unbiased Safepoint-Based Stack Walking. I’ll revisit this topic in this week’s short blog post because Markus Grönlund took Erik’s idea and started implementing it for the standard JFR method sampler:

Continue reading →

CAP in the pocket: Developing Java Applications on your Phone

Posted on by

Smartphones are more powerful then ever, with processors rivaling old laptops. So let’s try to use them like a laptop to develop web-applications on the go. In this weeks blog post I’ll show you how to do use run and develop a CAP Java Spring Boot application on your smartphone and how to run VSCode locally to develop and modify it. This, of course, works only on Android phones, as they are a Linux at their core.

Continue reading →