Hot-patching the JVM to hook native Java agents

Over a year ago, I wrote a blog post called Who instruments the instrumenters? together with Mikaël Francoeur on how we debugged the Java instrumentation code. In the meantime, I gave a more detailed talk on this topic at VoxxedDays Amsterdam. The meta-agent that I developed for this worked well for Java agents/instrumenters, but what about native agents? Marco Sussitz found my agent and asked exactly this question. Native agents are agents that utilize the JVMTI API to, for example, modify class bytecode; however, they are not written in Java. With this blog post, I’m proud to announce that the meta-agent now supports instrumenting native agents.

TL;DR: Meta-agent allows you to see how an agent, native or Java, transforms bytecode.

There are many examples of native agents, like DynaTrace‘s monitoring agent or async-profiler‘s method tracer. I’m using the latter in my example here, as it’s open-source and readily available. The method tracer instruments the Java bytecode to trace the execution time of specific methods. You can find more about it in the async-profiler forum.

As a sample program, we use Loop.java:

public class Loop {
    public static void main(String[] args) 
      throws InterruptedException {
        while (true) Thread.sleep(1000);
    }
}

Let’s trace the Thrread.sleep method and use the meta-agent to see what async-profiler does with the bytecode:

java -agentpath:native/libnative_agent.dylib \
     -javaagent:target/meta-agent.jar=server \
     -agentpath:libasyncProfiler.dylib=start,trace=java.lang.Thread.sleep,file=duration.html \
     Loop.java

This opens a server at localhost:7071 and we check how async-profiler modified the Thread class:

So we can now instrument native agents like any other Java agent. And the part: As all Java agents are built on top of the libinstrument native agent, we can also see what any Java agent is doing. For example, we can see that the Java instrumentation agent instruments itself:

So I finally built an instrumenter that can essentially instrument my instrumentation agent, which in turn instruments other instrumentation agents. Another benefit is that the instrumenter can find every modification of any Java agent.

Background

Before I show you how I implemented the new meta-agent features, I want to show you how a typical native agent is implemented. In my blog post Instrumenting Java Code to Find and Handle Unused Classes, I showed how to develop your own instrumenting agent in Java. It’s pretty simple, we just create an implementation of the ClassFileTransformer interface, and implement its transform method to transform classes when they are loaded or a retransform is triggered.

But how would you do this in C/C++ using the native API? We have the JVM Tool Interface (JVMTI) at our disposal. This interface allows the creation of native agents. Commonly used native agents include, for example, the JDWP agent for debugging or the libinstrument agent, which triggers all the Java agents.

In the following, I’ll show you how to implement a tiny native agent; you can find the full source code on GitHub. In contrast to the Java agent, where we implement the ClassFileTransformer#transform method, we implement here a hook for the ClassFileLoad JVMTI event:

void JNICALL ClassFileLoadHook(
    jvmtiEnv *jvmti,
    JNIEnv *jni,
    jclass class_being_redefined,
    jobject loader,
    const char *name,
    jobject protection_domain,
    jint class_data_len,
    const unsigned char *class_data,
    jint *new_class_data_len,
    unsigned char **new_class_data
) {
   if (name) {
        printf("[Agent] Class loaded: %s\n", name);
    } else {
        printf("[Agent] Anonymous class loaded.\n");
    }
}

We don’t transform any of the bytecode here, as there is no proper native bytecode library, and all agents have to essentially reimplement their own (some are based on the old java_crw_demo.c).

Now we just need to create an agent and register the hook. For this, we implement the Agent_OnLoad method that is called when the agent is first loaded:

JNIEXPORT jint JNICALL Agent_OnLoad(JavaVM *vm, char *options, void *reserved) {
    // We define some variables where JVMTI environment
    // and the error state is later stored
    jvmtiEnv *jvmti;
    jvmtiError err;

    printf("[Agent] Agent_OnLoad called.\n");

    // We get the JVMTI environment and fail if we had a problem
    if ((*vm)->GetEnv(vm, (void **)&jvmti, JVMTI_VERSION_1_2) != JNI_OK) {
        printf("[Agent] Unable to get JVMTI environment.\n");
        return JNI_ERR;
    }

    // We set capabilities of this agent, requesting access to ClassFileLoadHooks
    jvmtiCapabilities caps;
    memset(&caps, 0, sizeof(caps));
    caps.can_generate_all_class_hook_events = 1;
    err = (*jvmti)->AddCapabilities(jvmti, &caps);
    if (err != JVMTI_ERROR_NONE) {
        printf("[Agent] AddCapabilities failed: %d\n", err);
        return JNI_ERR;
    }

    // We register the ClassFileLoadHook
    jvmtiEventCallbacks callbacks;
    memset(&callbacks, 0, sizeof(callbacks));
    callbacks.ClassFileLoadHook = &ClassFileLoadHook;
    err = (*jvmti)->SetEventCallbacks(jvmti, &callbacks, sizeof(callbacks));
    if (err != JVMTI_ERROR_NONE) {
        printf("[Agent] SetEventCallbacks failed: %d\n", err);
        return JNI_ERR;
    }

    // We finally enable the ClassFileLoadHook event
    err = (*jvmti)->SetEventNotificationMode(jvmti, JVMTI_ENABLE, JVMTI_EVENT_CLASS_FILE_LOAD_HOOK, NULL);
    if (err != JVMTI_ERROR_NONE) {
        printf("[Agent] SetEventNotificationMode failed: %d\n", err);
        return JNI_ERR;
    }

    // and return
    printf("[Agent] ClassFileLoadHook registered.\n");
    return JNI_OK;
}

We now have a simple native agent that informs us whenever a class is loaded or retransformed. After we built it, we can use it with the Loop.java file from before:

> java -agentpath:native/libagent_minimal_cfh.dylib Loop.java               
[Agent] Agent_OnLoad called.
[Agent] ClassFileLoadHook registered.
[Agent] Class loaded: jdk/internal/vm/ContinuationSupport
[Agent] Class loaded: jdk/internal/vm/Continuation$Pinned
[Agent] Class loaded: sun/launcher/LauncherHelper
[Agent] Class loaded: jdk/internal/loader/BuiltinClassLoader$2
...

It is important to note that the JVM calls the Agent_OnLoad method of every native agent before the JVM starts loading our application (or the compiler in our example) and before the JVM starts any Java agent.

Now the question is: How can we instrument the native agent? In the Java case, we just transformed every call to Instrumentation#addTransformer to call a meta-agent method instead, which wrapped that Transformer before adding it.

Our goal now is to achieve a similar outcome with native agents. In an ideal world, we would just transform all calls to SetEventCallbacks. But can we? Without any vtable hacks or binary trickery? This is where an observation comes in handy:

Patching JVMTI

The SetEventCallbacks method belongs to the JVMTI environment struct that our agent obtained from the JVM by calling GetEnv. This struct is defined in the generated JVMTI header as follows (for C source code, it’s different for C++):

typedef struct jvmtiInterface_1_ {

  /*   1 :  RESERVED */
  void *reserved1;

  /*   2 : Set Event Notification Mode */
  jvmtiError (JNICALL *SetEventNotificationMode) (jvmtiEnv* env,
    jvmtiEventMode mode,
    jvmtiEvent event_type,
    jthread event_thread,
     ...);
  // ...
  
    /*   122 : Set Event Callbacks */
  jvmtiError (JNICALL *SetEventCallbacks) (jvmtiEnv* env,
    const jvmtiEventCallbacks* callbacks,
    jint size_of_callbacks);
  // ...
}

A tiny agent that prints the address of (*jvmti)->SetEventCallbacks later we can confirm that the method pointers for all agents point to the same address. So we can just override this method pointer via

*(void**)&((*jvmti)->SetEventCallbacks) = &SetEventCallbacks;

And now our wrapper

jvmtiError
SetEventCallbacks(jvmtiEnv* env, 
  jvmtiEventCallbacks* callbacks, 
  jint size_of_callbacks) { ... }

is called whenever a follow-up agent calls this method. Therefore, our native wrapping agent must always be the first agent. So we’re essentially hot-patching the JVM.

In the most basic case of only supporting wrapping one agent, we can implement a basic wrapper method as follows (GitHub):

// Original ClassFileLoadHook callback from the wrapped agent
static void (*original_ClassFileLoadHook)(jvmtiEnv *jvmti, JNIEnv *jni, 
                                          jclass class_being_redefined, jobject loader, 
                                          const char *name, jobject protection_domain, 
                                          jint class_data_len, const unsigned char *class_data, 
                                          jint *new_class_data_len, unsigned char **new_class_data) = NULL;

// Our wrapper for ClassFileLoadHook
static void JNICALL
wrapped_ClassFileLoadHook(jvmtiEnv *jvmti, JNIEnv *jni, jclass class_being_redefined,
                          jobject loader, const char *name, jobject protection_domain,
                          jint class_data_len, const unsigned char *class_data,
                          jint *new_class_data_len, unsigned char **new_class_data) {
    printf("[WRAPPER] ClassFileLoadHook called for class: %s\n", name ? name : "NULL");
    // Call the original ClassFileLoadHook
    original_ClassFileLoadHook(jvmti, jni, class_being_redefined, loader, name,
                                protection_domain, class_data_len, class_data,
                                new_class_data_len, new_class_data);
}

// Our wrapper for SetEventCallbacks
jvmtiError
SetEventCallbacks(jvmtiEnv* env, jvmtiEventCallbacks* callbacks, jint size_of_callbacks) {
    printf("[WRAPPER] SetEventCallbacks called\n");
    
    if (callbacks != NULL && callbacks->ClassFileLoadHook != NULL) {
        printf("[WRAPPER] Intercepting ClassFileLoadHook callback\n");
        
        // Store the original ClassFileLoadHook callback
        original_ClassFileLoadHook = callbacks->ClassFileLoadHook;

        // Replace with our wrapped version
        callbacks->ClassFileLoadHook = wrapped_ClassFileLoadHook;

        // Call original SetEventCallbacks
        return original_SetEventCallbacks(env, callbacks, size_of_callbacks);
    }
    
    // No ClassFileLoadHook to wrap, just pass through
    return original_SetEventCallbacks(env, callbacks, size_of_callbacks);
}

And that’s all. The implementation of this native agent instrumentation is relatively simple, yet it took me the better part of a morning, somewhere in Bulgaria, to come up with it.

Supporting Multiple Agents

Now we just need to support multiple agents. We could, in theory, create function objects on the heap, but this would require us to write or copy a small amount of custom assembly code onto a part of the heap that we would have to mark as executable. However, there is a more straightforward, albeit less elegant solution: we can limit the number of supported agent loadings to a large, fixed number, such as 4096, and probably be fine.

Now we start by defining an array of a ClassFileLoadHookInfo struct which includes both the wrapped load hook and the name of the agent:

typedef struct {
    void (*callback)(jvmtiEnv *jvmti, JNIEnv *jni, jclass class_being_redefined, 
                     jobject loader, const char *name, jobject protection_domain, 
                     jint class_data_len, const unsigned char *class_data, 
                     jint *new_class_data_len, unsigned char **new_class_data);
    char name[MAX_AGENT_NAME_LEN];
} ClassFileLoadHookInfo;

static ClassFileLoadHookInfo agent_info[MAX_AGENTS];
static int next_agent_slot = 0;

This is where we store all the wrapped load hooks. The idea is then to create 4096 functions, where function 0 wraps the load hook from agent_info[0], … . All these functions are created using macros and their addresses in a static WrapperFunc wrapper_functions[MAX_AGENTS] so that we later easily get the nth function address.

But how does the wrapping function then communicate with our meta-agent?

Communicating with the meta-agent

Unfortunately, we cannot simply use JVMTI to invoke the meta-agent’s Java code and record the collected bytecode transformations within the meta-agent. The problem is that we would need to do this within a ClassFileLoadHook, which would easily create circular class loading errors.

Instead, we create a temporary folder /tmp/njvm<pid> and atomically create numbered files in there for every recorded transformation. The files have the following format:

• Line 1: agent_name (e.g., “agent_minimal_cfh”)
• Line 2: class_name (e.g., “java/lang/String” or “unknown”)
• Line 3: old_len (decimal number, e.g., “1234”)
• Line 4: new_len (decimal number, e.g., “1456”)
• Binary data: old_len bytes of original class data
• Binary data: new_len bytes of transformed class data

The meta-agent has a loop that iterates regularly over the folder, parsing all files within and removing duplicates, ensuring that we don’t encounter duplicates.

Using the Native Agent

But first: Is this safe? Yes (but don’t use it in production), and the implementation itself probably still contains some errors. However, it’s platform-independent and doesn’t use undefined behavior in C, nor does it use any assembly code.

To use it, just clone the meta-agent repository and build the native agent via make:

git clone https://github.com/parttimenerd/meta-agent
cd meta-agent
(cd native; make)
# for good measure also build the Java meta-agent
mvn package -DskipTests
# Now you can use it via
java -agentpath:native/libnative_agent.dylib=log=verbose \
     -javaagent:target/meta-agent.jar=server ...

Conclusion

In this blog post, I showcased the new support for native agents in the meta-agent project and how I implemented the feature using JVM hot patching. This is particularly beneficial for debugging native agents and examining what existing agents, such as async-profiler’s method tracer, do. Of course, we could extend this to also wrap other JVMTI methods, but for now, this is out of scope.

Thanks for coming so far. I hope to see you in two weeks with something else.

P.S.: Here is a picture of a cat from the Pompeii museum.

This blog post is part of my work in the SapMachine team at SAP, making profiling easier for everyone. Thanks to Marco Sussitz for the idea of instrumenting native agents and for showcasing my meta-agent at JavaZone.