Tag Archives: sched_ext

Hello eBPF: A scheduler controlled by sound (20)

Posted on by
Reply

Welcome back to my hello-ebpf series. Last week, I was an attendee at a scheduling and power management summit (and to bring someone a sched-ext-themed birthday cake), and the Chemnitz Linux Days, so this blog post will be a bit shorter but cover a small scheduler that I wrote for Chemnitz.

In all of my eBPF presentations, I quote Brendan Gregg with his statement, “eBPF is a crazy technology, it’s like putting JavaScript into the Linux kernel.” and a picture of him shouting at hard drives. I took this picture from the following video where he demonstrates that his disk-read-latency measurement tool can detect if someone shouts at the hard drives:

Andrea Righi came up with the idea of writing a tiny scheduler that reacts to sound. The great thing about sched-ext and eBPF is that we can write experimental schedulers without much effort, especially if you use my hello-ebpf library and access the vast Java ecosystem.

For this scheduler, we scale the number of cores any task can use by the loudness level. So shouting at your computer makes your application run faster:

In this scenario, I ran the slow roads game while my system was exhausted by stress-ng, so more cores mean a less overcommitted system and, thereby, a more fluent gaming experience.

Continue reading →

Hello eBPF: Concurrency Testing using Custom Linux Schedulers (19)

Posted on by
1

Welcome back to my hello-ebpf series. I was a FOSDEM earlier this month and gave three(ish) talks. This blog post covers one of them, which is related to concurrency testing.

Backstory

The whole story started in the summer of last year when I thought about the potential use cases for custom Linux schedulers. The only two prominent use cases at the time were:

  1. Improve performance on servers and heterogenous systems
  2. Allow people to write their own schedulers for educational purposes

Of course, the latter is also important for the hello-ebpf project, which allows Java developers to create their own schedulers. But I had another idea back then: Why can’t we use custom Linux schedulers for concurrency testing? I presented this idea in my keynote at the eBPF summit in September, which you can watch here.

I ended up giving this talk because of a certain Bill Mulligan, who liked the idea of showing my eBPF SDK in Java and sched-ext in a single talk. For whatever reason, I foolishly agreed to give the talk and spent a whole weekend in Oslo before JavaZone frantically implementing sched-ext support in hello-ebpf.

My idea then was one could use a custom scheduler to test specific scheduling orders to find (and possibly) reproduce erroneous behavior of concurrent code. One of the examples in my talk was dead-locks:

Continue reading →

Sched-ext Scheduler Contest FOSDEM’25

Posted on by

Writing custom Linux schedulers is pretty easy using sched-ext. You can write your own tiny scheduler in a few lines of code using C, Rust, or even Java. We’re so confident in sched-ext that we’re starting a Scheduler Contest for FOSDEM’25. Think you can craft the ultimate scheduler? A scheduler that does something interesting, helpful, or fun? Join our sched-ext contest and show us what you’ve got, with the chance of winning hand-crafted sched-ext swag!

How to participate:
Submit your scheduler using sched-ext as a pull request to the repository fosdem25, ensuring:

  • It runs with a 6.12 Linux kernel.
  • It’s GPLv2-licensed
  • It compiles and is understandable. We’re programming language agnostic; just make sure to include a script so we can build and run it.

The implementations.md document provides details on the submission format.

You can also submit a unique scheduling policy idea to ideas.md.

Try to surprise us…

Deadlines & Announcement:

The submission deadline is Sunday, 2 February, at 10:00 AM (CET).

Winners will be announced during Andrea Righi’s talk on the Kernel track, “Level Up Your Linux Gaming: How Sched-ext Can Save Your FPS,” from 12:30 to 13:10.

Prizes:

  • Best Scheduler: A sched-ext shirt and a cup!
  • Best Idea: A sched-ext shirt!

Important:

To claim your prize, you must be present during Andrea’s talk. By participating, you agree to share your submission under GPL licensing and allow us to showcase it.

Legal Note:
This contest is for fun! We, Andrea and Johannes, select the winners based on our personal preferences and our own definition of “best”. All decisions are final, and we’re not liable beyond delivering the prizes to the winners. If you have any questions, feel free to create a GitHub issue.

There will be multiple talks at FOSDEM’25 on sched-ext:

Come by and get yourself a sched-ext sticker and valuable information on this fascinating topic.

In the meantime, you can learn about sched-ext from the resources listed in the wiki and the videos in the sched-ext playlist on YouTube.

Hello eBPF: Writing a Lottery Scheduler in Pure Java with bpf_for_each Support (18)

Posted on by

Welcome back to my hello-ebpf series. Last week, we learned about lottery schedulers and how to implement one with sched-ext and hello-ebpf. But there was one major issue when writing the scheduler in Java: hello-ebpf lacked support for the bpf_for_each macro. This is what this weeks short blog post is all about.

To loop over all elements in the scheduling queue, to pick a random one, we had to write inline C code. A simplified version without scheduling priorities looks like the following:

@Override
public void dispatch(int cpu, Ptr<TaskDefinitions.task_struct> prev) {
    String CODE = """
        // pick a random task struct index
        s32 random = bpf_get_prandom_u32() % 
            scx_bpf_dsq_nr_queued(SHARED_DSQ_ID);
        struct task_struct *p = NULL;
        bpf_for_each(scx_dsq, p, SHARED_DSQ_ID, 0) {
            // iterate till we get to this number
            random = random - 1;
            // and try to dispatch it
            if (random <= 0 && 
              tryDispatching(BPF_FOR_EACH_ITER, p, cpu)) {
                return 0;
            }
       };
       return 0;
       """;
}

This surely works, but inline C code is not the most readable and writing it is error-prone. I’ve written on this topic in my post Hello eBPF: Write your eBPF application in Pure Java (12) before. So what was keeping us from supporting the bpf_for_each macro? Support for lambdas as arguments to built-in bpf functions.

Lottery Scheduler in Pure Java

May I introduce you know the support for directly passed lambda expressions: This let’s us define a bpf_for_each_dsq function and to write a simple lottery scheduler in pure Java:

Continue reading →

Hello eBPF: Writing a Lottery Scheduler in Java with sched-ext (17)

Posted on by

Welcome back to my hello-ebpf series. Last week, I showed you how to create a custom Linux scheduler with a simple web API to control the task scheduling using sched-ext. Just so you know: A talk based on this blog post got accepted in the Testing and Continuous Delivery dev room at FOSDEM 2025. But this is not what I’m going to talk about in this blog post; instead, this article is on a topic that I covered almost exactly eleven years ago on this very blog: Lottery scheduling. We’ll extend the taskcontrol project covered in last week’s post with a new scheduler.

Foundations and Retrospektive

Eleven years ago, I was a student in Professor Frank Bellosa‘s operating systems course, learning about different schedulers. I wrote a blog post on a scheduler that I found particularly interesting: the lottery scheduler. I could write a new introduction to lottery scheduling, but I can also delegate this task to my old self:

I currently have a course covering operating systems at university. We learn in this course several scheduling algorithms. An operating system needs these kind of algorithms to determine which process to run at which time to allow these process to be executed “simultaniously” (from the users view).

Good scheduling algorithms have normally some very nice features:

  • They avoid the starving of one process (that one process can’t run at all and therefore makes no progress).
  • That all processes run the approriate percentage (defined by it’s priority) of time in each bigger time interval.

But they are maybe not only useful for operating systems but also for people like me. Probably they could help me to improve the scheduling of my learning.

An algorithm that seems to be suited best for this purpose is called lottery scheduling (pdf). It’s an algorithm that gives a each process some lottery tickets (their number resembles the priority of the process). Every time the algorithm has to choose a new process to run, it simply picks a lottery ticket randomly and returns the process that owns the ticket. A useful addition is (in scenarios with only a few tickets) to remove the tickets that are picked temporarily from the lottery pot and put them back again, when the pot is empty.

Real life practice in lottery scheduling

We can visualize this similarly to the FIFO scheduler from last week’s blog post:

Visualization of lottery scheduling with sched-ext

This is a truly random scheduler, making it great for testing applications with random scheduling orders. This is why it’s a great addition to the taskcontrol project.

Continue reading →

Hello eBPF: Control task scheduling with a custom scheduler written in Java (16)

Posted on by

Welcome back to my hello-ebpf series. A few weeks ago, I showed you how to write a minimal Linux scheduler in both Java and C using the new Linux scheduler extensions. Before we start: The Linux kernel with sched-ext support has finally been released, so the next Ubuntu version will probably have support for writing custom Linux schedulers out-of-the-box.

In this week’s blog posts, I’ll show you how to use a custom scheduler to control the execution of tasks and processes. The main idea is that I want to be able to tell a task (a single thread of a process) or a task group (usually a process) to stop for a few seconds. This can be helpful for testing the behaviour of programs in scheduling edge cases, when for example a consumer thread isn’t scheduled as it used to be normally.

Of course, we can achieve this without a custom Linux scheduler: We can just send the task a POSIX SIGSTOP signal and a SIGCONT signal when we want to continue it. But where is the fun in that? Also, some applications (I look at you, OpenJDK) don’t like it when you send signals to them, and they will behave strangely, as they can observe the signals.

Idea

I showed before that we can write schedulers ourselves, so why not use them for this case? The main idea is to only schedule processes in the scheduler that are allowed to be scheduled, according to a task (and task group) settings BPF map:

This is essentially our minimal scheduler, with one slight modification that I show you later.

And yes, the tasks are not stopped immediately, but with a maximum of 5ms scheduling intervals, we only have a small delay.

I implemented all this in the taskcontrol project, which you can find on GitHub. This is where you also find information on the required dependencies and how to install a 6.12 Kernel if you’re on Ubuntu 24.10.

Continue reading →

A Minimal Scheduler with eBPF, sched_ext and C

Posted on by

The following is a tutorial written for the sched_ext project, that gives you an introduction into writing a Linux scheduler directly in C. The tutorial is also present on the sched_ext scx wiki.

In the following a short tutorial for creating a minimal scheduler written with sched_ext in C. This scheduler uses a global scheduling queue from which every CPU gets its tasks to run for a time slice. The scheduler order is First-In-First-Out. So it essentially implements a round-robin scheduler:

Round Robin Diagram

This short tutorial covers the basics; to learn more, visit the resources from the scx wiki.

Continue reading →