Building the Raspberry Pi Kernel for eBPF (Beyla)

So I wanted to run Beyla on my Raspberry Pi 5 which is part of my home lab. But the stock kernel that ships doesn't really have all the eBPF features required for Beyla, and it fails with the following error:

time=2024-11-15T16:03:32.999Z level=ERROR msg="couldn't trace process. Stopping process tracer" component=ebpf.ProcessTracer path=/usr/local/bin/cephcsi pid=2675 error="loading and assigning BPF objects: field UprobeServeHTTP: program uprobe_ServeHTTP: apply CO-RE relocations: load kernel spec: no BTF found for kernel version 6.6.51+rpt-rpi-2712: not supported"

This guide walks you through all the steps required to make Beyla work on the Pi.

Requirements

We need the kernel to have the following capabilities when being compiled:

CONFIG_ARCH_SUPPORTS_UPROBES=y
CONFIG_UPROBES=y
CONFIG_ARCH_HAS_SUBPAGE_FAULTS=y
CONFIG_KPROBES=y
CONFIG_JUMP_LABEL=y
# CONFIG_STATIC_KEYS_SELFTEST is not set
CONFIG_UPROBES=y
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
CONFIG_KRETPROBES=y
CONFIG_HAVE_IOREMAP_PROT=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_ARCH_CORRECT_STACKTRACE_ON_KRETPROBE=y
CONFIG_DEBUG_INFO_BTF=y

Building the kernel

We need to follow this excellent guide: https://www.raspberrypi.com/documentation/computers/linux_kernel.html

git clone --depth=1 https://github.com/raspberrypi/linux
cd linux
export KERNEL=kernel_2712
make bcm2712_defconfig

This will generate the default .config file which we need to extend.

I like to use menuconfig to then configure it.

sudo apt install libncurses5-dev
make menuconfig

Step 1: Add a custom version to the kernel

In General setup > Local version - append to kernel release, set the local version name to: -v8-beyla-compatible. This will help us separate the kernel from the stock kernel later.

Step 2: Enable BTF

In Kernel Hacking > Compile-time checks and compiler options > Debug information (Disable debug information), select the following:

It will then enable the BTF option in Kernel Hacking > Compile-time checks and compiler options. Enable that:

Exit menuconfig to save the current state. For some reason, it blanks out in the next step if I don't exit it here.

Step 3: Enable Uprobes

Run make menuconfig again to enable Uprobes.

In Kernel hacking > Tracers, enable the Enable uprobes-based dynamic events option:

Step 4: Compile the kernel

Now you're all set to compile the kernel.

make -j6 Image.gz modules dtbs

Step 5: Install the kernel

sudo make -j6 modules_install

sudo cp /boot/firmware/$KERNEL.img /boot/firmware/$KERNEL-backup.img
sudo cp arch/arm64/boot/Image.gz /boot/firmware/$KERNEL.img
sudo cp arch/arm64/boot/dts/broadcom/*.dtb /boot/firmware/
sudo cp arch/arm64/boot/dts/overlays/*.dtb* /boot/firmware/overlays/
sudo cp arch/arm64/boot/dts/overlays/README /boot/firmware/overlays/

Finally, reboot by running sudo reboot. You should now have a kernel that can run Beyla.