Hi Jose,
On 15/01/26 21:31, Jose E. Marchesi wrote:
Hi Piyush.
This patch adds the bpf-vmtest-tool subdirectory under contrib which tests
BPF programs under a live kernel using a QEMU VM. It can build the
specified kernel version with eBPF support enabled
and stores it under $VMTEST_DIR
It can also compile BPF C source files or BPF bytecode objects and
test them against the kernel verifier for errors. When a BPF program
is rejected by the kernel verifier, the verifier logs are displayed.
The BPF bytecode objects don't get compiled I suppose.
True, the original wording is confusing. It should say that the tool can
compile BPF C source files and verify BPF bytecode objects against the
kernel verifier.
$ python3 main.py -k 6.15 --bpf-src assets/ebpf-programs/fail.c
BPF program failed to load
Verifier logs:
btf_vmlinux is malformed
0: R1=ctx() R10=fp0
0: (81) r0 = *(s32 *)(r10 +4)
invalid read from stack R10 off=4 size=4
processed 1 insns (limit 1000000) max_states_per_insn 0 total_states 0
peak_states 0 mark_read 0
See the README for more examples.
The script uses vmtest (https://github.com/danobi/vmtest) to boot
the VM and run the program. By default, it uses the host's root
("/") as the VM rootfs via the 9p filesystem, so only the kernel is
replaced during testing.
Tested with Python 3.9 and above.
contrib/ChangeLog:
* bpf-vmtest-tool/README: New file.
* bpf-vmtest-tool/bpf.py: New file.
* bpf-vmtest-tool/config.py: New file.
* bpf-vmtest-tool/kernel.py: New file.
* bpf-vmtest-tool/main.py: New file.
* bpf-vmtest-tool/pyproject.toml: New file.
* bpf-vmtest-tool/tests/test_cli.py: New file.
* bpf-vmtest-tool/utils.py: New file.
* bpf-vmtest-tool/vm.py: New file.
Signed-off-by: Piyush Raj <[email protected]>
---
contrib/bpf-vmtest-tool/README | 157 ++++++++++++
contrib/bpf-vmtest-tool/bpf.py | 221 +++++++++++++++++
contrib/bpf-vmtest-tool/config.py | 50 ++++
contrib/bpf-vmtest-tool/kernel.py | 290 ++++++++++++++++++++++
contrib/bpf-vmtest-tool/main.py | 285 +++++++++++++++++++++
contrib/bpf-vmtest-tool/pyproject.toml | 36 +++
contrib/bpf-vmtest-tool/tests/test_cli.py | 219 ++++++++++++++++
contrib/bpf-vmtest-tool/utils.py | 31 +++
contrib/bpf-vmtest-tool/vm.py | 169 +++++++++++++
9 files changed, 1458 insertions(+)
create mode 100644 contrib/bpf-vmtest-tool/README
create mode 100644 contrib/bpf-vmtest-tool/bpf.py
create mode 100644 contrib/bpf-vmtest-tool/config.py
create mode 100644 contrib/bpf-vmtest-tool/kernel.py
create mode 100644 contrib/bpf-vmtest-tool/main.py
create mode 100644 contrib/bpf-vmtest-tool/pyproject.toml
create mode 100644 contrib/bpf-vmtest-tool/tests/test_cli.py
create mode 100644 contrib/bpf-vmtest-tool/utils.py
create mode 100644 contrib/bpf-vmtest-tool/vm.py
diff --git a/contrib/bpf-vmtest-tool/README b/contrib/bpf-vmtest-tool/README
new file mode 100644
index 00000000000..552b2a3e1c8
--- /dev/null
+++ b/contrib/bpf-vmtest-tool/README
@@ -0,0 +1,157 @@
+BPF vmtest Tool
+===============
+https://gcc.gnu.org/wiki/BPFRunTimeTests
+
+This directory contains a Python script to run BPF programs or shell commands
+under a live Linux kernel using QEMU virtual machines.
+
+USAGE
+=====
+
+Initial Setup
+-------------
+
+Before using the tool, you must set the directory where vmtest will look for
+kernels and store kernel artifacts. You can do this in two ways:
+
+1. Set the VMTEST_DIR environment variable
+2. Use the --vmtest-dir flag with each command
+
+Note: This is required to use the tool.
+
+Available Options
+-----------------
+
+View all supported commands using the --help option:
+
+ usage: main.py [-h] [-v DEBUG|INFO|WARNING|ERROR] [--vmtest-dir DIR]
{bpf,vmtest,kernel} ...
+
+ BPF vmtest tool
+
+ positional arguments:
+ {bpf,vmtest,kernel} Available commands
+ bpf BPF program management
+ vmtest Run VM tests
+ kernel Kernel management
+
+ options:
+ -h, --help show this help message and exit
+ -v DEBUG|INFO|WARNING|ERROR, --log-level DEBUG|INFO|WARNING|ERROR
+ Log level
+ --vmtest-dir DIR Directory for vmtest artifacts (or set VMTEST_DIR
env variable)
+
+ Examples:
+ # Compile BPF source to bytecode
+ main.py bpf compile my_prog.bpf.c -o my_prog.bpf.o
+
+ # Run BPF program in VM
+ main.py vmtest --kernel 6.15-x86_64 --bpf-src my_prog.bpf.c
+
+ # List available kernels
+ main.py kernel list
+
+
+COMMANDS
+========
+
+kernel subcommand
+-----------------
+
+You must build a kernel before using it.
+
+Build a kernel:
+
+ python main.py --vmtest-dir="/home/user/.bpf-vmtest-tool" kernel build 6.16
+
+The tool will download and build the specified kernel version from
+https://www.kernel.org/pub/linux/kernel and store the build artifacts in
+$VMTEST_DIR/kernels/linux-6.15-x86_64. Specifically, it stores bpftool,
+bzImage-6.15-x86_64, and vmlinux.h, which are used when compiling BPF programs
+instead of relying on the host system.
+
+List available kernels:
+
+ python main.py --vmtest-dir="/home/user/.bpf-vmtest-tool" kernel list
+
+Remove kernels:
+
+ python main.py --vmtest-dir="/home/user/.bpf-vmtest-tool" kernel remove
6.15-x86_64
+
+Note: If the architecture is omitted, the host architecture is assumed. For
+example, "6.15" will be treated as "6.15-x86_64" on an x86_64 system.
+
+vmtest subcommand
+-----------------
+
+Run a shell command inside a live kernel VM:
+
+ python main.py vmtest -k 6.15 -r / -c "uname -a"
I see that -r/-rootfs is not documented, nor the other options which are
specific to subcommands. Can we have a little description of each?
Sure, I will add descriptions for the subcommand-specific options and
explain how they are used internally, so the usage is clearer.
+
+Run a BPF source file in the VM:
+
+ python main.py vmtest -k 6.15 --bpf-src fail.c
+
+Run a precompiled BPF object file:
+
+ python main.py vmtest -k 6.15 --bpf-obj fail.bpf.o
+
+bpf subcommand
+--------------
+
+You can compile BPF source to bytecode using the kernel-specific bpftool and
+vmlinux.h stored in $VMTEST_DIR:
+
+ python main.py --vmtest-dir="/home/user/.bpf-vmtest-tool" bpf compile
invalid-memory-access.c -k 6.15 -o /tmp/invalid-memory-access.bpf.o
+
+
+LIMITATIONS
+===========
+
+- Only x86_64 architecture is currently supported
+- Only "/" (the root filesystem) is currently supported as the VM rootfs when
+ running or testing BPF programs using --bpf-src or --bpf-obj
+
+
+DEPENDENCIES
+============
+
+- Python >= 3.9
+- vmtest >= v0.18.0 (https://github.com/danobi/vmtest)
+ - QEMU
+ - qemu-guest-agent
+
+For compiling kernels:
+- pahole
+- https://docs.kernel.org/process/changes.html#current-minimal-requirements
+
+For compiling and loading BPF programs:
+- libbpf
+- gcc-bpf-unknown-none
(https://gcc.gnu.org/wiki/BPFBackEnd#Where_to_find_GCC_BPF)
So libbpf is still a host dependency. Would it be possible to use the
libbpf of the built kernel instead? Much like bpftool.
Yes, the host’s libbpf is currently used. If we want to use the libbpf
from the built kernel, I can try implementing that in the next revision
(via a static libbpf build) and will update if there are any limitations.
+
+
+BUILD FLAGS
+===========
+
+You can customize compiler settings using environment variables:
+
+- BPF_CC: Compiler for the BPF program (default: bpf-unknown-none-gcc)
+- BPF_CFLAGS: Extra flags for BPF program compilation (default: "-O2")
+- BPF_INCLUDES: Include paths for BPF (default: "-I/usr/local/include
-I/usr/include")
+- VMTEST_CC: Compiler for the user-space loader (default: gcc)
+- VMTEST_CFLAGS: Flags for compiling the loader (default: "-g -Wall")
+- VMTEST_LDFLAGS: Linker flags for the loader (default: "-lelf -lz -lbpf")
+
+Example usage:
+
+ BPF_CFLAGS="-O3 -g" BPF_CC="/bpf-gcc-build/gcc/xgcc" python main.py vmtest
-k 6.15 --bpf-src fail.c
+
+
+DEVELOPMENT
+===========
+
+Development dependencies are specified in pyproject.toml, which can be used
+with any suitable Python virtual environment manager.
+
+To run the test suite:
+
+ python3 -m pytest
\ No newline at end of file
Please add a newline character here.
Sure, I’ll add the newline here and in other places as well.
Thanks
