On 10/20/26 23:30, Gabriele Monaco wrote:
On Tue, 2026-10-20 at 03:18 +0800, Wen Yang wrote:
there's been a conversation at the LPC conference about completely migrating
RV
to BPF.
While we aren't sure of its feasibility yet, and it will take time, it's
something to keep in mind for new contributions.
Understood, and I have been following that direction. I spent the last
few days reading the v2 "rv: Add support for BPF monitors" series in
detail, so the comparison below is against that code rather than the
general idea.
Great, I'm going to review your series when I get some time.
First of all, would dropping RV in-kernel support negatively affect your
current
workflow?
Yes, it would. Some concrete points, all from the current tree.
(a) Event ABI. event_da_monitor / error_env_* in
kernel/trace/rv/rv_trace.h are plain tracepoints, so trace-cmd,
perf and anything else reading tracefs consume them today with no
extra userspace. The BPF series delivers events through a ring
buffer read by the rv tool, so the two backends are not
interchangeable for existing consumers.
Good point, you might have noticed a similar discussion is happening also on the
RV-BPF thread. We cannot have the absolute same ABI in BPF, but we probably need
tracepoints or anything that can easily interact with other tracing tools.
(b) Build and deployment. On the images we target, BPF is often not
compiled in or not permitted:
CONFIG_BPF_SYSCALL default n
CONFIG_BPF_EVENTS needs BPF_SYSCALL
CONFIG_DEBUG_INFO_BTF needed for CO-RE; depends on
!DEBUG_INFO_REDUCED and pahole
LOCKDOWN_BPF_READ_KERNEL disables the probe_read helpers
Those are product constraints, not local build issues. tracefs/RV
works in all of these configurations.
There is also a size cost: BPF implies CONFIG_DEBUG_INFO_BTF, and
.BTF is resident in RO_DATA and kept in the boot image - about
5 MB uncompressed on a full kernel, 1.2-1.6 MB compressed, plus
roughly 25% per module with DEBUG_INFO_BTF_MODULES. On a
16-32 MB kernel partition that matters.
I've vaguely heard automotive folks saying they don't want to build BPF support
several times. Finally I get a proper technical reason!
Now I think this isn't easy to sort out, if that's a hard requirement, we are
probably going to need to keep in-kernel monitors around.
(c) Timing semantics. tlob arms a per-task hrtimer with
HRTIMER_MODE_REL_HARD. bpf_timer is fixed to
HRTIMER_MODE_REL_SOFT and the verifier states "bpf_timer callbacks
are never sleepable" . On PREEMPT_RT the callback runs in a
threaded
softirq, so the expiry time carries scheduler latency.
I haven't looked into this yet, but from what you say, the bpf_timers code
doesn't really support PREEMPT_RT.
For a budget monitor the expiry time is the output, so this is a
semantic change, not just overhead.
Well, the expiry time you get from an hrtimer is what you configured + some
system jitter. The jitter here is just going to be less predictable but is it
really the output you need in your monitor?
Or am I missing your point?
Hi Gabriele,
Thanks for the feedback.
For tlob, the primary goal is detecting deadline overruns on the order
of milliseconds or sub-milliseconds (e.g. 10 ms budget).
Under HRTIMER_MODE_REL_SOFT on PREEMPT_RT, the timer callback runs in a
threaded softirq (ksoftirqd or threaded irq context). If the system is
under CPU or scheduler pressure, the softirq latency itself can easily
add several milliseconds of unpredictable delay. When that happens, it
blurs attribution: users cannot distinguish whether the monitored task
exceeded its budget, or whether the task finished on time but the timer
callback was delayed in the scheduler queue. HARD hrtimer avoids this
by firing directly in hardirq context at the physical deadline.
What a BPF tlob would look like once those exist:
- model: still tlob.dot, with rvgen -b extended to the hybrid
backend;
- state: one record per task in BPF_MAP_TYPE_TASK_STORAGE (or
the per-object hash map the series has today), holding the
state, the binding id and the budget;
Yeah I wasn't aware of BPF_MAP_TYPE_TASK_STORAGE before, I have a draft V3
(coming some time after the merge window) using that.
But I'd assume this type of storage would simplify the design of your monitor,
which is currently using a PER_OBJ just to add more things to a PER_TASK.
I'm going to keep in mind your use-case where you essentially extend the storage
(which is perfectly fine in BPF since we don't use the task_struct slot).
Thanks.
On the in-kernel side, your DA_OBJ_EMBEDDED_TYPE PoC delivered
the exact same simplification.
In v8, we adopted DA_OBJ_EMBEDDED_TYPE so struct tlob_record is embedded
directly into struct da_monitor_storage:
- Deleted the private mempool and custom allocator entirely; tlob now
relies 100% on the framework pre-allocated pool.
- Deleted the parallel state machine (OPEN / CLOSED / RETIRED) and
tlob_binding_lock. The window lifecycle is now driven purely by the
automaton states (stopped, running, waiting, sleeping).
- Overall, tlob.c shrunk from ~1065 lines to ~660 lines (-38%), with no
locking on the scheduler hot path.
We are currently testing and will send out v8 soon.
--
Best wishes,
Wen
- window: two uprobe programs attached to PATH:START and
PATH:STOP, with the binding id and budget passed through the
attach cookie or a small config map populated before load;
- expiry: a bpf_timer in the same record, armed on state
transitions and cancelled on STOP, submitting an error entry
to the monitor's ringbuf;
- output: the rv tool reads the ringbuf and prints it, and the
reactor is linked in with bpf_linker as v2 already does.
The DA engine and the ringbuf/reactor plumbing would carry over
unchanged. The three missing pieces are the uprobe
attach/configuration path, the timer and env handling in
da_monitor_bpf.h, and the hybrid backend in rvgen.
None of this argues against migrating. It argues that the BPF
backend has to reach parity on (a)-(c) before the in-kernel path
can be removed.
Thanks for taking time to assess this, it looks like the right direction.
Gabriele
