On Mon, Jul 1, 2024 at 6:01 PM Masami Hiramatsu wrote:
>
> On Mon, 1 Jul 2024 15:15:56 -0700
> Andrii Nakryiko wrote:
>
> > On Mon, Jul 1, 2024 at 10:55 AM Andrii Nakryiko
> > wrote:
> > >
> > > On Sat, Jun 29, 2024 at 4:30 PM Masami Hiramatsu
> > > wrote:
> > > >
> > > > On Fri, 28 Jun 2024
On Mon, 1 Jul 2024 15:15:56 -0700
Andrii Nakryiko wrote:
> On Mon, Jul 1, 2024 at 10:55 AM Andrii Nakryiko
> wrote:
> >
> > On Sat, Jun 29, 2024 at 4:30 PM Masami Hiramatsu
> > wrote:
> > >
> > > On Fri, 28 Jun 2024 09:34:26 -0700
> > > Andrii Nakryiko wrote:
> > >
> > > > On Thu, Jun 27,
On Wed, 12 Jun 2024 09:11:56 +0800
Hongbo Li wrote:
> @@ -934,6 +943,12 @@ static int hugetlbfs_setattr(struct mnt_idmap *idmap,
> if (error)
> return error;
>
> + trace_hugetlbfs_setattr(inode, dentry->d_name.len, dentry->d_name.name,
> +
When tracing user functions with uprobe functionality, it's common to
install the probe (e.g., a BPF program) at the first instruction of the
function. This is often going to be `push %rbp` instruction in function
preamble, which means that within that function frame pointer hasn't
been
With all the batch uprobe APIs work we are now finally ready to reap the
benefits. Switch uprobes_treelock from reader-writer spinlock to a much
more efficient and scalable per-CPU RW semaphore.
Benchmarks and numbers time. I've used BPF selftests' bench tool,
trig-uprobe-nop benchmark
Switch internals of BPF multi-uprobes to batched version of uprobe
registration and unregistration APIs.
This also simplifies BPF clean up code a bit thanks to all-or-nothing
guarantee of uprobes_register_batch().
Signed-off-by: Andrii Nakryiko
---
kernel/trace/bpf_trace.c | 23
Similarly to what we did for uprobes_register_batch(), split
uprobe_unregister_batch() into two separate phases with different
locking needs.
First, all the VMA unregistration is performed while holding
a per-uprobe register_rwsem.
Then, we take a batched uprobes_treelock once to __put_uprobe()
Now that we have a good separate of each registration step, take
uprobes_treelock just once for relevant registration step, and then
process all relevant uprobes in one go.
Even if writer lock introduces a relatively large delay (as might happen
with per-CPU RW semaphore), this will keep overall
Now we are ready to split alloc-and-insert coupled step into two
separate phases.
First, we allocate and prepare all potentially-to-be-inserted uprobe
instances, assuming corresponding uprobes are not yet in uprobes_tree.
This is needed so that we don't do memory allocations under
To allow unbundling alloc-uprobe-and-insert step which is currently
tightly coupled, inline alloc_uprobe() logic into
uprobe_register_batch() loop. It's called from one place, so we don't
really lose much in terms of maintainability.
No functional changes.
Signed-off-by: Andrii Nakryiko
---
Introduce batch versions of uprobe registration (attachment) and
unregistration (detachment) APIs.
Unregistration is presumed to never fail, so that's easy.
Batch registration can fail, and so the semantics of
uprobe_register_batch() is such that either all uprobe_consumers are
successfully
Simplify uprobe registration/unregistration interfaces by making offset
and ref_ctr_offset part of uprobe_consumer "interface". In practice, all
existing users already store these fields somewhere in uprobe_consumer's
containing structure, so this doesn't pose any problem. We just move
some fields
Return -ENOMEM instead of NULL, which makes caller's error handling just
a touch simpler.
Signed-off-by: Andrii Nakryiko
---
kernel/events/uprobes.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index
Revamp how struct uprobe is refcounted, and thus how its lifetime is
managed.
Right now, there are a few possible "owners" of uprobe refcount:
- uprobes_tree RB tree assumes one refcount when uprobe is registered
and added to the lookup tree;
- while uprobe is triggered and kernel is
It seems like uprobe_write_opcode() doesn't require writer locked
mmap_sem, any lock (reader or writer) should be sufficient. This was
established in a discussion in [0] and looking through existing code
seems to confirm that there is no need for write-locked mmap_sem.
Fix the comment to state
There is no task_struct passed into get_user_pages_remote() anymore,
drop the parts of comment mentioning NULL tsk, it's just confusing at
this point.
Signed-off-by: Andrii Nakryiko
---
kernel/events/uprobes.c | 6 ++
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git
This patch set, ultimately, switches global uprobes_treelock from RW spinlock
to per-CPU RW semaphore, which has better performance and scales better under
contention and multiple parallel threads triggering lots of uprobes.
To make this work well with attaching multiple uprobes (through BPF
On Mon, Jul 1, 2024 at 10:55 AM Andrii Nakryiko
wrote:
>
> On Sat, Jun 29, 2024 at 4:30 PM Masami Hiramatsu wrote:
> >
> > On Fri, 28 Jun 2024 09:34:26 -0700
> > Andrii Nakryiko wrote:
> >
> > > On Thu, Jun 27, 2024 at 11:28 PM Masami Hiramatsu
> > > wrote:
> > > >
> > > > On Thu, 27 Jun 2024
On Thu, Jun 27, 2024 at 9:43 AM Andrii Nakryiko
wrote:
>
> On Wed, Jun 26, 2024 at 7:30 PM Masami Hiramatsu wrote:
> >
> > On Mon, 24 Jun 2024 17:21:36 -0700
> > Andrii Nakryiko wrote:
> >
> > > Anyways, under exclusive writer lock, we double-check that refcount
> > > didn't change and is still
On Sat, Jun 29, 2024 at 4:30 PM Masami Hiramatsu wrote:
>
> On Fri, 28 Jun 2024 09:34:26 -0700
> Andrii Nakryiko wrote:
>
> > On Thu, Jun 27, 2024 at 11:28 PM Masami Hiramatsu
> > wrote:
> > >
> > > On Thu, 27 Jun 2024 09:47:10 -0700
> > > Andrii Nakryiko wrote:
> > >
> > > > On Thu, Jun 27,
20 matches
Mail list logo
