On 05/13/2015 05:22 AM, Masami Hiramatsu wrote:
> On 2015/05/12 21:48, William Cohen wrote:
>> Hi Dave,
>>
>> In some of the previous diagnostic output it looked like things would go
>> wrong
>> in the entry.S when the D bit was cleared and the debug interrupts were
>> unmasksed. I wonder if some of the issue might be due to the starting the
>> kprobe for the trampoline, but leaving things in an odd state when another
>> set of krpobe/kretporbes are hit when the trampoline is running.
>
> Hmm, does this mean we have a trouble if a user kprobe handler calls the
> function which is probed by other kprobe? Or, is this just a problem
> only for kretprobes?
Hi Masami,
I wrote an example based off of sample/kprobes/kprobes_sample.c to force the
reentry issue for kprobes (the attached kprobe_rentry_example.c). That seemed
to run fine. I think the reason that the trampoline handler got into trouble
is because of the reset_current_kprobe() before the possible call to kfree, but
I haven't verified it. It seems like that should be at the end of trampoline
handler just before the return. Other architectures have similar trampoline
handlers, so I am surprised that the other architectures haven't encountered
this issue with kretprobes. Maybe this is due to specific of arm64 exception
handling.
# modprobe kprobe_reentry_example
[ 909.617295] Planted kprobe at fffffe00000b7b34
[ 909.623873] Planted kprobe at fffffe000032d34c
# rmmod kprobe_reentry_example
[ 1482.647504] kprobe at fffffe00000b7b34 unregistered
[ 1482.687506] kprobe at fffffe000032d34c unregistered
[ 1482.692361] y = 42
[ 1482.694361] z = 0
# grep \ int_sqrt$ /proc/kallsyms
fffffe000032d34c T int_sqrt
# grep \ do_fork$ /proc/kallsyms
fffffe00000b7b34 T do_fork
>
>> As Dave
>> mentioned the proposed trampoline patch avoids using a kprobe in the
>> trampoline and directly calls the trampoline handler. Attached is the
>> current version of the patch which was able to run the systemtap testsuite.
>> Systemtap does exercise the kprobe/kretprobe infrastructure, but it would
>> be good to have additional raw kprobe tests to check that kprobe reentry
>> works as expected.
>
> Actually, Will's patch looks like the same thing what I did on x86,
> as the kretprobe-booster. So I'm OK for that. But if the above problem
> is not solved, we need to fix that, since kprobes can be used from
> different sources.
The patch should look similar to the x86 code. The x86 code was used as a model.
-Will
/*
* NOTE: This example is designed to check that the kprobe reentry work.
*
* For more information on theory of operation of kprobes, see
* Documentation/kprobes.txt
*
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/kprobes.h>
/* For each probe you need to allocate a kprobe structure */
static struct kprobe kp = {
.symbol_name = "do_fork",
};
static struct kprobe kp_re = {
.symbol_name = "int_sqrt",
};
static unsigned long y=0;
static unsigned long z=0;
/* kprobe pre_handler: called just before the probed instruction is executed */
static int handler_pre(struct kprobe *p, struct pt_regs *regs)
{
/* call another function that is instrumented with a kprobe to
ensure that reentry works */
unsigned long x=1764;
y = int_sqrt(x);
return 0;
}
/* kprobe post_handler: called after the probed instruction is executed */
static void handler_post(struct kprobe *p, struct pt_regs *regs,
unsigned long flags)
{
return;
}
/* kprobe pre_handler: called just before the probed instruction is executed */
static int handler_pre_re(struct kprobe *p, struct pt_regs *regs)
{
/* if reentry is working as expected this code may not be executed */
z = 0xdeadbeef;
return 0;
}
/* kprobe post_handler: called after the probed instruction is executed */
static void handler_post_re(struct kprobe *p, struct pt_regs *regs,
unsigned long flags)
{
return;
}
/*
* fault_handler: this is called if an exception is generated for any
* instruction within the pre- or post-handler, or when Kprobes
* single-steps the probed instruction.
*/
static int handler_fault(struct kprobe *p, struct pt_regs *regs, int trapnr)
{
printk(KERN_INFO "fault_handler: p->addr = 0x%p, trap #%dn",
p->addr, trapnr);
/* Return 0 because we don't handle the fault. */
return 0;
}
static int __init kprobe_init(void)
{
int ret;
kp.pre_handler = handler_pre;
kp.post_handler = handler_post;
kp.fault_handler = handler_fault;
ret = register_kprobe(&kp);
if (ret < 0) {
printk(KERN_INFO "register_kprobe failed, returned %d\n", ret);
return ret;
}
printk(KERN_INFO "Planted kprobe at %p\n", kp.addr);
kp_re.pre_handler = handler_pre_re;
kp_re.post_handler = handler_post_re;
kp_re.fault_handler = handler_fault;
ret = register_kprobe(&kp_re);
if (ret < 0) {
printk(KERN_INFO "register_kprobe failed, returned %d\n", ret);
return ret;
}
printk(KERN_INFO "Planted kprobe at %p\n", kp_re.addr);
return 0;
}
static void __exit kprobe_exit(void)
{
unregister_kprobe(&kp);
printk(KERN_INFO "kprobe at %p unregistered\n", kp.addr);
unregister_kprobe(&kp_re);
printk(KERN_INFO "kprobe at %p unregistered\n", kp_re.addr);
printk(KERN_INFO "y = %ld\n", y);
printk(KERN_INFO "z = %lx\n", z);
}
module_init(kprobe_init)
module_exit(kprobe_exit)
MODULE_LICENSE("GPL");
