Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2014-02-10 Thread Masami Hiramatsu
(2014/02/10 13:02), Masami Hiramatsu wrote: > (2014/02/09 23:37), Ingo Molnar wrote: >> >> * Masami Hiramatsu wrote: >> >>> I guess the second reason is why the stap takes so long time to set >>> probes. stap tries to register kprobes without disabled flag, that >>> means we enables thousands

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2014-02-10 Thread Masami Hiramatsu
(2014/02/10 13:02), Masami Hiramatsu wrote: (2014/02/09 23:37), Ingo Molnar wrote: * Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: I guess the second reason is why the stap takes so long time to set probes. stap tries to register kprobes without disabled flag, that means we

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2014-02-09 Thread Masami Hiramatsu
(2014/02/09 23:37), Ingo Molnar wrote: > > * Masami Hiramatsu wrote: > >> I guess the second reason is why the stap takes so long time to set >> probes. stap tries to register kprobes without disabled flag, that >> means we enables thousands of probes (overheads). >> >> So the similar thing

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2014-02-09 Thread Frank Ch. Eigler
Hi - > > So the similar thing happens when we enables events as below; > > > > # for i in /sys/kernel/debug/tracing/events/kprobes/* ; do date; echo 1 > > > $i; done > > Wed Jan 29 10:44:50 UTC 2014 > > ... > > > > I tried it and canceled after 4 min passed. It enabled about 17k > >

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2014-02-09 Thread Ingo Molnar
* Masami Hiramatsu wrote: > I guess the second reason is why the stap takes so long time to set > probes. stap tries to register kprobes without disabled flag, that > means we enables thousands of probes (overheads). > > So the similar thing happens when we enables events as below; > > #

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2014-02-09 Thread Ingo Molnar
* Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: I guess the second reason is why the stap takes so long time to set probes. stap tries to register kprobes without disabled flag, that means we enables thousands of probes (overheads). So the similar thing happens when we enables

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2014-02-09 Thread Frank Ch. Eigler
Hi - So the similar thing happens when we enables events as below; # for i in /sys/kernel/debug/tracing/events/kprobes/* ; do date; echo 1 $i; done Wed Jan 29 10:44:50 UTC 2014 ... I tried it and canceled after 4 min passed. It enabled about 17k events and slowed down

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2014-02-09 Thread Masami Hiramatsu
(2014/02/09 23:37), Ingo Molnar wrote: * Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: I guess the second reason is why the stap takes so long time to set probes. stap tries to register kprobes without disabled flag, that means we enables thousands of probes (overheads). So

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2014-01-29 Thread Masami Hiramatsu
Hi Ingo, (2013/12/20 19:46), Ingo Molnar wrote: > > * Masami Hiramatsu wrote: > >> (2013/12/20 17:20), Ingo Molnar wrote: >>> >>> * Masami Hiramatsu wrote: >>> > But a closer look indicates that the insertion of kprobes is > taking about three (!!) orders of magnitude longer than

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2014-01-29 Thread Masami Hiramatsu
Hi Ingo, (2013/12/20 19:46), Ingo Molnar wrote: * Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: (2013/12/20 17:20), Ingo Molnar wrote: * Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: But a closer look indicates that the insertion of kprobes is taking about three

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-23 Thread Masami Hiramatsu
(2013/12/23 22:04), Ingo Molnar wrote: > > * Masami Hiramatsu wrote: > >>> I can reproduce one weirdness, with just 13 probes added, 'perf probe >>> -l' [which should really be 'perf probe list'!] executes very slowly: >>> >>> # perf stat --null --repeat 3 perf probe -l >>> >>> Performance

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-23 Thread Ingo Molnar
* Masami Hiramatsu wrote: > > I can reproduce one weirdness, with just 13 probes added, 'perf probe > > -l' [which should really be 'perf probe list'!] executes very slowly: > > > > # perf stat --null --repeat 3 perf probe -l > > > > Performance counter stats for 'perf probe -l' (3 runs):

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-23 Thread Ingo Molnar
* Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: I can reproduce one weirdness, with just 13 probes added, 'perf probe -l' [which should really be 'perf probe list'!] executes very slowly: # perf stat --null --repeat 3 perf probe -l Performance counter stats for 'perf

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-23 Thread Masami Hiramatsu
(2013/12/23 22:04), Ingo Molnar wrote: * Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: I can reproduce one weirdness, with just 13 probes added, 'perf probe -l' [which should really be 'perf probe list'!] executes very slowly: # perf stat --null --repeat 3 perf probe -l

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-22 Thread Masami Hiramatsu
(2013/12/20 22:40), Frank Ch. Eigler wrote: > Hi - > > mingo wrote: >> [...] >> For example a hash table (hashed by probe address) could be used in >> addition to the list, to speed up basic operations. > > In the past, when this sort of behavior popped up, it was due to > machine-wide

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-22 Thread Masami Hiramatsu
(2013/12/20 19:46), Ingo Molnar wrote: > > * Masami Hiramatsu wrote: > >> (2013/12/20 17:20), Ingo Molnar wrote: >>> >>> * Masami Hiramatsu wrote: >>> > But a closer look indicates that the insertion of kprobes is > taking about three (!!) orders of magnitude longer than before, as

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-22 Thread Masami Hiramatsu
(2013/12/20 19:46), Ingo Molnar wrote: * Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: (2013/12/20 17:20), Ingo Molnar wrote: * Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: But a closer look indicates that the insertion of kprobes is taking about three (!!) orders

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-22 Thread Masami Hiramatsu
(2013/12/20 22:40), Frank Ch. Eigler wrote: Hi - mingo wrote: [...] For example a hash table (hashed by probe address) could be used in addition to the list, to speed up basic operations. In the past, when this sort of behavior popped up, it was due to machine-wide halt/sync operations

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-20 Thread Frank Ch. Eigler
Hi - mingo wrote: > [...] > For example a hash table (hashed by probe address) could be used in > addition to the list, to speed up basic operations. In the past, when this sort of behavior popped up, it was due to machine-wide halt/sync operations being done too eagerly. At one point, the

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-20 Thread Ingo Molnar
* Masami Hiramatsu wrote: > (2013/12/20 17:20), Ingo Molnar wrote: > > > > * Masami Hiramatsu wrote: > > > >>> But a closer look indicates that the insertion of kprobes is > >>> taking about three (!!) orders of magnitude longer than before, as > >>> judged by the rate of increase of 'wc

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-20 Thread Masami Hiramatsu
(2013/12/20 17:20), Ingo Molnar wrote: > > * Masami Hiramatsu wrote: > >>> But a closer look indicates that the insertion of kprobes is >>> taking about three (!!) orders of magnitude longer than before, as >>> judged by the rate of increase of 'wc -l >>> /sys/kernel/debug/kprobes/list'. >>

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-20 Thread Ingo Molnar
* Masami Hiramatsu wrote: > > But a closer look indicates that the insertion of kprobes is > > taking about three (!!) orders of magnitude longer than before, as > > judged by the rate of increase of 'wc -l > > /sys/kernel/debug/kprobes/list'. > > Right, because kprobes are not designed

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-20 Thread Ingo Molnar
* Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: But a closer look indicates that the insertion of kprobes is taking about three (!!) orders of magnitude longer than before, as judged by the rate of increase of 'wc -l /sys/kernel/debug/kprobes/list'. Right, because

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-20 Thread Masami Hiramatsu
(2013/12/20 17:20), Ingo Molnar wrote: * Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: But a closer look indicates that the insertion of kprobes is taking about three (!!) orders of magnitude longer than before, as judged by the rate of increase of 'wc -l

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-20 Thread Ingo Molnar
* Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: (2013/12/20 17:20), Ingo Molnar wrote: * Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: But a closer look indicates that the insertion of kprobes is taking about three (!!) orders of magnitude longer than before,

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-20 Thread Frank Ch. Eigler
Hi - mingo wrote: [...] For example a hash table (hashed by probe address) could be used in addition to the list, to speed up basic operations. In the past, when this sort of behavior popped up, it was due to machine-wide halt/sync operations being done too eagerly. At one point, the

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-19 Thread Masami Hiramatsu
(2013/12/20 5:46), Frank Ch. Eigler wrote: > > Hi, Masami - > > > masami.hiramatsu.pt wrote: > >> Here is the version 6 of NOKPROBE_SYMBOL series. :) >> [...] > > Some preliminary results from building these on top of tip/master on > x86-64. > > # stap -te "probe kprobe.function("*") {}" >

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-19 Thread Frank Ch. Eigler
Hi, Masami - masami.hiramatsu.pt wrote: > Here is the version 6 of NOKPROBE_SYMBOL series. :) > [...] Some preliminary results from building these on top of tip/master on x86-64. # stap -te "probe kprobe.function("*") {}" starts up OK, without crashes, which looks like great progress.

[PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-19 Thread Masami Hiramatsu
Hi, Here is the version 6 of NOKPROBE_SYMBOL series. :) This includes small updates and introducing nokprobe_inline macro to prevent probing on the static/inlined small functions since NOKPROBE_SYMBOL will inhibit inlining by referring function address. This macro is more self-describing than

[PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-19 Thread Masami Hiramatsu
Hi, Here is the version 6 of NOKPROBE_SYMBOL series. :) This includes small updates and introducing nokprobe_inline macro to prevent probing on the static/inlined small functions since NOKPROBE_SYMBOL will inhibit inlining by referring function address. This macro is more self-describing than

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-19 Thread Frank Ch. Eigler
Hi, Masami - masami.hiramatsu.pt wrote: Here is the version 6 of NOKPROBE_SYMBOL series. :) [...] Some preliminary results from building these on top of tip/master on x86-64. # stap -te probe kprobe.function(*) {} starts up OK, without crashes, which looks like great progress. But a

Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup and fixes crash bugs

2013-12-19 Thread Masami Hiramatsu
(2013/12/20 5:46), Frank Ch. Eigler wrote: Hi, Masami - masami.hiramatsu.pt wrote: Here is the version 6 of NOKPROBE_SYMBOL series. :) [...] Some preliminary results from building these on top of tip/master on x86-64. # stap -te probe kprobe.function(*) {} starts up OK,