Re: Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
On Mon, Dec 01, 2014 at 12:28:08PM -0500, Jérémie Galarneau wrote: > On Fri, Nov 28, 2014 at 6:26 AM, Jiri Olsa wrote: > > On Wed, Nov 26, 2014 at 06:37:21PM +0100, Sebastian Andrzej Siewior wrote: > >> * Alexandre Montplaisir | 2014-11-12 17:14:45 [-0500]: > > > > SNIP > > > >> > >> >This was based on the most recent file format I was aware of, we will > >> >update it accordingly if required. > >> > > >> >Testing welcome! > >> > >> I pushed the perf changes I mentioned to > >> > >> git://git.breakpoint.cc/bigeasy/linux.git ctf_convert_7 > >> > > > > from perf data convert point of view (and libbabeltrace).. ;-) > > > > I tried to convert big perf.data (2GB) and my laptop got stuck > > for few minutes.. the reason is that perf allocated too much memory > > > > I needed to add occasional bt_ctf_stream_flush into the > > processing (patch attached) > > > > What I do now is checking if we processed given amount of events > > for the stream and once the value is crossed I flush it. > > > > My question is if there's a way to find out the allocated memory > > for the stream? It'd be nicer to setup maximum allocation size > > rather than the number of events. > > > > Not currently, although I agree that this would be a nice feature. > Perhaps by setting a maximal packet size, CTF Writer could handle the > automatic flushing. > > Thoughts? hum, it looks like 2 separated things to me.. the packet size is result format related size.. I'm more interested in the size of the 'struct ctf_stream' within the writer during the processing it seems it wont be trivial to track this AFAICS from sources, but I guess I can live so far with tracking the the number of events, until we find something else thanks, jirka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
On Fri, Nov 28, 2014 at 6:26 AM, Jiri Olsa wrote:
> On Wed, Nov 26, 2014 at 06:37:21PM +0100, Sebastian Andrzej Siewior wrote:
>> * Alexandre Montplaisir | 2014-11-12 17:14:45 [-0500]:
>
> SNIP
>
>>
>> >This was based on the most recent file format I was aware of, we will
>> >update it accordingly if required.
>> >
>> >Testing welcome!
>>
>> I pushed the perf changes I mentioned to
>>
>> git://git.breakpoint.cc/bigeasy/linux.git ctf_convert_7
>>
>
> from perf data convert point of view (and libbabeltrace).. ;-)
>
> I tried to convert big perf.data (2GB) and my laptop got stuck
> for few minutes.. the reason is that perf allocated too much memory
>
> I needed to add occasional bt_ctf_stream_flush into the
> processing (patch attached)
>
> What I do now is checking if we processed given amount of events
> for the stream and once the value is crossed I flush it.
>
> My question is if there's a way to find out the allocated memory
> for the stream? It'd be nicer to setup maximum allocation size
> rather than the number of events.
>
Not currently, although I agree that this would be a nice feature.
Perhaps by setting a maximal packet size, CTF Writer could handle the
automatic flushing.
Thoughts?
Jérémie
> thanks,
> jirka
>
>
>
> diff --git a/tools/perf/util/data-bt.c b/tools/perf/util/data-bt.c
> index c5720e13d8f1..981e8ff2c32a 100644
> --- a/tools/perf/util/data-bt.c
> +++ b/tools/perf/util/data-bt.c
> @@ -39,10 +39,15 @@ struct evsel_priv {
>
> #define MAX_CPUS 4096
>
> +struct ctf_stream {
> + struct bt_ctf_stream *stream;
> + u64 count;
> +};
> +
> struct ctf_writer {
> /* writer primitives */
> struct bt_ctf_writer*writer;
> - struct bt_ctf_stream*stream[MAX_CPUS];
> + struct ctf_stream *stream[MAX_CPUS];
> struct bt_ctf_stream_class *stream_class;
> struct bt_ctf_clock *clock;
> u32 last_cpu;
> @@ -377,6 +382,73 @@ static int add_generic_values(struct ctf_writer *cw,
> return 0;
> }
>
> +#define STREAM_FLUSH_COUNT 1000
> +
> +static bool is_flush_needed(struct ctf_stream *cstream)
> +{
> + return cstream && (cstream->count >= STREAM_FLUSH_COUNT);
> +}
> +
> +static int flush_stream(struct ctf_writer *cw, int cpu)
> +{
> + struct ctf_stream *cstream = cw->stream[cpu];
> + int err = 0;
> +
> + if (cstream) {
> + err = bt_ctf_stream_flush(cstream->stream);
> + if (err)
> + pr_err("CTF stream %d flush failed\n", cpu);
> +
> + cstream->count = 0;
> + }
> +
> + return err;
> +}
> +
> +static int create_stream(struct ctf_writer *cw, int cpu)
> +{
> + struct ctf_stream *cstream;
> + struct bt_ctf_field *pkt_ctx;
> + struct bt_ctf_field *cpu_field;
> + struct bt_ctf_stream *stream;
> + u32 i = cpu;
> + int ret;
> +
> + cstream = zalloc(sizeof(*cstream));
> + if (!cstream) {
> + pr_err("Failed to allocate ctf stream\n");
> + return -1;
> + }
> +
> + stream = bt_ctf_writer_create_stream(cw->writer, cw->stream_class);
> + if (!stream)
> + return -1;
> +
> + pkt_ctx = bt_ctf_stream_get_packet_context(stream);
> + if (!pkt_ctx) {
> + pr_err("Failed to obtain packet context\n");
> + return -1;
> + }
> +
> + cpu_field = bt_ctf_field_structure_get_field(pkt_ctx, "cpu_id");
> + bt_ctf_field_put(pkt_ctx);
> + if (!cpu_field) {
> + pr_err("Failed to obtain cpu field\n");
> + return -1;
> + }
> +
> + ret = bt_ctf_field_unsigned_integer_set_value(cpu_field, i);
> + if (ret) {
> + pr_err("Failed to update CPU number\n");
> + return -1;
> + }
> + bt_ctf_field_put(cpu_field);
> +
> + cstream->stream = stream;
> + cw->stream[i] = cstream;
> + return 0;
> +}
> +
> static int process_sample_event(struct perf_tool *tool,
> union perf_event *_event __maybe_unused,
> struct perf_sample *sample,
> @@ -431,40 +503,14 @@ static int process_sample_event(struct perf_tool *tool,
> cpu = 0;
> }
>
> - if (!cw->stream[cpu]) {
> - struct bt_ctf_field *pkt_ctx;
> - struct bt_ctf_field *cpu_field;
> - struct bt_ctf_stream *stream;
> - u32 i = sample->cpu;
> -
> - stream = bt_ctf_writer_create_stream(cw->writer,
> cw->stream_class);
> - if (!stream)
> - return -1;
> -
> - pkt_ctx = bt_ctf_stream_get_packet_context(stream);
> - if (!pkt_ctx) {
> - pr_err("Failed to obtain packet context\n");
> - return -1;
> - }
> -
> -
Re: Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
On Fri, Nov 28, 2014 at 6:26 AM, Jiri Olsa jo...@redhat.com wrote:
On Wed, Nov 26, 2014 at 06:37:21PM +0100, Sebastian Andrzej Siewior wrote:
* Alexandre Montplaisir | 2014-11-12 17:14:45 [-0500]:
SNIP
This was based on the most recent file format I was aware of, we will
update it accordingly if required.
Testing welcome!
I pushed the perf changes I mentioned to
git://git.breakpoint.cc/bigeasy/linux.git ctf_convert_7
from perf data convert point of view (and libbabeltrace).. ;-)
I tried to convert big perf.data (2GB) and my laptop got stuck
for few minutes.. the reason is that perf allocated too much memory
I needed to add occasional bt_ctf_stream_flush into the
processing (patch attached)
What I do now is checking if we processed given amount of events
for the stream and once the value is crossed I flush it.
My question is if there's a way to find out the allocated memory
for the stream? It'd be nicer to setup maximum allocation size
rather than the number of events.
Not currently, although I agree that this would be a nice feature.
Perhaps by setting a maximal packet size, CTF Writer could handle the
automatic flushing.
Thoughts?
Jérémie
thanks,
jirka
diff --git a/tools/perf/util/data-bt.c b/tools/perf/util/data-bt.c
index c5720e13d8f1..981e8ff2c32a 100644
--- a/tools/perf/util/data-bt.c
+++ b/tools/perf/util/data-bt.c
@@ -39,10 +39,15 @@ struct evsel_priv {
#define MAX_CPUS 4096
+struct ctf_stream {
+ struct bt_ctf_stream *stream;
+ u64 count;
+};
+
struct ctf_writer {
/* writer primitives */
struct bt_ctf_writer*writer;
- struct bt_ctf_stream*stream[MAX_CPUS];
+ struct ctf_stream *stream[MAX_CPUS];
struct bt_ctf_stream_class *stream_class;
struct bt_ctf_clock *clock;
u32 last_cpu;
@@ -377,6 +382,73 @@ static int add_generic_values(struct ctf_writer *cw,
return 0;
}
+#define STREAM_FLUSH_COUNT 1000
+
+static bool is_flush_needed(struct ctf_stream *cstream)
+{
+ return cstream (cstream-count = STREAM_FLUSH_COUNT);
+}
+
+static int flush_stream(struct ctf_writer *cw, int cpu)
+{
+ struct ctf_stream *cstream = cw-stream[cpu];
+ int err = 0;
+
+ if (cstream) {
+ err = bt_ctf_stream_flush(cstream-stream);
+ if (err)
+ pr_err(CTF stream %d flush failed\n, cpu);
+
+ cstream-count = 0;
+ }
+
+ return err;
+}
+
+static int create_stream(struct ctf_writer *cw, int cpu)
+{
+ struct ctf_stream *cstream;
+ struct bt_ctf_field *pkt_ctx;
+ struct bt_ctf_field *cpu_field;
+ struct bt_ctf_stream *stream;
+ u32 i = cpu;
+ int ret;
+
+ cstream = zalloc(sizeof(*cstream));
+ if (!cstream) {
+ pr_err(Failed to allocate ctf stream\n);
+ return -1;
+ }
+
+ stream = bt_ctf_writer_create_stream(cw-writer, cw-stream_class);
+ if (!stream)
+ return -1;
+
+ pkt_ctx = bt_ctf_stream_get_packet_context(stream);
+ if (!pkt_ctx) {
+ pr_err(Failed to obtain packet context\n);
+ return -1;
+ }
+
+ cpu_field = bt_ctf_field_structure_get_field(pkt_ctx, cpu_id);
+ bt_ctf_field_put(pkt_ctx);
+ if (!cpu_field) {
+ pr_err(Failed to obtain cpu field\n);
+ return -1;
+ }
+
+ ret = bt_ctf_field_unsigned_integer_set_value(cpu_field, i);
+ if (ret) {
+ pr_err(Failed to update CPU number\n);
+ return -1;
+ }
+ bt_ctf_field_put(cpu_field);
+
+ cstream-stream = stream;
+ cw-stream[i] = cstream;
+ return 0;
+}
+
static int process_sample_event(struct perf_tool *tool,
union perf_event *_event __maybe_unused,
struct perf_sample *sample,
@@ -431,40 +503,14 @@ static int process_sample_event(struct perf_tool *tool,
cpu = 0;
}
- if (!cw-stream[cpu]) {
- struct bt_ctf_field *pkt_ctx;
- struct bt_ctf_field *cpu_field;
- struct bt_ctf_stream *stream;
- u32 i = sample-cpu;
-
- stream = bt_ctf_writer_create_stream(cw-writer,
cw-stream_class);
- if (!stream)
- return -1;
-
- pkt_ctx = bt_ctf_stream_get_packet_context(stream);
- if (!pkt_ctx) {
- pr_err(Failed to obtain packet context\n);
- return -1;
- }
-
- cpu_field = bt_ctf_field_structure_get_field(pkt_ctx,
cpu_id);
- bt_ctf_field_put(pkt_ctx);
- if (!cpu_field) {
-
Re: Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
On Mon, Dec 01, 2014 at 12:28:08PM -0500, Jérémie Galarneau wrote: On Fri, Nov 28, 2014 at 6:26 AM, Jiri Olsa jo...@redhat.com wrote: On Wed, Nov 26, 2014 at 06:37:21PM +0100, Sebastian Andrzej Siewior wrote: * Alexandre Montplaisir | 2014-11-12 17:14:45 [-0500]: SNIP This was based on the most recent file format I was aware of, we will update it accordingly if required. Testing welcome! I pushed the perf changes I mentioned to git://git.breakpoint.cc/bigeasy/linux.git ctf_convert_7 from perf data convert point of view (and libbabeltrace).. ;-) I tried to convert big perf.data (2GB) and my laptop got stuck for few minutes.. the reason is that perf allocated too much memory I needed to add occasional bt_ctf_stream_flush into the processing (patch attached) What I do now is checking if we processed given amount of events for the stream and once the value is crossed I flush it. My question is if there's a way to find out the allocated memory for the stream? It'd be nicer to setup maximum allocation size rather than the number of events. Not currently, although I agree that this would be a nice feature. Perhaps by setting a maximal packet size, CTF Writer could handle the automatic flushing. Thoughts? hum, it looks like 2 separated things to me.. the packet size is result format related size.. I'm more interested in the size of the 'struct ctf_stream' within the writer during the processing it seems it wont be trivial to track this AFAICS from sources, but I guess I can live so far with tracking the the number of events, until we find something else thanks, jirka -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [tracecompass-dev] Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
Hi Alexandre,
Catching with this thread of email: could you please share some links to
documentation/screenshot about perf support with trace compass ?
Regards,
Jerome
-Original Message-
From: tracecompass-dev-boun...@eclipse.org
[mailto:tracecompass-dev-boun...@eclipse.org] On Behalf Of Alexandre Montplaisir
Sent: Thursday, November 27, 2014 5:28 AM
To: Sebastian Andrzej Siewior
Cc: Jeremie Galarneau; linux-kernel@vger.kernel.org; Arnaldo Carvalho de Melo;
Tom Zanussi; Mathieu Desnoyers; David Ahern; Dominique Toupin; Trace Compass
Developer Discussions; Jiri Olsa; andr...@linutronix.de
Subject: Re: [tracecompass-dev] Support for Perf CTF traces now in master (was
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
On 2014-11-26 12:37 PM, Sebastian Andrzej Siewior wrote:
> * Alexandre Montplaisir | 2014-11-12 17:14:45 [-0500]:
>
>> Just a quick note, this branch is now merged to master. So anyone who
>> pulls the code from the master branch at
>> git://git.eclipse.org/gitroot/tracecompass/org.eclipse.tracecompass.g
>> it should be able to load perf-CTF traces in the viewer. The trace
>> type is now called "Common Trace Format -> Linux Kernel Trace" and
>> should support both LTTng kernel and perf traces in CTF format
>> (although auto-detection should work in most cases).
> Thank you for all the work.
> Let me try to reply to the emails at once here:
> - I added to the environment metadata the following (comparing to the
>last version):
> domain => kernel
> tracer_name => perf
>
>There is no tracer_major + minor. Instead I added
>version => perf's version
>On my system I have:
> release = "3.16.0-4-amd64";
> version = "3.18.rc3.g91405a";
>
>Because I run Debian's v3.16 and recorded the trace with perf from the
>kernel 3.18.rc3.
>There is no version of perf doing the convert of perf.data => ctf.
>Any objections, rename of the version member?
>
> - Mathieu decided that it makes no sense to add the kernel version to
>each event we trace. Instead each event should have its own version
>with a major/minor member. Once the event is changed the "ABI"
>version should be adjusted. I second this since it makes sense.
>Therefore there are no changes that were made to the converter.
>
> - Alexandre (you) noticed that there are syscall names in the events
>recorded via "sys_enter and sys_exit". This is true, but there is a
>hint. There is an event for instance:
>
> [03:37:07.579969498] (+?.?) raw_syscalls:sys_enter: { cpu_id =
> 2 }, { perf_ip = 0x81020EBC, perf_tid = 30004, perf_pid =
> 30004, perf_id = 382, perf_period = 1, common_type = 76, common_flags
> = 0, common_preempt_count = 0, common_pid = 30004, id = 16, args = [
> [0] = 0xE, [1] = 0x2400, [2] = 0x0, [3] = 0x0, [4] = 0xA20F00, [5] =
> 0xA1FDA0 ] }
Oh ok, so this "id" field is really the system call ID then. Good to know,
thanks!
>
>By the end you notice id=16 and args. args are the Arguments passed
>to syscall and id is the syscall number. Together with machine =
>x86_64 you know which architecture you need to lookup the number 16.
>The numbers are from unistd.h (and may be different between architectures,
>even between i386 & x86_64). strace has for instance the following [0]
> table.
>
> { 3,TD, sys_read, "read" }, /* 0 */
> …
> { 3,TD, sys_ioctl, "ioctl" }, /* 16 */
> …
>
> So 16 is ioctl. strace has those tables for a bunch of architectures
> so it might be helpful to suck them in. I know no other way to ease
> things here.
Indeed. Well this information could be part of the trace metadata too, but I
guess that wouldn't be very practical.
We'll just need to add a way for each supported tracer to advertize how it gets
its system call names.
>
> [0] https://github.com/bnoordhuis/strace/blob/master/linux/x86_64/syscallent.h
>
> The same thing is true for softirq_entry events for instance. This event
> will give you you only vec=9 and you need to lookup that 9 => RCU. That
> one is easy however:
>
> const char * const softirq_to_name[NR_SOFTIRQS] = {
> "HI", "TIMER", "NET_TX", "NET_RX", "BLOCK", "BLOCK_IOPOLL",
> "TASKLET", "SCHED", "HRTIMER", "RCU"
> };
>
> this has been taken from kernel/softirq.c.
Oh, that's right, we never got around to getting/showing the actual
names of the soft IRQs. Thanks for reminding us. ;)
>> This was based on the most recent file format I was aware of, we will
&g
RE: [tracecompass-dev] Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
Hi Alexandre,
Catching with this thread of email: could you please share some links to
documentation/screenshot about perf support with trace compass ?
Regards,
Jerome
-Original Message-
From: tracecompass-dev-boun...@eclipse.org
[mailto:tracecompass-dev-boun...@eclipse.org] On Behalf Of Alexandre Montplaisir
Sent: Thursday, November 27, 2014 5:28 AM
To: Sebastian Andrzej Siewior
Cc: Jeremie Galarneau; linux-kernel@vger.kernel.org; Arnaldo Carvalho de Melo;
Tom Zanussi; Mathieu Desnoyers; David Ahern; Dominique Toupin; Trace Compass
Developer Discussions; Jiri Olsa; andr...@linutronix.de
Subject: Re: [tracecompass-dev] Support for Perf CTF traces now in master (was
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
On 2014-11-26 12:37 PM, Sebastian Andrzej Siewior wrote:
* Alexandre Montplaisir | 2014-11-12 17:14:45 [-0500]:
Just a quick note, this branch is now merged to master. So anyone who
pulls the code from the master branch at
git://git.eclipse.org/gitroot/tracecompass/org.eclipse.tracecompass.g
it should be able to load perf-CTF traces in the viewer. The trace
type is now called Common Trace Format - Linux Kernel Trace and
should support both LTTng kernel and perf traces in CTF format
(although auto-detection should work in most cases).
Thank you for all the work.
Let me try to reply to the emails at once here:
- I added to the environment metadata the following (comparing to the
last version):
domain = kernel
tracer_name = perf
There is no tracer_major + minor. Instead I added
version = perf's version
On my system I have:
release = 3.16.0-4-amd64;
version = 3.18.rc3.g91405a;
Because I run Debian's v3.16 and recorded the trace with perf from the
kernel 3.18.rc3.
There is no version of perf doing the convert of perf.data = ctf.
Any objections, rename of the version member?
- Mathieu decided that it makes no sense to add the kernel version to
each event we trace. Instead each event should have its own version
with a major/minor member. Once the event is changed the ABI
version should be adjusted. I second this since it makes sense.
Therefore there are no changes that were made to the converter.
- Alexandre (you) noticed that there are syscall names in the events
recorded via sys_enter and sys_exit. This is true, but there is a
hint. There is an event for instance:
[03:37:07.579969498] (+?.?) raw_syscalls:sys_enter: { cpu_id =
2 }, { perf_ip = 0x81020EBC, perf_tid = 30004, perf_pid =
30004, perf_id = 382, perf_period = 1, common_type = 76, common_flags
= 0, common_preempt_count = 0, common_pid = 30004, id = 16, args = [
[0] = 0xE, [1] = 0x2400, [2] = 0x0, [3] = 0x0, [4] = 0xA20F00, [5] =
0xA1FDA0 ] }
Oh ok, so this id field is really the system call ID then. Good to know,
thanks!
By the end you notice id=16 and args. args are the Arguments passed
to syscall and id is the syscall number. Together with machine =
x86_64 you know which architecture you need to lookup the number 16.
The numbers are from unistd.h (and may be different between architectures,
even between i386 x86_64). strace has for instance the following [0]
table.
{ 3,TD, sys_read, read }, /* 0 */
…
{ 3,TD, sys_ioctl, ioctl }, /* 16 */
…
So 16 is ioctl. strace has those tables for a bunch of architectures
so it might be helpful to suck them in. I know no other way to ease
things here.
Indeed. Well this information could be part of the trace metadata too, but I
guess that wouldn't be very practical.
We'll just need to add a way for each supported tracer to advertize how it gets
its system call names.
[0] https://github.com/bnoordhuis/strace/blob/master/linux/x86_64/syscallent.h
The same thing is true for softirq_entry events for instance. This event
will give you you only vec=9 and you need to lookup that 9 = RCU. That
one is easy however:
const char * const softirq_to_name[NR_SOFTIRQS] = {
HI, TIMER, NET_TX, NET_RX, BLOCK, BLOCK_IOPOLL,
TASKLET, SCHED, HRTIMER, RCU
};
this has been taken from kernel/softirq.c.
Oh, that's right, we never got around to getting/showing the actual
names of the soft IRQs. Thanks for reminding us. ;)
This was based on the most recent file format I was aware of, we will
update it accordingly if required.
Testing welcome!
I pushed the perf changes I mentioned to
git://git.breakpoint.cc/bigeasy/linux.git ctf_convert_7
It is now based on Arnaldo's perf/core. If everything goes well from
the compass side and nobody complains here in any way, the next step
would be to present the patches on the mailing list and compass as a
user.
I took you tree and added the patch below. I uploaded the following
files to https://breakpoint.cc/perf-ctf/:
- ctf-out6.tar.xz from perf6.data.xz
shows nothing
Hmm, indeed it throws exceptions
Re: Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
On Wed, Nov 26, 2014 at 06:37:21PM +0100, Sebastian Andrzej Siewior wrote:
> * Alexandre Montplaisir | 2014-11-12 17:14:45 [-0500]:
SNIP
>
> >This was based on the most recent file format I was aware of, we will
> >update it accordingly if required.
> >
> >Testing welcome!
>
> I pushed the perf changes I mentioned to
>
> git://git.breakpoint.cc/bigeasy/linux.git ctf_convert_7
>
from perf data convert point of view (and libbabeltrace).. ;-)
I tried to convert big perf.data (2GB) and my laptop got stuck
for few minutes.. the reason is that perf allocated too much memory
I needed to add occasional bt_ctf_stream_flush into the
processing (patch attached)
What I do now is checking if we processed given amount of events
for the stream and once the value is crossed I flush it.
My question is if there's a way to find out the allocated memory
for the stream? It'd be nicer to setup maximum allocation size
rather than the number of events.
thanks,
jirka
diff --git a/tools/perf/util/data-bt.c b/tools/perf/util/data-bt.c
index c5720e13d8f1..981e8ff2c32a 100644
--- a/tools/perf/util/data-bt.c
+++ b/tools/perf/util/data-bt.c
@@ -39,10 +39,15 @@ struct evsel_priv {
#define MAX_CPUS 4096
+struct ctf_stream {
+ struct bt_ctf_stream *stream;
+ u64 count;
+};
+
struct ctf_writer {
/* writer primitives */
struct bt_ctf_writer*writer;
- struct bt_ctf_stream*stream[MAX_CPUS];
+ struct ctf_stream *stream[MAX_CPUS];
struct bt_ctf_stream_class *stream_class;
struct bt_ctf_clock *clock;
u32 last_cpu;
@@ -377,6 +382,73 @@ static int add_generic_values(struct ctf_writer *cw,
return 0;
}
+#define STREAM_FLUSH_COUNT 1000
+
+static bool is_flush_needed(struct ctf_stream *cstream)
+{
+ return cstream && (cstream->count >= STREAM_FLUSH_COUNT);
+}
+
+static int flush_stream(struct ctf_writer *cw, int cpu)
+{
+ struct ctf_stream *cstream = cw->stream[cpu];
+ int err = 0;
+
+ if (cstream) {
+ err = bt_ctf_stream_flush(cstream->stream);
+ if (err)
+ pr_err("CTF stream %d flush failed\n", cpu);
+
+ cstream->count = 0;
+ }
+
+ return err;
+}
+
+static int create_stream(struct ctf_writer *cw, int cpu)
+{
+ struct ctf_stream *cstream;
+ struct bt_ctf_field *pkt_ctx;
+ struct bt_ctf_field *cpu_field;
+ struct bt_ctf_stream *stream;
+ u32 i = cpu;
+ int ret;
+
+ cstream = zalloc(sizeof(*cstream));
+ if (!cstream) {
+ pr_err("Failed to allocate ctf stream\n");
+ return -1;
+ }
+
+ stream = bt_ctf_writer_create_stream(cw->writer, cw->stream_class);
+ if (!stream)
+ return -1;
+
+ pkt_ctx = bt_ctf_stream_get_packet_context(stream);
+ if (!pkt_ctx) {
+ pr_err("Failed to obtain packet context\n");
+ return -1;
+ }
+
+ cpu_field = bt_ctf_field_structure_get_field(pkt_ctx, "cpu_id");
+ bt_ctf_field_put(pkt_ctx);
+ if (!cpu_field) {
+ pr_err("Failed to obtain cpu field\n");
+ return -1;
+ }
+
+ ret = bt_ctf_field_unsigned_integer_set_value(cpu_field, i);
+ if (ret) {
+ pr_err("Failed to update CPU number\n");
+ return -1;
+ }
+ bt_ctf_field_put(cpu_field);
+
+ cstream->stream = stream;
+ cw->stream[i] = cstream;
+ return 0;
+}
+
static int process_sample_event(struct perf_tool *tool,
union perf_event *_event __maybe_unused,
struct perf_sample *sample,
@@ -431,40 +503,14 @@ static int process_sample_event(struct perf_tool *tool,
cpu = 0;
}
- if (!cw->stream[cpu]) {
- struct bt_ctf_field *pkt_ctx;
- struct bt_ctf_field *cpu_field;
- struct bt_ctf_stream *stream;
- u32 i = sample->cpu;
-
- stream = bt_ctf_writer_create_stream(cw->writer,
cw->stream_class);
- if (!stream)
- return -1;
-
- pkt_ctx = bt_ctf_stream_get_packet_context(stream);
- if (!pkt_ctx) {
- pr_err("Failed to obtain packet context\n");
- return -1;
- }
-
- cpu_field = bt_ctf_field_structure_get_field(pkt_ctx, "cpu_id");
- bt_ctf_field_put(pkt_ctx);
- if (!cpu_field) {
- pr_err("Failed to obtain cpu field\n");
- return -1;
- }
+ if (!cw->stream[cpu] && create_stream(cw, cpu))
+ return -1;
- ret = bt_ctf_field_unsigned_integer_set_value(cpu_field, i);
- if (ret) {
- pr_err("Failed to
Re: Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
On Thu, Nov 27, 2014 at 01:31:38PM -0500, Alexandre Montplaisir wrote: > > On 11/27/2014 10:43 AM, Jiri Olsa wrote: > >On Wed, Nov 12, 2014 at 05:14:45PM -0500, Alexandre Montplaisir wrote: > >> > >>Testing welcome! > >hi, > >any other way besides compiling eclipse to test this? For pure mortals > >with Fedora eclipse rpm.. ;-) > > If you already have an Eclipse installation, you can use the update site: > http://download.eclipse.org/tracecompass/master/nightly/ > This would install the plugins into that Eclipse install. > > We will also put nightly builds of the stand-alone version on the download > page Very Soon(TM). We just have a few issues left to figure out. cool, I'll check thanks, jirka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
On Thu, Nov 27, 2014 at 01:31:38PM -0500, Alexandre Montplaisir wrote: On 11/27/2014 10:43 AM, Jiri Olsa wrote: On Wed, Nov 12, 2014 at 05:14:45PM -0500, Alexandre Montplaisir wrote: Testing welcome! hi, any other way besides compiling eclipse to test this? For pure mortals with Fedora eclipse rpm.. ;-) If you already have an Eclipse installation, you can use the update site: http://download.eclipse.org/tracecompass/master/nightly/ This would install the plugins into that Eclipse install. We will also put nightly builds of the stand-alone version on the download page Very Soon(TM). We just have a few issues left to figure out. cool, I'll check thanks, jirka -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
On Wed, Nov 26, 2014 at 06:37:21PM +0100, Sebastian Andrzej Siewior wrote:
* Alexandre Montplaisir | 2014-11-12 17:14:45 [-0500]:
SNIP
This was based on the most recent file format I was aware of, we will
update it accordingly if required.
Testing welcome!
I pushed the perf changes I mentioned to
git://git.breakpoint.cc/bigeasy/linux.git ctf_convert_7
from perf data convert point of view (and libbabeltrace).. ;-)
I tried to convert big perf.data (2GB) and my laptop got stuck
for few minutes.. the reason is that perf allocated too much memory
I needed to add occasional bt_ctf_stream_flush into the
processing (patch attached)
What I do now is checking if we processed given amount of events
for the stream and once the value is crossed I flush it.
My question is if there's a way to find out the allocated memory
for the stream? It'd be nicer to setup maximum allocation size
rather than the number of events.
thanks,
jirka
diff --git a/tools/perf/util/data-bt.c b/tools/perf/util/data-bt.c
index c5720e13d8f1..981e8ff2c32a 100644
--- a/tools/perf/util/data-bt.c
+++ b/tools/perf/util/data-bt.c
@@ -39,10 +39,15 @@ struct evsel_priv {
#define MAX_CPUS 4096
+struct ctf_stream {
+ struct bt_ctf_stream *stream;
+ u64 count;
+};
+
struct ctf_writer {
/* writer primitives */
struct bt_ctf_writer*writer;
- struct bt_ctf_stream*stream[MAX_CPUS];
+ struct ctf_stream *stream[MAX_CPUS];
struct bt_ctf_stream_class *stream_class;
struct bt_ctf_clock *clock;
u32 last_cpu;
@@ -377,6 +382,73 @@ static int add_generic_values(struct ctf_writer *cw,
return 0;
}
+#define STREAM_FLUSH_COUNT 1000
+
+static bool is_flush_needed(struct ctf_stream *cstream)
+{
+ return cstream (cstream-count = STREAM_FLUSH_COUNT);
+}
+
+static int flush_stream(struct ctf_writer *cw, int cpu)
+{
+ struct ctf_stream *cstream = cw-stream[cpu];
+ int err = 0;
+
+ if (cstream) {
+ err = bt_ctf_stream_flush(cstream-stream);
+ if (err)
+ pr_err(CTF stream %d flush failed\n, cpu);
+
+ cstream-count = 0;
+ }
+
+ return err;
+}
+
+static int create_stream(struct ctf_writer *cw, int cpu)
+{
+ struct ctf_stream *cstream;
+ struct bt_ctf_field *pkt_ctx;
+ struct bt_ctf_field *cpu_field;
+ struct bt_ctf_stream *stream;
+ u32 i = cpu;
+ int ret;
+
+ cstream = zalloc(sizeof(*cstream));
+ if (!cstream) {
+ pr_err(Failed to allocate ctf stream\n);
+ return -1;
+ }
+
+ stream = bt_ctf_writer_create_stream(cw-writer, cw-stream_class);
+ if (!stream)
+ return -1;
+
+ pkt_ctx = bt_ctf_stream_get_packet_context(stream);
+ if (!pkt_ctx) {
+ pr_err(Failed to obtain packet context\n);
+ return -1;
+ }
+
+ cpu_field = bt_ctf_field_structure_get_field(pkt_ctx, cpu_id);
+ bt_ctf_field_put(pkt_ctx);
+ if (!cpu_field) {
+ pr_err(Failed to obtain cpu field\n);
+ return -1;
+ }
+
+ ret = bt_ctf_field_unsigned_integer_set_value(cpu_field, i);
+ if (ret) {
+ pr_err(Failed to update CPU number\n);
+ return -1;
+ }
+ bt_ctf_field_put(cpu_field);
+
+ cstream-stream = stream;
+ cw-stream[i] = cstream;
+ return 0;
+}
+
static int process_sample_event(struct perf_tool *tool,
union perf_event *_event __maybe_unused,
struct perf_sample *sample,
@@ -431,40 +503,14 @@ static int process_sample_event(struct perf_tool *tool,
cpu = 0;
}
- if (!cw-stream[cpu]) {
- struct bt_ctf_field *pkt_ctx;
- struct bt_ctf_field *cpu_field;
- struct bt_ctf_stream *stream;
- u32 i = sample-cpu;
-
- stream = bt_ctf_writer_create_stream(cw-writer,
cw-stream_class);
- if (!stream)
- return -1;
-
- pkt_ctx = bt_ctf_stream_get_packet_context(stream);
- if (!pkt_ctx) {
- pr_err(Failed to obtain packet context\n);
- return -1;
- }
-
- cpu_field = bt_ctf_field_structure_get_field(pkt_ctx, cpu_id);
- bt_ctf_field_put(pkt_ctx);
- if (!cpu_field) {
- pr_err(Failed to obtain cpu field\n);
- return -1;
- }
+ if (!cw-stream[cpu] create_stream(cw, cpu))
+ return -1;
- ret = bt_ctf_field_unsigned_integer_set_value(cpu_field, i);
- if (ret) {
- pr_err(Failed to update CPU number\n);
- return
Re: Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
On 11/27/2014 10:43 AM, Jiri Olsa wrote: On Wed, Nov 12, 2014 at 05:14:45PM -0500, Alexandre Montplaisir wrote: Testing welcome! hi, any other way besides compiling eclipse to test this? For pure mortals with Fedora eclipse rpm.. ;-) If you already have an Eclipse installation, you can use the update site: http://download.eclipse.org/tracecompass/master/nightly/ This would install the plugins into that Eclipse install. We will also put nightly builds of the stand-alone version on the download page Very Soon(TM). We just have a few issues left to figure out. Cheers, Alexandre -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
On 11/27/2014 04:43 PM, Jiri Olsa wrote: > hi, > any other way besides compiling eclipse to test this? For pure mortals > with Fedora eclipse rpm.. ;-) > > or any instructions for the compilation.. I actually haven't checked yet | mvn clean install -Pbuild-rcp -Dmaven.test.skip=true does the trick. It took a while to complete especially since it sucked some jars at 30KiB/sec That archive at https://breakpoint.cc/perf-ctf/trace-compass-0.1.0-20141126-1744-linux.gtk.x86_64.tar.xz contains the standalone application "tracecompass" without eclipse. It contains also the patch I quoted in the thread. > thanks, > jirka Sebastian -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
On Wed, Nov 12, 2014 at 05:14:45PM -0500, Alexandre Montplaisir wrote: > > On 11/09/2014 08:31 PM, Alexandre Montplaisir wrote: > >On 2014-11-05 10:25 PM, Alexandre Montplaisir wrote: > >> > >>> > But if you could for example tell me the perf equivalents of all the > strings in that file, I could hack together such wrapper. With that, > in theory, perf traces should behave exactly the same as LTTng traces > in the viewer! > >>>Oooh, that would be awesome. So I installed maven but didn't get much > >>>further. Let me gather this for you. > >> > >>Awesome, thanks! > >> > >>I am travelling this week, so I'm a bit busy, but I will try to > >>prototype a "wrapper" for the kernel analysis, and adding support for > >>the perf events, whenever I have a chance. I'll keep you posted. > > > >Ok, some good news! > > > >I managed to get the CTF traces from perf working in Trace Compass! See > >attached screenshots. This is showing the "ctf-out2" trace from your > >previous email. The other trace seems to have less events enabled, so it > >would only show some WAIT_FOR_CPU states in the view. > > > >If anybody wishes to try it, you can grab the whole branch ending at > >https://git.eclipse.org/r/#/c/36200/ . Or run: > >$ git fetch > >git://git.eclipse.org/gitroot/tracecompass/org.eclipse.tracecompass > >refs/changes/00/36200/3 && git checkout FETCH_HEAD > > Just a quick note, this branch is now merged to master. So anyone who pulls > the code from the master branch at > git://git.eclipse.org/gitroot/tracecompass/org.eclipse.tracecompass.git > should be able to load perf-CTF traces in the viewer. The trace type is now > called "Common Trace Format -> Linux Kernel Trace" and should support both > LTTng kernel and perf traces in CTF format (although auto-detection should > work in most cases). > > This was based on the most recent file format I was aware of, we will update > it accordingly if required. > > Testing welcome! hi, any other way besides compiling eclipse to test this? For pure mortals with Fedora eclipse rpm.. ;-) or any instructions for the compilation.. I actually haven't checked yet thanks, jirka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
On Wed, Nov 12, 2014 at 05:14:45PM -0500, Alexandre Montplaisir wrote: On 11/09/2014 08:31 PM, Alexandre Montplaisir wrote: On 2014-11-05 10:25 PM, Alexandre Montplaisir wrote: But if you could for example tell me the perf equivalents of all the strings in that file, I could hack together such wrapper. With that, in theory, perf traces should behave exactly the same as LTTng traces in the viewer! Oooh, that would be awesome. So I installed maven but didn't get much further. Let me gather this for you. Awesome, thanks! I am travelling this week, so I'm a bit busy, but I will try to prototype a wrapper for the kernel analysis, and adding support for the perf events, whenever I have a chance. I'll keep you posted. Ok, some good news! I managed to get the CTF traces from perf working in Trace Compass! See attached screenshots. This is showing the ctf-out2 trace from your previous email. The other trace seems to have less events enabled, so it would only show some WAIT_FOR_CPU states in the view. If anybody wishes to try it, you can grab the whole branch ending at https://git.eclipse.org/r/#/c/36200/ . Or run: $ git fetch git://git.eclipse.org/gitroot/tracecompass/org.eclipse.tracecompass refs/changes/00/36200/3 git checkout FETCH_HEAD Just a quick note, this branch is now merged to master. So anyone who pulls the code from the master branch at git://git.eclipse.org/gitroot/tracecompass/org.eclipse.tracecompass.git should be able to load perf-CTF traces in the viewer. The trace type is now called Common Trace Format - Linux Kernel Trace and should support both LTTng kernel and perf traces in CTF format (although auto-detection should work in most cases). This was based on the most recent file format I was aware of, we will update it accordingly if required. Testing welcome! hi, any other way besides compiling eclipse to test this? For pure mortals with Fedora eclipse rpm.. ;-) or any instructions for the compilation.. I actually haven't checked yet thanks, jirka -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
On 11/27/2014 04:43 PM, Jiri Olsa wrote: hi, any other way besides compiling eclipse to test this? For pure mortals with Fedora eclipse rpm.. ;-) or any instructions for the compilation.. I actually haven't checked yet | mvn clean install -Pbuild-rcp -Dmaven.test.skip=true does the trick. It took a while to complete especially since it sucked some jars at 30KiB/sec That archive at https://breakpoint.cc/perf-ctf/trace-compass-0.1.0-20141126-1744-linux.gtk.x86_64.tar.xz contains the standalone application tracecompass without eclipse. It contains also the patch I quoted in the thread. thanks, jirka Sebastian -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
On 11/27/2014 10:43 AM, Jiri Olsa wrote: On Wed, Nov 12, 2014 at 05:14:45PM -0500, Alexandre Montplaisir wrote: Testing welcome! hi, any other way besides compiling eclipse to test this? For pure mortals with Fedora eclipse rpm.. ;-) If you already have an Eclipse installation, you can use the update site: http://download.eclipse.org/tracecompass/master/nightly/ This would install the plugins into that Eclipse install. We will also put nightly builds of the stand-alone version on the download page Very Soon(TM). We just have a few issues left to figure out. Cheers, Alexandre -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
On 2014-11-26 12:37 PM, Sebastian Andrzej Siewior wrote:
* Alexandre Montplaisir | 2014-11-12 17:14:45 [-0500]:
Just a quick note, this branch is now merged to master. So anyone who
pulls the code from the master branch at
git://git.eclipse.org/gitroot/tracecompass/org.eclipse.tracecompass.git
should be able to load perf-CTF traces in the viewer. The trace type
is now called "Common Trace Format -> Linux Kernel Trace" and should
support both LTTng kernel and perf traces in CTF format (although
auto-detection should work in most cases).
Thank you for all the work.
Let me try to reply to the emails at once here:
- I added to the environment metadata the following (comparing to the
last version):
domain => kernel
tracer_name => perf
There is no tracer_major + minor. Instead I added
version => perf's version
On my system I have:
release = "3.16.0-4-amd64";
version = "3.18.rc3.g91405a";
Because I run Debian's v3.16 and recorded the trace with perf from the
kernel 3.18.rc3.
There is no version of perf doing the convert of perf.data => ctf.
Any objections, rename of the version member?
- Mathieu decided that it makes no sense to add the kernel version to
each event we trace. Instead each event should have its own version
with a major/minor member. Once the event is changed the "ABI"
version should be adjusted. I second this since it makes sense.
Therefore there are no changes that were made to the converter.
- Alexandre (you) noticed that there are syscall names in the events
recorded via "sys_enter and sys_exit". This is true, but there is a
hint. There is an event for instance:
[03:37:07.579969498] (+?.?) raw_syscalls:sys_enter: { cpu_id = 2 }, {
perf_ip = 0x81020EBC, perf_tid = 30004, perf_pid = 30004, perf_id =
382, perf_period = 1, common_type = 76, common_flags = 0, common_preempt_count
= 0, common_pid = 30004, id = 16, args = [ [0] = 0xE, [1] = 0x2400, [2] = 0x0,
[3] = 0x0, [4] = 0xA20F00, [5] = 0xA1FDA0 ] }
Oh ok, so this "id" field is really the system call ID then. Good to
know, thanks!
By the end you notice id=16 and args. args are the Arguments passed
to syscall and id is the syscall number. Together with machine =
x86_64 you know which architecture you need to lookup the number 16.
The numbers are from unistd.h (and may be different between architectures,
even between i386 & x86_64). strace has for instance the following [0] table.
{ 3,TD, sys_read, "read" }, /* 0 */
…
{ 3,TD, sys_ioctl, "ioctl" }, /* 16 */
…
So 16 is ioctl. strace has those tables for a bunch of architectures
so it might be helpful to suck them in. I know no other way to ease
things here.
Indeed. Well this information could be part of the trace metadata too,
but I guess that wouldn't be very practical.
We'll just need to add a way for each supported tracer to advertize how
it gets its system call names.
[0] https://github.com/bnoordhuis/strace/blob/master/linux/x86_64/syscallent.h
The same thing is true for softirq_entry events for instance. This event
will give you you only vec=9 and you need to lookup that 9 => RCU. That
one is easy however:
const char * const softirq_to_name[NR_SOFTIRQS] = {
"HI", "TIMER", "NET_TX", "NET_RX", "BLOCK", "BLOCK_IOPOLL",
"TASKLET", "SCHED", "HRTIMER", "RCU"
};
this has been taken from kernel/softirq.c.
Oh, that's right, we never got around to getting/showing the actual
names of the soft IRQs. Thanks for reminding us. ;)
This was based on the most recent file format I was aware of, we will
update it accordingly if required.
Testing welcome!
I pushed the perf changes I mentioned to
git://git.breakpoint.cc/bigeasy/linux.git ctf_convert_7
It is now based on Arnaldo's perf/core. If everything goes well from
the compass side and nobody complains here in any way, the next step
would be to present the patches on the mailing list and compass as a
user.
I took you tree and added the patch below. I uploaded the following
files to https://breakpoint.cc/perf-ctf/:
- ctf-out6.tar.xz from perf6.data.xz
shows nothing
Hmm, indeed it throws exceptions in the console when trying to validate
the trace. It seems to read a packet size in perf_stream_0 as a negative
value. Babeltrace handles it fine though, so we're probably reading it
wrong. We'll investigate.
Cheers,
Alexandre
- ctf-out7.tar.xz from perf7.data.xz
shows something
The only obvious difference is the size of the CTF data. The size of out6
is almost 300MiB and it contains 3,259,929 events. The out7 is has only
15MiB and contains 152,900 events.
Cheers,
Alexandre
✂
From 7ffa619d918f2010046b391ae29063ffc5329468 Mon Sep 17 00:00:00 2001
From: Sebastian Andrzej Siewior
Date: Wed, 26 Nov 2014 18:04:53 +0100
Subject: [PATCH] CTF: use tracer_name for perf-CTF traces
domain will be set to kernel for both, perf and lttng traces. The
tracer_name
Re: Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
* Alexandre Montplaisir | 2014-11-12 17:14:45 [-0500]:
>Just a quick note, this branch is now merged to master. So anyone who
>pulls the code from the master branch at
>git://git.eclipse.org/gitroot/tracecompass/org.eclipse.tracecompass.git
>should be able to load perf-CTF traces in the viewer. The trace type
>is now called "Common Trace Format -> Linux Kernel Trace" and should
>support both LTTng kernel and perf traces in CTF format (although
>auto-detection should work in most cases).
Thank you for all the work.
Let me try to reply to the emails at once here:
- I added to the environment metadata the following (comparing to the
last version):
domain => kernel
tracer_name => perf
There is no tracer_major + minor. Instead I added
version => perf's version
On my system I have:
release = "3.16.0-4-amd64";
version = "3.18.rc3.g91405a";
Because I run Debian's v3.16 and recorded the trace with perf from the
kernel 3.18.rc3.
There is no version of perf doing the convert of perf.data => ctf.
Any objections, rename of the version member?
- Mathieu decided that it makes no sense to add the kernel version to
each event we trace. Instead each event should have its own version
with a major/minor member. Once the event is changed the "ABI"
version should be adjusted. I second this since it makes sense.
Therefore there are no changes that were made to the converter.
- Alexandre (you) noticed that there are syscall names in the events
recorded via "sys_enter and sys_exit". This is true, but there is a
hint. There is an event for instance:
[03:37:07.579969498] (+?.?) raw_syscalls:sys_enter: { cpu_id = 2 }, {
perf_ip = 0x81020EBC, perf_tid = 30004, perf_pid = 30004, perf_id =
382, perf_period = 1, common_type = 76, common_flags = 0, common_preempt_count
= 0, common_pid = 30004, id = 16, args = [ [0] = 0xE, [1] = 0x2400, [2] = 0x0,
[3] = 0x0, [4] = 0xA20F00, [5] = 0xA1FDA0 ] }
By the end you notice id=16 and args. args are the Arguments passed
to syscall and id is the syscall number. Together with machine =
x86_64 you know which architecture you need to lookup the number 16.
The numbers are from unistd.h (and may be different between architectures,
even between i386 & x86_64). strace has for instance the following [0] table.
{ 3,TD, sys_read, "read" }, /* 0 */
…
{ 3,TD, sys_ioctl, "ioctl" }, /* 16 */
…
So 16 is ioctl. strace has those tables for a bunch of architectures
so it might be helpful to suck them in. I know no other way to ease
things here.
[0] https://github.com/bnoordhuis/strace/blob/master/linux/x86_64/syscallent.h
The same thing is true for softirq_entry events for instance. This event
will give you you only vec=9 and you need to lookup that 9 => RCU. That
one is easy however:
const char * const softirq_to_name[NR_SOFTIRQS] = {
"HI", "TIMER", "NET_TX", "NET_RX", "BLOCK", "BLOCK_IOPOLL",
"TASKLET", "SCHED", "HRTIMER", "RCU"
};
this has been taken from kernel/softirq.c.
>This was based on the most recent file format I was aware of, we will
>update it accordingly if required.
>
>Testing welcome!
I pushed the perf changes I mentioned to
git://git.breakpoint.cc/bigeasy/linux.git ctf_convert_7
It is now based on Arnaldo's perf/core. If everything goes well from
the compass side and nobody complains here in any way, the next step
would be to present the patches on the mailing list and compass as a
user.
I took you tree and added the patch below. I uploaded the following
files to https://breakpoint.cc/perf-ctf/:
- ctf-out6.tar.xz from perf6.data.xz
shows nothing
- ctf-out7.tar.xz from perf7.data.xz
shows something
The only obvious difference is the size of the CTF data. The size of out6
is almost 300MiB and it contains 3,259,929 events. The out7 is has only
15MiB and contains 152,900 events.
>Cheers,
>Alexandre
✂
>From 7ffa619d918f2010046b391ae29063ffc5329468 Mon Sep 17 00:00:00 2001
From: Sebastian Andrzej Siewior
Date: Wed, 26 Nov 2014 18:04:53 +0100
Subject: [PATCH] CTF: use tracer_name for perf-CTF traces
domain will be set to kernel for both, perf and lttng traces. The
tracer_name will be set to perf if the trace is generated by perf and
otherwise lttng-modules if created by thet lttng tool.
Signed-off-by: Sebastian Andrzej Siewior
---
.../tracecompass/lttng2/kernel/core/trace/LttngKernelTrace.java | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git
a/org.eclipse.tracecompass.lttng2.kernel.core/src/org/eclipse/tracecompass/lttng2/kernel/core/trace/LttngKernelTrace.java
b/org.eclipse.tracecompass.lttng2.kernel.core/src/org/eclipse/tracecompass/lttng2/kernel/core/trace/LttngKernelTrace.java
index a58269f..03a09b9 100644
---
a/org.eclipse.tracecompass.lttng2.kernel.core/src/org/eclipse/tracecompass/lttng2/kernel/core/trace/LttngKernelTrace.java
+++
Re: Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
* Alexandre Montplaisir | 2014-11-12 17:14:45 [-0500]:
Just a quick note, this branch is now merged to master. So anyone who
pulls the code from the master branch at
git://git.eclipse.org/gitroot/tracecompass/org.eclipse.tracecompass.git
should be able to load perf-CTF traces in the viewer. The trace type
is now called Common Trace Format - Linux Kernel Trace and should
support both LTTng kernel and perf traces in CTF format (although
auto-detection should work in most cases).
Thank you for all the work.
Let me try to reply to the emails at once here:
- I added to the environment metadata the following (comparing to the
last version):
domain = kernel
tracer_name = perf
There is no tracer_major + minor. Instead I added
version = perf's version
On my system I have:
release = 3.16.0-4-amd64;
version = 3.18.rc3.g91405a;
Because I run Debian's v3.16 and recorded the trace with perf from the
kernel 3.18.rc3.
There is no version of perf doing the convert of perf.data = ctf.
Any objections, rename of the version member?
- Mathieu decided that it makes no sense to add the kernel version to
each event we trace. Instead each event should have its own version
with a major/minor member. Once the event is changed the ABI
version should be adjusted. I second this since it makes sense.
Therefore there are no changes that were made to the converter.
- Alexandre (you) noticed that there are syscall names in the events
recorded via sys_enter and sys_exit. This is true, but there is a
hint. There is an event for instance:
[03:37:07.579969498] (+?.?) raw_syscalls:sys_enter: { cpu_id = 2 }, {
perf_ip = 0x81020EBC, perf_tid = 30004, perf_pid = 30004, perf_id =
382, perf_period = 1, common_type = 76, common_flags = 0, common_preempt_count
= 0, common_pid = 30004, id = 16, args = [ [0] = 0xE, [1] = 0x2400, [2] = 0x0,
[3] = 0x0, [4] = 0xA20F00, [5] = 0xA1FDA0 ] }
By the end you notice id=16 and args. args are the Arguments passed
to syscall and id is the syscall number. Together with machine =
x86_64 you know which architecture you need to lookup the number 16.
The numbers are from unistd.h (and may be different between architectures,
even between i386 x86_64). strace has for instance the following [0] table.
{ 3,TD, sys_read, read }, /* 0 */
…
{ 3,TD, sys_ioctl, ioctl }, /* 16 */
…
So 16 is ioctl. strace has those tables for a bunch of architectures
so it might be helpful to suck them in. I know no other way to ease
things here.
[0] https://github.com/bnoordhuis/strace/blob/master/linux/x86_64/syscallent.h
The same thing is true for softirq_entry events for instance. This event
will give you you only vec=9 and you need to lookup that 9 = RCU. That
one is easy however:
const char * const softirq_to_name[NR_SOFTIRQS] = {
HI, TIMER, NET_TX, NET_RX, BLOCK, BLOCK_IOPOLL,
TASKLET, SCHED, HRTIMER, RCU
};
this has been taken from kernel/softirq.c.
This was based on the most recent file format I was aware of, we will
update it accordingly if required.
Testing welcome!
I pushed the perf changes I mentioned to
git://git.breakpoint.cc/bigeasy/linux.git ctf_convert_7
It is now based on Arnaldo's perf/core. If everything goes well from
the compass side and nobody complains here in any way, the next step
would be to present the patches on the mailing list and compass as a
user.
I took you tree and added the patch below. I uploaded the following
files to https://breakpoint.cc/perf-ctf/:
- ctf-out6.tar.xz from perf6.data.xz
shows nothing
- ctf-out7.tar.xz from perf7.data.xz
shows something
The only obvious difference is the size of the CTF data. The size of out6
is almost 300MiB and it contains 3,259,929 events. The out7 is has only
15MiB and contains 152,900 events.
Cheers,
Alexandre
✂
From 7ffa619d918f2010046b391ae29063ffc5329468 Mon Sep 17 00:00:00 2001
From: Sebastian Andrzej Siewior bige...@linutronix.de
Date: Wed, 26 Nov 2014 18:04:53 +0100
Subject: [PATCH] CTF: use tracer_name for perf-CTF traces
domain will be set to kernel for both, perf and lttng traces. The
tracer_name will be set to perf if the trace is generated by perf and
otherwise lttng-modules if created by thet lttng tool.
Signed-off-by: Sebastian Andrzej Siewior bige...@linutronix.de
---
.../tracecompass/lttng2/kernel/core/trace/LttngKernelTrace.java | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git
a/org.eclipse.tracecompass.lttng2.kernel.core/src/org/eclipse/tracecompass/lttng2/kernel/core/trace/LttngKernelTrace.java
b/org.eclipse.tracecompass.lttng2.kernel.core/src/org/eclipse/tracecompass/lttng2/kernel/core/trace/LttngKernelTrace.java
index a58269f..03a09b9 100644
---
a/org.eclipse.tracecompass.lttng2.kernel.core/src/org/eclipse/tracecompass/lttng2/kernel/core/trace/LttngKernelTrace.java
+++
Re: Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
On 2014-11-26 12:37 PM, Sebastian Andrzej Siewior wrote:
* Alexandre Montplaisir | 2014-11-12 17:14:45 [-0500]:
Just a quick note, this branch is now merged to master. So anyone who
pulls the code from the master branch at
git://git.eclipse.org/gitroot/tracecompass/org.eclipse.tracecompass.git
should be able to load perf-CTF traces in the viewer. The trace type
is now called Common Trace Format - Linux Kernel Trace and should
support both LTTng kernel and perf traces in CTF format (although
auto-detection should work in most cases).
Thank you for all the work.
Let me try to reply to the emails at once here:
- I added to the environment metadata the following (comparing to the
last version):
domain = kernel
tracer_name = perf
There is no tracer_major + minor. Instead I added
version = perf's version
On my system I have:
release = 3.16.0-4-amd64;
version = 3.18.rc3.g91405a;
Because I run Debian's v3.16 and recorded the trace with perf from the
kernel 3.18.rc3.
There is no version of perf doing the convert of perf.data = ctf.
Any objections, rename of the version member?
- Mathieu decided that it makes no sense to add the kernel version to
each event we trace. Instead each event should have its own version
with a major/minor member. Once the event is changed the ABI
version should be adjusted. I second this since it makes sense.
Therefore there are no changes that were made to the converter.
- Alexandre (you) noticed that there are syscall names in the events
recorded via sys_enter and sys_exit. This is true, but there is a
hint. There is an event for instance:
[03:37:07.579969498] (+?.?) raw_syscalls:sys_enter: { cpu_id = 2 }, {
perf_ip = 0x81020EBC, perf_tid = 30004, perf_pid = 30004, perf_id =
382, perf_period = 1, common_type = 76, common_flags = 0, common_preempt_count
= 0, common_pid = 30004, id = 16, args = [ [0] = 0xE, [1] = 0x2400, [2] = 0x0,
[3] = 0x0, [4] = 0xA20F00, [5] = 0xA1FDA0 ] }
Oh ok, so this id field is really the system call ID then. Good to
know, thanks!
By the end you notice id=16 and args. args are the Arguments passed
to syscall and id is the syscall number. Together with machine =
x86_64 you know which architecture you need to lookup the number 16.
The numbers are from unistd.h (and may be different between architectures,
even between i386 x86_64). strace has for instance the following [0] table.
{ 3,TD, sys_read, read }, /* 0 */
…
{ 3,TD, sys_ioctl, ioctl }, /* 16 */
…
So 16 is ioctl. strace has those tables for a bunch of architectures
so it might be helpful to suck them in. I know no other way to ease
things here.
Indeed. Well this information could be part of the trace metadata too,
but I guess that wouldn't be very practical.
We'll just need to add a way for each supported tracer to advertize how
it gets its system call names.
[0] https://github.com/bnoordhuis/strace/blob/master/linux/x86_64/syscallent.h
The same thing is true for softirq_entry events for instance. This event
will give you you only vec=9 and you need to lookup that 9 = RCU. That
one is easy however:
const char * const softirq_to_name[NR_SOFTIRQS] = {
HI, TIMER, NET_TX, NET_RX, BLOCK, BLOCK_IOPOLL,
TASKLET, SCHED, HRTIMER, RCU
};
this has been taken from kernel/softirq.c.
Oh, that's right, we never got around to getting/showing the actual
names of the soft IRQs. Thanks for reminding us. ;)
This was based on the most recent file format I was aware of, we will
update it accordingly if required.
Testing welcome!
I pushed the perf changes I mentioned to
git://git.breakpoint.cc/bigeasy/linux.git ctf_convert_7
It is now based on Arnaldo's perf/core. If everything goes well from
the compass side and nobody complains here in any way, the next step
would be to present the patches on the mailing list and compass as a
user.
I took you tree and added the patch below. I uploaded the following
files to https://breakpoint.cc/perf-ctf/:
- ctf-out6.tar.xz from perf6.data.xz
shows nothing
Hmm, indeed it throws exceptions in the console when trying to validate
the trace. It seems to read a packet size in perf_stream_0 as a negative
value. Babeltrace handles it fine though, so we're probably reading it
wrong. We'll investigate.
Cheers,
Alexandre
- ctf-out7.tar.xz from perf7.data.xz
shows something
The only obvious difference is the size of the CTF data. The size of out6
is almost 300MiB and it contains 3,259,929 events. The out7 is has only
15MiB and contains 152,900 events.
Cheers,
Alexandre
✂
From 7ffa619d918f2010046b391ae29063ffc5329468 Mon Sep 17 00:00:00 2001
From: Sebastian Andrzej Siewior bige...@linutronix.de
Date: Wed, 26 Nov 2014 18:04:53 +0100
Subject: [PATCH] CTF: use tracer_name for perf-CTF traces
domain will be set to kernel for both, perf and lttng traces. The
tracer_name will be set to perf if
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
adding Matthew Khouzam to the loop jirka On Thu, Nov 13, 2014 at 08:24:48PM +0100, Sebastian Andrzej Siewior wrote: > I try to get through my ctf mailbox and I hoped I can finish it today > but I don't make it completely… > > On 11/06/2014 04:25 AM, Alexandre Montplaisir wrote: > > "mvn clean install". It is the Maven equivalent of "./configure && make" ;) > > > > Or if you want to build a standalone application (RCP): > > mvn clean install -Pbuild-rcp -Dmaven.test.skip=true > > thanks. > > > Yes, the state dump is something specific to LTTng. It allows us to know > > about processes that exist on the system, even if they are sleeping for > > the whole duration of the trace (and thus, would not show up in the > > trace at all). > > > > But even if these events are not present, we can still know about active > > processes when they do shed_switch's for example. > > good to know. > > > IIRC, compat_sys is for instance for 32-bit system calls on a 64-bit > > kernel. Perhaps the "compat" system calls are recorded as standard > > system call events with perf? We could test it once we get the base > > things working. > > I booted a x86-64 in a 32bit userland and I expected to see them > somewhere but nothing. However if the task has a compat flag then it > would be the same information, right? > > >> - static final String SYS_CLONE = "sys_clone"; > >>here we have > >> syscalls:sys_enter_clone > >> syscalls:sys_exit_clone > >>I guess the enter is what you are looking for. > >> > >> For the fields, this is one event with alle the members we have. Please > >> note that lttng saves the members with the _ prefix and I haven't seen > >> that prefix in that .java file. > > > > As Mathieu explained in his reply, in LTTng-CTF they have a _ before > > field names. In our parser, we take out the first character if it is an > > underscore. So it should still work with underscore-less fields. > > I see. I think it started working once I added the underline prefix but > I might be wrong. Let me see what Mathieu says if I may leave that > prefix out. > > > Cheers, > > Alexandre > Sebastian -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
adding Matthew Khouzam to the loop jirka On Thu, Nov 13, 2014 at 08:24:48PM +0100, Sebastian Andrzej Siewior wrote: I try to get through my ctf mailbox and I hoped I can finish it today but I don't make it completely… On 11/06/2014 04:25 AM, Alexandre Montplaisir wrote: mvn clean install. It is the Maven equivalent of ./configure make ;) Or if you want to build a standalone application (RCP): mvn clean install -Pbuild-rcp -Dmaven.test.skip=true thanks. Yes, the state dump is something specific to LTTng. It allows us to know about processes that exist on the system, even if they are sleeping for the whole duration of the trace (and thus, would not show up in the trace at all). But even if these events are not present, we can still know about active processes when they do shed_switch's for example. good to know. IIRC, compat_sys is for instance for 32-bit system calls on a 64-bit kernel. Perhaps the compat system calls are recorded as standard system call events with perf? We could test it once we get the base things working. I booted a x86-64 in a 32bit userland and I expected to see them somewhere but nothing. However if the task has a compat flag then it would be the same information, right? - static final String SYS_CLONE = sys_clone; here we have syscalls:sys_enter_clone syscalls:sys_exit_clone I guess the enter is what you are looking for. For the fields, this is one event with alle the members we have. Please note that lttng saves the members with the _ prefix and I haven't seen that prefix in that .java file. As Mathieu explained in his reply, in LTTng-CTF they have a _ before field names. In our parser, we take out the first character if it is an underscore. So it should still work with underscore-less fields. I see. I think it started working once I added the underline prefix but I might be wrong. Let me see what Mathieu says if I may leave that prefix out. Cheers, Alexandre Sebastian -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
I try to get through my ctf mailbox and I hoped I can finish it today but I don't make it completely… On 11/06/2014 04:25 AM, Alexandre Montplaisir wrote: > "mvn clean install". It is the Maven equivalent of "./configure && make" ;) > > Or if you want to build a standalone application (RCP): > mvn clean install -Pbuild-rcp -Dmaven.test.skip=true thanks. > Yes, the state dump is something specific to LTTng. It allows us to know > about processes that exist on the system, even if they are sleeping for > the whole duration of the trace (and thus, would not show up in the > trace at all). > > But even if these events are not present, we can still know about active > processes when they do shed_switch's for example. good to know. > IIRC, compat_sys is for instance for 32-bit system calls on a 64-bit > kernel. Perhaps the "compat" system calls are recorded as standard > system call events with perf? We could test it once we get the base > things working. I booted a x86-64 in a 32bit userland and I expected to see them somewhere but nothing. However if the task has a compat flag then it would be the same information, right? >> - static final String SYS_CLONE = "sys_clone"; >>here we have >> syscalls:sys_enter_clone >> syscalls:sys_exit_clone >>I guess the enter is what you are looking for. >> >> For the fields, this is one event with alle the members we have. Please >> note that lttng saves the members with the _ prefix and I haven't seen >> that prefix in that .java file. > > As Mathieu explained in his reply, in LTTng-CTF they have a _ before > field names. In our parser, we take out the first character if it is an > underscore. So it should still work with underscore-less fields. I see. I think it started working once I added the underline prefix but I might be wrong. Let me see what Mathieu says if I may leave that prefix out. > Cheers, > Alexandre Sebastian -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
I try to get through my ctf mailbox and I hoped I can finish it today but I don't make it completely… On 11/06/2014 04:25 AM, Alexandre Montplaisir wrote: mvn clean install. It is the Maven equivalent of ./configure make ;) Or if you want to build a standalone application (RCP): mvn clean install -Pbuild-rcp -Dmaven.test.skip=true thanks. Yes, the state dump is something specific to LTTng. It allows us to know about processes that exist on the system, even if they are sleeping for the whole duration of the trace (and thus, would not show up in the trace at all). But even if these events are not present, we can still know about active processes when they do shed_switch's for example. good to know. IIRC, compat_sys is for instance for 32-bit system calls on a 64-bit kernel. Perhaps the compat system calls are recorded as standard system call events with perf? We could test it once we get the base things working. I booted a x86-64 in a 32bit userland and I expected to see them somewhere but nothing. However if the task has a compat flag then it would be the same information, right? - static final String SYS_CLONE = sys_clone; here we have syscalls:sys_enter_clone syscalls:sys_exit_clone I guess the enter is what you are looking for. For the fields, this is one event with alle the members we have. Please note that lttng saves the members with the _ prefix and I haven't seen that prefix in that .java file. As Mathieu explained in his reply, in LTTng-CTF they have a _ before field names. In our parser, we take out the first character if it is an underscore. So it should still work with underscore-less fields. I see. I think it started working once I added the underline prefix but I might be wrong. Let me see what Mathieu says if I may leave that prefix out. Cheers, Alexandre Sebastian -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
On 11/09/2014 08:31 PM, Alexandre Montplaisir wrote: On 2014-11-05 10:25 PM, Alexandre Montplaisir wrote: But if you could for example tell me the perf equivalents of all the strings in that file, I could hack together such wrapper. With that, in theory, perf traces should behave exactly the same as LTTng traces in the viewer! Oooh, that would be awesome. So I installed maven but didn't get much further. Let me gather this for you. Awesome, thanks! I am travelling this week, so I'm a bit busy, but I will try to prototype a "wrapper" for the kernel analysis, and adding support for the perf events, whenever I have a chance. I'll keep you posted. Ok, some good news! I managed to get the CTF traces from perf working in Trace Compass! See attached screenshots. This is showing the "ctf-out2" trace from your previous email. The other trace seems to have less events enabled, so it would only show some WAIT_FOR_CPU states in the view. If anybody wishes to try it, you can grab the whole branch ending at https://git.eclipse.org/r/#/c/36200/ . Or run: $ git fetch git://git.eclipse.org/gitroot/tracecompass/org.eclipse.tracecompass refs/changes/00/36200/3 && git checkout FETCH_HEAD Just a quick note, this branch is now merged to master. So anyone who pulls the code from the master branch at git://git.eclipse.org/gitroot/tracecompass/org.eclipse.tracecompass.git should be able to load perf-CTF traces in the viewer. The trace type is now called "Common Trace Format -> Linux Kernel Trace" and should support both LTTng kernel and perf traces in CTF format (although auto-detection should work in most cases). This was based on the most recent file format I was aware of, we will update it accordingly if required. Testing welcome! Cheers, Alexandre It reuses much of the code from the LTTng analysis, which is why it was relatively quick to do. For now, it looks for the domain in the CTF environment to be "kernel-perf". But this will be easy to update, if needed, once the final format is decided. Maybe I missed it, but I couldn't find the system call names in the trace. Using the sys_enter and sys_exit events, the viewer is able to determine the kernel-mode states (in blue), but we cannot show the exact system call names like we do with LTTng. There is also something weird with the arrows in the Control Flow View (disabled in the screenshot), I don't know if it's due to the particularity of the trace or to a bug in the view. We'll investigate. Feedback is very welcome. Cheers, Alexandre -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Support for Perf CTF traces now in master (was Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion)
On 11/09/2014 08:31 PM, Alexandre Montplaisir wrote: On 2014-11-05 10:25 PM, Alexandre Montplaisir wrote: But if you could for example tell me the perf equivalents of all the strings in that file, I could hack together such wrapper. With that, in theory, perf traces should behave exactly the same as LTTng traces in the viewer! Oooh, that would be awesome. So I installed maven but didn't get much further. Let me gather this for you. Awesome, thanks! I am travelling this week, so I'm a bit busy, but I will try to prototype a wrapper for the kernel analysis, and adding support for the perf events, whenever I have a chance. I'll keep you posted. Ok, some good news! I managed to get the CTF traces from perf working in Trace Compass! See attached screenshots. This is showing the ctf-out2 trace from your previous email. The other trace seems to have less events enabled, so it would only show some WAIT_FOR_CPU states in the view. If anybody wishes to try it, you can grab the whole branch ending at https://git.eclipse.org/r/#/c/36200/ . Or run: $ git fetch git://git.eclipse.org/gitroot/tracecompass/org.eclipse.tracecompass refs/changes/00/36200/3 git checkout FETCH_HEAD Just a quick note, this branch is now merged to master. So anyone who pulls the code from the master branch at git://git.eclipse.org/gitroot/tracecompass/org.eclipse.tracecompass.git should be able to load perf-CTF traces in the viewer. The trace type is now called Common Trace Format - Linux Kernel Trace and should support both LTTng kernel and perf traces in CTF format (although auto-detection should work in most cases). This was based on the most recent file format I was aware of, we will update it accordingly if required. Testing welcome! Cheers, Alexandre It reuses much of the code from the LTTng analysis, which is why it was relatively quick to do. For now, it looks for the domain in the CTF environment to be kernel-perf. But this will be easy to update, if needed, once the final format is decided. Maybe I missed it, but I couldn't find the system call names in the trace. Using the sys_enter and sys_exit events, the viewer is able to determine the kernel-mode states (in blue), but we cannot show the exact system call names like we do with LTTng. There is also something weird with the arrows in the Control Flow View (disabled in the screenshot), I don't know if it's due to the particularity of the trace or to a bug in the view. We'll investigate. Feedback is very welcome. Cheers, Alexandre -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
On 11/05/2014 01:50 PM, Sebastian Andrzej Siewior wrote:
[...]
If the trace events from both LTTng and perf represent the same thing
(and I assume they should, since they come from the same tracepoints,
right?), then we could just add a wrapper on the viewer side to
decide which event/field names to use, depending on the trace type.
Right now, we only define LTTng event and field names:
http://git.eclipse.org/c/tracecompass/org.eclipse.tracecompass.git/tree/org.eclipse.tracecompass.lttng2.kernel.core/src/org/eclipse/tracecompass/internal/lttng2/kernel/core/LttngStrings.java
Okay. So I found this file for linuxtools now let me try tracecompass.
The basic renaming should do the job. Then I have to figure out how to
compile this thingy…
"mvn clean install". It is the Maven equivalent of "./configure && make" ;)
Or if you want to build a standalone application (RCP):
mvn clean install -Pbuild-rcp -Dmaven.test.skip=true
see the README file in the git tree for details.
There is this one thing where you go for "tid" while perf says "pid". I
guess I could figure that out once I have the rename done.
We don't have lttng_statedump_process_state, this look lttng specific. I
would have to look if there is a replacement event in perf.
Yes, the state dump is something specific to LTTng. It allows us to know
about processes that exist on the system, even if they are sleeping for
the whole duration of the trace (and thus, would not show up in the
trace at all).
But even if these events are not present, we can still know about active
processes when they do shed_switch's for example.
I have no idea what we could do about the "unknown" events, say someone
enbales skb tracing. But this is probably something for once we are
done with the basic integration.
But if you could for example tell me the perf equivalents of all the
strings in that file, I could hack together such wrapper. With that,
in theory, perf traces should behave exactly the same as LTTng traces
in the viewer!
Oooh, that would be awesome. So I installed maven but didn't get much
further. Let me gather this for you.
Awesome, thanks!
I am travelling this week, so I'm a bit busy, but I will try to
prototype a "wrapper" for the kernel analysis, and adding support for
the perf events, whenever I have a chance. I'll keep you posted.
So first the renaming:
diff --git a/LttngStrings.java b/LttngStrings.java
--- a/LttngStrings.java
+++ b/LttngStrings.java
@@ -27,17 +27,17 @@ public interface LttngStrings {
/* Event names */
static final String EXIT_SYSCALL = "exit_syscall";
-static final String IRQ_HANDLER_ENTRY = "irq_handler_entry";
-static final String IRQ_HANDLER_EXIT = "irq_handler_exit";
-static final String SOFTIRQ_ENTRY = "softirq_entry";
-static final String SOFTIRQ_EXIT = "softirq_exit";
-static final String SOFTIRQ_RAISE = "softirq_raise";
-static final String SCHED_SWITCH = "sched_switch";
-static final String SCHED_WAKEUP = "sched_wakeup";
-static final String SCHED_WAKEUP_NEW = "sched_wakeup_new";
-static final String SCHED_PROCESS_FORK = "sched_process_fork";
-static final String SCHED_PROCESS_EXIT = "sched_process_exit";
-static final String SCHED_PROCESS_FREE = "sched_process_free";
+static final String IRQ_HANDLER_ENTRY = "irq:irq_handler_entry";
+static final String IRQ_HANDLER_EXIT = "irq:irq_handler_exit";
+static final String SOFTIRQ_ENTRY = "irq:softirq_entry";
+static final String SOFTIRQ_EXIT = "irq:softirq_exit";
+static final String SOFTIRQ_RAISE = "irq:softirq_raise";
+static final String SCHED_SWITCH = "sched:sched_switch";
+static final String SCHED_WAKEUP = "sched:sched_wakeup";
+static final String SCHED_WAKEUP_NEW = "sched:sched_wakeup_new";
+static final String SCHED_PROCESS_FORK = "sched:sched_process_fork";
+static final String SCHED_PROCESS_EXIT = "sched:sched_process_exit";
+static final String SCHED_PROCESS_FREE = "sched:sched_process_free";
static final String STATEDUMP_PROCESS_STATE =
"lttng_statedump_process_state";
/* System call names */
I have no idea how exit_syscall is different from irq_handler_exit and I
think we have to skip for now on STATEDUMP_PROCESS_STATE.
For the syscalls:
- static final String SYSCALL_PREFIX = "sys_";
It is bassicaly:
syscalls:sys_enter_
syscalls:sys_exit_
depending what you are looking for.
- static final String COMPAT_SYSCALL_PREFIX = "compat_sys_";
I haven't found this. Could it be that we don't record the compat_sys
at all?
IIRC, compat_sys is for instance for 32-bit system calls on a 64-bit
kernel. Perhaps the "compat" system calls are recorded as standard
system call events with perf? We could test it once we get the base
things working.
- static final String SYS_CLONE = "sys_clone";
here we have
syscalls:sys_enter_clone
syscalls:sys_exit_clone
I guess the enter is what you are looking
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
* Alexandre Montplaisir | 2014-11-04 02:20:10 [+0100]:
>Hi Sebastian,
Hi Alexandre,
>On 11/03/2014 06:58 PM, Sebastian Andrzej Siewior wrote:
>This is really great! Initially, I had believed that we would have
>needed to add a separate parser plugin, and to consider "perf traces"
>as a completely different beast from LTTng traces. However if you can
>get this close to they way LTTng presents its data, then we can
>probably re-use most of the existing code. In which case we could
>rename the "LTTng Kernel Trace" type in the UI to simply "Linux
>Kernel Trace". And that would cover both LTTng kernel traces and
>CTF-perf traces.
we have now CTF here. So lets see what we do about the naming
convention.
>>The cpu_id field change will be addressed soon on our side.
>>Now, the remaining things:
>>The "domain = kernel" thingy (or another identifier if desired) is
>>something we could add.
>
>Unless the event data is exactly the same, it would be easier to use
>a different name. Like "kernel-perf" for instance?
Some kind of a namespace / identifier is probably not wrong. The lttng
tracer added a tracer version probably in case the format changes
between version for some reason. Perf comes with the kernel so for this
the kernel version should sufficient.
>From the user's point of view, both would still be Linux Kernel
>Traces, but we could use the domain internally to determine which
>event/field layout to use.
>
>Mathieu, any thoughts on how CTF domains should be namespaced?
>
>>Now that I identified the differences between the CTF from lttng and
>>perf, any suggestions / ideas how this could be solved?
>
>I suppose it would be better/cleaner if the event and field names
>would remain the same, or at least be similar, in the perf.data and
>perf-CTF formats.
Yes, that would be cool. Especially if we teach perf to record straight
to CTF.
>If the trace events from both LTTng and perf represent the same thing
>(and I assume they should, since they come from the same tracepoints,
>right?), then we could just add a wrapper on the viewer side to
>decide which event/field names to use, depending on the trace type.
>
>Right now, we only define LTTng event and field names:
>http://git.eclipse.org/c/tracecompass/org.eclipse.tracecompass.git/tree/org.eclipse.tracecompass.lttng2.kernel.core/src/org/eclipse/tracecompass/internal/lttng2/kernel/core/LttngStrings.java
Okay. So I found this file for linuxtools now let me try tracecompass.
The basic renaming should do the job. Then I have to figure out how to
compile this thingy…
There is this one thing where you go for "tid" while perf says "pid". I
guess I could figure that out once I have the rename done.
We don't have lttng_statedump_process_state, this look lttng specific. I
would have to look if there is a replacement event in perf.
I have no idea what we could do about the "unknown" events, say someone
enbales skb tracing. But this is probably something for once we are
done with the basic integration.
>But if you could for example tell me the perf equivalents of all the
>strings in that file, I could hack together such wrapper. With that,
>in theory, perf traces should behave exactly the same as LTTng traces
>in the viewer!
Oooh, that would be awesome. So I installed maven but didn't get much
further. Let me gather this for you.
So first the renaming:
diff --git a/LttngStrings.java b/LttngStrings.java
--- a/LttngStrings.java
+++ b/LttngStrings.java
@@ -27,17 +27,17 @@ public interface LttngStrings {
/* Event names */
static final String EXIT_SYSCALL = "exit_syscall";
-static final String IRQ_HANDLER_ENTRY = "irq_handler_entry";
-static final String IRQ_HANDLER_EXIT = "irq_handler_exit";
-static final String SOFTIRQ_ENTRY = "softirq_entry";
-static final String SOFTIRQ_EXIT = "softirq_exit";
-static final String SOFTIRQ_RAISE = "softirq_raise";
-static final String SCHED_SWITCH = "sched_switch";
-static final String SCHED_WAKEUP = "sched_wakeup";
-static final String SCHED_WAKEUP_NEW = "sched_wakeup_new";
-static final String SCHED_PROCESS_FORK = "sched_process_fork";
-static final String SCHED_PROCESS_EXIT = "sched_process_exit";
-static final String SCHED_PROCESS_FREE = "sched_process_free";
+static final String IRQ_HANDLER_ENTRY = "irq:irq_handler_entry";
+static final String IRQ_HANDLER_EXIT = "irq:irq_handler_exit";
+static final String SOFTIRQ_ENTRY = "irq:softirq_entry";
+static final String SOFTIRQ_EXIT = "irq:softirq_exit";
+static final String SOFTIRQ_RAISE = "irq:softirq_raise";
+static final String SCHED_SWITCH = "sched:sched_switch";
+static final String SCHED_WAKEUP = "sched:sched_wakeup";
+static final String SCHED_WAKEUP_NEW = "sched:sched_wakeup_new";
+static final String SCHED_PROCESS_FORK = "sched:sched_process_fork";
+static final String SCHED_PROCESS_EXIT = "sched:sched_process_exit";
+static final String SCHED_PROCESS_FREE =
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
* Alexandre Montplaisir | 2014-11-04 02:20:10 [+0100]:
Hi Sebastian,
Hi Alexandre,
On 11/03/2014 06:58 PM, Sebastian Andrzej Siewior wrote:
This is really great! Initially, I had believed that we would have
needed to add a separate parser plugin, and to consider perf traces
as a completely different beast from LTTng traces. However if you can
get this close to they way LTTng presents its data, then we can
probably re-use most of the existing code. In which case we could
rename the LTTng Kernel Trace type in the UI to simply Linux
Kernel Trace. And that would cover both LTTng kernel traces and
CTF-perf traces.
we have now CTF here. So lets see what we do about the naming
convention.
The cpu_id field change will be addressed soon on our side.
Now, the remaining things:
The domain = kernel thingy (or another identifier if desired) is
something we could add.
Unless the event data is exactly the same, it would be easier to use
a different name. Like kernel-perf for instance?
Some kind of a namespace / identifier is probably not wrong. The lttng
tracer added a tracer version probably in case the format changes
between version for some reason. Perf comes with the kernel so for this
the kernel version should sufficient.
From the user's point of view, both would still be Linux Kernel
Traces, but we could use the domain internally to determine which
event/field layout to use.
Mathieu, any thoughts on how CTF domains should be namespaced?
Now that I identified the differences between the CTF from lttng and
perf, any suggestions / ideas how this could be solved?
I suppose it would be better/cleaner if the event and field names
would remain the same, or at least be similar, in the perf.data and
perf-CTF formats.
Yes, that would be cool. Especially if we teach perf to record straight
to CTF.
If the trace events from both LTTng and perf represent the same thing
(and I assume they should, since they come from the same tracepoints,
right?), then we could just add a wrapper on the viewer side to
decide which event/field names to use, depending on the trace type.
Right now, we only define LTTng event and field names:
http://git.eclipse.org/c/tracecompass/org.eclipse.tracecompass.git/tree/org.eclipse.tracecompass.lttng2.kernel.core/src/org/eclipse/tracecompass/internal/lttng2/kernel/core/LttngStrings.java
Okay. So I found this file for linuxtools now let me try tracecompass.
The basic renaming should do the job. Then I have to figure out how to
compile this thingy…
There is this one thing where you go for tid while perf says pid. I
guess I could figure that out once I have the rename done.
We don't have lttng_statedump_process_state, this look lttng specific. I
would have to look if there is a replacement event in perf.
I have no idea what we could do about the unknown events, say someone
enbales skb tracing. But this is probably something for once we are
done with the basic integration.
But if you could for example tell me the perf equivalents of all the
strings in that file, I could hack together such wrapper. With that,
in theory, perf traces should behave exactly the same as LTTng traces
in the viewer!
Oooh, that would be awesome. So I installed maven but didn't get much
further. Let me gather this for you.
So first the renaming:
diff --git a/LttngStrings.java b/LttngStrings.java
--- a/LttngStrings.java
+++ b/LttngStrings.java
@@ -27,17 +27,17 @@ public interface LttngStrings {
/* Event names */
static final String EXIT_SYSCALL = exit_syscall;
-static final String IRQ_HANDLER_ENTRY = irq_handler_entry;
-static final String IRQ_HANDLER_EXIT = irq_handler_exit;
-static final String SOFTIRQ_ENTRY = softirq_entry;
-static final String SOFTIRQ_EXIT = softirq_exit;
-static final String SOFTIRQ_RAISE = softirq_raise;
-static final String SCHED_SWITCH = sched_switch;
-static final String SCHED_WAKEUP = sched_wakeup;
-static final String SCHED_WAKEUP_NEW = sched_wakeup_new;
-static final String SCHED_PROCESS_FORK = sched_process_fork;
-static final String SCHED_PROCESS_EXIT = sched_process_exit;
-static final String SCHED_PROCESS_FREE = sched_process_free;
+static final String IRQ_HANDLER_ENTRY = irq:irq_handler_entry;
+static final String IRQ_HANDLER_EXIT = irq:irq_handler_exit;
+static final String SOFTIRQ_ENTRY = irq:softirq_entry;
+static final String SOFTIRQ_EXIT = irq:softirq_exit;
+static final String SOFTIRQ_RAISE = irq:softirq_raise;
+static final String SCHED_SWITCH = sched:sched_switch;
+static final String SCHED_WAKEUP = sched:sched_wakeup;
+static final String SCHED_WAKEUP_NEW = sched:sched_wakeup_new;
+static final String SCHED_PROCESS_FORK = sched:sched_process_fork;
+static final String SCHED_PROCESS_EXIT = sched:sched_process_exit;
+static final String SCHED_PROCESS_FREE = sched:sched_process_free;
static final String STATEDUMP_PROCESS_STATE =
lttng_statedump_process_state;
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
On 11/05/2014 01:50 PM, Sebastian Andrzej Siewior wrote:
[...]
If the trace events from both LTTng and perf represent the same thing
(and I assume they should, since they come from the same tracepoints,
right?), then we could just add a wrapper on the viewer side to
decide which event/field names to use, depending on the trace type.
Right now, we only define LTTng event and field names:
http://git.eclipse.org/c/tracecompass/org.eclipse.tracecompass.git/tree/org.eclipse.tracecompass.lttng2.kernel.core/src/org/eclipse/tracecompass/internal/lttng2/kernel/core/LttngStrings.java
Okay. So I found this file for linuxtools now let me try tracecompass.
The basic renaming should do the job. Then I have to figure out how to
compile this thingy…
mvn clean install. It is the Maven equivalent of ./configure make ;)
Or if you want to build a standalone application (RCP):
mvn clean install -Pbuild-rcp -Dmaven.test.skip=true
see the README file in the git tree for details.
There is this one thing where you go for tid while perf says pid. I
guess I could figure that out once I have the rename done.
We don't have lttng_statedump_process_state, this look lttng specific. I
would have to look if there is a replacement event in perf.
Yes, the state dump is something specific to LTTng. It allows us to know
about processes that exist on the system, even if they are sleeping for
the whole duration of the trace (and thus, would not show up in the
trace at all).
But even if these events are not present, we can still know about active
processes when they do shed_switch's for example.
I have no idea what we could do about the unknown events, say someone
enbales skb tracing. But this is probably something for once we are
done with the basic integration.
But if you could for example tell me the perf equivalents of all the
strings in that file, I could hack together such wrapper. With that,
in theory, perf traces should behave exactly the same as LTTng traces
in the viewer!
Oooh, that would be awesome. So I installed maven but didn't get much
further. Let me gather this for you.
Awesome, thanks!
I am travelling this week, so I'm a bit busy, but I will try to
prototype a wrapper for the kernel analysis, and adding support for
the perf events, whenever I have a chance. I'll keep you posted.
So first the renaming:
diff --git a/LttngStrings.java b/LttngStrings.java
--- a/LttngStrings.java
+++ b/LttngStrings.java
@@ -27,17 +27,17 @@ public interface LttngStrings {
/* Event names */
static final String EXIT_SYSCALL = exit_syscall;
-static final String IRQ_HANDLER_ENTRY = irq_handler_entry;
-static final String IRQ_HANDLER_EXIT = irq_handler_exit;
-static final String SOFTIRQ_ENTRY = softirq_entry;
-static final String SOFTIRQ_EXIT = softirq_exit;
-static final String SOFTIRQ_RAISE = softirq_raise;
-static final String SCHED_SWITCH = sched_switch;
-static final String SCHED_WAKEUP = sched_wakeup;
-static final String SCHED_WAKEUP_NEW = sched_wakeup_new;
-static final String SCHED_PROCESS_FORK = sched_process_fork;
-static final String SCHED_PROCESS_EXIT = sched_process_exit;
-static final String SCHED_PROCESS_FREE = sched_process_free;
+static final String IRQ_HANDLER_ENTRY = irq:irq_handler_entry;
+static final String IRQ_HANDLER_EXIT = irq:irq_handler_exit;
+static final String SOFTIRQ_ENTRY = irq:softirq_entry;
+static final String SOFTIRQ_EXIT = irq:softirq_exit;
+static final String SOFTIRQ_RAISE = irq:softirq_raise;
+static final String SCHED_SWITCH = sched:sched_switch;
+static final String SCHED_WAKEUP = sched:sched_wakeup;
+static final String SCHED_WAKEUP_NEW = sched:sched_wakeup_new;
+static final String SCHED_PROCESS_FORK = sched:sched_process_fork;
+static final String SCHED_PROCESS_EXIT = sched:sched_process_exit;
+static final String SCHED_PROCESS_FREE = sched:sched_process_free;
static final String STATEDUMP_PROCESS_STATE =
lttng_statedump_process_state;
/* System call names */
I have no idea how exit_syscall is different from irq_handler_exit and I
think we have to skip for now on STATEDUMP_PROCESS_STATE.
For the syscalls:
- static final String SYSCALL_PREFIX = sys_;
It is bassicaly:
syscalls:sys_enter_
syscalls:sys_exit_
depending what you are looking for.
- static final String COMPAT_SYSCALL_PREFIX = compat_sys_;
I haven't found this. Could it be that we don't record the compat_sys
at all?
IIRC, compat_sys is for instance for 32-bit system calls on a 64-bit
kernel. Perhaps the compat system calls are recorded as standard
system call events with perf? We could test it once we get the base
things working.
- static final String SYS_CLONE = sys_clone;
here we have
syscalls:sys_enter_clone
syscalls:sys_exit_clone
I guess the enter is what you are looking for.
For the fields, this is one event with alle the members we have.
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
Hi Sebastian, On 11/03/2014 06:58 PM, Sebastian Andrzej Siewior wrote: [...] I did on the linux side: |perf record \ | -e sched:sched_switch \ | -a This gave me perf.data trace. No with the new extension I converted it into CTF data stream (perf data convert -i perf.data --to-ctf ctf) and imported it into eclipse as a new trace. By setting 'domain = "kernel"' I managed to get it accepted as a kernel trace. Additionally I had to: - rename sched:sched_switch -> sched_switch (I dropped the other events) - rename "perf_cpu" to "cpu_id" and move it within "packet context" (had a patch for that but we wanted to merge this later) - I added "prev_tid" with the content of "prev_pid" and I added "next_tid" with the content of "next_pid". Now exclipse does not "freeze" after loading the first entry - I prefixed every entry with _ (so "prev_tid" becomes "_prev_tid") and now it seems to be recognized by the tracing plugin. puh. Now I have something in "cpu usage", "control flow" windows. This is really great! Initially, I had believed that we would have needed to add a separate parser plugin, and to consider "perf traces" as a completely different beast from LTTng traces. However if you can get this close to they way LTTng presents its data, then we can probably re-use most of the existing code. In which case we could rename the "LTTng Kernel Trace" type in the UI to simply "Linux Kernel Trace". And that would cover both LTTng kernel traces and CTF-perf traces. The cpu_id field change will be addressed soon on our side. Now, the remaining things: The "domain = kernel" thingy (or another identifier if desired) is something we could add. Unless the event data is exactly the same, it would be easier to use a different name. Like "kernel-perf" for instance? From the user's point of view, both would still be Linux Kernel Traces, but we could use the domain internally to determine which event/field layout to use. Mathieu, any thoughts on how CTF domains should be namespaced? What do we do on the naming convention? The converter takes basically the event names the way they come from perf. That is why we have a "system" prefix. For the member fields, we take all the specific ones as perf serves them. The only exception is for the generic fields which get a "perf_" prefix in order not to clash with the specific ones. And there is this _ prefix in front of every data member. Now that I identified the differences between the CTF from lttng and perf, any suggestions / ideas how this could be solved? I suppose it would be better/cleaner if the event and field names would remain the same, or at least be similar, in the perf.data and perf-CTF formats. If the trace events from both LTTng and perf represent the same thing (and I assume they should, since they come from the same tracepoints, right?), then we could just add a wrapper on the viewer side to decide which event/field names to use, depending on the trace type. Right now, we only define LTTng event and field names: http://git.eclipse.org/c/tracecompass/org.eclipse.tracecompass.git/tree/org.eclipse.tracecompass.lttng2.kernel.core/src/org/eclipse/tracecompass/internal/lttng2/kernel/core/LttngStrings.java But if you could for example tell me the perf equivalents of all the strings in that file, I could hack together such wrapper. With that, in theory, perf traces should behave exactly the same as LTTng traces in the viewer! Cheers, Alexandre ps. fyi, we have recently moved the trace viewer code to its own project, now called "Trace Compass". We will still support the Linux Tools plugins for the time being, but all new developments are done in Trace Compass. If you want to check it out: http://eclipse.org/tracecompass -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
On 08/20/2014 09:14 PM, Alexandre Montplaisir wrote: > On 08/20/2014 05:28 AM, Jiri Olsa wrote: >> >> ok, easy enough ;-) so I'm guessing this governs the expected >> CTF layout for event/stream headers/contexts, right? > > Correct, if the domain is "kernel" we then assume that the rest of the > trace contains the expected elements of a kernel trace. So that is one thing that I had to add. > Of course, one could craft a CTF trace to advertize itself as "kernel" > or "ust", and not actually have the layout of that trace type, in which > case it would fail parsing later on. > >> Also judging from the trace display, you have hardcoded specific >> displays/actions for specific events? That's all connected/specific >> under trace type? > > Yes the trace type is the main "provider" of functionality. I could go > into more details, but we use Eclipse extension points to define which > columns to put in the event table, which views are available, etc. for > each supported trace type. > >>> Once we have some views or analysis specific to perf CTF traces, we >>> could >>> definitely add a separate trace type for those too. >> I guess tracepoints and breakpoints should display something like >> the standard kernel trace. As for HW events it's usual to display >> profile infomation as the perf report does: >> >> https://perf.wiki.kernel.org/index.php/Tutorial#Sampling_with_perf_record > > Interesting, I haven't tried the perf CTF output yet, but I could see it > using the Statistics view (which by default just prints the % of events, > per event type) to print the results of the different "perf reports", > calculated from the CTF events. Eventually with pie charts! I did on the linux side: |perf record \ | -e sched:sched_switch \ | -a This gave me perf.data trace. No with the new extension I converted it into CTF data stream (perf data convert -i perf.data --to-ctf ctf) and imported it into eclipse as a new trace. By setting 'domain = "kernel"' I managed to get it accepted as a kernel trace. Additionally I had to: - rename sched:sched_switch -> sched_switch (I dropped the other events) - rename "perf_cpu" to "cpu_id" and move it within "packet context" (had a patch for that but we wanted to merge this later) - I added "prev_tid" with the content of "prev_pid" and I added "next_tid" with the content of "next_pid". Now exclipse does not "freeze" after loading the first entry - I prefixed every entry with _ (so "prev_tid" becomes "_prev_tid") and now it seems to be recognized by the tracing plugin. puh. Now I have something in "cpu usage", "control flow" windows. The cpu_id field change will be addressed soon on our side. Now, the remaining things: The "domain = kernel" thingy (or another identifier if desired) is something we could add. What do we do on the naming convention? The converter takes basically the event names the way they come from perf. That is why we have a "system" prefix. For the member fields, we take all the specific ones as perf serves them. The only exception is for the generic fields which get a "perf_" prefix in order not to clash with the specific ones. And there is this _ prefix in front of every data member. Now that I identified the differences between the CTF from lttng and perf, any suggestions / ideas how this could be solved? > Cheers, > Alexandre Sebastian -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
On 08/20/2014 09:14 PM, Alexandre Montplaisir wrote: On 08/20/2014 05:28 AM, Jiri Olsa wrote: ok, easy enough ;-) so I'm guessing this governs the expected CTF layout for event/stream headers/contexts, right? Correct, if the domain is kernel we then assume that the rest of the trace contains the expected elements of a kernel trace. So that is one thing that I had to add. Of course, one could craft a CTF trace to advertize itself as kernel or ust, and not actually have the layout of that trace type, in which case it would fail parsing later on. Also judging from the trace display, you have hardcoded specific displays/actions for specific events? That's all connected/specific under trace type? Yes the trace type is the main provider of functionality. I could go into more details, but we use Eclipse extension points to define which columns to put in the event table, which views are available, etc. for each supported trace type. Once we have some views or analysis specific to perf CTF traces, we could definitely add a separate trace type for those too. I guess tracepoints and breakpoints should display something like the standard kernel trace. As for HW events it's usual to display profile infomation as the perf report does: https://perf.wiki.kernel.org/index.php/Tutorial#Sampling_with_perf_record Interesting, I haven't tried the perf CTF output yet, but I could see it using the Statistics view (which by default just prints the % of events, per event type) to print the results of the different perf reports, calculated from the CTF events. Eventually with pie charts! I did on the linux side: |perf record \ | -e sched:sched_switch \ | -a This gave me perf.data trace. No with the new extension I converted it into CTF data stream (perf data convert -i perf.data --to-ctf ctf) and imported it into eclipse as a new trace. By setting 'domain = kernel' I managed to get it accepted as a kernel trace. Additionally I had to: - rename sched:sched_switch - sched_switch (I dropped the other events) - rename perf_cpu to cpu_id and move it within packet context (had a patch for that but we wanted to merge this later) - I added prev_tid with the content of prev_pid and I added next_tid with the content of next_pid. Now exclipse does not freeze after loading the first entry - I prefixed every entry with _ (so prev_tid becomes _prev_tid) and now it seems to be recognized by the tracing plugin. puh. Now I have something in cpu usage, control flow windows. The cpu_id field change will be addressed soon on our side. Now, the remaining things: The domain = kernel thingy (or another identifier if desired) is something we could add. What do we do on the naming convention? The converter takes basically the event names the way they come from perf. That is why we have a system prefix. For the member fields, we take all the specific ones as perf serves them. The only exception is for the generic fields which get a perf_ prefix in order not to clash with the specific ones. And there is this _ prefix in front of every data member. Now that I identified the differences between the CTF from lttng and perf, any suggestions / ideas how this could be solved? Cheers, Alexandre Sebastian -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
Hi Sebastian, On 11/03/2014 06:58 PM, Sebastian Andrzej Siewior wrote: [...] I did on the linux side: |perf record \ | -e sched:sched_switch \ | -a This gave me perf.data trace. No with the new extension I converted it into CTF data stream (perf data convert -i perf.data --to-ctf ctf) and imported it into eclipse as a new trace. By setting 'domain = kernel' I managed to get it accepted as a kernel trace. Additionally I had to: - rename sched:sched_switch - sched_switch (I dropped the other events) - rename perf_cpu to cpu_id and move it within packet context (had a patch for that but we wanted to merge this later) - I added prev_tid with the content of prev_pid and I added next_tid with the content of next_pid. Now exclipse does not freeze after loading the first entry - I prefixed every entry with _ (so prev_tid becomes _prev_tid) and now it seems to be recognized by the tracing plugin. puh. Now I have something in cpu usage, control flow windows. This is really great! Initially, I had believed that we would have needed to add a separate parser plugin, and to consider perf traces as a completely different beast from LTTng traces. However if you can get this close to they way LTTng presents its data, then we can probably re-use most of the existing code. In which case we could rename the LTTng Kernel Trace type in the UI to simply Linux Kernel Trace. And that would cover both LTTng kernel traces and CTF-perf traces. The cpu_id field change will be addressed soon on our side. Now, the remaining things: The domain = kernel thingy (or another identifier if desired) is something we could add. Unless the event data is exactly the same, it would be easier to use a different name. Like kernel-perf for instance? From the user's point of view, both would still be Linux Kernel Traces, but we could use the domain internally to determine which event/field layout to use. Mathieu, any thoughts on how CTF domains should be namespaced? What do we do on the naming convention? The converter takes basically the event names the way they come from perf. That is why we have a system prefix. For the member fields, we take all the specific ones as perf serves them. The only exception is for the generic fields which get a perf_ prefix in order not to clash with the specific ones. And there is this _ prefix in front of every data member. Now that I identified the differences between the CTF from lttng and perf, any suggestions / ideas how this could be solved? I suppose it would be better/cleaner if the event and field names would remain the same, or at least be similar, in the perf.data and perf-CTF formats. If the trace events from both LTTng and perf represent the same thing (and I assume they should, since they come from the same tracepoints, right?), then we could just add a wrapper on the viewer side to decide which event/field names to use, depending on the trace type. Right now, we only define LTTng event and field names: http://git.eclipse.org/c/tracecompass/org.eclipse.tracecompass.git/tree/org.eclipse.tracecompass.lttng2.kernel.core/src/org/eclipse/tracecompass/internal/lttng2/kernel/core/LttngStrings.java But if you could for example tell me the perf equivalents of all the strings in that file, I could hack together such wrapper. With that, in theory, perf traces should behave exactly the same as LTTng traces in the viewer! Cheers, Alexandre ps. fyi, we have recently moved the trace viewer code to its own project, now called Trace Compass. We will still support the Linux Tools plugins for the time being, but all new developments are done in Trace Compass. If you want to check it out: http://eclipse.org/tracecompass -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
On Thu, Aug 21, 2014 at 04:03:01PM -0400, Alexandre Montplaisir wrote: > > On 08/21/2014 12:58 PM, Jiri Olsa wrote: > >hum, I've got nothing from babeltrace: > > > >[jolsa@krava ~]$ su > >Password: > >[root@krava jolsa]# lttng create perf > >Spawning a session daemon > >Session perf created. > >Traces will be written in /root/lttng-traces/perf-20140821-184956 > >[root@krava jolsa]# lttng add-context -k -t prio -t perf:cpu:cycles > > Oh I see the problem, you don't have any events enabled! In LTTng terms, a > "context" is a piece of information that gets attached to every event. But > if you don't have any event at all, you're not gonna see much context > information. ;) > > Try adding a > # lttng enable-event -a -k > before starting the session. This should give you some output in the > viewers. ugh ;-) 'perf:cpu:cycles' already sounded like event to me.. will try thanks, jirka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
On Thu, Aug 21, 2014 at 04:03:01PM -0400, Alexandre Montplaisir wrote: On 08/21/2014 12:58 PM, Jiri Olsa wrote: hum, I've got nothing from babeltrace: [jolsa@krava ~]$ su Password: [root@krava jolsa]# lttng create perf Spawning a session daemon Session perf created. Traces will be written in /root/lttng-traces/perf-20140821-184956 [root@krava jolsa]# lttng add-context -k -t prio -t perf:cpu:cycles Oh I see the problem, you don't have any events enabled! In LTTng terms, a context is a piece of information that gets attached to every event. But if you don't have any event at all, you're not gonna see much context information. ;) Try adding a # lttng enable-event -a -k before starting the session. This should give you some output in the viewers. ugh ;-) 'perf:cpu:cycles' already sounded like event to me.. will try thanks, jirka -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
On 08/21/2014 12:58 PM, Jiri Olsa wrote: hum, I've got nothing from babeltrace: [jolsa@krava ~]$ su Password: [root@krava jolsa]# lttng create perf Spawning a session daemon Session perf created. Traces will be written in /root/lttng-traces/perf-20140821-184956 [root@krava jolsa]# lttng add-context -k -t prio -t perf:cpu:cycles Oh I see the problem, you don't have any events enabled! In LTTng terms, a "context" is a piece of information that gets attached to every event. But if you don't have any event at all, you're not gonna see much context information. ;) Try adding a # lttng enable-event -a -k before starting the session. This should give you some output in the viewers. Cheers, Alexandre kernel context prio added to all channels kernel context perf:cpu:cycles added to all channels [root@krava jolsa]# lttng start Tracing started for session perf [root@krava jolsa]# lttng stop Waiting for data availability. Tracing stopped for session perf [root@krava jolsa]# lttng destroy Session perf destroyed [root@krava jolsa]# babeltrace ~/lttng-traces/perf-20140821-184956/ [root@krava jolsa]# babeltrace ~/lttng-traces/perf-20140821-184956/kernel/ [root@krava jolsa]# and empty view in eclipse thanks, jirka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
On Wed, Aug 20, 2014 at 03:14:20PM -0400, Alexandre Montplaisir wrote: > On 08/20/2014 05:28 AM, Jiri Olsa wrote: > > > >ok, easy enough ;-) so I'm guessing this governs the expected > >CTF layout for event/stream headers/contexts, right? > > Correct, if the domain is "kernel" we then assume that the rest of the trace > contains the expected elements of a kernel trace. > > Of course, one could craft a CTF trace to advertize itself as "kernel" or > "ust", and not actually have the layout of that trace type, in which case it > would fail parsing later on. > > >Also judging from the trace display, you have hardcoded specific > >displays/actions for specific events? That's all connected/specific > >under trace type? > > Yes the trace type is the main "provider" of functionality. I could go into > more details, but we use Eclipse extension points to define which columns to > put in the event table, which views are available, etc. for each supported > trace type. > > >>Once we have some views or analysis specific to perf CTF traces, we could > >>definitely add a separate trace type for those too. > >I guess tracepoints and breakpoints should display something like > >the standard kernel trace. As for HW events it's usual to display > >profile infomation as the perf report does: > > https://perf.wiki.kernel.org/index.php/Tutorial#Sampling_with_perf_record > > Interesting, I haven't tried the perf CTF output yet, but I could see it > using the Statistics view (which by default just prints the % of events, per > event type) to print the results of the different "perf reports", calculated > from the CTF events. Eventually with pie charts! Basically, perf monitors single HW event and reports its hits/samples distribution across the workload processes. Just by running: $ perf record ls ; perf report you'll get report of HW event 'cycles' distribution over/during the ls process life: Samples: 29 of event 'cycles', Event count (approx.): 3763985 9.65% ls [kernel.kallsyms] [k] find_get_page 5.09% ls [kernel.kallsyms] [k] perf_event_context_sched_in 5.09% ls ls [.] calculate_columns 5.08% ls [kernel.kallsyms] [k] tty_insert_flip_string_fixed_flag 5.07% ls libc-2.17.so [.] get_next_seq 5.06% ls [kernel.kallsyms] [k] down_read_trylock 5.04% ls ls [.] xstrcoll_name 5.03% ls libc-2.17.so [.] __memmove_sse2 5.03% ls libc-2.17.so [.] _dl_addr 5.00% ls [kernel.kallsyms] [k] ext4_release_file 4.99% ls [kernel.kallsyms] [k] filemap_fault 4.88% ls ld-2.17.so [.] _dl_map_object_from_fd > > >I tried to record/display lttng event perf:cpu:cycles, but got nothing > >displayed in eclipse. Looks like this data provides only summary count > >of the event for the workload? > > Just to be sure I understand, you recorded an LTTng kernel trace, in which > you enabled the "perf:cpu:cycles" context? Did this trace display anything > in Babeltrace? > It should display the same in the Eclipse viewer, the value of the context > will be visible in the "Contents" column in the the table (and in the > Properties view), although for now we don't make any use of it. hum, I've got nothing from babeltrace: [jolsa@krava ~]$ su Password: [root@krava jolsa]# lttng create perf Spawning a session daemon Session perf created. Traces will be written in /root/lttng-traces/perf-20140821-184956 [root@krava jolsa]# lttng add-context -k -t prio -t perf:cpu:cycles kernel context prio added to all channels kernel context perf:cpu:cycles added to all channels [root@krava jolsa]# lttng start Tracing started for session perf [root@krava jolsa]# lttng stop Waiting for data availability. Tracing stopped for session perf [root@krava jolsa]# lttng destroy Session perf destroyed [root@krava jolsa]# babeltrace ~/lttng-traces/perf-20140821-184956/ [root@krava jolsa]# babeltrace ~/lttng-traces/perf-20140821-184956/kernel/ [root@krava jolsa]# and empty view in eclipse thanks, jirka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
On Wed, Aug 20, 2014 at 03:14:20PM -0400, Alexandre Montplaisir wrote: On 08/20/2014 05:28 AM, Jiri Olsa wrote: ok, easy enough ;-) so I'm guessing this governs the expected CTF layout for event/stream headers/contexts, right? Correct, if the domain is kernel we then assume that the rest of the trace contains the expected elements of a kernel trace. Of course, one could craft a CTF trace to advertize itself as kernel or ust, and not actually have the layout of that trace type, in which case it would fail parsing later on. Also judging from the trace display, you have hardcoded specific displays/actions for specific events? That's all connected/specific under trace type? Yes the trace type is the main provider of functionality. I could go into more details, but we use Eclipse extension points to define which columns to put in the event table, which views are available, etc. for each supported trace type. Once we have some views or analysis specific to perf CTF traces, we could definitely add a separate trace type for those too. I guess tracepoints and breakpoints should display something like the standard kernel trace. As for HW events it's usual to display profile infomation as the perf report does: https://perf.wiki.kernel.org/index.php/Tutorial#Sampling_with_perf_record Interesting, I haven't tried the perf CTF output yet, but I could see it using the Statistics view (which by default just prints the % of events, per event type) to print the results of the different perf reports, calculated from the CTF events. Eventually with pie charts! Basically, perf monitors single HW event and reports its hits/samples distribution across the workload processes. Just by running: $ perf record ls ; perf report you'll get report of HW event 'cycles' distribution over/during the ls process life: Samples: 29 of event 'cycles', Event count (approx.): 3763985 9.65% ls [kernel.kallsyms] [k] find_get_page 5.09% ls [kernel.kallsyms] [k] perf_event_context_sched_in 5.09% ls ls [.] calculate_columns 5.08% ls [kernel.kallsyms] [k] tty_insert_flip_string_fixed_flag 5.07% ls libc-2.17.so [.] get_next_seq 5.06% ls [kernel.kallsyms] [k] down_read_trylock 5.04% ls ls [.] xstrcoll_name 5.03% ls libc-2.17.so [.] __memmove_sse2 5.03% ls libc-2.17.so [.] _dl_addr 5.00% ls [kernel.kallsyms] [k] ext4_release_file 4.99% ls [kernel.kallsyms] [k] filemap_fault 4.88% ls ld-2.17.so [.] _dl_map_object_from_fd I tried to record/display lttng event perf:cpu:cycles, but got nothing displayed in eclipse. Looks like this data provides only summary count of the event for the workload? Just to be sure I understand, you recorded an LTTng kernel trace, in which you enabled the perf:cpu:cycles context? Did this trace display anything in Babeltrace? It should display the same in the Eclipse viewer, the value of the context will be visible in the Contents column in the the table (and in the Properties view), although for now we don't make any use of it. hum, I've got nothing from babeltrace: [jolsa@krava ~]$ su Password: [root@krava jolsa]# lttng create perf Spawning a session daemon Session perf created. Traces will be written in /root/lttng-traces/perf-20140821-184956 [root@krava jolsa]# lttng add-context -k -t prio -t perf:cpu:cycles kernel context prio added to all channels kernel context perf:cpu:cycles added to all channels [root@krava jolsa]# lttng start Tracing started for session perf [root@krava jolsa]# lttng stop Waiting for data availability. Tracing stopped for session perf [root@krava jolsa]# lttng destroy Session perf destroyed [root@krava jolsa]# babeltrace ~/lttng-traces/perf-20140821-184956/ [root@krava jolsa]# babeltrace ~/lttng-traces/perf-20140821-184956/kernel/ [root@krava jolsa]# and empty view in eclipse thanks, jirka -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
On 08/21/2014 12:58 PM, Jiri Olsa wrote: hum, I've got nothing from babeltrace: [jolsa@krava ~]$ su Password: [root@krava jolsa]# lttng create perf Spawning a session daemon Session perf created. Traces will be written in /root/lttng-traces/perf-20140821-184956 [root@krava jolsa]# lttng add-context -k -t prio -t perf:cpu:cycles Oh I see the problem, you don't have any events enabled! In LTTng terms, a context is a piece of information that gets attached to every event. But if you don't have any event at all, you're not gonna see much context information. ;) Try adding a # lttng enable-event -a -k before starting the session. This should give you some output in the viewers. Cheers, Alexandre kernel context prio added to all channels kernel context perf:cpu:cycles added to all channels [root@krava jolsa]# lttng start Tracing started for session perf [root@krava jolsa]# lttng stop Waiting for data availability. Tracing stopped for session perf [root@krava jolsa]# lttng destroy Session perf destroyed [root@krava jolsa]# babeltrace ~/lttng-traces/perf-20140821-184956/ [root@krava jolsa]# babeltrace ~/lttng-traces/perf-20140821-184956/kernel/ [root@krava jolsa]# and empty view in eclipse thanks, jirka -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
On 08/20/2014 05:28 AM, Jiri Olsa wrote: ok, easy enough ;-) so I'm guessing this governs the expected CTF layout for event/stream headers/contexts, right? Correct, if the domain is "kernel" we then assume that the rest of the trace contains the expected elements of a kernel trace. Of course, one could craft a CTF trace to advertize itself as "kernel" or "ust", and not actually have the layout of that trace type, in which case it would fail parsing later on. Also judging from the trace display, you have hardcoded specific displays/actions for specific events? That's all connected/specific under trace type? Yes the trace type is the main "provider" of functionality. I could go into more details, but we use Eclipse extension points to define which columns to put in the event table, which views are available, etc. for each supported trace type. Once we have some views or analysis specific to perf CTF traces, we could definitely add a separate trace type for those too. I guess tracepoints and breakpoints should display something like the standard kernel trace. As for HW events it's usual to display profile infomation as the perf report does: https://perf.wiki.kernel.org/index.php/Tutorial#Sampling_with_perf_record Interesting, I haven't tried the perf CTF output yet, but I could see it using the Statistics view (which by default just prints the % of events, per event type) to print the results of the different "perf reports", calculated from the CTF events. Eventually with pie charts! I tried to record/display lttng event perf:cpu:cycles, but got nothing displayed in eclipse. Looks like this data provides only summary count of the event for the workload? Just to be sure I understand, you recorded an LTTng kernel trace, in which you enabled the "perf:cpu:cycles" context? Did this trace display anything in Babeltrace? It should display the same in the Eclipse viewer, the value of the context will be visible in the "Contents" column in the the table (and in the Properties view), although for now we don't make any use of it. From what I understand, yes, the value of the different perf:* contexts represents the value of the perf counter at the moment the tracepoint was hit. So you cannot get perf data "in between" trace events when using this method. Cheers, Alexandre -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
On Tue, Aug 19, 2014 at 01:42:12PM -0400, Alexandre Montplaisir wrote: > Hi Jiri, > > (sorry for breaking the thread, I didn't have the original email, but was > forwarded it) np, some other folks to the CC > > I work on the Eclipse viewer (a.k.a. Trace Compass). > > > so I've put perf converted CTF stream into the eclipse CTF trace viewer > and got it displayed with the 'Generic CTF trace' type > > The other type 'LTTng kernel trace' seems to set some rules for the CTF > fields/format the trace has to obey. Is this described somewhere? > > To identify an LTTng kernel trace, we check if the domain defined in the > metadata is "kernel" [1]. I don't think we documented this anywhere though, > we probably should! ok, easy enough ;-) so I'm guessing this governs the expected CTF layout for event/stream headers/contexts, right? Also judging from the trace display, you have hardcoded specific displays/actions for specific events? That's all connected/specific under trace type? > > Once we have some views or analysis specific to perf CTF traces, we could > definitely add a separate trace type for those too. I guess tracepoints and breakpoints should display something like the standard kernel trace. As for HW events it's usual to display profile infomation as the perf report does: https://perf.wiki.kernel.org/index.php/Tutorial#Sampling_with_perf_record I tried to record/display lttng event perf:cpu:cycles, but got nothing displayed in eclipse. Looks like this data provides only summary count of the event for the workload? thanks, jirka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
On Tue, Aug 19, 2014 at 01:42:12PM -0400, Alexandre Montplaisir wrote: Hi Jiri, (sorry for breaking the thread, I didn't have the original email, but was forwarded it) np, some other folks to the CC I work on the Eclipse viewer (a.k.a. Trace Compass). so I've put perf converted CTF stream into the eclipse CTF trace viewer and got it displayed with the 'Generic CTF trace' type The other type 'LTTng kernel trace' seems to set some rules for the CTF fields/format the trace has to obey. Is this described somewhere? To identify an LTTng kernel trace, we check if the domain defined in the metadata is kernel [1]. I don't think we documented this anywhere though, we probably should! ok, easy enough ;-) so I'm guessing this governs the expected CTF layout for event/stream headers/contexts, right? Also judging from the trace display, you have hardcoded specific displays/actions for specific events? That's all connected/specific under trace type? Once we have some views or analysis specific to perf CTF traces, we could definitely add a separate trace type for those too. I guess tracepoints and breakpoints should display something like the standard kernel trace. As for HW events it's usual to display profile infomation as the perf report does: https://perf.wiki.kernel.org/index.php/Tutorial#Sampling_with_perf_record I tried to record/display lttng event perf:cpu:cycles, but got nothing displayed in eclipse. Looks like this data provides only summary count of the event for the workload? thanks, jirka -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion
On 08/20/2014 05:28 AM, Jiri Olsa wrote: ok, easy enough ;-) so I'm guessing this governs the expected CTF layout for event/stream headers/contexts, right? Correct, if the domain is kernel we then assume that the rest of the trace contains the expected elements of a kernel trace. Of course, one could craft a CTF trace to advertize itself as kernel or ust, and not actually have the layout of that trace type, in which case it would fail parsing later on. Also judging from the trace display, you have hardcoded specific displays/actions for specific events? That's all connected/specific under trace type? Yes the trace type is the main provider of functionality. I could go into more details, but we use Eclipse extension points to define which columns to put in the event table, which views are available, etc. for each supported trace type. Once we have some views or analysis specific to perf CTF traces, we could definitely add a separate trace type for those too. I guess tracepoints and breakpoints should display something like the standard kernel trace. As for HW events it's usual to display profile infomation as the perf report does: https://perf.wiki.kernel.org/index.php/Tutorial#Sampling_with_perf_record Interesting, I haven't tried the perf CTF output yet, but I could see it using the Statistics view (which by default just prints the % of events, per event type) to print the results of the different perf reports, calculated from the CTF events. Eventually with pie charts! I tried to record/display lttng event perf:cpu:cycles, but got nothing displayed in eclipse. Looks like this data provides only summary count of the event for the workload? Just to be sure I understand, you recorded an LTTng kernel trace, in which you enabled the perf:cpu:cycles context? Did this trace display anything in Babeltrace? It should display the same in the Eclipse viewer, the value of the context will be visible in the Contents column in the the table (and in the Properties view), although for now we don't make any use of it. From what I understand, yes, the value of the different perf:* contexts represents the value of the perf counter at the moment the tracepoint was hit. So you cannot get perf data in between trace events when using this method. Cheers, Alexandre -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
