[lttng-dev] Processing large trace files

Hello All, I was wondering if anyone had any suggestions on how to speedup trace processing. Right now I am using babeltrace in python to read the traces and process them. Some of the traces I am dealing with reach up to 10GB. I read the trace, extract the info I need and write them to a csv gzip

Re: [lttng-dev] Processing large trace files

Hi Shehab, On Mon, Jul 09, 2018 at 06:48:05PM -0400, Shehab Elsayed wrote: > Hello All, > > I was wondering if anyone had any suggestions on how to speedup trace > processing. Right now I am using babeltrace in python to read the traces > and process them. Some of the traces I am dealing with rea

Re: [lttng-dev] Processing large trace files

I haven't tried the C API yet. A smaller trace of about 2.7GB took +5 hrs before I had to kill the process. As for trace cutting, that sounds promising. How broken is it in babeltrace 2? I mean, can I use it to some extent or is it not functioning at all? Thanks! Shehab Y. Elsayed, MSc. PhD Stu

Re: [lttng-dev] Processing large trace files

On Tue, Jul 10, 2018 at 11:33 AM Shehab Elsayed wrote: > > I haven't tried the C API yet. > > A smaller trace of about 2.7GB took +5 hrs before I had to kill the process. Wow. The Python bindings are slow by nature, but that's a lot of time! > > As for trace cutting, that sounds promising. How b

Re: [lttng-dev] Processing large trace files

Thank you very much, Phil. I will give them a try and see if they help Shehab Y. Elsayed, MSc. PhD Student The Edwards S. Rogers Sr. Dept. of Electrical and Computer Engineering University of Toronto E-mail: shehaby...@gmail.com O