Hello, Alan, Sorry if I'm not right, but this patch is multi-purpose and therefore very big! ;) Can it be splited somehow for more efficient discussion/review? It's ok, if no...
Thanks, Vasily. Alan D. Brunelle wrote: > Hi Jens - > > I had submitted this back at the end of last year, but the patch was > bad - and I promised to redo it. So, here's another try. > > Alan > ------------------------------------------------------------------------ > > Lots of changes to how we handle traces - adds robustness & quicker > > From: Alan D. Brunelle <[EMAIL PROTECTED]> > Signed-off-by: Alan D. Brunelle <[EMAIL PROTECTED]> > > This large patch contains the following changes to the trace handling > aspects of btt: > > 1. Use larger buffers for output options. > > 2. Use mmap to handle the input of trace data. > > 3. More precise btt statistics are output at the end. > > 4. Added in (under DEBUG) the display of unhandled traces. I was running > into the problem where traces were not being connected, and the rb trees > would get quite large. This would slow things down considerably. (See > below for details on why traces weren't being handled.) > > 5. Sprinkled some ASSERTs (under DEBUG). > > 6. Added a new btt-specific trace type: "links" - since 'A' (remaps) > contain two separate pieces of information, I broke them up into a link > and a remap trace. [Thus, it is easy to find either end of the remap.] > > 7. Added in the notion of retries of completes (and requeues). I'm finding > some discrepencies in the time stamps, in order to make btt handle these > better, I've added the notion of keeping the trace around for a bit, > to see if it gets linked up later. > > 8. Separated trace streams into: simple IOs, and remapped IOs. > > 9. Fixed up D2C averages - Q2I + I2D + D2C should equal Q2C averages. > > ---------------------------------------------------------------------------- > I do not understand why it is so, but I am seeing two 'C' (complete) > traces for the same IO track at times. The sequence number is different > (+1 for the second one), and the time stamps are different (100's of > microseconds apart). I'm investigating this. > > At least on an IA64, I am seeing time inconsistencies amongst CPUs on very > heavy loads (48 disks, 4 CPUs, almost 300 million traces). I find the 'D' > (issue) and 'C' (complete) traces coming out ahead of the associate 'I' > (insert) and 'M' (merge) traces. It would be good to get this fixed in > the kernel, but I figure it is also goodness to attempt to account for > it in post-processing as well. > > ---------------------------------------------------------------------------- > This work was done in order to handle some of these large data sets, and > I've found that the performance is reasonable - here are some stats for > very large file (the largest of which used to take well over 12 minutes, > now it takes about 5 1/2 minutes - and a lot of that is just getting the > 18GiB of data read in): > > Size Real User System > ----- -------- -------- ------- > 7GiB 123.445s 80.188s 11.392s > 10GiB 179.148s 137.456s 16.680s > 13GiB 237.561s 156.992s 21.968s > 16GiB 283.262s 187.468s 26.748s > 18GiB 336.345s 225.084s 31.200s > --- > > btt/Makefile | 5 + > btt/args.c | 8 +- > btt/bt_timeline.c | 47 +++++++---- > btt/devs.c | 60 ++++++++++++++ > btt/dip_rb.c | 5 + > btt/globals.h | 86 +++++++++++--------- > btt/inlines.h | 214 > +++++++++++++++++++++++++++++++++++++++++++------- > btt/misc.c | 34 -------- > btt/mmap.c | 144 ++++++++++++++++++++++++++++++++++ > btt/trace.c | 106 +++++++++++-------------- > btt/trace_complete.c | 204 ++++++++++++++++++++++++------------------------ > btt/trace_im.c | 109 ++++++++++++++++++------- > btt/trace_issue.c | 121 ++++++++++++++-------------- > btt/trace_queue.c | 83 ++++++++----------- > btt/trace_remap.c | 147 ++++++++++++++++------------------ > btt/trace_requeue.c | 50 ++++-------- > 16 files changed, 876 insertions(+), 547 deletions(-) > > - To unsubscribe from this list: send the line "unsubscribe linux-btrace" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
