What's the difference between runs? Do you start a new process or are those repeated executions on the same cluster or are those different documents parsed in the same Hyracks job? What would the 3rd run look like? I'm wondering if we see a start-up cost here or if the numbers are just inconsistent.
Also, at some level the difference between the first run of the original and the alternate seems to make sense, as we spend more time building up additional structures and less time writing (copying). Would you agree? On Thu, Feb 20, 2014 at 12:05 PM, Eldon Carman <[email protected]> wrote: > Yes, the traverses are how you described it. > > The profiler can very form run to run. Here are the numbers from the last > few snapshots. > > Original > 45% parsing > 30% writing > > Alternate 1st run > 50% parsing > 25% writing > > Alternate 2nd run > 30% parsing > 30% writing > > > On Thu, Feb 20, 2014 at 10:39 AM, Till Westmann <[email protected]> > wrote: > > > > > On Feb 20, 2014, at 10:26 AM, Eldon Carman <[email protected]> wrote: > > > > > Here are some of my observations: > > > > > > Things that do NOT change between implementations: > > > > > > -------------------------------------------------------------------------------- > > > AbstractSAXParser parsing time (45 and 50% of the total time) > > > Writing information with ByteArrayAccessibleOutputStream (25 and 30% of > > the > > > total time) > > > Dictionary Creation (small amount of time) > > > > Are the higher percentages for the new implementation or the old one? > > > > > Things that DO change: > > > > > > -------------------------------------------------------------------------------- > > > The alternate method must traverse the nodes twice. The original did > this > > > in one pass. > > > > When you say it traverses twice, you mean that it > > a) traverses the nodes using the events form the parser once and that it > > b) traverses the new intermediate structures a second time? > > > > Thanks, > > Till > > > > > The alternate method makes fewer copies (30% less). > > > The alternate stores additional structure details for later processing. > > > > > > > > > The rest of the programming is basically shuffled around to make the > > > alternate process work. > > > > > > On Wed, Feb 19, 2014 at 6:13 PM, Till Westmann <[email protected]> > > wrote: > > > > > >> Ok, so this is actually quite a bit slower. > > >> Do you see an indication what is slow about it? > > >> > > >> Till > > >> > > >> > > >> On Wed, Feb 19, 2014 at 5:54 PM, Eldon Carman <[email protected]> > wrote: > > >> > > >>> The alternative SAX content handler does fewer copies by ~30%. Yet > the > > >>> reduce copy time does not save more than the introduced overhead to > > track > > >>> nodes for the single copy. > > >>> > > >>> Deep XML Tree > > >>> original 228 copies > > >>> alternative 167 copies > > >>> > > >>> Micro XML Tree > > >>> original 30 copies > > >>> alternative 19 copies > > >>> > > >>> The alternate query times: > > >>> > > >>> Whole Execution Time (start up, compile, 10 queries executed) > > >>> Time in minutes and seconds. > > >>> > > >>> Query q00 (500mb) > > >>> --------------- > > >>> 4m24.469s VXQuery - 1 partition > > >>> 2m26.115s VXQuery - 2 partitions > > >>> 1m34.063s VXQuery - 4 partitions > > >>> > > >>> Average Execution time (average of 4-10 queries executed) > > >>> Time in milliseconds. > > >>> > > >>> Query q01 (500mb) > > >>> --------------- > > >>> 24,724ms VXQuery - 1 partition > > >>> 12,727ms VXQuery - 2 partitions > > >>> 8,364ms VXQuery - 4 partitions > > >>> > > >> > > > > >
