Yes, the traverses are how you described it. The profiler can very form run to run. Here are the numbers from the last few snapshots.
Original 45% parsing 30% writing Alternate 1st run 50% parsing 25% writing Alternate 2nd run 30% parsing 30% writing On Thu, Feb 20, 2014 at 10:39 AM, Till Westmann <[email protected]> wrote: > > On Feb 20, 2014, at 10:26 AM, Eldon Carman <[email protected]> wrote: > > > Here are some of my observations: > > > > Things that do NOT change between implementations: > > > -------------------------------------------------------------------------------- > > AbstractSAXParser parsing time (45 and 50% of the total time) > > Writing information with ByteArrayAccessibleOutputStream (25 and 30% of > the > > total time) > > Dictionary Creation (small amount of time) > > Are the higher percentages for the new implementation or the old one? > > > Things that DO change: > > > -------------------------------------------------------------------------------- > > The alternate method must traverse the nodes twice. The original did this > > in one pass. > > When you say it traverses twice, you mean that it > a) traverses the nodes using the events form the parser once and that it > b) traverses the new intermediate structures a second time? > > Thanks, > Till > > > The alternate method makes fewer copies (30% less). > > The alternate stores additional structure details for later processing. > > > > > > The rest of the programming is basically shuffled around to make the > > alternate process work. > > > > On Wed, Feb 19, 2014 at 6:13 PM, Till Westmann <[email protected]> > wrote: > > > >> Ok, so this is actually quite a bit slower. > >> Do you see an indication what is slow about it? > >> > >> Till > >> > >> > >> On Wed, Feb 19, 2014 at 5:54 PM, Eldon Carman <[email protected]> wrote: > >> > >>> The alternative SAX content handler does fewer copies by ~30%. Yet the > >>> reduce copy time does not save more than the introduced overhead to > track > >>> nodes for the single copy. > >>> > >>> Deep XML Tree > >>> original 228 copies > >>> alternative 167 copies > >>> > >>> Micro XML Tree > >>> original 30 copies > >>> alternative 19 copies > >>> > >>> The alternate query times: > >>> > >>> Whole Execution Time (start up, compile, 10 queries executed) > >>> Time in minutes and seconds. > >>> > >>> Query q00 (500mb) > >>> --------------- > >>> 4m24.469s VXQuery - 1 partition > >>> 2m26.115s VXQuery - 2 partitions > >>> 1m34.063s VXQuery - 4 partitions > >>> > >>> Average Execution time (average of 4-10 queries executed) > >>> Time in milliseconds. > >>> > >>> Query q01 (500mb) > >>> --------------- > >>> 24,724ms VXQuery - 1 partition > >>> 12,727ms VXQuery - 2 partitions > >>> 8,364ms VXQuery - 4 partitions > >>> > >> > >
