Re: AW: some XSLT benchmarks
(cross posted to xalan-dev) > But can you use XSLTC with SAX Events? Yes. Some quick responses to Stefano's note... > 5) i haven't tested Xalan Translets, which, along with compiled XML > might be *the* way to go for Cocoon production environments Yes, I think so. I'm surprised that people on the Cocoon side haven't done more experimenting with Translets. Remember that XSLTC now implements that jaxp.xml.transformer interface, so you should be able to Run with them fairly easily. > 2) XT is still spectacularly fast, even if abandoned a long time ago But not XSLT 1.0 complient... the edge cases in the spec do indeed eat cycles. One thing I do is compile XT on the same compiler that Xalan is compiled with, and use it with Xerces. (Also, with SAXON, use it with Xerces). > is still *way* faster than Xalan (twice as fast, normally) > worries me very much. Yes. Believe me that I've spent a lot of time both with the DataPower benchmarks and in JProbe. Some of the difference is in the serializers (I don't think Cocoon is using Xalan's serializers anyway), which I've been working on. Some of the difference is probably in source tree construction. Probably a major difference is in the instrumentation that Xalan provides, and the state tracking that it does. I'm working at improving this particular issue by using derived classes when instrumentation is needed. Some other performance improvements that *haven't* been checked in yet or are on a branch: 1) General reduction of state tracking 2) Reduction of redundant expressions (4x improvement on one of the Cocoon files that Davinum gave us, but fairly little diff on DataPower benchmarks). 3) Serialization improvements A major issue with Xalan is the source tree representation, which is currently the DTM. XSLTC uses a format very similar to the DTM (the current DTM is partly copied from XSLTC, with development ongoing Xalan interpretive and compiled to use the same source tree). Even SAXON has a DTM-like source tree, originally copied from the Xalan notion (I'm pretty sure). But the DTM has some serious drawbacks too (subtree pruning is hard, and array access in Java tends to be a bit slow). > Flamesuit on, fire at need. Not interested in flame wars. If you would like to discuss the technical details of what's going on, I would be glad to. The issues aren't simple, though the results and the perception of the results are. -scott Jörn Heid n.de>cc: (bcc: Scott Boag/Cambridge/IBM) Subject: AW: some XSLT benchmarks 02/17/2002 06:25 PM Please respond to cocoon-dev Electric XML is not SAX or DOM based. For that it won't make sense I think. But XSLTC will be a nice alternative. But can you use XSLTC with SAX Events? -Ursprüngliche Nachricht- Von: Ivanov, Ivelin [mailto:[EMAIL PROTECTED]] Gesendet: Sonntag, 17. Februar 2002 23:21 An: 'Stefano Mazzocchi '; 'Apache Cocoon ' Cc: 'Apache Xalan ' Betreff: RE: some XSLT benchmarks Thесе benchmarkс make it clear that Xalan J is far from the winner. For completeness however, wouldn't it be fare to include XSLTC as well. Also Electric XML is a fast free parser which should probably be taken into account. http://www.themindelectric.com/products/xml/benchmarks.html Regards, Ivelin -Original Message- From: Stefano Mazzocchi To: Apache Cocoon Cc: Apache Xalan Sent: 16.2.2002 a. 10:58 Subject: some XSLT benchmarks [crossposting on xalan-dev since they might be interested in these results] I want to have numbers to know whether or not a native implementation of an XSLT transformer for Cocoon would make sense, so I did some benchmarks. I used XSLTMark and rerun the test on my machine (old lapto
Re: AW: some XSLT benchmarks
Stefano, A new xmdrivers.jar is attached with an updated drived for XSLTC. You can now run your tests again. You'll need a bunch of jars in "runxsltmark" scripts: XSLTCDriver) CLASSPATH=$WORKDIR/third-party/xalan-j/bin/BCEL.jar CLASSPATH=$WORKDIR/third-party/xalan-j/bin/xsltc.jar:$CLASSPATH CLASSPATH=$WORKDIR/third-party/xalan-j/bin/java_cup.jar:$CLASSPATH CLASSPATH=$WORKDIR/third-party/xalan-j/bin/JLex.jar:$CLASSPATH CLASSPATH=$WORKDIR/third-party/xalan-j/bin/regexp.jar:$CLASSPATH CLASSPATH=$WORKDIR/third-party/xalan-j/bin/runtime.jar:$CLASSPATH ;; (regexp is from jakarta) You may also need to modify default.conf in 'testcases' dir. For instance I have disabled 'dbonerow' which a very long time for translets (and I don't yet know why). If you have any trouble with running the benchmark send me email. --Jacek On Monday 18 February 2002 10:22 am, [EMAIL PROTECTED] wrote: > (cross posted to xalan-dev) > > > But can you use XSLTC with SAX Events? > > Yes. > > Some quick responses to Stefano's note... > > > 5) i haven't tested Xalan Translets, which, along with compiled XML > > might be *the* way to go for Cocoon production environments > > Yes, I think so. I'm surprised that people on the Cocoon side haven't done > more experimenting with Translets. > > Remember that XSLTC now implements that jaxp.xml.transformer interface, so > you should be able to Run with them fairly easily. > > > 2) XT is still spectacularly fast, even if abandoned a long time ago > > But not XSLT 1.0 complient... the edge cases in the spec do indeed eat > cycles. > > One thing I do is compile XT on the same compiler that Xalan is compiled > with, and use it with Xerces. (Also, with SAXON, use it with Xerces). > > > is still *way* faster than Xalan (twice as fast, normally) > > worries me very much. > > Yes. Believe me that I've spent a lot of time both with the DataPower > benchmarks and in JProbe. Some of the difference is in the serializers (I > don't think Cocoon is using Xalan's serializers anyway), which I've been > working on. Some of the difference is probably in source tree > construction. Probably a major difference is in the instrumentation that > Xalan provides, and the state tracking that it does. I'm working at > improving this particular issue by using derived classes when > instrumentation is needed. > > Some other performance improvements that *haven't* been checked in yet or > are on a branch: > 1) General reduction of state tracking > 2) Reduction of redundant expressions (4x improvement on one of the Cocoon > files that Davinum gave us, but fairly little diff on DataPower > benchmarks). > 3) Serialization improvements > > A major issue with Xalan is the source tree representation, which is > currently the DTM. XSLTC uses a format very similar to the DTM (the > current DTM is partly copied from XSLTC, with development ongoing Xalan > interpretive and compiled to use the same source tree). Even SAXON has a > DTM-like source tree, originally copied from the Xalan notion (I'm pretty > sure). But the DTM has some serious drawbacks too (subtree pruning is > hard, and array access in Java tends to be a bit slow). > > > Flamesuit on, fire at need. > > Not interested in flame wars. If you would like to discuss the technical > details of what's going on, I would be glad to. The issues aren't simple, > though the results and the perception of the results are. > > -scott > > > > > > Jörn Heid ><[EMAIL PROTECTED]> n.de>cc: (bcc: Scott > Boag/Cambridge/IBM) Subject: AW: some XSLT benchmarks 02/17/2002 06:25 > PM > Please respond to > cocoon-dev > > > > > > > Electric XML is not SAX or DOM based. For that it won't make sense I think. > But XSLTC will be a nice alternative. But can you use XSLTC with SAX > Events? > > -Ursprüngliche Nachricht- > Von: Ivanov, Ivelin [mailto:[EMAIL PROTECTED]] > Gesendet: Sonntag, 17. Februar 2002 23:21 > An: 'Stefano Mazzocchi '; 'Apache Cocoon ' > Cc: 'Apache Xalan ' > Betreff: RE: some XSLT benchmarks > > > > Thесе benchmarkс make it clear that Xalan J is far > from the winner. > For completeness however, wouldn't it be fare to include XSLTC as well. > Also Electric XML is a fast free parser which should probably be taken into > account. http://www.themindelectric.com/products/xml/benchmarks.html > > > Regards, > > Ivelin > > -Original Message- > From: Stefano Mazzocchi > To: Apache Cocoon > Cc: Apache Xalan > Sent: 16.2.2002 a. 10:58 > Subject: some XSLT benchmarks > > [crossposting on xalan-dev since they might be interested in these > results] > > I want to have numbers to know whether or not a native implementation of > an XSLT transformer for Cocoon would make sense, so I did some > benchmarks. > > I used XSLTMark and rerun the test on my machine (old lap
Re: AW: some XSLT benchmarks
Scott, FYI, Right now the Translets are unusable for C2 :( http://nagoya.apache.org/bugzilla/show_bug.cgi?id=6312 Thanks, dims --- [EMAIL PROTECTED] wrote: > > 5) i haven't tested Xalan Translets, which, along with compiled XML > > might be *the* way to go for Cocoon production environments > > Yes, I think so. I'm surprised that people on the Cocoon side haven't done > more experimenting with Translets. > > Remember that XSLTC now implements that jaxp.xml.transformer interface, so > you should be able to Run with them fairly easily. > = Davanum Srinivas - http://jguru.com/dims/ __ Do You Yahoo!? Yahoo! Sports - Coverage of the 2002 Olympic Games http://sports.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
Re: AW: some XSLT benchmarks
"Jacek R. Ambroziak" wrote: > > Stefano, > > A new xmdrivers.jar is attached with an updated drived for XSLTC. > You can now run your tests again. Ok, ran the tests on my laptop, same hardware/software/condition as before. Results: Xalan 2.3 XSLTC --- Sun 1.3.1 [1]118,09 Sun 1.3.1 [2]257,73 Sun 1.3.1 [3]390,13 Sun 1.4.0 [3]353,04 IBM 1.3.0 [3]345,93 XT -- Sun 1.3.1 [1]241,52 Sun 1.3.1 [2]205,10 Sun 1.3.1 [3]202,33 Sun 1.4.0 [3]193,85 IBM 1.3.0 [3]202,48 MSXML 3.0 - Sun 1.3.1 [1]317,67 Sun 1.3.1 [2]241,40 Sun 1.3.1 [3]222,21 IBM 1.3.0 [3]221,57 Sun 1.4.0 [3]221,67 NOTES: [1] all tests are run [2] all tests but dbonerow (XSLTC is *very* slow on this, about 10 times slower than XT) [3] all tests but dbonerow, decoy, number, patterns The third test is somewhat 'XSLTC-oriented' since I removed the tests where it appears to be unreasonably slower (dbonerow, decoy and patterns) and the one that triggers an exception (number) [please, guys, fix this! I can provide stacktraces if you want] COMMENTS: - XSLTC is *fast*!!! It's even faster than native code! I also tried to see if the test (running in java) was impacting the native performance but the three tests on the different JVMs show that the native code has always the same performance. MSXML gains a lot in dbonerow (10 secs vs. 200 secs, a really huge difference! but I think this is a synchronization issue since the CPU is not going at full speed, or could be a Java I/O problem since it's working on the biggest file) Final result: - All these processors aren't exactly top class as far as compliance goes. If you want a compliant processor use Xalan or Saxon, no way out. But if you want to run *fast* and you don't need to stress every function of XSLT, but perform normal stylesheets (like normally done in cocoon, since some of the XSLT functionality if factored out in other places), XSLTC is *without*any*doubt* the way to go. Interesting enough: Java 1.3 is faster than Java 1.4 (all tests were done with client JVM, since the tests aren't long enough to benefit from the hotspot server VM) and faster than IBM (which is normally 10% faster than SUN 1.3 in almost anything). I think this could be due to the hand-written java assembly code that XSLTC generates. In fact, no decompiler is able to decompile a translet since no javac pattern is recognized... I assume they make extensive use of direct jumping (which is prohibited in java since 'goto' is reserved but not used, while is allowed at the bytecode level. Too bad I don't have time to jump into their code, but should be fun for an old assembly freak like me :) finally able to tune the JVM bytecode by hand :) Seriously: hand-optimized java bytecode it's a dangerous approach since every JVM implementation is different. Ideal should for the XSLTC engine to recognize the JVM it runs in (via system properties) and tune the generated bytecode on the running JVM. I assume this could give some 20/30% more speed improvement, but it's a potentially harmful thing to do since it might duplicate code and requires *lots* of guesses on how the JVM works internally. Anyway, seriously, XSLTC *is* a solution to the XSLT bottleneck problem. Now: only one thing to add: let's make it work on Cocoon. Xalan-guys, please, can you give us a hand there? -- Stefano Mazzocchi One must still have chaos in oneself to be able to give birth to a dancing star. <[EMAIL PROTECTED]> Friedrich Nietzsche - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
Re: AW: some XSLT benchmarks
Stefano Mazzocchi wrote: > "Jacek R. Ambroziak" wrote: > > Ideal should for the XSLTC engine to recognize the JVM it runs in (via > system properties) and tune the generated bytecode on the running JVM. I > assume this could give some 20/30% more speed improvement, but it's a > potentially harmful thing to do since it might duplicate code and > requires *lots* of guesses on how the JVM works internally. > > Anyway, seriously, XSLTC *is* a solution to the XSLT bottleneck problem. > > Now: only one thing to add: let's make it work on Cocoon. > > Xalan-guys, please, can you give us a hand there? My question is this: how does it _scale_. For instance, The ECM is quite resonable with only a few threads, but in a server environment where there can be as many as 150 or more concurrent threads it slows down so badly it is unusable. A fresh approach reaping the benefits of the lessons of the ECM and Phoenix proved that the new approach can handle the load *much* more gracefully. I would like to see the *same* tests with 100 threads each performing 500 translations (yes they can have their own instance of the translet in each thread as is necessary). I would like to see that in comparison to Xalan and Saxon. That is the most important lesson we can learn. Sign Up for NetZero Platinum Today Only $9.95 per month! http://my.netzero.net/s/signup?r=platinum&refcd=PT97 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
Re: AW: some XSLT benchmarks
Stefano, that is cool! Except for the mysterious 'dbonerow'. I will attempt to fix it an in general I am going continue to follow my original vision to make XSLTC a good technology for people to actually use. You are right, there are multiple 'goto's' in the generated bytecodes (although the bytecodes are not hand-optimized, right? :-) but automatically generated), so that translets do not correspond to any valid Java program. Decompiling to FORTRAN would be a different thing :-) This is one of the reasons why I chose to generate bytecodes directly instead of Java code (which would make compilation longer, too). As to you other suggestions... I think a lot of time is spent *around* transformation, ie. first building the (special) DOM, and then serializing (or otherwise using) transformation output. In other words, even if transformation would take exactly 0 ms, we'd see a lot of time going to broadly conceived of input/output. That's why translets shine when the same DOM objects can be reused, perhaps to generate different 'views' of a document, eg. personalize) -- at least building-the-DOM part is factored out. Next I'd like to address 'dbonerow'; the benchmark kills XSLTC :-( so that it ends the race last w/ 10 iterations of the test. --Jacek --- Stefano Mazzocchi <[EMAIL PROTECTED]> wrote: > "Jacek R. Ambroziak" wrote: > > > > Stefano, > > > > A new xmdrivers.jar is attached with an updated > drived for XSLTC. > > You can now run your tests again. > > Ok, ran the tests on my laptop, same > hardware/software/condition as > before. > > Results: > > Xalan 2.3 XSLTC > --- > > Sun 1.3.1 [1]118,09 > Sun 1.3.1 [2]257,73 > Sun 1.3.1 [3]390,13 > Sun 1.4.0 [3]353,04 > IBM 1.3.0 [3]345,93 > > XT > -- > > Sun 1.3.1 [1]241,52 > Sun 1.3.1 [2]205,10 > Sun 1.3.1 [3]202,33 > Sun 1.4.0 [3]193,85 > IBM 1.3.0 [3]202,48 > > MSXML 3.0 > - > > Sun 1.3.1 [1]317,67 > Sun 1.3.1 [2]241,40 > Sun 1.3.1 [3]222,21 > IBM 1.3.0 [3]221,57 > Sun 1.4.0 [3]221,67 > > NOTES: > > [1] all tests are run > [2] all tests but dbonerow (XSLTC is *very* slow on > this, about 10 times > slower than XT) > [3] all tests but dbonerow, decoy, number, patterns > > The third test is somewhat 'XSLTC-oriented' since I > removed the tests > where it appears to be unreasonably slower > (dbonerow, decoy and > patterns) and the one that triggers an exception > (number) [please, guys, > fix this! I can provide stacktraces if you want] > > COMMENTS: > - > > XSLTC is *fast*!!! It's even faster than native > code! > > I also tried to see if the test (running in java) > was impacting the > native performance but the three tests on the > different JVMs show that > the native code has always the same performance. > > MSXML gains a lot in dbonerow (10 secs vs. 200 secs, > a really huge > difference! but I think this is a synchronization > issue since the CPU is > not going at full speed, or could be a Java I/O > problem since it's > working on the biggest file) > > Final result: > - > > All these processors aren't exactly top class as far > as compliance goes. > If you want a compliant processor use Xalan or > Saxon, no way out. > > But if you want to run *fast* and you don't need to > stress every > function of XSLT, but perform normal stylesheets > (like normally done in > cocoon, since some of the XSLT functionality if > factored out in other > places), XSLTC is *without*any*doubt* the way to go. > > Interesting enough: Java 1.3 is faster than Java 1.4 > (all tests were > done with client JVM, since the tests aren't long > enough to benefit from > the hotspot server VM) and faster than IBM (which is > normally 10% faster > than SUN 1.3 in almost anything). > > I think this could be due to the hand-written java > assembly code that > XSLTC generates. In fact, no decompiler is able to > decompile a translet > since no javac pattern is recognized... I assume > they make extensive use > of direct jumping (which is prohibited in java since > 'goto' is reserved > but not used, while is allowed at the bytecode > level. Too bad I don't > have time to jump into their code, but should be fun > for an old assembly > freak like me :) finally able to tune the JVM > bytecode by hand :) > > Seriously: hand-optimized java bytecode it's a > dangerous approach since > every JVM implementation is different. > > Ideal should for the XSLTC engine to recognize the > JVM it runs in (via > system properties) and tune the generated bytecode > on the running JVM. I > assume this could give some 20/30% more speed > improvement, but it's a > potentially harmful thing to do since it might > duplicate code and > requires *lots* of guesses on how the JVM works > internally. > > Anyway, seriously, XSLTC *is* a solution to the XSLT > bottlene
Re: AW: some XSLT benchmarks
--- Berin Loritsch <[EMAIL PROTECTED]> wrote: > Stefano Mazzocchi wrote: > > "Jacek R. Ambroziak" wrote: > > > > Anyway, seriously, XSLTC *is* a solution to the > XSLT bottleneck problem. > > > My question is this: how does it _scale_. > > I would like to see the *same* tests with 100 > threads each performing > 500 translations (yes they can have their own > instance of the translet > in each thread as is necessary). I would like to > see that in comparison > to Xalan and Saxon. > Have you tested Xalan/Saxon/XSLTC in your multithreaded environment? I don't see a priori reasons why translets should scale worse... --Jacek __ Do You Yahoo!? Yahoo! Sports - Coverage of the 2002 Olympic Games http://sports.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
Re: AW: some XSLT benchmarks
Jacek Ambroziak wrote: > --- Berin Loritsch <[EMAIL PROTECTED]> wrote: > >>Stefano Mazzocchi wrote: >> >>>"Jacek R. Ambroziak" wrote: >>> >>> > >>>Anyway, seriously, XSLTC *is* a solution to the >>> >>XSLT bottleneck problem. >> > >>My question is this: how does it _scale_. >> >> > >>I would like to see the *same* tests with 100 >>threads each performing >>500 translations (yes they can have their own >>instance of the translet >>in each thread as is necessary). I would like to >>see that in comparison >>to Xalan and Saxon. >> >> > > Have you tested Xalan/Saxon/XSLTC in your > multithreaded environment? > > I don't see a priori reasons why translets should > scale worse... I have run Xalan and Saxon (As have just about every other Cocooner). For the low volume highly personalized webapps I generally create, it is ok. However, I have seen much room for improvement--server side caching helps. Sign Up for NetZero Platinum Today Only $9.95 per month! http://my.netzero.net/s/signup?r=platinum&refcd=PT97 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
RE: AW: some XSLT benchmarks
> Results: > Xalan 2.3 XSLTC > XT > MSXML 3.0 And what about SAXON and MSXML 4.0 ? How are the results of the benchmarks with these two transformers ? Thanks Peter - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
Re: AW: some XSLT benchmarks
MORAVEK Peter wrote: > > > Results: > > Xalan 2.3 XSLTC > > XT > > MSXML 3.0 > > And what about SAXON and MSXML 4.0 ? How are the results of the benchmarks > with these two transformers ? Saxon was way slower (around 100, check my previous post on the matter) I couldn't test MSXML 4.0 because I didn't have a suitable driver for XSLTMark, if you write one for me I'll redo the tests as well :) -- Stefano Mazzocchi One must still have chaos in oneself to be able to give birth to a dancing star. <[EMAIL PROTECTED]> Friedrich Nietzsche - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
Re: AW: some XSLT benchmarks
No assumptions about JVMs have been made; in fact translets used to run faster on IBM's VMs (esp. 1.1.8) which was a bit embarassing for me as a Sun-er at the time. I am sure translets can be further optimized but JVM tuning would be the last (if at all) place to look at, maybe except for small devices. --Jacek --- Stefano Mazzocchi <[EMAIL PROTECTED]> wrote: > Jacek Ambroziak wrote: > > > > Stefano, > > > > that is cool! Except for the mysterious > 'dbonerow'. I > > will attempt to fix it > > an in general I am going continue to follow my > > original vision > > to make XSLTC a good technology for people to > actually > > use. > > Cool. > > > You are right, there are multiple 'goto's' in the > > generated bytecodes > > (although the bytecodes are not hand-optimized, > right? > > :-) > > but automatically generated) > > yes, automatically generated, but the 'XSLT to > bytecode' patterns have > been manually crafted, right? and I'm pretty sure > that you guys crafted > them based on the assumption on how the underlying > JVM was going to > interpret them. Which shows why it is faster on the > Sun 1.3.1 JVM (but > this is just my hypotesis) > > My point was: it would be cool to implement a way > for the 'XSLT 2 > bytecode' patterns to be choosen based on the JVM > system that is > currently running (or using a configurations, for > those translets that > have to be deployed on platforms different from > where the translet > assembly takes place) > > >, so that translets do not > > correspond > > to any valid Java program. Decompiling to FORTRAN > > would be a different > > thing :-) This is one of the reasons why I chose > to > > generate bytecodes > > directly instead of Java code (which would make > > compilation longer, too). > > Yes, this is a very intriguing technology and I > think it would be a > valuable tool for speed-optimizing java code for > specific cases. > > > As to you other suggestions... I think a lot of > time > > is spent *around* > > transformation, ie. first building the (special) > DOM, > > and then > > serializing (or otherwise using) transformation > > output. > > In other words, even if transformation would take > > exactly 0 ms, > > we'd see a lot of time going to broadly conceived > of > > input/output. > > That's why translets shine when the same DOM > objects > > can be reused, > > perhaps to generate different 'views' of a > document, > > eg. personalize) > > -- at least building-the-DOM part is factored out. > > Good point. > > > Next I'd like to address 'dbonerow'; the benchmark > > kills XSLTC :-( > > so that it ends the race last w/ 10 iterations of > the > > test. > > Please look also into the 'numbers' test that throws > an exception > also you might want to run the test to see why there > are some invalid > results out of the transformation (compliancy is > less than Xalan's) > > -- > Stefano Mazzocchi One must still have chaos in > oneself to be > able to give birth to a > dancing star. > <[EMAIL PROTECTED]> > Friedrich Nietzsche > > > __ Do You Yahoo!? Yahoo! Sports - Coverage of the 2002 Olympic Games http://sports.yahoo.com
Re: AW: some XSLT benchmarks
Jacek Ambroziak wrote: > > Stefano, > > that is cool! Except for the mysterious 'dbonerow'. I > will attempt to fix it > an in general I am going continue to follow my > original vision > to make XSLTC a good technology for people to actually > use. Cool. > You are right, there are multiple 'goto's' in the > generated bytecodes > (although the bytecodes are not hand-optimized, right? > :-) > but automatically generated) yes, automatically generated, but the 'XSLT to bytecode' patterns have been manually crafted, right? and I'm pretty sure that you guys crafted them based on the assumption on how the underlying JVM was going to interpret them. Which shows why it is faster on the Sun 1.3.1 JVM (but this is just my hypotesis) My point was: it would be cool to implement a way for the 'XSLT 2 bytecode' patterns to be choosen based on the JVM system that is currently running (or using a configurations, for those translets that have to be deployed on platforms different from where the translet assembly takes place) >, so that translets do not > correspond > to any valid Java program. Decompiling to FORTRAN > would be a different > thing :-) This is one of the reasons why I chose to > generate bytecodes > directly instead of Java code (which would make > compilation longer, too). Yes, this is a very intriguing technology and I think it would be a valuable tool for speed-optimizing java code for specific cases. > As to you other suggestions... I think a lot of time > is spent *around* > transformation, ie. first building the (special) DOM, > and then > serializing (or otherwise using) transformation > output. > In other words, even if transformation would take > exactly 0 ms, > we'd see a lot of time going to broadly conceived of > input/output. > That's why translets shine when the same DOM objects > can be reused, > perhaps to generate different 'views' of a document, > eg. personalize) > -- at least building-the-DOM part is factored out. Good point. > Next I'd like to address 'dbonerow'; the benchmark > kills XSLTC :-( > so that it ends the race last w/ 10 iterations of the > test. Please look also into the 'numbers' test that throws an exception also you might want to run the test to see why there are some invalid results out of the transformation (compliancy is less than Xalan's) -- Stefano Mazzocchi One must still have chaos in oneself to be able to give birth to a dancing star. <[EMAIL PROTECTED]> Friedrich Nietzsche - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
Re: AW: some XSLT benchmarks
Santiago Pericas wrote: > > > yes, automatically generated, but the 'XSLT to bytecode' patterns have > > been manually crafted, right? and I'm pretty sure that you guys crafted > > them based on the assumption on how the underlying JVM was going to > > interpret them. Which shows why it is faster on the Sun 1.3.1 JVM (but > > this is just my hypotesis) > > I think this is more of a coincidence. Ah > Translets try to allocate as few > objects as possible. Other JVM's may be better optimized for object > creation/initialization. Sun's, on the other hand, does a pretty good > job finding the "hot spots" and this could be the reason why translets > run fast. yes, the 'bytecode tight loops' that translets have might really use the power of hotspot. > As to why 1.4 is slower? This is puzzling. This is the reason why I didn't think of hotspot vs. IBM JIT. ASAIK, 1.4 uses the same code (tuned) of 1.3.1 > The first thing that comes > to mind is the new assertion mechanism but, as far as I know, it is > completely turned off by default (or isn't?). They should be. -- Stefano Mazzocchi One must still have chaos in oneself to be able to give birth to a dancing star. <[EMAIL PROTECTED]> Friedrich Nietzsche - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
Re: AW: some XSLT benchmarks
You guys may find another data point interesting... (I am using Sun's newest 1.4 on Linux) I reran the test with '-verbosegc' flag gathering GC info into files. and then counted Full garbage collections grep Full gcXSLTC | wc -l grep Full gcXT| wc -l and so on. The results: XSLTC21 XT 140 Saxon 308 Xajan2J 349 In terms of total GCs (full & incr.) 2177 gcXT 3466 gcXSLTC 3931 gcXalanJ 4562 gcSaxon Ratios full GCs/all GCs: XSLTC 0.006 XT 0.064 Saxon 0.067 XalanJ 0.089 Incemental GCc (for which JVMs optimize) are about 2 orders of magnitude faster, so It Is A Good Thing :-) if the majority of GC work can be taken care of via incremental GC. The fewer GCs improve performance in a number of ways. Especially 1) when memory is scarce anyway: PDA, cell phone 2) when on a server XSLT is put to truly heavy duty transformation work (heavy load) --Jacek __ Do You Yahoo!? Yahoo! Sports - Coverage of the 2002 Olympic Games http://sports.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
Re: AW: some XSLT benchmarks
Yes, I agree with Berin on this, though I also agree with Jacek that there's little reason that it should not scale well. Another factor is "incremental" output, which Xalan interpretive does a lot of work to do well (and tends to take penalty for), and XSLTC may have a much harder time at. On the other hand, especially given cacheing, incrementality may not matter at all. On the other hand, given Cocoon pipelines, it may matter a lot. (Hopefully, XSLTC can eventually be given incremental capabilities... though certainly not at the expense of any performance). I would like to eventually see much more sophisticated benchmarks than XSLTMark, which I think only tells about 20% of the performance story. -scott Berin Loritsch cc: "Jacek R. Ambroziak" <[EMAIL PROTECTED]>, [EMAIL PROTECTED], [EMAIL PROTECTED], Santiago Pericas <[EMAIL PROTECTED]> 02/19/2002 11:22 Subject: Re: AW: some XSLT benchmarks PM Please respond to xalan-dev Stefano Mazzocchi wrote: > "Jacek R. Ambroziak" wrote: > > Ideal should for the XSLTC engine to recognize the JVM it runs in (via > system properties) and tune the generated bytecode on the running JVM. I > assume this could give some 20/30% more speed improvement, but it's a > potentially harmful thing to do since it might duplicate code and > requires *lots* of guesses on how the JVM works internally. > > Anyway, seriously, XSLTC *is* a solution to the XSLT bottleneck problem. > > Now: only one thing to add: let's make it work on Cocoon. > > Xalan-guys, please, can you give us a hand there? My question is this: how does it _scale_. For instance, The ECM is quite resonable with only a few threads, but in a server environment where there can be as many as 150 or more concurrent threads it slows down so badly it is unusable. A fresh approach reaping the benefits of the lessons of the ECM and Phoenix proved that the new approach can handle the load *much* more gracefully. I would like to see the *same* tests with 100 threads each performing 500 translations (yes they can have their own instance of the translet in each thread as is necessary). I would like to see that in comparison to Xalan and Saxon. That is the most important lesson we can learn. Sign Up for NetZero Platinum Today Only $9.95 per month! http://my.netzero.net/s/signup?r=platinum&refcd=PT97 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
Re: AW: some XSLT benchmarks
> yes, automatically generated, but the 'XSLT to bytecode' patterns have > been manually crafted, right? and I'm pretty sure that you guys crafted > them based on the assumption on how the underlying JVM was going to > interpret them. Which shows why it is faster on the Sun 1.3.1 JVM (but > this is just my hypotesis) I think this is more of a coincidence. Translets try to allocate as few objects as possible. Other JVM's may be better optimized for object creation/initialization. Sun's, on the other hand, does a pretty good job finding the "hot spots" and this could be the reason why translets run fast. As to why 1.4 is slower? This is puzzling. The first thing that comes to mind is the new assertion mechanism but, as far as I know, it is completely turned off by default (or isn't?). -- Santiago Pericas - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
Re: AW: some XSLT benchmarks
Santiago Pericas wrote: >> yes, automatically generated, but the 'XSLT to bytecode' patterns have >> been manually crafted, right? and I'm pretty sure that you guys crafted >> them based on the assumption on how the underlying JVM was going to >> interpret them. Which shows why it is faster on the Sun 1.3.1 JVM (but >> this is just my hypotesis) > > > I think this is more of a coincidence. Translets try to allocate as few > objects as possible. Other JVM's may be better optimized for object > creation/initialization. Sun's, on the other hand, does a pretty good > job finding the "hot spots" and this could be the reason why translets > run fast. > As to why 1.4 is slower? This is puzzling. The first thing that comes > to mind is the new assertion mechanism but, as far as I know, it is > completely turned off by default (or isn't?). Yes it is. It is also only compiled in if you use the -source 1.4 parameter on javac. -- "They that give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." - Benjamin Franklin - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
Re: AW: some XSLT benchmarks
[EMAIL PROTECTED] wrote: > > Yes, I agree with Berin on this, though I also agree with Jacek that > there's little reason that it should not scale well. > > Another factor is "incremental" output, which Xalan interpretive does a lot > of work to do well (and tends to take penalty for), and XSLTC may have a > much harder time at. On the other hand, especially given cacheing, > incrementality may not matter at all. On the other hand, given Cocoon > pipelines, it may matter a lot. > > (Hopefully, XSLTC can eventually be given incremental capabilities... > though certainly not at the expense of any performance). > > I would like to eventually see much more sophisticated benchmarks than > XSLTMark, which I think only tells about 20% of the performance story. Well, this is the same old tune as for all benchmarks. Give me a more sofisticated one and I'll be happy to run the tests again. But one thing is for sure, XSLTC is way faster than Xalan and, given choice, I would use the first for performance-critical applications. No offense, of course, just the plain facts. -- Stefano Mazzocchi One must still have chaos in oneself to be able to give birth to a dancing star. <[EMAIL PROTECTED]> Friedrich Nietzsche - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
Re: AW: some XSLT benchmarks
> But one thing is for sure, XSLTC is way faster than Xalan and, given
> choice, I would use the first for performance-critical applications.
>
> No offense, of course, just the plain facts.
Please, Stefano. No offense taken, and I was not trying to be defensive.
I agree 100% that XSLTC is an ideal choice for high performance
applications. If I didn't think so it wouldn't have been brought into the
Xalan project. We put a lot of our hopes into XSLTC. I agree 100% with
your conclusions.
But we need to have discussions about incrementality, large document size,
and a few other issues that XSLTC may have without getting into
us-versus-them or whatever.
> Well, this is the same old tune as for all benchmarks. Give me a more
> sofisticated one and I'll be happy to run the tests again.
Sigh. I'm not criticizing the results of the DataPower/XSLTMark
benchmarks... they tell an important story and I've been running them since
they first appeared.
-scott
Stefano Mazzocchi
<[EMAIL PROTECTED]To: [EMAIL PROTECTED]
rg> cc: [EMAIL PROTECTED],
(bcc: Scott Boag/Cambridge/IBM)
Subject: Re: AW: some XSLT benchmarks
02/20/2002 02:01
PM
Please respond to
cocoon-dev
[EMAIL PROTECTED] wrote:
>
> Yes, I agree with Berin on this, though I also agree with Jacek that
> there's little reason that it should not scale well.
>
> Another factor is "incremental" output, which Xalan interpretive does a
lot
> of work to do well (and tends to take penalty for), and XSLTC may have a
> much harder time at. On the other hand, especially given cacheing,
> incrementality may not matter at all. On the other hand, given Cocoon
> pipelines, it may matter a lot.
>
> (Hopefully, XSLTC can eventually be given incremental capabilities...
> though certainly not at the expense of any performance).
>
> I would like to eventually see much more sophisticated benchmarks than
> XSLTMark, which I think only tells about 20% of the performance story.
Well, this is the same old tune as for all benchmarks. Give me a more
sofisticated one and I'll be happy to run the tests again.
But one thing is for sure, XSLTC is way faster than Xalan and, given
choice, I would use the first for performance-critical applications.
No offense, of course, just the plain facts.
--
Stefano Mazzocchi One must still have chaos in oneself to be
able to give birth to a dancing star.
<[EMAIL PROTECTED]> Friedrich Nietzsche
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, email: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, email: [EMAIL PROTECTED]
Fwd: Re: AW: some XSLT benchmarks
Note: forwarded message attached. __ Do You Yahoo!? Yahoo! Sports - Coverage of the 2002 Olympic Games http://sports.yahoo.com --- Begin Message --- No assumptions about JVMs have been made; in fact translets used to run faster on IBM's VMs (esp. 1.1.8) which was a bit embarassing for me as a Sun-er at the time. I am sure translets can be further optimized but JVM tuning would be the last (if at all) place to look at, maybe except for small devices. --Jacek --- Stefano Mazzocchi <[EMAIL PROTECTED]> wrote: > Jacek Ambroziak wrote: > > > > Stefano, > > > > that is cool! Except for the mysterious > 'dbonerow'. I > > will attempt to fix it > > an in general I am going continue to follow my > > original vision > > to make XSLTC a good technology for people to > actually > > use. > > Cool. > > > You are right, there are multiple 'goto's' in the > > generated bytecodes > > (although the bytecodes are not hand-optimized, > right? > > :-) > > but automatically generated) > > yes, automatically generated, but the 'XSLT to > bytecode' patterns have > been manually crafted, right? and I'm pretty sure > that you guys crafted > them based on the assumption on how the underlying > JVM was going to > interpret them. Which shows why it is faster on the > Sun 1.3.1 JVM (but > this is just my hypotesis) > > My point was: it would be cool to implement a way > for the 'XSLT 2 > bytecode' patterns to be choosen based on the JVM > system that is > currently running (or using a configurations, for > those translets that > have to be deployed on platforms different from > where the translet > assembly takes place) > > >, so that translets do not > > correspond > > to any valid Java program. Decompiling to FORTRAN > > would be a different > > thing :-) This is one of the reasons why I chose > to > > generate bytecodes > > directly instead of Java code (which would make > > compilation longer, too). > > Yes, this is a very intriguing technology and I > think it would be a > valuable tool for speed-optimizing java code for > specific cases. > > > As to you other suggestions... I think a lot of > time > > is spent *around* > > transformation, ie. first building the (special) > DOM, > > and then > > serializing (or otherwise using) transformation > > output. > > In other words, even if transformation would take > > exactly 0 ms, > > we'd see a lot of time going to broadly conceived > of > > input/output. > > That's why translets shine when the same DOM > objects > > can be reused, > > perhaps to generate different 'views' of a > document, > > eg. personalize) > > -- at least building-the-DOM part is factored out. > > Good point. > > > Next I'd like to address 'dbonerow'; the benchmark > > kills XSLTC :-( > > so that it ends the race last w/ 10 iterations of > the > > test. > > Please look also into the 'numbers' test that throws > an exception > also you might want to run the test to see why there > are some invalid > results out of the transformation (compliancy is > less than Xalan's) > > -- > Stefano Mazzocchi One must still have chaos in > oneself to be > able to give birth to a > dancing star. > <[EMAIL PROTECTED]> > Friedrich Nietzsche > > > __ Do You Yahoo!? Yahoo! Sports - Coverage of the 2002 Olympic Games http://sports.yahoo.com --- End Message --- - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
