Tom:- Did you try xt.exe vrs Xalan vrs XSLTC ?
Best Regards -patrick -----Original Message----- From: Tom Amiro [mailto:[EMAIL PROTECTED]] Sent: Thursday, July 18, 2002 2:50 PM To: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: XSLTMark benchmark results Xalan vs XSLTC Hi, I ran the XSLTMark benchmark with the latest versions of Xalan and XSLTC off the main trunk at Apache and got results that can't be true, or reflect a servious regression in Xalan's performance. In short, I get KB's/sec Xalan 2.5 XSLTC 7161 Something is really wrong somewhere. Here's the detail. ************************************************************* XALAN ************************************************************* XSLTMark 2.0 (Feb 28 2001) running on SunOS 5.8 sparc Java Runtime Environment is Sun Microsystems Inc. 1.3.1_03 Current time is 14:17:07 EDT, 18 Jul 2002 loading test driver: XalanJDriver... Loaded Xalan2JDriver No configurations specified, using default Reading configuration from testcases/default.conf dummy initialization run: xslbench3: xslbench3.xsl xslbenchdream.xml ... done in 3160ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found benchmark run: alphabetize: alphabetize.xsl db100.xml ... done in 52170ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found attsets: attsets.xsl chart.xml ... done in 21806ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found avts: avts.xsl db100.xml ... done in 37321ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found axis: axis.xsl axis.xml ... done in 22164ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found backwards: backwards.xsl game.xml ... done in 26767ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found bottles: bottles.xsl bottles.xml ... done in 34419ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found breadth: find.xsl breadth.xml ... done in 23475ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found brutal: brutal.xsl brutal.xml ... done in 22260ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found chart: chart.xsl chart.xml ... done in 21782ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found creation: creation.xsl db100.xml ... done in 33204ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found current: current.xsl current.xml ... done in 19765ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found dbonerow: dbonerow.xsl db10000.xml ... done in 96359ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found dbtail: dbtail.xsl db100.xml ... done in 23368ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found decoy: decoy.xsl db100.xml ... done in 56668ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found depth: find.xsl depth.xml ... done in 20518ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found encrypt: encrypt.xsl db100.xml ... done in 4607ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found functions: functions.xsl db100.xml ... done in 54484ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found game: game.xsl game.xml ... done in 18754ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found html: html.xsl html.xml ... done in 17782ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found identity: identity.xsl db1000.xml ... done in 18522ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found inventory: inventory.xsl inventory.xml ... done in 21633ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found metric: metric.xsl metric.xml ... done in 23545ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found number: number.xsl number.xml ... done in 20917ms. oddtemplate: oddtemplate.xsl oddtemplate.xml ... done in 20029ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found patterns: patterns.xsl db100.xml ... done in 47092ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found prettyprint: prettyprint.xsl db100.xml ... done in 12658ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found priority: priority.xsl priority.xml ... done in 20193ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found products: products.xsl products.xml ... done in 21631ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found queens: queens.xsl queens.xml ... done in 5009ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found reverser: reverser.xsl gettysburg.xml ... done in 20336ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found stringsort: stringsort.xsl db1000.xml ... done in 17345ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found summarize: summarize.xsl queens.xsl ... done in 15747ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found total: total.xsl chart.xml ... done in 14948ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found tower: tower.xsl tower.xml ... done in 12956ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found trend: trend.xsl trend.xml ... done in 86370ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found union: union.xsl union.xml ... done in 16682ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found xpath: xpath.xsl xpath.xml ... done in 16278ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found xslbench1: xslbench1.xsl xslbench1.xml ... done in 16932ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found xslbench2: xslbench2.xsl xslbenchdream.xml ... done in 18297ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found xslbench3: xslbench3.xsl xslbenchdream.xml ... done in 9782ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found XSLTMark 2.0 (Feb 28 2001) running on SunOS 5.8 sparc Java Runtime Environment is Sun Microsystems Inc. 1.3.1_03 Current time is 14:17:07 EDT, 18 Jul 2002 Results for driver Xalan2JDriver prep exec Test Component iter ms ms KB In KB Out KB/s status alphabetize 100 192 52170 1966 1566 33.86 CHK OUTPUT attsets 100 94 21806 121 255 8.65 CHK OUTPUT avts 100 209 37321 1966 1156 41.84 CHK OUTPUT axis 100 138 22164 35 60 2.17 CHK OUTPUT backwards 100 100 26767 260 261 9.76 CHK OUTPUT bottles 100 131 34419 4 1130 16.49 CHK OUTPUT breadth 100 54 23475 415 200 13.14 CHK OUTPUT brutal 100 143 22260 130 394 11.78 CHK OUTPUT chart 100 59 21782 121 359 11.04 CHK OUTPUT creation 50 36 33204 983 1520 37.7 CHK OUTPUT current 100 268 19765 18 29 1.24 CHK OUTPUT dbonerow 10 45 96359 19621 8 101.86 CHK OUTPUT dbtail 100 35 23368 1966 503 52.85 CHK OUTPUT decoy 100 187 56668 1966 1974 34.77 CHK OUTPUT depth 100 25 20518 588 373 23.46 CHK OUTPUT encrypt 10 109 4607 196 196 42.69 CHK OUTPUT functions 100 160 54484 1966 618 23.72 CHK OUTPUT game 100 56 18754 260 43 8.1 CHK OUTPUT html 100 52 17782 29 47 2.14 CHK OUTPUT identity 10 30 18522 1962 1962 105.96 CHK OUTPUT inventory 100 37 21633 184 202 8.94 CHK OUTPUT metric 100 58 23545 88 65 3.28 CHK OUTPUT number 100 38 20917 18 79 2.35 NO REF oddtemplate 100 266 20029 16 15 0.81 CHK OUTPUT patterns 100 34 47092 1966 1974 41.84 CHK OUTPUT prettyprint 10 65 12658 196 1405 63.27 CHK OUTPUT priority 100 98 20193 91 57 3.67 CHK OUTPUT products 100 68 21631 196 40 5.47 CHK OUTPUT queens 10 127 5009 0 17 1.75 CHK OUTPUT reverser 100 37 20336 129 129 6.37 CHK OUTPUT stringsort 10 22 17345 1962 1933 112.3 CHK OUTPUT summarize 100 178 15747 712 29 23.56 CHK OUTPUT total 100 25 14948 121 33 5.18 CHK OUTPUT tower 20 66 12956 1 1381 53.35 CHK OUTPUT trend 100 126 86370 176 806 5.69 CHK OUTPUT union 100 123 16682 16 11 0.86 CHK OUTPUT xpath 100 25 16278 28 20 1.51 CHK OUTPUT xslbench1 100 176 16932 36 665 20.74 CHK OUTPUT xslbench2 20 34 18297 2920 2139 138.26 CHK OUTPUT xslbench3 20 38 9782 2920 25 150.56 CHK OUTPUT Aggr. Results 100 38 20917 18 79 2.35 Arithmetic Mean 38 2.35 Geometric Mean 2.35 Harmonic Mean 2.35 Conformance Score (OK+NOREF+SKIPPED divided by TOTAL) 1/40 = 2.5% ************************************************************* XSLTC ************************************************************* XSLTMark 2.0 (Feb 28 2001) running on SunOS 5.8 sparc Java Runtime Environment is Sun Microsystems Inc. 1.3.1_03 Current time is 14:37:47 EDT, 18 Jul 2002 loading test driver: XSLTCDriver... Loaded XSLTCDriver No configurations specified, using default Reading configuration from testcases/default.conf dummy initialization run: xslbench3: xslbench3.xsl xslbenchdream.xml ... done in 291ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found benchmark run: alphabetize: alphabetize.xsl db100.xml ... done in 13844ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found attsets: attsets.xsl chart.xml ... done in 1002ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found avts: avts.xsl db100.xml ... done in 1522ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found axis: axis.xsl axis.xml ... done in 741ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found backwards: backwards.xsl game.xml ... done in 1458ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found bottles: bottles.xsl bottles.xml ... done in 1433ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found breadth: find.xsl breadth.xml ... done in 703ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found brutal: brutal.xsl brutal.xml ... done in 856ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found chart: chart.xsl chart.xml ... done in 836ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found creation: creation.xsl db100.xml ... done in 2024ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found current: current.xsl current.xml ... done in 287ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found dbonerow: dbonerow.xsl db10000.xml ... done in 2115ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found dbtail: dbtail.xsl db100.xml ... done in 746ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found decoy: decoy.xsl db100.xml ... done in 5665ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found depth: find.xsl depth.xml ... done in 596ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found encrypt: encrypt.xsl db100.xml ... done in 574ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found functions: functions.xsl db100.xml ... done in 7624ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found game: game.xsl game.xml ... done in 225ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found html: html.xsl html.xml ... done in 282ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found identity: identity.xsl db1000.xml ... done in 1701ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found inventory: inventory.xsl inventory.xml ... done in 487ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found metric: metric.xsl metric.xml ... done in 519ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found number: number.xsl number.xml ... done in 557ms. oddtemplate: oddtemplate.xsl oddtemplate.xml ... done in 239ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found patterns: patterns.xsl db100.xml ... done in 2781ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found prettyprint: prettyprint.xsl db100.xml ... done in 5573ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found priority: priority.xsl priority.xml ... done in 264ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found products: products.xsl products.xml ... done in 675ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found queens: queens.xsl queens.xml ... done in 1348ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found reverser: reverser.xsl gettysburg.xml ... done in 778ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found stringsort: stringsort.xsl db1000.xml ... done in 2653ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found summarize: summarize.xsl queens.xsl ... done in 1013ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found total: total.xsl chart.xml ... done in 146ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found tower: tower.xsl tower.xml ... done in 6095ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found trend: trend.xsl trend.xml ... done in 11740ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found union: union.xsl union.xml ... done in 304ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found xpath: xpath.xsl xpath.xml ... done in 339ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found xslbench1: xslbench1.xsl xslbench1.xml ... done in 717ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found xslbench2: xslbench2.xsl xslbenchdream.xml ... done in 1901ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found xslbench3: xslbench3.xsl xslbenchdream.xml ... done in 171ms. dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found XSLTMark 2.0 (Feb 28 2001) running on SunOS 5.8 sparc Java Runtime Environment is Sun Microsystems Inc. 1.3.1_03 Current time is 14:37:47 EDT, 18 Jul 2002 Results for driver XSLTCDriver prep exec Test Component iter ms ms KB In KB Out KB/s status alphabetize 100 692 13844 1966 156699 5730.47 CHK OUTPUT attsets 100 374 1002 121 25595 12832.83 CHK OUTPUT avts 100 294 1522 1966 115693 38652.88 CHK OUTPUT axis 100 485 741 35 6093 4135.56 CHK OUTPUT backwards 100 462 1458 260 26181 9067.93 CHK OUTPUT bottles 100 335 1433 4 113046 39445.66 CHK OUTPUT breadth 100 147 703 415 20078 14576.13 CHK OUTPUT brutal 100 944 856 130 39990 23434.99 CHK OUTPUT chart 100 381 836 121 40732 24434.04 CHK OUTPUT creation 50 268 2024 983 76042 19028.04 CHK OUTPUT current 100 687 287 18 2998 5255.91 CHK OUTPUT dbonerow 10 277 2115 19621 89 4659.7 CHK OUTPUT dbtail 100 358 746 1966 50390 35091.58 CHK OUTPUT decoy 100 377 5665 1966 197460 17601.67 CHK OUTPUT depth 100 150 596 588 37363 31839.13 CHK OUTPUT encrypt 10 177 574 196 1967 1885.09 CHK OUTPUT functions 100 313 7624 1966 61845 4184.92 CHK OUTPUT game 100 202 225 260 4658 10930.34 CHK OUTPUT html 100 563 282 29 4863 8674.26 CHK OUTPUT identity 10 254 1701 1962 19626 6345.96 CHK OUTPUT inventory 100 322 487 184 20205 20934.23 CHK OUTPUT metric 100 292 519 88 6582 6426.31 CHK OUTPUT number 100 189 557 18 7958 7161.43 NO REF oddtemplate 100 230 239 16 1552 3283.54 CHK OUTPUT patterns 100 394 2781 1966 197460 35855.26 CHK OUTPUT prettyprint 10 290 5573 196 14363 1306.32 CHK OUTPUT priority 100 379 264 91 5722 11010.93 CHK OUTPUT products 100 361 675 196 4101 3183.52 CHK OUTPUT queens 10 357 1348 0 172 64.24 CHK OUTPUT reverser 100 276 778 129 12939 8399.32 CHK OUTPUT stringsort 10 605 2653 1962 19333 4013.55 CHK OUTPUT summarize 100 362 1013 712 2939 1802.54 CHK OUTPUT total 100 174 146 121 3408 12087.3 CHK OUTPUT tower 20 281 6095 1 27628 2266.54 CHK OUTPUT trend 100 1535 11740 176 80683 3443.78 CHK OUTPUT union 100 922 304 16 1181 1971.27 CHK OUTPUT xpath 100 1158 339 28 2021 3024.17 CHK OUTPUT xslbench1 100 528 717 36 72285 50433.66 CHK OUTPUT xslbench2 20 320 1901 2920 42801 12025.66 CHK OUTPUT xslbench3 20 273 171 2920 503 10010.28 CHK OUTPUT Aggr. Results 100 189 557 18 7958 7161.43 Arithmetic Mean 189 7161.43 Geometric Mean 7161.43 Harmonic Mean 7161.43 -- Tom Amiro -- SQA Engineer Sun XML Technology Development voice: 781-442-0589 Fax: 781-442-1437 eMail: [email protected]
