Stefano, It is a bit unfair to compare *debug* build of Harmony with other release versions :) I suppose all VMs where run in default mode (i.e. no special cmd-line switches)?
2006/11/17, Stefano Mazzocchi <[EMAIL PROTECTED]>:
There are lies, damn lies and benchmarks.... which don't really tell you if an implementation of a program is *faster* but at least it tells you where you're at. So, as Geir managed to get the DSO linking problem go away in DRLVM, I was able to start running some benchmarks. The machine is the following: Linux harmony-em64t 2.6.15-27-amd64-generic #1 SMP PREEMPT Sat Sep 16 01:50:50 UTC 2006 x86_64 GNU/Linux dual Intel(R) Pentium(R) D CPU 3.20GHz bogomips 6410.31 (per CPU) There is nothing else running on the machine (load is 0.04 at the time of testing). The various virtual machines tested are: harmony ------- Apache Harmony Launcher : (c) Copyright 1991, 2006 The Apache Software Foundation or its licensors, as applicable. java version "1.5.0" pre-alpha : not complete or compatible svn = r476006, (Nov 16 2006), Linux/em64t/gcc 4.0.3, debug build sun5 --- java version "1.5.0_09" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_09-b03) Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_09-b03, mixed mode) sun6 ---- java version "1.6.0-rc" Java(TM) SE Runtime Environment (build 1.6.0-rc-b104) Java HotSpot(TM) 64-Bit Server VM (build 1.6.0-rc-b104, mixed mode) ibm --- java version "1.5.0" Java(TM) 2 Runtime Environment, Standard Edition (build pxa64dev-20061002a (SR3) ) IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux amd64-64 j9vmxa6423-20061001 (JIT enabled) J9VM - 20060915_08260_LHdSMr JIT - 20060908_1811_r8 GC - 20060906_AA) JCL - 20061002 bea --- java version "1.5.0_06" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05) BEA JRockit(R) (build R26.4.0-63-63688-1.5.0_06-20060626-2259-linux-x86_64, ) -------------------------------------------------------------------------- Test #1: java scimark2 (http://math.nist.gov/scimark2/) command: java jnt.scimark2.commandline NOTE: bigger number is better Sun6 Composite Score: 364.5832265230057 FFT (1024): 220.8458713892794 SOR (100x100): 696.1542342357722 Monte Carlo : 149.37978088875656 Sparse matmult (N=1000, nz=5000): 326.37451873283845 LU (100x100): 430.1617273683819 BEA Composite Score: 359.13480378697835 FFT (1024): 303.8746880751562 SOR (100x100): 454.25628897202307 Monte Carlo : 93.23913192138497 Sparse matmult (N=1000, nz=5000): 530.44112637391 LU (100x100): 413.8627835924175 Sun5 Composite Score: 332.84987587548574 FFT (1024): 216.5144595799027 SOR (100x100): 689.429322146947 Monte Carlo : 25.791262124978065 Sparse matmult (N=1000, nz=5000): 317.5193965699373 LU (100x100): 414.99493895566377 IBM Composite Score: 259.8249218693683 FFT (1024): 296.8415012789055 SOR (100x100): 428.974881649179 Monte Carlo : 89.15159857584082 Sparse matmult (N=1000, nz=5000): 144.3524241203982 LU (100x100): 339.8042037225181 Harmony Composite Score: 113.65082278962575 FFT (1024): 203.76641991778123 SOR (100x100): 224.37761309236748 Monte Carlo : 9.063866256533116 Sparse matmult (N=1000, nz=5000): 65.4051866327227 LU (100x100): 65.6410280487242 In this test harmony is clearly lagging behind... at about 30% performance of the best JVM, it's a little crappy. Please note how FFT's performance is not so bad awhile monte carlo is pretty bad compared to BEA or IBM. Overall, it seems like there is some serious work to do here to catch up. -------------------------------------------------------------------------- Test 2: Dhrystones (http://www.c-creators.co.jp/okayan/DhrystoneApplet/) command: java dhry 100000000 NOTE: bigger is better NB: I modified the code to accept the count at input from the command line! sun6: 8552856 dhrystones/sec sun5: 6605892 bea: 5678914 harmony: 669734 ibm: 501562 The performance here is horrific but what's surprising is that J9 is even worse. No idea what's going on but it seems like something is not working as it should (in both harmony and J9) -------------------------------------------------------------------------- Test 3: Sieve (part of http://www.sax.de/~adlibit/tya18.tgz) command: java Sieve 30 NB: I modified the test to run for a configurable amount of seconds. sun6 8545 sieves/sec sun5 8364 bea 6174 harmony 1836 ibm 225 IBM J9 clearly has something wrong on x86_64 but harmony is clearly lagging behind. Stay tuned for more tests. -- Stefano.
