Stefano, thank you for pointing peroformance issues out!
Although that was a debug build, it uncovers that Harmony is not properly tuned for FP-intensive workloads as SciMark. One of the possible reasons of this (known:) limitation is also raised in the thread "[drlvm][jit][ia-32]register-based fast calling convention", where Alex vounteers to help with more intelligent XMM register usage. I consider this a quite-high priority, fully support Alex, hope for the best, ready to help, etc etc. On the 0x224 day of Apache Harmony Stefano Mazzocchi wrote: > There are lies, damn lies and benchmarks.... which don't really tell you > if an implementation of a program is *faster* but at least it tells you > where you're at. > > So, as Geir managed to get the DSO linking problem go away in DRLVM, I > was able to start running some benchmarks. > > The machine is the following: > > Linux harmony-em64t 2.6.15-27-amd64-generic #1 SMP PREEMPT Sat Sep 16 > 01:50:50 UTC 2006 x86_64 GNU/Linux > > dual Intel(R) Pentium(R) D CPU 3.20GHz > bogomips 6410.31 (per CPU) > > There is nothing else running on the machine (load is 0.04 at the time > of testing). > > The various virtual machines tested are: > > harmony > ------- > Apache Harmony Launcher : (c) Copyright 1991, 2006 The Apache Software > Foundation or its licensors, as applicable. > java version "1.5.0" > pre-alpha : not complete or compatible > svn = r476006, (Nov 16 2006), Linux/em64t/gcc 4.0.3, debug build > > sun5 > --- > java version "1.5.0_09" > Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_09-b03) > Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_09-b03, mixed mode) > > sun6 > ---- > java version "1.6.0-rc" > Java(TM) SE Runtime Environment (build 1.6.0-rc-b104) > Java HotSpot(TM) 64-Bit Server VM (build 1.6.0-rc-b104, mixed mode) > > ibm > --- > java version "1.5.0" > Java(TM) 2 Runtime Environment, Standard Edition (build > pxa64dev-20061002a (SR3) ) > IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux amd64-64 > j9vmxa6423-20061001 (JIT enabled) > J9VM - 20060915_08260_LHdSMr > JIT - 20060908_1811_r8 > GC - 20060906_AA) > JCL - 20061002 > > bea > --- > java version "1.5.0_06" > Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05) > BEA JRockit(R) (build > R26.4.0-63-63688-1.5.0_06-20060626-2259-linux-x86_64, ) > > > -------------------------------------------------------------------------- > > Test #1: java scimark2 (http://math.nist.gov/scimark2/) > > command: java jnt.scimark2.commandline > > NOTE: bigger number is better > > Sun6 > Composite Score: 364.5832265230057 > FFT (1024): 220.8458713892794 > SOR (100x100): 696.1542342357722 > Monte Carlo : 149.37978088875656 > Sparse matmult (N=1000, nz=5000): 326.37451873283845 > LU (100x100): 430.1617273683819 > > BEA > Composite Score: 359.13480378697835 > FFT (1024): 303.8746880751562 > SOR (100x100): 454.25628897202307 > Monte Carlo : 93.23913192138497 > Sparse matmult (N=1000, nz=5000): 530.44112637391 > LU (100x100): 413.8627835924175 > > Sun5 > Composite Score: 332.84987587548574 > FFT (1024): 216.5144595799027 > SOR (100x100): 689.429322146947 > Monte Carlo : 25.791262124978065 > Sparse matmult (N=1000, nz=5000): 317.5193965699373 > LU (100x100): 414.99493895566377 > > IBM > Composite Score: 259.8249218693683 > FFT (1024): 296.8415012789055 > SOR (100x100): 428.974881649179 > Monte Carlo : 89.15159857584082 > Sparse matmult (N=1000, nz=5000): 144.3524241203982 > LU (100x100): 339.8042037225181 > > Harmony > Composite Score: 113.65082278962575 > FFT (1024): 203.76641991778123 > SOR (100x100): 224.37761309236748 > Monte Carlo : 9.063866256533116 > Sparse matmult (N=1000, nz=5000): 65.4051866327227 > LU (100x100): 65.6410280487242 > > In this test harmony is clearly lagging behind... at about 30% > performance of the best JVM, it's a little crappy. Please note how FFT's > performance is not so bad awhile monte carlo is pretty bad compared to > BEA or IBM. > > Overall, it seems like there is some serious work to do here to catch up. > > -------------------------------------------------------------------------- > > Test 2: Dhrystones (http://www.c-creators.co.jp/okayan/DhrystoneApplet/) > > command: java dhry 100000000 > > NOTE: bigger is better > > NB: I modified the code to accept the count at input from the command line! > > sun6: 8552856 dhrystones/sec > sun5: 6605892 > bea: 5678914 > harmony: 669734 > ibm: 501562 > > The performance here is horrific but what's surprising is that J9 is > even worse. No idea what's going on but it seems like something is not > working as it should (in both harmony and J9) > > -------------------------------------------------------------------------- > > Test 3: Sieve (part of http://www.sax.de/~adlibit/tya18.tgz) > > command: java Sieve 30 > > NB: I modified the test to run for a configurable amount of seconds. > > sun6 8545 sieves/sec > sun5 8364 > bea 6174 > harmony 1836 > ibm 225 > > IBM J9 clearly has something wrong on x86_64 but harmony is clearly > lagging behind. > > Stay tuned for more tests. > > -- > Stefano. > > -- Egor Pasko
