You can find performance measurements SableVM compared to other free/non-free JVMs running some real benchmarks (SPECjvm, Soot, SableCC) in my Ph.D. Thesis. Look for the Chapter on performance measurements. More precisely, you might be interested by Tables 9.1 and 9.2 on page 118.
http://www.sable.mcgill.ca/publications/thesis/phd-gagnon/sable-thesis-2002-phd-gagnon.ps.gz
I would summarize SableVM's situation as follows: - It is an interpreter, so do not expect it to perform as a JIT. Interpreters have dispatch overhead. (Yet SableVM is able to get rid of much dispatch overhead using clever tricks). - As an interpreter, SableVM performs wery well. It is much faster than Kaffe's interpreter (which has not bee designed for efficiency, to be fair). It has comparable performance to interpreters written partly in assembly language. - The goal of SableVM is *NOT* to achieve upmost performance. It is to provide an efficient yet highly portable, suitable for doing research (profiling, language extensions, new type systems, new bytecodes, etc.) - The very preliminary work is again not targeted at providing the maximal performance, but to provide an additional choice on the performance/portability tradeof scale, with an *easily* retargetable JIT. [The aim is for few hours or very few days work by a single person to port to a new platform].
I insist: SableVM's goal is *NOT* to beat other VMs in micro/mini/macro-benchmark contests, but to provide a highly versatile and portable, yet *efficient* framework.
;-)
Have fun!
Etienne
Mark Wielaard wrote:
Hi all,
Before going to the Libre Software Meeting and LinuxTag <http://www.libresoftwaremeeting.org/> <http://www.linuxtag.org/2003/en/index.html> I wanted to know how the different free VMs were doing. So I did some micro benchmarks against them. I didn't have time to do more complete benchmarks and I couldn't get Orp and JRVM working (which is a shame since those are claimed to be the fasted VMs there are). So maybe someone with more time (or access to working Orp/JRVM VMs) can do some bigger tests.
Orp didn't compile since it has some hardcoded values for glibc 2.1 and glibc 2.2, but I am using glibc 2.3. And it needs several fixes to get it to compile with gcc 3.x. JRVM did compile, but needs a specific version of a proprietary VM to create the boot image. The resulting JikesRVM binary and image keep seg faulting however on my machine.
The machine that ran the test had a Athlon 1600+ (1.4Ghz) and 768MB. It was lightly used and I don't claim to have done scientific benchmarking. All times were measured with /usr/bin/time -p and the best of three runs is given. (Note that some tests measure elapsed time themselves so the time -p values are only for reference.) The benchmarks were mentioned earlier on shudo.net. If someone has more suggestions please let me know.
VMs used: kissme 0.0.30 (Jun 24 2003) SableVM version 1.0.8 kaffe 1.1.x-cvs (checkout today) ikvm (Latest ikvmbin snapshot plus Mono JIT compiler version 0.25) gij/gcj 3.3.1 20030626 (Debian prerelease) (For gcj tests were compiled from source using just -O2)
Looking at the results (see below) there are clearly three groups: the plain interpreters (kissme, gij and sablevm), the jitters (kaffe and ikvm) and the ahead of time compiler (gcj). The jitters are clearly much faster then the interpreters, and the ahead of time compiler is clearly much faster then the jitters.
Interesting is that ikvm has results comparable with kaffe, but often takes more time "preparing" the classes. This can be clearly seen with the Linpack benchmark where the ikvm score and kaffe score are almost identical, but when looking at the real time spend you see that ikvm consumes much more processor cycles.
SableVM recently got a "portable" jit compiler, it would be interesting to see how it matches up to kaffe and ikvm.
Since gcj compiled code seems to be very fast but the gij interpreter is not very fast it might be interesting to combine gcj with one of the jitters or make one of the jitters be able to use precompiled binaries produced by gcj (for example to precompile all core classes used by the VM).
Let me know if you have other benchmark/speed test stories.
Cheers,
Mark
The results:
http://math.nist.gov/scimark2/
Kissme:
Composite Score: 3.0156149099413163E174
FFT (1024): 1.5078077289324666E175
SOR (100x100): -1.1107352312578372E61
Monte Carlo : 1.6082285944663933E129
Sparse matmult (N=1000, nz=5000): 2.6108808678857117E25
LU (100x100): 3.5030103117925084
real 26.99
user 26.82
sys 0.05
gij:
Composite Score: 5.145989276632694
FFT (1024): 2.648052903699278
SOR (100x100): 8.881613884448855
Monte Carlo : 1.387464107626074
Sparse matmult (N=1000, nz=5000): 5.551639995055836
LU (100x100): 7.261175492333425
real 31.30
user 31.19
sys 0.10
SableVM:
Composite Score: 7.329552111085732
FFT (1024): 4.404292204203279
SOR (100x100): 13.062330882926906
Monte Carlo : 2.369663222573336
Sparse matmult (N=1000, nz=5000): 7.329992764243664
LU (100x100): 9.481481481481481
real 28.23
user 26.95
sys 0.11
Kaffe:
Composite Score: 60.46979156068268
FFT (1024): 30.891199299334463
SOR (100x100): 148.2273083076923
Monte Carlo : 12.628691263262526
Sparse matmult (N=1000, nz=5000): 54.10832237609153
LU (100x100): 56.49343655703252
real 31.17
user 31.10
sys 0.00
ikvm:
Composite Score: 100.9979186697824
FFT (1024): 59.79980402491375
SOR (100x100): 224.2731618120265
Monte Carlo : 34.02223834505183
Sparse matmult (N=1000, nz=5000): 83.7413726183124
LU (100x100): 103.1530165486078
real 35.60
user 35.42
sys 0.09
gcj:
Composite Score: 217.51089337171615
FFT (1024): 225.23768378150965
SOR (100x100): 305.57485467536026
Monte Carlo : 15.335663953302584
Sparse matmult (N=1000, nz=5000): 235.82583613959164
LU (100x100): 305.5804283088167
real 28.49
user 28.44
sys 0.06
Sieve.java from TYA <http://sax.sax.de/~adlibit/>
Kissme: 634 iterations in 10.008 seconds Sieve score = 63 real 10.10 user 9.98 sys 0.02
SableVM: 2355 iterations in 10.0 seconds Sieve score = 236 real 10.08 user 10.08 sys 0.01
gij: 2692 iterations in 10.0 seconds Sieve score = 269 real 10.10 user 10.05 sys 0.05
ikvm: 40937 iterations in 10.0 seconds Sieve score = 4094 real 11.32 user 11.24 sys 0.05
Kaffe: 48611 iterations in 10.0 seconds Sieve score = 4861 real 10.11 user 10.10 sys 0.01
gcj: 91621 iterations in 10.0 seconds Sieve score = 9162 real 10.03 user 9.95 sys 0.06
http://www.netlib.org/benchmark/linpackjava/
Kissme: Mflops/s: 2.289 Time: 36.62s Norm Res: 0.0 Precision: 2.220446049250313E-16 real 40.73 user 40.62 sys 0.10
gij: Mflops/s: 6.078 Time: 13.79s Norm Res: 5.81 Precision: 2.220446049250313E-16 real 14.74 user 14.59 sys 0.14
SableVM: Mflops/s: 8.45 Time: 9.92s Norm Res: 5.81 Precision: 2.220446049250313E-16 real 10.88 user 10.86 sys 0.02
ikvm: Mflops/s: 60.312 Time: 1.39s Norm Res: 5.68 Precision: 2.220446049250313E-16.0 real 5.03 user 4.92 sys 0.06
Kaffe: Mflops/s: 61.282 Time: 1.37s Norm Res: 4.28 Precision: 2.220446049250313E-16 real 1.92 user 1.89 sys 0.03
gcj: Mflops/s: 79.163 Time: 1.06s Norm Res: 4.28 Precision: 2.220446049250313E-16 real 1.41 user 1.37 sys 0.03
_______________________________________________ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
-- Etienne M. Gagnon, Ph.D. http://www.info.uqam.ca/~egagnon/ SableVM: http://www.sablevm.org/ SableCC: http://www.sablecc.org/
_______________________________________________ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath