Re: [Mono-list] CLI-Grande Test Results

2004-06-09 Thread Marcus 
I think that the idea is to test overall performance, independent of JIT time. 
When real users run the applications, the JIT time is part of the performance 
issue.

The results are available at http://mylinuxisp.com/~mathpup/results.sxc

I tried to attach them to the email, but that did not seem to work.

Of course, you're welcome to run the tests themselves. The total battery of 
tests takes quite a while to run -- perhaps 45 minutes. I'm not entirely sure 
because I run them while I'm away from the machine.

___
Mono-list maillist  -  [EMAIL PROTECTED]
http://lists.ximian.com/mailman/listinfo/mono-list

Re: [Mono-list] CLI-Grande Test Results

2004-06-09 Thread Ben Maurer 
Hello Marcus,

Thank you, as always, for your interest in performance. I really do
appreciate your dedication.

On Wed, 2004-06-09 at 02:42, Marcus wrote:
> I observed that using full optimization (-O=all) with Mono sometimes made 
> results worse.
This may be due to abcrem. That optimization generally takes alot of
time. I think at one point, we talked about removing abcrem from -O=all.
We should consider this again.

To see the effect of abcrem, try running --compile-all on corlib with
-O=all and -O=all,-abcrem. The difference is quite large.

IMHO, these tests should have some sort of `warm up period' in which
they invoke the methods so that they can be jitted. If we want to test
the speed of jitting, we should do so in another benchmark. It would be
interesting to see the effects of different -O options on --compile-all
on a few assemblies.

> Also, the Create:Array tests run so much faster with full 
> optimization that I am not sure how to interpret the results.
Maybe the abcrem optimization was effective there?


Could you give us any insight to the performance of these benchmarks
over time? I know you run them every so often, and would love to hear
about what has and has not helped the benchmarks.

Someday, I hope we can get benchmarks run automatically on a server,
like our NUnit tests are now, so that we can see trends over time. Your
informal reports can be a first step in this process.

Thanks again,
-- Ben

___
Mono-list maillist  -  [EMAIL PROTECTED]
http://lists.ximian.com/mailman/listinfo/mono-list

Re: [Mono-list] CLI-Grande Test Results

2004-06-08 Thread KiOrKY 
can you posty me the results ;-)
regards

Marcus <[EMAIL PROTECTED]> a écrit :

> I recently obtained a copy of the CLI-Grande benchmark suite, which is a port
> 
> of Java-Grande. It is available from CVS 
> at :pserver:[EMAIL PROTECTED]:/cvs.
> Check out module "cli-grande". I wanted 
> to post the results in case anyone was interested.
> 
> I observed that using full optimization (-O=all) with Mono sometimes made 
> results worse. Also, the Create:Array tests run so much faster with full 
> optimization that I am not sure how to interpret the results.
> 
> Finally, ilrun failed during the Serial:Reading:BinaryTree test, which aborted
> 
> the benchmark. So the remaining 4 tests did not run.

-- 

I{ i () r |{ Y
Debian User
-
-- Mathieu PASQUET 
-- Testeur Debian SID
-- irc: #debian-fr / irc.freenode.net port: 6667 
-- Site: www.fr.debian.org 
-


___
Mono-list maillist  -  [EMAIL PROTECTED]
http://lists.ximian.com/mailman/listinfo/mono-list

[Mono-list] CLI-Grande Test Results

2004-06-08 Thread Marcus 
I recently obtained a copy of the CLI-Grande benchmark suite, which is a port 
of Java-Grande. It is available from CVS 
at :pserver:[EMAIL PROTECTED]:/cvs. Check out module "cli-grande". I wanted 
to post the results in case anyone was interested.

I observed that using full optimization (-O=all) with Mono sometimes made 
results worse. Also, the Create:Array tests run so much faster with full 
optimization that I am not sure how to interpret the results.

Finally, ilrun failed during the Serial:Reading:BinaryTree test, which aborted 
the benchmark. So the remaining 4 tests did not run.



"CLI Grande Benchmark Suite - Version 2.0 - Section 1",
"OS: Unix 2.4.21.144 CLI: 1.1.4322.573",
,"mono -O=all","mono (default)","ilrun","mint","mono-opt/mono-default","ilrun/mono-opt","ilrun/mint",,
,
"Section1:Arith:Add:Int",221190500,235948900,206204280,9584203,0.937450863301333,0.93224745185711,21.5150159069043,," (adds/s) "
"Section1:Arith:Add:Long   ",179951900,197343700,6903719.5,8671307,0.911870508154048,0.0383642490020944,0.796156738540107,," (adds/s) "
"Section1:Arith:Add:Float  ",110815900,111283400,57196664,6694502,0.995799014048816,0.516141311851458,8.54382656096002,," (adds/s) "
"Section1:Arith:Add:Double ",108419600,46860440,58760308,7984470,2.31367012345595,0.541971267187852,7.35932478924713,," (adds/s) "
"Section1:Arith:Mult:Int   ",161486500,158719100,119963360,9613009,1.01743583475461,0.742869280094621,12.4792726190103,," (multiplies/s) "
"Section1:Arith:Mult:Long  ",39700480,37110090,6116030,8089773,1.06980284876701,0.154054308663271,0.756019977321984,," (multiplies/s) "
"Section1:Arith:Mult:Float ",88500380,89552900,54192044,7431890,0.98824694677671,0.612336850983013,7.29182536340016,," (multiplies/s) "
"Section1:Arith:Mult:Double",88517490,50837480,54735644,8945791,1.74118563705361,0.618359648471731,6.11859186068622,," (multiplies/s) "
"Section1:Arith:Div:Int",26305810,26304620,24554168,7652796,1.4523920133,0.933412352632365,3.2085224798884,," (divides/s) "
"Section1:Arith:Div:Long   ",8042728,7919320,3874991,4986261,1.01558315612957,0.481800578112302,0.777133607727313,," (divides/s) "
"Section1:Arith:Div:Float  ",37746460,37105660,27710100,6782400,1.01726960253503,0.734111225264568,4.08558917197452,," (divides/s) "
"Section1:Arith:Div:Double ",53415090,37523440,39626440,8809686,1.42351260971809,0.741858527243893,4.49805361961823,," (divides/s) "
"Section1:Assign:Same:Scalar:Local ",620827500,846611700,599959420,16967150,0.733308434079047,0.966386669404947,35.3600587016676,," (assignments/s) "
"Section1:Assign:Same:Scalar:Instance  ",793268200,787358500,217401870,8975077,1.00750572960094,0.274058471019007,24.2228417650344,," (assignments/s) "
"Section1:Assign:Same:Scalar:Class ",341011900,342017900,21406904,4912664,0.997058633480879,0.0627746539050397,4.35749401953808,," (assignments/s) "
"Section1:Assign:Same:Array:Local  ",119135500,125996300,138440090,4694022,0.945547607350374,1.16203893885534,29.4928506939252,," (assignments/s) "
"Section1:Assign:Same:Array:Instance   ",63705430,59535560,44387680,1829450,1.07003998954574,0.696764467330336,24.2628549564077,," (assignments/s) "
"Section1:Assign:Same:Array:Class  ",131722600,131707300,17462705,4151987,1.00011616668173,0.132571821388281,4.20586697405363,," (assignments/s) "
"Section1:Assign:Other:Scalar:Instance ",814437800,763728900,190996190,9770939,1.06639646607586,0.234512924129995,19.5473730825666,," (assignments/s) "
"Section1:Assign:Other:Scalar:Class",667766800,667856900,4392104,4898740,0.999865090860033,0.00657730213601515,0.896578303808735,," (assignments/s) "
"Section1:Assign:Other:Array:Instance  ",126032400,125998800,70286504,3703004,1.0002920637,0.557685991856062,18.980941959555,," (assignments/s) "
"Section1:Assign:Other:Array:Class ",131748200,130704700,4193471.25,4150006,1.00798364557663,0.0318294386564674,1.0104735390744,," (assignments/s) "
"Section1:Cast:IntFloat",57291080,53036550,16287968,12643340,1.0802188302218,0.28430198907055,1.28826465158732,," (casts/s) "
"Section1:Cast:IntDouble   ",57300250,44932720,17423172,12417110,1.27524552263918,0.304067992722545,1.40315838387515,," (casts/s) "
"Section1:Cast:LongFloat   ",32967550,31849230,9498018,10277280,1.03511293679627,0.288102027599867,0.924176241184438,," (casts/s) "
"Section1:Cast:LongDouble  ",36798260,31699070,9509921,10260570,1.16086244801504,0.258433985737369,0.926841393801709,," (casts/s) "
"Section1:Create:Array:Int:1   ",8685521000,1926683,1209995,1054746,4508.01766559418,0.000139311735012787,1.14719088766395,," (arrays/s) "
"Section1:Create:Array:Int:2   ",8693961000,1924005,1214885.25,1053181,4518.67900551194,0.000139738980885698,1.15353889787226,," (arrays/s) "
"Section1:Create:Array:Int:4

[Mono-list] CLI-Grande Test Results

2004-06-07 Thread Marcus 
I recently obtained a copy of the CLI-Grande benchmark suite, which is a port 
of Java-Grande. It is available from CVS 
at :pserver:[EMAIL PROTECTED]:/cvs. Check out module "cli-grande". I wanted 
to post the results in case anyone was interested.

I observed that using full optimization (-O=all) with Mono sometimes made 
results worse. Also, the Create:Array tests run so much faster with full 
optimization that I am not sure how to interpret the results.

Finally, ilrun failed during the Serial:Reading:BinaryTree test, which aborted 
the benchmark. So the remaining 4 tests did not run.



results.sxc
Description: OpenOffice Calc spreadsheet