[Mono-dev] substantial performance regression between 2.10 and 2.6 or impl diff?
Hi,I was doing some benchmarks of struct vs class based creation (I have an application that will generate millions of small objects). I was doing the tests in a ubuntu 11.4 VM on my mac pro and found the following: mono2.6.7 was 4x faster on my linux VM than 2.10.4 running on OSX (same machine)I don't know whether this may be because of one of the following:- performance in 2.10.4 regressed vs 2.6.7- mono JIT implementation for OSX has a completely different JiT codebase and does not perform- difference in GC (only relevant for second part of the test)Note that I tried this with separate compilations with mcs -optimize+ on both environments as well as running the same exe on both.I can live with slower performance on OSX, but want to make sure that linux and windows versions of mono 2.10.x have the performance of 2.6.7 or better.Can someone clue me in? I've included the simple test code with this posting.ThanksJonathan Main.cs Description: Binary data ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] substantial performance regression between 2.10 and 2.6 or impl diff?
My machine is an old 2006 Mac Pro 1,1 2 x Xeon 5130 (64 bit) running OSX lion. Here are the respective versions of mono: Mono on OSX: Mono JIT compiler version 2.10.4 (tarball Mon Aug 8 22:03:39 EDT 2011) Copyright (C) 2002-2011 Novell, Inc, Xamarin, Inc and Contributors. www.mono-project.com TLS: normal SIGSEGV: normal Notification: kqueue Architecture: x86 Disabled:none Misc: debugger softdebug LLVM: yes(2.9svn-mono) GC: Included Boehm (with typed GC) Mono on Ubuntu 11.04 (running in VMWare VM on same machine): Mono JIT compiler version 2.6.7 (Debian 2.6.7-5ubuntu3) Copyright (C) 2002-2010 Novell, Inc and Contributors. www.mono-project.com TLS: __thread GC:Included Boehm (with typed GC and Parallel Mark) SIGSEGV: altstack Notifications: epoll Architecture: amd64 Disabled: none Here is the Mac OSX mono 2.10.4 run: $ mono main.exe Running benchmark struct sum: 589998356.48, time: 9.010549 secs class sum: 589998356.48, time: 30.67357 secs Here is the Ubuntu 11.04 mono 2.6.7 run: $ mono main.exe Running benchmark struct sum: 589998356.48, time: 2.737732 secs class sum: 589998356.48, time: 7.83984 secs Note that the running time for mono 2.6.7 is ~4x faster than mono 2.10.4 on the same box (and the linux run has the disadvantage of running on a VM). The struct test is most likely not exercising the GC and the later is. I suspect given the consistent performance difference is *not* a GC issue, rather a difference in the JIT code generation. Let me know if there is other information I can provide. Thanks. Jonathan On Aug 27, 2011, at 1:52 PM, Slide wrote: On Sat, Aug 27, 2011 at 10:27 AM, Jonathan Shore jonathan.sh...@gmail.com wrote: Hi, I was doing some benchmarks of struct vs class based creation (I have an application that will generate millions of small objects). I was doing the tests in a ubuntu 11.4 VM on my mac pro and found the following: mono 2.6.7 was 4x faster on my linux VM than 2.10.4 running on OSX (same machine) I don't know whether this may be because of one of the following: - performance in 2.10.4 regressed vs 2.6.7 - mono JIT implementation for OSX has a completely different JiT codebase and does not perform - difference in GC (only relevant for second part of the test) Note that I tried this with separate compilations with mcs -optimize+ on both environments as well as running the same exe on both. I can live with slower performance on OSX, but want to make sure that linux and windows versions of mono 2.10.x have the performance of 2.6.7 or better. Can someone clue me in? I've included the simple test code with this posting. Thanks Jonathan Can you publish your benchmark numbers and for what machines you are running on? slide -- slide-o-blog http://slide-o-blog.blogspot.com/ ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] substantial performance regression between 2.10 and 2.6 or impl diff?
On Aug 27, 2011, at 2:20 PM, Bojan Rajkovic wrote: On Aug 27, 2011, at 2:15 PM, Jonathan Shore wrote: My machine is an old 2006 Mac Pro 1,1 2 x Xeon 5130 (64 bit) running OSX lion. Here are the respective versions of mono: Would 32 vs 64 really make that much of a difference? I have used 32 bit C applications without any noticable perf difference. I am worried that there may also be a problem with 2.10.x on other platforms. It would put my mind at ease if this is just a OSX implementation issue. I don't know if I can reasonably install both 2.6.x and 2.10.x on linux without hand compiling and doing some magic in config.Would be nice to know that this benchmark performs the same or better from 2.6 to 2.10. Jonathan ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] substantial performance regression between 2.10 and 2.6 or impl diff?
On Aug 27, 2011, at 2:30 PM, Jonathan Shore wrote: On Aug 27, 2011, at 2:20 PM, Bojan Rajkovic wrote: On Aug 27, 2011, at 2:15 PM, Jonathan Shore wrote: My machine is an old 2006 Mac Pro 1,1 2 x Xeon 5130 (64 bit) running OSX lion. Here are the respective versions of mono: Would 32 vs 64 really make that much of a difference? I have used 32 bit C applications without any noticable perf difference. I am worried that there may also be a problem with 2.10.x on other platforms. It would put my mind at ease if this is just a OSX implementation issue. I don't know if I can reasonably install both 2.6.x and 2.10.x on linux without hand compiling and doing some magic in config.Would be nice to know that this benchmark performs the same or better from 2.6 to 2.10. Jonathan Might make a difference with respect to performance if your struct can fit in the registers on 64-bit, but cannot on 32-bit. :-) If you could post your benchmark code, more people could test. —Bojan ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] substantial performance regression between 2.10 and 2.6 or impl diff?
I guess the code was trimmed off in the mailer (I sent in the initial note). Here it is inlined as text: using System; namespace TestHeap { public interface IDatum { longTimestamp { get; } } public struct SDelta : IDatum { public SDelta (long timestamp, long id, double size, double price, double maxsize = 0) { _timestamp = timestamp; _id = id; _size = size; _price = price; _maxsize = maxsize; } // Properties public long Timestamp { get { return _timestamp; } } public long Id { get { return _id; } } public double Size { get { return _size; } } public double Price { get { return _price; } } public double MaxSize { get { return _maxsize; } } // Variables private long_timestamp; private long_id; private double _size; private double _price; private double _maxsize; } public class CDelta : IDatum { public CDelta (long timestamp, long id, double size, double price, double maxsize = 0) { _timestamp = timestamp; _id = id; _size = size; _price = price; _maxsize = maxsize; } // Properties public long Timestamp { get { return _timestamp; } } public long Id { get { return _id; } } public double Size { get { return _size; } } public double Price { get { return _price; } } public double MaxSize { get { return _maxsize; } } // Variables private long_timestamp; private long_id; private double _size; private double _price; private double _maxsize; } class MainClass { private static SDelta f (int i) { return new SDelta (i*1000, i, 1e6, 1.400 + i / 1000); } private static CDelta g (int i) { return new CDelta (i*1000, i, 1e6, 1.400 + i / 1000); } public static void Main (string[] args) { Console.WriteLine (Running benchmark); // struct test long Tnow1 = DateTime.Now.Ticks; double sum1 = 0; for (int i = 0 ; i 1 ; i++) sum1 += f(i).Price; long Tend1 = DateTime.Now.Ticks; // class test long Tnow2 = DateTime.Now.Ticks; double sum2 = 0; for (int i = 0 ; i 1 ; i++) sum2 += g(i).Price; long Tend2 = DateTime.Now.Ticks; Console.WriteLine (struct sum: + sum1 + , time: + ((Tend1 - Tnow1) / 1e7) + secs); Console.WriteLine (class sum: + sum2 + , time: + ((Tend2 - Tnow2) / 1e7) + secs); } } } On Aug 27, 2011, at 2:34 PM, Bojan Rajkovic wrote: On Aug 27, 2011, at 2:30 PM, Jonathan Shore wrote: On Aug 27, 2011, at 2:20 PM, Bojan Rajkovic wrote: On Aug 27, 2011, at 2:15 PM, Jonathan Shore wrote: My machine is an old 2006 Mac Pro 1,1 2 x Xeon 5130 (64 bit) running OSX lion. Here are the respective versions of mono: Would 32 vs 64 really make that much of a difference? I have used 32 bit C applications without any noticable perf difference. I am worried that there may also be a problem with 2.10.x on other platforms. It would put my mind at ease if this is just a OSX implementation issue. I don't know if I can
Re: [Mono-dev] substantial performance regression between 2.10 and 2.6 or impl diff?
Ok. When you have a chance can you indicate your marks CPU? I expect a reasonably modern CPU to be 2 - 6x fast than my sluggish cpu. Thanks. So for instance my mac X5130 rates at 12.7 CINT 2006 vs 28.6 On Aug 27, 2011, at 2:57 PM, Slide wrote: I just ran on ubuntu 64bit with mono 2.10.1 and got better numbers than your 2.6.7. I had to run somewhere quick but will publish the numbers when I get back. On Aug 27, 2011 11:16 AM, Jonathan Shore jonathan.sh...@gmail.com wrote: My machine is an old 2006 Mac Pro 1,1 2 x Xeon 5130 (64 bit) running OSX lion. Here are the respective versions of mono: Mono on OSX: Mono JIT compiler version 2.10.4 (tarball Mon Aug 8 22:03:39 EDT 2011) Copyright (C) 2002-2011 Novell, Inc, Xamarin, Inc and Contributors. www.mono-project.com TLS: normal SIGSEGV: normal Notification: kqueue Architecture: x86 Disabled: none Misc: debugger softdebug LLVM: yes(2.9svn-mono) GC: Included Boehm (with typed GC) Mono on Ubuntu 11.04 (running in VMWare VM on same machine): Mono JIT compiler version 2.6.7 (Debian 2.6.7-5ubuntu3) Copyright (C) 2002-2010 Novell, Inc and Contributors. www.mono-project.com TLS: __thread GC: Included Boehm (with typed GC and Parallel Mark) SIGSEGV: altstack Notifications: epoll Architecture: amd64 Disabled: none Here is the Mac OSX mono 2.10.4 run: $ mono main.exe Running benchmark struct sum: 589998356.48, time: 9.010549 secs class sum: 589998356.48, time: 30.67357 secs Here is the Ubuntu 11.04 mono 2.6.7 run: $ mono main.exe Running benchmark struct sum: 589998356.48, time: 2.737732 secs class sum: 589998356.48, time: 7.83984 secs Note that the running time for mono 2.6.7 is ~4x faster than mono 2.10.4 on the same box (and the linux run has the disadvantage of running on a VM). The struct test is most likely not exercising the GC and the later is. I suspect given the consistent performance difference is *not* a GC issue, rather a difference in the JIT code generation. Let me know if there is other information I can provide. Thanks. Jonathan On Aug 27, 2011, at 1:52 PM, Slide wrote: On Sat, Aug 27, 2011 at 10:27 AM, Jonathan Shore jonathan.sh...@gmail.com wrote: Hi, I was doing some benchmarks of struct vs class based creation (I have an application that will generate millions of small objects). I was doing the tests in a ubuntu 11.4 VM on my mac pro and found the following: mono 2.6.7 was 4x faster on my linux VM than 2.10.4 running on OSX (same machine) I don't know whether this may be because of one of the following: - performance in 2.10.4 regressed vs 2.6.7 - mono JIT implementation for OSX has a completely different JiT codebase and does not perform - difference in GC (only relevant for second part of the test) Note that I tried this with separate compilations with mcs -optimize+ on both environments as well as running the same exe on both. I can live with slower performance on OSX, but want to make sure that linux and windows versions of mono 2.10.x have the performance of 2.6.7 or better. Can someone clue me in? I've included the simple test code with this posting. Thanks Jonathan Can you publish your benchmark numbers and for what machines you are running on? slide -- slide-o-blog http://slide-o-blog.blogspot.com/ ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] substantial performance regression between 2.10 and 2.6 or impl diff?
Here are my results when running your benchmark. My PC: a very old a cheap Windows XP Celeron. I compiled the application with .NET, and ran it with Mono. I ran the benchmark 3 times to make sure results were reliable. Mono 2.6.3: structs: just below 15 seconds classes: just below 40 seconds Mono 2.10.3: structs: just above 19 seconds classes: just above 40 seconds So classes are a tiny bit slower, and structs are a fair bit slower. Jonathan Shore wrote: I guess the code was trimmed off in the mailer (I sent in the initial note). Here it is inlined as text: using System; namespace TestHeap { public interface IDatum { longTimestamp { get; } } public struct SDelta : IDatum { public SDelta (long timestamp, long id, double size, double price, double maxsize = 0) { _timestamp = timestamp; _id = id; _size = size; _price = price; _maxsize = maxsize; } // Properties public long Timestamp { get { return _timestamp; } } public long Id { get { return _id; } } public double Size { get { return _size; } } public double Price { get { return _price; } } public double MaxSize { get { return _maxsize; } } // Variables private long_timestamp; private long_id; private double _size; private double _price; private double _maxsize; } public class CDelta : IDatum { public CDelta (long timestamp, long id, double size, double price, double maxsize = 0) { _timestamp = timestamp; _id = id; _size = size; _price = price; _maxsize = maxsize; } // Properties public long Timestamp { get { return _timestamp; } } public long Id { get { return _id; } } public double Size { get { return _size; } } public double Price { get { return _price; } } public double MaxSize { get { return _maxsize; } } // Variables private long_timestamp; private long_id; private double _size; private double _price; private double _maxsize; } class MainClass { private static SDelta f (int i) { return new SDelta (i*1000, i, 1e6, 1.400 + i / 1000); } private static CDelta g (int i) { return new CDelta (i*1000, i, 1e6, 1.400 + i / 1000); } public static void Main (string[] args) { Console.WriteLine (Running benchmark); // struct test long Tnow1 = DateTime.Now.Ticks; double sum1 = 0; for (int i = 0 ; i 1 ; i++) sum1 += f(i).Price; long Tend1 = DateTime.Now.Ticks; // class test long Tnow2 = DateTime.Now.Ticks; double sum2 = 0; for (int i = 0 ; i 1 ; i++) sum2 += g(i).Price; long Tend2 = DateTime.Now.Ticks; Console.WriteLine (struct sum: + sum1 + , time: + ((Tend1 - Tnow1) / 1e7) + secs); Console.WriteLine (class sum: + sum2 + , time: + ((Tend2 - Tnow2) / 1e7) + secs); } } } On Aug 27, 2011, at 2:34 PM, Bojan Rajkovic wrote: On Aug 27, 2011, at 2:30 PM, Jonathan Shore wrote: On Aug 27, 2011, at 2:20 PM, Bojan Rajkovic wrote: On Aug 27, 2011, at 2:15 PM, Jonathan Shore wrote: My machine is an old 2006 Mac Pro 1,1 2 x Xeon 5130 (64 bit) running OSX lion.
Re: [Mono-dev] substantial performance regression between 2.10 and 2.6 or impl diff?
You could probably install the Mac OSX 2.6.7 Mono pretty quickly to see if it's a difference caused by architectures or by a change between Mono versions. http://www.go-mono.com/mono-downloads/download.html Jonathan On 8/27/2011 2:36 PM, Jonathan Shore wrote: Ok. When you have a chance can you indicate your marks CPU? I expect a reasonably modern CPU to be 2 - 6x fast than my sluggish cpu. Thanks. So for instance my mac X5130 rates at 12.7 CINT 2006 vs 28.6 On Aug 27, 2011, at 2:57 PM, Slide wrote: I just ran on ubuntu 64bit with mono 2.10.1 and got better numbers than your 2.6.7. I had to run somewhere quick but will publish the numbers when I get back. On Aug 27, 2011 11:16 AM, Jonathan Shore jonathan.sh...@gmail.com mailto:jonathan.sh...@gmail.com wrote: My machine is an old 2006 Mac Pro 1,1 2 x Xeon 5130 (64 bit) running OSX lion. Here are the respective versions of mono: Mono on OSX: Mono JIT compiler version 2.10.4 (tarball Mon Aug 8 22:03:39 EDT 2011) Copyright (C) 2002-2011 Novell, Inc, Xamarin, Inc and Contributors. www.mono-project.com http://www.mono-project.com/ TLS: normal SIGSEGV: normal Notification: kqueue Architecture: x86 Disabled: none Misc: debugger softdebug LLVM: yes(2.9svn-mono) GC: Included Boehm (with typed GC) Mono on Ubuntu 11.04 (running in VMWare VM on same machine): Mono JIT compiler version 2.6.7 (Debian 2.6.7-5ubuntu3) Copyright (C) 2002-2010 Novell, Inc and Contributors. www.mono-project.com http://www.mono-project.com/ TLS: __thread GC: Included Boehm (with typed GC and Parallel Mark) SIGSEGV: altstack Notifications: epoll Architecture: amd64 Disabled: none Here is the Mac OSX mono 2.10.4 run: $ mono main.exe Running benchmark struct sum: 589998356.48, time: 9.010549 secs class sum: 589998356.48, time: 30.67357 secs Here is the Ubuntu 11.04 mono 2.6.7 run: $ mono main.exe Running benchmark struct sum: 589998356.48, time: 2.737732 secs class sum: 589998356.48, time: 7.83984 secs Note that the running time for mono 2.6.7 is ~4x faster than mono 2.10.4 on the same box (and the linux run has the disadvantage of running on a VM). The struct test is most likely not exercising the GC and the later is. I suspect given the consistent performance difference is *not* a GC issue, rather a difference in the JIT code generation. Let me know if there is other information I can provide. Thanks. Jonathan On Aug 27, 2011, at 1:52 PM, Slide wrote: On Sat, Aug 27, 2011 at 10:27 AM, Jonathan Shore jonathan.sh...@gmail.com mailto:jonathan.sh...@gmail.com wrote: Hi, I was doing some benchmarks of struct vs class based creation (I have an application that will generate millions of small objects). I was doing the tests in a ubuntu 11.4 VM on my mac pro and found the following: mono 2.6.7 was 4x faster on my linux VM than 2.10.4 running on OSX (same machine) I don't know whether this may be because of one of the following: - performance in 2.10.4 regressed vs 2.6.7 - mono JIT implementation for OSX has a completely different JiT codebase and does not perform - difference in GC (only relevant for second part of the test) Note that I tried this with separate compilations with mcs -optimize+ on both environments as well as running the same exe on both. I can live with slower performance on OSX, but want to make sure that linux and windows versions of mono 2.10.x have the performance of 2.6.7 or better. Can someone clue me in? I've included the simple test code with this posting. Thanks Jonathan Can you publish your benchmark numbers and for what machines you are running on? slide -- slide-o-blog http://slide-o-blog.blogspot.com/ ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] substantial performance regression between 2.10 and 2.6 or impl diff?
Hi all, Here's my results from 64-bit Mono master on OS X 10.7.1. The CPU is an Intel Core i7 @ 2.0 GHz, with 8 GB of RAM backing it: struct sum: 589998356.48, time: 1.593511 secs class sum: 589998356.48, time: 14.413891 secs Classes are almost twice as slow, which is somewhat odd. —Bojan On Aug 27, 2011, at 3:36 PM, Jonathan Shore wrote: Ok. When you have a chance can you indicate your marks CPU? I expect a reasonably modern CPU to be 2 - 6x fast than my sluggish cpu. Thanks. So for instance my mac X5130 rates at 12.7 CINT 2006 vs 28.6 On Aug 27, 2011, at 2:57 PM, Slide wrote: I just ran on ubuntu 64bit with mono 2.10.1 and got better numbers than your 2.6.7. I had to run somewhere quick but will publish the numbers when I get back. On Aug 27, 2011 11:16 AM, Jonathan Shore jonathan.sh...@gmail.com wrote: My machine is an old 2006 Mac Pro 1,1 2 x Xeon 5130 (64 bit) running OSX lion. Here are the respective versions of mono: Mono on OSX: Mono JIT compiler version 2.10.4 (tarball Mon Aug 8 22:03:39 EDT 2011) Copyright (C) 2002-2011 Novell, Inc, Xamarin, Inc and Contributors. www.mono-project.com TLS: normal SIGSEGV: normal Notification: kqueue Architecture: x86 Disabled: none Misc: debugger softdebug LLVM: yes(2.9svn-mono) GC: Included Boehm (with typed GC) Mono on Ubuntu 11.04 (running in VMWare VM on same machine): Mono JIT compiler version 2.6.7 (Debian 2.6.7-5ubuntu3) Copyright (C) 2002-2010 Novell, Inc and Contributors. www.mono-project.com TLS: __thread GC: Included Boehm (with typed GC and Parallel Mark) SIGSEGV: altstack Notifications: epoll Architecture: amd64 Disabled: none Here is the Mac OSX mono 2.10.4 run: $ mono main.exe Running benchmark struct sum: 589998356.48, time: 9.010549 secs class sum: 589998356.48, time: 30.67357 secs Here is the Ubuntu 11.04 mono 2.6.7 run: $ mono main.exe Running benchmark struct sum: 589998356.48, time: 2.737732 secs class sum: 589998356.48, time: 7.83984 secs Note that the running time for mono 2.6.7 is ~4x faster than mono 2.10.4 on the same box (and the linux run has the disadvantage of running on a VM). The struct test is most likely not exercising the GC and the later is. I suspect given the consistent performance difference is *not* a GC issue, rather a difference in the JIT code generation. Let me know if there is other information I can provide. Thanks. Jonathan On Aug 27, 2011, at 1:52 PM, Slide wrote: On Sat, Aug 27, 2011 at 10:27 AM, Jonathan Shore jonathan.sh...@gmail.com wrote: Hi, I was doing some benchmarks of struct vs class based creation (I have an application that will generate millions of small objects). I was doing the tests in a ubuntu 11.4 VM on my mac pro and found the following: mono 2.6.7 was 4x faster on my linux VM than 2.10.4 running on OSX (same machine) I don't know whether this may be because of one of the following: - performance in 2.10.4 regressed vs 2.6.7 - mono JIT implementation for OSX has a completely different JiT codebase and does not perform - difference in GC (only relevant for second part of the test) Note that I tried this with separate compilations with mcs -optimize+ on both environments as well as running the same exe on both. I can live with slower performance on OSX, but want to make sure that linux and windows versions of mono 2.10.x have the performance of 2.6.7 or better. Can someone clue me in? I've included the simple test code with this posting. Thanks Jonathan Can you publish your benchmark numbers and for what machines you are running on? slide -- slide-o-blog http://slide-o-blog.blogspot.com/ ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] substantial performance regression between 2.10 and 2.6 or impl diff?
I think all of these results point to a regression in performance between 2.10.x and 2.6.x.I ran the 2.6.x version on OSX and found it to be faster than 2.10.x. I will run on mu core i7 linux box and see how that fairs. On Aug 27, 2011, at 4:09 PM, Bojan Rajkovic wrote: Hi all, Here's my results from 64-bit Mono master on OS X 10.7.1. The CPU is an Intel Core i7 @ 2.0 GHz, with 8 GB of RAM backing it: struct sum: 589998356.48, time: 1.593511 secs class sum: 589998356.48, time: 14.413891 secs Classes are almost twice as slow, which is somewhat odd. —Bojan On Aug 27, 2011, at 3:36 PM, Jonathan Shore wrote: Ok. When you have a chance can you indicate your marks CPU? I expect a reasonably modern CPU to be 2 - 6x fast than my sluggish cpu. Thanks. So for instance my mac X5130 rates at 12.7 CINT 2006 vs 28.6 On Aug 27, 2011, at 2:57 PM, Slide wrote: I just ran on ubuntu 64bit with mono 2.10.1 and got better numbers than your 2.6.7. I had to run somewhere quick but will publish the numbers when I get back. On Aug 27, 2011 11:16 AM, Jonathan Shore jonathan.sh...@gmail.com wrote: My machine is an old 2006 Mac Pro 1,1 2 x Xeon 5130 (64 bit) running OSX lion. Here are the respective versions of mono: Mono on OSX: Mono JIT compiler version 2.10.4 (tarball Mon Aug 8 22:03:39 EDT 2011) Copyright (C) 2002-2011 Novell, Inc, Xamarin, Inc and Contributors. www.mono-project.com TLS: normal SIGSEGV: normal Notification: kqueue Architecture: x86 Disabled: none Misc: debugger softdebug LLVM: yes(2.9svn-mono) GC: Included Boehm (with typed GC) Mono on Ubuntu 11.04 (running in VMWare VM on same machine): Mono JIT compiler version 2.6.7 (Debian 2.6.7-5ubuntu3) Copyright (C) 2002-2010 Novell, Inc and Contributors. www.mono-project.com TLS: __thread GC: Included Boehm (with typed GC and Parallel Mark) SIGSEGV: altstack Notifications: epoll Architecture: amd64 Disabled: none Here is the Mac OSX mono 2.10.4 run: $ mono main.exe Running benchmark struct sum: 589998356.48, time: 9.010549 secs class sum: 589998356.48, time: 30.67357 secs Here is the Ubuntu 11.04 mono 2.6.7 run: $ mono main.exe Running benchmark struct sum: 589998356.48, time: 2.737732 secs class sum: 589998356.48, time: 7.83984 secs Note that the running time for mono 2.6.7 is ~4x faster than mono 2.10.4 on the same box (and the linux run has the disadvantage of running on a VM). The struct test is most likely not exercising the GC and the later is. I suspect given the consistent performance difference is *not* a GC issue, rather a difference in the JIT code generation. Let me know if there is other information I can provide. Thanks. Jonathan On Aug 27, 2011, at 1:52 PM, Slide wrote: On Sat, Aug 27, 2011 at 10:27 AM, Jonathan Shore jonathan.sh...@gmail.com wrote: Hi, I was doing some benchmarks of struct vs class based creation (I have an application that will generate millions of small objects). I was doing the tests in a ubuntu 11.4 VM on my mac pro and found the following: mono 2.6.7 was 4x faster on my linux VM than 2.10.4 running on OSX (same machine) I don't know whether this may be because of one of the following: - performance in 2.10.4 regressed vs 2.6.7 - mono JIT implementation for OSX has a completely different JiT codebase and does not perform - difference in GC (only relevant for second part of the test) Note that I tried this with separate compilations with mcs -optimize+ on both environments as well as running the same exe on both. I can live with slower performance on OSX, but want to make sure that linux and windows versions of mono 2.10.x have the performance of 2.6.7 or better. Can someone clue me in? I've included the simple test code with this posting. Thanks Jonathan Can you publish your benchmark numbers and for what machines you are running on? slide -- slide-o-blog http://slide-o-blog.blogspot.com/ ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] substantial performance regression between 2.10 and 2.6 or impl diff?
Follow-up: Reminder, my results were: Mono 2.6.3: structs: just below 15 seconds classes: just below 40 seconds Mono 2.10.3: structs: just above 19 seconds classes: just above 40 seconds And this is with Mono 2.8.1: structs: just below 15 seconds classes: just above 40 seconds So structs are as fast in Mono 2.6 and 2.8 here, but classes are a tiny bit slower in 2.8, the same as with Mono 2.10. Looking at my results only, you'd think there were 2 regressions: classes got slower between 2.6 and 2.8, and structs got slower between 2.8 and 2.10. Jonathan Shore wrote: I think all of these results point to a regression in performance between 2.10.x and 2.6.x.I ran the 2.6.x version on OSX and found it to be faster than 2.10.x. I will run on mu core i7 linux box and see how that fairs. On Aug 27, 2011, at 4:09 PM, Bojan Rajkovic wrote: Hi all, Here's my results from 64-bit Mono master on OS X 10.7.1. The CPU is an Intel Core i7 @ 2.0 GHz, with 8 GB of RAM backing it: struct sum: 589998356.48, time: 1.593511 secs class sum: 589998356.48, time: 14.413891 secs Classes are almost twice as slow, which is somewhat odd. —Bojan On Aug 27, 2011, at 3:36 PM, Jonathan Shore wrote: Ok. When you have a chance can you indicate your marks CPU? I expect a reasonably modern CPU to be 2 - 6x fast than my sluggish cpu. Thanks. So for instance my mac X5130 rates at 12.7 CINT 2006 vs 28.6 On Aug 27, 2011, at 2:57 PM, Slide wrote: I just ran on ubuntu 64bit with mono 2.10.1 and got better numbers than your 2.6.7. I had to run somewhere quick but will publish the numbers when I get back. On Aug 27, 2011 11:16 AM, Jonathan Shore lt;jonathan.sh...@gmail.comgt; wrote: My machine is an old 2006 Mac Pro 1,1 2 x Xeon 5130 (64 bit) running OSX lion. Here are the respective versions of mono: Mono on OSX: Mono JIT compiler version 2.10.4 (tarball Mon Aug 8 22:03:39 EDT 2011) Copyright (C) 2002-2011 Novell, Inc, Xamarin, Inc and Contributors. www.mono-project.com TLS: normal SIGSEGV: normal Notification: kqueue Architecture: x86 Disabled: none Misc: debugger softdebug LLVM: yes(2.9svn-mono) GC: Included Boehm (with typed GC) Mono on Ubuntu 11.04 (running in VMWare VM on same machine): Mono JIT compiler version 2.6.7 (Debian 2.6.7-5ubuntu3) Copyright (C) 2002-2010 Novell, Inc and Contributors. www.mono-project.com TLS: __thread GC: Included Boehm (with typed GC and Parallel Mark) SIGSEGV: altstack Notifications: epoll Architecture: amd64 Disabled: none Here is the Mac OSX mono 2.10.4 run: $ mono main.exe Running benchmark struct sum: 589998356.48, time: 9.010549 secs class sum: 589998356.48, time: 30.67357 secs Here is the Ubuntu 11.04 mono 2.6.7 run: $ mono main.exe Running benchmark struct sum: 589998356.48, time: 2.737732 secs class sum: 589998356.48, time: 7.83984 secs Note that the running time for mono 2.6.7 is ~4x faster than mono 2.10.4 on the same box (and the linux run has the disadvantage of running on a VM). The struct test is most likely not exercising the GC and the later is. I suspect given the consistent performance difference is *not* a GC issue, rather a difference in the JIT code generation. Let me know if there is other information I can provide. Thanks. Jonathan On Aug 27, 2011, at 1:52 PM, Slide wrote: On Sat, Aug 27, 2011 at 10:27 AM, Jonathan Shore lt;jonathan.sh...@gmail.comgt; wrote: Hi, I was doing some benchmarks of struct vs class based creation (I have an application that will generate millions of small objects). I was doing the tests in a ubuntu 11.4 VM on my mac pro and found the following: mono 2.6.7 was 4x faster on my linux VM than 2.10.4 running on OSX (same machine) I don't know whether this may be because of one of the following: - performance in 2.10.4 regressed vs 2.6.7 - mono JIT implementation for OSX has a completely different JiT codebase and does not perform - difference in GC (only relevant for second part of the test) Note that I tried this with separate compilations with mcs -optimize+ on both environments as well as running the same exe on both. I can live with slower performance on OSX, but want to make sure that linux and windows versions of mono 2.10.x have the performance of 2.6.7 or better. Can someone clue me in? I've included the simple test code with this posting. Thanks Jonathan Can you publish your benchmark numbers and for what machines you are running on? slide -- slide-o-blog http://slide-o-blog.blogspot.com/ ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing
Re: [Mono-dev] substantial performance regression between 2.10 and 2.6 or impl diff?
There are a few points to note here. Linux with the default GC is much faster than OSX since the laster can't use fast object allocation. OSX 2.6 shipped with default parallel mark enabled on OSX, but it was later disabled due to bugs in boehm. Given no one is actively working on boehm, this was the best option. My suggestion is that you try using the new GC try running 2.10 with --gc=sgen. Performance will be substantially better for your test. On Sat, Aug 27, 2011 at 2:27 PM, Jonathan Shore jonathan.sh...@gmail.comwrote: Hi, I was doing some benchmarks of struct vs class based creation (I have an application that will generate millions of small objects). I was doing the tests in a ubuntu 11.4 VM on my mac pro and found the following: mono *2.6.7 was 4x faster* on my linux VM *than 2.10.4 *running on OSX (same machine) I don't know whether this may be because of one of the following: - performance in 2.10.4 regressed vs 2.6.7 - mono JIT implementation for OSX has a completely different JiT codebase and does not perform - difference in GC (only relevant for second part of the test) Note that I tried this with separate compilations with mcs -optimize+ on both environments as well as running the same exe on both. I can live with slower performance on OSX, but want to make sure that linux and windows versions of mono 2.10.x have the performance of 2.6.7 or better. Can someone clue me in? I've included the simple test code with this posting. Thanks Jonathan ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] substantial performance regression between 2.10 and 2.6 or impl diff?
Rodrigo, The class based object part of the test is definitely GC sensitive. However, I and others have noted a performance drop for the struct based test (the other test in this simple benchmark), which should not be impacted by GC. I did two tests of the struct-based approach, one with 2.6.x OSX and 2.10.x and there was a consistent difference. I also have a linux box, so am trying to build 2.10.x (as the packages available are 2.6.x). I expect, as others have found, that the struct performance will be lower in 2.10.x, even on linux. I will post results later if I manage to build this successfully. Jonathan On Aug 27, 2011, at 7:09 PM, Rodrigo Kumpera wrote: There are a few points to note here. Linux with the default GC is much faster than OSX since the laster can't use fast object allocation. OSX 2.6 shipped with default parallel mark enabled on OSX, but it was later disabled due to bugs in boehm. Given no one is actively working on boehm, this was the best option. My suggestion is that you try using the new GC try running 2.10 with --gc=sgen. Performance will be substantially better for your test. On Sat, Aug 27, 2011 at 2:27 PM, Jonathan Shore jonathan.sh...@gmail.com wrote: Hi, I was doing some benchmarks of struct vs class based creation (I have an application that will generate millions of small objects). I was doing the tests in a ubuntu 11.4 VM on my mac pro and found the following: mono 2.6.7 was 4x faster on my linux VM than 2.10.4 running on OSX (same machine) I don't know whether this may be because of one of the following: - performance in 2.10.4 regressed vs 2.6.7 - mono JIT implementation for OSX has a completely different JiT codebase and does not perform - difference in GC (only relevant for second part of the test) Note that I tried this with separate compilations with mcs -optimize+ on both environments as well as running the same exe on both. I can live with slower performance on OSX, but want to make sure that linux and windows versions of mono 2.10.x have the performance of 2.6.7 or better. Can someone clue me in? I've included the simple test code with this posting. Thanks Jonathan ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] substantial performance regression between 2.10 and 2.6 or impl diff?
On Aug 27, 2011, at 7:09 PM, Rodrigo Kumpera wrote: There are a few points to note here. Linux with the default GC is much faster than OSX since the laster can't use fast object allocation. OSX 2.6 shipped with default parallel mark enabled on OSX, but it was later disabled due to bugs in boehm. Given no one is actively working on boehm, this was the best option. My suggestion is that you try using the new GC try running 2.10 with --gc=sgen. Performance will be substantially better for your test. Thanks for the pointer re: sgen. sgen is dramatically faster. I will also try this on linux. ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] substantial performance regression between 2.10 and 2.6 or impl diff?
Here are the #s for ubuntu 11.4 on a core i7 box with different versions of mono. Mono 2.6.7 (default Boehm GC) struct sum: 589998356.48, time: 1.7 secs class sum: 589998356.48, time: 9.5 secs Mono 2.6.10 (default Boehm GC) struct sum: 589998356.48, time: 1.7 secs class sum: 589998356.48, time: 17.2 secs Mono 2.6.10 (default Generational GC) struct sum: 589998356.48, time: 1.7 secs class sum: 589998356.48, time: 2.0 secs The linux implementations seem to have the same performance across the struct-based benchmarks. The default Boehm GC in 2.6.10 is about 2x slower than the implementation or parameters in 2.6.7. Finally, the generational GC is more than 4x faster than the best Boehm GC time (for this trivial benchmark). So I with gc=sgen, quite fine with the results. I'm not sure if sgen is stable or not, but perhaps it should be the default on osx. The osx performance is very poor (not shown here). On Aug 27, 2011, at 7:09 PM, Rodrigo Kumpera wrote: There are a few points to note here. Linux with the default GC is much faster than OSX since the laster can't use fast object allocation. OSX 2.6 shipped with default parallel mark enabled on OSX, but it was later disabled due to bugs in boehm. Given no one is actively working on boehm, this was the best option. My suggestion is that you try using the new GC try running 2.10 with --gc=sgen. Performance will be substantially better for your test. On Sat, Aug 27, 2011 at 2:27 PM, Jonathan Shore jonathan.sh...@gmail.com wrote: Hi, I was doing some benchmarks of struct vs class based creation (I have an application that will generate millions of small objects). I was doing the tests in a ubuntu 11.4 VM on my mac pro and found the following: mono 2.6.7 was 4x faster on my linux VM than 2.10.4 running on OSX (same machine) I don't know whether this may be because of one of the following: - performance in 2.10.4 regressed vs 2.6.7 - mono JIT implementation for OSX has a completely different JiT codebase and does not perform - difference in GC (only relevant for second part of the test) Note that I tried this with separate compilations with mcs -optimize+ on both environments as well as running the same exe on both. I can live with slower performance on OSX, but want to make sure that linux and windows versions of mono 2.10.x have the performance of 2.6.7 or better. Can someone clue me in? I've included the simple test code with this posting. Thanks Jonathan ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] substantial performance regression between 2.10 and 2.6 or impl diff?
On Sat, Aug 27, 2011 at 10:27 AM, Jonathan Shore jonathan.sh...@gmail.comwrote: Hi, I was doing some benchmarks of struct vs class based creation (I have an application that will generate millions of small objects). I was doing the tests in a ubuntu 11.4 VM on my mac pro and found the following: mono *2.6.7 was 4x faster* on my linux VM *than 2.10.4 *running on OSX (same machine) I don't know whether this may be because of one of the following: - performance in 2.10.4 regressed vs 2.6.7 - mono JIT implementation for OSX has a completely different JiT codebase and does not perform - difference in GC (only relevant for second part of the test) Note that I tried this with separate compilations with mcs -optimize+ on both environments as well as running the same exe on both. I can live with slower performance on OSX, but want to make sure that linux and windows versions of mono 2.10.x have the performance of 2.6.7 or better. Can someone clue me in? I've included the simple test code with this posting. Thanks Jonathan Can you publish your benchmark numbers and for what machines you are running on? slide -- slide-o-blog http://slide-o-blog.blogspot.com/ ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] substantial performance regression between 2.10 and 2.6 or impl diff?
I just ran on ubuntu 64bit with mono 2.10.1 and got better numbers than your 2.6.7. I had to run somewhere quick but will publish the numbers when I get back. On Aug 27, 2011 11:16 AM, Jonathan Shore jonathan.sh...@gmail.com wrote: My machine is an old 2006 Mac Pro 1,1 2 x Xeon 5130 (64 bit) running OSX lion. Here are the respective versions of mono: Mono on OSX: Mono JIT compiler version 2.10.4 (tarball Mon Aug 8 22:03:39 EDT 2011) Copyright (C) 2002-2011 Novell, Inc, Xamarin, Inc and Contributors. www.mono-project.com TLS: normal SIGSEGV: normal Notification: kqueue Architecture: x86 Disabled: none Misc: debugger softdebug LLVM: yes(2.9svn-mono) GC: Included Boehm (with typed GC) Mono on Ubuntu 11.04 (running in VMWare VM on same machine): Mono JIT compiler version 2.6.7 (Debian 2.6.7-5ubuntu3) Copyright (C) 2002-2010 Novell, Inc and Contributors. www.mono-project.com TLS: __thread GC: Included Boehm (with typed GC and Parallel Mark) SIGSEGV: altstack Notifications: epoll Architecture: amd64 Disabled: none Here is the Mac OSX mono 2.10.4 run: $ mono main.exe Running benchmark struct sum: 589998356.48, time: 9.010549 secs class sum: 589998356.48, time: 30.67357 secs Here is the Ubuntu 11.04 mono 2.6.7 run: $ mono main.exe Running benchmark struct sum: 589998356.48, time: 2.737732 secs class sum: 589998356.48, time: 7.83984 secs Note that the running time for mono 2.6.7 is ~4x faster than mono 2.10.4 on the same box (and the linux run has the disadvantage of running on a VM). The struct test is most likely not exercising the GC and the later is. I suspect given the consistent performance difference is *not* a GC issue, rather a difference in the JIT code generation. Let me know if there is other information I can provide. Thanks. Jonathan On Aug 27, 2011, at 1:52 PM, Slide wrote: On Sat, Aug 27, 2011 at 10:27 AM, Jonathan Shore jonathan.sh...@gmail.com wrote: Hi, I was doing some benchmarks of struct vs class based creation (I have an application that will generate millions of small objects). I was doing the tests in a ubuntu 11.4 VM on my mac pro and found the following: mono 2.6.7 was 4x faster on my linux VM than 2.10.4 running on OSX (same machine) I don't know whether this may be because of one of the following: - performance in 2.10.4 regressed vs 2.6.7 - mono JIT implementation for OSX has a completely different JiT codebase and does not perform - difference in GC (only relevant for second part of the test) Note that I tried this with separate compilations with mcs -optimize+ on both environments as well as running the same exe on both. I can live with slower performance on OSX, but want to make sure that linux and windows versions of mono 2.10.x have the performance of 2.6.7 or better. Can someone clue me in? I've included the simple test code with this posting. Thanks Jonathan Can you publish your benchmark numbers and for what machines you are running on? slide -- slide-o-blog http://slide-o-blog.blogspot.com/ ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] substantial performance regression between 2.10 and 2.6 or impl diff?
On Aug 27, 2011, at 2:15 PM, Jonathan Shore wrote: My machine is an old 2006 Mac Pro 1,1 2 x Xeon 5130 (64 bit) running OSX lion. Here are the respective versions of mono: Mono on OSX: Mono JIT compiler version 2.10.4 (tarball Mon Aug 8 22:03:39 EDT 2011) Copyright (C) 2002-2011 Novell, Inc, Xamarin, Inc and Contributors. www.mono-project.com TLS: normal SIGSEGV: normal Notification: kqueue Architecture: x86 Disabled:none Misc: debugger softdebug LLVM: yes(2.9svn-mono) GC: Included Boehm (with typed GC) Mono on Ubuntu 11.04 (running in VMWare VM on same machine): Mono JIT compiler version 2.6.7 (Debian 2.6.7-5ubuntu3) Copyright (C) 2002-2010 Novell, Inc and Contributors. www.mono-project.com TLS: __thread GC:Included Boehm (with typed GC and Parallel Mark) SIGSEGV: altstack Notifications: epoll Architecture: amd64 Disabled: none Here is the Mac OSX mono 2.10.4 run: $ mono main.exe Running benchmark struct sum: 589998356.48, time: 9.010549 secs class sum: 589998356.48, time: 30.67357 secs Here is the Ubuntu 11.04 mono 2.6.7 run: $ mono main.exe Running benchmark struct sum: 589998356.48, time: 2.737732 secs class sum: 589998356.48, time: 7.83984 secs Note that the running time for mono 2.6.7 is ~4x faster than mono 2.10.4 on the same box (and the linux run has the disadvantage of running on a VM). The struct test is most likely not exercising the GC and the later is. I suspect given the consistent performance difference is *not* a GC issue, rather a difference in the JIT code generation. Let me know if there is other information I can provide. Thanks. Jonathan On Aug 27, 2011, at 1:52 PM, Slide wrote: On Sat, Aug 27, 2011 at 10:27 AM, Jonathan Shore jonathan.sh...@gmail.com wrote: Hi, I was doing some benchmarks of struct vs class based creation (I have an application that will generate millions of small objects). I was doing the tests in a ubuntu 11.4 VM on my mac pro and found the following: mono 2.6.7 was 4x faster on my linux VM than 2.10.4 running on OSX (same machine) I don't know whether this may be because of one of the following: - performance in 2.10.4 regressed vs 2.6.7 - mono JIT implementation for OSX has a completely different JiT codebase and does not perform - difference in GC (only relevant for second part of the test) Note that I tried this with separate compilations with mcs -optimize+ on both environments as well as running the same exe on both. I can live with slower performance on OSX, but want to make sure that linux and windows versions of mono 2.10.x have the performance of 2.6.7 or better. Can someone clue me in? I've included the simple test code with this posting. Thanks Jonathan Can you publish your benchmark numbers and for what machines you are running on? slide -- slide-o-blog http://slide-o-blog.blogspot.com/ Hi Jonathan, Slide I suspect this is a difference between x86 and amd64 codegen. Mono on OS X is distributed as 32-bit only for now, though there is (somewhat-experimental) amd64/Darwin support in master. You can build it following the instructions at http://www.mono-project.com/Compiling_Mono_on_OSX. Try benchmarking that. —Bojan___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list