[Mono-dev] Remoting and thread pool limits
Hi,
I've found the following difference working with .NET and Mono remoting. In
.NET when the remoting ThreadPool reaches the pool's maximum (25 threads per
process), it is able to continue creating new threads. In fact, you can end
up having a remoting server with hundreds of threads.
In mono the behaviour is different. Once the limit is reached it starts
refusing connections.
The other drawback of the standard (1.1) remoting (.NET) is that once the
thread pool limit is reached, each new thread takes a lng time. In fact,
replacing the standard TCP channel by, for instance, the GenuineChannels
one, performance gets increased under heavy load.
The attached code creates a remoting server which implements one method. The
method is really simple:
public string GetVal()
{
Console.WriteLine("GetVal() - ThId:{0}",
Thread.CurrentThread.GetHashCode());
Thread.Sleep(1000);
return "Hi There";
}
Well, if you have a client launching 200 threads at the same time and
calling the server, it should take about 1 second to complete. Here are my
results using .NET (it doesn't finish with mono)
1 - Time 1102 ms
5 - Time 1011 ms
10 - Time 1002 ms
20 - Time 1011 ms
40 - Time 2003 ms
50 - Time 2013 ms
200 - Time 6019 ms
Using GenuineChannels (the difference here is how they implement the
threadpool), all get about 1 sec to finish.
Please find the sample code at
http://www.codicesoftware.com/testing/remotingtransmission.rar
Regards,
pablo
___
Mono-devel-list mailing list
Mono-devel-list@lists.ximian.com
http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Remoting and thread pool limits
Hi, pablosantosluac wrote: > Hi, > > I've found the following difference working with .NET and Mono remoting. In > .NET when the remoting ThreadPool reaches the pool's maximum (25 threads per > process), it is able to continue creating new threads. In fact, you can end > up having a remoting server with hundreds of threads. > > In mono the behaviour is different. Once the limit is reached it starts > refusing connections. This is unrelated to mono's remoting thread pool. You have probably ran your tests on Windows, where mono indeed fails with a GC failure when too many threads are created because the GC has a hard coded max thread count limit. On Linux (x86_64, Mono 1.2.6) I can finish the tests: Linux Mono client -> Linux Mono server (same machine) poseidon [~/foo] $ mono client/bin/Debug/client.exe tcp://localhost:8084/remote 1 - Time 1206 ms 5 - Time 2045 ms 10 - Time 3575 ms 20 - Time 8174 ms 40 - Time 12055 ms 50 - Time 8185 ms 200 - Time 46150 ms MS.NET client -> Linux Mono server (different machines) troll:/cygdrive/u/foo/client/bin/Debug $ ./client.exe tcp://poseidon:8084/remote 1 - Time 1297 ms 5 - Time 2000 ms 10 - Time 3515 ms 20 - Time 6016 ms 40 - Time 11031 ms 50 - Time 6016 ms 200 - Time 94375 ms The numbers don't look very well, though. I promise to look at this when you file a bug report so that it doesn't get overlooked. Robert ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Remoting and thread pool limits
Well, it is not a *bug* but a feature. I wonder if it should be changed. If you look into RemotingThreadPool.cs there is a line like: threadDone.WaitOne(PoolGrowDelay, false); This one is actually the one making the process too slow. I'm afraid it must be something similar on the .NET code too! Of course removing this line the problem gets solved, but I guess there is a reason in the channel to do that. pablo - Original Message - From: "Robert Jordan" <[EMAIL PROTECTED]> To: Sent: Saturday, December 15, 2007 4:12 PM Subject: Re: [Mono-dev] Remoting and thread pool limits > Hi, > > pablosantosluac wrote: >> Hi, >> >> I've found the following difference working with .NET and Mono remoting. >> In >> .NET when the remoting ThreadPool reaches the pool's maximum (25 threads >> per >> process), it is able to continue creating new threads. In fact, you can >> end >> up having a remoting server with hundreds of threads. >> >> In mono the behaviour is different. Once the limit is reached it starts >> refusing connections. > > This is unrelated to mono's remoting thread pool. You have probably > ran your tests on Windows, where mono indeed fails with a GC failure > when too many threads are created because the GC has a hard coded > max thread count limit. > > On Linux (x86_64, Mono 1.2.6) I can finish the tests: > > Linux Mono client -> Linux Mono server (same machine) > > poseidon [~/foo] $ mono client/bin/Debug/client.exe > tcp://localhost:8084/remote > 1 - Time 1206 ms > 5 - Time 2045 ms > 10 - Time 3575 ms > 20 - Time 8174 ms > 40 - Time 12055 ms > 50 - Time 8185 ms > 200 - Time 46150 ms > > > MS.NET client -> Linux Mono server (different machines) > > troll:/cygdrive/u/foo/client/bin/Debug $ ./client.exe > tcp://poseidon:8084/remote > 1 - Time 1297 ms > 5 - Time 2000 ms > 10 - Time 3515 ms > 20 - Time 6016 ms > 40 - Time 11031 ms > 50 - Time 6016 ms > 200 - Time 94375 ms > > > The numbers don't look very well, though. I promise to look at > this when you file a bug report so that it doesn't get > overlooked. > > Robert > > ___ > Mono-devel-list mailing list > Mono-devel-list@lists.ximian.com > http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Remoting and thread pool limits
pablosantosluac wrote: > Well, it is not a *bug* but a feature. I wonder if it should be changed. If > you look into RemotingThreadPool.cs there is a line like: > > > threadDone.WaitOne(PoolGrowDelay, false); > > > This one is actually the one making the process too slow. I'm afraid it must > be something similar on the .NET code too! > > Of course removing this line the problem gets solved, but I guess there is a > reason in the channel to do that. I noticed that too, but this doesn't solve the real problem: the 200 thread test is still too slow on my machine. The degradation seems to start after 60-80 threads on my pretty weak SMP machine. Just to be sure, I've replaced RemotingThreadPool with an own, simple version based on the standard BCL ThreadPool => same issue, although I've raised the env var MONO_THREADS_PER_CPU to an insane value. Robert > > > pablo > > > - Original Message - > From: "Robert Jordan" <[EMAIL PROTECTED]> > To: > Sent: Saturday, December 15, 2007 4:12 PM > Subject: Re: [Mono-dev] Remoting and thread pool limits > > >> Hi, >> >> pablosantosluac wrote: >>> Hi, >>> >>> I've found the following difference working with .NET and Mono remoting. >>> In >>> .NET when the remoting ThreadPool reaches the pool's maximum (25 threads >>> per >>> process), it is able to continue creating new threads. In fact, you can >>> end >>> up having a remoting server with hundreds of threads. >>> >>> In mono the behaviour is different. Once the limit is reached it starts >>> refusing connections. >> This is unrelated to mono's remoting thread pool. You have probably >> ran your tests on Windows, where mono indeed fails with a GC failure >> when too many threads are created because the GC has a hard coded >> max thread count limit. >> >> On Linux (x86_64, Mono 1.2.6) I can finish the tests: >> >> Linux Mono client -> Linux Mono server (same machine) >> >> poseidon [~/foo] $ mono client/bin/Debug/client.exe >> tcp://localhost:8084/remote >> 1 - Time 1206 ms >> 5 - Time 2045 ms >> 10 - Time 3575 ms >> 20 - Time 8174 ms >> 40 - Time 12055 ms >> 50 - Time 8185 ms >> 200 - Time 46150 ms >> >> >> MS.NET client -> Linux Mono server (different machines) >> >> troll:/cygdrive/u/foo/client/bin/Debug $ ./client.exe >> tcp://poseidon:8084/remote >> 1 - Time 1297 ms >> 5 - Time 2000 ms >> 10 - Time 3515 ms >> 20 - Time 6016 ms >> 40 - Time 11031 ms >> 50 - Time 6016 ms >> 200 - Time 94375 ms >> >> >> The numbers don't look very well, though. I promise to look at >> this when you file a bug report so that it doesn't get >> overlooked. >> >> Robert >> >> ___ >> Mono-devel-list mailing list >> Mono-devel-list@lists.ximian.com >> http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Remoting and thread pool limits
Hi Robert, Here are my numbers on a Linux (Xeon) box: [EMAIL PROTECTED] client]$ /opt/mono/bin/mono client.exe tcp://localhost:10020/remote 1 - Time 1296 ms 5 - Time 1023 ms 10 - Time 1025 ms 20 - Time 1031 ms 40 - Time 1041 ms 50 - Time 1041 ms 200 - Time 10126 ms As you mentioned I can't currently understand the "hit" when creating 200 threads because if you try to create them in a standalone application (a loop launching 1000 threads), it works perfectly... but doing remoting (I don't know where the problem can be), the impact is quite high... ¿Something socket related? (I doubt it!) I'll take another look into the channel's code. Regards, pablo - Original Message - From: "Robert Jordan" <[EMAIL PROTECTED]> To: Sent: Saturday, December 15, 2007 9:25 PM Subject: Re: [Mono-dev] Remoting and thread pool limits > pablosantosluac wrote: >> Well, it is not a *bug* but a feature. I wonder if it should be changed. >> If >> you look into RemotingThreadPool.cs there is a line like: >> >> >> threadDone.WaitOne(PoolGrowDelay, false); >> >> >> This one is actually the one making the process too slow. I'm afraid it >> must >> be something similar on the .NET code too! >> >> Of course removing this line the problem gets solved, but I guess there >> is a >> reason in the channel to do that. > > I noticed that too, but this doesn't solve the real problem: the 200 > thread test is still too slow on my machine. The degradation seems to > start after 60-80 threads on my pretty weak SMP machine. > > Just to be sure, I've replaced RemotingThreadPool with an own, simple > version based on the standard BCL ThreadPool => same issue, although > I've raised the env var MONO_THREADS_PER_CPU to an insane value. > > Robert > >> >> >> pablo >> >> >> - Original Message - >> From: "Robert Jordan" <[EMAIL PROTECTED]> >> To: >> Sent: Saturday, December 15, 2007 4:12 PM >> Subject: Re: [Mono-dev] Remoting and thread pool limits >> >> >>> Hi, >>> >>> pablosantosluac wrote: >>>> Hi, >>>> >>>> I've found the following difference working with .NET and Mono >>>> remoting. >>>> In >>>> .NET when the remoting ThreadPool reaches the pool's maximum (25 >>>> threads >>>> per >>>> process), it is able to continue creating new threads. In fact, you can >>>> end >>>> up having a remoting server with hundreds of threads. >>>> >>>> In mono the behaviour is different. Once the limit is reached it starts >>>> refusing connections. >>> This is unrelated to mono's remoting thread pool. You have probably >>> ran your tests on Windows, where mono indeed fails with a GC failure >>> when too many threads are created because the GC has a hard coded >>> max thread count limit. >>> >>> On Linux (x86_64, Mono 1.2.6) I can finish the tests: >>> >>> Linux Mono client -> Linux Mono server (same machine) >>> >>> poseidon [~/foo] $ mono client/bin/Debug/client.exe >>> tcp://localhost:8084/remote >>> 1 - Time 1206 ms >>> 5 - Time 2045 ms >>> 10 - Time 3575 ms >>> 20 - Time 8174 ms >>> 40 - Time 12055 ms >>> 50 - Time 8185 ms >>> 200 - Time 46150 ms >>> >>> >>> MS.NET client -> Linux Mono server (different machines) >>> >>> troll:/cygdrive/u/foo/client/bin/Debug $ ./client.exe >>> tcp://poseidon:8084/remote >>> 1 - Time 1297 ms >>> 5 - Time 2000 ms >>> 10 - Time 3515 ms >>> 20 - Time 6016 ms >>> 40 - Time 11031 ms >>> 50 - Time 6016 ms >>> 200 - Time 94375 ms >>> >>> >>> The numbers don't look very well, though. I promise to look at >>> this when you file a bug report so that it doesn't get >>> overlooked. >>> >>> Robert >>> >>> ___ >>> Mono-devel-list mailing list >>> Mono-devel-list@lists.ximian.com >>> http://lists.ximian.com/mailman/listinfo/mono-devel-list > > ___ > Mono-devel-list mailing list > Mono-devel-list@lists.ximian.com > http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Remoting and thread pool limits
Hi Robert, Do you think it is related to the thread creation? I'll run some tests with the mono profiler on Linux and try to figure out where the time is being lost... pablo - Original Message - From: "Robert Jordan" <[EMAIL PROTECTED]> To: Sent: Saturday, December 15, 2007 9:25 PM Subject: Re: [Mono-dev] Remoting and thread pool limits > pablosantosluac wrote: >> Well, it is not a *bug* but a feature. I wonder if it should be changed. >> If >> you look into RemotingThreadPool.cs there is a line like: >> >> >> threadDone.WaitOne(PoolGrowDelay, false); >> >> >> This one is actually the one making the process too slow. I'm afraid it >> must >> be something similar on the .NET code too! >> >> Of course removing this line the problem gets solved, but I guess there >> is a >> reason in the channel to do that. > > I noticed that too, but this doesn't solve the real problem: the 200 > thread test is still too slow on my machine. The degradation seems to > start after 60-80 threads on my pretty weak SMP machine. > > Just to be sure, I've replaced RemotingThreadPool with an own, simple > version based on the standard BCL ThreadPool => same issue, although > I've raised the env var MONO_THREADS_PER_CPU to an insane value. > > Robert > >> >> >> pablo >> >> >> - Original Message - >> From: "Robert Jordan" <[EMAIL PROTECTED]> >> To: >> Sent: Saturday, December 15, 2007 4:12 PM >> Subject: Re: [Mono-dev] Remoting and thread pool limits >> >> >>> Hi, >>> >>> pablosantosluac wrote: >>>> Hi, >>>> >>>> I've found the following difference working with .NET and Mono >>>> remoting. >>>> In >>>> .NET when the remoting ThreadPool reaches the pool's maximum (25 >>>> threads >>>> per >>>> process), it is able to continue creating new threads. In fact, you can >>>> end >>>> up having a remoting server with hundreds of threads. >>>> >>>> In mono the behaviour is different. Once the limit is reached it starts >>>> refusing connections. >>> This is unrelated to mono's remoting thread pool. You have probably >>> ran your tests on Windows, where mono indeed fails with a GC failure >>> when too many threads are created because the GC has a hard coded >>> max thread count limit. >>> >>> On Linux (x86_64, Mono 1.2.6) I can finish the tests: >>> >>> Linux Mono client -> Linux Mono server (same machine) >>> >>> poseidon [~/foo] $ mono client/bin/Debug/client.exe >>> tcp://localhost:8084/remote >>> 1 - Time 1206 ms >>> 5 - Time 2045 ms >>> 10 - Time 3575 ms >>> 20 - Time 8174 ms >>> 40 - Time 12055 ms >>> 50 - Time 8185 ms >>> 200 - Time 46150 ms >>> >>> >>> MS.NET client -> Linux Mono server (different machines) >>> >>> troll:/cygdrive/u/foo/client/bin/Debug $ ./client.exe >>> tcp://poseidon:8084/remote >>> 1 - Time 1297 ms >>> 5 - Time 2000 ms >>> 10 - Time 3515 ms >>> 20 - Time 6016 ms >>> 40 - Time 11031 ms >>> 50 - Time 6016 ms >>> 200 - Time 94375 ms >>> >>> >>> The numbers don't look very well, though. I promise to look at >>> this when you file a bug report so that it doesn't get >>> overlooked. >>> >>> Robert >>> >>> ___ >>> Mono-devel-list mailing list >>> Mono-devel-list@lists.ximian.com >>> http://lists.ximian.com/mailman/listinfo/mono-devel-list > > ___ > Mono-devel-list mailing list > Mono-devel-list@lists.ximian.com > http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Remoting and thread pool limits
Hi Pablo, pablosantosluac wrote: > Hi Robert, > > Do you think it is related to the thread creation? I don't think so. Most of the time is spent in TcpChannel: System.Runtime.Remoting.Channels.Tcp.ClientConnection::ProcessMessages() System.Runtime.Remoting.Channels.Tcp.TcpMessageIO::ReceiveMessageStatus(Stream,byte[]) System.Runtime.Remoting.Channels.Tcp.TcpMessageIO::StreamRead(Stream,byte[],int) When I stop the test at 40 threads, these methods are not at the top anymore. Robert > > I'll run some tests with the mono profiler on Linux and try to figure out > where the time is being lost... > > > pablo > > > > - Original Message - > From: "Robert Jordan" <[EMAIL PROTECTED]> > To: > Sent: Saturday, December 15, 2007 9:25 PM > Subject: Re: [Mono-dev] Remoting and thread pool limits > > >> pablosantosluac wrote: >>> Well, it is not a *bug* but a feature. I wonder if it should be changed. >>> If >>> you look into RemotingThreadPool.cs there is a line like: >>> >>> >>> threadDone.WaitOne(PoolGrowDelay, false); >>> >>> >>> This one is actually the one making the process too slow. I'm afraid it >>> must >>> be something similar on the .NET code too! >>> >>> Of course removing this line the problem gets solved, but I guess there >>> is a >>> reason in the channel to do that. >> I noticed that too, but this doesn't solve the real problem: the 200 >> thread test is still too slow on my machine. The degradation seems to >> start after 60-80 threads on my pretty weak SMP machine. >> >> Just to be sure, I've replaced RemotingThreadPool with an own, simple >> version based on the standard BCL ThreadPool => same issue, although >> I've raised the env var MONO_THREADS_PER_CPU to an insane value. >> >> Robert >> >>> >>> pablo >>> >>> >>> - Original Message - >>> From: "Robert Jordan" <[EMAIL PROTECTED]> >>> To: >>> Sent: Saturday, December 15, 2007 4:12 PM >>> Subject: Re: [Mono-dev] Remoting and thread pool limits >>> >>> >>>> Hi, >>>> >>>> pablosantosluac wrote: >>>>> Hi, >>>>> >>>>> I've found the following difference working with .NET and Mono >>>>> remoting. >>>>> In >>>>> .NET when the remoting ThreadPool reaches the pool's maximum (25 >>>>> threads >>>>> per >>>>> process), it is able to continue creating new threads. In fact, you can >>>>> end >>>>> up having a remoting server with hundreds of threads. >>>>> >>>>> In mono the behaviour is different. Once the limit is reached it starts >>>>> refusing connections. >>>> This is unrelated to mono's remoting thread pool. You have probably >>>> ran your tests on Windows, where mono indeed fails with a GC failure >>>> when too many threads are created because the GC has a hard coded >>>> max thread count limit. >>>> >>>> On Linux (x86_64, Mono 1.2.6) I can finish the tests: >>>> >>>> Linux Mono client -> Linux Mono server (same machine) >>>> >>>> poseidon [~/foo] $ mono client/bin/Debug/client.exe >>>> tcp://localhost:8084/remote >>>> 1 - Time 1206 ms >>>> 5 - Time 2045 ms >>>> 10 - Time 3575 ms >>>> 20 - Time 8174 ms >>>> 40 - Time 12055 ms >>>> 50 - Time 8185 ms >>>> 200 - Time 46150 ms >>>> >>>> >>>> MS.NET client -> Linux Mono server (different machines) >>>> >>>> troll:/cygdrive/u/foo/client/bin/Debug $ ./client.exe >>>> tcp://poseidon:8084/remote >>>> 1 - Time 1297 ms >>>> 5 - Time 2000 ms >>>> 10 - Time 3515 ms >>>> 20 - Time 6016 ms >>>> 40 - Time 11031 ms >>>> 50 - Time 6016 ms >>>> 200 - Time 94375 ms >>>> >>>> >>>> The numbers don't look very well, though. I promise to look at >>>> this when you file a bug report so that it doesn't get >>>> overlooked. >>>> >>>> Robert >>>> >>>> ___ >>>> Mono-devel-list mailing list >>>> Mono-devel-list@lists.ximian.com >>>> http://lists.ximian.com/mailman/listinfo/mono-devel-list >> ___ >> Mono-devel-list mailing list >> Mono-devel-list@lists.ximian.com >> http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Remoting and thread pool limits
Ok, so this are the ones to inspect then: ProcessMessages, ReceiveMessageStatus and StreamRead. Well, looks ok that they are the ones eating CPU (the system is processing requests), but... why they go worse when dealing with more than 80 threads? Looking at the code I can't see anything at these methods... :-( I'll try to look into it again in more detail... Thanks, pablo - Original Message - From: "Robert Jordan" <[EMAIL PROTECTED]> To: Sent: Monday, December 17, 2007 11:45 PM Subject: Re: [Mono-dev] Remoting and thread pool limits > Hi Pablo, > > pablosantosluac wrote: >> Hi Robert, >> >> Do you think it is related to the thread creation? > > I don't think so. Most of the time is spent in TcpChannel: > > System.Runtime.Remoting.Channels.Tcp.ClientConnection::ProcessMessages() > System.Runtime.Remoting.Channels.Tcp.TcpMessageIO::ReceiveMessageStatus(Stream,byte[]) > System.Runtime.Remoting.Channels.Tcp.TcpMessageIO::StreamRead(Stream,byte[],int) > > When I stop the test at 40 threads, these methods are not at the top > anymore. > > Robert > >> >> I'll run some tests with the mono profiler on Linux and try to figure out >> where the time is being lost... >> >> >> pablo >> >> >> >> - Original Message - >> From: "Robert Jordan" <[EMAIL PROTECTED]> >> To: >> Sent: Saturday, December 15, 2007 9:25 PM >> Subject: Re: [Mono-dev] Remoting and thread pool limits >> >> >>> pablosantosluac wrote: >>>> Well, it is not a *bug* but a feature. I wonder if it should be >>>> changed. >>>> If >>>> you look into RemotingThreadPool.cs there is a line like: >>>> >>>> >>>> threadDone.WaitOne(PoolGrowDelay, false); >>>> >>>> >>>> This one is actually the one making the process too slow. I'm afraid it >>>> must >>>> be something similar on the .NET code too! >>>> >>>> Of course removing this line the problem gets solved, but I guess there >>>> is a >>>> reason in the channel to do that. >>> I noticed that too, but this doesn't solve the real problem: the 200 >>> thread test is still too slow on my machine. The degradation seems to >>> start after 60-80 threads on my pretty weak SMP machine. >>> >>> Just to be sure, I've replaced RemotingThreadPool with an own, simple >>> version based on the standard BCL ThreadPool => same issue, although >>> I've raised the env var MONO_THREADS_PER_CPU to an insane value. >>> >>> Robert >>> >>>> >>>> pablo >>>> >>>> >>>> - Original Message - >>>> From: "Robert Jordan" <[EMAIL PROTECTED]> >>>> To: >>>> Sent: Saturday, December 15, 2007 4:12 PM >>>> Subject: Re: [Mono-dev] Remoting and thread pool limits >>>> >>>> >>>>> Hi, >>>>> >>>>> pablosantosluac wrote: >>>>>> Hi, >>>>>> >>>>>> I've found the following difference working with .NET and Mono >>>>>> remoting. >>>>>> In >>>>>> .NET when the remoting ThreadPool reaches the pool's maximum (25 >>>>>> threads >>>>>> per >>>>>> process), it is able to continue creating new threads. In fact, you >>>>>> can >>>>>> end >>>>>> up having a remoting server with hundreds of threads. >>>>>> >>>>>> In mono the behaviour is different. Once the limit is reached it >>>>>> starts >>>>>> refusing connections. >>>>> This is unrelated to mono's remoting thread pool. You have probably >>>>> ran your tests on Windows, where mono indeed fails with a GC failure >>>>> when too many threads are created because the GC has a hard coded >>>>> max thread count limit. >>>>> >>>>> On Linux (x86_64, Mono 1.2.6) I can finish the tests: >>>>> >>>>> Linux Mono client -> Linux Mono server (same machine) >>>>> >>>>> poseidon [~/foo] $ mono client/bin/Debug/client.exe >>>>> tcp://localhost:8084/remote >>>>> 1 - Time 1206 ms >>>>> 5 - Time 2045 ms >>>>> 10 - Time 3575 ms >>>>> 20 - Time 8174 ms >>>>>
Re: [Mono-dev] Remoting and thread pool limits
Hi Robert, I'm trying this together with a SSL remoting channel. I'm observing huge variations: I don't see anymore the "80" connections problems: I can run the test launching 200 threads and doing 200 connections in less than 2 seconds. (mono/Linux) When I use the SSL channel on Mono/Linux I can get about 7 seconds to complete the test. But the "bad" thing is that from time to time the same test takes about 40 secs for tcp and about 180 for ssl!!! I still didn't find out why there's such a difference!! Do you think it could be related to the tcp stack? Maybe it is needed to create sockets with different params or something like that? Thanks, pablo - Original Message - From: "pablosantosluac" <[EMAIL PROTECTED]> To: ; "Robert Jordan" <[EMAIL PROTECTED]> Sent: Monday, December 17, 2007 11:59 PM Subject: Re: [Mono-dev] Remoting and thread pool limits > Ok, so this are the ones to inspect then: ProcessMessages, > ReceiveMessageStatus and StreamRead. Well, looks ok that they are the ones > eating CPU (the system is processing requests), but... why they go worse > when dealing with more than 80 threads? > > Looking at the code I can't see anything at these methods... :-( I'll try > to > look into it again in more detail... > > Thanks, > > pablo > > - Original Message - > From: "Robert Jordan" <[EMAIL PROTECTED]> > To: > Sent: Monday, December 17, 2007 11:45 PM > Subject: Re: [Mono-dev] Remoting and thread pool limits > > >> Hi Pablo, >> >> pablosantosluac wrote: >>> Hi Robert, >>> >>> Do you think it is related to the thread creation? >> >> I don't think so. Most of the time is spent in TcpChannel: >> >> System.Runtime.Remoting.Channels.Tcp.ClientConnection::ProcessMessages() >> System.Runtime.Remoting.Channels.Tcp.TcpMessageIO::ReceiveMessageStatus(Stream,byte[]) >> System.Runtime.Remoting.Channels.Tcp.TcpMessageIO::StreamRead(Stream,byte[],int) >> >> When I stop the test at 40 threads, these methods are not at the top >> anymore. >> >> Robert >> >>> >>> I'll run some tests with the mono profiler on Linux and try to figure >>> out >>> where the time is being lost... >>> >>> >>> pablo >>> >>> >>> >>> - Original Message - >>> From: "Robert Jordan" <[EMAIL PROTECTED]> >>> To: >>> Sent: Saturday, December 15, 2007 9:25 PM >>> Subject: Re: [Mono-dev] Remoting and thread pool limits >>> >>> >>>> pablosantosluac wrote: >>>>> Well, it is not a *bug* but a feature. I wonder if it should be >>>>> changed. >>>>> If >>>>> you look into RemotingThreadPool.cs there is a line like: >>>>> >>>>> >>>>> threadDone.WaitOne(PoolGrowDelay, false); >>>>> >>>>> >>>>> This one is actually the one making the process too slow. I'm afraid >>>>> it >>>>> must >>>>> be something similar on the .NET code too! >>>>> >>>>> Of course removing this line the problem gets solved, but I guess >>>>> there >>>>> is a >>>>> reason in the channel to do that. >>>> I noticed that too, but this doesn't solve the real problem: the 200 >>>> thread test is still too slow on my machine. The degradation seems to >>>> start after 60-80 threads on my pretty weak SMP machine. >>>> >>>> Just to be sure, I've replaced RemotingThreadPool with an own, simple >>>> version based on the standard BCL ThreadPool => same issue, although >>>> I've raised the env var MONO_THREADS_PER_CPU to an insane value. >>>> >>>> Robert >>>> >>>>> >>>>> pablo >>>>> >>>>> >>>>> - Original Message - >>>>> From: "Robert Jordan" <[EMAIL PROTECTED]> >>>>> To: >>>>> Sent: Saturday, December 15, 2007 4:12 PM >>>>> Subject: Re: [Mono-dev] Remoting and thread pool limits >>>>> >>>>> >>>>>> Hi, >>>>>> >>>>>> pablosantosluac wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I've found the following difference working with .NET and Mono >>>>>>> remoting. >
