Re: [twsocket] ICS Message handlers
Dear Arno, Did you test that in exactly the same environment, same box, both service or GUI applications, both non or RDP sessions etc.. ? I tested on the same box, GUI and service. Non RDP, all local desktop I will provide much more data, I´m preparing a document for you and a sample program, I´m just curious about this findins, your comments are really appreciated Can I send attachment to this group? If not, how do I send? Eric -- From: "Arno Garrels" Sent: Thursday, August 12, 2010 2:56 PM To: "ICS support mailing" Subject: Re: [twsocket] ICS Message handlers Eric, I found a problem in our software that is limiting the total ammount of TCP sockets that I can create. Did you test that in exactly the same environment, same box, both service or GUI applications, both non or RDP sessions etc.. ? Would you please provide a small test application that demonstrates this? -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be Nenhum virus encontrado nessa mensagem recebida. Verificado por AVG - www.avgbrasil.com.br Versao: 9.0.851 / Banco de dados de virus: 271.1.1/3066 - Data de Lancamento: 08/12/10 03:34:00 -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
>Can I send attachment to this group? If not, how do I send? >Eric Please don't send attachments even if it is possible. Upload to hotfile.com or some similar service (there is a bunch of them!) like that and link in your message... all of such services offer you a "delete" link which you can use later to delete your files if you don't want them present at later time. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
Eric, > I will provide much more data, I´m preparing a document for you and a > sample program, I´m just curious about this findins, your comments > are really appreciated It is just important that you compare apples with apples. > Can I send attachment to this group? If not, how do I send? No, the list server will eat it, either upload it to some website and post the link here or send me a private e-mail, thanks. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
François Windows limit the number of windows handles to 10.000. That's why ICS V7 is designed to use only one handler for a lot of TWSocket so that you can create a lot of sockets. Which OS are you using which allows creating 1.000.000 handles ? After more testing, and a clear mind of a new day, I found that I was doing some things wrong.. The limit is really 10.000 handles, and if I create 10.000 handles I can´t create any more TWSocket or anything, so you are right, sorry about that Classes.AllocateHwnd and TIcsWndHandler.AllocateHWnd both use the same API (CreateWindowEx ) to create the handle so both should be able to create the same number of those. You are right here too, it doesn´t matter using your allocatehandle or classes.allocateHwnd, the results are the same What I do found is that windows imposes a lesser limit (Less than 2.000 handles) for service applications that runs under LocalSystem account and do not have desktop interaction, so, by the end of the day, the problem was not related to ICS itself, but to windows service management. If you make a simple service application that tries to create 10.000 handles, it won´t create even 2.000, but if you activate the Deskop Interaction flag in the service, than you can create almost 10.000 Sorry about the confusion 10.000 is a huge number and we will never use that amount of resources, it was just for testing, just to make it clear Thanks Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
On 08/12/2010 12:56, Eric Fleming Bonilha wrote: Since my app can have many handles and I need many sockets, I did many tests and found that procedure TIcsWndHandler.AllocateHWnd; create its own windows handle, instead of using classes.allocatehwnd How does this scheme works? As I understood it, Classes.AllocateHWnd() is not thread-safe (or at least it wasn't in older Delphi versions). I think this was the primary reason why ICS had to avoid it and create its own thread-safe version. I encountered the same problem when creating timers and other fancy things in a multi-threaded service some time ago. I'm still stuck with Delphi 7 at the office, so I don't know if this has changed during the years. However, looking at the source code of TIcsWndControl.AllocateHWnd(), it appears that it's still locking the resource in order to create new handles. My guess is, no. dZ. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
Hi Eric, I hope you received my reply to you and got everything ok between us. I have a question for you: On Fri, Aug 13, 2010 at 4:13 PM, Eric Fleming Bonilha wrote: > What I do found is that windows imposes a lesser limit (Less than 2.000 > handles) for service applications that runs under LocalSystem account and do > not have desktop interaction, so, by the end of the day, the problem was not > related to ICS itself, but to windows service management. > > If you make a simple service application that tries to create 10.000 > handles, it won´t create even 2.000, but if you activate the Deskop > Interaction flag in the service, than you can create almost 10.000 > > I have tried to find this finding before here but nobody seemed to know it. So it is very interesting. Did you find this by testing or from a Microsoft KB? If so, which one? Best Regards, SubZero -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
Hello Eric, so you are right, sorry about that You are right here too Sorry about the confusion No problem. Happy to know that I'm still OK. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be - Original Message - From: "Eric Fleming Bonilha" To: "ICS support mailing" Sent: Friday, August 13, 2010 3:13 PM Subject: Re: [twsocket] ICS Message handlers François Windows limit the number of windows handles to 10.000. That's why ICS V7 is designed to use only one handler for a lot of TWSocket so that you can create a lot of sockets. Which OS are you using which allows creating 1.000.000 handles ? After more testing, and a clear mind of a new day, I found that I was doing some things wrong.. The limit is really 10.000 handles, and if I create 10.000 handles I can´t create any more TWSocket or anything, so you are right, sorry about that Classes.AllocateHwnd and TIcsWndHandler.AllocateHWnd both use the same API (CreateWindowEx ) to create the handle so both should be able to create the same number of those. You are right here too, it doesn´t matter using your allocatehandle or classes.allocateHwnd, the results are the same What I do found is that windows imposes a lesser limit (Less than 2.000 handles) for service applications that runs under LocalSystem account and do not have desktop interaction, so, by the end of the day, the problem was not related to ICS itself, but to windows service management. If you make a simple service application that tries to create 10.000 handles, it won´t create even 2.000, but if you activate the Deskop Interaction flag in the service, than you can create almost 10.000 Sorry about the confusion 10.000 is a huge number and we will never use that amount of resources, it was just for testing, just to make it clear Thanks Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
I'm still stuck with Delphi 7 at the office, so I don't know if this has changed during the years. However, looking at the source code of TIcsWndControl.AllocateHWnd(), it appears that it's still locking the resource in order to create new handles. My guess is, no. I think that they lock in order to threads to be able to create the handlers, because as I understood they share the handles between classes, which is very very clever, I liked the design, to reuse handles, that is why I can create 130.000 instances of TWSocket! They share resources when they can, because each instance will have a different message number for the same messages, that is a very smart idea and I will try to use the same thinking in my software. I have already created a class called TSharedTimer, which allows me to create virtually unlimited number of timers, since they will all share the same resource, and it is working pretty fine. I will try to make something similar to ICS to reduce the amount of handles that I will consume in my software, by sharing the handles and defining different ranges of messages per handle Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
Hi SubZero,
I hope you received my reply to you and got everything ok between us. I
have
a question for you:
I got it, don´t worry, I wasn´t mad, I was just thinking that I was asking
something obvious hehe, I thought that it was a silly question, since I
don´t have much knowlege on how TWSocket works internally ;)
I have tried to find this finding before here but nobody seemed to know
it.
So it is very interesting. Did you find this by testing or from a
Microsoft
KB? If so, which one?
Humm, it was just testing, and it is pretty easy to test, and it just
happens in service application, so, I don´t really know why does it happens,
here is a simple code for a service application:
InitializeDebugLog;
for i := 0 to 1 do
begin
if AllocateHWnd(WndProc) = 0 then
DebugMessage('Error creating handle - ' + IntToStr(i));
end;
DebugLog is a routine that I have just to write debug messages into a text
file
The result of this application when running in a service as LocalSystem
account without desktop interaction is:
13/08/2010 - 10:48:41: Error creating handle - 2265
13/08/2010 - 10:48:41: Error creating handle - 2266
13/08/2010 - 10:48:41: Error creating handle - 2267
...
13/08/2010 - 10:48:41: Error creating handle - 1
The result of this application when running in a service as LocalSystem
account WITH desktop interaction is:
13/08/2010 - 10:49:44: Error creating handle - 9982
13/08/2010 - 10:49:44: Error creating handle - 9983
13/08/2010 - 10:49:44: Error creating handle - 9984
...
13/08/2010 - 10:49:44: Error creating handle - 1
So, when running without desktop interaction I can create at most 2264
handles, and with desktop interaction I can create 9981 handles!
That is why I can create 139.733 instances of TWSocket in a service app with
desktop interaction and 30.239 instances of TWSocket in a service app
without desktop interaction
Since TWSocket share the same handle among different instances, it can
create MANY sockets, that was a nice idea
Now I just want to know, WHY does windows limit the handles in a service app
without desktop interaction (Maybe to protect other services from running
out of resources?)
Eric
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
I will try to make something similar to ICS to reduce the amount of handles that I will consume in my software, by sharing the handles and defining different ranges of messages per handle Reusing the design I've done is very easy. Just derive your class from TIcsWndControl and override the methods AllocateMsgHandlers, FreeMsgHandlers and MsgHandlersCount to get your own messages. See how it is done in many ICS components. very easy. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
> I have already created a class called TSharedTimer, which allows me > to create virtually unlimited number of timers, since they will all > share the same resource, and it is working pretty fine. Arno already added that functionality to ICS, unit OverbyteIcsThreadTimer and TIcsThreadTimer, it supports 10,000 timers. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
Eric, > What I do found is that windows imposes a lesser limit (Less than > 2.000 handles) for service applications that runs under LocalSystem > account and do not have desktop interaction, I did know that services limit the maximum number, but did not know that interactive services allow more. Just tested on Win7 64-bit 8 GB RAM: interactive = 9982 vs. 2277 non-interactive service. Good to know! -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
François Reusing the design I've done is very easy. Just derive your class from TIcsWndControl and override the methods AllocateMsgHandlers, FreeMsgHandlers and MsgHandlersCount to get your own messages. See how it is done in many ICS components. very easy. It is really easy to use and it is already done and tested! So I will definitely use it ;) I really liked this implementation, congratulations on that Thanks Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
Angus Robertson - Magenta Systems Ltd wrote: >> I have already created a class called TSharedTimer, which allows me >> to create virtually unlimited number of timers, since they will all >> share the same resource, and it is working pretty fine. > > Arno already added that functionality to ICS, unit > OverbyteIcsThreadTimer and TIcsThreadTimer, it supports 10,000 timers. TIcsThreadTimer is a good choice for 10,000+ of lazy timers, when accuracy isn't very important. TIcsTimer is a 'normal' timer that requires a TIcsWndControl instance to work. There are demos of both in the MiscDemos directory. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
TIcsThreadTimer is a good choice for 10,000+ of lazy timers, when accuracy isn't very important. TIcsTimer is a 'normal' timer that requires a TIcsWndControl instance to work. There are demos of both in the MiscDemos directory. What I did was something similar, but not using messages I actually have instances of TSharedTimer, and I have a manager with a list of all timers created, this manager has a TTimer with 10ms loop, and it calls the OnTimer event of the timers when they are ready to be triggered The limitation is that the OnTimer events of SharedTimer will be called from within the main thread only. It is working really good and I can create 10.000+ timers even with high resolution (not THAT high, but as low as 30ms), without using system resources as handles, only memory Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] OT: ICS-based IQ Proxy Server supports 60K concurrent connections on single port now!
Hello, What's new in IQ Proxy Server 3.4.2R - On single port, IQP now supports 60,000 simultaneous connections http://www.iqproxyserver.com Just FYI. With thanks to Eric. Best Regards, SubZero -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] WM_USER vs. WM_APP messages
>From what I could see, ICS uses a lot of WM_USER+n messages range but the use of these custom messages is not recommended. Instead you should use WM_APP+n. http://msdn.microsoft.com/en-us/library/ms644930%28VS.85%29.aspx The reason to use WM_APP is because WM_USER can conflict with system messages - this is especially true on older Windows so WM_APP range should be safer. Any thoughts on that? -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] OT: ICS-based IQ Proxy Server supports 60K concurrent connections on single port now!
SubZero What's new in IQ Proxy Server 3.4.2R - On single port, IQP now supports 60,000 simultaneous connections That is really nice, did you get that by just activating the desktop interaction flag on? Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] WM_USER vs. WM_APP messages
From what I could see, ICS uses a lot of WM_USER+n messages range but the use of these custom messages is not recommended. Instead you should use WM_APP+n. http://msdn.microsoft.com/en-us/library/ms644930%28VS.85%29.aspx The reason to use WM_APP is because WM_USER can conflict with system messages - this is especially true on older Windows so WM_APP range should be safer. Any thoughts on that? There is no problem since ICS component use their own window which only receive messages from the components. BTW: In ICS V7, you can select the starting message number with GWndHandlerMsgLow global variable. GWndHandlerMsgLow can be initialized before any ICS component is created Typically it is done from the project source file. If you don't initialize it, it will be initialized to WM_USER + 1 the first time it is needed. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] OT: ICS-based IQ Proxy Server supports 60Kconcurrent connections on single port now!
Eric Fleming Bonilha wrote: > SubZero > >> What's new in IQ Proxy Server 3.4.2R >> - On single port, IQP now supports 60,000 simultaneous connections >> > > That is really nice, did you get that by just activating the desktop > interaction flag on? I bet that. I only wonder how long this option will be available since interactive services have been deprecated since Vista. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
Eric Fleming Bonilha wrote: >> TIcsThreadTimer is a good choice for 10,000+ of lazy timers, >> when accuracy isn't very important. >> >> TIcsTimer is a 'normal' timer that requires a TIcsWndControl instance >> to work. There are demos of both in the MiscDemos directory. > > What I did was something similar, but not using messages > > I actually have instances of TSharedTimer, and I have a manager with > a list of all timers created, this manager has a TTimer with 10ms > loop, and it calls the OnTimer event of the timers when they are > ready to be triggered > > The limitation is that the OnTimer events of SharedTimer will be > called from within the main thread only. > > It is working really good and I can create 10.000+ timers even with > high resolution (not THAT high, but as low as 30ms), without using > system resources as handles, only memory One limitation with such kind of implementation (might not be relevant) is that the application blocks until entire list of events is processed. Like iterating over a list of n server clients blocks all clients. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] OT: ICS-based IQ Proxy Server supports 60K concurrent connections on single port now!
Yes. Plus I noticed IIS runs as a NETWORK_SERVICE account for this purpose. On Fri, Aug 13, 2010 at 6:19 PM, Eric Fleming Bonilha wrote: > SubZero > > > What's new in IQ Proxy Server 3.4.2R >> - On single port, IQP now supports 60,000 simultaneous connections >> >> > That is really nice, did you get that by just activating the desktop > interaction flag on? > > Eric > > > > -- > To unsubscribe or change your settings for TWSocket mailing list > please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket > Visit our website at http://www.overbyte.be > -- Fastream Technologies Software IQ: Innovation & Quality www.fastream.com | www.iqproxyserver.com Email: sa...@fastream.com | supp...@fastream.com Pbx: +90-312-223-2830 | MSN: g_a...@hotmail.com Join IQWF Server Yahoo group at http://groups.yahoo.com/group/IQWFServer Join IQ Proxy Server Yahoo group at http://groups.yahoo.com/group/IQProxyServer THIS IS NOT A NONSENSE SIGNATURE--PLEASE DO JOIN OUR YAHOO GROUPS FOR ANNOUNCEMENTS ABOUT OUR PRODUCTS! -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
One limitation with such kind of implementation (might not be relevant) is that the application blocks until entire list of events is processed. Like iterating over a list of n server clients blocks all clients. Yes, that is true, but I try not to have long processing in timers, just routines that are fast to be executed, but it is true ;) Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
