Re: [twsocket] bug in TCustomWSocket? (regarding wsoNoReceiveLoop)
Merijn Bosma wrote: I've been using ICS for quite a while now, and a few days ago I came across something strange. One of our larger customers had some extra modules installed, causing a lot of more tcp clients connecting to the tcp server. Every now and then (somewhere between 1 minute and 3 hours) the tcp server would get up to 100% cpu utilization out of the blue. After some searching we discovered that this was caused by the OnDataAvailable event for one of the connected clients being called continuously. Now the mystery: If I understood correct, you can get an endless loop if you don't read all received data in the OnDataAvailable event. You get a real infinite loop only if you called Receive() with a buffer size of zero. The event will keep being fired until all data is read, unless you add wsoNoReceiveLoop. With option wsoNoReceiveLoop turned on OnDataAvailable will also trigger again and again as long as data is available. The difference is that the app. is given a chance to call its message pump between two OnDataAvailable events. As a result CPU utilization becomes lower. This may however also lower transfer rates a bit. The thing is that we _always_ call ReceiveStr() in the OnDataAvailable event, I think this should always read all data received. Depends on LineMode, if LineMode is TRUE ReceiveStr() just reads a single line. Is this a bug in ReceiveStr() or a bug regarding wsoNoReceiveLoop? I don't think it's a bug, wsoNoReceiveLoop is an option turned off by default for backwards compatibility. -- Arno Garrels [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] bug in TCustomWSocket? (regarding wsoNoReceiveLoop)
Thanks for your reply! Arno Garrels wrote: You get a real infinite loop only if you called Receive() with a buffer size of zero. Which is not the case The thing is that we _always_ call ReceiveStr() in the OnDataAvailable event, I think this should always read all data received. Depends on LineMode, if LineMode is TRUE ReceiveStr() just reads a single line. LineMode is false I don't think it's a bug, wsoNoReceiveLoop is an option turned off by default for backwards compatibility. I'm really looking for an explanation of what happened, so I can confirm the bug is really fixed. Facts: - 100% cpu utilization caused by OnDataAvailable being called in a loop (sync, so no change for the message pump to work) - Linemode = false - we call ReceiveStr() on _every_ OnDataAvailable event - when wsoNoReceiveLoop is set, we don't have this problem anymore these things seem to contradict each other? It of course is possible that with wsoNoReceiveLoop set we still have the same problem, but we don't notice it because the message pump has a change to kick in. Is there any way I can get my fingers around this? Reading it all again, I'm still coming to the same conclusion: either bug in ReceiveStr(), bug in handling of wsoNoReceiveLoop, or me being stupid and don't understanding things correctly. Can anybody shed some light? :p -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Proposal for new content-type in function DocumentToContentType in OverbyteIcsHttpSrv.pas
On Apr 9, 2008, at 03:45, Bjørnar Nielsen wrote: The comment about the registry was there before I made any changes. My only change was to add 'text/xml' as content-type for extension 'xml'. For now my only need is this extra content-type, not another way of managing the list of content-types. When using Flash, if you have the need to do cross-domain-scripting, you have to have a file called crossdomain.xml on the root, defining the rules for crossdomain-scripting. And the content-type has to be text/xml when downloaded to flash from a server in order to work. I understand, but perhaps ICS could benefit from having automatic MIME-Type detection/handling, not only for the HTTP components, but for the SMTP as well. Currently, you have to set the MIME-Type manually on the SMTPCli component; and any attachments are automatically set to application/octet-stream, regardless of whether they are plain text files or binaries. So, if there is a plan to add such functionality to the ICS components, it is my opinion that using the Registry is the wrong thing to do. Indy does this, and then you depend on the host computer having the proper types registered. dZ. -- DZ-Jay [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] bug in TCustomWSocket? (regarding wsoNoReceiveLoop)
Merijn Bosma wrote: again, I'm still coming to the same conclusion: either bug in ReceiveStr(), bug in handling of wsoNoReceiveLoop, Again I don't agree because as you described it works as documented. Quoted from the unit comment section: Added ComponentOptions property with currently only one options wsoNoReceiveLoop which disable a receive loop in AsyncReceive. This loop breaking was suggested by Davie [EMAIL PROTECTED] to lower resource usage with really fast LAN and large transfers. By default, this option is disabled so there is no change needed in current code. -- Arno Garrels [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] bug in TCustomWSocket? (regarding wsoNoReceiveLoop)
I'll reply to both your mails in a single one: One possible explanation were that anytime OnDataAvailable returns, just a few _new_ bytes are available in winsock buffer. Could be possible due to the background thread winsock creates internally with non-blocking sockets, though I'm not aware of how winsock works under the hood. When in this situation, we got a few hundred calls a second, for an unlimited time, where normally it's one every few seconds. I agree with you, it could be a possible cause, but very unlikely. Arno Garrels wrote: Merijn Bosma wrote: again, I'm still coming to the same conclusion: either bug in ReceiveStr(), bug in handling of wsoNoReceiveLoop, Again I don't agree because as you described it works as documented. Quoted from the unit comment section: Added ComponentOptions property with currently only one options wsoNoReceiveLoop which disable a receive loop in AsyncReceive. This loop breaking was suggested by Davie [EMAIL PROTECTED] to lower resource usage with really fast LAN and large transfers. By default, this option is disabled so there is no change needed in current code. Ok, but can the behavior be explained concerning what's said above? tx, Merijn -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] bug in TCustomWSocket? (regarding wsoNoReceiveLoop)
One possible explanation were that anytime OnDataAvailable returns, just a few _new_ bytes are available in winsock buffer. Could be possible due to the background thread winsock creates internally with non-blocking sockets, though I'm not aware of how winsock works under the hood. When in this situation, we got a few hundred calls a second, for an unlimited time, where normally it's one every few seconds. I agree with you, it could be a possible cause, but very unlikely. Do you receive data each time you call ReceiveStr ? If yes, then probably you have a fast network, and your application becomes CPU bound. btw: Since you don't use line mode, I strongly suggest you avoid ReceiveStr and instead use Receive with a fixed buffer. Calling ReceiveStr will force the runtime to create new string and copy data into it, and probably youthen copy data elsewhere for processing. All this is very CPU intensive. The best way to design a high performance application is to use a fixed buffer (which may be dynamically allocated but not each time you receive data). And use the buffer as close as possible of the processing each time you copy data, each time you consume CPU and memory for nothing. -- [EMAIL PROTECTED] The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
