Re: [twsocket] HTTP Server creates extremly high CPU load
Lars Gehre wrote:
> I think, I found something.
Thanks for reporting, I just checked in a fix.
Available via SVN now or included in next daily snapshot
here: http://wiki.overbyte.be/wiki/index.php/ICS_Download
--
Arno Garrels
> The problem seems to be the head request. Normaly you do a HEAD and
> close the connection, but in my case it stays open and the http
> server gets stuck in an endless loop.
>
> The culprits are:
>
> procedure THttpConnection.SendDocument(SendType : THttpSendType;const
> CustomHeaders : String);
> and
> procedure THttpConnection.ConnectionDataSent(Sender : TObject; Error :
> WORD);
>
> In SendDocument the fdocsize gets set to the size of the stream and a
>
> if SendType = httpSendHead then
>FDocStream.Seek(0, soFromEnd)
>
> And
>if SendType = httpSendDoc then
>SendStream
>else
>Send(nil, 0); { Added 15/04/02 } // this one triggers the
> ConnectionDataSent.
>
>
> In ConnectionDataSent FDataSent >= FDocSize is the end trigger but
>
> ToSend := FDocSize - FDataSent; //fdatasent = 0
>if ToSend > FSndBlkSize then
>ToSend := FSndBlkSize;
>Count := FDocStream.Read(FDocBuf^, ToSend); // =0 because of
> FDocStream.Seek(0, soFromEnd) in SendDocument
>FDataSent := FDataSent + Count; { Count data which is sent
> } // 0 + 0 = 0
>if State = wsConnected then { Be sure to be still
> connected... }
>Send(FDocBuf, Count);{ before actually send any
> data. } // starts it all over again...
>
> Maybe a
>if (FDataSent >= FDocSize) or
> (FDocStream.Position=FDocStream.Size) then begin
>
> might be a solution, but I don't know enough of the inner workings of
> the http component to foresee possible side effects...
>
> Lars
>
>
>
>> -Original Message-
>> From: twsocket-boun...@elists.org
>> [mailto:twsocket-boun...@elists.org] On Behalf Of Lars Gehre
>> Sent: Thursday, June 30, 2011 9:49 PM
>> To: 'ICS support mailing'
>> Subject: Re: [twsocket] HTTP Server creates extremly high CPU load
>>
>> Well I was a little to fast with my conclusion. It only
>> happens to files in the demo because they have a proper
>> header, the virtual documents only give a 404 for a HEAD request.
>>
>> The download is not of matter, the demo files are 500 bytes
>> or so. And the server load stays as long as the client is connected.
>>
>> As I said, I was able to replicate this on two different
>> machines and it is not limited to local access. cross tests
>> between the machines also show this behaviour.
>>
>> Lars
>>
>>
>>> -Original Message-
>>> From: twsocket-boun...@elists.org
>>> [mailto:twsocket-boun...@elists.org] On Behalf Of Arno Garrels
>>> Sent: Thursday, June 30, 2011 3:49 PM
>>> To: ICS support mailing
>>> Subject: Re: [twsocket] HTTP Server creates extremly high CPU load
>>>
>>> Lars Gehre wrote:
I noticed one thing: This only happens if files are involved, for
virtual documents it does not happen.
>>>
>>> I cannot reproduce it, 0.0% CPU here.
>>> I used the OverbyteIcsWebServ demo running on a Win7 quadcore.
>>> However if I download a file with Firefox (100 Mbit/s LAN)
>> at around 6
>>> MB/s CPU 0 uses ~25-30% and CPU 2 8-15%, overall usage of
>> the server
>>> process ~8-10% reported by Process Explorer.
>>>
>>> --
>>> Arno Garrels
>>>
>>>
>>>
For the http server demo it means you need to open Index.html or
Form.html.
Lars
> -Original Message-
> From: twsocket-boun...@elists.org
> [mailto:twsocket-boun...@elists.org] On Behalf Of Lars Gehre
> Sent: Thursday, June 30, 2011 11:21 AM
> To: 'ICS support mailing'
> Subject: [twsocket] HTTP Server creates extremly high CPU load
>
> Hello all,
>
> I have a serious problem with the http server component from
> ICS7 (downloaded from the repository last week).
>
> I'm using the wininet api on the client side to open a (GET)
> connection to a ICS http Server.
> For certain reasons I need to query the HEAD in intervals (while
> the other get is still in progress).
> As soon as I send the second request the CPU load of the http
> server maxes out one CPU core.
>
> Steps to reproduce:
>
> - Run the http server demo from ics.
>
> - Download the demo
> www.dvbviewer.tv/icsproblem/icsproblem.zip source (delphi 7)
>
> - Run the demo, enter the url of the demo server and press the
> button.
>
> - As soon as the call to GetFileSize is finished you see the CPU
> load of the webserver going up.
>
>
> I checked it on two windows 7 (multicore) machines with the same
> result.
>
> I also checked it against several other http servers and none of
> them had a problem with this kind of usage.
>
>
> Thanks
> Lars
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
>
Re: [twsocket] HTTP Server creates extremly high CPU load
I think, I found something.
The problem seems to be the head request. Normaly you do a HEAD and close
the connection, but in my case it stays open and the http server gets stuck
in an endless loop.
The culprits are:
procedure THttpConnection.SendDocument(SendType : THttpSendType;const
CustomHeaders : String);
and
procedure THttpConnection.ConnectionDataSent(Sender : TObject; Error :
WORD);
In SendDocument the fdocsize gets set to the size of the stream and a
if SendType = httpSendHead then
FDocStream.Seek(0, soFromEnd)
And
if SendType = httpSendDoc then
SendStream
else
Send(nil, 0); { Added 15/04/02 } // this one triggers the
ConnectionDataSent.
In ConnectionDataSent FDataSent >= FDocSize is the end trigger but
ToSend := FDocSize - FDataSent; //fdatasent = 0
if ToSend > FSndBlkSize then
ToSend := FSndBlkSize;
Count := FDocStream.Read(FDocBuf^, ToSend); // =0 because of
FDocStream.Seek(0, soFromEnd) in SendDocument
FDataSent := FDataSent + Count; { Count data which is sent
} // 0 + 0 = 0
if State = wsConnected then { Be sure to be still connected...
}
Send(FDocBuf, Count);{ before actually send any data.
} // starts it all over again...
Maybe a
if (FDataSent >= FDocSize) or (FDocStream.Position=FDocStream.Size) then
begin
might be a solution, but I don't know enough of the inner workings of the
http component to foresee possible side effects...
Lars
> -Original Message-
> From: twsocket-boun...@elists.org
> [mailto:twsocket-boun...@elists.org] On Behalf Of Lars Gehre
> Sent: Thursday, June 30, 2011 9:49 PM
> To: 'ICS support mailing'
> Subject: Re: [twsocket] HTTP Server creates extremly high CPU load
>
> Well I was a little to fast with my conclusion. It only
> happens to files in the demo because they have a proper
> header, the virtual documents only give a 404 for a HEAD request.
>
> The download is not of matter, the demo files are 500 bytes
> or so. And the server load stays as long as the client is connected.
>
> As I said, I was able to replicate this on two different
> machines and it is not limited to local access. cross tests
> between the machines also show this behaviour.
>
> Lars
>
>
> > -Original Message-
> > From: twsocket-boun...@elists.org
> > [mailto:twsocket-boun...@elists.org] On Behalf Of Arno Garrels
> > Sent: Thursday, June 30, 2011 3:49 PM
> > To: ICS support mailing
> > Subject: Re: [twsocket] HTTP Server creates extremly high CPU load
> >
> > Lars Gehre wrote:
> > > I noticed one thing: This only happens if files are involved, for
> > > virtual documents it does not happen.
> >
> > I cannot reproduce it, 0.0% CPU here.
> > I used the OverbyteIcsWebServ demo running on a Win7 quadcore.
> > However if I download a file with Firefox (100 Mbit/s LAN)
> at around 6
> > MB/s CPU 0 uses ~25-30% and CPU 2 8-15%, overall usage of
> the server
> > process ~8-10% reported by Process Explorer.
> >
> > --
> > Arno Garrels
> >
> >
> >
> > >
> > > For the http server demo it means you need to open Index.html or
> > > Form.html.
> > >
> > > Lars
> > >
> > >> -Original Message-
> > >> From: twsocket-boun...@elists.org
> > >> [mailto:twsocket-boun...@elists.org] On Behalf Of Lars Gehre
> > >> Sent: Thursday, June 30, 2011 11:21 AM
> > >> To: 'ICS support mailing'
> > >> Subject: [twsocket] HTTP Server creates extremly high CPU load
> > >>
> > >> Hello all,
> > >>
> > >> I have a serious problem with the http server component from
> > >> ICS7 (downloaded from the repository last week).
> > >>
> > >> I'm using the wininet api on the client side to open a (GET)
> > >> connection to a ICS http Server.
> > >> For certain reasons I need to query the HEAD in intervals (while
> > >> the other get is still in progress).
> > >> As soon as I send the second request the CPU load of the http
> > >> server maxes out one CPU core.
> > >>
> > >> Steps to reproduce:
> > >>
> > >> - Run the http server demo from ics.
> > >>
> > >> - Download the demo
> > >> www.dvbviewer.tv/icsproblem/icsproblem.zip source (delphi 7)
> > >>
> > >> - Run the demo, enter the url of the demo server and press the
> > >> button.
> > >>
> > >> - As soon as the call to GetFileSize is finished you see the CPU
> > >> load of the webserver going up.
> > >>
> > >>
> > >> I checked it on two windows 7 (multicore) machines with the same
> > >> result.
> > >>
> > >> I also checked it against several other http servers and none of
> > >> them had a problem with this kind of usage.
> > >>
> > >>
> > >> Thanks
> > >> Lars
> > >>
> > >> --
> > >> To unsubscribe or change your settings for TWSocket mailing list
> > >> please goto
> > >> http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> > >> Visit our website at http://www.overbyte.be
> > --
> > To unsubscribe or change your settings for TWSocket mailing list
> > please goto
> http:
[twsocket] Showstopped bug in ICS
Hi
I believe I found a showstopper bug in ICS that lead my software to crash in
many customers.
The problem is with wsoNoReceiveLoop option from socket!
In my program, I have to use wsoNoReceiveLoop in order to better process the
data, I don´t want to process the data from the thread that fires the
OnDataAvailable event, instead, I want to read the data from another thread...
Thats fine... but.. I found this:
procedure TCustomWSocket.Do_FD_CLOSE(var msg: TMessage);
begin
{ In some strange situations I found that we receive a FD_CLOSE }
{ during the connection phase, breaking the connection early ! }
{ This occurs for example after a failed FTP transfert Probably }
{ something related to bugged winsock. Doesn't hurt with good}
{ winsock. So let the code there ! }
if (FState <> wsConnecting) and (FHSocket <> INVALID_SOCKET) then begin
{ Check if we have something arrived, if yes, process it }
{ DLR, since we are closing MAKE SURE WE LOOP in the receive }
{ function to get ALL remaining data }
ASyncReceive(0, RemoveOption(FComponentOptions, wsoNoReceiveLoop));
if not FCloseInvoked then begin
FCloseInvoked := TRUE;
TriggerSessionClosed(HiWord(msg.LParam));
end;
if FState <> wsClosed then
Close;
end;
end;
On routine Do_FD_CLOSE, the ASyncReceive (Which triggers my OnDataAvailable
event handler) can be called WITHOUT the wsoNoReceiveLoop.. guess what is
happening? My OnDataAvailable event handler DOES NOT process the data from the
socket, and since ASyncReceive was called with wsoNoReceiveLoop it is expecting
that my event handler DO receive the data (But it doesn´t) and then, it locks
up on a loop!
What can I do??
Eric
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] HTTP Server creates extremly high CPU load
Well I was a little to fast with my conclusion. It only happens to files in the demo because they have a proper header, the virtual documents only give a 404 for a HEAD request. The download is not of matter, the demo files are 500 bytes or so. And the server load stays as long as the client is connected. As I said, I was able to replicate this on two different machines and it is not limited to local access. cross tests between the machines also show this behaviour. Lars > -Original Message- > From: twsocket-boun...@elists.org > [mailto:twsocket-boun...@elists.org] On Behalf Of Arno Garrels > Sent: Thursday, June 30, 2011 3:49 PM > To: ICS support mailing > Subject: Re: [twsocket] HTTP Server creates extremly high CPU load > > Lars Gehre wrote: > > I noticed one thing: This only happens if files are involved, for > > virtual documents it does not happen. > > I cannot reproduce it, 0.0% CPU here. > I used the OverbyteIcsWebServ demo running on a Win7 quadcore. > However if I download a file with Firefox (100 Mbit/s LAN) at > around 6 MB/s CPU 0 uses ~25-30% and CPU 2 8-15%, overall > usage of the server process ~8-10% reported by Process Explorer. > > -- > Arno Garrels > > > > > > > For the http server demo it means you need to open > > Index.html or Form.html. > > > > Lars > > > >> -Original Message- > >> From: twsocket-boun...@elists.org > >> [mailto:twsocket-boun...@elists.org] On Behalf Of Lars Gehre > >> Sent: Thursday, June 30, 2011 11:21 AM > >> To: 'ICS support mailing' > >> Subject: [twsocket] HTTP Server creates extremly high CPU load > >> > >> Hello all, > >> > >> I have a serious problem with the http server component from > >> ICS7 (downloaded from the repository last week). > >> > >> I'm using the wininet api on the client side to open a (GET) > >> connection to a ICS http Server. > >> For certain reasons I need to query the HEAD in intervals > >> (while the other get is still in progress). > >> As soon as I send the second request the CPU load of the http > >> server maxes out one CPU core. > >> > >> Steps to reproduce: > >> > >> - Run the http server demo from ics. > >> > >> - Download the demo > >> www.dvbviewer.tv/icsproblem/icsproblem.zip source (delphi 7) > >> > >> - Run the demo, enter the url of the demo server and press the > >> button. > >> > >> - As soon as the call to GetFileSize is finished you see the > >> CPU load of the webserver going up. > >> > >> > >> I checked it on two windows 7 (multicore) machines with the > >> same result. > >> > >> I also checked it against several other http servers and none > >> of them had a problem with this kind of usage. > >> > >> > >> Thanks > >> Lars > >> > >> -- > >> To unsubscribe or change your settings for TWSocket mailing > >> list please goto > >> http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket > >> Visit our website at http://www.overbyte.be > -- > To unsubscribe or change your settings for TWSocket mailing list > please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket > Visit our website at http://www.overbyte.be > -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HTTP Server creates extremly high CPU load
Lars Gehre wrote: > I noticed one thing: This only happens if files are involved, for > virtual documents it does not happen. I cannot reproduce it, 0.0% CPU here. I used the OverbyteIcsWebServ demo running on a Win7 quadcore. However if I download a file with Firefox (100 Mbit/s LAN) at around 6 MB/s CPU 0 uses ~25-30% and CPU 2 8-15%, overall usage of the server process ~8-10% reported by Process Explorer. -- Arno Garrels > > For the http server demo it means you need to open > Index.html or Form.html. > > Lars > >> -Original Message- >> From: twsocket-boun...@elists.org >> [mailto:twsocket-boun...@elists.org] On Behalf Of Lars Gehre >> Sent: Thursday, June 30, 2011 11:21 AM >> To: 'ICS support mailing' >> Subject: [twsocket] HTTP Server creates extremly high CPU load >> >> Hello all, >> >> I have a serious problem with the http server component from >> ICS7 (downloaded from the repository last week). >> >> I'm using the wininet api on the client side to open a (GET) >> connection to a ICS http Server. >> For certain reasons I need to query the HEAD in intervals >> (while the other get is still in progress). >> As soon as I send the second request the CPU load of the http >> server maxes out one CPU core. >> >> Steps to reproduce: >> >> - Run the http server demo from ics. >> >> - Download the demo >> www.dvbviewer.tv/icsproblem/icsproblem.zip source (delphi 7) >> >> - Run the demo, enter the url of the demo server and press the >> button. >> >> - As soon as the call to GetFileSize is finished you see the >> CPU load of the webserver going up. >> >> >> I checked it on two windows 7 (multicore) machines with the >> same result. >> >> I also checked it against several other http servers and none >> of them had a problem with this kind of usage. >> >> >> Thanks >> Lars >> >> -- >> To unsubscribe or change your settings for TWSocket mailing >> list please goto >> http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket >> Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HTTP Server creates extremly high CPU load
I noticed one thing: This only happens if files are involved, for virtual documents it does not happen. For the http server demo it means you need to open Index.html or Form.html. Lars > -Original Message- > From: twsocket-boun...@elists.org > [mailto:twsocket-boun...@elists.org] On Behalf Of Lars Gehre > Sent: Thursday, June 30, 2011 11:21 AM > To: 'ICS support mailing' > Subject: [twsocket] HTTP Server creates extremly high CPU load > > Hello all, > > I have a serious problem with the http server component from > ICS7 (downloaded from the repository last week). > > I'm using the wininet api on the client side to open a (GET) > connection to a ICS http Server. > For certain reasons I need to query the HEAD in intervals > (while the other get is still in progress). > As soon as I send the second request the CPU load of the http > server maxes out one CPU core. > > Steps to reproduce: > > - Run the http server demo from ics. > > - Download the demo > www.dvbviewer.tv/icsproblem/icsproblem.zip source (delphi 7) > > - Run the demo, enter the url of the demo server and press the button. > > - As soon as the call to GetFileSize is finished you see the > CPU load of the webserver going up. > > > I checked it on two windows 7 (multicore) machines with the > same result. > > I also checked it against several other http servers and none > of them had a problem with this kind of usage. > > > Thanks > Lars > > -- > To unsubscribe or change your settings for TWSocket mailing > list please goto > http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket > Visit our website at http://www.overbyte.be > -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] HTTP Server creates extremly high CPU load
Hello all, I have a serious problem with the http server component from ICS7 (downloaded from the repository last week). I'm using the wininet api on the client side to open a (GET) connection to a ICS http Server. For certain reasons I need to query the HEAD in intervals (while the other get is still in progress). As soon as I send the second request the CPU load of the http server maxes out one CPU core. Steps to reproduce: - Run the http server demo from ics. - Download the demo www.dvbviewer.tv/icsproblem/icsproblem.zip source (delphi 7) - Run the demo, enter the url of the demo server and press the button. - As soon as the call to GetFileSize is finished you see the CPU load of the webserver going up. I checked it on two windows 7 (multicore) machines with the same result. I also checked it against several other http servers and none of them had a problem with this kind of usage. Thanks Lars -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
