Re[2]: [twsocket] Receive result of the first ReceiveStr() call twice-not resolved
Hello Francois,
Thanks for the quick reply. You are right. I should check in
the code. But in this particular test environment, I am receiving
HTTP/1.x. This I checked from the log.
I looked at the httpcli code briefly yesterday but failed to
find the part that parses the http headers (I know it should be there.)
I have been displaying the logs in a TMemo and yes, I do receive
a part of the string twice as shown in the log. I didn't call any
of the functions you mentioned in your earlier emails about what
could mess up the message pump.
I have further stripped the test project - very simple now.
I will email it to you. Please take a look when you have time.
BTW, are attachments allowed here, or should I send you a private mail?
--
Best regards,
Jack
Thursday, June 2, 2005, 6:43:30 AM, you wrote:
> To avoid difficulties, you should first correctly write your
> GetHTTPReponseStatusCode function to
> correctly parse the answer. You should not simply remove
> "HTTP/1.1" but actually check you really
> have that string. I'm sure you think you receive data twice while
> you aren't. You think you receive
> it twice because you don't parse it correctly and you overlooked
> the mechanism that makes TCP stream
> split into several packets and OnDataAvailable events.
> You can have look at the HTTP client component source code (if
> you don't want to use it, look at the
> code and copy what you need to reinvent your wheel).
> Use OutputDebugString to display what you receive in your
> OnDataAvailable event. You'll see that you
> never receive data twice, unless you mess up with the message pump.
> --
> [EMAIL PROTECTED]
> Auteur du freeware ICS - Internet Component Suite
> Auteur du freeware MidWare - Multi-tiers framework
> http://www.overbyte.be
> - Original Message -
> From: "Jack" <[EMAIL PROTECTED]>
> To: "ICS support mailing"
> Sent: Thursday, June 02, 2005 12:39 AM
> Subject: Re: [twsocket] Receive result of the first ReceiveStr() call
> twice-not resolved
>> This problem is still bugging me. I did some further testing
>> and find some strange things. In the OnDataAvailable event
>> handler, I call the function below to get HTTP status code.
>> The HttpReply is the string variable I get from ReceiveStr() call.
>> I find that as long as I call this function, I get a part of
>> the HTTP reply twice from ReceiveStr() call. Further testing
>> shows that this line is teh culprit:
>>
>> Result := StrToInt(s);
>>
>> If I comment out this line, I don't have a problem (then I don't
>> get a result from the function either.)
>>
>> Any ideas? (Below is the whole function.)
>>
>> function GetHTTPReponseStatusCode(const HttpReply: String): Integer;
>> var i: Integer; s: String;
>> begin
>> Result := -1;
>> i := Pos(' ', HttpReply);
>> if i > 0 then
>> begin
>> s := Copy(HttpReply, i + 1, Length(HttpReply)); // Remove "HTTP/1.1 " at
>> the beginning
>> i := Pos(' ', s); // Remove the trailing text
>> if i > 0 then
>> begin
>> s := Copy(s, 1, i - 1);
>> Result := StrToInt(s);
>> end;
>> end;
>> end;
>>
>> --
>> Best regards,
>> Jack
>>
>> >> I celebrated a little too early. I am getting this problem again
>> >> in further testing after I changed Close() to CloseDelayed();
>> >> The difference is, I'm not getting result of the first ReceiveStr()
>> >> twice. Instead, I get the result of the second ReceiveStr() twice.
>> >> I think this is because CloseDelayed() delayed the Close().
>> >> What I don't understand is why the buffer isn't cleared after
>> >> ReceiveStr() in both scenarios of Close() and CloseDelayed().
>> >>
>> >> I reason I thought it was fixed is because I tested from
>> >> an allowed IP earlier and got a very short response back from
>> >> the proxy server. The whole message was retrieved in the first
>> >> ReceiveStr() call so I didn't get a chance to call ReceiveStr()
>> >> twice. I then run the same app on a blocked IP. The proxy sends
>> >> a longer message (the error message) and it is retrieved in
>> >> two ReceiveStr() calls. Then I found that the second ReceiveStr()
>> >> result was duplicated.
>> >>
>> >> --
>> >> Best regards,
>> >> Jack
>> >>
>> >> Thursday, May 19, 2005, 10:53:49 AM, you wrote:
>> >>
>> >> J> Francois, thanks! This fixed the problem! It's actually in the FAQ:
>> >>
>> >> J> - CloseDelayed
>> >>
>> >> J> Is in most cases the preferred way. It will post a message to
>> >> J> itself to close the TWSocket. This means that the socket closure
>> >> J> is done outside the code that calls the CloseDelayed, meaning a
>> >> J> while later. The message handler will call Close.
>> >>
>> >> J> - Close
>> >>
>> >> J> Attempt to gracefully close the socket. If there is still some
>> >> J> data waiting in the buffers it will try to send it. ***Do not use
>> >> J> Close from within any TWSocket events, instead use CloseDelayed.***
>> >>
>> >>
>> >> J> Thursday, May 19, 2005, 2:46:47 AM, you wrote:
>> >>
>> >> FP>> Use CloseD
Re: Re[2]: [twsocket] Receive result of the first ReceiveStr() call twice -not resolved
Jack, The logic in your WSocketDataAvailable is broken. That's the problem. You used LineMode which is fine, and you selected double CRLF as end of line. So you received the complete header into the first OnDataAvailable event. Then you receive subsequent OnDataAvailable depending on what the server replies (your lines are defined with double CRLF). To fix your code, simply add a flag in your TProxySocket class and use it to ignore anything after header. Here is the code: procedure TProxyChecker.WSocketDataAvailable(Sender: TObject; ErrCode: Word); var s, Status: String; StatusCode: Integer; begin if ErrCode = 0 then with TProxySocket(Sender) do begin s := ReceiveStr(); if FHeaderFlag then Exit; FHeaderFlag := TRUE; Log(s); CloseDelayed(); //ShutDown(-1); GetHTTPReponseStatusCode(s, StatusCode, Status); if StatusCode = 200 then AddGoodIP(Host); end; end; In your GetHTTPReponseStatusCode routine, you should check for valid HTTP reply in case you connect to a broken server or not a HTTP server. -- [EMAIL PROTECTED] http://www.overbyte.be - Original Message - From: "Jack" To: "Francois Piette" <[EMAIL PROTECTED]> Sent: Friday, May 20, 2005 10:58 PM Subject: Re[2]: [twsocket] Receive result of the first ReceiveStr() call twice -not resolved > Hello Francois, > > Thanks for offering to look at my code. > > I zipped the (simplified) project files. Since it's > small (10KB), I'll just email it to you. Please see attached. > When you run it, the log window displays some logs. > Below is what I get. Please notice that I'm getting this > line twice: > > 5/20/2005 4:46:47 PM Access denied due to Proxy+'s Security settings! > Description: REJECTED - by default (no rule allowed the access) > Request details: HTTPS; 68.38.201.173:63664 -> 192.168.2.36:4480 > Proxy+ 3.00 (Build #246), Date: Fri, 20 May 2005 16:47:22 GMT > > Also, in the end, I'm getting a WSocketBgException. It mentions > 'size="5"', which is a part of the duplicated string. So I suspect > it has something to do with the dup string: > > 5/20/2005 4:46:47 PM WSocketBgException idx=0 addr=chatsvr.no-ip.org:4480 Exception='size="5"' is not a valid integer value > > The IP I'm connecting to (chatsvr.no-ip.org) is a proxy server > I set up for testing using commercial proxy server ProxyPlus. > The client IP is disallowed so that the application will get > a longer HTTP reply in order to reproduce this problem. > > -- > Best regards, > Jack > > > 5/20/2005 4:46:47 PM GetIP: chatsvr.no-ip.org:4480 > 5/20/2005 4:46:47 PM HTTP/1.0 403 Forbidden > Date: Fri, 20 May 2005 20:47:22 GMT > Content-Type: text/html > Content-Length: 1213 > > > 5/20/2005 4:46:47 PM Access denied due to Proxy+'s Security settings! > Description: REJECTED - by default (no rule allowed the access) > Request details: HTTPS; 68.38.201.173:63664 -> 192.168.2.36:4480 > Proxy+ 3.00 (Build #246), Date: Fri, 20 May 2005 16:47:22 GMT > > > 5/20/2005 4:46:47 PM Access denied due to Proxy+'s Security settings! > Description: REJECTED - by default (no rule allowed the access) > Request details: HTTPS; 68.38.201.173:63664 -> 192.168.2.36:4480 > Proxy+ 3.00 (Build #246), Date: Fri, 20 May 2005 16:47:22 GMT > > > > > > 5/20/2005 4:46:47 PM WSocketBgException idx=0 addr=chatsvr.no-ip.org:4480 Exception='size="5"' is not a valid integer value > > > Friday, May 20, 2005, 3:50:18 AM, you wrote: > > FP> I still persist saying the you probably call the message pump > FP> directly or indirectly (for example > FP> your Log() function, or any component you use for display). > FP> I suggest you make a stripped down version of your program, > FP> just he bare minimum in a small project. > FP> If the problem persist with this small project, make it > FP> available for download somewhere. If I have > FP> time, I'll look at it. > > FP> -- > FP> Contribute to the SSL Effort. Visit > FP> http://www.overbyte.be/eng/ssl.html > FP> -- > FP> [EMAIL PROTECTED] > FP> Author of ICS (Internet Component Suite, freeware) > FP> Author of MidWare (Multi-tier framework, freeware) > FP> http://www.overbyte.be > > > FP> - Original Message - > FP> From: "Jack" <[EMAIL PROTECTED]> > FP> To: "ICS support mailing" > FP> Sent: Thursday, May 19, 2005 9:31 PM > FP> Subject: [twsocket] Receive result of the first ReceiveStr() call twice -not resolved > > > >> I celebrated a little too early. I am getting this problem again > >> in further
Re[2]: [twsocket] Receive result of the first ReceiveStr() call twice -not resolved
Thanks Francois. I double-checked and didn't see any calls to ProcessMessages or ShowMessage, etc. I'll make a simplified version and send the code in private email. Jack Friday, May 20, 2005, 3:50:18 AM, you wrote: FP> I still persist saying the you probably call the message pump FP> directly or indirectly (for example FP> your Log() function, or any component you use for display). FP> I suggest you make a stripped down version of your program, FP> just he bare minimum in a small project. FP> If the problem persist with this small project, make it FP> available for download somewhere. If I have FP> time, I'll look at it. FP> -- FP> Contribute to the SSL Effort. Visit FP> http://www.overbyte.be/eng/ssl.html FP> -- FP> [EMAIL PROTECTED] FP> Author of ICS (Internet Component Suite, freeware) FP> Author of MidWare (Multi-tier framework, freeware) FP> http://www.overbyte.be FP> - Original Message - FP> From: "Jack" <[EMAIL PROTECTED]> FP> To: "ICS support mailing" FP> Sent: Thursday, May 19, 2005 9:31 PM FP> Subject: [twsocket] Receive result of the first ReceiveStr() call twice -not resolved >> I celebrated a little too early. I am getting this problem again >> in further testing after I changed Close() to CloseDelayed(); >> The difference is, I'm not getting result of the first ReceiveStr() >> twice. Instead, I get the result of the second ReceiveStr() twice. >> I think this is because CloseDelayed() delayed the Close(). >> What I don't understand is why the buffer isn't cleared after >> ReceiveStr() in both scenarios of Close() and CloseDelayed(). >> >> I reason I thought it was fixed is because I tested from >> an allowed IP earlier and got a very short response back from >> the proxy server. The whole message was retrieved in the first >> ReceiveStr() call so I didn't get a chance to call ReceiveStr() >> twice. I then run the same app on a blocked IP. The proxy sends >> a longer message (the error message) and it is retrieved in >> two ReceiveStr() calls. Then I found that the second ReceiveStr() >> result was duplicated. >> >> -- >> Best regards, >> Jack >> >> Thursday, May 19, 2005, 10:53:49 AM, you wrote: >> >> J> Francois, thanks! This fixed the problem! It's actually in the FAQ: >> >> J> - CloseDelayed >> >> J> Is in most cases the preferred way. It will post a message to >> J> itself to close the TWSocket. This means that the socket closure >> J> is done outside the code that calls the CloseDelayed, meaning a >> J> while later. The message handler will call Close. >> >> J> - Close >> >> J> Attempt to gracefully close the socket. If there is still some >> J> data waiting in the buffers it will try to send it. ***Do not use >> J> Close from within any TWSocket events, instead use CloseDelayed.*** >> >> >> J> Thursday, May 19, 2005, 2:46:47 AM, you wrote: >> >> FP>> Use CloseDelayed instead of Close. >> >> >>> Hello Wilfried, >> >>> >> >>> Sure. Below is my event handler code. Hope I'm doing something >> >>> wrong, otherwise, it's really strange. TProxyChecker is the owner >> >>> object of the TProxySocket object array. TProxySocket is derived >> >>> from TWSocket. >> >>> >> >>> procedure TProxyChecker.WSocketDataAvailable(Sender: TObject; ErrCode: >> >>> Word); >> >>> var s: String; >> >>> begin >> >>> if ErrCode = 0 then >> >>> with TProxySocket(Sender) do >> >>> begin >> >>> s := ReceiveStr(); >> >>> Log(s); >> >>> Close(); >> >>> end; >> >>> end; >> >>> >> >>> The destination server is a commercial http proxy server. >> >>> I tried using #13#10 and #13#10#13#10, I different strings >> >>> read back from ReceiveStr. But the second string includes >> >>> the first string in both cases. So it shouldn't be the server >> >>> sending the string twice. >> >>> >> >>> BTW, I'm using the latest ICS. >> >>> >> >>> -- >> >>> Best regards, >> >>> Jack >> >>> >> >>> Wednesday, May 18, 2005, 3:31:32 PM, you wrote: >> >>> >> >>> WM> Hello Jack, >> >>> >> >>> WM> Can you show your OnDataAvailable handler ? >> >>> WM> Eventually download SocketSpy from 'user made' page and 'hang' it >> >>> WM> between client and server. Then you see exacly what is sent by >> >>> server. >> >>> >> >>> WM> --- >> >>> WM> Rgds, Wilfried >> >>> WM> http://www.mestdagh.biz >> >>> >> >>> WM> Wednesday, May 18, 2005, 20:48, Jack wrote: >> >>> >> >>> >> Hello Francois and all, >> >>> >> >>> >> I'm using a TWSocket client in LineMode with LineEnd set to #13#10. >> >>> >> I connect to a HTTP proxy server using CONNECT command. I then get >> >>> >> a reply back from the HTTP proxy from ReceiveStr() >> >>> >> >>> >> Things look OK except that I am receiving the data from the first >> >>> >> ReceiveStr() call twice. I get two WSocketDataAvailable messages for >> >>> >> 3 lines in the HTTP response: >> >>> >> >>> >> Line 1: HTTP/1.0 200 Connection established >> >>> >> Line 2: Proxy-agent: Proxy+ 3.00 >> >>> >> Line 3: (Blank line) >> >>> >> >>> >> However, I'm getting the first Recei
