[twsocket] Multi-threaded VCL architecture question
I am trying to trace memory leaks reported by AQTime and I suspect it may
have something to do with the design of my application, the HTTPCli
component, and the multi-threaded limitations of the VCL.
I have an EXE which loads a dynamic DLL. Both of them built with C++Builder
2009 and using the latest ICS V7 (from
http://www.overbyte.be/cgi-bin/redirect.cgi?url=http://www.overbyte.be/arch/
OverbyteIcsV7.zip).
In the DLL, I have a link checker that uses ICS. It creates multiple worker
threads and multiple HTTPCli objects and uses asynchronous calls.
AQTime reports small memory leaks. Originally it was when calling
Application->ProcessMessages(), but I think that was wrong since I know now
that should only be called from the main VCL thread.
So I created my own message loop:
void AWProcessMessages(void) {
MSG Msg;
while(PeekMessage(&Msg, NULL, 0, 0, PM_REMOVE) > 0) {
TranslateMessage(&Msg);
DispatchMessage(&Msg); // memory leaks?
}
}
I call that instead of ProcessMessages() in the DLL threads, but AQTime is
still reporting memory leaks, I believe for the line DispatchMessage().
1. First, is there a problem creating HTTPCli components in DLL created
threads that are not the main VCL thread? If so, could this be the problem
and is there a workaround? I suspect processing the messages may be
resulting in VCL functions being called by a non-VCL safe thread (not the
main VCL thread).
2. Any other ideas what might be causing these small memory leaks AQTime is
reporting? Or perhaps they could be false?
Thanks!
--
Albert Wiersch
AI Internet Solutions
supp...@htmlvalidator.com
http://www.htmlvalidator.com/
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] Problem with smtp file attach send
Hello Andrea, > the smtp server is my provider smtp relay ( TELECOM ITALIA) .it's > public smtp relay for each adsl connection . > Im my Outlook client i use it from the same dvelop PC and all mail > with all attachment < 10 Mb will sent OK. > > so , i have put your code in your demo app ..and the result it's the > same. Then my guess was obviously wrong :( > > only if i check progress the email will sent ( with large attach).. > it's like during the submission process, if I insert a small delay the > server can process the email ( with large attach).. > > in fact if i put a small loop in SmtpClientMessageDataSent procedure > or sleep(150) in myapp and it send email correctly .with all attach. > but i can't do it in my real app , because there are a lot > simultaneous thread ..e it impact a lot in GUI and in run time > process. Are you able to reproduce the problem with a different mail server? Also have you tried without pumping the large attachment through the memo (property OnDisplay not assigned)? Do you have any background AV scanner running or some internet security suite? I'm actually running out of ideas since I am not able to reproduce the issue at all, tested with D7 and D2010, maybe someone else can? However did not pump the attachment thru the memo. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HTTP persistent connections
RTT wrote: > On 07-03-2011 06:31, Arno Garrels wrote: >> TWSocket property KeepAliveOnOff may be used to setup winsock to send >> keep-alive packets in the background in order to detect such brocken >> connections, however that's not reliable since both peers must >> support it and routers have to route the keep-alive packets. > And, at least by this post > http://www.mail-archive.com/twsocket@elists.org/msg02934.html > don't seems to be working at winsock level either I tested it at that time on XP and it worked fine in my LAN. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HTTP persistent connections
RTT wrote: > On 07-03-2011 06:31, Arno Garrels wrote: >> That's sounds like a bug. I would expect that THttpCli is reset to >> default values after that error? > I don't see any reference to "10053" in any of the ICS code, so I > suppose this situation is not being handled internally by the > THttpCli. Because the THttpCli try to reuse the connection, it should > be also the responsible to retry with a new connection too, if while > reusing it resulted in this specific error. The THttpCli should not handle reconnects in such cases, however the default values should be restored whenever the connection closed unexpectedly. It's another story if an exception raised, after that the component user is responsible to reset it to a consistent state, Abort might work in most cases however recreate the object from scratch is the secure way. In order to debug and replay this issue try disconnecting the network cable while a connection persists, do a request and try to find the reason why the state is not reset after the failure, just an idea? -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HTTP persistent connections
On 07-03-2011 06:31, Arno Garrels wrote: TWSocket property KeepAliveOnOff may be used to setup winsock to send keep-alive packets in the background in order to detect such brocken connections, however that's not reliable since both peers must support it and routers have to route the keep-alive packets. And, at least by this post http://www.mail-archive.com/twsocket@elists.org/msg02934.html don't seems to be working at winsock level either -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HTTP persistent connections
On 07-03-2011 06:31, Arno Garrels wrote: That's sounds like a bug. I would expect that THttpCli is reset to default values after that error? I don't see any reference to "10053" in any of the ICS code, so I suppose this situation is not being handled internally by the THttpCli. Because the THttpCli try to reuse the connection, it should be also the responsible to retry with a new connection too, if while reusing it resulted in this specific error. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [OT] Upgrading a (very) old Delphi to Delphi XE
Any news for 64-bit RAD Studio? SubZero On Mon, Mar 7, 2011 at 2:45 PM, Francois PIETTE wrote: > This is out of topic, well almost, but I wanted to let you know that > Embarcadero is currently offering several unique opportunities until march > 31: > > 1. Upgrade Amnesty > > Users of all earlier versions of Delphi, C++Builder, RAD Studioand Borland > Developer Studio qualify for upgrade pricing > (normally only 2007-2010 users qualify) > http://www.embarcadero.com/radspecial > > > 2. XE Pro to XE Enterprise Upgrade Special Prices > > Delphi, C++Builder and RAD Studio XE Pro owners get $700-$1,000 off > upgrades to XE Enterprise by purchasing discounted XE Pro to XE Ent upgrade > SKUs. With this offer, Pro users can get DataSnap and more. > http://www.embarcadero.com/radoffer > > > 3. XE Enterprise Upgrade Special Prices > > Users of any earlier version, any edition, of Delphi, C++Builder, RAD > Studio or BDS get $300 off Delphi XE, C++Builder XE and RAD Studio XE > Enterprise Upgrades. Delphi or C++Builder upgrades were $1,299 now $999. RAD > Studio was $1,799 now $1,499. http://www.embarcadero.com/radoffer > > > -- > francois.pie...@overbyte.be > The author of the freeware multi-tier middleware MidWare > The author of the freeware Internet Component Suite (ICS) > http://www.overbyte.be > > -- > To unsubscribe or change your settings for TWSocket mailing list > please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket > Visit our website at http://www.overbyte.be > -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] [OT] Upgrading a (very) old Delphi to Delphi XE
This is out of topic, well almost, but I wanted to let you know that Embarcadero is currently offering several unique opportunities until march 31: 1. Upgrade Amnesty Users of all earlier versions of Delphi, C++Builder, RAD Studioand Borland Developer Studio qualify for upgrade pricing (normally only 2007-2010 users qualify) http://www.embarcadero.com/radspecial 2. XE Pro to XE Enterprise Upgrade Special Prices Delphi, C++Builder and RAD Studio XE Pro owners get $700-$1,000 off upgrades to XE Enterprise by purchasing discounted XE Pro to XE Ent upgrade SKUs. With this offer, Pro users can get DataSnap and more. http://www.embarcadero.com/radoffer 3. XE Enterprise Upgrade Special Prices Users of any earlier version, any edition, of Delphi, C++Builder, RAD Studio or BDS get $300 off Delphi XE, C++Builder XE and RAD Studio XE Enterprise Upgrades. Delphi or C++Builder upgrades were $1,299 now $999. RAD Studio was $1,799 now $1,499. http://www.embarcadero.com/radoffer -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Problem with smtp file attach send
hi Arno
the smtp server is my provider smtp relay ( TELECOM ITALIA) .it's public
smtp relay for each adsl connection .
Im my Outlook client i use it from the same dvelop PC and all mail with all
attachment < 10 Mb will sent OK.
so , i have put your code in your demo app ..and the result it's the same.
only if i check progress the email will sent ( with large attach)..
it's like during the submission process, if I insert a small delay the
server can process the email ( with large attach)..
in fact if i put a small loop in SmtpClientMessageDataSent procedure or
sleep(150) in myapp and it send email correctly .with all attach.
but i can't do it in my real app , because there are a lot simultaneous
thread ..e it impact a lot in GUI and in run time process.
>
> but i have put a sleep(150) on SmtpClientDisplay procedure and all
> work well ..so this is fix..
I do not think so, there must be some reason for error #10004.
> it's how the send streaming it's so fast like the server want..
That's impossible you cannot send data too fast.
>
> without "fix" i receive this error ( but i can't use it because
> freeze my APP)
What's freezing the GUI is displaying the log messages in the Memo
that causes 100% CPU and slows down transfers dramatically.
>
> All message sent OK at the last after send file i have this error
>
>>
>> --= Multipart Boundary 0305110951--
>> .
> RequestDone Rq=5 Error=10004
> RequestDone Rq=6 Error=0
Then the server probably drops the connection without a hint why.
If it were a good mail server the log would show something like:
{code}
>
> --= Multipart Boundary 0305111618--
.
> .
552 Message size exceeds fixed maximium message size
< 552 Message size exceeds fixed maximium message size
RequestDone Rq=5 Error: 552 Message size exceeds fixed maximium message size
Error, stoped All-In-One demo
RequestDone Rq=6 No Error
{code}
> i have see the calcmsg like you say but there isn't any example ...
> and i don't know how use it in my procedure
There are multiple ways, this is just a simple example:
{code}
case RqType of
smtpConnect:
begin
SmtpClient.MessageSize := 0;
SmtpClient.Ehlo;
end;
smtpEhlo:
begin
if SmtpClient.Username <> '' then
begin
SmtpClient.AuthType := smtpAuthAutoSelect;
SmtpClient.Auth;
end
else if SmtpClient.SizeSupported then
SmtpClient.CalcMsgSize
else
SmtpClient.MailFrom;
end;
smtpCalcMsgSize:
SmtpClient.MailFromSIZE;
smtpAuth:
begin
if SmtpClient.SizeSupported then
SmtpClient.CalcMsgSize
else
SmtpClient.MailFrom;
end;
smtpMailFrom:
SmtpClient.RcptTo;
smtpRcptTo:
SmtpClient.Data;
smtpData:
SmtpClient.Quit;
smtpQuit:
Display('All-In-One done !');
end;
{code}
Note that like most samples it lacks proper error checks.
> i have see in your demo and you use it in PrepareProgressBar.
>
> so i have insert a procedure like PrepaeProgressBar and now it
> work...with all file...(i hope ..next day i test it with more file on
> large develop case)
That doesn't make sense IMO.
> conclusion
> i have reproduce it in your demo app...if i send a very small file
> all work without check progress bar
> with large file i need to check progress bar ...(same in my app)..
> i have see that this problem can repruduce with very larg body
> text..(html not testing but i think is the same)
It is very likely that the server accepts only a certain amount of
data from the same IP within some time frame. Try my example above
with the demo and see what happens.
>
> i think that calcmsg(sync) could be a default setting in your
> componet..when file attachment or body text is to large..
I don't think so, you are free to use it in designs like the
All-In-One demo.
--
Arno Garrels
--
___
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
End of TWSocket Digest, Vol 413, Issue 11
*
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
[twsocket] Bug in THttpCli or where?
Hello,
I have reported this before and I know it is rather strange but it persists.
When I make consecutive POSTs to a HTTPS server (both ICS code), it
sometimes returns 10053. It happens on Win7 and Win2008, with the latter
more often. Anybody sending multiple consecutive POSTs from/to ICS can see
it I believe. Our code is below:
class WebClient
{
private:
TSslContext *sslContext;
AdminClient *adminClient;
bool aborted;
bool reallyDone;
TMemoryStream *sendStream;
public:
TMemoryStream *receivedStream;
TSslHttpCli *HTTPClient;
void *HandleToPost;
__fastcall WebClient(AdminClient *adminClient)
{
this->adminClient = adminClient;
aborted = false;
sendStream = new TMemoryStream();
receivedStream = new TMemoryStream();
HTTPClient = new TSslHttpCli(NULL);
sslContext = new TSslContext(NULL);
HTTPClient->Agent = "Fastream IQWF/IQProxy";
HTTPClient->Connection = "close";
HTTPClient->Timeout = 60;
HTTPClient->SslContext = sslContext;
HTTPClient->SslContext->SslVerifyPeer = false;
HTTPClient->SslContext->SslVerifyPeerModes << SslVerifyMode_PEER;
HTTPClient->SslContext->SslOptions << sslOpt_MICROSOFT_SESS_ID_BUG <<
sslOpt_NETSCAPE_CHALLENGE_BUG << sslOpt_NETSCAPE_REUSE_CIPHER_CHANGE_BUG <<
sslOpt_SSLREF2_REUSE_CERT_TYPE_BUG << sslOpt_MICROSOFT_BIG_SSLV3_BUFFER <<
sslOpt_SSLEAY_080_CLIENT_DH_BUG << sslOpt_TLS_D5_BUG <<
sslOpt_TLS_BLOCK_PADDING_BUG, sslOpt_TLS_ROLLBACK_BUG << sslOpt_NO_SSLv2 <<
sslOpt_NETSCAPE_CA_DN_BUG << sslOpt_NETSCAPE_DEMO_CIPHER_CHANGE_BUG;
HTTPClient->SslContext->InitContext();
HTTPClient->SendStream = sendStream;
HTTPClient->RcvdStream = receivedStream;
HTTPClient->OnRequestDone = RequestDone;
}
__fastcall ~WebClient()
{
HTTPClient->OnRequestDone = NULL;
HTTPClient->Abort();
delete HTTPClient;
sslContext->DeInitContext();
delete sslContext;
delete sendStream;
delete receivedStream;
}
void __fastcall SendPOST(const String &URL, TMemoryStream *sendStream)
{
receivedStream->Clear();
HTTPClient->URL = URL;
HTTPClient->SendStream = sendStream;
aborted = false;
reallyDone = false;
try
{
HTTPClient->PostASync(); // sync
}
catch(Exception &e)
{
}
while(!reallyDone) // for SSL to work well!
Forms::Application->ProcessMessages();
}
void __fastcall RequestDone(TObject *Sender, THttpRequest RqType, WORD
ErrCode)
{
delete HTTPClient->SendStream;
HTTPClient->SendStream = NULL;
PostMessage(HandleToPost, WM_ADMIN_HTTP_POST_DONE, ErrCode, 0);
}
void __fastcall finalizeRequest(WORD ErrCode)
{
if(ErrCode)
adminClient->HTTPClientHostUnreachable(ErrCode);
else if(aborted)
adminClient->HTTPClientWaitTimeoutExpired();
else if(HTTPClient->StatusCode != 200)
adminClient->HTTPClientOnHTTPError();
else
adminClient->HTTPClientDone(receivedStream);
reallyDone = true;
}
};
// and the server
adminHTTPSSLServer = new TSslHttpServer(NULL);
adminHTTPSSLServer->OnGetDocument = HTTPServerGetDocument;
adminHTTPSSLServer->OnHeadDocument = HTTPServerHeadDocument;
adminHTTPSSLServer->OnPostDocument = HTTPServerPostDocument;
adminHTTPSSLServer->OnPostedData = HTTPServerPostedData;
adminHTTPSSLServer->OnClientConnect = HTTPServerClientConnected;
adminHTTPSSLServer->OnClientDisconnect = HTTPServerClientDisconnected;
adminHTTPSSLServer->OnHttpRequestDone = HTTPServerClientRequestDone;
adminHTTPSSLServer->MaxClients = 50;
adminHTTPSSLServer->ListenBacklog = 200;
adminHTTPSSLContext = new TSslContext(NULL);
adminHTTPSSLContext->SslVerifyPeer = false;
adminHTTPSSLContext->SslOptions << sslOpt_MICROSOFT_SESS_ID_BUG <<
sslOpt_NETSCAPE_CHALLENGE_BUG << sslOpt_NETSCAPE_REUSE_CIPHER_CHANGE_BUG <<
sslOpt_SSLREF2_REUSE_CERT_TYPE_BUG << sslOpt_MICROSOFT_BIG_SSLV3_BUFFER <<
sslOpt_SSLEAY_080_CLIENT_DH_BUG << sslOpt_TLS_D5_BUG <<
sslOpt_TLS_BLOCK_PADDING_BUG, sslOpt_TLS_ROLLBACK_BUG << sslOpt_NO_SSLv2 <<
sslOpt_NETSCAPE_CA_DN_BUG << sslOpt_NETSCAPE_DEMO_CIPHER_CHANGE_BUG;
adminHTTPSSLContext->SslVerifyPeerModes << SslVerifyMode_PEER;
adminHTTPSSLContext->SslSessionCacheModes = TSslSessCacheModes() <<
sslSESS_CACHE_NO_INTERNAL_LOOKUP << sslSESS_CACHE_NO_INTERNAL_STORE;
adminHTTPSSLContext->SslCipherList = "ALL:!ADH:!DES:RC4+RSA:@STRENGTH";
adminHTTPSSLContext->SslVersionMethod = sslV23_SERVER;
adminHTTPSSLContext->SslSessionTimeout = 300;
adminHTTPSSLContext->SslSessionCacheSize = 20480;
adminHTTPSSLContext->SslDefaultSessionIDContext = "IQP_ADMIN_SERVER";
...
void __fastcall AdminServer::HTTPServerGetDocument(TObject *Sender, TObject
*Client, Overbyteicshttpsrv::THttpGetFlag &Flags)
{
Flags = hg403;
}
//---
void __fastcall AdminServer::HTTPServerHeadDocument(TObject *Sender, TObject
*Client, Overbyteicshttp
