Thanks Oleg, I supposed it could be a problem. I will change crawler
architecture a little, to generate less connections simultaneously. So
OS will have more time to release the sockets. I saw that it is possible
to change the number of client ports in WinXP but I'm not sure if it
would be a proper way to deal with this kind of problem.
Cheers,
Tomek
Oleg Kalnichevski wrote:
On Wed, 2008-09-03 at 16:03 -0400, Tomek Maciejewski wrote:
Hi Oleg,
Thank for your reply. I have got the latest code from repository, but
the problem didn't disappear. I saved the logs here:
http://pentium.hopto.org/~thinred/files/output.zip
At the end of the crawler execution I started to get BindException. I
would appreciate if you could see this logs, because actually I don't
understand what is a relation between not reused connections and this
bind exception. What is the nature of this problem, and why it tries to
bind to 'address already in use'? Is there any way to solve this problem?
Cheers,
Tomek
It is all pretty simple. Please examine response messages in the log.
Most of them (if not all) have "Connection: close" header, which means
the server does not want the connection to be kept alive. As a result
connections do not get reused. Every time a new request is executed, a
new socket has to be open. Every time a new socket is open, a client
port has to be allocated. Windows (per default) allocates the range from
1024 to 5000 for client ports. It usually takes a while before a used
client port gets released back to the pool. If you execute 4000 requests
in a tight loop you are pretty much bound to run out of client ports and
end up with a bunch of BindExceptions
Hope this helps
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
