On Wed, 2008-09-03 at 16:03 -0400, Tomek Maciejewski wrote: > Hi Oleg, > > Thank for your reply. I have got the latest code from repository, but > the problem didn't disappear. I saved the logs here: > http://pentium.hopto.org/~thinred/files/output.zip > At the end of the crawler execution I started to get BindException. I > would appreciate if you could see this logs, because actually I don't > understand what is a relation between not reused connections and this > bind exception. What is the nature of this problem, and why it tries to > bind to 'address already in use'? Is there any way to solve this problem? > > Cheers, > Tomek > >
It is all pretty simple. Please examine response messages in the log. Most of them (if not all) have "Connection: close" header, which means the server does not want the connection to be kept alive. As a result connections do not get reused. Every time a new request is executed, a new socket has to be open. Every time a new socket is open, a client port has to be allocated. Windows (per default) allocates the range from 1024 to 5000 for client ports. It usually takes a while before a used client port gets released back to the pool. If you execute 4000 requests in a tight loop you are pretty much bound to run out of client ports and end up with a bunch of BindExceptions Hope this helps Oleg --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
