Thanks Julius. I need to handle slow servers too. Actually I tried using the methodParams.setSoTimeout(5000)//5 seconds with 3.0beta1 and the servlet sleeps for 30 seconds before writing some response. However no exception was thrown. Any ideas??
-----Original Message----- From: Julius Davies [mailto:[EMAIL PROTECTED] Sent: Wednesday, February 02, 2005 5:22 PM To: HttpClient User Discussion Subject: RE: HttpClient under load Hi, Prasad, You can use this to catch sockets that stop writing back to your stream: ========================== HttpClient client = new HttpClient(); client.setTimeout( 60000 ); // socket timeout of 60 seconds ========================== I don't think it will catch sockets that write back slowly (e.g. 1 character per second). For that you probably need to implement your own timeout. yours, Julius On Wed, 2005-02-02 at 17:05 -0700, Prasad Sethumadhavan wrote: > Hi Mike, > Thanks for your input. I am now using a single HttpClient instance and > a multi-threaded connection manager that is shared by all threads. It > is not failing any more. > > I am trying to implement a "fetch timeout" feature using the http > client. Basically it means that if I have not received a complete > response within "n" seconds then I need to return a fetch timeout. > Is there any built in mechanism in HttpClient for this or do I need to > have a monitor thread that calls HttpMethod.abort() after the timeout > period (if the method has not finished execution). What would be the > result of calling the abort method on the worker thread? (what > exceptions etc..). Please advise if there are any better ways to > implement the same. > > Thanks in advance > Prasad > > -----Original Message----- > From: Michael Becke [mailto:[EMAIL PROTECTED] > Sent: Wednesday, February 02, 2005 6:39 AM > To: HttpClient User Discussion > Subject: Re: HttpClient under load > > Hi Prasad, > > Given the scenario you describe below you are more likely leaking one > connection/socket per request. You definitely want to reuse instances > of HttpClient and especially instances of a ConnectionManager. You > can either use the multi-threaded connection manager from all threads > or have a dedicated SimpleHttpConnectionManager per thread. Please > have a look at the performance guide for some more ideas > <http://jakarta.apache.org/commons/httpclient/3.0/performance.html>. > > Also please note that calling manager.closeIdleConnections(1000L) only > closes connections that have already been idle for 1000ms. It does > not continuously poll for idle connections, closing them when they've > been idle for 1000ms. > > Mike > > On Feb 1, 2005, at 11:14 PM, Prasad Sethumadhavan wrote: > > > Hi, > > I need some help or pointers regarding using the HttpClient under > > heavy load. I am currently using the HttpClient under moderate load > > (50 concurrent threads). Each thread instatiates a new HttpClient > > client, uses it and does the method.releaseConnection() before > exiting. > > > > SimpleHttpConnectionManager manager = new > > SimpleHttpConnectionManager(); > > manager.getParams().setStaleCheckingEnabled(true); > > manager.closeIdleConnections(1000L); > > manager.getParams().setConnectionTimeout( > > (int) DEFAULT_CONNECTION_TIMEOUT); > > > > httpclient = new HttpClient(manager); > > httpclient.getParams().setCookiePolicy( > > CookiePolicy.BROWSER_COMPATIBILITY); > > > > However when I run this setup under load, I keep getting socket > > exceptions on the client side. I am using tomcat web server and I > > have > > > increased the max-processor & access-count. I don't see any errors > > or exception in the tomcat log. > > > > However I keep getting a lot of these client socket bind exceptions > > after sometime. I guess it is happening due to the unavailability of > > ports. I would like to know some best practices in using the > > HttpClient under load. I am sure tha it is my bad, but wanted to get > > some input from the user/dev community. Should I have a pool of > > HttpClient objects and reuse them? Should I use the multi-threaded > > connection manager? I need to run this under heavier load and need > > to fix this soon. Any help, code snippets, or guidance would be > > greatly appreciated. Please let me know if you need any other information. > > > > Thanks in advance > > Prasad > > > > > > Client StackTrace: > > > > [ERROR] 01 Feb 2005 20:43:56,177 NDC[] T[Thread-20] HTTP HTTP > > I/O transport exception occurred > > java.net.BindException: Address already in use: connect > > at java.net.PlainSocketImpl.socketConnect(Native Method) > > at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333) > > at > > java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195) > > at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182) > > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:364) > > at java.net.Socket.connect(Socket.java:507) > > at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) > > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcce > > ss > > o > > r > > Impl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:585) > > at > > org.apache.commons.httpclient.protocol.ReflectionSocketFactory.creat > > eS > > o > > c > > ket(ReflectionSocketFactory.java:128) > > at > > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory. > > cr > > e > > a > > teSocket(DefaultProtocolSocketFactory.java:124) > > at > > org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java: > > > 70 > > 5) > > at > > org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(Ht > > tp > > M > > e > > thodDirector.java:359) > > at > > org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpM > > et > > h > > o > > dDirector.java:168) > > at > > > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java: > > 3 > > 93) > > at > > > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java: > > 3 > > 24) > > > > > > > > > > -------------------------------------------------------------------- > > - To unsubscribe, e-mail: > > [EMAIL PROTECTED] > > For additional commands, e-mail: > > [EMAIL PROTECTED] > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: > [EMAIL PROTECTED] > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: > [EMAIL PROTECTED] > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
