On Fri, 2008-06-27 at 11:25 -0700, Ken DeLong wrote: > All of our connections are to the exact same host - in fact, the sending and > receiving processes are on the same actual server. The same dns name is used > for each request (we use the dns name of the machine, not the string > "localhost"). >
Ken, Try disabling the stale connection check. It can happen, especially with TLS/SSL, that perfectly good connections are reported as stale. You can get a better insight as to why connections get dropped by turning on the context logging. For details see http://hc.apache.org/httpclient-3.x/logging.html Oleg > We'll try the multithreaded connection manager next. (I was pretty sure the > calling app was single threaded but I'm not 100% sure). > > Thanks! > Ken > > > -----Original Message----- > From: Oleg Kalnichevski [mailto:[EMAIL PROTECTED] > Sent: Friday, June 27, 2008 11:20 AM > To: HttpClient User Discussion > Subject: RE: Too Many Connections > > On Fri, 2008-06-27 at 09:08 -0700, Ken DeLong wrote: > > That didn't help. We changed to > > > > private HttpClient client = new HttpClient(new > > SimpleHttpConnectionManager()); > > > > Ken, > > SimpleHttpConnectionManager re-uses the underlying connection object but it > cannot reuse the network socket when connecting to a different host. Ii needs > to close the old socket and open a new one every time it connects to a > different host. If you would like to keep alive connections to multiple > hosts, consider using MultiThreadedHttpConnectionManager > > Regarding connections in TIME_WAIT state please see > > http://wiki.apache.org/HttpComponents/FrequentlyAskedConnectionManagementQuestions > > Oleg > > > > But we still see the same behavior. > > > > Turning on the debug logs shows messages like > > > > $ fgrep 'Connection is locked. Call to releaseConnection() ignored.' > > server.log | wc -l > > 176 > > > > $ fgrep 'Releasing connection back to connection manager.' server.log > > | wc -l > > 264 > > > > > > What would cause the connection to be locked when releaseConnection() > > is called? > > > > > > > > -----Original Message----- > > From: Pete Keyes [mailto:[EMAIL PROTECTED] > > Sent: Thursday, June 26, 2008 3:27 PM > > To: HttpClient User Discussion > > Cc: Stephen Hiley > > Subject: RE: Too Many Connections > > > > You need to use either of these: > > - org.apache.commons.httpclient.MultiThreadedHttpConnectionManager > > - org.apache.commons.httpclient.SimpleHttpConnectionManager > > There are examples of both in the HttpClient site. > > > > In addition to that you may need to tune the O/S to formally release > > the CLOSE_WAIT files sooner. There is a delay between when an > > application closes a socket and when the O/S can formally release that > > socket to avoid orphan packets. > > > > See the documentation for your O/S on setting that value. > > > > > > ...Pete > > Starbucks Coffee Co. > > 2601 Utah Ave S. > > Seattle, WA. 98134 > > (w) 206-318-5933 > > (c) 206-914-4134 > > > > -----Original Message----- > > From: Ken DeLong [mailto:[EMAIL PROTECTED] > > Sent: Thursday, June 26, 2008 3:17 PM > > To: httpclient-users@hc.apache.org > > Cc: Stephen Hiley > > Subject: Too Many Connections > > > > We have a method in our app that calls httpclient like this: > > > > private HttpClient client = new HttpClient(); > > > > public String send(String urlString, String xml) > > { > > PostMethod postMethod = new PostMethod(urlString); > > > > // Put the XML in the request > > try > > { > > RequestEntity entity = new > > StringRequestEntity(xml, "text/xml", "UTF-8"); > > postMethod.setRequestEntity(entity); > > } > > catch(UnsupportedEncodingException e1) > > { > > logger.warn("Couldn't encode the xml?", e1); > > return "" + > > IContentPublishingService.ACTION_ERROR; > > } > > > > // Send the data and get the response > > String response = null; > > try > > { > > client.executeMethod(postMethod); > > byte[] responseBody = > > postMethod.getResponseBody(); > > response = new String(responseBody); > > } > > catch(HttpException e) > > { > > logger.warn("Fatal protocol violation: " + > > e.getMessage()); > > return "" + > > IContentPublishingService.ACTION_ERROR; > > } > > catch(IOException e) > > { > > logger.warn("Fatal transport error: " + e.getMessage()); > > return "" + > > IContentPublishingService.ACTION_ERROR; > > } > > finally > > { > > // Release the connection. > > postMethod.releaseConnection(); > > } > > > > return response; > > } > > > > > > This method is called rapidly by a single thread in our application. > > After 1500 calls or so, we start to see java.net.SocketException, too > > many open files. Netstat reveals that there are many sockets in > > CLOSE_WAIT and TIME_WAIT states. It seems that httpclient is not > > reusing the connection managed by the HttpClient instance, and instead > > is closing the connection and creating a new one. This appears to be > > in conflict with all the documentation that we could find. > > > > An interesting wrinkle is that the sending and receiving side of the > > connection are both on the same machine. The URL is constructed using > > the machine's dns name, not "localhost". > > > > ---------------------------------------------------------------------- > > -- > > -- > > Kenneth DeLong > > Software Architect > > Engineering > > BabyCenter, LLC > > > > [EMAIL PROTECTED] > > p: 415.344.7616 > > AIM: kenwdelong > > > > http://www.babycenter.com > > > > > > > > This email message is for the sole use of the intended recipient(s) > > and may contain confidential and privileged information. Any > > unauthorized review, use, disclosure or distribution is prohibited. If > > you are not the intended recipient, please contact the sender by reply > > email and destroy all copies of the original message. If you are the > > intended recipient, please be advised that the content of this message > > is subject to access, review and disclosure by the sender's Email > > System Administrator. > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
