RE: read timeout with persistent connection

[Oleg Kalnichevski]

 The only solution I can see is that perhaps HTTPClient could follow IE's
 approach and close  remove free connections from the connection manager
 pool after a configurable period of inactivity? 
 

Matthew,
This sounds like a reasonable addition to HttpClient connection
management capabilities. Michael Becke  Sam Berlin have been recently
discussing such a feature and possible options of implementing it.
There's a good chance that it will eventually be added, but it may take
a while

Oleg


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

read timeout with persistent connection

[Matthew McGowan]

Greetings,

With HttpClient, I'm using the MultiThreadedHttpConnectionManager, with
all timeout values set to 30s, and I'm seeing a problem against a server
that discards unused connections after several minutes. It seems the
connection close isn't detected by the HttpConnection.isStale() method,
which causes a delay in processing as stream reading blocks waiting for
the socket timeout.
In HttpConnection.isStale, the stream read throws a
SocketTimeoutException which indicates the socket is still alive, so the
connection is reused in HttpMethodBase.processRequest without closing
the socket first.

The request is successfully written via writeRequest, but readResponse
throws a java.net.SocketTimeoutException, after delaying for 30s. 

private void processRequest(HttpState state, HttpConnection
connection)
throws HttpException, IOException {
LOG.trace(enter HttpMethodBase.processRequest(HttpState,
HttpConnection));

...

if (!connection.isOpen()) {
// returns true
LOG.debug(Opening the connection.);
connection.open();
}
writeRequest(state, connection);
requestSent = true;
readResponse(state, connection);
// the response read times out
// the method has successfully executed
used = true; 
break;

There is a pattern - this happens only after connections have not been
used for a certain period of time, where I presume the origin server is
closing the unused persistent connection. If connections are reused
frequently, then there are no problems - the timeout occurs only when
connections lay idle for a few minutes. 

The externally visible behavior to users is that the request takes at
least 30s to execute - the request is retried, so it eventually
succeeds. 

Any ideas how I go about fixing this? I'm not entirely sure if this is a
problem with HTTPClient or with the origin server (which is running MS
IIS 4.0, not under my control.) 

I've included below a simple test case to demonstrate the problem. 

Thanks for any help!
mat mcgowan


import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.*;

import java.io.IOException;

import junit.framework.TestCase;

public class TestJakartaHttpClient   extends TestCase
{
public void testNorgeNo() throws InterruptedException,
IOException
{
httpServer(http://norge.no;, 5*1000*60);
}

public void httpServer(String uri, int delay) throws
IOException, InterruptedException
{
HttpClient client = new HttpClient(new
MultiThreadedHttpConnectionManager());
int timeout = 30*1000;
client.setConnectionTimeout(timeout);
client.setHttpConnectionFactoryTimeout(timeout) ;
client.setTimeout(timeout);
GetMethod m = new GetMethod(uri);
m.setStrictMode(false);
m.setFollowRedirects(false);
m.setDoAuthentication(false);
m.setHttp11(true);
// read data
client.executeMethod(m);
m.getResponseBody();
Thread.sleep(delay);

try
{
m = new GetMethod(uri);
m.setStrictMode(false);
m.setFollowRedirects(false);
m.setDoAuthentication(false);
m.setHttp11(true);
client.executeMethod(m);
m.getResponseBody();
}
catch (HttpRecoverableException ex)
{
fail(ex.toString());
}
}
}














-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: read timeout with persistent connection

[Ortwin Glück]

There is an option to disable stale checking.

Matthew McGowan wrote:
Any ideas how I go about fixing this? 


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

RE: read timeout with persistent connection

[Matthew McGowan]

Ortwin Glück write:
 There is an option to disable stale checking.

Thanks, though turning stale checking off doesn't change the outcome -
the readResponse call still blocks for 30s. I don't think it's the stale
checking that's causing the problem, but for some reason the server
ignores or closes the connection after several minutes of inactivity.

Incidentally, I've tried to emulate this using IE 6, and I couldn't
reproduce the problem. I believe this is because IE closes persistent
connections itself after 60 seconds of inactivity. (See
http://www.cs.wisc.edu/~cao/papers/persistent-connection.html. The paper
discusses IE 4.01 - though using a network monitor I can confirm this is
still true for IE 6.) 

But, Netscape 7.1 does have the problem. If I load a page from the
server, wait a few minutes, and then try a reload, the browser hangs
until the socket times out, just as HttpClient does. The network monitor
shows the connection closed after a few minutes. As the socket is
clearly closed, I have no idea why both netscape and HttpClient can't
detect that the socket is closed. 

I hacked HttpConnection to include a check to call the methods
Socket.isClosed/isConnected/isInputShutdown/isOutputShutdown to see if
these help determine if the socket is close - all of these indicate that
the socket is still alive, yet my network monitor indicates that the
connection is closed. 

The only solution I can see is that perhaps HTTPClient could follow IE's
approach and close  remove free connections from the connection manager
pool after a configurable period of inactivity? 

Cheers,
mat.



-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]