Re: Persistent HTTPS connections

2004-05-14 Thread Jesus M. Salvo Jr. 
Hi Oleg,

Thanks for that. I have now implemented my own MethodRetryHandler and 
got rid of my custom loop / retry.
Cleaner log files for my application as well.

John

Kalnichevski, Oleg wrote:

John,

Please correct me if I am wrong (which may well be the case) SO_TIMEOUT only affects socket read operations. I thought it had nothing to do with SSL inactivity timeout. But it looks like it might.

There's another way to deal with recoverable exceptions. You can provide a custom implementation of MethodRetryHandler

http://jakarta.apache.org/commons/httpclient/apidocs/org/apache/commons/httpclient/MethodRetryHandler.html

The default implementation of the MethodRetryHandler is quite conservative. It does not allow the method to be retried if the request has been transmitted in its entirety. 

http://jakarta.apache.org/commons/httpclient/xref/org/apache/commons/httpclient/DefaultMethodRetryHandler.html#62

Oleg

-Original Message-
From: Jesus M. Salvo Jr. [mailto:[EMAIL PROTECTED]
Sent: Thursday, May 13, 2004 7:40
To: [EMAIL PROTECTED]
Subject: Persistent HTTPS connections


Using HttpClient 2.0
JDK 1.4.2_04 on Fedora
Is there anything special that I have to do to make use of persistent
HTTP(S) connections with HttpClient other than using
MultiThreadedHttpConnectionManager ?
Basically, what I am doing is the following ( more explanation after the
snippet of the source code ):
   MultiThreadedHttpConnectionManager connectionManager =
 new MultiThreadedHttpConnectionManager();
   connectionManager.setConnectionStaleCheckingEnabled( true );
   connectionManager.setMaxConnectionsPerHost( 10 );
   connectionManager.setMaxTotalConnections( 100 );
   this.httpClient = new HttpClient( connectionManager );
   this.httpClient.setConnectionTimeout( 3 );
   this.httpClient.setTimeout( 3 );
 and then within a thread, the thread does this:

 String content;
 HttpMethod method;
  
   try {
 method = new PostMethod( connectionURL );
  
 method.setDoAuthentication( true );
 method.setRequestHeader( Content-Type, contentType + ;
charset= + this.outboundEncoding);
 if( method instanceof PostMethod ) {
   PostMethod postMethod = (PostMethod) method;
   postMethod.setRequestBody( content );
 }
  int responseCode = this.httpClient.executeMethod( method );
  String response = method.getResponseBodyAsString();
  .
   }
   catch( Exception ex ) {}
   finally {
  if( method != null ) method.releaseConnection();
   }

What I am doing, then, from a JUnit test class, is:

1) Send one HTTP POST to a URL, which works and I get the response.
2) Sleep for 40 seconds ( which is greater than the SO_TIMEOUT of 30
seconds )
3) Send another HTTP POST to the same URL.
What I am seeing with ethereal is that, after 30 seconds of no activity
( no TCP ACKs whatever on the socket ), the web server sends a a TLS alert.
So what actually happens is this:
1) Send one HTTP POST to a URL, which works and I get the response.
2) Sleep for 40 seconds ( which is greater than the SO_TIMEOUT of 30
seconds )
3) Web server sends a TLS alert after 30 seconds of inactivity. Web
server sends a TCP FIN, Java sends a TCP ACK _only_ .
4) Wake up from sleep at the 40 second mark, send another HTTP POST to
the same URL.
5) Receive a TLS alert instead of a HTTP response. The TLS alert
according to JSSE's debug mode is:
   main, RECV TLSv1 ALERT:  warning, close_notify
   main, called closeInternal(false)
   main, SEND TLSv1 ALERT:  warning, description = close_notify
6) HttpClient throws an HttpRecoverableException, shown below:
   org.apache.commons.httpclient.HttpRecoverableException:
org.apache.commons.httpclient.HttpRecoverableException: Error in parsing
the status  line from the response: unable to find line starting with HTTP
Question is, was I correct in initially assuming that
MultiThreadedHttpConnectionManager should have handled this case ?
e.g... .detected that the exception, and retried the HTTP POST by
creating a new HTTPS socket ?


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
***
The information in this email is confidential and may be legally privileged.  Access 
to this email by anyone other than the intended addressee is unauthorized.  If you are 
not the intended recipient of this message, any review, disclosure, copying, 
distribution, retention, or any action taken or omitted to be taken in reliance on it 
is prohibited and may be unlawful.  If you are not the intended recipient, please 
reply to or forward a copy of this message to the sender and delete the message, any 
attachments, and any copies thereof from your system.
***

Re: Persistent HTTPS connections

2004-05-14 Thread Jesus M. Salvo Jr. 
One small thing though. Why is setMethodRetryHandler() defined in 
HttpMethodBase class instead of the HttpMethod interface ?

Jesus M. Salvo Jr. wrote:

Hi Oleg,

Thanks for that. I have now implemented my own MethodRetryHandler and 
got rid of my custom loop / retry.
Cleaner log files for my application as well.

John

Kalnichevski, Oleg wrote:

John,

Please correct me if I am wrong (which may well be the case) 
SO_TIMEOUT only affects socket read operations. I thought it had 
nothing to do with SSL inactivity timeout. But it looks like it might.

There's another way to deal with recoverable exceptions. You can 
provide a custom implementation of MethodRetryHandler

http://jakarta.apache.org/commons/httpclient/apidocs/org/apache/commons/httpclient/MethodRetryHandler.html 

The default implementation of the MethodRetryHandler is quite 
conservative. It does not allow the method to be retried if the 
request has been transmitted in its entirety.
http://jakarta.apache.org/commons/httpclient/xref/org/apache/commons/httpclient/DefaultMethodRetryHandler.html#62 

Oleg

-Original Message-
From: Jesus M. Salvo Jr. [mailto:[EMAIL PROTECTED]
Sent: Thursday, May 13, 2004 7:40
To: [EMAIL PROTECTED]
Subject: Persistent HTTPS connections


Using HttpClient 2.0
JDK 1.4.2_04 on Fedora
Is there anything special that I have to do to make use of persistent
HTTP(S) connections with HttpClient other than using
MultiThreadedHttpConnectionManager ?
Basically, what I am doing is the following ( more explanation after the
snippet of the source code ):
   MultiThreadedHttpConnectionManager connectionManager =
 new MultiThreadedHttpConnectionManager();
   connectionManager.setConnectionStaleCheckingEnabled( true );
   connectionManager.setMaxConnectionsPerHost( 10 );
   connectionManager.setMaxTotalConnections( 100 );
   this.httpClient = new HttpClient( connectionManager );
   this.httpClient.setConnectionTimeout( 3 );
   this.httpClient.setTimeout( 3 );
 and then within a thread, the thread does this:

 String content;
 HttpMethod method;
  
   try {
 method = new PostMethod( connectionURL );
   method.setDoAuthentication( true );
 method.setRequestHeader( Content-Type, contentType + ;
charset= + this.outboundEncoding);
 if( method instanceof PostMethod ) {
   PostMethod postMethod = (PostMethod) method;
   postMethod.setRequestBody( content );
 }
  int responseCode = this.httpClient.executeMethod( method );
  String response = method.getResponseBodyAsString();
  .
   }
   catch( Exception ex ) {}
   finally {
  if( method != null ) method.releaseConnection();
   }
What I am doing, then, from a JUnit test class, is:

1) Send one HTTP POST to a URL, which works and I get the response.
2) Sleep for 40 seconds ( which is greater than the SO_TIMEOUT of 30
seconds )
3) Send another HTTP POST to the same URL.
What I am seeing with ethereal is that, after 30 seconds of no activity
( no TCP ACKs whatever on the socket ), the web server sends a a TLS 
alert.
So what actually happens is this:

1) Send one HTTP POST to a URL, which works and I get the response.
2) Sleep for 40 seconds ( which is greater than the SO_TIMEOUT of 30
seconds )
3) Web server sends a TLS alert after 30 seconds of inactivity. Web
server sends a TCP FIN, Java sends a TCP ACK _only_ .
4) Wake up from sleep at the 40 second mark, send another HTTP POST to
the same URL.
5) Receive a TLS alert instead of a HTTP response. The TLS alert
according to JSSE's debug mode is:
   main, RECV TLSv1 ALERT:  warning, close_notify
   main, called closeInternal(false)
   main, SEND TLSv1 ALERT:  warning, description = close_notify
6) HttpClient throws an HttpRecoverableException, shown below:
   org.apache.commons.httpclient.HttpRecoverableException:
org.apache.commons.httpclient.HttpRecoverableException: Error in parsing
the status  line from the response: unable to find line starting with 
HTTP

Question is, was I correct in initially assuming that
MultiThreadedHttpConnectionManager should have handled this case ?
e.g... .detected that the exception, and retried the HTTP POST by
creating a new HTTPS socket ?


-
To unsubscribe, e-mail: 
[EMAIL PROTECTED]
For additional commands, e-mail: 
[EMAIL PROTECTED]

*** 

The information in this email is confidential and may be legally 
privileged.  Access to this email by anyone other than the intended 
addressee is unauthorized.  If you are not the intended recipient of 
this message, any review, disclosure, copying, distribution, 
retention, or any action taken or omitted to be taken in reliance on 
it is prohibited and may be unlawful.  If you are not the intended 
recipient, please reply to or forward a copy of this message to the 
sender and delete the message, any

Re: Persistent HTTPS connections

2004-05-14 Thread Oleg Kalnichevski 
John
No reason other than HttpMethod  HttpMethodBase being horrible
monstrosities badly in need of a compete redesign

Oleg


On Fri, 2004-05-14 at 09:02, Jesus M. Salvo Jr. wrote:
 One small thing though. Why is setMethodRetryHandler() defined in 
 HttpMethodBase class instead of the HttpMethod interface ?
 
 
 Jesus M. Salvo Jr. wrote:
 
 
  Hi Oleg,
 
  Thanks for that. I have now implemented my own MethodRetryHandler and 
  got rid of my custom loop / retry.
  Cleaner log files for my application as well.
 
 
  John
 
  Kalnichevski, Oleg wrote:
 
  John,
 
  Please correct me if I am wrong (which may well be the case) 
  SO_TIMEOUT only affects socket read operations. I thought it had 
  nothing to do with SSL inactivity timeout. But it looks like it might.
 
  There's another way to deal with recoverable exceptions. You can 
  provide a custom implementation of MethodRetryHandler
 
  http://jakarta.apache.org/commons/httpclient/apidocs/org/apache/commons/httpclient/MethodRetryHandler.html
 
 
  The default implementation of the MethodRetryHandler is quite 
  conservative. It does not allow the method to be retried if the 
  request has been transmitted in its entirety.
  http://jakarta.apache.org/commons/httpclient/xref/org/apache/commons/httpclient/DefaultMethodRetryHandler.html#62
 
 
  Oleg
 
 
  -Original Message-
  From: Jesus M. Salvo Jr. [mailto:[EMAIL PROTECTED]
  Sent: Thursday, May 13, 2004 7:40
  To: [EMAIL PROTECTED]
  Subject: Persistent HTTPS connections
 
 
 
  Using HttpClient 2.0
  JDK 1.4.2_04 on Fedora
 
  Is there anything special that I have to do to make use of persistent
  HTTP(S) connections with HttpClient other than using
  MultiThreadedHttpConnectionManager ?
 
  Basically, what I am doing is the following ( more explanation after the
  snippet of the source code ):
 
 MultiThreadedHttpConnectionManager connectionManager =
   new MultiThreadedHttpConnectionManager();
 connectionManager.setConnectionStaleCheckingEnabled( true );
 connectionManager.setMaxConnectionsPerHost( 10 );
 connectionManager.setMaxTotalConnections( 100 );
 
 this.httpClient = new HttpClient( connectionManager );
 this.httpClient.setConnectionTimeout( 3 );
 this.httpClient.setTimeout( 3 );
 
   and then within a thread, the thread does this:
 
   String content;
   HttpMethod method;

 try {
   method = new PostMethod( connectionURL );
 method.setDoAuthentication( true );
   method.setRequestHeader( Content-Type, contentType + ;
  charset= + this.outboundEncoding);
   if( method instanceof PostMethod ) {
 PostMethod postMethod = (PostMethod) method;
 postMethod.setRequestBody( content );
   }
int responseCode = this.httpClient.executeMethod( method );
String response = method.getResponseBodyAsString();
.
 }
 catch( Exception ex ) {}
 finally {
if( method != null ) method.releaseConnection();
 }
 
 
  What I am doing, then, from a JUnit test class, is:
 
  1) Send one HTTP POST to a URL, which works and I get the response.
  2) Sleep for 40 seconds ( which is greater than the SO_TIMEOUT of 30
  seconds )
  3) Send another HTTP POST to the same URL.
 
  What I am seeing with ethereal is that, after 30 seconds of no activity
  ( no TCP ACKs whatever on the socket ), the web server sends a a TLS 
  alert.
  So what actually happens is this:
 
 
  1) Send one HTTP POST to a URL, which works and I get the response.
  2) Sleep for 40 seconds ( which is greater than the SO_TIMEOUT of 30
  seconds )
  3) Web server sends a TLS alert after 30 seconds of inactivity. Web
  server sends a TCP FIN, Java sends a TCP ACK _only_ .
  4) Wake up from sleep at the 40 second mark, send another HTTP POST to
  the same URL.
  5) Receive a TLS alert instead of a HTTP response. The TLS alert
  according to JSSE's debug mode is:
 main, RECV TLSv1 ALERT:  warning, close_notify
 main, called closeInternal(false)
 main, SEND TLSv1 ALERT:  warning, description = close_notify
  6) HttpClient throws an HttpRecoverableException, shown below:
 
 org.apache.commons.httpclient.HttpRecoverableException:
  org.apache.commons.httpclient.HttpRecoverableException: Error in parsing
  the status  line from the response: unable to find line starting with 
  HTTP
 
  Question is, was I correct in initially assuming that
  MultiThreadedHttpConnectionManager should have handled this case ?
  e.g... .detected that the exception, and retried the HTTP POST by
  creating a new HTTPS socket ?
 
 
 
 
  -
  To unsubscribe, e-mail: 
  [EMAIL PROTECTED]
  For additional commands, e-mail: 
  [EMAIL PROTECTED]
 
 
  ***
   
 
  The information in this email is confidential and may be legally

Re: Persistent HTTPS connections

2004-05-13 Thread Jesus M. Salvo Jr. 
Some clarifications below.

Jesus M. Salvo Jr. wrote:

What I am seeing with ethereal is that, after 30 seconds of no 
activity ( no TCP ACKs whatever on the socket ), the web server sends 
a a TLS alert.
So what actually happens is this:

1) Send one HTTP POST to a URL, which works and I get the response.
2) Sleep for 40 seconds ( which is greater than the SO_TIMEOUT of 30 
seconds )
3) Web server sends a TLS alert after 30 seconds of inactivity. Web 
server sends a TCP FIN, Java sends a TCP ACK _only_ . 
At this point, local socket is in CLOSE_WAIT state.

4) Wake up from sleep at the 40 second mark, send another HTTP POST to 
the same URL.
5) Receive a TLS alert instead of a HTTP response. The TLS alert 
according to JSSE's debug mode is:
   main, RECV TLSv1 ALERT:  warning, close_notify
   main, called closeInternal(false)
   main, SEND TLSv1 ALERT:  warning, description = close_notify 
Item (5) is actually not the webserver sending a TLS alert, but JSSE 
sending the TLS alert

6) HttpClient throws an HttpRecoverableException, shown below:

   org.apache.commons.httpclient.HttpRecoverableException: 
org.apache.commons.httpclient.HttpRecoverableException: Error in 
parsing the status  line from the response: unable to find line 
starting with HTTP

Question is, was I correct in initially assuming that 
MultiThreadedHttpConnectionManager should have handled this case ? 
e.g... .detected that the exception, and retried the HTTP POST by 
creating a new HTTPS socket ?



-
To unsubscribe, e-mail: 
[EMAIL PROTECTED]
For additional commands, e-mail: 
[EMAIL PROTECTED]





-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Persistent HTTPS connections

2004-05-13 Thread Jesus M. Salvo Jr. 
Jesus M. Salvo Jr. wrote:

Question is, was I correct in initially assuming that 
MultiThreadedHttpConnectionManager should have handled this case ? 
e.g... .detected that the exception, and retried the HTTP POST by 
creating a new HTTPS socket ?


What I have done now is, if I get a HttpRecoverableException, I retry 
the HTTP call.
The executeMethod() is therefore in a while loop. I am now to put a hard 
limit on the number of retries.
Just like to ask what have everyone else doing.
Is this the best pratice with HttpClient to handle persistent 
connections ?



Regards,

John

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

RE: Persistent HTTPS connections

2004-05-13 Thread Kalnichevski, Oleg 

John,

Please correct me if I am wrong (which may well be the case) SO_TIMEOUT only affects 
socket read operations. I thought it had nothing to do with SSL inactivity timeout. 
But it looks like it might.

There's another way to deal with recoverable exceptions. You can provide a custom 
implementation of MethodRetryHandler

http://jakarta.apache.org/commons/httpclient/apidocs/org/apache/commons/httpclient/MethodRetryHandler.html

The default implementation of the MethodRetryHandler is quite conservative. It does 
not allow the method to be retried if the request has been transmitted in its 
entirety. 

http://jakarta.apache.org/commons/httpclient/xref/org/apache/commons/httpclient/DefaultMethodRetryHandler.html#62

Oleg


-Original Message-
From: Jesus M. Salvo Jr. [mailto:[EMAIL PROTECTED]
Sent: Thursday, May 13, 2004 7:40
To: [EMAIL PROTECTED]
Subject: Persistent HTTPS connections



Using HttpClient 2.0
JDK 1.4.2_04 on Fedora

Is there anything special that I have to do to make use of persistent
HTTP(S) connections with HttpClient other than using
MultiThreadedHttpConnectionManager ?

Basically, what I am doing is the following ( more explanation after the
snippet of the source code ):

MultiThreadedHttpConnectionManager connectionManager =
  new MultiThreadedHttpConnectionManager();
connectionManager.setConnectionStaleCheckingEnabled( true );
connectionManager.setMaxConnectionsPerHost( 10 );
connectionManager.setMaxTotalConnections( 100 );

this.httpClient = new HttpClient( connectionManager );
this.httpClient.setConnectionTimeout( 3 );
this.httpClient.setTimeout( 3 );

 and then within a thread, the thread does this:

  String content;
  HttpMethod method;
   
try {
  method = new PostMethod( connectionURL );
   
  method.setDoAuthentication( true );
  method.setRequestHeader( Content-Type, contentType + ;
charset= + this.outboundEncoding);
  if( method instanceof PostMethod ) {
PostMethod postMethod = (PostMethod) method;
postMethod.setRequestBody( content );
  }
   int responseCode = this.httpClient.executeMethod( method );
   String response = method.getResponseBodyAsString();
   .
}
catch( Exception ex ) {}
finally {
   if( method != null ) method.releaseConnection();
}


What I am doing, then, from a JUnit test class, is:

1) Send one HTTP POST to a URL, which works and I get the response.
2) Sleep for 40 seconds ( which is greater than the SO_TIMEOUT of 30
seconds )
3) Send another HTTP POST to the same URL.

What I am seeing with ethereal is that, after 30 seconds of no activity
( no TCP ACKs whatever on the socket ), the web server sends a a TLS alert.
So what actually happens is this:


1) Send one HTTP POST to a URL, which works and I get the response.
2) Sleep for 40 seconds ( which is greater than the SO_TIMEOUT of 30
seconds )
3) Web server sends a TLS alert after 30 seconds of inactivity. Web
server sends a TCP FIN, Java sends a TCP ACK _only_ .
4) Wake up from sleep at the 40 second mark, send another HTTP POST to
the same URL.
5) Receive a TLS alert instead of a HTTP response. The TLS alert
according to JSSE's debug mode is:
main, RECV TLSv1 ALERT:  warning, close_notify
main, called closeInternal(false)
main, SEND TLSv1 ALERT:  warning, description = close_notify
6) HttpClient throws an HttpRecoverableException, shown below:

org.apache.commons.httpclient.HttpRecoverableException:
org.apache.commons.httpclient.HttpRecoverableException: Error in parsing
the status  line from the response: unable to find line starting with HTTP

Question is, was I correct in initially assuming that
MultiThreadedHttpConnectionManager should have handled this case ?
e.g... .detected that the exception, and retried the HTTP POST by
creating a new HTTPS socket ?




-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


***
The information in this email is confidential and may be legally privileged.  Access 
to this email by anyone other than the intended addressee is unauthorized.  If you are 
not the intended recipient of this message, any review, disclosure, copying, 
distribution, retention, or any action taken or omitted to be taken in reliance on it 
is prohibited and may be unlawful.  If you are not the intended recipient, please 
reply to or forward a copy of this message to the sender and delete the message, any 
attachments, and any copies thereof from your system.
***

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL