Re: Help/Advise debugging SocketException in BHttpConnectionBase.fillInputBuffer()

2016-06-22 Thread Mark Rekveld
Hi Oleg,

Thanks for your quick reply.
I'll try upgrading to see if that resolves this issue,

On Wed, Jun 22, 2016 at 5:23 PM Oleg Kalnichevski  wrote:

> On Wed, 2016-06-22 at 13:00 +0200, Mark Rekveld wrote:
> > Hi,
> >
> > I would like to ask for your assistance to resolve an issue that we are
> > facing that has the nature of being a race condition.
> >
> > The problem is that a SocketException is thrown in the finally clause of
> > BHttpConnectionBase.fillInputBuffer() [
> >
> https://github.com/apache/httpcore/blob/4.3.x/httpcore/src/main/java/org/apache/http/impl/BHttpConnectionBase.java#L326
> > ]
> >
> > We use 4.3.6 of the HttpClient with 4.3.3 of HttpCore, and have a custom
> > extension of org.apache.http.protocol.HttpRequestExecutor that sends the
> > request entity asynchronously and immediately try to read the response
> from
> > the sever.
> >
>
> We can only adequately support two branches of HttpClient: stable
> (currently 4.5.x) and dev (currently 5.0-alpha)
>
> Please consider upgrading to HttpClient 4.5. There have been numerous
> changes since 4.3.6.
>
> Oleg
>
> > The exception occurs within this custom HttpRequestExecutor when the
> > conn.isResponseAvailable(1000 * 5) call is made in this code snippet. See
> > stack trace below
> >
> > private void sendRequestEntity(final HttpEntityEnclosingRequest req)
> throws
> > IOException {
> >   Thread entitySender = new Thread(getClass().getSimpleName() + "-" +
> > System.currentTimeMillis())
> >   {
> > @Override
> > public void run() {
> >   try {
> > conn.sendRequestEntity(req);
> > conn.flush();
> > context.setAttribute(HttpCoreContext.HTTP_REQ_SENT,
> Boolean.TRUE);
> >   }
> >   catch (Throwable t) {
> > error = t;
> >   }
> > }
> >   };
> >   entitySender.start();
> >
> >   while (entitySender.isAlive() && !conn.isResponseAvailable(1000 * 5)) {
> > /*
> >  * Wait here until either we have fully uploaded the request entity
> or
> > the server sends a response. If
> >  * we were to return immediately and called doReceiveResponse() right
> > away we would provoke socket
> >  * timeouts.
> >  */
> >   }
> > }
> >
> > The full stack trace:
> >
> > 2016-06-03 02:36:04,149+1000 ERROR [qtp761509273-12469 - PUT
> > /rest/ci/scan/treasury-web-deployable] jenkins-server
> > com.sonatype.insight.brain.hds.HdsClient - Socket is closed
> > java.net.SocketException: Socket is closed
> >  at java.net.Socket.setSoTimeout(Socket.java:1137) ~[na:1.8.0_91]
> >  at
> >
> sun.security.ssl.BaseSSLSocketImpl.setSoTimeout(BaseSSLSocketImpl.java:633)
> > ~[na:1.8.0_91]
> >  at sun.security.ssl.SSLSocketImpl.setSoTimeout(SSLSocketImpl.java:2526)
> > ~[na:1.8.0_91]
> >  at
> > org.apache.http.impl.BHttpConnectionBase.fillInputBuffer(SourceFile:326)
> > ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
> >  at org.apache.http.impl.BHttpConnectionBase.awaitInput(SourceFile:334)
> > ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
> >  at
> >
> org.apache.http.impl.DefaultBHttpClientConnection.isResponseAvailable(SourceFile:131)
> > ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
> >  at
> > org.apache.http.impl.conn.CPoolProxy.isResponseAvailable(SourceFile:141)
> > ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
> >  at
> >
> com.sonatype.insight.client.utils.HttpRequestExecutor$HttpExchange.sendRequestEntity(SourceFile:201)
> > ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
> >  at
> >
> com.sonatype.insight.client.utils.HttpRequestExecutor$HttpExchange.doSendRequest(SourceFile:177)
> > ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
> >  at
> >
> com.sonatype.insight.client.utils.HttpRequestExecutor$HttpExchange.perform(SourceFile:108)
> > ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
> >  at
> >
> com.sonatype.insight.client.utils.HttpRequestExecutor.execute(SourceFile:53)
> > ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
> >  at org.apache.http.impl.execchain.MainClientExec.execute(SourceFile:254)
> > ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
> >  at org.apache.http.impl.execchain.ProtocolExec.execute(SourceFile:195)
> > ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
> >  at org.apache.http.impl.execchain.RetryExec.execute(SourceFile:86)
> > ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
> >  at org.apache.http.impl.execchain.RedirectExec.execute(SourceFile:108)
> > ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
> >  at
> > org.apache.http.impl.client.InternalHttpClient.doExecute(SourceFile:184)
> > ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
> >  at
> org.apache.http.impl.client.CloseableHttpClient.execute(SourceFile:82)
> > ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
> >  at
> org.apache.http.impl.client.CloseableHttpClient.execute(SourceFile:106)
> > ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
> >  at
> org.apache.http.impl.client.CloseableHttpClient.execute(SourceFile:57)
> > ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
> >  at com.sonatype.insight.brain.hds.HdsClient.execute(SourceFile:380)
> > [nexus-iq-server-1.20.0-02.jar:1

Re: Telling HC to make new connections all the time

2016-06-22 Thread Oleg Kalnichevski
On Wed, 2016-06-22 at 07:07 -0400, Benson Margulies wrote:
> On Wed, Jun 22, 2016 at 3:05 AM, Stefan Magnus Landrø
>  wrote:
> > Hi Benson,
> >
> > Have you considered implementing a custom
> > https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/conn/DnsResolver.html
> > that handles DNS changes in a better way than the default implementation?
> > You could probably use http://www.xbill.org/dnsjava/ and lookup names
> > directly with the amazon NS?
> 
> Thanks for reminding me to ask about that. Does HC call the resolver
> as part of deciding whether to keep a connection or not? 

No, it does not, but one could make it do so by using a custom
ConnectionReuseStrategy.

Oleg

> The problem
> as we see it is this: the DNS mapping for an ELB changes, and a minute
> later the connections to it become useless but don't necessarily get
> properly closed with a FIN by the ELB. I've written some goofy code
> that notices when the address resolution changes for any of the ELB's
> we care about, and when it does it calls 'closeIdleConnections' with a
> time of 0. On the other hand, I don't think we're currently having any
> problems getting InetAddress.getByName to keep up with Amazon, but
> your suggestion would be safer.
> 
> 
> >
> > Stefan
> >
> > 2016-06-20 19:57 GMT+02:00 Benson Margulies :
> >
> >> We use the async http components library to connect various components
> >> in a system that is deployed, in some cases, on AWS. When it's
> >> deployed on AWS, there are ELBs involved.
> >>
> >> Like some other people, we've having various trauma dealing with the
> >> very dynamic environment of AWS ELBs, which includes constant DNS
> >> changes and some other issues. So far, we've been making our HC
> >> configuration more and more complex. I'm wondering if we should back
> >> off and make it simpler -- make a new connection for each request.
> >> Obviously, this will be somewhat slower and more expensive, but at the
> >> moment we might prefer a slower and more expensive approach that works
> >> to one that occasionally gets lost.
> >>
> >> I appreciate that even making new connections all the time won't
> >> compensate for all possible DNS problems -- if Java itself caches a
> >> stale name->address translation, making a new connection will fail
> >> just as effectively as anything else.
> >>
> >> Does anyone out there have advice to share (other than running away from
> >> ELB)?
> >>
> >> -
> >> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> >> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> >>
> >>
> >
> >
> > --
> > BEKK Open
> > http://open.bekk.no
> >
> > TesTcl - a unit test framework for iRules
> > http://testcl.com
> 
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> 



-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org



Re: Help/Advise debugging SocketException in BHttpConnectionBase.fillInputBuffer()

2016-06-22 Thread Oleg Kalnichevski
On Wed, 2016-06-22 at 13:00 +0200, Mark Rekveld wrote:
> Hi,
> 
> I would like to ask for your assistance to resolve an issue that we are
> facing that has the nature of being a race condition.
> 
> The problem is that a SocketException is thrown in the finally clause of
> BHttpConnectionBase.fillInputBuffer() [
> https://github.com/apache/httpcore/blob/4.3.x/httpcore/src/main/java/org/apache/http/impl/BHttpConnectionBase.java#L326
> ]
> 
> We use 4.3.6 of the HttpClient with 4.3.3 of HttpCore, and have a custom
> extension of org.apache.http.protocol.HttpRequestExecutor that sends the
> request entity asynchronously and immediately try to read the response from
> the sever.
> 

We can only adequately support two branches of HttpClient: stable
(currently 4.5.x) and dev (currently 5.0-alpha)

Please consider upgrading to HttpClient 4.5. There have been numerous
changes since 4.3.6.  

Oleg

> The exception occurs within this custom HttpRequestExecutor when the
> conn.isResponseAvailable(1000 * 5) call is made in this code snippet. See
> stack trace below
> 
> private void sendRequestEntity(final HttpEntityEnclosingRequest req) throws
> IOException {
>   Thread entitySender = new Thread(getClass().getSimpleName() + "-" +
> System.currentTimeMillis())
>   {
> @Override
> public void run() {
>   try {
> conn.sendRequestEntity(req);
> conn.flush();
> context.setAttribute(HttpCoreContext.HTTP_REQ_SENT, Boolean.TRUE);
>   }
>   catch (Throwable t) {
> error = t;
>   }
> }
>   };
>   entitySender.start();
> 
>   while (entitySender.isAlive() && !conn.isResponseAvailable(1000 * 5)) {
> /*
>  * Wait here until either we have fully uploaded the request entity or
> the server sends a response. If
>  * we were to return immediately and called doReceiveResponse() right
> away we would provoke socket
>  * timeouts.
>  */
>   }
> }
> 
> The full stack trace:
> 
> 2016-06-03 02:36:04,149+1000 ERROR [qtp761509273-12469 - PUT
> /rest/ci/scan/treasury-web-deployable] jenkins-server
> com.sonatype.insight.brain.hds.HdsClient - Socket is closed
> java.net.SocketException: Socket is closed
>  at java.net.Socket.setSoTimeout(Socket.java:1137) ~[na:1.8.0_91]
>  at
> sun.security.ssl.BaseSSLSocketImpl.setSoTimeout(BaseSSLSocketImpl.java:633)
> ~[na:1.8.0_91]
>  at sun.security.ssl.SSLSocketImpl.setSoTimeout(SSLSocketImpl.java:2526)
> ~[na:1.8.0_91]
>  at
> org.apache.http.impl.BHttpConnectionBase.fillInputBuffer(SourceFile:326)
> ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
>  at org.apache.http.impl.BHttpConnectionBase.awaitInput(SourceFile:334)
> ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
>  at
> org.apache.http.impl.DefaultBHttpClientConnection.isResponseAvailable(SourceFile:131)
> ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
>  at
> org.apache.http.impl.conn.CPoolProxy.isResponseAvailable(SourceFile:141)
> ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
>  at
> com.sonatype.insight.client.utils.HttpRequestExecutor$HttpExchange.sendRequestEntity(SourceFile:201)
> ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
>  at
> com.sonatype.insight.client.utils.HttpRequestExecutor$HttpExchange.doSendRequest(SourceFile:177)
> ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
>  at
> com.sonatype.insight.client.utils.HttpRequestExecutor$HttpExchange.perform(SourceFile:108)
> ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
>  at
> com.sonatype.insight.client.utils.HttpRequestExecutor.execute(SourceFile:53)
> ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
>  at org.apache.http.impl.execchain.MainClientExec.execute(SourceFile:254)
> ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
>  at org.apache.http.impl.execchain.ProtocolExec.execute(SourceFile:195)
> ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
>  at org.apache.http.impl.execchain.RetryExec.execute(SourceFile:86)
> ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
>  at org.apache.http.impl.execchain.RedirectExec.execute(SourceFile:108)
> ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
>  at
> org.apache.http.impl.client.InternalHttpClient.doExecute(SourceFile:184)
> ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
>  at org.apache.http.impl.client.CloseableHttpClient.execute(SourceFile:82)
> ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
>  at org.apache.http.impl.client.CloseableHttpClient.execute(SourceFile:106)
> ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
>  at org.apache.http.impl.client.CloseableHttpClient.execute(SourceFile:57)
> ~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
>  at com.sonatype.insight.brain.hds.HdsClient.execute(SourceFile:380)
> [nexus-iq-server-1.20.0-02.jar:1.20.0-02]
>  at com.sonatype.insight.brain.hds.HdsClient.execute(SourceFile:329)
> [nexus-iq-server-1.20.0-02.jar:1.20.0-02]
>  at com.sonatype.insight.brain.hds.HdsClient.getResponse(SourceFile:132)
> [nexus-iq-server-1.20.0-02.jar:1.20.0-02]
>  at com.sonatype.insight.brain.hds.HdsClient.get(SourceFile:178)
> [nexus-iq-server-1.20.0-02.jar:1.20.0-02]
>  at com.sonatype.insight.brain.hds.Sc

Re: Telling HC to make new connections all the time

2016-06-22 Thread Benson Margulies
On Wed, Jun 22, 2016 at 3:05 AM, Stefan Magnus Landrø
 wrote:
> Hi Benson,
>
> Have you considered implementing a custom
> https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/conn/DnsResolver.html
> that handles DNS changes in a better way than the default implementation?
> You could probably use http://www.xbill.org/dnsjava/ and lookup names
> directly with the amazon NS?

Thanks for reminding me to ask about that. Does HC call the resolver
as part of deciding whether to keep a connection or not? The problem
as we see it is this: the DNS mapping for an ELB changes, and a minute
later the connections to it become useless but don't necessarily get
properly closed with a FIN by the ELB. I've written some goofy code
that notices when the address resolution changes for any of the ELB's
we care about, and when it does it calls 'closeIdleConnections' with a
time of 0. On the other hand, I don't think we're currently having any
problems getting InetAddress.getByName to keep up with Amazon, but
your suggestion would be safer.


>
> Stefan
>
> 2016-06-20 19:57 GMT+02:00 Benson Margulies :
>
>> We use the async http components library to connect various components
>> in a system that is deployed, in some cases, on AWS. When it's
>> deployed on AWS, there are ELBs involved.
>>
>> Like some other people, we've having various trauma dealing with the
>> very dynamic environment of AWS ELBs, which includes constant DNS
>> changes and some other issues. So far, we've been making our HC
>> configuration more and more complex. I'm wondering if we should back
>> off and make it simpler -- make a new connection for each request.
>> Obviously, this will be somewhat slower and more expensive, but at the
>> moment we might prefer a slower and more expensive approach that works
>> to one that occasionally gets lost.
>>
>> I appreciate that even making new connections all the time won't
>> compensate for all possible DNS problems -- if Java itself caches a
>> stale name->address translation, making a new connection will fail
>> just as effectively as anything else.
>>
>> Does anyone out there have advice to share (other than running away from
>> ELB)?
>>
>> -
>> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
>> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>>
>>
>
>
> --
> BEKK Open
> http://open.bekk.no
>
> TesTcl - a unit test framework for iRules
> http://testcl.com

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org



Help/Advise debugging SocketException in BHttpConnectionBase.fillInputBuffer()

2016-06-22 Thread Mark Rekveld
Hi,

I would like to ask for your assistance to resolve an issue that we are
facing that has the nature of being a race condition.

The problem is that a SocketException is thrown in the finally clause of
BHttpConnectionBase.fillInputBuffer() [
https://github.com/apache/httpcore/blob/4.3.x/httpcore/src/main/java/org/apache/http/impl/BHttpConnectionBase.java#L326
]

We use 4.3.6 of the HttpClient with 4.3.3 of HttpCore, and have a custom
extension of org.apache.http.protocol.HttpRequestExecutor that sends the
request entity asynchronously and immediately try to read the response from
the sever.

The exception occurs within this custom HttpRequestExecutor when the
conn.isResponseAvailable(1000 * 5) call is made in this code snippet. See
stack trace below

private void sendRequestEntity(final HttpEntityEnclosingRequest req) throws
IOException {
  Thread entitySender = new Thread(getClass().getSimpleName() + "-" +
System.currentTimeMillis())
  {
@Override
public void run() {
  try {
conn.sendRequestEntity(req);
conn.flush();
context.setAttribute(HttpCoreContext.HTTP_REQ_SENT, Boolean.TRUE);
  }
  catch (Throwable t) {
error = t;
  }
}
  };
  entitySender.start();

  while (entitySender.isAlive() && !conn.isResponseAvailable(1000 * 5)) {
/*
 * Wait here until either we have fully uploaded the request entity or
the server sends a response. If
 * we were to return immediately and called doReceiveResponse() right
away we would provoke socket
 * timeouts.
 */
  }
}

The full stack trace:

2016-06-03 02:36:04,149+1000 ERROR [qtp761509273-12469 - PUT
/rest/ci/scan/treasury-web-deployable] jenkins-server
com.sonatype.insight.brain.hds.HdsClient - Socket is closed
java.net.SocketException: Socket is closed
 at java.net.Socket.setSoTimeout(Socket.java:1137) ~[na:1.8.0_91]
 at
sun.security.ssl.BaseSSLSocketImpl.setSoTimeout(BaseSSLSocketImpl.java:633)
~[na:1.8.0_91]
 at sun.security.ssl.SSLSocketImpl.setSoTimeout(SSLSocketImpl.java:2526)
~[na:1.8.0_91]
 at
org.apache.http.impl.BHttpConnectionBase.fillInputBuffer(SourceFile:326)
~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at org.apache.http.impl.BHttpConnectionBase.awaitInput(SourceFile:334)
~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at
org.apache.http.impl.DefaultBHttpClientConnection.isResponseAvailable(SourceFile:131)
~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at
org.apache.http.impl.conn.CPoolProxy.isResponseAvailable(SourceFile:141)
~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at
com.sonatype.insight.client.utils.HttpRequestExecutor$HttpExchange.sendRequestEntity(SourceFile:201)
~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at
com.sonatype.insight.client.utils.HttpRequestExecutor$HttpExchange.doSendRequest(SourceFile:177)
~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at
com.sonatype.insight.client.utils.HttpRequestExecutor$HttpExchange.perform(SourceFile:108)
~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at
com.sonatype.insight.client.utils.HttpRequestExecutor.execute(SourceFile:53)
~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at org.apache.http.impl.execchain.MainClientExec.execute(SourceFile:254)
~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at org.apache.http.impl.execchain.ProtocolExec.execute(SourceFile:195)
~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at org.apache.http.impl.execchain.RetryExec.execute(SourceFile:86)
~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at org.apache.http.impl.execchain.RedirectExec.execute(SourceFile:108)
~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at
org.apache.http.impl.client.InternalHttpClient.doExecute(SourceFile:184)
~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at org.apache.http.impl.client.CloseableHttpClient.execute(SourceFile:82)
~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at org.apache.http.impl.client.CloseableHttpClient.execute(SourceFile:106)
~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at org.apache.http.impl.client.CloseableHttpClient.execute(SourceFile:57)
~[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at com.sonatype.insight.brain.hds.HdsClient.execute(SourceFile:380)
[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at com.sonatype.insight.brain.hds.HdsClient.execute(SourceFile:329)
[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at com.sonatype.insight.brain.hds.HdsClient.getResponse(SourceFile:132)
[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at com.sonatype.insight.brain.hds.HdsClient.get(SourceFile:178)
[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at com.sonatype.insight.brain.hds.ScanUploader.upload(SourceFile:69)
[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at com.sonatype.insight.brain.hds.CIResource.putScan(SourceFile:50)
[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at
com.sonatype.insight.brain.hds.CIResource$$EnhancerByGuice$$88667ab8.CGLIB$putScan$0()
[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at
com.sonatype.insight.brain.hds.CIResource$$EnhancerByGuice$$88667ab8$$FastClassByGuice$$99131cf1.invoke()
[nexus-iq-server-1.20.0-02.jar:1.20.0-02]
 at
com.google.injec

Re: Telling HC to make new connections all the time

2016-06-22 Thread Stefan Magnus Landrø
Hi Benson,

Have you considered implementing a custom
https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/conn/DnsResolver.html
that handles DNS changes in a better way than the default implementation?
You could probably use http://www.xbill.org/dnsjava/ and lookup names
directly with the amazon NS?

Stefan

2016-06-20 19:57 GMT+02:00 Benson Margulies :

> We use the async http components library to connect various components
> in a system that is deployed, in some cases, on AWS. When it's
> deployed on AWS, there are ELBs involved.
>
> Like some other people, we've having various trauma dealing with the
> very dynamic environment of AWS ELBs, which includes constant DNS
> changes and some other issues. So far, we've been making our HC
> configuration more and more complex. I'm wondering if we should back
> off and make it simpler -- make a new connection for each request.
> Obviously, this will be somewhat slower and more expensive, but at the
> moment we might prefer a slower and more expensive approach that works
> to one that occasionally gets lost.
>
> I appreciate that even making new connections all the time won't
> compensate for all possible DNS problems -- if Java itself caches a
> stale name->address translation, making a new connection will fail
> just as effectively as anything else.
>
> Does anyone out there have advice to share (other than running away from
> ELB)?
>
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>
>


-- 
BEKK Open
http://open.bekk.no

TesTcl - a unit test framework for iRules
http://testcl.com