On Fri, 2019-06-07 at 19:43 +0200, Joan grupoventus wrote:
> Hi,
>
> > Now the current issue. Something similar is happening. After
> > several
> > days of execution, requests start to throw an exception because
> > the
> > container asynchronous timeout is reached. Tracing http, we saw
> > the
> > pool exhausted ... so it's a behavior similar to issue nr. 1.
> > 08:14:03.054 Connection request: [route: {}->
> > http://10.15.15.53:8083][total kept alive: 0; route allocated: 107
> > of
> > 1500; total allocated: 1500 of 1500]
> >
>
> I could not find it anywhere in the logs that you have provided.
> --> It's here, https://github.com/joanbalaguero/HttpClient , in the
> http.log I uploaded.
>
But the pool is already been completely exhausted at the beginning of
the log
---
Connection request: [route: {}->http://10.15.15.22:8081][total kept
alive: 0; route allocated: 9 of 1500; total allocated: 1500 of 1500]
---
The log contains no information as to what has caused the pool
exhaustion in the first place. It is useless.
> >
> I need to see a log of such 'hanging' message exchnage.
> --> Ok, to do this I need to enable the http traces for days waiting
> the "hang" to happen. But with the log I have enabled now there is a
> significant increase in cpu and io (because the responses are usually
> quite big). Is there any way to enable a kind of logging useful for
> you but avoiding trace the request and response messages?
>
Try disabling I/O session and wire logging. That should massively
reduce the amount of chatter in the log.
---
<Configuration status="WARN" name="XMLConfigTest">
<Appenders>
<Console name="STDOUT">
<!--<PatternLayout pattern="%msg%n%xThrowable"/>-->
<PatternLayout pattern="%d %-5level
[%t][%logger]%notEmpty{[%markerSimpleName]} %msg%n%xThrowable"/>
</Console>
</Appenders>
<Loggers>
<Root level="warn">
<AppenderRef ref="STDOUT"/>
</Root>
<Logger name="org.apache.hc.client5" level="debug"/>
<Logger name="org.apache.hc.client5.http.impl" level="debug"/>
<Logger name="org.apache.hc.client5.http.headers" level="debug"/>
<Logger name="org.apache.hc.client5.http.impl.io" level="debug"/>
<Logger name="org.apache.hc.client5.http.impl.nio" level="debug"/>
<Logger name="org.apache.hc.client5.http.impl.cache" level="debug"/>
<Logger name="org.apache.hc.client5.http.wire" level="error"/>
<Logger name="org.apache.hc.client5.http2" level="debug"/>
<Logger name="org.apache.hc.client5.http2.frame" level="debug"/>
<Logger name="org.apache.hc.client5.http2.frame.payload" level="error"/>
<Logger name="org.apache.hc.client5.http2.flow" level="debug"/>
</Loggers>
</Configuration>
---
Oleg
> Joan.
>
>
> > Thanks,
> >
> > Joan.
> >
> > -----Original Message-----
> > From: Oleg Kalnichevski <[email protected]>
> > Sent: Friday, June 7, 2019 4:00 PM
> > To: HttpClient User Discussion <[email protected]>
> > Subject: Re: Response timeout issue
> >
> > On Thu, 2019-06-06 at 18:56 +0200, Joan grupoventus wrote:
> > > Hi again,
> > >
> > > Not sure if we have seen something strange in the current test
> > > with
> > > the keep alive connections. It's just a GET method to get a gif
> > > (this pool has no keep-alive, that means a 100ms keep-alive).
> > > Here
> > > the complete http log:
> > > https://github.com/joanbalaguero/HttpClient.git
> > > (it's the "http_get.log")
> >
> > Joan
> >
> > I am confused. I looked at http_get.log and I can see the exact
> > same
> > number of connection lease requests and that of released
> > connections.
> > There are two 253 message exchanges and the same number of
> > connection
> > lease / release operations.
> >
> > I am not seeing any evidence of connections being leaked. I am
> > lost
> > now. What is exactly the issue you are having?
> >
> > Oleg
> >
> >
> > > Previously to this test, this pool received 4 requests today
> > > 06/06/2019 around 11:15:00 (so 6 hours before the test), these 4
> > > requests ended up with a 4xx status code.
> > >
> > > This is the first request we sent to this pool using this route.
> > > Should not keep-alive be 0 and total allocated 1 of 1000?
> > > ---------------------------------------------------------------
> > > ----
> > > ------------
> > > 17:23:02.692 ex-0105B9B8: preparing request execution
> > > 17:23:02.692 Target auth state: UNCHALLENGED
> > > 17:23:02.692 Proxy auth state: UNCHALLENGED
> > > 17:23:02.692 ex-0105B9B8: acquiring connection with route {}->
> > > http://andalucia.viajesolympia.com:80
> > > 17:23:02.692 Connection request: [route: {}->
> > > http://andalucia.viajesolympia.com:80][total kept alive: 1; route
> > > allocated: 0 of 1000; total allocated: 1 of 1000]
> > > 17:23:02.692 Connection leased: [route: {}->
> > > http://andalucia.viajesolympia.com:80][total kept alive: 1; route
> > > allocated: 1 of 1000; total allocated: 2 of 1000]
> > > 17:23:02.692 ep-010526BF: acquired null
> > > 17:23:02.692 ex-0105B9B8: connection acquired
> > >
> > > This is the second:
> > > ------------------------
> > > 17:23:02.954 ex-0105B9B9: preparing request execution
> > > 17:23:02.955 Target auth state: UNCHALLENGED
> > > 17:23:02.955 Proxy auth state: UNCHALLENGED
> > > 17:23:02.955 ex-0105B9B9: acquiring connection with route {}->
> > > http://andalucia.viajesolympia.com:80
> > > 17:23:02.955 Connection request: [route: {}->
> > > http://andalucia.viajesolympia.com:80][total kept alive: 2; route
> > > allocated: 1 of 1000; total allocated: 2 of 1000]
> > > 17:23:02.955 Connection leased: [route: {}->
> > > http://andalucia.viajesolympia.com:80][total kept alive: 1; route
> > > allocated: 1 of 1000; total allocated: 2 of 1000]
> > > 17:23:02.955 ep-010526C0: acquired null
> > > 17:23:02.955 ex-0105B9B9: connection acquired
> > > 17:23:02.976 ep-010526C0: connected i/o-0000434A
> > > 17:23:02.976 Connected to target
> > > 17:23:02.976 Route fully established
> > >
> > > ( . . . )
> > >
> > > Now a pool reboot
> > > -------------------------
> > > 17:27:29.996 Shutdown connection pool GRACEFUL
> > > 17:27:29.996 i/o-000043AF: Shutdown connection GRACEFUL
> > > 17:27:29.996 i/o-00002D9A: Shutdown connection GRACEFUL
> > > 17:27:29.996 Connection pool shut down
> > > 17:27:29.997 Shutdown GRACEFUL
> > >
> > > 1st. request after rebooting. Here keep-alive = 0 and
> > > total_allocated = 1
> > > -------------------------------------
> > > 17:27:53.585 ex-0105BA49: preparing request execution
> > > 17:27:53.585 Target auth state: UNCHALLENGED
> > > 17:27:53.585 Proxy auth state: UNCHALLENGED
> > > 17:27:53.585 ex-0105BA49: acquiring connection with route {}->
> > > http://andalucia.viajesolympia.com:80
> > > 17:27:53.585 Connection request: [route: {}->
> > > http://andalucia.viajesolympia.com:80][total kept alive: 0; route
> > > allocated: 0 of 1000; total allocated: 0 of 1000]
> > > 17:27:53.585 Connection leased: [route: {}->
> > > http://andalucia.viajesolympia.com:80][total kept alive: 0; route
> > > allocated: 1 of 1000; total allocated: 1 of 1000]
> > > 17:27:53.585 ep-01052750: acquired null
> > > 17:27:53.585 ex-0105BA49: connection acquired
> > > 17:27:53.609 ep-01052750: connected i/o-000043B0
> > > 17:27:53.609 Connected to target
> > > 17:27:53.609 Route fully established
> > >
> > >
> > > Thanks,
> > > Joan.
> > >
> > >
> > >
> > > -----------------------------------------------------------------
> > > ----
> > > To unsubscribe, e-mail:
> > > [email protected]
> > > For additional commands, e-mail:
> > > [email protected]
> > >
> >
> >
> > -----------------------------------------------------------------
> > ----
> > To unsubscribe, e-mail: [email protected]
> > For additional commands, e-mail:
> > [email protected]
> >
> >
> >
> >
> > -----------------------------------------------------------------
> > ----
> > To unsubscribe, e-mail: [email protected]
> > For additional commands, e-mail:
> > [email protected]
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
