Re: HttpServer on shutdown waits for entire grace period when client is connected using keep-alive
On Sun, 2020-08-09 at 20:24 +0200, Niklas Lochschmidt wrote: > Thank you, Oleg. Just to confirm. The code you send is equivalent to > just a lower grace period in the 4.x API, e.g. it will terminate > legitimate inflight connections as well, not only the idle > connections, right? Correct. > At least that is what I am seeing from my tests. > > Some background: I actually want to have one particular service built > on http4k, that is scaled up and back down to zero pretty frequently. > Sometimes it has to serve API requests that take up to 10 or 15 > seconds to complete (doing a larger amount of fan-out http requests > to other APIs). That means I would want to set a higher grace period > even, but that will mean the service will always take long to shut > down. > > Using http4k I can easily switch to a different server backend, but I > was using this one and was very happy with it's performance. Kudos > for that! > > Of course the "real" solution is to switch to a background worker > setup with queues, but so far it was unnecessary. > > If I can piece together a good testcase I'll open an issue on Jira. I > guess this would be something that would only land in 5.x and not be > backported into 4.x anymore? > We are not planning to add new features to 4.x or back-port features from 5.x. All our limited resources will have to allocated to building 5.1 and maintaining 5.0. Oleg > Niklas > > Oleg Kalnichevski schrieb am So., 9. Aug. 2020, > 13:00: > > On Fri, 2020-08-07 at 20:04 +0200, Niklas Lochschmidt wrote: > > > I am trying to use the graceful shutdown of HttpServer server > > for > > > http4k, using HttpCore 4.4.13 and 5.0.1. I am creating the server > > > simply > > > using `ServerBootstrap.bootstrap().register(...).create()` and > > > shutting > > > down with `shutdown(5, TimeUnit.SECONDS)` and > > > `close(CloseMode.GRACEFUL)` respectively. Here is what I am > > seeing: > > > > > > - When the server has never been requested it shuts down > > instantly > > > ✔️ > > > - When the server has been requested by a client without keep- > > alive > > > it > > > shuts down instantly ✔️ > > > - When the server has been requested by a client with keep-alive > > and > > > the > > > request is in-flight during shutdown it shuts down as soon as > > the > > > request is done ✔️ > > > > > > The problem is, when I connected with a client that uses keep- > > alive, > > > and > > > the request has been handled by the server and now the client is > > > holding > > > the connection open, then when I shutdown the server, the server > > > will > > > take the whole grace period to shutdown at which point the keep- > > > alive > > > connections are finally being force-closed. > > > > > > Is it possible to somehow speed up the shutdown also in this > > case > > > without terminating in-flight requests? Is there maybe a > > workaround > > > to > > > terminate all idle connections with keep-alives immediately > > before > > > shutdown? > > > > > > > The problem is there is no way of reliably testing whether a > > connection > > is idle or not, as connections can become active immediately after > > passing inactivity test. > > > > What could be a reasonable solution in many cases is to reduce the > > default grace period timeout. > > > > This will make the server await connection termination for 1 second > > instead of 5 seconds. > > ``` > > HttpServer server = ServerBootstrap.bootstrap().create(); > > server.initiateShutdown(); > > server.awaitTermination(TimeValue.ofSeconds(1)); > > server.close(CloseMode.IMMEDIATE); > > ``` > > > > Having said that there might be a better way to detect idle server- > > side > > connections as well. Feel free to raise a request in JIRA if you > > want > > me to look into that. > > > > Oleg > > > > - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: HttpServer on shutdown waits for entire grace period when client is connected using keep-alive
Thank you, Oleg. Just to confirm. The code you send is equivalent to just a lower grace period in the 4.x API, e.g. it will terminate legitimate inflight connections as well, not only the idle connections, right? At least that is what I am seeing from my tests. Some background: I actually want to have one particular service built on http4k, that is scaled up and back down to zero pretty frequently. Sometimes it has to serve API requests that take up to 10 or 15 seconds to complete (doing a larger amount of fan-out http requests to other APIs). That means I would want to set a higher grace period even, but that will mean the service will always take long to shut down. Using http4k I can easily switch to a different server backend, but I was using this one and was very happy with it's performance. Kudos for that! Of course the "real" solution is to switch to a background worker setup with queues, but so far it was unnecessary. If I can piece together a good testcase I'll open an issue on Jira. I guess this would be something that would only land in 5.x and not be backported into 4.x anymore? Niklas Oleg Kalnichevski schrieb am So., 9. Aug. 2020, 13:00: > On Fri, 2020-08-07 at 20:04 +0200, Niklas Lochschmidt wrote: > > I am trying to use the graceful shutdown of HttpServer server for > > http4k, using HttpCore 4.4.13 and 5.0.1. I am creating the server > > simply > > using `ServerBootstrap.bootstrap().register(...).create()` and > > shutting > > down with `shutdown(5, TimeUnit.SECONDS)` and > > `close(CloseMode.GRACEFUL)` respectively. Here is what I am seeing: > > > > - When the server has never been requested it shuts down instantly > > ✔️ > > - When the server has been requested by a client without keep-alive > > it > > shuts down instantly ✔️ > > - When the server has been requested by a client with keep-alive and > > the > > request is in-flight during shutdown it shuts down as soon as the > > request is done ✔️ > > > > The problem is, when I connected with a client that uses keep-alive, > > and > > the request has been handled by the server and now the client is > > holding > > the connection open, then when I shutdown the server, the server > > will > > take the whole grace period to shutdown at which point the keep- > > alive > > connections are finally being force-closed. > > > > Is it possible to somehow speed up the shutdown also in this case > > without terminating in-flight requests? Is there maybe a workaround > > to > > terminate all idle connections with keep-alives immediately before > > shutdown? > > > > The problem is there is no way of reliably testing whether a connection > is idle or not, as connections can become active immediately after > passing inactivity test. > > What could be a reasonable solution in many cases is to reduce the > default grace period timeout. > > This will make the server await connection termination for 1 second > instead of 5 seconds. > ``` > HttpServer server = ServerBootstrap.bootstrap().create(); > server.initiateShutdown(); > server.awaitTermination(TimeValue.ofSeconds(1)); > server.close(CloseMode.IMMEDIATE); > ``` > > Having said that there might be a better way to detect idle server-side > connections as well. Feel free to raise a request in JIRA if you want > me to look into that. > > Oleg > > >
Re: HttpServer on shutdown waits for entire grace period when client is connected using keep-alive
On Fri, 2020-08-07 at 20:04 +0200, Niklas Lochschmidt wrote: > I am trying to use the graceful shutdown of HttpServer server for > http4k, using HttpCore 4.4.13 and 5.0.1. I am creating the server > simply > using `ServerBootstrap.bootstrap().register(...).create()` and > shutting > down with `shutdown(5, TimeUnit.SECONDS)` and > `close(CloseMode.GRACEFUL)` respectively. Here is what I am seeing: > > - When the server has never been requested it shuts down instantly > ✔️ > - When the server has been requested by a client without keep-alive > it > shuts down instantly ✔️ > - When the server has been requested by a client with keep-alive and > the > request is in-flight during shutdown it shuts down as soon as the > request is done ✔️ > > The problem is, when I connected with a client that uses keep-alive, > and > the request has been handled by the server and now the client is > holding > the connection open, then when I shutdown the server, the server > will > take the whole grace period to shutdown at which point the keep- > alive > connections are finally being force-closed. > > Is it possible to somehow speed up the shutdown also in this case > without terminating in-flight requests? Is there maybe a workaround > to > terminate all idle connections with keep-alives immediately before > shutdown? > The problem is there is no way of reliably testing whether a connection is idle or not, as connections can become active immediately after passing inactivity test. What could be a reasonable solution in many cases is to reduce the default grace period timeout. This will make the server await connection termination for 1 second instead of 5 seconds. ``` HttpServer server = ServerBootstrap.bootstrap().create(); server.initiateShutdown(); server.awaitTermination(TimeValue.ofSeconds(1)); server.close(CloseMode.IMMEDIATE); ``` Having said that there might be a better way to detect idle server-side connections as well. Feel free to raise a request in JIRA if you want me to look into that. Oleg - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
