Thanks, Bryan! Based on the discussion with Bryan on slack, below is the newer
proposed config changes:
1) Rename "proxy.config.net.max_active_connections_in" to
"proxy.config.net.max_requests_in"
2) Rename "proxy.config.http.server_max_connections" to
"proxy.config.net.max_connections_out"
3) "proxy.config.net.max_connections_in" - This config will stay put to limit
the max number of in-flight requests (concurrency) + idle (socket) connections
4) "proxy.config.net.connections_throttle" - This config will stay put to limit
the max number of open connections (FD, rlimits etc), but eventually will be
deprecated and replaced with {{ "proxy.config.net.max_connections_in" +
"proxy.config.net.max_connections_out" }}
Please provide any feedback/comments/concerns.
Thanks,
Sudheer
On Friday, May 15, 2020, 11:11:53 AM PDT, Bryan Call <[email protected]>
wrote:
After seeing the GitHub PR I think we should keep
proxy.config.net.max_connections_in since it is the number of socket
connections (active + idle) and then rename
proxy.config.net.max_active_connections_in, as you suggested in your email, to
proxy.config.net.max_active_requests_in or proxy.config.net.max_requests_in (as
Sudheer suggested on Slack).
I would like to see us get rid of proxy.config.net.connections_throttle in the
long term and use max_connections_in/out if possible.
-Bryan
> On May 12, 2020, at 5:17 PM, Sudheer Vinukonda
> <[email protected]> wrote:
>
> 9.x (re)introduces a feature (it was accidentally "lost" during some
> unrelated refactoring) that will allow to configure limits on max "active"
> connections that can be allowed at any given time
> (https://github.com/apache/trafficserver/pull/6754)
> Given that this feature is based on tracking "active connections", it should
> work very well with HTTP/1.1 traffic (where, 1 active connection = 1
> request). However, with HTTP/2 and stream multiplexing, this is no longer
> true and it isn't nearly as effective. One can still use an average number of
> concurrent streams/connection to configure the limits, but, ideally, what
> would work for both HTTP/1.1 and HTTP/2 (and going forward with QUIC/HTTP/3)
> would be to track and (rate) limit request level concurrency. There is some
> ongoing work on this and to align better with that work, we are proposing to
> make the below changes to existing configs with 9.x
>
> 1) Rename "proxy.config.net.max_connections_in" to
> "proxy.config.net.max_requests_in"
> 2) Rename "proxy.config.net.max_active_connections_in" to
> "proxy.config.net.max_active_requests_in"
> 3) proxy.config.net.connections_throttle - This config will stay put to limit
> the max number of open connections (FD, rlimits etc)
>
> More context on the new changes -
>
>
> The idea is to tune active connections that can be handled (based on the
> available resources/capacity (CPU, Memory, Network bandwidth) and
> minimize/remove dependency on having to tune connection timeouts
> (active/inactive/keep-alive etc) which are very hard to tune.
>
> The primary goal is to limit the max active connections allowed at any given
> instant. The resource requirement for an idle/inactive vs active connections
> are completely different - For e.g an idle connection really only consumes
> memory resource, while an active connection consumes CPU, network besides
> memory. And allowing to tune/cap the max active connections based on the
> deployed capacity for the resources available, would make timeout tuning
> almost redundant and no op. Otherwise, we'd have to tune the timeouts to
> estimate throughput which is very hard as it's hard to justify how large or
> small we want the active timeout to be or keep alive timeout to be. For e.g
> in a non-peak hour, we could let the active timeout be much higher than the
> default, while during peak hour, we'd want to limit it to ensure we are not
> starving resources on one connection.
> Note: there's one little TODO item here. PluginVC's connections are not
> tracked in the NetVC's active queue because these are bogus internal
> connections. However, for some users, internal traffic is significant (e.g
> sideways calls, or background fetches etc) and does consume plenty of
> resources. Since these internal requests don't impact ingress network, and
> have a slightly different resource requirements than client originated
> requests, it might be better to track these using a separate config for
> internal requests. Will follow that part up with a separate PR.
>
>
