pooling connection manager: changing max per route

[Pete Keyes]

version: 4.4.1
App Server: TomEE7

We use PoolingHttpClientConnectionManager with the following defaults:

  *   max total connections: 100,000
  *   default max per route: 50

we create the HttpClient and PoolingHttpClientConnectionManager as static 
singletons at container startup.
private static Thread globalConnManagerThread;
private static CloseableHttpClient globalHttpClient;
static {
        // create global pooling connection manager
        globalConnManagerPool = new PoolingHttpClientConnectionManager(
            registry
        );
        globalConnManagerPool.setDefaultMaxPerRoute(50);
        globalConnManagerPool.setMaxTotal(100000);

        // create global HttpClient instance
        final HttpClientBuilder builder = HttpClients.custom()
            .setConnectionManager(globalConnManagerPool)
            .setConnectionManagerShared(true)
            .setConnectionTimeToLive(5, TimeUnit.MINUTES)
            .setDefaultConnectionConfig(connConfig)
            .setDefaultRequestConfig(requestConfig)
            .setMaxConnPerRoute(50)  // <<== default per route is 50
            .setDefaultSocketConfig(socketConfig)
            .setMaxConnTotal(100000)
            .setRetryHandler(httpRetryHandler)
            .useSystemProperties()
        ;
        CloseableHttpClient globalHttpClient = builder.build();
}

We set the max per specific route with the following code snippet:

@PostConstruct public void init() {
    List<URL> allUrls = getAllUrls();
    for(final URL url : allUrls) {
        HttpServiceConfig hsc = getHttpServiceConfig(url);

        final String host = url.getHost();
        final int portNo = url.getPort() == -1 ? url.getDefaultPort() : 
url.getPort();
        final String protocol = url.getProtocol();
        final HttpHost httpHost = new HttpHost(host, portNo, protocol);
        final HttpRoute httpRoute = new HttpRoute(httpHost);
        globalConnManagerPool.setMaxPerRoute(httpRoute, 200);  // <<== max set 
to 200 for route
        log.info(gMarker, "set route max-conns:"
            + "; max-allowed=" + hsc.getMaxConns()
            + "; protocol-host-port=" + protocol + "://" + host + ":" + portNo
            + "; httpRoute=" + String.valueOf(httpRoute)
        );
    }
}


we see each route specific max connection log message at startup.  for instance:
set route max-conns: conns-max=200; protocol-host-port=https://some-host:443; 
httpRoute={}->https://some-host:443

we have a background thread that monitors the apache-hc connection pool 
statistics and emits log messages.  we see the following apache-hc connection 
manager pool “route specific” statistics logged:
private void logPoolStats() {
    Set<HttpRoute> routes = globalConnManagerPool.getRoutes();
    for(final HttpRoute r : routes) {
        final PoolStats ps = cm.getStats(r);
        final int available = ps.getAvailable();
        final int active = ps.getLeased();
        final int limit = ps.getMax();
        final int blocking = ps.getPending();

        String s = "emit statistic:"
            + " type=hc-conn-pool"
            + "; route=" + hostname
            + "; available=" + available
            + "; active=" + active
            + "; blocking=" + blocking
            + "; max-allowed=" + limit
            + "; server-nm=" + Helpers.getLocalHost() + ":" + 
Helpers.getServerHttpsPort()
        ;
        log.info(marker, s);
    }
}

we see log messages like the following from the apache-hc connection pool 
monitor logging:
emit statistic: type=hc-conn-pool; route=some-host; available=1; active=0; 
blocking=0; max-allowed=50

The connection manager isn’t respecting the per-route connection settings.  
Above we see that the “max-allowed” per route reported by the pool-manager is 
50 for a route that specifically set the max-allowed to 200.

Any hints about what we are doing wrong to override the limit on a per-route 
basis?

—
Pete Keyes