Connection leakage from Migrating 4.2.3 -> 4.3.3

[Nick Chang]

Hello,

We are using httpclient to build a web proxy application in our product. We
use 4.2.3 version and it has been working flawlessly. Now I am trying to
migrate the product's httpclient library from 4.2.3 to 4.3.3 to achieve
SNI-Extension support with JDK1.7. We have to re-factor several deprecated
methods and classes in 4.2.3, such as changing
PoolingClientConnectionManager to PoolingHttpClientConnectionManager,
changing DefaultHttpClient to CloseableHttpClient.

However, I am running into an issue that connection is not being released
and eventually execute() got the exception in waiting for a connection
(Exception Text:org.apache.http.conn.ConnectionPoolTimeoutException:
Timeout waiting for connection from pool).

Below is the code sniplet in a worker thread.

The 4.2.3 code

        m_httpClient = new DefaultHttpClient(m_connectionManager, params);
        ...
        m_httpResponse = m_httpClient.execute(m_httpCommand, localContext);
        ...
        BasicManagedEntity entity =
(BasicManagedEntity)m_httpResponse.getEntity();
        if (entity!= null) {
            m_inputStream = entity.getContent();
        }
        .. process inputStream ...
        if (m_inputStream != null) {
            m_inputStream.close();
        }

        m_httpClient.getConnectionManager().closeIdleConnections(5,
TimeUnit.SECONDS);

The 4.3.3 code
        ...
        m_httpClient = HttpClients.custom()
            .setConnectionManager(m_connectionManager)
            .setDefaultCredentialsProvider(credentialsProvider)
            .setDefaultRequestConfig(m_defaultRequestConfig)
            .disableCookieManagement()
            .disableRedirectHandling()
            .disableAutomaticRetries()
            .setKeepAliveStrategy(keepAliveStrategy)
            .build();
        ...
        m_httpResponse = m_httpClient.execute(m_httpCommand);
        ...
        m_entity = (HttpEntity)m_httpResponse.getEntity();
        if (m_entity!= null) {
              m_inputStream = m_entity.getContent();
        }
        .. process inputStream ...
        try {
            EntityUtils.consume(m_entity);
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    ...
        try {
            if (m_inputStream != null) {
                m_inputStream.close();
            }

        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    ...
        try {
            m_httpResponse.close();
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

(I've followed the instruction in this email.
http://mail-archives.apache.org/mod_mbox/hc-httpclient-users/201211.mbox/%3C1353079080.20463.12.camel%40ubuntu%3Eto
close the connection.)

The 4.2.3 code works flawlessly. The number of leased connection always
goes back to zero. But in the 4.3.3 code, the number of leased connection
gradually increased.

I added IdleConnectionMonitorThread class as in
http://hc.apache.org/httpcomponents-client-4.3.x/tutorial/html/connmgmt.html#d5e405It
did help reducing the number of leased
connection, but the number of leased connections still gradually increased.

Hence, here are my questions:

1. How would I track down this connection leakage?
2. Is there a suggestion or best practice to migrate the deprecated classes
and methods in 4.2.3 to the current ones in 4.3.3?
3. Is a separate thread, IdleConnectionMonitorThread, required to clean up
the leased connection in 4.3.3? If so, why is it not needed in 4.2.3?

Thanks in advance,

Nick