AW: Why are many TCP open connections created on OS (Linux)?

[Gnerlich, Manuel (IPS PROJECTS GMBH)]

Hello Bernd,

thank you for your reply. 

I see them as ESTABLISHED and TIME_WAIT in netstat. The connections with 
ESTABLISHED state are ok and most under 100, but the connections with TIME_WAIT 
state are always more than 200. I mean on the client. On the server is ok and 
has little open connections. 

Gruss ;) Manuel

Here some important  code, how I set it up:

public class NetworkClient implements AutoCloseable {
...
public NetworkClient() {
    ConnectingIOReactor ioReactor;
    try {
      // Create I/O reactor configuration
      IOReactorConfig ioReactorConfig = IOReactorConfig.custom()
          .setIoThreadCount(Runtime.getRuntime().availableProcessors())
          
.setConnectTimeout(config.getConnectTimeout()).setSoTimeout(config.getSoTimeout())
          .build();
      ioReactor = new DefaultConnectingIOReactor(ioReactorConfig);

      PoolingNHttpClientConnectionManager cm = new 
PoolingNHttpClientConnectionManager(ioReactor);
      cm.setMaxTotal(config.getConnectionMaxTotal());
      cm.setDefaultMaxPerRoute(config.getConnectionDefaultMaxPerRoute());

      RequestConfig globalConfig = 
RequestConfig.custom().setCookieSpec(CookieSpecs.IGNORE_COOKIES)
          .build();

      HttpProcessor httpprocessor = HttpProcessorBuilder.create().build();
      httpClient = 
HttpAsyncClients.custom().setDefaultRequestConfig(globalConfig)
          .setHttpProcessor(httpprocessor).setConnectionManager(cm).build();
    } catch (IOReactorException e) {
      LOGGER.error("Could not create ConnectionIOReactor", e);
    }

    initClientTransport();
  }

  private void initClientTransport() {
    httpClient.start();
  }

....
....
public Integer send(String destination, int port, HttpRequest httpRequest, 
String operationName) {
    Assert.notNull(destination);
    Assert.notNull(httpRequest);

    startTime = startTime == 0 ? System.nanoTime() : startTime;

    HttpHost target = new HttpHost(destination, port);

    NcResponse ncResponse = getNewNcResponse(operationName);

    NcResponseProducer responseProducer = new NcResponseProducer(target, 
httpRequest, ncResponse,
        requestCounter);
    NcResponseConsumer responseConsumer = new NcResponseConsumer(ncResponse);

    httpClient.execute(responseProducer, responseConsumer, new 
ClientCallback(responseCounter,
        failCounter, connectionErrorMessageList, getRequestLine(httpRequest)));

    long clientId = requestClientId.getAndIncrement();
    return ((Long) clientId).intValue();
  }
..
}

public class NcResponseProducer extends BasicAsyncRequestProducer {

  private final NcResponse ncResponse;

  private static final Logger LOGGER = LogManager.getLogger();

  private final AtomicLong requestCounter;

  private final Header[] allHeaders;

  public NcResponseProducer(HttpHost target, HttpRequest request, NcResponse 
ncResponse,
      AtomicLong requestCounter) {
    super(target, request);
    this.allHeaders = request.getAllHeaders();
    this.ncResponse = ncResponse;
    this.requestCounter = requestCounter;
  }

  @Override
  public void requestCompleted(HttpContext context) {
    super.requestCompleted(context);
    ncResponse.setStartTimeInNano(System.nanoTime());
    LOGGER.debug("Sent request to client {}. request ID={} headers={} count={}",
        getTarget().toHostString(), ncResponse.getRequestID(), 
Arrays.asList(allHeaders),
        requestCounter.incrementAndGet());
  }

}

public class NcResponseConsumer extends BasicAsyncResponseConsumer {

  private final NcResponse ncResponse;

  public NcResponseConsumer(NcResponse ncResponse) {
    this.ncResponse = ncResponse;
  }

  @Override
  protected void onResponseReceived(HttpResponse response) throws IOException {
    super.onResponseReceived(response);
    ncResponse.setEndTimeInNano(System.nanoTime());
    ncResponse.setReturnCode(response.getStatusLine().getStatusCode());
  }

}


-----Ursprüngliche Nachricht-----
Von: Bernd Eckenfels [mailto:e...@zusammenkunft.net] 
Gesendet: Dienstag, 4. Oktober 2016 21:19
An: Gnerlich, Manuel (IPS PROJECTS GMBH) <manuel.gnerl...@otto.de>
Cc: httpclient-users@hc.apache.org
Betreff: Re: Why are many TCP open connections created on OS (Linux)?

Am Tue, 4 Oct 2016 14:00:38 +0000
schrieb "Gnerlich, Manuel (IPS PROJECTS GMBH)"
<manuel.gnerl...@otto.de>:

> I am using Apache AsychHttpClient with connection pool to send more 
> than 1 mio requests and configured maxinum number of connection (200) 
> and of connection per Host (100).
> 
> However there are many open TCP connections (about 30.000) on linux.
> Why? How can I control number of connection over Java on linux?

Define open for us, please. Do you mean you see them as ESTABLISHED in netstat 
or do you mean you see them as socket handles for the Java process (in lsof). 
Do you mean on the client or the server?

Can you show some code how you set it up and use the client.

Gruss
Bernd

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org