[
https://issues.apache.org/jira/browse/HTTPCORE-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16356178#comment-16356178
]
Elijah Zupancic commented on HTTPCORE-509:
------------------------------------------
Hello Oleg and Gary,
A non-system property implementation is very clean from an API perspective, but
it doesn't solve the problem for end users. For example, in the use case where
I hit this problem, I was given a Tomcat application that was running a web
application container (standard WAR distribution). This application depended on
HTTP Components HTTP Aysnc Client which in turn depends on HTTP Core Components
NIO. I don't have access to modify the source code for the WAR app, I don't
have access to modify the source code for HTTP Async (although I could file a
bug and ask them for a system parameter). I think this type of problem is quite
common in Java applications. This is why you see Cassandra take an approach
like:
[https://github.com/apache/cassandra/blob/6fc3699adb4f609f78a40888af9797bb205216b0/src/java/org/apache/cassandra/utils/FBUtilities.java#L87]
Netty took this approach:
[https://github.com/netty/netty/blob/98beb777f81f092aa0fddf49ed08b426b2c72f01/transport/src/main/java/io/netty/channel/MultithreadEventLoopGroup.java#L40]
This is a well documented problem with containers and the JVM:
[https://github.com/fabianenardon/docker-java-issues-demo/tree/master/cpu-sample]
The issue has been fixed in Java for Docker cgroups on Linux only:
[https://bugs.openjdk.java.net/browse/JDK-8140793]
However, there is no fix for Solaris zones nor FreeBSD jails. Also, if I recall
correctly, even if you set processor affinity in Windows, the JVM will still
detect all of the cores on the machine.
>From a user perspective being able to pass a flag to the JVM is the only
>reliable way to get applications built with a specific library to configure
>their threads correctly when deploying an app on an older JVM or on a platform
>that does CPU scheduling without cgroups.
I also share your dislike for Java system properties. However, if there was an
exception to never using them, this would be it.
> AVAIL_PROCS is autoconfigured based on core count
> -------------------------------------------------
>
> Key: HTTPCORE-509
> URL: https://issues.apache.org/jira/browse/HTTPCORE-509
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore NIO
> Affects Versions: 4.4.9
> Reporter: Elijah Zupancic
> Priority: Major
>
> In [IOReactorConfig:39|
> https://github.com/apache/httpcomponents-core/blob/92c4c6ff64a9a4a0b0677338c03ebd24bfe1b219/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java#L39
> ], the default value for thread count is based on the detected cores on the
> system. This is usually acceptable. However, if the library is being run
> within a container where the reported number of CPUs does not match the
> actual amount of CPU available, you will see too many threads created in the
> pool leading to performance degradation.
> Of course a direct user of this library will be able to set this value to a
> sensible number. However, you will typically see httpcore-nio 4-5 steps down
> the dependency graph and it is being used by other open source components
> which in turn are being used by other components.
> I would suggest that we optionally read a system parameter for the value of
> AVAIL_PROCS. If that system parameter is not available, then we use the value
> of Runtime.getRuntime().availableProcessors();
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
