[jira] [Commented] (HTTPCORE-509) AVAIL_PROCS is autoconfigured based on core count
[
https://issues.apache.org/jira/browse/HTTPCORE-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357373#comment-16357373
]
Oleg Kalnichevski commented on HTTPCORE-509:
{quote}Now, I'm wondering if I should file an issue with HTTP Components Async
Client asking for a system parameter.
{quote}
@[~elijah] You could but you would end up talking to the same bunch of people
(us). There is no need to do anything about HttpAsyncClient. One can just
upgrade its HttpCore dependency.
Oleg
> AVAIL_PROCS is autoconfigured based on core count
> -
>
> Key: HTTPCORE-509
> URL: https://issues.apache.org/jira/browse/HTTPCORE-509
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore NIO
>Affects Versions: 4.4.9
>Reporter: Elijah Zupancic
>Priority: Major
>
> In [IOReactorConfig:39|
> https://github.com/apache/httpcomponents-core/blob/92c4c6ff64a9a4a0b0677338c03ebd24bfe1b219/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java#L39
> ], the default value for thread count is based on the detected cores on the
> system. This is usually acceptable. However, if the library is being run
> within a container where the reported number of CPUs does not match the
> actual amount of CPU available, you will see too many threads created in the
> pool leading to performance degradation.
> Of course a direct user of this library will be able to set this value to a
> sensible number. However, you will typically see httpcore-nio 4-5 steps down
> the dependency graph and it is being used by other open source components
> which in turn are being used by other components.
> I would suggest that we optionally read a system parameter for the value of
> AVAIL_PROCS. If that system parameter is not available, then we use the value
> of Runtime.getRuntime().availableProcessors();
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-509) AVAIL_PROCS is autoconfigured based on core count
[ https://issues.apache.org/jira/browse/HTTPCORE-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357368#comment-16357368 ] Oleg Kalnichevski commented on HTTPCORE-509: I personally could do without a getter but will not insist. Oleg > AVAIL_PROCS is autoconfigured based on core count > - > > Key: HTTPCORE-509 > URL: https://issues.apache.org/jira/browse/HTTPCORE-509 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore NIO >Affects Versions: 4.4.9 >Reporter: Elijah Zupancic >Priority: Major > > In [IOReactorConfig:39| > https://github.com/apache/httpcomponents-core/blob/92c4c6ff64a9a4a0b0677338c03ebd24bfe1b219/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java#L39 > ], the default value for thread count is based on the detected cores on the > system. This is usually acceptable. However, if the library is being run > within a container where the reported number of CPUs does not match the > actual amount of CPU available, you will see too many threads created in the > pool leading to performance degradation. > Of course a direct user of this library will be able to set this value to a > sensible number. However, you will typically see httpcore-nio 4-5 steps down > the dependency graph and it is being used by other open source components > which in turn are being used by other components. > I would suggest that we optionally read a system parameter for the value of > AVAIL_PROCS. If that system parameter is not available, then we use the value > of Runtime.getRuntime().availableProcessors(); -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-509) AVAIL_PROCS is autoconfigured based on core count
[
https://issues.apache.org/jira/browse/HTTPCORE-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357237#comment-16357237
]
Gary Gregory commented on HTTPCORE-509:
---
There are two new public APIs:
- {{IOReactorConfig.setDefaultMaxIoThreadCount(int)}}
- {{IOReactorConfig.getDefaultMaxIoThreadCount()}}
I do not see how to do with less. We could skip
{{IOReactorConfig.getDefaultMaxIoThreadCount()}} but it seems like a fail to
have a setter without a getter.
> AVAIL_PROCS is autoconfigured based on core count
> -
>
> Key: HTTPCORE-509
> URL: https://issues.apache.org/jira/browse/HTTPCORE-509
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore NIO
>Affects Versions: 4.4.9
>Reporter: Elijah Zupancic
>Priority: Major
>
> In [IOReactorConfig:39|
> https://github.com/apache/httpcomponents-core/blob/92c4c6ff64a9a4a0b0677338c03ebd24bfe1b219/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java#L39
> ], the default value for thread count is based on the detected cores on the
> system. This is usually acceptable. However, if the library is being run
> within a container where the reported number of CPUs does not match the
> actual amount of CPU available, you will see too many threads created in the
> pool leading to performance degradation.
> Of course a direct user of this library will be able to set this value to a
> sensible number. However, you will typically see httpcore-nio 4-5 steps down
> the dependency graph and it is being used by other open source components
> which in turn are being used by other components.
> I would suggest that we optionally read a system parameter for the value of
> AVAIL_PROCS. If that system parameter is not available, then we use the value
> of Runtime.getRuntime().availableProcessors();
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-509) AVAIL_PROCS is autoconfigured based on core count
[ https://issues.apache.org/jira/browse/HTTPCORE-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357225#comment-16357225 ] Elijah Zupancic commented on HTTPCORE-509: -- That patch looks good. As for my use case, a default value works fine. Now, I'm wondering if I should file an issue with HTTP Components Async Client asking for a system parameter. > AVAIL_PROCS is autoconfigured based on core count > - > > Key: HTTPCORE-509 > URL: https://issues.apache.org/jira/browse/HTTPCORE-509 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore NIO >Affects Versions: 4.4.9 >Reporter: Elijah Zupancic >Priority: Major > > In [IOReactorConfig:39| > https://github.com/apache/httpcomponents-core/blob/92c4c6ff64a9a4a0b0677338c03ebd24bfe1b219/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java#L39 > ], the default value for thread count is based on the detected cores on the > system. This is usually acceptable. However, if the library is being run > within a container where the reported number of CPUs does not match the > actual amount of CPU available, you will see too many threads created in the > pool leading to performance degradation. > Of course a direct user of this library will be able to set this value to a > sensible number. However, you will typically see httpcore-nio 4-5 steps down > the dependency graph and it is being used by other open source components > which in turn are being used by other components. > I would suggest that we optionally read a system parameter for the value of > AVAIL_PROCS. If that system parameter is not available, then we use the value > of Runtime.getRuntime().availableProcessors(); -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-509) AVAIL_PROCS is autoconfigured based on core count
[ https://issues.apache.org/jira/browse/HTTPCORE-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357215#comment-16357215 ] Oleg Kalnichevski commented on HTTPCORE-509: Works for me. Please do try to reduce API visible changes, though. Oleg > AVAIL_PROCS is autoconfigured based on core count > - > > Key: HTTPCORE-509 > URL: https://issues.apache.org/jira/browse/HTTPCORE-509 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore NIO >Affects Versions: 4.4.9 >Reporter: Elijah Zupancic >Priority: Major > > In [IOReactorConfig:39| > https://github.com/apache/httpcomponents-core/blob/92c4c6ff64a9a4a0b0677338c03ebd24bfe1b219/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java#L39 > ], the default value for thread count is based on the detected cores on the > system. This is usually acceptable. However, if the library is being run > within a container where the reported number of CPUs does not match the > actual amount of CPU available, you will see too many threads created in the > pool leading to performance degradation. > Of course a direct user of this library will be able to set this value to a > sensible number. However, you will typically see httpcore-nio 4-5 steps down > the dependency graph and it is being used by other open source components > which in turn are being used by other components. > I would suggest that we optionally read a system parameter for the value of > AVAIL_PROCS. If that system parameter is not available, then we use the value > of Runtime.getRuntime().availableProcessors(); -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-509) AVAIL_PROCS is autoconfigured based on core count
[ https://issues.apache.org/jira/browse/HTTPCORE-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357129#comment-16357129 ] Gary Gregory commented on HTTPCORE-509: --- Aside from picking a number like 4, how about starting with the patch https://issues.apache.org/jira/browse/HTTPCORE-509?focusedCommentId=16355607=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16355607 ? > AVAIL_PROCS is autoconfigured based on core count > - > > Key: HTTPCORE-509 > URL: https://issues.apache.org/jira/browse/HTTPCORE-509 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore NIO >Affects Versions: 4.4.9 >Reporter: Elijah Zupancic >Priority: Major > > In [IOReactorConfig:39| > https://github.com/apache/httpcomponents-core/blob/92c4c6ff64a9a4a0b0677338c03ebd24bfe1b219/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java#L39 > ], the default value for thread count is based on the detected cores on the > system. This is usually acceptable. However, if the library is being run > within a container where the reported number of CPUs does not match the > actual amount of CPU available, you will see too many threads created in the > pool leading to performance degradation. > Of course a direct user of this library will be able to set this value to a > sensible number. However, you will typically see httpcore-nio 4-5 steps down > the dependency graph and it is being used by other open source components > which in turn are being used by other components. > I would suggest that we optionally read a system parameter for the value of > AVAIL_PROCS. If that system parameter is not available, then we use the value > of Runtime.getRuntime().availableProcessors(); -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-509) AVAIL_PROCS is autoconfigured based on core count
[ https://issues.apache.org/jira/browse/HTTPCORE-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356997#comment-16356997 ] Oleg Kalnichevski commented on HTTPCORE-509: If you ask me I think 4 would be a pretty reasonable default for applications one has no control over. For all other cases where would be a static setter. Oleg > AVAIL_PROCS is autoconfigured based on core count > - > > Key: HTTPCORE-509 > URL: https://issues.apache.org/jira/browse/HTTPCORE-509 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore NIO >Affects Versions: 4.4.9 >Reporter: Elijah Zupancic >Priority: Major > > In [IOReactorConfig:39| > https://github.com/apache/httpcomponents-core/blob/92c4c6ff64a9a4a0b0677338c03ebd24bfe1b219/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java#L39 > ], the default value for thread count is based on the detected cores on the > system. This is usually acceptable. However, if the library is being run > within a container where the reported number of CPUs does not match the > actual amount of CPU available, you will see too many threads created in the > pool leading to performance degradation. > Of course a direct user of this library will be able to set this value to a > sensible number. However, you will typically see httpcore-nio 4-5 steps down > the dependency graph and it is being used by other open source components > which in turn are being used by other components. > I would suggest that we optionally read a system parameter for the value of > AVAIL_PROCS. If that system parameter is not available, then we use the value > of Runtime.getRuntime().availableProcessors(); -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-509) AVAIL_PROCS is autoconfigured based on core count
[
https://issues.apache.org/jira/browse/HTTPCORE-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356984#comment-16356984
]
Gary Gregory commented on HTTPCORE-509:
---
I am not sure there is a reasonable default. Even if we call something like
{{Thread.getAllStackTraces()}} to count threads, then what? That's only a
snapshot of what is happening now. It seems that in a larger application stack,
each library needs to provide enough configuration hooks in order to create a
profile for the whole stack. Only the user/developer once knowing the behavior
of that stack can decide who should allocate how much.
> AVAIL_PROCS is autoconfigured based on core count
> -
>
> Key: HTTPCORE-509
> URL: https://issues.apache.org/jira/browse/HTTPCORE-509
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore NIO
>Affects Versions: 4.4.9
>Reporter: Elijah Zupancic
>Priority: Major
>
> In [IOReactorConfig:39|
> https://github.com/apache/httpcomponents-core/blob/92c4c6ff64a9a4a0b0677338c03ebd24bfe1b219/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java#L39
> ], the default value for thread count is based on the detected cores on the
> system. This is usually acceptable. However, if the library is being run
> within a container where the reported number of CPUs does not match the
> actual amount of CPU available, you will see too many threads created in the
> pool leading to performance degradation.
> Of course a direct user of this library will be able to set this value to a
> sensible number. However, you will typically see httpcore-nio 4-5 steps down
> the dependency graph and it is being used by other open source components
> which in turn are being used by other components.
> I would suggest that we optionally read a system parameter for the value of
> AVAIL_PROCS. If that system parameter is not available, then we use the value
> of Runtime.getRuntime().availableProcessors();
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-509) AVAIL_PROCS is autoconfigured based on core count
[ https://issues.apache.org/jira/browse/HTTPCORE-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356664#comment-16356664 ] Oleg Kalnichevski commented on HTTPCORE-509: @[~elijah] Your arguments are very well articulated and are hard to disagree with. However making an exception to a rule makes it much harder to enforce in the future and I would hate to break our policy on the use of system properties for what is essentially a bug in other people's code. Could you live with a sensible default instead? Oleg > AVAIL_PROCS is autoconfigured based on core count > - > > Key: HTTPCORE-509 > URL: https://issues.apache.org/jira/browse/HTTPCORE-509 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore NIO >Affects Versions: 4.4.9 >Reporter: Elijah Zupancic >Priority: Major > > In [IOReactorConfig:39| > https://github.com/apache/httpcomponents-core/blob/92c4c6ff64a9a4a0b0677338c03ebd24bfe1b219/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java#L39 > ], the default value for thread count is based on the detected cores on the > system. This is usually acceptable. However, if the library is being run > within a container where the reported number of CPUs does not match the > actual amount of CPU available, you will see too many threads created in the > pool leading to performance degradation. > Of course a direct user of this library will be able to set this value to a > sensible number. However, you will typically see httpcore-nio 4-5 steps down > the dependency graph and it is being used by other open source components > which in turn are being used by other components. > I would suggest that we optionally read a system parameter for the value of > AVAIL_PROCS. If that system parameter is not available, then we use the value > of Runtime.getRuntime().availableProcessors(); -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-509) AVAIL_PROCS is autoconfigured based on core count
[ https://issues.apache.org/jira/browse/HTTPCORE-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356229#comment-16356229 ] Gary Gregory commented on HTTPCORE-509: --- I'd like [~olegk] to chime in here. If it were up to me, I would commit the above code-only solution and then discuss a property solution which would be layered on top of the code-only solution. > AVAIL_PROCS is autoconfigured based on core count > - > > Key: HTTPCORE-509 > URL: https://issues.apache.org/jira/browse/HTTPCORE-509 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore NIO >Affects Versions: 4.4.9 >Reporter: Elijah Zupancic >Priority: Major > > In [IOReactorConfig:39| > https://github.com/apache/httpcomponents-core/blob/92c4c6ff64a9a4a0b0677338c03ebd24bfe1b219/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java#L39 > ], the default value for thread count is based on the detected cores on the > system. This is usually acceptable. However, if the library is being run > within a container where the reported number of CPUs does not match the > actual amount of CPU available, you will see too many threads created in the > pool leading to performance degradation. > Of course a direct user of this library will be able to set this value to a > sensible number. However, you will typically see httpcore-nio 4-5 steps down > the dependency graph and it is being used by other open source components > which in turn are being used by other components. > I would suggest that we optionally read a system parameter for the value of > AVAIL_PROCS. If that system parameter is not available, then we use the value > of Runtime.getRuntime().availableProcessors(); -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-509) AVAIL_PROCS is autoconfigured based on core count
[ https://issues.apache.org/jira/browse/HTTPCORE-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356178#comment-16356178 ] Elijah Zupancic commented on HTTPCORE-509: -- Hello Oleg and Gary, A non-system property implementation is very clean from an API perspective, but it doesn't solve the problem for end users. For example, in the use case where I hit this problem, I was given a Tomcat application that was running a web application container (standard WAR distribution). This application depended on HTTP Components HTTP Aysnc Client which in turn depends on HTTP Core Components NIO. I don't have access to modify the source code for the WAR app, I don't have access to modify the source code for HTTP Async (although I could file a bug and ask them for a system parameter). I think this type of problem is quite common in Java applications. This is why you see Cassandra take an approach like: [https://github.com/apache/cassandra/blob/6fc3699adb4f609f78a40888af9797bb205216b0/src/java/org/apache/cassandra/utils/FBUtilities.java#L87] Netty took this approach: [https://github.com/netty/netty/blob/98beb777f81f092aa0fddf49ed08b426b2c72f01/transport/src/main/java/io/netty/channel/MultithreadEventLoopGroup.java#L40] This is a well documented problem with containers and the JVM: [https://github.com/fabianenardon/docker-java-issues-demo/tree/master/cpu-sample] The issue has been fixed in Java for Docker cgroups on Linux only: [https://bugs.openjdk.java.net/browse/JDK-8140793] However, there is no fix for Solaris zones nor FreeBSD jails. Also, if I recall correctly, even if you set processor affinity in Windows, the JVM will still detect all of the cores on the machine. >From a user perspective being able to pass a flag to the JVM is the only >reliable way to get applications built with a specific library to configure >their threads correctly when deploying an app on an older JVM or on a platform >that does CPU scheduling without cgroups. I also share your dislike for Java system properties. However, if there was an exception to never using them, this would be it. > AVAIL_PROCS is autoconfigured based on core count > - > > Key: HTTPCORE-509 > URL: https://issues.apache.org/jira/browse/HTTPCORE-509 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore NIO >Affects Versions: 4.4.9 >Reporter: Elijah Zupancic >Priority: Major > > In [IOReactorConfig:39| > https://github.com/apache/httpcomponents-core/blob/92c4c6ff64a9a4a0b0677338c03ebd24bfe1b219/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java#L39 > ], the default value for thread count is based on the detected cores on the > system. This is usually acceptable. However, if the library is being run > within a container where the reported number of CPUs does not match the > actual amount of CPU available, you will see too many threads created in the > pool leading to performance degradation. > Of course a direct user of this library will be able to set this value to a > sensible number. However, you will typically see httpcore-nio 4-5 steps down > the dependency graph and it is being used by other open source components > which in turn are being used by other components. > I would suggest that we optionally read a system parameter for the value of > AVAIL_PROCS. If that system parameter is not available, then we use the value > of Runtime.getRuntime().availableProcessors(); -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-509) AVAIL_PROCS is autoconfigured based on core count
[
https://issues.apache.org/jira/browse/HTTPCORE-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355607#comment-16355607
]
Gary Gregory commented on HTTPCORE-509:
---
The alternative I propose is then this:
{noformat}
diff --git
a/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java
b/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java
index 26fb11ea5..51d38e820 100644
---
a/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java
+++
b/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java
@@ -27,6 +27,9 @@
package org.apache.http.impl.nio.reactor;
+import java.security.AccessController;
+import java.security.PrivilegedAction;
+
import org.apache.http.util.Args;
/**
@@ -59,7 +62,7 @@ public IOReactorConfig() {
this.selectInterval = 1000;
this.shutdownGracePeriod = 500;
this.interestOpQueued = false;
-this.ioThreadCount = Runtime.getRuntime().availableProcessors();
+this.ioThreadCount = Builder.getDefaultMaxIoThreadCount();
this.soTimeout = 0;
this.soReuseAddress = false;
this.soLinger = -1;
@@ -376,6 +379,33 @@ public static Builder copy(final IOReactorConfig config) {
public static class Builder {
+private static int DefaultMaxIoThreadCount = -1;
+
+/**
+ * Gets the default value for {@code ioThreadCount}. Returns
+ * {@link Runtime#availableProcessors()} if
+ * {@link #setDefaultMaxIoThreadCount(int)} was called with a value
<=0.
+ *
+ * @return the default value for ioThreadCount.
+ * @since 4.4.10
+ */
+public static int getDefaultMaxIoThreadCount() {
+return DefaultMaxIoThreadCount > 0 ? DefaultMaxIoThreadCount :
Runtime.getRuntime().availableProcessors();
+}
+
+/**
+ * Sets the default value for {@code ioThreadCount}. Use a value <= 0
to
+ * cause {@link #getDefaultMaxIoThreadCount()} to return
+ * {@link Runtime#availableProcessors()}.
+ *
+ * @param defaultMaxIoThreadCount
+ *the default value for ioThreadCount.
+ * @since 4.4.10
+ */
+public static void setDefaultMaxIoThreadCount(int
defaultMaxIoThreadCount) {
+DefaultMaxIoThreadCount = defaultMaxIoThreadCount;
+}
+
private long selectInterval;
private long shutdownGracePeriod;
private boolean interestOpQueued;
@@ -394,7 +424,7 @@ public static Builder copy(final IOReactorConfig config) {
this.selectInterval = 1000;
this.shutdownGracePeriod = 500;
this.interestOpQueued = false;
-this.ioThreadCount = Runtime.getRuntime().availableProcessors();
+this.ioThreadCount = getDefaultMaxIoThreadCount();
this.soTimeout = 0;
this.soReuseAddress = false;
this.soLinger = -1;
{noformat}
> AVAIL_PROCS is autoconfigured based on core count
> -
>
> Key: HTTPCORE-509
> URL: https://issues.apache.org/jira/browse/HTTPCORE-509
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore NIO
>Affects Versions: 4.4.9
>Reporter: Elijah Zupancic
>Priority: Major
>
> In [IOReactorConfig:39|
> https://github.com/apache/httpcomponents-core/blob/92c4c6ff64a9a4a0b0677338c03ebd24bfe1b219/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java#L39
> ], the default value for thread count is based on the detected cores on the
> system. This is usually acceptable. However, if the library is being run
> within a container where the reported number of CPUs does not match the
> actual amount of CPU available, you will see too many threads created in the
> pool leading to performance degradation.
> Of course a direct user of this library will be able to set this value to a
> sensible number. However, you will typically see httpcore-nio 4-5 steps down
> the dependency graph and it is being used by other open source components
> which in turn are being used by other components.
> I would suggest that we optionally read a system parameter for the value of
> AVAIL_PROCS. If that system parameter is not available, then we use the value
> of Runtime.getRuntime().availableProcessors();
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-509) AVAIL_PROCS is autoconfigured based on core count
[ https://issues.apache.org/jira/browse/HTTPCORE-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355567#comment-16355567 ] Gary Gregory commented on HTTPCORE-509: --- Hi Oleg, That's fine with me and I like that better. We can layer a system property later if we really need to. I wonder if a code solution will work for the reporter of this issue. [~elijah] ? Gary > AVAIL_PROCS is autoconfigured based on core count > - > > Key: HTTPCORE-509 > URL: https://issues.apache.org/jira/browse/HTTPCORE-509 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore NIO >Affects Versions: 4.4.9 >Reporter: Elijah Zupancic >Priority: Major > > In [IOReactorConfig:39| > https://github.com/apache/httpcomponents-core/blob/92c4c6ff64a9a4a0b0677338c03ebd24bfe1b219/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java#L39 > ], the default value for thread count is based on the detected cores on the > system. This is usually acceptable. However, if the library is being run > within a container where the reported number of CPUs does not match the > actual amount of CPU available, you will see too many threads created in the > pool leading to performance degradation. > Of course a direct user of this library will be able to set this value to a > sensible number. However, you will typically see httpcore-nio 4-5 steps down > the dependency graph and it is being used by other open source components > which in turn are being used by other components. > I would suggest that we optionally read a system parameter for the value of > AVAIL_PROCS. If that system parameter is not available, then we use the value > of Runtime.getRuntime().availableProcessors(); -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-509) AVAIL_PROCS is autoconfigured based on core count
[ https://issues.apache.org/jira/browse/HTTPCORE-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355193#comment-16355193 ] Oleg Kalnichevski commented on HTTPCORE-509: Gary Up to this very point we have had a semi-official policy of not using system properties for configuration. I would like keep it that way if possible. Would a static setter for default value also be OK for you? Oleg > AVAIL_PROCS is autoconfigured based on core count > - > > Key: HTTPCORE-509 > URL: https://issues.apache.org/jira/browse/HTTPCORE-509 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore NIO >Affects Versions: 4.4.9 >Reporter: Elijah Zupancic >Priority: Major > > In [IOReactorConfig:39| > https://github.com/apache/httpcomponents-core/blob/92c4c6ff64a9a4a0b0677338c03ebd24bfe1b219/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java#L39 > ], the default value for thread count is based on the detected cores on the > system. This is usually acceptable. However, if the library is being run > within a container where the reported number of CPUs does not match the > actual amount of CPU available, you will see too many threads created in the > pool leading to performance degradation. > Of course a direct user of this library will be able to set this value to a > sensible number. However, you will typically see httpcore-nio 4-5 steps down > the dependency graph and it is being used by other open source components > which in turn are being used by other components. > I would suggest that we optionally read a system parameter for the value of > AVAIL_PROCS. If that system parameter is not available, then we use the value > of Runtime.getRuntime().availableProcessors(); -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-509) AVAIL_PROCS is autoconfigured based on core count
[
https://issues.apache.org/jira/browse/HTTPCORE-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354471#comment-16354471
]
Gary Gregory commented on HTTPCORE-509:
---
[~olegk]: What do you think about addressing this ticket as follows:
{noformat}
diff --git
a/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java
b/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java
index 26fb11ea5..ecda462e5 100644
---
a/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java
+++
b/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java
@@ -27,6 +27,9 @@
package org.apache.http.impl.nio.reactor;
+import java.security.AccessController;
+import java.security.PrivilegedAction;
+
import org.apache.http.util.Args;
/**
@@ -36,6 +39,19 @@
*/
public final class IOReactorConfig implements Cloneable {
+static int getIntProperty(final String key, final int defaultValue) {
+return AccessController.doPrivileged(new PrivilegedAction() {
+@Override
+public Integer run() {
+final String strValue= System.getProperty(key);
+if (strValue == null) {
+return Integer.valueOf(defaultValue);
+}
+return Integer.valueOf(strValue);
+}
+});
+}
+
public static final IOReactorConfig DEFAULT = new Builder().build();
// TODO: make final
@@ -59,7 +75,9 @@ public IOReactorConfig() {
this.selectInterval = 1000;
this.shutdownGracePeriod = 500;
this.interestOpQueued = false;
-this.ioThreadCount = Runtime.getRuntime().availableProcessors();
+this.ioThreadCount = Args.positive(
+getIntProperty("http.IOReactorConfig.ioThreadCount",
Runtime.getRuntime().availableProcessors()),
+"ioThreadCount");
this.soTimeout = 0;
this.soReuseAddress = false;
this.soLinger = -1;
@@ -394,7 +412,9 @@ public static Builder copy(final IOReactorConfig config) {
this.selectInterval = 1000;
this.shutdownGracePeriod = 500;
this.interestOpQueued = false;
-this.ioThreadCount = Runtime.getRuntime().availableProcessors();
+this.ioThreadCount = Args.positive(
+getIntProperty("http.IOReactorConfig.ioThreadCount",
Runtime.getRuntime().availableProcessors()),
+"ioThreadCount");
this.soTimeout = 0;
this.soReuseAddress = false;
this.soLinger = -1;
{noformat}
?
> AVAIL_PROCS is autoconfigured based on core count
> -
>
> Key: HTTPCORE-509
> URL: https://issues.apache.org/jira/browse/HTTPCORE-509
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore NIO
>Affects Versions: 4.4.9
>Reporter: Elijah Zupancic
>Priority: Major
>
> In [IOReactorConfig:39|
> https://github.com/apache/httpcomponents-core/blob/92c4c6ff64a9a4a0b0677338c03ebd24bfe1b219/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java#L39
> ], the default value for thread count is based on the detected cores on the
> system. This is usually acceptable. However, if the library is being run
> within a container where the reported number of CPUs does not match the
> actual amount of CPU available, you will see too many threads created in the
> pool leading to performance degradation.
> Of course a direct user of this library will be able to set this value to a
> sensible number. However, you will typically see httpcore-nio 4-5 steps down
> the dependency graph and it is being used by other open source components
> which in turn are being used by other components.
> I would suggest that we optionally read a system parameter for the value of
> AVAIL_PROCS. If that system parameter is not available, then we use the value
> of Runtime.getRuntime().availableProcessors();
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-509) AVAIL_PROCS is autoconfigured based on core count
[ https://issues.apache.org/jira/browse/HTTPCORE-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16353997#comment-16353997 ] Gary Gregory commented on HTTPCORE-509: --- Also, our storing this value in a static does not allow for us adapting to the value changing... > AVAIL_PROCS is autoconfigured based on core count > - > > Key: HTTPCORE-509 > URL: https://issues.apache.org/jira/browse/HTTPCORE-509 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore NIO >Affects Versions: 4.4.9 >Reporter: Elijah Zupancic >Priority: Major > > In [IOReactorConfig:39| > https://github.com/apache/httpcomponents-core/blob/92c4c6ff64a9a4a0b0677338c03ebd24bfe1b219/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java#L39 > ], the default value for thread count is based on the detected cores on the > system. This is usually acceptable. However, if the library is being run > within a container where the reported number of CPUs does not match the > actual amount of CPU available, you will see too many threads created in the > pool leading to performance degradation. > Of course a direct user of this library will be able to set this value to a > sensible number. However, you will typically see httpcore-nio 4-5 steps down > the dependency graph and it is being used by other open source components > which in turn are being used by other components. > I would suggest that we optionally read a system parameter for the value of > AVAIL_PROCS. If that system parameter is not available, then we use the value > of Runtime.getRuntime().availableProcessors(); -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-509) AVAIL_PROCS is autoconfigured based on core count
[ https://issues.apache.org/jira/browse/HTTPCORE-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16353359#comment-16353359 ] Elijah Zupancic commented on HTTPCORE-509: -- Yes, exactly something like that. > AVAIL_PROCS is autoconfigured based on core count > - > > Key: HTTPCORE-509 > URL: https://issues.apache.org/jira/browse/HTTPCORE-509 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore NIO >Affects Versions: 4.4.9 >Reporter: Elijah Zupancic >Priority: Major > > In [IOReactorConfig:39| > https://github.com/apache/httpcomponents-core/blob/92c4c6ff64a9a4a0b0677338c03ebd24bfe1b219/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java#L39 > ], the default value for thread count is based on the detected cores on the > system. This is usually acceptable. However, if the library is being run > within a container where the reported number of CPUs does not match the > actual amount of CPU available, you will see too many threads created in the > pool leading to performance degradation. > Of course a direct user of this library will be able to set this value to a > sensible number. However, you will typically see httpcore-nio 4-5 steps down > the dependency graph and it is being used by other open source components > which in turn are being used by other components. > I would suggest that we optionally read a system parameter for the value of > AVAIL_PROCS. If that system parameter is not available, then we use the value > of Runtime.getRuntime().availableProcessors(); -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-509) AVAIL_PROCS is autoconfigured based on core count
[
https://issues.apache.org/jira/browse/HTTPCORE-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16353353#comment-16353353
]
Gary Gregory commented on HTTPCORE-509:
---
Do you mean something like {{java ...
-Dorg.apache.http.impl.nio.reactor.IOReactorConfig.ioThreadCount=3 ...}}
> AVAIL_PROCS is autoconfigured based on core count
> -
>
> Key: HTTPCORE-509
> URL: https://issues.apache.org/jira/browse/HTTPCORE-509
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore NIO
>Affects Versions: 4.4.9
>Reporter: Elijah Zupancic
>Priority: Major
>
> In [IOReactorConfig:39|
> https://github.com/apache/httpcomponents-core/blob/92c4c6ff64a9a4a0b0677338c03ebd24bfe1b219/httpcore-nio/src/main/java/org/apache/http/impl/nio/reactor/IOReactorConfig.java#L39
> ], the default value for thread count is based on the detected cores on the
> system. This is usually acceptable. However, if the library is being run
> within a container where the reported number of CPUs does not match the
> actual amount of CPU available, you will see too many threads created in the
> pool leading to performance degradation.
> Of course a direct user of this library will be able to set this value to a
> sensible number. However, you will typically see httpcore-nio 4-5 steps down
> the dependency graph and it is being used by other open source components
> which in turn are being used by other components.
> I would suggest that we optionally read a system parameter for the value of
> AVAIL_PROCS. If that system parameter is not available, then we use the value
> of Runtime.getRuntime().availableProcessors();
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
