[ https://issues.apache.org/jira/browse/ZOOKEEPER-1504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13526976#comment-13526976 ]
Hadoop QA commented on ZOOKEEPER-1504: -------------------------------------- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12560001/ZOOKEEPER-1504.patch against trunk revision 1418555. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1289//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1289//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1289//console This message is automatically generated. > Multi-thread NIOServerCnxn > -------------------------- > > Key: ZOOKEEPER-1504 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1504 > Project: ZooKeeper > Issue Type: Improvement > Components: server > Affects Versions: 3.4.3, 3.4.4, 3.5.0 > Reporter: Jay Shrauner > Assignee: Jay Shrauner > Labels: performance, scaling > Fix For: 3.5.0 > > Attachments: ZOOKEEPER-1504.patch, ZOOKEEPER-1504.patch, > ZOOKEEPER-1504.patch, ZOOKEEPER-1504.patch, ZOOKEEPER-1504.patch > > > NIOServerCnxnFactory is single threaded, which doesn't scale well to large > numbers of clients. This is particularly noticeable when thousands of clients > connect. I propose multi-threading this code as follows: > - 1 acceptor thread, for accepting new connections > - 1-N selector threads > - 0-M I/O worker threads > Numbers of threads are configurable, with defaults scaling according to > number of cores. Communication with the selector threads is handled via > LinkedBlockingQueues, and connections are permanently assigned to a > particular selector thread so that all potentially blocking SelectionKey > operations can be performed solely by the selector thread. An ExecutorService > is used for the worker threads. > On a 32 core machine running Linux 2.6.38, achieved best performance with 4 > selector threads and 64 worker threads for a 70% +/- 5% improvement in > throughput. > This patch incorporates and supersedes the patches for > https://issues.apache.org/jira/browse/ZOOKEEPER-517 > https://issues.apache.org/jira/browse/ZOOKEEPER-1444 > New classes introduced in this patch are: > - ExpiryQueue (from ZOOKEEPER-1444): factor out the logic from > SessionTrackerImpl used to expire sessions so that the same logic can be used > to expire connections > - RateLogger (from ZOOKEEPER-517): rate limit error message logging, > currently only used to throttle rate of logging "out of file descriptors" > errors > - WorkerService (also in ZOOKEEPER-1505): ExecutorService wrapper that > makes worker threads daemon threads and names then in an easily debuggable > manner. Supports assignable threads (as used by CommitProcessor) and > non-assignable threads (as used here). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira