[ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13525991#comment-13525991 ]
Vijay edited comment on CASSANDRA-4718 at 12/6/12 11:23 PM: ------------------------------------------------------------ How about Per thread queue? was curious about this ticket and ran a test and actually it performs better than Executors.newFixedThreadPool... with arg[0] = 100000 {code} Making JIT happy TimeTaken: 109 Real test starts! running per thread exec Test! TimeTaken: 8 running Plain old exec Test! TimeTaken: 82 Test with multiple of 10! TimeTaken: 152 running Plain old exec Test! TimeTaken: 334 Test with multiple of 100! TimeTaken: 48341 running Plain old exec Test! TimeTaken: 52195 {code} I have to also note that the stages content when we have a lot of CPU's (was testing with 64 core machines) was (Author: vijay2...@yahoo.com): How about Per thread queue? was curious about this ticket and ran a test and actually it performs better than Executors.newFixedThreadPool... {code} Making JIT happy TimeTaken: 120 Real test starts! running per thread exec Test! TimeTaken: 19 running Plain old exec Test! TimeTaken: 68 Test with multiple of 10! TimeTaken: 166 running Plain old exec Test! TimeTaken: 312 Test with multiple of 100! TimeTaken: 54381 running Plain old exec Test! TimeTaken: 55445 {code} I have to also note that the stages content when we have a lot of CPU's (was testing with 64 core machines) > More-efficient ExecutorService for improved throughput > ------------------------------------------------------ > > Key: CASSANDRA-4718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4718 > Project: Cassandra > Issue Type: Improvement > Reporter: Jonathan Ellis > Priority: Minor > Attachments: PerThreadQueue.java > > > Currently all our execution stages dequeue tasks one at a time. This can > result in contention between producers and consumers (although we do our best > to minimize this by using LinkedBlockingQueue). > One approach to mitigating this would be to make consumer threads do more > work in "bulk" instead of just one task per dequeue. (Producer threads tend > to be single-task oriented by nature, so I don't see an equivalent > opportunity there.) > BlockingQueue has a drainTo(collection, int) method that would be perfect for > this. However, no ExecutorService in the jdk supports using drainTo, nor > could I google one. > What I would like to do here is create just such a beast and wire it into (at > least) the write and read stages. (Other possible candidates for such an > optimization, such as the CommitLog and OutboundTCPConnection, are not > ExecutorService-based and will need to be one-offs.) > AbstractExecutorService may be useful. The implementations of > ICommitLogExecutorService may also be useful. (Despite the name these are not > actual ExecutorServices, although they share the most important properties of > one.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira