[GitHub] activemq-artemis pull request #2484: Use a specific executor for pageSyncTim...

[qihongxu]

GitHub user qihongxu opened a pull request:

    https://github.com/apache/activemq-artemis/pull/2484

    Use a specific executor for pageSyncTimer

    Improve paging throughput by using a specific executor for pageSyncTimer
    
    Improving throughput on paging mode is one of our concerns since our 
cluster uses paging a lot.
    We found that pageSyncTimer in PagingStoreImpl shared the same executor 
with pageCursorProvider from thread pool. In heavy load scenario like hundreds 
of consumers receiving messages simultaneously, it became difficult for 
pageSyncTimer to get the executor due to race condition. Therefore page sync 
was delayed and producers suffered low throughput.
    
    To achieve higher performance we assign a specific executor to 
pageSyncTimer to avoid racing. And we run a small-scale test on a single 
modified broker.
    
    Broker: 4C/8G/500G SSD
    Producer: 200 threads, non-transactional send
    Consumer 200 threads, transactional receive
    Message text size: 100-200 bytes randomly
    AddressFullPolicy: PAGE
    
    Test result：
    
      | Only Send TPS | Only Receive TPS | Send&Receive TPS
    -- | -- | -- | --
    Original ver | 38k | 33k | 3k/30k
    Modified ver | 38k | 34k | 30k/12.5k
    
    
    The chart above shows that on modified broker send TPS improves from 
“poor” to “extremely fast”, while receive TPS drops from “extremely 
fast” to “not-bad” under heavy load. Considering consumer systems usually 
have a long processing chain after receiving messages, we don’t need too fast 
receive TPS. Instead, we want to guarantee send TPS to cope with traffic peak 
and lower producer’s delay time. Moreover, send and receive TPS in total 
raises from 33k to about 43k. From all above this trade-off seems beneficial 
and acceptable.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/qihongxu/activemq-artemis 
pageSyncTimer_executor

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/activemq-artemis/pull/2484.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2484
    
----
commit 01a09f2f2bee98643df06a4eb93588047fea6527
Author: Qihong Xu <qihongxu@...>
Date:   2018-12-28T11:59:41Z

    Use a specific executor for pageSyncTimer

----


---