[
https://issues.apache.org/jira/browse/HBASE-14540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941335#comment-14941335
]
Elliott Clark commented on HBASE-14540:
---------------------------------------
To expand a bit more this is the classic throughput vs response time trade off.
We already have the smart batching method described in the linked article. If
requests come in bunches we'll sync fewer times with larger payloads. This just
adds more wait time.
The attached class will wait longer than the described smart batching
algorithm. That will increase throughput as it will mean less round trips to
HDFS. However since on average everything will twice for while going through
the ring it will increase latency. For clusters that optimize for throughput (
benchmarks and olap based workloads ) that's a good trade off. For people with
an online workload this will set a baseline that they can't go faster than 2ms
on average.
> Write Ahead Log Batching Optimization
> -------------------------------------
>
> Key: HBASE-14540
> URL: https://issues.apache.org/jira/browse/HBASE-14540
> Project: HBase
> Issue Type: Improvement
> Reporter: John Leach
> Attachments: HBaseWALBlockingWaitStrategy.java
>
>
> The new write ahead log mechanism seems to batch too few mutations when
> running inside the disruptor. As we scaled our load up (many threads with
> small writes), we saw the number of hdfs sync operations grow in concert with
> the number of writes. Generally, one would expect the size of the batches to
> grow but the number of actual sync operations to settle.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
