[ https://issues.apache.org/jira/browse/HBASE-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13863843#comment-13863843 ]
Sergey Shelukhin commented on HBASE-10278: ------------------------------------------ Skimmed the doc, looks really nice. I do think that out-of-order WAL should eventually become ok (we will get per-region mvcc from seqId-mvcc merge, and mvcc in WAL from this or several other jiras). One thing I might have missed - since it currently requires log rolling, would it need throttling for switching? If there's a long sequence of network hiccups from the machine (i.e. to both files), it might roll lots of tiny logs. > Provide better write predictability > ----------------------------------- > > Key: HBASE-10278 > URL: https://issues.apache.org/jira/browse/HBASE-10278 > Project: HBase > Issue Type: New Feature > Reporter: Himanshu Vashishtha > Assignee: Himanshu Vashishtha > Attachments: Multiwaldesigndoc.pdf > > > Currently, HBase has one WAL per region server. > Whenever there is any latency in the write pipeline (due to whatever reasons > such as n/w blip, a node in the pipeline having a bad disk, etc), the overall > write latency suffers. > Jonathan Hsieh and I analyzed various approaches to tackle this issue. We > also looked at HBASE-5699, which talks about adding concurrent multi WALs. > Along with performance numbers, we also focussed on design simplicity, > minimum impact on MTTR & Replication, and compatibility with 0.96 and 0.98. > Considering all these parameters, we propose a new HLog implementation with > WAL Switching functionality. > Please find attached the design doc for the same. It introduces the WAL > Switching feature, and experiments/results of a prototype implementation, > showing the benefits of this feature. > The second goal of this work is to serve as a building block for concurrent > multiple WALs feature. > Please review the doc. -- This message was sent by Atlassian JIRA (v6.1.5#6160)