[
https://issues.apache.org/jira/browse/HBASE-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13865014#comment-13865014
]
Liang Xie commented on HBASE-10278:
-----------------------------------
nice design, especially considering MTTR! +100 :) very impressive about the
table under "Switch Threshold vs N/W Hiccup" section!
bq. Its not taking any writes, so no throttling is required. If the hiccup
stays for long, the new WAL might switch too. In future, we could use some
heuristic to monitor switches (current WAL size, last switch time, etc).
To me, i think it would be better if we have a throttling config. e.g. there is
a long rack level n/w outage, then lots of RS's log switchings will put not low
pressure to NN. and those tiny logs seems unhappy to every hdfs ops:)
In the section "Cost of HLog Rolling’s Open / Close with data", i can't
understand the results for "Open/1k write/Close 1000 files concurrently: ~300ms"
Results:
● Open/1k write/Close 1 file: ~340 ms (avg).
● Open/1k write/Close 1000 files concurrently: ~300ms
○ 4 sec; 568ops takes > 1sec (2,3,4 sec)
○ 56.8%tile is >1sec
the "~300ms" is avg ??? why it's smaller than "1 file" scenario ?
could you guys add more detailed stuff about how to handle the 2+ opening log
files on replication path ?
> Provide better write predictability
> -----------------------------------
>
> Key: HBASE-10278
> URL: https://issues.apache.org/jira/browse/HBASE-10278
> Project: HBase
> Issue Type: New Feature
> Reporter: Himanshu Vashishtha
> Assignee: Himanshu Vashishtha
> Attachments: Multiwaldesigndoc.pdf
>
>
> Currently, HBase has one WAL per region server.
> Whenever there is any latency in the write pipeline (due to whatever reasons
> such as n/w blip, a node in the pipeline having a bad disk, etc), the overall
> write latency suffers.
> Jonathan Hsieh and I analyzed various approaches to tackle this issue. We
> also looked at HBASE-5699, which talks about adding concurrent multi WALs.
> Along with performance numbers, we also focussed on design simplicity,
> minimum impact on MTTR & Replication, and compatibility with 0.96 and 0.98.
> Considering all these parameters, we propose a new HLog implementation with
> WAL Switching functionality.
> Please find attached the design doc for the same. It introduces the WAL
> Switching feature, and experiments/results of a prototype implementation,
> showing the benefits of this feature.
> The second goal of this work is to serve as a building block for concurrent
> multiple WALs feature.
> Please review the doc.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
