High Availability Framework for HDFS Namenode in 2.0.0
It sounds like an exciting feature. Does anyone have tried this in practice? How does the hot standby namenode perform and how reliable is the HDFS recovery? Is it now a good chance to migrate to 2.0.0, in your opinions? Best, Shi
Re: High Availability Framework for HDFS Namenode in 2.0.0
Hey Shi Yu, Some questions of yours are answered at this comment: https://issues.apache.org/jira/browse/HDFS-1623?focusedCommentId=13215309page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13215309 (and below) and at was tracked at https://issues.apache.org/jira/browse/HDFS-3010 The version of HA available already does work reasonably well, although my own personal tests with DFS+HBase were local and not long-running (as they may call it, non-prod). Unsure on what you mean by recovery exactly. However, 2.0.0 is a release yet to be prepared, but should come out soon per recent activity on the general lists (branch-2 is your best shot for now for the latest usable HDFS). Docs for available HA in 0.23.1 is available in the tar, or https://issues.apache.org/jira/secure/attachment/12513484/HDFSHighAvailability.html. Doesn't look like it was compiled/linked onto the 0.23.1 live docs, though it was present. Do give it a whirl and let us know! Also checkout https://issues.apache.org/jira/browse/HDFS-3042 for the remaining part of failover work. On Thu, May 3, 2012 at 7:58 PM, Shi Yu sh...@uchicago.edu wrote: It sounds like an exciting feature. Does anyone have tried this in practice? How does the hot standby namenode perform and how reliable is the HDFS recovery? Is it now a good chance to migrate to 2.0.0, in your opinions? Best, Shi -- Harsh J
Re: High Availability Framework for HDFS Namenode in 2.0.0
Hi Harsh J, It seems that the 20% performance lost is not that bad, at least some smart people are still working to improve it. I will keep an eye on this interesting trend. Shi
Re: High Availability Framework for HDFS Namenode in 2.0.0
Hi Shi, The 20% regression was prior to implementing a few optimizations on the branch. Here's the later comment: https://issues.apache.org/jira/browse/HDFS-1623?focusedCommentId=13218813page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13218813 Also, the 20% measurement was on a particular stress test designed to target exactly the metadata code path we figured there might be performance impact. So on typical clusters with typical workloads, you shouldn't find any measurable difference. -Todd On Thu, May 3, 2012 at 8:38 AM, Shi Yu sh...@uchicago.edu wrote: Hi Harsh J, It seems that the 20% performance lost is not that bad, at least some smart people are still working to improve it. I will keep an eye on this interesting trend. Shi -- Todd Lipcon Software Engineer, Cloudera
Re: High Availability Framework for HDFS Namenode in 2.0.0
Hi Todd, Okay, that sounds really good (sorry didn't grab all the information in that long page). Shi
