[ https://issues.apache.org/jira/browse/HDFS-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13143369#comment-13143369 ]
Tsz Wo (Nicholas), SZE commented on HDFS-2535: ---------------------------------------------- +1 great work! > A Model for Data Durability > --------------------------- > > Key: HDFS-2535 > URL: https://issues.apache.org/jira/browse/HDFS-2535 > Project: Hadoop HDFS > Issue Type: Task > Components: documentation > Reporter: Robert Chansler > Assignee: Robert Chansler > Attachments: LosingBlocks.xlsx > > > This is a statistical model considering data durability in HDFS in the > presence of Data Node failures. The attached spreadsheet considers the > probability of losing a block with three replicas in the case of uncorrelated > failures of DNs. Also included is a section that looks at the consequences of > simultaneous failures. > The model parameters reflect experience at Yahoo with a large cluster. But it > is easy to change the parameters in the spreadsheet. Number of replicas is > not a easily adjusted parameter. And while published reports (the Google > papers) suggest that node failures are not really uncorrelated, this does > give some practical insight into HDFS durability. > I and others have quoted from this work in the past. I thought it good to > make the details conveniently available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira