[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12730481#action_12730481 ]
Flavio Paiva Junqueira commented on HDFS-234: --------------------------------------------- When you read from a BookKeeper ledger, it seems that you are reading the whole ledger before processing it. I can see two potential problems with it: # If the ledger is large, then the process reading the ledger might end up consuming too much memory; # It does not overlap reading the ledger with processing. It might be best to try to overlap reading from the ledger with processing the edits. I don't know what changes this suggestion implies, though. > Integration with BookKeeper logging system > ------------------------------------------ > > Key: HDFS-234 > URL: https://issues.apache.org/jira/browse/HDFS-234 > Project: Hadoop HDFS > Issue Type: New Feature > Reporter: Luca Telloli > Assignee: Luca Telloli > Attachments: create.png, HADOOP-5189-trunk-preview.patch, > HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, > HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.patch, > zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar > > > BookKeeper is a system to reliably log streams of records > (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a > natural target for such a system for being the metadata repository of the > entire file system for HDFS. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.