Hi all, Just an update on this project: - The current list of uncommitted patches up for review is:
1bea9d3 HDFS-1979. Fix BackupNode and CheckpointNode 32db384 Amend HDFS-2011. Fix TestCheckpoint test for double close/abort of ELFOS b6a55a4 HDFS-2101. Update remaining unit tests for new layout ca0ace6 HDFS-2133. Address TODOs left in code b46825d HDFS-1780. reduce need to rewrite fsimage on statrtup 30c858d HDFS-2104. Add flag to SecondaryNameNode to format it during startup 942eaef HDFS-2135. Fix regression of HDFS-1955 in branch I believe Eli is going to work on reviewing these this week. - I've set up a Hudson job for the branch here: https://builds.apache.org/job/Hadoop-Hdfs-1073-branch/ It's currently failing because it's missing some of the patches above. After the above patches go in, I expect a pretty clean build, modulo maybe one or two things that are environment issues, which I'll tackle later this week. - BackupNode and CheckpointNode are working. I've done some basic functional testing by pounding edits into the NN while both a 2NN and a BN are checkpointing every 2 seconds. - I merged with trunk as of this morning, so I think we should be up-to-date with trunk patches. Aaron was very helpful and went through all NN-related patches in trunk from the last 3 months to make sure we didn't inadvertently regress anything - he discovered one bug but everything else looks good. Once the above patches are in the branch, I would like to merge. So, if you plan on reviewing pre-merge, please do so *this week*. Of course, if you don't have time and you find issues post-merge, I absolutely plan on fixing them ASAP ;-) Thanks -Todd On Thu, Jun 30, 2011 at 12:11 AM, Todd Lipcon <t...@cloudera.com> wrote: > Hey all, > > Work on the HDFS-1073 branch has been progressing steadily, and I believe > we're coming close to the point where it can be merged. To briefly summarize > the status: > - NameNode and SecondaryNameNode are both fully working and have undergone > some stress/fault testing in addition to a over 3000 lines worth of new unit > tests. > - Most of the existing unit tests have been updated, though a few more need > some small tweaks (HDFS-2101) > - The BackupNode and CheckpointNode are not currently working, though I am > working on it locally and making good progress (HDFS-1979) > - There are a few various and sundry small improvements that should > probably be done before release, but I think could be done either before or > after merge (eg HDFS-2104) > > Given this, I am expecting that we can merge this into trunk by the end of > July if not earlier, as soon as the BN/CN work is complete. If you are > hoping to review the code or tests before merge time, this is your early > warning! Please do so now! > > Thanks! > > -Todd > P.S. I will also be giving a short talk about the motivations and current > status of this project at Friday's contributor meeting, for those who are > able to attend. If we're lucky, maybe even a demo! > -- > Todd Lipcon > Software Engineer, Cloudera > -- Todd Lipcon Software Engineer, Cloudera
