With 4 binding +1 votes, the vote to merge the branch HDFS-2832 into trunk passes. The code will be merged soon. We will address any remaining issues in trunk.
Thanks to everyone who voted and provided their feedback. Regards, Arpit On Mon, Dec 9, 2013 at 10:51 AM, Andrew Wang <andrew.w...@cloudera.com>wrote: > Thanks for clarifying that Arpit. I'm a +0.9 since I haven't reviewed > enough to +1, but everything thus far looks great. > > Andrew > > > On Fri, Dec 6, 2013 at 5:35 PM, Chen He <airb...@gmail.com> wrote: > > > +1 nice feature for HDFS > > > > > > On Fri, Dec 6, 2013 at 7:32 PM, Arpit Agarwal <aagar...@hortonworks.com > > >wrote: > > > > > Hi Andrew, > > > > > > Our plan as stated back in August was to do this work principally in > two > > > phases. > > > > > > https://issues.apache.org/jira/browse/HDFS-2832?focusedCommentId=13739041 > > > > > > For the second phase which includes API support, we also need quota > > > management. For changes of this scope, to do all the work at once while > > > keeping the feature branch in sync with ongoing development in trunk is > > > unmanageable. Hence we'd like to stick with the initial plan and > develop > > in > > > phases. > > > > > > Even for datanode caching the initial merge did not include the quota > > > management changes which are happening subsequently. > > > > > > Going forward, we will stabilize the current changes in trunk in the > 2.4 > > > time frame. Next we will add quota management and API support which can > > > align with the 2.5 time frame, with the second merge potentially in > > > March/April. > > > > > > Arpit > > > > > > > > > On Fri, Dec 6, 2013 at 3:15 PM, Andrew Wang <andrew.w...@cloudera.com > > > >wrote: > > > > > > > Hi everyone, > > > > > > > > I'm still getting up to speed on the changes here (my fault for not > > > > following development more closely, other priorities etc etc), but > the > > > > branch thus far is already quite impressive. It's quite an > undertaking > > to > > > > turn the DN into a collection of Storages, along with the > corresponding > > > > datastructure, tracking, and other changes in the NN and DN. > > > > > > > > Correct me if I'm wrong though, but this still leaves a substantial > > part > > > of > > > > the design doc to be implemented. Looking at the list of remaining > > > > subtasks, it seems like we still can't specify a storage type for a > > file > > > > (HDFS-5229) or write a file to a given storage type (HDFS-5391), > along > > > with > > > > the corresponding client protocol changes. This leads me to two > > > questions: > > > > > > > > - If this is merged, what can I do with the new code? Without client > > > > changes or the ability to create a file on a different storage type, > I > > > > don't know how (for example) I could hand this to our QA team to > test. > > > I'm > > > > wondering why we want to merge now rather than when the branch is > more > > > > feature complete. > > > > - What's the plan for the implementation of the remaining features? > How > > > > many phases? What's the timeline for these phases? Particularly, > > related > > > to > > > > the use cases presented in section 2 of the design doc. > > > > > > > > I'm also going to post some design doc questions to the JIRA, there > > are a > > > > few technical q's I'd like to get clarification on. > > > > > > > > Thanks, > > > > Andrew > > > > > > > > > > > > On Wed, Dec 4, 2013 at 7:21 AM, Sirianni, Eric < > > eric.siria...@netapp.com > > > > >wrote: > > > > > > > > > +1 > > > > > > > > > > My team has been developing and testing against the HDFS-2832 > branch > > > for > > > > > the past month. It has proven to be quite stable. > > > > > > > > > > Eric > > > > > > > > > > -----Original Message----- > > > > > From: Arpit Agarwal [mailto:aagar...@hortonworks.com] > > > > > Sent: Monday, December 02, 2013 7:07 PM > > > > > To: hdfs-...@hadoop.apache.org; common-dev@hadoop.apache.org > > > > > Subject: [VOTE] Merge HDFS-2832 Heterogeneous Storage Phase 1 to > > trunk > > > > > > > > > > Hello all, > > > > > > > > > > I would like to call a vote to merge phase 1 of the Heterogeneous > > > Storage > > > > > feature into trunk. > > > > > > > > > > *Scope of the changes:* > > > > > The changes allow exposing the DataNode as a collection of storages > > and > > > > set > > > > > the foundation for subsequent work to present Heterogeneous > Storages > > to > > > > > applications. This allows DataNodes to send block and storage > reports > > > > > per-storage. In addition this change introduces the ability to add > a > > > > > 'storage type' tag to the storage directories. This enables > > supporting > > > > > different types of storages in addition to disk storage. > > > > > > > > > > Development of the feature is tracked in the jira > > > > > https://issues.apache.org/jira/browse/HDFS-2832. > > > > > > > > > > *Details of development and testing:* > > > > > Development has been done in a separate branch - > > > > > https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832. > > The > > > > > updated design is posted at - > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/attachment/12615761/20131125-HeterogeneousStorage.pdf > > > > > . > > > > > The changes involve ~6K changed lines of code, with a third of > those > > > > > changes being to tests. > > > > > > > > > > Please see the test plan > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/attachment/12616642/20131202-HeterogeneousStorage-TestPlan.pdffor > > > > > the details. Once the feature is > > > > > merged into trunk, we will continue to test and fix any bugs that > may > > > be > > > > > found on trunk as well as add further tests as outlined in the test > > > plan. > > > > > > > > > > The bulk of the design and implementation was done by Suresh > > Srinivas, > > > > > Sanjay Radia, Nicholas Sze, Junping Du and me. Also, thanks to Eric > > > > > Sirianni, Chris Nauroth, Steve Loughran, Bikas Saha, Andrew Wang > and > > > Todd > > > > > Lipcon for providing feedback on the Jiras and in discussions. > > > > > > > > > > This vote runs for a week and closes on 12/9/2013 at 11:59 pm PT. > > > > > > > > > > Thanks, > > > > > Arpit > > > > > > > > > > -- > > > > > CONFIDENTIALITY NOTICE > > > > > NOTICE: This message is intended for the use of the individual or > > > entity > > > > to > > > > > which it is addressed and may contain information that is > > confidential, > > > > > privileged and exempt from disclosure under applicable law. If the > > > reader > > > > > of this message is not the intended recipient, you are hereby > > notified > > > > that > > > > > any printing, copying, dissemination, distribution, disclosure or > > > > > forwarding of this communication is strictly prohibited. If you > have > > > > > received this communication in error, please contact the sender > > > > immediately > > > > > and delete it from your system. Thank You. > > > > > > > > > > > > > > > -- > > > CONFIDENTIALITY NOTICE > > > NOTICE: This message is intended for the use of the individual or > entity > > to > > > which it is addressed and may contain information that is confidential, > > > privileged and exempt from disclosure under applicable law. If the > reader > > > of this message is not the intended recipient, you are hereby notified > > that > > > any printing, copying, dissemination, distribution, disclosure or > > > forwarding of this communication is strictly prohibited. If you have > > > received this communication in error, please contact the sender > > immediately > > > and delete it from your system. Thank You. > > > > > > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
