+1 to releasing often. Also, like Vinod's idea of labeling individual features alpha/beta/stable.
I am +1 to releasing 2.4 end of this month and including YARN-149 as "beta". I think the associated configs are stable; we might add new ones to make it simpler. We have been testing it internally in non-secure and secure clusters, and it has been holding okay, except when running Oozie workflows in a secure cluster. I hope to address these issues in the next week or two. Other sub-tasks Vinod mentioned are all making progress. Given the plan is to release often, I am sure we can roll out 2.5 next month with RM HA marked "stable". On Wed, Jan 22, 2014 at 12:04 PM, Andrew Wang <andrew.w...@cloudera.com>wrote: > Thanks for the comments everyone. > > Vinod, if you think YARN-149 isn't ready yet, we can leave it out. > Alternatively, we could release note it as "beta" with said known issues, > and let people kick the tires. It looks like a bunch of the core > functionality is already in place. > > Unless anyone else objects, I plan to cut a 2.4 branch later this week. > Steve, Stack, if you think those two JIRAs can wrap up in that time frame > we can include it. Since this more experiment will hopefully go well, we > can include them in a 2.5 next month. > > Best, > Andrew > > > On Wed, Jan 22, 2014 at 11:28 AM, Stack <st...@duboce.net> wrote: > > > Thanks Andrew for bringing this up. > > > > +1 on more frequent releases and an effort at (roughly time-based > release. > > > > We are working to get 'HDFS-5776 Support 'hedged' reads in DFSClient' to > > land in time for 2.4 (but don't hold up the release for us!) > > > > St.Ack > > > > > > On Tue, Jan 21, 2014 at 3:51 PM, Vinod Kumar Vavilapalli < > > vino...@hortonworks.com> wrote: > > > > > If the timeline is to cut one next week, I don't think we can ship > > > YARN-149 as part of that and call it stable. There are a bunch of major > > > things that are still missing there: YARN-1202, YARN-1410, YARN-1525 > and > > > YARN-1611/YARN-1459. > > > > > > We need to start labeling individual features alpha/beta/stable now > that > > > we have a stable 2.2 base. > > > > > > Thanks > > > +Vinod > > > > > > On Jan 21, 2014, at 1:26 PM, Andrew Wang <andrew.w...@cloudera.com> > > wrote: > > > > > > > So, as per my earlier email, I think a 2.4 with just HDFS-4949, > > > HDFS-2832, > > > > and YARN-149 would be an attractive and stable release, and is > > something > > > we > > > > could actually cut this week and vote on before the month is out. The > > > other > > > > stuff we can ship in Feb or March when it's gotten a chance to bake > > for a > > > > bit, and culturally speaking, the fact that it's in 2.5 rather than > 2.4 > > > > shouldn't be a big deal. > > > > > > > > Thanks, > > > > Andrew > > > > > > > > > > > > On Tue, Jan 21, 2014 at 11:37 AM, Suresh Srinivas < > > > sur...@hortonworks.com>wrote: > > > > > > > >> There is not much progress on symlinks issue. I think we should move > > > >> forward with 2.4 release with symlinks disabled. > > > >> > > > >> Status of 2.4 features from HDFS so far: > > > >> - HDFS-2832 Heterogeneous storage support has been merged > > > >> - HDFS-5535 rolling upgrades work is in progress > > > >> - HDFS-4685 ACL related work is close to completion > > > >> - HDFS-4949 As Andrew has proposed, this will be soon merged into > 2.4 > > > >> > > > >> Regards, > > > >> Suresh > > > >> > > > >> On Tue, Jan 21, 2014 at 11:12 AM, Arun C Murthy < > a...@hortonworks.com> > > > >> wrote: > > > >> > > > >>> Andrew, > > > >>> > > > >>> I'm almost ready to push out rc0 for 2.3 (been testing it > overnight), > > > >> I'm > > > >>> pretty sure I'll get that out tonight. > > > >>> > > > >>> However, AHS (YARN-321) is very close (merge vote going on) … so > that > > > >>> will definitely make it in very soon. > > > >>> > > > >>> So, my plan is essentially the same i.e. release 2.4 end of the > month > > > >>> (after a bit more testing of RM HA in secure mode). Thanks for the > > > offer, > > > >>> I'll ping you if I need any help. > > > >>> > > > >>> OTOH, can someone from HDFS chime in on status of symlinks? > > > >>> > > > >>> Arun > > > >>> > > > >>> On Jan 20, 2014, at 4:19 PM, Andrew Wang <andrew.w...@cloudera.com > > > > > >> wrote: > > > >>> > > > >>>> Hi all, > > > >>>> > > > >>>> I'm pretty excited to see a 2.4 this month if possible. Since I > > think > > > >>>> people were favorable to the idea of time-based releases, how do > we > > > >> feel > > > >>>> about just cutting branch-2 and spinning up the release process > for > > > our > > > >>>> January goal? > > > >>>> > > > >>>> Looking at the roadmap (https://wiki.apache.org/hadoop/Roadmap), > on > > > >> the > > > >>>> HDFS side, I plan to post a branch-2 patch for HDFS-4949 this > week, > > > and > > > >>>> HDFS-2832 is already in. On the YARN side, it appears that RM HA > is > > > in, > > > >>> but > > > >>>> the other three features (AHS, unmanaged containers, and dynamic > > > >> resource > > > >>>> configuration) remain unresolved. > > > >>>> > > > >>>> I think a 2.4 with HDFS-4949, HDFS-2832, and YARN-149 is already a > > > >> pretty > > > >>>> nice release. If it'd help, I'm willing to volunteer as release > > > manager > > > >>> to > > > >>>> help get this out the door. > > > >>>> > > > >>>> Thoughts welcome! > > > >>>> > > > >>>> Thanks, > > > >>>> Andrew > > > >>> > > > >>> -- > > > >>> Arun C. Murthy > > > >>> Hortonworks Inc. > > > >>> http://hortonworks.com/ > > > >>> > > > >>> > > > >>> > > > >>> -- > > > >>> CONFIDENTIALITY NOTICE > > > >>> NOTICE: This message is intended for the use of the individual or > > > entity > > > >> to > > > >>> which it is addressed and may contain information that is > > confidential, > > > >>> privileged and exempt from disclosure under applicable law. If the > > > reader > > > >>> of this message is not the intended recipient, you are hereby > > notified > > > >> that > > > >>> any printing, copying, dissemination, distribution, disclosure or > > > >>> forwarding of this communication is strictly prohibited. If you > have > > > >>> received this communication in error, please contact the sender > > > >> immediately > > > >>> and delete it from your system. Thank You. > > > >>> > > > >> > > > >> > > > >> > > > >> -- > > > >> http://hortonworks.com/download/ > > > >> > > > >> -- > > > >> CONFIDENTIALITY NOTICE > > > >> NOTICE: This message is intended for the use of the individual or > > > entity to > > > >> which it is addressed and may contain information that is > > confidential, > > > >> privileged and exempt from disclosure under applicable law. If the > > > reader > > > >> of this message is not the intended recipient, you are hereby > notified > > > that > > > >> any printing, copying, dissemination, distribution, disclosure or > > > >> forwarding of this communication is strictly prohibited. If you have > > > >> received this communication in error, please contact the sender > > > immediately > > > >> and delete it from your system. Thank You. > > > >> > > > > > > > > > -- > > > CONFIDENTIALITY NOTICE > > > NOTICE: This message is intended for the use of the individual or > entity > > to > > > which it is addressed and may contain information that is confidential, > > > privileged and exempt from disclosure under applicable law. If the > reader > > > of this message is not the intended recipient, you are hereby notified > > that > > > any printing, copying, dissemination, distribution, disclosure or > > > forwarding of this communication is strictly prohibited. If you have > > > received this communication in error, please contact the sender > > immediately > > > and delete it from your system. Thank You. > > > > > >
