On Jan 22, 2014, at 5:14 PM, Andrew Wang <andrew.w...@cloudera.com> wrote:
> Another option (not to step on anyone's toes) is to scratch the current 2.3 > branch and make HDFS-2832+HDFS-4949 the new 2.3, since the current 2.3 has > sort of lost its way as a bugfix-only release, and has diverged quite a bit > from branch-2. Agree 2.3 is now very stale. So, as I understand, your motivation is to get HDFS-2832 & HDFS-4949 out, correct? If so, that could be a better option - I can abandon the 2.3rc, and then release current branch-2 as 2.3. Would that be better? Arun > > As to a more frequent release cadence, it seems like so far there's a lot > of support. I think downstreams would actually appreciate more frequent > releases. HBase at least only builds against release artifacts, so they end > up scrambling whenever we drop a new release. Releasing more often will > force us to streamline the integration testing process, which in turn means > we'll do it more and reveal incompatibilities sooner. Since I'm > volunteering to RM some of these, that part of the burden goes away as well. > > Overall, I'm pushing for this because I'd really like to see high-quality, > compatible releases come out of the 2.x line, and this seems like one of > the best way to make that happen. I plan to cut branch-2.4 tomorrow and > start firming things up. > > Thanks, > Andrew > > On Wed, Jan 22, 2014 at 12:48 PM, Vinod Kumar Vavilapalli < > vino...@apache.org> wrote: > >> >> I think YARN-149 will be ready if we do a 2.4 release sometime next month. >> The same goes for YARN-321 also. The merge-to-trunk vote is happening now >> and it should be in a good shape soon *if *we stick to the early plan >> that is laid out in the roadmap. Overall, a bunch of us in the YARN side >> have been working on features like YARN-149 and YARN-321 assuming a target >> release of 2.4. Release numbers don't make a big deal, but we need to be >> sure about the perceived and publicized stability of these features. For >> e.g., what is the story of compatibility if we ship alpha/beta YARN-149 in >> 2.4 by this month-end? I'd like us all to agree labeling features as >> alpha/beta/stable formally before making an offhanded decision. Will start >> a separate thread for this. >> >> Also, we are yet to release a 2.3? The RC is supposedly a WIP and we'll >> take a week for its release. Which will already be end of the month. Shall >> we wait till that goes out? Cutting a branch for 2.4 next week or end of >> this month seems reasonable and inline with the previous plans. What are >> the timelines for the 2.4 release accordingly if 2.5 were to be started >> middle of next month. >> >> Overall, may be I am missing some threads, but, have we discussed and >> already decided on this new way of monthly release-plans? Even if we did, I >> don't recall us saying it will start with 2.4 and don't see an urgency for >> it to be. Let's start that new process with 2.5 if there is an apparent >> agreement. We will also need to figure out more things before we take this >> plunge. Some them include: >> - What happens to WIP features? Shall we start calling them >> alpha/beta/stable? >> - How do we communicate the non-readiness of some WIP features? >> - What about the compatibility story of WIP features across these monthly >> releases? >> - A monthly release with a release-vote that takes a week? That seems >> like a terrible overhead. >> - What about downstream projects? BigTop is in with these monthly cycles? >> >> Thanks, >> +vinod >> >> >> On Jan 22, 2014, at 12:04 PM, Andrew Wang <andrew.w...@cloudera.com> >> wrote: >> >> Thanks for the comments everyone. >> >> Vinod, if you think YARN-149 isn't ready yet, we can leave it out. >> Alternatively, we could release note it as "beta" with said known issues, >> and let people kick the tires. It looks like a bunch of the core >> functionality is already in place. >> >> Unless anyone else objects, I plan to cut a 2.4 branch later this week. >> Steve, Stack, if you think those two JIRAs can wrap up in that time frame >> we can include it. Since this more experiment will hopefully go well, we >> can include them in a 2.5 next month. >> >> Best, >> Andrew >> >> >> On Wed, Jan 22, 2014 at 11:28 AM, Stack <st...@duboce.net> wrote: >> >> Thanks Andrew for bringing this up. >> >> +1 on more frequent releases and an effort at (roughly time-based release. >> >> We are working to get 'HDFS-5776 Support 'hedged' reads in DFSClient' to >> land in time for 2.4 (but don't hold up the release for us!) >> >> St.Ack >> >> >> On Tue, Jan 21, 2014 at 3:51 PM, Vinod Kumar Vavilapalli < >> vino...@hortonworks.com> wrote: >> >> If the timeline is to cut one next week, I don't think we can ship >> YARN-149 as part of that and call it stable. There are a bunch of major >> things that are still missing there: YARN-1202, YARN-1410, YARN-1525 and >> YARN-1611/YARN-1459. >> >> We need to start labeling individual features alpha/beta/stable now that >> we have a stable 2.2 base. >> >> Thanks >> +Vinod >> >> On Jan 21, 2014, at 1:26 PM, Andrew Wang <andrew.w...@cloudera.com> >> >> wrote: >> >> >> So, as per my earlier email, I think a 2.4 with just HDFS-4949, >> >> HDFS-2832, >> >> and YARN-149 would be an attractive and stable release, and is >> >> something >> >> we >> >> could actually cut this week and vote on before the month is out. The >> >> other >> >> stuff we can ship in Feb or March when it's gotten a chance to bake >> >> for a >> >> bit, and culturally speaking, the fact that it's in 2.5 rather than 2.4 >> shouldn't be a big deal. >> >> Thanks, >> Andrew >> >> >> On Tue, Jan 21, 2014 at 11:37 AM, Suresh Srinivas < >> >> sur...@hortonworks.com>wrote: >> >> >> There is not much progress on symlinks issue. I think we should move >> forward with 2.4 release with symlinks disabled. >> >> Status of 2.4 features from HDFS so far: >> - HDFS-2832 Heterogeneous storage support has been merged >> - HDFS-5535 rolling upgrades work is in progress >> - HDFS-4685 ACL related work is close to completion >> - HDFS-4949 As Andrew has proposed, this will be soon merged into 2.4 >> >> Regards, >> Suresh >> >> On Tue, Jan 21, 2014 at 11:12 AM, Arun C Murthy <a...@hortonworks.com> >> wrote: >> >> Andrew, >> >> I'm almost ready to push out rc0 for 2.3 (been testing it overnight), >> >> I'm >> >> pretty sure I'll get that out tonight. >> >> However, AHS (YARN-321) is very close (merge vote going on) … so that >> will definitely make it in very soon. >> >> So, my plan is essentially the same i.e. release 2.4 end of the month >> (after a bit more testing of RM HA in secure mode). Thanks for the >> >> offer, >> >> I'll ping you if I need any help. >> >> OTOH, can someone from HDFS chime in on status of symlinks? >> >> Arun >> >> On Jan 20, 2014, at 4:19 PM, Andrew Wang <andrew.w...@cloudera.com> >> >> wrote: >> >> >> Hi all, >> >> I'm pretty excited to see a 2.4 this month if possible. Since I >> >> think >> >> people were favorable to the idea of time-based releases, how do we >> >> feel >> >> about just cutting branch-2 and spinning up the release process for >> >> our >> >> January goal? >> >> Looking at the roadmap (https://wiki.apache.org/hadoop/Roadmap), on >> >> the >> >> HDFS side, I plan to post a branch-2 patch for HDFS-4949 this week, >> >> and >> >> HDFS-2832 is already in. On the YARN side, it appears that RM HA is >> >> in, >> >> but >> >> the other three features (AHS, unmanaged containers, and dynamic >> >> resource >> >> configuration) remain unresolved. >> >> I think a 2.4 with HDFS-4949, HDFS-2832, and YARN-149 is already a >> >> pretty >> >> nice release. If it'd help, I'm willing to volunteer as release >> >> manager >> >> to >> >> help get this out the door. >> >> Thoughts welcome! >> >> Thanks, >> Andrew >> >> >> -- >> Arun C. Murthy >> Hortonworks Inc. >> http://hortonworks.com/ >> >> >> >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or >> >> entity >> >> to >> >> which it is addressed and may contain information that is >> >> confidential, >> >> privileged and exempt from disclosure under applicable law. If the >> >> reader >> >> of this message is not the intended recipient, you are hereby >> >> notified >> >> that >> >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender >> >> immediately >> >> and delete it from your system. Thank You. >> >> >> >> >> -- >> http://hortonworks.com/download/ >> >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or >> >> entity to >> >> which it is addressed and may contain information that is >> >> confidential, >> >> privileged and exempt from disclosure under applicable law. If the >> >> reader >> >> of this message is not the intended recipient, you are hereby notified >> >> that >> >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender >> >> immediately >> >> and delete it from your system. Thank You. >> >> >> >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> >> to >> >> which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified >> >> that >> >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender >> >> immediately >> >> and delete it from your system. Thank You. >> >> >> >> >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. >> -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
