I think starting a 3.0 alpha soon would be a great idea. As some other people commented, this would come with no compatibility guarantees, so that we can iron out any issues.
Colin On Mon, Feb 22, 2016 at 1:26 PM, Zhe Zhang <zhezh...@cloudera.com> wrote: > Thanks Andrew for driving the effort! > > +1 (non-binding) on starting the 3.0 release process now with 3.0 as an > alpha. > > I wanted to echo Andrew's point that backporting EC to branch-2 is a lot of > work. Considering that no concrete backporting plan has been proposed, it > seems quite uncertain whether / when it can be released in 2.9. I think we > should rather concentrate our EC dev efforts to harden key features under > the follow-on umbrella HDFS-8031 and make it solid for a 3.0 release. > > Sincerely, > Zhe > > On Mon, Feb 22, 2016 at 9:25 AM Colin P. McCabe <cmcc...@apache.org> wrote: > >> +1 for a release of 3.0. There are a lot of significant, >> compatibility-breaking, but necessary changes in this release... we've >> touched on some of them in this thread. >> >> +1 for a parallel release of 2.8 as well. I think we are pretty close >> to this, barring a dozen or so blockers. >> >> best, >> Colin >> >> On Mon, Feb 22, 2016 at 2:56 AM, Steve Loughran <ste...@hortonworks.com> >> wrote: >> > >> >> On 20 Feb 2016, at 15:34, Junping Du <j...@hortonworks.com> wrote: >> >> >> >> Shall we consolidate effort for 2.8.0 and 3.0.0? It doesn't sounds >> reasonable to have two alpha releases to go in parallel. Is EC feature the >> main motivation of releasing hadoop 3 here? If so, I don't understand why >> this feature cannot land on 2.8.x or 2.9.x as an alpha feature. >> > >> > >> > >> >> If we release 3.0 in a month like plan proposed below, it means we will >> have 4 active releases going in parallel - two alpha releases (2.8 and 3.0) >> and two stable releases (2.6.x and 2.7.x). It brings a lot of challenges in >> issues tracking and patch committing, not even mention the tremendous >> effort of release verification and voting. >> >> I would like to propose to wait 2.8 release become stable (may be 2nd >> release in 2.8 branch cause first release is alpha due to discussion in >> another email thread), then we can move to 3.0 as the only alpha release. >> In the meantime, we can bring more significant features (like ATS v2, etc.) >> to trunk and consolidate stable releases in 2.6.x and 2.7.x. I believe that >> make life easier. :) >> >> Thoughts? >> >> >> > >> > 2.8.0 is relatively close to shipping. I say relatively as I'm doing >> some work with ATS 1.5 downstream and I'd like to make sure all that works. >> There's also a large collection of S3 and swift patches needing attention >> from any reviewers with time and credentials. >> > >> > 3.x is going to take multiple iterations to stabilise, and with more >> changes, more significant a rollout. I'd also like to do a complete update >> of all the dependencies before a final release, so we can have less >> pressure to upgrade for a while, and get Sean's classloader patch in so >> it's slightly less visible. >> > >> > That means 3.0 is going to be an alpha release, not final. >> > >> > one thing that could be shared is any build.xml automation of the >> release process, to at least take away most of the manual steps in the >> process, to have something more repeatable. >> > >> > -steve >> > >> > >> >> Thanks, >> >> >> >> Junping >> >> ________________________________________ >> >> From: Yongjun Zhang <yzh...@cloudera.com> >> >> Sent: Friday, February 19, 2016 8:05 PM >> >> To: hdfs-...@hadoop.apache.org >> >> Cc: common-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; >> yarn-...@hadoop.apache.org >> >> Subject: Re: Looking to a Hadoop 3 release >> >> >> >> Thanks Andrew for initiating the effort! >> >> >> >> +1 on pushing 3.x with extended alpha cycle, and continuing the more >> stable >> >> 2.x releases. >> >> >> >> --Yongjun >> >> >> >> On Thu, Feb 18, 2016 at 5:58 PM, Andrew Wang <andrew.w...@cloudera.com> >> >> wrote: >> >> >> >>> Hi Kai, >> >>> >> >>> Sure, I'm open to it. It's a new major release, so we're allowed to >> make >> >>> these kinds of big changes. The idea behind the extended alpha cycle is >> >>> that downstreams can give us feedback. This way if we do anything too >> >>> radical, we can address it in the next alpha and have downstreams >> re-test. >> >>> >> >>> Best, >> >>> Andrew >> >>> >> >>> On Thu, Feb 18, 2016 at 5:23 PM, Zheng, Kai <kai.zh...@intel.com> >> wrote: >> >>> >> >>>> Thanks Andrew for driving this. Wonder if it's a good chance for >> >>>> HADOOP-12579 (Deprecate and remove WriteableRPCEngine) to be in. Note >> >>> it's >> >>>> not an incompatible change, but feel better to be done in the major >> >>> release. >> >>>> >> >>>> Regards, >> >>>> Kai >> >>>> >> >>>> -----Original Message----- >> >>>> From: Andrew Wang [mailto:andrew.w...@cloudera.com] >> >>>> Sent: Friday, February 19, 2016 7:04 AM >> >>>> To: hdfs-...@hadoop.apache.org; Kihwal Lee <kih...@yahoo-inc.com> >> >>>> Cc: mapreduce-dev@hadoop.apache.org; common-...@hadoop.apache.org; >> >>>> yarn-...@hadoop.apache.org >> >>>> Subject: Re: Looking to a Hadoop 3 release >> >>>> >> >>>> Hi Kihwal, >> >>>> >> >>>> I think there's still value in continuing the 2.x releases. 3.x comes >> >>> with >> >>>> the incompatible bump to a JDK8 runtime, and also the fact that 3.x >> won't >> >>>> be beta or GA for some number of months. In the meanwhile, it'd be >> good >> >>> to >> >>>> keep putting out regular, stable 2.x releases. >> >>>> >> >>>> Best, >> >>>> Andrew >> >>>> >> >>>> >> >>>> On Thu, Feb 18, 2016 at 2:50 PM, Kihwal Lee >> <kih...@yahoo-inc.com.invalid >> >>>> >> >>>> wrote: >> >>>> >> >>>>> Moving Hadoop 3 forward sounds fine. If EC is one of the main >> >>>>> motivations, are we getting rid of branch-2.8? >> >>>>> >> >>>>> Kihwal >> >>>>> >> >>>>> From: Andrew Wang <andrew.w...@cloudera.com> >> >>>>> To: "common-...@hadoop.apache.org" <common-...@hadoop.apache.org> >> >>>>> Cc: "yarn-...@hadoop.apache.org" <yarn-...@hadoop.apache.org>; " >> >>>>> mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org>; >> >>>>> hdfs-dev <hdfs-...@hadoop.apache.org> >> >>>>> Sent: Thursday, February 18, 2016 4:35 PM >> >>>>> Subject: Re: Looking to a Hadoop 3 release >> >>>>> >> >>>>> Hi all, >> >>>>> >> >>>>> Reviving this thread. I've seen renewed interest in a trunk release >> >>>>> since HDFS erasure coding has not yet made it to branch-2. Along with >> >>>>> JDK8, the shell script rewrite, and many other improvements, I think >> >>>>> it's time to revisit Hadoop 3.0 release plans. >> >>>>> >> >>>>> My overall plan is still the same as in my original email: a series >> of >> >>>>> regular alpha releases leading up to beta and GA. Alpha releases make >> >>>>> it easier for downstreams to integrate with our code, and making them >> >>>>> regular means features can be included when they are ready. >> >>>>> >> >>>>> I know there are some incompatible changes waiting in the wings (i.e. >> >>>>> HDFS-6984 making FileStatus a PB rather than Writable, some of >> >>>>> HADOOP-9991 bumping dependency versions) that would be good to get >> in. >> >>>>> If you have changes like this, please set the target version to 3.0.0 >> >>>>> and mark them "Incompatible". We can use this JIRA query to track: >> >>>>> >> >>>>> >> >>>>> >> https://issues.apache.org/jira/issues/?jql=project%20in%20(HADOOP%2C%2 >> >>>>> >> 0HDFS%2C%20YARN%2C%20MAPREDUCE)%20and%20%22Target%20Version%2Fs%22%20% >> >>>>> >> 3D%20%223.0.0%22%20and%20resolution%3D%22unresolved%22%20and%20%22Hado >> >>>>> op%20Flags%22%3D%22Incompatible%20change%22%20order%20by%20priority >> >>>>> >> >>>>> There's some release-related stuff that needs to be sorted out >> >>>>> (namely, the new CHANGES.txt and release note generation from Yetus), >> >>>>> but I'd tentatively like to roll the first alpha a month out, so >> third >> >>>>> week of March. >> >>>>> >> >>>>> Best, >> >>>>> Andrew >> >>>>> >> >>>>> On Mon, Mar 9, 2015 at 7:23 PM, Raymie Stata <rst...@altiscale.com> >> >>>> wrote: >> >>>>> >> >>>>>> Avoiding the use of JDK8 language features (and, presumably, APIs) >> >>>>>> means you've abandoned #1, i.e., you haven't (really) bumped the JDK >> >>>>>> source version to JDK8. >> >>>>>> >> >>>>>> Also, note that releasing from trunk is a way of achieving #3, it's >> >>>>>> not a way of abandoning it. >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> On Mon, Mar 9, 2015 at 7:10 PM, Andrew Wang >> >>>>>> <andrew.w...@cloudera.com> >> >>>>>> wrote: >> >>>>>>> Hi Raymie, >> >>>>>>> >> >>>>>>> Konst proposed just releasing off of trunk rather than cutting a >> >>>>>> branch-2, >> >>>>>>> and there was general agreement there. So, consider #3 abandoned. >> >>>>>>> 1&2 >> >>>>> can >> >>>>>>> be achieved at the same time, we just need to avoid using JDK8 >> >>>>>>> language features in trunk so things can be backported. >> >>>>>>> >> >>>>>>> Best, >> >>>>>>> Andrew >> >>>>>>> >> >>>>>>> On Mon, Mar 9, 2015 at 7:01 PM, Raymie Stata >> >>>>>>> <rst...@altiscale.com> >> >>>>>> wrote: >> >>>>>>> >> >>>>>>>> In this (and the related threads), I see the following three >> >>>>>> requirements: >> >>>>>>>> >> >>>>>>>> 1. "Bump the source JDK version to JDK8" (ie, drop JDK7 support). >> >>>>>>>> >> >>>>>>>> 2. "We'll still be releasing 2.x releases for a while, with >> >>>>>>>> similar feature sets as 3.x." >> >>>>>>>> >> >>>>>>>> 3. Avoid the "risk of split-brain behavior" by "minimize >> >>>>>>>> backporting headaches. Pulling trunk > branch-2 > branch-2.x is >> >>>> already tedious. >> >>>>>>>> Adding a branch-3, branch-3.x would be obnoxious." >> >>>>>>>> >> >>>>>>>> These three cannot be achieved at the same time. Which do we >> >>>> abandon? >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> On Mon, Mar 9, 2015 at 12:45 PM, sanjay Radia >> >>>>>>>> <sanjayo...@gmail.com> >> >>>>>>>> wrote: >> >>>>>>>>> >> >>>>>>>>>> On Mar 5, 2015, at 3:21 PM, Siddharth Seth <ss...@apache.org> >> >>>>> wrote: >> >>>>>>>>>> >> >>>>>>>>>> 2) Simplification of configs - potentially separating client >> >>>>>>>>>> side >> >>>>>>>> configs >> >>>>>>>>>> and those used by daemons. This is another source of perpetual >> >>>>>> confusion >> >>>>>>>>>> for users. >> >>>>>>>>> + 1 on this. >> >>>>>>>>> >> >>>>>>>>> sanjay >> >>>>>>>> >> >>>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>> >> >>> >> >> >> > >>
