On 15 September 2014 18:48, Allen Wittenauer <a...@altiscale.com> wrote:
> > It’s now September. With the passage of time, I have a lot of > doubts about this plan and where that trajectory takes us. > > * The list of changes that are already in branch-2 scare the crap out of > any risk adverse person (Hello to my fellow operations people!). Not only > are the number of changes extremely high, but in addition there are a lot > of major, blockbuster features in what is supposed to be a minor release. > Combined with the fact that we’ve had to do some micro releases, it seems > to hint that branch-2 is getting less stable over time. > I don't agree. Certainly the stuff I got into Hadoop 2.5 nailed down the filesystem binding with more tests than ever before. There are changes coming in 2.6, but I see most of them as extensions, rather than big reworks of the internals > > * One of the plans talked about was rolling a 2.7 release that drops JDK6 > and makes JDK7 the standard. If 2.7 comes after 2.6 in October, date wise > makes it somewhere around January 2015. JDK7 EOL’s in April 2015. So > we’ll have a viable JDK7 release for exactly 3 months. Frankly, it is too > late for us to talk about JDK7 and we need to start thinking about JDK8. > > I think my stance on JDK7 is well known. I'm personally in favour of moving to it ASAP, allowing Hadoop to improve, and to allow downstream projects to move up secure in the knowledge that hadoop 2.7 ==> Java 7+ I also think the language improvements of Java 8 look appealing; we can do stuff that currently you only gain from by moving to groovy, and with the improved concurrency stuff you can do it way faster with code is easier to maintain. A fair few downstream projects use groovy as the test language, Bigtop and Slider being the two I know of. We get the language improvements & better assertions, at a price. Java 8 would let us improve the Junit tests in core hadoop as well as the production code. But I am also aware of large organisations that are still on Java 6. Giving a clear roadmap "move to Java 7 now, java 8 in "XX months" can help them plan. > * trunk is currently sitting at 3 years old. There is a lot of stuff that > has been hanging around that really needs to get into people hands so that > we can start stabilizing it for a “real” release. > > > To me this all says one thing: > > Drop the 2.6.0 release, branch trunk, and start rolling a > 3.0.0-alpha with JDK8 as the minimum. 2.5.1 becomes the base for all > sustaining work. This gives the rest of the community time to move to JDK8 > if they haven’t already. For downstream vendors, it gives a roadmap for > their customers who will be asking about JDK8 sooner rather than later. By > the time 3.0 stabilizes, we’re probably looking at April, which is perfect > timing. > > That delays getting stuff out too much; if april slips it becomes a long time since an ASF release came out. Saying "you must run on Java 8 for this" will only scare people off and hold back adoption of 3.x, leaving 2.5 as the last release that ends up being used for a while; the new 1.0.4 Here's an alternative -2.6 on java 6, announce EOL for Java 6 support -2.7 on Java 7, state that the lifespan of j7 support will be some bounded time period (12-18 mo) -trunk to build and test on Java 8 in jenkins alongside java 7. For that to be useful someone needs to volunteer to care about build failures. are you volunteering, Allen? -steve -we switch trunk to Java 7 NOW. That doesn't mean a rewrite fest going through all catch() statements making them multicatch, and the same for string case. What it does mean is that trunk can -adopt the Java 7 APIs where needed for improvements, such as in the native filesystem IO -prepare for the java 8 migration by fixing things that don't work on java 8 and which are incompatible with a java 7 codebase. Doing that in trunk does mean those features can't seamlessly migrate to hadoop 2.6, so people who hope to do that had better stay in java-6 compatibility mode. Those bits of the code where we know java 6 is crippling it can move up, announcing it clearly. > One of the issues I’ve heard mention is that 3.0 doesn’t have > anything “compelling” in it. Well, dropping 2.6 makes the feature list the > carrot, JDK8 support is obviously the stick. > > Thoughts? > > > > > On Aug 15, 2014, at 6:07 PM, Subramaniam Krishnan <su...@apache.org> > wrote: > > > Thanks for initiating the thread Arun. > > > > Can we add YARN-1051 <https://issues.apache.org/jira/browse/YARN-1051> > to > > the list? We have most of the patches for the sub-JIRAs under review and > > have committed a couple. > > > > -Subru > > > > ---------- Forwarded message ---------- > > > > From: Arun C Murthy <a...@hortonworks.com> > > > > Date: Tue, Aug 12, 2014 at 1:34 PM > > > > Subject: Thinking ahead to hadoop-2.6 > > > > To: "common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>, " > > hdfs-...@hadoop.apache.org" <hdfs-...@hadoop.apache.org>, " > > mapreduce-...@hadoop.apache.org" <mapreduce-...@hadoop.apache.org>, > > > > "yarn-...@hadoop.apache.org" <yarn-...@hadoop.apache.org> > > > > > > > > > > > > Folks, > > > > > > > > With hadoop-2.5 nearly done, it's time to start thinking ahead to > > hadoop-2.6. > > > > > > > > Currently, here is the Roadmap per the wiki: > > > > > > > > • HADOOP > > > > • Credential provider HADOOP-10607 > > > > • HDFS > > > > • Heterogeneous storage (Phase 2) - Support APIs for using > > storage tiers by the applications HDFS-5682 > > > > • Memory as storage tier HDFS-5851 > > > > • YARN > > > > • Dynamic Resource Configuration YARN-291 > > > > • NodeManager Restart YARN-1336 > > > > • ResourceManager HA Phase 2 YARN-556 > > > > • Support for admin-specified labels in YARN YARN-796 > > > > • Support for automatic, shared cache for YARN application > > artifacts YARN-1492 > > > > • Support NodeGroup layer topology on YARN YARN-18 > > > > • Support for Docker containers in YARN YARN-1964 > > > > • YARN service registry YARN-913 > > > > > > > > My suspicion is, as is normal, some will make the cut and some won't. > > > > Please do add/subtract from the list as appropriate. Ideally, it would be > > good to ship hadoop-2.6 in a 6-8 weeks (say, October) to keep up a > cadence. > > > > > > > > More importantly, as we discussed previously, we'd like hadoop-2.6 to be > > the *last* Apache Hadoop 2.x release which support JDK6. I'll start a > > discussion with other communities (HBase, Pig, Hive, Oozie etc.) and see > > how they feel about this. > > > > > > > > thanks, > > > > Arun > > > > > > > > > > > > -- > > > > CONFIDENTIALITY NOTICE > > > > NOTICE: This message is intended for the use of the individual or entity > to > > which it is addressed and may contain information that is confidential, > > privileged and exempt from disclosure under applicable law. If the reader > > of this message is not the intended recipient, you are hereby notified > that > > any printing, copying, dissemination, distribution, disclosure or > > forwarding of this communication is strictly prohibited. If you have > > received this communication in error, please contact the sender > immediately > > and delete it from your system. Thank You. > > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
