On 5 April 2014 11:53, Colin McCabe <cmcc...@alumni.cmu.edu> wrote: > I've been using JDK7 for Hadoop development for a while now, and I > know a lot of other folks have as well. Correct me if I'm wrong, but > what we're talking about here is not "moving towards JDK7" but > "breaking compatibility with JDK6." >
+1 > > There are a lot of good reasons to ditch JDK6. It would let us use > new APIs in JDK7, especially the new file APIs. It would let us > update a few dependencies to newer versions. > > +1 > I don't like the idea of breaking compatibility with JDK6 in trunk, > but not in branch-2. The traditional reason for putting something in > trunk but not in branch-2 is that it is new code that needs some time > to prove itself. +1. branch-2 must continue to run on JDK6 > This doesn't really apply to incrementing min.jdk-- > we could do that easily whenever we like. Meanwhile, if trunk starts > accumulating jdk7-only code and dependencies, backports from trunk to > branch-2 will become harder and harder over time. > I agree, but note that trunk diverges from branch-2 over time anyway -it's happening. > > Since we make stable releases off of branch-2, and not trunk, I don't > see any upside to this. To be honest, I see only negatives here. > More time backporting, more issues that show up only in production > (branch-2) and not on dev machines (trunk). > > Maybe it's time to start thinking about what version of branch-2 will > drop jdk6 support. But until there is such a version, I don't think > trunk should do it. > 1. Let's assume that branch-2 will never drop JDK6 -clusters are committed to it, and saying "JDK updated needed" will simply stop updates. 2. By the hadoop 3.0 ships -2015?- JDK6 will be EOL, java 8 will be in common use, and even JDK7 seen as trailing edge. 3. JDK7 improves JVM performance: NUMA, nativeIO &c -which you get for free -as we're confident its stable there's no reason to not move to it in production. 4. As we update the dependencies on hadoop 3, we'll end up upgrading to libraries that are JDK7+ only (jetty!), so JDK6 is implicitly abandoned. 5. There are new packages and APIs in Java7 which we can adopt to make our lives better and development more productive -as well as improving the user experience. as a case in point, java.io.File.mkdirs() says "true if and only if the directory was created; false otherwise " , and returns false in either of the two cases: -the path resolves to a directory that exists -the path resolves to a file think about that, anyone using local filesystems could write code that assumes that mkdir()==0 is harmless, because if you apply it more than once on a directory it is. But call it on a file and you don't get told its only a file until you try to do something under it, and then things stop behaving. In comparison, java.nio.files.Files differentiates this case by declaring "FileAlreadyExistsException - if dir exists but is not a directory". Which is the kind of thing that would make RawLocalFS behave a lot more like HDFS. Similarly, if we could switch to Files.moveTo(), then the destination file would stop being overwritten if it existed, so RawLocalFS's rename() semantics would come closer to HDFS. These are things we just can't do while retaining Java 6 compatibility. -and why I am looking forward to the time when we can stop caring about Java7. Now, assuming that Hadoop 3.x will be Java7+ only, we have the option between now and its future ship date to move to those Java7 APIs. So when to make the move? 1. It can be done late -in which case few changes will happen, nobody sees much benefit. 2. We can do it now, and have 12+ months to adopt the new features, make the move -and be set up for Java 8 migration in later versions. Yes, code that uses the new APIs won't work on Java6, but that doesn't mean it shouldn't happen Hadoop made the jump from Java 5 to Java 6 after all. -Steve -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
