+1 for option 3 to drop Java 7 and start using Java8 features in code as well
On Fri, Jun 3, 2016 at 9:38 PM, Reuben Kuhnert <reuben.kuhn...@cloudera.com> wrote: > +1 Drop Java7 support, start using JDK8 features. > > On Fri, Jun 3, 2016 at 12:39 PM, Prasanth Jayachandran < > pjayachand...@hortonworks.com> wrote: > > > +1 for using jdk8 as minimum required version. > > > > Thanks > > Prasanth > > > > > > > > > > On Fri, Jun 3, 2016 at 10:21 AM -0700, "Siddharth Seth" < > ss...@apache.org > > <mailto:ss...@apache.org>> wrote: > > > > +1. Drop Java7 support, and start using JDK8 features. > > > > On Fri, Jun 3, 2016 at 8:13 AM, Ashutosh Chauhan <hashut...@apache.org> > > wrote: > > > > > What I meant was that we start compiling using jdk8 as well. That will > > > allow devs to use jdk8 only features (lambda functions etc.) > > > > > > On Fri, Jun 3, 2016 at 8:10 AM, Sergio Pena <sergio.p...@cloudera.com> > > > wrote: > > > > > > > Hey Ashutosh, > > > > > > > > I switched to JDK8 in master last weekend. Jenkins has been running > all > > > > tests in Java8 successfully for a week now. > > > > There are still a few tests we need to fix, but so far is looking > good. > > > > > > > > Sergio > > > > > > > > On Fri, Jun 3, 2016 at 12:02 AM, Ashutosh Chauhan < > > hashut...@apache.org> > > > > wrote: > > > > > > > > > Now that branch-2.1 has been cut, I think its an opportune time to > > drop > > > > > support of jdk7 altogether in master. That is we compile using jdk8 > > as > > > > > well. What do others think? > > > > > > > > > > Ashutosh > > > > > > > > > > On Fri, May 27, 2016 at 7:41 PM, Sergio Pena < > > sergio.p...@cloudera.com > > > > > > > > > wrote: > > > > > > > > > > > I did the change to JDK8 on Ptest now. > > > > > > > > > > > > Please let me tknow if there are issues with Java8. > > > > > > > > > > > > - Sergio > > > > > > > > > > > > On Fri, May 27, 2016 at 2:45 PM, Sergio Pena < > > > sergio.p...@cloudera.com > > > > > > > > > > > wrote: > > > > > > > > > > > > > Thanks Mohit. > > > > > > > > > > > > > > I will plan to do the JDK8 change on Jenkins today EOD and > > monitor > > > > any > > > > > > > issues through the weekend. > > > > > > > > > > > > > > - Sergio > > > > > > > > > > > > > > > > > > > > > On Fri, May 27, 2016 at 2:58 AM, Mohit Sabharwal < > > > mo...@cloudera.com > > > > > > > > > > > > wrote: > > > > > > > > > > > > > >> Update on moving Hive2 tests to JDK8: I've addressed almost > all > > > test > > > > > > >> failures in HIVE-13547 on java8 branch. There is one remaining > > > > > > >> open item (HIVE-13834) which is currently assigned. Given > > current > > > > > > >> state of flaky test runs, there might be few more. > > > > > > >> > > > > > > >> I will work with Sergio to merge the test fixes to master and > > > switch > > > > > > >> the Hive2 pre-commit job to use JDK8, hopefully sometime > > tomorrow. > > > > > > >> > > > > > > >> After Hive2 tests switch, if your patch sees ordering related > > test > > > > > > >> failures in pre-commit run, it's likely JDK version related > and > > > > you'll > > > > > > >> need to build & re-run the test using JDK8. Number of such > tests > > > > > > >> should be relatively small. > > > > > > >> > > > > > > >> On Tue, Apr 19, 2016 at 10:43 AM, Mohit Sabharwal < > > > > mo...@cloudera.com > > > > > > > > > > > > >> wrote: > > > > > > >> > > > > > > >> > Created HIVE-13547 to track switching 2x tests to JDK8. > > > > > > >> > > > > > > > >> > On Wed, Apr 13, 2016 at 10:02 AM, Sergio Pena < > > > > > > sergio.p...@cloudera.com > > > > > > >> > > > > > > > >> > wrote: > > > > > > >> > > > > > > > >> >> I agree with such change as JDK7 is not longer supported. > > > > > > >> >> > > > > > > >> >> Changes on Jenkins and Hive PTest shouldn't be hard. We > just > > > need > > > > > to > > > > > > >> >> replace the path from java7 to java8. But I think we should > > fix > > > > all > > > > > > >> JDK8 > > > > > > >> >> issues or most of them before doing the change or we will > end > > > up > > > > > > >> having a > > > > > > >> >> lot of failures on all JIRAs running pre-commit tests. > > > > > > >> >> > > > > > > >> >> +1 with the change. > > > > > > >> >> > > > > > > >> >> On Mon, Apr 11, 2016 at 9:34 PM, Siddharth Seth < > > > > ss...@apache.org> > > > > > > >> wrote: > > > > > > >> >> > > > > > > >> >> > Option 3 sounds good. I'd ideally like to make JDK8 the > > > minimum > > > > > > >> >> requirement > > > > > > >> >> > soon as well. > > > > > > >> >> > > > > > > > >> >> > On Mon, Apr 11, 2016 at 4:59 PM, Szehon Ho < > > > > sze...@cloudera.com> > > > > > > >> wrote: > > > > > > >> >> > > > > > > > >> >> > > Sounds like a good plan, +1 > > > > > > >> >> > > > > > > > > >> >> > > On Mon, Apr 11, 2016 at 4:31 PM, Mohit Sabharwal < > > > > > > >> mo...@cloudera.com> > > > > > > >> >> > > wrote: > > > > > > >> >> > > > > > > > > >> >> > > > Thanks, Ashutosh. Makes sense to keep the source and > > > target > > > > > as > > > > > > >> 1.7 > > > > > > >> >> > since > > > > > > >> >> > > > we're not using any JDK8 specific features yet. So, > > > option > > > > > (3) > > > > > > >> >> > > essentially > > > > > > >> >> > > > just means using JDK8 exclusively to build & test > > Hive2. > > > > > > >> >> > > > > > > > > > >> >> > > > On Sat, Apr 9, 2016 at 12:23 PM, Ashutosh Chauhan < > > > > > > >> >> > hashut...@apache.org> > > > > > > >> >> > > > wrote: > > > > > > >> >> > > > > > > > > > >> >> > > > > Hi Mohit, > > > > > > >> >> > > > > > > > > > > >> >> > > > > I also think option 3 makes sense. We should strive > > to > > > > keep > > > > > > >> test > > > > > > >> >> > matrix > > > > > > >> >> > > > > small so that we can do fast QA runs for dev > patches. > > > > > > >> >> > > > > We can just use jdk7 to build & test Hive1 and jdk8 > > to > > > > > build > > > > > > & > > > > > > >> >> test > > > > > > >> >> > > > Hive2. > > > > > > >> >> > > > > However, I am not sure of explicitly dropping > support > > > > > > >> altogether > > > > > > >> >> for > > > > > > >> >> > > jdk7 > > > > > > >> >> > > > > on Hive2. We should make sure that in pom.xml java > > > > source & > > > > > > >> target > > > > > > >> >> > > > > compatibility is still 1.7 (which already is the > case > > > > > > >> currently) > > > > > > >> >> so > > > > > > >> >> > > that > > > > > > >> >> > > > > Hive2 is still compatible with jdk7. Unit tests as > I > > > said > > > > > we > > > > > > >> can > > > > > > >> >> run > > > > > > >> >> > on > > > > > > >> >> > > > > jdk8. > > > > > > >> >> > > > > > > > > > > >> >> > > > > Thanks, > > > > > > >> >> > > > > Ashutosh > > > > > > >> >> > > > > > > > > > > >> >> > > > > On Fri, Apr 8, 2016 at 4:34 PM, Mohit Sabharwal < > > > > > > >> >> mo...@cloudera.com> > > > > > > >> >> > > > > wrote: > > > > > > >> >> > > > > > > > > > > >> >> > > > > > Hi folks, > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > Oracle EOL'ed (ended public updates) for Java 7 > in > > > > April > > > > > > >> 2015. > > > > > > >> >> > > > > > In order to support Java 8, we fixed quite a > bunch > > of > > > > > tests > > > > > > >> in > > > > > > >> >> > > > > > HIVE-8607 (*) early last year. However, since our > > > > > > pre-commit > > > > > > >> >> tests > > > > > > >> >> > > run > > > > > > >> >> > > > > > on JDK7 only, any JDK8 test failures are getting > > > > ignored. > > > > > > As > > > > > > >> a > > > > > > >> >> > > result, > > > > > > >> >> > > > > > the count has crept up > > > > > > >> >> > > > > > < > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > > >> >> > > > > > > > >> >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/HIVE-TRUNK-JAVA8/ > > > > > > >> >> > > > > > > > > > > > > >> >> > > > > > back > > > > > > >> >> > > > > > from zero to 125. > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > Some options to address this: > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > (1) Run pre-commit tests on both JDK7 and JDK8, > in > > > both > > > > > 1.x > > > > > > >> >> > > > > > and 2.x: This will further slow down the > pre-commit > > > > run. > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > (2) Alternate pre-commit test runs between JDK7 > and > > > > JDK8 > > > > > > (in > > > > > > >> >> > > > > > both 1.x and 2.x): It's a cheap hack. More messy, > > > > though > > > > > > >> >> failures > > > > > > >> >> > > > > > won't get ignored. > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > (3) Drop support for JDK7 in Hive 2.x, i.e. build > > and > > > > > test > > > > > > >> Hive > > > > > > >> >> 2.x > > > > > > >> >> > > on > > > > > > >> >> > > > > > JDK8 only. For Hive 1.x, continue running JDK7 > > > > pre-commit > > > > > > >> tests. > > > > > > >> >> > > > > > The pros are: > > > > > > >> >> > > > > > - Simple test matrix. > > > > > > >> >> > > > > > - No need to slow down test run or maintain > version > > > > > > specific > > > > > > >> >> golden > > > > > > >> >> > > > files > > > > > > >> >> > > > > > (**). > > > > > > >> >> > > > > > - 2.x looks like the logical place to move to > JDK8. > > > > > > >> >> > > > > > - Users transitioning to JDK8 for all other > > services > > > do > > > > > not > > > > > > >> >> have to > > > > > > >> >> > > > > > maintain > > > > > > >> >> > > > > > multiple java versions on the cluster. > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > Option (3) looks most attractive to me. > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > Moving to JDK8 also lines us up better for Java > 9, > > > > which > > > > > is > > > > > > >> on > > > > > > >> >> the > > > > > > >> >> > > > > > horizon (Oracle will end public updates for Java > 8 > > in > > > > Sep > > > > > > >> 2017) > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > Around 100 of the latest crop of failures are due > > to > > > > one > > > > > > >> cause > > > > > > >> >> > > > > > (HIVE-13409). > > > > > > >> >> > > > > > I can take a pass at triaging the rest if there > is > > > > > > consensus > > > > > > >> >> around > > > > > > >> >> > > > what > > > > > > >> >> > > > > > to do overall. > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > Thanks, > > > > > > >> >> > > > > > Mohit > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > (*) Most test failures are due to JDK8 using a > > > > different > > > > > > hash > > > > > > >> >> > > function > > > > > > >> >> > > > > for > > > > > > >> >> > > > > > HashMap compared to JDK7. This results in > (mostly > > > > > benign, > > > > > > >> but > > > > > > >> >> hard > > > > > > >> >> > > > > > to debug) ordering differences in q-file output > > > related > > > > > to > > > > > > >> >> > > > serialization > > > > > > >> >> > > > > > order of map entries, numbering of stages in > query > > > > plan, > > > > > > etc. > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > (**) In some cases, hash function related > ordering > > > > > > >> differences > > > > > > >> >> > > > originate > > > > > > >> >> > > > > > in external libraries like Avro, antlr, json > > > ,Thrift's > > > > > map, > > > > > > >> etc. > > > > > > >> >> > for > > > > > > >> >> > > > > which > > > > > > >> >> > > > > > code > > > > > > >> >> > > > > > changes are either more involved or led to more > > test > > > > > > >> failures. > > > > > > >> >> To > > > > > > >> >> > > > > address > > > > > > >> >> > > > > > this, > > > > > > >> >> > > > > > we added support for version specific golden > files > > > > > > >> (HIVE-9109). > > > > > > >> >> > > > > Currently, > > > > > > >> >> > > > > > there > > > > > > >> >> > > > > > are ~40 golden files with ".java1.8.out" > extension. > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > > >> >> > > > > > > > >> >> > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
