Sounds like a good plan, +1 On Mon, Apr 11, 2016 at 4:31 PM, Mohit Sabharwal <mo...@cloudera.com> wrote:
> Thanks, Ashutosh. Makes sense to keep the source and target as 1.7 since > we're not using any JDK8 specific features yet. So, option (3) essentially > just means using JDK8 exclusively to build & test Hive2. > > On Sat, Apr 9, 2016 at 12:23 PM, Ashutosh Chauhan <hashut...@apache.org> > wrote: > > > Hi Mohit, > > > > I also think option 3 makes sense. We should strive to keep test matrix > > small so that we can do fast QA runs for dev patches. > > We can just use jdk7 to build & test Hive1 and jdk8 to build & test > Hive2. > > However, I am not sure of explicitly dropping support altogether for jdk7 > > on Hive2. We should make sure that in pom.xml java source & target > > compatibility is still 1.7 (which already is the case currently) so that > > Hive2 is still compatible with jdk7. Unit tests as I said we can run on > > jdk8. > > > > Thanks, > > Ashutosh > > > > On Fri, Apr 8, 2016 at 4:34 PM, Mohit Sabharwal <mo...@cloudera.com> > > wrote: > > > > > Hi folks, > > > > > > Oracle EOL'ed (ended public updates) for Java 7 in April 2015. > > > In order to support Java 8, we fixed quite a bunch of tests in > > > HIVE-8607 (*) early last year. However, since our pre-commit tests run > > > on JDK7 only, any JDK8 test failures are getting ignored. As a result, > > > the count has crept up > > > < > > > > > > http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/HIVE-TRUNK-JAVA8/ > > > > > > > back > > > from zero to 125. > > > > > > Some options to address this: > > > > > > (1) Run pre-commit tests on both JDK7 and JDK8, in both 1.x > > > and 2.x: This will further slow down the pre-commit run. > > > > > > (2) Alternate pre-commit test runs between JDK7 and JDK8 (in > > > both 1.x and 2.x): It's a cheap hack. More messy, though failures > > > won't get ignored. > > > > > > (3) Drop support for JDK7 in Hive 2.x, i.e. build and test Hive 2.x on > > > JDK8 only. For Hive 1.x, continue running JDK7 pre-commit tests. > > > The pros are: > > > - Simple test matrix. > > > - No need to slow down test run or maintain version specific golden > files > > > (**). > > > - 2.x looks like the logical place to move to JDK8. > > > - Users transitioning to JDK8 for all other services do not have to > > > maintain > > > multiple java versions on the cluster. > > > > > > Option (3) looks most attractive to me. > > > > > > Moving to JDK8 also lines us up better for Java 9, which is on the > > > horizon (Oracle will end public updates for Java 8 in Sep 2017) > > > > > > Around 100 of the latest crop of failures are due to one cause > > > (HIVE-13409). > > > I can take a pass at triaging the rest if there is consensus around > what > > > to do overall. > > > > > > Thanks, > > > Mohit > > > > > > (*) Most test failures are due to JDK8 using a different hash function > > for > > > HashMap compared to JDK7. This results in (mostly benign, but hard > > > to debug) ordering differences in q-file output related to > serialization > > > order of map entries, numbering of stages in query plan, etc. > > > > > > (**) In some cases, hash function related ordering differences > originate > > > in external libraries like Avro, antlr, json ,Thrift's map, etc. for > > which > > > code > > > changes are either more involved or led to more test failures. To > > address > > > this, > > > we added support for version specific golden files (HIVE-9109). > > Currently, > > > there > > > are ~40 golden files with ".java1.8.out" extension. > > > > > >
