Changes for Build #1144 [jvs] HIVE-1040 [jira] use sed rather than diff for masking out noise in diff-based tests (Marek Sapota via John Sichi)
Summary: Replace diff -I with regex masking in Java The current diff -I approach has two problems: (1) it does not allow resolution finer than line-level, so it's impossible to mask out pattern occurrences within a line, and (2) it produces unmasked files, so if you run diff on the command line to compare the result .q.out with the checked-in file, you see the noise. My suggestion is to first run sed to replace noise patterns with an unlikely-to-occur string like ZYZZYZVA, and then diff the pre-masked files without using any -I. This would require a one-time hit to update all existing .q.out files so that they would contain the pre-masked results. Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: jsichi Differential Revision: 597 Changes for Build #1145 Changes for Build #1146 [namit] HIVE-2640 Add alterPartition to AlterHandler interface (Kevin Wilfong via namit) Changes for Build #1147 [namit] HIVE-2617 Insert overwrite table db.tname fails if partition already exists (Chinna Rao Lalam via namit) Changes for Build #1148 [heyongqiang] HIVE-2651 [jira] The variable hive.exec.mode.local.auto.tasks.max should be changed (Namit Jain via Yongqiang He) Summary: HIVE-2651 It should be called hive.exec.mode.local.auto.input.files.max instead. The number of input files are checked currently. Test Plan: EMPTY Reviewers: JIRA, heyongqiang Reviewed By: heyongqiang CC: heyongqiang Differential Revision: 861 [cws] HIVE-727. Hive Server getSchema() returns wrong schema for 'Explain' queries (Prasad Mujumdar via cws) [namit] HIVE-2611 Make index table output of create index command if index is table based (Kevin Wilfong via namit) Changes for Build #1150 [jvs] HIVE-2657 [jira] builtins JAR is not being published to Maven repo & hive-cli POM does not depend on it either (Carl Steinbach via John Sichi) Summary: Make hive-cli and hive-ql depend on hive-builtins Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: jsichi Differential Revision: 897 [namit] HIVE-2654 "hive.querylog.location" requires parent directory to be exist or else folder creation fails (Chinna Rao Lalam via namit) Changes for Build #1151 [hashutosh] HIVE-1892 : show functions also returns internal operators (Priyadarshini via Ashutosh Chauhan) Changes for Build #1152 Changes for Build #1153 [namit] HIVE-2660 Need better exception handling in RCFile tolerate corruptions mode (Ramkumar Vadali via namit) Changes for Build #1154 [cws] HIVE-2631. Make Hive work with Hadoop 1.0.0 (Ashutosh Chauhan via cws) Changes for Build #1155 [cws] HIVE-BUILD. Update RELEASE_NOTES.txt with 0.8.0 release information (cws) Changes for Build #1156 Changes for Build #1157 Changes for Build #1158 [namit] HIVE-2602 add support for insert partition overwrite(...) if not exists (Chinna Rao Lalam via namit) Changes for Build #1159 Changes for Build #1160 [cws] HIVE-2005. Implement BETWEEN operator (Navis via cws) Changes for Build #1161 [jvs] HIVE-2433. add DOAP file for Hive Changes for Build #1162 Changes for Build #1163 Changes for Build #1164 [heyongqiang] HIVE-2666 [jira] StackOverflowError when using custom UDF in map join (Kevin Wilfong via Yongqiang He) Summary: Resource files are now added to the class path as soon as they are added via the CLI. This fixes the stack overflow error mentioned in the JIRA by ensuring a consistent class loader between serializers and deserializers for the same query. Note that now serdes which contain a static block to register themselves are now registered twice, once when adding the file to the class loader, and once when an instance of the class is created. Previously, registering a serde twice resulted in an exception, to avoid this, I have downgraded it to a warning. When a custom UDF is used as part of a join which is converted to a map join, the XMLEncoder enters an infinite loop when serializing the map reduce task for the second time, as part of sending it to be executed. This results in a stack overflow error. Test Plan: I ran the unit tests to verify nothing was broken. I ran several queries which used custom UDFs and involved a join which was converted to a map join. I verified these completed successfully consistently Reviewers: JIRA, heyongqiang Reviewed By: heyongqiang CC: heyongqiang, kevinwilfong Differential Revision: 957 [namit] HIVE-2642 fix Hive-2566 and make union optimization more aggressive (Yongqiang He via namit) Changes for Build #1166 Changes for Build #1167 Changes for Build #1168 [heyongqiang] HIVE-2600: Enable/Add type-specific compression for rcfile (Krishna Kumar via He Yongqiang) Changes for Build #1169 Changes for Build #1170 [cws] HIVE-1877. Add java_method() as a synonym for the reflect() UDF (Zhenxiao Luo via cws) Changes for Build #1171 Changes for Build #1172 Changes for Build #1173 Changes for Build #1174 Changes for Build #1175 [hashutosh] HIVE-2681 : SUCESS is misspelled (jonchang via Ashutosh Chauhan) Changes for Build #1176 [hashutosh] HIVE-2616 : Passing user identity from metastore client to server in non-secure mode (Ashutosh Chauhan) Changes for Build #1177 Changes for Build #1178 Changes for Build #1179 Changes for Build #1180 Changes for Build #1181 3 tests failed. REGRESSION: org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1(TestNegativeCliDriver.java:9476) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) FAILED: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert2_overwrite_partitions Error Message: Unexpected exception See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert2_overwrite_partitions(TestCliDriver.java:16920) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) FAILED: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_inputddl5 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_inputddl5(TestCliDriver.java:16632) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1181) Status: Failure Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1181/ to view the results.