[jira] [Updated] (HIVE-2477) Use name of original expression for name of CAST output
[ https://issues.apache.org/jira/browse/HIVE-2477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2477: Attachment: HIVE-2477.1.patch.txt Simple patch. If a column is function with single param and the param is a column, use the column name as alias. Use name of original expression for name of CAST output --- Key: HIVE-2477 URL: https://issues.apache.org/jira/browse/HIVE-2477 Project: Hive Issue Type: Improvement Reporter: Adam Kramer Priority: Minor Attachments: HIVE-2477.1.patch.txt CAST(foo AS INT) should, by default, consider itself a column named foo if unspecified/unaliased. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2629) Make a single Hive binary work with both 0.20.x and 0.23.0
[ https://issues.apache.org/jira/browse/HIVE-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amareshwari Sriramadasu updated HIVE-2629: -- Resolution: Fixed Status: Resolved (was: Patch Available) Committed this to branch-0.8-r2. Make a single Hive binary work with both 0.20.x and 0.23.0 -- Key: HIVE-2629 URL: https://issues.apache.org/jira/browse/HIVE-2629 Project: Hive Issue Type: Bug Components: Shims Reporter: Carl Steinbach Assignee: Thomas Weise Fix For: 0.8.1, 0.9.0 Attachments: HIVE-2629-InputSplitShim.patch, HIVE-2629.3.patch, HIVE-2629.D711.1.patch, HIVE-2629.D711.2.patch, HIVE-2629.D711.3.patch, HIVE-2629.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1207 - Still Failing
Changes for Build #1174 Changes for Build #1175 [hashutosh] HIVE-2681 : SUCESS is misspelled (jonchang via Ashutosh Chauhan) Changes for Build #1176 [hashutosh] HIVE-2616 : Passing user identity from metastore client to server in non-secure mode (Ashutosh Chauhan) Changes for Build #1177 Changes for Build #1178 Changes for Build #1179 Changes for Build #1180 Changes for Build #1181 Changes for Build #1182 [heyongqiang] HIVE-2621:Allow multiple group bys with the same input data and spray keys to be run on the same reducer. (Kevin via He Yongqiang) Changes for Build #1184 [namit] HIVE-2690 a bug in 'alter table concatenate' that causes filenames getting double url encoded (He Yongqiang via namit) Changes for Build #1185 Changes for Build #1186 Changes for Build #1187 Changes for Build #1188 Changes for Build #1189 Changes for Build #1190 [amareshwari] HIVE-2629. Make a single Hive binary work with both 0.20.x and 0.23.0. (Thomas Weise via amareshwari) Changes for Build #1191 [amareshwari] HIVE-2629. Reverting previous commit Changes for Build #1192 [heyongqiang] HIVE-2706 [jira] StackOverflowError when using custom UDF after adding archive after adding jars (Kevin Wilfong via Yongqiang He) Summary: https://issues.apache.org/jira/browse/HIVE-2706 The issue was that the current thread's classloader and the classloader in the conf differed due to the prehook updating only the current thread's classloader with new jars. Now, it updates both classloaders, fixing the issue. When a custom UDF is used in a query after add an archive, such as a zip file, after adding jars, the XMLEncoder enters an infinite loop when serializing the map reduce task, as part of sending it to be executed. This results in a stack overflow error. Test Plan: Verified it fixed the stack overflow error. Reviewers: JIRA, heyongqiang, njain Reviewed By: heyongqiang CC: heyongqiang Differential Revision: https://reviews.facebook.net/D1167 Changes for Build #1193 [hashutosh] HIVE-2705: SemanticAnalyzer twice swallows an exception it shouldn't (jghoman via hashutosh) Changes for Build #1194 Changes for Build #1195 [hashutosh] HIVE-2589: Newly created partition should inherit properties from table (Ashutosh Chauhan) [hashutosh] HIVE-2682: Clean-up logs (Rajat Goel via Ashutosh Chauhan) Changes for Build #1196 [amareshwari] HIVE-2629. Make a single Hive binary work with both 0.20.x and 0.23.0. (Thomas Weise via amareshwari) Changes for Build #1197 Changes for Build #1198 [namit] HIVE-2504 Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory (Chinna Rao Lalam via namit) [namit] HIVE-2695 Add PRINTF() Udf (Zhenxiao Luo via namit) Changes for Build #1199 Changes for Build #1200 Changes for Build #1201 Changes for Build #1202 Changes for Build #1203 Changes for Build #1204 [cws] HIVE-2719. Revert HIVE-2589 (He Yongqiang via cws) Changes for Build #1205 Changes for Build #1207 [namit] HIVE-2718 NPE in union followed by join (He Yongqiang via namit) 2 tests failed. REGRESSION: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppr_pushdown Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppr_pushdown(TestCliDriver.java:25903) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) FAILED: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_inputddl5 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace:
[jira] [Commented] (HIVE-2718) NPE in union followed by join
[ https://issues.apache.org/jira/browse/HIVE-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13189069#comment-13189069 ] Hudson commented on HIVE-2718: -- Integrated in Hive-trunk-h0.21 #1207 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1207/]) HIVE-2718 NPE in union followed by join (He Yongqiang via namit) namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1233127 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java * /hive/trunk/ql/src/test/queries/clientpositive/union27.q * /hive/trunk/ql/src/test/results/clientpositive/union27.q.out NPE in union followed by join - Key: HIVE-2718 URL: https://issues.apache.org/jira/browse/HIVE-2718 Project: Hive Issue Type: Bug Reporter: He Yongqiang Assignee: He Yongqiang Attachments: HIVE-2718.D1245.1.patch, HIVE-2718.D1245.2.patch, Hive-2718.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2629) Make a single Hive binary work with both 0.20.x and 0.23.0
[ https://issues.apache.org/jira/browse/HIVE-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13189118#comment-13189118 ] Hudson commented on HIVE-2629: -- Integrated in Hive-0.8.1-SNAPSHOT-h0.21 #166 (See [https://builds.apache.org/job/Hive-0.8.1-SNAPSHOT-h0.21/166/]) HIVE-2629. Merge -r 1230390:1230391 from trunk. amareshwari : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1233262 Files : * /hive/branches/branch-0.8-r2/build-common.xml * /hive/branches/branch-0.8-r2/build.properties * /hive/branches/branch-0.8-r2/shims/build.xml * /hive/branches/branch-0.8-r2/shims/ivy.xml * /hive/branches/branch-0.8-r2/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java * /hive/branches/branch-0.8-r2/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/DelegationTokenIdentifier.java * /hive/branches/branch-0.8-r2/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/DelegationTokenSecretManager.java * /hive/branches/branch-0.8-r2/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/DelegationTokenSelector.java * /hive/branches/branch-0.8-r2/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java * /hive/branches/branch-0.8-r2/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java * /hive/branches/branch-0.8-r2/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java * /hive/branches/branch-0.8-r2/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java * /hive/branches/branch-0.8-r2/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/client/TUGIAssumingTransport.java * /hive/branches/branch-0.8-r2/shims/src/0.20S/java/org/apache/hadoop/security/token/delegation/HiveDelegationTokenSupport.java * /hive/branches/branch-0.8-r2/shims/src/0.23/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java * /hive/branches/branch-0.8-r2/shims/src/0.23/java/org/apache/hadoop/hive/thrift/DelegationTokenIdentifier23.java * /hive/branches/branch-0.8-r2/shims/src/0.23/java/org/apache/hadoop/hive/thrift/DelegationTokenSelector23.java * /hive/branches/branch-0.8-r2/shims/src/common-secure * /hive/branches/branch-0.8-r2/shims/src/common-secure/java * /hive/branches/branch-0.8-r2/shims/src/common-secure/java/org * /hive/branches/branch-0.8-r2/shims/src/common-secure/java/org/apache * /hive/branches/branch-0.8-r2/shims/src/common-secure/java/org/apache/hadoop * /hive/branches/branch-0.8-r2/shims/src/common-secure/java/org/apache/hadoop/hive * /hive/branches/branch-0.8-r2/shims/src/common-secure/java/org/apache/hadoop/hive/shims * /hive/branches/branch-0.8-r2/shims/src/common-secure/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java * /hive/branches/branch-0.8-r2/shims/src/common-secure/java/org/apache/hadoop/hive/thrift * /hive/branches/branch-0.8-r2/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DelegationTokenIdentifier.java * /hive/branches/branch-0.8-r2/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DelegationTokenSecretManager.java * /hive/branches/branch-0.8-r2/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DelegationTokenSelector.java * /hive/branches/branch-0.8-r2/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java * /hive/branches/branch-0.8-r2/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java * /hive/branches/branch-0.8-r2/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java * /hive/branches/branch-0.8-r2/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java * /hive/branches/branch-0.8-r2/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/client * /hive/branches/branch-0.8-r2/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/client/TUGIAssumingTransport.java * /hive/branches/branch-0.8-r2/shims/src/common-secure/java/org/apache/hadoop/security * /hive/branches/branch-0.8-r2/shims/src/common-secure/java/org/apache/hadoop/security/token * /hive/branches/branch-0.8-r2/shims/src/common-secure/java/org/apache/hadoop/security/token/delegation * /hive/branches/branch-0.8-r2/shims/src/common-secure/java/org/apache/hadoop/security/token/delegation/HiveDelegationTokenSupport.java * /hive/branches/branch-0.8-r2/shims/src/common/java/org/apache/hadoop/hive/shims/ShimLoader.java Make a single Hive binary work with both 0.20.x and 0.23.0 -- Key: HIVE-2629 URL: https://issues.apache.org/jira/browse/HIVE-2629 Project: Hive Issue Type: Bug Components: Shims Reporter: Carl Steinbach Assignee: Thomas Weise Fix For: 0.8.1, 0.9.0 Attachments: HIVE-2629-InputSplitShim.patch, HIVE-2629.3.patch, HIVE-2629.D711.1.patch,
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13189119#comment-13189119 ] Hudson commented on HIVE-2616: -- Integrated in Hive-0.8.1-SNAPSHOT-h0.21 #166 (See [https://builds.apache.org/job/Hive-0.8.1-SNAPSHOT-h0.21/166/]) HIVE-2616. Merge -r 1225682:1225683 https://svn.apache.org/repos/asf/hive/trunk . amareshwari : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1233260 Files : * /hive/branches/branch-0.8-r2/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/branches/branch-0.8-r2/conf/hive-default.xml.template * /hive/branches/branch-0.8-r2/metastore/if/hive_metastore.thrift * /hive/branches/branch-0.8-r2/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp * /hive/branches/branch-0.8-r2/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h * /hive/branches/branch-0.8-r2/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp * /hive/branches/branch-0.8-r2/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java * /hive/branches/branch-0.8-r2/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php * /hive/branches/branch-0.8-r2/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote * /hive/branches/branch-0.8-r2/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py * /hive/branches/branch-0.8-r2/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb * /hive/branches/branch-0.8-r2/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java * /hive/branches/branch-0.8-r2/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java * /hive/branches/branch-0.8-r2/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java * /hive/branches/branch-0.8-r2/metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java * /hive/branches/branch-0.8-r2/metastore/src/test/org/apache/hadoop/hive/metastore/TestSetUGIOnBothClientServer.java * /hive/branches/branch-0.8-r2/metastore/src/test/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyClient.java * /hive/branches/branch-0.8-r2/metastore/src/test/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyServer.java * /hive/branches/branch-0.8-r2/shims/ivy.xml * /hive/branches/branch-0.8-r2/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java * /hive/branches/branch-0.8-r2/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java * /hive/branches/branch-0.8-r2/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java * /hive/branches/branch-0.8-r2/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/client * /hive/branches/branch-0.8-r2/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/client/TUGIAssumingTransport.java * /hive/branches/branch-0.8-r2/shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java * /hive/branches/branch-0.8-r2/shims/src/common/java/org/apache/hadoop/hive/thrift/TFilterTransport.java * /hive/branches/branch-0.8-r2/shims/src/common/java/org/apache/hadoop/hive/thrift/TUGIContainingTransport.java Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.8.1, 0.9.0 Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch, hive-2616_4.patch, hive-2616_5.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1208 - Still Failing
Changes for Build #1174 Changes for Build #1175 [hashutosh] HIVE-2681 : SUCESS is misspelled (jonchang via Ashutosh Chauhan) Changes for Build #1176 [hashutosh] HIVE-2616 : Passing user identity from metastore client to server in non-secure mode (Ashutosh Chauhan) Changes for Build #1177 Changes for Build #1178 Changes for Build #1179 Changes for Build #1180 Changes for Build #1181 Changes for Build #1182 [heyongqiang] HIVE-2621:Allow multiple group bys with the same input data and spray keys to be run on the same reducer. (Kevin via He Yongqiang) Changes for Build #1184 [namit] HIVE-2690 a bug in 'alter table concatenate' that causes filenames getting double url encoded (He Yongqiang via namit) Changes for Build #1185 Changes for Build #1186 Changes for Build #1187 Changes for Build #1188 Changes for Build #1189 Changes for Build #1190 [amareshwari] HIVE-2629. Make a single Hive binary work with both 0.20.x and 0.23.0. (Thomas Weise via amareshwari) Changes for Build #1191 [amareshwari] HIVE-2629. Reverting previous commit Changes for Build #1192 [heyongqiang] HIVE-2706 [jira] StackOverflowError when using custom UDF after adding archive after adding jars (Kevin Wilfong via Yongqiang He) Summary: https://issues.apache.org/jira/browse/HIVE-2706 The issue was that the current thread's classloader and the classloader in the conf differed due to the prehook updating only the current thread's classloader with new jars. Now, it updates both classloaders, fixing the issue. When a custom UDF is used in a query after add an archive, such as a zip file, after adding jars, the XMLEncoder enters an infinite loop when serializing the map reduce task, as part of sending it to be executed. This results in a stack overflow error. Test Plan: Verified it fixed the stack overflow error. Reviewers: JIRA, heyongqiang, njain Reviewed By: heyongqiang CC: heyongqiang Differential Revision: https://reviews.facebook.net/D1167 Changes for Build #1193 [hashutosh] HIVE-2705: SemanticAnalyzer twice swallows an exception it shouldn't (jghoman via hashutosh) Changes for Build #1194 Changes for Build #1195 [hashutosh] HIVE-2589: Newly created partition should inherit properties from table (Ashutosh Chauhan) [hashutosh] HIVE-2682: Clean-up logs (Rajat Goel via Ashutosh Chauhan) Changes for Build #1196 [amareshwari] HIVE-2629. Make a single Hive binary work with both 0.20.x and 0.23.0. (Thomas Weise via amareshwari) Changes for Build #1197 Changes for Build #1198 [namit] HIVE-2504 Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory (Chinna Rao Lalam via namit) [namit] HIVE-2695 Add PRINTF() Udf (Zhenxiao Luo via namit) Changes for Build #1199 Changes for Build #1200 Changes for Build #1201 Changes for Build #1202 Changes for Build #1203 Changes for Build #1204 [cws] HIVE-2719. Revert HIVE-2589 (He Yongqiang via cws) Changes for Build #1205 Changes for Build #1207 [namit] HIVE-2718 NPE in union followed by join (He Yongqiang via namit) Changes for Build #1208 2 tests failed. REGRESSION: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert2_overwrite_partitions Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert2_overwrite_partitions(TestCliDriver.java:17167) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) FAILED: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_inputddl5 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false
[jira] [Commented] (HIVE-2380) Add Binary Datatype in Hive
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13189346#comment-13189346 ] Hojjat Jafarpour commented on HIVE-2380: Is there an example for defining a table with binary and timestamp columns and loading daata into the table and querying it? Add Binary Datatype in Hive --- Key: HIVE-2380 URL: https://issues.apache.org/jira/browse/HIVE-2380 Project: Hive Issue Type: New Feature Components: Serializers/Deserializers Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.8.0 Attachments: hive-2380.patch, hive-2380_1.patch, hive-2380_2.patch, hive-2380_3.patch, hive-2380_4.patch Add bytearray as a primitive data type. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2674) get_partitions_ps throws TApplicationException if table doesn't exist
[ https://issues.apache.org/jira/browse/HIVE-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13189354#comment-13189354 ] Kevin Wilfong commented on HIVE-2674: - Ran svn up, and updated the diff. get_partitions_ps throws TApplicationException if table doesn't exist - Key: HIVE-2674 URL: https://issues.apache.org/jira/browse/HIVE-2674 Project: Hive Issue Type: Bug Components: Metastore Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2674.D987.1.patch, HIVE-2674.D987.2.patch If the table passed to get_partition_ps doesn't exist, a NPE is thrown by getPartitionPsQueryResults. There should be a check here, which throws a NoSuchObjectException if the table doesn't exist. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2674) get_partitions_ps throws TApplicationException if table doesn't exist
[ https://issues.apache.org/jira/browse/HIVE-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2674: -- Attachment: HIVE-2674.D987.2.patch kevinwilfong updated the revision HIVE-2674 [jira] get_partitions_ps throws TApplicationException if table doesn't exist. Reviewers: JIRA Ran svn up. REVISION DETAIL https://reviews.facebook.net/D987 AFFECTED FILES metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php metastore/if/hive_metastore.thrift get_partitions_ps throws TApplicationException if table doesn't exist - Key: HIVE-2674 URL: https://issues.apache.org/jira/browse/HIVE-2674 Project: Hive Issue Type: Bug Components: Metastore Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2674.D987.1.patch, HIVE-2674.D987.2.patch If the table passed to get_partition_ps doesn't exist, a NPE is thrown by getPartitionPsQueryResults. There should be a check here, which throws a NoSuchObjectException if the table doesn't exist. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2674) get_partitions_ps throws TApplicationException if table doesn't exist
[ https://issues.apache.org/jira/browse/HIVE-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-2674: Status: Patch Available (was: Open) get_partitions_ps throws TApplicationException if table doesn't exist - Key: HIVE-2674 URL: https://issues.apache.org/jira/browse/HIVE-2674 Project: Hive Issue Type: Bug Components: Metastore Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2674.D987.1.patch, HIVE-2674.D987.2.patch If the table passed to get_partition_ps doesn't exist, a NPE is thrown by getPartitionPsQueryResults. There should be a check here, which throws a NoSuchObjectException if the table doesn't exist. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2727) add a testcase for partitioned view on union and base tables have index
[ https://issues.apache.org/jira/browse/HIVE-2727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13189363#comment-13189363 ] Phabricator commented on HIVE-2727: --- njain has commented on the revision HIVE-2727 [jira] add a testcase for partitioned view on union and base tables have index. INLINE COMMENTS ql/src/test/queries/clientpositive/union_view.q:27 The code changes look OK. But, can you change the tests ? Create 2 partitions in each base table. Change the explain to Explain Extended for all the queries, to verify that we are only looking at the relevant partitions of the index. Can you unset all the hooks for this (printhook, lineagehook etc.) Also remove the merge (both map-only and map-reduce only). It will make the test output much smaller, and easier to validate REVISION DETAIL https://reviews.facebook.net/D1323 add a testcase for partitioned view on union and base tables have index --- Key: HIVE-2727 URL: https://issues.apache.org/jira/browse/HIVE-2727 Project: Hive Issue Type: Test Reporter: He Yongqiang Assignee: He Yongqiang Attachments: HIVE-2727.1.patch, HIVE-2727.D1323.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2674) get_partitions_ps throws TApplicationException if table doesn't exist
[ https://issues.apache.org/jira/browse/HIVE-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13189362#comment-13189362 ] Namit Jain commented on HIVE-2674: -- +1 get_partitions_ps throws TApplicationException if table doesn't exist - Key: HIVE-2674 URL: https://issues.apache.org/jira/browse/HIVE-2674 Project: Hive Issue Type: Bug Components: Metastore Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2674.D987.1.patch, HIVE-2674.D987.2.patch If the table passed to get_partition_ps doesn't exist, a NPE is thrown by getPartitionPsQueryResults. There should be a check here, which throws a NoSuchObjectException if the table doesn't exist. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2674) get_partitions_ps throws TApplicationException if table doesn't exist
[ https://issues.apache.org/jira/browse/HIVE-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13189364#comment-13189364 ] Namit Jain commented on HIVE-2674: -- running tests get_partitions_ps throws TApplicationException if table doesn't exist - Key: HIVE-2674 URL: https://issues.apache.org/jira/browse/HIVE-2674 Project: Hive Issue Type: Bug Components: Metastore Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2674.D987.1.patch, HIVE-2674.D987.2.patch If the table passed to get_partition_ps doesn't exist, a NPE is thrown by getPartitionPsQueryResults. There should be a check here, which throws a NoSuchObjectException if the table doesn't exist. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2730) ObjectInspectors improperly converted if a constantOI is returned from a nondeterministic function.
[ https://issues.apache.org/jira/browse/HIVE-2730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Chang reassigned HIVE-2730: Assignee: Jonathan Chang ObjectInspectors improperly converted if a constantOI is returned from a nondeterministic function. --- Key: HIVE-2730 URL: https://issues.apache.org/jira/browse/HIVE-2730 Project: Hive Issue Type: Bug Reporter: Jonathan Chang Assignee: Jonathan Chang The logic for whether constants can be folded is not consistent. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2730) ObjectInspectors improperly converted if a constantOI is returned from a nondeterministic function.
ObjectInspectors improperly converted if a constantOI is returned from a nondeterministic function. --- Key: HIVE-2730 URL: https://issues.apache.org/jira/browse/HIVE-2730 Project: Hive Issue Type: Bug Reporter: Jonathan Chang The logic for whether constants can be folded is not consistent. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2731) Precommit builds at Jenkins
Precommit builds at Jenkins Key: HIVE-2731 URL: https://issues.apache.org/jira/browse/HIVE-2731 Project: Hive Issue Type: Improvement Components: Build Infrastructure, Testing Infrastructure Reporter: Enis Soztutar Precommit builds, similar to what Hadoop/Hbase/ZK do is a great developer tool, so that we can enforce tests, test coverage, audit checks, etc. I guess we can ask Yahoo folks to use one of the Hadoop build machines. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2612) support hive table/partitions coexistes in more than one clusters
[ https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13189526#comment-13189526 ] Carl Steinbach commented on HIVE-2612: -- bq. A table T1's primary cluster is C1 meaning :1) C1 contains all data that is available in all other clusters. Does this mean that if T1's primary cluster is C1, then all of the partitions in T1 must also have have their primary partition set to C1? If that's the case then primary cluster should probably be a table level property, and the list of replica clusters can be a table/partition level property. bq. 2) write is only allowed in this cluster for table C1. but need to allow exceptions here What are the exceptions? bq. 4) all data changes to T1 happened in the primary cluster should be replicated to other clusters if there are any secondary clusters. but there should be a conf to disable it as there are some exception situations. What are the exceptions? How will dynamic partitions work? Where will new partitions get the list of replica clusters from? Will they inherit it from the table definition? Hive now supports insert-append into a partition (HIVE-306). Suppose that the metadata for a particular partition indicates that it is replicated to clusters C2 and C3. If I insert new data into the partition in the primary cluster C1, then the metadata is now invalid. How is this going to be handled? bq. 2) add new interfaces which do exactly the same set of functionalities as old ones but using a different name (use _on_cluster suffix maybe?) and have a cluster parameter This is going to introduce new codepaths that need to be tested separately, and also double the amount of work people need to do every time a new metastore API call is created. I don't think this is a good approach. bq. 3) overwrite database name for the purpose of cluster name. And allow a table co-exist in multiple databases. But that require to promote table to top level citizen, and degrade database. For example, show tables used to scan all tables in current db, but now need to scan all tables in all databases. I don't think this is an option since it breaks backwards compatibility and effectively changes the whole notion of what a db/schema is. A lot of people in the community already depend on this feature. bq. 1) add a cluster parameter to existing thrift interfaces This sounds like the best option to me. I think Thrift supports API evolution via default values for missing parameters, but setting a default value in this case may be a little tricky. Also, instead of modifying the Thrift interface, is it possible that you could instead leverage the work that's being done in HIVE-2720? support hive table/partitions coexistes in more than one clusters - Key: HIVE-2612 URL: https://issues.apache.org/jira/browse/HIVE-2612 Project: Hive Issue Type: New Feature Components: Metastore Reporter: He Yongqiang Assignee: He Yongqiang 1) add cluster object into hive metastore 2) each partition/table has a creation cluster and a list of living clusters, and also data location in each cluster -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2720) Merge MetaStoreListener and HiveMetaHook interfaces
[ https://issues.apache.org/jira/browse/HIVE-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13189551#comment-13189551 ] Phabricator commented on HIVE-2720: --- cwsteinbach has requested changes to the revision HIVE-2720 [jira] Merge MetaStoreListener and HiveMetaHook interfaces. This also needs some test coverage. In particular I'd like to see testcases that cover the scenario where multiple hooks are registered. INLINE COMMENTS metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaHook.java:64 Right now pre-methods can also modify the input parameters before they are seen by the metastore. I don't think this should be allowed. We can prevent it by only passing the pre-methods copies of the input parameters, but that may be expensive. At the very least I think the javadoc needs to stipulate that pre-methods are *not* allowed to mutate the input parameters. metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaHook.java:54 and for some of the of the operations metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaHook.java:66 If multiple metahooks are registered, then it's also possible that the first metahook will raise an exception and the later metahooks won't see the event at all. This seems pretty bad from the standpoint of a person trying to write a audit plugin, since they risk not seeing a lot of operations that are failed by hooks that precede them in the list. Also, what happens if the second prehook throws an exception? Will the first posthook still get called with a failure? It's possible that the first prehook created state that will get cleared by the posthook, but this won't happen if the posthook is not called. I think the javadoc either needs to say that you (a) can't assume a posthook will get called just because the prehook was called, or else (b) say that the posthook will always get called if the prehook is called. Option (b) will make it much easier to write hooks at the expense of more complexity on the Hive side. metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaHook.java:65 What happens if a post-method throws an exception on an operation that has already been committed to the metastore? What does that exception mean, and how is Hive supposed to handle it? metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java:883 Where's the prehook? I noticed missing prehooks elsewhere as well. metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java:1287 This is a hack and is going to be broken by someone who doesn't understand that you're only calling get_database to trigger the hook. Also, the fact that pre/post hooks aren't symmetrical is going to cause a lot of grief for hook implementors. This needs to be fixed. This also introduces a time-of-check-to-time-of-use bug since you're not grabbing the DB and table definitions from within the same txn. metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java:1458 Is this a separate bug? Is it being tracked in a different JIRA? metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java:1624 This convention for triggering the hooks is not acceptable. I think code analysis tools will probably flag this since the db and tbl variables are not read. metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java:970 Implementations of what? Not sure if this method belongs here in the first place. metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java:42 What is the plan for removing this interface? Can we remove it on trunk as soon as HCatalog has migrated to the new interface? REVISION DETAIL https://reviews.facebook.net/D1299 Merge MetaStoreListener and HiveMetaHook interfaces --- Key: HIVE-2720 URL: https://issues.apache.org/jira/browse/HIVE-2720 Project: Hive Issue Type: Sub-task Components: JDBC, Metastore, ODBC, Security Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HIVE-2720.D1299.1.patch, HIVE-2720.D1299.2.patch, HIVE-2720.D1299.3.patch MetaStoreListener and HiveMetaHook both serve as a notification mechanism for metastore-related events. The former is used by hcat and the latter is by the hbase-storage handler, and invoked by the client. I propose to merge these interfaces, and extend the MetaStoreListener, to add most of the on- and pre- methods at the Thrift interface. This way, extending metastore will be easier, and validation, storage-driver notification, and enforcement can be delegated to individual listeners. Besides, more functionality can be plugged-in by Hcat at this level. -- This message is automatically generated by JIRA. If you think it was sent
[jira] [Updated] (HIVE-2727) add a testcase for partitioned view on union and base tables have index
[ https://issues.apache.org/jira/browse/HIVE-2727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2727: -- Attachment: HIVE-2727.D1323.2.patch heyongqiang updated the revision HIVE-2727 [jira] add a testcase for partitioned view on union and base tables have index. Reviewers: JIRA refresh to address namit's comments. REVISION DETAIL https://reviews.facebook.net/D1323 AFFECTED FILES ql/src/test/results/clientpositive/union_view.q.out ql/src/test/queries/clientpositive/union_view.q ql/src/java/org/apache/hadoop/hive/ql/optimizer/IndexUtils.java ql/src/java/org/apache/hadoop/hive/ql/index/IndexMetadataChangeTask.java ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java add a testcase for partitioned view on union and base tables have index --- Key: HIVE-2727 URL: https://issues.apache.org/jira/browse/HIVE-2727 Project: Hive Issue Type: Test Reporter: He Yongqiang Assignee: He Yongqiang Attachments: HIVE-2727.1.patch, HIVE-2727.D1323.1.patch, HIVE-2727.D1323.2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
