[jira] [Work started] (HIVE-2791) filter is still removed due to regression of HIVE-1538 althougth HIVE-2344
[ https://issues.apache.org/jira/browse/HIVE-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-2791 started by binlijin. filter is still removed due to regression of HIVE-1538 althougth HIVE-2344 -- Key: HIVE-2791 URL: https://issues.apache.org/jira/browse/HIVE-2791 Project: Hive Issue Type: Bug Reporter: binlijin Assignee: binlijin Attachments: HIVE-2791.2.patch, HIVE-2791.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2791) filter is still removed due to regression of HIVE-1538 althougth HIVE-2344
[ https://issues.apache.org/jira/browse/HIVE-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] binlijin updated HIVE-2791: --- Attachment: HIVE-2791.2.patch filter is still removed due to regression of HIVE-1538 althougth HIVE-2344 -- Key: HIVE-2791 URL: https://issues.apache.org/jira/browse/HIVE-2791 Project: Hive Issue Type: Bug Reporter: binlijin Assignee: binlijin Attachments: HIVE-2791.2.patch, HIVE-2791.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2791) filter is still removed due to regression of HIVE-1538 althougth HIVE-2344
[ https://issues.apache.org/jira/browse/HIVE-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] binlijin updated HIVE-2791: --- Status: Patch Available (was: In Progress) filter is still removed due to regression of HIVE-1538 althougth HIVE-2344 -- Key: HIVE-2791 URL: https://issues.apache.org/jira/browse/HIVE-2791 Project: Hive Issue Type: Bug Reporter: binlijin Assignee: binlijin Attachments: HIVE-2791.2.patch, HIVE-2791.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1250 - Still Failing
Changes for Build #1207 [namit] HIVE-2718 NPE in union followed by join (He Yongqiang via namit) Changes for Build #1208 Changes for Build #1209 Changes for Build #1210 [namit] HIVE-2674 get_partitions_ps throws TApplicationException if table doesn't exist (Kevin Wilfong via namit) Changes for Build #1211 [cws] HIVE-2203. Extend concat_ws() UDF to support arrays of strings (Zhenxiao Luo via cws) [cws] HIVE-2279. Implement sort(array) UDF (Zhenxiao Luo via cws) Changes for Build #1212 [hashutosh] HIVE-2589 : Newly created partition should inherit properties from table (Ashutosh Chauhan) Changes for Build #1213 Changes for Build #1214 Changes for Build #1215 Changes for Build #1216 [cws] HIVE-2724. Remove unused lib/log4j-1.2.15.jar (Andrew Bayer via cws) Changes for Build #1217 Changes for Build #1218 Changes for Build #1219 [hashutosh] HIVE-2665 : Support for metastore service specific HADOOP_OPTS environment setting (thw via hashutosh) Changes for Build #1220 [namit] HIVE-2727 add a testcase for partitioned view on union and base tables have index (He Yongqiang via namit) Changes for Build #1221 [hashutosh] HIVE-2746 : Metastore client doesn't log properly in case of connection failure to server (hashutosh) [cws] HIVE-2698 [jira] Enable Hadoop-1.0.0 in Hive (Enis Söztutar via Carl Steinbach) Summary: third version of the patch Hadoop-1.0.0 is recently released, which is AFAIK, API compatible to the 0.20S release. Test Plan: EMPTY Reviewers: JIRA, cwsteinbach Reviewed By: cwsteinbach CC: cwsteinbach, enis Differential Revision: https://reviews.facebook.net/D1389 Changes for Build #1222 [namit] HIVE-2750 Hive multi group by single reducer optimization causes invalid column reference error (Kevin Wilfong via namit) Changes for Build #1223 Changes for Build #1224 [cws] HIVE-2734 [jira] Fix some nondeterministic test output (Zhenxiao Luo via Carl Steinbach) Summary: HIVE-2734: Fix some nondeterministic test output Many Hive query tests lack an ORDER BY clause, and consequently the ordering of the rows in the result set is nondeterministic: groupby1_limit input11_limit input1_limit input_lazyserde join18_multi_distinct join_1to1 join_casesensitive join_filters join_nulls merge3 rcfile_columnar rcfile_lazydecompress rcfile_union sample10 udf_sentences union24 columnarserde_create_shortcut combine1 global_limit Test Plan: EMPTY Reviewers: JIRA, cwsteinbach Reviewed By: cwsteinbach CC: zhenxiao, cwsteinbach Differential Revision: https://reviews.facebook.net/D1449 [namit] HIVE-2754 NPE in union with lateral view (Yongqiang He via namit) Changes for Build #1225 Changes for Build #1226 Changes for Build #1227 [namit] HIVE-2755 union follwowed by union_subq does not work if the subquery union has reducers (He Yongqiang via namit) Changes for Build #1228 Changes for Build #1229 [hashutosh] HIVE-2735: PlanUtils.configureTableJobPropertiesForStorageHandler() is not called for partitioned table (sushanth via ashutosh) Changes for Build #1230 [cws] HIVE-2760 [jira] TestCliDriver should log elapsed time Summary: HIVE-2760. TestCliDriver should log elapsed time Test Plan: EMPTY Reviewers: JIRA, ashutoshc Reviewed By: ashutoshc CC: ashutoshc, cwsteinbach Differential Revision: https://reviews.facebook.net/D1503 [cws] HIVE-2662 [jira] Add Ant configuration property for dumping classpath of tests Summary: HIVE-2662. Add Ant configuration property for dumping classpath of tests Test Plan: EMPTY Reviewers: JIRA, jsichi, ashutoshc Reviewed By: ashutoshc CC: ashutoshc Differential Revision: https://reviews.facebook.net/D903 Changes for Build #1231 [hashutosh] HIVE_2645: Hive Web Server startup messages logs incorrect path it is searching for WAR (Edward Capriolo via Ashutosh Chauhan) Changes for Build #1232 Changes for Build #1233 [sdong] HIVE-2249 When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double (Zhiqiu Kong via Siying Dong) Changes for Build #1234 Changes for Build #1235 [heyongqiang] HIVE-2765 hbase handler uses ZooKeeperConnectionException which is not compatible with HBase versions other than 0.89 (Pei Yue via He Yongqiang) Changes for Build #1236 Changes for Build #1237 Changes for Build #1238 [heyongqiang] HIVE-2772 [jira] make union31.q deterministic (Namit Jain via Yongqiang He) Summary: https://issues.apache.org/jira/browse/HIVE-2772 HIVE-2772 Test Plan: EMPTY Reviewers: JIRA, ashutoshc Reviewed By: ashutoshc CC: ashutoshc Differential Revision: https://reviews.facebook.net/D1557 [kevinwilfong] HIVE-2758 Metastore is caching too aggressively (Kevin Wilfong reviewed by Carl Steinbach) Changes for Build #1239 Changes for Build #1240 [namit] HIVE-2762 Alter Table Partition Concatenate Fails On Certain Characters (Kevin Wilfong via namit) Changes for Build #1241 [namit] HIVE-2756 Views should be added to
[jira] [Commented] (HIVE-2749) CONV returns incorrect results sometimes
[ https://issues.apache.org/jira/browse/HIVE-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13204428#comment-13204428 ] Hudson commented on HIVE-2749: -- Integrated in Hive-trunk-h0.21 #1250 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1250/]) HIVE-2749 CONV returns incorrect results sometimes (Jonathan Chang via namit) namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=124 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFConv.java * /hive/trunk/ql/src/test/queries/clientpositive/udf_conv.q * /hive/trunk/ql/src/test/results/clientpositive/udf_conv.q.out CONV returns incorrect results sometimes Key: HIVE-2749 URL: https://issues.apache.org/jira/browse/HIVE-2749 Project: Hive Issue Type: Bug Reporter: Jonathan Chang Assignee: Jonathan Chang Attachments: HIVE-2749.D1437.1.patch, HIVE-2749.D1437.2.patch, HIVE-2749.D1437.2.patch ...because it fails to reset state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2749) CONV returns incorrect results sometimes
[ https://issues.apache.org/jira/browse/HIVE-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2749: --- Component/s: UDF Fix Version/s: 0.9.0 CONV returns incorrect results sometimes Key: HIVE-2749 URL: https://issues.apache.org/jira/browse/HIVE-2749 Project: Hive Issue Type: Bug Components: UDF Reporter: Jonathan Chang Assignee: Jonathan Chang Fix For: 0.9.0 Attachments: HIVE-2749.D1437.1.patch, HIVE-2749.D1437.2.patch, HIVE-2749.D1437.2.patch ...because it fails to reset state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2676) The row count that loaded to a table may not right
[ https://issues.apache.org/jira/browse/HIVE-2676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] binlijin updated HIVE-2676: --- Status: Patch Available (was: Open) The row count that loaded to a table may not right - Key: HIVE-2676 URL: https://issues.apache.org/jira/browse/HIVE-2676 Project: Hive Issue Type: Improvement Reporter: binlijin Priority: Minor Labels: patch Attachments: HIVE-2676.patch create table tablename as SELECT *** At the end hive will print a number that show how many Rows loaded to the tablename, but sometimes the number is not right. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2790) Grouping on constant expressions is broken on 0.23
[ https://issues.apache.org/jira/browse/HIVE-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2790: - Component/s: Query Processor Grouping on constant expressions is broken on 0.23 -- Key: HIVE-2790 URL: https://issues.apache.org/jira/browse/HIVE-2790 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Zhenxiao Luo Assignee: Zhenxiao Luo Query result(-30.33) is missing in the running output: [junit] Begin query: groupby_neg_float.q [junit] 12/01/24 10:57:53 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir [junit] 12/01/24 10:57:53 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir [junit] 12/01/24 10:57:58 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir [junit] 12/01/24 10:57:58 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/groupby_neg_float.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/groupby_neg_float.q.out [junit] 14c14,15 [junit] POSTHOOK: Output: file:/tmp/cloudera/hive_2012-01-24_10-57-51_652_2049459353574360528/-mr-1 [junit] — [junit] POSTHOOK: Output: file:/data/users/njain/hive5/hive5/build/ql/tmp/815331462/1 [junit] -30.33 [junit] 28c29,30 [junit] POSTHOOK: Output: file:/tmp/cloudera/hive_2012-01-24_10-57-56_909_7738265803221872068/-mr-1 [junit] — [junit] POSTHOOK: Output: file:/data/users/njain/hive5/hive5/build/ql/tmp/2082702070/1 [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1 [junit] See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. [junit] at junit.framework.Assert.fail(Assert.java:50) [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_neg_float(TestCliDriver.java:129) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [junit] at java.lang.reflect.Method.invoke(Method.java:616) [junit] at junit.framework.TestCase.runTest(TestCase.java:168) [junit] at junit.framework.TestCase.runBare(TestCase.java:134) [junit] at junit.framework.TestResult$1.protect(TestResult.java:110) [junit] at junit.framework.TestResult.runProtected(TestResult.java:128) [junit] at junit.framework.TestResult.run(TestResult.java:113) [junit] at junit.framework.TestCase.run(TestCase.java:124) [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243) [junit] at junit.framework.TestSuite.run(TestSuite.java:238) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768) [junit] Exception: Client execution results failed with error code = 1 [junit] See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. [junit] -30.33 [junit] See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs.) When trying the following(GROUP BY on Floating Point type and string type): FROM src SELECT -30.33 GROUP BY -30.33 LIMIT 1; FROM src SELECT '-30.33' GROUP BY '-30.33' LIMIT 1; Get NULL in result -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2790) Grouping on constant expressions is broken on 0.23
[ https://issues.apache.org/jira/browse/HIVE-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2790: - Summary: Grouping on constant expressions is broken on 0.23 (was: When integrating into MapReduce2, GROUP BY on Floating Point and String types are broken) Grouping on constant expressions is broken on 0.23 -- Key: HIVE-2790 URL: https://issues.apache.org/jira/browse/HIVE-2790 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Zhenxiao Luo Assignee: Zhenxiao Luo Query result(-30.33) is missing in the running output: [junit] Begin query: groupby_neg_float.q [junit] 12/01/24 10:57:53 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir [junit] 12/01/24 10:57:53 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir [junit] 12/01/24 10:57:58 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir [junit] 12/01/24 10:57:58 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/groupby_neg_float.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/groupby_neg_float.q.out [junit] 14c14,15 [junit] POSTHOOK: Output: file:/tmp/cloudera/hive_2012-01-24_10-57-51_652_2049459353574360528/-mr-1 [junit] — [junit] POSTHOOK: Output: file:/data/users/njain/hive5/hive5/build/ql/tmp/815331462/1 [junit] -30.33 [junit] 28c29,30 [junit] POSTHOOK: Output: file:/tmp/cloudera/hive_2012-01-24_10-57-56_909_7738265803221872068/-mr-1 [junit] — [junit] POSTHOOK: Output: file:/data/users/njain/hive5/hive5/build/ql/tmp/2082702070/1 [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1 [junit] See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. [junit] at junit.framework.Assert.fail(Assert.java:50) [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_neg_float(TestCliDriver.java:129) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [junit] at java.lang.reflect.Method.invoke(Method.java:616) [junit] at junit.framework.TestCase.runTest(TestCase.java:168) [junit] at junit.framework.TestCase.runBare(TestCase.java:134) [junit] at junit.framework.TestResult$1.protect(TestResult.java:110) [junit] at junit.framework.TestResult.runProtected(TestResult.java:128) [junit] at junit.framework.TestResult.run(TestResult.java:113) [junit] at junit.framework.TestCase.run(TestCase.java:124) [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243) [junit] at junit.framework.TestSuite.run(TestSuite.java:238) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768) [junit] Exception: Client execution results failed with error code = 1 [junit] See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. [junit] -30.33 [junit] See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs.) When trying the following(GROUP BY on Floating Point type and string type): FROM src SELECT -30.33 GROUP BY -30.33 LIMIT 1; FROM src SELECT '-30.33' GROUP BY '-30.33' LIMIT 1; Get NULL in result -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Post-gres Hive Metastore
Hello, Does anyone still use postgres as the Hive metastore SQL backend? I'm in the process of writing scripts to upgrade the metastore schema as part of https://issues.apache.org/jira/browse/HIVE-2612 I noticed that the postgres metastore upgrade scripts are missing the script for HIVE-2246. This patch added a new table COLUMNS_V2 and completely rearranged the way column information is stored in the metastore. Without this change, I don't know how anyone could continue to use postgres for Hive after upgrading to HIVE-2246. Thanks, Kevin Wilfong
FW: Post-gres Hive Metastore
-Original Message- From: Kevin Wilfong [mailto:kevinwilf...@fb.com] Sent: Thursday, February 09, 2012 10:59 AM To: dev@hive.apache.org Subject: Post-gres Hive Metastore Hello, Does anyone still use postgres as the Hive metastore SQL backend? I'm in the process of writing scripts to upgrade the metastore schema as part of https://issues.apache.org/jira/browse/HIVE-2612 I noticed that the postgres metastore upgrade scripts are missing the script for HIVE-2246. This patch added a new table COLUMNS_V2 and completely rearranged the way column information is stored in the metastore. Without this change, I don't know how anyone could continue to use postgres for Hive after upgrading to HIVE-2246. Thanks, Kevin Wilfong
Hive-trunk-h0.21 - Build # 1251 - Still Failing
Changes for Build #1208 Changes for Build #1209 Changes for Build #1210 [namit] HIVE-2674 get_partitions_ps throws TApplicationException if table doesn't exist (Kevin Wilfong via namit) Changes for Build #1211 [cws] HIVE-2203. Extend concat_ws() UDF to support arrays of strings (Zhenxiao Luo via cws) [cws] HIVE-2279. Implement sort(array) UDF (Zhenxiao Luo via cws) Changes for Build #1212 [hashutosh] HIVE-2589 : Newly created partition should inherit properties from table (Ashutosh Chauhan) Changes for Build #1213 Changes for Build #1214 Changes for Build #1215 Changes for Build #1216 [cws] HIVE-2724. Remove unused lib/log4j-1.2.15.jar (Andrew Bayer via cws) Changes for Build #1217 Changes for Build #1218 Changes for Build #1219 [hashutosh] HIVE-2665 : Support for metastore service specific HADOOP_OPTS environment setting (thw via hashutosh) Changes for Build #1220 [namit] HIVE-2727 add a testcase for partitioned view on union and base tables have index (He Yongqiang via namit) Changes for Build #1221 [hashutosh] HIVE-2746 : Metastore client doesn't log properly in case of connection failure to server (hashutosh) [cws] HIVE-2698 [jira] Enable Hadoop-1.0.0 in Hive (Enis Söztutar via Carl Steinbach) Summary: third version of the patch Hadoop-1.0.0 is recently released, which is AFAIK, API compatible to the 0.20S release. Test Plan: EMPTY Reviewers: JIRA, cwsteinbach Reviewed By: cwsteinbach CC: cwsteinbach, enis Differential Revision: https://reviews.facebook.net/D1389 Changes for Build #1222 [namit] HIVE-2750 Hive multi group by single reducer optimization causes invalid column reference error (Kevin Wilfong via namit) Changes for Build #1223 Changes for Build #1224 [cws] HIVE-2734 [jira] Fix some nondeterministic test output (Zhenxiao Luo via Carl Steinbach) Summary: HIVE-2734: Fix some nondeterministic test output Many Hive query tests lack an ORDER BY clause, and consequently the ordering of the rows in the result set is nondeterministic: groupby1_limit input11_limit input1_limit input_lazyserde join18_multi_distinct join_1to1 join_casesensitive join_filters join_nulls merge3 rcfile_columnar rcfile_lazydecompress rcfile_union sample10 udf_sentences union24 columnarserde_create_shortcut combine1 global_limit Test Plan: EMPTY Reviewers: JIRA, cwsteinbach Reviewed By: cwsteinbach CC: zhenxiao, cwsteinbach Differential Revision: https://reviews.facebook.net/D1449 [namit] HIVE-2754 NPE in union with lateral view (Yongqiang He via namit) Changes for Build #1225 Changes for Build #1226 Changes for Build #1227 [namit] HIVE-2755 union follwowed by union_subq does not work if the subquery union has reducers (He Yongqiang via namit) Changes for Build #1228 Changes for Build #1229 [hashutosh] HIVE-2735: PlanUtils.configureTableJobPropertiesForStorageHandler() is not called for partitioned table (sushanth via ashutosh) Changes for Build #1230 [cws] HIVE-2760 [jira] TestCliDriver should log elapsed time Summary: HIVE-2760. TestCliDriver should log elapsed time Test Plan: EMPTY Reviewers: JIRA, ashutoshc Reviewed By: ashutoshc CC: ashutoshc, cwsteinbach Differential Revision: https://reviews.facebook.net/D1503 [cws] HIVE-2662 [jira] Add Ant configuration property for dumping classpath of tests Summary: HIVE-2662. Add Ant configuration property for dumping classpath of tests Test Plan: EMPTY Reviewers: JIRA, jsichi, ashutoshc Reviewed By: ashutoshc CC: ashutoshc Differential Revision: https://reviews.facebook.net/D903 Changes for Build #1231 [hashutosh] HIVE_2645: Hive Web Server startup messages logs incorrect path it is searching for WAR (Edward Capriolo via Ashutosh Chauhan) Changes for Build #1232 Changes for Build #1233 [sdong] HIVE-2249 When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double (Zhiqiu Kong via Siying Dong) Changes for Build #1234 Changes for Build #1235 [heyongqiang] HIVE-2765 hbase handler uses ZooKeeperConnectionException which is not compatible with HBase versions other than 0.89 (Pei Yue via He Yongqiang) Changes for Build #1236 Changes for Build #1237 Changes for Build #1238 [heyongqiang] HIVE-2772 [jira] make union31.q deterministic (Namit Jain via Yongqiang He) Summary: https://issues.apache.org/jira/browse/HIVE-2772 HIVE-2772 Test Plan: EMPTY Reviewers: JIRA, ashutoshc Reviewed By: ashutoshc CC: ashutoshc Differential Revision: https://reviews.facebook.net/D1557 [kevinwilfong] HIVE-2758 Metastore is caching too aggressively (Kevin Wilfong reviewed by Carl Steinbach) Changes for Build #1239 Changes for Build #1240 [namit] HIVE-2762 Alter Table Partition Concatenate Fails On Certain Characters (Kevin Wilfong via namit) Changes for Build #1241 [namit] HIVE-2756 Views should be added to the inputs of queries. (Yongqiang He via namit) Changes for Build #1242 Changes for Build #1243
[jira] [Created] (HIVE-2795) View partitions do not have a storage descriptor
View partitions do not have a storage descriptor Key: HIVE-2795 URL: https://issues.apache.org/jira/browse/HIVE-2795 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Kevin Wilfong -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2795) View partitions do not have a storage descriptor
[ https://issues.apache.org/jira/browse/HIVE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-2795: Description: Besides being an inconsistency, it causes errors. Calling describe formatted on a view partition throws an exception java.lang.NullPointerException at org.apache.hadoop.hive.ql.metadata.Partition.getCols(Partition.java:505) at org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:2570) because it does not have a column descriptor, which is part of the storage descriptor. View partitions do not have a storage descriptor Key: HIVE-2795 URL: https://issues.apache.org/jira/browse/HIVE-2795 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Kevin Wilfong Besides being an inconsistency, it causes errors. Calling describe formatted on a view partition throws an exception java.lang.NullPointerException at org.apache.hadoop.hive.ql.metadata.Partition.getCols(Partition.java:505) at org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:2570) because it does not have a column descriptor, which is part of the storage descriptor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
partitioned views
We found a bug recently, that partitioned views do not have storage descriptors. https://issues.apache.org/jira/browse/HIVE-2795 We will fix this bug, but the existing data needs to be upgraded (if you really care about the describe output of a view partition). It seems fairly difficult to come up with a sql script to change the existing data. One other way to fix this problem would be to list all view partitions and then add them again after the patch. For all partitions p of view v: alter view v drop partition p; alter view v add partition p; How many people are using partitioned views ? Is the above solution OK for them ? Thanks, -namit
[jira] [Updated] (HIVE-2795) View partitions do not have a storage descriptor
[ https://issues.apache.org/jira/browse/HIVE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2795: -- Attachment: HIVE-2795.D1683.1.patch kevinwilfong requested code review of HIVE-2795 [jira] View partitions do not have a storage descriptor. Reviewers: JIRA https://issues.apache.org/jira/browse/HIVE-2795 Updated the metastore so that if a partition is added to a view and the partition's StorageDescriptor is null it will use a copy of the table's StorageDescriptor. Besides being an inconsistency, it causes errors. Calling describe formatted on a view partition throws an exception java.lang.NullPointerException at org.apache.hadoop.hive.ql.metadata.Partition.getCols(Partition.java:505) at org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:2570) because it does not have a column descriptor, which is part of the storage descriptor. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D1683 AFFECTED FILES metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/3591/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. View partitions do not have a storage descriptor Key: HIVE-2795 URL: https://issues.apache.org/jira/browse/HIVE-2795 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Kevin Wilfong Attachments: HIVE-2795.D1683.1.patch Besides being an inconsistency, it causes errors. Calling describe formatted on a view partition throws an exception java.lang.NullPointerException at org.apache.hadoop.hive.ql.metadata.Partition.getCols(Partition.java:505) at org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:2570) because it does not have a column descriptor, which is part of the storage descriptor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2795) View partitions do not have a storage descriptor
[ https://issues.apache.org/jira/browse/HIVE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-2795: Status: Patch Available (was: Open) View partitions do not have a storage descriptor Key: HIVE-2795 URL: https://issues.apache.org/jira/browse/HIVE-2795 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Kevin Wilfong Attachments: HIVE-2795.D1683.1.patch Besides being an inconsistency, it causes errors. Calling describe formatted on a view partition throws an exception java.lang.NullPointerException at org.apache.hadoop.hive.ql.metadata.Partition.getCols(Partition.java:505) at org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:2570) because it does not have a column descriptor, which is part of the storage descriptor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2795) View partitions do not have a storage descriptor
[ https://issues.apache.org/jira/browse/HIVE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2795: -- Attachment: HIVE-2795.D1683.2.patch kevinwilfong updated the revision HIVE-2795 [jira] View partitions do not have a storage descriptor. Reviewers: JIRA, njain Added a test to verify calling describe formatted on a view partition succeeds. REVISION DETAIL https://reviews.facebook.net/D1683 AFFECTED FILES metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java ql/src/test/results/clientpositive/describe_formatted_view_partitioned.q.out ql/src/test/queries/clientpositive/describe_formatted_view_partitioned.q View partitions do not have a storage descriptor Key: HIVE-2795 URL: https://issues.apache.org/jira/browse/HIVE-2795 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Kevin Wilfong Attachments: HIVE-2795.D1683.1.patch, HIVE-2795.D1683.2.patch Besides being an inconsistency, it causes errors. Calling describe formatted on a view partition throws an exception java.lang.NullPointerException at org.apache.hadoop.hive.ql.metadata.Partition.getCols(Partition.java:505) at org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:2570) because it does not have a column descriptor, which is part of the storage descriptor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2795) View partitions do not have a storage descriptor
[ https://issues.apache.org/jira/browse/HIVE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205038#comment-13205038 ] Phabricator commented on HIVE-2795: --- njain has accepted the revision HIVE-2795 [jira] View partitions do not have a storage descriptor. REVISION DETAIL https://reviews.facebook.net/D1683 View partitions do not have a storage descriptor Key: HIVE-2795 URL: https://issues.apache.org/jira/browse/HIVE-2795 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Kevin Wilfong Attachments: HIVE-2795.D1683.1.patch, HIVE-2795.D1683.2.patch Besides being an inconsistency, it causes errors. Calling describe formatted on a view partition throws an exception java.lang.NullPointerException at org.apache.hadoop.hive.ql.metadata.Partition.getCols(Partition.java:505) at org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:2570) because it does not have a column descriptor, which is part of the storage descriptor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2612) support hive table/partitions exists in more than one region
[ https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2612: - Summary: support hive table/partitions exists in more than one region (was: support hive table/partitions coexists in more than one region) support hive table/partitions exists in more than one region Key: HIVE-2612 URL: https://issues.apache.org/jira/browse/HIVE-2612 Project: Hive Issue Type: New Feature Components: Metastore Reporter: He Yongqiang Assignee: Namit Jain Attachments: HIVE-2612.1.patch, HIVE-2612.2.patch.txt, HIVE-2612.3.patch.txt, HIVE-2612.4.patch.txt, HIVE-2612.D1569.1.patch, HIVE-2612.D1569.2.patch, HIVE-2612.D1569.3.patch, HIVE-2612.D1569.4.patch, HIVE-2612.D1569.5.patch, HIVE-2612.D1569.6.patch, HIVE-2612.D1569.7.patch 1) add region object into hive metastore 2) each partition/table has a primary region and a list of living regions, and also data location in each region -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2796) Support auto completion for hive configs in CliDriver
Support auto completion for hive configs in CliDriver - Key: HIVE-2796 URL: https://issues.apache.org/jira/browse/HIVE-2796 Project: Hive Issue Type: Improvement Reporter: Navis Priority: Trivial It's very cumbersome to memorize hive conf vars. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2796) Support auto completion for hive configs in CliDriver
[ https://issues.apache.org/jira/browse/HIVE-2796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2796: -- Attachment: HIVE-2796.D1689.1.patch navis requested code review of HIVE-2796 [jira] Support auto completion for hive configs in CliDriver. Reviewers: JIRA DPAL-747 Support auto completion for hive configs in CliDriver It's very cumbersome to memorize hive conf vars. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D1689 AFFECTED FILES cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/3603/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. Support auto completion for hive configs in CliDriver - Key: HIVE-2796 URL: https://issues.apache.org/jira/browse/HIVE-2796 Project: Hive Issue Type: Improvement Reporter: Navis Priority: Trivial Attachments: HIVE-2796.D1689.1.patch It's very cumbersome to memorize hive conf vars. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2795) View partitions do not have a storage descriptor
[ https://issues.apache.org/jira/browse/HIVE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2795: -- Attachment: HIVE-2795.D1683.3.patch kevinwilfong updated the revision HIVE-2795 [jira] View partitions do not have a storage descriptor. Reviewers: JIRA, njain Included a Python script to go through the metastore and update all view partitions to have a Storage Descriptor. It does so by dropping the partition and adding the partition. As the only view partition data that can be modified is the name, no data will be lost by doing this. Detailed instructions and warnings are included in the script. REVISION DETAIL https://reviews.facebook.net/D1683 AFFECTED FILES metastore/scripts/upgrade/001-HIVE-2795.update_view_partitions.py metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java ql/src/test/results/clientpositive/describe_formatted_view_partitioned.q.out ql/src/test/queries/clientpositive/describe_formatted_view_partitioned.q View partitions do not have a storage descriptor Key: HIVE-2795 URL: https://issues.apache.org/jira/browse/HIVE-2795 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Kevin Wilfong Attachments: HIVE-2795.D1683.1.patch, HIVE-2795.D1683.2.patch, HIVE-2795.D1683.3.patch Besides being an inconsistency, it causes errors. Calling describe formatted on a view partition throws an exception java.lang.NullPointerException at org.apache.hadoop.hive.ql.metadata.Partition.getCols(Partition.java:505) at org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:2570) because it does not have a column descriptor, which is part of the storage descriptor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2795) View partitions do not have a storage descriptor
[ https://issues.apache.org/jira/browse/HIVE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2795: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed. Thanks Kevin View partitions do not have a storage descriptor Key: HIVE-2795 URL: https://issues.apache.org/jira/browse/HIVE-2795 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Kevin Wilfong Attachments: HIVE-2795.D1683.1.patch, HIVE-2795.D1683.2.patch, HIVE-2795.D1683.3.patch Besides being an inconsistency, it causes errors. Calling describe formatted on a view partition throws an exception java.lang.NullPointerException at org.apache.hadoop.hive.ql.metadata.Partition.getCols(Partition.java:505) at org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:2570) because it does not have a column descriptor, which is part of the storage descriptor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2796) Support auto completion for hive configs in CliDriver
[ https://issues.apache.org/jira/browse/HIVE-2796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2796: -- Attachment: HIVE-2796.D1689.2.patch navis updated the revision HIVE-2796 [jira] Support auto completion for hive configs in CliDriver. Reviewers: JIRA 1. Removed trailing space after completion 2. Fix not showing s-started keywords REVISION DETAIL https://reviews.facebook.net/D1689 AFFECTED FILES cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java Support auto completion for hive configs in CliDriver - Key: HIVE-2796 URL: https://issues.apache.org/jira/browse/HIVE-2796 Project: Hive Issue Type: Improvement Reporter: Navis Priority: Trivial Attachments: HIVE-2796.D1689.1.patch, HIVE-2796.D1689.2.patch It's very cumbersome to memorize hive conf vars. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
