[jira] [Commented] (HIVE-2651) The variable hive.exec.mode.local.auto.tasks.max should be changed
[ https://issues.apache.org/jira/browse/HIVE-2651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170013#comment-13170013 ] Phabricator commented on HIVE-2651: --- heyongqiang has committed the revision HIVE-2651 [jira] The variable hive.exec.mode.local.auto.tasks.max should be changed. REVISION DETAIL https://reviews.facebook.net/D861 COMMIT https://reviews.facebook.net/rHIVE1214647 The variable hive.exec.mode.local.auto.tasks.max should be changed -- Key: HIVE-2651 URL: https://issues.apache.org/jira/browse/HIVE-2651 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Namit Jain Attachments: HIVE-2651.D861.1.patch It should be called hive.exec.mode.local.auto.input.files.max instead. The number of input files are checked currently. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-727) Hive Server getSchema() returns wrong schema for Explain queries
[ https://issues.apache.org/jira/browse/HIVE-727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170036#comment-13170036 ] Hudson commented on HIVE-727: - Integrated in Hive-trunk-h0.23.0 #26 (See [https://builds.apache.org/job/Hive-trunk-h0.23.0/26/]) HIVE-727. Hive Server getSchema() returns wrong schema for 'Explain' queries (Prasad Mujumdar via cws) cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1214636 Files : * /hive/trunk/jdbc/src/test/org/apache/hadoop/hive/jdbc/TestJdbcDriver.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExplainTask.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ExplainSemanticAnalyzer.java Hive Server getSchema() returns wrong schema for Explain queries -- Key: HIVE-727 URL: https://issues.apache.org/jira/browse/HIVE-727 Project: Hive Issue Type: Bug Components: JDBC Reporter: Eric Hwang Assignee: Prasad Mujumdar Fix For: 0.9.0 Attachments: HIVE-727.patch, HIVE-727.patch.1, Hive-727.patch The Hive Server's getSchema() function will return a schema with zero fields when executing an Explain... query. A quick fix might be to set the default schema to be exactly one column of type string. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2617) Insert overwrite table db.tname fails if partition already exists
[ https://issues.apache.org/jira/browse/HIVE-2617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170054#comment-13170054 ] Hudson commented on HIVE-2617: -- Integrated in Hive-trunk-h0.21 #1147 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1147/]) HIVE-2617 Insert overwrite table db.tname fails if partition already exists (Chinna Rao Lalam via namit) namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1214592 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java * /hive/trunk/ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q * /hive/trunk/ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out Insert overwrite table db.tname fails if partition already exists -- Key: HIVE-2617 URL: https://issues.apache.org/jira/browse/HIVE-2617 Project: Hive Issue Type: Bug Components: Metastore Reporter: Aniket Mokashi Assignee: Chinna Rao Lalam Attachments: HIVE-2617.1.patch, HIVE-2617.D843.1.patch, HIVE-2617.patch Insert Overwrite table db.tname fails if partition already exists. For example- insert overwrite table db.tname PARTITION(part='p') select .. from t2 where part='p'; fails if partition 'p' already exists. Workaround is - use db; and the fire the command. From the source code- alterPartition(tbl.getTableName(), new Partition(tbl, tpart)); takes String tablename as argument and loses db information. Table table = newTable(tablename) is called to retrieve table from name. But, it relies on currentDatabase value (hence the workaround). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1147 - Still Failing
Changes for Build #1144 [jvs] HIVE-1040 [jira] use sed rather than diff for masking out noise in diff-based tests (Marek Sapota via John Sichi) Summary: Replace diff -I with regex masking in Java The current diff -I approach has two problems: (1) it does not allow resolution finer than line-level, so it's impossible to mask out pattern occurrences within a line, and (2) it produces unmasked files, so if you run diff on the command line to compare the result .q.out with the checked-in file, you see the noise. My suggestion is to first run sed to replace noise patterns with an unlikely-to-occur string like ZYZZYZVA, and then diff the pre-masked files without using any -I. This would require a one-time hit to update all existing .q.out files so that they would contain the pre-masked results. Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: jsichi Differential Revision: 597 Changes for Build #1145 Changes for Build #1146 [namit] HIVE-2640 Add alterPartition to AlterHandler interface (Kevin Wilfong via namit) Changes for Build #1147 [namit] HIVE-2617 Insert overwrite table db.tname fails if partition already exists (Chinna Rao Lalam via namit) 2 tests failed. REGRESSION: org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1(TestNegativeCliDriver.java:9440) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) FAILED: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_inputddl5 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_inputddl5(TestCliDriver.java:16628) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1147) Status: Still Failing Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1147/ to view the results.
[jira] [Updated] (HIVE-2602) add support for insert partition overwrite(...) if not exists
[ https://issues.apache.org/jira/browse/HIVE-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinna Rao Lalam updated HIVE-2602: --- Attachment: HIVE-2602.3.patch add support for insert partition overwrite(...) if not exists - Key: HIVE-2602 URL: https://issues.apache.org/jira/browse/HIVE-2602 Project: Hive Issue Type: New Feature Reporter: Namit Jain Assignee: Chinna Rao Lalam Attachments: HIVE-2602.1.patch, HIVE-2602.2.patch, HIVE-2602.3.patch, HIVE-2602.D579.1.patch, HIVE-2602.D879.1.patch, HIVE-2602.patch INSERT OVERWRITE TABLE X PARTITION (a=b, c=d) IF NOT EXISTS ... The partition should be created and written if and only if it's not there already. The support can be added for dynamic partitions in the future, but this jira is for adding this support for static partitions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2602) add support for insert partition overwrite(...) if not exists
[ https://issues.apache.org/jira/browse/HIVE-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinna Rao Lalam updated HIVE-2602: --- Status: Patch Available (was: Open) add support for insert partition overwrite(...) if not exists - Key: HIVE-2602 URL: https://issues.apache.org/jira/browse/HIVE-2602 Project: Hive Issue Type: New Feature Reporter: Namit Jain Assignee: Chinna Rao Lalam Attachments: HIVE-2602.1.patch, HIVE-2602.2.patch, HIVE-2602.3.patch, HIVE-2602.D579.1.patch, HIVE-2602.D879.1.patch, HIVE-2602.patch INSERT OVERWRITE TABLE X PARTITION (a=b, c=d) IF NOT EXISTS ... The partition should be created and written if and only if it's not there already. The support can be added for dynamic partitions in the future, but this jira is for adding this support for static partitions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1892) show functions also returns internal operators
[ https://issues.apache.org/jira/browse/HIVE-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyadarshini updated HIVE-1892: Attachment: HIVE-1892.patch show functions also returns internal operators -- Key: HIVE-1892 URL: https://issues.apache.org/jira/browse/HIVE-1892 Project: Hive Issue Type: Bug Components: Security Reporter: Namit Jain Attachments: HIVE-1892.patch show functions: returns bigint etc. in its outputs, which are not valid external functions -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1892) show functions also returns internal operators
[ https://issues.apache.org/jira/browse/HIVE-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyadarshini updated HIVE-1892: Status: Patch Available (was: Open) show functions also returns internal operators -- Key: HIVE-1892 URL: https://issues.apache.org/jira/browse/HIVE-1892 Project: Hive Issue Type: Bug Components: Security Reporter: Namit Jain Attachments: HIVE-1892.patch show functions: returns bigint etc. in its outputs, which are not valid external functions -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1892) show functions also returns internal operators
[ https://issues.apache.org/jira/browse/HIVE-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170198#comment-13170198 ] Priyadarshini commented on HIVE-1892: - FunctionRegistry.getFunctionNames() will return the set of all registered functions including the primitive constants. So before sending to outstream, primitives will be removed and only the registered function names will be wriiten to outStream. show functions also returns internal operators -- Key: HIVE-1892 URL: https://issues.apache.org/jira/browse/HIVE-1892 Project: Hive Issue Type: Bug Components: Security Reporter: Namit Jain Attachments: HIVE-1892.patch show functions: returns bigint etc. in its outputs, which are not valid external functions -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2651) The variable hive.exec.mode.local.auto.tasks.max should be changed
[ https://issues.apache.org/jira/browse/HIVE-2651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170229#comment-13170229 ] Hudson commented on HIVE-2651: -- Integrated in Hive-trunk-h0.21 #1148 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1148/]) HIVE-2651 [jira] The variable hive.exec.mode.local.auto.tasks.max should be changed (Namit Jain via Yongqiang He) Summary: HIVE-2651 It should be called hive.exec.mode.local.auto.input.files.max instead. The number of input files are checked currently. Test Plan: EMPTY Reviewers: JIRA, heyongqiang Reviewed By: heyongqiang CC: heyongqiang Differential Revision: 861 heyongqiang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1214647 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java * /hive/trunk/ql/src/test/queries/clientpositive/sample_islocalmode_hook.q The variable hive.exec.mode.local.auto.tasks.max should be changed -- Key: HIVE-2651 URL: https://issues.apache.org/jira/browse/HIVE-2651 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Namit Jain Attachments: HIVE-2651.D861.1.patch It should be called hive.exec.mode.local.auto.input.files.max instead. The number of input files are checked currently. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2611) Make index table output of create index command if index is table based
[ https://issues.apache.org/jira/browse/HIVE-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170230#comment-13170230 ] Hudson commented on HIVE-2611: -- Integrated in Hive-trunk-h0.21 #1148 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1148/]) HIVE-2611 Make index table output of create index command if index is table based (Kevin Wilfong via namit) namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1214622 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java * /hive/trunk/ql/src/test/results/clientnegative/alter_concatenate_indexed_table.q.out * /hive/trunk/ql/src/test/results/clientnegative/index_bitmap_no_map_aggr.q.out * /hive/trunk/ql/src/test/results/clientnegative/index_compact_entry_limit.q.out * /hive/trunk/ql/src/test/results/clientnegative/index_compact_size_limit.q.out * /hive/trunk/ql/src/test/results/clientnegative/merge_negative_1.q.out * /hive/trunk/ql/src/test/results/clientpositive/alter_concatenate_indexed_table.q.out * /hive/trunk/ql/src/test/results/clientpositive/alter_index.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_auth.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_auto.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_auto_empty.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_auto_file_format.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_auto_multiple.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_auto_partitioned.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_auto_self_join.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_auto_unused.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_auto_update.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_bitmap.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_bitmap1.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_bitmap2.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_bitmap3.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_bitmap_auto.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_bitmap_compression.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_bitmap_rc.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_compact.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_compact_1.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_compact_2.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_compact_3.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_compact_binary_search.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_compression.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_creation.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_stale.q.out * /hive/trunk/ql/src/test/results/clientpositive/index_stale_partitioned.q.out * /hive/trunk/ql/src/test/results/clientpositive/ql_rewrite_gbtoidx.q.out * /hive/trunk/ql/src/test/results/clientpositive/show_indexes_edge_cases.q.out * /hive/trunk/ql/src/test/results/clientpositive/show_indexes_syntax.q.out Make index table output of create index command if index is table based --- Key: HIVE-2611 URL: https://issues.apache.org/jira/browse/HIVE-2611 Project: Hive Issue Type: Improvement Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2611.1.patch.txt, HIVE-2611.D705.1.patch, HIVE-2611.D705.2.patch, HIVE-2611.D705.3.patch If an index is table based, when that index is created a table is created to contain that index. This should be listed in the output of the command. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1148 - Still Failing
Changes for Build #1144 [jvs] HIVE-1040 [jira] use sed rather than diff for masking out noise in diff-based tests (Marek Sapota via John Sichi) Summary: Replace diff -I with regex masking in Java The current diff -I approach has two problems: (1) it does not allow resolution finer than line-level, so it's impossible to mask out pattern occurrences within a line, and (2) it produces unmasked files, so if you run diff on the command line to compare the result .q.out with the checked-in file, you see the noise. My suggestion is to first run sed to replace noise patterns with an unlikely-to-occur string like ZYZZYZVA, and then diff the pre-masked files without using any -I. This would require a one-time hit to update all existing .q.out files so that they would contain the pre-masked results. Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: jsichi Differential Revision: 597 Changes for Build #1145 Changes for Build #1146 [namit] HIVE-2640 Add alterPartition to AlterHandler interface (Kevin Wilfong via namit) Changes for Build #1147 [namit] HIVE-2617 Insert overwrite table db.tname fails if partition already exists (Chinna Rao Lalam via namit) Changes for Build #1148 [heyongqiang] HIVE-2651 [jira] The variable hive.exec.mode.local.auto.tasks.max should be changed (Namit Jain via Yongqiang He) Summary: HIVE-2651 It should be called hive.exec.mode.local.auto.input.files.max instead. The number of input files are checked currently. Test Plan: EMPTY Reviewers: JIRA, heyongqiang Reviewed By: heyongqiang CC: heyongqiang Differential Revision: 861 [cws] HIVE-727. Hive Server getSchema() returns wrong schema for 'Explain' queries (Prasad Mujumdar via cws) [namit] HIVE-2611 Make index table output of create index command if index is table based (Kevin Wilfong via namit) 2 tests failed. REGRESSION: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert2_overwrite_partitions Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert2_overwrite_partitions(TestCliDriver.java:16868) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785) FAILED: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_inputddl5 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_inputddl5(TestCliDriver.java:16628) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at
[jira] [Updated] (HIVE-2589) Newly created partition should inherit properties from table
[ https://issues.apache.org/jira/browse/HIVE-2589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2589: --- Status: Patch Available (was: Open) Patch Available Newly created partition should inherit properties from table Key: HIVE-2589 URL: https://issues.apache.org/jira/browse/HIVE-2589 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.9.0 Attachments: hive-2589.patch, hive-2589.patch, hive-2589_1.patch, hive-2589_2.patch This will make all the info contained in table properties available to partitions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
A bug or work as design ? - MetaStore, createDatabase()
Hi, Devs When I ran Hive UT with the candidate build of Hive-0.8.0, I found that TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore always FAILED with ROOT account while PASS with NON-ROOT account. I took a look at the source code of TestHiveMetaStore, and found that fs.mkdirs( new Path(HiveConf.getVar(hiveConf, HiveConf.ConfVars.METASTOREWAREHOUSE) + /test), new FsPermission((short) 0)); client.createDatabase(db); // always create the db with ROOT I tried the similar scenario in Linux OS, executed the following commands: 1. mkdir user_home/try 2. sudo chmod 000 user_home/try 3. mkdir user_home/try/again If the user is root, directory user_home/try/again could be executed successfully; if the user is a non-root account, it always reports Permission Deny, even if ls command. If it has the same rule in HDFS, there should be a limitation of HIVE UT, is it? Thanks, - Bing
[jira] [Created] (HIVE-2657) builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either
builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either - Key: HIVE-2657 URL: https://issues.apache.org/jira/browse/HIVE-2657 Project: Hive Issue Type: Bug Affects Versions: 0.9.0 Reporter: Alejandro Abdelnur Fix For: 0.9.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-1892) show functions also returns internal operators
[ https://issues.apache.org/jira/browse/HIVE-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan reassigned HIVE-1892: -- Assignee: Priyadarshini +1 Running Tests show functions also returns internal operators -- Key: HIVE-1892 URL: https://issues.apache.org/jira/browse/HIVE-1892 Project: Hive Issue Type: Bug Components: Security Reporter: Namit Jain Assignee: Priyadarshini Attachments: HIVE-1892.patch show functions: returns bigint etc. in its outputs, which are not valid external functions -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2602) add support for insert partition overwrite(...) if not exists
[ https://issues.apache.org/jira/browse/HIVE-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2602: -- Attachment: HIVE-2602.D579.2.patch njain updated the revision HIVE-2602 [jira] add support for insert partition overwrite(...) if not exists. Reviewers: JIRA HIVE-2602 REVISION DETAIL https://reviews.facebook.net/D579 AFFECTED FILES ql/src/test/results/clientnegative/insertover_dynapart_ifnotexists.q.out ql/src/test/results/clientpositive/insert1_overwrite_partitions.q.out ql/src/test/queries/clientnegative/insertover_dynapart_ifnotexists.q ql/src/test/queries/clientpositive/insert1_overwrite_partitions.q ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java add support for insert partition overwrite(...) if not exists - Key: HIVE-2602 URL: https://issues.apache.org/jira/browse/HIVE-2602 Project: Hive Issue Type: New Feature Reporter: Namit Jain Assignee: Chinna Rao Lalam Attachments: HIVE-2602.1.patch, HIVE-2602.2.patch, HIVE-2602.3.patch, HIVE-2602.D579.1.patch, HIVE-2602.D579.2.patch, HIVE-2602.D879.1.patch, HIVE-2602.patch INSERT OVERWRITE TABLE X PARTITION (a=b, c=d) IF NOT EXISTS ... The partition should be created and written if and only if it's not there already. The support can be added for dynamic partitions in the future, but this jira is for adding this support for static partitions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: bucketing in hive
Hi Ranjith I'm not aware of any Dynamic Bucketing in hive where as there is definitely Dynamic Partitions available. Your partitions/sub partitions would be generated on the fly/dynamically based on the value of a particular column .The records with same values for that column would go into the same partition. But Dynamic Partition load can't happen with a LOAD DATA statement as it requires running mapreduce job, You can utilize dynamic partitions in 2 steps for delimited files - Load delimited file into a non partitioned table in hive using LOAD DATA - Load data into destination table from the source table using INSERT OVERWRITE - here a MR job would be triggered that would do the job for you. I have scribbled something down on the same, check whether it'd be useful for you. http://kickstarthadoop.blogspot.com/2011/06/how-to-speed-up-your-hive-queries-in.html Regards Bejoy.K.S From: Raghunath, Ranjith ranjith.raghuna...@usaa.com To: u...@hive.apache.org u...@hive.apache.org; hive dev list dev@hive.apache.org Sent: Thursday, December 15, 2011 7:53 AM Subject: bucketing in hive Can one use bucketing in hive to emulate hash partitions on a database? Is there also a way to segment data into buckets dynamically based on values in the column. For example, Col1 Col2 Apple 1 Orange 2 Apple 2 Banana 1 If the file above were inserted into a table with Col1 as the bucket column, can we dynamically allow all of the rows with “Apple” in one file and “Orange” in one file and so on. Is there a way to do this without specifying the bucket size to be 3. Thank you, Ranjith
Re: Review Request: HIVE-2503: create per-session HiveConf instance
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2820/#review3926 --- We should include a test case for regression purposes. You can achieve this by playing with test.service.standalone.server. If that doesn't work out, then you can always extend TestHiveServer and setup the test the way you want. - Ashutosh On 2011-11-14 08:54:00, Navis Ryu wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2820/ --- (Updated 2011-11-14 08:54:00) Review request for hive and Carl Steinbach. Summary --- We uses multiple HiveClient connected to single HiveServer. After configuring connections, we've found the environment variables are different from expected. Current hive server uses single instance of HiveConf and this seemed to make the problem. This addresses bug HIVE-2503. https://issues.apache.org/jira/browse/HIVE-2503 Diffs - service/src/java/org/apache/hadoop/hive/service/HiveServer.java a2d599f Diff: https://reviews.apache.org/r/2820/diff Testing --- If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex. Thanks, Navis
[jira] [Commented] (HIVE-2503) HiveServer should provide per session configuration
[ https://issues.apache.org/jira/browse/HIVE-2503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170424#comment-13170424 ] jirapos...@reviews.apache.org commented on HIVE-2503: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2820/#review3926 --- We should include a test case for regression purposes. You can achieve this by playing with test.service.standalone.server. If that doesn't work out, then you can always extend TestHiveServer and setup the test the way you want. - Ashutosh On 2011-11-14 08:54:00, Navis Ryu wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/2820/ bq. --- bq. bq. (Updated 2011-11-14 08:54:00) bq. bq. bq. Review request for hive and Carl Steinbach. bq. bq. bq. Summary bq. --- bq. bq. We uses multiple HiveClient connected to single HiveServer. After configuring connections, we've found the environment variables are different from expected. bq. Current hive server uses single instance of HiveConf and this seemed to make the problem. bq. bq. bq. This addresses bug HIVE-2503. bq. https://issues.apache.org/jira/browse/HIVE-2503 bq. bq. bq. Diffs bq. - bq. bq.service/src/java/org/apache/hadoop/hive/service/HiveServer.java a2d599f bq. bq. Diff: https://reviews.apache.org/r/2820/diff bq. bq. bq. Testing bq. --- bq. bq. If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex. bq. bq. bq. Thanks, bq. bq. Navis bq. bq. HiveServer should provide per session configuration --- Key: HIVE-2503 URL: https://issues.apache.org/jira/browse/HIVE-2503 Project: Hive Issue Type: Bug Components: CLI, Server Infrastructure Affects Versions: 0.9.0 Reporter: Navis Assignee: Navis Fix For: 0.9.0 Attachments: HIVE-2503.1.patch.txt Currently ThriftHiveProcessorFactory returns same HiveConf instance to HiveServerHandler, making impossible to use per sesssion configuration. Just wrapping 'conf' - 'new HiveConf(conf)' seemed to solve this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-2544: Nullpointer on registering udfs.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2702/#review3927 --- trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java https://reviews.apache.org/r/2702/#comment8870 Is it possible to write a test case, which spins 10s of threads doing operations. Thats not fool-proof but will result in some coverage. Unless you have a better idea for writing test case for this. trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java https://reviews.apache.org/r/2702/#comment8869 Its LinkedHashMap right now, changing it to Concurrent will violate the contract of LinkedHashMap which you can iterate in same order as your inserts. Not sure if violating that is alright here. - Ashutosh On 2011-11-03 13:47:48, Bennie Schut wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2702/ --- (Updated 2011-11-03 13:47:48) Review request for hive. Summary --- HIVE-2544: Nullpointer on registering udfs. This addresses bug HIVE-2544. https://issues.apache.org/jira/browse/HIVE-2544 Diffs - trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1196618 Diff: https://reviews.apache.org/r/2702/diff Testing --- Thanks, Bennie
[jira] [Commented] (HIVE-2544) Nullpointer on registering udfs.
[ https://issues.apache.org/jira/browse/HIVE-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170431#comment-13170431 ] jirapos...@reviews.apache.org commented on HIVE-2544: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2702/#review3927 --- trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java https://reviews.apache.org/r/2702/#comment8870 Is it possible to write a test case, which spins 10s of threads doing operations. Thats not fool-proof but will result in some coverage. Unless you have a better idea for writing test case for this. trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java https://reviews.apache.org/r/2702/#comment8869 Its LinkedHashMap right now, changing it to Concurrent will violate the contract of LinkedHashMap which you can iterate in same order as your inserts. Not sure if violating that is alright here. - Ashutosh On 2011-11-03 13:47:48, Bennie Schut wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/2702/ bq. --- bq. bq. (Updated 2011-11-03 13:47:48) bq. bq. bq. Review request for hive. bq. bq. bq. Summary bq. --- bq. bq. HIVE-2544: Nullpointer on registering udfs. bq. bq. bq. This addresses bug HIVE-2544. bq. https://issues.apache.org/jira/browse/HIVE-2544 bq. bq. bq. Diffs bq. - bq. bq.trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1196618 bq. bq. Diff: https://reviews.apache.org/r/2702/diff bq. bq. bq. Testing bq. --- bq. bq. bq. Thanks, bq. bq. Bennie bq. bq. Nullpointer on registering udfs. Key: HIVE-2544 URL: https://issues.apache.org/jira/browse/HIVE-2544 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.9.0 Reporter: Bennie Schut Assignee: Bennie Schut Fix For: 0.9.0 Attachments: HIVE-2544.1.patch.txt Currently the Function registry can throw NullPointers when multiple threads are trying to register the same function. The normal put() will replace the existing registered function object even if it's exactly the same function. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2657) builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either
[ https://issues.apache.org/jira/browse/HIVE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2657: -- Attachment: HIVE-2657.D897.1.patch cwsteinbach requested code review of HIVE-2657 [jira] builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either. Reviewers: JIRA Make hive-cli and hive-ql depend on hive-builtins TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D897 AFFECTED FILES cli/ivy.xml ql/ivy.xml MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/1887/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either - Key: HIVE-2657 URL: https://issues.apache.org/jira/browse/HIVE-2657 Project: Hive Issue Type: Bug Affects Versions: 0.9.0 Reporter: Alejandro Abdelnur Fix For: 0.9.0 Attachments: HIVE-2657.D897.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2657) builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either
[ https://issues.apache.org/jira/browse/HIVE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2657: - Attachment: HIVE-2657.D897.1.patch builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either - Key: HIVE-2657 URL: https://issues.apache.org/jira/browse/HIVE-2657 Project: Hive Issue Type: Bug Affects Versions: 0.9.0 Reporter: Alejandro Abdelnur Fix For: 0.9.0 Attachments: HIVE-2657.D897.1.patch, HIVE-2657.D897.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-0.8.0-SNAPSHOT-h0.21 - Build # 131 - Failure
Changes for Build #131 No tests ran. The Apache Jenkins build system has built Hive-0.8.0-SNAPSHOT-h0.21 (build #131) Status: Failure Check console output at https://builds.apache.org/job/Hive-0.8.0-SNAPSHOT-h0.21/131/ to view the results.
[jira] [Commented] (HIVE-1487) parallelize test query runs
[ https://issues.apache.org/jira/browse/HIVE-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170491#comment-13170491 ] John Sichi commented on HIVE-1487: -- https://cwiki.apache.org/confluence/display/Hive/Unit+Test+Parallel+Execution parallelize test query runs --- Key: HIVE-1487 URL: https://issues.apache.org/jira/browse/HIVE-1487 Project: Hive Issue Type: Improvement Components: Testing Infrastructure Reporter: Joydeep Sen Sarma Assignee: Marek Sapota Fix For: 0.9.0 Attachments: HIVE-1487.D603.1.patch, HIVE-1487.D603.3.patch, HIVE-1487.D603.4.patch, HIVE-1487.D603.4.patch HIVE-1464 speeded up serial runs somewhat - but looks like it's still too slow. we should use parallel junit or some similar setup to run test queries in parallel. this should be really easy as well need to just use a separate warehouse/metadb and potentiall mapred system dir location. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
parallel execution for Hive unit tests
Hey all, Marek Sapota has put together a doc on the new scripts for spreading Hive unit test execution across a cluster: https://cwiki.apache.org/confluence/display/Hive/Unit+Test+Parallel+Execution Whether you are a committer or someone contributing patches, if you are currently frustrated by waiting for Hive tests to complete for each patch, please set this up and give it a try. JVS
[jira] [Commented] (HIVE-1487) parallelize test query runs
[ https://issues.apache.org/jira/browse/HIVE-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170497#comment-13170497 ] Ashutosh Chauhan commented on HIVE-1487: Awesome work! Thanks, Marek and everyone else involved for making life better for us : ) parallelize test query runs --- Key: HIVE-1487 URL: https://issues.apache.org/jira/browse/HIVE-1487 Project: Hive Issue Type: Improvement Components: Testing Infrastructure Reporter: Joydeep Sen Sarma Assignee: Marek Sapota Fix For: 0.9.0 Attachments: HIVE-1487.D603.1.patch, HIVE-1487.D603.3.patch, HIVE-1487.D603.4.patch, HIVE-1487.D603.4.patch HIVE-1464 speeded up serial runs somewhat - but looks like it's still too slow. we should use parallel junit or some similar setup to run test queries in parallel. this should be really easy as well need to just use a separate warehouse/metadb and potentiall mapred system dir location. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2657) builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either
[ https://issues.apache.org/jira/browse/HIVE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170504#comment-13170504 ] Phabricator commented on HIVE-2657: --- jsichi has accepted the revision HIVE-2657 [jira] builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either. +1, committing to trunk. REVISION DETAIL https://reviews.facebook.net/D897 builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either - Key: HIVE-2657 URL: https://issues.apache.org/jira/browse/HIVE-2657 Project: Hive Issue Type: Bug Affects Versions: 0.9.0 Reporter: Alejandro Abdelnur Fix For: 0.9.0 Attachments: HIVE-2657.D897.1.patch, HIVE-2657.D897.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2657) builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either
[ https://issues.apache.org/jira/browse/HIVE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi updated HIVE-2657: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk, thanks Carl! Are you gonna backport this one for 0.8.1? builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either - Key: HIVE-2657 URL: https://issues.apache.org/jira/browse/HIVE-2657 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.9.0 Reporter: Alejandro Abdelnur Fix For: 0.9.0 Attachments: HIVE-2657.D897.1.patch, HIVE-2657.D897.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2657) builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either
[ https://issues.apache.org/jira/browse/HIVE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi updated HIVE-2657: - Component/s: Build Infrastructure builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either - Key: HIVE-2657 URL: https://issues.apache.org/jira/browse/HIVE-2657 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.9.0 Reporter: Alejandro Abdelnur Fix For: 0.9.0 Attachments: HIVE-2657.D897.1.patch, HIVE-2657.D897.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2657) builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either
[ https://issues.apache.org/jira/browse/HIVE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170511#comment-13170511 ] Phabricator commented on HIVE-2657: --- cwsteinbach has committed the revision HIVE-2657 [jira] builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either. Change committed by jvs. REVISION DETAIL https://reviews.facebook.net/D897 COMMIT https://reviews.facebook.net/rHIVE1214961 builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either - Key: HIVE-2657 URL: https://issues.apache.org/jira/browse/HIVE-2657 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.9.0 Reporter: Alejandro Abdelnur Fix For: 0.9.0 Attachments: HIVE-2657.D897.1.patch, HIVE-2657.D897.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2658) add a option in hive to skip corrupted data entirely
add a option in hive to skip corrupted data entirely Key: HIVE-2658 URL: https://issues.apache.org/jira/browse/HIVE-2658 Project: Hive Issue Type: New Feature Reporter: Namit Jain Assignee: He Yongqiang Add a new parameter: hive.skip.corrupted.data This is independent of the type of the underlying data. The idea is as follows: We have some corrupted data in our cluster right now. We will run hive over all the corrupted partitions: use bucketizedhiveinputformat set hive.skip.corrupted.data=true insert overwrite table T partition P select * from T where P This way, T@P will be regenerated with all the data that can be read. If HiveRecordReader gets a exception getting the next row, the mapper will behave as if no more data is present in the file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170521#comment-13170521 ] John Sichi commented on HIVE-2616: -- OK, can you submit the thrift patch? If we'll be able to uptake that eventually, then I'm fine with the current approach. I was thinking dynamic proxy would allow us to do the necessary method interception without needing to futz with method accessibility. Carl, are you good with this? Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2656) Add a command line flag to specify a jar whence to load UDFs
[ https://issues.apache.org/jira/browse/HIVE-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170522#comment-13170522 ] John Sichi commented on HIVE-2656: -- Note that the jar needs to have been produced via Hive's PDK in order to have the necessary metadata in the manifest. Add a command line flag to specify a jar whence to load UDFs Key: HIVE-2656 URL: https://issues.apache.org/jira/browse/HIVE-2656 Project: Hive Issue Type: New Feature Reporter: Jonathan Chang Assignee: Jonathan Chang To extend the functionality of Hive, it is useful to be able to easily deploy packages of UDFs and register the functions therein. Right now it is a bit cumbersome to do ADD JAR and CREATE TEMPORARY FUNCTION. This can be greatly simplified by adding a simple command line flag that will leverage the code in HIVE-2523 to load all the functions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2621) Allow multiple group bys with the same input data and spray keys to be run on the same reducer.
[ https://issues.apache.org/jira/browse/HIVE-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2621: -- Attachment: HIVE-2621.D567.2.patch kevinwilfong updated the revision HIVE-2621 [jira] Allow multiple group bys with the same input data and spray keys to be run on the same reducer.. Reviewers: JIRA Updated to avoid merge conflicts. REVISION DETAIL https://reviews.facebook.net/D567 AFFECTED FILES common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ql/src/test/results/clientpositive/groupby7_noskew_multi_single_reducer.q.out ql/src/test/results/clientpositive/groupby_multi_single_reducer.q.out ql/src/test/results/clientpositive/groupby_complex_types_multi_single_reducer.q.out ql/src/test/queries/clientpositive/groupby_multi_single_reducer.q ql/src/test/queries/clientpositive/groupby7_noskew_multi_single_reducer.q ql/src/test/queries/clientpositive/groupby_complex_types_multi_single_reducer.q ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDesc.java ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java Allow multiple group bys with the same input data and spray keys to be run on the same reducer. --- Key: HIVE-2621 URL: https://issues.apache.org/jira/browse/HIVE-2621 Project: Hive Issue Type: New Feature Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2621.1.patch.txt, HIVE-2621.D567.1.patch, HIVE-2621.D567.2.patch Currently, when a user runs a query, such as a multi-insert, where each insertion subclause consists of a simple query followed by a group by, the group bys for each clause are run on a separate reducer. This requires writing the data for each group by clause to an intermediate file, and then reading it back. This uses a significant amount of the total CPU consumed by the query for an otherwise simple query. If the subclauses are grouped by their distinct expressions and group by keys, with all of the group by expressions for a group of subclauses run on a single reducer, this would reduce the amount of reading/writing to intermediate files for some queries. To do this, for each group of subclauses, in the mapper we would execute a the filters for each subclause 'or'd together (provided each subclause has a filter) followed by a reduce sink. In the reducer, the child operators would be each subclauses filter followed by the group by and any subsequent operations. Note that this would require turning off map aggregation, so we would need to make using this type of plan configurable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2657) builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either
[ https://issues.apache.org/jira/browse/HIVE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170541#comment-13170541 ] Hudson commented on HIVE-2657: -- Integrated in Hive-trunk-h0.23.0 #28 (See [https://builds.apache.org/job/Hive-trunk-h0.23.0/28/]) HIVE-2657 [jira] builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either (Carl Steinbach via John Sichi) Summary: Make hive-cli and hive-ql depend on hive-builtins Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: jsichi Differential Revision: 897 jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1214961 Files : * /hive/trunk/cli/ivy.xml * /hive/trunk/ql/ivy.xml builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either - Key: HIVE-2657 URL: https://issues.apache.org/jira/browse/HIVE-2657 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.9.0 Reporter: Alejandro Abdelnur Fix For: 0.9.0 Attachments: HIVE-2657.D897.1.patch, HIVE-2657.D897.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2654) hive.querylog.location requires parent directory to be exist or else folder creation fails
[ https://issues.apache.org/jira/browse/HIVE-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170540#comment-13170540 ] Hudson commented on HIVE-2654: -- Integrated in Hive-trunk-h0.23.0 #28 (See [https://builds.apache.org/job/Hive-trunk-h0.23.0/28/]) HIVE-2654 hive.querylog.location requires parent directory to be exist or else folder creation fails (Chinna Rao Lalam via namit) namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1214891 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/history/HiveHistory.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/history/TestHiveHistory.java hive.querylog.location requires parent directory to be exist or else folder creation fails Key: HIVE-2654 URL: https://issues.apache.org/jira/browse/HIVE-2654 Project: Hive Issue Type: Bug Components: Query Processor Environment: Hadoop 0.20.1, Hive0.9.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5). Reporter: Chinna Rao Lalam Assignee: Chinna Rao Lalam Attachments: HIVE-2654.D885.1.patch, HIVE-2654.patch if value of hive.querylog.location is '/tmp/root/hive123/test' if the parent directories not exist the creation of the folder is failed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2631) Make Hive work with Hadoop 1.0.0
[ https://issues.apache.org/jira/browse/HIVE-2631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170554#comment-13170554 ] Ashutosh Chauhan commented on HIVE-2631: All the unit tests passed. Since, its a straight forward patch will commit unless some one has an objection. Make Hive work with Hadoop 1.0.0 Key: HIVE-2631 URL: https://issues.apache.org/jira/browse/HIVE-2631 Project: Hive Issue Type: Bug Components: Shims Affects Versions: 0.8.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2631.patch With Hadoop 1.0.0 around the corner ( http://mail-archives.apache.org/mod_mbox/hadoop-general/20.mbox/%3C9D6B6144-F4E0-4A31-883F-2AC504727A1F%40hortonworks.com%3E ), it will be useful to make Hive work with it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2486) Phabricator for code review
[ https://issues.apache.org/jira/browse/HIVE-2486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marek Sapota updated HIVE-2486: --- Attachment: my_uploaded_file Phabricator for code review --- Key: HIVE-2486 URL: https://issues.apache.org/jira/browse/HIVE-2486 Project: Hive Issue Type: Bug Reporter: Marek Sapota Assignee: Marek Sapota Fix For: 0.8.0, 0.9.0 Attachments: D135.1.patch, D135.2.patch, D387.1.patch, D51.1.patch, D51.2.patch, D51.2.patch, D57.1.patch, D57.2.patch, D63.1.patch, D63.2.patch, D63.3.patch, D69.1.patch, D69.2.patch, D69.3.patch, D75.1.patch, D75.2.patch, D9.1.diff, D9.2.diff, HIVE-2486.D387.2.patch, SOAP, SOAP, TEST, my_uploaded_file, my_uploaded_file, x, xyz Replacing Review Board with Phabricator and Differential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2486) Phabricator for code review
[ https://issues.apache.org/jira/browse/HIVE-2486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ala Makota updated HIVE-2486: - Attachment: xyz Phabricator for code review --- Key: HIVE-2486 URL: https://issues.apache.org/jira/browse/HIVE-2486 Project: Hive Issue Type: Bug Reporter: Marek Sapota Assignee: Marek Sapota Fix For: 0.8.0, 0.9.0 Attachments: D135.1.patch, D135.2.patch, D387.1.patch, D51.1.patch, D51.2.patch, D51.2.patch, D57.1.patch, D57.2.patch, D63.1.patch, D63.2.patch, D63.3.patch, D69.1.patch, D69.2.patch, D69.3.patch, D75.1.patch, D75.2.patch, D9.1.diff, D9.2.diff, HIVE-2486.D387.2.patch, SOAP, SOAP, TEST, my_uploaded_file, my_uploaded_file, x, xyz Replacing Review Board with Phabricator and Differential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2631) Make Hive work with Hadoop 1.0.0
[ https://issues.apache.org/jira/browse/HIVE-2631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170572#comment-13170572 ] Carl Steinbach commented on HIVE-2631: -- +1. Will commit if tests pass. bq. Since, its a straight forward patch will commit unless some one has an objection. Hive committers are not allowed to commit their own patches, right? Make Hive work with Hadoop 1.0.0 Key: HIVE-2631 URL: https://issues.apache.org/jira/browse/HIVE-2631 Project: Hive Issue Type: Bug Components: Shims Affects Versions: 0.8.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2631.patch With Hadoop 1.0.0 around the corner ( http://mail-archives.apache.org/mod_mbox/hadoop-general/20.mbox/%3C9D6B6144-F4E0-4A31-883F-2AC504727A1F%40hortonworks.com%3E ), it will be useful to make Hive work with it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2631) Make Hive work with Hadoop 1.0.0
[ https://issues.apache.org/jira/browse/HIVE-2631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170577#comment-13170577 ] John Sichi commented on HIVE-2631: -- Correct, bylaws say one +1 from a committer who has not authored the patch followed by a Lazy approval (not counting the vote of the contributor), moving to lazy majority if a -1 is received. Make Hive work with Hadoop 1.0.0 Key: HIVE-2631 URL: https://issues.apache.org/jira/browse/HIVE-2631 Project: Hive Issue Type: Bug Components: Shims Affects Versions: 0.8.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2631.patch With Hadoop 1.0.0 around the corner ( http://mail-archives.apache.org/mod_mbox/hadoop-general/20.mbox/%3C9D6B6144-F4E0-4A31-883F-2AC504727A1F%40hortonworks.com%3E ), it will be useful to make Hive work with it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2486) Phabricator for code review
[ https://issues.apache.org/jira/browse/HIVE-2486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170582#comment-13170582 ] John Sichi commented on HIVE-2486: -- Was that patch ASF-granted via Phabricator? :) Phabricator for code review --- Key: HIVE-2486 URL: https://issues.apache.org/jira/browse/HIVE-2486 Project: Hive Issue Type: Bug Reporter: Marek Sapota Assignee: Marek Sapota Fix For: 0.8.0, 0.9.0 Attachments: D135.1.patch, D135.2.patch, D387.1.patch, D51.1.patch, D51.2.patch, D51.2.patch, D57.1.patch, D57.2.patch, D63.1.patch, D63.2.patch, D63.3.patch, D69.1.patch, D69.2.patch, D69.3.patch, D75.1.patch, D75.2.patch, D9.1.diff, D9.2.diff, HIVE-2486.D387.2.patch, SOAP, SOAP, TEST, my_uploaded_file, my_uploaded_file, x, xyz Replacing Review Board with Phabricator and Differential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2631) Make Hive work with Hadoop 1.0.0
[ https://issues.apache.org/jira/browse/HIVE-2631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170596#comment-13170596 ] Alan Gates commented on HIVE-2631: -- FWIW, other communities with these same rules (Pig and Hadoop) interpret this to mean another committer must review a patch, but not that another committer must commit it. Once a second committer has reviewed it the original contributor is free to commit it. Make Hive work with Hadoop 1.0.0 Key: HIVE-2631 URL: https://issues.apache.org/jira/browse/HIVE-2631 Project: Hive Issue Type: Bug Components: Shims Affects Versions: 0.8.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2631.patch With Hadoop 1.0.0 around the corner ( http://mail-archives.apache.org/mod_mbox/hadoop-general/20.mbox/%3C9D6B6144-F4E0-4A31-883F-2AC504727A1F%40hortonworks.com%3E ), it will be useful to make Hive work with it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [VOTE] Apache Hive 0.8.0 Release Candidate 4
Apache Hive 0.8.0 RC4 has been approved with three binding +1 votes and zero -1 votes. I will send out an official release announcement once the 0.8.0 artifacts have had a chance to propagate to the Apache mirrors. Thanks to everyone involved for their help in getting this release out! Carl On Tue, Dec 13, 2011 at 12:46 PM, Edward Capriolo edlinuxg...@gmail.comwrote: +1 Good job all. On Tue, Dec 13, 2011 at 3:28 PM, Carl Steinbach c...@cloudera.com wrote: I'm +1 on rc4. On Dec 13, 2011 11:58 AM, John Sichi jsi...@fb.com wrote: +1 from me on the release candidate. JVS On Mon, Dec 12, 2011 at 4:55 PM, Carl Steinbach c...@cloudera.com wrote: Apache Hive 0.8.0 Release Candidate 4 is available here: http://people.apache.org/~cws/hive-0.8.0-candidate-4/ Changes since RC3: * HIVE-2643. Recent patch prevents Hadoop confs from loading in 0.20.204 Please test and vote. Thanks. Carl
[jira] [Commented] (HIVE-2631) Make Hive work with Hadoop 1.0.0
[ https://issues.apache.org/jira/browse/HIVE-2631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170650#comment-13170650 ] John Sichi commented on HIVE-2631: -- Right, I meant the +1 vote (not the mechanics of committing). Make Hive work with Hadoop 1.0.0 Key: HIVE-2631 URL: https://issues.apache.org/jira/browse/HIVE-2631 Project: Hive Issue Type: Bug Components: Shims Affects Versions: 0.8.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2631.patch With Hadoop 1.0.0 around the corner ( http://mail-archives.apache.org/mod_mbox/hadoop-general/20.mbox/%3C9D6B6144-F4E0-4A31-883F-2AC504727A1F%40hortonworks.com%3E ), it will be useful to make Hive work with it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170682#comment-13170682 ] Ashutosh Chauhan commented on HIVE-2616: Sure. I uploaded the patch on THRIFT-1465 Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170685#comment-13170685 ] Carl Steinbach commented on HIVE-2616: -- @John: I'm looking at the patch now. Will respond soon with some comments. Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1892) show functions also returns internal operators
[ https://issues.apache.org/jira/browse/HIVE-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-1892: --- Resolution: Fixed Fix Version/s: 0.9.0 Status: Resolved (was: Patch Available) Tests passed. Committed to trunk. Thanks, Priyadarshini. show functions also returns internal operators -- Key: HIVE-1892 URL: https://issues.apache.org/jira/browse/HIVE-1892 Project: Hive Issue Type: Bug Components: Security Reporter: Namit Jain Assignee: Priyadarshini Fix For: 0.9.0 Attachments: HIVE-1892.patch show functions: returns bigint etc. in its outputs, which are not valid external functions -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-2005 Implement BETWEEN operator
On 2011-12-15 08:06:32, Carl Steinbach wrote: ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g, line 2007 https://reviews.apache.org/r/3008/diff/4/?file=63145#file63145line2007 I was able to get 'between' to work without 'btw' by adding KW_BETWEEN to the list of sysFuncNames below. Please see if you can get this to work too, and if so then remove any mention of 'btw' from the patch. It's much better. Thanks. - Navis --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3008/#review3923 --- On 2011-12-08 04:21:44, Navis Ryu wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3008/ --- (Updated 2011-12-08 04:21:44) Review request for hive, John Sichi and Carl Steinbach. Summary --- Revised version using 'between' UDF This addresses bug HIVE-2005. https://issues.apache.org/jira/browse/HIVE-2005 Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e1c47b4 ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g eecd9e7 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java PRE-CREATION ql/src/test/queries/clientpositive/udf_between.q PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out e63516d ql/src/test/results/clientpositive/udf_between.q.out PRE-CREATION Diff: https://reviews.apache.org/r/3008/diff Testing --- added new test : udf_between.q Thanks, Navis
Re: Review Request: HIVE-2005 Implement BETWEEN operator
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3008/ --- (Updated 2011-12-16 02:09:08.375278) Review request for hive, John Sichi and Carl Steinbach. Changes --- Renamed 'btw' to 'between' as Carl mentioned. Summary --- Revised version using 'between' UDF This addresses bug HIVE-2005. https://issues.apache.org/jira/browse/HIVE-2005 Diffs (updated) - ql/src/test/queries/clientpositive/udf_between.q PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out e63516d ql/src/test/results/clientpositive/udf_between.q.out PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e1c47b4 ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g eecd9e7 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java PRE-CREATION Diff: https://reviews.apache.org/r/3008/diff Testing --- added new test : udf_between.q Thanks, Navis
[jira] [Commented] (HIVE-2005) Implement BETWEEN operator
[ https://issues.apache.org/jira/browse/HIVE-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170694#comment-13170694 ] jirapos...@reviews.apache.org commented on HIVE-2005: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3008/ --- (Updated 2011-12-16 02:09:08.375278) Review request for hive, John Sichi and Carl Steinbach. Changes --- Renamed 'btw' to 'between' as Carl mentioned. Summary --- Revised version using 'between' UDF This addresses bug HIVE-2005. https://issues.apache.org/jira/browse/HIVE-2005 Diffs (updated) - ql/src/test/queries/clientpositive/udf_between.q PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out e63516d ql/src/test/results/clientpositive/udf_between.q.out PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e1c47b4 ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g eecd9e7 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java PRE-CREATION Diff: https://reviews.apache.org/r/3008/diff Testing --- added new test : udf_between.q Thanks, Navis Implement BETWEEN operator -- Key: HIVE-2005 URL: https://issues.apache.org/jira/browse/HIVE-2005 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.9.0 Reporter: Carl Steinbach Assignee: Navis Fix For: 0.9.0 Attachments: HIVE-2005.1.patch.txt Reference: http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#operator_between -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2005) Implement BETWEEN operator
[ https://issues.apache.org/jira/browse/HIVE-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170693#comment-13170693 ] jirapos...@reviews.apache.org commented on HIVE-2005: - bq. On 2011-12-15 08:06:32, Carl Steinbach wrote: bq. ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g, line 2007 bq. https://reviews.apache.org/r/3008/diff/4/?file=63145#file63145line2007 bq. bq. I was able to get 'between' to work without 'btw' by adding KW_BETWEEN to the list of sysFuncNames below. Please see if you can get this to work too, and if so then remove any mention of 'btw' from the patch. It's much better. Thanks. - Navis --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3008/#review3923 --- On 2011-12-08 04:21:44, Navis Ryu wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/3008/ bq. --- bq. bq. (Updated 2011-12-08 04:21:44) bq. bq. bq. Review request for hive, John Sichi and Carl Steinbach. bq. bq. bq. Summary bq. --- bq. bq. Revised version using 'between' UDF bq. bq. bq. This addresses bug HIVE-2005. bq. https://issues.apache.org/jira/browse/HIVE-2005 bq. bq. bq. Diffs bq. - bq. bq.ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e1c47b4 bq.ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g eecd9e7 bq.ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java PRE-CREATION bq.ql/src/test/queries/clientpositive/udf_between.q PRE-CREATION bq.ql/src/test/results/clientpositive/show_functions.q.out e63516d bq.ql/src/test/results/clientpositive/udf_between.q.out PRE-CREATION bq. bq. Diff: https://reviews.apache.org/r/3008/diff bq. bq. bq. Testing bq. --- bq. bq. added new test : udf_between.q bq. bq. bq. Thanks, bq. bq. Navis bq. bq. Implement BETWEEN operator -- Key: HIVE-2005 URL: https://issues.apache.org/jira/browse/HIVE-2005 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.9.0 Reporter: Carl Steinbach Assignee: Navis Fix For: 0.9.0 Attachments: HIVE-2005.1.patch.txt Reference: http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#operator_between -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1150 - Still Failing
Changes for Build #1144 [jvs] HIVE-1040 [jira] use sed rather than diff for masking out noise in diff-based tests (Marek Sapota via John Sichi) Summary: Replace diff -I with regex masking in Java The current diff -I approach has two problems: (1) it does not allow resolution finer than line-level, so it's impossible to mask out pattern occurrences within a line, and (2) it produces unmasked files, so if you run diff on the command line to compare the result .q.out with the checked-in file, you see the noise. My suggestion is to first run sed to replace noise patterns with an unlikely-to-occur string like ZYZZYZVA, and then diff the pre-masked files without using any -I. This would require a one-time hit to update all existing .q.out files so that they would contain the pre-masked results. Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: jsichi Differential Revision: 597 Changes for Build #1145 Changes for Build #1146 [namit] HIVE-2640 Add alterPartition to AlterHandler interface (Kevin Wilfong via namit) Changes for Build #1147 [namit] HIVE-2617 Insert overwrite table db.tname fails if partition already exists (Chinna Rao Lalam via namit) Changes for Build #1148 [heyongqiang] HIVE-2651 [jira] The variable hive.exec.mode.local.auto.tasks.max should be changed (Namit Jain via Yongqiang He) Summary: HIVE-2651 It should be called hive.exec.mode.local.auto.input.files.max instead. The number of input files are checked currently. Test Plan: EMPTY Reviewers: JIRA, heyongqiang Reviewed By: heyongqiang CC: heyongqiang Differential Revision: 861 [cws] HIVE-727. Hive Server getSchema() returns wrong schema for 'Explain' queries (Prasad Mujumdar via cws) [namit] HIVE-2611 Make index table output of create index command if index is table based (Kevin Wilfong via namit) Changes for Build #1150 [jvs] HIVE-2657 [jira] builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either (Carl Steinbach via John Sichi) Summary: Make hive-cli and hive-ql depend on hive-builtins Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: jsichi Differential Revision: 897 [namit] HIVE-2654 hive.querylog.location requires parent directory to be exist or else folder creation fails (Chinna Rao Lalam via namit) 1 tests failed. FAILED: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_inputddl5 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_inputddl5(TestCliDriver.java:16628) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1150) Status: Still Failing Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1150/ to view the results.
[jira] [Commented] (HIVE-2654) hive.querylog.location requires parent directory to be exist or else folder creation fails
[ https://issues.apache.org/jira/browse/HIVE-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170711#comment-13170711 ] Hudson commented on HIVE-2654: -- Integrated in Hive-trunk-h0.21 #1150 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1150/]) HIVE-2654 hive.querylog.location requires parent directory to be exist or else folder creation fails (Chinna Rao Lalam via namit) namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1214891 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/history/HiveHistory.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/history/TestHiveHistory.java hive.querylog.location requires parent directory to be exist or else folder creation fails Key: HIVE-2654 URL: https://issues.apache.org/jira/browse/HIVE-2654 Project: Hive Issue Type: Bug Components: Query Processor Environment: Hadoop 0.20.1, Hive0.9.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5). Reporter: Chinna Rao Lalam Assignee: Chinna Rao Lalam Attachments: HIVE-2654.D885.1.patch, HIVE-2654.patch if value of hive.querylog.location is '/tmp/root/hive123/test' if the parent directories not exist the creation of the folder is failed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2657) builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either
[ https://issues.apache.org/jira/browse/HIVE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170712#comment-13170712 ] Hudson commented on HIVE-2657: -- Integrated in Hive-trunk-h0.21 #1150 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1150/]) HIVE-2657 [jira] builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either (Carl Steinbach via John Sichi) Summary: Make hive-cli and hive-ql depend on hive-builtins Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: jsichi Differential Revision: 897 jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1214961 Files : * /hive/trunk/cli/ivy.xml * /hive/trunk/ql/ivy.xml builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either - Key: HIVE-2657 URL: https://issues.apache.org/jira/browse/HIVE-2657 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.9.0 Reporter: Alejandro Abdelnur Fix For: 0.9.0 Attachments: HIVE-2657.D897.1.patch, HIVE-2657.D897.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: Pass user identity in metastore connection in unsecure mode
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2975/#review3939
---
trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
https://reviews.apache.org/r/2975/#comment8885
All properties that appear in HiveConf also need to appear in
conf/hive-default.xml.template along with a description.
trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
https://reviews.apache.org/r/2975/#comment8893
I think it would make sense to change the name to
'hive.metastore.client.enable.setugi'. Also, I think this feature should be
disabled by default.
trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
https://reviews.apache.org/r/2975/#comment8894
Please add a new property hive.metastore.server.enable.setugi that allows
this RPC to be disabled on the server side, and set the default value to false.
trunk/metastore/if/hive_metastore.thrift
https://reviews.apache.org/r/2975/#comment8895
Please add a comment explaining what this call does.
trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
https://reviews.apache.org/r/2975/#comment8886
When I apply your changes and run the thriftif ant target I see a small
diff in this file. Did you use Thrift 0.7.0 to generate these files?
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
https://reviews.apache.org/r/2975/#comment8887
Indentation.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
https://reviews.apache.org/r/2975/#comment
So instead of checking the hive.metastore.sasl.enabled property we now just
check to see if we're using a security enabled shim, and if so assume that the
user wants to enable security? I don't think this is correct behavior since the
fact that we're using a secure version of Hadoop does not necessarily imply
that we actually have security enabled.
Also, it looks like this change deprecates the hive.metastore.sasl.enabled
configuration property. In line with my comment above I think it makes sense to
leave this property in, but if you do remove it then you need to release note
the change and remove this property from HiveConf and
conf/hive-default.xml.template.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
https://reviews.apache.org/r/2975/#comment8896
Indentation.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
https://reviews.apache.org/r/2975/#comment8897
We're initializing SASL even if isSecure=false?
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
https://reviews.apache.org/r/2975/#comment8889
Formatting.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
https://reviews.apache.org/r/2975/#comment8891
Formatting: please add spaces.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
https://reviews.apache.org/r/2975/#comment8898
new client talking to old metastore seems to imply that we're able to
determine whether or not we're talking to an old server, which isn't true. In
reality, the onus is on the admin to ensure that both sides support this
feature. What happens if the client calls set_ugi(), but the server doesn't
support it? Is the error message helpful?
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
https://reviews.apache.org/r/2975/#comment8892
Should this be Failed to login to the MetaStore Server...?
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java
https://reviews.apache.org/r/2975/#comment8901
I think it's more accurate to say that the processor *checks* to see if
the first call is to set_ugi()... instead of saying that it *expects* the
first call to be to set_ugi().
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java
https://reviews.apache.org/r/2975/#comment8902
+1 to referencing the THRIFT JIRA. I think the class comment should call
out that this is a temporary workaround cite a TODO.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java
https://reviews.apache.org/r/2975/#comment8899
Formatting: '} else {'
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java
https://reviews.apache.org/r/2975/#comment8900
There's a TAB here. Please remove.
trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java
https://reviews.apache.org/r/2975/#comment8903
Spacing.
trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java
https://reviews.apache.org/r/2975/#comment8904
Please change the name of this method to
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[
https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170726#comment-13170726
]
jirapos...@reviews.apache.org commented on HIVE-2616:
-
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2975/#review3939
---
trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
https://reviews.apache.org/r/2975/#comment8885
All properties that appear in HiveConf also need to appear in
conf/hive-default.xml.template along with a description.
trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
https://reviews.apache.org/r/2975/#comment8893
I think it would make sense to change the name to
'hive.metastore.client.enable.setugi'. Also, I think this feature should be
disabled by default.
trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
https://reviews.apache.org/r/2975/#comment8894
Please add a new property hive.metastore.server.enable.setugi that allows
this RPC to be disabled on the server side, and set the default value to false.
trunk/metastore/if/hive_metastore.thrift
https://reviews.apache.org/r/2975/#comment8895
Please add a comment explaining what this call does.
trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
https://reviews.apache.org/r/2975/#comment8886
When I apply your changes and run the thriftif ant target I see a small
diff in this file. Did you use Thrift 0.7.0 to generate these files?
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
https://reviews.apache.org/r/2975/#comment8887
Indentation.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
https://reviews.apache.org/r/2975/#comment
So instead of checking the hive.metastore.sasl.enabled property we now just
check to see if we're using a security enabled shim, and if so assume that the
user wants to enable security? I don't think this is correct behavior since the
fact that we're using a secure version of Hadoop does not necessarily imply
that we actually have security enabled.
Also, it looks like this change deprecates the hive.metastore.sasl.enabled
configuration property. In line with my comment above I think it makes sense to
leave this property in, but if you do remove it then you need to release note
the change and remove this property from HiveConf and
conf/hive-default.xml.template.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
https://reviews.apache.org/r/2975/#comment8896
Indentation.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
https://reviews.apache.org/r/2975/#comment8897
We're initializing SASL even if isSecure=false?
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
https://reviews.apache.org/r/2975/#comment8889
Formatting.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
https://reviews.apache.org/r/2975/#comment8891
Formatting: please add spaces.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
https://reviews.apache.org/r/2975/#comment8898
new client talking to old metastore seems to imply that we're able to
determine whether or not we're talking to an old server, which isn't true. In
reality, the onus is on the admin to ensure that both sides support this
feature. What happens if the client calls set_ugi(), but the server doesn't
support it? Is the error message helpful?
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
https://reviews.apache.org/r/2975/#comment8892
Should this be Failed to login to the MetaStore Server...?
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java
https://reviews.apache.org/r/2975/#comment8901
I think it's more accurate to say that the processor *checks* to see if
the first call is to set_ugi()... instead of saying that it *expects* the
first call to be to set_ugi().
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java
https://reviews.apache.org/r/2975/#comment8902
+1 to referencing the THRIFT JIRA. I think the class comment should call
out that this is a temporary workaround cite a TODO.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java
https://reviews.apache.org/r/2975/#comment8899
Formatting: '} else {'
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java
https://reviews.apache.org/r/2975/#comment8900
There's a TAB here. Please remove.
Re: Review Request: HIVE-2005 Implement BETWEEN operator
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3008/#review3943 --- Please update udf_bewteen.q and then attach the updated patch to HIVE-2005. I think this will be ready to commit once this change is made. Thanks! ql/src/test/queries/clientpositive/udf_between.q https://reviews.apache.org/r/3008/#comment8915 I'm not sure why you changed this testcase. The original version was better. * The first two lines should be: DESCRIBE FUNCTION between DESCRIBE FUNCTION EXTENDED between; * Please call EXPLAIN on each query before running it. * Is there any reason in particular why you create the T1 table instead of reusing the 'src' table that gets created automatically for all CliDriver tests? Loading data will definitely make the test run more slowly. - Carl On 2011-12-16 02:09:08, Navis Ryu wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3008/ --- (Updated 2011-12-16 02:09:08) Review request for hive, John Sichi and Carl Steinbach. Summary --- Revised version using 'between' UDF This addresses bug HIVE-2005. https://issues.apache.org/jira/browse/HIVE-2005 Diffs - ql/src/test/queries/clientpositive/udf_between.q PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out e63516d ql/src/test/results/clientpositive/udf_between.q.out PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e1c47b4 ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g eecd9e7 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java PRE-CREATION Diff: https://reviews.apache.org/r/3008/diff Testing --- added new test : udf_between.q Thanks, Navis
[jira] [Commented] (HIVE-2005) Implement BETWEEN operator
[ https://issues.apache.org/jira/browse/HIVE-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170736#comment-13170736 ] jirapos...@reviews.apache.org commented on HIVE-2005: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3008/#review3943 --- Please update udf_bewteen.q and then attach the updated patch to HIVE-2005. I think this will be ready to commit once this change is made. Thanks! ql/src/test/queries/clientpositive/udf_between.q https://reviews.apache.org/r/3008/#comment8915 I'm not sure why you changed this testcase. The original version was better. * The first two lines should be: DESCRIBE FUNCTION between DESCRIBE FUNCTION EXTENDED between; * Please call EXPLAIN on each query before running it. * Is there any reason in particular why you create the T1 table instead of reusing the 'src' table that gets created automatically for all CliDriver tests? Loading data will definitely make the test run more slowly. - Carl On 2011-12-16 02:09:08, Navis Ryu wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/3008/ bq. --- bq. bq. (Updated 2011-12-16 02:09:08) bq. bq. bq. Review request for hive, John Sichi and Carl Steinbach. bq. bq. bq. Summary bq. --- bq. bq. Revised version using 'between' UDF bq. bq. bq. This addresses bug HIVE-2005. bq. https://issues.apache.org/jira/browse/HIVE-2005 bq. bq. bq. Diffs bq. - bq. bq.ql/src/test/queries/clientpositive/udf_between.q PRE-CREATION bq.ql/src/test/results/clientpositive/show_functions.q.out e63516d bq.ql/src/test/results/clientpositive/udf_between.q.out PRE-CREATION bq.ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e1c47b4 bq.ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g eecd9e7 bq.ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java PRE-CREATION bq. bq. Diff: https://reviews.apache.org/r/3008/diff bq. bq. bq. Testing bq. --- bq. bq. added new test : udf_between.q bq. bq. bq. Thanks, bq. bq. Navis bq. bq. Implement BETWEEN operator -- Key: HIVE-2005 URL: https://issues.apache.org/jira/browse/HIVE-2005 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.9.0 Reporter: Carl Steinbach Assignee: Navis Fix For: 0.9.0 Attachments: HIVE-2005.1.patch.txt Reference: http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#operator_between -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2659) SHOW FUNCTIONS still returns internal operators
SHOW FUNCTIONS still returns internal operators --- Key: HIVE-2659 URL: https://issues.apache.org/jira/browse/HIVE-2659 Project: Hive Issue Type: Bug Components: UDF Reporter: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1892) show functions also returns internal operators
[ https://issues.apache.org/jira/browse/HIVE-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170739#comment-13170739 ] Hudson commented on HIVE-1892: -- Integrated in Hive-trunk-h0.23.0 #29 (See [https://builds.apache.org/job/Hive-trunk-h0.23.0/29/]) HIVE-1892 : show functions also returns internal operators (Priyadarshini via Ashutosh Chauhan) hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1215021 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java * /hive/trunk/ql/src/test/results/clientpositive/show_functions.q.out show functions also returns internal operators -- Key: HIVE-1892 URL: https://issues.apache.org/jira/browse/HIVE-1892 Project: Hive Issue Type: Bug Components: Security Reporter: Namit Jain Assignee: Priyadarshini Fix For: 0.9.0 Attachments: HIVE-1892.patch show functions: returns bigint etc. in its outputs, which are not valid external functions -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2659) SHOW FUNCTIONS still returns internal operators
[ https://issues.apache.org/jira/browse/HIVE-2659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170744#comment-13170744 ] Carl Steinbach commented on HIVE-2659: -- HIVE-1892 implemented a mechanism that filters the output of the SHOW FUNCTIONS command in order to remove type operators, e.g. DOUBLE, BOOLEAN, etc. However, we're still not removing other non-function functions such as '+', '', 'if', 'in', etc. Instead of filtering the output of SHOW FUNCTIONS, I think a better approach is to add registerInternalUDF and registerInternalGenericUDF methods to FunctionRegistry, and then use the information captured by these methods to influence the generation of the list of functions returned to SHOW FUNCTIONS. SHOW FUNCTIONS still returns internal operators --- Key: HIVE-2659 URL: https://issues.apache.org/jira/browse/HIVE-2659 Project: Hive Issue Type: Bug Components: UDF Reporter: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-2005 Implement BETWEEN operator
On 2011-12-16 03:26:25, Carl Steinbach wrote: ql/src/test/queries/clientpositive/udf_between.q, line 1 https://reviews.apache.org/r/3008/diff/5/?file=64857#file64857line1 I'm not sure why you changed this testcase. The original version was better. * The first two lines should be: DESCRIBE FUNCTION between DESCRIBE FUNCTION EXTENDED between; * Please call EXPLAIN on each query before running it. * Is there any reason in particular why you create the T1 table instead of reusing the 'src' table that gets created automatically for all CliDriver tests? Loading data will definitely make the test run more slowly. Cause renaming 'btw' to 'between' made impossible to compile queries like 'SELECT between(false, 'b', 'a', 'c') FROM src LIMIT 1;'. Could you give me some better test queries? I'll add DESCRIBE part and EXPLAIN part, shortly. - Navis --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3008/#review3943 --- On 2011-12-16 02:09:08, Navis Ryu wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3008/ --- (Updated 2011-12-16 02:09:08) Review request for hive, John Sichi and Carl Steinbach. Summary --- Revised version using 'between' UDF This addresses bug HIVE-2005. https://issues.apache.org/jira/browse/HIVE-2005 Diffs - ql/src/test/queries/clientpositive/udf_between.q PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out e63516d ql/src/test/results/clientpositive/udf_between.q.out PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e1c47b4 ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g eecd9e7 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java PRE-CREATION Diff: https://reviews.apache.org/r/3008/diff Testing --- added new test : udf_between.q Thanks, Navis
[jira] [Commented] (HIVE-2005) Implement BETWEEN operator
[ https://issues.apache.org/jira/browse/HIVE-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170747#comment-13170747 ] jirapos...@reviews.apache.org commented on HIVE-2005: - bq. On 2011-12-16 03:26:25, Carl Steinbach wrote: bq. ql/src/test/queries/clientpositive/udf_between.q, line 1 bq. https://reviews.apache.org/r/3008/diff/5/?file=64857#file64857line1 bq. bq. I'm not sure why you changed this testcase. The original version was better. bq. bq. * The first two lines should be: bq. DESCRIBE FUNCTION between bq. DESCRIBE FUNCTION EXTENDED between; bq. bq. * Please call EXPLAIN on each query before running it. bq. bq. * Is there any reason in particular why you create the T1 table instead of reusing the 'src' table that gets created automatically for all CliDriver tests? Loading data will definitely make the test run more slowly. Cause renaming 'btw' to 'between' made impossible to compile queries like 'SELECT between(false, 'b', 'a', 'c') FROM src LIMIT 1;'. Could you give me some better test queries? I'll add DESCRIBE part and EXPLAIN part, shortly. - Navis --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3008/#review3943 --- On 2011-12-16 02:09:08, Navis Ryu wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/3008/ bq. --- bq. bq. (Updated 2011-12-16 02:09:08) bq. bq. bq. Review request for hive, John Sichi and Carl Steinbach. bq. bq. bq. Summary bq. --- bq. bq. Revised version using 'between' UDF bq. bq. bq. This addresses bug HIVE-2005. bq. https://issues.apache.org/jira/browse/HIVE-2005 bq. bq. bq. Diffs bq. - bq. bq.ql/src/test/queries/clientpositive/udf_between.q PRE-CREATION bq.ql/src/test/results/clientpositive/show_functions.q.out e63516d bq.ql/src/test/results/clientpositive/udf_between.q.out PRE-CREATION bq.ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e1c47b4 bq.ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g eecd9e7 bq.ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java PRE-CREATION bq. bq. Diff: https://reviews.apache.org/r/3008/diff bq. bq. bq. Testing bq. --- bq. bq. added new test : udf_between.q bq. bq. bq. Thanks, bq. bq. Navis bq. bq. Implement BETWEEN operator -- Key: HIVE-2005 URL: https://issues.apache.org/jira/browse/HIVE-2005 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.9.0 Reporter: Carl Steinbach Assignee: Navis Fix For: 0.9.0 Attachments: HIVE-2005.1.patch.txt Reference: http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#operator_between -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-2005 Implement BETWEEN operator
On 2011-12-16 03:26:25, Carl Steinbach wrote: ql/src/test/queries/clientpositive/udf_between.q, line 1 https://reviews.apache.org/r/3008/diff/5/?file=64857#file64857line1 I'm not sure why you changed this testcase. The original version was better. * The first two lines should be: DESCRIBE FUNCTION between DESCRIBE FUNCTION EXTENDED between; * Please call EXPLAIN on each query before running it. * Is there any reason in particular why you create the T1 table instead of reusing the 'src' table that gets created automatically for all CliDriver tests? Loading data will definitely make the test run more slowly. Navis Ryu wrote: Cause renaming 'btw' to 'between' made impossible to compile queries like 'SELECT between(false, 'b', 'a', 'c') FROM src LIMIT 1;'. Could you give me some better test queries? I'll add DESCRIBE part and EXPLAIN part, shortly. Here's what I think it should look like: DESCRIBE FUNCTION between; DESCRIBE FUNCTION EXTENDED between; EXPLAIN SELECT * FROM src WHERE key BETWEEN 2 AND 7; SELECT * FROM src WHERE key BETWEEN 2 AND 7; EXPLAIN SELECT * FROM src WHERE key NOT BETWEEN 2 AND 7; SELECT * FROM src WHERE key NOT BETWEEN 2 AND 7; - Carl --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3008/#review3943 --- On 2011-12-16 02:09:08, Navis Ryu wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3008/ --- (Updated 2011-12-16 02:09:08) Review request for hive, John Sichi and Carl Steinbach. Summary --- Revised version using 'between' UDF This addresses bug HIVE-2005. https://issues.apache.org/jira/browse/HIVE-2005 Diffs - ql/src/test/queries/clientpositive/udf_between.q PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out e63516d ql/src/test/results/clientpositive/udf_between.q.out PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e1c47b4 ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g eecd9e7 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java PRE-CREATION Diff: https://reviews.apache.org/r/3008/diff Testing --- added new test : udf_between.q Thanks, Navis
[jira] [Commented] (HIVE-2005) Implement BETWEEN operator
[ https://issues.apache.org/jira/browse/HIVE-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170749#comment-13170749 ] jirapos...@reviews.apache.org commented on HIVE-2005: - bq. On 2011-12-16 03:26:25, Carl Steinbach wrote: bq. ql/src/test/queries/clientpositive/udf_between.q, line 1 bq. https://reviews.apache.org/r/3008/diff/5/?file=64857#file64857line1 bq. bq. I'm not sure why you changed this testcase. The original version was better. bq. bq. * The first two lines should be: bq. DESCRIBE FUNCTION between bq. DESCRIBE FUNCTION EXTENDED between; bq. bq. * Please call EXPLAIN on each query before running it. bq. bq. * Is there any reason in particular why you create the T1 table instead of reusing the 'src' table that gets created automatically for all CliDriver tests? Loading data will definitely make the test run more slowly. bq. bq. Navis Ryu wrote: bq. Cause renaming 'btw' to 'between' made impossible to compile queries like 'SELECT between(false, 'b', 'a', 'c') FROM src LIMIT 1;'. Could you give me some better test queries? bq. bq. I'll add DESCRIBE part and EXPLAIN part, shortly. Here's what I think it should look like: DESCRIBE FUNCTION between; DESCRIBE FUNCTION EXTENDED between; EXPLAIN SELECT * FROM src WHERE key BETWEEN 2 AND 7; SELECT * FROM src WHERE key BETWEEN 2 AND 7; EXPLAIN SELECT * FROM src WHERE key NOT BETWEEN 2 AND 7; SELECT * FROM src WHERE key NOT BETWEEN 2 AND 7; - Carl --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3008/#review3943 --- On 2011-12-16 02:09:08, Navis Ryu wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/3008/ bq. --- bq. bq. (Updated 2011-12-16 02:09:08) bq. bq. bq. Review request for hive, John Sichi and Carl Steinbach. bq. bq. bq. Summary bq. --- bq. bq. Revised version using 'between' UDF bq. bq. bq. This addresses bug HIVE-2005. bq. https://issues.apache.org/jira/browse/HIVE-2005 bq. bq. bq. Diffs bq. - bq. bq.ql/src/test/queries/clientpositive/udf_between.q PRE-CREATION bq.ql/src/test/results/clientpositive/show_functions.q.out e63516d bq.ql/src/test/results/clientpositive/udf_between.q.out PRE-CREATION bq.ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e1c47b4 bq.ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g eecd9e7 bq.ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java PRE-CREATION bq. bq. Diff: https://reviews.apache.org/r/3008/diff bq. bq. bq. Testing bq. --- bq. bq. added new test : udf_between.q bq. bq. bq. Thanks, bq. bq. Navis bq. bq. Implement BETWEEN operator -- Key: HIVE-2005 URL: https://issues.apache.org/jira/browse/HIVE-2005 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.9.0 Reporter: Carl Steinbach Assignee: Navis Fix For: 0.9.0 Attachments: HIVE-2005.1.patch.txt Reference: http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#operator_between -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-2005 Implement BETWEEN operator
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3008/ --- (Updated 2011-12-16 04:23:03.804398) Review request for hive, John Sichi and Carl Steinbach. Summary --- Revised version using 'between' UDF This addresses bug HIVE-2005. https://issues.apache.org/jira/browse/HIVE-2005 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e1c47b4 ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g eecd9e7 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java PRE-CREATION ql/src/test/queries/clientpositive/udf_between.q PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out e63516d ql/src/test/results/clientpositive/udf_between.q.out PRE-CREATION Diff: https://reviews.apache.org/r/3008/diff Testing --- added new test : udf_between.q Thanks, Navis
[jira] [Commented] (HIVE-2005) Implement BETWEEN operator
[ https://issues.apache.org/jira/browse/HIVE-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170760#comment-13170760 ] jirapos...@reviews.apache.org commented on HIVE-2005: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3008/ --- (Updated 2011-12-16 04:23:03.804398) Review request for hive, John Sichi and Carl Steinbach. Summary --- Revised version using 'between' UDF This addresses bug HIVE-2005. https://issues.apache.org/jira/browse/HIVE-2005 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e1c47b4 ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g eecd9e7 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java PRE-CREATION ql/src/test/queries/clientpositive/udf_between.q PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out e63516d ql/src/test/results/clientpositive/udf_between.q.out PRE-CREATION Diff: https://reviews.apache.org/r/3008/diff Testing --- added new test : udf_between.q Thanks, Navis Implement BETWEEN operator -- Key: HIVE-2005 URL: https://issues.apache.org/jira/browse/HIVE-2005 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.9.0 Reporter: Carl Steinbach Assignee: Navis Fix For: 0.9.0 Attachments: HIVE-2005.1.patch.txt Reference: http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#operator_between -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2659) SHOW FUNCTIONS still returns internal operators
[ https://issues.apache.org/jira/browse/HIVE-2659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinna Rao Lalam reassigned HIVE-2659: -- Assignee: Priyadarshini SHOW FUNCTIONS still returns internal operators --- Key: HIVE-2659 URL: https://issues.apache.org/jira/browse/HIVE-2659 Project: Hive Issue Type: Bug Components: UDF Reporter: Carl Steinbach Assignee: Priyadarshini -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-2005 Implement BETWEEN operator
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3008/#review3946 --- Ship it! +1. Please upload this version of the patch to JIRA and click the box that grants license rights to the ASF. I will test and then commit if everything passes. Thanks. - Carl On 2011-12-16 04:23:03, Navis Ryu wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3008/ --- (Updated 2011-12-16 04:23:03) Review request for hive, John Sichi and Carl Steinbach. Summary --- Revised version using 'between' UDF This addresses bug HIVE-2005. https://issues.apache.org/jira/browse/HIVE-2005 Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e1c47b4 ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g eecd9e7 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java PRE-CREATION ql/src/test/queries/clientpositive/udf_between.q PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out e63516d ql/src/test/results/clientpositive/udf_between.q.out PRE-CREATION Diff: https://reviews.apache.org/r/3008/diff Testing --- added new test : udf_between.q Thanks, Navis
[jira] [Commented] (HIVE-2005) Implement BETWEEN operator
[ https://issues.apache.org/jira/browse/HIVE-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170783#comment-13170783 ] jirapos...@reviews.apache.org commented on HIVE-2005: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3008/#review3946 --- Ship it! +1. Please upload this version of the patch to JIRA and click the box that grants license rights to the ASF. I will test and then commit if everything passes. Thanks. - Carl On 2011-12-16 04:23:03, Navis Ryu wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/3008/ bq. --- bq. bq. (Updated 2011-12-16 04:23:03) bq. bq. bq. Review request for hive, John Sichi and Carl Steinbach. bq. bq. bq. Summary bq. --- bq. bq. Revised version using 'between' UDF bq. bq. bq. This addresses bug HIVE-2005. bq. https://issues.apache.org/jira/browse/HIVE-2005 bq. bq. bq. Diffs bq. - bq. bq.ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e1c47b4 bq.ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g eecd9e7 bq.ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java PRE-CREATION bq.ql/src/test/queries/clientpositive/udf_between.q PRE-CREATION bq.ql/src/test/results/clientpositive/show_functions.q.out e63516d bq.ql/src/test/results/clientpositive/udf_between.q.out PRE-CREATION bq. bq. Diff: https://reviews.apache.org/r/3008/diff bq. bq. bq. Testing bq. --- bq. bq. added new test : udf_between.q bq. bq. bq. Thanks, bq. bq. Navis bq. bq. Implement BETWEEN operator -- Key: HIVE-2005 URL: https://issues.apache.org/jira/browse/HIVE-2005 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.9.0 Reporter: Carl Steinbach Assignee: Navis Fix For: 0.9.0 Attachments: HIVE-2005.1.patch.txt Reference: http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#operator_between -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-2503: create per-session HiveConf instance
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2820/ --- (Updated 2011-12-16 05:44:15.242903) Review request for hive and Carl Steinbach. Changes --- I know the test code is very awkward, but that was what I've expected from HiveServer. Summary --- We uses multiple HiveClient connected to single HiveServer. After configuring connections, we've found the environment variables are different from expected. Current hive server uses single instance of HiveConf and this seemed to make the problem. This addresses bug HIVE-2503. https://issues.apache.org/jira/browse/HIVE-2503 Diffs (updated) - service/src/java/org/apache/hadoop/hive/service/HiveServer.java 854cc99 service/src/test/org/apache/hadoop/hive/service/TestHiveServerSessions.java PRE-CREATION Diff: https://reviews.apache.org/r/2820/diff Testing --- If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex. Thanks, Navis
[jira] [Commented] (HIVE-2503) HiveServer should provide per session configuration
[ https://issues.apache.org/jira/browse/HIVE-2503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170793#comment-13170793 ] jirapos...@reviews.apache.org commented on HIVE-2503: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2820/ --- (Updated 2011-12-16 05:44:15.242903) Review request for hive and Carl Steinbach. Changes --- I know the test code is very awkward, but that was what I've expected from HiveServer. Summary --- We uses multiple HiveClient connected to single HiveServer. After configuring connections, we've found the environment variables are different from expected. Current hive server uses single instance of HiveConf and this seemed to make the problem. This addresses bug HIVE-2503. https://issues.apache.org/jira/browse/HIVE-2503 Diffs (updated) - service/src/java/org/apache/hadoop/hive/service/HiveServer.java 854cc99 service/src/test/org/apache/hadoop/hive/service/TestHiveServerSessions.java PRE-CREATION Diff: https://reviews.apache.org/r/2820/diff Testing --- If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex. Thanks, Navis HiveServer should provide per session configuration --- Key: HIVE-2503 URL: https://issues.apache.org/jira/browse/HIVE-2503 Project: Hive Issue Type: Bug Components: CLI, Server Infrastructure Affects Versions: 0.9.0 Reporter: Navis Assignee: Navis Fix For: 0.9.0 Attachments: HIVE-2503.1.patch.txt Currently ThriftHiveProcessorFactory returns same HiveConf instance to HiveServerHandler, making impossible to use per sesssion configuration. Just wrapping 'conf' - 'new HiveConf(conf)' seemed to solve this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2005) Implement BETWEEN operator
[ https://issues.apache.org/jira/browse/HIVE-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2005: Attachment: (was: HIVE-2005.1.patch.txt) Implement BETWEEN operator -- Key: HIVE-2005 URL: https://issues.apache.org/jira/browse/HIVE-2005 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.9.0 Reporter: Carl Steinbach Assignee: Navis Fix For: 0.9.0 Attachments: HIVE-2005.6.patch.txt Reference: http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#operator_between -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2005) Implement BETWEEN operator
[ https://issues.apache.org/jira/browse/HIVE-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2005: Attachment: HIVE-2005.6.patch.txt Implement BETWEEN operator -- Key: HIVE-2005 URL: https://issues.apache.org/jira/browse/HIVE-2005 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.9.0 Reporter: Carl Steinbach Assignee: Navis Fix For: 0.9.0 Attachments: HIVE-2005.6.patch.txt Reference: http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#operator_between -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2019) Implement NOW() UDF
[ https://issues.apache.org/jira/browse/HIVE-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170798#comment-13170798 ] Priyadarshini commented on HIVE-2019: - I am working on this. I have few doubts to clarify with respect to this. select now(),now() from tname The above query shud return the same time in both ofthe columns ? I find that in the reference link it does it. One more doubt is if we want of this format, select now()+0 from tname i.e adding some integers to the now() function, it shuould be possible that now() fucntion should take argument like which format user want. So for operations for adding some integer to now() funciton, we may hav to pass the format also Implement NOW() UDF --- Key: HIVE-2019 URL: https://issues.apache.org/jira/browse/HIVE-2019 Project: Hive Issue Type: New Feature Components: UDF Reporter: Carl Steinbach Reference: http://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_now -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2019) Implement NOW() UDF
[ https://issues.apache.org/jira/browse/HIVE-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170801#comment-13170801 ] Priyadarshini commented on HIVE-2019: - I will implement 2 possible uses One with default - now() Other that takes a format as an argument and supports adding integer to it i.e seelct now(format string) + 1 from tname Implement NOW() UDF --- Key: HIVE-2019 URL: https://issues.apache.org/jira/browse/HIVE-2019 Project: Hive Issue Type: New Feature Components: UDF Reporter: Carl Steinbach Reference: http://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_now -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1151 - Still Failing
Changes for Build #1144 [jvs] HIVE-1040 [jira] use sed rather than diff for masking out noise in diff-based tests (Marek Sapota via John Sichi) Summary: Replace diff -I with regex masking in Java The current diff -I approach has two problems: (1) it does not allow resolution finer than line-level, so it's impossible to mask out pattern occurrences within a line, and (2) it produces unmasked files, so if you run diff on the command line to compare the result .q.out with the checked-in file, you see the noise. My suggestion is to first run sed to replace noise patterns with an unlikely-to-occur string like ZYZZYZVA, and then diff the pre-masked files without using any -I. This would require a one-time hit to update all existing .q.out files so that they would contain the pre-masked results. Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: jsichi Differential Revision: 597 Changes for Build #1145 Changes for Build #1146 [namit] HIVE-2640 Add alterPartition to AlterHandler interface (Kevin Wilfong via namit) Changes for Build #1147 [namit] HIVE-2617 Insert overwrite table db.tname fails if partition already exists (Chinna Rao Lalam via namit) Changes for Build #1148 [heyongqiang] HIVE-2651 [jira] The variable hive.exec.mode.local.auto.tasks.max should be changed (Namit Jain via Yongqiang He) Summary: HIVE-2651 It should be called hive.exec.mode.local.auto.input.files.max instead. The number of input files are checked currently. Test Plan: EMPTY Reviewers: JIRA, heyongqiang Reviewed By: heyongqiang CC: heyongqiang Differential Revision: 861 [cws] HIVE-727. Hive Server getSchema() returns wrong schema for 'Explain' queries (Prasad Mujumdar via cws) [namit] HIVE-2611 Make index table output of create index command if index is table based (Kevin Wilfong via namit) Changes for Build #1150 [jvs] HIVE-2657 [jira] builtins JAR is not being published to Maven repo hive-cli POM does not depend on it either (Carl Steinbach via John Sichi) Summary: Make hive-cli and hive-ql depend on hive-builtins Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: jsichi Differential Revision: 897 [namit] HIVE-2654 hive.querylog.location requires parent directory to be exist or else folder creation fails (Chinna Rao Lalam via namit) Changes for Build #1151 [hashutosh] HIVE-1892 : show functions also returns internal operators (Priyadarshini via Ashutosh Chauhan) 2 tests failed. REGRESSION: org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1(TestNegativeCliDriver.java:9440) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) FAILED: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_inputddl5 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_inputddl5(TestCliDriver.java:16628) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at
[jira] [Commented] (HIVE-1892) show functions also returns internal operators
[ https://issues.apache.org/jira/browse/HIVE-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170809#comment-13170809 ] Hudson commented on HIVE-1892: -- Integrated in Hive-trunk-h0.21 #1151 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1151/]) HIVE-1892 : show functions also returns internal operators (Priyadarshini via Ashutosh Chauhan) hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1215021 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java * /hive/trunk/ql/src/test/results/clientpositive/show_functions.q.out show functions also returns internal operators -- Key: HIVE-1892 URL: https://issues.apache.org/jira/browse/HIVE-1892 Project: Hive Issue Type: Bug Components: Security Reporter: Namit Jain Assignee: Priyadarshini Fix For: 0.9.0 Attachments: HIVE-1892.patch show functions: returns bigint etc. in its outputs, which are not valid external functions -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2660) Need better exception handling in RCFile tolerate corruptions mode
Need better exception handling in RCFile tolerate corruptions mode -- Key: HIVE-2660 URL: https://issues.apache.org/jira/browse/HIVE-2660 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Ramkumar Vadali Assignee: Ramkumar Vadali Priority: Minor The exception handling in nextKeyValueTolerateCorruptions treats IOException as follows: - if EOFException, corrupt, can be tolerated - If CheckSumException, corrupt, can be tolerated - else not a corruption, re-throw But the compression code can also throw IOException in case of corruption, which will get re-thrown in this case. The correct way of handling IOException is: - if BlockMissingException, re-throw. - if not BlockMissingException - corruption, can be tolerated -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2486) Phabricator for code review
[ https://issues.apache.org/jira/browse/HIVE-2486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ala Makota updated HIVE-2486: - Attachment: xyz Phabricator for code review --- Key: HIVE-2486 URL: https://issues.apache.org/jira/browse/HIVE-2486 Project: Hive Issue Type: Bug Reporter: Marek Sapota Assignee: Marek Sapota Fix For: 0.8.0, 0.9.0 Attachments: D135.1.patch, D135.2.patch, D387.1.patch, D51.1.patch, D51.2.patch, D51.2.patch, D57.1.patch, D57.2.patch, D63.1.patch, D63.2.patch, D63.3.patch, D69.1.patch, D69.2.patch, D69.3.patch, D75.1.patch, D75.2.patch, D9.1.diff, D9.2.diff, HIVE-2486.D387.2.patch, SOAP, SOAP, TEST, my_uploaded_file, my_uploaded_file, x, xyz, xyz Replacing Review Board with Phabricator and Differential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2486) Phabricator for code review
[ https://issues.apache.org/jira/browse/HIVE-2486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marek Sapota resolved HIVE-2486. Resolution: Fixed Phabricator for code review --- Key: HIVE-2486 URL: https://issues.apache.org/jira/browse/HIVE-2486 Project: Hive Issue Type: Bug Reporter: Marek Sapota Assignee: Marek Sapota Fix For: 0.8.0, 0.9.0 Attachments: D135.1.patch, D135.2.patch, D387.1.patch, D51.1.patch, D51.2.patch, D51.2.patch, D57.1.patch, D57.2.patch, D63.1.patch, D63.2.patch, D63.3.patch, D69.1.patch, D69.2.patch, D69.3.patch, D75.1.patch, D75.2.patch, D9.1.diff, D9.2.diff, HIVE-2486.D387.2.patch, SOAP, SOAP, TEST, my_uploaded_file, my_uploaded_file, x, xyz, xyz Replacing Review Board with Phabricator and Differential. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
