[jira] [Updated] (HIVE-2901) Hive union with NULL constant and string in same column returns all null
[ https://issues.apache.org/jira/browse/HIVE-2901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2901: -- Attachment: HIVE-2901.D2541.1.patch navis requested code review of HIVE-2901 [jira] Hive union with NULL constant and string in same column returns all null. Reviewers: JIRA DPAL-1029 Hive union with NULL constant and string in same column returns all null select x from (select value as x from src union all select NULL as x from src)a; This query produces all nulls, where value is a string column. Notably, select x from (select key as x from src union all select NULL as x from src)a; where key is a string, but can be cast to a double, the query returns correct results. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D2541 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java ql/src/test/queries/clientpositive/union_null.q MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/5775/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. Hive union with NULL constant and string in same column returns all null Key: HIVE-2901 URL: https://issues.apache.org/jira/browse/HIVE-2901 Project: Hive Issue Type: Bug Reporter: Kevin Wilfong Priority: Critical Attachments: HIVE-2901.D2541.1.patch select x from (select value as x from src union all select NULL as x from src)a; This query produces all nulls, where value is a string column. Notably, select x from (select key as x from src union all select NULL as x from src)a; where key is a string, but can be cast to a double, the query returns correct results. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2901) Hive union with NULL constant and string in same column returns all null
[ https://issues.apache.org/jira/browse/HIVE-2901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis reassigned HIVE-2901: --- Assignee: Navis Hive union with NULL constant and string in same column returns all null Key: HIVE-2901 URL: https://issues.apache.org/jira/browse/HIVE-2901 Project: Hive Issue Type: Bug Reporter: Kevin Wilfong Assignee: Navis Priority: Critical Attachments: HIVE-2901.D2541.1.patch, HIVE-2901.D2541.2.patch select x from (select value as x from src union all select NULL as x from src)a; This query produces all nulls, where value is a string column. Notably, select x from (select key as x from src union all select NULL as x from src)a; where key is a string, but can be cast to a double, the query returns correct results. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2901) Hive union with NULL constant and string in same column returns all null
[ https://issues.apache.org/jira/browse/HIVE-2901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2901: -- Attachment: HIVE-2901.D2541.2.patch navis updated the revision HIVE-2901 [jira] Hive union with NULL constant and string in same column returns all null. Reviewers: JIRA Missed test result REVISION DETAIL https://reviews.facebook.net/D2541 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java ql/src/test/queries/clientpositive/union_null.q ql/src/test/results/clientpositive/union_null.q.out Hive union with NULL constant and string in same column returns all null Key: HIVE-2901 URL: https://issues.apache.org/jira/browse/HIVE-2901 Project: Hive Issue Type: Bug Reporter: Kevin Wilfong Assignee: Navis Priority: Critical Attachments: HIVE-2901.D2541.1.patch, HIVE-2901.D2541.2.patch select x from (select value as x from src union all select NULL as x from src)a; This query produces all nulls, where value is a string column. Notably, select x from (select key as x from src union all select NULL as x from src)a; where key is a string, but can be cast to a double, the query returns correct results. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2913) BlockMergeTask Doesn't Honor Job Configuration Properties when used directly
BlockMergeTask Doesn't Honor Job Configuration Properties when used directly Key: HIVE-2913 URL: https://issues.apache.org/jira/browse/HIVE-2913 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.8.1 Reporter: Thiruvel Thirumoolan Assignee: Thiruvel Thirumoolan Priority: Minor Fix For: 0.9.0 BlockMergeTask has a main() and when used directly (instead of say partition concatenate feature), the -jobconf arguments are not honored. This is not something most people directly use. Usage: BlockMergeTask -input colon seperated input paths -outputDir outputDir [-jobconf k1=v1 [-jobconf k2=v2] ...] To reproduce: Run BlockMergeTask with say -jobconf mapred.job.name=test and launched job will have a different name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2913) BlockMergeTask Doesn't Honor Job Configuration Properties when used directly
[ https://issues.apache.org/jira/browse/HIVE-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvel Thirumoolan updated HIVE-2913: --- Attachment: blockmerge.sh Simple shell script that helped to test. Place it under $HIVE_HOME/bin/ext. BlockMergeTask Doesn't Honor Job Configuration Properties when used directly Key: HIVE-2913 URL: https://issues.apache.org/jira/browse/HIVE-2913 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.8.1 Reporter: Thiruvel Thirumoolan Assignee: Thiruvel Thirumoolan Priority: Minor Fix For: 0.9.0 Attachments: HIVE-2913_1.patch, blockmerge.sh BlockMergeTask has a main() and when used directly (instead of say partition concatenate feature), the -jobconf arguments are not honored. This is not something most people directly use. Usage: BlockMergeTask -input colon seperated input paths -outputDir outputDir [-jobconf k1=v1 [-jobconf k2=v2] ...] To reproduce: Run BlockMergeTask with say -jobconf mapred.job.name=test and launched job will have a different name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2913) BlockMergeTask Doesn't Honor Job Configuration Properties when used directly
[ https://issues.apache.org/jira/browse/HIVE-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2913: -- Attachment: HIVE-2913.D2547.1.patch thiruvel requested code review of HIVE-2913 [jira] BlockMergeTask Doesn't Honor Job Configuration Properties when used directly. Reviewers: JIRA https://issues.apache.org/jira/browse/HIVE-2913 HIVE-2913: Running BlockMergeTask through its main() interface should honor jobconf properties BlockMergeTask has a main() and when used directly (instead of say partition concatenate feature), the -jobconf arguments are not honored. This is not something most people directly use. Usage: BlockMergeTask -input colon seperated input paths -outputDir outputDir [-jobconf k1=v1 [-jobconf k2=v2] ...] To reproduce: Run BlockMergeTask with say -jobconf mapred.job.name=test and launched job will have a different name. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D2547 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/BlockMergeTask.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/5793/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. BlockMergeTask Doesn't Honor Job Configuration Properties when used directly Key: HIVE-2913 URL: https://issues.apache.org/jira/browse/HIVE-2913 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.8.1 Reporter: Thiruvel Thirumoolan Assignee: Thiruvel Thirumoolan Priority: Minor Fix For: 0.9.0 Attachments: HIVE-2913.D2547.1.patch, HIVE-2913_1.patch, blockmerge.sh BlockMergeTask has a main() and when used directly (instead of say partition concatenate feature), the -jobconf arguments are not honored. This is not something most people directly use. Usage: BlockMergeTask -input colon seperated input paths -outputDir outputDir [-jobconf k1=v1 [-jobconf k2=v2] ...] To reproduce: Run BlockMergeTask with say -jobconf mapred.job.name=test and launched job will have a different name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2843) UDAF to convert an aggregation to a map
[ https://issues.apache.org/jira/browse/HIVE-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242286#comment-13242286 ] Thiruvel Thirumoolan commented on HIVE-2843: @David, It will be nice to get this in. It will also help when people import data from RDBMS/other stores and want to save space by converting first level columns into complex types and vice-versa using UDTF (explode()). Would naming it 'implode' help? UDAF to convert an aggregation to a map --- Key: HIVE-2843 URL: https://issues.apache.org/jira/browse/HIVE-2843 Project: Hive Issue Type: New Feature Components: UDF Affects Versions: 0.9.0 Reporter: David Worms Priority: Minor Labels: features, udf I propose the addition of two new Hive UDAF to help with maps in Apache Hive. The source code is available on GitHub at https://github.com/wdavidw/hive-udf in two Java classes: UDAFToMap and UDAFToOrderedMap. The first function convert an aggregation into a map and is internally using a Java `HashMap`. The second function extends the first one. It convert an aggregation into an ordered map and is internally using a Java `TreeMap`. They both extends the `AbstractGenericUDAFResolver` class. Also, I have covered the motivations and usages of those UDAF in a blog post at http://adaltas.com/blog/2012/03/06/hive-udaf-map-conversion/ If you are interested by my proposal, I'll take the time to update this issue while following the guideline posted on the wiki to create an appropriate path. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2843) UDAF to convert an aggregation to a map
[ https://issues.apache.org/jira/browse/HIVE-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242373#comment-13242373 ] David Worms commented on HIVE-2843: --- Thiruvel, I have been thinking to name it implode at first. Then i step back because of 2 reasons: 1. there are potentially different map implementations, for example, i'm personally intensively using the toOrderedMap UDAF which in my case would be much faster than using an 'order by' clause in my query. 2. implode is not restricted to map, and actually would probably be more appropriate for an array conversion. What do you think? UDAF to convert an aggregation to a map --- Key: HIVE-2843 URL: https://issues.apache.org/jira/browse/HIVE-2843 Project: Hive Issue Type: New Feature Components: UDF Affects Versions: 0.9.0 Reporter: David Worms Priority: Minor Labels: features, udf I propose the addition of two new Hive UDAF to help with maps in Apache Hive. The source code is available on GitHub at https://github.com/wdavidw/hive-udf in two Java classes: UDAFToMap and UDAFToOrderedMap. The first function convert an aggregation into a map and is internally using a Java `HashMap`. The second function extends the first one. It convert an aggregation into an ordered map and is internally using a Java `TreeMap`. They both extends the `AbstractGenericUDAFResolver` class. Also, I have covered the motivations and usages of those UDAF in a blog post at http://adaltas.com/blog/2012/03/06/hive-udaf-map-conversion/ If you are interested by my proposal, I'll take the time to update this issue while following the guideline posted on the wiki to create an appropriate path. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2909) SHOW COLUMNS table_name; to provide a comma-delimited list of columns.
[ https://issues.apache.org/jira/browse/HIVE-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242575#comment-13242575 ] Namit Jain commented on HIVE-2909: -- https://cwiki.apache.org/Hive/phabricatorcodereview.html SHOW COLUMNS table_name; to provide a comma-delimited list of columns. -- Key: HIVE-2909 URL: https://issues.apache.org/jira/browse/HIVE-2909 Project: Hive Issue Type: New Feature Reporter: Adam Kramer Assignee: Dikang Gu Priority: Minor Due to the way that SELECT * and partitioning works, it is frequently obnoxious to insert data into tables of the same schema. This could be fixed in a number of ways, all murky; this feature request reduces the obnoxicity of the current situation. SHOW COLUMNS foo; OK bar, baz, tball, ds ...then I could just copy the first three and not the last. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2909) SHOW COLUMNS table_name; to provide a comma-delimited list of columns.
[ https://issues.apache.org/jira/browse/HIVE-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dikang Gu reassigned HIVE-2909: --- Assignee: Dikang Gu SHOW COLUMNS table_name; to provide a comma-delimited list of columns. -- Key: HIVE-2909 URL: https://issues.apache.org/jira/browse/HIVE-2909 Project: Hive Issue Type: New Feature Reporter: Adam Kramer Assignee: Dikang Gu Priority: Minor Due to the way that SELECT * and partitioning works, it is frequently obnoxious to insert data into tables of the same schema. This could be fixed in a number of ways, all murky; this feature request reduces the obnoxicity of the current situation. SHOW COLUMNS foo; OK bar, baz, tball, ds ...then I could just copy the first three and not the last. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2816) Extend IMPORT/EXPORT to Support Metata Only Operations
[ https://issues.apache.org/jira/browse/HIVE-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242618#comment-13242618 ] Bejoy KS commented on HIVE-2816: Also this functionality needs to be supported in a database level as well in addition to table level. Metadata exports/imports can happen at selective tables and could be on whole dbs itself. Extend IMPORT/EXPORT to Support Metata Only Operations -- Key: HIVE-2816 URL: https://issues.apache.org/jira/browse/HIVE-2816 Project: Hive Issue Type: New Feature Components: Import/Export Reporter: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2914) HiveConnection constructor ignores passed-in properties object
HiveConnection constructor ignores passed-in properties object
--
Key: HIVE-2914
URL: https://issues.apache.org/jira/browse/HIVE-2914
Project: Hive
Issue Type: Bug
Components: JDBC
Reporter: Ashutosh Chauhan
In local mode HiveConf should initialize itself with passed in properties and
in remote mode, connection should execute series of {{set}} command for all the
properties.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-0.8.1-SNAPSHOT-h0.21 - Build # 238 - Failure
Changes for Build #238 1 tests failed. REGRESSION: org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1(TestNegativeCliDriver.java:9440) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785) The Apache Jenkins build system has built Hive-0.8.1-SNAPSHOT-h0.21 (build #238) Status: Failure Check console output at https://builds.apache.org/job/Hive-0.8.1-SNAPSHOT-h0.21/238/ to view the results.
build failed
Hi, After checking out the source code, I go to my eclipse workspace and execute ant clean package eclipse-files, but I receive the following message: Unable to locate tools.jar. Expected to find it in /usr/lib/jvm/java-6-openjdk/lib/tools.jar Buildfile: build.xml does not exist! Build failed Actually jar.tools is in my /usr/lib/jvm/java-1.5.0-gcj-4.4/lib. Can I configure it to look in the former directory? Thank you in advance for your assistance, Mahsa
[jira] [Updated] (HIVE-538) make hive_jdbc.jar self-containing
[ https://issues.apache.org/jira/browse/HIVE-538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-538: - Attachment: HIVE-538.D2553.1.patch ashutoshc requested code review of HIVE-538 [jira] make hive_jdbc.jar self-containing. Reviewers: JIRA https://issues.apache.org/jira/browse/HIVE-538 This patch introduces two new targets: a) jar-jdbc-combined : This target generates a jar file containing all the hive jars required for jdbc driver. b) jar-jdbc-rt-deps : This target generates a jar file which contains all the hive runtime dependcies in a single jar. Currently, most jars in hive/build/dist/lib and the hadoop-*-core.jar are required in the classpath to run jdbc applications on hive. We need to do atleast the following to get rid of most unnecessary dependencies: 1. get rid of dynamic serde and use a standard serialization format, maybe tab separated, json or avro 2. dont use hadoop configuration parameters 3. repackage thrift and fb303 classes into hive_jdbc.jar TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D2553 AFFECTED FILES build.xml ivy/libraries.properties ivy.xml MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/5799/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. make hive_jdbc.jar self-containing -- Key: HIVE-538 URL: https://issues.apache.org/jira/browse/HIVE-538 Project: Hive Issue Type: Improvement Components: JDBC Affects Versions: 0.3.0, 0.4.0, 0.6.0 Reporter: Raghotham Murthy Attachments: HIVE-538.D2553.1.patch Currently, most jars in hive/build/dist/lib and the hadoop-*-core.jar are required in the classpath to run jdbc applications on hive. We need to do atleast the following to get rid of most unnecessary dependencies: 1. get rid of dynamic serde and use a standard serialization format, maybe tab separated, json or avro 2. dont use hadoop configuration parameters 3. repackage thrift and fb303 classes into hive_jdbc.jar -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-538) make hive_jdbc.jar self-containing
[ https://issues.apache.org/jira/browse/HIVE-538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan reassigned HIVE-538: - Assignee: Ashutosh Chauhan make hive_jdbc.jar self-containing -- Key: HIVE-538 URL: https://issues.apache.org/jira/browse/HIVE-538 Project: Hive Issue Type: Improvement Components: JDBC Affects Versions: 0.3.0, 0.4.0, 0.6.0 Reporter: Raghotham Murthy Assignee: Ashutosh Chauhan Attachments: HIVE-538.D2553.1.patch Currently, most jars in hive/build/dist/lib and the hadoop-*-core.jar are required in the classpath to run jdbc applications on hive. We need to do atleast the following to get rid of most unnecessary dependencies: 1. get rid of dynamic serde and use a standard serialization format, maybe tab separated, json or avro 2. dont use hadoop configuration parameters 3. repackage thrift and fb303 classes into hive_jdbc.jar -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2900) Add target for fatjar
[ https://issues.apache.org/jira/browse/HIVE-2900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2900: --- Resolution: Duplicate Fix Version/s: (was: 0.9.0) Status: Resolved (was: Patch Available) Closed in favor of HIVE-538 Add target for fatjar - Key: HIVE-2900 URL: https://issues.apache.org/jira/browse/HIVE-2900 Project: Hive Issue Type: New Feature Components: Build Infrastructure Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-2900.D2469.1.patch Currently, hive generates many different jars and also has slew of deps. It would be useful to bundle all these jars together for easier downstream consumption. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-538) make hive_jdbc.jar self-containing
[ https://issues.apache.org/jira/browse/HIVE-538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242752#comment-13242752 ] Ashutosh Chauhan commented on HIVE-538: --- Patch for generating artifacts for jdbc drivers which makes it easier for folks using jdbc driver to include it in their projects. Note two noticeable omissions from hive-jdbc-rt-deps.jar datanucleus-core.jar and datanucleus-rdbms.jar If those are packaged in same jar then datanucleus have trouble loading them, so I excluded those. As a result, those still needs to put in application's classpath separately make hive_jdbc.jar self-containing -- Key: HIVE-538 URL: https://issues.apache.org/jira/browse/HIVE-538 Project: Hive Issue Type: Improvement Components: JDBC Affects Versions: 0.3.0, 0.4.0, 0.6.0 Reporter: Raghotham Murthy Attachments: HIVE-538.D2553.1.patch Currently, most jars in hive/build/dist/lib and the hadoop-*-core.jar are required in the classpath to run jdbc applications on hive. We need to do atleast the following to get rid of most unnecessary dependencies: 1. get rid of dynamic serde and use a standard serialization format, maybe tab separated, json or avro 2. dont use hadoop configuration parameters 3. repackage thrift and fb303 classes into hive_jdbc.jar -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HIVE-2504) Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory
[ https://issues.apache.org/jira/browse/HIVE-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise reopened HIVE-2504: Reopening per Rohini's request. Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory - Key: HIVE-2504 URL: https://issues.apache.org/jira/browse/HIVE-2504 Project: Hive Issue Type: Bug Components: Metastore Reporter: Carl Steinbach Assignee: Chinna Rao Lalam Fix For: 0.9.0 Attachments: HIVE-2504.patch When the Hive Metastore creates a subdirectory in the Hive warehouse for a new table it does so with the default HDFS permissions. Since the default dfs.umask value is 022, this means that the new subdirectory will not inherit the group write permissions of the hive warehouse directory. We should make the umask used by Warehouse.mkdirs() configurable, and set it to use a default value of 002. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2504) Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory
[ https://issues.apache.org/jira/browse/HIVE-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242787#comment-13242787 ] Rohini Palaniswamy commented on HIVE-2504: -- Had Thomas reopen the issue as I could not. Couple of issues with the fix: * The config that is actually being looked for is HIVE_FILES_UMASK_VALUE instead of hive.files.umask.value as the Enum name() function is used and not varname. short umaskVal = (short) conf.getInt(HiveConf.ConfVars.HIVE_FILES_UMASK_VALUE.name(), 0002); * Because of the new FsPermission(short mode) the value for the configuration has to be specified in decimal instead of a octal umask. Would be more happy if the fix did not resort to using this setting at all and instead applied the permissions of the parent directory. Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory - Key: HIVE-2504 URL: https://issues.apache.org/jira/browse/HIVE-2504 Project: Hive Issue Type: Bug Components: Metastore Reporter: Carl Steinbach Assignee: Chinna Rao Lalam Fix For: 0.9.0 Attachments: HIVE-2504.patch When the Hive Metastore creates a subdirectory in the Hive warehouse for a new table it does so with the default HDFS permissions. Since the default dfs.umask value is 022, this means that the new subdirectory will not inherit the group write permissions of the hive warehouse directory. We should make the umask used by Warehouse.mkdirs() configurable, and set it to use a default value of 002. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2908) Hive: Extend ALTER TABLE DROP PARTITION syntax to use all comparators
[ https://issues.apache.org/jira/browse/HIVE-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242791#comment-13242791 ] Sambavi Muthukrishnan commented on HIVE-2908: - This patch includes the fix to the test log for escape1.q.out. Per discussion with Kevin included that fix and uploaded the svn patch. Hive: Extend ALTER TABLE DROP PARTITION syntax to use all comparators - Key: HIVE-2908 URL: https://issues.apache.org/jira/browse/HIVE-2908 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Sambavi Muthukrishnan Assignee: Sambavi Muthukrishnan Priority: Minor Attachments: HIVE-2908.D2523.1.patch, HIVE-2908.D2523.2.patch, HIVE-2908.final.patch.txt Original Estimate: 96h Remaining Estimate: 96h To drop a partition from a Hive table, this works: ALTER TABLE foo DROP PARTITION(ds = 'date') ...but it should also work to drop all partitions prior to date. ALTER TABLE foo DROP PARTITION(ds 'date') This task is to implement ALTER TABLE DROP PARTITION for all of the comparators, = = = != instead of just for =. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2908) Hive: Extend ALTER TABLE DROP PARTITION syntax to use all comparators
[ https://issues.apache.org/jira/browse/HIVE-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sambavi Muthukrishnan updated HIVE-2908: Attachment: HIVE-2908.final.patch.txt Final patch after code review for HIVE-2908 Hive: Extend ALTER TABLE DROP PARTITION syntax to use all comparators - Key: HIVE-2908 URL: https://issues.apache.org/jira/browse/HIVE-2908 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Sambavi Muthukrishnan Assignee: Sambavi Muthukrishnan Priority: Minor Attachments: HIVE-2908.D2523.1.patch, HIVE-2908.D2523.2.patch, HIVE-2908.final.patch.txt Original Estimate: 96h Remaining Estimate: 96h To drop a partition from a Hive table, this works: ALTER TABLE foo DROP PARTITION(ds = 'date') ...but it should also work to drop all partitions prior to date. ALTER TABLE foo DROP PARTITION(ds 'date') This task is to implement ALTER TABLE DROP PARTITION for all of the comparators, = = = != instead of just for =. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2875) Renaming partition changes partition location prefix
[ https://issues.apache.org/jira/browse/HIVE-2875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2875: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed. Thanks Kevin Renaming partition changes partition location prefix Key: HIVE-2875 URL: https://issues.apache.org/jira/browse/HIVE-2875 Project: Hive Issue Type: Bug Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2875.D2349.1.patch, HIVE-2875.D2349.2.patch Renaming a partition changes the location of the partition to the default location of the table, followed by the partition specification. It should just change the partition specification of the path. If the path does not end with the old partition specification, we should probably throw an exception because renaming a partition should not change the path so dramatically, and not changing the path to reflect the new partition name could leave the partition in a very confusing state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2908) Hive: Extend ALTER TABLE DROP PARTITION syntax to use all comparators
[ https://issues.apache.org/jira/browse/HIVE-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242795#comment-13242795 ] Phabricator commented on HIVE-2908: --- kevinwilfong has accepted the revision HIVE-2908 [jira] Hive: Extend ALTER TABLE DROP PARTITION syntax to use all comparators. +1 Looks good, and thanks for uploading the patch with all the files to the JIRA. Will commit if tests pass. Will let you know about the documentation. REVISION DETAIL https://reviews.facebook.net/D2523 BRANCH svn Hive: Extend ALTER TABLE DROP PARTITION syntax to use all comparators - Key: HIVE-2908 URL: https://issues.apache.org/jira/browse/HIVE-2908 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Sambavi Muthukrishnan Assignee: Sambavi Muthukrishnan Priority: Minor Attachments: HIVE-2908.D2523.1.patch, HIVE-2908.D2523.2.patch, HIVE-2908.final.patch.txt Original Estimate: 96h Remaining Estimate: 96h To drop a partition from a Hive table, this works: ALTER TABLE foo DROP PARTITION(ds = 'date') ...but it should also work to drop all partitions prior to date. ALTER TABLE foo DROP PARTITION(ds 'date') This task is to implement ALTER TABLE DROP PARTITION for all of the comparators, = = = != instead of just for =. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1340 - Still Failing
Changes for Build #1332 [hashutosh] HIVE-2797: Make the IP address of a Thrift client available to HMSHandler. (Kevin Wilfong via Ashutosh Chauhan) Changes for Build #1333 Changes for Build #1334 Changes for Build #1335 Changes for Build #1336 [hashutosh] HIVE-2891: TextConverter for UDF's is inefficient if the input object is already Text or Lazy (Cliff Engle via Ashutosh Chauhan) Changes for Build #1337 Changes for Build #1338 [namit] HIVE-2518 pull junit jar from maven repos via ivy (Kevin Wilfong via namit) [hashutosh] HIVE-2898: Add nicer helper functions for adding and reading metadata from RCFiles (Owen Omalley via Ashutosh Chauhan) Changes for Build #1339 Changes for Build #1340 4 tests failed. REGRESSION: org.apache.hadoop.hive.cli.TestHBaseMinimrCliDriver.testCliDriver_hbase_bulk Error Message: Unexpected exception Stack Trace: junit.framework.AssertionFailedError: Unexpected exception at junit.framework.Assert.fail(Assert.java:47) at org.apache.hadoop.hive.cli.TestHBaseMinimrCliDriver.testCliDriver_hbase_bulk(TestHBaseMinimrCliDriver.java:113) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:154) at junit.framework.TestCase.runBare(TestCase.java:127) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:118) at junit.framework.TestSuite.runTest(TestSuite.java:208) at junit.framework.TestSuite.run(TestSuite.java:203) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.extensions.TestSetup.run(TestSetup.java:23) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) REGRESSION: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_add_part_exist Error Message: Unexpected exception in setup Stack Trace: junit.framework.AssertionFailedError: Unexpected exception in setup at junit.framework.Assert.fail(Assert.java:47) at org.apache.hadoop.hive.cli.TestCliDriver.setUp(TestCliDriver.java:60) at junit.framework.TestCase.runBare(TestCase.java:125) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:118) at junit.framework.TestSuite.runTest(TestSuite.java:208) at junit.framework.TestSuite.run(TestSuite.java:203) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) REGRESSION: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter1 Error Message: Unexpected exception in setup Stack Trace: junit.framework.AssertionFailedError: Unexpected exception in setup at junit.framework.Assert.fail(Assert.java:47) at org.apache.hadoop.hive.cli.TestCliDriver.setUp(TestCliDriver.java:60) at junit.framework.TestCase.runBare(TestCase.java:125) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:118) at junit.framework.TestSuite.runTest(TestSuite.java:208) at junit.framework.TestSuite.run(TestSuite.java:203) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) FAILED: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace:
[jira] [Created] (HIVE-2915) up-to-date metastore ER diagram
up-to-date metastore ER diagram --- Key: HIVE-2915 URL: https://issues.apache.org/jira/browse/HIVE-2915 Project: Hive Issue Type: Improvement Components: Metastore Affects Versions: 0.8.1 Environment: Ubuntu, MySQL workbech Reporter: Mahsa Mofidpoor Priority: Trivial Fix For: 0.8.1 The up-to-date ER diagram for metastore is not available in Hive wiki. Now there are 31 tables residing in metastore including the ones for indexes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2915) up-to-date metastore ER diagram
[ https://issues.apache.org/jira/browse/HIVE-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahsa Mofidpoor updated HIVE-2915: -- Attachment: HiveMetastoreCompleterelations.pdf up-to-date metastore ER diagram --- Key: HIVE-2915 URL: https://issues.apache.org/jira/browse/HIVE-2915 Project: Hive Issue Type: Improvement Components: Metastore Affects Versions: 0.8.1 Environment: Ubuntu, MySQL workbech Reporter: Mahsa Mofidpoor Priority: Trivial Labels: ER, Hive, metadata, metastore Fix For: 0.8.1 Attachments: HiveMetastoreCompleterelations.pdf Original Estimate: 2h Remaining Estimate: 2h The up-to-date ER diagram for metastore is not available in Hive wiki. Now there are 31 tables residing in metastore including the ones for indexes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2915) up-to-date metastore ER diagram
[ https://issues.apache.org/jira/browse/HIVE-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahsa Mofidpoor updated HIVE-2915: -- Release Note: Updated ER Diagram of the Meta-store has been attached with this ticket. Status: Patch Available (was: Open) Updated ER Diagram of the Meta-store has been attached with this ticket. up-to-date metastore ER diagram --- Key: HIVE-2915 URL: https://issues.apache.org/jira/browse/HIVE-2915 Project: Hive Issue Type: Improvement Components: Metastore Affects Versions: 0.8.1 Environment: Ubuntu, MySQL workbech Reporter: Mahsa Mofidpoor Priority: Trivial Labels: ER, Hive, metadata, metastore Fix For: 0.8.1 Attachments: HiveMetastoreCompleterelations.pdf Original Estimate: 2h Remaining Estimate: 2h The up-to-date ER diagram for metastore is not available in Hive wiki. Now there are 31 tables residing in metastore including the ones for indexes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2906) Support providing some table properties by user via SQL
[
https://issues.apache.org/jira/browse/HIVE-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2906:
Status: Patch Available (was: Open)
Support providing some table properties by user via SQL
---
Key: HIVE-2906
URL: https://issues.apache.org/jira/browse/HIVE-2906
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Attachments: HIVE-2906.D2499.1.patch, HIVE-2906.D2499.2.patch,
HIVE-2906.D2499.3.patch, HIVE-2906.D2499.4.patch
Some properties are needed to be provided to StorageHandler by user in
runtime. It might be an address for remote resource or retry count for access
or maximum version count(for hbase), etc.
For example,
{code}
select emp.empno, emp.ename from hbase_emp ('max.version'='3') emp;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2906) Support providing some table properties by user via SQL
[
https://issues.apache.org/jira/browse/HIVE-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-2906:
--
Attachment: HIVE-2906.D2499.5.patch
navis updated the revision HIVE-2906 [jira] Support providing some table
properties by user via SQL.
Reviewers: JIRA
1. Fixed NPE
2. Passed all tests
REVISION DETAIL
https://reviews.facebook.net/D2499
AFFECTED FILES
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/ResultWrapper.java
hbase-handler/src/test/queries/hbase_max_versions.q
hbase-handler/src/test/results/hbase_max_versions.q.out
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java
ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
ql/src/test/queries/clientpositive/serde_user_properties.q
ql/src/test/results/clientpositive/serde_user_properties.q.out
Support providing some table properties by user via SQL
---
Key: HIVE-2906
URL: https://issues.apache.org/jira/browse/HIVE-2906
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Attachments: HIVE-2906.D2499.1.patch, HIVE-2906.D2499.2.patch,
HIVE-2906.D2499.3.patch, HIVE-2906.D2499.4.patch, HIVE-2906.D2499.5.patch
Some properties are needed to be provided to StorageHandler by user in
runtime. It might be an address for remote resource or retry count for access
or maximum version count(for hbase), etc.
For example,
{code}
select emp.empno, emp.ename from hbase_emp ('max.version'='3') emp;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2711) Make the header of RCFile unique
[ https://issues.apache.org/jira/browse/HIVE-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242930#comment-13242930 ] Ashutosh Chauhan commented on HIVE-2711: That makes sense. I will take a look. Make the header of RCFile unique Key: HIVE-2711 URL: https://issues.apache.org/jira/browse/HIVE-2711 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: HIVE-2711.D2115.1.patch The RCFile implementation was copied from Hadoop's SequenceFile and copied the 'magic' string in the header. This means that you can't use the header to distinguish between RCFiles and SequenceFiles. I'd propose that we create a new header for RCFiles (RCF?) to replace the current SEQ. To maintain compatibility, we'll need to continue to accept the current 'SEQ\06' and just make new files contain the new header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2711) Make the header of RCFile unique
[ https://issues.apache.org/jira/browse/HIVE-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242932#comment-13242932 ] Phabricator commented on HIVE-2711: --- ashutoshc has commented on the revision HIVE-2711 [jira] Make the header of RCFile unique. It seems like you are backward compatible with SEQ6 but not for anything before that. If the intent is to break backward compatibility, then I think we should send an email on both dev and user list about this change, since folks might have historical data in this format, which can't be read then. INLINE COMMENTS ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java:1205 By doing this, you are effectively dropping the ability to read data before version SEQ6. This is backward incompatible change. ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java:1224 By removing this, you will not be able to read data before version SEQ2. ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java:1252 This removes ability to read before version 6. ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java:1256 This removes ability to read before SEQ1 ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java:1357 This removes ability to read before SEQ1 REVISION DETAIL https://reviews.facebook.net/D2115 Make the header of RCFile unique Key: HIVE-2711 URL: https://issues.apache.org/jira/browse/HIVE-2711 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: HIVE-2711.D2115.1.patch The RCFile implementation was copied from Hadoop's SequenceFile and copied the 'magic' string in the header. This means that you can't use the header to distinguish between RCFiles and SequenceFiles. I'd propose that we create a new header for RCFiles (RCF?) to replace the current SEQ. To maintain compatibility, we'll need to continue to accept the current 'SEQ\06' and just make new files contain the new header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2711) Make the header of RCFile unique
[ https://issues.apache.org/jira/browse/HIVE-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2711: --- Status: Open (was: Patch Available) Make the header of RCFile unique Key: HIVE-2711 URL: https://issues.apache.org/jira/browse/HIVE-2711 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: HIVE-2711.D2115.1.patch The RCFile implementation was copied from Hadoop's SequenceFile and copied the 'magic' string in the header. This means that you can't use the header to distinguish between RCFiles and SequenceFiles. I'd propose that we create a new header for RCFiles (RCF?) to replace the current SEQ. To maintain compatibility, we'll need to continue to accept the current 'SEQ\06' and just make new files contain the new header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2585) Collapse hive.metastore.uris and hive.metastore.local
[ https://issues.apache.org/jira/browse/HIVE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2585: -- Attachment: HIVE-2585.D2559.1.patch ashutoshc requested code review of HIVE-2585 [jira] Collapse hive.metastore.uris and hive.metastore.local. Reviewers: JIRA https://issues.apache.org/jira/browse/HIVE-2585 Get rid of hive.metastore.local. By default, it was assumed to be true as a result hive will try to connect to local metastore. That behavior is preserved since hive.metastore.uris is empty by default. We should just have hive.metastore.uris. If it is empty, we shall assume local mode, if non-empty we shall use that string to connect to remote metastore. Having two different keys for same information is confusing. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D2559 AFFECTED FILES shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java conf/hive-default.xml.template metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartitionRemote.java metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreAuthorization.java metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStoreIpAddress.java metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java data/conf/hive-site.xml common/src/java/org/apache/hadoop/hive/conf/HiveConf.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/5811/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. Collapse hive.metastore.uris and hive.metastore.local - Key: HIVE-2585 URL: https://issues.apache.org/jira/browse/HIVE-2585 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-2585.D2559.1.patch We should just have hive.metastore.uris. If it is empty, we shall assume local mode, if non-empty we shall use that string to connect to remote metastore. Having two different keys for same information is confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2585) Collapse hive.metastore.uris and hive.metastore.local
[ https://issues.apache.org/jira/browse/HIVE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan reassigned HIVE-2585: -- Assignee: Ashutosh Chauhan Collapse hive.metastore.uris and hive.metastore.local - Key: HIVE-2585 URL: https://issues.apache.org/jira/browse/HIVE-2585 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-2585.D2559.1.patch We should just have hive.metastore.uris. If it is empty, we shall assume local mode, if non-empty we shall use that string to connect to remote metastore. Having two different keys for same information is confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2676) The row count that loaded to a table may not right
[ https://issues.apache.org/jira/browse/HIVE-2676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2676: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed. Thanks binlijin The row count that loaded to a table may not right - Key: HIVE-2676 URL: https://issues.apache.org/jira/browse/HIVE-2676 Project: Hive Issue Type: Improvement Reporter: binlijin Priority: Minor Labels: patch Attachments: HIVE-2676.patch create table tablename as SELECT *** At the end hive will print a number that show how many Rows loaded to the tablename, but sometimes the number is not right. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2911) Move global .hiverc file
[
https://issues.apache.org/jira/browse/HIVE-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242951#comment-13242951
]
Namit Jain commented on HIVE-2911:
--
This is a backward incompatible change.
Move global .hiverc file
Key: HIVE-2911
URL: https://issues.apache.org/jira/browse/HIVE-2911
Project: Hive
Issue Type: Improvement
Components: CLI
Reporter: Owen O'Malley
Assignee: Owen O'Malley
Attachments: HIVE-2911.D2529.1.patch, HIVE-2911.D2529.2.patch
Currently, the .hiverc files are loaded from:
{code}
$HIVE_HOME/bin/.hiverc
~/.hiverc
{code}
It seems more ops-friendly to have it in the config directory.
{code}
$HIVE_HOME/bin/.hiverc - for backwards compatibility
$HIVE_CONF_DIR/.hiverc
~/.hiverc
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1341 - Fixed
Changes for Build #1332 [hashutosh] HIVE-2797: Make the IP address of a Thrift client available to HMSHandler. (Kevin Wilfong via Ashutosh Chauhan) Changes for Build #1333 Changes for Build #1334 Changes for Build #1335 Changes for Build #1336 [hashutosh] HIVE-2891: TextConverter for UDF's is inefficient if the input object is already Text or Lazy (Cliff Engle via Ashutosh Chauhan) Changes for Build #1337 Changes for Build #1338 [namit] HIVE-2518 pull junit jar from maven repos via ivy (Kevin Wilfong via namit) [hashutosh] HIVE-2898: Add nicer helper functions for adding and reading metadata from RCFiles (Owen Omalley via Ashutosh Chauhan) Changes for Build #1339 Changes for Build #1340 Changes for Build #1341 [namit] HIVE-2875 /Users/njain/hive/hive_commit1/svn-commit.tmp (Kevin Wilfong via namit) All tests passed The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1341) Status: Fixed Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1341/ to view the results.
[jira] [Created] (HIVE-2916) Hive Type Information for Table Columns
Hive Type Information for Table Columns --- Key: HIVE-2916 URL: https://issues.apache.org/jira/browse/HIVE-2916 Project: Hive Issue Type: Bug Environment: Hive-0.8.1 Reporter: Indrani Gorti Priority: Blocker Hi I am using Hive-0.8.1. I am trying to get type information for table columns from MetaStore. The Developer documentation says the components of ql/typeinfo is responsible for this. i.e, under the query processor section in https://cwiki.apache.org/confluence/display/Hive/DeveloperGuide#DeveloperGuide-QueryProcessor However the directory typeinfo is missing in hive.I understand from typeinfo has java files that is responsible for getting all struct fieldnames, categories, primitive types. etc. Indrani -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2916) Hive Type Information for Table Columns
[ https://issues.apache.org/jira/browse/HIVE-2916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indrani Gorti updated HIVE-2916: Description: Hi I am using Hive-0.8.1. I am trying to get type information for table columns from MetaStore. The Developer documentation says the components of ql/typeinfo is responsible for this. i.e, under the query processor section in https://cwiki.apache.org/confluence/display/Hive/DeveloperGuide#DeveloperGuide-QueryProcessor However the directory typeinfo is missing in hadoop/hive/ql.I understand from typeinfo has java files that is responsible for getting all struct fieldnames, categories, primitive types. etc. Indrani was: Hi I am using Hive-0.8.1. I am trying to get type information for table columns from MetaStore. The Developer documentation says the components of ql/typeinfo is responsible for this. i.e, under the query processor section in https://cwiki.apache.org/confluence/display/Hive/DeveloperGuide#DeveloperGuide-QueryProcessor However the directory typeinfo is missing in hive.I understand from typeinfo has java files that is responsible for getting all struct fieldnames, categories, primitive types. etc. Indrani Hive Type Information for Table Columns --- Key: HIVE-2916 URL: https://issues.apache.org/jira/browse/HIVE-2916 Project: Hive Issue Type: Bug Environment: Hive-0.8.1 Reporter: Indrani Gorti Priority: Blocker Hi I am using Hive-0.8.1. I am trying to get type information for table columns from MetaStore. The Developer documentation says the components of ql/typeinfo is responsible for this. i.e, under the query processor section in https://cwiki.apache.org/confluence/display/Hive/DeveloperGuide#DeveloperGuide-QueryProcessor However the directory typeinfo is missing in hadoop/hive/ql.I understand from typeinfo has java files that is responsible for getting all struct fieldnames, categories, primitive types. etc. Indrani -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2907) Hive error when dropping a table with large number of partitions
[ https://issues.apache.org/jira/browse/HIVE-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2907: -- Attachment: HIVE-2907.D2505.6.patch mousom updated the revision HIVE-2907 [jira] Hive error when dropping a table with large number of partitions. Reviewers: JIRA, kevinwilfong Fixed drop_table_failure3.q REVISION DETAIL https://reviews.facebook.net/D2505 AFFECTED FILES conf/hive-default.xml.template metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ql/src/test/results/clientnegative/drop_table_failure3.q.out ql/src/test/results/clientpositive/drop_table2.q.out ql/src/test/queries/clientnegative/drop_table_failure3.q ql/src/test/queries/clientpositive/drop_table2.q ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java Hive error when dropping a table with large number of partitions Key: HIVE-2907 URL: https://issues.apache.org/jira/browse/HIVE-2907 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.9.0 Environment: General. Hive Metastore bug. Reporter: Mousom Dhar Gupta Assignee: Mousom Dhar Gupta Priority: Minor Fix For: 0.9.0 Attachments: HIVE-2907.1.patch.txt, HIVE-2907.D2505.1.patch, HIVE-2907.D2505.2.patch, HIVE-2907.D2505.3.patch, HIVE-2907.D2505.4.patch, HIVE-2907.D2505.5.patch, HIVE-2907.D2505.6.patch Original Estimate: 10h Remaining Estimate: 10h Running into an Out Of Memory error when trying to drop a table with 128K partitions. The methods dropTable in metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java and dropTable in ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java encounter out of memory errors when dropping tables with lots of partitions because they try to load the metadata for every partition into memory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2916) Hive Type Information for Table Columns
[ https://issues.apache.org/jira/browse/HIVE-2916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indrani Gorti resolved HIVE-2916. - Resolution: Fixed Fix Version/s: 0.8.0 0.8.1 Hi I just realized that there is a package org.apache.hadoop.hive.serde2.typeinfo that is imported when file q/typeinfo has to be used. The package has the following which is what /q/typeinfo directory contains * ListTypeInfo.java * MapTypeInfo.java * PrimitiveTypeInfo.java * StructTypeInfo.java * TypeInfo.java * TypeInfoFactory.java * TypeInfoUtils.java one can see this in http://hive.apache.org/docs/r0.7.1/api/org/apache/hadoop/hive/serde2/typeinfo/package-tree.html and https://svn.apache.org/repos/asf/hive/branches/branch-0.2/ql/src/java/org/apache/hadoop/hive/ql/typeinfo/ Indrani Gorti Hive Type Information for Table Columns --- Key: HIVE-2916 URL: https://issues.apache.org/jira/browse/HIVE-2916 Project: Hive Issue Type: Bug Environment: Hive-0.8.1 Reporter: Indrani Gorti Priority: Blocker Fix For: 0.8.1, 0.8.0 Hi I am using Hive-0.8.1. I am trying to get type information for table columns from MetaStore. The Developer documentation says the components of ql/typeinfo is responsible for this. i.e, under the query processor section in https://cwiki.apache.org/confluence/display/Hive/DeveloperGuide#DeveloperGuide-QueryProcessor However the directory typeinfo is missing in hadoop/hive/ql.I understand from typeinfo has java files that is responsible for getting all struct fieldnames, categories, primitive types. etc. Indrani -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2866) Cache local map reduce job errors for additional logging
[ https://issues.apache.org/jira/browse/HIVE-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13243019#comment-13243019 ] Namit Jain commented on HIVE-2866: -- +1 running tests Cache local map reduce job errors for additional logging Key: HIVE-2866 URL: https://issues.apache.org/jira/browse/HIVE-2866 Project: Hive Issue Type: Improvement Components: Logging Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2866.D2277.1.patch Using the CachingPrintStream we are storing errors that occur locally in Hive, but because local map reduce jobs are run in a separate JVM we are not storing that occur for these. We can use this same construct to store errors written to the subprocesses error stream. This way, when we log failed queries, these will give us a decent idea of why those queries failed. See related issues: https://issues.apache.org/jira/browse/HIVE-2832 https://issues.apache.org/jira/browse/HIVE-2858 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
