[jira] [Created] (HIVE-3633) sort-merge join does not work with sub-queries
Namit Jain created HIVE-3633: Summary: sort-merge join does not work with sub-queries Key: HIVE-3633 URL: https://issues.apache.org/jira/browse/HIVE-3633 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Consider the following query: create table smb_bucket_1(key int, value string) CLUSTERED BY (key) SORTED BY (key) INTO 6 BUCKETS STORED AS TEXTFILE; create table smb_bucket_2(key int, value string) CLUSTERED BY (key) SORTED BY (key) INTO 6 BUCKETS STORED AS TEXTFILE; -- load the above tables set hive.optimize.bucketmapjoin = true; set hive.optimize.bucketmapjoin.sortedmerge = true; set hive.input.format = org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat; explain select count(*) from ( select /*+mapjoin(a)*/ a.key as key1, b.key as key2, a.value as value1, b.value as value2 from smb_bucket_1 a join smb_bucket_2 b on a.key = b.key) subq; The above query does not use sort-merge join. This would be very useful as we automatically convert the queries to use sorting and bucketing properties for join. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3623) Reset operator-id before executing parse tests
[ https://issues.apache.org/jira/browse/HIVE-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3623: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed. Thanks Navis Reset operator-id before executing parse tests -- Key: HIVE-3623 URL: https://issues.apache.org/jira/browse/HIVE-3623 Project: Hive Issue Type: Improvement Components: Tests Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-3623.D6213.1.patch Parse test in hive-exec outputs XML file containing operator ids, which can be vary by previous query history. Resetting it 0 before each test could make test result independent to history. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3607) Set mapreduce.task.classpath.user.precedence to true by default
[ https://issues.apache.org/jira/browse/HIVE-3607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485856#comment-13485856 ] Namit Jain commented on HIVE-3607: -- This seems to be logical - hive jars should always come before hadoop server side jars. What is the reason for giving preference to hadoop server jars ? Is it the issue that the client can submit malicious hadoop jars - anyway, the client today can submit any jar and write a UDF which accesses those jars. If we want to be conservative, we can come up with a blacklist of hadoop jars which cannot be provided by hive. Set mapreduce.task.classpath.user.precedence to true by default --- Key: HIVE-3607 URL: https://issues.apache.org/jira/browse/HIVE-3607 Project: Hive Issue Type: Improvement Components: Configuration Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong When queries are actually run in a Hadoop cluster, Hive's jars are appended to Hadoop's classpath. However, when we test/run jobs locally Hive's classpath comes first. This leads to issues like the one brought up here after the patch was committed HIVE-3581 where a change depended on a jar Hive includes which conflicted with one provided by Hadoop which is an older version in 0.20 It's possible that more of the jars we include are getting preceded by older jars in Hadoop, and we haven't noticed yet. If we add Hive jars to the beginning of Hadoop's classpath we will be in control in such situations where the jars are backwards compatible. We will be able to update the jars in Hive and these will be used at run time, instead of just compile time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3606) get_json_object and json_tuple no longer support single quotes after upgrade to Jackson
[ https://issues.apache.org/jira/browse/HIVE-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3606: - Status: Open (was: Patch Available) This cannot go in until we have a resolution on HIVE-3607 get_json_object and json_tuple no longer support single quotes after upgrade to Jackson --- Key: HIVE-3606 URL: https://issues.apache.org/jira/browse/HIVE-3606 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3606.1.patch.txt After the upgrade to Jackson, JSON objects that contain keys or values surrounded by single quotes are not considered valid JSON, this was supported before the switch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3634) Utilize sorted traits of columns provided by some storage handler (hbase, etc.)
Navis created HIVE-3634: --- Summary: Utilize sorted traits of columns provided by some storage handler (hbase, etc.) Key: HIVE-3634 URL: https://issues.apache.org/jira/browse/HIVE-3634 Project: Hive Issue Type: Bug Reporter: Navis Priority: Trivial Some storage handler can provide rows sorted/bucketized on columns but this is not utilized by hive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3634) Utilize sorted traits of columns provided by some storage handler (hbase, etc.)
[ https://issues.apache.org/jira/browse/HIVE-3634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-3634: -- Attachment: HIVE-3634.D6279.1.patch navis requested code review of HIVE-3634 [jira] Utilize sorted traits of columns provided by some storage handler (hbase, etc.). Reviewers: JIRA DPAL-1927 Utilize sorted traits of columns provided by some storage handler (hbase, etc.) Some storage handler can provide rows sorted/bucketized on columns but this is not utilized by hive. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D6279 AFFECTED FILES hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java hbase-handler/src/test/queries/positive/hbase_mapside_groupby.q hbase-handler/src/test/results/positive/hbase_mapside_groupby.q.out ql/src/java/org/apache/hadoop/hive/ql/metadata/SortedBucketizedStorageHandler.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/GroupByOptimizer.java ql/src/java/org/apache/hadoop/hive/ql/plan/TableScanDesc.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/14919/ To: JIRA, navis Utilize sorted traits of columns provided by some storage handler (hbase, etc.) --- Key: HIVE-3634 URL: https://issues.apache.org/jira/browse/HIVE-3634 Project: Hive Issue Type: Bug Reporter: Navis Priority: Trivial Attachments: HIVE-3634.D6279.1.patch Some storage handler can provide rows sorted/bucketized on columns but this is not utilized by hive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3471) Implement grouping sets in hive
[ https://issues.apache.org/jira/browse/HIVE-3471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3471: - Status: Open (was: Patch Available) Comments on phabricator Implement grouping sets in hive --- Key: HIVE-3471 URL: https://issues.apache.org/jira/browse/HIVE-3471 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Namit Jain Assignee: Ivan Gorbachev Attachments: jira-3471.0.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3570) Add/fix facility to collect operator specific statisticsin hive + add hash-in/hash-out counter for GroupBy Optr
[ https://issues.apache.org/jira/browse/HIVE-3570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485871#comment-13485871 ] Phabricator commented on HIVE-3570: --- njain has commented on the revision HIVE-3570 [jira] Hive changes for Optr level stats. INLINE COMMENTS ql/src/test/org/apache/hadoop/hive/ql/hooks/OptrStatGroupByHook.java:1 add apache header ql/src/test/org/apache/hadoop/hive/ql/hooks/OptrStatGroupByHook.java:29 add some comments - what are you trying to do ? ql/src/test/org/apache/hadoop/hive/ql/hooks/OptrStatGroupByHook.java:49 fix lint ql/src/test/queries/clientpositive/OptrStatGroupBy.q:1 add some comments in the test. What are you trying to achieve ? ql/src/test/queries/clientpositive/OptrStatGroupBy.q:1 The test names start with a lower case letter - can you rename this test file ? ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java:997 lint ?? REVISION DETAIL https://reviews.facebook.net/D5985 To: njain, sambavim, kevinwilfong, satadru Cc: JIRA, adobriyal Add/fix facility to collect operator specific statisticsin hive + add hash-in/hash-out counter for GroupBy Optr --- Key: HIVE-3570 URL: https://issues.apache.org/jira/browse/HIVE-3570 Project: Hive Issue Type: Improvement Components: Statistics Affects Versions: 0.9.0 Reporter: Satadru Pan Assignee: Satadru Pan Priority: Minor Attachments: HIVE-3570.1.patch.txt, HIVE-3570.D5985.1.patch, HIVE-3570.D5985.2.patch, HIVE-3570.D5985.3.patch, HIVE-3570.D5985.4.patch, HIVE-3570.D5985.5.patch Requirement: Collect Operator specific stats for hive queries. Use the counter framework available in Hive Operator.java to accomplish that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
Alexander Alten-Lorenz created HIVE-3635: Summary: allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type Key: HIVE-3635 URL: https://issues.apache.org/jira/browse/HIVE-3635 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3635: - Status: Patch Available (was: Open) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type --- Key: HIVE-3635 URL: https://issues.apache.org/jira/browse/HIVE-3635 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3635: - Attachment: HIVE-3635.patch allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type --- Key: HIVE-3635 URL: https://issues.apache.org/jira/browse/HIVE-3635 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3635.patch interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7759/ --- Review request for hive. Description --- interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way This addresses bug HIVE-3635. https://issues.apache.org/jira/browse/HIVE-3635 Diffs - serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyBoolean.java c741c3a Diff: https://reviews.apache.org/r/7759/diff/ Testing --- Thanks, Alexander Alten-Lorenz
[jira] [Commented] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485874#comment-13485874 ] Alexander Alten-Lorenz commented on HIVE-3635: -- https://reviews.apache.org/r/7759/ allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type --- Key: HIVE-3635 URL: https://issues.apache.org/jira/browse/HIVE-3635 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3635.patch interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3570) Add/fix facility to collect operator specific statisticsin hive + add hash-in/hash-out counter for GroupBy Optr
[ https://issues.apache.org/jira/browse/HIVE-3570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3570: - Affects Version/s: (was: 0.9.0) 0.10.0 Status: Open (was: Patch Available) minor comments on phabricator Add/fix facility to collect operator specific statisticsin hive + add hash-in/hash-out counter for GroupBy Optr --- Key: HIVE-3570 URL: https://issues.apache.org/jira/browse/HIVE-3570 Project: Hive Issue Type: Improvement Components: Statistics Affects Versions: 0.10.0 Reporter: Satadru Pan Assignee: Satadru Pan Priority: Minor Attachments: HIVE-3570.1.patch.txt, HIVE-3570.D5985.1.patch, HIVE-3570.D5985.2.patch, HIVE-3570.D5985.3.patch, HIVE-3570.D5985.4.patch, HIVE-3570.D5985.5.patch Requirement: Collect Operator specific stats for hive queries. Use the counter framework available in Hive Operator.java to accomplish that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3636) Catch the NPe when using ^D to exit from CLI
[ https://issues.apache.org/jira/browse/HIVE-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3636: - Fix Version/s: 0.10.0 Affects Version/s: 0.9.0 Status: Patch Available (was: Open) Catch the NPe when using ^D to exit from CLI Key: HIVE-3636 URL: https://issues.apache.org/jira/browse/HIVE-3636 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3636) Catch the NPe when using ^D to exit from CLI
Alexander Alten-Lorenz created HIVE-3636: Summary: Catch the NPe when using ^D to exit from CLI Key: HIVE-3636 URL: https://issues.apache.org/jira/browse/HIVE-3636 Project: Hive Issue Type: Improvement Components: CLI Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3636) Catch the NPe when using ^D to exit from CLI
[ https://issues.apache.org/jira/browse/HIVE-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3636: - Attachment: HIVE-3636.patch Catch the NPe when using ^D to exit from CLI Key: HIVE-3636 URL: https://issues.apache.org/jira/browse/HIVE-3636 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3636.patch The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: Catch the NPe when using ^D to exit from CLI
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7760/ --- Review request for hive. Description --- The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace This addresses bug HIVE-3636. https://issues.apache.org/jira/browse/HIVE-3636 Diffs - cli/src/java/org/apache/hadoop/hive/cli/CliSessionState.java dfb30e2 Diff: https://reviews.apache.org/r/7760/diff/ Testing --- Thanks, Alexander Alten-Lorenz
[jira] [Commented] (HIVE-3636) Catch the NPe when using ^D to exit from CLI
[ https://issues.apache.org/jira/browse/HIVE-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485877#comment-13485877 ] Alexander Alten-Lorenz commented on HIVE-3636: -- https://reviews.apache.org/r/7760/ Catch the NPe when using ^D to exit from CLI Key: HIVE-3636 URL: https://issues.apache.org/jira/browse/HIVE-3636 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3636.patch The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485878#comment-13485878 ] Esteban Gutierrez commented on HIVE-3635: - It might be good to add a property to maintain consistency if the user doesn't require this behavior by default. Probably something like 'hive.extended.boolean = [false|true]' allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type --- Key: HIVE-3635 URL: https://issues.apache.org/jira/browse/HIVE-3635 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3635.patch interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3636) Catch the NPe when using ^D to exit from CLI
[ https://issues.apache.org/jira/browse/HIVE-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485894#comment-13485894 ] Harsh J commented on HIVE-3636: --- This is no longer a problem on Hive trunk/recent releases. Was resolved (in a different manner) quite a while ago after a CLI refactor I think. Catch the NPe when using ^D to exit from CLI Key: HIVE-3636 URL: https://issues.apache.org/jira/browse/HIVE-3636 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3636.patch The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3499) add tests to use bucketing metadata for partitions
[ https://issues.apache.org/jira/browse/HIVE-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3499: - Attachment: hive.3499.2.patch add tests to use bucketing metadata for partitions -- Key: HIVE-3499 URL: https://issues.apache.org/jira/browse/HIVE-3499 Project: Hive Issue Type: Test Components: Tests Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3499.1.patch, hive.3499.2.patch What happens if the bucketing metadata is different for different partitions ? I dont mean, different number of buckets, but what happens if the partitions are bucketed on different keys. Do we get wrong results ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3499) add tests to use bucketing metadata for partitions
[ https://issues.apache.org/jira/browse/HIVE-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3499: - Status: Patch Available (was: Open) refreshed add tests to use bucketing metadata for partitions -- Key: HIVE-3499 URL: https://issues.apache.org/jira/browse/HIVE-3499 Project: Hive Issue Type: Test Components: Tests Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3499.1.patch, hive.3499.2.patch What happens if the bucketing metadata is different for different partitions ? I dont mean, different number of buckets, but what happens if the partitions are bucketed on different keys. Do we get wrong results ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3529) Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table
[ https://issues.apache.org/jira/browse/HIVE-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485942#comment-13485942 ] Namit Jain commented on HIVE-3529: -- +1 Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table -- Key: HIVE-3529 URL: https://issues.apache.org/jira/browse/HIVE-3529 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3529.1.patch.txt If you have a partition with bucket/sort metadata set, then you alter the table to have different bucket/sort metadata, and insert overwrite the partition with hive.enforce.bucketing=true and/or hive.enforce.sorting=true, the partition data will be bucketed/sorted by the table's metadata, but the partition will have the same metadata. This could result in wrong results. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3276) optimize union sub-queries
[ https://issues.apache.org/jira/browse/HIVE-3276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3276: - Attachment: hive.3276.14.patch optimize union sub-queries -- Key: HIVE-3276 URL: https://issues.apache.org/jira/browse/HIVE-3276 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3276.10.patch, hive.3276.11.patch, hive.3276.12.patch, hive.3276.13.patch, hive.3276.14.patch, HIVE-3276.1.patch, hive.3276.2.patch, hive.3276.3.patch, hive.3276.4.patch, hive.3276.5.patch, hive.3276.6.patch, hive.3276.7.patch, hive.3276.8.patch, hive.3276.9.patch It might be a good idea to optimize simple union queries containing map-reduce jobs in at least one of the sub-qeuries. For eg: a query like: insert overwrite table T1 partition P1 select * from ( subq1 union all subq2 ) u; today creates 3 map-reduce jobs, one for subq1, another for subq2 and the final one for the union. It might be a good idea to optimize this. Instead of creating the union task, it might be simpler to create a move task (or something like a move task), where the outputs of the two sub-queries will be moved to the final directory. This can easily extend to more than 2 sub-queries in the union. This is very useful if there is a select * followed by filesink after the union. This can be independently useful, and also be used to optimize the skewed joins https://cwiki.apache.org/Hive/skewed-join-optimization.html. If there is a select, filter between the union and the filesink, the select and the filter can be moved before the union, and the follow-up job can still be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3276) optimize union sub-queries
[ https://issues.apache.org/jira/browse/HIVE-3276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3276: - Status: Patch Available (was: Open) addressed comments - added new tests for double/bigint conversion refreshed patch + test outputs optimize union sub-queries -- Key: HIVE-3276 URL: https://issues.apache.org/jira/browse/HIVE-3276 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3276.10.patch, hive.3276.11.patch, hive.3276.12.patch, hive.3276.13.patch, hive.3276.14.patch, HIVE-3276.1.patch, hive.3276.2.patch, hive.3276.3.patch, hive.3276.4.patch, hive.3276.5.patch, hive.3276.6.patch, hive.3276.7.patch, hive.3276.8.patch, hive.3276.9.patch It might be a good idea to optimize simple union queries containing map-reduce jobs in at least one of the sub-qeuries. For eg: a query like: insert overwrite table T1 partition P1 select * from ( subq1 union all subq2 ) u; today creates 3 map-reduce jobs, one for subq1, another for subq2 and the final one for the union. It might be a good idea to optimize this. Instead of creating the union task, it might be simpler to create a move task (or something like a move task), where the outputs of the two sub-queries will be moved to the final directory. This can easily extend to more than 2 sub-queries in the union. This is very useful if there is a select * followed by filesink after the union. This can be independently useful, and also be used to optimize the skewed joins https://cwiki.apache.org/Hive/skewed-join-optimization.html. If there is a select, filter between the union and the filesink, the select and the filter can be moved before the union, and the follow-up job can still be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-933) Infer bucketing/sorting properties
[ https://issues.apache.org/jira/browse/HIVE-933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485966#comment-13485966 ] Namit Jain commented on HIVE-933: - For bucketing/sorting properties, partition based metadata is used now. So, inferring these properties should help in better plans for downstream queries. Infer bucketing/sorting properties -- Key: HIVE-933 URL: https://issues.apache.org/jira/browse/HIVE-933 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Namit Jain Assignee: Kevin Wilfong This is a long-term plan, and may require major changes. From the query, we can figure out the sorting/bucketing properties, and change the metadata of the destination at that time. However, this means that different partitions may have different metadata. Currently, the query plan is same for all the partitions of the table - we can do the following: 1. In the first cut, have a simple approach where you take the union all metadata, and create the most defensive plan. 2. Enhance mapredWork() to include partition specific operator trees. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3628) Provide a way to use counters in Hive through UDF
[ https://issues.apache.org/jira/browse/HIVE-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485967#comment-13485967 ] Viji commented on HIVE-3628: Hive UDFs do not allow a developer to use the Hadoop Reporter object to generate more Hadoop job counters of their own. Provide a way to use counters in Hive through UDF - Key: HIVE-3628 URL: https://issues.apache.org/jira/browse/HIVE-3628 Project: Hive Issue Type: Improvement Components: UDF Affects Versions: 0.7.0 Reporter: Viji Priority: Minor Currently it is not possible to generate counters through UDF. We should support this. Pig currently allows this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3623) Reset operator-id before executing parse tests
[ https://issues.apache.org/jira/browse/HIVE-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486050#comment-13486050 ] Hudson commented on HIVE-3623: -- Integrated in Hive-trunk-h0.21 #1762 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1762/]) HIVE-3623 Reset operator-id before executing parse tests (Navis via namit) (Revision 1403176) Result = FAILURE namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1403176 Files : * /hive/trunk/ql/src/test/results/compiler/plan/cast1.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/groupby1.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/groupby2.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/groupby3.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/groupby4.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/groupby5.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/groupby6.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input1.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input2.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input20.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input3.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input5.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input6.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input7.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input8.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input9.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input_part1.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input_testsequencefile.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input_testxpath.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input_testxpath2.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/join4.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/join5.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/join6.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/join7.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/subq.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/udf1.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/udf4.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/udf6.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/udf_case.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/udf_when.q.xml * /hive/trunk/ql/src/test/templates/TestParse.vm Reset operator-id before executing parse tests -- Key: HIVE-3623 URL: https://issues.apache.org/jira/browse/HIVE-3623 Project: Hive Issue Type: Improvement Components: Tests Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-3623.D6213.1.patch Parse test in hive-exec outputs XML file containing operator ids, which can be vary by previous query history. Resetting it 0 before each test could make test result independent to history. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1762 - Still Failing
Changes for Build #1747 Changes for Build #1748 [namit] HIVE-3544 union involving double column with a map join subquery will fail or give wrong results (Kevin Wilfong via namit) [cws] HIVE-3590. TCP KeepAlive and connection timeout for the HiveServer (Esteban Gutierrez via cws) Changes for Build #1749 Changes for Build #1750 [ecapriolo] HIVE-3599 missing return of compression codec to pool (Owen O'Malley via egc) Changes for Build #1751 Changes for Build #1752 Changes for Build #1753 Changes for Build #1754 Changes for Build #1755 [kevinwilfong] HIVE-3152. Disallow certain character patterns in partition names. (Ivan Gorbachev via kevinwilfong) Changes for Build #1756 [hashutosh] HIVE-3519 : partition to directory comparison in CombineHiveInputFormat needs to accept partitions dir without scheme (Thejas Nair via Ashutosh Chauhan) Changes for Build #1757 Changes for Build #1758 [kevinwilfong] HIVE-3433. Implement CUBE and ROLLUP operators in Hive. (Ivan Gorbachev and Namit Jain via kevinwilfong) Changes for Build #1759 Changes for Build #1760 Changes for Build #1761 Changes for Build #1762 [namit] HIVE-3623 Reset operator-id before executing parse tests (Navis via namit) 5 tests failed. REGRESSION: org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisherOneStat Error Message: null Stack Trace: junit.framework.AssertionFailedError: null at junit.framework.Assert.fail(Assert.java:47) at junit.framework.Assert.assertTrue(Assert.java:20) at junit.framework.Assert.assertTrue(Assert.java:27) at org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisherOneStat(TestStatsPublisherEnhanced.java:81) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:232) at junit.framework.TestSuite.run(TestSuite.java:227) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) REGRESSION: org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisher Error Message: null Stack Trace: junit.framework.AssertionFailedError: null at junit.framework.Assert.fail(Assert.java:47) at junit.framework.Assert.assertTrue(Assert.java:20) at junit.framework.Assert.assertTrue(Assert.java:27) at org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisher(TestStatsPublisherEnhanced.java:129) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:232) at junit.framework.TestSuite.run(TestSuite.java:227) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) REGRESSION: org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisherMultipleUpdates Error Message: null Stack Trace: junit.framework.AssertionFailedError: null at junit.framework.Assert.fail(Assert.java:47) at junit.framework.Assert.assertTrue(Assert.java:20) at junit.framework.Assert.assertTrue(Assert.java:27) at org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisherMultipleUpdates(TestStatsPublisherEnhanced.java:190)
[jira] [Updated] (HIVE-3529) Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table
[ https://issues.apache.org/jira/browse/HIVE-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3529: - Resolution: Fixed Fix Version/s: 0.10.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed. Thanks Kevin Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table -- Key: HIVE-3529 URL: https://issues.apache.org/jira/browse/HIVE-3529 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Fix For: 0.10.0 Attachments: HIVE-3529.1.patch.txt If you have a partition with bucket/sort metadata set, then you alter the table to have different bucket/sort metadata, and insert overwrite the partition with hive.enforce.bucketing=true and/or hive.enforce.sorting=true, the partition data will be bucketed/sorted by the table's metadata, but the partition will have the same metadata. This could result in wrong results. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #182
See https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/182/ -- [...truncated 5055 lines...] A ql/src/gen/thrift/gen-py/queryplan/ttypes.py A ql/src/gen/thrift/gen-py/queryplan/constants.py A ql/src/gen/thrift/gen-py/queryplan/__init__.py A ql/src/gen/thrift/gen-cpp A ql/src/gen/thrift/gen-cpp/queryplan_constants.h A ql/src/gen/thrift/gen-cpp/queryplan_types.cpp A ql/src/gen/thrift/gen-cpp/queryplan_types.h A ql/src/gen/thrift/gen-cpp/queryplan_constants.cpp A ql/src/gen/thrift/gen-rb A ql/src/gen/thrift/gen-rb/queryplan_types.rb A ql/src/gen/thrift/gen-rb/queryplan_constants.rb A ql/src/gen/thrift/gen-javabean A ql/src/gen/thrift/gen-javabean/org A ql/src/gen/thrift/gen-javabean/org/apache A ql/src/gen/thrift/gen-javabean/org/apache/hadoop A ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive A ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql A ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan A ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api A ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/QueryPlan.java A ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Adjacency.java A ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Graph.java A ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Task.java A ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/AdjacencyType.java A ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Stage.java A ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/TaskType.java A ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Query.java A ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/StageType.java A ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/NodeType.java A ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Operator.java A ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/OperatorType.java A ql/src/gen/thrift/gen-php A ql/src/gen/thrift/gen-php/queryplan A ql/src/gen/thrift/gen-php/queryplan/queryplan_types.php A ql/src/gen-javabean A ql/src/gen-javabean/org A ql/src/gen-javabean/org/apache A ql/src/gen-javabean/org/apache/hadoop A ql/src/gen-javabean/org/apache/hadoop/hive A ql/src/gen-javabean/org/apache/hadoop/hive/ql A ql/src/gen-javabean/org/apache/hadoop/hive/ql/plan A ql/src/gen-javabean/org/apache/hadoop/hive/ql/plan/api A ql/src/gen-php A ql/build.xml A ql/if A ql/if/queryplan.thrift A pdk A pdk/ivy.xml A pdk/scripts A pdk/scripts/class-registration.xsl A pdk/scripts/build-plugin.xml A pdk/scripts/README A pdk/src A pdk/src/java A pdk/src/java/org A pdk/src/java/org/apache A pdk/src/java/org/apache/hive A pdk/src/java/org/apache/hive/pdk A pdk/src/java/org/apache/hive/pdk/FunctionExtractor.java A pdk/src/java/org/apache/hive/pdk/HivePdkUnitTest.java A pdk/src/java/org/apache/hive/pdk/HivePdkUnitTests.java A pdk/src/java/org/apache/hive/pdk/PluginTest.java A pdk/test-plugin A pdk/test-plugin/test A pdk/test-plugin/test/cleanup.sql A pdk/test-plugin/test/onerow.txt A pdk/test-plugin/test/setup.sql A pdk/test-plugin/src A pdk/test-plugin/src/org A pdk/test-plugin/src/org/apache A pdk/test-plugin/src/org/apache/hive A pdk/test-plugin/src/org/apache/hive/pdktest A pdk/test-plugin/src/org/apache/hive/pdktest/Rot13.java A pdk/test-plugin/build.xml A pdk/build.xml A build-offline.xml U. At revision 1403392 no change for http://svn.apache.org/repos/asf/hive/branches/branch-0.9 since the previous build [hive] $ /home/hudson/tools/ant/apache-ant-1.8.1/bin/ant -Dversion=0.9.1-SNAPSHOT very-clean tar binary Buildfile: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/build.xml ivy-init-dirs: [echo] Project: hive [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/182/artifact/hive/build/ivy [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/182/artifact/hive/build/ivy/lib [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/182/artifact/hive/build/ivy/report [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/182/artifact/hive/build/ivy/maven
[jira] [Commented] (HIVE-3610) Add a command Explain dependency ...
[
https://issues.apache.org/jira/browse/HIVE-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486123#comment-13486123
]
Sambavi Muthukrishnan commented on HIVE-3610:
-
Namit: the set of inputs should be available through hooks. But this seems
like a generically useful feature - to be able to determine the dependencies
for a given query, which are the sets of inputs that are processed, hence
decided to make it a command that is available in HIVE.
Add a command Explain dependency ...
--
Key: HIVE-3610
URL: https://issues.apache.org/jira/browse/HIVE-3610
Project: Hive
Issue Type: New Feature
Components: Query Processor
Affects Versions: 0.9.0
Reporter: Sambavi Muthukrishnan
Assignee: Sambavi Muthukrishnan
Priority: Minor
Attachments: explain_dependency.1.patch, explain_dependency.2.patch
Add a new command EXPLAIN DEPENDENCY.
Any query can be passed to EXPLAIN DEPENDENCY as with EXPLAIN
(FORMATTED/EXTENDED). The output of this command will be JSON that provides
the list of tables and partitions that the query depends on.
One possible use case is to determine the set of tables/views that are used
by a view, and the set of partitions that are used by a given query on that
view. This will allow a view to be replicated from one Hive instance to
another, since we can determine the set of objects that need to be replicated
for replication of the view to be successful.
Example output:
{input_tables:[{tablename: default@test_sambavi_v2, tabletype:
EXTERNAL_TABLE}, {tablename: default@test_sambavi_v1, tabletype:
TABLE}], input
partitions:[default@srcpart@ds=2008-04-08/hr=11,default@srcpart@ds=2008-04-08/hr=12]}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3621) Make prompt in Hive CLI configurable
[ https://issues.apache.org/jira/browse/HIVE-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingwei Lu updated HIVE-3621: - Assignee: Jingwei Lu Make prompt in Hive CLI configurable Key: HIVE-3621 URL: https://issues.apache.org/jira/browse/HIVE-3621 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Jingwei Lu Assignee: Jingwei Lu Priority: Minor Fix For: 0.10.0 Original Estimate: 48h Remaining Estimate: 48h Right now the Hive CLI prompt just says hive, for users (primarily power users) who run in different clusters it can be easy to forget which cluster your Hive CLI is pointing to. If we change the Hive CLI prompt to be something like hive(silver) it would be much clearer. We could potentially extend this to namespaces as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1362) column level statistics
[ https://issues.apache.org/jira/browse/HIVE-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486223#comment-13486223 ] Carl Steinbach commented on HIVE-1362: -- +1. Will commit if tests pass. @Namit: Let me know if you need more time to review this. If so I'll hold off until you're ready. Thanks. column level statistics --- Key: HIVE-1362 URL: https://issues.apache.org/jira/browse/HIVE-1362 Project: Hive Issue Type: Sub-task Components: Statistics Reporter: Ning Zhang Assignee: Shreepadma Venugopalan Attachments: HIVE-1362.1.patch.txt, HIVE-1362.2.patch.txt, HIVE-1362.3.patch.txt, HIVE-1362.4.patch.txt, HIVE-1362.5.patch.txt, HIVE-1362-gen_thrift.1.patch.txt, HIVE-1362-gen_thrift.2.patch.txt, HIVE-1362-gen_thrift.3.patch.txt, HIVE-1362-gen_thrift.4.patch.txt, HIVE-1362-gen_thrift.5.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3610) Add a command Explain dependency ...
[
https://issues.apache.org/jira/browse/HIVE-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Wilfong updated HIVE-3610:
Resolution: Fixed
Fix Version/s: 0.10.0
Release Note:
This introduces a new command
EXPLAIN DEPENDENCY query
which outputs a JSON object containing the input tables and partitions the
query will access without actually running the query.
Status: Resolved (was: Patch Available)
Committed, thanks Sambavi.
Add a command Explain dependency ...
--
Key: HIVE-3610
URL: https://issues.apache.org/jira/browse/HIVE-3610
Project: Hive
Issue Type: New Feature
Components: Query Processor
Affects Versions: 0.9.0
Reporter: Sambavi Muthukrishnan
Assignee: Sambavi Muthukrishnan
Priority: Minor
Fix For: 0.10.0
Attachments: explain_dependency.1.patch, explain_dependency.2.patch
Add a new command EXPLAIN DEPENDENCY.
Any query can be passed to EXPLAIN DEPENDENCY as with EXPLAIN
(FORMATTED/EXTENDED). The output of this command will be JSON that provides
the list of tables and partitions that the query depends on.
One possible use case is to determine the set of tables/views that are used
by a view, and the set of partitions that are used by a given query on that
view. This will allow a view to be replicated from one Hive instance to
another, since we can determine the set of objects that need to be replicated
for replication of the view to be successful.
Example output:
{input_tables:[{tablename: default@test_sambavi_v2, tabletype:
EXTERNAL_TABLE}, {tablename: default@test_sambavi_v1, tabletype:
TABLE}], input
partitions:[default@srcpart@ds=2008-04-08/hr=11,default@srcpart@ds=2008-04-08/hr=12]}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3637) groupby_neg_float.q fails with Hadoop23
Chris Drome created HIVE-3637: - Summary: groupby_neg_float.q fails with Hadoop23 Key: HIVE-3637 URL: https://issues.apache.org/jira/browse/HIVE-3637 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.10.0, 0.9.1 Reporter: Chris Drome Hive creates an empty file as a hack to get Hadoop to run a mapper. This no longer works with Hadoop23, causing this test to fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3623) Reset operator-id before executing parse tests
[ https://issues.apache.org/jira/browse/HIVE-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-3623: --- Fix Version/s: 0.10.0 Reset operator-id before executing parse tests -- Key: HIVE-3623 URL: https://issues.apache.org/jira/browse/HIVE-3623 Project: Hive Issue Type: Improvement Components: Tests Reporter: Navis Assignee: Navis Priority: Trivial Fix For: 0.10.0 Attachments: HIVE-3623.D6213.1.patch Parse test in hive-exec outputs XML file containing operator ids, which can be vary by previous query history. Resetting it 0 before each test could make test result independent to history. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3614) TestParse_Union is failing on trunk
[ https://issues.apache.org/jira/browse/HIVE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486404#comment-13486404 ] Ashutosh Chauhan commented on HIVE-3614: [~navis] Now that HIVE-3623 is fixed, I believe this test will succeed. If so, can you resolve this as fixed. TestParse_Union is failing on trunk --- Key: HIVE-3614 URL: https://issues.apache.org/jira/browse/HIVE-3614 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Ashutosh Chauhan Test is generating output different then expected and thus failing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21 #182
See https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/182/ -- [...truncated 36564 lines...] [junit] POSTHOOK: query: select count(1) as cnt from testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/hudson/hive_2012-10-29_14-53-33_335_5184603201371397244/-mr-1 [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: default@testhivedrivertable [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Hive history file=https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/182/artifact/hive/build/service/tmp/hive_job_log_hudson_201210291453_1831127891.txt [junit] Copying file: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/data/files/kv1.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: load data local inpath 'https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Output: default@testhivedrivertable [junit] Copying data from https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/data/files/kv1.txt [junit] Loading data to table default.testhivedrivertable [junit] POSTHOOK: query: load data local inpath 'https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: select * from testhivedrivertable limit 10 [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: file:/tmp/hudson/hive_2012-10-29_14-53-38_134_2141645953625203041/-mr-1 [junit] POSTHOOK: query: select * from testhivedrivertable limit 10 [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/hudson/hive_2012-10-29_14-53-38_134_2141645953625203041/-mr-1 [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: default@testhivedrivertable [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Hive history file=https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/182/artifact/hive/build/service/tmp/hive_job_log_hudson_201210291453_538157493.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: default@testhivedrivertable [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Hive history file=https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/182/artifact/hive/build/service/tmp/hive_job_log_hudson_201210291453_1787631488.txt [junit] Hive history file=https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/182/artifact/hive/build/service/tmp/hive_job_log_hudson_201210291453_603809046.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type:
[jira] [Updated] (HIVE-3519) partition to directory comparison in CombineHiveInputFormat needs to accept partitions dir without scheme
[
https://issues.apache.org/jira/browse/HIVE-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-3519:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
partition to directory comparison in CombineHiveInputFormat needs to accept
partitions dir without scheme
-
Key: HIVE-3519
URL: https://issues.apache.org/jira/browse/HIVE-3519
Project: Hive
Issue Type: Bug
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Fix For: 0.10.0
Attachments: HIVE-3519.1.patch, HIVE-3519.2.patch
TestSymlinkTextInputFormat.testCombine throws following exception. The test
case is just printing out the stacktrace when that happens instead of failing.
{code}
java.io.IOException: cannot find dir =
file:/Users/thejas/hive-trunk/ql/TestSymlinkTextInputFormat/datadir1/combinefile1_1
in pathToPartitionInfo:
[/Users/thejas/hive-trunk/ql/TestSymlinkTextInputFormat/datadir2/combinefile2_1,
/Users/thejas/hive-trunk/ql/TestSymlinkTextInputFormat/datadir1/combinefile1_1]
at
org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getPartitionDescFromPathRecursively(HiveFileFormatUtils.java:288)
at
org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getPartitionDescFromPathRecursively(HiveFileFormatUtils.java:256)
at
org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:289)
at
org.apache.hadoop.hive.ql.io.TestSymlinkTextInputFormat.testCombine(TestSymlinkTextInputFormat.java:186)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:232)
at junit.framework.TestSuite.run(TestSuite.java:227)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:520)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1060)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:911)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3441) testcases escape1,escape2 fail on windows
[ https://issues.apache.org/jira/browse/HIVE-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486412#comment-13486412 ] Ashutosh Chauhan commented on HIVE-3441: +1 will commit if tests pass. testcases escape1,escape2 fail on windows - Key: HIVE-3441 URL: https://issues.apache.org/jira/browse/HIVE-3441 Project: Hive Issue Type: Bug Reporter: Thejas M Nair Assignee: Thejas M Nair Fix For: 0.10.0 Attachments: HIVE-3441.1.patch, HIVE-3441.2.patch, HIVE-3441.3.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3529) Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table
[ https://issues.apache.org/jira/browse/HIVE-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486502#comment-13486502 ] Hudson commented on HIVE-3529: -- Integrated in Hive-trunk-h0.21 #1763 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1763/]) HIVE-3529 Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table (Kevin Wilfong via namit) (Revision 1403363) Result = SUCCESS namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1403363 Files : * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java * /hive/trunk/ql/src/test/queries/clientpositive/alter_numbuckets_partitioned_table2.q * /hive/trunk/ql/src/test/queries/clientpositive/alter_table_serde2.q * /hive/trunk/ql/src/test/results/clientpositive/alter_numbuckets_partitioned_table.q.out * /hive/trunk/ql/src/test/results/clientpositive/alter_numbuckets_partitioned_table2.q.out * /hive/trunk/ql/src/test/results/clientpositive/alter_table_serde2.q.out Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table -- Key: HIVE-3529 URL: https://issues.apache.org/jira/browse/HIVE-3529 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Fix For: 0.10.0 Attachments: HIVE-3529.1.patch.txt If you have a partition with bucket/sort metadata set, then you alter the table to have different bucket/sort metadata, and insert overwrite the partition with hive.enforce.bucketing=true and/or hive.enforce.sorting=true, the partition data will be bucketed/sorted by the table's metadata, but the partition will have the same metadata. This could result in wrong results. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1763 - Fixed
Changes for Build #1747 Changes for Build #1748 [namit] HIVE-3544 union involving double column with a map join subquery will fail or give wrong results (Kevin Wilfong via namit) [cws] HIVE-3590. TCP KeepAlive and connection timeout for the HiveServer (Esteban Gutierrez via cws) Changes for Build #1749 Changes for Build #1750 [ecapriolo] HIVE-3599 missing return of compression codec to pool (Owen O'Malley via egc) Changes for Build #1751 Changes for Build #1752 Changes for Build #1753 Changes for Build #1754 Changes for Build #1755 [kevinwilfong] HIVE-3152. Disallow certain character patterns in partition names. (Ivan Gorbachev via kevinwilfong) Changes for Build #1756 [hashutosh] HIVE-3519 : partition to directory comparison in CombineHiveInputFormat needs to accept partitions dir without scheme (Thejas Nair via Ashutosh Chauhan) Changes for Build #1757 Changes for Build #1758 [kevinwilfong] HIVE-3433. Implement CUBE and ROLLUP operators in Hive. (Ivan Gorbachev and Namit Jain via kevinwilfong) Changes for Build #1759 Changes for Build #1760 Changes for Build #1761 Changes for Build #1762 [namit] HIVE-3623 Reset operator-id before executing parse tests (Navis via namit) Changes for Build #1763 [namit] HIVE-3529 Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table (Kevin Wilfong via namit) All tests passed The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1763) Status: Fixed Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1763/ to view the results.
[jira] [Commented] (HIVE-3637) groupby_neg_float.q fails with Hadoop23
[ https://issues.apache.org/jira/browse/HIVE-3637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486531#comment-13486531 ] Shreepadma Venugopalan commented on HIVE-3637: -- @Chris: Can you update the JIRA with the failure you see on Hadoop23? Thanks. groupby_neg_float.q fails with Hadoop23 --- Key: HIVE-3637 URL: https://issues.apache.org/jira/browse/HIVE-3637 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.10.0, 0.9.1 Reporter: Chris Drome Hive creates an empty file as a hack to get Hadoop to run a mapper. This no longer works with Hadoop23, causing this test to fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3276) optimize union sub-queries
[ https://issues.apache.org/jira/browse/HIVE-3276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486554#comment-13486554 ] Kevin Wilfong commented on HIVE-3276: - +1 optimize union sub-queries -- Key: HIVE-3276 URL: https://issues.apache.org/jira/browse/HIVE-3276 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3276.10.patch, hive.3276.11.patch, hive.3276.12.patch, hive.3276.13.patch, hive.3276.14.patch, HIVE-3276.1.patch, hive.3276.2.patch, hive.3276.3.patch, hive.3276.4.patch, hive.3276.5.patch, hive.3276.6.patch, hive.3276.7.patch, hive.3276.8.patch, hive.3276.9.patch It might be a good idea to optimize simple union queries containing map-reduce jobs in at least one of the sub-qeuries. For eg: a query like: insert overwrite table T1 partition P1 select * from ( subq1 union all subq2 ) u; today creates 3 map-reduce jobs, one for subq1, another for subq2 and the final one for the union. It might be a good idea to optimize this. Instead of creating the union task, it might be simpler to create a move task (or something like a move task), where the outputs of the two sub-queries will be moved to the final directory. This can easily extend to more than 2 sub-queries in the union. This is very useful if there is a select * followed by filesink after the union. This can be independently useful, and also be used to optimize the skewed joins https://cwiki.apache.org/Hive/skewed-join-optimization.html. If there is a select, filter between the union and the filesink, the select and the filter can be moved before the union, and the follow-up job can still be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3638) metadataonly1.q test fails with Hadoop23
[ https://issues.apache.org/jira/browse/HIVE-3638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486555#comment-13486555 ] Shreepadma Venugopalan commented on HIVE-3638: -- @Chris: Can you update the JIRA with the change in behavior? In the case of this particular test, Hive deviates from the SQL standard and outputs an empty result set instead of a NULL result set when an aggregate function is run on an empty table/partition. I believe MR2 generates splits even though the input directory in empty and Hive in turn outputs an empty result set. The behavior on MR1 used to be the same as MR2, until a recent change that went into HDFS. It would really help to see the failure you are seeing on hadoop23. metadataonly1.q test fails with Hadoop23 Key: HIVE-3638 URL: https://issues.apache.org/jira/browse/HIVE-3638 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.10.0, 0.9.1 Reporter: Chris Drome Hive creates an empty file as a hack to get Hadoop to run a mapper. This no longer works with Hadoop23, causing this test to fail. Note that this tests empty partitions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486563#comment-13486563 ] Shreepadma Venugopalan commented on HIVE-3635: -- The changes to LazyBoolean.java look good. +1 to what Esteban said. It would be good to have a way to turn this off. allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type --- Key: HIVE-3635 URL: https://issues.apache.org/jira/browse/HIVE-3635 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3635.patch interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3636) Catch the NPe when using ^D to exit from CLI
[ https://issues.apache.org/jira/browse/HIVE-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486564#comment-13486564 ] Shreepadma Venugopalan commented on HIVE-3636: -- Yup, this is not a problem on the latest trunk. Catch the NPe when using ^D to exit from CLI Key: HIVE-3636 URL: https://issues.apache.org/jira/browse/HIVE-3636 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3636.patch The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-1362: Support for column statistics in Hive
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/6878/ --- (Updated Oct. 30, 2012, 1:24 a.m.) Review request for hive and Carl Steinbach. Description --- This patch implements version 1 of the column statistics project in Hive. It adds support for computing and persisting statistical summary of column values in Hive Tables and Partitions. In order to support column statistics in Hive, this patch does the following, * Adds a new compute stats UDAF to compute scalar statistics for all primitive Hive data types. In version 1 of the project, we support the following scalar statistics on primitive types - estimate of number of distinct values, number of null values, number of trues/falses for boolean typed columsn, max and avg length for string and binary typed columns, max and min value for long and double typed columns. Note that version 1 of the column stats project includes support for column statistics both at the table and partition level. * Adds Metastore schema tables to persist the newly added statistics both at table and partition level. * Adds Metastore Thrift API to persist, retrieve and delete column statistics at both table and partition level. Please refer to the following wiki link for the details of the schema and the Thrift API changes - https://cwiki.apache.org/confluence/display/Hive/Column+Statistics+in+Hive * Extends the analyze table compute statistics statement to trigger statistics computation and persistence for one or more columns. Please note that statistics for multiple columns is computed through a single scan of the table data. Please refer to the following wiki link for the syntax changes - https://cwiki.apache.org/confluence/display/Hive/Column+Statistics+in+Hive One thing missing from the patch at this point is the metastore upgrade scrips for MySQL/Derby/Postgres/Oracle. I'm waiting for the review to finalize the metastore schema changes before I go ahead and add the upgrade scripts. In a follow on patch, as part of version 2 of the column statistics project, we will add support for computing, persisting and retrieving histograms on long and double typed column values. Generated Thrift files have been removed for viewing pleasure. JIRA page has the patch with the generated Thrift files. This addresses bug HIVE-1362. https://issues.apache.org/jira/browse/HIVE-1362 Diffs (updated) - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 211f474 conf/hive-default.xml.template 93a86ec data/files/UserVisits.dat PRE-CREATION data/files/binary.txt PRE-CREATION data/files/bool.txt PRE-CREATION data/files/double.txt PRE-CREATION data/files/employee.dat PRE-CREATION data/files/employee2.dat PRE-CREATION data/files/int.txt PRE-CREATION metastore/if/hive_metastore.thrift d4fad72 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 915a5cf metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 17b986c metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 3883b5b metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java a49aecd metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java bf5ae3a metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java 77d1caa metastore/src/model/org/apache/hadoop/hive/metastore/model/MPartitionColumnStatistics.java PRE-CREATION metastore/src/model/org/apache/hadoop/hive/metastore/model/MTableColumnStatistics.java PRE-CREATION metastore/src/model/package.jdo 38ce6d5 metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java 528a100 metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 925938d ql/if/queryplan.thrift 05fbf58 ql/ivy.xml 2c4410a ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsTask.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 425900d ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java 4c8831f ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java 4446952 ql/src/java/org/apache/hadoop/hive/ql/exec/TaskFactory.java 79b87f1 ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java de9fc04 ql/src/java/org/apache/hadoop/hive/ql/optimizer/index/RewriteParseContextGenerator.java 0b55ac4 ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java c9e356a ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 5fc6a4f ql/src/java/org/apache/hadoop/hive/ql/parse/ExportSemanticAnalyzer.java 61bc7fd ql/src/java/org/apache/hadoop/hive/ql/parse/FunctionSemanticAnalyzer.java 6024dd4 ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 65f748c ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java 09ef969
[jira] [Updated] (HIVE-1362) column level statistics
[ https://issues.apache.org/jira/browse/HIVE-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shreepadma Venugopalan updated HIVE-1362: - Attachment: HIVE-1362.6.patch.txt column level statistics --- Key: HIVE-1362 URL: https://issues.apache.org/jira/browse/HIVE-1362 Project: Hive Issue Type: Sub-task Components: Statistics Reporter: Ning Zhang Assignee: Shreepadma Venugopalan Attachments: HIVE-1362.1.patch.txt, HIVE-1362.2.patch.txt, HIVE-1362.3.patch.txt, HIVE-1362.4.patch.txt, HIVE-1362.5.patch.txt, HIVE-1362.6.patch.txt, HIVE-1362-gen_thrift.1.patch.txt, HIVE-1362-gen_thrift.2.patch.txt, HIVE-1362-gen_thrift.3.patch.txt, HIVE-1362-gen_thrift.4.patch.txt, HIVE-1362-gen_thrift.5.patch.txt, HIVE-1362-gen_thrift.6.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1362) column level statistics
[ https://issues.apache.org/jira/browse/HIVE-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486586#comment-13486586 ] Shreepadma Venugopalan commented on HIVE-1362: -- Attached new patch rebased off the tip of current trunk. column level statistics --- Key: HIVE-1362 URL: https://issues.apache.org/jira/browse/HIVE-1362 Project: Hive Issue Type: Sub-task Components: Statistics Reporter: Ning Zhang Assignee: Shreepadma Venugopalan Attachments: HIVE-1362.1.patch.txt, HIVE-1362.2.patch.txt, HIVE-1362.3.patch.txt, HIVE-1362.4.patch.txt, HIVE-1362.5.patch.txt, HIVE-1362.6.patch.txt, HIVE-1362-gen_thrift.1.patch.txt, HIVE-1362-gen_thrift.2.patch.txt, HIVE-1362-gen_thrift.3.patch.txt, HIVE-1362-gen_thrift.4.patch.txt, HIVE-1362-gen_thrift.5.patch.txt, HIVE-1362-gen_thrift.6.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1362) column level statistics
[ https://issues.apache.org/jira/browse/HIVE-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shreepadma Venugopalan updated HIVE-1362: - Attachment: HIVE-1362-gen_thrift.6.patch.txt column level statistics --- Key: HIVE-1362 URL: https://issues.apache.org/jira/browse/HIVE-1362 Project: Hive Issue Type: Sub-task Components: Statistics Reporter: Ning Zhang Assignee: Shreepadma Venugopalan Attachments: HIVE-1362.1.patch.txt, HIVE-1362.2.patch.txt, HIVE-1362.3.patch.txt, HIVE-1362.4.patch.txt, HIVE-1362.5.patch.txt, HIVE-1362.6.patch.txt, HIVE-1362-gen_thrift.1.patch.txt, HIVE-1362-gen_thrift.2.patch.txt, HIVE-1362-gen_thrift.3.patch.txt, HIVE-1362-gen_thrift.4.patch.txt, HIVE-1362-gen_thrift.5.patch.txt, HIVE-1362-gen_thrift.6.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3638) metadataonly1.q test fails with Hadoop23
[ https://issues.apache.org/jira/browse/HIVE-3638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486597#comment-13486597 ] Chris Drome commented on HIVE-3638: --- @Shreepadma: I had traced the code a while back and found that Hive was creating an empty file for these types of queries. Hadoop20 didn't care that the file was empty and would create a split, which would get a mapper. With Hadoop23 I noticed that there is a condition which checks to see whether the file is empty or not. If it is empty it doesn't create a split and hence doesn't get a mapper. In this way Hive could trick Hadoop20 into running an MR job, but this tactic doesn't work on Hadoop23. I don't remember the classes off-hand. Here is the diff of the generated vs expected output. If I remember correctly, when no splits are generated it returns NULL. Line 84: select max(ds) from TEST1; (no partitions exist) Line 211: alter table TEST1 add partition (ds='1'); select max(ds) from TEST1; Line 337: select count(distinct ds) from TEST1; Line 1080: alter table TEST2 add partition (ds='1', hr='1'); alter table TEST2 add partition (ds='1', hr='2'); alter table TEST2 add partition (ds='1', hr='3'); select ds, count(distinct hr) from TEST2 group by ds; Line 1453: alter table TEST1 add partition (ds='2'); select max(ds) from TEST1; [junit] diff -a /export/crawlspace/cdrome/workspace/hive/build/ql/test/logs/clientpositive/metadataonly1.q.out /export/crawlspace/cdrome/workspace/hive/ql/src/test/results/clientpositive/metadataonly1.q.out [junit] 84c84 [junit] NULL [junit] --- [junit] [junit] 211c211 [junit] NULL [junit] --- [junit] 1 [junit] 337c337 [junit] 0 [junit] --- [junit] 1 [junit] 1080a1081 [junit] 1 3 [junit] 1453c1454 [junit] NULL [junit] --- [junit] 2 metadataonly1.q test fails with Hadoop23 Key: HIVE-3638 URL: https://issues.apache.org/jira/browse/HIVE-3638 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.10.0, 0.9.1 Reporter: Chris Drome Hive creates an empty file as a hack to get Hadoop to run a mapper. This no longer works with Hadoop23, causing this test to fail. Note that this tests empty partitions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3638) metadataonly1.q test fails with Hadoop23
[ https://issues.apache.org/jira/browse/HIVE-3638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486602#comment-13486602 ] Shreepadma Venugopalan commented on HIVE-3638: -- Yes, thats correct. When no splits are generated Hive takes a different code path and a NULL is emitted from the closeOp() of the groupByOperator. I originally noticed this problem with hadoop20 after HDFS-3672 was committed. I believe MAPREDUCE-4470 fixes the behavior on hadoop23 i.e., hadoop23 generates 1 split even when the directory is empty. I checked the behavior on hadoop20 and hadoop23 a month or so back and behavior seemed consistent with the empty result set. However, NULL is the correct behavior as per the SQL standard. Do you see this difference in behavior on the latest trunk? Thanks. metadataonly1.q test fails with Hadoop23 Key: HIVE-3638 URL: https://issues.apache.org/jira/browse/HIVE-3638 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.10.0, 0.9.1 Reporter: Chris Drome Hive creates an empty file as a hack to get Hadoop to run a mapper. This no longer works with Hadoop23, causing this test to fail. Note that this tests empty partitions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3638) metadataonly1.q test fails with Hadoop23
[ https://issues.apache.org/jira/browse/HIVE-3638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486609#comment-13486609 ] Navis commented on HIVE-3638: - HIVE-2955 started writing dummy value for empty file preventing being removed at CombineHiveInputFormatter. Could you try run test with the patch? And.. First one (at line 84) should be null, IMHO. It could be a different issue. metadataonly1.q test fails with Hadoop23 Key: HIVE-3638 URL: https://issues.apache.org/jira/browse/HIVE-3638 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.10.0, 0.9.1 Reporter: Chris Drome Hive creates an empty file as a hack to get Hadoop to run a mapper. This no longer works with Hadoop23, causing this test to fail. Note that this tests empty partitions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1362) column level statistics
[ https://issues.apache.org/jira/browse/HIVE-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486620#comment-13486620 ] Namit Jain commented on HIVE-1362: -- @Carl, Can you hold off for just 1 day ? I will definitely take a look today. column level statistics --- Key: HIVE-1362 URL: https://issues.apache.org/jira/browse/HIVE-1362 Project: Hive Issue Type: Sub-task Components: Statistics Reporter: Ning Zhang Assignee: Shreepadma Venugopalan Attachments: HIVE-1362.1.patch.txt, HIVE-1362.2.patch.txt, HIVE-1362.3.patch.txt, HIVE-1362.4.patch.txt, HIVE-1362.5.patch.txt, HIVE-1362.6.patch.txt, HIVE-1362-gen_thrift.1.patch.txt, HIVE-1362-gen_thrift.2.patch.txt, HIVE-1362-gen_thrift.3.patch.txt, HIVE-1362-gen_thrift.4.patch.txt, HIVE-1362-gen_thrift.5.patch.txt, HIVE-1362-gen_thrift.6.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HIVE-1362) column level statistics
[ https://issues.apache.org/jira/browse/HIVE-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486650#comment-13486650 ] Namit Jain edited comment on HIVE-1362 at 10/30/12 4:58 AM: https://reviews.facebook.net/D6321 Created the phabricator entry for easy review from HIVE-1362.6.patch was (Author: namit): https://reviews.facebook.net/D6321 Crated the phabricator entry for easy review from HIVE-1362.6.patch column level statistics --- Key: HIVE-1362 URL: https://issues.apache.org/jira/browse/HIVE-1362 Project: Hive Issue Type: Sub-task Components: Statistics Reporter: Ning Zhang Assignee: Shreepadma Venugopalan Attachments: HIVE-1362.1.patch.txt, HIVE-1362.2.patch.txt, HIVE-1362.3.patch.txt, HIVE-1362.4.patch.txt, HIVE-1362.5.patch.txt, HIVE-1362.6.patch.txt, HIVE-1362-gen_thrift.1.patch.txt, HIVE-1362-gen_thrift.2.patch.txt, HIVE-1362-gen_thrift.3.patch.txt, HIVE-1362-gen_thrift.4.patch.txt, HIVE-1362-gen_thrift.5.patch.txt, HIVE-1362-gen_thrift.6.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1362) column level statistics
[ https://issues.apache.org/jira/browse/HIVE-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486650#comment-13486650 ] Namit Jain commented on HIVE-1362: -- https://reviews.facebook.net/D6321 Crated the phabricator entry for easy review from HIVE-1362.6.patch column level statistics --- Key: HIVE-1362 URL: https://issues.apache.org/jira/browse/HIVE-1362 Project: Hive Issue Type: Sub-task Components: Statistics Reporter: Ning Zhang Assignee: Shreepadma Venugopalan Attachments: HIVE-1362.1.patch.txt, HIVE-1362.2.patch.txt, HIVE-1362.3.patch.txt, HIVE-1362.4.patch.txt, HIVE-1362.5.patch.txt, HIVE-1362.6.patch.txt, HIVE-1362-gen_thrift.1.patch.txt, HIVE-1362-gen_thrift.2.patch.txt, HIVE-1362-gen_thrift.3.patch.txt, HIVE-1362-gen_thrift.4.patch.txt, HIVE-1362-gen_thrift.5.patch.txt, HIVE-1362-gen_thrift.6.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1362) column level statistics
[ https://issues.apache.org/jira/browse/HIVE-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486653#comment-13486653 ] Namit Jain commented on HIVE-1362: -- Looks like this patch file is old. I couldn't see any reference to StatsSemanticAnalyzer.java. Can you load the latest patch file ? If you dont mind, can you create a phabricator entry ? I know, there is no official policy, but phabricator is much more usable. column level statistics --- Key: HIVE-1362 URL: https://issues.apache.org/jira/browse/HIVE-1362 Project: Hive Issue Type: Sub-task Components: Statistics Reporter: Ning Zhang Assignee: Shreepadma Venugopalan Attachments: HIVE-1362.1.patch.txt, HIVE-1362.2.patch.txt, HIVE-1362.3.patch.txt, HIVE-1362.4.patch.txt, HIVE-1362.5.patch.txt, HIVE-1362.6.patch.txt, HIVE-1362-gen_thrift.1.patch.txt, HIVE-1362-gen_thrift.2.patch.txt, HIVE-1362-gen_thrift.3.patch.txt, HIVE-1362-gen_thrift.4.patch.txt, HIVE-1362-gen_thrift.5.patch.txt, HIVE-1362-gen_thrift.6.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3554) Hive List Bucketing - Query logic
[ https://issues.apache.org/jira/browse/HIVE-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486659#comment-13486659 ] Namit Jain commented on HIVE-3554: -- Can you upload the latest patch file ? Hive List Bucketing - Query logic - Key: HIVE-3554 URL: https://issues.apache.org/jira/browse/HIVE-3554 Project: Hive Issue Type: New Feature Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: HIVE-3554.patch.1, HIVE-3554.patch.2, HIVE-3554.patch.3, HIVE-3554.patch.4, HIVE-3554.patch.5, HIVE-3554.patch.7 This is part of efforts for list bucketing feature: https://cwiki.apache.org/Hive/listbucketing.html This patch includes: 1. Query logic: hive chooses right sub-directory instead of partition directory. 2. alter table grammar which is required to support query logic This patch doesn't include list bucketing DML. Main reasons: 1. risk. w/o DML, this patch won't impact any existing hive regression features since no touch on any data manipulation so that very low risk. 2. manageability. w/ DML, patch is getting bigger and hard to review. Removing DML, it's easy to review. We still disable hive feature by default since DML is not in yet. DML will be in follow-up patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1362) column level statistics
[ https://issues.apache.org/jira/browse/HIVE-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486661#comment-13486661 ] Shreepadma Venugopalan commented on HIVE-1362: -- @Namit: Thanks for creating the phabricator entry. HIVE-1362.6.patch is the latest patch. StatsSemanticAnalyzer is referenced in SemanticAnalyzerFactory.java. I'll try to create a phabricator entry in the future. In the past when I've tried to use it, I've had problems with it. column level statistics --- Key: HIVE-1362 URL: https://issues.apache.org/jira/browse/HIVE-1362 Project: Hive Issue Type: Sub-task Components: Statistics Reporter: Ning Zhang Assignee: Shreepadma Venugopalan Attachments: HIVE-1362.1.patch.txt, HIVE-1362.2.patch.txt, HIVE-1362.3.patch.txt, HIVE-1362.4.patch.txt, HIVE-1362.5.patch.txt, HIVE-1362.6.patch.txt, HIVE-1362-gen_thrift.1.patch.txt, HIVE-1362-gen_thrift.2.patch.txt, HIVE-1362-gen_thrift.3.patch.txt, HIVE-1362-gen_thrift.4.patch.txt, HIVE-1362-gen_thrift.5.patch.txt, HIVE-1362-gen_thrift.6.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3636) Catch the NPe when using ^D to exit from CLI
[ https://issues.apache.org/jira/browse/HIVE-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3636: - Resolution: Fixed Status: Resolved (was: Patch Available) Catch the NPe when using ^D to exit from CLI Key: HIVE-3636 URL: https://issues.apache.org/jira/browse/HIVE-3636 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3636.patch The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3635: - Description: interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. (was: interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. Originally created by Frank Fejes. ) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type --- Key: HIVE-3635 URL: https://issues.apache.org/jira/browse/HIVE-3635 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3635.patch interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486668#comment-13486668 ] Alexander Alten-Lorenz commented on HIVE-3635: -- +1 - will do allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type --- Key: HIVE-3635 URL: https://issues.apache.org/jira/browse/HIVE-3635 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3635.patch interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3636) Catch the NPe when using ^D to exit from CLI
[ https://issues.apache.org/jira/browse/HIVE-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3636: - Description: The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace. (was: The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace. Originally created by Frank Fejes.) Catch the NPe when using ^D to exit from CLI Key: HIVE-3636 URL: https://issues.apache.org/jira/browse/HIVE-3636 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3636.patch The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3636) Catch the NPe when using ^D to exit from CLI
[ https://issues.apache.org/jira/browse/HIVE-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486669#comment-13486669 ] Alexander Alten-Lorenz commented on HIVE-3636: -- Thanks, set to Fixed Catch the NPe when using ^D to exit from CLI Key: HIVE-3636 URL: https://issues.apache.org/jira/browse/HIVE-3636 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3636.patch The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
