[jira] [Resolved] (HIVE-2249) When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
[
https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siying Dong resolved HIVE-2249.
---
Resolution: Fixed
> When creating constant expression for numbers, try to infer type from another
> comparison operand, instead of trying to use integer first, and then long and
> double
> --
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
>Reporter: Siying Dong
>Assignee: Zhiqiu Kong
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt,
> HIVE-2249.D1383.1.patch, HIVE-2249.D1383.2.patch, HIVE-2249.D1383.3.patch,
> HIVE-2249.D1383.4.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE = 0", or "WHERE
> = 0", we always have to do a type conversion when comparing, which is
> unnecessary if it is slightly smarter to choose type when creating the
> constant expression. We can simply walk one level up the tree, find another
> comparison party and use the same type with that one if it is possible. For
> user's wrong query like '=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2249) When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
[
https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196740#comment-13196740
]
Siying Dong commented on HIVE-2249:
---
committed. Thanks Zhiqiu!
> When creating constant expression for numbers, try to infer type from another
> comparison operand, instead of trying to use integer first, and then long and
> double
> --
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
>Reporter: Siying Dong
>Assignee: Zhiqiu Kong
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt,
> HIVE-2249.D1383.1.patch, HIVE-2249.D1383.2.patch, HIVE-2249.D1383.3.patch,
> HIVE-2249.D1383.4.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE = 0", or "WHERE
> = 0", we always have to do a type conversion when comparing, which is
> unnecessary if it is slightly smarter to choose type when creating the
> constant expression. We can simply walk one level up the tree, find another
> comparison party and use the same type with that one if it is possible. For
> user's wrong query like '=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2612) support hive table/partitions coexistes in more than one clusters
[ https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Yongqiang updated HIVE-2612: --- Attachment: HIVE-2612.1.patch a preliminary patch. Namit will take over the development. Things left: 1) finish drop table command 2) alter table add partition 3) drop partition 4) alter table add cluster 5) insert overwrite... 6) set table primary cluster > support hive table/partitions coexistes in more than one clusters > - > > Key: HIVE-2612 > URL: https://issues.apache.org/jira/browse/HIVE-2612 > Project: Hive > Issue Type: New Feature > Components: Metastore >Reporter: He Yongqiang >Assignee: Namit Jain > Attachments: HIVE-2612.1.patch > > > 1) add cluster object into hive metastore > 2) each partition/table has a creation cluster and a list of living clusters, > and also data location in each cluster -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2249) When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2249: -- Attachment: HIVE-2249.D1383.4.patch zhiqiu updated the revision "HIVE-2249 [jira] When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double". Reviewers: njain, kevinwilfong, heyongqiang, JIRA, stuart983 Fix insert1_overwrite_partitions.q and insert2_overwrite_partitions.q thus the results are deterministic REVISION DETAIL https://reviews.facebook.net/D1383 AFFECTED FILES contrib/src/test/results/clientpositive/dboutput.q.out contrib/src/test/results/clientpositive/serde_typedbytes4.q.out data/files/infer_const_type.txt ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java ql/src/test/queries/clientpositive/infer_const_type.q ql/src/test/queries/clientpositive/insert1_overwrite_partitions.q ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q ql/src/test/queries/clientpositive/ppr_pushdown.q ql/src/test/results/clientpositive/auto_join0.q.out ql/src/test/results/clientpositive/auto_join11.q.out ql/src/test/results/clientpositive/auto_join12.q.out ql/src/test/results/clientpositive/auto_join13.q.out ql/src/test/results/clientpositive/auto_join14.q.out ql/src/test/results/clientpositive/auto_join16.q.out ql/src/test/results/clientpositive/auto_join20.q.out ql/src/test/results/clientpositive/auto_join21.q.out ql/src/test/results/clientpositive/auto_join23.q.out ql/src/test/results/clientpositive/auto_join27.q.out ql/src/test/results/clientpositive/auto_join28.q.out ql/src/test/results/clientpositive/auto_join29.q.out ql/src/test/results/clientpositive/auto_join4.q.out ql/src/test/results/clientpositive/auto_join5.q.out ql/src/test/results/clientpositive/auto_join6.q.out ql/src/test/results/clientpositive/auto_join7.q.out ql/src/test/results/clientpositive/auto_join8.q.out ql/src/test/results/clientpositive/cast1.q.out ql/src/test/results/clientpositive/cluster.q.out ql/src/test/results/clientpositive/create_view.q.out ql/src/test/results/clientpositive/groupby_multi_single_reducer.q.out ql/src/test/results/clientpositive/having.q.out ql/src/test/results/clientpositive/index_auto.q.out ql/src/test/results/clientpositive/index_auto_empty.q.out ql/src/test/results/clientpositive/index_auto_file_format.q.out ql/src/test/results/clientpositive/index_auto_mult_tables.q.out ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out ql/src/test/results/clientpositive/index_auto_multiple.q.out ql/src/test/results/clientpositive/index_auto_partitioned.q.out ql/src/test/results/clientpositive/index_auto_self_join.q.out ql/src/test/results/clientpositive/index_auto_unused.q.out ql/src/test/results/clientpositive/index_auto_update.q.out ql/src/test/results/clientpositive/index_bitmap3.q.out ql/src/test/results/clientpositive/index_bitmap_auto.q.out ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out ql/src/test/results/clientpositive/index_bitmap_compression.q.out ql/src/test/results/clientpositive/index_compression.q.out ql/src/test/results/clientpositive/index_stale.q.out ql/src/test/results/clientpositive/index_stale_partitioned.q.out ql/src/test/results/clientpositive/infer_const_type.q.out ql/src/test/results/clientpositive/input11.q.out ql/src/test/results/clientpositive/input11_limit.q.out ql/src/test/results/clientpositive/input12.q.out ql/src/test/results/clientpositive/input13.q.out ql/src/test/results/clientpositive/input14.q.out ql/src/test/results/clientpositive/input14_limit.q.out ql/src/test/results/clientpositive/input18.q.out ql/src/test/results/clientpositive/input1_limit.q.out ql/src/test/results/clientpositive/input2_limit.q.out ql/src/test/results/clientpositive/input42.q.out ql/src/test/results/clientpositive/input_part1.q.out ql/src/test/results/clientpositive/input_part2.q.out ql/src/test/results/clientpositive/input_part5.q.out ql/src/test/results/clientpositive/input_part7.q.out ql/src/test/results/clientpositive/insert1_overwrite_partitions.q.out ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out ql/src/test/results/clientpositive/join0.q.out ql/src/test/results/clientpositive/join11.q.out ql/src/test/results/clientpositive/join12.q.out ql/src/test/results/clientpositive/join13.q.out ql/src/test/results/clientpositive/join14.q.out ql/src/test/results/clientpositive/join16.q.out ql/src/test/results/clientpositive/join20.q.out ql/src/test/results/clientpositive/join21.q.out ql/src/test/results/clientpositive/join23.q.out ql/src/test/results/clientpositive/join34.q.out ql/src/test/results/clientpositive/join35.q.out ql/src/test/results/clientpositive/join38.q.out ql
Re: HCat security design
Oh, I though you were referring to the commands at the background section. Anyway, I've added 'RW' there. Thanks, Enis On Mon, Jan 30, 2012 at 6:21 PM, Ted Yu wrote: > I meant to say that the following example should be modified: > > grant '@goo' 'hbase_table' > > > > On Mon, Jan 30, 2012 at 5:06 PM, Enis Söztutar wrote: > > > Yeah, but column family and column qualifier are optional. > > > > On Mon, Jan 30, 2012 at 5:03 PM, Ted Yu wrote: > > > > > Thanks for the heads up. > > > According to src/main/ruby/shell/commands/grant.rb, the syntax for > grant > > in > > > HBase should be: > > >hbase> grant 'bobsmith', 'RW', 't1', 'f1', 'col1' > > > > > > Cheers > > > > > > On Mon, Jan 30, 2012 at 4:51 PM, Enis Söztutar > wrote: > > > > > > > Hi devs, > > > > > > > > We have been spending some time to figure out how to approach > security > > > > related issues in Hcat/Hive + HBase and Hdfs. There has already been > > > great > > > > work, but there is still some issues to be done. I created a wiki > page > > at > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/HCATALOG/Hcat+Security+Designwhich > > > > contains a base for further discussion. All feedback, suggestions are > > > > more than welcome. > > > > > > > > Umbrella jira is https://issues.apache.org/jira/browse/HCATALOG-244. > > > > > > > > Thanks, > > > > Enis > > > > > > > > > >
Re: HCat security design
I meant to say that the following example should be modified: grant '@goo' 'hbase_table' On Mon, Jan 30, 2012 at 5:06 PM, Enis Söztutar wrote: > Yeah, but column family and column qualifier are optional. > > On Mon, Jan 30, 2012 at 5:03 PM, Ted Yu wrote: > > > Thanks for the heads up. > > According to src/main/ruby/shell/commands/grant.rb, the syntax for grant > in > > HBase should be: > >hbase> grant 'bobsmith', 'RW', 't1', 'f1', 'col1' > > > > Cheers > > > > On Mon, Jan 30, 2012 at 4:51 PM, Enis Söztutar wrote: > > > > > Hi devs, > > > > > > We have been spending some time to figure out how to approach security > > > related issues in Hcat/Hive + HBase and Hdfs. There has already been > > great > > > work, but there is still some issues to be done. I created a wiki page > at > > > > > > > > > https://cwiki.apache.org/confluence/display/HCATALOG/Hcat+Security+Designwhich > > > contains a base for further discussion. All feedback, suggestions are > > > more than welcome. > > > > > > Umbrella jira is https://issues.apache.org/jira/browse/HCATALOG-244. > > > > > > Thanks, > > > Enis > > > > > >
[jira] [Commented] (HIVE-2743) Enable SASL mode for HiveServer
[ https://issues.apache.org/jira/browse/HIVE-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196627#comment-13196627 ] Carl Steinbach commented on HIVE-2743: -- I'm planning to work on it, but probably won't be able to start for a couple of weeks. > Enable SASL mode for HiveServer > --- > > Key: HIVE-2743 > URL: https://issues.apache.org/jira/browse/HIVE-2743 > Project: Hive > Issue Type: New Feature > Components: Security, Server Infrastructure >Reporter: Carl Steinbach >Assignee: Carl Steinbach > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2743) Enable SASL mode for HiveServer
[ https://issues.apache.org/jira/browse/HIVE-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196621#comment-13196621 ] Enis Soztutar commented on HIVE-2743: - Carl, will you work on this? > Enable SASL mode for HiveServer > --- > > Key: HIVE-2743 > URL: https://issues.apache.org/jira/browse/HIVE-2743 > Project: Hive > Issue Type: New Feature > Components: Security, Server Infrastructure >Reporter: Carl Steinbach >Assignee: Carl Steinbach > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2765) hbase handler uses ZooKeeperConnectionException which is not compatible with HBase versions other than 0.89
hbase handler uses ZooKeeperConnectionException which is not compatible with HBase versions other than 0.89 Key: HIVE-2765 URL: https://issues.apache.org/jira/browse/HIVE-2765 Project: Hive Issue Type: Improvement Components: HBase Handler Affects Versions: 0.8.0, 0.7.0 Reporter: Pei Yue It cannot integrate with HBase0.21 and may not be able to integrate with hbase0.9x -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2743) Enable SASL mode for HiveServer
[ https://issues.apache.org/jira/browse/HIVE-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reassigned HIVE-2743: Assignee: Carl Steinbach > Enable SASL mode for HiveServer > --- > > Key: HIVE-2743 > URL: https://issues.apache.org/jira/browse/HIVE-2743 > Project: Hive > Issue Type: New Feature > Components: Security, Server Infrastructure >Reporter: Carl Steinbach >Assignee: Carl Steinbach > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2764) Obtain delegation tokens for MR jobs in secure hbase setup
Obtain delegation tokens for MR jobs in secure hbase setup Key: HIVE-2764 URL: https://issues.apache.org/jira/browse/HIVE-2764 Project: Hive Issue Type: Improvement Components: HBase Handler, Security Reporter: Enis Soztutar Assignee: Enis Soztutar As discussed in HCATALOG-244, in a secure hbase setup with 0.92, we need to obtain delegation tokens for hbase and save it in jobconf, so that tasks can access region servers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: HCat security design
Yeah, but column family and column qualifier are optional. On Mon, Jan 30, 2012 at 5:03 PM, Ted Yu wrote: > Thanks for the heads up. > According to src/main/ruby/shell/commands/grant.rb, the syntax for grant in > HBase should be: >hbase> grant 'bobsmith', 'RW', 't1', 'f1', 'col1' > > Cheers > > On Mon, Jan 30, 2012 at 4:51 PM, Enis Söztutar wrote: > > > Hi devs, > > > > We have been spending some time to figure out how to approach security > > related issues in Hcat/Hive + HBase and Hdfs. There has already been > great > > work, but there is still some issues to be done. I created a wiki page at > > > > > https://cwiki.apache.org/confluence/display/HCATALOG/Hcat+Security+Designwhich > > contains a base for further discussion. All feedback, suggestions are > > more than welcome. > > > > Umbrella jira is https://issues.apache.org/jira/browse/HCATALOG-244. > > > > Thanks, > > Enis > > >
Re: HCat security design
Thanks for the heads up. According to src/main/ruby/shell/commands/grant.rb, the syntax for grant in HBase should be: hbase> grant 'bobsmith', 'RW', 't1', 'f1', 'col1' Cheers On Mon, Jan 30, 2012 at 4:51 PM, Enis Söztutar wrote: > Hi devs, > > We have been spending some time to figure out how to approach security > related issues in Hcat/Hive + HBase and Hdfs. There has already been great > work, but there is still some issues to be done. I created a wiki page at > > https://cwiki.apache.org/confluence/display/HCATALOG/Hcat+Security+Designwhich > contains a base for further discussion. All feedback, suggestions are > more than welcome. > > Umbrella jira is https://issues.apache.org/jira/browse/HCATALOG-244. > > Thanks, > Enis >
HCat security design
Hi devs, We have been spending some time to figure out how to approach security related issues in Hcat/Hive + HBase and Hdfs. There has already been great work, but there is still some issues to be done. I created a wiki page at https://cwiki.apache.org/confluence/display/HCATALOG/Hcat+Security+Designwhich contains a base for further discussion. All feedback, suggestions are more than welcome. Umbrella jira is https://issues.apache.org/jira/browse/HCATALOG-244. Thanks, Enis
[jira] [Created] (HIVE-2763) CREATE TABLE IF NOT EXISTS should match schemas.
CREATE TABLE IF NOT EXISTS should match schemas. Key: HIVE-2763 URL: https://issues.apache.org/jira/browse/HIVE-2763 Project: Hive Issue Type: Improvement Reporter: Adam Kramer CREATE TABLE foo (bar BIGINT); CREATE TABLE IF NOT EXISTS foo (baz BIGINT, extra STRING); ...the latter should throw a warning, because the table _as specified_ does not exist. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2689) ObjectInspectorConverters cannot convert Void types to Array/Map/Struct types.
[ https://issues.apache.org/jira/browse/HIVE-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Chang updated HIVE-2689: - Status: Patch Available (was: Open) > ObjectInspectorConverters cannot convert Void types to Array/Map/Struct types. > -- > > Key: HIVE-2689 > URL: https://issues.apache.org/jira/browse/HIVE-2689 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Chang >Assignee: Jonathan Chang >Priority: Minor > Attachments: HIVE-2689.D1083.1.patch > > > In a bunch of places we rely on ObjectInspectorConverters to implicitly > convert types. Unfortunately, an exception will needlessly be thrown if one > of the types is a complex type and the other is a void type. For example, > SELECT ARRAY(ARRAY(), NULL) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2656) Add a command line flag to specify a jar whence to load UDFs
[ https://issues.apache.org/jira/browse/HIVE-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Chang updated HIVE-2656: - Status: Patch Available (was: Open) > Add a command line flag to specify a jar whence to load UDFs > > > Key: HIVE-2656 > URL: https://issues.apache.org/jira/browse/HIVE-2656 > Project: Hive > Issue Type: New Feature >Reporter: Jonathan Chang >Assignee: Jonathan Chang > Attachments: HIVE-2656.D1023.1.patch, HIVE-2656.D1023.2.patch > > > To extend the functionality of Hive, it is useful to be able to easily deploy > packages of UDFs and register the functions therein. Right now it is a bit > cumbersome to do ADD JAR and CREATE TEMPORARY FUNCTION. This can be greatly > simplified by adding a simple command line flag that will leverage the code > in HIVE-2523 to load all the functions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2655) Ability to define functions in HQL
[ https://issues.apache.org/jira/browse/HIVE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Chang updated HIVE-2655: - Status: Patch Available (was: Open) > Ability to define functions in HQL > -- > > Key: HIVE-2655 > URL: https://issues.apache.org/jira/browse/HIVE-2655 > Project: Hive > Issue Type: New Feature > Components: SQL >Reporter: Jonathan Perlow >Assignee: Jonathan Chang > Attachments: HIVE-2655.D915.1.patch, HIVE-2655.D915.2.patch, > HIVE-2655.D915.3.patch, HIVE-2655.D915.4.patch > > > Ability to create functions in HQL as a substitute for creating them in Java. > Jonathan Chang requested I create this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2714) Lots of special characters are not handled in LIKE
[ https://issues.apache.org/jira/browse/HIVE-2714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Chang updated HIVE-2714: - Status: Patch Available (was: Open) > Lots of special characters are not handled in LIKE > -- > > Key: HIVE-2714 > URL: https://issues.apache.org/jira/browse/HIVE-2714 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Chang >Assignee: Jonathan Chang > Attachments: HIVE-2714.D1239.1.patch > > > Currently LIKE converts the string to a regular expression. It tries to > handle special characters but many are not correctly handled, e.g. +, ?, |, > etc. That code should just use Pattern.quote. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2726) COALESCE(NULL, NULL) NPEs
[ https://issues.apache.org/jira/browse/HIVE-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Chang updated HIVE-2726: - Assignee: Jonathan Chang Status: Patch Available (was: Open) > COALESCE(NULL, NULL) NPEs > - > > Key: HIVE-2726 > URL: https://issues.apache.org/jira/browse/HIVE-2726 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Chang >Assignee: Jonathan Chang >Priority: Minor > Attachments: HIVE-2726.D1317.1.patch > > > ReturnOIResolver returns null when it gets no non-void arguments. This is > bad. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2749) CONV returns incorrect results sometimes
[ https://issues.apache.org/jira/browse/HIVE-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196549#comment-13196549 ] Jonathan Chang commented on HIVE-2749: -- Oh sorry, I thought that was only for when the final patch gets approved. Lemme change all my other JIRAs. > CONV returns incorrect results sometimes > > > Key: HIVE-2749 > URL: https://issues.apache.org/jira/browse/HIVE-2749 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Chang >Assignee: Jonathan Chang > Attachments: HIVE-2749.D1437.1.patch, HIVE-2749.D1437.2.patch > > > ...because it fails to reset state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2749) CONV returns incorrect results sometimes
[ https://issues.apache.org/jira/browse/HIVE-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Chang updated HIVE-2749: - Assignee: Jonathan Chang Status: Patch Available (was: Open) > CONV returns incorrect results sometimes > > > Key: HIVE-2749 > URL: https://issues.apache.org/jira/browse/HIVE-2749 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Chang >Assignee: Jonathan Chang > Attachments: HIVE-2749.D1437.1.patch, HIVE-2749.D1437.2.patch > > > ...because it fails to reset state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2762) Alter Table Partition Concatenate Fails On Certain Characters
[ https://issues.apache.org/jira/browse/HIVE-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196536#comment-13196536 ] Phabricator commented on HIVE-2762: --- kevinwilfong has commented on the revision "HIVE-2762 [jira] Alter Table Partition Concatenate Fails On Certain Characters". I'm not sure why escape2.q.out was considered a binary file by arcanist, but I attached a patch to the JIRA which includes the contents of that file. REVISION DETAIL https://reviews.facebook.net/D1533 > Alter Table Partition Concatenate Fails On Certain Characters > - > > Key: HIVE-2762 > URL: https://issues.apache.org/jira/browse/HIVE-2762 > Project: Hive > Issue Type: Bug >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2762.1.patch.txt, HIVE-2762.D1533.1.patch, > HIVE-2762.D1533.2.patch > > > Alter table partition concatenate creates a Java URI object for the location > of a partition. If the partition name contains certain characters, such as } > or space ' ', the object constructor fails, causing the query to fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2762) Alter Table Partition Concatenate Fails On Certain Characters
[ https://issues.apache.org/jira/browse/HIVE-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-2762: Status: Patch Available (was: Open) > Alter Table Partition Concatenate Fails On Certain Characters > - > > Key: HIVE-2762 > URL: https://issues.apache.org/jira/browse/HIVE-2762 > Project: Hive > Issue Type: Bug >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2762.1.patch.txt, HIVE-2762.D1533.1.patch, > HIVE-2762.D1533.2.patch > > > Alter table partition concatenate creates a Java URI object for the location > of a partition. If the partition name contains certain characters, such as } > or space ' ', the object constructor fails, causing the query to fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2762) Alter Table Partition Concatenate Fails On Certain Characters
[ https://issues.apache.org/jira/browse/HIVE-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-2762: Attachment: HIVE-2762.1.patch.txt > Alter Table Partition Concatenate Fails On Certain Characters > - > > Key: HIVE-2762 > URL: https://issues.apache.org/jira/browse/HIVE-2762 > Project: Hive > Issue Type: Bug >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2762.1.patch.txt, HIVE-2762.D1533.1.patch, > HIVE-2762.D1533.2.patch > > > Alter table partition concatenate creates a Java URI object for the location > of a partition. If the partition name contains certain characters, such as } > or space ' ', the object constructor fails, causing the query to fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2249) When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
[
https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196535#comment-13196535
]
Phabricator commented on HIVE-2249:
---
stuart983 has accepted the revision "HIVE-2249 [jira] When creating constant
expression for numbers, try to infer type from another comparison operand,
instead of trying to use integer first, and then long and double".
REVISION DETAIL
https://reviews.facebook.net/D1383
> When creating constant expression for numbers, try to infer type from another
> comparison operand, instead of trying to use integer first, and then long and
> double
> --
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
>Reporter: Siying Dong
>Assignee: Zhiqiu Kong
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt,
> HIVE-2249.D1383.1.patch, HIVE-2249.D1383.2.patch, HIVE-2249.D1383.3.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE = 0", or "WHERE
> = 0", we always have to do a type conversion when comparing, which is
> unnecessary if it is slightly smarter to choose type when creating the
> constant expression. We can simply walk one level up the tree, find another
> comparison party and use the same type with that one if it is possible. For
> user's wrong query like '=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2762) Alter Table Partition Concatenate Fails On Certain Characters
[ https://issues.apache.org/jira/browse/HIVE-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2762: -- Attachment: HIVE-2762.D1533.2.patch kevinwilfong updated the revision "HIVE-2762 [jira] Alter Table Partition Concatenate Fails On Certain Characters". Reviewers: JIRA Missed a file in the last diff. REVISION DETAIL https://reviews.facebook.net/D1533 AFFECTED FILES ql/src/test/results/clientpositive/escape2.q.out ql/src/test/queries/clientpositive/escape2.q ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java > Alter Table Partition Concatenate Fails On Certain Characters > - > > Key: HIVE-2762 > URL: https://issues.apache.org/jira/browse/HIVE-2762 > Project: Hive > Issue Type: Bug >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2762.D1533.1.patch, HIVE-2762.D1533.2.patch > > > Alter table partition concatenate creates a Java URI object for the location > of a partition. If the partition name contains certain characters, such as } > or space ' ', the object constructor fails, causing the query to fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2762) Alter Table Partition Concatenate Fails On Certain Characters
[ https://issues.apache.org/jira/browse/HIVE-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2762: -- Attachment: HIVE-2762.D1533.1.patch kevinwilfong requested code review of "HIVE-2762 [jira] Alter Table Partition Concatenate Fails On Certain Characters". Reviewers: JIRA https://issues.apache.org/jira/browse/HIVE-2762 Changed the way the URI is created for a partition file as part of alter table partition concatenate queries. This seems to resolve the issue. Also added a new test which checks alter table partition concsatenate queries on partitions containing most ASCII characters. Alter table partition concatenate creates a Java URI object for the location of a partition. If the partition name contains certain characters, such as } or space ' ', the object constructor fails, causing the query to fail. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D1533 AFFECTED FILES ql/src/test/results/clientpositive/escape2.q.out ql/src/test/queries/clientpositive/escape2.q MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/3201/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. > Alter Table Partition Concatenate Fails On Certain Characters > - > > Key: HIVE-2762 > URL: https://issues.apache.org/jira/browse/HIVE-2762 > Project: Hive > Issue Type: Bug >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Attachments: HIVE-2762.D1533.1.patch > > > Alter table partition concatenate creates a Java URI object for the location > of a partition. If the partition name contains certain characters, such as } > or space ' ', the object constructor fails, causing the query to fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2249) When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
[
https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siying Dong reassigned HIVE-2249:
-
Assignee: Zhiqiu Kong (was: Joseph Barillari)
> When creating constant expression for numbers, try to infer type from another
> comparison operand, instead of trying to use integer first, and then long and
> double
> --
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
>Reporter: Siying Dong
>Assignee: Zhiqiu Kong
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt,
> HIVE-2249.D1383.1.patch, HIVE-2249.D1383.2.patch, HIVE-2249.D1383.3.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE = 0", or "WHERE
> = 0", we always have to do a type conversion when comparing, which is
> unnecessary if it is slightly smarter to choose type when creating the
> constant expression. We can simply walk one level up the tree, find another
> comparison party and use the same type with that one if it is possible. For
> user's wrong query like '=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2749) CONV returns incorrect results sometimes
[ https://issues.apache.org/jira/browse/HIVE-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196528#comment-13196528 ] Carl Steinbach commented on HIVE-2749: -- Great! Can you change the status to "Patch Available"? > CONV returns incorrect results sometimes > > > Key: HIVE-2749 > URL: https://issues.apache.org/jira/browse/HIVE-2749 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Chang > Attachments: HIVE-2749.D1437.1.patch, HIVE-2749.D1437.2.patch > > > ...because it fails to reset state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2762) Alter Table Partition Concatenate Fails On Certain Characters
Alter Table Partition Concatenate Fails On Certain Characters - Key: HIVE-2762 URL: https://issues.apache.org/jira/browse/HIVE-2762 Project: Hive Issue Type: Bug Reporter: Kevin Wilfong Assignee: Kevin Wilfong Alter table partition concatenate creates a Java URI object for the location of a partition. If the partition name contains certain characters, such as } or space ' ', the object constructor fails, causing the query to fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HIVE-2612) support hive table/partitions coexistes in more than one clusters
[ https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196447#comment-13196447 ] Namit Jain edited comment on HIVE-2612 at 1/30/12 9:41 PM: --- bq. write is only allowed in this cluster for table C1. but need to allow exceptions here. What are the exceptions ? Currently, there should be no exceptions. Eventually, if we provide something in hive to do a cross-cluster write, that should be like an exception. There may be a hive command like, Replicate T@P from cluster1 to c1uster2. bq. all data changes to T1 happened in the primary cluster should be replicated to other clusters if there are any secondary clusters. but there should be a conf to disable it as there are some exception situations. This question should not be relevant now. A much simpler to visualize this is: for every table, there is a primary cluster, and a list of secondary clusters. All the partitions belong to the primary cluster, and may belong to one or more secondary clusters. Every hive session has a current cluster, and the read happens from the current cluster. An error is thrown if the partition is missing from the current cluster, but is present in the primary cluster. I will write a new wiki, and attach it - it might be simpler to understand that way. Dynamic partitions should not require anything different. bq. overwrite database name for the purpose of cluster name. And allow a table co-exist in multiple databases. But that require to promote table to top level citizen, and degrade database. For example, "show tables" used to scan all tables in current db, but now need to scan all tables in all databases. I don't think this is an option since it breaks backwards compatibility and effectively changes the whole notion of what a db/schema is. A lot of people in the community already depend on this feature. Agreed. bq. add a cluster parameter to existing thrift interfaces. This sounds like the best option to me. I think Thrift supports API evolution via default values for missing parameters, but setting a default value in this case may be a little tricky. Agreed bq. Also, instead of modifying the Thrift interface, is it possible that you could instead leverage the work that's being done in HIVE-2720? Will look into it was (Author: namit): .bq. write is only allowed in this cluster for table C1. but need to allow exceptions here. What are the exceptions ? Currently, there should be no exceptions. Eventually, if we provide something in hive to do a cross-cluster write, that should be like an exception. There may be a hive command like, Replicate T@P from cluster1 to c1uster2. .bq. all data changes to T1 happened in the primary cluster should be replicated to other clusters if there are any secondary clusters. but there should be a conf to disable it as there are some exception situations. This question should not be relevant now. A much simpler to visualize this is: for every table, there is a primary cluster, and a list of secondary clusters. All the partitions belong to the primary cluster, and may belong to one or more secondary clusters. Every hive session has a current cluster, and the read happens from the current cluster. An error is thrown if the partition is missing from the current cluster, but is present in the primary cluster. I will write a new wiki, and attach it - it might be simpler to understand that way. Dynamic partitions should not require anything different. .bq. overwrite database name for the purpose of cluster name. And allow a table co-exist in multiple databases. But that require to promote table to top level citizen, and degrade database. For example, "show tables" used to scan all tables in current db, but now need to scan all tables in all databases. I don't think this is an option since it breaks backwards compatibility and effectively changes the whole notion of what a db/schema is. A lot of people in the community already depend on this feature. Agreed. .bq. add a cluster parameter to existing thrift interfaces. This sounds like the best option to me. I think Thrift supports API evolution via default values for missing parameters, but setting a default value in this case may be a little tricky. Agreed .bq. Also, instead of modifying the Thrift interface, is it possible that you could instead leverage the work that's being done in HIVE-2720? Will look into it > support hive table/partitions coexistes in more than one clusters > - > > Key: HIVE-2612 > URL: https://issues.apache.org/jira/browse/HIVE-2612 > Project: Hive > Issue Type: New Feature > Components: Metastore >Reporter: He Yongqiang >Assignee:
[jira] [Issue Comment Edited] (HIVE-2612) support hive table/partitions coexistes in more than one clusters
[ https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196447#comment-13196447 ] Namit Jain edited comment on HIVE-2612 at 1/30/12 9:40 PM: --- .bq. write is only allowed in this cluster for table C1. but need to allow exceptions here. What are the exceptions ? Currently, there should be no exceptions. Eventually, if we provide something in hive to do a cross-cluster write, that should be like an exception. There may be a hive command like, Replicate T@P from cluster1 to c1uster2. .bq. all data changes to T1 happened in the primary cluster should be replicated to other clusters if there are any secondary clusters. but there should be a conf to disable it as there are some exception situations. This question should not be relevant now. A much simpler to visualize this is: for every table, there is a primary cluster, and a list of secondary clusters. All the partitions belong to the primary cluster, and may belong to one or more secondary clusters. Every hive session has a current cluster, and the read happens from the current cluster. An error is thrown if the partition is missing from the current cluster, but is present in the primary cluster. I will write a new wiki, and attach it - it might be simpler to understand that way. Dynamic partitions should not require anything different. .bq. overwrite database name for the purpose of cluster name. And allow a table co-exist in multiple databases. But that require to promote table to top level citizen, and degrade database. For example, "show tables" used to scan all tables in current db, but now need to scan all tables in all databases. I don't think this is an option since it breaks backwards compatibility and effectively changes the whole notion of what a db/schema is. A lot of people in the community already depend on this feature. Agreed. .bq. add a cluster parameter to existing thrift interfaces. This sounds like the best option to me. I think Thrift supports API evolution via default values for missing parameters, but setting a default value in this case may be a little tricky. Agreed .bq. Also, instead of modifying the Thrift interface, is it possible that you could instead leverage the work that's being done in HIVE-2720? Will look into it was (Author: namit): .bq write is only allowed in this cluster for table C1. but need to allow exceptions here. What are the exceptions? Currently, there should be no exceptions. Eventually, if we provide something in hive to do a cross-cluster write, that should be like an exception. There may be a hive command like, Replicate T@P from cluster1 to c1uster2. .bq all data changes to T1 happened in the primary cluster should be replicated to other clusters if there are any secondary clusters. but there should be a conf to disable it as there are some exception situations. This question should not be relevant now. A much simpler to visualize this is: for every table, there is a primary cluster, and a list of secondary clusters. All the partitions belong to the primary cluster, and may belong to one or more secondary clusters. Every hive session has a current cluster, and the read happens from the current cluster. An error is thrown if the partition is missing from the current cluster, but is present in the primary cluster. I will write a new wiki, and attach it - it might be simpler to understand that way. Dynamic partitions should not require anything different. .bq overwrite database name for the purpose of cluster name. And allow a table co-exist in multiple databases. But that require to promote table to top level citizen, and degrade database. For example, "show tables" used to scan all tables in current db, but now need to scan all tables in all databases. I don't think this is an option since it breaks backwards compatibility and effectively changes the whole notion of what a db/schema is. A lot of people in the community already depend on this feature. Agreed. .bq add a cluster parameter to existing thrift interfaces. This sounds like the best option to me. I think Thrift supports API evolution via default values for missing parameters, but setting a default value in this case may be a little tricky. Agreed .bq Also, instead of modifying the Thrift interface, is it possible that you could instead leverage the work that's being done in HIVE-2720? Will look into it > support hive table/partitions coexistes in more than one clusters > - > > Key: HIVE-2612 > URL: https://issues.apache.org/jira/browse/HIVE-2612 > Project: Hive > Issue Type: New Feature > Components: Metastore >Reporter: He Yongqiang >Assignee:
[jira] [Commented] (HIVE-2612) support hive table/partitions coexistes in more than one clusters
[ https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196447#comment-13196447 ] Namit Jain commented on HIVE-2612: -- .bq write is only allowed in this cluster for table C1. but need to allow exceptions here. What are the exceptions? Currently, there should be no exceptions. Eventually, if we provide something in hive to do a cross-cluster write, that should be like an exception. There may be a hive command like, Replicate T@P from cluster1 to c1uster2. .bq all data changes to T1 happened in the primary cluster should be replicated to other clusters if there are any secondary clusters. but there should be a conf to disable it as there are some exception situations. This question should not be relevant now. A much simpler to visualize this is: for every table, there is a primary cluster, and a list of secondary clusters. All the partitions belong to the primary cluster, and may belong to one or more secondary clusters. Every hive session has a current cluster, and the read happens from the current cluster. An error is thrown if the partition is missing from the current cluster, but is present in the primary cluster. I will write a new wiki, and attach it - it might be simpler to understand that way. Dynamic partitions should not require anything different. .bq overwrite database name for the purpose of cluster name. And allow a table co-exist in multiple databases. But that require to promote table to top level citizen, and degrade database. For example, "show tables" used to scan all tables in current db, but now need to scan all tables in all databases. I don't think this is an option since it breaks backwards compatibility and effectively changes the whole notion of what a db/schema is. A lot of people in the community already depend on this feature. Agreed. .bq add a cluster parameter to existing thrift interfaces. This sounds like the best option to me. I think Thrift supports API evolution via default values for missing parameters, but setting a default value in this case may be a little tricky. Agreed .bq Also, instead of modifying the Thrift interface, is it possible that you could instead leverage the work that's being done in HIVE-2720? Will look into it > support hive table/partitions coexistes in more than one clusters > - > > Key: HIVE-2612 > URL: https://issues.apache.org/jira/browse/HIVE-2612 > Project: Hive > Issue Type: New Feature > Components: Metastore >Reporter: He Yongqiang >Assignee: Namit Jain > > 1) add cluster object into hive metastore > 2) each partition/table has a creation cluster and a list of living clusters, > and also data location in each cluster -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2749) CONV returns incorrect results sometimes
[ https://issues.apache.org/jira/browse/HIVE-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196446#comment-13196446 ] Jonathan Chang commented on HIVE-2749: -- Yep! > CONV returns incorrect results sometimes > > > Key: HIVE-2749 > URL: https://issues.apache.org/jira/browse/HIVE-2749 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Chang > Attachments: HIVE-2749.D1437.1.patch, HIVE-2749.D1437.2.patch > > > ...because it fails to reset state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2753) Remove empty java files
[
https://issues.apache.org/jira/browse/HIVE-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196428#comment-13196428
]
Carl Steinbach commented on HIVE-2753:
--
@Owen: Can you try submitting this patch with arc? It would be good to see if
arc handles deleted files properly. Thanks.
> Remove empty java files
> ---
>
> Key: HIVE-2753
> URL: https://issues.apache.org/jira/browse/HIVE-2753
> Project: Hive
> Issue Type: Bug
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Attachments: h-2753.patch
>
>
> When looking at the 0.8.1 rc1, I discovered there were a set of empty Java
> files that were likely left over from using 'patch' without the -E.
> {quote}
> jdbc/src/java/org/apache/hadoop/hive/jdbc/JdbcSessionState.java
> ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeIndexEvaluator.java
> ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinObject.java
> ql/src/java/org/apache/hadoop/hive/ql/exec/PathUtil.java
> ql/src/java/org/apache/hadoop/hive/ql/exec/TypedBytesRecordReader.java
> ql/src/java/org/apache/hadoop/hive/ql/plan/AlterPartitionProtectModeDesc.java
> ql/src/java/org/apache/hadoop/hive/ql/plan/TouchDesc.java
> ql/src/test/org/apache/hadoop/hive/ql/plan/TestAddPartition.java
> serde/src/gen-java/org/apache/hadoop/hive/serde/test/Constants.java
> shims/src/0.20/java/org/apache/hadoop/fs/ProxyFileSystem.java
> shims/src/0.20/java/org/apache/hadoop/fs/ProxyLocalFileSystem.java
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2749) CONV returns incorrect results sometimes
[ https://issues.apache.org/jira/browse/HIVE-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196426#comment-13196426 ] Carl Steinbach commented on HIVE-2749: -- Is this ready for review? > CONV returns incorrect results sometimes > > > Key: HIVE-2749 > URL: https://issues.apache.org/jira/browse/HIVE-2749 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Chang > Attachments: HIVE-2749.D1437.1.patch, HIVE-2749.D1437.2.patch > > > ...because it fails to reset state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2612) support hive table/partitions coexistes in more than one clusters
[ https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196422#comment-13196422 ] Namit Jain commented on HIVE-2612: -- bq. A table T1's primary cluster is C1 meaning :1) C1 contains all data that is available in all other clusters. Does this mean that if T1's primary cluster is C1, then all of the partitions in T1 must also have have their primary partition set to C1? If that's the case then primary cluster should probably be a table level property, and the list of replica clusters can be a table/partition level property. I agree > support hive table/partitions coexistes in more than one clusters > - > > Key: HIVE-2612 > URL: https://issues.apache.org/jira/browse/HIVE-2612 > Project: Hive > Issue Type: New Feature > Components: Metastore >Reporter: He Yongqiang >Assignee: Namit Jain > > 1) add cluster object into hive metastore > 2) each partition/table has a creation cluster and a list of living clusters, > and also data location in each cluster -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-0.8.1-SNAPSHOT-h0.21 - Build # 180 - Fixed
Changes for Build #179 Changes for Build #180 All tests passed The Apache Jenkins build system has built Hive-0.8.1-SNAPSHOT-h0.21 (build #180) Status: Fixed Check console output at https://builds.apache.org/job/Hive-0.8.1-SNAPSHOT-h0.21/180/ to view the results.
Hive-trunk-h0.21 - Build # 1232 - Still Failing
Changes for Build #1189 Changes for Build #1190 [amareshwari] HIVE-2629. Make a single Hive binary work with both 0.20.x and 0.23.0. (Thomas Weise via amareshwari) Changes for Build #1191 [amareshwari] HIVE-2629. Reverting previous commit Changes for Build #1192 [heyongqiang] HIVE-2706 [jira] StackOverflowError when using custom UDF after adding archive after adding jars (Kevin Wilfong via Yongqiang He) Summary: https://issues.apache.org/jira/browse/HIVE-2706 The issue was that the current thread's classloader and the classloader in the conf differed due to the prehook updating only the current thread's classloader with new jars. Now, it updates both classloaders, fixing the issue. When a custom UDF is used in a query after add an archive, such as a zip file, after adding jars, the XMLEncoder enters an infinite loop when serializing the map reduce task, as part of sending it to be executed. This results in a stack overflow error. Test Plan: Verified it fixed the stack overflow error. Reviewers: JIRA, heyongqiang, njain Reviewed By: heyongqiang CC: heyongqiang Differential Revision: https://reviews.facebook.net/D1167 Changes for Build #1193 [hashutosh] HIVE-2705: SemanticAnalyzer twice swallows an exception it shouldn't (jghoman via hashutosh) Changes for Build #1194 Changes for Build #1195 [hashutosh] HIVE-2589: Newly created partition should inherit properties from table (Ashutosh Chauhan) [hashutosh] HIVE-2682: Clean-up logs (Rajat Goel via Ashutosh Chauhan) Changes for Build #1196 [amareshwari] HIVE-2629. Make a single Hive binary work with both 0.20.x and 0.23.0. (Thomas Weise via amareshwari) Changes for Build #1197 Changes for Build #1198 [namit] HIVE-2504 Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory (Chinna Rao Lalam via namit) [namit] HIVE-2695 Add PRINTF() Udf (Zhenxiao Luo via namit) Changes for Build #1199 Changes for Build #1200 Changes for Build #1201 Changes for Build #1202 Changes for Build #1203 Changes for Build #1204 [cws] HIVE-2719. Revert HIVE-2589 (He Yongqiang via cws) Changes for Build #1205 Changes for Build #1207 [namit] HIVE-2718 NPE in union followed by join (He Yongqiang via namit) Changes for Build #1208 Changes for Build #1209 Changes for Build #1210 [namit] HIVE-2674 get_partitions_ps throws TApplicationException if table doesn't exist (Kevin Wilfong via namit) Changes for Build #1211 [cws] HIVE-2203. Extend concat_ws() UDF to support arrays of strings (Zhenxiao Luo via cws) [cws] HIVE-2279. Implement sort(array) UDF (Zhenxiao Luo via cws) Changes for Build #1212 [hashutosh] HIVE-2589 : Newly created partition should inherit properties from table (Ashutosh Chauhan) Changes for Build #1213 Changes for Build #1214 Changes for Build #1215 Changes for Build #1216 [cws] HIVE-2724. Remove unused lib/log4j-1.2.15.jar (Andrew Bayer via cws) Changes for Build #1217 Changes for Build #1218 Changes for Build #1219 [hashutosh] HIVE-2665 : Support for metastore service specific HADOOP_OPTS environment setting (thw via hashutosh) Changes for Build #1220 [namit] HIVE-2727 add a testcase for partitioned view on union and base tables have index (He Yongqiang via namit) Changes for Build #1221 [hashutosh] HIVE-2746 : Metastore client doesn't log properly in case of connection failure to server (hashutosh) [cws] HIVE-2698 [jira] Enable Hadoop-1.0.0 in Hive (Enis Söztutar via Carl Steinbach) Summary: third version of the patch Hadoop-1.0.0 is recently released, which is AFAIK, API compatible to the 0.20S release. Test Plan: EMPTY Reviewers: JIRA, cwsteinbach Reviewed By: cwsteinbach CC: cwsteinbach, enis Differential Revision: https://reviews.facebook.net/D1389 Changes for Build #1222 [namit] HIVE-2750 Hive multi group by single reducer optimization causes invalid column reference error (Kevin Wilfong via namit) Changes for Build #1223 Changes for Build #1224 [cws] HIVE-2734 [jira] Fix some nondeterministic test output (Zhenxiao Luo via Carl Steinbach) Summary: HIVE-2734: Fix some nondeterministic test output Many Hive query tests lack an ORDER BY clause, and consequently the ordering of the rows in the result set is nondeterministic: groupby1_limit input11_limit input1_limit input_lazyserde join18_multi_distinct join_1to1 join_casesensitive join_filters join_nulls merge3 rcfile_columnar rcfile_lazydecompress rcfile_union sample10 udf_sentences union24 columnarserde_create_shortcut combine1 global_limit Test Plan: EMPTY Reviewers: JIRA, cwsteinbach Reviewed By: cwsteinbach CC: zhenxiao, cwsteinbach Differential Revision: https://reviews.facebook.net/D1449 [namit] HIVE-2754 NPE in union with lateral view (Yongqiang He via namit) Changes for Build #1225 Changes for Build #1226 Changes for Build #1227 [namit] HIVE-2755 union follwowed by union_subq does not work if the subquery union has reducers (He Yongqiang via namit) Changes for Build #1228
[jira] [Updated] (HIVE-2577) Expose the HiveConf in HiveConnection API
[ https://issues.apache.org/jira/browse/HIVE-2577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Lalevée updated HIVE-2577: -- Affects Version/s: 0.8.0 Status: Patch Available (was: Open) > Expose the HiveConf in HiveConnection API > - > > Key: HIVE-2577 > URL: https://issues.apache.org/jira/browse/HIVE-2577 > Project: Hive > Issue Type: Improvement >Affects Versions: 0.8.0 >Reporter: Nicolas Lalevée > Attachments: HIVE-2577-r1201637.patch > > > When running the jdbc code in a local mode, there no way to programatically > manage the hive conf. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2576) HiveDataSource doesn't get a proper connection
[ https://issues.apache.org/jira/browse/HIVE-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Lalevée updated HIVE-2576: -- Affects Version/s: 0.8.0 Status: Patch Available (was: Open) > HiveDataSource doesn't get a proper connection > -- > > Key: HIVE-2576 > URL: https://issues.apache.org/jira/browse/HIVE-2576 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 0.8.0 >Reporter: Nicolas Lalevée > Attachments: HIVE-2576-r1201637.patch > > > The HiveDataSource is creating a HiveConnection with as an URL "", but the > connection expects to start with "jdbc:hive://" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2738) NPE in ExprNodeGenericFuncEvaluator
[
https://issues.apache.org/jira/browse/HIVE-2738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Lalevée updated HIVE-2738:
--
Status: Patch Available (was: Open)
> NPE in ExprNodeGenericFuncEvaluator
> ---
>
> Key: HIVE-2738
> URL: https://issues.apache.org/jira/browse/HIVE-2738
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.8.0
>Reporter: Nicolas Lalevée
> Attachments: 750c8966-6402-465a-b011-903469fe56da.xml,
> HIVE-2738-r1237763.patch, MapMaxUDF.java, MapToJsonUDF.java, hive_job_logs.txt
>
>
> Here is the query:
> bq. {{SELECT t.lid, '2011-12-12',
> s_map2json(s_maxmap(UNION_MAP(t.categoryCount), 100)) FROM ( SELECT theme_lid
> AS theme_lid, MAP(s_host(referer), COUNT( * )) AS categoryCount FROM
> PageViewEvent WHERE day >= '20130104' AND day <= '20130112' AND date_ >=
> '2012-01-04' AND date_ < '2012-01-13' AND lid IS NOT NULL GROUP BY lid,
> s_host(referer) ) t GROUP BY t.lid}}
> Removing the call s_map2json make it work but not by removing s_maxmap, but I
> don't understand what could be wrong with the implementation of my udf. And I
> don't know how to debug remote hadoop jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2738) NPE in ExprNodeGenericFuncEvaluator
[
https://issues.apache.org/jira/browse/HIVE-2738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Lalevée updated HIVE-2738:
--
Attachment: HIVE-2738-r1237763.patch
> NPE in ExprNodeGenericFuncEvaluator
> ---
>
> Key: HIVE-2738
> URL: https://issues.apache.org/jira/browse/HIVE-2738
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.8.0
>Reporter: Nicolas Lalevée
> Attachments: 750c8966-6402-465a-b011-903469fe56da.xml,
> HIVE-2738-r1237763.patch, MapMaxUDF.java, MapToJsonUDF.java, hive_job_logs.txt
>
>
> Here is the query:
> bq. {{SELECT t.lid, '2011-12-12',
> s_map2json(s_maxmap(UNION_MAP(t.categoryCount), 100)) FROM ( SELECT theme_lid
> AS theme_lid, MAP(s_host(referer), COUNT( * )) AS categoryCount FROM
> PageViewEvent WHERE day >= '20130104' AND day <= '20130112' AND date_ >=
> '2012-01-04' AND date_ < '2012-01-13' AND lid IS NOT NULL GROUP BY lid,
> s_host(referer) ) t GROUP BY t.lid}}
> Removing the call s_map2json make it work but not by removing s_maxmap, but I
> don't understand what could be wrong with the implementation of my udf. And I
> don't know how to debug remote hadoop jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2738) NPE in ExprNodeGenericFuncEvaluator
[
https://issues.apache.org/jira/browse/HIVE-2738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196216#comment-13196216
]
Nicolas Lalevée commented on HIVE-2738:
---
Actually there was some classpath issues, the version of jackson use by hadoop
and mine are conflicting. The result is that my genericUDF failed to be
instanciated, and the XMLDecoder, rather than reporting the error, returns
null. Hence the NPE.
I suggest the XMLDecoder should always fail on error. See the attached patch.
> NPE in ExprNodeGenericFuncEvaluator
> ---
>
> Key: HIVE-2738
> URL: https://issues.apache.org/jira/browse/HIVE-2738
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.8.0
>Reporter: Nicolas Lalevée
> Attachments: 750c8966-6402-465a-b011-903469fe56da.xml,
> HIVE-2738-r1237763.patch, MapMaxUDF.java, MapToJsonUDF.java, hive_job_logs.txt
>
>
> Here is the query:
> bq. {{SELECT t.lid, '2011-12-12',
> s_map2json(s_maxmap(UNION_MAP(t.categoryCount), 100)) FROM ( SELECT theme_lid
> AS theme_lid, MAP(s_host(referer), COUNT( * )) AS categoryCount FROM
> PageViewEvent WHERE day >= '20130104' AND day <= '20130112' AND date_ >=
> '2012-01-04' AND date_ < '2012-01-13' AND lid IS NOT NULL GROUP BY lid,
> s_host(referer) ) t GROUP BY t.lid}}
> Removing the call s_map2json make it work but not by removing s_maxmap, but I
> don't understand what could be wrong with the implementation of my udf. And I
> don't know how to debug remote hadoop jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2604) Add UberCompressor Serde/Codec to contrib which allows per-column compression strategies
[ https://issues.apache.org/jira/browse/HIVE-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2604: -- Attachment: HIVE-2604.D1011.2.patch krishnakumar updated the revision "HIVE-2604 [jira] Add UberCompressor Serde/Codec to contrib which allows per-column compression strategies". Reviewers: JIRA, heyongqiang Addressing review comments REVISION DETAIL https://reviews.facebook.net/D1011 AFFECTED FILES contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/InputReader.java contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/OutputWriter.java contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/TypeSpecificCompressor.java contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/UberCompressionCodec.java contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/UberCompressionInputStream.java contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/UberCompressionOutputStream.java contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/UberCompressorColumnConfig.java contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/UberCompressorConfig.java contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/UberCompressorSerde.java contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/UberCompressorSerdeField.java contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/UberCompressorUtils.java contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/compressors/DummyIntegerCompressor.java contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/dsalg/Tuple.java contrib/src/test/queries/clientpositive/ubercompressor.q contrib/src/test/results/clientpositive/ubercompressor.q.out > Add UberCompressor Serde/Codec to contrib which allows per-column compression > strategies > > > Key: HIVE-2604 > URL: https://issues.apache.org/jira/browse/HIVE-2604 > Project: Hive > Issue Type: Sub-task > Components: Contrib >Reporter: Krishna Kumar >Assignee: Krishna Kumar > Attachments: HIVE-2604.D1011.1.patch, HIVE-2604.D1011.2.patch, > HIVE-2604.v0.patch, HIVE-2604.v1.patch, HIVE-2604.v2.patch > > > The strategies supported are > 1. using a specified codec on the column > 2. using a specific codec on the column which is serialized via a specific > serde > 3. using a specific "TypeSpecificCompressor" instance -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1231 - Still Failing
Changes for Build #1189 Changes for Build #1190 [amareshwari] HIVE-2629. Make a single Hive binary work with both 0.20.x and 0.23.0. (Thomas Weise via amareshwari) Changes for Build #1191 [amareshwari] HIVE-2629. Reverting previous commit Changes for Build #1192 [heyongqiang] HIVE-2706 [jira] StackOverflowError when using custom UDF after adding archive after adding jars (Kevin Wilfong via Yongqiang He) Summary: https://issues.apache.org/jira/browse/HIVE-2706 The issue was that the current thread's classloader and the classloader in the conf differed due to the prehook updating only the current thread's classloader with new jars. Now, it updates both classloaders, fixing the issue. When a custom UDF is used in a query after add an archive, such as a zip file, after adding jars, the XMLEncoder enters an infinite loop when serializing the map reduce task, as part of sending it to be executed. This results in a stack overflow error. Test Plan: Verified it fixed the stack overflow error. Reviewers: JIRA, heyongqiang, njain Reviewed By: heyongqiang CC: heyongqiang Differential Revision: https://reviews.facebook.net/D1167 Changes for Build #1193 [hashutosh] HIVE-2705: SemanticAnalyzer twice swallows an exception it shouldn't (jghoman via hashutosh) Changes for Build #1194 Changes for Build #1195 [hashutosh] HIVE-2589: Newly created partition should inherit properties from table (Ashutosh Chauhan) [hashutosh] HIVE-2682: Clean-up logs (Rajat Goel via Ashutosh Chauhan) Changes for Build #1196 [amareshwari] HIVE-2629. Make a single Hive binary work with both 0.20.x and 0.23.0. (Thomas Weise via amareshwari) Changes for Build #1197 Changes for Build #1198 [namit] HIVE-2504 Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory (Chinna Rao Lalam via namit) [namit] HIVE-2695 Add PRINTF() Udf (Zhenxiao Luo via namit) Changes for Build #1199 Changes for Build #1200 Changes for Build #1201 Changes for Build #1202 Changes for Build #1203 Changes for Build #1204 [cws] HIVE-2719. Revert HIVE-2589 (He Yongqiang via cws) Changes for Build #1205 Changes for Build #1207 [namit] HIVE-2718 NPE in union followed by join (He Yongqiang via namit) Changes for Build #1208 Changes for Build #1209 Changes for Build #1210 [namit] HIVE-2674 get_partitions_ps throws TApplicationException if table doesn't exist (Kevin Wilfong via namit) Changes for Build #1211 [cws] HIVE-2203. Extend concat_ws() UDF to support arrays of strings (Zhenxiao Luo via cws) [cws] HIVE-2279. Implement sort(array) UDF (Zhenxiao Luo via cws) Changes for Build #1212 [hashutosh] HIVE-2589 : Newly created partition should inherit properties from table (Ashutosh Chauhan) Changes for Build #1213 Changes for Build #1214 Changes for Build #1215 Changes for Build #1216 [cws] HIVE-2724. Remove unused lib/log4j-1.2.15.jar (Andrew Bayer via cws) Changes for Build #1217 Changes for Build #1218 Changes for Build #1219 [hashutosh] HIVE-2665 : Support for metastore service specific HADOOP_OPTS environment setting (thw via hashutosh) Changes for Build #1220 [namit] HIVE-2727 add a testcase for partitioned view on union and base tables have index (He Yongqiang via namit) Changes for Build #1221 [hashutosh] HIVE-2746 : Metastore client doesn't log properly in case of connection failure to server (hashutosh) [cws] HIVE-2698 [jira] Enable Hadoop-1.0.0 in Hive (Enis Söztutar via Carl Steinbach) Summary: third version of the patch Hadoop-1.0.0 is recently released, which is AFAIK, API compatible to the 0.20S release. Test Plan: EMPTY Reviewers: JIRA, cwsteinbach Reviewed By: cwsteinbach CC: cwsteinbach, enis Differential Revision: https://reviews.facebook.net/D1389 Changes for Build #1222 [namit] HIVE-2750 Hive multi group by single reducer optimization causes invalid column reference error (Kevin Wilfong via namit) Changes for Build #1223 Changes for Build #1224 [cws] HIVE-2734 [jira] Fix some nondeterministic test output (Zhenxiao Luo via Carl Steinbach) Summary: HIVE-2734: Fix some nondeterministic test output Many Hive query tests lack an ORDER BY clause, and consequently the ordering of the rows in the result set is nondeterministic: groupby1_limit input11_limit input1_limit input_lazyserde join18_multi_distinct join_1to1 join_casesensitive join_filters join_nulls merge3 rcfile_columnar rcfile_lazydecompress rcfile_union sample10 udf_sentences union24 columnarserde_create_shortcut combine1 global_limit Test Plan: EMPTY Reviewers: JIRA, cwsteinbach Reviewed By: cwsteinbach CC: zhenxiao, cwsteinbach Differential Revision: https://reviews.facebook.net/D1449 [namit] HIVE-2754 NPE in union with lateral view (Yongqiang He via namit) Changes for Build #1225 Changes for Build #1226 Changes for Build #1227 [namit] HIVE-2755 union follwowed by union_subq does not work if the subquery union has reducers (He Yongqiang via namit) Changes for Build #1228
[jira] [Updated] (HIVE-2750) Hive multi group by single reducer optimization causes invalid column reference error
[ https://issues.apache.org/jira/browse/HIVE-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amareshwari Sriramadasu updated HIVE-2750: -- Resolution: Fixed Fix Version/s: 0.9.0 Status: Resolved (was: Patch Available) Seems the issue missed resolution. Resolving. > Hive multi group by single reducer optimization causes invalid column > reference error > - > > Key: HIVE-2750 > URL: https://issues.apache.org/jira/browse/HIVE-2750 > Project: Hive > Issue Type: Bug >Reporter: Kevin Wilfong >Assignee: Kevin Wilfong > Fix For: 0.9.0 > > Attachments: HIVE-2750.D1455.1.patch > > > After the optimization, if two query blocks have the same distinct clause and > the same group by keys, but the first query block does not reference all the > rows the second query block does, an invalid column reference error is raised > for the columns unreferenced in the first query block. > E.g. > FROM src > INSERT OVERWRITE TABLE dest_g2 SELECT substr(src.key,1,1), count(DISTINCT > src.key) WHERE substr(src.key,1,1) >= 5 GROUP BY substr(src.key,1,1) > INSERT OVERWRITE TABLE dest_g3 SELECT substr(src.key,1,1), count(DISTINCT > src.key), count(src.value) WHERE substr(src.key,1,1) < 5 GROUP BY > substr(src.key,1,1); > This results in an invalid column reference error on src.value -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
