[jira] [Updated] (HIVE-4144) Add select database() command to show the current database
[
https://issues.apache.org/jira/browse/HIVE-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-4144:
--
Attachment: HIVE-4144.D9597.1.patch
navis requested code review of HIVE-4144 [jira] Add select database()
command to show the current database.
Reviewers: JIRA
HIVE-4144 Add select database() command to show the current database
A recent hive-user mailing list conversation asked about having a command to
show the current database.
http://mail-archives.apache.org/mod_mbox/hive-user/201303.mbox/%3CCAMGr+0i+CRY69m3id=DxthmUCWLf0NxpKMCtROb=uauh2va...@mail.gmail.com%3E
MySQL seems to have a command to do so:
select database();
http://dev.mysql.com/doc/refman/5.0/en/information-functions.html#function_database
We should look into having something similar in Hive.
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D9597
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/UDFCurrentDB.java
ql/src/test/queries/clientpositive/udf_current_database.q
ql/src/test/results/clientpositive/udf_current_database.q.out
MANAGE HERALD RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/22923/
To: JIRA, navis
Add select database() command to show the current database
Key: HIVE-4144
URL: https://issues.apache.org/jira/browse/HIVE-4144
Project: Hive
Issue Type: Bug
Components: SQL
Reporter: Mark Grover
Assignee: Navis
Attachments: HIVE-4144.D9597.1.patch
A recent hive-user mailing list conversation asked about having a command to
show the current database.
http://mail-archives.apache.org/mod_mbox/hive-user/201303.mbox/%3CCAMGr+0i+CRY69m3id=DxthmUCWLf0NxpKMCtROb=uauh2va...@mail.gmail.com%3E
MySQL seems to have a command to do so:
{code}
select database();
{code}
http://dev.mysql.com/doc/refman/5.0/en/information-functions.html#function_database
We should look into having something similar in Hive.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4144) Add select database() command to show the current database
[
https://issues.apache.org/jira/browse/HIVE-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-4144:
Assignee: Navis
Status: Patch Available (was: Open)
'database' cannot be used cause it's a keyword. so used 'current_database'
instead.
Add select database() command to show the current database
Key: HIVE-4144
URL: https://issues.apache.org/jira/browse/HIVE-4144
Project: Hive
Issue Type: Bug
Components: SQL
Reporter: Mark Grover
Assignee: Navis
Attachments: HIVE-4144.D9597.1.patch
A recent hive-user mailing list conversation asked about having a command to
show the current database.
http://mail-archives.apache.org/mod_mbox/hive-user/201303.mbox/%3CCAMGr+0i+CRY69m3id=DxthmUCWLf0NxpKMCtROb=uauh2va...@mail.gmail.com%3E
MySQL seems to have a command to do so:
{code}
select database();
{code}
http://dev.mysql.com/doc/refman/5.0/en/information-functions.html#function_database
We should look into having something similar in Hive.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3381) Result of outer join is not valid
[
https://issues.apache.org/jira/browse/HIVE-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-3381:
--
Attachment: HIVE-3381.D5565.5.patch
navis updated the revision HIVE-3381 [jira] Result of outer join is not valid.
Rebased to trunk. running test
Reviewers: JIRA
REVISION DETAIL
https://reviews.facebook.net/D5565
CHANGE SINCE LAST DIFF
https://reviews.facebook.net/D5565?vs=22947id=30225#toc
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/exec/CommonJoinOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/HashTableSinkOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/JoinUtil.java
ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinObjectValue.java
ql/src/test/results/clientpositive/auto_join21.q.out
ql/src/test/results/clientpositive/auto_join29.q.out
ql/src/test/results/clientpositive/auto_join7.q.out
ql/src/test/results/clientpositive/auto_join_filters.q.out
ql/src/test/results/clientpositive/join21.q.out
ql/src/test/results/clientpositive/join7.q.out
ql/src/test/results/clientpositive/join_1to1.q.out
ql/src/test/results/clientpositive/join_filters.q.out
ql/src/test/results/clientpositive/join_filters_overlap.q.out
To: JIRA, navis
Cc: njain
Result of outer join is not valid
-
Key: HIVE-3381
URL: https://issues.apache.org/jira/browse/HIVE-3381
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.10.0
Reporter: Navis
Assignee: Navis
Priority: Critical
Attachments: HIVE-3381.D5565.3.patch, HIVE-3381.D5565.4.patch,
HIVE-3381.D5565.5.patch
Outer joins, especially full outer joins or outer join with filter on 'ON
clause' is not showing proper results. For example, query in test join_1to1.q
{code}
SELECT * FROM join_1to1_1 a full outer join join_1to1_2 b on a.key1 = b.key1
and a.value = 66 and b.value = 66 ORDER BY a.key1 ASC, a.key2 ASC, a.value
ASC, b.key1 ASC, b.key2 ASC, b.value ASC;
{code}
results
{code}
NULL NULLNULLNULLNULL66
NULL NULLNULLNULL10050 66
NULL NULLNULL10 10010 66
NULL NULLNULL30 10030 88
NULL NULLNULL35 10035 88
NULL NULLNULL40 10040 88
NULL NULLNULL40 10040 88
NULL NULLNULL50 10050 88
NULL NULLNULL50 10050 88
NULL NULLNULL50 10050 88
NULL NULLNULL70 10040 88
NULL NULLNULL70 10040 88
NULL NULLNULL70 10040 88
NULL NULLNULL70 10040 88
NULL NULL66 NULLNULLNULL
NULL 10050 66 NULLNULLNULL
5 10005 66 5 10005 66
1510015 66 NULLNULLNULL
2010020 66 20 10020 66
2510025 88 NULLNULLNULL
3010030 66 NULLNULLNULL
3510035 88 NULLNULLNULL
4010040 66 NULLNULLNULL
4010040 66 40 10040 66
4010040 88 NULLNULLNULL
4010040 88 NULLNULLNULL
5010050 66 NULLNULLNULL
5010050 66 50 10050 66
5010050 66 50 10050 66
5010050 88 NULLNULLNULL
5010050 88 NULLNULLNULL
5010050 88 NULLNULLNULL
5010050 88 NULLNULLNULL
5010050 88 NULLNULLNULL
5010050 88 NULLNULLNULL
6010040 66 60 10040 66
6010040 66 60 10040 66
6010040 66 60 10040 66
6010040 66 60 10040 66
7010040 66 NULLNULLNULL
7010040 66 NULLNULLNULL
7010040 66 NULLNULLNULL
7010040 66 NULLNULLNULL
8010040 88 NULLNULLNULL
8010040 88 NULLNULLNULL
8010040 88 NULLNULLNULL
8010040 88 NULLNULLNULL
{code}
but it seemed not right. This should be
{code}
NULL NULLNULLNULLNULL66
NULL NULLNULLNULL10050 66
NULL NULLNULL10 10010 66
NULL NULLNULL25 10025 66
NULL NULLNULL30 10030 88
NULL NULLNULL35 10035 88
NULL NULLNULL40 10040 88
NULL NULLNULL50 10050 88
NULL NULLNULL70 10040 88
NULL NULLNULL70 10040 88
NULL NULLNULL80 10040 66
NULL NULLNULL80 10040 66
NULL NULL
[jira] [Updated] (HIVE-4206) Sort merge join does not work for outer joins for 7 inputs
[ https://issues.apache.org/jira/browse/HIVE-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4206: - Summary: Sort merge join does not work for outer joins for 7 inputs (was: Sort merge join does not work for more than 7 inputs) Sort merge join does not work for outer joins for 7 inputs -- Key: HIVE-4206 URL: https://issues.apache.org/jira/browse/HIVE-4206 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4206) Sort merge join does not work for outer joins for 7 inputs
[ https://issues.apache.org/jira/browse/HIVE-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13608776#comment-13608776 ] Namit Jain commented on HIVE-4206: -- https://reviews.facebook.net/D9603 Sort merge join does not work for outer joins for 7 inputs -- Key: HIVE-4206 URL: https://issues.apache.org/jira/browse/HIVE-4206 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.4206.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4206) Sort merge join does not work for outer joins for 7 inputs
[ https://issues.apache.org/jira/browse/HIVE-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4206: - Attachment: hive.4206.1.patch Sort merge join does not work for outer joins for 7 inputs -- Key: HIVE-4206 URL: https://issues.apache.org/jira/browse/HIVE-4206 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.4206.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Deleted] (HIVE-4206) Sort merge join does not work for outer joins for 7 inputs
[ https://issues.apache.org/jira/browse/HIVE-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4206: - Comment: was deleted (was: set hive.optimize.bucketmapjoin = true; set hive.optimize.bucketmapjoin.sortedmerge = true; set hive.input.format = org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat; set hive.enforce.bucketing=true; set hive.enforce.sorting=true; set hive.exec.reducers.max = 1; set hive.merge.mapfiles=false; set hive.merge.mapredfiles=false; -- Create bucketed and sorted tables CREATE TABLE test_table1 (key INT, value STRING) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; CREATE TABLE test_table2 (key INT, value STRING) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; CREATE TABLE test_table3 (key INT, value STRING) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; CREATE TABLE test_table4 (key INT, value STRING) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; CREATE TABLE test_table5 (key INT, value STRING) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; CREATE TABLE test_table6 (key INT, value STRING) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; CREATE TABLE test_table7 (key INT, value STRING) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; FROM src INSERT OVERWRITE TABLE test_table1 SELECT *; FROM src INSERT OVERWRITE TABLE test_table2 SELECT *; FROM src INSERT OVERWRITE TABLE test_table3 SELECT *; FROM src INSERT OVERWRITE TABLE test_table4 SELECT *; FROM src INSERT OVERWRITE TABLE test_table5 SELECT *; FROM src INSERT OVERWRITE TABLE test_table6 SELECT *; FROM src INSERT OVERWRITE TABLE test_table7 SELECT *; -- Mapjoin followed by a aggregation should be performed in a single MR job EXPLAIN SELECT /*+mapjoin(b)*/ count(*) FROM test_table1 a JOIN test_table2 b ON a.key = b.key JOIN test_table3 c ON a.key = c.key JOIN test_table4 d ON a.key = d.key JOIN test_table5 e ON a.key = e.key JOIN test_table6 f ON a.key = f.key JOIN test_table6 g ON a.key = g.key; The above query does not use sort-merge join.) Sort merge join does not work for outer joins for 7 inputs -- Key: HIVE-4206 URL: https://issues.apache.org/jira/browse/HIVE-4206 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4206) Sort merge join does not work for outer joins for 7 inputs
[ https://issues.apache.org/jira/browse/HIVE-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4206: - Attachment: hive.4206.2.patch Sort merge join does not work for outer joins for 7 inputs -- Key: HIVE-4206 URL: https://issues.apache.org/jira/browse/HIVE-4206 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.4206.1.patch, hive.4206.2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4196) Support for Streaming Partitions in Hive
[ https://issues.apache.org/jira/browse/HIVE-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13608800#comment-13608800 ] eric baldeschwieler commented on HIVE-4196: --- Maybe we should just return both? Support for Streaming Partitions in Hive Key: HIVE-4196 URL: https://issues.apache.org/jira/browse/HIVE-4196 Project: Hive Issue Type: New Feature Components: Database/Schema, HCatalog Affects Versions: 0.10.1 Reporter: Roshan Naik Assignee: Roshan Naik Motivation: Allow Hive users to immediately query data streaming in through clients such as Flume. Currently Hive partitions must be created after all the data for the partition is available. Thereafter, data in the partitions is considered immutable. This proposal introduces the notion of a streaming partition into which new files an be committed periodically and made available for queries before the partition is closed and converted into a standard partition. The admin enables streaming partition on a table using DDL. He provides the following pieces of information: - Name of the partition in the table on which streaming is enabled - Frequency at which the streaming partition should be closed and converted into a standard partition. Tables with streaming partition enabled will be partitioned by one and only one column. It is assumed that this column will contain a timestamp. Closing the current streaming partition converts it into a standard partition. Based on the specified frequency, the current streaming partition is closed and a new one created for future writes. This is referred to as 'rolling the partition'. A streaming partition's life cycle is as follows: - A new streaming partition is instantiated for writes - Streaming clients request (via webhcat) for a HDFS file name into which they can write a chunk of records for a specific table. - Streaming clients write a chunk (via webhdfs) to that file and commit it(via webhcat). Committing merely indicates that the chunk has been written completely and ready for serving queries. - When the partition is rolled, all committed chunks are swept into single directory and a standard partition pointing to that directory is created. The streaming partition is closed and new streaming partition is created. Rolling the partition is atomic. Streaming clients are agnostic of partition rolling. - Hive queries will be able to query the partition that is currently open for streaming. only committed chunks will be visible. read consistency will be ensured so that repeated reads of the same partition will be idempotent for the lifespan of the query. Partition rolling requires an active agent/thread running to check when it is time to roll and trigger the roll. This could be either be achieved by using an external agent such as Oozie (preferably) or an internal agent. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4211) Common column and partition column are defined the same type and union them, it will hints Schema of both sides of union should match.
Daisy.Yuan created HIVE-4211: Summary: Common column and partition column are defined the same type and union them, it will hints Schema of both sides of union should match. Key: HIVE-4211 URL: https://issues.apache.org/jira/browse/HIVE-4211 Project: Hive Issue Type: Bug Components: SQL Affects Versions: 0.9.0, 0.11.0 Reporter: Daisy.Yuan create table UnionBoolA (id boolean, no boolean) row format delimited fields terminated by ' '; load data local inpath '/opt/files/unionboola.txt' into table UnionBoolA; create table UnionPartionBool (id int) partitioned by (no boolean) row format delimited fields terminated by ' '; load data local inpath '/opt/files/unionpartint.txt' into table UnionPartionBool partition(no=true); unionboola.txt: true true false true true true false true unionpartint.txt: 111 444 1122 44 when I execute select * from( select no from UnionBoolA union all select no from UnionPartionBool) unionResult, it is failed. The exception info is as follows: FAILED: Error in semantic analysis: 1:66 Schema of both sides of union should match: Column no is of type boolean on first table and type string on second table. Error encountered near token 'UnionPartionBool' org.apache.hadoop.hive.ql.parse.SemanticException: 1:66 Schema of both sides of union should match: Column no is of type boolean on first table and type string on second table. Error encountered near token 'UnionPartionBool' at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genUnionPlan(SemanticAnalyzer.java:6295) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6733) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6748) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:7556) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:244) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:621) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:525) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1153) at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:226) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:630) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:618) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge20S.java:535) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge20S.java:532) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge20S.java:532) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) So I execute explain select no from UnionPartionBool to see the partition column, and find the partition column type is string. And all the partition column type is changed to be TypeInfoFactory.stringTypeInfo. It is in org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genTablePlan(). And it is todo task. Now I modify it to be TypeInfoFactory.getPrimitiveTypeInfo(part_col.getType()).It can fix this bug. And you can see what I modified in the patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4211) Common column and partition column are defined the same type and union them, it will hints Schema of both sides of union should match.
[ https://issues.apache.org/jira/browse/HIVE-4211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daisy.Yuan updated HIVE-4211: - Status: Patch Available (was: Open) Common column and partition column are defined the same type and union them, it will hints Schema of both sides of union should match. --- Key: HIVE-4211 URL: https://issues.apache.org/jira/browse/HIVE-4211 Project: Hive Issue Type: Bug Components: SQL Affects Versions: 0.9.0, 0.11.0 Reporter: Daisy.Yuan Labels: patch Attachments: PartitionColumnTypInfo.patch create table UnionBoolA (id boolean, no boolean) row format delimited fields terminated by ' '; load data local inpath '/opt/files/unionboola.txt' into table UnionBoolA; create table UnionPartionBool (id int) partitioned by (no boolean) row format delimited fields terminated by ' '; load data local inpath '/opt/files/unionpartint.txt' into table UnionPartionBool partition(no=true); unionboola.txt: true true false true true true false true unionpartint.txt: 111 444 1122 44 when I execute select * from( select no from UnionBoolA union all select no from UnionPartionBool) unionResult, it is failed. The exception info is as follows: FAILED: Error in semantic analysis: 1:66 Schema of both sides of union should match: Column no is of type boolean on first table and type string on second table. Error encountered near token 'UnionPartionBool' org.apache.hadoop.hive.ql.parse.SemanticException: 1:66 Schema of both sides of union should match: Column no is of type boolean on first table and type string on second table. Error encountered near token 'UnionPartionBool' at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genUnionPlan(SemanticAnalyzer.java:6295) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6733) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6748) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:7556) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:244) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:621) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:525) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1153) at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:226) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:630) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:618) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge20S.java:535) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge20S.java:532) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge20S.java:532) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) So I execute explain select no from UnionPartionBool to see the partition column, and find the partition column type is string. And all the partition column type is changed to be TypeInfoFactory.stringTypeInfo. It is in org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genTablePlan(). And it is todo task. Now I modify it to be TypeInfoFactory.getPrimitiveTypeInfo(part_col.getType()).It can fix this bug. And you can see what I modified in the patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4211) Common column and partition column are defined the same type and union them, it will hints Schema of both sides of union should match.
[ https://issues.apache.org/jira/browse/HIVE-4211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daisy.Yuan updated HIVE-4211: - Attachment: PartitionColumnTypInfo.patch Common column and partition column are defined the same type and union them, it will hints Schema of both sides of union should match. --- Key: HIVE-4211 URL: https://issues.apache.org/jira/browse/HIVE-4211 Project: Hive Issue Type: Bug Components: SQL Affects Versions: 0.9.0, 0.11.0 Reporter: Daisy.Yuan Labels: patch Attachments: PartitionColumnTypInfo.patch create table UnionBoolA (id boolean, no boolean) row format delimited fields terminated by ' '; load data local inpath '/opt/files/unionboola.txt' into table UnionBoolA; create table UnionPartionBool (id int) partitioned by (no boolean) row format delimited fields terminated by ' '; load data local inpath '/opt/files/unionpartint.txt' into table UnionPartionBool partition(no=true); unionboola.txt: true true false true true true false true unionpartint.txt: 111 444 1122 44 when I execute select * from( select no from UnionBoolA union all select no from UnionPartionBool) unionResult, it is failed. The exception info is as follows: FAILED: Error in semantic analysis: 1:66 Schema of both sides of union should match: Column no is of type boolean on first table and type string on second table. Error encountered near token 'UnionPartionBool' org.apache.hadoop.hive.ql.parse.SemanticException: 1:66 Schema of both sides of union should match: Column no is of type boolean on first table and type string on second table. Error encountered near token 'UnionPartionBool' at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genUnionPlan(SemanticAnalyzer.java:6295) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6733) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6748) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:7556) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:244) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:621) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:525) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1153) at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:226) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:630) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:618) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge20S.java:535) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge20S.java:532) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge20S.java:532) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) So I execute explain select no from UnionPartionBool to see the partition column, and find the partition column type is string. And all the partition column type is changed to be TypeInfoFactory.stringTypeInfo. It is in org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genTablePlan(). And it is todo task. Now I modify it to be TypeInfoFactory.getPrimitiveTypeInfo(part_col.getType()).It can fix this bug. And you can see what I modified in the patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4206) Sort merge join does not work for outer joins for 7 inputs
[ https://issues.apache.org/jira/browse/HIVE-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4206: - Status: Patch Available (was: Open) Sort merge join does not work for outer joins for 7 inputs -- Key: HIVE-4206 URL: https://issues.apache.org/jira/browse/HIVE-4206 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.4206.1.patch, hive.4206.2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4212) sort merge join should work for outer joins for more than 8 inputs
Namit Jain created HIVE-4212: Summary: sort merge join should work for outer joins for more than 8 inputs Key: HIVE-4212 URL: https://issues.apache.org/jira/browse/HIVE-4212 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Namit Jain -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4212) sort merge join should work for outer joins for more than 8 inputs
[ https://issues.apache.org/jira/browse/HIVE-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4212: - Component/s: Query Processor sort merge join should work for outer joins for more than 8 inputs -- Key: HIVE-4212 URL: https://issues.apache.org/jira/browse/HIVE-4212 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4212) sort merge join should work for outer joins for more than 8 inputs
[ https://issues.apache.org/jira/browse/HIVE-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4212: - Attachment: hive.4212.1.patch sort merge join should work for outer joins for more than 8 inputs -- Key: HIVE-4212 URL: https://issues.apache.org/jira/browse/HIVE-4212 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.4212.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4148) Cleanup aisle ivy
[ https://issues.apache.org/jira/browse/HIVE-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13608857#comment-13608857 ] Hudson commented on HIVE-4148: -- Integrated in Hive-trunk-h0.21 #2025 (See [https://builds.apache.org/job/Hive-trunk-h0.21/2025/]) HIVE-4187. QL build-grammar target fails after HIVE-4148 (Gunther Hagleitner via cws) (Revision 1459014) Result = FAILURE cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1459014 Files : * /hive/trunk/ivy/libraries.properties * /hive/trunk/metastore/ivy.xml Cleanup aisle ivy --- Key: HIVE-4148 URL: https://issues.apache.org/jira/browse/HIVE-4148 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Fix For: 0.11.0 Attachments: HIVE-4148.patch Lot's of duplicated dependencies in the modules' ivy configs. Makes compiling slow and maintenance hard. This patch cleans up these dependencies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4187) QL build-grammar target fails after HIVE-4148
[ https://issues.apache.org/jira/browse/HIVE-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13608858#comment-13608858 ] Hudson commented on HIVE-4187: -- Integrated in Hive-trunk-h0.21 #2025 (See [https://builds.apache.org/job/Hive-trunk-h0.21/2025/]) HIVE-4187. QL build-grammar target fails after HIVE-4148 (Gunther Hagleitner via cws) (Revision 1459014) Result = FAILURE cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1459014 Files : * /hive/trunk/ivy/libraries.properties * /hive/trunk/metastore/ivy.xml QL build-grammar target fails after HIVE-4148 - Key: HIVE-4187 URL: https://issues.apache.org/jira/browse/HIVE-4187 Project: Hive Issue Type: Bug Components: Build Infrastructure Reporter: Carl Steinbach Assignee: Gunther Hagleitner Priority: Critical Fix For: 0.11.0 Attachments: HIVE-4187.1.patch, HIVE-4187.2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4015) Add ORC file to the grammar as a file format
[ https://issues.apache.org/jira/browse/HIVE-4015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13608859#comment-13608859 ] Hudson commented on HIVE-4015: -- Integrated in Hive-trunk-h0.21 #2025 (See [https://builds.apache.org/job/Hive-trunk-h0.21/2025/]) HIVE-4015. Add ORC file to the grammar as a file format. (Gunther Hagleitner via kevinwilfong) (Revision 1459030) Result = FAILURE kevinwilfong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1459030 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g * /hive/trunk/ql/src/test/queries/clientpositive/orc_create.q * /hive/trunk/ql/src/test/queries/clientpositive/orc_createas1.q * /hive/trunk/ql/src/test/results/clientpositive/orc_create.q.out * /hive/trunk/ql/src/test/results/clientpositive/orc_createas1.q.out Add ORC file to the grammar as a file format Key: HIVE-4015 URL: https://issues.apache.org/jira/browse/HIVE-4015 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Gunther Hagleitner Fix For: 0.11.0 Attachments: HIVE-4015.1.patch, HIVE-4015.2.patch, HIVE-4015.3.patch, HIVE-4015.4.patch, HIVE-4015.5.patch It would be much more convenient for users if we enable them to use ORC as a file format in the HQL grammar. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4139) MiniDFS shim does not work for hadoop 2
[ https://issues.apache.org/jira/browse/HIVE-4139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13608860#comment-13608860 ] Hudson commented on HIVE-4139: -- Integrated in Hive-trunk-h0.21 #2025 (See [https://builds.apache.org/job/Hive-trunk-h0.21/2025/]) HIVE-4139 : MiniDFS shim does not work for hadoop 2 (Gunther Hagleitner via Ashutosh Chauhan) (Revision 1459072) Result = FAILURE hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1459072 Files : * /hive/trunk/build-common.xml * /hive/trunk/build.properties * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java * /hive/trunk/shims/ivy.xml * /hive/trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java * /hive/trunk/shims/src/0.23/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java * /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java MiniDFS shim does not work for hadoop 2 --- Key: HIVE-4139 URL: https://issues.apache.org/jira/browse/HIVE-4139 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Fix For: 0.11.0 Attachments: HIVE-4139.1.patch, HIVE-4139.2.patch, HIVE-4139.3.patch, HIVE-4139.4.patch There's an incompatibility between hadoop 1 2 wrt to the MiniDfsCluster class. That causes the hadoop 2 line Minimr tests to fail with a MethodNotFound exception. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 2025 - Failure
Changes for Build #2012 [hashutosh] HIVE-3862 : testHBaseNegativeCliDriver_cascade_dbdrop fails on hadoop-1 (Gunther Hagleitner via Ashutosh Chauhan) Changes for Build #2013 [kevinwilfong] HIVE-4125. Expose metastore JMX metrics. (Samuel Yuan via kevinwilfong) [hashutosh] HIVE-2935 : Implement HiveServer2 Core code changes (4th patch of 4) (Carl Steinbach and others via Ashutosh Chauhan) [kevinwilfong] HIVE-4096. problem in hive.map.groupby.sorted with distincts. (njain via kevinwilfong) [hashutosh] HIVE-2935 : Implement HiveServer2 Beeline .q.out files (3rd patch of 4) (Carl Steinbach and others via Ashutosh Chauhan) [hashutosh] HIVE-2935 : Implement HiveServer2 Beeline code changes (2nd patch of 4) (Carl Steinbach and others via Ashutosh Chauhan) [hashutosh] HIVE-2935 : Implement HiveServer2 (1st patch of 4) (Carl Steinbach and others via Ashutosh Chauhan) [hashutosh] HIVE-3717 : Hive wont compile with -Dhadoop.mr.rev=20S (Gunther Hagleitner via Ashutosh Chauhan) [hashutosh] HIVE-4148 : Cleanup aisle ivy (Gunther Hagleitner via Ashutosh Chauhan) Changes for Build #2014 Changes for Build #2015 Changes for Build #2016 [ecapriolo] Hive-4141 InspectorFactories use static HashMaps which fail in concurrent modification (Brock Noland via egc) Submitted by: Brock Noland Reviewed by: Edward Capriolo Approved by: Edward Capriolo [kevinwilfong] HIVE-4176. disable TestBeeLineDriver in ptest util. (kevinwilfong reviewed by njain, ashutoshc) [hashutosh] HIVE-4169 : union_remove_*.q fail on hadoop 2 (Gunther Hagleitner via Ashutosh Chauhan) Changes for Build #2017 [cws] HIVE-4145. Create hcatalog stub directory and add it to the build (Carl Steinbach via cws) [kevinwilfong] HIVE-4162. disable TestBeeLineDriver. (Thejas M Nair via kevinwilfong) Changes for Build #2018 Changes for Build #2019 Changes for Build #2020 Changes for Build #2021 Changes for Build #2022 Changes for Build #2023 [namit] HIVE-4154 NPE reading column of empty string from ORC file (Kevin Wilfong via namit) [hashutosh] HIVE-4186 : NPE in ReduceSinkDeDuplication (Harish Butani via Ashutosh Chauhan) Changes for Build #2024 [namit] HIVE-4146 bug with hive.auto.convert.join.noconditionaltask with outer joins (Namit via Gang Tim Liu) Changes for Build #2025 [hashutosh] HIVE-4139 : MiniDFS shim does not work for hadoop 2 (Gunther Hagleitner via Ashutosh Chauhan) [kevinwilfong] HIVE-4015. Add ORC file to the grammar as a file format. (Gunther Hagleitner via kevinwilfong) [cws] HIVE-4187. QL build-grammar target fails after HIVE-4148 (Gunther Hagleitner via cws) 5 tests failed. FAILED: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_create_table_alter Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:47) at org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:5935) at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_create_table_alter(TestCliDriver.java:4416) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:154) at junit.framework.TestCase.runBare(TestCase.java:127) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:118) at junit.framework.TestSuite.runTest(TestSuite.java:208) at junit.framework.TestSuite.run(TestSuite.java:203) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) FAILED: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_create_table_db_table Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:47) at org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:5935) at
[jira] [Updated] (HIVE-3820) Consider creating a literal like D or BD for representing Decimal type constants
[
https://issues.apache.org/jira/browse/HIVE-3820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-3820:
---
Resolution: Fixed
Fix Version/s: 0.11.0
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks, Gunther!
Consider creating a literal like D or BD for representing Decimal type
constants
Key: HIVE-3820
URL: https://issues.apache.org/jira/browse/HIVE-3820
Project: Hive
Issue Type: Bug
Reporter: Mark Grover
Assignee: Gunther Hagleitner
Fix For: 0.11.0
Attachments: HIVE-3820.1.patch, HIVE-3820.2.patch,
HIVE-3820.D8823.1.patch
When the HIVE-2693 gets committed, users are going to see this behavior:
{code}
hive select cast(3.14 as decimal) from decimal_3 limit 1;
3.140124344978758017532527446746826171875
{code}
That's intuitively incorrect but is the case because 3.14 (double) is being
converted to BigDecimal because of which there is a precision mismatch.
We should consider creating a new literal for expressing constants of Decimal
type as Gunther suggested in HIVE-2693.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3348) semi-colon in comments in .q file does not work
[ https://issues.apache.org/jira/browse/HIVE-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Collins updated HIVE-3348: --- Affects Version/s: 0.10.0 Status: Patch Available (was: Open) semi-colon in comments in .q file does not work --- Key: HIVE-3348 URL: https://issues.apache.org/jira/browse/HIVE-3348 Project: Hive Issue Type: Bug Components: Testing Infrastructure Affects Versions: 0.10.0 Reporter: Namit Jain Assignee: Namit Jain -- comment ; -- comment select count(1) from src; The above test file fails -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3348) semi-colon in comments in .q file does not work
[ https://issues.apache.org/jira/browse/HIVE-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Collins updated HIVE-3348: --- Affects Version/s: (was: 0.10.0) semi-colon in comments in .q file does not work --- Key: HIVE-3348 URL: https://issues.apache.org/jira/browse/HIVE-3348 Project: Hive Issue Type: Bug Components: Testing Infrastructure Reporter: Namit Jain Assignee: Namit Jain Attachments: semicolon.patch -- comment ; -- comment select count(1) from src; The above test file fails -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3348) semi-colon in comments in .q file does not work
[ https://issues.apache.org/jira/browse/HIVE-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Collins updated HIVE-3348: --- Attachment: semicolon.patch semi-colon in comments in .q file does not work --- Key: HIVE-3348 URL: https://issues.apache.org/jira/browse/HIVE-3348 Project: Hive Issue Type: Bug Components: Testing Infrastructure Reporter: Namit Jain Assignee: Namit Jain Attachments: semicolon.patch -- comment ; -- comment select count(1) from src; The above test file fails -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3348) semi-colon in comments in .q file does not work
[ https://issues.apache.org/jira/browse/HIVE-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Collins updated HIVE-3348: --- Attachment: (was: semicolon.patch) semi-colon in comments in .q file does not work --- Key: HIVE-3348 URL: https://issues.apache.org/jira/browse/HIVE-3348 Project: Hive Issue Type: Bug Components: Testing Infrastructure Reporter: Namit Jain Assignee: Namit Jain Attachments: hive-3348.patch -- comment ; -- comment select count(1) from src; The above test file fails -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3348) semi-colon in comments in .q file does not work
[ https://issues.apache.org/jira/browse/HIVE-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Collins updated HIVE-3348: --- Attachment: hive-3348.patch semi-colon in comments in .q file does not work --- Key: HIVE-3348 URL: https://issues.apache.org/jira/browse/HIVE-3348 Project: Hive Issue Type: Bug Components: Testing Infrastructure Reporter: Namit Jain Assignee: Namit Jain Attachments: hive-3348.patch -- comment ; -- comment select count(1) from src; The above test file fails -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-4180) Filter getting dropped with PTFOperator
[ https://issues.apache.org/jira/browse/HIVE-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan resolved HIVE-4180. Resolution: Fixed Committed to branch. Thanks, Harish for review! Filter getting dropped with PTFOperator --- Key: HIVE-4180 URL: https://issues.apache.org/jira/browse/HIVE-4180 Project: Hive Issue Type: Bug Components: PTF-Windowing Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Another case where filter push down was not handled correctly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4206) Sort merge join does not work for outer joins for 7 inputs
[ https://issues.apache.org/jira/browse/HIVE-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609121#comment-13609121 ] Gang Tim Liu commented on HIVE-4206: +1 Sort merge join does not work for outer joins for 7 inputs -- Key: HIVE-4206 URL: https://issues.apache.org/jira/browse/HIVE-4206 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.4206.1.patch, hive.4206.2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4188) TestJdbcDriver2.testDescribeTable failing consistently
[ https://issues.apache.org/jira/browse/HIVE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-4188: Resolution: Fixed Fix Version/s: 0.11.0 Status: Resolved (was: Patch Available) Committed, thanks Prasad. TestJdbcDriver2.testDescribeTable failing consistently -- Key: HIVE-4188 URL: https://issues.apache.org/jira/browse/HIVE-4188 Project: Hive Issue Type: Bug Components: HiveServer2, Tests Affects Versions: 0.11.0 Reporter: Kevin Wilfong Assignee: Prasad Mujumdar Fix For: 0.11.0 Attachments: HIVE-4188-1.patch, HIVE-4188-2.patch Running in Linux on a clean checkout after running ant very-clean package, the test TestJdbcDriver2.testDescribeTable fails consistently with Column name 'under_col' not found expected:under_col but was:# col_name junit.framework.ComparisonFailure: Column name 'under_col' not found expected:under_col but was:# col_name at junit.framework.Assert.assertEquals(Assert.java:81) at org.apache.hive.jdbc.TestJdbcDriver2.testDescribeTable(TestJdbcDriver2.java:815) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:154) at junit.framework.TestCase.runBare(TestCase.java:127) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:118) at junit.framework.TestSuite.runTest(TestSuite.java:208) at junit.framework.TestSuite.run(TestSuite.java:203) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4206) Sort merge join does not work for outer joins for 7 inputs
[ https://issues.apache.org/jira/browse/HIVE-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4206: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed. Thanks Tim Sort merge join does not work for outer joins for 7 inputs -- Key: HIVE-4206 URL: https://issues.apache.org/jira/browse/HIVE-4206 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.4206.1.patch, hive.4206.2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4206) Sort merge join does not work for outer joins for 7 inputs
[ https://issues.apache.org/jira/browse/HIVE-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4206: - Affects Version/s: 0.11.0 Sort merge join does not work for outer joins for 7 inputs -- Key: HIVE-4206 URL: https://issues.apache.org/jira/browse/HIVE-4206 Project: Hive Issue Type: Improvement Components: Query Processor Affects Versions: 0.11.0 Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.4206.1.patch, hive.4206.2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4193) OVER clauses with BETWEEN in the window definition produce wrong results
[
https://issues.apache.org/jira/browse/HIVE-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-4193:
-
Attachment: Range_5_diff
A diff from last night's run for the query:
select i, f, avg(d) over (partition by i order by f desc rows between 5
preceding and 5 following) from over100k;
Based on the output this doesn't look like a sort stability issue.
OVER clauses with BETWEEN in the window definition produce wrong results
Key: HIVE-4193
URL: https://issues.apache.org/jira/browse/HIVE-4193
Project: Hive
Issue Type: Bug
Components: PTF-Windowing
Affects Versions: 0.11.0
Reporter: Alan Gates
Attachments: Range_5_diff
Window queries that define a windowing clause that has a termination row
often (though not all) return incorrect results. For example, from our test
queries all of the following return incorrect results:
{code}
select s, sum(f) over (partition by t order by b
rows between current row and unbounded following)
from over100k;
select s, avg(f) over (partition by b order by d
rows between 5 preceding and current row)
from over100k;
select s, avg(f) over (partition by bin order by s
rows between current row and 5 following)
from over100k;
select s, avg(d) over (partition by i order by f desc
rows between 5 preceding and 5 following)
from over100k;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4212) sort merge join should work for outer joins for more than 8 inputs
[ https://issues.apache.org/jira/browse/HIVE-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609144#comment-13609144 ] Namit Jain commented on HIVE-4212: -- https://reviews.facebook.net/D9615 sort merge join should work for outer joins for more than 8 inputs -- Key: HIVE-4212 URL: https://issues.apache.org/jira/browse/HIVE-4212 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.4212.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4193) OVER clauses with BETWEEN in the window definition produce wrong results
[
https://issues.apache.org/jira/browse/HIVE-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609148#comment-13609148
]
Alan Gates commented on HIVE-4193:
--
Note, other than the query listed in the above comments all of the other
queries using ROWS BETWEEN now produce correct results.
OVER clauses with BETWEEN in the window definition produce wrong results
Key: HIVE-4193
URL: https://issues.apache.org/jira/browse/HIVE-4193
Project: Hive
Issue Type: Bug
Components: PTF-Windowing
Affects Versions: 0.11.0
Reporter: Alan Gates
Attachments: Range_5_diff
Window queries that define a windowing clause that has a termination row
often (though not all) return incorrect results. For example, from our test
queries all of the following return incorrect results:
{code}
select s, sum(f) over (partition by t order by b
rows between current row and unbounded following)
from over100k;
select s, avg(f) over (partition by b order by d
rows between 5 preceding and current row)
from over100k;
select s, avg(f) over (partition by bin order by s
rows between current row and 5 following)
from over100k;
select s, avg(d) over (partition by i order by f desc
rows between 5 preceding and 5 following)
from over100k;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4193) OVER clauses with BETWEEN in the window definition produce wrong results
[
https://issues.apache.org/jira/browse/HIVE-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609156#comment-13609156
]
Ashutosh Chauhan commented on HIVE-4193:
One way this query is different from others is it has desc. Alan, can you run
this query without desc and see if it still produces wrong results. If not,
likely we have a bug in order by desc.
OVER clauses with BETWEEN in the window definition produce wrong results
Key: HIVE-4193
URL: https://issues.apache.org/jira/browse/HIVE-4193
Project: Hive
Issue Type: Bug
Components: PTF-Windowing
Affects Versions: 0.11.0
Reporter: Alan Gates
Attachments: Range_5_diff
Window queries that define a windowing clause that has a termination row
often (though not all) return incorrect results. For example, from our test
queries all of the following return incorrect results:
{code}
select s, sum(f) over (partition by t order by b
rows between current row and unbounded following)
from over100k;
select s, avg(f) over (partition by b order by d
rows between 5 preceding and current row)
from over100k;
select s, avg(f) over (partition by bin order by s
rows between current row and 5 following)
from over100k;
select s, avg(d) over (partition by i order by f desc
rows between 5 preceding and 5 following)
from over100k;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-4193) OVER clauses with BETWEEN in the window definition produce wrong results
[
https://issues.apache.org/jira/browse/HIVE-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates resolved HIVE-4193.
--
Resolution: Invalid
The desc definitely causes an issue when you forget to put it into the
verification query. Once you do that all is good. I'm closing this as invalid
because the rest of the queries work now (the result of other patches checked
in I assume) and this last one was a test error.
OVER clauses with BETWEEN in the window definition produce wrong results
Key: HIVE-4193
URL: https://issues.apache.org/jira/browse/HIVE-4193
Project: Hive
Issue Type: Bug
Components: PTF-Windowing
Affects Versions: 0.11.0
Reporter: Alan Gates
Attachments: Range_5_diff
Window queries that define a windowing clause that has a termination row
often (though not all) return incorrect results. For example, from our test
queries all of the following return incorrect results:
{code}
select s, sum(f) over (partition by t order by b
rows between current row and unbounded following)
from over100k;
select s, avg(f) over (partition by b order by d
rows between 5 preceding and current row)
from over100k;
select s, avg(f) over (partition by bin order by s
rows between current row and 5 following)
from over100k;
select s, avg(d) over (partition by i order by f desc
rows between 5 preceding and 5 following)
from over100k;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: HiveServer2 released in some branch
Hi Thejas, thank you for the explanation. So, HiveServer2 is going to be part of Hive 0.11, and I have read that HCatalog is going to be merged in version 0.11 too. What is the target date for Hive 0.11 ? I searched this information or a schedule, but did not find. Do you, or someone, know this information ? Or can point me to some schedule page, etc. Regards, Rodrigo On 03/20/2013 05:03 PM, Thejas Nair wrote: Rodrigo, New features usually go into new version (0.xx) . (0.xx.x versions are usually meant for releases with bug fixes). So HiveServer2 is likely to be part of next release (0.11), and not part of a (0.9.x or 0.10.x) . This is the usual practice that I have seen, and haven't heard of any plans to contrary. -Thejas (not a hive PMC member/committer) On 3/20/13 6:14 AM, Rodrigo Trujillo wrote: Hi all, I see that the patches for HiveServer2 are committed in the trunk. Any idea whether this feature is going to be released in current branches ( 0.9.x or 0.10.x ) ? Best regards, Rodrigo Trujillo
[jira] [Created] (HIVE-4213) List bucketing error too restrictive
Mark Grover created HIVE-4213:
-
Summary: List bucketing error too restrictive
Key: HIVE-4213
URL: https://issues.apache.org/jira/browse/HIVE-4213
Project: Hive
Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Mark Grover
Fix For: 0.11.0
With the introduction of List bucketing, we introduced a config validation step
where we say:
{code}
SUPPORT_DIR_MUST_TRUE_FOR_LIST_BUCKETING(
10199,
hive.mapred.supports.subdirectories must be true
+ if any one of following is true:
hive.internal.ddl.list.bucketing.enable,
+ hive.optimize.listbucketing and mapred.input.dir.recursive),
{code}
This seems overly restrictive to because there are use cases where people may
want to use {{mapred.input.dir.recursive}} to {{true}} even when they don't
care about list bucketing.
Is that not true?
For example, here is the unit test code for {{clientpositive/recursive_dir.q}}
{code}
CREATE TABLE fact_daily(x int) PARTITIONED BY (ds STRING);
CREATE TABLE fact_tz(x int) PARTITIONED BY (ds STRING, hr STRING)
LOCATION 'pfile:${system:test.tmp.dir}/fact_tz';
INSERT OVERWRITE TABLE fact_tz PARTITION (ds='1', hr='1')
SELECT key+11 FROM src WHERE key=484;
ALTER TABLE fact_daily SET TBLPROPERTIES('EXTERNAL'='TRUE');
ALTER TABLE fact_daily ADD PARTITION (ds='1')
LOCATION 'pfile:${system:test.tmp.dir}/fact_tz/ds=1';
set hive.mapred.supports.subdirectories=true;
set mapred.input.dir.recursive=true;
set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;
SELECT * FROM fact_daily WHERE ds='1';
SELECT count(1) FROM fact_daily WHERE ds='1';
{code}
The unit test doesn't seem to be concerned about list bucketing but wants to
set {{mapred.input.dir.recursive}} to {{true}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4213) List bucketing error too restrictive
[
https://issues.apache.org/jira/browse/HIVE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609262#comment-13609262
]
Mark Grover commented on HIVE-4213:
---
[~gangtimliu] I would love to hear your thoughts on this! Thanks in advance!
List bucketing error too restrictive
Key: HIVE-4213
URL: https://issues.apache.org/jira/browse/HIVE-4213
Project: Hive
Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Mark Grover
Fix For: 0.11.0
With the introduction of List bucketing, we introduced a config validation
step where we say:
{code}
SUPPORT_DIR_MUST_TRUE_FOR_LIST_BUCKETING(
10199,
hive.mapred.supports.subdirectories must be true
+ if any one of following is true:
hive.internal.ddl.list.bucketing.enable,
+ hive.optimize.listbucketing and mapred.input.dir.recursive),
{code}
This seems overly restrictive to because there are use cases where people may
want to use {{mapred.input.dir.recursive}} to {{true}} even when they don't
care about list bucketing.
Is that not true?
For example, here is the unit test code for {{clientpositive/recursive_dir.q}}
{code}
CREATE TABLE fact_daily(x int) PARTITIONED BY (ds STRING);
CREATE TABLE fact_tz(x int) PARTITIONED BY (ds STRING, hr STRING)
LOCATION 'pfile:${system:test.tmp.dir}/fact_tz';
INSERT OVERWRITE TABLE fact_tz PARTITION (ds='1', hr='1')
SELECT key+11 FROM src WHERE key=484;
ALTER TABLE fact_daily SET TBLPROPERTIES('EXTERNAL'='TRUE');
ALTER TABLE fact_daily ADD PARTITION (ds='1')
LOCATION 'pfile:${system:test.tmp.dir}/fact_tz/ds=1';
set hive.mapred.supports.subdirectories=true;
set mapred.input.dir.recursive=true;
set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;
SELECT * FROM fact_daily WHERE ds='1';
SELECT count(1) FROM fact_daily WHERE ds='1';
{code}
The unit test doesn't seem to be concerned about list bucketing but wants to
set {{mapred.input.dir.recursive}} to {{true}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4070) Like operator in Hive is case sensitive while in MySQL (and most likely other DBs) it's case insensitive
[ https://issues.apache.org/jira/browse/HIVE-4070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609278#comment-13609278 ] Mark Grover commented on HIVE-4070: --- Thanks Gwen. In that case, I am ok with documenting it and resolving this JIRA as won't fix. Is that ok with you as well, [~mackrorysd]? Like operator in Hive is case sensitive while in MySQL (and most likely other DBs) it's case insensitive Key: HIVE-4070 URL: https://issues.apache.org/jira/browse/HIVE-4070 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.10.0 Reporter: Mark Grover Assignee: Mark Grover Priority: Trivial Fix For: 0.11.0 Hive's like operator seems to be case sensitive. See https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLike.java#L164 However, MySQL's like operator is case insensitive. I don't have other DB's (like PostgreSQL) installed and handy but I am guessing their LIKE is case insensitive as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HIVE-4180) Filter getting dropped with PTFOperator
[ https://issues.apache.org/jira/browse/HIVE-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan reopened HIVE-4180: Previous patch was too restrictive, it was leaving the filter in its original location. Whereas what we want is to filter to come up to PTFOp and don't get pushed beyond PTFOp. This is exactly what ScriptPPD does, so we can reuse it. LimitPPD also uses ScriptPPD for similar reason. Filter getting dropped with PTFOperator --- Key: HIVE-4180 URL: https://issues.apache.org/jira/browse/HIVE-4180 Project: Hive Issue Type: Bug Components: PTF-Windowing Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Another case where filter push down was not handled correctly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4214) OVER accepts general expression instead of just function
Alan Gates created HIVE-4214: Summary: OVER accepts general expression instead of just function Key: HIVE-4214 URL: https://issues.apache.org/jira/browse/HIVE-4214 Project: Hive Issue Type: Bug Components: PTF-Windowing Affects Versions: 0.11.0 Reporter: Alan Gates The query: select s, i, avg(d) / 10.0 over (partition by s order by i) from over100k; runs (and produces meaningless output). Over should not allow the arithmetic expression. Only a UDAF or PTF function should be valid there. The correct way to write this query should be select s, i, avg(d) over (partition by s order by i) / 10. 0 from over100k; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4214) OVER accepts general expression instead of just function
[
https://issues.apache.org/jira/browse/HIVE-4214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609307#comment-13609307
]
Alan Gates commented on HIVE-4214:
--
When the query is re-written to properly put the / 10.0 after the OVER, it
fails with:
{code}
NoViableAltException(15@[129:7: ( ( ( KW_AS )? identifier ) | ( KW_AS LPAREN
identifier ( COMMA identifier )* RPAREN ) )?])
{code}
OVER accepts general expression instead of just function
Key: HIVE-4214
URL: https://issues.apache.org/jira/browse/HIVE-4214
Project: Hive
Issue Type: Bug
Components: PTF-Windowing
Affects Versions: 0.11.0
Reporter: Alan Gates
The query:
select s, i, avg(d) / 10.0 over (partition by s order by i) from over100k;
runs (and produces meaningless output).
Over should not allow the arithmetic expression. Only a UDAF or PTF function
should be valid there. The correct way to write this query should be
select s, i, avg(d) over (partition by s order by i) / 10. 0 from over100k;
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4180) Filter getting dropped with PTFOperator
[
https://issues.apache.org/jira/browse/HIVE-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609318#comment-13609318
]
Ashutosh Chauhan commented on HIVE-4180:
Easier to explain via an example. Consider following query:
{noformat}
select s_store_name,
i_product_name,
rnk
from (
select ss_store_sk,
ss_item_sk,
rank() over (partition by ss_store_sk order by item_total desc) as
rnk
from (
select ss_store_sk,
ss_item_sk,
sum(ss_sales_price) as item_total
from store_sales
join date_dim on (store_sales.ss_sold_date_sk = date_dim.d_date_sk)
where date_dim.d_year = 2000
and date_dim.d_moy = 12
and date_dim.d_dom = 24
group by ss_store_sk,
ss_item_sk
) item_sales
) item_rank
join item on (item.i_item_sk = item_rank.ss_item_sk)
join store on (item_rank.ss_store_sk = store.s_store_sk)
where rnk = 3;
{noformat}
Running explain before and after latest patch will prove this point. With the
previous patch, filter (rnk = 3) will not be moved and will be applied last
after the outermost joins, whereas what we really want is this filter to get
pushed up till PTFOperator and applied right after it. Latest patch gets us
that plan.
Filter getting dropped with PTFOperator
---
Key: HIVE-4180
URL: https://issues.apache.org/jira/browse/HIVE-4180
Project: Hive
Issue Type: Bug
Components: PTF-Windowing
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
Another case where filter push down was not handled correctly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4215) Refactor ppd.OpProcFactory
Ashutosh Chauhan created HIVE-4215: -- Summary: Refactor ppd.OpProcFactory Key: HIVE-4215 URL: https://issues.apache.org/jira/browse/HIVE-4215 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Ashutosh Chauhan Priority: Minor I was working on predicate pushdown recently and found code was bit hard to follow. Refactored to get rid of all compiler warnings and easier to read. No, semantic changes in code itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4215) Refactor ppd.OpProcFactory
[ https://issues.apache.org/jira/browse/HIVE-4215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-4215: --- Assignee: Ashutosh Chauhan Refactor ppd.OpProcFactory -- Key: HIVE-4215 URL: https://issues.apache.org/jira/browse/HIVE-4215 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Priority: Minor I was working on predicate pushdown recently and found code was bit hard to follow. Refactored to get rid of all compiler warnings and easier to read. No, semantic changes in code itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4215) Refactor ppd.OpProcFactory
[ https://issues.apache.org/jira/browse/HIVE-4215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609343#comment-13609343 ] Ashutosh Chauhan commented on HIVE-4215: https://reviews.facebook.net/D9639 Refactor ppd.OpProcFactory -- Key: HIVE-4215 URL: https://issues.apache.org/jira/browse/HIVE-4215 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Priority: Minor I was working on predicate pushdown recently and found code was bit hard to follow. Refactored to get rid of all compiler warnings and easier to read. No, semantic changes in code itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4215) Refactor ppd.OpProcFactory
[ https://issues.apache.org/jira/browse/HIVE-4215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609367#comment-13609367 ] Ashutosh Chauhan commented on HIVE-4215: Saw some failures while running tests. Investigating. Refactor ppd.OpProcFactory -- Key: HIVE-4215 URL: https://issues.apache.org/jira/browse/HIVE-4215 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Priority: Minor I was working on predicate pushdown recently and found code was bit hard to follow. Refactored to get rid of all compiler warnings and easier to read. No, semantic changes in code itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4216) TestHBaseMinimrCliDriver throws wierd error with HBase 0.94.5 and Hadoop 23 and test is stuck infinitely
Viraj Bhat created HIVE-4216:
Summary: TestHBaseMinimrCliDriver throws wierd error with HBase
0.94.5 and Hadoop 23 and test is stuck infinitely
Key: HIVE-4216
URL: https://issues.apache.org/jira/browse/HIVE-4216
Project: Hive
Issue Type: Bug
Components: StorageHandler
Affects Versions: 0.9.0
Reporter: Viraj Bhat
After upgrading to Hadoop 23 and HBase 0.94.5 compiled for Hadoop 23. The
TestHBaseMinimrCliDriver, fails after performing the following steps
Update hbase_bulk.m with the following properties
set mapreduce.totalorderpartitioner.naturalorder=false;
set mapreduce.totalorderpartitioner.path=/tmp/hbpartition.lst;
Otherwise I keep seeing: _partition.lst not found exception in the mappers,
even though set total.order.partitioner.path=/tmp/hbpartition.lst is set.
When the test runs, the 3 reducer phase of the second query fails with the
following error, but the MiniMRCluster keeps spinning up new reducer and the
test is stuck infinitely.
{code}
insert overwrite table hbsort
select distinct value,
case when key=103 then cast(null as string) else key end,
case when key=103 then ''
else cast(key+1 as string) end
from src
cluster by value;
{code}
The stack trace I see in the syslog for the Node Manager is the following:
{quote}
13-03-20 16:26:48,942 FATAL [IPC Server handler 17 on 55996]
org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
attempt_1363821864968_0003_r_02_0 - exited : java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row (tag=0)
{key:{reducesinkkey0:val_200},value:{_col0:val_200,_col1:200,_col2:201.0},alias:0}
at
org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:268)
at
org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:448)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:399)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:157)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:152)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
while processing row (tag=0)
{key:{reducesinkkey0:val_200},value:{_col0:val_200,_col1:200,_col2:201.0},alias:0}
at
org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:256)
... 7 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveRecordWriter(HiveFileFormatUtils.java:237)
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:477)
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:525)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.ExtractOperator.processOp(ExtractOperator.java:45)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at
org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:247)
... 7 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.mapreduce.TaskID$CharTaskTypeMaps.getRepresentingCharacter(TaskID.java:265)
at org.apache.hadoop.mapreduce.TaskID.appendTo(TaskID.java:153)
at
org.apache.hadoop.mapreduce.TaskAttemptID.appendTo(TaskAttemptID.java:119)
at
org.apache.hadoop.mapreduce.TaskAttemptID.toString(TaskAttemptID.java:151)
at java.lang.String.valueOf(String.java:2826)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.init(FileOutputCommitter.java:69)
at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.getRecordWriter(HFileOutputFormat.java:90)
at
org.apache.hadoop.hive.hbase.HiveHFileOutputFormat.getFileWriter(HiveHFileOutputFormat.java:67)
at
org.apache.hadoop.hive.hbase.HiveHFileOutputFormat.getHiveRecordWriter(HiveHFileOutputFormat.java:104)
at
org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getRecordWriter(HiveFileFormatUtils.java:246)
at
org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveRecordWriter(HiveFileFormatUtils.java:234)
... 14 more
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4216) TestHBaseMinimrCliDriver throws weird error with HBase 0.94.5 and Hadoop 23 and test is stuck infinitely
[
https://issues.apache.org/jira/browse/HIVE-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Bhat updated HIVE-4216:
-
Summary: TestHBaseMinimrCliDriver throws weird error with HBase 0.94.5 and
Hadoop 23 and test is stuck infinitely (was: TestHBaseMinimrCliDriver throws
wierd error with HBase 0.94.5 and Hadoop 23 and test is stuck infinitely)
TestHBaseMinimrCliDriver throws weird error with HBase 0.94.5 and Hadoop 23
and test is stuck infinitely
Key: HIVE-4216
URL: https://issues.apache.org/jira/browse/HIVE-4216
Project: Hive
Issue Type: Bug
Components: StorageHandler
Affects Versions: 0.9.0
Reporter: Viraj Bhat
After upgrading to Hadoop 23 and HBase 0.94.5 compiled for Hadoop 23. The
TestHBaseMinimrCliDriver, fails after performing the following steps
Update hbase_bulk.m with the following properties
set mapreduce.totalorderpartitioner.naturalorder=false;
set mapreduce.totalorderpartitioner.path=/tmp/hbpartition.lst;
Otherwise I keep seeing: _partition.lst not found exception in the mappers,
even though set total.order.partitioner.path=/tmp/hbpartition.lst is set.
When the test runs, the 3 reducer phase of the second query fails with the
following error, but the MiniMRCluster keeps spinning up new reducer and the
test is stuck infinitely.
{code}
insert overwrite table hbsort
select distinct value,
case when key=103 then cast(null as string) else key end,
case when key=103 then ''
else cast(key+1 as string) end
from src
cluster by value;
{code}
The stack trace I see in the syslog for the Node Manager is the following:
{quote}
13-03-20 16:26:48,942 FATAL [IPC Server handler 17 on 55996]
org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
attempt_1363821864968_0003_r_02_0 - exited : java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row (tag=0)
{key:{reducesinkkey0:val_200},value:{_col0:val_200,_col1:200,_col2:201.0},alias:0}
at
org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:268)
at
org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:448)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:399)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:157)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:152)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row (tag=0)
{key:{reducesinkkey0:val_200},value:{_col0:val_200,_col1:200,_col2:201.0},alias:0}
at
org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:256)
... 7 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveRecordWriter(HiveFileFormatUtils.java:237)
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:477)
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:525)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.ExtractOperator.processOp(ExtractOperator.java:45)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at
org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:247)
... 7 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.mapreduce.TaskID$CharTaskTypeMaps.getRepresentingCharacter(TaskID.java:265)
at org.apache.hadoop.mapreduce.TaskID.appendTo(TaskID.java:153)
at
org.apache.hadoop.mapreduce.TaskAttemptID.appendTo(TaskAttemptID.java:119)
at
org.apache.hadoop.mapreduce.TaskAttemptID.toString(TaskAttemptID.java:151)
at java.lang.String.valueOf(String.java:2826)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.init(FileOutputCommitter.java:69)
at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.getRecordWriter(HFileOutputFormat.java:90)
at
org.apache.hadoop.hive.hbase.HiveHFileOutputFormat.getFileWriter(HiveHFileOutputFormat.java:67)
at
[jira] [Updated] (HIVE-4216) TestHBaseMinimrCliDriver throws weird error with HBase 0.94.5 and Hadoop 23 and test is stuck infinitely
[
https://issues.apache.org/jira/browse/HIVE-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Bhat updated HIVE-4216:
-
Description:
After upgrading to Hadoop 23 and HBase 0.94.5 compiled for Hadoop 23. The
TestHBaseMinimrCliDriver, fails after performing the following steps
Update hbase_bulk.m with the following properties
set mapreduce.totalorderpartitioner.naturalorder=false;
set mapreduce.totalorderpartitioner.path=/tmp/hbpartition.lst;
Otherwise I keep seeing: _partition.lst not found exception in the mappers,
even though set total.order.partitioner.path=/tmp/hbpartition.lst is set.
When the test runs, the 3 reducer phase of the second query fails with the
following error, but the MiniMRCluster keeps spinning up new reducer and the
test is stuck infinitely.
{code}
insert overwrite table hbsort
select distinct value,
case when key=103 then cast(null as string) else key end,
case when key=103 then ''
else cast(key+1 as string) end
from src
cluster by value;
{code}
The stack trace I see in the syslog for the Node Manager is the following:
==
13-03-20 16:26:48,942 FATAL [IPC Server handler 17 on 55996]
org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
attempt_1363821864968_0003_r_02_0 - exited : java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row (tag=0)
{key:{reducesinkkey0:val_200},value:{_col0:val_200,_col1:200,_col2:201.0},alias:0}
at
org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:268)
at
org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:448)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:399)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:157)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:152)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
while processing row (tag=0)
{key:{reducesinkkey0:val_200},value:{_col0:val_200,_col1:200,_col2:201.0},alias:0}
at
org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:256)
... 7 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveRecordWriter(HiveFileFormatUtils.java:237)
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:477)
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:525)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.ExtractOperator.processOp(ExtractOperator.java:45)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at
org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:247)
... 7 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.mapreduce.TaskID$CharTaskTypeMaps.getRepresentingCharacter(TaskID.java:265)
at org.apache.hadoop.mapreduce.TaskID.appendTo(TaskID.java:153)
at
org.apache.hadoop.mapreduce.TaskAttemptID.appendTo(TaskAttemptID.java:119)
at
org.apache.hadoop.mapreduce.TaskAttemptID.toString(TaskAttemptID.java:151)
at java.lang.String.valueOf(String.java:2826)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.init(FileOutputCommitter.java:69)
at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.getRecordWriter(HFileOutputFormat.java:90)
at
org.apache.hadoop.hive.hbase.HiveHFileOutputFormat.getFileWriter(HiveHFileOutputFormat.java:67)
at
org.apache.hadoop.hive.hbase.HiveHFileOutputFormat.getHiveRecordWriter(HiveHFileOutputFormat.java:104)
at
org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getRecordWriter(HiveFileFormatUtils.java:246)
at
org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveRecordWriter(HiveFileFormatUtils.java:234)
... 14 more
==
was:
After upgrading to Hadoop 23 and HBase 0.94.5 compiled for Hadoop 23. The
TestHBaseMinimrCliDriver, fails after performing the following steps
Update hbase_bulk.m with the following properties
set mapreduce.totalorderpartitioner.naturalorder=false;
set mapreduce.totalorderpartitioner.path=/tmp/hbpartition.lst;
Otherwise I keep seeing:
[jira] [Created] (HIVE-4217) Fix show_create_table_*.q test failures
Carl Steinbach created HIVE-4217: Summary: Fix show_create_table_*.q test failures Key: HIVE-4217 URL: https://issues.apache.org/jira/browse/HIVE-4217 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Carl Steinbach Assignee: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4217) Fix show_create_table_*.q test failures
[ https://issues.apache.org/jira/browse/HIVE-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609471#comment-13609471 ] Carl Steinbach commented on HIVE-4217: -- The TestCliDriver.show_create_table_*.q tests are currently failing on trunk. These failures are caused by HIVE-4187 which I did not adequately test before committing. Fix show_create_table_*.q test failures --- Key: HIVE-4217 URL: https://issues.apache.org/jira/browse/HIVE-4217 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Carl Steinbach Assignee: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: HIVE-4217. Fix show_create_table_*.q test failures
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10066/ --- Review request for hive. Description --- This is a fix for HIVE-4217. The patch modifies DDLTask to use the StringTemplate4 instead of relying on the old version which has a dependency on antlr-2.7.7. This addresses bug HIVE-4217. https://issues.apache.org/jira/browse/HIVE-4217 Diffs - ivy/libraries.properties 49b4b61 metastore/ivy.xml a13cff7 ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 6895759 Diff: https://reviews.apache.org/r/10066/diff/ Testing --- Thanks, Carl Steinbach
[jira] [Updated] (HIVE-4217) Fix show_create_table_*.q test failures
[ https://issues.apache.org/jira/browse/HIVE-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-4217: - Attachment: HIVE-4217.1.patch.txt Review request: https://reviews.apache.org/r/10066/ Fix show_create_table_*.q test failures --- Key: HIVE-4217 URL: https://issues.apache.org/jira/browse/HIVE-4217 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Carl Steinbach Assignee: Carl Steinbach Attachments: HIVE-4217.1.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4217) Fix show_create_table_*.q test failures
[ https://issues.apache.org/jira/browse/HIVE-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-4217: - Status: Patch Available (was: Open) Fix show_create_table_*.q test failures --- Key: HIVE-4217 URL: https://issues.apache.org/jira/browse/HIVE-4217 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Carl Steinbach Assignee: Carl Steinbach Attachments: HIVE-4217.1.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4217) Fix show_create_table_*.q test failures
[ https://issues.apache.org/jira/browse/HIVE-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609542#comment-13609542 ] Kevin Wilfong commented on HIVE-4217: - Can you remove the entry in eclipse-templates/.classpath for the stringtemplate jar as well? Fix show_create_table_*.q test failures --- Key: HIVE-4217 URL: https://issues.apache.org/jira/browse/HIVE-4217 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Carl Steinbach Assignee: Carl Steinbach Attachments: HIVE-4217.1.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4213) List bucketing error too restrictive
[
https://issues.apache.org/jira/browse/HIVE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609586#comment-13609586
]
Gang Tim Liu commented on HIVE-4213:
[~mgrover]
I am a little confused. Please correct me. The current logic is not
restrictive.
For example, it is legal for the following case:
set hive.mapred.supports.subdirectories=true;
set mapred.input.dir.recursive=true;
set hive.optimize.listbucketing=false;
List bucketing error too restrictive
Key: HIVE-4213
URL: https://issues.apache.org/jira/browse/HIVE-4213
Project: Hive
Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Mark Grover
Fix For: 0.11.0
With the introduction of List bucketing, we introduced a config validation
step where we say:
{code}
SUPPORT_DIR_MUST_TRUE_FOR_LIST_BUCKETING(
10199,
hive.mapred.supports.subdirectories must be true
+ if any one of following is true:
hive.internal.ddl.list.bucketing.enable,
+ hive.optimize.listbucketing and mapred.input.dir.recursive),
{code}
This seems overly restrictive to because there are use cases where people may
want to use {{mapred.input.dir.recursive}} to {{true}} even when they don't
care about list bucketing.
Is that not true?
For example, here is the unit test code for {{clientpositive/recursive_dir.q}}
{code}
CREATE TABLE fact_daily(x int) PARTITIONED BY (ds STRING);
CREATE TABLE fact_tz(x int) PARTITIONED BY (ds STRING, hr STRING)
LOCATION 'pfile:${system:test.tmp.dir}/fact_tz';
INSERT OVERWRITE TABLE fact_tz PARTITION (ds='1', hr='1')
SELECT key+11 FROM src WHERE key=484;
ALTER TABLE fact_daily SET TBLPROPERTIES('EXTERNAL'='TRUE');
ALTER TABLE fact_daily ADD PARTITION (ds='1')
LOCATION 'pfile:${system:test.tmp.dir}/fact_tz/ds=1';
set hive.mapred.supports.subdirectories=true;
set mapred.input.dir.recursive=true;
set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;
SELECT * FROM fact_daily WHERE ds='1';
SELECT count(1) FROM fact_daily WHERE ds='1';
{code}
The unit test doesn't seem to be concerned about list bucketing but wants to
set {{mapred.input.dir.recursive}} to {{true}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-4217. Fix show_create_table_*.q test failures
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10066/ --- (Updated March 21, 2013, 10:26 p.m.) Review request for hive. Changes --- Updated eclipse classpath template. Description --- This is a fix for HIVE-4217. The patch modifies DDLTask to use the StringTemplate4 instead of relying on the old version which has a dependency on antlr-2.7.7. This addresses bug HIVE-4217. https://issues.apache.org/jira/browse/HIVE-4217 Diffs (updated) - eclipse-templates/.classpath a94463e ivy/libraries.properties 49b4b61 metastore/ivy.xml a13cff7 ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 6895759 Diff: https://reviews.apache.org/r/10066/diff/ Testing --- Thanks, Carl Steinbach
[jira] [Updated] (HIVE-4217) Fix show_create_table_*.q test failures
[ https://issues.apache.org/jira/browse/HIVE-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-4217: - Attachment: HIVE-4217.2.patch.txt New patch with updated eclipse classpath template. Fix show_create_table_*.q test failures --- Key: HIVE-4217 URL: https://issues.apache.org/jira/browse/HIVE-4217 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Carl Steinbach Assignee: Carl Steinbach Attachments: HIVE-4217.1.patch.txt, HIVE-4217.2.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4217) Fix show_create_table_*.q test failures
[ https://issues.apache.org/jira/browse/HIVE-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609598#comment-13609598 ] Kevin Wilfong commented on HIVE-4217: - +1 Fix show_create_table_*.q test failures --- Key: HIVE-4217 URL: https://issues.apache.org/jira/browse/HIVE-4217 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Carl Steinbach Assignee: Carl Steinbach Attachments: HIVE-4217.1.patch.txt, HIVE-4217.2.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4217) Fix show_create_table_*.q test failures
[ https://issues.apache.org/jira/browse/HIVE-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-4217: - Resolution: Fixed Fix Version/s: 0.11.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I went ahead an committed this to trunk without waiting for the tests to finish. I figured this was acceptable since the build is already broken, and this patch aims to fix it. Fix show_create_table_*.q test failures --- Key: HIVE-4217 URL: https://issues.apache.org/jira/browse/HIVE-4217 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Carl Steinbach Assignee: Carl Steinbach Fix For: 0.11.0 Attachments: HIVE-4217.1.patch.txt, HIVE-4217.2.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HIVE-4217) Fix show_create_table_*.q test failures
[ https://issues.apache.org/jira/browse/HIVE-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609652#comment-13609652 ] Carl Steinbach edited comment on HIVE-4217 at 3/21/13 11:01 PM: I went ahead and committed this to trunk without waiting for the tests to finish. I figured this was acceptable since the build is already broken, and this patch aims to fix it. was (Author: cwsteinbach): I went ahead an committed this to trunk without waiting for the tests to finish. I figured this was acceptable since the build is already broken, and this patch aims to fix it. Fix show_create_table_*.q test failures --- Key: HIVE-4217 URL: https://issues.apache.org/jira/browse/HIVE-4217 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Carl Steinbach Assignee: Carl Steinbach Fix For: 0.11.0 Attachments: HIVE-4217.1.patch.txt, HIVE-4217.2.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Review request: HIVE-4119
https://issues.apache.org/jira/browse/HIVE-4119 Thanks. Shreepadma
[jira] [Commented] (HIVE-4214) OVER accepts general expression instead of just function
[
https://issues.apache.org/jira/browse/HIVE-4214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609681#comment-13609681
]
Harish Butani commented on HIVE-4214:
-
The behavior today is similar to UDAF evaluation and group by:
- the UDAF is evaluated in the PTFOp
- the non UDAF parts of an expression are evaluated in the child Select
Operator.
- the over clause is associated with all UDAFs in the Select Clause.
- So you can express things like (see are test cases from
windowing_expressions.q)
{noformat}
select p_mfgr, p_retailprice, p_size,
round(sum(p_retailprice),2) = round((sum(lag(p_retailprice,1)) -
first_value(p_retailprice)) + last_value(p_retailprice),2) over(distribute by
p_mfgr sort by p_retailprice),
max(p_retailprice) - min(p_retailprice) = last_value(p_retailprice) -
first_value(p_retailprice)
over(distribute by p_mfgr sort by p_retailprice)
from part;
select p_mfgr, p_retailprice, p_size,
rank() over (distribute by p_mfgr sort by p_retailprice) as r,
sum(p_retailprice) over (distribute by p_mfgr sort by p_retailprice rows
between unbounded preceding and current row) as s2,
sum(p_retailprice) - 5 over (distribute by p_mfgr sort by p_retailprice rows
between unbounded preceding and current row) as s1
from part;
{noformat}
- Can you post what output you are expecting for the first query you posted?
- Have to look at whether we can switch the second form you posted. This will
make the first e.g. much more cumbersome
to write.
OVER accepts general expression instead of just function
Key: HIVE-4214
URL: https://issues.apache.org/jira/browse/HIVE-4214
Project: Hive
Issue Type: Bug
Components: PTF-Windowing
Affects Versions: 0.11.0
Reporter: Alan Gates
The query:
select s, i, avg(d) / 10.0 over (partition by s order by i) from over100k;
runs (and produces meaningless output).
Over should not allow the arithmetic expression. Only a UDAF or PTF function
should be valid there. The correct way to write this query should be
select s, i, avg(d) over (partition by s order by i) / 10. 0 from over100k;
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4218) Store resource the user has explicitly added
Jinyi Yao created HIVE-4218: --- Summary: Store resource the user has explicitly added Key: HIVE-4218 URL: https://issues.apache.org/jira/browse/HIVE-4218 Project: Hive Issue Type: Improvement Reporter: Jinyi Yao Priority: Minor It would be useful to track what resources the user has explicitly added. This can be logged and later enable usage pattern analysis to optimize system performance. It needs to differentiate built-in resource from that explicitly added by the user via add jar, add file etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4144) Add select database() command to show the current database
[
https://issues.apache.org/jira/browse/HIVE-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-4144:
--
Attachment: HIVE-4144.D9597.2.patch
navis updated the revision HIVE-4144 [jira] Add select database() command to
show the current database.
Extended to accept limit clause
Reviewers: JIRA
REVISION DETAIL
https://reviews.facebook.net/D9597
CHANGE SINCE LAST DIFF
https://reviews.facebook.net/D9597?vs=30219id=30321#toc
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
ql/src/java/org/apache/hadoop/hive/ql/io/NullRowsInputFormat.java
ql/src/java/org/apache/hadoop/hive/ql/io/OneNullRowInputFormat.java
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/UDFCurrentDB.java
ql/src/test/queries/clientpositive/select_dummy_source.q
ql/src/test/queries/clientpositive/udf_current_database.q
ql/src/test/results/clientpositive/select_dummy_source.q.out
ql/src/test/results/clientpositive/udf_current_database.q.out
To: JIRA, navis
Add select database() command to show the current database
Key: HIVE-4144
URL: https://issues.apache.org/jira/browse/HIVE-4144
Project: Hive
Issue Type: Bug
Components: SQL
Reporter: Mark Grover
Assignee: Navis
Attachments: HIVE-4144.D9597.1.patch, HIVE-4144.D9597.2.patch
A recent hive-user mailing list conversation asked about having a command to
show the current database.
http://mail-archives.apache.org/mod_mbox/hive-user/201303.mbox/%3CCAMGr+0i+CRY69m3id=DxthmUCWLf0NxpKMCtROb=uauh2va...@mail.gmail.com%3E
MySQL seems to have a command to do so:
{code}
select database();
{code}
http://dev.mysql.com/doc/refman/5.0/en/information-functions.html#function_database
We should look into having something similar in Hive.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4219) explain dependency does not capture the input table
Namit Jain created HIVE-4219:
Summary: explain dependency does not capture the input table
Key: HIVE-4219
URL: https://issues.apache.org/jira/browse/HIVE-4219
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Namit Jain
Assignee: Namit Jain
hive explain dependency select * from srcpart where ds is not null;
OK
{input_partitions:[{partitionName:default@srcpart@ds=2008-04-08/hr=11},{partitionName:default@srcpart@ds=2008-04-08/hr=12},{partitionName:default@srcpart@ds=2008-04-09/hr=11},{partitionName:default@srcpart@ds=2008-04-09/hr=12}],input_tables:[]}
input_tables should contain srcpart
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3820) Consider creating a literal like D or BD for representing Decimal type constants
[
https://issues.apache.org/jira/browse/HIVE-3820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609941#comment-13609941
]
Hudson commented on HIVE-3820:
--
Integrated in Hive-trunk-h0.21 #2026 (See
[https://builds.apache.org/job/Hive-trunk-h0.21/2026/])
HIVE-3820 Consider creating a literal like D or BD for representing Decimal
type constants (Gunther Hagleitner via Ashutosh Chauhan) (Revision 1459298)
Result = FAILURE
hashutosh :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1459298
Files :
*
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g
*
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java
* /hive/trunk/ql/src/test/queries/clientpositive/literal_decimal.q
* /hive/trunk/ql/src/test/results/clientpositive/literal_decimal.q.out
*
/hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaBigDecimalObjectInspector.java
Consider creating a literal like D or BD for representing Decimal type
constants
Key: HIVE-3820
URL: https://issues.apache.org/jira/browse/HIVE-3820
Project: Hive
Issue Type: Bug
Reporter: Mark Grover
Assignee: Gunther Hagleitner
Fix For: 0.11.0
Attachments: HIVE-3820.1.patch, HIVE-3820.2.patch,
HIVE-3820.D8823.1.patch
When the HIVE-2693 gets committed, users are going to see this behavior:
{code}
hive select cast(3.14 as decimal) from decimal_3 limit 1;
3.140124344978758017532527446746826171875
{code}
That's intuitively incorrect but is the case because 3.14 (double) is being
converted to BigDecimal because of which there is a precision mismatch.
We should consider creating a new literal for expressing constants of Decimal
type as Gunther suggested in HIVE-2693.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4219) explain dependency does not capture the input table
[
https://issues.apache.org/jira/browse/HIVE-4219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609944#comment-13609944
]
Namit Jain commented on HIVE-4219:
--
This is a problem only for 'select * from T where condition on partition'
queries, where a map-reduce job is not invoked.
explain dependency does not capture the input table
---
Key: HIVE-4219
URL: https://issues.apache.org/jira/browse/HIVE-4219
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Namit Jain
Assignee: Namit Jain
hive explain dependency select * from srcpart where ds is not null;
OK
{input_partitions:[{partitionName:default@srcpart@ds=2008-04-08/hr=11},{partitionName:default@srcpart@ds=2008-04-08/hr=12},{partitionName:default@srcpart@ds=2008-04-09/hr=11},{partitionName:default@srcpart@ds=2008-04-09/hr=12}],input_tables:[]}
input_tables should contain srcpart
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 2026 - Still Failing
Changes for Build #2012 [hashutosh] HIVE-3862 : testHBaseNegativeCliDriver_cascade_dbdrop fails on hadoop-1 (Gunther Hagleitner via Ashutosh Chauhan) Changes for Build #2013 [kevinwilfong] HIVE-4125. Expose metastore JMX metrics. (Samuel Yuan via kevinwilfong) [hashutosh] HIVE-2935 : Implement HiveServer2 Core code changes (4th patch of 4) (Carl Steinbach and others via Ashutosh Chauhan) [kevinwilfong] HIVE-4096. problem in hive.map.groupby.sorted with distincts. (njain via kevinwilfong) [hashutosh] HIVE-2935 : Implement HiveServer2 Beeline .q.out files (3rd patch of 4) (Carl Steinbach and others via Ashutosh Chauhan) [hashutosh] HIVE-2935 : Implement HiveServer2 Beeline code changes (2nd patch of 4) (Carl Steinbach and others via Ashutosh Chauhan) [hashutosh] HIVE-2935 : Implement HiveServer2 (1st patch of 4) (Carl Steinbach and others via Ashutosh Chauhan) [hashutosh] HIVE-3717 : Hive wont compile with -Dhadoop.mr.rev=20S (Gunther Hagleitner via Ashutosh Chauhan) [hashutosh] HIVE-4148 : Cleanup aisle ivy (Gunther Hagleitner via Ashutosh Chauhan) Changes for Build #2014 Changes for Build #2015 Changes for Build #2016 [ecapriolo] Hive-4141 InspectorFactories use static HashMaps which fail in concurrent modification (Brock Noland via egc) Submitted by: Brock Noland Reviewed by: Edward Capriolo Approved by: Edward Capriolo [kevinwilfong] HIVE-4176. disable TestBeeLineDriver in ptest util. (kevinwilfong reviewed by njain, ashutoshc) [hashutosh] HIVE-4169 : union_remove_*.q fail on hadoop 2 (Gunther Hagleitner via Ashutosh Chauhan) Changes for Build #2017 [cws] HIVE-4145. Create hcatalog stub directory and add it to the build (Carl Steinbach via cws) [kevinwilfong] HIVE-4162. disable TestBeeLineDriver. (Thejas M Nair via kevinwilfong) Changes for Build #2018 Changes for Build #2019 Changes for Build #2020 Changes for Build #2021 Changes for Build #2022 Changes for Build #2023 [namit] HIVE-4154 NPE reading column of empty string from ORC file (Kevin Wilfong via namit) [hashutosh] HIVE-4186 : NPE in ReduceSinkDeDuplication (Harish Butani via Ashutosh Chauhan) Changes for Build #2024 [namit] HIVE-4146 bug with hive.auto.convert.join.noconditionaltask with outer joins (Namit via Gang Tim Liu) Changes for Build #2025 [hashutosh] HIVE-4139 : MiniDFS shim does not work for hadoop 2 (Gunther Hagleitner via Ashutosh Chauhan) [kevinwilfong] HIVE-4015. Add ORC file to the grammar as a file format. (Gunther Hagleitner via kevinwilfong) [cws] HIVE-4187. QL build-grammar target fails after HIVE-4148 (Gunther Hagleitner via cws) Changes for Build #2026 [hashutosh] HIVE-3820 Consider creating a literal like D or BD for representing Decimal type constants (Gunther Hagleitner via Ashutosh Chauhan) 2 tests failed. REGRESSION: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union12 Error Message: Timeout occurred. Please note the time in the report does not reflect the time until the timeout. Stack Trace: junit.framework.AssertionFailedError: Timeout occurred. Please note the time in the report does not reflect the time until the timeout. at net.sf.antcontrib.logic.ForTask.doSequentialIteration(ForTask.java:259) at net.sf.antcontrib.logic.ForTask.doToken(ForTask.java:268) at net.sf.antcontrib.logic.ForTask.doTheTasks(ForTask.java:299) at net.sf.antcontrib.logic.ForTask.execute(ForTask.java:244) REGRESSION: org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_aggregator_error_1 Error Message: Forked Java VM exited abnormally. Please note the time in the report does not reflect the time until the VM exit. Stack Trace: junit.framework.AssertionFailedError: Forked Java VM exited abnormally. Please note the time in the report does not reflect the time until the VM exit. at net.sf.antcontrib.logic.ForTask.doSequentialIteration(ForTask.java:259) at net.sf.antcontrib.logic.ForTask.doToken(ForTask.java:268) at net.sf.antcontrib.logic.ForTask.doTheTasks(ForTask.java:299) at net.sf.antcontrib.logic.ForTask.execute(ForTask.java:244) The Apache Jenkins build system has built Hive-trunk-h0.21 (build #2026) Status: Still Failing Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/2026/ to view the results.
[jira] [Commented] (HIVE-4219) explain dependency does not capture the input table
[
https://issues.apache.org/jira/browse/HIVE-4219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609951#comment-13609951
]
Namit Jain commented on HIVE-4219:
--
https://reviews.facebook.net/D9657
explain dependency does not capture the input table
---
Key: HIVE-4219
URL: https://issues.apache.org/jira/browse/HIVE-4219
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Namit Jain
Assignee: Namit Jain
Attachments: hive.4219.1.patch
hive explain dependency select * from srcpart where ds is not null;
OK
{input_partitions:[{partitionName:default@srcpart@ds=2008-04-08/hr=11},{partitionName:default@srcpart@ds=2008-04-08/hr=12},{partitionName:default@srcpart@ds=2008-04-09/hr=11},{partitionName:default@srcpart@ds=2008-04-09/hr=12}],input_tables:[]}
input_tables should contain srcpart
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4219) explain dependency does not capture the input table
[
https://issues.apache.org/jira/browse/HIVE-4219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Namit Jain updated HIVE-4219:
-
Attachment: hive.4219.1.patch
explain dependency does not capture the input table
---
Key: HIVE-4219
URL: https://issues.apache.org/jira/browse/HIVE-4219
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Namit Jain
Assignee: Namit Jain
Attachments: hive.4219.1.patch
hive explain dependency select * from srcpart where ds is not null;
OK
{input_partitions:[{partitionName:default@srcpart@ds=2008-04-08/hr=11},{partitionName:default@srcpart@ds=2008-04-08/hr=12},{partitionName:default@srcpart@ds=2008-04-09/hr=11},{partitionName:default@srcpart@ds=2008-04-09/hr=12}],input_tables:[]}
input_tables should contain srcpart
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4220) TimestampWritable.toString throws array index exception sometimes
Navis created HIVE-4220:
---
Summary: TimestampWritable.toString throws array index exception
sometimes
Key: HIVE-4220
URL: https://issues.apache.org/jira/browse/HIVE-4220
Project: Hive
Issue Type: Bug
Reporter: Navis
Assignee: Navis
{noformat}
org.apache.hive.service.cli.HiveSQLException: java.io.IOException:
java.lang.ArrayIndexOutOfBoundsException: 45
at
org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:215)
at
org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:170)
at
org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:288)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:348)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1553)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1538)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 45
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:194)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1449)
at
org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:193)
... 11 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 45
at
sun.util.calendar.BaseCalendar.getCalendarDateFromFixedDate(BaseCalendar.java:436)
at
java.util.GregorianCalendar.computeFields(GregorianCalendar.java:2081)
at
java.util.GregorianCalendar.computeFields(GregorianCalendar.java:1996)
at java.util.Calendar.setTimeInMillis(Calendar.java:1110)
at java.util.Calendar.setTime(Calendar.java:1076)
at java.text.SimpleDateFormat.format(SimpleDateFormat.java:875)
at java.text.SimpleDateFormat.format(SimpleDateFormat.java:868)
at java.text.DateFormat.format(DateFormat.java:316)
at
org.apache.hadoop.hive.serde2.io.TimestampWritable.toString(TimestampWritable.java:327)
at
org.apache.hadoop.hive.serde2.lazy.LazyTimestamp.writeUTF8(LazyTimestamp.java:95)
at
org.apache.hadoop.hive.serde2.lazy.LazyUtils.writePrimitiveUTF8(LazyUtils.java:234)
at
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe.java:427)
at
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serializeField(LazySimpleSerDe.java:381)
at
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe.java:365)
at
org.apache.hadoop.hive.ql.exec.ListSinkOperator.processOp(ListSinkOperator.java:96)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:487)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:821)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:487)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:821)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:90)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:487)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:474)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:468)
at
org.apache.hadoop.hive.ql.exec.FetchTask.fetchAndPush(FetchTask.java:222)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:188)
... 13 more
{noformat}
data formatter in TimestampWritable is declared static and shared but it's not
thread-safe.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3348) semi-colon in comments in .q file does not work
[ https://issues.apache.org/jira/browse/HIVE-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3348: - Status: Open (was: Patch Available) Have you run all the tests ? The output file is empty. semi-colon in comments in .q file does not work --- Key: HIVE-3348 URL: https://issues.apache.org/jira/browse/HIVE-3348 Project: Hive Issue Type: Bug Components: Testing Infrastructure Reporter: Namit Jain Assignee: Namit Jain Attachments: hive-3348.patch -- comment ; -- comment select count(1) from src; The above test file fails -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-3348) semi-colon in comments in .q file does not work
[ https://issues.apache.org/jira/browse/HIVE-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain reassigned HIVE-3348: Assignee: Nick Collins (was: Namit Jain) semi-colon in comments in .q file does not work --- Key: HIVE-3348 URL: https://issues.apache.org/jira/browse/HIVE-3348 Project: Hive Issue Type: Bug Components: Testing Infrastructure Reporter: Namit Jain Assignee: Nick Collins Attachments: hive-3348.patch -- comment ; -- comment select count(1) from src; The above test file fails -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
