[jira] [Commented] (HIVE-11401) Predicate push down does not work with Parquet when partitions are in the expression
[
https://issues.apache.org/jira/browse/HIVE-11401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14649299#comment-14649299
]
Sergio Peña commented on HIVE-11401:
The tests are not related with this patch. I run them in my local system, and
they're working correct.
Predicate push down does not work with Parquet when partitions are in the
expression
Key: HIVE-11401
URL: https://issues.apache.org/jira/browse/HIVE-11401
Project: Hive
Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Sergio Peña
Assignee: Sergio Peña
Attachments: HIVE-11401.1.patch, HIVE-11401.2.patch
When filtering Parquet tables using a partition column, the query fails
saying the column does not exist:
{noformat}
hive create table part1 (id int, content string) partitioned by (p string)
stored as parquet;
hive alter table part1 add partition (p='p1');
hive insert into table part1 partition (p='p1') values (1, 'a'), (2, 'b');
hive select id from part1 where p='p1';
Failed with exception java.io.IOException:java.lang.IllegalArgumentException:
Column [p] was not found in schema!
Time taken: 0.151 seconds
{noformat}
It is correct that the partition column is not part of the Parquet schema.
So, the fix should be to remove such expression from the Parquet PPD.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11401) Predicate push down does not work with Parquet when partitions are in the expression
[
https://issues.apache.org/jira/browse/HIVE-11401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14648100#comment-14648100
]
Szehon Ho commented on HIVE-11401:
--
+1 makes sense from my end.
Predicate push down does not work with Parquet when partitions are in the
expression
Key: HIVE-11401
URL: https://issues.apache.org/jira/browse/HIVE-11401
Project: Hive
Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Sergio Peña
Assignee: Sergio Peña
Attachments: HIVE-11401.1.patch
When filtering Parquet tables using a partition column, the query fails
saying the column does not exist:
{noformat}
hive create table part1 (id int, content string) partitioned by (p string)
stored as parquet;
hive alter table part1 add partition (p='p1');
hive insert into table part1 partition (p='p1') values (1, 'a'), (2, 'b');
hive select id from part1 where p='p1';
Failed with exception java.io.IOException:java.lang.IllegalArgumentException:
Column [p] was not found in schema!
Time taken: 0.151 seconds
{noformat}
It is correct that the partition column is not part of the Parquet schema.
So, the fix should be to remove such expression from the Parquet PPD.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
