[
https://issues.apache.org/jira/browse/HIVE-20284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hui Huang updated HIVE-20284:
-----------------------------
Description:
In strict mode and the hive.optimize.constant.propagation is set to true, the
following sql will failed:
{code:java}
hive> desc employee_part;
OK
col_name data_type comment
eid int
name string
dept string
year string
month string
# Partition Information
# col_name data_type comment
year string
month string
Time taken: 0.564 seconds, Fetched: 11 row(s)
hive> set hive.mapred.mode=strict;
hive> select * from employee_part where false and concat(year,month)='201807';
FAILED: SemanticException Queries against partitioned tables without a
partition filter are disabled for safety reasons. If you know what you are
doing, please sethive.strict.checks.large.query to false and that
hive.mapred.mode is not set to 'strict' to proceed. Note that if you may get
errors or incorrect results if you make a mistake while using some of the
unsafe features. No partition predicate for Alias "employee_part" Table
"employee_part"
{code}
The above error message is confusing because the expression
concat(year,month)='201807' is a partition filter。
The reason is during logic optimization, the ConstantPropagate optimizer is
running before partitionPruner optimizer, when found a express like 'false and
concat(year,month)=xxxx', the express is replaced with 'fasle' , and the
partition filter is dropped. So the PartitionPruner can not get the partition
filter.
Users can remove the constant express that always has true/false values to work
around.
When views used, if some columns are constant values, users will be confusing.
So we should add some more message in the error msg returned.
was:
In strict mode and the hive.optimize.constant.propagation is set to true, the
following sql will failed:
{code:java}
hive> desc employee_part;
OK
col_name data_type comment
eid int
name string
dept string
year string
month string
# Partition Information
# col_name data_type comment
year string
month string
Time taken: 0.564 seconds, Fetched: 11 row(s)
hive> set hive.mapred.mode=strict;
hive> select * from employee_part where false and concat(year,month)='201807';
FAILED: SemanticException Queries against partitioned tables without a
partition filter are disabled for safety reasons. If you know what you are
doing, please sethive.strict.checks.large.query to false and that
hive.mapred.mode is not set to 'strict' to proceed. Note that if you may get
errors or incorrect results if you make a mistake while using some of the
unsafe features. No partition predicate for Alias "employee_part" Table
"employee_part"
{code}
The above error msg is confusing， concat(year,month)='201807' is the partition
filter。
The reason is during logic optimization, the ConstantPropagate optimizer is
running before partitionPruner optimizer, when found a express like 'false and
concat(year,month)=xxxx', the express is replaced with 'fasle' , and the
partition filter is dropped. So the PartitionPruner can not get the partition
filter.
Users can remove the constant express that always has true/false values to work
around.
When views used, if some columns are constant values, users will be confusing.
So we should add some more message in the error msg returned.
> In strict mode, if constant propagation is enable, the partition filter may
> be folded before partition pruner lead to error "No partition predicate for
> Alias"
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-20284
> URL: https://issues.apache.org/jira/browse/HIVE-20284
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Affects Versions: 1.2.1, 2.3.3
> Reporter: Hui Huang
> Assignee: Hui Huang
> Priority: Trivial
> Fix For: 2.3.3, 4.0.0
>
> Attachments: HIVE-20284.1.patch, HIVE-20284.2.patch, HIVE-20284.patch
>
>
> In strict mode and the hive.optimize.constant.propagation is set to true, the
> following sql will failed:
> {code:java}
> hive> desc employee_part;
> OK
> col_name data_type comment
> eid int
> name string
> dept string
> year string
> month string
> # Partition Information
> # col_name data_type comment
> year string
> month string
> Time taken: 0.564 seconds, Fetched: 11 row(s)
> hive> set hive.mapred.mode=strict;
> hive> select * from employee_part where false and concat(year,month)='201807';
> FAILED: SemanticException Queries against partitioned tables without a
> partition filter are disabled for safety reasons. If you know what you are
> doing, please sethive.strict.checks.large.query to false and that
> hive.mapred.mode is not set to 'strict' to proceed. Note that if you may get
> errors or incorrect results if you make a mistake while using some of the
> unsafe features. No partition predicate for Alias "employee_part" Table
> "employee_part"
> {code}
> The above error message is confusing because the expression
> concat(year,month)='201807' is a partition filter。
> The reason is during logic optimization, the ConstantPropagate optimizer is
> running before partitionPruner optimizer, when found a express like 'false
> and concat(year,month)=xxxx', the express is replaced with 'fasle' , and the
> partition filter is dropped. So the PartitionPruner can not get the partition
> filter.
> Users can remove the constant express that always has true/false values to
> work around.
> When views used, if some columns are constant values, users will be
> confusing.
> So we should add some more message in the error msg returned.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
