[ 
https://issues.apache.org/jira/browse/HIVE-20284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hui Huang updated HIVE-20284:
-----------------------------
    Description: 
In strict mode and the hive.optimize.constant.propagation is set to true, the 
following sql will failed:

{code:java}
hive> desc employee_part;
OK
col_name        data_type       comment
eid                     int
name                    string
dept                    string
year                    string
month                   string

# Partition Information
# col_name              data_type               comment

year                    string
month                   string
Time taken: 0.564 seconds, Fetched: 11 row(s)
hive> set hive.mapred.mode=strict;
hive> select * from employee_part where false and concat(year,month)='201807';
FAILED: SemanticException Queries against partitioned tables without a 
partition filter are disabled for safety reasons. If you know what you are 
doing, please sethive.strict.checks.large.query to false and that 
hive.mapred.mode is not set to 'strict' to proceed. Note that if you may get 
errors or incorrect results if you make a mistake while using some of the 
unsafe features. No partition predicate for Alias "employee_part" Table 
"employee_part"
{code}

The above error message is confusing because the expression 
concat(year,month)='201807' is a partition filter。

The reason is during logic optimization, the ConstantPropagate optimizer is 
running before partitionPruner optimizer, when found a express like 'false and 
concat(year,month)=xxxx', the express is replaced with 'fasle' , and the 
partition filter is dropped. So the PartitionPruner can not get the partition 
filter.

Users can remove the constant express that always has true/false values to work 
around.

When views used, if some columns are constant values, users  will be  confusing.

So we should add some more message in the error msg returned.


  was:
In strict mode and the hive.optimize.constant.propagation is set to true, the 
following sql will failed:

{code:java}
hive> desc employee_part;
OK
col_name        data_type       comment
eid                     int
name                    string
dept                    string
year                    string
month                   string

# Partition Information
# col_name              data_type               comment

year                    string
month                   string
Time taken: 0.564 seconds, Fetched: 11 row(s)
hive> set hive.mapred.mode=strict;
hive> select * from employee_part where false and concat(year,month)='201807';
FAILED: SemanticException Queries against partitioned tables without a 
partition filter are disabled for safety reasons. If you know what you are 
doing, please sethive.strict.checks.large.query to false and that 
hive.mapred.mode is not set to 'strict' to proceed. Note that if you may get 
errors or incorrect results if you make a mistake while using some of the 
unsafe features. No partition predicate for Alias "employee_part" Table 
"employee_part"
{code}

The above error msg is confusing,  concat(year,month)='201807' is the partition 
filter。

The reason is during logic optimization, the ConstantPropagate optimizer is 
running before partitionPruner optimizer, when found a express like 'false and 
concat(year,month)=xxxx', the express is replaced with 'fasle' , and the 
partition filter is dropped. So the PartitionPruner can not get the partition 
filter.

Users can remove the constant express that always has true/false values to work 
around.

When views used, if some columns are constant values, users  will be  confusing.

So we should add some more message in the error msg returned.



> In strict mode, if constant propagation is enable, the partition filter may 
> be folded before partition pruner lead to error "No partition predicate for 
> Alias"  
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-20284
>                 URL: https://issues.apache.org/jira/browse/HIVE-20284
>             Project: Hive
>          Issue Type: Improvement
>          Components: CLI
>    Affects Versions: 1.2.1, 2.3.3
>            Reporter: Hui Huang
>            Assignee: Hui Huang
>            Priority: Trivial
>             Fix For: 2.3.3, 4.0.0
>
>         Attachments: HIVE-20284.1.patch, HIVE-20284.2.patch, HIVE-20284.patch
>
>
> In strict mode and the hive.optimize.constant.propagation is set to true, the 
> following sql will failed:
> {code:java}
> hive> desc employee_part;
> OK
> col_name      data_type       comment
> eid                   int
> name                  string
> dept                  string
> year                  string
> month                 string
> # Partition Information
> # col_name                    data_type               comment
> year                  string
> month                 string
> Time taken: 0.564 seconds, Fetched: 11 row(s)
> hive> set hive.mapred.mode=strict;
> hive> select * from employee_part where false and concat(year,month)='201807';
> FAILED: SemanticException Queries against partitioned tables without a 
> partition filter are disabled for safety reasons. If you know what you are 
> doing, please sethive.strict.checks.large.query to false and that 
> hive.mapred.mode is not set to 'strict' to proceed. Note that if you may get 
> errors or incorrect results if you make a mistake while using some of the 
> unsafe features. No partition predicate for Alias "employee_part" Table 
> "employee_part"
> {code}
> The above error message is confusing because the expression 
> concat(year,month)='201807' is a partition filter。
> The reason is during logic optimization, the ConstantPropagate optimizer is 
> running before partitionPruner optimizer, when found a express like 'false 
> and concat(year,month)=xxxx', the express is replaced with 'fasle' , and the 
> partition filter is dropped. So the PartitionPruner can not get the partition 
> filter.
> Users can remove the constant express that always has true/false values to 
> work around.
> When views used, if some columns are constant values, users  will be  
> confusing.
> So we should add some more message in the error msg returned.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to