[jira] [Updated] (HIVE-11375) Broken processing of queries containing NOT (x IS NOT NULL and x <> 0)
[
https://issues.apache.org/jira/browse/HIVE-11375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-11375:
Target Version/s: 1.3.0
> Broken processing of queries containing NOT (x IS NOT NULL and x <> 0)
> --
>
> Key: HIVE-11375
> URL: https://issues.apache.org/jira/browse/HIVE-11375
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Affects Versions: 2.0.0
>Reporter: Mariusz Sakowski
>Assignee: Aihua Xu
> Fix For: 2.0.0
>
> Attachments: HIVE-11375.2.patch, HIVE-11375.3.patch,
> HIVE-11375.4.patch, HIVE-11375.branch-1.patch, HIVE-11375.patch
>
>
> When running query like this:
> {code}explain select * from test where (val is not null and val <> 0);{code}
> hive will simplify expression in parenthesis and omit is not null check:
> {code}
> Filter Operator
> predicate: (val <> 0) (type: boolean)
> {code}
> which is fine.
> but if we negate condition using NOT operator:
> {code}explain select * from test where not (val is not null and val <>
> 0);{code}
> hive will also simplify thing, but now it will break stuff:
> {code}
> Filter Operator
> predicate: (not (val <> 0)) (type: boolean)
> {code}
> because valid predicate should be *val == 0 or val is null*, while above row
> is equivalent to *val == 0* only, filtering away rows where val is null
> simple example:
> {code}
> CREATE TABLE example (
> val bigint
> );
> INSERT INTO example VALUES (1), (NULL), (0);
> -- returns 2 rows - NULL and 0
> select * from example where (val is null or val == 0);
> -- returns 1 row - 0
> select * from example where not (val is not null and val <> 0);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11375) Broken processing of queries containing NOT (x IS NOT NULL and x <> 0)
[
https://issues.apache.org/jira/browse/HIVE-11375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-11375:
Should this issue be backported to branch-1? It looks like a bug.
> Broken processing of queries containing NOT (x IS NOT NULL and x <> 0)
> --
>
> Key: HIVE-11375
> URL: https://issues.apache.org/jira/browse/HIVE-11375
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Affects Versions: 2.0.0
>Reporter: Mariusz Sakowski
>Assignee: Aihua Xu
> Fix For: 2.0.0
>
> Attachments: HIVE-11375.2.patch, HIVE-11375.3.patch,
> HIVE-11375.4.patch, HIVE-11375.branch-1.patch, HIVE-11375.patch
>
>
> When running query like this:
> {code}explain select * from test where (val is not null and val <> 0);{code}
> hive will simplify expression in parenthesis and omit is not null check:
> {code}
> Filter Operator
> predicate: (val <> 0) (type: boolean)
> {code}
> which is fine.
> but if we negate condition using NOT operator:
> {code}explain select * from test where not (val is not null and val <>
> 0);{code}
> hive will also simplify thing, but now it will break stuff:
> {code}
> Filter Operator
> predicate: (not (val <> 0)) (type: boolean)
> {code}
> because valid predicate should be *val == 0 or val is null*, while above row
> is equivalent to *val == 0* only, filtering away rows where val is null
> simple example:
> {code}
> CREATE TABLE example (
> val bigint
> );
> INSERT INTO example VALUES (1), (NULL), (0);
> -- returns 2 rows - NULL and 0
> select * from example where (val is null or val == 0);
> -- returns 1 row - 0
> select * from example where not (val is not null and val <> 0);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11375) Broken processing of queries containing NOT (x IS NOT NULL and x <> 0)
[
https://issues.apache.org/jira/browse/HIVE-11375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-11375:
Attachment: HIVE-11375.branch-1.patch
Attached the patch for branch-1 branch.
> Broken processing of queries containing NOT (x IS NOT NULL and x <> 0)
> --
>
> Key: HIVE-11375
> URL: https://issues.apache.org/jira/browse/HIVE-11375
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Affects Versions: 2.0.0
>Reporter: Mariusz Sakowski
>Assignee: Aihua Xu
> Fix For: 2.0.0
>
> Attachments: HIVE-11375.2.patch, HIVE-11375.3.patch,
> HIVE-11375.4.patch, HIVE-11375.branch-1.patch, HIVE-11375.patch
>
>
> When running query like this:
> {code}explain select * from test where (val is not null and val <> 0);{code}
> hive will simplify expression in parenthesis and omit is not null check:
> {code}
> Filter Operator
> predicate: (val <> 0) (type: boolean)
> {code}
> which is fine.
> but if we negate condition using NOT operator:
> {code}explain select * from test where not (val is not null and val <>
> 0);{code}
> hive will also simplify thing, but now it will break stuff:
> {code}
> Filter Operator
> predicate: (not (val <> 0)) (type: boolean)
> {code}
> because valid predicate should be *val == 0 or val is null*, while above row
> is equivalent to *val == 0* only, filtering away rows where val is null
> simple example:
> {code}
> CREATE TABLE example (
> val bigint
> );
> INSERT INTO example VALUES (1), (NULL), (0);
> -- returns 2 rows - NULL and 0
> select * from example where (val is null or val == 0);
> -- returns 1 row - 0
> select * from example where not (val is not null and val <> 0);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11375) Broken processing of queries containing NOT (x IS NOT NULL and x <> 0)
[
https://issues.apache.org/jira/browse/HIVE-11375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-11375:
Attachment: HIVE-11375.4.patch
> Broken processing of queries containing NOT (x IS NOT NULL and x <> 0)
> --
>
> Key: HIVE-11375
> URL: https://issues.apache.org/jira/browse/HIVE-11375
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Affects Versions: 2.0.0
>Reporter: Mariusz Sakowski
>Assignee: Aihua Xu
> Fix For: 2.0.0
>
> Attachments: HIVE-11375.2.patch, HIVE-11375.3.patch,
> HIVE-11375.4.patch, HIVE-11375.patch
>
>
> When running query like this:
> {code}explain select * from test where (val is not null and val <> 0);{code}
> hive will simplify expression in parenthesis and omit is not null check:
> {code}
> Filter Operator
> predicate: (val <> 0) (type: boolean)
> {code}
> which is fine.
> but if we negate condition using NOT operator:
> {code}explain select * from test where not (val is not null and val <>
> 0);{code}
> hive will also simplify thing, but now it will break stuff:
> {code}
> Filter Operator
> predicate: (not (val <> 0)) (type: boolean)
> {code}
> because valid predicate should be *val == 0 or val is null*, while above row
> is equivalent to *val == 0* only, filtering away rows where val is null
> simple example:
> {code}
> CREATE TABLE example (
> val bigint
> );
> INSERT INTO example VALUES (1), (NULL), (0);
> -- returns 2 rows - NULL and 0
> select * from example where (val is null or val == 0);
> -- returns 1 row - 0
> select * from example where not (val is not null and val <> 0);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11375) Broken processing of queries containing NOT (x IS NOT NULL and x <> 0)
[
https://issues.apache.org/jira/browse/HIVE-11375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-11375:
Attachment: HIVE-11375.3.patch
Unit tests updated. A new negative() function is added for supported UDF
function.
> Broken processing of queries containing NOT (x IS NOT NULL and x <> 0)
> --
>
> Key: HIVE-11375
> URL: https://issues.apache.org/jira/browse/HIVE-11375
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Affects Versions: 2.0.0
>Reporter: Mariusz Sakowski
>Assignee: Aihua Xu
> Fix For: 2.0.0
>
> Attachments: HIVE-11375.2.patch, HIVE-11375.3.patch, HIVE-11375.patch
>
>
> When running query like this:
> {code}explain select * from test where (val is not null and val <> 0);{code}
> hive will simplify expression in parenthesis and omit is not null check:
> {code}
> Filter Operator
> predicate: (val <> 0) (type: boolean)
> {code}
> which is fine.
> but if we negate condition using NOT operator:
> {code}explain select * from test where not (val is not null and val <>
> 0);{code}
> hive will also simplify thing, but now it will break stuff:
> {code}
> Filter Operator
> predicate: (not (val <> 0)) (type: boolean)
> {code}
> because valid predicate should be *val == 0 or val is null*, while above row
> is equivalent to *val == 0* only, filtering away rows where val is null
> simple example:
> {code}
> CREATE TABLE example (
> val bigint
> );
> INSERT INTO example VALUES (1), (NULL), (0);
> -- returns 2 rows - NULL and 0
> select * from example where (val is null or val == 0);
> -- returns 1 row - 0
> select * from example where not (val is not null and val <> 0);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11375) Broken processing of queries containing NOT (x IS NOT NULL and x <> 0)
[
https://issues.apache.org/jira/browse/HIVE-11375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-11375:
Attachment: HIVE-11375.2.patch
> Broken processing of queries containing NOT (x IS NOT NULL and x <> 0)
> --
>
> Key: HIVE-11375
> URL: https://issues.apache.org/jira/browse/HIVE-11375
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Affects Versions: 2.0.0
>Reporter: Mariusz Sakowski
>Assignee: Aihua Xu
> Fix For: 2.0.0
>
> Attachments: HIVE-11375.2.patch, HIVE-11375.patch
>
>
> When running query like this:
> {code}explain select * from test where (val is not null and val <> 0);{code}
> hive will simplify expression in parenthesis and omit is not null check:
> {code}
> Filter Operator
> predicate: (val <> 0) (type: boolean)
> {code}
> which is fine.
> but if we negate condition using NOT operator:
> {code}explain select * from test where not (val is not null and val <>
> 0);{code}
> hive will also simplify thing, but now it will break stuff:
> {code}
> Filter Operator
> predicate: (not (val <> 0)) (type: boolean)
> {code}
> because valid predicate should be *val == 0 or val is null*, while above row
> is equivalent to *val == 0* only, filtering away rows where val is null
> simple example:
> {code}
> CREATE TABLE example (
> val bigint
> );
> INSERT INTO example VALUES (1), (NULL), (0);
> -- returns 2 rows - NULL and 0
> select * from example where (val is null or val == 0);
> -- returns 1 row - 0
> select * from example where not (val is not null and val <> 0);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11375) Broken processing of queries containing NOT (x IS NOT NULL and x <> 0)
[
https://issues.apache.org/jira/browse/HIVE-11375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-11375:
Attachment: HIVE-11375.patch
> Broken processing of queries containing NOT (x IS NOT NULL and x <> 0)
> --
>
> Key: HIVE-11375
> URL: https://issues.apache.org/jira/browse/HIVE-11375
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Affects Versions: 2.0.0
>Reporter: Mariusz Sakowski
>Assignee: Aihua Xu
> Attachments: HIVE-11375.patch
>
>
> When running query like this:
> {code}explain select * from test where (val is not null and val <> 0);{code}
> hive will simplify expression in parenthesis and omit is not null check:
> {code}
> Filter Operator
> predicate: (val <> 0) (type: boolean)
> {code}
> which is fine.
> but if we negate condition using NOT operator:
> {code}explain select * from test where not (val is not null and val <>
> 0);{code}
> hive will also simplify thing, but now it will break stuff:
> {code}
> Filter Operator
> predicate: (not (val <> 0)) (type: boolean)
> {code}
> because valid predicate should be *val == 0 or val is null*, while above row
> is equivalent to *val == 0* only, filtering away rows where val is null
> simple example:
> {code}
> CREATE TABLE example (
> val bigint
> );
> INSERT INTO example VALUES (1), (NULL), (0);
> -- returns 2 rows - NULL and 0
> select * from example where (val is null or val == 0);
> -- returns 1 row - 0
> select * from example where not (val is not null and val <> 0);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11375) Broken processing of queries containing NOT (x IS NOT NULL and x <> 0)
[
https://issues.apache.org/jira/browse/HIVE-11375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-11375:
Component/s: (was: SQL)
Logical Optimizer
> Broken processing of queries containing NOT (x IS NOT NULL and x <> 0)
> --
>
> Key: HIVE-11375
> URL: https://issues.apache.org/jira/browse/HIVE-11375
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Affects Versions: 2.0.0
>Reporter: Mariusz Sakowski
>Assignee: Aihua Xu
>
> When running query like this:
> {code}explain select * from test where (val is not null and val <> 0);{code}
> hive will simplify expression in parenthesis and omit is not null check:
> {code}
> Filter Operator
> predicate: (val <> 0) (type: boolean)
> {code}
> which is fine.
> but if we negate condition using NOT operator:
> {code}explain select * from test where not (val is not null and val <>
> 0);{code}
> hive will also simplify thing, but now it will break stuff:
> {code}
> Filter Operator
> predicate: (not (val <> 0)) (type: boolean)
> {code}
> because valid predicate should be *val == 0 or val is null*, while above row
> is equivalent to *val == 0* only, filtering away rows where val is null
> simple example:
> {code}
> CREATE TABLE example (
> val bigint
> );
> INSERT INTO example VALUES (1), (NULL), (0);
> -- returns 2 rows - NULL and 0
> select * from example where (val is null or val == 0);
> -- returns 1 row - 0
> select * from example where not (val is not null and val <> 0);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11375) Broken processing of queries containing NOT (x IS NOT NULL and x <> 0)
[
https://issues.apache.org/jira/browse/HIVE-11375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-11375:
Affects Version/s: 2.0.0
> Broken processing of queries containing NOT (x IS NOT NULL and x <> 0)
> --
>
> Key: HIVE-11375
> URL: https://issues.apache.org/jira/browse/HIVE-11375
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Affects Versions: 2.0.0
>Reporter: Mariusz Sakowski
>Assignee: Aihua Xu
>
> When running query like this:
> {code}explain select * from test where (val is not null and val <> 0);{code}
> hive will simplify expression in parenthesis and omit is not null check:
> {code}
> Filter Operator
> predicate: (val <> 0) (type: boolean)
> {code}
> which is fine.
> but if we negate condition using NOT operator:
> {code}explain select * from test where not (val is not null and val <>
> 0);{code}
> hive will also simplify thing, but now it will break stuff:
> {code}
> Filter Operator
> predicate: (not (val <> 0)) (type: boolean)
> {code}
> because valid predicate should be *val == 0 or val is null*, while above row
> is equivalent to *val == 0* only, filtering away rows where val is null
> simple example:
> {code}
> CREATE TABLE example (
> val bigint
> );
> INSERT INTO example VALUES (1), (NULL), (0);
> -- returns 2 rows - NULL and 0
> select * from example where (val is null or val == 0);
> -- returns 1 row - 0
> select * from example where not (val is not null and val <> 0);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
