[jira] [Updated] (HIVE-27335) Backport of HIVE-25468: Authorization for Create/Drop functions in HMS

2023-05-23 Thread Sankar Hariappan (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sankar Hariappan updated HIVE-27335:

Summary: Backport of HIVE-25468: Authorization for Create/Drop functions in 
HMS  (was: Backport of HIVE-25468: Authorization for Create/Drop functions in 
HMS(Saihemanth Gantasala via Naveen Gangam))

> Backport of HIVE-25468: Authorization for Create/Drop functions in HMS
> --
>
> Key: HIVE-27335
> URL: https://issues.apache.org/jira/browse/HIVE-27335
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Diksha
>Assignee: Diksha
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.2.0
>
>
> Backport of HIVE-25468: Authorization for Create/Drop functions in 
> HMS(Saihemanth Gantasala via Naveen Gangam)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-27334) Backport of HIVE-25726: Upgrade velocity to 2.3 due to CVE-2020-13936

2023-05-23 Thread Sankar Hariappan (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sankar Hariappan updated HIVE-27334:

Summary: Backport of HIVE-25726: Upgrade velocity to 2.3 due to 
CVE-2020-13936  (was: Backport of HIVE-25726: Upgrade velocity to 2.3 due to 
CVE-2020-13936 (Sourabh Goyal via Naveen Gangam))

> Backport of HIVE-25726: Upgrade velocity to 2.3 due to CVE-2020-13936
> -
>
> Key: HIVE-27334
> URL: https://issues.apache.org/jira/browse/HIVE-27334
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Diksha
>Assignee: Diksha
>Priority: Major
> Fix For: 3.2.0
>
>
> Backport of HIVE-25726: Upgrade velocity to 2.3 due to CVE-2020-13936 
> (Sourabh Goyal via Naveen Gangam)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (HIVE-27334) Backport of HIVE-25726: Upgrade velocity to 2.3 due to CVE-2020-13936 (Sourabh Goyal via Naveen Gangam)

2023-05-23 Thread Sankar Hariappan (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sankar Hariappan resolved HIVE-27334.
-
Resolution: Fixed

> Backport of HIVE-25726: Upgrade velocity to 2.3 due to CVE-2020-13936 
> (Sourabh Goyal via Naveen Gangam)
> ---
>
> Key: HIVE-27334
> URL: https://issues.apache.org/jira/browse/HIVE-27334
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Diksha
>Assignee: Diksha
>Priority: Major
> Fix For: 3.2.0
>
>
> Backport of HIVE-25726: Upgrade velocity to 2.3 due to CVE-2020-13936 
> (Sourabh Goyal via Naveen Gangam)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-27334) Backport of HIVE-25726: Upgrade velocity to 2.3 due to CVE-2020-13936 (Sourabh Goyal via Naveen Gangam)

2023-05-23 Thread Sankar Hariappan (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sankar Hariappan updated HIVE-27334:

Fix Version/s: 3.2.0

> Backport of HIVE-25726: Upgrade velocity to 2.3 due to CVE-2020-13936 
> (Sourabh Goyal via Naveen Gangam)
> ---
>
> Key: HIVE-27334
> URL: https://issues.apache.org/jira/browse/HIVE-27334
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Diksha
>Assignee: Diksha
>Priority: Major
> Fix For: 3.2.0
>
>
> Backport of HIVE-25726: Upgrade velocity to 2.3 due to CVE-2020-13936 
> (Sourabh Goyal via Naveen Gangam)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (HIVE-27335) Backport of HIVE-25468: Authorization for Create/Drop functions in HMS(Saihemanth Gantasala via Naveen Gangam)

2023-05-23 Thread Sankar Hariappan (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sankar Hariappan resolved HIVE-27335.
-
Resolution: Fixed

> Backport of HIVE-25468: Authorization for Create/Drop functions in 
> HMS(Saihemanth Gantasala via Naveen Gangam)
> --
>
> Key: HIVE-27335
> URL: https://issues.apache.org/jira/browse/HIVE-27335
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Diksha
>Assignee: Diksha
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.2.0
>
>
> Backport of HIVE-25468: Authorization for Create/Drop functions in 
> HMS(Saihemanth Gantasala via Naveen Gangam)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-27335) Backport of HIVE-25468: Authorization for Create/Drop functions in HMS(Saihemanth Gantasala via Naveen Gangam)

2023-05-23 Thread Sankar Hariappan (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sankar Hariappan updated HIVE-27335:

Fix Version/s: 3.2.0

> Backport of HIVE-25468: Authorization for Create/Drop functions in 
> HMS(Saihemanth Gantasala via Naveen Gangam)
> --
>
> Key: HIVE-27335
> URL: https://issues.apache.org/jira/browse/HIVE-27335
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Diksha
>Assignee: Diksha
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.2.0
>
>
> Backport of HIVE-25468: Authorization for Create/Drop functions in 
> HMS(Saihemanth Gantasala via Naveen Gangam)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (HIVE-27314) Backport of HIVE-25600: Compaction job creates redundant base/delta folder within base/delta folder

2023-05-23 Thread Sankar Hariappan (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sankar Hariappan resolved HIVE-27314.
-
Resolution: Fixed

> Backport of HIVE-25600: Compaction job creates redundant base/delta folder 
> within base/delta folder
> ---
>
> Key: HIVE-27314
> URL: https://issues.apache.org/jira/browse/HIVE-27314
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Diksha
>Assignee: Diksha
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.2.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Backport of HIVE-25600: Compaction job creates redundant base/delta folder 
> within base/delta folder



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-27314) Backport of HIVE-25600: Compaction job creates redundant base/delta folder within base/delta folder

2023-05-23 Thread Sankar Hariappan (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sankar Hariappan updated HIVE-27314:

Fix Version/s: 3.2.0

> Backport of HIVE-25600: Compaction job creates redundant base/delta folder 
> within base/delta folder
> ---
>
> Key: HIVE-27314
> URL: https://issues.apache.org/jira/browse/HIVE-27314
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Diksha
>Assignee: Diksha
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.2.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Backport of HIVE-25600: Compaction job creates redundant base/delta folder 
> within base/delta folder



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (HIVE-22601) Some columns will be lost when a UDTF has multiple aliases in some cases

2023-05-23 Thread okumin (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

okumin resolved HIVE-22601.
---
Resolution: Duplicate

I verified this issue doesn't happen with Hive 4.

 
{code:java}
0: jdbc:hive2://hive-hiveserver2:1/defaul> SELECT stack(1, 'a', 'b', 'c') 
AS (c1, c2, c3)
. . . . . . . . . . . . . . . . . . . . . . .> UNION ALL
. . . . . . . . . . . . . . . . . . . . . . .> SELECT stack(1, 'd', 'e', 'f') 
AS (c1, c2, c3);
...
+-+-+-+
| c1  | c2  | c3  |
+-+-+-+
| a   | b   | c   |
| d   | e   | f   |
+-+-+-+ {code}
Looks like, HIVE-25754 resolved this issue. Thanks!

 

> Some columns will be lost when a UDTF has multiple aliases in some cases
> 
>
> Key: HIVE-22601
> URL: https://issues.apache.org/jira/browse/HIVE-22601
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 2.1.1, 2.2.0, 2.3.6, 3.1.2
>Reporter: okumin
>Assignee: okumin
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-22601.1.patch, HIVE-22601.2.patch, 
> HIVE-22601.3.patch, HIVE-22601.patch
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Only one column will be retained when putting UDTFs with multiple aliases and 
> a top-level UNION together.
> For example, the result of the following SQL should have three columns, c1, 
> c2 and c3.
> {code:java}
> SELECT stack(1, 'a', 'b', 'c') AS (c1, c2, c3)
> UNION ALL
> SELECT stack(1, 'd', 'e', 'f') AS (c1, c2, c3);
> {code}
> However, It's only the c3 column which I can get.
> {code:java}
> +-+
> | _u1.c3  |
> +-+
> | c   |
> | f   |
> +-+
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-22601) Some columns will be lost when a UDTF has multiple aliases in some cases

2023-05-23 Thread okumin (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

okumin updated HIVE-22601:
--
Status: Open  (was: Patch Available)

> Some columns will be lost when a UDTF has multiple aliases in some cases
> 
>
> Key: HIVE-22601
> URL: https://issues.apache.org/jira/browse/HIVE-22601
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 3.1.2, 2.3.6, 2.2.0, 2.1.1
>Reporter: okumin
>Assignee: okumin
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-22601.1.patch, HIVE-22601.2.patch, 
> HIVE-22601.3.patch, HIVE-22601.patch
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Only one column will be retained when putting UDTFs with multiple aliases and 
> a top-level UNION together.
> For example, the result of the following SQL should have three columns, c1, 
> c2 and c3.
> {code:java}
> SELECT stack(1, 'a', 'b', 'c') AS (c1, c2, c3)
> UNION ALL
> SELECT stack(1, 'd', 'e', 'f') AS (c1, c2, c3);
> {code}
> However, It's only the c3 column which I can get.
> {code:java}
> +-+
> | _u1.c3  |
> +-+
> | c   |
> | f   |
> +-+
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (HIVE-27369) Replace log4j 1.x with reload4j

2023-05-23 Thread Devaspati Krishnatri (Jira)
Devaspati Krishnatri created HIVE-27369:
---

 Summary: Replace log4j 1.x with reload4j
 Key: HIVE-27369
 URL: https://issues.apache.org/jira/browse/HIVE-27369
 Project: Hive
  Issue Type: Task
Reporter: Devaspati Krishnatri
Assignee: Devaspati Krishnatri






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-27365) Fix test acid_bloom_filter_orc_file_dump

2023-05-23 Thread Krisztian Kasa (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krisztian Kasa updated HIVE-27365:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Merged to master. Thanks [~zabetak], [~kokila19], [~aturoczy] for review.

> Fix test acid_bloom_filter_orc_file_dump
> 
>
> Key: HIVE-27365
> URL: https://issues.apache.org/jira/browse/HIVE-27365
> Project: Hive
>  Issue Type: Bug
>Reporter: Kokila N
>Assignee: Krisztian Kasa
>Priority: Minor
>  Labels: pull-request-available
>
> This test dumps orc file data and the table in the test has 2 delta 
> directories with one orc file in each
> Hive has a posthook which scans all the directories of a table and dumps all 
> orc files to the output but the order of the directory list is not 
> deterministic.
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/hooks/PostExecOrcFileDump.java



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-27088) Incorrect results when inner and outer joins with post join filters are merged

2023-05-23 Thread Krisztian Kasa (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krisztian Kasa updated HIVE-27088:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Merged to master. Thanks [~ryu_kobayashi] for the fix.

> Incorrect results when inner and outer joins with post join filters are merged
> --
>
> Key: HIVE-27088
> URL: https://issues.apache.org/jira/browse/HIVE-27088
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Reporter: Ryu Kobayashi
>Assignee: Ryu Kobayashi
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 4h
>  Remaining Estimate: 0h
>
> When hive.merge.nway.joins is set to `true` and JOIN is performed with INNER 
> and OUTER and a filter exists, normal results cannot be obtained.
> For example:
> {code:java}
> -- Data preparation
> create temporary table foo (id bigint, code string) stored as orc;
> create temporary table bar (id bigint, code string) stored as orc;
> create temporary table baz (id bigint) stored as orc;
> INSERT INTO foo values
>   (252073, '01'),
>   (252107, '01'),
>   (252111, '01'),
>   (252112, '01'),
>   (252113, '01'),
>   (252114, '01'),
>   (252071, '01A'),
>   (252072, '01A'),
>   (252116, '01A'),
>   (252117, '01A'),
>   (252118, '01A'),
>   (252119, '01A'),
>   (252120, '01A'),
>   (252076, '06'),
>   (252074, '06A'),
>   (252075, '06A');INSERT INTO bar values
>   (252071, '01'),
>   (252072, '01'),
>   (252073, '01'),
>   (252116, '01'),
>   (252117, '01'),
>   (252071, '01A'),
>   (252072, '01A'),
>   (252073, '01A'),
>   (252116, '01AS'),
>   (252117, '01AS'),
>   (252071, '01B'),
>   (252072, '01B'),
>   (252073, '01B'),
>   (252116, '01BS'),
>   (252117, '01BS'),
>   (252071, '01C'),
>   (252072, '01C'),
>   (252073, '01C7'),
>   (252116, '01CS'),
>   (252117, '01CS'),
>   (252071, '01D'),
>   (252072, '01D'),
>   (252073, '01D'),
>   (252116, '01DS'),
>   (252117, '01DS');INSERT INTO baz values
>   (252071),
>   (252072),
>   (252073),
>   (252074),
>   (252075),
>   (252076),
>   (252107),
>   (252111),
>   (252112),
>   (252113),
>   (252114),
>   (252116),
>   (252117),
>   (252118),
>   (252119),
>   (252120);{code}
> Normal works(set hive.merge.nway.joins=false):
> {code:java}
> hive> set hive.merge.nway.joins=false;
> hive> SELECT
>   a.id,
>   b.code,
>   c.id
> FROM bar AS a
> INNER JOIN foo AS b
> ON a.id = b.id
>   AND (a.code = '01AS' OR b.code = '01BS')
> LEFT OUTER JOIN baz AS c
> ON a.id = c.id;
> OK
> 252116  01A     252116
> 252117  01A     252117 {code}
> Abnormal works(set hive.merge.nway.joins=true):
> {code:java}
> hive> set hive.merge.nway.joins=true;
> hive> SELECT
>   a.id,
>   b.code,
>   c.id
> FROM bar AS a
> INNER JOIN foo AS b
> ON a.id = b.id
>   AND (a.code = '01AS' OR b.code = '01BS')
> LEFT OUTER JOIN baz AS c
> ON a.id = c.id;
> OK 252071  01A     NULL
> 252072  01A     NULL
> 252073  01      NULL
> 252116  01A     NULL
> 252117  01A     NULL
> 252071  01A     NULL
> 252072  01A     NULL
> 252073  01      NULL
> 252116  01A     252116
> 252117  01A     252117
> 252071  01A     NULL
> 252072  01A     NULL
> 252073  01      NULL
> 252116  01A     NULL
> 252117  01A     NULL
> 252071  01A     NULL
> 252072  01A     NULL
> 252073  01      NULL
> 252116  01A     NULL
> 252117  01A     NULL
> 252071  01A     NULL
> 252072  01A     NULL
> 252073  01      NULL
> 252116  01A     NULL
> 252117  01A     NULL   {code}
>  
> I think this is also related to the next ticket: 
> https://issues.apache.org/jira/browse/HIVE-21322



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-27088) Incorrect results when inner and outer joins with post join filters are merged

2023-05-23 Thread Krisztian Kasa (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krisztian Kasa updated HIVE-27088:
--
Summary: Incorrect results when inner and outer joins with post join 
filters are merged  (was: Incorrect result when inner and outer joins with post 
join filters are merged)

> Incorrect results when inner and outer joins with post join filters are merged
> --
>
> Key: HIVE-27088
> URL: https://issues.apache.org/jira/browse/HIVE-27088
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Reporter: Ryu Kobayashi
>Assignee: Ryu Kobayashi
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 4h
>  Remaining Estimate: 0h
>
> When hive.merge.nway.joins is set to `true` and JOIN is performed with INNER 
> and OUTER and a filter exists, normal results cannot be obtained.
> For example:
> {code:java}
> -- Data preparation
> create temporary table foo (id bigint, code string) stored as orc;
> create temporary table bar (id bigint, code string) stored as orc;
> create temporary table baz (id bigint) stored as orc;
> INSERT INTO foo values
>   (252073, '01'),
>   (252107, '01'),
>   (252111, '01'),
>   (252112, '01'),
>   (252113, '01'),
>   (252114, '01'),
>   (252071, '01A'),
>   (252072, '01A'),
>   (252116, '01A'),
>   (252117, '01A'),
>   (252118, '01A'),
>   (252119, '01A'),
>   (252120, '01A'),
>   (252076, '06'),
>   (252074, '06A'),
>   (252075, '06A');INSERT INTO bar values
>   (252071, '01'),
>   (252072, '01'),
>   (252073, '01'),
>   (252116, '01'),
>   (252117, '01'),
>   (252071, '01A'),
>   (252072, '01A'),
>   (252073, '01A'),
>   (252116, '01AS'),
>   (252117, '01AS'),
>   (252071, '01B'),
>   (252072, '01B'),
>   (252073, '01B'),
>   (252116, '01BS'),
>   (252117, '01BS'),
>   (252071, '01C'),
>   (252072, '01C'),
>   (252073, '01C7'),
>   (252116, '01CS'),
>   (252117, '01CS'),
>   (252071, '01D'),
>   (252072, '01D'),
>   (252073, '01D'),
>   (252116, '01DS'),
>   (252117, '01DS');INSERT INTO baz values
>   (252071),
>   (252072),
>   (252073),
>   (252074),
>   (252075),
>   (252076),
>   (252107),
>   (252111),
>   (252112),
>   (252113),
>   (252114),
>   (252116),
>   (252117),
>   (252118),
>   (252119),
>   (252120);{code}
> Normal works(set hive.merge.nway.joins=false):
> {code:java}
> hive> set hive.merge.nway.joins=false;
> hive> SELECT
>   a.id,
>   b.code,
>   c.id
> FROM bar AS a
> INNER JOIN foo AS b
> ON a.id = b.id
>   AND (a.code = '01AS' OR b.code = '01BS')
> LEFT OUTER JOIN baz AS c
> ON a.id = c.id;
> OK
> 252116  01A     252116
> 252117  01A     252117 {code}
> Abnormal works(set hive.merge.nway.joins=true):
> {code:java}
> hive> set hive.merge.nway.joins=true;
> hive> SELECT
>   a.id,
>   b.code,
>   c.id
> FROM bar AS a
> INNER JOIN foo AS b
> ON a.id = b.id
>   AND (a.code = '01AS' OR b.code = '01BS')
> LEFT OUTER JOIN baz AS c
> ON a.id = c.id;
> OK 252071  01A     NULL
> 252072  01A     NULL
> 252073  01      NULL
> 252116  01A     NULL
> 252117  01A     NULL
> 252071  01A     NULL
> 252072  01A     NULL
> 252073  01      NULL
> 252116  01A     252116
> 252117  01A     252117
> 252071  01A     NULL
> 252072  01A     NULL
> 252073  01      NULL
> 252116  01A     NULL
> 252117  01A     NULL
> 252071  01A     NULL
> 252072  01A     NULL
> 252073  01      NULL
> 252116  01A     NULL
> 252117  01A     NULL
> 252071  01A     NULL
> 252072  01A     NULL
> 252073  01      NULL
> 252116  01A     NULL
> 252117  01A     NULL   {code}
>  
> I think this is also related to the next ticket: 
> https://issues.apache.org/jira/browse/HIVE-21322



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-27088) Incorrect result when inner and outer joins with post join filters are merged

2023-05-23 Thread Krisztian Kasa (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krisztian Kasa updated HIVE-27088:
--
Summary: Incorrect result when inner and outer joins with post join filters 
are merged  (was: Using MergeJoin and using filters does not work)

> Incorrect result when inner and outer joins with post join filters are merged
> -
>
> Key: HIVE-27088
> URL: https://issues.apache.org/jira/browse/HIVE-27088
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Reporter: Ryu Kobayashi
>Assignee: Ryu Kobayashi
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 4h
>  Remaining Estimate: 0h
>
> When hive.merge.nway.joins is set to `true` and JOIN is performed with INNER 
> and OUTER and a filter exists, normal results cannot be obtained.
> For example:
> {code:java}
> -- Data preparation
> create temporary table foo (id bigint, code string) stored as orc;
> create temporary table bar (id bigint, code string) stored as orc;
> create temporary table baz (id bigint) stored as orc;
> INSERT INTO foo values
>   (252073, '01'),
>   (252107, '01'),
>   (252111, '01'),
>   (252112, '01'),
>   (252113, '01'),
>   (252114, '01'),
>   (252071, '01A'),
>   (252072, '01A'),
>   (252116, '01A'),
>   (252117, '01A'),
>   (252118, '01A'),
>   (252119, '01A'),
>   (252120, '01A'),
>   (252076, '06'),
>   (252074, '06A'),
>   (252075, '06A');INSERT INTO bar values
>   (252071, '01'),
>   (252072, '01'),
>   (252073, '01'),
>   (252116, '01'),
>   (252117, '01'),
>   (252071, '01A'),
>   (252072, '01A'),
>   (252073, '01A'),
>   (252116, '01AS'),
>   (252117, '01AS'),
>   (252071, '01B'),
>   (252072, '01B'),
>   (252073, '01B'),
>   (252116, '01BS'),
>   (252117, '01BS'),
>   (252071, '01C'),
>   (252072, '01C'),
>   (252073, '01C7'),
>   (252116, '01CS'),
>   (252117, '01CS'),
>   (252071, '01D'),
>   (252072, '01D'),
>   (252073, '01D'),
>   (252116, '01DS'),
>   (252117, '01DS');INSERT INTO baz values
>   (252071),
>   (252072),
>   (252073),
>   (252074),
>   (252075),
>   (252076),
>   (252107),
>   (252111),
>   (252112),
>   (252113),
>   (252114),
>   (252116),
>   (252117),
>   (252118),
>   (252119),
>   (252120);{code}
> Normal works(set hive.merge.nway.joins=false):
> {code:java}
> hive> set hive.merge.nway.joins=false;
> hive> SELECT
>   a.id,
>   b.code,
>   c.id
> FROM bar AS a
> INNER JOIN foo AS b
> ON a.id = b.id
>   AND (a.code = '01AS' OR b.code = '01BS')
> LEFT OUTER JOIN baz AS c
> ON a.id = c.id;
> OK
> 252116  01A     252116
> 252117  01A     252117 {code}
> Abnormal works(set hive.merge.nway.joins=true):
> {code:java}
> hive> set hive.merge.nway.joins=true;
> hive> SELECT
>   a.id,
>   b.code,
>   c.id
> FROM bar AS a
> INNER JOIN foo AS b
> ON a.id = b.id
>   AND (a.code = '01AS' OR b.code = '01BS')
> LEFT OUTER JOIN baz AS c
> ON a.id = c.id;
> OK 252071  01A     NULL
> 252072  01A     NULL
> 252073  01      NULL
> 252116  01A     NULL
> 252117  01A     NULL
> 252071  01A     NULL
> 252072  01A     NULL
> 252073  01      NULL
> 252116  01A     252116
> 252117  01A     252117
> 252071  01A     NULL
> 252072  01A     NULL
> 252073  01      NULL
> 252116  01A     NULL
> 252117  01A     NULL
> 252071  01A     NULL
> 252072  01A     NULL
> 252073  01      NULL
> 252116  01A     NULL
> 252117  01A     NULL
> 252071  01A     NULL
> 252072  01A     NULL
> 252073  01      NULL
> 252116  01A     NULL
> 252117  01A     NULL   {code}
>  
> I think this is also related to the next ticket: 
> https://issues.apache.org/jira/browse/HIVE-21322



--
This message was sent by Atlassian Jira
(v8.20.10#820010)