[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver

2019-12-16 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22608:
--
Attachment: HIVE-22608.01.patch

> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong 
> there, or should not be public.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver

2019-12-16 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22608:
--
Attachment: (was: HIVE-22608.01.patch)

> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong 
> there, or should not be public.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver

2019-12-16 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22608:
--
Attachment: HIVE-22608.01.patch

> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong 
> there, or should not be public.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver

2019-12-16 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22608:
--
Attachment: (was: HIVE-22608.01.patch)

> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong 
> there, or should not be public.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22608) Reduce the number of public methods in Driver

2019-12-16 Thread Miklos Gergely (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16997080#comment-16997080
 ] 

Miklos Gergely commented on HIVE-22608:
---

No, they have nothing to do with the patch. The same patch was tested at the 
first run, and that time 

TestHiveCli did not fail.

> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong 
> there, or should not be public.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22638) Fix insert statement issue with return path

2019-12-16 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22638:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Fix insert statement issue with return path
> ---
>
> Key: HIVE-22638
> URL: https://issues.apache.org/jira/browse/HIVE-22638
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-22638.01.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Insert statements were not handled properly with return path. It was revealed 
> during examining why TestUpgradeTool is not working with return path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver

2019-12-15 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22608:
--
Attachment: (was: HIVE-22608.01.patch)

> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong 
> there, or should not be public.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver

2019-12-15 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22608:
--
Attachment: HIVE-22608.01.patch

> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong 
> there, or should not be public.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver

2019-12-14 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22608:
--
Attachment: (was: HIVE-22608.01.patch)

> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong 
> there, or should not be public.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver

2019-12-14 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22608:
--
Attachment: HIVE-22608.01.patch

> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong 
> there, or should not be public.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver

2019-12-14 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22608:
--
Attachment: (was: HIVE-22608.01.patch)

> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong 
> there, or should not be public.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver

2019-12-14 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22608:
--
Attachment: HIVE-22608.01.patch

> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong 
> there, or should not be public.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver

2019-12-14 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22608:
--
Attachment: (was: HIVE-22608.01.patch)

> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong 
> there, or should not be public.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver

2019-12-14 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22608:
--
Attachment: HIVE-22608.01.patch

> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong 
> there, or should not be public.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22638) Fix insert statement issue with return path

2019-12-13 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22638:
--
Attachment: HIVE-22638.01.patch

> Fix insert statement issue with return path
> ---
>
> Key: HIVE-22638
> URL: https://issues.apache.org/jira/browse/HIVE-22638
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-22638.01.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Insert statements were not handled properly with return path. It was revealed 
> during examining why TestUpgradeTool is not working with return path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22638) Fix insert statement issue with return path

2019-12-13 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22638:
--
Attachment: (was: HIVE-22638.01.patch)

> Fix insert statement issue with return path
> ---
>
> Key: HIVE-22638
> URL: https://issues.apache.org/jira/browse/HIVE-22638
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Insert statements were not handled properly with return path. It was revealed 
> during examining why TestUpgradeTool is not working with return path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver

2019-12-13 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22608:
--
Attachment: HIVE-22608.01.patch

> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong 
> there, or should not be public.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver

2019-12-13 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22608:
--
Attachment: (was: HIVE-22608.01.patch)

> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong 
> there, or should not be public.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22644) Break up DDLSemanticAnalyzer - extract Table column analyzers

2019-12-13 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22644:
--
Attachment: HIVE-22644.01.patch

> Break up DDLSemanticAnalyzer - extract Table column analyzers
> -
>
> Key: HIVE-22644
> URL: https://issues.apache.org/jira/browse/HIVE-22644
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22644.01.patch
>
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #11: extract the table column related analyzers from 
> DDLSemanticAnalyzer, and move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22644) Break up DDLSemanticAnalyzer - extract Table column analyzers

2019-12-13 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22644:
--
Status: Patch Available  (was: Open)

> Break up DDLSemanticAnalyzer - extract Table column analyzers
> -
>
> Key: HIVE-22644
> URL: https://issues.apache.org/jira/browse/HIVE-22644
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22644.01.patch
>
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #11: extract the table column related analyzers from 
> DDLSemanticAnalyzer, and move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (HIVE-22644) Break up DDLSemanticAnalyzer - extract Table column analyzers

2019-12-13 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely reassigned HIVE-22644:
-


> Break up DDLSemanticAnalyzer - extract Table column analyzers
> -
>
> Key: HIVE-22644
> URL: https://issues.apache.org/jira/browse/HIVE-22644
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available, refactor-ddl
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #10: extract the table constraints related analyzers from 
> DDLSemanticAnalyzer, and move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22644) Break up DDLSemanticAnalyzer - extract Table column analyzers

2019-12-13 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22644:
--
Description: 
DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is to 
refactor it in order to have everything cut into more handleable classes under 
the package  org.apache.hadoop.hive.ql.exec.ddl:
 * have a separate class for each analyzers
 * have a package for each operation, containing an analyzer, a description, 
and an operation, so the amount of classes under a package is more manageable

Step #11: extract the table column related analyzers from DDLSemanticAnalyzer, 
and move them under the new package.

  was:
DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is to 
refactor it in order to have everything cut into more handleable classes under 
the package  org.apache.hadoop.hive.ql.exec.ddl:
 * have a separate class for each analyzers
 * have a package for each operation, containing an analyzer, a description, 
and an operation, so the amount of classes under a package is more manageable

Step #10: extract the table constraints related analyzers from 
DDLSemanticAnalyzer, and move them under the new package.


> Break up DDLSemanticAnalyzer - extract Table column analyzers
> -
>
> Key: HIVE-22644
> URL: https://issues.apache.org/jira/browse/HIVE-22644
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available, refactor-ddl
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #11: extract the table column related analyzers from 
> DDLSemanticAnalyzer, and move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22557) Break up DDLSemanticAnalyzer - extract Table constraints analyzers

2019-12-12 Thread Miklos Gergely (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995436#comment-16995436
 ] 

Miklos Gergely commented on HIVE-22557:
---

Fixed the type, thanks!

> Break up DDLSemanticAnalyzer - extract Table constraints analyzers
> --
>
> Key: HIVE-22557
> URL: https://issues.apache.org/jira/browse/HIVE-22557
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22557.01.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #10: extract the table constraints related analyzers from 
> DDLSemanticAnalyzer, and move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22557) Break up DDLSemanticAnalyzer - extract Table constraints analyzers

2019-12-12 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22557:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Break up DDLSemanticAnalyzer - extract Table constraints analyzers
> --
>
> Key: HIVE-22557
> URL: https://issues.apache.org/jira/browse/HIVE-22557
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22557.01.patch
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #10: extract the table constraints related analyzers from 
> DDLSemanticAnalyzer, and move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22638) Fix insert statement issue with return path

2019-12-12 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22638:
--
Attachment: HIVE-22638.01.patch

> Fix insert statement issue with return path
> ---
>
> Key: HIVE-22638
> URL: https://issues.apache.org/jira/browse/HIVE-22638
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-22638.01.patch
>
>
> Insert statements were not handled properly with return path. It was revealed 
> during examining why TestUpgradeTool is not working with return path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22638) Fix insert statement issue with return path

2019-12-12 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22638:
--
Attachment: (was: HIVE-22638.01.patch)

> Fix insert statement issue with return path
> ---
>
> Key: HIVE-22638
> URL: https://issues.apache.org/jira/browse/HIVE-22638
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>
> Insert statements were not handled properly with return path. It was revealed 
> during examining why TestUpgradeTool is not working with return path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22638) Fix insert statement issue with return path

2019-12-12 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22638:
--
Status: Patch Available  (was: Open)

> Fix insert statement issue with return path
> ---
>
> Key: HIVE-22638
> URL: https://issues.apache.org/jira/browse/HIVE-22638
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-22638.01.patch
>
>
> Insert statements were not handled properly with return path. It was revealed 
> during examining why TestUpgradeTool is not working with return path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22638) Fix insert statement issue with return path

2019-12-12 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22638:
--
Attachment: HIVE-22638.01.patch

> Fix insert statement issue with return path
> ---
>
> Key: HIVE-22638
> URL: https://issues.apache.org/jira/browse/HIVE-22638
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-22638.01.patch
>
>
> Insert statements were not handled properly with return path. It was revealed 
> during examining why TestUpgradeTool is not working with return path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22638) Fix insert statement issue with return path

2019-12-12 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22638:
--
Parent: HIVE-9132
Issue Type: Sub-task  (was: Bug)

> Fix insert statement issue with return path
> ---
>
> Key: HIVE-22638
> URL: https://issues.apache.org/jira/browse/HIVE-22638
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>
> Insert statements were not handled properly with return path. It was revealed 
> during examining why TestUpgradeTool is not working with return path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (HIVE-22638) Fix insert statement issue with return path

2019-12-12 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely reassigned HIVE-22638:
-


> Fix insert statement issue with return path
> ---
>
> Key: HIVE-22638
> URL: https://issues.apache.org/jira/browse/HIVE-22638
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>
> Insert statements were not handled properly with return path. It was revealed 
> during examining why TestUpgradeTool is not working with return path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver

2019-12-09 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22608:
--
Status: Patch Available  (was: Open)

> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong 
> there, or should not be public.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver

2019-12-09 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22608:
--
Attachment: HIVE-22608.01.patch

> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong 
> there, or should not be public.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (HIVE-22608) Reduce the number of public methods in Driver

2019-12-09 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely reassigned HIVE-22608:
-


> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
>
> There are 33 public methods in Driver, some of them either don't belong 
> there, or should not be public.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-12-03 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, 
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch, 
> HIVE-22526.06.patch, HIVE-22526.07.patch, HIVE-22526.08.patch, 
> HIVE-22526.09.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-12-01 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: HIVE-22526.09.patch

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, 
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch, 
> HIVE-22526.06.patch, HIVE-22526.07.patch, HIVE-22526.08.patch, 
> HIVE-22526.09.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-12-01 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: (was: HIVE-22526.09.patch)

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, 
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch, 
> HIVE-22526.06.patch, HIVE-22526.07.patch, HIVE-22526.08.patch, 
> HIVE-22526.09.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-11-30 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: (was: HIVE-22526.09.patch)

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, 
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch, 
> HIVE-22526.06.patch, HIVE-22526.07.patch, HIVE-22526.08.patch, 
> HIVE-22526.09.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-11-30 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: HIVE-22526.09.patch

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, 
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch, 
> HIVE-22526.06.patch, HIVE-22526.07.patch, HIVE-22526.08.patch, 
> HIVE-22526.09.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-11-30 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: HIVE-22526.09.patch

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, 
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch, 
> HIVE-22526.06.patch, HIVE-22526.07.patch, HIVE-22526.08.patch, 
> HIVE-22526.09.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-11-30 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: HIVE-22526.08.patch

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, 
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch, 
> HIVE-22526.06.patch, HIVE-22526.07.patch, HIVE-22526.08.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-11-30 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: HIVE-22526.07.patch

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, 
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch, 
> HIVE-22526.06.patch, HIVE-22526.07.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-11-30 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: (was: HIVE-22526.07.patch)

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, 
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch, 
> HIVE-22526.06.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-11-30 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: HIVE-22526.07.patch

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, 
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch, 
> HIVE-22526.06.patch, HIVE-22526.07.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-11-30 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: HIVE-22526.06.patch

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, 
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch, 
> HIVE-22526.06.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-11-30 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: (was: HIVE-22526.06.patch)

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, 
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-11-30 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: HIVE-22526.06.patch

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, 
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch, 
> HIVE-22526.06.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-11-30 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: (was: HIVE-22526.06.patch)

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, 
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch, 
> HIVE-22526.06.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-11-29 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: HIVE-22526.06.patch

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, 
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch, 
> HIVE-22526.06.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22557) Break up DDLSemanticAnalyzer - extract Table constraints analyzers

2019-11-29 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22557:
--
Attachment: (was: HIVE-22557.01.patch)

> Break up DDLSemanticAnalyzer - extract Table constraints analyzers
> --
>
> Key: HIVE-22557
> URL: https://issues.apache.org/jira/browse/HIVE-22557
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: refactor-ddl
> Attachments: HIVE-22557.01.patch
>
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #10: extract the table constraints related analyzers from 
> DDLSemanticAnalyzer, and move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22557) Break up DDLSemanticAnalyzer - extract Table constraints analyzers

2019-11-29 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22557:
--
Attachment: HIVE-22557.01.patch

> Break up DDLSemanticAnalyzer - extract Table constraints analyzers
> --
>
> Key: HIVE-22557
> URL: https://issues.apache.org/jira/browse/HIVE-22557
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: refactor-ddl
> Attachments: HIVE-22557.01.patch
>
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #10: extract the table constraints related analyzers from 
> DDLSemanticAnalyzer, and move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22536) Improve return path enabling/disabling

2019-11-29 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22536:
--
Attachment: HIVE-22536.03.patch

> Improve return path enabling/disabling
> --
>
> Key: HIVE-22536
> URL: https://issues.apache.org/jira/browse/HIVE-22536
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22536.01.patch, HIVE-22536.02.patch, 
> HIVE-22536.03.patch
>
>
> Instead of having a boolean for hive.cbo.returnpath.hiveop it should be 
> on/off/supported. In case of "supported" it should be used for a subset of 
> commands which are already verified to be able to work with return path. This 
> is a temporary solution for the time while we are developing return path, 
> before making it the only way to handle commands.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22557) Break up DDLSemanticAnalyzer - extract Table constraints analyzers

2019-11-27 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22557:
--
Status: Patch Available  (was: Open)

> Break up DDLSemanticAnalyzer - extract Table constraints analyzers
> --
>
> Key: HIVE-22557
> URL: https://issues.apache.org/jira/browse/HIVE-22557
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22557.01.patch
>
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #10: extract the table constraints related analyzers from 
> DDLSemanticAnalyzer, and move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22557) Break up DDLSemanticAnalyzer - extract Table constraints analyzers

2019-11-27 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22557:
--
Labels: refactor-ddl  (was: pull-request-available refactor-ddl)

> Break up DDLSemanticAnalyzer - extract Table constraints analyzers
> --
>
> Key: HIVE-22557
> URL: https://issues.apache.org/jira/browse/HIVE-22557
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: refactor-ddl
> Attachments: HIVE-22557.01.patch
>
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #10: extract the table constraints related analyzers from 
> DDLSemanticAnalyzer, and move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22557) Break up DDLSemanticAnalyzer - extract Table constraints analyzers

2019-11-27 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22557:
--
Attachment: HIVE-22557.01.patch

> Break up DDLSemanticAnalyzer - extract Table constraints analyzers
> --
>
> Key: HIVE-22557
> URL: https://issues.apache.org/jira/browse/HIVE-22557
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22557.01.patch
>
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #10: extract the table constraints related analyzers from 
> DDLSemanticAnalyzer, and move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22557) Break up DDLSemanticAnalyzer - extract Table constraints analyzers

2019-11-27 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22557:
--
Description: 
DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is to 
refactor it in order to have everything cut into more handleable classes under 
the package  org.apache.hadoop.hive.ql.exec.ddl:
 * have a separate class for each analyzers
 * have a package for each operation, containing an analyzer, a description, 
and an operation, so the amount of classes under a package is more manageable

Step #10: extract the table constraints related analyzers from 
DDLSemanticAnalyzer, and move them under the new package.

  was:
DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is to 
refactor it in order to have everything cut into more handleable classes under 
the package  org.apache.hadoop.hive.ql.exec.ddl:
 * have a separate class for each analyzers
 * have a package for each operation, containing an analyzer, a description, 
and an operation, so the amount of classes under a package is more manageable

Step #9: extract the table creationanalyzers from DDLSemanticAnalyzer, and move 
them under the new package.


> Break up DDLSemanticAnalyzer - extract Table constraints analyzers
> --
>
> Key: HIVE-22557
> URL: https://issues.apache.org/jira/browse/HIVE-22557
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available, refactor-ddl
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #10: extract the table constraints related analyzers from 
> DDLSemanticAnalyzer, and move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (HIVE-22557) Break up DDLSemanticAnalyzer - extract Table constraints analyzers

2019-11-27 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely reassigned HIVE-22557:
-


> Break up DDLSemanticAnalyzer - extract Table constraints analyzers
> --
>
> Key: HIVE-22557
> URL: https://issues.apache.org/jira/browse/HIVE-22557
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available, refactor-ddl
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #9: extract the table creationanalyzers from DDLSemanticAnalyzer, and 
> move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22536) Improve return path enabling/disabling

2019-11-27 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22536:
--
Attachment: (was: HIVE-22536.02.patch)

> Improve return path enabling/disabling
> --
>
> Key: HIVE-22536
> URL: https://issues.apache.org/jira/browse/HIVE-22536
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22536.01.patch, HIVE-22536.02.patch
>
>
> Instead of having a boolean for hive.cbo.returnpath.hiveop it should be 
> on/off/supported. In case of "supported" it should be used for a subset of 
> commands which are already verified to be able to work with return path. This 
> is a temporary solution for the time while we are developing return path, 
> before making it the only way to handle commands.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22536) Improve return path enabling/disabling

2019-11-27 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22536:
--
Attachment: HIVE-22536.02.patch

> Improve return path enabling/disabling
> --
>
> Key: HIVE-22536
> URL: https://issues.apache.org/jira/browse/HIVE-22536
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22536.01.patch, HIVE-22536.02.patch
>
>
> Instead of having a boolean for hive.cbo.returnpath.hiveop it should be 
> on/off/supported. In case of "supported" it should be used for a subset of 
> commands which are already verified to be able to work with return path. This 
> is a temporary solution for the time while we are developing return path, 
> before making it the only way to handle commands.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22488) Break up DDLSemanticAnalyzer - extract Table creation analyzers

2019-11-27 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22488:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Break up DDLSemanticAnalyzer - extract Table creation analyzers
> ---
>
> Key: HIVE-22488
> URL: https://issues.apache.org/jira/browse/HIVE-22488
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22488.01.patch, HIVE-22488.03.patch
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #9: extract the table creationanalyzers from DDLSemanticAnalyzer, and 
> move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22536) Improve return path enabling/disabling

2019-11-26 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22536:
--
Attachment: HIVE-22536.02.patch

> Improve return path enabling/disabling
> --
>
> Key: HIVE-22536
> URL: https://issues.apache.org/jira/browse/HIVE-22536
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22536.01.patch, HIVE-22536.02.patch
>
>
> Instead of having a boolean for hive.cbo.returnpath.hiveop it should be 
> on/off/supported. In case of "supported" it should be used for a subset of 
> commands which are already verified to be able to work with return path. This 
> is a temporary solution for the time while we are developing return path, 
> before making it the only way to handle commands.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22488) Break up DDLSemanticAnalyzer - extract Table creation analyzers

2019-11-26 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22488:
--
Attachment: (was: HIVE-22488.03.patch)

> Break up DDLSemanticAnalyzer - extract Table creation analyzers
> ---
>
> Key: HIVE-22488
> URL: https://issues.apache.org/jira/browse/HIVE-22488
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22488.01.patch, HIVE-22488.03.patch
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #9: extract the table creationanalyzers from DDLSemanticAnalyzer, and 
> move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22536) Improve return path enabling/disabling

2019-11-26 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22536:
--
Attachment: (was: HIVE-22536.02.patch)

> Improve return path enabling/disabling
> --
>
> Key: HIVE-22536
> URL: https://issues.apache.org/jira/browse/HIVE-22536
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22536.01.patch
>
>
> Instead of having a boolean for hive.cbo.returnpath.hiveop it should be 
> on/off/supported. In case of "supported" it should be used for a subset of 
> commands which are already verified to be able to work with return path. This 
> is a temporary solution for the time while we are developing return path, 
> before making it the only way to handle commands.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22488) Break up DDLSemanticAnalyzer - extract Table creation analyzers

2019-11-26 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22488:
--
Attachment: HIVE-22488.03.patch

> Break up DDLSemanticAnalyzer - extract Table creation analyzers
> ---
>
> Key: HIVE-22488
> URL: https://issues.apache.org/jira/browse/HIVE-22488
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22488.01.patch, HIVE-22488.03.patch
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #9: extract the table creationanalyzers from DDLSemanticAnalyzer, and 
> move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22488) Break up DDLSemanticAnalyzer - extract Table creation analyzers

2019-11-26 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22488:
--
Attachment: (was: HIVE-22488.03.patch)

> Break up DDLSemanticAnalyzer - extract Table creation analyzers
> ---
>
> Key: HIVE-22488
> URL: https://issues.apache.org/jira/browse/HIVE-22488
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22488.01.patch, HIVE-22488.03.patch
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #9: extract the table creationanalyzers from DDLSemanticAnalyzer, and 
> move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22488) Break up DDLSemanticAnalyzer - extract Table creation analyzers

2019-11-26 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22488:
--
Attachment: HIVE-22488.03.patch

> Break up DDLSemanticAnalyzer - extract Table creation analyzers
> ---
>
> Key: HIVE-22488
> URL: https://issues.apache.org/jira/browse/HIVE-22488
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22488.01.patch, HIVE-22488.03.patch
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #9: extract the table creationanalyzers from DDLSemanticAnalyzer, and 
> move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22488) Break up DDLSemanticAnalyzer - extract Table creation analyzers

2019-11-26 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22488:
--
Attachment: HIVE-22488.03.patch

> Break up DDLSemanticAnalyzer - extract Table creation analyzers
> ---
>
> Key: HIVE-22488
> URL: https://issues.apache.org/jira/browse/HIVE-22488
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22488.01.patch, HIVE-22488.03.patch
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #9: extract the table creationanalyzers from DDLSemanticAnalyzer, and 
> move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22536) Improve return path enabling/disabling

2019-11-26 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22536:
--
Attachment: HIVE-22536.02.patch

> Improve return path enabling/disabling
> --
>
> Key: HIVE-22536
> URL: https://issues.apache.org/jira/browse/HIVE-22536
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22536.01.patch, HIVE-22536.02.patch
>
>
> Instead of having a boolean for hive.cbo.returnpath.hiveop it should be 
> on/off/supported. In case of "supported" it should be used for a subset of 
> commands which are already verified to be able to work with return path. This 
> is a temporary solution for the time while we are developing return path, 
> before making it the only way to handle commands.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22536) Improve return path enabling/disabling

2019-11-25 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22536:
--
Status: Patch Available  (was: Open)

> Improve return path enabling/disabling
> --
>
> Key: HIVE-22536
> URL: https://issues.apache.org/jira/browse/HIVE-22536
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22536.01.patch
>
>
> Instead of having a boolean for hive.cbo.returnpath.hiveop it should be 
> on/off/supported. In case of "supported" it should be used for a subset of 
> commands which are already verified to be able to work with return path. This 
> is a temporary solution for the time while we are developing return path, 
> before making it the only way to handle commands.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22536) Improve return path enabling/disabling

2019-11-25 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22536:
--
Attachment: HIVE-22536.01.patch

> Improve return path enabling/disabling
> --
>
> Key: HIVE-22536
> URL: https://issues.apache.org/jira/browse/HIVE-22536
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22536.01.patch
>
>
> Instead of having a boolean for hive.cbo.returnpath.hiveop it should be 
> on/off/supported. In case of "supported" it should be used for a subset of 
> commands which are already verified to be able to work with return path. This 
> is a temporary solution for the time while we are developing return path, 
> before making it the only way to handle commands.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (HIVE-22536) Improve return path enabling/disabling

2019-11-25 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely reassigned HIVE-22536:
-


> Improve return path enabling/disabling
> --
>
> Key: HIVE-22536
> URL: https://issues.apache.org/jira/browse/HIVE-22536
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
>
> Instead of having a boolean for hive.cbo.returnpath.hiveop it should be 
> on/off/supported. In case of "supported" it should be used for a subset of 
> commands which are already verified to be able to work with return path. This 
> is a temporary solution for the time while we are developing return path, 
> before making it the only way to handle commands.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22516) TestScheduledQueryIntegration fails occasionally

2019-11-25 Thread Miklos Gergely (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981417#comment-16981417
 ] 

Miklos Gergely commented on HIVE-22516:
---

+1

 

> TestScheduledQueryIntegration fails occasionally
> 
>
> Key: HIVE-22516
> URL: https://issues.apache.org/jira/browse/HIVE-22516
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22516.01.patch, HIVE-22516.02.patch
>
>
> failure seems to be caused by some filesystem level operation:
> {code}
> Failed
> org.apache.hadoop.hive.schq.TestScheduledQueryIntegration.testScheduledQueryExecutionImpersonation
> Failing for the past 2 builds (Since Failed#19506 )
> Took 21 sec.
> Error Message
> java.io.IOException: ExitCodeException exitCode=1: chmod: cannot access 
> ‘/home/hiveptest/35.224.52.88-hiveptest-0/apache-github-source-source/target/tmp/junit9072291964634791171/scratchdir/hiveptest/_tez_session_dir/d1aa15eb-d23c-4248-b509-0b29c456a1cd/.tez/application_1574237195383_0001_wd/localmode-log-dir’:
>  No such file or directory
> Stacktrace
> java.lang.RuntimeException: 
> java.io.IOException: ExitCodeException exitCode=1: chmod: cannot access 
> ‘/home/hiveptest/35.224.52.88-hiveptest-0/apache-github-source-source/target/tmp/junit9072291964634791171/scratchdir/hiveptest/_tez_session_dir/d1aa15eb-d23c-4248-b509-0b29c456a1cd/.tez/application_1574237195383_0001_wd/localmode-log-dir’:
>  No such file or directory
>   at 
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:701)
>   at 
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:606)
>   at 
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:586)
>   at 
> org.apache.hadoop.hive.schq.TestScheduledQueryIntegration.createDriver(TestScheduledQueryIntegration.java:164)
>   at 
> org.apache.hadoop.hive.schq.TestScheduledQueryIntegration.runAsUser(TestScheduledQueryIntegration.java:132)
>   at 
> org.apache.hadoop.hive.schq.TestScheduledQueryIntegration.testScheduledQueryExecutionImpersonation(TestScheduledQueryIntegration.java:115)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
>   at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>   at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
>   at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
>   at 
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
>   at 
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
>   at 
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
>   at 
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
>   at 
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
>   at 
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
>   at 
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
>   at 
> 

[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-11-24 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: HIVE-22526.05.patch

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, 
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-11-24 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: (was: HIVE-22526.05.patch)

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, 
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-11-24 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: HIVE-22526.05.patch

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, 
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-11-24 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: HIVE-22526.04.patch

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, 
> HIVE-22526.03.patch, HIVE-22526.04.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-11-23 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: HIVE-22526.03.patch

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, 
> HIVE-22526.03.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22525) Refactor HiveOpConverter

2019-11-23 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22525:
--
Attachment: HIVE-22525.02.patch

> Refactor HiveOpConverter
> 
>
> Key: HIVE-22525
> URL: https://issues.apache.org/jira/browse/HIVE-22525
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22525.01.patch, HIVE-22525.02.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> HiveOpConverter is on it's way to become a monster class. It is already ~1300 
> lines long, and expected to grow. It should be refactored, cut into multiple 
> classes in a reasonable way. It is a natural way to do this is to create 
> separate visitor classes for the different RelNodes, which are already 
> handled in different functions within HiveOpConverter. That way 
> HiveOpConverter can be the dispatcher among those visitor classes, while each 
> of them are handling some specific work, potentially requesting sub nodes to 
> be dispatched by HiveOpConverter. The functions used by multiple visitors 
> should be put into some utility class.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22525) Refactor HiveOpConverter

2019-11-23 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22525:
--
Attachment: (was: HIVE-22525.02.patch)

> Refactor HiveOpConverter
> 
>
> Key: HIVE-22525
> URL: https://issues.apache.org/jira/browse/HIVE-22525
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22525.01.patch, HIVE-22525.02.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> HiveOpConverter is on it's way to become a monster class. It is already ~1300 
> lines long, and expected to grow. It should be refactored, cut into multiple 
> classes in a reasonable way. It is a natural way to do this is to create 
> separate visitor classes for the different RelNodes, which are already 
> handled in different functions within HiveOpConverter. That way 
> HiveOpConverter can be the dispatcher among those visitor classes, while each 
> of them are handling some specific work, potentially requesting sub nodes to 
> be dispatched by HiveOpConverter. The functions used by multiple visitors 
> should be put into some utility class.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-11-22 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: HIVE-22526.02.patch

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22525) Refactor HiveOpConverter

2019-11-22 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22525:
--
Attachment: HIVE-22525.02.patch

> Refactor HiveOpConverter
> 
>
> Key: HIVE-22525
> URL: https://issues.apache.org/jira/browse/HIVE-22525
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22525.01.patch, HIVE-22525.02.patch
>
>
> HiveOpConverter is on it's way to become a monster class. It is already ~1300 
> lines long, and expected to grow. It should be refactored, cut into multiple 
> classes in a reasonable way. It is a natural way to do this is to create 
> separate visitor classes for the different RelNodes, which are already 
> handled in different functions within HiveOpConverter. That way 
> HiveOpConverter can be the dispatcher among those visitor classes, while each 
> of them are handling some specific work, potentially requesting sub nodes to 
> be dispatched by HiveOpConverter. The functions used by multiple visitors 
> should be put into some utility class.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22524) CommandProcessorException should utilize standard Exception fields

2019-11-22 Thread Miklos Gergely (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16980141#comment-16980141
 ] 

Miklos Gergely commented on HIVE-22524:
---

+1, thanks for making the code cleaner

> CommandProcessorException should utilize standard Exception fields
> --
>
> Key: HIVE-22524
> URL: https://issues.apache.org/jira/browse/HIVE-22524
> Project: Hive
>  Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22524.01.patch
>
>
> CommandProcessorException right now has:
> * getCause() inherited from Exception
> * getException() local implementation
> * getMessage() inherited from Exception
> * getErrorMessage() local implementation



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-11-21 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Attachment: HIVE-22526.01.patch

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22526) Extract Compiler from Driver

2019-11-21 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22526:
--
Status: Patch Available  (was: Open)

> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (HIVE-22526) Extract Compiler from Driver

2019-11-21 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely reassigned HIVE-22526:
-


> Extract Compiler from Driver
> 
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
>
> The Driver class contains ~600 lines of code responsible for compiling the 
> command. That means that from the command String a Plan needs to be created, 
> and also a transaction needs to be started (in most of the cases). This is a 
> thing done by the compile function, which has a lot of sub functions to help 
> this task, while itself is also really big. All these codes should be put 
> into a separate class, where it can do it's job without getting mixed with 
> the other codes in the Driver.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22525) Refactor HiveOpConverter

2019-11-21 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22525:
--
Status: Patch Available  (was: Open)

> Refactor HiveOpConverter
> 
>
> Key: HIVE-22525
> URL: https://issues.apache.org/jira/browse/HIVE-22525
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22525.01.patch
>
>
> HiveOpConverter is on it's way to become a monster class. It is already ~1300 
> lines long, and expected to grow. It should be refactored, cut into multiple 
> classes in a reasonable way. It is a natural way to do this is to create 
> separate visitor classes for the different RelNodes, which are already 
> handled in different functions within HiveOpConverter. That way 
> HiveOpConverter can be the dispatcher among those visitor classes, while each 
> of them are handling some specific work, potentially requesting sub nodes to 
> be dispatched by HiveOpConverter. The functions used by multiple visitors 
> should be put into some utility class.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22525) Refactor HiveOpConverter

2019-11-21 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22525:
--
Attachment: HIVE-22525.01.patch

> Refactor HiveOpConverter
> 
>
> Key: HIVE-22525
> URL: https://issues.apache.org/jira/browse/HIVE-22525
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22525.01.patch
>
>
> HiveOpConverter is on it's way to become a monster class. It is already ~1300 
> lines long, and expected to grow. It should be refactored, cut into multiple 
> classes in a reasonable way. It is a natural way to do this is to create 
> separate visitor classes for the different RelNodes, which are already 
> handled in different functions within HiveOpConverter. That way 
> HiveOpConverter can be the dispatcher among those visitor classes, while each 
> of them are handling some specific work, potentially requesting sub nodes to 
> be dispatched by HiveOpConverter. The functions used by multiple visitors 
> should be put into some utility class.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (HIVE-22525) Refactor HiveOpConverter

2019-11-21 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely reassigned HIVE-22525:
-


> Refactor HiveOpConverter
> 
>
> Key: HIVE-22525
> URL: https://issues.apache.org/jira/browse/HIVE-22525
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
>
> HiveOpConverter is on it's way to become a monster class. It is already ~1300 
> lines long, and expected to grow. It should be refactored, cut into multiple 
> classes in a reasonable way. It is a natural way to do this is to create 
> separate visitor classes for the different RelNodes, which are already 
> handled in different functions within HiveOpConverter. That way 
> HiveOpConverter can be the dispatcher among those visitor classes, while each 
> of them are handling some specific work, potentially requesting sub nodes to 
> be dispatched by HiveOpConverter. The functions used by multiple visitors 
> should be put into some utility class.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22369) Handle HiveTableFunctionScan at return path

2019-11-21 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22369:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Handle HiveTableFunctionScan at return path
> ---
>
> Key: HIVE-22369
> URL: https://issues.apache.org/jira/browse/HIVE-22369
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22369.01.patch, HIVE-22369.02.patch
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> The 
> [optimizedOptiqPlan|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L1573]
>  at CalcitePlanner.getOptimizedHiveOPDag is ultimately generated by 
> CalcitePlanner.internalGenSelectLogicalPlan, which may either provide a 
> [HiveProject|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L4831]
>  or a 
> [HiveTableFunctionScan|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L4776].
>  When HiveCalciteUtil.getTopLevelSelect is invoked on this it is looking for 
> a 
> [HiveProject|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveCalciteUtil.java#L633]
>  node in the tree, which if won't find in case of a HiveTableFunctionScan was 
> returned. This is why TestNewGetSplitsFormat is failing with return path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22369) Handle HiveTableFunctionScan at return path

2019-11-21 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22369:
--
Attachment: HIVE-22369.02.patch

> Handle HiveTableFunctionScan at return path
> ---
>
> Key: HIVE-22369
> URL: https://issues.apache.org/jira/browse/HIVE-22369
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22369.01.patch, HIVE-22369.02.patch
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> The 
> [optimizedOptiqPlan|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L1573]
>  at CalcitePlanner.getOptimizedHiveOPDag is ultimately generated by 
> CalcitePlanner.internalGenSelectLogicalPlan, which may either provide a 
> [HiveProject|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L4831]
>  or a 
> [HiveTableFunctionScan|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L4776].
>  When HiveCalciteUtil.getTopLevelSelect is invoked on this it is looking for 
> a 
> [HiveProject|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveCalciteUtil.java#L633]
>  node in the tree, which if won't find in case of a HiveTableFunctionScan was 
> returned. This is why TestNewGetSplitsFormat is failing with return path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22369) Handle HiveTableFunctionScan at return path

2019-11-21 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22369:
--
Attachment: (was: HIVE-22369.02.patch)

> Handle HiveTableFunctionScan at return path
> ---
>
> Key: HIVE-22369
> URL: https://issues.apache.org/jira/browse/HIVE-22369
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22369.01.patch
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> The 
> [optimizedOptiqPlan|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L1573]
>  at CalcitePlanner.getOptimizedHiveOPDag is ultimately generated by 
> CalcitePlanner.internalGenSelectLogicalPlan, which may either provide a 
> [HiveProject|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L4831]
>  or a 
> [HiveTableFunctionScan|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L4776].
>  When HiveCalciteUtil.getTopLevelSelect is invoked on this it is looking for 
> a 
> [HiveProject|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveCalciteUtil.java#L633]
>  node in the tree, which if won't find in case of a HiveTableFunctionScan was 
> returned. This is why TestNewGetSplitsFormat is failing with return path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22369) Handle HiveTableFunctionScan at return path

2019-11-20 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22369:
--
Attachment: HIVE-22369.02.patch

> Handle HiveTableFunctionScan at return path
> ---
>
> Key: HIVE-22369
> URL: https://issues.apache.org/jira/browse/HIVE-22369
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22369.01.patch, HIVE-22369.02.patch
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> The 
> [optimizedOptiqPlan|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L1573]
>  at CalcitePlanner.getOptimizedHiveOPDag is ultimately generated by 
> CalcitePlanner.internalGenSelectLogicalPlan, which may either provide a 
> [HiveProject|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L4831]
>  or a 
> [HiveTableFunctionScan|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L4776].
>  When HiveCalciteUtil.getTopLevelSelect is invoked on this it is looking for 
> a 
> [HiveProject|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveCalciteUtil.java#L633]
>  node in the tree, which if won't find in case of a HiveTableFunctionScan was 
> returned. This is why TestNewGetSplitsFormat is failing with return path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22369) Handle HiveTableFunctionScan at return path

2019-11-20 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22369:
--
Attachment: (was: HIVE-22369.02.patch)

> Handle HiveTableFunctionScan at return path
> ---
>
> Key: HIVE-22369
> URL: https://issues.apache.org/jira/browse/HIVE-22369
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22369.01.patch, HIVE-22369.02.patch
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> The 
> [optimizedOptiqPlan|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L1573]
>  at CalcitePlanner.getOptimizedHiveOPDag is ultimately generated by 
> CalcitePlanner.internalGenSelectLogicalPlan, which may either provide a 
> [HiveProject|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L4831]
>  or a 
> [HiveTableFunctionScan|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L4776].
>  When HiveCalciteUtil.getTopLevelSelect is invoked on this it is looking for 
> a 
> [HiveProject|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveCalciteUtil.java#L633]
>  node in the tree, which if won't find in case of a HiveTableFunctionScan was 
> returned. This is why TestNewGetSplitsFormat is failing with return path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22369) Handle HiveTableFunctionScan at return path

2019-11-20 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22369:
--
Attachment: HIVE-22369.02.patch

> Handle HiveTableFunctionScan at return path
> ---
>
> Key: HIVE-22369
> URL: https://issues.apache.org/jira/browse/HIVE-22369
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22369.01.patch, HIVE-22369.02.patch
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> The 
> [optimizedOptiqPlan|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L1573]
>  at CalcitePlanner.getOptimizedHiveOPDag is ultimately generated by 
> CalcitePlanner.internalGenSelectLogicalPlan, which may either provide a 
> [HiveProject|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L4831]
>  or a 
> [HiveTableFunctionScan|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L4776].
>  When HiveCalciteUtil.getTopLevelSelect is invoked on this it is looking for 
> a 
> [HiveProject|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveCalciteUtil.java#L633]
>  node in the tree, which if won't find in case of a HiveTableFunctionScan was 
> returned. This is why TestNewGetSplitsFormat is failing with return path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22329) Create DriverContext

2019-11-20 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22329:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Create DriverContext
> 
>
> Key: HIVE-22329
> URL: https://issues.apache.org/jira/browse/HIVE-22329
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22329.01.patch, HIVE-22329.02.patch, 
> HIVE-22329.03.patch, HIVE-22329.04.patch
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> The current DriverContext class is not really a context, rather a queue, thus 
> it should be renamed to DriverQueue. At the same time Driver has a a lot of 
> fields, they should be in a real context class called DriverContext, which 
> would also facilitate the extraction of independent parts of the Driver class.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22369) Handle HiveTableFunctionScan at return path

2019-11-18 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22369:
--
Status: Patch Available  (was: Open)

> Handle HiveTableFunctionScan at return path
> ---
>
> Key: HIVE-22369
> URL: https://issues.apache.org/jira/browse/HIVE-22369
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22369.01.patch
>
>
> The 
> [optimizedOptiqPlan|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L1573]
>  at CalcitePlanner.getOptimizedHiveOPDag is ultimately generated by 
> CalcitePlanner.internalGenSelectLogicalPlan, which may either provide a 
> [HiveProject|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L4831]
>  or a 
> [HiveTableFunctionScan|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L4776].
>  When HiveCalciteUtil.getTopLevelSelect is invoked on this it is looking for 
> a 
> [HiveProject|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveCalciteUtil.java#L633]
>  node in the tree, which if won't find in case of a HiveTableFunctionScan was 
> returned. This is why TestNewGetSplitsFormat is failing with return path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22369) Handle HiveTableFunctionScan at return path

2019-11-18 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22369:
--
Attachment: HIVE-22369.01.patch

> Handle HiveTableFunctionScan at return path
> ---
>
> Key: HIVE-22369
> URL: https://issues.apache.org/jira/browse/HIVE-22369
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22369.01.patch
>
>
> The 
> [optimizedOptiqPlan|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L1573]
>  at CalcitePlanner.getOptimizedHiveOPDag is ultimately generated by 
> CalcitePlanner.internalGenSelectLogicalPlan, which may either provide a 
> [HiveProject|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L4831]
>  or a 
> [HiveTableFunctionScan|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L4776].
>  When HiveCalciteUtil.getTopLevelSelect is invoked on this it is looking for 
> a 
> [HiveProject|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveCalciteUtil.java#L633]
>  node in the tree, which if won't find in case of a HiveTableFunctionScan was 
> returned. This is why TestNewGetSplitsFormat is failing with return path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22488) Break up DDLSemanticAnalyzer - extract Table creation analyzers

2019-11-16 Thread Miklos Gergely (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975752#comment-16975752
 ] 

Miklos Gergely commented on HIVE-22488:
---

All the tests have passed, could you please review, [~jcamachorodriguez]?

> Break up DDLSemanticAnalyzer - extract Table creation analyzers
> ---
>
> Key: HIVE-22488
> URL: https://issues.apache.org/jira/browse/HIVE-22488
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22488.01.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #9: extract the table creationanalyzers from DDLSemanticAnalyzer, and 
> move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22488) Break up DDLSemanticAnalyzer - extract Table creation analyzers

2019-11-16 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22488:
--
Attachment: (was: HIVE-22488.01.patch)

> Break up DDLSemanticAnalyzer - extract Table creation analyzers
> ---
>
> Key: HIVE-22488
> URL: https://issues.apache.org/jira/browse/HIVE-22488
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22488.01.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #9: extract the table creationanalyzers from DDLSemanticAnalyzer, and 
> move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22488) Break up DDLSemanticAnalyzer - extract Table creation analyzers

2019-11-16 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22488:
--
Attachment: HIVE-22488.01.patch

> Break up DDLSemanticAnalyzer - extract Table creation analyzers
> ---
>
> Key: HIVE-22488
> URL: https://issues.apache.org/jira/browse/HIVE-22488
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22488.01.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #9: extract the table creationanalyzers from DDLSemanticAnalyzer, and 
> move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22329) Create DriverContext

2019-11-16 Thread Miklos Gergely (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-22329:
--
Attachment: HIVE-22329.04.patch

> Create DriverContext
> 
>
> Key: HIVE-22329
> URL: https://issues.apache.org/jira/browse/HIVE-22329
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22329.01.patch, HIVE-22329.02.patch, 
> HIVE-22329.03.patch, HIVE-22329.04.patch
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> The current DriverContext class is not really a context, rather a queue, thus 
> it should be renamed to DriverQueue. At the same time Driver has a a lot of 
> fields, they should be in a real context class called DriverContext, which 
> would also facilitate the extraction of independent parts of the Driver class.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


<    4   5   6   7   8   9   10   11   12   13   >