[jira] [Work logged] (HIVE-22978) Fix decimal precision and scale inference for aggregate rewriting in Calcite

2020-03-10 Thread ASF GitHub Bot (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-22978?focusedWorklogId=400939=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-400939
 ]

ASF GitHub Bot logged work on HIVE-22978:
-

Author: ASF GitHub Bot
Created on: 10/Mar/20 19:19
Start Date: 10/Mar/20 19:19
Worklog Time Spent: 10m 
  Work Description: asfgit commented on pull request #938: HIVE-22978
URL: https://github.com/apache/hive/pull/938
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 400939)
Time Spent: 50m  (was: 40m)

> Fix decimal precision and scale inference for aggregate rewriting in Calcite
> 
>
> Key: HIVE-22978
> URL: https://issues.apache.org/jira/browse/HIVE-22978
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>  Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22978.01.patch, HIVE-22978.02.patch, 
> HIVE-22978.patch
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Calcite rules can do rewritings of aggregate functions, e.g., {{avg}} into 
> {{sum/count}}. When type of {{avg}} is decimal, inference of intermediate 
> precision and scale for the division is not done correctly. The reason is 
> that we miss support for some types in method {{getDefaultPrecision}} in 
> {{HiveTypeSystemImpl}}. Additionally, {{deriveSumType}} should be overridden 
> in {{HiveTypeSystemImpl}} to abide by the Hive semantics for sum aggregate 
> type inference.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (HIVE-22978) Fix decimal precision and scale inference for aggregate rewriting in Calcite

2020-03-09 Thread ASF GitHub Bot (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-22978?focusedWorklogId=400501=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-400501
 ]

ASF GitHub Bot logged work on HIVE-22978:
-

Author: ASF GitHub Bot
Created on: 10/Mar/20 01:19
Start Date: 10/Mar/20 01:19
Worklog Time Spent: 10m 
  Work Description: jcamachor commented on pull request #938: HIVE-22978
URL: https://github.com/apache/hive/pull/938#discussion_r390045244
 
 

 ##
 File path: 
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveTypeSystemImpl.java
 ##
 @@ -37,6 +39,14 @@
   private static final int MAX_BINARY_PRECISION  = Integer.MAX_VALUE;
   private static final int MAX_TIMESTAMP_PRECISION   = 9;
   private static final int MAX_TIMESTAMP_WITH_LOCAL_TIME_ZONE_PRECISION = 15; 
// Up to nanos
+  private static final int DEFAULT_BOOLEAN_PRECISION  = 1;
 
 Review comment:
   I think you are right. Implementation in Calcite has it defined for Boolean 
since I think they consider precision as length... However, I do not think it 
makes much sense. I have removed it.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 400501)
Time Spent: 40m  (was: 0.5h)

> Fix decimal precision and scale inference for aggregate rewriting in Calcite
> 
>
> Key: HIVE-22978
> URL: https://issues.apache.org/jira/browse/HIVE-22978
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-22978.01.patch, HIVE-22978.patch
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Calcite rules can do rewritings of aggregate functions, e.g., {{avg}} into 
> {{sum/count}}. When type of {{avg}} is decimal, inference of intermediate 
> precision and scale for the division is not done correctly. The reason is 
> that we miss support for some types in method {{getDefaultPrecision}} in 
> {{HiveTypeSystemImpl}}. Additionally, {{deriveSumType}} should be overridden 
> in {{HiveTypeSystemImpl}} to abide by the Hive semantics for sum aggregate 
> type inference.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (HIVE-22978) Fix decimal precision and scale inference for aggregate rewriting in Calcite

2020-03-09 Thread ASF GitHub Bot (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-22978?focusedWorklogId=400471=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-400471
 ]

ASF GitHub Bot logged work on HIVE-22978:
-

Author: ASF GitHub Bot
Created on: 10/Mar/20 00:25
Start Date: 10/Mar/20 00:25
Worklog Time Spent: 10m 
  Work Description: vineetgarg02 commented on pull request #938: HIVE-22978
URL: https://github.com/apache/hive/pull/938#discussion_r390027730
 
 

 ##
 File path: 
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveTypeSystemImpl.java
 ##
 @@ -148,4 +172,17 @@ public boolean isSchemaCaseSensitive() {
 return false;
   }
 
+  @Override
+  public RelDataType deriveSumType(RelDataTypeFactory typeFactory,
 
 Review comment:
   Can you add a comment explaining why is this being done? Adding 10 seems 
arbitrary without the comment. 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 400471)
Time Spent: 0.5h  (was: 20m)

> Fix decimal precision and scale inference for aggregate rewriting in Calcite
> 
>
> Key: HIVE-22978
> URL: https://issues.apache.org/jira/browse/HIVE-22978
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-22978.01.patch, HIVE-22978.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Calcite rules can do rewritings of aggregate functions, e.g., {{avg}} into 
> {{sum/count}}. When type of {{avg}} is decimal, inference of intermediate 
> precision and scale for the division is not done correctly. The reason is 
> that we miss support for some types in method {{getDefaultPrecision}} in 
> {{HiveTypeSystemImpl}}. Additionally, {{deriveSumType}} should be overridden 
> in {{HiveTypeSystemImpl}} to abide by the Hive semantics for sum aggregate 
> type inference.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (HIVE-22978) Fix decimal precision and scale inference for aggregate rewriting in Calcite

2020-03-09 Thread ASF GitHub Bot (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-22978?focusedWorklogId=400470=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-400470
 ]

ASF GitHub Bot logged work on HIVE-22978:
-

Author: ASF GitHub Bot
Created on: 10/Mar/20 00:25
Start Date: 10/Mar/20 00:25
Worklog Time Spent: 10m 
  Work Description: vineetgarg02 commented on pull request #938: HIVE-22978
URL: https://github.com/apache/hive/pull/938#discussion_r389846825
 
 

 ##
 File path: 
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveTypeSystemImpl.java
 ##
 @@ -37,6 +39,14 @@
   private static final int MAX_BINARY_PRECISION  = Integer.MAX_VALUE;
   private static final int MAX_TIMESTAMP_PRECISION   = 9;
   private static final int MAX_TIMESTAMP_WITH_LOCAL_TIME_ZONE_PRECISION = 15; 
// Up to nanos
+  private static final int DEFAULT_BOOLEAN_PRECISION  = 1;
 
 Review comment:
   What does BOOLEAN with precision 1 mean? I thought precision only pertain to 
numeric data type.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 400470)
Time Spent: 20m  (was: 10m)

> Fix decimal precision and scale inference for aggregate rewriting in Calcite
> 
>
> Key: HIVE-22978
> URL: https://issues.apache.org/jira/browse/HIVE-22978
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-22978.01.patch, HIVE-22978.patch
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Calcite rules can do rewritings of aggregate functions, e.g., {{avg}} into 
> {{sum/count}}. When type of {{avg}} is decimal, inference of intermediate 
> precision and scale for the division is not done correctly. The reason is 
> that we miss support for some types in method {{getDefaultPrecision}} in 
> {{HiveTypeSystemImpl}}. Additionally, {{deriveSumType}} should be overridden 
> in {{HiveTypeSystemImpl}} to abide by the Hive semantics for sum aggregate 
> type inference.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (HIVE-22978) Fix decimal precision and scale inference for aggregate rewriting in Calcite

2020-03-05 Thread ASF GitHub Bot (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-22978?focusedWorklogId=398827=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-398827
 ]

ASF GitHub Bot logged work on HIVE-22978:
-

Author: ASF GitHub Bot
Created on: 06/Mar/20 01:06
Start Date: 06/Mar/20 01:06
Worklog Time Spent: 10m 
  Work Description: jcamachor commented on pull request #938: HIVE-22978
URL: https://github.com/apache/hive/pull/938
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 398827)
Remaining Estimate: 0h
Time Spent: 10m

> Fix decimal precision and scale inference for aggregate rewriting in Calcite
> 
>
> Key: HIVE-22978
> URL: https://issues.apache.org/jira/browse/HIVE-22978
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-22978.01.patch, HIVE-22978.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Calcite rules can do rewritings of aggregate functions, e.g., {{avg}} into 
> {{sum/count}}. When type of {{avg}} is decimal, inference of intermediate 
> precision and scale for the division is not done correctly. The reason is 
> that we miss support for some types in method {{getDefaultPrecision}} in 
> {{HiveTypeSystemImpl}}. Additionally, {{deriveSumType}} should be overridden 
> in {{HiveTypeSystemImpl}} to abide by the Hive semantics for sum aggregate 
> type inference.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)