[jira] [Commented] (PIG-4128) New logical optimizer rule: ConstantCalculator

2014-09-17 Thread Daniel Dai (JIRA)

[ 
https://issues.apache.org/jira/browse/PIG-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14138394#comment-14138394
 ] 

Daniel Dai commented on PIG-4128:
-

Thanks Cheolsoo, really glad to know this!

> New logical optimizer rule: ConstantCalculator
> --
>
> Key: PIG-4128
> URL: https://issues.apache.org/jira/browse/PIG-4128
> Project: Pig
>  Issue Type: New Feature
>  Components: impl
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: 0.14.0
>
> Attachments: PIG-4128-1.patch, PIG-4128-2.patch, PIG-4128-3.patch
>
>
> Pig used to have a LogicExpressionSimplifier to simplify expression which 
> also calculates constant expression. The optimizer rule is buggy and we 
> disable it by default in PIG-2316.
> However, we do need this feature especially in partition/predicate push down, 
> since both does not deal with complex constant expression, we'd like to 
> replace the expression with constant before the actual push down. Yes, user 
> may manually do the calculation and rewrite the query, but even rewrite is 
> sometimes not possible. Consider the case user want to push a datetime 
> predicate, user have to write a ToDate udf since Pig does not have datetime 
> constant.
> In this Jira, I provide a new rule: ConstantCalculator, which is much simpler 
> and much less error prone, to replace LogicExpressionSimplifier.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (PIG-4128) New logical optimizer rule: ConstantCalculator

2014-09-16 Thread Cheolsoo Park (JIRA)

[ 
https://issues.apache.org/jira/browse/PIG-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14136539#comment-14136539
 ] 

Cheolsoo Park commented on PIG-4128:


[~daijy], I want to say thank you for this patch. One of common mistakes that I 
see is that partition column is long type, and user specifies a filter 
expression like {{ == }}. The partition used to 
be not pushed down in this case because Pig inserts a cast expression into the 
filer expression, and this was always confusing to users. But with your patch, 
this filter expression just works.

> New logical optimizer rule: ConstantCalculator
> --
>
> Key: PIG-4128
> URL: https://issues.apache.org/jira/browse/PIG-4128
> Project: Pig
>  Issue Type: New Feature
>  Components: impl
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: 0.14.0
>
> Attachments: PIG-4128-1.patch, PIG-4128-2.patch, PIG-4128-3.patch
>
>
> Pig used to have a LogicExpressionSimplifier to simplify expression which 
> also calculates constant expression. The optimizer rule is buggy and we 
> disable it by default in PIG-2316.
> However, we do need this feature especially in partition/predicate push down, 
> since both does not deal with complex constant expression, we'd like to 
> replace the expression with constant before the actual push down. Yes, user 
> may manually do the calculation and rewrite the query, but even rewrite is 
> sometimes not possible. Consider the case user want to push a datetime 
> predicate, user have to write a ToDate udf since Pig does not have datetime 
> constant.
> In this Jira, I provide a new rule: ConstantCalculator, which is much simpler 
> and much less error prone, to replace LogicExpressionSimplifier.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (PIG-4128) New logical optimizer rule: ConstantCalculator

2014-08-27 Thread Daniel Dai (JIRA)

[ 
https://issues.apache.org/jira/browse/PIG-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14112498#comment-14112498
 ] 

Daniel Dai commented on PIG-4128:
-

Oh, yes, I delete it because it is empty. I bring it back.

> New logical optimizer rule: ConstantCalculator
> --
>
> Key: PIG-4128
> URL: https://issues.apache.org/jira/browse/PIG-4128
> Project: Pig
>  Issue Type: New Feature
>  Components: impl
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: 0.14.0
>
> Attachments: PIG-4128-1.patch, PIG-4128-2.patch, PIG-4128-3.patch
>
>
> Pig used to have a LogicExpressionSimplifier to simplify expression which 
> also calculates constant expression. The optimizer rule is buggy and we 
> disable it by default in PIG-2316.
> However, we do need this feature especially in partition/predicate push down, 
> since both does not deal with complex constant expression, we'd like to 
> replace the expression with constant before the actual push down. Yes, user 
> may manually do the calculation and rewrite the query, but even rewrite is 
> sometimes not possible. Consider the case user want to push a datetime 
> predicate, user have to write a ToDate udf since Pig does not have datetime 
> constant.
> In this Jira, I provide a new rule: ConstantCalculator, which is much simpler 
> and much less error prone, to replace LogicExpressionSimplifier.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (PIG-4128) New logical optimizer rule: ConstantCalculator

2014-08-27 Thread Cheolsoo Park (JIRA)

[ 
https://issues.apache.org/jira/browse/PIG-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14112366#comment-14112366
 ] 

Cheolsoo Park commented on PIG-4128:


[~daijy], did you delete {{excluded-tests-23}} intentionally? It broke the 
build-
{code}
/home/jenkins/jenkins-slave/workspace/Pig-trunk/build.xml:905: Excludesfile 
/home/jenkins/jenkins-slave/workspace/Pig-trunk/test/excluded-tests-23 not 
found.
{code}
https://builds.apache.org/job/Pig-trunk/1639/console

> New logical optimizer rule: ConstantCalculator
> --
>
> Key: PIG-4128
> URL: https://issues.apache.org/jira/browse/PIG-4128
> Project: Pig
>  Issue Type: New Feature
>  Components: impl
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: 0.14.0
>
> Attachments: PIG-4128-1.patch, PIG-4128-2.patch, PIG-4128-3.patch
>
>
> Pig used to have a LogicExpressionSimplifier to simplify expression which 
> also calculates constant expression. The optimizer rule is buggy and we 
> disable it by default in PIG-2316.
> However, we do need this feature especially in partition/predicate push down, 
> since both does not deal with complex constant expression, we'd like to 
> replace the expression with constant before the actual push down. Yes, user 
> may manually do the calculation and rewrite the query, but even rewrite is 
> sometimes not possible. Consider the case user want to push a datetime 
> predicate, user have to write a ToDate udf since Pig does not have datetime 
> constant.
> In this Jira, I provide a new rule: ConstantCalculator, which is much simpler 
> and much less error prone, to replace LogicExpressionSimplifier.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (PIG-4128) New logical optimizer rule: ConstantCalculator

2014-08-26 Thread Thejas M Nair (JIRA)

[ 
https://issues.apache.org/jira/browse/PIG-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14111505#comment-14111505
 ] 

Thejas M Nair commented on PIG-4128:


+1

> New logical optimizer rule: ConstantCalculator
> --
>
> Key: PIG-4128
> URL: https://issues.apache.org/jira/browse/PIG-4128
> Project: Pig
>  Issue Type: New Feature
>  Components: impl
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: 0.14.0
>
> Attachments: PIG-4128-1.patch, PIG-4128-2.patch, PIG-4128-3.patch
>
>
> Pig used to have a LogicExpressionSimplifier to simplify expression which 
> also calculates constant expression. The optimizer rule is buggy and we 
> disable it by default in PIG-2316.
> However, we do need this feature especially in partition/predicate push down, 
> since both does not deal with complex constant expression, we'd like to 
> replace the expression with constant before the actual push down. Yes, user 
> may manually do the calculation and rewrite the query, but even rewrite is 
> sometimes not possible. Consider the case user want to push a datetime 
> predicate, user have to write a ToDate udf since Pig does not have datetime 
> constant.
> In this Jira, I provide a new rule: ConstantCalculator, which is much simpler 
> and much less error prone, to replace LogicExpressionSimplifier.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (PIG-4128) New logical optimizer rule: ConstantCalculator

2014-08-17 Thread Daniel Dai (JIRA)

[ 
https://issues.apache.org/jira/browse/PIG-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14100183#comment-14100183
 ] 

Daniel Dai commented on PIG-4128:
-

RB link: https://reviews.apache.org/r/24789/

> New logical optimizer rule: ConstantCalculator
> --
>
> Key: PIG-4128
> URL: https://issues.apache.org/jira/browse/PIG-4128
> Project: Pig
>  Issue Type: New Feature
>  Components: impl
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: 0.14.0
>
> Attachments: PIG-4128-1.patch
>
>
> Pig used to have a LogicExpressionSimplifier to simplify expression which 
> also calculates constant expression. The optimizer rule is buggy and we 
> disable it by default in PIG-2316.
> However, we do need this feature especially in partition/predicate push down, 
> since both does not deal with complex constant expression, we'd like to 
> replace the expression with constant before the actual push down. Yes, user 
> may manually do the calculation and rewrite the query, but even rewrite is 
> sometimes not possible. Consider the case user want to push a datetime 
> predicate, user have to write a ToDate udf since Pig does not have datetime 
> constant.
> In this Jira, I provide a new rule: ConstantCalculator, which is much simpler 
> and much less error prone, to replace LogicExpressionSimplifier.



--
This message was sent by Atlassian JIRA
(v6.2#6252)