[
https://issues.apache.org/jira/browse/PIG-3461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13774108#comment-13774108
]
Cheolsoo Park commented on PIG-3461:
------------------------------------
(Apache RB is not responsive now, so I am leaving comments here.)
Excellent work, [~aniket486]! Your code is a lot more readable and
maintainable! I just have few minor comments as follows:
# Can you make the old visitor default and the new one optional in 0.12? I
think that's safer given we haven't tested the new code enough. Do you agree?
# Can you annotate the old visitor class (and its unit test) as deprecated so
that we can delete them perhaps in 0.13?
# Can you make FilterExtractor extend PlanVisitor? It doesn't seem necessary,
but I find it helpful to view it in the type hierarchy of PlanVisitor in IDE.
# Please remove trailing whites paces.
# Lastly, I have a question about backward incompatibility.
{code}
* Flag to use old PartitionFilterOptimizer in case NewPartitionFilterOptimizer
is not backwards compatible
* (A known case is "filter a by 1 == 0").
{code}
As far as I understand, your visitor is simply superior to the old one. (i.e.
more filter expressions can be pushed down.) Is there any filter expression
that used to be pushed down is no longer pushed down? The "1 == 0" case in your
comment is pushed down now while it wasn't before, so I see it more as an
improvement than a backward incompatibility. I am trying to understand whether
there are real backward incompatibilities before I deploy your code in
production.
Thank you very much!
> Rewrite PartitionFilterOptimizer to make it work for all the cases
> ------------------------------------------------------------------
>
> Key: PIG-3461
> URL: https://issues.apache.org/jira/browse/PIG-3461
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.11.1
> Reporter: Aniket Mokashi
> Assignee: Aniket Mokashi
> Fix For: 0.12
>
> Attachments: PIG-3461-2.patch, PIG-3461-4.patch
>
>
> Current algorithm for Partition Filter pushdown identification fails in
> several corner cases. We need to rewrite its logic so that it works in all
> cases and does the maximum possible filter pushdown.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
