[
https://issues.apache.org/jira/browse/IMPALA-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17454310#comment-17454310
]
Wenzhe Zhou commented on IMPALA-8041:
-------------------------------------
Comments from [Kurt Deschler|https://app.slack.com/team/UL853Q97Y]: it may be
(more) practical to implement hashing and partition the expressions lists which
can reduce the effective number of columns by orders of magnitude and mitigate
algorithm complexity issues. For the JNI function, we should look into bulk
interfaces, either materializing as a row or adding a new array interface.
> Migrate rewrite rules to occur during expression analysis
> ---------------------------------------------------------
>
> Key: IMPALA-8041
> URL: https://issues.apache.org/jira/browse/IMPALA-8041
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Affects Versions: Impala 3.1.0
> Reporter: Paul Rogers
> Priority: Minor
>
> A number of JIRAs have been filed about issues with, or improvements to, the
> expression rewrite rules implemented in the analyzer. Work in this area
> revealed that changes in this area are quite difficult because of the way the
> code is currently structured.
> * The analyzer makes one pass over the AST to resolve references, compute
> types and so on.
> * The rewrite rules make a pass over the tree to rewrite expressions.
> * The prior analysis is discarded, and a second analysis pass is done over
> the rewritten expressions.
> Work was done to handle this two-step process. Special code exists in some
> clauses to tuck away the underwritten SQL for use in error messages after
> rewrite, though the implementation is inconsistent across clauses.
> The analyzer often makes copies of expressions for various purposes. It is
> quite hard to keep things in sync when doing rewrites as new copies must be
> made (as part of the second analysis pass.)
> The goal of this ticket is to migrate rewrites into the expression analysis
> step. For each node:
> * Analyze and rewrite children.
> * Resolve references.
> * Rewrite the node itself.
> * Compute costs and selectivity.
> Once completed, rewrites will be just another step in expression analysis.
> The analyzer will make just one analysis pass. Copies of expressions will be
> made after analysis/rewrite to that they stay in sync.
> The work will be done as a series of small patches. Those that are just
> refactoring will use this this JIRA ticket with a (Part i) designation. Those
> that make functional changes have their own JIRA tickets.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
