Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15157 )
Change subject: IMPALA-9358: Query slowdown with inline views and hundreds of columns ...................................................................... IMPALA-9358: Query slowdown with inline views and hundreds of columns IMPALA-8386 introduced an expensive precondition check using the function ExprSubstitutionMap.checkComposedFrom(). This check has significant performance impact on statements that contain inline views with hundreds of columns. Most of the cost is in the get() calls used to find expressions in the local substitution map. The fix is to add a getWithHint() call that uses the current loop index as a starting point to search for expressions. This leverages the fact that expressions have identical positions in both substitution maps in most common cases. A more generic approach would be to accelerate expression equality search using hash functions but that would be a much riskier fix and Impala currently lacks the infrasturucture to so. Testing: Performance testing with a query with 1000 expressions of the following form: with a as (select c1 c1, c1 c2, c1 c3, ... from t) select c1, c2, c3, ... from a; repro query went from 12 sec to 1 sec. There was no noticeable time spent in the precondition now. Change-Id: I77423d9c10e1edbb505cb210b5c072281b5d7cfc Reviewed-on: http://gerrit.cloudera.org:8080/15157 Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- M fe/src/main/java/org/apache/impala/analysis/ExprSubstitutionMap.java 1 file changed, 16 insertions(+), 2 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/15157 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I77423d9c10e1edbb505cb210b5c072281b5d7cfc Gerrit-Change-Number: 15157 Gerrit-PatchSet: 5 Gerrit-Owner: Kurt Deschler <kdesc...@cloudera.com> Gerrit-Reviewer: Anurag Mantripragada <anu...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Kurt Deschler <kdesc...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Vihang Karajgaonkar <vih...@cloudera.com>