Steve Carlin has posted comments on this change. ( http://gerrit.cloudera.org:8080/21109 )
Change subject: IMPALA-12872: Use Calcite for optimization - part 1: simple queries ...................................................................... Patch Set 15: (2 comments) http://gerrit.cloudera.org:8080/#/c/21109/15/java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteJniFrontend.java File java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteJniFrontend.java: http://gerrit.cloudera.org:8080/#/c/21109/15/java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteJniFrontend.java@56 PS15, Line 56: public class CalciteJniFrontend extends JniFrontend { > Future: One path forward is that as the Calcite planner gains functionality This totally makes sense. The nice thing (as you probably are aware) about keeping it in its own JniFrontend right now is to remove any kind of dependency needed while this is not near production ready. But yeah, once we are ready to move this over, then there should only be one frontend. It also does make sense to have a query option to run on the current planner in production even before the planners get combined. I've filed a subtask for this: IMPALA-12946 http://gerrit.cloudera.org:8080/#/c/21109/15/java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteMetadataHandler.java File java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteMetadataHandler.java: http://gerrit.cloudera.org:8080/#/c/21109/15/java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteMetadataHandler.java@93 PS15, Line 93: // load the relevant tables in the query from catalogd : this.stmtTableCache_ = stmtMetadataLoader.loadTables(tableVisitor.tableNames_); > Future: When this supports views, are we thinking that StmtMetadataLoader w Admittedly, I haven't looked very closely at how views are going to be handled yet. And I'm not sure I understand what you mean here and where StatementBase is being used. Perhaps you are suggesting a refactor somewhere? Yeah, currently, the purpose behind the code here is to make sure impalad has retrieved all the metadata information from catalogd. The stmtTableCache_ object contains this metadata and will be used by the Calcite Schema to get various information (e.g. statistics for the table). I'm presuming views fall somewhere under this umbrella though in terms of metadata from catalogd. -- To view, visit http://gerrit.cloudera.org:8080/21109 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I453fd75b7b705f4d7de1ed73c3e24cafad0b8c98 Gerrit-Change-Number: 21109 Gerrit-PatchSet: 15 Gerrit-Owner: Steve Carlin <scar...@cloudera.com> Gerrit-Reviewer: Aman Sinha <amsi...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Michael Smith <michael.sm...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Steve Carlin <scar...@cloudera.com> Gerrit-Comment-Date: Wed, 27 Mar 2024 15:31:36 +0000 Gerrit-HasComments: Yes
