Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/17387 )
Change subject: IMPALA-10681: Improve inner join cardinality estimates ...................................................................... Patch Set 6: (3 comments) Thanks Aman for the comments on formula and background info. http://gerrit.cloudera.org:8080/#/c/17387/3/fe/src/main/java/org/apache/impala/planner/JoinNode.java File fe/src/main/java/org/apache/impala/planner/JoinNode.java: http://gerrit.cloudera.org:8080/#/c/17387/3/fe/src/main/java/org/apache/impala/planner/JoinNode.java@430 PS3, Line 430: lt; > Actually, all the join cardinality methods in this file are using NDV so I Okay, Thanks for the background info. Really helpful. In this case, maybe we should still use getGenericJoinCardinality() since the core computation formula is the same. http://gerrit.cloudera.org:8080/#/c/17387/3/fe/src/main/java/org/apache/impala/planner/JoinNode.java@441 PS3, Line 441: long lhsCard, long rhsCard) { > Note that I kept the calculation the same as getGenericJoinCardinality() . Okay. http://gerrit.cloudera.org:8080/#/c/17387/3/fe/src/main/java/org/apache/impala/planner/JoinNode.java@448 PS3, Line 448: // Adjust the NDVs on both sides to account for predicates. Intuitively, the NDVs : // should onl > The formula on line 448 is the same as the one in getGenericJoinCardinality Okay. Looks like my formula missed one term min(NdvL, NdvR) and should be (CardL / NdvL) * (CardR / NdvR) * min(NdvL, NdvR), which is exactly your formula. -- To view, visit http://gerrit.cloudera.org:8080/17387 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8aa9d3b8f3c4848b3e9414fe19ad7ad348d12ecc Gerrit-Change-Number: 17387 Gerrit-PatchSet: 6 Gerrit-Owner: Aman Sinha <amsi...@cloudera.com> Gerrit-Reviewer: Aman Sinha <amsi...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Comment-Date: Thu, 20 May 2021 13:43:48 +0000 Gerrit-HasComments: Yes
