wangning has posted comments on this change. ( http://gerrit.cloudera.org:8080/16674 )
Change subject: KUDU-1644 hash-partition based in-list predicate optimization ...................................................................... Patch Set 18: (1 comment) http://gerrit.cloudera.org:8080/#/c/16674/18//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16674/18//COMMIT_MSG@16 PS18, Line 16: Before: : To each tablet, time complexity to complete hash-key based in-list query is: : V * LOG(N/R) * N : : After: : Complexity becomes: : V/P * LOG(N/R) * N > I'm curious, do you also have numbers, in the same way that you did in the https://docs.google.com/document/d/1WO4TT2ZqGsvlgogyKOsChpinEeupZCkxn9OI5xulryM/edit?usp=sharing We patched this idea in our company's fork in other form. But it's hard to paste our log here or it is really hard to explain what does the log do. So today, I did a little scale experiment to explain this idea with benchmark. The number, it's depend on how many partitions the table have as I think. E.g. 9 partitions may have 3 times speed up roughly. And I also explained why it can not reach this number in my experiment, actually it managed to made the number in our product. BTW, In commit msg, it's was my mistake to made it wrong. I removed rowset concept in order to have a better explain. It should be Before: To each tablet, time complexity to complete hash-key based in-list query is: LOG(V) * N After: Complexity becomes: LOG(V/P) * N -- To view, visit http://gerrit.cloudera.org:8080/16674 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I202001535669a72de7fbb9e766dbc27db48e0aa2 Gerrit-Change-Number: 16674 Gerrit-PatchSet: 18 Gerrit-Owner: wangning <1994wangn...@gmail.com> Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com> Gerrit-Reviewer: Bankim Bhavsar <ban...@cloudera.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy <mre...@cloudera.com> Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: wangning <1994wangn...@gmail.com> Gerrit-Comment-Date: Wed, 11 Nov 2020 13:58:40 +0000 Gerrit-HasComments: Yes
