Hello Kudu Jenkins, Andrew Wong, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/14706
to look at the new patch set (#9).
Change subject: [java] KUDU-1644 Simplify IN-list predicate values during token
building
......................................................................
[java] KUDU-1644 Simplify IN-list predicate values during token building
This commit intends to improve the performance of token-based in-list
scan in case of only one hash partition.
By reducing the number of values pushed to server, it can reduce
the complexity of token-based in-list logarithmically.
The commit implemented the idea by filtering the unnesscessary values
when building the scanner.
Here I attach an simple benchmark which implemented by java client and
calculate qps by a 10s-scheduled timer.
Some settings,
client 4core, 8g memory
server 4core, 8g memory, HDD
in-list size, 100000
table settings, 10M rows, 24 partitions
Before tuning:
Mon Nov 11 19:09:53 CST 2019, qps :2528
Mon Nov 11 19:10:03 CST 2019, qps :3715
Mon Nov 11 19:10:13 CST 2019, qps :2509
Mon Nov 11 19:10:23 CST 2019, qps :2900
Mon Nov 11 19:10:33 CST 2019, qps :2081
Mon Nov 11 19:10:43 CST 2019, qps :3635
After tuning
Mon Nov 11 19:15:01 CST 2019, qps :13735
Mon Nov 11 19:15:11 CST 2019, qps :16264
Mon Nov 11 19:15:21 CST 2019, qps :13702
Mon Nov 11 19:15:31 CST 2019, qps :15469
Mon Nov 11 19:15:41 CST 2019, qps :16269
Change-Id: I05affbd10804b4d07478f7abdde5380dd70ec116
---
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduScanner.java
M java/kudu-client/src/main/java/org/apache/kudu/client/KuduPredicate.java
M java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanToken.java
M java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanner.java
M java/kudu-client/src/main/java/org/apache/kudu/client/PartitionPruner.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestPartitionPruner.java
6 files changed, 190 insertions(+), 16 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/06/14706/9
--
To view, visit http://gerrit.cloudera.org:8080/14706
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I05affbd10804b4d07478f7abdde5380dd70ec116
Gerrit-Change-Number: 14706
Gerrit-PatchSet: 9
Gerrit-Owner: wangning <[email protected]>
Gerrit-Reviewer: Andrew Wong <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <[email protected]>
Gerrit-Reviewer: wangning <[email protected]>
