Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4308
to look at the new patch set (#2).
Change subject: [c++ client] AUTO_FLUSH_BACKGROUND optimizations
......................................................................
[c++ client] AUTO_FLUSH_BACKGROUND optimizations
Some optimizations after initial performance testing of the
Kudu C++ client library with AUTO_FLUSH_BACKGROUND flush mode support.
The most important tuning is the default flush watermark
for the mutation buffer. Changing it from 80% to 50% gave good
performance boost.
The data below is for workloads of 8M rows like
(int64, int32, string, string, int) where strings are about 32 bytes
long in average. Each thread ran its own single-session KuduClient,
where each session was running in AUTO_FLUSH_BACKGROUND flush mode.
1-thread insertion (8M rows per thread)
80% watermark:
total : 35229.7 ms
per row: 0.00440372 ms
50% watermark:
total : 22562.8 ms
per row: 0.00282035 ms
2-thread insertion (4M rows per thread)
80% watermark:
total : 19683.6 ms
per row: 0.00246046 ms
50% watermark:
total : 12931.8 ms
per row: 0.00161647 ms
4-thread insertion (2M rows per thread)
80% watermark:
total : 11941.9 ms
per row: 0.00149274 ms
50% watermark:
total : 7724.68 ms
per row: 0.000965585 ms
Other related session parameters:
mutation buffer size: 7M (default)
maximum number of batchers: 2 (default)
time-based flush interval: 1 second (default)
The tests were run at ve0518.halxg.cloudera.com against binaries
built in release configuration at the same machine.
Change-Id: I1f0aa6d02c51bb063498709e8570e8c7214a31a0
---
M src/kudu/client/batcher.cc
M src/kudu/client/batcher.h
M src/kudu/client/client.h
M src/kudu/client/session-internal.cc
M src/kudu/client/write_op.cc
5 files changed, 10 insertions(+), 13 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/08/4308/2
--
To view, visit http://gerrit.cloudera.org:8080/4308
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f0aa6d02c51bb063498709e8570e8c7214a31a0
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <aser...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <a...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <aser...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
