Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/5736
to look at the new patch set (#6).
Change subject: WIP: KUDU-1835 (part 2): enable WAL compression
......................................................................
WIP: KUDU-1835 (part 2): enable WAL compression
To test the efficiency, I loaded some data into a local tablet server
using kudu-ts and the influxdb benchmark as a data generator. I think
restarted the server with different configurations, and disabled log
segment prealloocation in order to see the _actual_ length of the WALs
instead of the preallocated length.
Compression Size
NONE 311M
LZ4 98M - 3.17x
SNAPPY 91M - 3.41x
ZLIB 59M - 5.27x
Based on some results in the JIRA I expect more significant improvements
in some other workloads, since kudu-ts already does its own sort of
"dictionary encoding" by normalizing out the tagset IDs.
WIP items:
- need to make sure that this is compatible when set to NONE and
incompatible when enabled
- may want to use a common dictionary at the top of the file in order to
improve performance for small log appends
- need to update log-test and add other coverage
Change-Id: I0fc790ddc72a519a82bbb6b71a902418051b35a5
---
M src/kudu/consensus/CMakeLists.txt
M src/kudu/consensus/log-test.cc
M src/kudu/consensus/log.cc
M src/kudu/consensus/log.h
M src/kudu/consensus/log.proto
M src/kudu/consensus/log_anchor_registry.h
M src/kudu/consensus/log_reader.h
M src/kudu/consensus/log_util.cc
M src/kudu/consensus/log_util.h
M src/kudu/integration-tests/disk_reservation-itest.cc
M src/kudu/integration-tests/raft_consensus-itest.cc
M src/kudu/util/compression/compression_codec.cc
M src/kudu/util/compression/compression_codec.h
13 files changed, 248 insertions(+), 59 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/36/5736/6
--
To view, visit http://gerrit.cloudera.org:8080/5736
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0fc790ddc72a519a82bbb6b71a902418051b35a5
Gerrit-PatchSet: 6
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <t...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
