Hello Tidy Bot, Yingchun Lai, Yifan Zhang, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19327
to look at the new patch set (#9).
Change subject: [Tool] KUDU-3318 Compact container metadata file manually
......................................................................
[Tool] KUDU-3318 Compact container metadata file manually
Some old versions of Kudu do not support compacting log container
metadata file at runtime. A long time later, metadata file
becomes very large, for example, more than 100G. When restarting
Kudu tablet server, it will read all metadata files to check which
blocks are still alive and compact metadata files into new small
metadata files which only record alive blocks. As the metadata
files are very big, compacting metadata files will cost a lot of
time, maybe 2-3 hours. That will cause the service unavailabe
for a long time, which is unacceptable.
Runtime metadata file compacting has already been supported
above version of 1.16.0, see jira: KUDU-3318. Upgrading old
version to above Kudu 1.16.0 still needs to restart Kudu.
This tool helps to compact metdata file for old versions of
Kudu. It supports 4 operations:
--print: Statistic the total blocks and alive blocks for
every container.
--compact: Compact metadata files when Kudu is still running
--merge: Stop Kudu and write incremental records into compacted
metadata files. Then backup old metadata files.
--clear: Delete metadata backup files.
Restarting old version of Kudu maybe cost 2-3 hours, but using
compact tool, service stopped only lasts 3 minutes.
Change-Id: Id61cb6c88d7a093a8448e2497639f5c5c094efa4
---
M src/kudu/fs/log_block_manager-test.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/fs/log_block_manager.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_fs.cc
M src/kudu/util/pb_util.cc
M src/kudu/util/pb_util.h
7 files changed, 677 insertions(+), 11 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/27/19327/9
--
To view, visit http://gerrit.cloudera.org:8080/19327
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id61cb6c88d7a093a8448e2497639f5c5c094efa4
Gerrit-Change-Number: 19327
Gerrit-PatchSet: 9
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Attila Bukor <abu...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang <chinazhangyi...@163.com>
Gerrit-Reviewer: Yingchun Lai <laiyingc...@apache.org>
