Hello Tidy Bot, Alexey Serbin, Yingchun Lai, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19670
to look at the new patch set (#11).
Change subject: [tablet] GC ancient, fully deleted rowsets witout live row
count stats
......................................................................
[tablet] GC ancient, fully deleted rowsets witout live row count stats
We added a background op to GC ancient, fully deleted rowsets for
KUDU-1625 base on live row count. That patch is very useful, but
does not work for older versions that do not support live row count
stats. And during the upgrade process from a lower version to a
higher version, live row count feature cannot be enabled for already
existing data.
To resolve this issue on a lower version of the kudu cluster, I submitted
this patch. The main reason is to replace the use of live row count.
However, due to the lack of a more accurate counting method, this patch
may only release part of the storage space for ancient, fully deleted rows.
Therefore, this feature can alleviate the storage space tension of older
versions to a certain extent.
If you need to enable this feature, enable the flag
--enable_gc_deleted_rowsets_without_live_row_count and restart tservers.
There's still room for improvement in this implementation in that, currently,
we ignored the delete operation in DMS. I will resolve this issue in a follow-up
patch.
I ran this on a real cluster, the storage space of deleted rowsets that was not
previously freed can be GCed as expected. And I also add unit test case to
ensure
it make sense.
Change-Id: Iacdff107b8b07cbd56f47f296a93f4bcfbf56b41
---
M src/kudu/tablet/delta_tracker.cc
M src/kudu/tablet/delta_tracker.h
M src/kudu/tablet/diskrowset-test-base.h
M src/kudu/tablet/diskrowset-test.cc
M src/kudu/tablet/diskrowset.cc
M src/kudu/tablet/diskrowset.h
M src/kudu/tablet/tablet.cc
M src/kudu/tablet/tablet_history_gc-test.cc
8 files changed, 167 insertions(+), 12 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/70/19670/11
--
To view, visit http://gerrit.cloudera.org:8080/19670
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iacdff107b8b07cbd56f47f296a93f4bcfbf56b41
Gerrit-Change-Number: 19670
Gerrit-PatchSet: 11
Gerrit-Owner: KeDeng <kdeng...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <ale...@apache.org>
Gerrit-Reviewer: KeDeng <kdeng...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <laiyingc...@apache.org>
