Hello Todd Lipcon,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/5852
to review the following change.
Change subject: KUDU-1856: always truncate containers when they get full
......................................................................
KUDU-1856: always truncate containers when they get full
On a production cluster, I found that XFS's speculative preallocation
feature often winds up permanently growing LBM containers without increasing
their file sizes. It's not clear why this happens; I don't think we're
running afoul of any of the conditions spelled out in the XFS FAQ [1].
Nevertheless, on this cluster the preallocated space accounted for almost a
third of the total data consumption, so we need to address it.
This patch changes the existing LBM container truncation behavior such that
at startup, the container's file size no longer determines whether to
truncate. Instead, all full containers are truncated unconditionally. The
assumption here is that a no-op truncate should be free.
We don't always run on XFS, and triggering its speculative preallocation is
hard. But, we can approximate it by passing FALLOC_FL_KEEP_SIZE to
fallocate(), which grows the file without changing its size.
1. http://xfs.org/index.php/XFS_FAQ#Q:_Is_speculative_preallocation_permanent.3F
Change-Id: Ic959c59489a08f92efa2df5c85b22e56740f1346
---
M src/kudu/fs/block_manager-test.cc
M src/kudu/fs/data_dirs.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/util/env.h
M src/kudu/util/env_posix.cc
M src/kudu/util/file_cache.cc
6 files changed, 59 insertions(+), 29 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/52/5852/1
--
To view, visit http://gerrit.cloudera.org:8080/5852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic959c59489a08f92efa2df5c85b22e56740f1346
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo <a...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <t...@apache.org>
