Adar Dembo has posted comments on this change. Change subject: log block manager: use extent maps to decide whether to truncate containers ......................................................................
Patch Set 6: > Have you done any tests of how this affects startup time, > particularly after a full drop_caches on a server with a large > number of containers/extents? Am worried that we might regress > startup time significantly I've been working on an itest that, amongst other things, produces an enormous number of blocks and LBM containers, many of them full. The itest can also benchmark LBM startup time after first calling sync() and writing '3' to /proc/sys/vm/drop_caches. I tested it on my Ubuntu 16.04 laptop (ssd) as well as an el6.6 physical machine (hdd), with this patch series and without. Here are the full results from my first attempt: https://gist.github.com/adembo/0caccb85541789572e02e9e683341359. I was feeling OKish about the (not too significant) change, but then I noticed that I wasn't running sync() before dropping the caches. Once I added that, startup time became dramatically worse (50% slowdown). Full results here: https://gist.github.com/adembo/d4159f9610fc6c7b950bfff9272c0683. I know you weren't a fan of using extent maps to fix inconsistencies from the get go, (and I tip my proverbial cap to your intuition) but I do think it's worth preserving this in some form. Without the extent maps, we can't provide as good information during a repair as to how much excess space was preallocated at the end of a file, or how much space was "unpunched" (when that's implemented). So, what do you think of gating the (expensive) repair behind a gflag? It'd be off during regular startup (so only "cheap" repairs will be done), and on during "kudu fs check". -- To view, visit http://gerrit.cloudera.org:8080/6585 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I3541f70721328d5af911f4889f8b4bcf0de3ebf8 Gerrit-PatchSet: 6 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Adar Dembo <a...@cloudera.com> Gerrit-Reviewer: Adar Dembo <a...@cloudera.com> Gerrit-Reviewer: David Ribeiro Alves <dral...@apache.org> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon <t...@apache.org> Gerrit-HasComments: No
