Hello Dan Burkert, Mike Percy, Todd Lipcon, I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/8090 to review the following change. Change subject: KUDU-1125: issue one catalog write per tablet report ...................................................................... KUDU-1125: issue one catalog write per tablet report This commit addresses a long-standing issue in the catalog manager where an independent catalog write is performed for each reported tablet. When the master is configured to fsync WAL writes, this can add a lot of load during election storms and when the master is restarted. To tackle this, I fully rewrote ProcessTabletReport and friends. I had higher hopes for the final product, but all of the dependent control flow complicates decomposition. Still, I managed to make some improvements. For example, all RPCs are now sent at the end in one go rather than piecemeal. I also rewrote all of the comments so that they can serve as a map to the function and to emphasize the actions performed by the corresponding code. Here are the actual semantic changes being made: - Table and tablet locks are now acquired en masse. For tablets, this is required for correctness; I've documented why I did the same for tables. - We no longer check for non-running tables. AFAICT this was a useless check because a non-running table must be a deleted table, and there's a check for that just before. - We no longer wake the BgTasks thread upon completion. This is because: 1. WakeIfHasPendingUpdates() was semi-broken; pending_updates_ is never set in ProcessTabletReport. 2. Regardless, there's no work for the BgTasks thread to do. Change-Id: Ie6f5cf0e4b1cd1160b3b310d89c6dbf3dd62e43b --- M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h 2 files changed, 380 insertions(+), 408 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/90/8090/1 -- To view, visit http://gerrit.cloudera.org:8080/8090 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ie6f5cf0e4b1cd1160b3b310d89c6dbf3dd62e43b Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Adar Dembo <a...@cloudera.com> Gerrit-Reviewer: Dan Burkert <danburk...@apache.org> Gerrit-Reviewer: Mike Percy <mpe...@apache.org> Gerrit-Reviewer: Todd Lipcon <t...@apache.org>