Adar Dembo has submitted this change and it was merged. Change subject: KUDU-1442 log number of open log block containers ......................................................................
KUDU-1442 log number of open log block containers Tservers with large numbers of log block containers can appear to hang for a long time without logging anything during initialization. This commit adds a log message every 10 seconds per data dir thread stating how many containers have been opened. No automated tests were added because this change only affects the contents of INFO logs, and no existing log block manager tests check log output. It was tested manually by running a tserver with 3 data dirs, creating a table with 5,000,000 rows, restarting the tserver, and inspecting its log output, which contained these lines: I0608 11:30:39.587874 6090 log_block_manager.cc:1488] Data dir /data/8/samuel.okrent/kudu/build/debug/data2/data is on an ext4 filesystem vulnerable to KUDU-1508 with block size 2048 I0608 11:30:39.587934 6090 log_block_manager.cc:1498] Limiting containers on data directory /data/8/samuel.okrent/kudu/build/debug/data2/data to 1353 blocks I0608 11:30:39.588039 6090 log_block_manager.cc:1488] Data dir /data/8/samuel.okrent/kudu/build/debug/data3/data is on an ext4 filesystem vulnerable to KUDU-1508 with block size 2048 I0608 11:30:39.588049 6090 log_block_manager.cc:1498] Limiting containers on data directory /data/8/samuel.okrent/kudu/build/debug/data3/data to 1353 blocks I0608 11:30:39.588080 6090 log_block_manager.cc:1488] Data dir /data/8/samuel.okrent/kudu/build/debug/data4/data is on an ext4 filesystem vulnerable to KUDU-1508 with block size 2048 I0608 11:30:39.588088 6090 log_block_manager.cc:1498] Limiting containers on data directory /data/8/samuel.okrent/kudu/build/debug/data4/data to 1353 blocks I0608 11:30:39.590297 6097 log_block_manager.cc:2022] /data/8/samuel.okrent/kudu/build/debug/data2/data: opened 1 log block containers I0608 11:30:39.590458 6098 log_block_manager.cc:2022] /data/8/samuel.okrent/kudu/build/debug/data3/data: opened 1 log block containers I0608 11:30:39.590482 6099 log_block_manager.cc:2022] /data/8/samuel.okrent/kudu/build/debug/data4/data: opened 1 log block containers I0608 11:30:39.591612 6097 log_block_manager.cc:2022] /data/8/samuel.okrent/kudu/build/debug/data2/data: opened 3 log block containers I0608 11:30:39.591668 6098 log_block_manager.cc:2022] /data/8/samuel.okrent/kudu/build/debug/data3/data: opened 3 log block containers I0608 11:30:39.591806 6099 log_block_manager.cc:2022] /data/8/samuel.okrent/kudu/build/debug/data4/data: opened 3 log block containers I0608 11:30:39.593066 6090 fs_manager.cc:263] Time spent opening block manager: real 0.007s user 0.003s sys 0.001s This test was run with a logging interval of 1ms instead of 10s (by changing the hardcoded value in log_block_manager.cc, due to the size of the dataset needed to delay initialization by 10s). This resulted in periodic logging from each data dir thread of how many containers had been opened. When the same test was run with the normal parameter of 10s, the usual output was produced, without any of the "Opened x log block containers..." messages Change-Id: I3dd5eccd166ae81be26d0aa75758539f84d62656 Reviewed-on: http://gerrit.cloudera.org:8080/7111 Reviewed-by: Adar Dembo <[email protected]> Tested-by: Adar Dembo <[email protected]> --- M src/kudu/fs/log_block_manager.cc 1 file changed, 10 insertions(+), 0 deletions(-) Approvals: Adar Dembo: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/7111 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I3dd5eccd166ae81be26d0aa75758539f84d62656 Gerrit-PatchSet: 7 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Sam Okrent <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Sam Okrent <[email protected]>
