Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/7111
to look at the new patch set (#3).
Change subject: KUDU-1442 log number of open log block containers
......................................................................
KUDU-1442 log number of open log block containers
Tservers with large numbers of log block containers can appear to
hang for a long time without logging anything during initialization.
This commit adds a log message every 10 seconds per data dir thread
stating how many containers have been opened.
No automated tests were added because this change only affects the
contents of INFO logs, and no existing log block manager tests check
log output. It was tested manually by running a tserver with 3 data
dirs, creating a table with 5,000,000 rows, restarting the tserver,
and inspecting its log output, which contained these lines:
I0607 16:23:05.822599 27752 log_block_manager.cc:1488] Data dir
/data/8/samuel.okrent/kudu/build/debug/data2/data is on an ext4 filesystem
vulnerable to KUDU-1508 with block size 2048
I0607 16:23:05.822681 27752 log_block_manager.cc:1498] Limiting containers on
data directory /data/8/samuel.okrent/kudu/build/debug/data2/data to 1353 blocks
I0607 16:23:05.822835 27752 log_block_manager.cc:1488] Data dir
/data/8/samuel.okrent/kudu/build/debug/data3/data is on an ext4 filesystem
vulnerable to KUDU-1508 with block size 2048
I0607 16:23:05.822851 27752 log_block_manager.cc:1498] Limiting containers on
data directory /data/8/samuel.okrent/kudu/build/debug/data3/data to 1353 blocks
I0607 16:23:05.822892 27752 log_block_manager.cc:1488] Data dir
/data/8/samuel.okrent/kudu/build/debug/data4/data is on an ext4 filesystem
vulnerable to KUDU-1508 with block size 2048
I0607 16:23:05.822903 27752 log_block_manager.cc:1498] Limiting containers on
data directory /data/8/samuel.okrent/kudu/build/debug/data4/data to 1353 blocks
I0607 16:23:05.824937 27759 log_block_manager.cc:2019] Opened 1 log block
containers in /data/8/samuel.okrent/kudu/build/debug/data2/data
I0607 16:23:05.825098 27760 log_block_manager.cc:2019] Opened 1 log block
containers in /data/8/samuel.okrent/kudu/build/debug/data3/data
I0607 16:23:05.825115 27761 log_block_manager.cc:2019] Opened 1 log block
containers in /data/8/samuel.okrent/kudu/build/debug/data4/data
I0607 16:23:05.826077 27759 log_block_manager.cc:2019] Opened 3 log block
containers in /data/8/samuel.okrent/kudu/build/debug/data2/data
I0607 16:23:05.826352 27760 log_block_manager.cc:2019] Opened 3 log block
containers in /data/8/samuel.okrent/kudu/build/debug/data3/data
I0607 16:23:05.826354 27761 log_block_manager.cc:2019] Opened 3 log block
containers in /data/8/samuel.okrent/kudu/build/debug/data4/data
I0607 16:23:05.827477 27752 fs_manager.cc:265] Time spent opening block
manager: real 0.008s user 0.002s sys 0.002s
This test was run with a logging interval of 1ms instead of 10s (by
changing the hardcoded value in log_block_manager.cc, due to the size
of the dataset needed to delay initialization by 10s). This resulted
in periodic logging from each data dir thread of how many containers
had been opened. When the same test was run with the normal parameter
of 10s, the usual output was produced, without any of the
"Opened x log block containers..." messages.
Change-Id: I3dd5eccd166ae81be26d0aa75758539f84d62656
---
M src/kudu/fs/log_block_manager.cc
1 file changed, 10 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/11/7111/3
--
To view, visit http://gerrit.cloudera.org:8080/7111
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3dd5eccd166ae81be26d0aa75758539f84d62656
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Sam Okrent <[email protected]>
Gerrit-Reviewer: Adar Dembo <[email protected]>
Gerrit-Reviewer: Alexey Serbin <[email protected]>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Sam Okrent <[email protected]>
