[jira] [Commented] (HDFS-4080) Add an option to disable block-level state change logging
[ https://issues.apache.org/jira/browse/HDFS-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494172#comment-13494172 ] Suresh Srinivas commented on HDFS-4080: --- +1 for the patch. Add an option to disable block-level state change logging - Key: HDFS-4080 URL: https://issues.apache.org/jira/browse/HDFS-4080 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 3.0.0, 2.0.3-alpha Reporter: Kihwal Lee Assignee: Kihwal Lee Attachments: hdfs-4080.1.patch, hdfs-4080-branch-0.23.patch, hdfs-4080.patch, hdfs-4080.patch, hdfs-4080-trunk.patch Although the block-level logging in namenode is useful for debugging, it can add a significant overhead to busy hdfs clusters since they are done while the namespace write lock is held. One example is shown in HDFS-4075. In this example, the write lock was held for 5 minutes while logging 11 million log messages for 5.5 million block invalidation events. It will be useful if we have an option to disable these block-level log messages and keep other state change messages going. If others feel that they can turned into DEBUG (with addition of isDebugEnabled() checks), that may also work too, but there might be people depending on the messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4080) Add an option to disable block-level state change logging
[
https://issues.apache.org/jira/browse/HDFS-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493250#comment-13493250
]
Kihwal Lee commented on HDFS-4080:
--
bq. instead of the logger name org.apache.hadoop.hdfs.StateChange.block, which
is a change from the existing name, should be just change it to
BlockStateChange. This potentially saves a logging from printing the entire
package name?
I think keeping the hierarchical naming is better. To reduce the amount of log
data, the layout can include something like %c{2} or omit c entirely since
these lines already have a distinct signature. Do you think it is better to add
the logger config with this layout change to
hadoop-common-project/hadoop-common/src/main/conf/log4j.properties? If so, as a
part of this jira or a separate common one?
Add an option to disable block-level state change logging
-
Key: HDFS-4080
URL: https://issues.apache.org/jira/browse/HDFS-4080
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node
Affects Versions: 3.0.0, 2.0.3-alpha
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Attachments: hdfs-4080.1.patch, hdfs-4080.patch, hdfs-4080.patch
Although the block-level logging in namenode is useful for debugging, it can
add a significant overhead to busy hdfs clusters since they are done while
the namespace write lock is held. One example is shown in HDFS-4075. In this
example, the write lock was held for 5 minutes while logging 11 million log
messages for 5.5 million block invalidation events.
It will be useful if we have an option to disable these block-level log
messages and keep other state change messages going. If others feel that
they can turned into DEBUG (with addition of isDebugEnabled() checks), that
may also work too, but there might be people depending on the messages.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4080) Add an option to disable block-level state change logging
[ https://issues.apache.org/jira/browse/HDFS-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493351#comment-13493351 ] Suresh Srinivas commented on HDFS-4080: --- We should add logger config as a part of this jira. That way a single jira tracks the related changes. As regards to org.apache.hadoop.hdfs.StateChange.block I am okay keeping the name longer. Though the named logger (not based on class) can have any name, the name you have used is similar to a class name. Consider changing the name to org.apache.hadoop.hdfs.StateChange.Block (block-Block). Add an option to disable block-level state change logging - Key: HDFS-4080 URL: https://issues.apache.org/jira/browse/HDFS-4080 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 3.0.0, 2.0.3-alpha Reporter: Kihwal Lee Assignee: Kihwal Lee Attachments: hdfs-4080.1.patch, hdfs-4080.patch, hdfs-4080.patch Although the block-level logging in namenode is useful for debugging, it can add a significant overhead to busy hdfs clusters since they are done while the namespace write lock is held. One example is shown in HDFS-4075. In this example, the write lock was held for 5 minutes while logging 11 million log messages for 5.5 million block invalidation events. It will be useful if we have an option to disable these block-level log messages and keep other state change messages going. If others feel that they can turned into DEBUG (with addition of isDebugEnabled() checks), that may also work too, but there might be people depending on the messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4080) Add an option to disable block-level state change logging
[ https://issues.apache.org/jira/browse/HDFS-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493511#comment-13493511 ] Kihwal Lee commented on HDFS-4080: -- On a second thought, making the category shorter may be better, since having to use a different appender to omit or shorten the category is more invasive. I will follow your original suggestion. :) Add an option to disable block-level state change logging - Key: HDFS-4080 URL: https://issues.apache.org/jira/browse/HDFS-4080 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 3.0.0, 2.0.3-alpha Reporter: Kihwal Lee Assignee: Kihwal Lee Attachments: hdfs-4080.1.patch, hdfs-4080.patch, hdfs-4080.patch Although the block-level logging in namenode is useful for debugging, it can add a significant overhead to busy hdfs clusters since they are done while the namespace write lock is held. One example is shown in HDFS-4075. In this example, the write lock was held for 5 minutes while logging 11 million log messages for 5.5 million block invalidation events. It will be useful if we have an option to disable these block-level log messages and keep other state change messages going. If others feel that they can turned into DEBUG (with addition of isDebugEnabled() checks), that may also work too, but there might be people depending on the messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4080) Add an option to disable block-level state change logging
[ https://issues.apache.org/jira/browse/HDFS-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493538#comment-13493538 ] Kihwal Lee commented on HDFS-4080: -- A new trunk patch and a branch-0.23 patch are attached. Add an option to disable block-level state change logging - Key: HDFS-4080 URL: https://issues.apache.org/jira/browse/HDFS-4080 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 3.0.0, 2.0.3-alpha Reporter: Kihwal Lee Assignee: Kihwal Lee Attachments: hdfs-4080.1.patch, hdfs-4080-branch-0.23.patch, hdfs-4080.patch, hdfs-4080.patch, hdfs-4080-trunk.patch Although the block-level logging in namenode is useful for debugging, it can add a significant overhead to busy hdfs clusters since they are done while the namespace write lock is held. One example is shown in HDFS-4075. In this example, the write lock was held for 5 minutes while logging 11 million log messages for 5.5 million block invalidation events. It will be useful if we have an option to disable these block-level log messages and keep other state change messages going. If others feel that they can turned into DEBUG (with addition of isDebugEnabled() checks), that may also work too, but there might be people depending on the messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4080) Add an option to disable block-level state change logging
[
https://issues.apache.org/jira/browse/HDFS-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493650#comment-13493650
]
Hadoop QA commented on HDFS-4080:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12552722/hdfs-4080-trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/3469//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3469//console
This message is automatically generated.
Add an option to disable block-level state change logging
-
Key: HDFS-4080
URL: https://issues.apache.org/jira/browse/HDFS-4080
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node
Affects Versions: 3.0.0, 2.0.3-alpha
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Attachments: hdfs-4080.1.patch, hdfs-4080-branch-0.23.patch,
hdfs-4080.patch, hdfs-4080.patch, hdfs-4080-trunk.patch
Although the block-level logging in namenode is useful for debugging, it can
add a significant overhead to busy hdfs clusters since they are done while
the namespace write lock is held. One example is shown in HDFS-4075. In this
example, the write lock was held for 5 minutes while logging 11 million log
messages for 5.5 million block invalidation events.
It will be useful if we have an option to disable these block-level log
messages and keep other state change messages going. If others feel that
they can turned into DEBUG (with addition of isDebugEnabled() checks), that
may also work too, but there might be people depending on the messages.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4080) Add an option to disable block-level state change logging
[
https://issues.apache.org/jira/browse/HDFS-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491951#comment-13491951
]
Suresh Srinivas commented on HDFS-4080:
---
Kihwal, the patch looks good.
One question - instead of the logger name
{{org.apache.hadoop.hdfs.StateChange.block}}, which is a change from the
existing name, should be just change it to BlockStateChange. This potentially
saves a logging from printing the entire package name?
This should also be marked incompatible to indicate the change in logger name?
Add an option to disable block-level state change logging
-
Key: HDFS-4080
URL: https://issues.apache.org/jira/browse/HDFS-4080
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node
Affects Versions: 3.0.0, 2.0.3-alpha
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Attachments: hdfs-4080.1.patch, hdfs-4080.patch, hdfs-4080.patch
Although the block-level logging in namenode is useful for debugging, it can
add a significant overhead to busy hdfs clusters since they are done while
the namespace write lock is held. One example is shown in HDFS-4075. In this
example, the write lock was held for 5 minutes while logging 11 million log
messages for 5.5 million block invalidation events.
It will be useful if we have an option to disable these block-level log
messages and keep other state change messages going. If others feel that
they can turned into DEBUG (with addition of isDebugEnabled() checks), that
may also work too, but there might be people depending on the messages.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4080) Add an option to disable block-level state change logging
[
https://issues.apache.org/jira/browse/HDFS-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13487962#comment-13487962
]
Hadoop QA commented on HDFS-4080:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12551558/hdfs-4080.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/3433//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3433//console
This message is automatically generated.
Add an option to disable block-level state change logging
-
Key: HDFS-4080
URL: https://issues.apache.org/jira/browse/HDFS-4080
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node
Affects Versions: 3.0.0, 2.0.3-alpha
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Attachments: hdfs-4080.1.patch, hdfs-4080.patch, hdfs-4080.patch
Although the block-level logging in namenode is useful for debugging, it can
add a significant overhead to busy hdfs clusters since they are done while
the namespace write lock is held. One example is shown in HDFS-4075. In this
example, the write lock was held for 5 minutes while logging 11 million log
messages for 5.5 million block invalidation events.
It will be useful if we have an option to disable these block-level log
messages and keep other state change messages going. If others feel that
they can turned into DEBUG (with addition of isDebugEnabled() checks), that
may also work too, but there might be people depending on the messages.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4080) Add an option to disable block-level state change logging
[
https://issues.apache.org/jira/browse/HDFS-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486105#comment-13486105
]
Hadoop QA commented on HDFS-4080:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12551200/hdfs-4080.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3417//console
This message is automatically generated.
Add an option to disable block-level state change logging
-
Key: HDFS-4080
URL: https://issues.apache.org/jira/browse/HDFS-4080
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node
Affects Versions: 3.0.0, 2.0.3-alpha
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Attachments: hdfs-4080.patch
Although the block-level logging in namenode is useful for debugging, it can
add a significant overhead to busy hdfs clusters since they are done while
the namespace write lock is held. One example is shown in HDFS-4075. In this
example, the write lock was held for 5 minutes while logging 11 million log
messages for 5.5 million block invalidation events.
It will be useful if we have an option to disable these block-level log
messages and keep other state change messages going. If others feel that
they can turned into DEBUG (with addition of isDebugEnabled() checks), that
may also work too, but there might be people depending on the messages.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4080) Add an option to disable block-level state change logging
[
https://issues.apache.org/jira/browse/HDFS-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486308#comment-13486308
]
Hadoop QA commented on HDFS-4080:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12551221/hdfs-4080.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/3418//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3418//console
This message is automatically generated.
Add an option to disable block-level state change logging
-
Key: HDFS-4080
URL: https://issues.apache.org/jira/browse/HDFS-4080
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node
Affects Versions: 3.0.0, 2.0.3-alpha
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Attachments: hdfs-4080.1.patch, hdfs-4080.patch
Although the block-level logging in namenode is useful for debugging, it can
add a significant overhead to busy hdfs clusters since they are done while
the namespace write lock is held. One example is shown in HDFS-4075. In this
example, the write lock was held for 5 minutes while logging 11 million log
messages for 5.5 million block invalidation events.
It will be useful if we have an option to disable these block-level log
messages and keep other state change messages going. If others feel that
they can turned into DEBUG (with addition of isDebugEnabled() checks), that
may also work too, but there might be people depending on the messages.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4080) Add an option to disable block-level state change logging
[ https://issues.apache.org/jira/browse/HDFS-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486408#comment-13486408 ] Suresh Srinivas commented on HDFS-4080: --- Kihwal, quick comment, having come out of my own log cleanup work :-). It is probably a good idea to retain Block * and not change to Dir *. BTW I have removed unnecessary prefixes such NameNode. from the logs - see HDFS-4122. Sorry you will have to rebase this patch. Add an option to disable block-level state change logging - Key: HDFS-4080 URL: https://issues.apache.org/jira/browse/HDFS-4080 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 3.0.0, 2.0.3-alpha Reporter: Kihwal Lee Assignee: Kihwal Lee Attachments: hdfs-4080.1.patch, hdfs-4080.patch Although the block-level logging in namenode is useful for debugging, it can add a significant overhead to busy hdfs clusters since they are done while the namespace write lock is held. One example is shown in HDFS-4075. In this example, the write lock was held for 5 minutes while logging 11 million log messages for 5.5 million block invalidation events. It will be useful if we have an option to disable these block-level log messages and keep other state change messages going. If others feel that they can turned into DEBUG (with addition of isDebugEnabled() checks), that may also work too, but there might be people depending on the messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4080) Add an option to disable block-level state change logging
[ https://issues.apache.org/jira/browse/HDFS-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13482346#comment-13482346 ] Kihwal Lee commented on HDFS-4080: -- Thanks Nicholas. I think that will address most of problems for now. Once we have locking refinements such as HDFS-3937, we will be able to do what Aaron suggested as well. Add an option to disable block-level state change logging - Key: HDFS-4080 URL: https://issues.apache.org/jira/browse/HDFS-4080 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 3.0.0, 2.0.3-alpha Reporter: Kihwal Lee Although the block-level logging in namenode is useful for debugging, it can add a significant overhead to busy hdfs clusters since they are done while the namespace write lock is held. One example is shown in HDFS-4075. In this example, the write lock was held for 5 minutes while logging 11 million log messages for 5.5 million block invalidation events. It will be useful if we have an option to disable these block-level log messages and keep other state change messages going. If others feel that they can turned into DEBUG (with addition of isDebugEnabled() checks), that may also work too, but there might be people depending on the messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4080) Add an option to disable block-level state change logging
[ https://issues.apache.org/jira/browse/HDFS-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481390#comment-13481390 ] Kihwal Lee commented on HDFS-4080: -- Many are being logged while the lock is held by an object far up in the call tree, so it is not as simple as HDFS-4052. The ultimate solution may be finer granule locking. Add an option to disable block-level state change logging - Key: HDFS-4080 URL: https://issues.apache.org/jira/browse/HDFS-4080 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 3.0.0, 2.0.3-alpha Reporter: Kihwal Lee Although the block-level logging in namenode is useful for debugging, it can add a significant overhead to busy hdfs clusters since they are done while the namespace write lock is held. One example is shown in HDFS-4075. In this example, the write lock was held for 5 minutes while logging 11 million log messages for 5.5 million block invalidation events. It will be useful if we have an option to disable these block-level log messages and keep other state change messages going. If others feel that they can turned into DEBUG (with addition of isDebugEnabled() checks), that may also work too, but there might be people depending on the messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4080) Add an option to disable block-level state change logging
[ https://issues.apache.org/jira/browse/HDFS-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481970#comment-13481970 ] Tsz Wo (Nicholas), SZE commented on HDFS-4080: -- Let's put the block-level log in BlockManager state change log (a new log). Then, it can be set independently of the NN state change log. Add an option to disable block-level state change logging - Key: HDFS-4080 URL: https://issues.apache.org/jira/browse/HDFS-4080 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 3.0.0, 2.0.3-alpha Reporter: Kihwal Lee Although the block-level logging in namenode is useful for debugging, it can add a significant overhead to busy hdfs clusters since they are done while the namespace write lock is held. One example is shown in HDFS-4075. In this example, the write lock was held for 5 minutes while logging 11 million log messages for 5.5 million block invalidation events. It will be useful if we have an option to disable these block-level log messages and keep other state change messages going. If others feel that they can turned into DEBUG (with addition of isDebugEnabled() checks), that may also work too, but there might be people depending on the messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4080) Add an option to disable block-level state change logging
[ https://issues.apache.org/jira/browse/HDFS-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13480139#comment-13480139 ] Aaron T. Myers commented on HDFS-4080: -- Would it be possible to just move the logging outside of the FSNS lock? Jing did a similar change recently in HDFS-4052. Add an option to disable block-level state change logging - Key: HDFS-4080 URL: https://issues.apache.org/jira/browse/HDFS-4080 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 3.0.0, 2.0.3-alpha Reporter: Kihwal Lee Although the block-level logging in namenode is useful for debugging, it can add a significant overhead to busy hdfs clusters since they are done while the namespace write lock is held. One example is shown in HDFS-4075. In this example, the write lock was held for 5 minutes while logging 11 million log messages for 5.5 million block invalidation events. It will be useful if we have an option to disable these block-level log messages and keep other state change messages going. If others feel that they can turned into DEBUG (with addition of isDebugEnabled() checks), that may also work too, but there might be people depending on the messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
