[jira] [Commented] (CASSANDRA-14326) Handle verbose logging at a different level than DEBUG
[ https://issues.apache.org/jira/browse/CASSANDRA-14326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16439032#comment-16439032 ] Kurt Greaves commented on CASSANDRA-14326: -- As a user/operator I can't see the need to split out a subset of logs to a verbose-system.log. If it helps the operator it should just be in the system.log. If those logs are making system.log too noisy then thought needs to go into better log messages. A bit of code refactoring for clearer/less noisy log messages is preferable to just shipping them off to another file. More log files is just a nightmare for operators in my experience, and most will just end up pumping them all into journald/[insert log service here] defeating the purpose anyway. The proposals so far mostly just sound to me like keeping the debug logging but with a different name. Debug logs are for developers, not operators. > Handle verbose logging at a different level than DEBUG > -- > > Key: CASSANDRA-14326 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14326 > Project: Cassandra > Issue Type: Improvement >Reporter: Alexander Dejanovski >Priority: Major > Fix For: 4.x > > > CASSANDRA-10241 introduced debug logging turned on by default to act as a > verbose system.log and help troubleshoot production issues. > One of the consequence was to severely affect read performance in 2.2 as > contributors weren't all up to speed on how to use logging levels > (CASSANDRA-14318). > As DEBUG level has a very specific meaning in dev, it is confusing to use it > for always on verbose logging and should probably not be used this way in > Cassandra. > Options so far are : > # Bring back common loggings to INFO level (compactions, flushes, etc...) > and disable debug logging by default > # Use files named as verbose-system.log instead of debug.log and use a > custom logging level instead of DEBUG for verbose tracing, that would be > enabled by default. Debug logging would still exist and be disabled by > default in the root logger (not just filtered at the appender level). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14326) Handle verbose logging at a different level than DEBUG
[ https://issues.apache.org/jira/browse/CASSANDRA-14326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406721#comment-16406721 ] Alexander Dejanovski commented on CASSANDRA-14326: -- I agree it would be nice to keep incremental loggings indeed so that verbose contains info + verbose, and debug contains info + verbose + debug, but then we would have to do 2 changes to enable debug logging at will : * change to * uncomment the ASYNCDEBUGLOG appender Otherwise : * if the appender is there we always have something that's written to debug.log (all INFO level stuff) * and if o.a.c is at DEBUG all the time, any call to logger.debug() will have to be in a conditional block to avoid the performance penalty of interpreting the calls and have the appender filter out debug stuff. Unless there's a better way of achieving this ? > Handle verbose logging at a different level than DEBUG > -- > > Key: CASSANDRA-14326 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14326 > Project: Cassandra > Issue Type: Improvement >Reporter: Alexander Dejanovski >Priority: Major > Fix For: 4.x > > > CASSANDRA-10241 introduced debug logging turned on by default to act as a > verbose system.log and help troubleshoot production issues. > One of the consequence was to severely affect read performance in 2.2 as > contributors weren't all up to speed on how to use logging levels > (CASSANDRA-14318). > As DEBUG level has a very specific meaning in dev, it is confusing to use it > for always on verbose logging and should probably not be used this way in > Cassandra. > Options so far are : > # Bring back common loggings to INFO level (compactions, flushes, etc...) > and disable debug logging by default > # Use files named as verbose-system.log instead of debug.log and use a > custom logging level instead of DEBUG for verbose tracing, that would be > enabled by default. Debug logging would still exist and be disabled by > default in the root logger (not just filtered at the appender level). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14326) Handle verbose logging at a different level than DEBUG
[ https://issues.apache.org/jira/browse/CASSANDRA-14326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406447#comment-16406447 ] Jeremiah Jordan commented on CASSANDRA-14326: - One thing I would suggest is to still keep DEBUG filtered to the asynchronous log, even after this change. So that when debug logging is enabled by someone it has less impact in the default logging setup. I think it is useful to keep all messages in the verbose log, otherwise you need to some how merge or otherwise correlate things between the two files. > Handle verbose logging at a different level than DEBUG > -- > > Key: CASSANDRA-14326 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14326 > Project: Cassandra > Issue Type: Improvement >Reporter: Alexander Dejanovski >Priority: Major > Fix For: 4.x > > > CASSANDRA-10241 introduced debug logging turned on by default to act as a > verbose system.log and help troubleshoot production issues. > One of the consequence was to severely affect read performance in 2.2 as > contributors weren't all up to speed on how to use logging levels > (CASSANDRA-14318). > As DEBUG level has a very specific meaning in dev, it is confusing to use it > for always on verbose logging and should probably not be used this way in > Cassandra. > Options so far are : > # Bring back common loggings to INFO level (compactions, flushes, etc...) > and disable debug logging by default > # Use files named as verbose-system.log instead of debug.log and use a > custom logging level instead of DEBUG for verbose tracing, that would be > enabled by default. Debug logging would still exist and be disabled by > default and the root logger level (not just filtered at the appender level). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14326) Handle verbose logging at a different level than DEBUG
[ https://issues.apache.org/jira/browse/CASSANDRA-14326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406387#comment-16406387 ] Paulo Motta commented on CASSANDRA-14326: - bq. The only thing I can think of that will be a bit odd, is to have two different logs, both having INFO messages with one being a superset of the other. We could add a new marker INFO-VERBOSE, that is logged asynchronously to verbose-system.log (former debug.log). Admins would typically only look at system.log, and only go to verbose-system.log when facing problems, troubleshooting issues or wanting to perform advanced tuning, etc. Ultimately advanced operators could still disable the system-verbose.log if/when they're not interested in that. > Handle verbose logging at a different level than DEBUG > -- > > Key: CASSANDRA-14326 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14326 > Project: Cassandra > Issue Type: Improvement >Reporter: Alexander Dejanovski >Priority: Major > Fix For: 4.x > > > CASSANDRA-10241 introduced debug logging turned on by default to act as a > verbose system.log and help troubleshoot production issues. > One of the consequence was to severely affect read performance in 2.2 as > contributors weren't all up to speed on how to use logging levels > (CASSANDRA-14318). > As DEBUG level has a very specific meaning in dev, it is confusing to use it > for always on verbose logging and should probably not be used this way in > Cassandra. > Options so far are : > # Bring back common loggings to INFO level (compactions, flushes, etc...) > and disable debug logging by default > # Use files named as verbose-system.log instead of debug.log and use a > custom logging level instead of DEBUG for verbose tracing, that would be > enabled by default. Debug logging would still exist and be disabled by > default and the root logger level (not just filtered at the appender level). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14326) Handle verbose logging at a different level than DEBUG
[ https://issues.apache.org/jira/browse/CASSANDRA-14326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406374#comment-16406374 ] Stefan Podkowinski commented on CASSANDRA-14326: I'd prefer option #2 over what we have now, if we can use a SLF4J marker for filtering messages in a practical way, as suggested by [~jjordan]. The only thing I can think of that will be a bit odd, is to have two different logs, both having INFO messages with one being a superset of the other. Although I'm aware of the reasons behind this, it's probably not that obvious why you need to keep system.log around with verbose-system.log enabled, too. But this is really more a minor issue, compared to the benefit of preventing performance regressions by just adding a debug statement in hot code paths. > Handle verbose logging at a different level than DEBUG > -- > > Key: CASSANDRA-14326 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14326 > Project: Cassandra > Issue Type: Improvement >Reporter: Alexander Dejanovski >Priority: Major > Fix For: 4.x > > > CASSANDRA-10241 introduced debug logging turned on by default to act as a > verbose system.log and help troubleshoot production issues. > One of the consequence was to severely affect read performance in 2.2 as > contributors weren't all up to speed on how to use logging levels > (CASSANDRA-14318). > As DEBUG level has a very specific meaning in dev, it is confusing to use it > for always on verbose logging and should probably not be used this way in > Cassandra. > Options so far are : > # Bring back common loggings to INFO level (compactions, flushes, etc...) > and disable debug logging by default > # Use files named as verbose-system.log instead of debug.log and use a > custom logging level instead of DEBUG for verbose tracing, that would be > enabled by default. Debug logging would still exist and be disabled by > default and the root logger level (not just filtered at the appender level). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14326) Handle verbose logging at a different level than DEBUG
[ https://issues.apache.org/jira/browse/CASSANDRA-14326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406317#comment-16406317 ] Paulo Motta commented on CASSANDRA-14326: - I think we should go with 2 otherwise we lose the ability introduced by CASSANDRA-10421 to have a human-readable system.log vs verbose troubleshooting log, which I think it has been useful to troubleshoot hard-to-reproduce issues after the fact without affecting performance (best-effort async-appender logging) and cluttering system.log readability (separation of system vs debug log), but I'd be interested in hearing more on what operators from 3.0+ clusters think of the current separation of system.log and debug.log. In any case, the first step of this ticket is to go through all messages being logged at DEBUG and reclassify important ones as INFO, and then we can decide to go with 1 or 2 later. > Handle verbose logging at a different level than DEBUG > -- > > Key: CASSANDRA-14326 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14326 > Project: Cassandra > Issue Type: Improvement >Reporter: Alexander Dejanovski >Priority: Major > Fix For: 4.0 > > > CASSANDRA-10241 introduced debug logging turned on by default to act as a > verbose system.log and help troubleshoot production issues. > One of the consequence was to severely affect read performance in 2.2 as > contributors weren't all up to speed on how to use logging levels > (CASSANDRA-14318). > As DEBUG level has a very specific meaning in dev, it is confusing to use it > for always on verbose logging and should probably not be used this way in > Cassandra. > Options so far are : > # Bring back common loggings to INFO level (compactions, flushes, etc...) > and disable debug logging by default > # Use files named as verbose-system.log instead of debug.log and use a > custom logging level instead of DEBUG for verbose tracing, that would be > enabled by default. Debug logging would still exist and be disabled by > default and the root logger level (not just filtered at the appender level). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
