[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13243038#comment-13243038 ] Hudson commented on HBASE-5655: --- Integrated in HBase-TRUNK-security #155 (See [https://builds.apache.org/job/HBase-TRUNK-security/155/]) HBASE-5655 Cap space usage of default log4j rolling policy (Himanshu) (Revision 1307453) Result = SUCCESS tedyu : Files : * /hbase/trunk/bin/hbase-daemon.sh * /hbase/trunk/conf/hbase-env.sh * /hbase/trunk/conf/log4j.properties > Cap space usage of default log4j rolling policy > --- > > Key: HBASE-5655 > URL: https://issues.apache.org/jira/browse/HBASE-5655 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.1 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Fix For: 0.96.0 > > Attachments: 5655-v1.patch, HBase-5655-v2.patch, HBase-5655-v3.patch > > > The current default log4j policy is to use Daily Rolling File Appender > (DRFA). At times, its good to have a cap on the maximum size of the logs in > order to limit its disk usage. Here is a proposal to set a new file appemder > (RFA) as the default appender. It can be configured via env so that existing > tools can use the current behavior of using DRFA instead. > This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242927#comment-13242927 ] Himanshu Vashishtha commented on HBASE-5655: Is it ok to mark it resolved now? > Cap space usage of default log4j rolling policy > --- > > Key: HBASE-5655 > URL: https://issues.apache.org/jira/browse/HBASE-5655 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.1 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Fix For: 0.96.0 > > Attachments: 5655-v1.patch, HBase-5655-v2.patch, HBase-5655-v3.patch > > > The current default log4j policy is to use Daily Rolling File Appender > (DRFA). At times, its good to have a cap on the maximum size of the logs in > order to limit its disk usage. Here is a proposal to set a new file appemder > (RFA) as the default appender. It can be configured via env so that existing > tools can use the current behavior of using DRFA instead. > This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242476#comment-13242476 ] Hudson commented on HBASE-5655: --- Integrated in HBase-TRUNK #2698 (See [https://builds.apache.org/job/HBase-TRUNK/2698/]) HBASE-5655 Cap space usage of default log4j rolling policy (Himanshu) (Revision 1307453) Result = FAILURE tedyu : Files : * /hbase/trunk/bin/hbase-daemon.sh * /hbase/trunk/conf/hbase-env.sh * /hbase/trunk/conf/log4j.properties > Cap space usage of default log4j rolling policy > --- > > Key: HBASE-5655 > URL: https://issues.apache.org/jira/browse/HBASE-5655 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.1 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Fix For: 0.96.0 > > Attachments: 5655-v1.patch, HBase-5655-v2.patch, HBase-5655-v3.patch > > > The current default log4j policy is to use Daily Rolling File Appender > (DRFA). At times, its good to have a cap on the maximum size of the logs in > order to limit its disk usage. Here is a proposal to set a new file appemder > (RFA) as the default appender. It can be configured via env so that existing > tools can use the current behavior of using DRFA instead. > This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242394#comment-13242394 ] Zhihong Yu commented on HBASE-5655: --- Integrated to TRUNK. Thanks for the patch Himanshu. Thanks for the review Stack. > Cap space usage of default log4j rolling policy > --- > > Key: HBASE-5655 > URL: https://issues.apache.org/jira/browse/HBASE-5655 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.1 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Fix For: 0.96.0 > > Attachments: 5655-v1.patch, HBase-5655-v2.patch, HBase-5655-v3.patch > > > The current default log4j policy is to use Daily Rolling File Appender > (DRFA). At times, its good to have a cap on the maximum size of the logs in > order to limit its disk usage. Here is a proposal to set a new file appemder > (RFA) as the default appender. It can be configured via env so that existing > tools can use the current behavior of using DRFA instead. > This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241989#comment-13241989 ] Hadoop QA commented on HBASE-5655: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12520536/HBase-5655-v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestImportTsv Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1349//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1349//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1349//console This message is automatically generated. > Cap space usage of default log4j rolling policy > --- > > Key: HBASE-5655 > URL: https://issues.apache.org/jira/browse/HBASE-5655 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.1 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Attachments: 5655-v1.patch, HBase-5655-v2.patch, HBase-5655-v3.patch > > > The current default log4j policy is to use Daily Rolling File Appender > (DRFA). At times, its good to have a cap on the maximum size of the logs in > order to limit its disk usage. Here is a proposal to set a new file appemder > (RFA) as the default appender. It can be configured via env so that existing > tools can use the current behavior of using DRFA instead. > This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[
https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241942#comment-13241942
]
Zhihong Yu commented on HBASE-5655:
---
Patch looks good.
Minor comment:
{code}
+# In case one need to do log
{code}
The above should read 'one needs to ...'
The 2 of the test failures should have been fixed by MAPREDUCE-3583.
> Cap space usage of default log4j rolling policy
> ---
>
> Key: HBASE-5655
> URL: https://issues.apache.org/jira/browse/HBASE-5655
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.92.1
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
> Attachments: 5655-v1.patch, HBase-5655-v2.patch
>
>
> The current default log4j policy is to use Daily Rolling File Appender
> (DRFA). At times, its good to have a cap on the maximum size of the logs in
> order to limit its disk usage. Here is a proposal to set a new file appemder
> (RFA) as the default appender. It can be configured via env so that existing
> tools can use the current behavior of using DRFA instead.
> This is in parallel with jira Hadoop-8149.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241894#comment-13241894 ] David S. Wang commented on HBASE-5655: -- There is already a JIRA filed for this: HBASE-5608. I highly doubt that your patch is causing these tests to fail. > Cap space usage of default log4j rolling policy > --- > > Key: HBASE-5655 > URL: https://issues.apache.org/jira/browse/HBASE-5655 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.1 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Attachments: 5655-v1.patch, HBase-5655-v2.patch > > > The current default log4j policy is to use Daily Rolling File Appender > (DRFA). At times, its good to have a cap on the maximum size of the logs in > order to limit its disk usage. Here is a proposal to set a new file appemder > (RFA) as the default appender. It can be configured via env so that existing > tools can use the current behavior of using DRFA instead. > This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241883#comment-13241883 ] Himanshu Vashishtha commented on HBASE-5655: It will be good to know whether these failures are caused by this patch or are they just flaky. Looking at the logs for TestTAbleMapReduce's failures, it seems metrics2 were not loaded, and it gave out NFE: "attempt_20120329220930011_0001_r_02_0: 2012-03-29 22:09:50,378 WARN [main] impl.MetricsSystemImpl(137): Metrics system not started: Cannot locate configuration: tried hadoop-metrics2-reducetask.properties, hadoop-metrics2.properties" Shall I look into it, or are they just flaky. I can't correlate these failures with log4j rolling policy change as of now. > Cap space usage of default log4j rolling policy > --- > > Key: HBASE-5655 > URL: https://issues.apache.org/jira/browse/HBASE-5655 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.1 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Attachments: 5655-v1.patch, HBase-5655-v2.patch > > > The current default log4j policy is to use Daily Rolling File Appender > (DRFA). At times, its good to have a cap on the maximum size of the logs in > order to limit its disk usage. Here is a proposal to set a new file appemder > (RFA) as the default appender. It can be configured via env so that existing > tools can use the current behavior of using DRFA instead. > This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241875#comment-13241875 ] Hadoop QA commented on HBASE-5655: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12520517/HBase-5655-v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1348//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1348//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1348//console This message is automatically generated. > Cap space usage of default log4j rolling policy > --- > > Key: HBASE-5655 > URL: https://issues.apache.org/jira/browse/HBASE-5655 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.1 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Attachments: 5655-v1.patch, HBase-5655-v2.patch > > > The current default log4j policy is to use Daily Rolling File Appender > (DRFA). At times, its good to have a cap on the maximum size of the logs in > order to limit its disk usage. Here is a proposal to set a new file appemder > (RFA) as the default appender. It can be configured via env so that existing > tools can use the current behavior of using DRFA instead. > This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[
https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241536#comment-13241536
]
stack commented on HBASE-5655:
--
Did you intend to change this H?
{code}
-hbase.root.logger=INFO,console
+hbase.root.logger=DEBUG,console
{code}
I think you should add more comment in hbase-env.sh suggesting DRFA if folks
want the old behavior (though it has issues -- point to this issue?).
Good stuff
> Cap space usage of default log4j rolling policy
> ---
>
> Key: HBASE-5655
> URL: https://issues.apache.org/jira/browse/HBASE-5655
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.92.1
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
> Attachments: 5655-v1.patch
>
>
> The current default log4j policy is to use Daily Rolling File Appender
> (DRFA). At times, its good to have a cap on the maximum size of the logs in
> order to limit its disk usage. Here is a proposal to set a new file appemder
> (RFA) as the default appender. It can be configured via env so that existing
> tools can use the current behavior of using DRFA instead.
> This is in parallel with jira Hadoop-8149.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241512#comment-13241512 ] Hadoop QA commented on HBASE-5655: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12520446/5655-v1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1344//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1344//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1344//console This message is automatically generated. > Cap space usage of default log4j rolling policy > --- > > Key: HBASE-5655 > URL: https://issues.apache.org/jira/browse/HBASE-5655 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.1 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Attachments: 5655-v1.patch > > > The current default log4j policy is to use Daily Rolling File Appender > (DRFA). At times, its good to have a cap on the maximum size of the logs in > order to limit its disk usage. Here is a proposal to set a new file appemder > (RFA) as the default appender. It can be configured via env so that existing > tools can use the current behavior of using DRFA instead. > This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241460#comment-13241460 ] Himanshu Vashishtha commented on HBASE-5655: I have tested this by running HBase with this settings. It uses the RFA policy. Using an env property with HBASE_ROOT_LOGGER=DRFA takes the old DRFA policy. > Cap space usage of default log4j rolling policy > --- > > Key: HBASE-5655 > URL: https://issues.apache.org/jira/browse/HBASE-5655 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.1 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Attachments: 5655-v1.patch > > > The current default log4j policy is to use Daily Rolling File Appender > (DRFA). At times, its good to have a cap on the maximum size of the logs in > order to limit its disk usage. Here is a proposal to set a new file appemder > (RFA) as the default appender. It can be configured via env so that existing > tools can use the current behavior of using DRFA instead. > This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240955#comment-13240955 ] stack commented on HBASE-5655: -- OK. Lets target the singularity then. Will need fat release note and yeah, an incompatible change but for the sake of better usability. Thanks lads. > Cap space usage of default log4j rolling policy > --- > > Key: HBASE-5655 > URL: https://issues.apache.org/jira/browse/HBASE-5655 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.1 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > > The current default log4j policy is to use Daily Rolling File Appender > (DRFA). At times, its good to have a cap on the maximum size of the logs in > order to limit its disk usage. Here is a proposal to set a new file appemder > (RFA) as the default appender. It can be configured via env so that existing > tools can use the current behavior of using DRFA instead. > This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240020#comment-13240020 ] Eli Collins commented on HBASE-5655: The issue is that the DRFA doesn't have the ability to cap the the #files or space usage, so people who aren't using host-level log rotation fill up their log partitions, their clusters crash hard. The idea here is to change the default to something safe (won't crash your cluster) and people who want to keep lots of logs can adjust the limit accordingly, eg they can bump up the default to the size of their log partition. Should make HBase more user-friendly. Given that the default is 5gb (lots of logs!) I suspect few people will need to do that. Ideally we'd be able to keep daily files AND limit space usage, unfortunately the DRFA doesn't support that =( Given that this is an incompatible change it may be better to target for 94 or the singularity. > Cap space usage of default log4j rolling policy > --- > > Key: HBASE-5655 > URL: https://issues.apache.org/jira/browse/HBASE-5655 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.1 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > > The current default log4j policy is to use Daily Rolling File Appender > (DRFA). At times, its good to have a cap on the maximum size of the logs in > order to limit its disk usage. Here is a proposal to set a new file appemder > (RFA) as the default appender. It can be configured via env so that existing > tools can use the current behavior of using DRFA instead. > This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240012#comment-13240012 ] Himanshu Vashishtha commented on HBASE-5655: Yes, it will not roll over the log on a date change. Disk space is not the only issue for DRFA; its java doc says there are also possible concurrency issues (or data loss). "DailyRollingFileAppender has been observed to exhibit synchronization issues and data loss. The log4j extras companion includes alternatives which should be considered for new deployments and which are discussed in the documentation for org.apache.log4j.rolling.RollingFileAppender." Therefore, I think it will be a positive move if we switch to RFA as default; and let users set an env param in case they do want to use DRFA one. Its just an idea, though :) > Cap space usage of default log4j rolling policy > --- > > Key: HBASE-5655 > URL: https://issues.apache.org/jira/browse/HBASE-5655 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.1 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > > The current default log4j policy is to use Daily Rolling File Appender > (DRFA). At times, its good to have a cap on the maximum size of the logs in > order to limit its disk usage. Here is a proposal to set a new file appemder > (RFA) as the default appender. It can be configured via env so that existing > tools can use the current behavior of using DRFA instead. > This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239836#comment-13239836 ] stack commented on HBASE-5655: -- It won't roll on the day boundary though, is that right? > Cap space usage of default log4j rolling policy > --- > > Key: HBASE-5655 > URL: https://issues.apache.org/jira/browse/HBASE-5655 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.1 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > > The current default log4j policy is to use Daily Rolling File Appender > (DRFA). At times, its good to have a cap on the maximum size of the logs in > order to limit its disk usage. Here is a proposal to set a new file appemder > (RFA) as the default appender. It can be configured via env so that existing > tools can use the current behavior of using DRFA instead. > This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239838#comment-13239838 ] stack commented on HBASE-5655: -- We could just write a section in reference guide suggesting you might want to change the default if you are worried about space or don't want to keep files around a long time. > Cap space usage of default log4j rolling policy > --- > > Key: HBASE-5655 > URL: https://issues.apache.org/jira/browse/HBASE-5655 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.1 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > > The current default log4j policy is to use Daily Rolling File Appender > (DRFA). At times, its good to have a cap on the maximum size of the logs in > order to limit its disk usage. Here is a proposal to set a new file appemder > (RFA) as the default appender. It can be configured via env so that existing > tools can use the current behavior of using DRFA instead. > This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239627#comment-13239627 ] Himanshu Vashishtha commented on HBASE-5655: You can define its maximum size and the number of history files. More specifically, log4j.appender.RFA.MaxFileSize=. log4j.appender.RFA.MaxBackupIndex= // This is similar to DRFA.MaxBackupIndex though. > Cap space usage of default log4j rolling policy > --- > > Key: HBASE-5655 > URL: https://issues.apache.org/jira/browse/HBASE-5655 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.1 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > > The current default log4j policy is to use Daily Rolling File Appender > (DRFA). At times, its good to have a cap on the maximum size of the logs in > order to limit its disk usage. Here is a proposal to set a new file appemder > (RFA) as the default appender. It can be configured via env so that existing > tools can use the current behavior of using DRFA instead. > This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239612#comment-13239612 ] stack commented on HBASE-5655: -- What does RFA do different from DRFA? > Cap space usage of default log4j rolling policy > --- > > Key: HBASE-5655 > URL: https://issues.apache.org/jira/browse/HBASE-5655 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.1 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > > The current default log4j policy is to use Daily Rolling File Appender > (DRFA). At times, its good to have a cap on the maximum size of the logs in > order to limit its disk usage. Here is a proposal to set a new file appemder > (RFA) as the default appender. It can be configured via env so that existing > tools can use the current behavior of using DRFA instead. > This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[
https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239578#comment-13239578
]
Himanshu Vashishtha commented on HBASE-5655:
It involves adding a new appender description in log4j.properties file, and
making the hbase-daemon.sh to set the root logger after reading from env rather
than hard coding.
More specifically, in hbase-daemon.sh:
using export HBASE_ROOT_LOGGER=${HBASE_ROOT_LOGGER:-"INFO,RFA"}, in place of
export HBASE_ROOT_LOGGER="INFO,DRFA"
I have done the changes and tested on my local. Please share your thoughts.
> Cap space usage of default log4j rolling policy
> ---
>
> Key: HBASE-5655
> URL: https://issues.apache.org/jira/browse/HBASE-5655
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.92.1
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
> Fix For: 0.96.0
>
>
> The current default log4j policy is to use Daily Rolling File Appender
> (DRFA). At times, its good to have a cap on the maximum size of the logs in
> order to limit its disk usage. Here is a proposal to set a new file appemder
> (RFA) as the default appender. It can be configured via env so that existing
> tools can use the current behavior of using DRFA instead.
> This is in parallel with jira Hadoop-8149.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
