Thanks Imesh, Isuru, Amila and Martin. As Martin mentioned, we had log enabled at DEBUG level. This change will additional level of protection besides having good logging instrumentation in the code. With the DailyRollingFileAppender, we can only hope that log file is rotated every 24 hr before the disk is filled up.
I've committed this change to both master and 4.0.0. Please feel free to revert or make adjustment to the size/number of files if you think necessary. Regards, -Jeffrey From: Imesh Gunaratne <im...@apache.org<mailto:im...@apache.org>> Reply-To: "dev@stratos.apache.org<mailto:dev@stratos.apache.org>" <dev@stratos.apache.org<mailto:dev@stratos.apache.org>> Date: Sunday, June 29, 2014 1:45 PM To: dev <dev@stratos.apache.org<mailto:dev@stratos.apache.org>> Subject: Re: Carbon log flooded causing disk full and reboot Hi Jeffrey, A great finding! Did this occur while running Stratos in single JVM mode without having any DEBUG level logs? Please send these changes to both master and 4.0.0 branches. Thanks On Fri, Jun 27, 2014 at 7:52 PM, Jeffrey Nguyen (jeffrngu) <jeffr...@cisco.com<mailto:jeffr...@cisco.com>> wrote: Hi, In our Stratos deployment, we experienced issue where wso2carbon.log file grew as much as 13 GB causing disk full and instance reboot and Stratos inaccessible. We're deploying Stratos using default profile. To fix this issue, we modified log4j.properties with the changes listed below. Please let me know if it's okay to push this changes upstream. If so, which branch(es) should the changes be checked in? Thanks, -Jeffrey diff --git a/products/stratos/modules/distribution/src/main/conf/log4j.properties b/products/stratos/modules/distribution/src index a85d346..bb37679 100644 --- a/products/stratos/modules/distribution/src/main/conf/log4j.properties +++ b/products/stratos/modules/distribution/src/main/conf/log4j.properties @@ -113,7 +113,9 @@ log4j.appender.CARBON_MEMORY.threshold=DEBUG # CARBON_LOGFILE is set to be a DailyRollingFileAppender using a PatternLayout. -log4j.appender.CARBON_LOGFILE=org.apache.log4j.DailyRollingFileAppender +log4j.appender.CARBON_LOGFILE=org.apache.log4j.RollingFileAppender +log4j.appender.CARBON_LOGFILE.MaxFileSize=10000KB +log4j.appender.CARBON_LOGFILE.MaxBackupIndex=10 # Log file will be overridden by the configuration setting in the DB # This path should be relative to WSO2 Carbon Home -- Imesh Gunaratne Technical Lead, WSO2 Committer & PPMC Member, Apache Stratos
