> On Aug. 2, 2012, 10:03 a.m., Mike Percy wrote:
> > The INFO Hadoop setting is certainly needed. Also, including the thread
> > name is great. However I have heard that DailyRollingFileAppender is
> > unreliable and can cause data loss at roll time. What's the reason behind
> > that change?
>
> Denny Ye wrote:
> Thanks Mike, I'm familiar with DatedFileAppender in my project and
> DailyRollingFileAppender in Hadoop. I think that's good in troubleshooting
> for me and QA to track issue at specified date. Log name suffixed with
> 'yyyy-MM-dd' is better than 'flume-{number}.log'. Another tip in my
> experience is we have tools to clean up log file automatically with expire,
> it verify file suffix.
>
> Also, following your concern about data loss in DailyRollingFileAppender.
> I found a Hadoop bug https://issues.apache.org/jira/browse/HADOOP-8149, the
> same problem and they changed DailyRollingFileAppender back to
> RollingFileAppender. I don't agree with them completely. It's hard to
> distinguish log at specified date.
>
> May be the DatedFileAppender is another choose. Do you think so?
>
> Mike Percy wrote:
> I've never used DatedFileAppender personally. I believe this is the
> original site: http://minaret.biz/tips/datedFileAppender.html and someone
> apparently made some modifications to it:
> http://sourceforge.net/p/log4j-dfa/home/Home/
>
> Do you know of anyone using it in production? If not, it seems a bit
> hasty to just make it the default for all users of Flume...
>
> Denny Ye wrote:
> We used DatedFileAppender in production two years. It's stable always.
> Configuration example in log4j.xml in my project:
> <param name="Threshold" value="DEBUG" />
> <param name="Directory" value="${catalina.base}/logs" />
> <param name="Prefix" value="xxxxx." />
> <param name="Suffix" value=".log" />
> <param name="Append" value="true" />
> <layout class="org.apache.log4j.PatternLayout">
> <param name="ConversionPattern" value="%d{dd MMM yyyy HH:mm:ss,SSS}
> [%t] %-5p (%C.%M:%L) %x - %m%n" />
> </layout>
>
> Log file will be : xxxxx.2012-08-03.log
>
> Denny Ye wrote:
> hi Mike, what's your suggestion or response?
Hi Denny, apologies for the delay as I was away for the weekend. Regarding the
format, I think "%-5p" should before the thread name since the thread names
vary in length. But that is just a nitpick. Otherwise the changes look good to
me.
If you are comfortable with DatedFileAppender and believe it is reliable then I
am OK with it, as long as we can pull it in via Maven. If we have to check
binaries into Git, or jump through hoops to build, I think we should defer
changes to the rolling mechanism, and go ahead with the rest of the changes in
your patch.
- Mike
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6316/#review9745
-----------------------------------------------------------
On Aug. 3, 2012, 10:18 a.m., Denny Ye wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/6316/
> -----------------------------------------------------------
>
> (Updated Aug. 3, 2012, 10:18 a.m.)
>
>
> Review request for Flume and Hari Shreedharan.
>
>
> Description
> -------
>
> Updated log4j.properties
> 1. Add 'logs' folder. It's better to aggregate logs into unified folder.
> 2. Log file using DailyRollingFileAppender
> 3. More useful log pattern. Add thread and code line information
> 4. Set debug level for Hadoop. If we use debug level, Hadoop log takes too
> many lines.
>
>
> This addresses bug FLUME-1418.
> https://issues.apache.org/jira/browse/FLUME-1418
>
>
> Diffs
> -----
>
> trunk/conf/log4j.properties 1363210
>
> Diff: https://reviews.apache.org/r/6316/diff/
>
>
> Testing
> -------
>
> That's OK in my environment
>
>
> Thanks,
>
> Denny Ye
>
>
