[
https://issues.apache.org/jira/browse/FLUME-743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103535#comment-13103535
]
[email protected] commented on FLUME-743:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1825/
-----------------------------------------------------------
Review request for Flume.
Summary
-------
Fix to allow the flume log4j avro appender to include the log4j pattern layout.
Updated the Log4JEventAdaptor constructor to require a layout object. As any
class that uses the Log4JEventAdaptor extends the log4j AppenderSkeleton the
layout object is always present. When a user wishes to configure a layout they
use the setLayout method when setting up the log4j appender, if unset the
layout will be defaulted to just displaying the event message body.
Updated the getBody() method of the Log4JEventAdaptor class to apply the layout
to the log4j event and output stacktrace.
Updated FlumeLog4jAvroAppender to set up a default layout to display the
message body if no user defined layout is specified.
Added a new test to TestFlumeLog4jAvroAppender to test a logging pattern.
Note that when using text sink only raw output will display stack traces on
multiple lines, other formats display the newline escape characters.
This addresses bug FLUME-743.
https://issues.apache.org/jira/browse/FLUME-743
Diffs
-----
/trunk/flume-core/src/main/java/com/cloudera/flume/handlers/log4j/FlumeSeqfileAppender.java
1170098
/trunk/flume-core/src/main/java/com/cloudera/flume/handlers/log4j/Log4JAppenderEventSink.java
1170098
/trunk/flume-core/src/main/java/com/cloudera/flume/handlers/log4j/Log4JEventAdaptor.java
1170098
/trunk/flume-log4j-appender/src/main/java/com/cloudera/flume/log4j/appender/FlumeLog4jAvroAppender.java
1170098
/trunk/flume-log4j-appender/src/test/java/com/cloudera/flume/log4j/appender/TestFlumeLog4jAvroAppender.java
1170098
Diff: https://reviews.apache.org/r/1825/diff
Testing
-------
Set up two physical nodes, one as the source and one as the collector. Set up
the source with an avro logical node listening on port 12345 with the sink
being the collector. Set up a text sink on the collector to log to a file with
the output format set to raw: text("/tmp/test.log","raw").
Use the test app code below to generate log events:
import com.cloudera.flume.log4j.appender.FlumeLog4jAvroAppender;
import org.apache.log4j.Logger;
import org.apache.log4j.MDC;
import org.apache.log4j.PatternLayout;
import java.util.UUID;
public class Main {
static Logger log = Logger.getLogger(Main.class);
public static void main(String[] args) {
try {
FlumeLog4jAvroAppender appender = new FlumeLog4jAvroAppender();
appender.setHostname("localhost");
appender.setPort(12345);
appender.setLayout(new PatternLayout("%d [%c] (%t) <%X{user}
%X{field}> %m"));
appender.setReconnectAttempts(100);
appender.activateOptions();
log.addAppender(appender);
MDC.put("user", "chris");
while (true) {
MDC.put("field", UUID.randomUUID().toString());
log.info("Hello World");
try {
throw new Exception("Testing");
} catch (Exception e) {
log.error("Gone wrong ", e);
}
}
}
catch (Exception e) {
e.printStackTrace();
}
}
}
Thanks,
Chris
> Log4J Avro Appender Ignores Logging Pattern
> -------------------------------------------
>
> Key: FLUME-743
> URL: https://issues.apache.org/jira/browse/FLUME-743
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v0.9.4
> Environment: Linux Ubuntu/Redhat
> Reporter: Chris Jansen
> Assignee: Chris Jansen
> Priority: Minor
> Labels: log4j
>
> The log4J Avro appender does not use the pattern layout to format event
> messages. It only takes the message body, meaning that important information
> such as thread name, MDC/NDC values and stacktrace are lost.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
