Hi,
thanks, worked for me, (using
http://stackoverflow.com/questions/6072389/how-to-create-a-own-appender-in-log4j):
public class MyFlumeAppender extends AppenderSkeleton {
MyRpcClientFacade client = new MyRpcClientFacade();
public MyFlumeAppender() {
super();
String flumeServer = "localhost";
Configuration conf = new Configuration();
conf.addResource("app-config.xml");
flumeServer = conf.get("flume.server");
client.init(flumeServer, 41415);
}
@Override
public void close() {
client.cleanUp();
}
@Override
public boolean requiresLayout() {
// TODO Auto-generated method stub
return false;
}
@Override
protected void append(LoggingEvent event) {
//String logger = event.getLoggerName();
//String level = event.getLevel().toString();
//Long ts = event.getTimeStamp();
//client.sendDataToFlume(ts.toString()+" - "+level+" -
"+event.getMessage().toString());
PatternLayout myLayout = new PatternLayout("%d{ISO8601} %-5p %c - %m");
client.sendDataToFlume(myLayout.format(event));
}
}
However, when I try to use the PatternLayout for formatting the output,
logging falls back to the container logs. If I build a string on my own
(uncommenting the first four lines in append) it's working. I know this
might be a bit off topic now, but maybe someone has an idea?
Cheers
Seb.
Am 19.05.2014 11:17, schrieb Harsh J:
Task JVMs in MR run with their own TaskLogAppender-configured
task-log4j.properties file. This is not overridable or configurable
currently.
Your best approach at the moment would be to configure your custom
appender in code at the beginning of your program.
On Sun, May 18, 2014 at 9:27 PM, Sebastian Gäde <[email protected]> wrote:
Hi,
I'd like to log events within my MR application using Flume writing to a
central log file. I've followed
http://flume.apache.org/releases/content/1.4.0/FlumeUserGuide.html#log4j-appender
putting the stuff into Hadoop's log4j.properties and copying it to Hadoop's
conf directory on all nodes.
However, my in-application events are only logged to stdout (I can see them
using the YARN webapp) but not to Flume. Although I thought Hadoop's conf
directory is added to classpath, it does not seem to work for me.
Could you please tell me where I should put the log4j.properties?
Cheers
Seb.
