Re: Hadoop+log4j

2008-11-24 Thread Brian Bockelman
On Nov 24, 2008, at 9:49 AM, Steve Loughran wrote: Scott Whitecross wrote: Thanks Brian. So you have had luck w/ log4j? We grab logs off machines by not using lo4j and routing to our own logging infrastructure that can feed events to other boxes via RMI and queues. This stuff slots in

Re: Hadoop+log4j

2008-11-24 Thread Steve Loughran
Scott Whitecross wrote: Thanks Brian. So you have had luck w/ log4j? We grab logs off machines by not using lo4j and routing to our own logging infrastructure that can feed events to other boxes via RMI and queues. This stuff slots in behind commons-logging, with a custom commons-logging

Re: Hadoop+log4j

2008-11-23 Thread Scott Whitecross
I've looked around for a while, but it seems there isn't a way to log from Hadoop, without going through the logs/userlogs/ and the 'attempt' directories? That would mean that for logging I'm restricted to writing to System.out and System.err, then collecting via scripts? Thanks.

Re: Hadoop+log4j

2008-11-23 Thread Scott Whitecross
Thanks Brian. I've played w/ the log4j.properties a bit, and haven't had any luck. Can you share how youve setup log4j? I am probably missing the obvious, but here is what I setup: log4j.logger.com.mycompany.hadoop=DEBUG,DX,console

Re: Hadoop+log4j

2008-11-23 Thread Scott Whitecross
Thanks Brian. So you have had luck w/ log4j? I haven't tried local mode. I will try it tonight and see how it goes for quick debugging. More so, I wanted to be able to easily log and watch events on a cluster, rather then digging through all the hadoop logging levels. I've also read

Hadoop+log4j

2008-11-11 Thread ZhiHong Fu
Hello, I'm very sorry to trouble you, I'm developing a MapReduce Application, And I can get Log.INFO in InputFormat ,but In Mapper or Reducer , I can't get anything . And Now an error occured in the reduce stage. Because the code is a little complicated, I can't find where is the mistake

Re: Hadoop+log4j

2008-11-11 Thread Alex Loddengaard
Have you seen this: http://wiki.apache.org/hadoop/HowToDebugMapReducePrograms Alex On Tue, Nov 11, 2008 at 6:03 PM, ZhiHong Fu [EMAIL PROTECTED] wrote: Hello, I'm very sorry to trouble you, I'm developing a MapReduce Application, And I can get Log.INFO in InputFormat ,but In Mapper