W. P., How are you running your Reducer? Is everything running in standalone mode (all mappers/reducers in the same process as the launching application)? Or are you running this in pseudo-distributed mode or on a remote cluster?
Depending on the application's configuration, log4j configuration could be read from one of many different places. Furthermore, where are you expecting your output? If you're running in pseudo-distributed (or fully distributed) mode, mapper / reducer tasks will not emit output back to the console of the launching application. That only happens in local mode. In the distributed flavors, you'll see a different file for each task attempt containing its log output, on the machine where the task executed. These files can be accessed through the web UI at http://jobtracker:50030/ -- click on the job, then the task, then the task attempt, then "syslog" in the right-most column. - Aaron On Mon, Dec 13, 2010 at 10:05 AM, W.P. McNeill <bill...@gmail.com> wrote: > I would like to use Hadoop's Log4j infrastructure to do logging from my > map/reduce application. I think I've got everything set up correctly, but > I > am still unable to specify the logging level I want. > > By default Hadoop is set up to log at level INFO. The first line of its > log4j.properties file looks like this: > > hadoop.root.logger=INFO,console > > > I have an application whose reducer looks like this: > > package com.me; > > public class MyReducer<...> extends Reducer<...> { > private static Logger logger = > Logger.getLogger(MyReducer.class.getName()); > > ... > protected void reduce(...) { > logger.debug("My message"); > ... > } > } > > > I've added the following line to the Hadoop log4j.properties file: > > log4j.logger.com.me.MyReducer=DEBUG > > > I expect the Hadoop system to log at level INFO, but my application to log > at level DEBUG, so that I see "My message" in the logs for the reducer > task. > However, my application does not produce any log4j output. If I change > the > line in my reducer to read logger.info("My message") the message does get > logged, so somehow I'm failing to specify that log level for this class. > > I've also tried changing the log4j line for my app to > read log4j.logger.com.me.MyReducer=DEBUG,console and get the same result. > > I've been through the Hadoop and log4j documentation and I can't figure out > what I'm doing wrong. Any suggestions? > > Thanks. >
