Thank you very much. But that does not help. I did try to symbolic link one into my working directory "-files conf/log4j.proeprties#mylog4j.properties", and then put a specified configuration in jvm options
<property> <name>mapred.child.java.opts</name> <value> -Dlog4j.configuration=mylog4j.properties</value> </property> This messed up the task log output in some degree, the "syslog" section completely has completely gone. Nevertheless, it does not work. It seems hadoop engine did some special log setting. What should I do? Thanks. Zhu, Guojun Modeling Sr Graduate 571-3824370 guojun_...@freddiemac.com Financial Engineering Freddie Mac Joey Echeverria <j...@cloudera.com> 02/29/2012 10:45 PM Please respond to mapreduce-user@hadoop.apache.org To "mapreduce-user@hadoop.apache.org" <mapreduce-user@hadoop.apache.org> cc "mapreduce-user@hadoop.apache.org" <mapreduce-user@hadoop.apache.org> Subject Re: no log function for map/red in a cluster setup Try adding the log4j.properties file to he distributed cache, e.g.: hadoop jar job.jar -config conf -files conf/log4j.properties my.package.Class arg1 -Joey On Feb 29, 2012, at 16:15, GUOJUN Zhu <guojun_...@freddiemac.com> wrote: What I found out is that the default conf/log4j.properties set root with INFO and indeed anything beyond INFO (hadoop's or my own codes') show up. However, I tried to put a new log4j.properties with lower threshold in the new conf directory and specify it with "--configure" option and it did not work (it did pick up other things such as mapreq-site.xml). Unfortunately, I am not the administrator and do not have the priviledge to modify the default log4j.properties. Do I have to ask the administrator to do it for me? Thanks. Zhu, Guojun Modeling Sr Graduate 571-3824370 guojun_...@freddiemac.com Financial Engineering Freddie Mac GUOJUN Zhu <guojun_...@freddiemac.com> 02/27/2012 11:34 AM Please respond to mapreduce-user@hadoop.apache.org To "mapreduce-user@hadoop.apache.org" <mapreduce-user@hadoop.apache.org> cc Subject no log function for map/red in a cluster setup Hi. We are testing hadoop. We are using hadoop (0.20.2-cdh3u3). I am using the cotomized conf directory with -"-config mypath". I modified the log4j.properties file in this path, adding " log4j.logger.com.mycompany=DEBUG". It works fine with our pseudo-one-node-cluster setup (1.00). But in the new cluster (with 32 data nodes/name node/secondary namenode/jobtracker/backup jobtracker), I can only see the log from hadoop (in the web interface, when I navigate all the way into the task node log), but no logs from my mapper/reducer (com.mycompany.***) show up. I can do System.out.println or System.err.println and see them in the same log file, but no logs from log4j show up. Is there any other configuration I missed? Thanks. Zhu, Guojun Modeling Sr Graduate 571-3824370 guojun_...@freddiemac.com Financial Engineering Freddie Mac