Thanks for your reply! Sorry for i forgot referring the spark which i'm using is *Version 1.0.2* instead of 1.1.0.
Also the document of 1.0.2 seems not same like 1.1.0: http://spark.apache.org/docs/1.0.2/running-on-yarn.html And i tried your suggestion(upload ) but did not work: *1. set my copy of log4j.properties like:* *log4j.rootCategory=DEBUG, console* *2. upload when using spark-submit script:* *./bin/spark-submit --class edu.bjut.spark.SparkPageRank --master yarn-cluster --num-executors 5 --executor-memory 2g --executor-cores 1 /data/hadoopspark/MySparkTest.jar hdfs://master:8000/srcdata/searchengine/* 5 5 hdfs://master:8000/resultdata/searchengine/2014102001/ * *--files log4j.properties* So plz point out my fault and any suggestion would be welcome Thanks! 2014-10-16 9:45 GMT+08:00 Marcelo Vanzin <van...@cloudera.com>: > Hi Eric, > > Check the "Debugging Your Application" section at: > http://spark.apache.org/docs/latest/running-on-yarn.html > > Long story short: upload your log4j.properties using the "--files" > argument of spark-submit. > > (Mental note: we could make the log level configurable via a system > property...) > > > On Wed, Oct 15, 2014 at 5:58 PM, eric wong <win19...@gmail.com> wrote: > > Hi, > > > > I want to check the DEBUG log of spark executor on YARN(using > yarn-cluster > > mode), but > > > > 1. yarn daemonlog setlevel DEBUG YarnChild.class > > 2. set log4j.properties in spark/conf folder on client node. > > > > no means above works. > > > > So how could i set the log level of spark executor on YARN container to > > DEBUG？ > > > > Thanks! > > > > > > > > > > -- > > Wang Haihua > > > > > > -- > Marcelo > -- 王海华
