Hi, I figure out that in standalone mode these configuration should add to worker process's configs, like adding the following line in spark-env.sh:
SPARK_WORKER_OPTS="-Dspark.executor.logs.rolling.strategy=time -Dspark.executor.logs.rolling.time.interval=daily -Dspark.executor.logs.rolling.maxRetainedFiles=3" Maybe in yarn mode the spark-defaults.conf would be sufficient, but I've not tested. On Tue, Nov 4, 2014 at 12:24 PM, Ji ZHANG <zhangj...@gmail.com> wrote: > Hi, > > I'm using Spark Streaming 1.1, and I have the following logs keep growing: > > /opt/spark-1.1.0-bin-cdh4/work/app-20141029175309-0005/2/stderr > > I think it is executor log, so I setup the following options in > spark-defaults.conf: > > spark.executor.logs.rolling.strategy time > spark.executor.logs.rolling.time.interval daily > spark.executor.logs.rolling.maxRetainedFiles 10 > > I can see these options on Web UI, so I suppose they are effective. > However, the stderr is still not rotated. > > Am I doing wrong? > > Thanks. > > -- > Jerry -- Jerry --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
