If the mentioned conf is enabled, the rolling of the stderr should work. If it is not, then there is probably some bug. Take a look at the Worker's logs and see if there is any error about rolling of the Executor's stderr. If there is a bug, then it needs to be fixed (maybe you can take a crack at it ;) ). Alternatively, you could roll using log4j by specifying your appender (not sure off the top of my head how to set it such that it works on workers), but it will break viewing of logs on the Spark web UI as the web UI will not know where to find the logs from.
TD On Thu, Feb 26, 2015 at 2:14 AM, Sean Owen <so...@cloudera.com> wrote: > I think that's up to you. You can make it log wherever you want, and > have some control over how log4j names the rolled log files by > configuring its file-based rolling appender. > > On Thu, Feb 26, 2015 at 10:05 AM, Jeffrey Jedele > <jeffrey.jed...@gmail.com> wrote: > > So the summarize (I had a similar question): > > Spark's log4j per default is configured to log to the console? Those > > messages end up in the stderr files and the approach does not support > > rolling? > > > > If I configure log4j to log to files, how can I keep the folder > structure? > > Should I use relative paths and assume that those end up in the same > folders > > the stderr files do? > > > > Regards, > > Jeff > > > > 2015-02-25 9:35 GMT+01:00 Sean Owen <so...@cloudera.com>: > >> > >> These settings don't control what happens to stderr, right? stderr is > >> up to the process that invoked the driver to control. You may wish to > >> configure log4j to log to files instead. > >> > >> On Wed, Nov 12, 2014 at 8:15 PM, Nguyen, Duc <duc.ngu...@pearson.com> > >> wrote: > >> > I've also tried setting the aforementioned properties using > >> > System.setProperty() as well as on the command line while submitting > the > >> > job > >> > using --conf key=value. All to no success. When I go to the Spark UI > and > >> > click on that particular streaming job and then the "Environment" > tab, I > >> > can > >> > see the properties are correctly set. But regardless of what I've > tried, > >> > the > >> > "stderr" log file on the worker nodes does not roll and continues to > >> > grow...leading to a crash of the cluster once it claims 100% of disk. > >> > Has > >> > anyone else encountered this? Anyone? > >> > > >> > > >> > > >> > On Fri, Nov 7, 2014 at 3:35 PM, Nguyen, Duc <duc.ngu...@pearson.com> > >> > wrote: > >> >> > >> >> We are running spark streaming jobs (version 1.1.0). After a > sufficient > >> >> amount of time, the stderr file grows until the disk is full at 100% > >> >> and > >> >> crashes the cluster. I've read this > >> >> > >> >> https://github.com/apache/spark/pull/895 > >> >> > >> >> and also read this > >> >> > >> >> > http://spark.apache.org/docs/latest/configuration.html#spark-streaming > >> >> > >> >> > >> >> So I've tried testing with this in an attempt to get the stderr log > >> >> file > >> >> to roll. > >> >> > >> >> sparkConf.set("spark.executor.logs.rolling.strategy", "size") > >> >> .set("spark.executor.logs.rolling.size.maxBytes", "1024") > >> >> .set("spark.executor.logs.rolling.maxRetainedFiles", "3") > >> >> > >> >> > >> >> Yet it does not roll and continues to grow. Am I missing something > >> >> obvious? > >> >> > >> >> > >> >> thanks, > >> >> Duc > >> >> > >> > > >> > > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > >> For additional commands, e-mail: user-h...@spark.apache.org > >> > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
