You can use RollingFileAppenders in log4j.properties. http://logging.apache.org/log4j/extras/apidocs/org/apache/log4j/rolling/RollingFileAppender.html
You can have other scripts delete old logs. TD On Mon, Mar 24, 2014 at 12:20 AM, Sourav Chandra < sourav.chan...@livestream.com> wrote: > Hi, > > I have few questions regarding log file management in spark: > > 1. Currently I did not find any way to modify the lof file name for > executor/drivers). Its hardcoded as stdout and stderr. Also there is no log > rotation. > > In case of streaming application this will grow forever and become > unmanageable. Is there any way to overcome this? > > Thanks, > -- > > Sourav Chandra > > Senior Software Engineer > > · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · > > sourav.chan...@livestream.com > > o: +91 80 4121 8723 > > m: +91 988 699 3746 > > skype: sourav.chandra > > Livestream > > "Ajmera Summit", First Floor, #3/D, 68 Ward, 3rd Cross, 7th C Main, 3rd > Block, Koramangala Industrial Area, > > Bangalore 560034 > > www.livestream.com >
