Hi, With the current design, eventlogs are not ideal for long running streaming applications. So, it is better then to disable the eventlogs. There was a proposal for splitting the eventlogs based on size/Job/query for long running applications, not sure about the followup for that.
Regards, Shahid On Tue, 16 Jul 2019, 3:38 pm raman gugnani, <ramangugnani....@gmail.com> wrote: > HI , > > I have long running spark streaming jobs. > Event log directories are getting filled with .inprogress files. > Is there fix or work around for spark streaming. > > There is also one jira raised for the same by one reporter. > > https://issues.apache.org/jira/browse/SPARK-22783 > > -- > Raman Gugnani > > 8588892293 > Principal Engineer > *ixigo.com <http://ixigo.com>* >
