Event log is enabled in my spark streaming app. My code runs in standalone mode
and the spark version is 1.3.1. I periodically stop and restart the streaming
context by calling ssc.stop(). However, from the web UI, when clicking on a
past job, it says the job is still in progress and does not show the event log.
The event log files have suffix .inprogress. Removing the suffix does not solve
the problem. Do I need to do anything here in order to view the event logs of
finished jobs? Or do I need to stop ssc differently?
In addition, the documentation seems to suggest history server is used for
Mesos or YARN mode. Does it mean I don't need to start history server if I only
use spark in standalone mode?
Thanks,Du
