Hi,
In Spark 1.3.0 I've enabled event logging to write to an existing HDFS
folder on a Standalone cluster. This is generally working, all the logs are
being written. However, from the Master Web UI, the vast majority of
completed applications are labeled as not having a history:
http://xxx.xxx.xxx.xxx:8080/history/not-found/?msg=Application+App+is+still+in+progress.&title=Application%20history%20not%20found%20(app-20150601160846-1914)
The log does exists though:
# hdfs dfs -ls -R /eventLogs/app-20150601160846-1914
-rw-rw---- 3 user group 1027848 2015-06-01 16:09
/eventLogs/app-20150601160846-1914
and `cat` the file ends with:
{"Event":"SparkListenerApplicationEnd","Timestamp":1433174955077}
This seems to indicate it saw and logged the application end.
Is there a known issue here or a workaround? Looking at the source code I
might have expected these files to end in `.inprogress` given the UI error
message, but they don't.
Thanks,
Richard
