Zhang, Liye created SPARK-6197:
----------------------------------
Summary: handle json parse exception for eventlog file not
finished writing
Key: SPARK-6197
URL: https://issues.apache.org/jira/browse/SPARK-6197
Project: Spark
Issue Type: Bug
Components: Spark Core, Web UI
Affects Versions: 1.4.0, 1.3.1
Reporter: Zhang, Liye
This is a following JIRA for
[SPARK-6107|https://issues.apache.org/jira/browse/SPARK-6107]. In
[SPARK-6107|https://issues.apache.org/jira/browse/SPARK-6107], webUI can
display event log files that with suffix *.inprogress*. However, the eventlog
file may be not finished writing for some abnormal cases (e.g. Ctrl+C), In
which case, the file maybe truncated in the last line, leading to the line
being not in valid Json format. Which will cause Json parse exception when
reading the file.
For this case, we can just ignore the last line content, since the history for
abnormal cases showed on web is only a reference for user, it can demonstrate
the past status of the app before terminated abnormally (we can not guarantee
the history can show exactly the last moment when app encounter the abnormal
situation).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
