[
https://issues.apache.org/jira/browse/SPARK-11801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15034827#comment-15034827
]
Marcelo Vanzin commented on SPARK-11801:
----------------------------------------
bq. But like I said earlier, I'm OK with generic executor lost msgs, the
problem is that there are plenty of times when the other confusing error msgs
do get back to the driver, which is really confusing.
I think to a certain extent those are different issues. The changes to
re-prioritize shutdown hooks in Srinivasa's PR should take care of avoiding
misleading messages (also, SPARK-11799), regardless of how OOM is handled,
right?
bq. The argument I see in favor of OnOutOfMemoryError is that it seems more
reliable.
That is true. So maybe leave both approaches, "just in case", and update
YarnAllocator to handle {{SparkExitCode.OOM}}?
> Notify driver when OOM is thrown before executor JVM is killed
> ---------------------------------------------------------------
>
> Key: SPARK-11801
> URL: https://issues.apache.org/jira/browse/SPARK-11801
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 1.5.1
> Reporter: Srinivasa Reddy Vundela
> Priority: Minor
>
> Here is some background for the issue.
> Customer got OOM exception in one of the task and executor got killed with
> kill %p. It is unclear in driver logs/Spark UI why the task is lost or
> executor is lost. Customer has to look into the executor logs to see OOM is
> the cause for the task/executor lost.
> It would be helpful if driver logs/spark UI shows the reason for task
> failures by making sure that task updates the driver with OOM.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
