[
https://issues.apache.org/jira/browse/SPARK-20547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shixiong Zhu updated SPARK-20547:
---------------------------------
Description:
ExecutorClassLoader's findClass may throw some transient exception. For
example, when a task is cancelled, if ExecutorClassLoader is running, you may
see InterruptedException or IOException, even if this class can be loaded. Then
the result of findClass will be cached by JVM, and later when the same class is
being loaded (note: in this case, this class may be still loadable), it will
just throw NoClassDefFoundError.
We should make ExecutorClassLoader retry on transient exceptions.
was:
ExecutorClassLoader's findClass may throw some transient exception. For
example, when a task is cancelled, if ExecutorClassLoader is running, you may
see InterruptedException or IOException, even if this class can be loaded. Then
the result of findClass will be cached by JVM, and later when the same class is
being loaded (note: in this case, this class may be still loadable), it will
just throw NoClassDefFoundError.
We should make ExecutorClassLoader's retry on transient exceptions.
> ExecutorClassLoader's findClass may not work correctly when a task is
> cancelled.
> --------------------------------------------------------------------------------
>
> Key: SPARK-20547
> URL: https://issues.apache.org/jira/browse/SPARK-20547
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.1.0
> Reporter: Shixiong Zhu
> Priority: Blocker
>
> ExecutorClassLoader's findClass may throw some transient exception. For
> example, when a task is cancelled, if ExecutorClassLoader is running, you may
> see InterruptedException or IOException, even if this class can be loaded.
> Then the result of findClass will be cached by JVM, and later when the same
> class is being loaded (note: in this case, this class may be still loadable),
> it will just throw NoClassDefFoundError.
> We should make ExecutorClassLoader retry on transient exceptions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
