Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16639#discussion_r100409667
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -340,6 +342,14 @@ private[spark] class Executor(
}
}
}
+ task.context.fetchFailed.foreach { fetchFailure =>
+ // uh-oh. it appears the user code has caught the fetch-failure
without throwing any
+ // other exceptions. Its *possible* this is what the user meant
to do (though highly
+ // unlikely). So we will log an error and keep going.
+ logError(s"TID ${taskId} completed successfully though
internally it encountered " +
+ s"unrecoverable fetch failures! Most likely this means user
code is incorrectly " +
+ s"swallowing Spark's internal exceptions", fetchFailure)
--- End diff --
Can you be explicit about what exception is getting swallowed here? (i.e.,
"incorrectly swallowing Spark's internal FetchFailedException") -- to possibly
simplify debugging/fixing this issue for a user who runs into it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
