[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12433 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-212034603 Merging in master. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user sameeragarwal commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-212033047 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/12433#discussion_r60180864
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -802,7 +807,12 @@ private[spark] class BlockManager(
logDebug("Put block %s locally took %s".format(blockId,
Utils.getUsedTimeMs(startTimeMs)))
if (level.replication > 1) {
// Wait for asynchronous replication to finish
-Await.ready(replicationFuture, Duration.Inf)
+try {
+ Await.ready(replicationFuture, Duration.Inf)
--- End diff --
@ScrapCodes, towards your other comment, I think that timeouts in this case
might already happen to be covered by network / RPC timeouts within the
`replicationFuture`'s code.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/12433#discussion_r60180666
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -260,7 +260,12 @@ private[spark] class BlockManager(
def waitForAsyncReregister(): Unit = {
val task = asyncReregisterTask
if (task != null) {
- Await.ready(task, Duration.Inf)
+ try {
+Await.ready(task, Duration.Inf)
--- End diff --
According to the Scaladoc (and actual usages), it looks like this
particular `waitForAsyncReregister` method is only used in test code and I'm
guessing that it's probably called from within an interrupt-based timeout block.
As for the other usages, we'd have to consider them on a case-by-case basis.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user ScrapCodes commented on a diff in the pull request:
https://github.com/apache/spark/pull/12433#discussion_r60173470
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -260,7 +260,12 @@ private[spark] class BlockManager(
def waitForAsyncReregister(): Unit = {
val task = asyncReregisterTask
if (task != null) {
- Await.ready(task, Duration.Inf)
+ try {
+Await.ready(task, Duration.Inf)
--- End diff --
Unrelated to this PR, But waiting for infinite time has a downside, that if
this (main)thread blocks then the app running will appear to have hanged with
no way to know unless one checks the thread dump somehow. However if it is for
finite time duration, an exception is thrown on timeout. In the case
`Duration.Inf` there is no exception ever thrown.
If I am correct about the above, I am not sure why it is being used widely
? I am just asking so I understand if there is some side to it that I do not
understand.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-211679436 cc @sameeragarwal for review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-211659107 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56154/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-211659104 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-211658925 **[Test build #56154 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56154/consoleFull)** for PR 12433 at commit [`d25cd4c`](https://github.com/apache/spark/commit/d25cd4cd54fff28e1d9f9c7b6475543afed2fc25). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-211633603 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56145/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-211633597 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-211633467 **[Test build #56145 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56145/consoleFull)** for PR 12433 at commit [`4760d3c`](https://github.com/apache/spark/commit/4760d3ce20408b6afdaf24a018ec409447520e60). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-211620068 **[Test build #56154 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56154/consoleFull)** for PR 12433 at commit [`d25cd4c`](https://github.com/apache/spark/commit/d25cd4cd54fff28e1d9f9c7b6475543afed2fc25). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-211598606 **[Test build #56145 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56145/consoleFull)** for PR 12433 at commit [`4760d3c`](https://github.com/apache/spark/commit/4760d3ce20408b6afdaf24a018ec409447520e60). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-211597667 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-211597673 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56144/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-211597659 **[Test build #56144 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56144/consoleFull)** for PR 12433 at commit [`0f06488`](https://github.com/apache/spark/commit/0f06488892f09c01fab8eab59e20f9a29ff8d86d). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-211597299 **[Test build #56144 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56144/consoleFull)** for PR 12433 at commit [`0f06488`](https://github.com/apache/spark/commit/0f06488892f09c01fab8eab59e20f9a29ff8d86d). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-211590743 **[Test build #56139 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56139/consoleFull)** for PR 12433 at commit [`c0bfc5e`](https://github.com/apache/spark/commit/c0bfc5ee022f76ed259c40cd13f9e776c31be45c). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-211590753 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-211590758 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56139/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-211589994 **[Test build #56139 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56139/consoleFull)** for PR 12433 at commit [`c0bfc5e`](https://github.com/apache/spark/commit/c0bfc5ee022f76ed259c40cd13f9e776c31be45c). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-211589074 Alright, I've updated this to add a `ThreadUtils.awaitResult` and added a linter rule to ban direct use of `Await.result`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user thunterdb commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-211493984 +1, because we can then add a linter rule that disallows `Await.result` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-210876468 Maybe instead of doing this, we should create a wrapper function that wraps around await.result? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12433#discussion_r59972246
--- Diff:
core/src/main/scala/org/apache/spark/network/BlockTransferService.scala ---
@@ -101,7 +102,12 @@ abstract class BlockTransferService extends
ShuffleClient with Closeable with Lo
}
})
-Await.result(result.future, Duration.Inf)
+try {
+ Await.result(result.future, Duration.Inf)
+} catch {
+ case NonFatal(t) =>
+throw new Exception("Exception occurred while fetching block", t)
--- End diff --
RuntimeException? or SparkException.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-210725264 Hmm, this is unfortunate. It looks like there are a bunch more assertions like the one in the `FutureAction` test suite which match on the class / name of the exception thrown from methods which call `Await.result`. I wonder whether this change of wrapping the exception is going to cause regressions in user code which tries to do the same thing. Should I try to use reflection to make a best-effort attempt to re-throw using the same exception? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-210721724 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/55985/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-210721722 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-210721684 **[Test build #55985 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55985/consoleFull)** for PR 12433 at commit [`975bbac`](https://github.com/apache/spark/commit/975bbacf06e1d089dd16ef2140dbb71c141a56cd). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-210701963 **[Test build #55985 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55985/consoleFull)** for PR 12433 at commit [`975bbac`](https://github.com/apache/spark/commit/975bbacf06e1d089dd16ef2140dbb71c141a56cd). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-210701761 I've decided to just leave the occurrence in `FutureAction` untouched for now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-210700398 Hmm, those are legitimate test failures. The problem was that wrapping something in `Exception` changed the exception matched by some test asserts. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-210699178 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-210699179 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/55972/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-210699113 **[Test build #55972 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55972/consoleFull)** for PR 12433 at commit [`23e7467`](https://github.com/apache/spark/commit/23e7467de7ba39b16dc92003c842ac1156f8). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12433#issuecomment-210676479 **[Test build #55972 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55972/consoleFull)** for PR 12433 at commit [`23e7467`](https://github.com/apache/spark/commit/23e7467de7ba39b16dc92003c842ac1156f8). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14676] Wrap and re-throw Await.result e...
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/12433 [SPARK-14676] Wrap and re-throw Await.result exceptions in order to capture full stacktrace When `Await.result` throws an exception which originated from a different thread, the resulting stacktrace doesn't include the path leading to the `Await.result` call itself, making it difficult to identify the impact of these exceptions. For example, I've seen cases where broadcast cleaning errors propagate to the main thread and crash it but the resulting stacktrace doesn't include any of the main thread's code, making it difficult to pinpoint which exception crashed that thread. This patch addresses this issue by explicitly catching, wrapping, and re-throwing exceptions that are thrown by `Await.result`. I tested this manually using https://github.com/JoshRosen/spark/commit/16b31c825197ee31a50214c6ba3c1df08148f403, a patch which reproduces an issue where an RPC exception which occurs while unpersisting RDDs manages to crash the main thread without any useful stacktrace, and verified that informative, full stacktraces were generated. /cc @rxin @nongli @yhuai @anabranch You can merge this pull request into a Git repository by running: $ git pull https://github.com/JoshRosen/spark wrap-and-rethrow-await-exceptions Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/12433.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #12433 commit 23e7467de7ba39b16dc92003c842ac1156f8 Author: Josh Rosen Date: 2016-04-15T22:48:06Z Wrap exceptions thrown by Await.result(). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
