GitHub user bouk opened a pull request:
https://github.com/apache/incubator-spark/pull/644
Catch depickling errors
This surroungs the complete worker code in a try/except block so we catch
any error that arrives. An example would be the depickling failing for some
reason
@JoshRosen
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Shopify/spark catch-depickling-errors
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-spark/pull/644.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #644
----
commit 0e4d504941528451af570b796a37c9a4c579d2b0
Author: Bouke van der Bijl <[email protected]>
Date: 2014-02-24T20:52:44Z
Surround the complete python worker with the try block
This is for making sure depickling and any other nasty errors are
captured when they arrive before the function is actually being executed
commit f0f67cc7b1478de82f750fc068695caff4079eb1
Author: Bouke van der Bijl <[email protected]>
Date: 2014-02-24T21:01:31Z
Lol indentation
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
