[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-19 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/14693 Merging this into master and 2.0 and 1.6 (hopefully), thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14693 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64027/ Test PASSed. ---

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14693 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14693 **[Test build #64027 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64027/consoleFull)** for PR 14693 at commit

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14693 **[Test build #64027 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64027/consoleFull)** for PR 14693 at commit

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/14693 I see, that makes sense. Updated the PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/14693 I'm sorry that did not pay enough attension on this part, thanks to the unit test! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/14693 @sitalkedia Your last commit is not correct, which fail this test. `upstream` is the one used for reading, `inMemIterator` is the one used for spilling, see #10142 --- If your project is set up

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/14693 Not sure why the test UnsafeExternalSorterSuite.forcedSpillingWithReadIterator has become flaky. I am not able to reproduce the issue on my laptop. @davies - any idea? --- If your project is

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14693 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64010/ Test FAILed. ---

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14693 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14693 **[Test build #64010 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64010/consoleFull)** for PR 14693 at commit

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14693 **[Test build #64010 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64010/consoleFull)** for PR 14693 at commit

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/14693 Weird, there are no test failure, still the job failed. May be Jenkins is unstable? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/14693 Jenkins test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14693 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14693 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64000/ Test FAILed. ---

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14693 **[Test build #64000 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64000/consoleFull)** for PR 14693 at commit

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14693 **[Test build #64000 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64000/consoleFull)** for PR 14693 at commit

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/14693 Jenkins test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/14693 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14693 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14693 **[Test build #63991 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63991/consoleFull)** for PR 14693 at commit

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14693 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63991/ Test FAILed. ---

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/14693 @rxin - I am not sure if there is an easy way to unit test this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14693 **[Test build #63991 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63991/consoleFull)** for PR 14693 at commit

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14693 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63964/ Test PASSed. ---

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14693 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14693 **[Test build #63964 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63964/consoleFull)** for PR 14693 at commit

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14693 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63960/ Test PASSed. ---

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14693 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14693 **[Test build #63960 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63960/consoleFull)** for PR 14693 at commit

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-17 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14693 Is there a way to add a unit test for this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-17 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/14693 @rxin - thanks updated the PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14693 **[Test build #63964 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63964/consoleFull)** for PR 14693 at commit

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-17 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14693 Can you update the pr description to say what the problem is? Also add some inline comment in the code. Thanks. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14693 **[Test build #63960 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63960/consoleFull)** for PR 14693 at commit

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-17 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/14693 cc - @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or