[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-10-10 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/19181 Merging to master. Thanks! Can you create a backport for Spark-2.2? --- - To unsubscribe, e-mail:

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19181 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82594/ Test PASSed. ---

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19181 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19181 **[Test build #82594 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82594/testReport)** for PR 19181 at commit

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-10-10 Thread eyalfa
Github user eyalfa commented on the issue: https://github.com/apache/spark/pull/19181 @hvanhovell , thanks :+1: --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19181 **[Test build #82594 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82594/testReport)** for PR 19181 at commit

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-10-10 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/19181 I will merge this when it passes tests. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-10-10 Thread juliuszsompolski
Github user juliuszsompolski commented on the issue: https://github.com/apache/spark/pull/19181 Looks good to me. What do you think @hvanhovell ? --- - To unsubscribe, e-mail:

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19181 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82372/ Test PASSed. ---

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19181 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19181 **[Test build #82372 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82372/testReport)** for PR 19181 at commit

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19181 **[Test build #82372 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82372/testReport)** for PR 19181 at commit

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-26 Thread eyalfa
Github user eyalfa commented on the issue: https://github.com/apache/spark/pull/19181 @hvanhovell ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19181 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81998/ Test PASSed. ---

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19181 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19181 **[Test build #81998 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81998/testReport)** for PR 19181 at commit

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19181 **[Test build #81998 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81998/testReport)** for PR 19181 at commit

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-20 Thread eyalfa
Github user eyalfa commented on the issue: https://github.com/apache/spark/pull/19181 @hvanhovell , PTAL let me now if there's anything that requires fixing here. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-15 Thread eyalfa
Github user eyalfa commented on the issue: https://github.com/apache/spark/pull/19181 I agree, but 'it is what it is'😎 We can probably come up with some mechanism that detects such scenarios and avoids invoking the spill method on an object whose already 'on the stack',

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-15 Thread juliuszsompolski
Github user juliuszsompolski commented on the issue: https://github.com/apache/spark/pull/19181 @eyalfa Thanks. I agree that this is a good fix to this issue and lgtm. I'm just worried that there are more lurking cases where a nested spill can trigger and cause something

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19181 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19181 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81782/ Test PASSed. ---

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19181 **[Test build #81782 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81782/testReport)** for PR 19181 at commit

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-14 Thread eyalfa
Github user eyalfa commented on the issue: https://github.com/apache/spark/pull/19181 @juliuszsompolski, if you comment the few added lines in the reset() method, you'd see that the test fails with a stack frame very similar to the one you pasted in JIRA. ---

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-14 Thread juliuszsompolski
Github user juliuszsompolski commented on the issue: https://github.com/apache/spark/pull/19181 LGTM. It'd be nicer if there was an invariant that a nested spill just never can happen, i.e. there are no places that can spill on the code path that already spills. But this fix

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19181 **[Test build #81782 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81782/testReport)** for PR 19181 at commit

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-14 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/19181 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-14 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/19181 @eyalfa you should be able to retrigger tests yourself: `jenkins retest this please` --- - To unsubscribe, e-mail:

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-14 Thread eyalfa
Github user eyalfa commented on the issue: https://github.com/apache/spark/pull/19181 @hvanhovell, @juliuszsompolski, failure seem unrelated to my work, can you please request a retest? --- - To unsubscribe,

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19181 **[Test build #81765 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81765/testReport)** for PR 19181 at commit

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19181 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81765/ Test FAILed. ---

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19181 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19181 **[Test build #81765 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81765/testReport)** for PR 19181 at commit

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19181 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19181 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81734/ Test PASSed. ---

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19181 **[Test build #81734 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81734/testReport)** for PR 19181 at commit

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-13 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/19181 cc @juliuszsompolski can you take a look, or perhaps take this for a test drive? --- - To unsubscribe, e-mail:

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19181 **[Test build #81734 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81734/testReport)** for PR 19181 at commit

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19181 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19181 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81672/ Test PASSed. ---

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19181 **[Test build #81672 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81672/testReport)** for PR 19181 at commit

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19181 **[Test build #81672 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81672/testReport)** for PR 19181 at commit

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-12 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/19181 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-10 Thread eyalfa
Github user eyalfa commented on the issue: https://github.com/apache/spark/pull/19181 @ericl , @davies , you guys seem to be the last ones to edit this area of the code, I'd appreciate if you could take a look. ---

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19181 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional