[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/23084 @cloud-fan,thanks --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23084 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23084 I think it's safer to only catch the spark-thrown OOM, not the system OOM, so LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/23084 so this PR is just code cleanup, not a real bug. thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/23084 I think `growAndRehash` always thrown `SparkOutOfMemoryError`. catch `SparkOutOfMemoryError` or `OutOfMemoryError` will make `canGrowArray = false`. please correct if i understand something wrong. thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23084 > for example, HashAggregateExec will carry out spill the map and fallback to sort-based Do you mean this patch change nothing for this case? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23084 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99046/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23084 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23084 **[Test build #99046 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99046/testReport)** for PR 23084 at commit [`400bcd5`](https://github.com/apache/spark/commit/400bcd5e6672281f352b36ef64219ec50bb2c2f9). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23084 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5174/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23084 **[Test build #99046 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99046/testReport)** for PR 23084 at commit [`400bcd5`](https://github.com/apache/spark/commit/400bcd5e6672281f352b36ef64219ec50bb2c2f9). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23084 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/23084 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23084 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99045/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23084 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23084 **[Test build #99045 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99045/testReport)** for PR 23084 at commit [`400bcd5`](https://github.com/apache/spark/commit/400bcd5e6672281f352b36ef64219ec50bb2c2f9). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/23084 @cloud-fan, I think not kill executor. the memory allocate fails, and the caller is notified to perform other operations. for example, HashAggregateExec will carry out spill the map and fallback to sort-based. please correct if i understand something wrong. thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23084 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5173/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23084 **[Test build #99045 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99045/testReport)** for PR 23084 at commit [`400bcd5`](https://github.com/apache/spark/commit/400bcd5e6672281f352b36ef64219ec50bb2c2f9). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23084 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23084 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99002/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23084 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23084 **[Test build #99002 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99002/testReport)** for PR 23084 at commit [`7345942`](https://github.com/apache/spark/commit/73459423d0c3a5070a143865bcf6557d5b73f423). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23084 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23084 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99006/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23084 **[Test build #99006 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99006/testReport)** for PR 23084 at commit [`7345942`](https://github.com/apache/spark/commit/73459423d0c3a5070a143865bcf6557d5b73f423). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/23084 I think that we need to take care of `UnsafeExternalSorterSuite.testGetIterator`, too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23084 **[Test build #99006 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99006/testReport)** for PR 23084 at commit [`7345942`](https://github.com/apache/spark/commit/73459423d0c3a5070a143865bcf6557d5b73f423). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23084 **[Test build #99002 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99002/testReport)** for PR 23084 at commit [`7345942`](https://github.com/apache/spark/commit/73459423d0c3a5070a143865bcf6557d5b73f423). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23084 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5144/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23084 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23084 good catch! thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23084 add to whitelist --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23084 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/23084 cc @cloud-fan @kiszk @dongjoon-hyun --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23084 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23084 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
