[GitHub] spark issue #15347: [SPARK-16827] Stop reporting spill metrics as shuffle me...
Github user dreamworks007 commented on the issue: https://github.com/apache/spark/pull/15347 @rxin , "Can you also work on reporting spill metrics?" --> here , you mean reporting spill time metrics , right ? Since I think write spill bytes is already included - https://github.com/facebook/FB-Spark/blob/fb-2.0/core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java#L232 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15347: [SPARK-16827] Stop reporting spill metrics as shuffle me...
Github user dafrista commented on the issue: https://github.com/apache/spark/pull/15347 @rxin Sure, I added the PR #15455 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15347: [SPARK-16827] Stop reporting spill metrics as shuffle me...
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15347 @dafrista can you create a backport for branch-2.0? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15347: [SPARK-16827] Stop reporting spill metrics as shuffle me...
Github user dafrista commented on the issue: https://github.com/apache/spark/pull/15347 @andrewor14 sure I think the one mentioned in the TODO covers it (SPARK-3577) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15347: [SPARK-16827] Stop reporting spill metrics as shuffle me...
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/15347 OK, this change by itself LGTM. @dafrista would you mind creating a separate JIRA (or point me to an existing one) about the TODO then? Merging this into master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15347: [SPARK-16827] Stop reporting spill metrics as shuffle me...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15347 **[Test build #3296 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3296/consoleFull)** for PR 15347 at commit [`92f3b89`](https://github.com/apache/spark/commit/92f3b899262f628925b23673baa0c37489049fb0). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15347: [SPARK-16827] Stop reporting spill metrics as shuffle me...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15347 **[Test build #3296 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3296/consoleFull)** for PR 15347 at commit [`92f3b89`](https://github.com/apache/spark/commit/92f3b899262f628925b23673baa0c37489049fb0). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15347: [SPARK-16827] Stop reporting spill metrics as shuffle me...
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15347 Jenkins, add to whitelist. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15347: [SPARK-16827] Stop reporting spill metrics as shuffle me...
Github user dafrista commented on the issue: https://github.com/apache/spark/pull/15347 Sure, having the actual spill metrics is something we're interested in as well. I'd like to work on it, but I might not get to it immediately. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15347: [SPARK-16827] Stop reporting spill metrics as shuffle me...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15347 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
