[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16291 what's the current status of this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user CodingCat commented on the issue: https://github.com/apache/spark/pull/16291 what's the current status of this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/16291 @tgravescs - I have not gotten time to work on this. Hopefully will get to it sometime next week. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/16291 @sitalkedia are you still working on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/16291 Thanks guys for your input. I have made changes to support grouping of listeners to preserve the existing synchronization behavior. Currently, the `ExecutionAllocationManager`, `HeartbeatReceiver` and `EventLogger` use their separate threads. All User developed `Listerners` use a separate thread and all other belong to a default group. Let me know what you think the approach. @rxin, @vanzin, @markhamstra , @mridulm . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70660/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70660 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70660/testReport)** for PR 16291 at commit [`e09d66e`](https://github.com/apache/spark/commit/e09d66ef58049481924dcdf88157bbb981cac7fd). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70660 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70660/testReport)** for PR 16291 at commit [`e09d66e`](https://github.com/apache/spark/commit/e09d66ef58049481924dcdf88157bbb981cac7fd). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/16291 Jenkins retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70656/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70656 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70656/testReport)** for PR 16291 at commit [`e09d66e`](https://github.com/apache/spark/commit/e09d66ef58049481924dcdf88157bbb981cac7fd). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70652/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70652 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70652/testReport)** for PR 16291 at commit [`6494cfe`](https://github.com/apache/spark/commit/6494cfe171dce7845d0db8d53f845a1df933d202). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70652 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70652/testReport)** for PR 16291 at commit [`6494cfe`](https://github.com/apache/spark/commit/6494cfe171dce7845d0db8d53f845a1df933d202). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70651 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70651/testReport)** for PR 16291 at commit [`9e5819d`](https://github.com/apache/spark/commit/9e5819df62a77a12bd0b08d59499cb30aa591862). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70651/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70651 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70651/testReport)** for PR 16291 at commit [`9e5819d`](https://github.com/apache/spark/commit/9e5819df62a77a12bd0b08d59499cb30aa591862). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70650/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70650 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70650/testReport)** for PR 16291 at commit [`f2c777d`](https://github.com/apache/spark/commit/f2c777d4b8f2eeca4cfe2c51601e5c96e4424880). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70650 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70650/testReport)** for PR 16291 at commit [`f2c777d`](https://github.com/apache/spark/commit/f2c777d4b8f2eeca4cfe2c51601e5c96e4424880). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/16291 jenkins test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70649/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70649 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70649/testReport)** for PR 16291 at commit [`162001a`](https://github.com/apache/spark/commit/162001a2cd6fd75cdff22b4e5620f007e483480a). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70649 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70649/testReport)** for PR 16291 at commit [`162001a`](https://github.com/apache/spark/commit/162001a2cd6fd75cdff22b4e5620f007e483480a). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/16291 I agree with @markhamstra and @vanzin - having ability to tag listeners into groups (default = spark listener group) and preserving current synchronized behavior within group would be ensure backward compatibility at fairly minimal additional complexity. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16291 > I think that will be an over-kill. Having a separate thread per listener simplifies the design. I don't think it's overkill, and it wouldn't really be that much of a change from your existing design. It also serves as a way of controlling how much extra memory you'll need for these things, since right now each listener would incur in its own thread + message queue. e.g. you could just have a tagging interface (if listener implements "X" then it gets its own dedicated dispatch thread, or if you want to get fancy, you could also define a "group" so that listener in the same group share a dispatch thread). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user markhamstra commented on the issue: https://github.com/apache/spark/pull/16291 > Each listener should in theory could work independent of each other and we should only guarantee ordered processing of the events within a listener. If we were starting from nothing, then yes, it would be valid and advisable to design the Listener infrastructure using only this weaker guarantee. The issue, though, is that we are not starting from nothing, but rather from a system that currently offers a much stronger guarantee on the synchronized behavior of Listeners. If it is the case that no Listeners currently rely on the stronger guarantee and thus could work completely correctly under the weaker guarantee of this PR, then we could make this change without much additional concern. But reaching that level of confidence in current Listeners is a difficult prerequisite -- strictly speaking, it's an impossible task. We could carefully work through all the internal behavior of Spark's Listeners to convince ourselves that they can work correctly under the new, weaker guarantee. At a bare minimum, we need to do that much before we can consider merging this PR -- but that's probably not enough. The problem is that Listeners aren't just internal to Spark. Users have also developed their own custom Listeners that either implement `SparkListenerInterface` or extend `SparkListener` or `SparkFirehoseListener`, and we can't just assume that those custom Listeners don't rely upon the current guarantee to either synchronize behavior with other custom Listeners or even with Spark internal Listeners. Since we can't know that user Listeners don't already rely upon the current, stronger guarantee, the question now becomes whether we even have the freedom to change that guarantee within the lifetime of Spark 2.x, or whether any such change would have to wait for Spark 3.x. `SparkListener` is still annotated as `@DeveloperAPI`, so if that were the only piece in play, then we could change its guarantee fairly freely. `SparkListenerInterface` is almost as good, since it includes the admonition in a comment to "[n]ote that this is an internal interface which might change in different Spark releases." The stickier issue is with `SparkFirehoseListener`, which carries no such annotations or comments, but is just a plain public class and API. So, after convincing ourselves that Spark's internal Listeners would be fine with this PR, we'd still have to convince the Spark PMC that changing the public `SparkFirehoseListener` (with prominent warnings in the release notes, of course) before Spark 3.x would be acceptable. And all of the above is still really only arguing about whether we *could* adopt this PR in essentially its present form. There are still questions of whether we *should* do this or maybe instead we should do something a little different or more. I can see some merit in Marcelo's "opt in" suggestion. If there is utility in having groups of Listeners that can rely upon synchronized behavior, then we should probably retain one or more threads running synchronized Listeners. For example, if Listener A relies upon synchronization with Listeners B and C while D needs to synchronize with E, but F, G and H are all independent, then there are a couple of things we could do. First, the independent Listeners (F, G and H) can each run in its own thread, providing the scalable performance that this PR is aiming for. After that, we could either have one synchronized Listener thread for all the other Listeners, or we could have one thread for A, B and C and one thread for D and E. Wheth er we support only one synchronized Listener group/thread or multiple, we'd still need some mechanism for Listeners to select into a synchronized group or to indicate that they can and should be run independently on their own thread. @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/16291 @markhamstra, @vanzin thanks for your comments! >> so it is entirely possible. for example, for one ListenerEventExecutor to process a task end event for a particular task before another ListenerEventExecutor has worked sufficiently through its eventQueue to have even seen the corresponding task start event. That is quite a bit different than the prior ordering guarantee implied by the comment "onPostEvent is guaranteed to be called in the same thread for all listeners." That is true. If a listener is slow in processing the events than it might not see the task start event where as other listeners might already have processed the corresponding task end event. I am not sure why would that cause any problem though. Each listener should in theory could work independent of each other and we should only guarantee ordered processing of the events within a listener. This is the only way I can think of to scale the driver for large workloads. >> but I wonder if a system where listeners can "opt in" to this behavior wouldn't be better. e.g., EventLoggingListener can slow down the bus, having it run in a separate thread would benefit all the others. I think that will be an over-kill. Having a separate thread per listener simplifies the design. Also, for user added listeners, it will be hard to decide if we want to have a shared or separate thread per listener. The only downside of having a separate thread per listener is increased memory footprint in worst case. But I would argue that since the average event processing latency is decreased by multi threaded event processor, the backlog on the event queues will clear faster which actually might decrease the driver memory footprint on average case. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16291 I'm not sure whether that change would be a problem, but I wonder if a system where listeners can "opt in" to this behavior wouldn't be better. e.g., `EventLoggingListener` can slow down the bus, having it run in a separate thread would benefit all the others. Similarly, the dynamic allocation listener could have its own dedicated thread since it needs lower latency, and doesn't really depend on any others. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user markhamstra commented on the issue: https://github.com/apache/spark/pull/16291 > You are right, that's why the executor service is single threaded which guarantees ordered processing of the events per listener. But this is still substantially different from the ordering guarantees in the current implementation, and I'm not sure that this PR wouldn't invalidate some assumptions made by current listeners. Events will end up in the same order in each `ListenerEventExecutor` (at least by my understanding, which someone else should double-check), but there is no synchronization between `ListenerEventExecutor`s of the processing of their `eventQueue`s; so it is entirely possible. for example, for one `ListenerEventExecutor` to process a task end event for a particular task before another `ListenerEventExecutor` has worked sufficiently through its `eventQueue` to have even seen the corresponding task start event. That is quite a bit different than the prior ordering guarantee implied by the comment "`onPostEvent` is guaranteed to be called in the same thread for all listeners." Whether this actually constitutes a problem or not, I'm not certain; but serializing event processing within each `ListenerEventExecutor` isn't enough if `Listener`s on different `ListenerEventExecutor` threads expect to share mutable state. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/16291 This change is ready for review @rxin, @zsxwing, @kayousterhout --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70360/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70360 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70360/testReport)** for PR 16291 at commit [`66e4f12`](https://github.com/apache/spark/commit/66e4f12c726d7cc49620f9c5747ab6b1d70d5bf3). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class StreamingQueryListenerBus(val sparkListenerBus: LiveListenerBus)` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70360 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70360/testReport)** for PR 16291 at commit [`66e4f12`](https://github.com/apache/spark/commit/66e4f12c726d7cc49620f9c5747ab6b1d70d5bf3). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/16291 Jenkins retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70345/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70345 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70345/testReport)** for PR 16291 at commit [`66e4f12`](https://github.com/apache/spark/commit/66e4f12c726d7cc49620f9c5747ab6b1d70d5bf3). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70325/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70325 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70325/testReport)** for PR 16291 at commit [`6763827`](https://github.com/apache/spark/commit/676382790b2e73c6ad3b4abe7c565ad266d9df1a). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class StreamingQueryListenerBus(val sparkListenerBus: LiveListenerBus)` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70325 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70325/testReport)** for PR 16291 at commit [`6763827`](https://github.com/apache/spark/commit/676382790b2e73c6ad3b4abe7c565ad266d9df1a). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70322/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70322 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70322/testReport)** for PR 16291 at commit [`fbcdcca`](https://github.com/apache/spark/commit/fbcdccab8845af1bd0b2ff31c71dee52d328f452). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class StreamingQueryListenerBus(val sparkListenerBus: LiveListenerBus)` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70322 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70322/testReport)** for PR 16291 at commit [`fbcdcca`](https://github.com/apache/spark/commit/fbcdccab8845af1bd0b2ff31c71dee52d328f452). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70298/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70299/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70299 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70299/testReport)** for PR 16291 at commit [`9b4968d`](https://github.com/apache/spark/commit/9b4968dc74157a781f1dfb9cf9f4c2c446c8fe22). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70298 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70298/testReport)** for PR 16291 at commit [`9b4968d`](https://github.com/apache/spark/commit/9b4968dc74157a781f1dfb9cf9f4c2c446c8fe22). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/16291 Jenkins test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70290/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70290 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70290/testReport)** for PR 16291 at commit [`20fbf2f`](https://github.com/apache/spark/commit/20fbf2f2978efdbbeedca1feba475877b9e1a8d1). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class StreamingQueryListenerBus(val sparkListenerBus: LiveListenerBus)` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70290 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70290/testReport)** for PR 16291 at commit [`20fbf2f`](https://github.com/apache/spark/commit/20fbf2f2978efdbbeedca1feba475877b9e1a8d1). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70289/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70289 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70289/testReport)** for PR 16291 at commit [`d687aaf`](https://github.com/apache/spark/commit/d687aaf0e56e327ca93709f9d4972b86b0b863c0). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class StreamingQueryListenerBus(val sparkListenerBus: LiveListenerBus)` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70289 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70289/testReport)** for PR 16291 at commit [`d687aaf`](https://github.com/apache/spark/commit/d687aaf0e56e327ca93709f9d4972b86b0b863c0). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70287 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70287/testReport)** for PR 16291 at commit [`c790861`](https://github.com/apache/spark/commit/c790861da9e5e15e23d1943f1e8ff406f77dce6b). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class StreamingQueryListenerBus(val sparkListenerBus: LiveListenerBus)` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16291 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70287/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16291 **[Test build #70287 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70287/testReport)** for PR 16291 at commit [`c790861`](https://github.com/apache/spark/commit/c790861da9e5e15e23d1943f1e8ff406f77dce6b). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
