[GitHub] spark issue #21038: [SPARK-22968][DStream] Fix Kafka partition revoked issue

2018-04-11 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21038 Thanks @koeninger , then I will just improve the exception message. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21038: [SPARK-22968][DStream] Fix Kafka partition revoked issue

2018-04-11 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/21038 I can't think of a valid reason to create a configuration to allow it. It just fundamentally doesn't make sense to run different apps with the same group id. Trying to catch and

[GitHub] spark issue #21038: [SPARK-22968][DStream] Fix Kafka partition revoked issue

2018-04-11 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21038 Thanks @koeninger for your comments. I think your suggestion is valid, the log here is just pasted from JIRA, but we also got the same issue from customer's report. Here in the PR

[GitHub] spark issue #21038: [SPARK-22968][DStream] Fix Kafka partition revoked issue

2018-04-11 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/21038 The log in the jira looks like it's from a consumer rebalance, i.e. more than one driver consumer was running with the same group id. Isn't the underlying problem here that the user is

[GitHub] spark issue #21038: [SPARK-22968][DStream] Fix Kafka partition revoked issue

2018-04-11 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21038 @koeninger would you please help to review, thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21038: [SPARK-22968][DStream] Fix Kafka partition revoked issue

2018-04-11 Thread Hackeruncle
Github user Hackeruncle commented on the issue: https://github.com/apache/spark/pull/21038 @jerryshao Thank you very much for this issue. I go to compile and test. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21038: [SPARK-22968][DStream] Fix Kafka partition revoked issue

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21038 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21038: [SPARK-22968][DStream] Fix Kafka partition revoked issue

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21038 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89177/ Test PASSed. ---

[GitHub] spark issue #21038: [SPARK-22968][DStream] Fix Kafka partition revoked issue

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21038 **[Test build #89177 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89177/testReport)** for PR 21038 at commit