[GitHub] spark pull request: KafKaDirectDstream should filter empty partiti...

Github user tdas commented on the pull request: https://github.com/apache/spark/pull/8237#issuecomment-132429469 Thanks! Appreciate your understanding. On Tue, Aug 18, 2015 at 7:51 PM, Shan Hu wrote: > @tdas Ok, as you already discussed befor

[GitHub] spark pull request: KafKaDirectDstream should filter empty partiti...

Github user suyanNone commented on the pull request: https://github.com/apache/spark/pull/8237#issuecomment-132427763 @tdas Ok, as you already discussed before, so let's stay the same. --- If your project is set up for it, you can reply to this email and have your reply appear on GitH

[GitHub] spark pull request: KafKaDirectDstream should filter empty partiti...

Github user suyanNone closed the pull request at: https://github.com/apache/spark/pull/8237 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[GitHub] spark pull request: KafKaDirectDstream should filter empty partiti...

Github user tdas commented on the pull request: https://github.com/apache/spark/pull/8237#issuecomment-132087668 I understand and I have debated about this before. This is one of those grey areas where there is no golden solution. In such cases, I think it is better to err towards hav

[GitHub] spark pull request: KafKaDirectDstream should filter empty partiti...

Github user suyanNone commented on the pull request: https://github.com/apache/spark/pull/8237#issuecomment-132054268 @tdas, yean, I agree with it will be confused in semantics, and for a batch streaming system, it will not blocked the next batch as long as it will finished quickly, e

[GitHub] spark pull request: KafKaDirectDstream should filter empty partiti...

Github user tdas commented on the pull request: https://github.com/apache/spark/pull/8237#issuecomment-132033574 We have rejected such ideas before because not generating an RDD in a batch will actually cause problems in semantics of the downstream operations. For example, if you are

[GitHub] spark pull request: KafKaDirectDstream should filter empty partiti...

Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/8237#discussion_r37255803 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala --- @@ -253,16 +253,19 @@ object KafkaRDD { fromOffsets:

[GitHub] spark pull request: KafKaDirectDstream should filter empty partiti...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8237#issuecomment-131855297 [Test build #41018 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41018/console) for PR 8237 at commit [`4e4764d`](https://github.

[GitHub] spark pull request: KafKaDirectDstream should filter empty partiti...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8237#issuecomment-131855341 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: KafKaDirectDstream should filter empty partiti...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8237#issuecomment-131855342 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41

[GitHub] spark pull request: KafKaDirectDstream should filter empty partiti...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8237#issuecomment-131847832 [Test build #41018 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41018/consoleFull) for PR 8237 at commit [`4e4764d`](https://gith

[GitHub] spark pull request: KafKaDirectDstream should filter empty partiti...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8237#issuecomment-131844793 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: KafKaDirectDstream should filter empty partiti...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8237#issuecomment-131802701 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: KafKaDirectDstream should filter empty partiti...

GitHub user suyanNone reopened a pull request: https://github.com/apache/spark/pull/8237 KafKaDirectDstream should filter empty partition task or rdd To avoid submit stages and tasks for 0 events batch. You can merge this pull request into a Git repository by running: $ git pul

[GitHub] spark pull request: KafKaDirectDstream should filter empty partiti...

Github user suyanNone closed the pull request at: https://github.com/apache/spark/pull/8237 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[GitHub] spark pull request: KafKaDirectDstream should filter empty partiti...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8237#issuecomment-131799163 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: KafKaDirectDstream should filter empty partiti...

GitHub user suyanNone opened a pull request: https://github.com/apache/spark/pull/8237 KafKaDirectDstream should filter empty partition task or rdd To avoid submit stages and tasks for 0 events batch. You can merge this pull request into a Git repository by running: $ git pull