Github user markgrover commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-191376232
This discussion is very closely tied to SPARK-12177 and so I agree with
@koeninger that it makes sense for this to decided when we resolved
SPARK-12177. So, for
Github user markgrover closed the pull request at:
https://github.com/apache/spark/pull/11143
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user mbonaci commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-190971237
@ijuma my bad, I was looking for `SimpleConsumer` in [the client
api](https://home.apache.org/~junrao/kafka-0.9.0.1-candidate1/javadoc/),
totally forgetting that it's
Github user koeninger commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-190914907
I really don't think it makes sense to discuss this PR outside of the
context of SPARK-12177 and the approach taken for supporting the new consumer.
Merging it as
Github user gwenshap commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-190909149
Agreeing with @mbonaci. Users who delay upgrading their Kafka brokers can
continue using older versions of Spark.
@markgrover If you have code that depends on
Github user ijuma commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-190883372
That's right @markgrover, the current approach used by Kafka preserves
compatibility for users, but makes it a bit complicated for libraries/systems
that want to support
Github user markgrover commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-190845440
Kafka 0.9 still has the old high and low level consumers (and the old
producers too).
Correct but they are not compatible when using the 0.9 client with 0.8
Github user ijuma commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-190818071
"When Kafka went from 0.8 to 0.9, they did not leave the old high and low
level consumer APIs around just because there are users that use them."
Kafka 0.9 still
Github user mbonaci commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-190808475
IMHO option 1 is cleaner.
Users who have to postpone upgrading Kafka brokers have the option of using
older versions of Spark.
When Kafka went from 0.8 to
Github user markgrover commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-189383164
Thanks @srowen and @koeninger for your thoughts.
I did take a look at what other projects are doing. In particular, I looked
at Storm and Flume since they
Github user koeninger commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-189365359
Kafka isn't just a library dependency that a user can try out a new version
of on a particular job and see if it works.
It's an infrastructure component that
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-189259048
@koeninger may be a better reviewer at this point.
I suspect that 2.0.0 is the only opportunity in the near future to move to
0.9, so I think it's a good idea. Any
Github user markgrover commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-189110299
Hi @tdas, I'd still appreciate if you could review this. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-189099111
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-189099110
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-189098928
**[Test build #52012 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52012/consoleFull)**
for PR 11143 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-189072415
**[Test build #52012 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52012/consoleFull)**
for PR 11143 at commit
Github user mariobriggs commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-182769749
FWIW, the [IBM Cloud Message Hub
service](https://www.ng.bluemix.net/docs/services/MessageHub/index.html#messagehub050)
which is Kafka, has already moved to 0.9.0
Github user markgrover commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-182996634
Thanks Mario. Do let us know what you think @tdas
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-182122561
**[Test build #51000 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51000/consoleFull)**
for PR 11143 at commit
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-182131389
Code LGTM.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-182136221
@markgrover What is the compatibility story of Kafka 0.8 and 0.9 with
existing Kafka installations? If Kafka 0.9 client API is not supposed to back
compatible with Kafka
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-182153559
**[Test build #51000 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51000/consoleFull)**
for PR 11143 at commit
Github user markgrover commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-182133935
Thanks for reviewing, @vanzin Much appreciated!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-182153877
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-182127663
**[Test build #51001 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51001/consoleFull)**
for PR 11143 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-182153879
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-182157427
**[Test build #51001 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51001/consoleFull)**
for PR 11143 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-182157962
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-182136615
**[Test build #51004 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51004/consoleFull)**
for PR 11143 at commit
Github user markgrover commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-182154267
Hi @tdas ! Thanks for reviewing.
I talk about the compatibility story at length in the related PR #10953 (in
particular,
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-182157957
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-182163030
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-182162862
**[Test build #51004 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51004/consoleFull)**
for PR 11143 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-182163032
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
35 matches
Mail list logo
