[
https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15110994#comment-15110994
]
Mario Briggs commented on SPARK-12177:
--------------------------------------
bq. If one uses the kafka v9 jar even when using the old consumer API, it can
only work a Kafka v9 broker.
I tried it on a single system setup (v0.9 client talking to v0.8 server-side)
and the consumers had a problem (old or new). The producers though worked fine.
So you are right. So then we will have kafka-assembly and
kafka-assembly-v09/new and each including their version of kafka jars
respectively right? ( I guess now, you were all along thinking 2 diff
assemblies, and i guessed the other way round. Duh, IRC might have been faster)
With the above confirmed, it automatically throws out 'The public API
signatures (of KafkaUtils in v0.9 subproject) are different and do not clash
(with KafkaUtils in original kafka subproject) and hence can be added to the
existing (original kafka subproject) KafkaUtils class.’
So the only thing left it seems is to use 'new' or a better term instead of
'v09', since we both agree on that. Great and thanks Mark.
How's the 'python/pyspark/streaming/kafka-v09(new).py' going
> Update KafkaDStreams to new Kafka 0.9 Consumer API
> --------------------------------------------------
>
> Key: SPARK-12177
> URL: https://issues.apache.org/jira/browse/SPARK-12177
> Project: Spark
> Issue Type: Improvement
> Components: Streaming
> Affects Versions: 1.6.0
> Reporter: Nikita Tarasenko
> Labels: consumer, kafka
>
> Kafka 0.9 already released and it introduce new consumer API that not
> compatible with old one. So, I added new consumer api. I made separate
> classes in package org.apache.spark.streaming.kafka.v09 with changed API. I
> didn't remove old classes for more backward compatibility. User will not need
> to change his old spark applications when he uprgade to new Spark version.
> Please rewiew my changes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
