[
https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15905886#comment-15905886
]
Shixiong Zhu commented on SPARK-18057:
--------------------------------------
> Based on previous kafka client upgrades I wouldn't expect them to be binary
> compatible, so it's likely to cause someone problems if they were also making
> use of kafka client libraries in their spark job. Still may be the path of
> least resistance.
I can confirm the APIs used by Kafka sink is source compatible since I didn't
change any core source codes (test codes have to be changed because of the
server APIs are changed). Since these APIs are Java APIs. I'm pretty sure they
are binary compatible. So for the user, even if we upgrade the Kafka client
version, they can still downgrade the Kafka client version if they want, and
just repackage the codes with the kafka client. It's a bit annoying that
"--packages" probably won't work but it's acceptable.
> Update structured streaming kafka from 10.0.1 to 10.2.0
> -------------------------------------------------------
>
> Key: SPARK-18057
> URL: https://issues.apache.org/jira/browse/SPARK-18057
> Project: Spark
> Issue Type: Improvement
> Components: Structured Streaming
> Reporter: Cody Koeninger
>
> There are a couple of relevant KIPs here,
> https://archive.apache.org/dist/kafka/0.10.1.0/RELEASE_NOTES.html
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
