[
https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16371174#comment-16371174
]
Sönke Liebau edited comment on SPARK-18057 at 2/21/18 10:10 AM:
----------------------------------------------------------------
I agree that we should not rename the existing kafka_10 package, as that would
probably cause people loads of pain.
It is however tempting to have "clean" naming - would it be an option to add a
package simply called kafka and update the kafka version in that? We could keep
the kafka_10 package around for now but deprecate it at some point in time.
I am a bit on the fence about this, as in principle the current Kafka story is:
"Any client with a version of 0.10.2.0 or later will support brokers of version
0.10.x or later, so simply upgrading the kafka version in the existing package
should not break anything, as 0.9.x is not currently supported anyway.
However there is a caveat:
{code}
If the burden of backwards compatibility becomes too large, at some point we
may need to break it.
{code}
So there is the possibility of the kafka_10 package becoming relevant again if
later Kafka versions stop supporting 0.10.x brokers.
was (Author: sliebau):
I agree that we should not rename the existing kafka_10 package, as that would
probably cause people loads of pain.
It is however tempting to have "clean" naming - would it be an option to add a
package simply called kafka and update the kafka version in that? We could keep
the kafka_10 package around for now but deprecate it at some point in time.
I am a bit on the fence about this, as in principle the current Kafka story is:
"Any client with a version of 0.10.2.0 or later will support brokers of version
0.10.x or later, so simply upgrading the kafka version in the existing package
should not break anything, as 0.9.x is not currently supported anyway.
However there is a caveat:
{code:java}
If the burden of backwards compatibility becomes too large, at some point we
may need to break it.
{code}
So there is the possibility of the kafka_10 package becoming relevant again if
later Kafka versions stop supporting 0.10.x brokers.
> Update structured streaming kafka from 10.0.1 to 10.2.0
> -------------------------------------------------------
>
> Key: SPARK-18057
> URL: https://issues.apache.org/jira/browse/SPARK-18057
> Project: Spark
> Issue Type: Improvement
> Components: Structured Streaming
> Reporter: Cody Koeninger
> Priority: Major
>
> There are a couple of relevant KIPs here,
> https://archive.apache.org/dist/kafka/0.10.1.0/RELEASE_NOTES.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
