[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16345963#comment-16345963 ]
John Cheng commented on SPARK-18057: ------------------------------------ Apache Kafka is now at version 1.0. For people who want to use Spark streaming against Kafka brokers on 1.0.0, it is preferable to use the `org.apache.kafka:kafka-clients:jar:1.0.0` client. "Most of the discussion on the performance impact of [upgrading to the 0.10.0 message format|https://kafka.apache.org/0110/documentation.html#upgrade_10_performance_impact] remains pertinent to the 0.11.0 upgrade. This mainly affects clusters that are not secured with TLS since "zero-copy" transfer is already not possible in that case. In order to avoid the cost of down-conversion, you should ensure that consumer applications are upgraded to the latest 0.11.0 client." > Update structured streaming kafka from 10.0.1 to 10.2.0 > ------------------------------------------------------- > > Key: SPARK-18057 > URL: https://issues.apache.org/jira/browse/SPARK-18057 > Project: Spark > Issue Type: Improvement > Components: Structured Streaming > Reporter: Cody Koeninger > Priority: Major > > There are a couple of relevant KIPs here, > https://archive.apache.org/dist/kafka/0.10.1.0/RELEASE_NOTES.html -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org