[jira] [Commented] (SPARK-33913) Upgrade Kafka to 2.7.0
[ https://issues.apache.org/jira/browse/SPARK-33913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17267028#comment-17267028 ] Apache Spark commented on SPARK-33913: -- User 'dongjoon-hyun' has created a pull request for this issue: https://github.com/apache/spark/pull/31223 > Upgrade Kafka to 2.7.0 > -- > > Key: SPARK-33913 > URL: https://issues.apache.org/jira/browse/SPARK-33913 > Project: Spark > Issue Type: Improvement > Components: Build, DStreams >Affects Versions: 3.2.0 >Reporter: dengziming >Priority: Major > > > The Apache Kafka community has released for Apache Kafka 2.7.0, some features > are useful for example the KAFKA-9893 > configurable TCP connection timeout, more details : > https://downloads.apache.org/kafka/2.7.0/RELEASE_NOTES.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-33913) Upgrade Kafka to 2.7.0
[ https://issues.apache.org/jira/browse/SPARK-33913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17267027#comment-17267027 ] Apache Spark commented on SPARK-33913: -- User 'dongjoon-hyun' has created a pull request for this issue: https://github.com/apache/spark/pull/31223 > Upgrade Kafka to 2.7.0 > -- > > Key: SPARK-33913 > URL: https://issues.apache.org/jira/browse/SPARK-33913 > Project: Spark > Issue Type: Improvement > Components: Build, DStreams >Affects Versions: 3.2.0 >Reporter: dengziming >Priority: Major > > > The Apache Kafka community has released for Apache Kafka 2.7.0, some features > are useful for example the KAFKA-9893 > configurable TCP connection timeout, more details : > https://downloads.apache.org/kafka/2.7.0/RELEASE_NOTES.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-33913) Upgrade Kafka to 2.7.0
[ https://issues.apache.org/jira/browse/SPARK-33913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17255395#comment-17255395 ] L. C. Hsieh commented on SPARK-33913: - Seems we cannot upgrade to Kafka 2.7.0. Because Kafka core inlines the Scala library, you can not use a different Scala patch version than what Kafka used to compile its jars: https://github.com/embeddedkafka/embedded-kafka/issues/202 Kafka 2.7.0 uses Scala 2.12.12 and currently Spark uses Scala 2.12.10, so there will be {{java.lang.NoClassDefFoundError: scala/math/Ordering$$anon$7}} errors. Due to an issue in Scala 2.12.12, Spark won't upgrade to Scala 2.12.12: https://github.com/scala/bug/issues/12096, and waits for Scala 2.12.13. So, seems for Kafka, Spark needs to wait for next Kafka version which uses Scala 2.12.13 too. > Upgrade Kafka to 2.7.0 > -- > > Key: SPARK-33913 > URL: https://issues.apache.org/jira/browse/SPARK-33913 > Project: Spark > Issue Type: Improvement > Components: Build, DStreams >Affects Versions: 3.2.0 >Reporter: dengziming >Priority: Major > > > The Apache Kafka community has released for Apache Kafka 2.7.0, some features > are useful for example the KAFKA-9893 > configurable TCP connection timeout, more details : > https://downloads.apache.org/kafka/2.7.0/RELEASE_NOTES.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-33913) Upgrade Kafka to 2.7.0
[ https://issues.apache.org/jira/browse/SPARK-33913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17255324#comment-17255324 ] Apache Spark commented on SPARK-33913: -- User 'viirya' has created a pull request for this issue: https://github.com/apache/spark/pull/30939 > Upgrade Kafka to 2.7.0 > -- > > Key: SPARK-33913 > URL: https://issues.apache.org/jira/browse/SPARK-33913 > Project: Spark > Issue Type: Improvement > Components: Build, DStreams >Affects Versions: 3.2.0 >Reporter: dengziming >Priority: Major > > > The Apache Kafka community has released for Apache Kafka 2.7.0, some features > are useful for example the KAFKA-9893 > configurable TCP connection timeout, more details : > https://downloads.apache.org/kafka/2.7.0/RELEASE_NOTES.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org