[jira] [Commented] (SPARK-25954) Upgrade to Kafka 2.1.0
[ https://issues.apache.org/jira/browse/SPARK-25954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16824550#comment-16824550 ] Dongjoon Hyun commented on SPARK-25954: --- Yep. [~ijuma]. I updated Spark already via SPARK-27260. > Upgrade to Kafka 2.1.0 > -- > > Key: SPARK-25954 > URL: https://issues.apache.org/jira/browse/SPARK-25954 > Project: Spark > Issue Type: Sub-task > Components: Structured Streaming >Affects Versions: 3.0.0 >Reporter: Dongjoon Hyun >Assignee: Dongjoon Hyun >Priority: Major > Fix For: 3.0.0 > > > Kafka 2.1.0 vote passed. Since this includes official KAFKA-7264 JDK 11 > support, we had better use that. > - > https://lists.apache.org/thread.html/9f487094491e512b556a1c9c3c6034ac642b088e3f797e3d192ebc9d@%3Cdev.kafka.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25954) Upgrade to Kafka 2.1.0
[ https://issues.apache.org/jira/browse/SPARK-25954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16824226#comment-16824226 ] Ismael Juma commented on SPARK-25954: - That's awesome, sorry for the noise. > Upgrade to Kafka 2.1.0 > -- > > Key: SPARK-25954 > URL: https://issues.apache.org/jira/browse/SPARK-25954 > Project: Spark > Issue Type: Sub-task > Components: Structured Streaming >Affects Versions: 3.0.0 >Reporter: Dongjoon Hyun >Assignee: Dongjoon Hyun >Priority: Major > Fix For: 3.0.0 > > > Kafka 2.1.0 vote passed. Since this includes official KAFKA-7264 JDK 11 > support, we had better use that. > - > https://lists.apache.org/thread.html/9f487094491e512b556a1c9c3c6034ac642b088e3f797e3d192ebc9d@%3Cdev.kafka.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25954) Upgrade to Kafka 2.1.0
[ https://issues.apache.org/jira/browse/SPARK-25954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16824188#comment-16824188 ] Gabor Somogyi commented on SPARK-25954: --- [~ijuma] On the actual master 2.2.0 kafka-clients used. > Upgrade to Kafka 2.1.0 > -- > > Key: SPARK-25954 > URL: https://issues.apache.org/jira/browse/SPARK-25954 > Project: Spark > Issue Type: Sub-task > Components: Structured Streaming >Affects Versions: 3.0.0 >Reporter: Dongjoon Hyun >Assignee: Dongjoon Hyun >Priority: Major > Fix For: 3.0.0 > > > Kafka 2.1.0 vote passed. Since this includes official KAFKA-7264 JDK 11 > support, we had better use that. > - > https://lists.apache.org/thread.html/9f487094491e512b556a1c9c3c6034ac642b088e3f797e3d192ebc9d@%3Cdev.kafka.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25954) Upgrade to Kafka 2.1.0
[ https://issues.apache.org/jira/browse/SPARK-25954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16824177#comment-16824177 ] Ismael Juma commented on SPARK-25954: - [~dongjoon] Kafka 2.2.0 has a fix for an issue that has caused problems for Spark in the past: https://cwiki.apache.org/confluence/display/KAFKA/KIP-207%3A+Offsets+returned+by+ListOffsetsResponse+should+be+monotonically+increasing+even+during+a+partition+leader+change It might be worth upgrading when you have a chance. > Upgrade to Kafka 2.1.0 > -- > > Key: SPARK-25954 > URL: https://issues.apache.org/jira/browse/SPARK-25954 > Project: Spark > Issue Type: Sub-task > Components: Structured Streaming >Affects Versions: 3.0.0 >Reporter: Dongjoon Hyun >Assignee: Dongjoon Hyun >Priority: Major > Fix For: 3.0.0 > > > Kafka 2.1.0 vote passed. Since this includes official KAFKA-7264 JDK 11 > support, we had better use that. > - > https://lists.apache.org/thread.html/9f487094491e512b556a1c9c3c6034ac642b088e3f797e3d192ebc9d@%3Cdev.kafka.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25954) Upgrade to Kafka 2.1.0
[ https://issues.apache.org/jira/browse/SPARK-25954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16696214#comment-16696214 ] Apache Spark commented on SPARK-25954: -- User 'dongjoon-hyun' has created a pull request for this issue: https://github.com/apache/spark/pull/23119 > Upgrade to Kafka 2.1.0 > -- > > Key: SPARK-25954 > URL: https://issues.apache.org/jira/browse/SPARK-25954 > Project: Spark > Issue Type: Sub-task > Components: Structured Streaming >Affects Versions: 3.0.0 >Reporter: Dongjoon Hyun >Assignee: Dongjoon Hyun >Priority: Major > Fix For: 3.0.0 > > > Kafka 2.1.0 vote passed. Since this includes official KAFKA-7264 JDK 11 > support, we had better use that. > - > https://lists.apache.org/thread.html/9f487094491e512b556a1c9c3c6034ac642b088e3f797e3d192ebc9d@%3Cdev.kafka.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25954) Upgrade to Kafka 2.1.0
[ https://issues.apache.org/jira/browse/SPARK-25954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16696215#comment-16696215 ] Apache Spark commented on SPARK-25954: -- User 'dongjoon-hyun' has created a pull request for this issue: https://github.com/apache/spark/pull/23119 > Upgrade to Kafka 2.1.0 > -- > > Key: SPARK-25954 > URL: https://issues.apache.org/jira/browse/SPARK-25954 > Project: Spark > Issue Type: Sub-task > Components: Structured Streaming >Affects Versions: 3.0.0 >Reporter: Dongjoon Hyun >Assignee: Dongjoon Hyun >Priority: Major > Fix For: 3.0.0 > > > Kafka 2.1.0 vote passed. Since this includes official KAFKA-7264 JDK 11 > support, we had better use that. > - > https://lists.apache.org/thread.html/9f487094491e512b556a1c9c3c6034ac642b088e3f797e3d192ebc9d@%3Cdev.kafka.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25954) Upgrade to Kafka 2.1.0
[ https://issues.apache.org/jira/browse/SPARK-25954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16693864#comment-16693864 ] Apache Spark commented on SPARK-25954: -- User 'dongjoon-hyun' has created a pull request for this issue: https://github.com/apache/spark/pull/23099 > Upgrade to Kafka 2.1.0 > -- > > Key: SPARK-25954 > URL: https://issues.apache.org/jira/browse/SPARK-25954 > Project: Spark > Issue Type: Sub-task > Components: Structured Streaming >Affects Versions: 3.0.0 >Reporter: Dongjoon Hyun >Priority: Major > > Kafka 2.1.0 vote passed. Since this includes official KAFKA-7264 JDK 11 > support, we had better use that. > - > https://lists.apache.org/thread.html/9f487094491e512b556a1c9c3c6034ac642b088e3f797e3d192ebc9d@%3Cdev.kafka.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25954) Upgrade to Kafka 2.1.0
[ https://issues.apache.org/jira/browse/SPARK-25954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16693849#comment-16693849 ] Dongjoon Hyun commented on SPARK-25954: --- I update the description because Kafka 2.1.0 vote passed today. > Upgrade to Kafka 2.1.0 > -- > > Key: SPARK-25954 > URL: https://issues.apache.org/jira/browse/SPARK-25954 > Project: Spark > Issue Type: Sub-task > Components: Structured Streaming >Affects Versions: 3.0.0 >Reporter: Dongjoon Hyun >Priority: Major > > Kafka 2.1.0 vote passed. Since this includes official KAFKA-7264 JDK 11 > support, we had better use that. > - > https://lists.apache.org/thread.html/9f487094491e512b556a1c9c3c6034ac642b088e3f797e3d192ebc9d@%3Cdev.kafka.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25954) Upgrade to Kafka 2.1.0
[ https://issues.apache.org/jira/browse/SPARK-25954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16677163#comment-16677163 ] Dongjoon Hyun commented on SPARK-25954: --- Thanks. Of course, it's for Spark 3.0 next year. > Upgrade to Kafka 2.1.0 > -- > > Key: SPARK-25954 > URL: https://issues.apache.org/jira/browse/SPARK-25954 > Project: Spark > Issue Type: Sub-task > Components: Structured Streaming >Affects Versions: 3.0.0 >Reporter: Dongjoon Hyun >Priority: Major > > Kafka 2.1.0 RC0 is started. Since this includes official KAFKA-7264 JDK 11 > support, we had better use that. > - > https://lists.apache.org/thread.html/8288f0afdfed4d329f1a8338320b6e24e7684a0593b4bbd6f1b79101@%3Cdev.kafka.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25954) Upgrade to Kafka 2.1.0
[ https://issues.apache.org/jira/browse/SPARK-25954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16677152#comment-16677152 ] Ted Yu commented on SPARK-25954: Looking at Kafka thread, message from Satish indicated there may be another RC coming. > Upgrade to Kafka 2.1.0 > -- > > Key: SPARK-25954 > URL: https://issues.apache.org/jira/browse/SPARK-25954 > Project: Spark > Issue Type: Sub-task > Components: Structured Streaming >Affects Versions: 3.0.0 >Reporter: Dongjoon Hyun >Priority: Major > > Kafka 2.1.0 RC0 is started. Since this includes official KAFKA-7264 JDK 11 > support, we had better use that. > - > https://lists.apache.org/thread.html/8288f0afdfed4d329f1a8338320b6e24e7684a0593b4bbd6f1b79101@%3Cdev.kafka.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25954) Upgrade to Kafka 2.1.0
[ https://issues.apache.org/jira/browse/SPARK-25954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16677107#comment-16677107 ] Dongjoon Hyun commented on SPARK-25954: --- Ping, [~te...@apache.org]. Are you interested in this since you did SPARK-18057? > Upgrade to Kafka 2.1.0 > -- > > Key: SPARK-25954 > URL: https://issues.apache.org/jira/browse/SPARK-25954 > Project: Spark > Issue Type: Sub-task > Components: Structured Streaming >Affects Versions: 3.0.0 >Reporter: Dongjoon Hyun >Priority: Major > > Kafka 2.1.0 RC0 is started. Since this includes official KAFKA-7264 JDK 11 > support, we had better use that. > - > https://lists.apache.org/thread.html/8288f0afdfed4d329f1a8338320b6e24e7684a0593b4bbd6f1b79101@%3Cdev.kafka.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
