[jira] [Updated] (FLINK-33361) Add Java 17 compatibility to Flink Kafka connector

2023-11-11 Thread ASF GitHub Bot (Jira) 


 [ 
https://issues.apache.org/jira/browse/FLINK-33361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated FLINK-33361:
---
Labels: pull-request-available  (was: )

> Add Java 17 compatibility to Flink Kafka connector
> --
>
> Key: FLINK-33361
> URL: https://issues.apache.org/jira/browse/FLINK-33361
> Project: Flink
>  Issue Type: Improvement
>  Components: Connectors / Kafka
>Affects Versions: kafka-3.0.1, kafka-3.1.0
>Reporter: Martijn Visser
>Priority: Major
>  Labels: pull-request-available
>
> When currently trying to {{mvn clean install -Dflink.version=1.18.0 
> -Dscala-2.12 -Prun-end-to-end-tests 
> -DdistDir=/Users/mvisser/Developer/flink-1.18.0 
> -Dflink.convergence.phase=install 
> -Dlog4j.configurationFile=tools/ci/log4j.properties}} this fails with errors 
> like:
> {code:java}
> [INFO] 
> [INFO] Results:
> [INFO] 
> [ERROR] Errors: 
> [ERROR] FlinkKafkaConsumerBaseMigrationTest.testRestore
> [ERROR]   Run 1: Exception while creating StreamOperatorStateContext.
> [ERROR]   Run 2: Exception while creating StreamOperatorStateContext.
> [ERROR]   Run 3: Exception while creating StreamOperatorStateContext.
> [ERROR]   Run 4: Exception while creating StreamOperatorStateContext.
> [ERROR]   Run 5: Exception while creating StreamOperatorStateContext.
> [ERROR]   Run 6: Exception while creating StreamOperatorStateContext.
> [ERROR]   Run 7: Exception while creating StreamOperatorStateContext.
> [ERROR]   Run 8: Exception while creating StreamOperatorStateContext.
> [ERROR]   Run 9: Exception while creating StreamOperatorStateContext.
> [INFO] 
> [ERROR]   
> FlinkKafkaConsumerBaseTest.testExplicitStateSerializerCompatibility:721 » 
> Runtime
> [ERROR]   FlinkKafkaConsumerBaseTest.testScaleDown:742->testRescaling:817 » 
> Checkpoint C...
> [ERROR]   FlinkKafkaConsumerBaseTest.testScaleUp:737->testRescaling:817 » 
> Checkpoint Cou...
> [ERROR]   UpsertKafkaDynamicTableFactoryTest.testBufferedTableSink:243 » 
> UncheckedIO jav...
> {code}
> Example stacktrace:
> {code:java}
> Test 
> testBufferedTableSink(org.apache.flink.streaming.connectors.kafka.table.UpsertKafkaDynamicTableFactoryTest)
>  failed with:
> java.io.UncheckedIOException: java.io.IOException: Serializing the source 
> elements failed: java.lang.reflect.InaccessibleObjectException: Unable to 
> make field private final java.lang.Object[] java.util.Arrays$ArrayList.a 
> accessible: module java.base does not "opens java.util" to unnamed module 
> @45b4c3a9
>   at 
> org.apache.flink.streaming.api.functions.source.FromElementsFunction.setOutputType(FromElementsFunction.java:162)
>   at 
> org.apache.flink.streaming.util.functions.StreamingFunctionUtils.trySetOutputType(StreamingFunctionUtils.java:84)
>   at 
> org.apache.flink.streaming.util.functions.StreamingFunctionUtils.setOutputType(StreamingFunctionUtils.java:60)
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.setOutputType(AbstractUdfStreamOperator.java:146)
>   at 
> org.apache.flink.streaming.api.operators.SimpleOperatorFactory.setOutputType(SimpleOperatorFactory.java:118)
>   at 
> org.apache.flink.streaming.api.graph.StreamGraph.addOperator(StreamGraph.java:434)
>   at 
> org.apache.flink.streaming.api.graph.StreamGraph.addOperator(StreamGraph.java:402)
>   at 
> org.apache.flink.streaming.api.graph.StreamGraph.addLegacySource(StreamGraph.java:356)
>   at 
> org.apache.flink.streaming.runtime.translators.LegacySourceTransformationTranslator.translateInternal(LegacySourceTransformationTranslator.java:66)
>   at 
> org.apache.flink.streaming.runtime.translators.LegacySourceTransformationTranslator.translateForStreamingInternal(LegacySourceTransformationTranslator.java:53)
>   at 
> org.apache.flink.streaming.runtime.translators.LegacySourceTransformationTranslator.translateForStreamingInternal(LegacySourceTransformationTranslator.java:40)
>   at 
> org.apache.flink.streaming.api.graph.SimpleTransformationTranslator.translateForStreaming(SimpleTransformationTranslator.java:62)
>   at 
> org.apache.flink.streaming.api.graph.StreamGraphGenerator.translate(StreamGraphGenerator.java:860)
>   at 
> org.apache.flink.streaming.api.graph.StreamGraphGenerator.transform(StreamGraphGenerator.java:590)
>   at 
> org.apache.flink.streaming.api.graph.StreamGraphGenerator.getParentInputIds(StreamGraphGenerator.java:881)
>   at 
> org.apache.flink.streaming.api.graph.StreamGraphGenerator.translate(StreamGraphGenerator.java:839)
>   at 
> org.apache.flink.streaming.api.graph.StreamGraphGenerator.transform(StreamGraphGenerator.java:590)
>   at 
> org.apache.flink.streaming.api.graph.StreamGraphGenerator.generate(StreamGraphGenerator.java:328)
>   at 
> org.apache.flink.strea

[jira] [Updated] (FLINK-33361) Add Java 17 compatibility to Flink Kafka connector

2023-10-25 Thread Martijn Visser (Jira) 


 [ 
https://issues.apache.org/jira/browse/FLINK-33361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Martijn Visser updated FLINK-33361:
---
Summary: Add Java 17 compatibility to Flink Kafka connector  (was: Add Java 
17 compatibility to Flink Kafka consumer)

> Add Java 17 compatibility to Flink Kafka connector
> --
>
> Key: FLINK-33361
> URL: https://issues.apache.org/jira/browse/FLINK-33361
> Project: Flink
>  Issue Type: Improvement
>  Components: Connectors / Kafka
>Affects Versions: kafka-3.0.1, kafka-3.1.0
>Reporter: Martijn Visser
>Priority: Major
>
> When currently trying to {{mvn clean install -Dflink.version=1.18.0 
> -Dscala-2.12 -Prun-end-to-end-tests 
> -DdistDir=/Users/mvisser/Developer/flink-1.18.0 
> -Dflink.convergence.phase=install 
> -Dlog4j.configurationFile=tools/ci/log4j.properties}} this fails with errors 
> like:
> {code:java}
> [INFO] 
> [INFO] Results:
> [INFO] 
> [ERROR] Errors: 
> [ERROR] FlinkKafkaConsumerBaseMigrationTest.testRestore
> [ERROR]   Run 1: Exception while creating StreamOperatorStateContext.
> [ERROR]   Run 2: Exception while creating StreamOperatorStateContext.
> [ERROR]   Run 3: Exception while creating StreamOperatorStateContext.
> [ERROR]   Run 4: Exception while creating StreamOperatorStateContext.
> [ERROR]   Run 5: Exception while creating StreamOperatorStateContext.
> [ERROR]   Run 6: Exception while creating StreamOperatorStateContext.
> [ERROR]   Run 7: Exception while creating StreamOperatorStateContext.
> [ERROR]   Run 8: Exception while creating StreamOperatorStateContext.
> [ERROR]   Run 9: Exception while creating StreamOperatorStateContext.
> [INFO] 
> [ERROR]   
> FlinkKafkaConsumerBaseTest.testExplicitStateSerializerCompatibility:721 » 
> Runtime
> [ERROR]   FlinkKafkaConsumerBaseTest.testScaleDown:742->testRescaling:817 » 
> Checkpoint C...
> [ERROR]   FlinkKafkaConsumerBaseTest.testScaleUp:737->testRescaling:817 » 
> Checkpoint Cou...
> [ERROR]   UpsertKafkaDynamicTableFactoryTest.testBufferedTableSink:243 » 
> UncheckedIO jav...
> {code}
> Example stacktrace:
> {code:java}
> Test 
> testBufferedTableSink(org.apache.flink.streaming.connectors.kafka.table.UpsertKafkaDynamicTableFactoryTest)
>  failed with:
> java.io.UncheckedIOException: java.io.IOException: Serializing the source 
> elements failed: java.lang.reflect.InaccessibleObjectException: Unable to 
> make field private final java.lang.Object[] java.util.Arrays$ArrayList.a 
> accessible: module java.base does not "opens java.util" to unnamed module 
> @45b4c3a9
>   at 
> org.apache.flink.streaming.api.functions.source.FromElementsFunction.setOutputType(FromElementsFunction.java:162)
>   at 
> org.apache.flink.streaming.util.functions.StreamingFunctionUtils.trySetOutputType(StreamingFunctionUtils.java:84)
>   at 
> org.apache.flink.streaming.util.functions.StreamingFunctionUtils.setOutputType(StreamingFunctionUtils.java:60)
>   at 
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.setOutputType(AbstractUdfStreamOperator.java:146)
>   at 
> org.apache.flink.streaming.api.operators.SimpleOperatorFactory.setOutputType(SimpleOperatorFactory.java:118)
>   at 
> org.apache.flink.streaming.api.graph.StreamGraph.addOperator(StreamGraph.java:434)
>   at 
> org.apache.flink.streaming.api.graph.StreamGraph.addOperator(StreamGraph.java:402)
>   at 
> org.apache.flink.streaming.api.graph.StreamGraph.addLegacySource(StreamGraph.java:356)
>   at 
> org.apache.flink.streaming.runtime.translators.LegacySourceTransformationTranslator.translateInternal(LegacySourceTransformationTranslator.java:66)
>   at 
> org.apache.flink.streaming.runtime.translators.LegacySourceTransformationTranslator.translateForStreamingInternal(LegacySourceTransformationTranslator.java:53)
>   at 
> org.apache.flink.streaming.runtime.translators.LegacySourceTransformationTranslator.translateForStreamingInternal(LegacySourceTransformationTranslator.java:40)
>   at 
> org.apache.flink.streaming.api.graph.SimpleTransformationTranslator.translateForStreaming(SimpleTransformationTranslator.java:62)
>   at 
> org.apache.flink.streaming.api.graph.StreamGraphGenerator.translate(StreamGraphGenerator.java:860)
>   at 
> org.apache.flink.streaming.api.graph.StreamGraphGenerator.transform(StreamGraphGenerator.java:590)
>   at 
> org.apache.flink.streaming.api.graph.StreamGraphGenerator.getParentInputIds(StreamGraphGenerator.java:881)
>   at 
> org.apache.flink.streaming.api.graph.StreamGraphGenerator.translate(StreamGraphGenerator.java:839)
>   at 
> org.apache.flink.streaming.api.graph.StreamGraphGenerator.transform(StreamGraphGenerator.java:590)
>   at 
> org.apache.flink.streaming.api.graph.StreamGraphGenerator.generate(StreamGraphGenerator.java:328)
>