[
https://issues.apache.org/jira/browse/FLINK-34581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824731#comment-17824731
]
Martijn Visser commented on FLINK-34581:
----------------------------------------
I've downgraded this from Blocker to Major ticket, and I actually think this
isn't necessarily a bug, but more a user questions. Also, keep in mind that
you're using the Iceberg Sink, but that's not owned by the Flink project but by
the Iceberg project. It could also be that there's actually a bug/missing
feature in that implementation.
> streaming code throws java.lang.reflect.InaccessibleObjectException
> -------------------------------------------------------------------
>
> Key: FLINK-34581
> URL: https://issues.apache.org/jira/browse/FLINK-34581
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
> Affects Versions: 1.18.1
> Reporter: Henning Schmiedehausen
> Priority: Critical
>
> I have a pretty simple test pipeline (read a bunch of tables from Apache
> Kafka, join and project them, then write to Apache Iceberg) that I run
> locally with Junit5 and the MiniClusterExtension. This works ok with Java 11
> and Flink 1.17.2 and Flink 1.18.1.
> With Java 17, I see
> {{[WARN ] IcebergStreamWriter (1/1)#1
> (78d9251dbbab3aae84bf303dfc080d23_626c1e687bcaad0c13c507629a5894a8_0_1)
> switched from RUNNING to FAILED with failure cause:}}
> {{java.io.IOException: Could not perform checkpoint 2 for operator
> IcebergStreamWriter (1/1)#1.}}
> {{at
> org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:1275)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.CheckpointBarrierHandler.notifyCheckpoint(CheckpointBarrierHandler.java:147)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.triggerCheckpoint(SingleCheckpointBarrierHandler.java:287)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.access$100(SingleCheckpointBarrierHandler.java:64)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler$ControllerImpl.triggerGlobalCheckpoint(SingleCheckpointBarrierHandler.java:488)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.AbstractAlignedBarrierHandlerState.triggerGlobalCheckpoint(AbstractAlignedBarrierHandlerState.java:74)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.AbstractAlignedBarrierHandlerState.barrierReceived(AbstractAlignedBarrierHandlerState.java:66)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.lambda$processBarrier$2(SingleCheckpointBarrierHandler.java:234)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.markCheckpointAlignedAndTransformState(SingleCheckpointBarrierHandler.java:262)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.processBarrier(SingleCheckpointBarrierHandler.java:231)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.CheckpointedInputGate.handleEvent(CheckpointedInputGate.java:181)}}
> {{at
> org.apache.flink.streaming.runtime.io.checkpointing.CheckpointedInputGate.pollNext(CheckpointedInputGate.java:159)}}
> {{at
> org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:118)}}
> {{at
> org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)}}
> {{at
> org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:562)}}
> {{at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)}}
> {{at
> org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:858)}}
> {{at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:807)}}
> {{at
> org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953)}}
> {{at
> org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932)}}
> {{at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746)}}
> {{at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)}}
> {{at java.base/java.lang.Thread.run(Thread.java:840)}}
> {{Caused by: java.lang.RuntimeException:
> java.lang.reflect.InaccessibleObjectException: Unable to make field private
> final java.lang.Object[] java.util.Arrays$ArrayList.a accessible: module
> java.base does not "opens java.util" to unnamed module @6af93788}}
> {{at
> com.twitter.chill.java.ArraysAsListSerializer.<init>(ArraysAsListSerializer.java:69)}}
> {{at
> org.apache.flink.api.java.typeutils.runtime.kryo.FlinkChillPackageRegistrar.registerSerializers(FlinkChillPackageRegistrar.java:67)}}
> {{at
> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.getKryoInstance(KryoSerializer.java:513)}}
> {{at
> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.checkKryoInitialized(KryoSerializer.java:522)}}
> {{at
> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.serialize(KryoSerializer.java:348)}}
> {{at
> org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.serialize(StreamElementSerializer.java:165)}}
> {{at
> org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.serialize(StreamElementSerializer.java:43)}}
> {{at
> org.apache.flink.runtime.plugable.SerializationDelegate.write(SerializationDelegate.java:54)}}
> {{at
> org.apache.flink.runtime.io.network.api.writer.RecordWriter.serializeRecord(RecordWriter.java:141)}}
> {{at
> org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:107)}}
> {{at
> org.apache.flink.runtime.io.network.api.writer.ChannelSelectorRecordWriter.emit(ChannelSelectorRecordWriter.java:55)}}
> {{at
> org.apache.flink.streaming.runtime.io.RecordWriterOutput.pushToRecordWriter(RecordWriterOutput.java:134)}}
> {{at
> org.apache.flink.streaming.runtime.io.RecordWriterOutput.collectAndCheckIfChained(RecordWriterOutput.java:114)}}
> {{at
> org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:95)}}
> {{at
> org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:48)}}
> {{at
> org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:59)}}
> {{at
> org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:31)}}
> {{at
> org.apache.iceberg.flink.sink.IcebergStreamWriter.flush(IcebergStreamWriter.java:113)}}
> {{at
> org.apache.iceberg.flink.sink.IcebergStreamWriter.prepareSnapshotPreBarrier(IcebergStreamWriter.java:66)}}
> {{at
> org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.prepareSnapshotPreBarrier(RegularOperatorChain.java:89)}}
> {{at
> org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointState(SubtaskCheckpointCoordinatorImpl.java:322)}}
> {{at
> org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$15(StreamTask.java:1318)}}
> {{at
> org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)}}
> {{at
> org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:1306)}}
> {{at
> org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:1263)}}
> {{... 22 more}}
> {{Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make
> field private final java.lang.Object[] java.util.Arrays$ArrayList.a
> accessible: module java.base does not "opens java.util" to unnamed module
> @6af93788}}
> {{at
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)}}
> {{at
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)}}
> {{at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)}}
> {{at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)}}
> {{at
> com.twitter.chill.java.ArraysAsListSerializer.<init>(ArraysAsListSerializer.java:67)}}
> {{... 46 more}}
> This is blocking us right now from upgrading to Java 17.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
