[
https://issues.apache.org/jira/browse/FLINK-17822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17112350#comment-17112350
]
Andrey Zagrebin commented on FLINK-17822:
-----------------------------------------
FLINK-15758 did not export jdk.internal.misc package used by reflection for GC
cleaners of managed memory. Our PR CI does not run Java 11 tests atm.
The package has to be exported by a JVM runtime arg: --add-opens
java.base/jdk.internal.misc=ALL-UNNAMED;
If this arg is set for Java 8, it fails the JVM process.
Therefore, the fix is complicated as we have to do it also for e.g. Yarn CLI
where client and cluster may run different Java versions.
An alternative quicker fix is to call directly the private method (has to be
made accessible via reflection):
- java.lang.ref.Reference.tryHandlePending(false) // for Java 8
- java.lang.ref.Reference.waitForReferenceProcessing() // for Java 11
This though leads to an annoying warning for Java 11:
{code:java}
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by
org.apache.flink.util.JavaGcCleanerWrapper$PendingCleanersRunnerProvider
(file:/Users/azagrebin/projects/flink/flink-core/target/classes/) to method
java.lang.ref.Reference.waitForReferenceProcessing()
WARNING: Please consider reporting this to the maintainers of
org.apache.flink.util.JavaGcCleanerWrapper$PendingCleanersRunnerProvider
WARNING: Use --illegal-access=warn to enable warnings of further illegal
reflective access operations
WARNING: All illegal access operations will be denied in a future release
{code}
We can do the quick fix and think how to fix the warning in a follow-up.
> Nightly Flink CLI end-to-end test failed with
> "JavaGcCleanerWrapper$PendingCleanersRunner cannot access class
> jdk.internal.misc.SharedSecrets" in Java 11
> ----------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-17822
> URL: https://issues.apache.org/jira/browse/FLINK-17822
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task, Tests
> Affects Versions: 1.11.0
> Reporter: Dian Fu
> Assignee: Andrey Zagrebin
> Priority: Blocker
> Labels: test-stability
> Fix For: 1.11.0
>
>
> Instance:
> https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_apis/build/builds/1887/logs/600
> {code}
> 2020-05-19T21:59:39.8829043Z 2020-05-19 21:59:25,193 ERROR
> org.apache.flink.util.JavaGcCleanerWrapper [] - FATAL
> UNEXPECTED - Failed to invoke waitForReferenceProcessing
> 2020-05-19T21:59:39.8829849Z java.lang.IllegalAccessException: class
> org.apache.flink.util.JavaGcCleanerWrapper$PendingCleanersRunner cannot
> access class jdk.internal.misc.SharedSecrets (in module java.base) because
> module java.base does not export jdk.internal.misc to unnamed module @54e3658c
> 2020-05-19T21:59:39.8830707Z at
> jdk.internal.reflect.Reflection.newIllegalAccessException(Reflection.java:361)
> ~[?:?]
> 2020-05-19T21:59:39.8831166Z at
> java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:591)
> ~[?:?]
> 2020-05-19T21:59:39.8831744Z at
> java.lang.reflect.Method.invoke(Method.java:558) ~[?:?]
> 2020-05-19T21:59:39.8832596Z at
> org.apache.flink.util.JavaGcCleanerWrapper$PendingCleanersRunner.getJavaLangRefAccess(JavaGcCleanerWrapper.java:362)
> ~[flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8833667Z at
> org.apache.flink.util.JavaGcCleanerWrapper$PendingCleanersRunner.tryRunPendingCleaners(JavaGcCleanerWrapper.java:351)
> ~[flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8834712Z at
> org.apache.flink.util.JavaGcCleanerWrapper$CleanerManager.tryRunPendingCleaners(JavaGcCleanerWrapper.java:207)
> ~[flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8835686Z at
> org.apache.flink.util.JavaGcCleanerWrapper.tryRunPendingCleaners(JavaGcCleanerWrapper.java:158)
> ~[flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8836652Z at
> org.apache.flink.runtime.memory.UnsafeMemoryBudget.reserveMemory(UnsafeMemoryBudget.java:94)
> ~[flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8838033Z at
> org.apache.flink.runtime.memory.UnsafeMemoryBudget.verifyEmpty(UnsafeMemoryBudget.java:64)
> ~[flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8839259Z at
> org.apache.flink.runtime.memory.MemoryManager.verifyEmpty(MemoryManager.java:172)
> ~[flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8840148Z at
> org.apache.flink.runtime.taskexecutor.slot.TaskSlot.verifyMemoryFreed(TaskSlot.java:311)
> ~[flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8841035Z at
> org.apache.flink.runtime.taskexecutor.slot.TaskSlot.lambda$closeAsync$1(TaskSlot.java:301)
> ~[flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8841603Z at
> java.util.concurrent.CompletableFuture.uniRunNow(CompletableFuture.java:815)
> ~[?:?]
> 2020-05-19T21:59:39.8842069Z at
> java.util.concurrent.CompletableFuture.uniRunStage(CompletableFuture.java:799)
> ~[?:?]
> 2020-05-19T21:59:39.8842844Z at
> java.util.concurrent.CompletableFuture.thenRun(CompletableFuture.java:2121)
> ~[?:?]
> 2020-05-19T21:59:39.8843828Z at
> org.apache.flink.runtime.taskexecutor.slot.TaskSlot.closeAsync(TaskSlot.java:300)
> ~[flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8844790Z at
> org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl.freeSlotInternal(TaskSlotTableImpl.java:404)
> ~[flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8845754Z at
> org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl.freeSlot(TaskSlotTableImpl.java:365)
> ~[flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8846842Z at
> org.apache.flink.runtime.taskexecutor.TaskExecutor.freeSlotInternal(TaskExecutor.java:1589)
> ~[flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8847711Z at
> org.apache.flink.runtime.taskexecutor.TaskExecutor.freeSlot(TaskExecutor.java:967)
> ~[flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8848295Z at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
> 2020-05-19T21:59:39.8848732Z at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:?]
> 2020-05-19T21:59:39.8849228Z at
> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:?]
> 2020-05-19T21:59:39.8849669Z at
> java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
> 2020-05-19T21:59:39.8850656Z at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:284)
> ~[flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8851589Z at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:199)
> ~[flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8852497Z at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:152)
> ~[flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8853486Z at
> akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26)
> [flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8854447Z at
> akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21)
> [flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8855190Z at
> scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
> [flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8855958Z at
> akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21)
> [flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8856723Z at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170)
> [flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8857663Z at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
> [flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8858452Z at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
> [flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8859214Z at
> akka.actor.Actor$class.aroundReceive(Actor.scala:517)
> [flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8860030Z at
> akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225)
> [flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8860825Z at
> akka.actor.ActorCell.receiveMessage(ActorCell.scala:592)
> [flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8861706Z at
> akka.actor.ActorCell.invoke(ActorCell.scala:561)
> [flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8862652Z at
> akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258)
> [flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8863414Z at akka.dispatch.Mailbox.run(Mailbox.scala:225)
> [flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8864132Z at
> akka.dispatch.Mailbox.exec(Mailbox.scala:235)
> [flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8864940Z at
> akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> [flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8865931Z at
> akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> [flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8866947Z at
> akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> [flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8867921Z at
> akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> [flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
> 2020-05-19T21:59:39.8868777Z 2020-05-19 21:59:25,242 INFO
> org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService [] - Remove job
> c968ba482982e36e3e30f058c488ac3f from job leader monitoring.
> 2020-05-19T21:59:39.8869653Z 2020-05-19 21:59:25,257 INFO
> org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Close
> JobManager connection for job c968ba482982e36e3e30f058c488ac3f.
> 2020-05-19T21:59:39.8870178Z WARNING: An illegal reflective access operation
> has occurred
> 2020-05-19T21:59:39.8871620Z WARNING: Illegal reflective access by
> org.apache.flink.shaded.akka.org.jboss.netty.util.internal.ByteBufferUtil
> (file:/home/vsts/work/1/s/flink-dist/target/flink-1.12-SNAPSHOT-bin/flink-1.12-SNAPSHOT/lib/flink-dist_2.11-1.12-SNAPSHOT.jar)
> to method java.nio.DirectByteBuffer.cleaner()
> 2020-05-19T21:59:39.8872448Z WARNING: Please consider reporting this to the
> maintainers of
> org.apache.flink.shaded.akka.org.jboss.netty.util.internal.ByteBufferUtil
> 2020-05-19T21:59:39.8873159Z WARNING: Use --illegal-access=warn to enable
> warnings of further illegal reflective access operations
> 2020-05-19T21:59:39.8873558Z WARNING: All illegal access operations will be
> denied in a future release
> 2020-05-19T21:59:39.8874019Z Checking for non-empty .out files...
> 2020-05-19T21:59:39.8874367Z No non-empty .out files.
> 2020-05-19T21:59:39.8874528Z
> 2020-05-19T21:59:39.8875304Z [FAIL] 'Flink CLI end-to-end test' failed after
> 0 minutes and 39 seconds! Test exited with exit code 0 but the logs contained
> errors, exceptions or non-empty .out files
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
