Matheus Felisberto created FLINK-32212:
------------------------------------------
Summary: Job restarting indefinitely after an
IllegalStateException from BlobLibraryCacheManager
Key: FLINK-32212
URL: https://issues.apache.org/jira/browse/FLINK-32212
Project: Flink
Issue Type: Bug
Components: Runtime / Task
Affects Versions: 1.16.1
Environment: I'm running my workload on Kubernetes using Operator v1.4
and Flink 1.16
Reporter: Matheus Felisberto
After running for a few hours the job starts to throw IllegalStateException and
I can't figure out why. The jar is built-in into the docker image, hence is
defined accordingly with the Operator's documentation:
{code:java}
// jarURI: local:///opt/flink/usrlib/my-job.jar {code}
I've tried to move it into /opt/flink/lib/my-job.jar but it didn't work either.
{code:java}
// Source: my-topic (1/2)#30587
(b82d2c7f9696449a2d9f4dc298c0a008_bc764cd8ddf7a0cff126f51c16239658_0_30587)
switched from DEPLOYING to FAILED with failure cause:
java.lang.IllegalStateException: The library registration references a
different set of library BLOBs than previous registrations for this job:
old:[p-5d91888083d38a3ff0b6c350f05a3013632137c6-7237ecbb12b0b021934b0c81aef78396]
new:[p-5d91888083d38a3ff0b6c350f05a3013632137c6-943737c6790a3ec6870cecd652b956c2]
at
org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$ResolvedClassLoader.verifyClassLoader(BlobLibraryCacheManager.java:419)
at
org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$ResolvedClassLoader.access$500(BlobLibraryCacheManager.java:359)
at
org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$LibraryCacheEntry.getOrResolveClassLoader(BlobLibraryCacheManager.java:235)
at
org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$LibraryCacheEntry.access$1100(BlobLibraryCacheManager.java:202)
at
org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$DefaultClassLoaderLease.getOrResolveClassLoader(BlobLibraryCacheManager.java:336)
at
org.apache.flink.runtime.taskmanager.Task.createUserCodeClassloader(Task.java:1024)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:612)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
at java.base/java.lang.Thread.run(Unknown Source) {code}
If there is any other information that can help to identify the problem, please
let me know.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
