We are also seeing something similar:
2024-04-26 16:30:44,401 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Power
Consumption:power_consumption -> Ingest Power Consumption -> PopSysFields ->
WindowingWatermarkPreCheck (1/1)
(cb8c425b6463b1ade9b8359c0514386b_28bc590bb7896e1df191c306d7cb6d23_0_11)
switched from DEPLOYING to FAILED on
f-a9ad4438-cddf-512f-94c6-c5f921f66078-taskmanager-1-1 @ 10.111.164.30
(dataPort=42621).
java.lang.IllegalStateException: The library registration references a
different set of library BLOBs than previous registrations for this job:
old:[p-1b01ac6374a137939ffa18432714b7c9af30dc3f-522999dc4412a76d33728a97225de573]
new:[p-1b01ac6374a137939ffa18432714b7c9af30dc3f-819e2ecffa2f22c5e0f4d88ef5789421]
at
org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$ResolvedClassLoader.verifyClassLoader(BlobLibraryCacheManager.java:437)
~[flink-dist-1.19.0.jar:1.19.0]
at
org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$ResolvedClassLoader.access$600(BlobLibraryCacheManager.java:373)
~[flink-dist-1.19.0.jar:1.19.0]
at
org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$LibraryCacheEntry.getOrResolveClassLoader(BlobLibraryCacheManager.java:249)
~[flink-dist-1.19.0.jar:1.19.0]
at
org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$LibraryCacheEntry.access$1200(BlobLibraryCacheManager.java:210)
~[flink-dist-1.19.0.jar:1.19.0]
at
org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$DefaultClassLoaderLease.getOrResolveClassLoader(BlobLibraryCacheManager.java:350)
~[flink-dist-1.19.0.jar:1.19.0]
at
org.apache.flink.runtime.taskmanager.Task.createUserCodeClassloader(Task.java:1047)
~[flink-dist-1.19.0.jar:1.19.0]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:628)
~[flink-dist-1.19.0.jar:1.19.0]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566)
~[flink-dist-1.19.0.jar:1.19.0]
at java.lang.Thread.run(Unknown Source) ~[?:?]
2024-04-26 16:30:44,402 INFO
org.apache.flink.runtime.scheduler.adaptive.AdaptiveScheduler [] - Restarting
job.
java.lang.IllegalStateException: The library registration references a
different set of library BLOBs than previous registrations for this job:
old:[p-1b01ac6374a137939ffa18432714b7c9af30dc3f-522999dc4412a76d33728a97225de573]
new:[p-1b01ac6374a137939ffa18432714b7c9af30dc3f-819e2ecffa2f22c5e0f4d88ef5789421]
at
org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$ResolvedClassLoader.verifyClassLoader(BlobLibraryCacheManager.java:437)
~[flink-dist-1.19.0.jar:1.19.0]
at
org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$ResolvedClassLoader.access$600(BlobLibraryCacheManager.java:373)
~[flink-dist-1.19.0.jar:1.19.0]
at
org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$LibraryCacheEntry.getOrResolveClassLoader(BlobLibraryCacheManager.java:249)
~[flink-dist-1.19.0.jar:1.19.0]
at
org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$LibraryCacheEntry.access$1200(BlobLibraryCacheManager.java:210)
~[flink-dist-1.19.0.jar:1.19.0]
at
org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$DefaultClassLoaderLease.getOrResolveClassLoader(BlobLibraryCacheManager.java:350)
~[flink-dist-1.19.0.jar:1.19.0]
at
org.apache.flink.runtime.taskmanager.Task.createUserCodeClassloader(Task.java:1047)
~[flink-dist-1.19.0.jar:1.19.0]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:628)
~[flink-dist-1.19.0.jar:1.19.0]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566)
~[flink-dist-1.19.0.jar:1.19.0]
at java.lang.Thread.run(Unknown Source) ~[?:?]
Flink 1.19, operator 1.9.0
. . . . . . . . . . . . . . . . . . . . .
Maxim Senin
Senior Backend Engineer
COGILITY<http://cogility.com/>
From: prashant parbhane <[email protected]>
Date: Tuesday, April 23, 2024 at 11:09 PM
To: [email protected] <[email protected]>
Subject: Regarding java.lang.IllegalStateException
Hello,
We have been facing this weird issue, where we are getting below exception and
the job is getting restarted with new task managers. We are using flink 1.17.
Same job works fine with lower number of task managers. (<10)
java.lang.IllegalStateException: The library registration references a
different set of library BLOBs than previous registrations for this job:
old:[p-ecf88f3c5c35842d7bca235cfd7592c53f9fdbe0-e06122dd51cccba866932318dc031d68]
new:[p-ecf88f3c5c35842d7bca235cfd7592c53f9fdbe0-7651f69109c915ac830aa42ce2ab67f0]
at
org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$ResolvedClassLoader.verifyClassLoader(BlobLibraryCacheManager.java:419)
at
org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$ResolvedClassLoader.access$500(BlobLibraryCacheManager.java:359)
at
org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$LibraryCacheEntry.getOrResolveClassLoader(BlobLibraryCacheManager.java:235)
at
org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$LibraryCacheEntry.access$1100(BlobLibraryCacheManager.java:202)
at
org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$DefaultClassLoaderLease.getOrResolveClassLoader(BlobLibraryCacheManager.java:336)
at
org.apache.flink.runtime.taskmanager.Task.createUserCodeClassloader(Task.java:1041)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:624)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)
at java.base/java.lang.Thread.run(Unknown Source)
Thanks,
Prashant
________________________________
COGILITY SOFTWARE CORPORATION LEGAL DISCLAIMER: The information in this email
is confidential and is intended solely for the addressee. Access to this email
by anyone else is unauthorized. If you are not the intended recipient, any
disclosure, copying, distribution or any action taken or omitted to be taken in
reliance on it, is prohibited and may be unlawful.
