Full stack trace: 2024-01-18 18:27:12.889 o.a.s.d.n.Nimbus pool-29-thread-22 [WARN] get blob meta exception. org.apache.storm.utils.WrappedKeyNotFoundException: topology-A-7-1704896101-stormjar.jar at org.apache.storm.blobstore. LocalFsBlobStore.getStoredBlobMeta(LocalFsBlobStore.java:258) ~[storm-server -2.3.0.jar:2.3.0] at org.apache.storm.blobstore.LocalFsBlobStore.getBlobMeta (LocalFsBlobStore.java:288) ~[storm-server-2.3.0.jar:2.3.0] at org.apache. storm.daemon.nimbus.Nimbus.getBlobMeta(Nimbus.java:3815) [storm-server-2.3.0 .jar:2.3.0] at org.apache.storm.generated.Nimbus$Processor$getBlobMeta. getResult(Nimbus.java:4278) [storm-client-2.3.0.jar:2.3.0] at org.apache. storm.generated.Nimbus$Processor$getBlobMeta.getResult(Nimbus.java:4257) [ storm-client-2.3.0.jar:2.3.0] at org.apache.storm.thrift.ProcessFunction. process(ProcessFunction.java:38) [storm-shaded-deps-2.3.0.jar:2.3.0] at org. apache.storm.thrift.TBaseProcessor.process(TBaseProcessor.java:38) [storm- shaded-deps-2.3.0.jar:2.3.0] at org.apache.storm.security.auth. SimpleTransportPlugin$SimpleWrapProcessor.process(SimpleTransportPlugin.java :172) [storm-client-2.3.0.jar:2.3.0] at org.apache.storm.thrift.server. AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java: 524) [storm-shaded-deps-2.3.0.jar:2.3.0] at org.apache.storm.thrift.server. Invocation.run(Invocation.java:18) [storm-shaded-deps-2.3.0.jar:2.3.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java: 1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run( ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:834) [?:?]
On Thu, Jan 18, 2024 at 11:26 AM Bipin Prasad <bipin_pra...@yahoo.com.invalid> wrote: > Can you post the full stack trace? I want to confirm that this is logged > while trying to obtain the heartbeat.This message is a warning message and > is not expected to shutdown nimbus. > > > Sent from Yahoo Mail for iPhone > > > On Thursday, January 18, 2024, 11:19 AM, Devendar Rao < > devendar.gu...@gmail.com> wrote: > > Hi, > We're constantly seeing issues in storm 2.3.0 with blobs with each > topology deployment. Supervisor/nimbus dies after seeing the below > exceptions. > Is this a known issue? Are we hitting any blob cache size limits? > Exceptions: > "Could not download the blob with key:" > o.a.s.d.n.Nimbus pool-29-thread-22 [WARN] > .org.apache.storm.utils.WrappedKeyNotFoundException > > > Can someone please shed some light on this? > Thanks > > > >
