NPE during printing failure information

Andrey Davydov Thu, 11 Jun 2020 01:38:15 -0700

Hello,

We start test our system on Ignite 2.8.1 and got very strange log. As I understand, It was error in system (on 2.7.6 this test works without any problem, I will investigate it later) and it was NPE during handling of this error:

[01:08:11] Configured failure handler: [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]]]

[01:08:11] Message queue limit is set to 0 which may lead to potential OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender and receiver sides.

2020-06-11 01:08:22,677 [tcp-disco-msg-worker-[crd]-#1225%TestNode-0%] WARN :119 - Possible failure suppressed accordingly to a configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=grid-nio-worker-tcp-comm-3, igniteInstanceName=TestNode-0, finished=false, heartbeatTs=1591837692022]]]

org.apache.ignite.IgniteException: GridWorker [name=grid-nio-worker-tcp-comm-3, igniteInstanceName=TestNode-0, finished=false, heartbeatTs=1591837692022]

at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$3.apply(IgnitionEx.java:1810) ~[ignite-core-2.8.1.jar:2.8.1]

at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$3.apply(IgnitionEx.java:1805) ~[ignite-core-2.8.1.jar:2.8.1]

at org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:234) ~[ignite-core-2.8.1.jar:2.8.1]

at org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297) ~[ignite-core-2.8.1.jar:2.8.1]

at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2858) ~[ignite-core-2.8.1.jar:2.8.1]

at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7759) [ignite-core-2.8.1.jar:2.8.1]

at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2946) [ignite-core-2.8.1.jar:2.8.1]

at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) [ignite-core-2.8.1.jar:2.8.1]

at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7697) [ignite-core-2.8.1.jar:2.8.1]

at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:61) [ignite-core-2.8.1.jar:2.8.1]

[01:08:22] Possible failure suppressed accordingly to a configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=grid-nio-worker-tcp-comm-3, igniteInstanceName=TestNode-0, finished=false, heartbeatTs=1591837692022]]]

2020-06-11 01:08:22,685 [tcp-disco-msg-worker-[crd]-#1225%TestNode-0%] WARN :119 - Possible failure suppressed accordingly to a configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=grid-nio-worker-tcp-comm-0, igniteInstanceName=TestNode-0, finished=false, heartbeatTs=1591837692022]]]

org.apache.ignite.IgniteException: GridWorker [name=grid-nio-worker-tcp-comm-0, igniteInstanceName=TestNode-0, finished=false, heartbeatTs=1591837692022]