[
https://issues.apache.org/jira/browse/IGNITE-23202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikita Amelchev reassigned IGNITE-23202:
----------------------------------------
Assignee: Nikita Amelchev (was: Nikolay Izhikov)
> Services: node crashes when node filter class was not found in class path
> -------------------------------------------------------------------------
>
> Key: IGNITE-23202
> URL: https://issues.apache.org/jira/browse/IGNITE-23202
> Project: Ignite
> Issue Type: Bug
> Components: managed services
> Affects Versions: 2.16
> Reporter: Ilya Shishkov
> Assignee: Nikita Amelchev
> Priority: Critical
> Labels: ise
> Fix For: 2.17
>
> Attachments: IGNITE-23202.patch
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> How to reproduce:
> # Start node(s) (eg. via {{ignite.sh}})
> # Start another node in order to deploy service with service node filter
> which is not present in classpath of server nodes. As example see patch of
> {{ServicesExample.java}}: [^IGNITE-23202.patch]
> # Some nodes (may be all) will fail.
> Stacktraces differ for anonymous class and lamda:
> {code:title=Lambda}
> [20:36:56,908][INFO][exchange-worker-#53][time] Started exchange init
> [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], crd=true,
> evt=NODE_FAILED, evtNode=97ae84a3-094e-47b7-92db-8f059f8a2512,
> customEvt=null, allowMerge=true, exchangeFreeSwitch=false]
> [20:36:56,909][SEVERE][tcp-disco-msg-worker-[97ae84a3
> 0:0:0:0:0:0:0:1%lo:47500]-#2-#45][] Critical system error detected. Will be
> handled accordingly to configured handler [hnd=StopNodeOrHaltFailureHandler
> [tryStop=false, timeout=0, super=AbstractFailureHandler
> [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
> SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext
> [type=SYSTEM_WORKER_TERMINATION, err=class o.a.i.IgniteException: Failed to
> unmarshal discovery custom message: TcpDiscoveryCustomEventMessage [msg=null,
> super=TcpDiscoveryAbstractMessage
> [sndNodeId=a952f13f-349e-4a69-bff8-cbcaeca9b1eb,
> id=2213f47e191-34c6795a-e1fe-40d6-b45d-229bbe2c358d,
> verifierNodeId=a952f13f-349e-4a69-bff8-cbcaeca9b1eb, topVer=4, pendingIdx=0,
> failedNodes=null, isClient=false]]]]
> class org.apache.ignite.IgniteException: Failed to unmarshal discovery custom
> message: TcpDiscoveryCustomEventMessage [msg=null,
> super=TcpDiscoveryAbstractMessage
> [sndNodeId=a952f13f-349e-4a69-bff8-cbcaeca9b1eb,
> id=2213f47e191-34c6795a-e1fe-40d6-b45d-229bbe2c358d,
> verifierNodeId=a952f13f-349e-4a69-bff8-cbcaeca9b1eb, topVer=4, pendingIdx=0,
> failedNodes=null, isClient=false]]
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.notifyDiscoveryListener(ServerImpl.java:6307)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processCustomMessage(ServerImpl.java:6097)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:3237)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2892)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7980)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:3066)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7911)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
> deserialize object with given class loader:
> jdk.internal.loader.ClassLoaders$AppClassLoader@55054057
> at
> org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:132)
> at
> org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:139)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:80)
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10807)
> at
> org.apache.ignite.spi.discovery.tcp.messages.TcpDiscoveryCustomEventMessage.message(TcpDiscoveryCustomEventMessage.java:103)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.notifyDiscoveryListener(ServerImpl.java:6304)
> ... 8 more
> Caused by: java.lang.ClassCastException: cannot assign instance of
> java.lang.invoke.SerializedLambda to field
> org.apache.ignite.services.ServiceConfiguration.nodeFilter of type
> org.apache.ignite.lang.IgnitePredicate in instance of
> org.apache.ignite.internal.processors.service.LazyServiceConfiguration
> at
> java.base/java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(ObjectStreamClass.java:2076)
> at
> java.base/java.io.ObjectStreamClass$FieldReflector.checkObjectFieldValueTypes(ObjectStreamClass.java:2039)
> at
> java.base/java.io.ObjectStreamClass.checkObjFieldValueTypes(ObjectStreamClass.java:1293)
> at
> java.base/java.io.ObjectInputStream.defaultCheckFieldValues(ObjectInputStream.java:2512)
> at
> java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2419)
> at
> java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2228)
> at
> java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1687)
> at
> java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2496)
> at
> java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2390)
> at
> java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2228)
> at
> java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1687)
> at
> java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:489)
> at
> java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:447)
> at java.base/java.util.ArrayList.readObject(ArrayList.java:929)
> at
> java.base/jdk.internal.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> java.base/java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1046)
> at
> java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2357)
> at
> java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2228)
> at
> java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1687)
> at
> java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2496)
> at
> java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2390)
> at
> java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2228)
> at
> java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1687)
> at
> java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2496)
> at
> java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2390)
> at
> java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2228)
> at
> java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1687)
> at
> java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:489)
> at
> java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:447)
> at
> org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:124)
> ... 13 more
> {code}
> {code:title=Anonymous class}
> [20:40:54,949][SEVERE][tcp-disco-msg-worker-[87a727e0
> 0:0:0:0:0:0:0:1%lo:47501 crd]-#2-#45][] Critical system error detected. Will
> be handled accordingly to configured handler
> [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0,
> super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet
> [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]],
> failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION, err=class
> o.a.i.IgniteException: Failed to unmarshal discovery custom message:
> TcpDiscoveryCustomEventMessage [msg=null, super=TcpDiscoveryAbstractMessage
> [sndNodeId=1c2bff3c-72c4-4cfc-832c-50bda9af1b10,
> id=4b4d257e191-1c2bff3c-72c4-4cfc-832c-50bda9af1b10,
> verifierNodeId=7146786b-cd3d-4ab7-aa7f-28293a1e1de8, topVer=3, pendingIdx=0,
> failedNodes=null, isClient=false]]]]
> class org.apache.ignite.IgniteException: Failed to unmarshal discovery custom
> message: TcpDiscoveryCustomEventMessage [msg=null,
> super=TcpDiscoveryAbstractMessage
> [sndNodeId=1c2bff3c-72c4-4cfc-832c-50bda9af1b10,
> id=4b4d257e191-1c2bff3c-72c4-4cfc-832c-50bda9af1b10,
> verifierNodeId=7146786b-cd3d-4ab7-aa7f-28293a1e1de8, topVer=3, pendingIdx=0,
> failedNodes=null, isClient=false]]
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.notifyDiscoveryListener(ServerImpl.java:6307)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processCustomMessage(ServerImpl.java:6097)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:3237)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2892)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7980)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:3066)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7911)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to find
> class with given class loader for unmarshalling (make sure same versions of
> all classes are available on all nodes or enable peer-class-loading)
> [clsLdr=jdk.internal.loader.ClassLoaders$AppClassLoader@55054057,
> cls=org.apache.ignite.examples.servicegrid.ServicesExample$1]
> at
> org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:129)
> at
> org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:139)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:80)
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10807)
> at
> org.apache.ignite.spi.discovery.tcp.messages.TcpDiscoveryCustomEventMessage.message(TcpDiscoveryCustomEventMessage.java:103)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.notifyDiscoveryListener(ServerImpl.java:6304)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.ignite.examples.servicegrid.ServicesExample$1
> at
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
> at
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
> at java.base/java.lang.Class.forName0(Native Method)
> at java.base/java.lang.Class.forName(Class.java:398)
> at
> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:9349)
> at
> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:9287)
> at
> org.apache.ignite.marshaller.jdk.JdkMarshallerObjectInputStream.resolveClass(JdkMarshallerObjectInputStream.java:59)
> at
> java.base/java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2003)
> at
> java.base/java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1870)
> at
> java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2201)
> at
> java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1687)
> at
> java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2496)
> at
> java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2390)
> at
> java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2228)
> at
> java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1687)
> at
> java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2496)
> at
> java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2390)
> at
> java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2228)
> at
> java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1687)
> at
> java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:489)
> at
> java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:447)
> at java.base/java.util.ArrayList.readObject(ArrayList.java:929)
> at
> java.base/jdk.internal.reflect.GeneratedMethodAccessor10.invoke(Unknown
> Source)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> java.base/java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1046)
> at
> java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2357)
> at
> java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2228)
> at
> java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1687)
> at
> java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2496)
> at
> java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2390)
> at
> java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2228)
> at
> java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1687)
> at
> java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2496)
> at
> java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2390)
> at
> java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2228)
> at
> java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1687)
> at
> java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:489)
> at
> java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:447)
> at
> org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:124)
> ... 13 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
