We have a 3 node server cluster ( Issue observed in 2.7.6, could not test in 2.8.0 because I am unable to bring up the dbeaver in 2.8.0 with securityplugin enabled )(http://apache-ignite-users.70518.x6.nabble.com/2-8-0-JDBC-Thin-Client-Unable-to-load-the-tables-via-DBeaver-td31681.html)
A 4th node joins as a client with a continuous query on a Table A( Transaction_mode = transactional ). Now If I bring the client down and issue an update to the Table A within failureDetectionTimeout 30000 , I get the following error and this error brings the server down since it causes a */unhandled exception in a critical thread/*: ====================================== /*ERROR [] - Critical system error detected. Will be handled accordingly to configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=CRITICAL_ERROR, err=java.lang.NoClassDefFoundError: com/companyname/projectname/modulename/helper/ContinuousQueryHelper]] java.lang.NoClassDefFoundError: com/companyname/projectname/modulename/helper/ContinuousQueryHelper*/ at com.companyname.projectname.modulename.helper.ContinuousQueryHelper$ModuleTableRemoteFilterFactory$1.evaluate(ContinuousQueryHelper.java:289) ~[?:?] at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.filter(CacheContinuousQueryHandler.java:833) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler$2.onEntryUpdated(CacheContinuousQueryHandler.java:422) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.onEntryUpdated(CacheContinuousQueryManager.java:426) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerSet(GridCacheMapEntry.java:1584) ~[ignite-core-2.7.6.jar:2.7.6] ================================= "(err) Failed to notify listener: GridDhtTxPrepareFuture Error" =================================== Basically the server , tries to update the record on the Table A, and tries to notify Client since it had registered a continuous query for Table A. But since the Client Node has been brought down, it undeploys the remotefilterfactory lambda. Hence the server is no longer able to complete the transaction . */This also brings the server down./ * How can I resolve this issue ? ======================================= Please find the complete stack trace for this error : 2020-03-13 17:13:40,145 sys-stripe-19-#20 ERROR [] - Critical system error detected. Will be handled accordingly to configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=CRITICAL_ERROR, err=java.lang.NoClassDefFoundError: com/companyname/projectname/Module/helper/ContinuousQueryHelper]] java.lang.NoClassDefFoundError: com/companyname/projectname/Module/helper/ContinuousQueryHelper at com.companyname.projectname.Module.helper.ContinuousQueryHelper$ModuleTableRemoteFilterFactory$1.evaluate(ContinuousQueryHelper.java:289) ~[?:?] at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.filter(CacheContinuousQueryHandler.java:833) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler$2.onEntryUpdated(CacheContinuousQueryHandler.java:422) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.onEntryUpdated(CacheContinuousQueryManager.java:426) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerSet(GridCacheMapEntry.java:1584) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:741) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.localFinish(GridNearTxLocal.java:3646) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.doFinish(GridNearTxFinishFuture.java:475) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.finish(GridNearTxFinishFuture.java:425) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$25.apply(GridNearTxLocal.java:3788) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$25.apply(GridNearTxLocal.java:3782) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:385) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:349) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:337) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:497) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.GridCacheCompoundFuture.onDone(GridCacheCompoundFuture.java:56) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:476) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.distributed.near.GridNearOptimisticTxPrepareFuture.onComplete(GridNearOptimisticTxPrepareFuture.java:307) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.distributed.near.GridNearOptimisticTxPrepareFuture.onDone(GridNearOptimisticTxPrepareFuture.java:285) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.distributed.near.GridNearOptimisticTxPrepareFuture.onDone(GridNearOptimisticTxPrepareFuture.java:79) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:453) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridCompoundFuture.checkComplete(GridCompoundFuture.java:285) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridCompoundFuture.apply(GridCompoundFuture.java:144) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridCompoundFuture.apply(GridCompoundFuture.java:45) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:385) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:349) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:337) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:497) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:476) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:453) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.distributed.near.GridNearOptimisticTxPrepareFuture$MiniFuture.onResult(GridNearOptimisticTxPrepareFuture.java:1046) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.distributed.near.GridNearOptimisticTxPrepareFuture$2.apply(GridNearOptimisticTxPrepareFuture.java:612) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.distributed.near.GridNearOptimisticTxPrepareFuture$2.apply(GridNearOptimisticTxPrepareFuture.java:609) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:385) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:349) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:337) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:497) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:476) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:453) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureChainListener.applyCallback(GridFutureChainListener.java:78) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureChainListener.apply(GridFutureChainListener.java:70) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureChainListener.apply(GridFutureChainListener.java:30) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:385) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:349) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:337) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:497) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.GridCacheCompoundFuture.onDone(GridCacheCompoundFuture.java:56) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:476) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.onComplete(GridDhtTxPrepareFuture.java:1006) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.onDone(GridDhtTxPrepareFuture.java:806) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.onDone(GridDhtTxPrepareFuture.java:110) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:453) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridCompoundFuture.checkComplete(GridCompoundFuture.java:285) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridCompoundFuture.apply(GridCompoundFuture.java:144) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridCompoundFuture.apply(GridCompoundFuture.java:45) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:385) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:349) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:337) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:497) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:476) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:453) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture$MiniFuture.onResult(GridDhtTxPrepareFuture.java:1948) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.onResult(GridDhtTxPrepareFuture.java:572) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processDhtTxPrepareResponse(IgniteTxHandler.java:798) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$500(IgniteTxHandler.java:119) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$6.apply(IgniteTxHandler.java:229) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$6.apply(IgniteTxHandler.java:227) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:505) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) ~[ignite-core-2.7.6.jar:2.7.6] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_241] Caused by: java.lang.ClassNotFoundException: Failed to peer load class [class=com.companyname.projectname.Module.helper.ContinuousQueryHelper, nodeClsLdrs=\{123a731e-fc4e-46cc-a9f7-bd72fbde82fb=c8be2e4d071-123a731e-fc4e-46cc-a9f7-bd72fbde82fb}, parentClsLoader=sun.misc.Launcher$AppClassLoader@73d16e93] at org.apache.ignite.internal.managers.deployment.GridDeploymentClassLoader.sendClassRequest(GridDeploymentClassLoader.java:661) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.managers.deployment.GridDeploymentClassLoader.findClass(GridDeploymentClassLoader.java:508) ~[ignite-core-2.7.6.jar:2.7.6] at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[?:1.8.0_241] at org.apache.ignite.internal.managers.deployment.GridDeploymentClassLoader.loadClass(GridDeploymentClassLoader.java:440) ~[ignite-core-2.7.6.jar:2.7.6] ... 80 more Caused by: org.apache.ignite.IgniteCheckedException: Failed to send message (node may have left the grid or TCP connection cannot be established due to firewall issues) [node=TcpDiscoveryNode [id=123a731e-fc4e-46cc-a9f7-bd72fbde82fb, addrs=[0:0:0:0:0:0:0:1, x.x.x.44, 127.0.0.1], sockAddrs=[machinename.companyname.LOCAL/x.x.x.44:0, /0:0:0:0:0:0:0:1:0, /127.0.0.1:0], discPort=0, order=4, intOrder=4, lastExchangeTime=1584119608194, loc=false, ver=2.7.6#20190911-sha1:21f7ca41, isClient=true], topic=TOPIC_CLASSLOAD, msg=GridDeploymentRequest [rsrcName=com/companyname/projectname/Module/helper/ContinuousQueryHelper.class, ldrId=c8be2e4d071-123a731e-fc4e-46cc-a9f7-bd72fbde82fb, isUndeploy=false, nodeIds=null], policy=1] at org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1667) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.managers.communication.GridIoManager.sendToGridTopic(GridIoManager.java:1731) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.managers.deployment.GridDeploymentCommunication.sendResourceRequest(GridDeploymentCommunication.java:454) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.managers.deployment.GridDeploymentClassLoader.sendClassRequest(GridDeploymentClassLoader.java:601) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.managers.deployment.GridDeploymentClassLoader.findClass(GridDeploymentClassLoader.java:508) ~[ignite-core-2.7.6.jar:2.7.6] at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[?:1.8.0_241] at org.apache.ignite.internal.managers.deployment.GridDeploymentClassLoader.loadClass(GridDeploymentClassLoader.java:440) ~[ignite-core-2.7.6.jar:2.7.6] ... 80 more Caused by: org.apache.ignite.spi.IgniteSpiException: Failed to send message to remote node: TcpDiscoveryNode [id=123a731e-fc4e-46cc-a9f7-bd72fbde82fb, addrs=[0:0:0:0:0:0:0:1, x.x.x.44, 127.0.0.1], sockAddrs=[machinename.companyname.LOCAL/x.x.x.44:0, /0:0:0:0:0:0:0:1:0, /127.0.0.1:0], discPort=0, order=4, intOrder=4, lastExchangeTime=1584119608194, loc=false, ver=2.7.6#20190911-sha1:21f7ca41, isClient=true] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2747) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:2672) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1656) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.managers.communication.GridIoManager.sendToGridTopic(GridIoManager.java:1731) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.managers.deployment.GridDeploymentCommunication.sendResourceRequest(GridDeploymentCommunication.java:454) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.managers.deployment.GridDeploymentClassLoader.sendClassRequest(GridDeploymentClassLoader.java:601) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.managers.deployment.GridDeploymentClassLoader.findClass(GridDeploymentClassLoader.java:508) ~[ignite-core-2.7.6.jar:2.7.6] at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[?:1.8.0_241] at org.apache.ignite.internal.managers.deployment.GridDeploymentClassLoader.loadClass(GridDeploymentClassLoader.java:440) ~[ignite-core-2.7.6.jar:2.7.6] ... 80 more Caused by: org.apache.ignite.IgniteCheckedException: Failed to connect to node (is node still alive?). Make sure that each ComputeTask and cache Transaction has a timeout set in order to prevent parties from waiting forever in case of network issues [nodeId=123a731e-fc4e-46cc-a9f7-bd72fbde82fb, addrs=[/127.0.0.1:47103, /0:0:0:0:0:0:0:1:47103, machinename.companyname.LOCAL/x.x.x.44:47103]] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3459) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createNioClient(TcpCommunicationSpi.java:2987) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2870) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.access$6000(TcpCommunicationSpi.java:271) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.processDisconnect(TcpCommunicationSpi.java:4489) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.body(TcpCommunicationSpi.java:4294) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$5.body(TcpCommunicationSpi.java:2237) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) ~[ignite-core-2.7.6.jar:2.7.6] Suppressed: org.apache.ignite.IgniteCheckedException: Failed to connect to address [addr=/127.0.0.1:47103, err=Connection refused: no further information] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3462) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createNioClient(TcpCommunicationSpi.java:2987) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2870) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.access$6000(TcpCommunicationSpi.java:271) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.processDisconnect(TcpCommunicationSpi.java:4489) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.body(TcpCommunicationSpi.java:4294) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$5.body(TcpCommunicationSpi.java:2237) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) ~[ignite-core-2.7.6.jar:2.7.6] Caused by: java.net.ConnectException: Connection refused: no further information at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:1.8.0_241] at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[?:1.8.0_241] at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:111) ~[?:1.8.0_241] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3299) ~[ignite-core-2.7.6.jar:2.7.6] ... 8 more Suppressed: org.apache.ignite.IgniteCheckedException: Failed to connect to address [addr=/0:0:0:0:0:0:0:1:47103, err=Connection refused: no further information] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3462) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createNioClient(TcpCommunicationSpi.java:2987) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2870) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.access$6000(TcpCommunicationSpi.java:271) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.processDisconnect(TcpCommunicationSpi.java:4489) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.body(TcpCommunicationSpi.java:4294) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$5.body(TcpCommunicationSpi.java:2237) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) ~[ignite-core-2.7.6.jar:2.7.6] Caused by: java.net.ConnectException: Connection refused: no further information at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:1.8.0_241] at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[?:1.8.0_241] at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:111) ~[?:1.8.0_241] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3299) ~[ignite-core-2.7.6.jar:2.7.6] ... 8 more Suppressed: org.apache.ignite.IgniteCheckedException: Failed to connect to address [addr=machinename.companyname.LOCAL/x.x.x.44:47103, err=Connection refused: no further information] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3462) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createNioClient(TcpCommunicationSpi.java:2987) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2870) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.access$6000(TcpCommunicationSpi.java:271) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.processDisconnect(TcpCommunicationSpi.java:4489) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.body(TcpCommunicationSpi.java:4294) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$5.body(TcpCommunicationSpi.java:2237) ~[ignite-core-2.7.6.jar:2.7.6] at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) ~[ignite-core-2.7.6.jar:2.7.6] Caused by: java.net.ConnectException: Connection refused: no further information at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:1.8.0_241] at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[?:1.8.0_241] at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:111) ~[?:1.8.0_241] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3299) ~[ignite-core-2.7.6.jar:2.7.6] ... 8 more 2 -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/