On the 1.8 problem stays the same (I switched off by mistake 48500 before - I switched it on again now): node connects, is listed as part of the cluster, discovery sends list of nodes with internal addresses and they are not translated on the local node, so even the node is listed as part of the cluster, exceptions appear that the client cannot connect to any of the nodes from the send list...
Now it fails on ctx.io().send(node, new GridDhtAffinityAssignmentRequest(key.get1(), key.get2()), at GridDhtAssignmentFetchFuture:185 trying to connect to port 48500 of the first node from the availableNodes list. Why is it connecting to discovery port, even it had fetched the cluster info already? <log> [02:25:57,195][INFO ][main][IgniteKernal] Non-loopback local IPs: 192.168.0.102, fe80:0:0:0:6e40:8ff:fe91:c0a4%en0, fe80:0:0:0:9485:8fff:fec1:9de8%awdl0 [02:25:57,195][INFO ][main][IgniteKernal] Enabled local MACs: 6C400891C0A4, 96858FC19DE8 [02:25:57,213][INFO ][main][IgnitePluginProcessor] Configured plugins: [02:25:57,213][INFO ][main][IgnitePluginProcessor] ^-- None [02:25:57,213][INFO ][main][IgnitePluginProcessor] [02:25:59,804][INFO ][main][TcpCommunicationSpi] Successfully bound to TCP port [port=47099, locHost=localhost/127.0.0.1] [02:26:00,451][WARN ][main][NoopCheckpointSpi] Checkpoints are disabled (to enable configure any GridCheckpointSpi implementation) [02:26:00,486][WARN ][main][GridCollisionManager] Collision resolution is disabled (all jobs will be activated upon arrival). [02:26:00,489][WARN ][main][NoopSwapSpaceSpi] Swap space is disabled. To enable use FileSwapSpaceSpi. [02:26:00,491][INFO ][main][IgniteKernal] Security status [authentication=off, tls/ssl=off] [02:26:00,776][INFO ][main][GridTcpRestProtocol] Command protocol successfully started [name=TCP binary, host=0.0.0.0/0.0.0.0, port=11211] [02:26:06,180][INFO ][main][GridCacheProcessor] Started cache [name=ignite-sys-cache, mode=REPLICATED] [02:26:06,194][INFO ][main][GridCacheProcessor] Started cache [name=ignite-atomics-sys-cache, mode=PARTITIONED] [02:26:06,214][INFO ][main][GridCacheProcessor] Started cache [name=ignite-marshaller-sys-cache, mode=REPLICATED] [02:26:22,657][WARN ][exchange-worker-#46%null%][TcpCommunicationSpi] Connect timed out (consider increasing 'failureDetectionTimeout' configuration property) [addr=/192.168.168.8:47100, failureDetectionTimeout=15000] [02:26:22,660][WARN ][exchange-worker-#46%null%][TcpCommunicationSpi] Connect timed out (consider increasing 'failureDetectionTimeout' configuration property) [addr=/127.0.0.1:47100, failureDetectionTimeout=15000] [02:26:22,661][WARN ][exchange-worker-#46%null%][TcpCommunicationSpi] Failed to connect to a remote node (make sure that destination node is alive and operating system firewall is disabled on local and remote hosts) [addrs=[/192.168.168.8:47100, /127.0.0.1:47100, 0:0:0:0:0:0:0:1%lo:47100]] [02:26:36,223][WARN ][main][GridCachePartitionExchangeManager] Failed to wait for initial partition map exchange. Possible reasons are: ^-- Transactions in deadlock. ^-- Long running transactions (ignore if this is the case). ^-- Unreleased explicit locks. [02:26:54,741][ERROR][exchange-worker-#46%null%][GridDhtAssignmentFetchFuture] Failed to request affinity assignment from remote node (will continue to another node): TcpDiscoveryNode [id=337d89f0-c14a-4b5d-8d33-b995331dcf63, addrs=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 192.168.168.8], sockAddrs=[0:0:0:0:0 :0:0:1%lo:48500, /127.0.0.1:48500, /192.168.168.8:48500], discPort=48500, order=218, intOrder=110, lastExchangeTime=1475627164474, loc=false, ver=1.8.0#20161004-sha1:a370bad1, isClient=false] class org.apache.ignite.IgniteCheckedException: Failed to send message (node may have left the grid or TCP connection cannot be established due to firewall issues) [node=TcpDiscoveryNode [id=337d89f0-c14a-4b5d-8d33-b995331dcf63, addrs=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 192.168.168.8], sockAddrs=[0:0:0: 0:0:0:0:1%lo:48500, /127.0.0.1:48500, /192.168.168.8:48500], discPort=48500, order=218, intOrder=110, lastExchangeTime=1475627164474, loc=false, ver=1.8.0#20161004-sha1:a370bad1, isClient=false], topic=TOPIC_CACHE, msg=GridDhtAffinityAssignmentRequest [topVer=AffinityTopologyVersion [topVer=276, min orTopVer=0]], policy=4] at org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1309) at org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1373) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.send(GridCacheIoManager.java:841) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtAssignmentFetchFuture.requestFromNextNode(GridDhtAssignmentFetchFuture.java:185) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtAssignmentFetchFuture.init(GridDhtAssignmentFetchFuture.java:107) at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.fetchAffinityOnJoin(CacheAffinitySharedManager.java:953) at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onClientEvent(CacheAffinitySharedManager.java:639) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onClientNodeEvent(GridDhtPartitionsExchangeFuture.java:619) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:464) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1447) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:745) Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to send message to remote node: TcpDiscoveryNode [id=337d89f0-c14a-4b5d-8d33-b995331dcf63, addrs=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 192.168.168.8], sockAddrs=[0:0:0:0:0:0:0:1%lo:48500, /127.0.0.1:48500, /192.168.168.8:48500], discPort=48 500, order=218, intOrder=110, lastExchangeTime=1475627164474, loc=false, ver=1.8.0#20161004-sha1:a370bad1, isClient=false] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2013) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:1951) at org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1304) </log> What am I missing? In 1.8 I am unable to have any DEBUG messages with the config that worked with 1.7, both log4j and ignite-log4j from 1.8 on the classpath. Cheers Krzysztof On Tue, Oct 4, 2016 at 11:27 PM, vkulichenko [via Apache Ignite Users] <ml-node+s70518n8090...@n6.nabble.com> wrote: > All nodes should run on the same version. > > What is exactly different in log4j behavior? > > -Val > > ________________________________ > If you reply to this email, your message will be added to the discussion > below: > http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cluster-Communication-with-SSH-Tunnels-tp273p8090.html > To unsubscribe from Ignite Cluster Communication with SSH Tunnels, click > here. > NAML -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cluster-Communication-with-SSH-Tunnels-tp273p8095.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
