Hello again, my problem's root was using Scaleway instances as server nodes . Scaleway developer instances do not know about their Public IPs , thus they were sending local addresses in the addr and sockAddr arrays. When I set up my server nodes on digital ocean instances , everything was solved.
example from scaleway , which is not working: addrs=ArrayList [0:0:0:0:0:0:0:1%lo, 10.18.122.155, 127.0.0.1, 2001:bc8:1824:154d:0:0:0:1%ens2], sockAddrs=HashSet [2001:bc8:1824:154d:0:0:0:1%ens2:47500, /10.18.122.155:47500, /0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500] example digital ocean node's arrays: addrs=ArrayList [0:0:0:0:0:0:0:1%lo, 10.15.0.5, 127.0.0.1, 188.166.219. <http://188.166.219.180/>XXX], sockAddrs=HashSet [/0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500, /10.15.0.5:47500, / 188.166.219.XXX:47500 <http://188.166.219.180:47500/>] How can we set scaleway server and client nodes' configurations , so that they send correct public IPs for tcp communication ? On Wed, Aug 12, 2020 at 12:53 PM Ilya Kasnacheev <ilya.kasnach...@gmail.com> wrote: > Hello! > > I recommend setting the additional JVM arg -Djava.net.preferIPv4Stack=true on > all nodes. You have IPv6 addresses there, which can cause issues in some > cases. > > Regards, > -- > Ilya Kasnacheev > > > > вт, 11 авг. 2020 г. в 23:40, Homer Kommrad <kommradho...@gmail.com>: > >> Thank you for the quick response. First of wall ,none of the servers have >> a firewall. And all the ignite instances are run as root, on machines that >> I solely run for ignite. So I really don't understand How they can have any >> connection permission problems at all. >> >> For now , I'll also provide the small java snippet I run for this test: >> >> IgniteConfiguration cfg = new IgniteConfiguration(); >> cfg.setClientMode(true); >> >> cfg.setPeerClassLoadingEnabled(true); >> >> cfg.setWorkDirectory("/tmp/"); >> >> TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder(); >> >> ipFinder.setAddresses(Arrays.asList("51.15.203.48","51.15.88.216")); >> >> cfg.setDiscoverySpi(new >> TcpDiscoverySpi().setLocalAddress("68.183.91.137").setIpFinder(ipFinder)); >> >> Ignite ignite=Ignition.start(cfg); >> >> On Tue, Aug 11, 2020 at 10:06 PM Evgenii Zhuravlev < >> e.zhuravlev...@gmail.com> wrote: >> >>> Hi, >>> >>> It looks like the node can't establish a connection using the >>> Communication channel to the remote node. I would recommend checking that >>> all ports are opened and there is no firewall. Also, you can check that >>> connection can be established using telnet or any other tool. If you're >>> sure, that connection can be established between these 2 nodes using the >>> communication port, please share full logs(not the console output) from all >>> nodes. >>> >>> Evgenii >>> >>> вт, 11 авг. 2020 г. в 10:14, Homer Kommrad <kommradho...@gmail.com>: >>> >>>> Hello , >>>> >>>> I have a cluster of 2 server nodes and I'm trying to connect them via a >>>> client node on another server. When I try to connect , my client node >>>> connects and I can see the topology changing , 1 Client up . But right >>>> after that , I keep getting the exception for socketTimeout . >>>> >>>> such is the log: >>>> >>>> Aug 11, 2020 5:03:47 PM java.util.logging.LogManager$RootLogger log >>>> WARNING: Failed to resolve default logging config file: >>>> config/java.util.logging.properties >>>> [17:03:48] __________ ________________ >>>> [17:03:48] / _/ ___/ |/ / _/_ __/ __/ >>>> [17:03:48] _/ // (7 7 // / / / / _/ >>>> [17:03:48] /___/\___/_/|_/___/ /_/ /___/ >>>> [17:03:48] >>>> [17:03:48] ver. 2.8.1#20200521-sha1:86422096 >>>> [17:03:48] 2020 Copyright(C) Apache Software Foundation >>>> [17:03:48] >>>> [17:03:48] Ignite documentation: http://ignite.apache.org >>>> [17:03:48] >>>> [17:03:48] Quiet mode. >>>> [17:03:48] ^-- Logging by 'JavaLogger [quiet=true, config=null]' >>>> [17:03:48] ^-- To see **FULL** console log here add >>>> -DIGNITE_QUIET=false or "-v" to ignite.{sh|bat} >>>> [17:03:48] >>>> [17:03:48] OS: Linux 4.15.0-112-generic amd64 >>>> [17:03:48] VM information: OpenJDK Runtime Environment >>>> 1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 Private Build OpenJDK 64-Bit Server >>>> VM 25.265-b01 >>>> [17:03:48] Please set system property '-Djava.net.preferIPv4Stack=true' >>>> to avoid possible problems in mixed environments. >>>> [17:03:48] Initial heap size is 16MB (should be no less than 512MB, use >>>> -Xms512m -Xmx512m). >>>> [17:03:48] Configured plugins: >>>> [17:03:48] ^-- None >>>> [17:03:48] >>>> [17:03:48] Configured failure handler: >>>> [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, >>>> super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet >>>> [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITIC >>>> AL_OPERATION_TIMEOUT]]]] >>>> [17:03:49] Message queue limit is set to 0 which may lead to potential >>>> OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due >>>> to message queues growth on sender and receiver sides. >>>> [17:03:49] Security status [authentication=off, tls/ssl=off] >>>> [17:03:50] REST protocols do not start on client node. To start the >>>> protocols on client node set '-DIGNITE_REST_START_ON_CLIENT=true' system >>>> property. >>>> [17:03:52] Nodes started on local machine require more than 80% of >>>> physical RAM what can lead to significant slowdown due to swapping (please >>>> decrease JVM heap size, data region size or checkpoint buffer >>>> size) [required=239MB, available=985MB] >>>> Aug 11, 2020 5:04:02 PM org.apache.ignite.logger.java.JavaLogger error >>>> SEVERE: Failed to send message to remote node [node=TcpDiscoveryNode >>>> [id=7899fe2d-bd77-4077-8cce-a0550f1cab62, consistentId=22222222222222, >>>> addrs=ArrayList [0:0:0:0:0:0:0:1%lo, 10.69.44.105, 127.0.0.1, 2 >>>> 001:bc8:628:1634:0:0:0:1%ens2], sockAddrs=HashSet [/10.69.44.105:47500, >>>> /0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500, >>>> 2001:bc8:628:1634:0:0:0:1%ens2:47500], discPort=47500, order=116, >>>> intOrder=59, lastExc >>>> hangeTime=1597165431145, loc=false, ver=2.8.1#20200521-sha1:86422096, >>>> isClient=false], msg=GridIoMessage [plc=2, topic=TOPIC_CACHE, topicOrd=8, >>>> ordered=false, timeout=0, skipOnTimeout=false, msg=GridDhtP >>>> artitionsSingleMessage [parts=null, partCntrs=null, partsSizes=null, >>>> partHistCntrs=null, err=null, client=true, exchangeStartTime=4152061099919, >>>> finishMsg=null, super=GridDhtPartitionsAbstractMessage [ex >>>> chId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion >>>> [topVer=201, minorTopVer=0], discoEvt=DiscoveryEvent >>>> [evtNode=TcpDiscoveryNode [id=5aa5cfc5-8722-4e2f-bddf-35af5df73808, >>>> consistentId=5aa5c >>>> fc5-8722-4e2f-bddf-35af5df73808, addrs=ArrayList [68.183.91.137], >>>> sockAddrs=HashSet [/68.183.91.137:0], discPort=0, order=201, >>>> intOrder=0, lastExchangeTime=1597165430185, loc=true, >>>> ver=2.8.1#20200521-sha >>>> 1:86422096, isClient=true], topVer=201, nodeId8=5aa5cfc5, msg=null, >>>> type=NODE_JOINED, tstamp=1597165432178], nodeId=5aa5cfc5, evt=NODE_JOINED], >>>> lastVer=GridCacheVersion [topVer=0, order=1597165429407, no >>>> deOrder=0], super=GridCacheMessage [msgId=1, depInfo=null, >>>> lastAffChangedTopVer=AffinityTopologyVersion [topVer=-1, minorTopVer=0], >>>> err=null, skipPrepare=false]]]]] >>>> class org.apache.ignite.IgniteCheckedException: Failed to connect to >>>> node (is node still alive?). Make sure that each ComputeTask and cache >>>> Transaction has a timeout set in order to prevent parties from >>>> waiting forever in case of network issues >>>> [nodeId=7899fe2d-bd77-4077-8cce-a0550f1cab62, addrs=[/ >>>> 10.69.44.105:47100, /0:0:0:0:0:0:0:1%lo:47100, /127.0.0.1:47100, >>>> 2001:bc8:628:1634:0:0:0:1%ens2:47100]] >>>> at >>>> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createNioSession(TcpCommunicationSpi.java:3738) >>>> >>>> at >>>> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3458) >>>> >>>> at >>>> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createCommunicationClient(TcpCommunicationSpi.java:3198) >>>> >>>> at >>>> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:3078) >>>> >>>> at >>>> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2918) >>>> >>>> at >>>> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:2877) >>>> >>>> at >>>> org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:2035) >>>> >>>> at >>>> org.apache.ignite.internal.managers.communication.GridIoManager.sendToGridTopic(GridIoManager.java:2132) >>>> >>>> at >>>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.send(GridCacheIoManager.java:1257) >>>> >>>> at >>>> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.sendLocalPartitions(GridDhtPartitionsExchangeFuture.java:2020) >>>> >>>> at >>>> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.clientOnlyExchange(GridDhtPartitionsExchangeFuture.java:1436) >>>> >>>> at >>>> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:903) >>>> >>>> at >>>> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:3214) >>>> >>>> at >>>> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:3063) >>>> >>>> at >>>> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) >>>> at java.lang.Thread.run(Thread.java:748) >>>> Caused by: java.net.SocketTimeoutException >>>> at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:129) >>>> at >>>> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createNioSession(TcpCommunicationSpi.java:3584) >>>> >>>> ... 15 more >>>> >>>> >>>>
