Re: I have an exception while trying to send the "evt=NODE_JOINED" message

2020-08-14 Thread Evgenii Zhuravlev
Hi,

You can use Address resolver for that:
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/BasicAddressResolver.html

Best Regards,
Evgenii

пт, 14 авг. 2020 г. в 14:30, Homer Kommrad :

> Hello again,
>
> my problem's root was using Scaleway instances as server nodes . Scaleway
> developer instances do not know about their Public IPs , thus they were
> sending local addresses in the addr and sockAddr arrays. When I set up my
> server nodes on digital ocean instances , everything was solved.
>
> example from scaleway , which is not working:
> addrs=ArrayList [0:0:0:0:0:0:0:1%lo, 10.18.122.155, 127.0.0.1,
> 2001:bc8:1824:154d:0:0:0:1%ens2], sockAddrs=HashSet
> [2001:bc8:1824:154d:0:0:0:1%ens2:47500, /10.18.122.155:47500,
> /0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500]
>
> example digital ocean node's arrays:
> addrs=ArrayList [0:0:0:0:0:0:0:1%lo, 10.15.0.5, 127.0.0.1, 188.166.219.
> XXX], sockAddrs=HashSet
> [/0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500, /10.15.0.5:47500, /
> 188.166.219.XXX:47500 ]
>
>
> How can we set scaleway server and client nodes' configurations , so that
> they send correct public IPs for tcp communication  ?
>
> On Wed, Aug 12, 2020 at 12:53 PM Ilya Kasnacheev <
> ilya.kasnach...@gmail.com> wrote:
>
>> Hello!
>>
>> I recommend setting the additional JVM arg -Djava.net.prefer
>> IPv4Stack=true on all nodes. You have IPv6 addresses there, which can
>> cause issues in some cases.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>>
>> вт, 11 авг. 2020 г. в 23:40, Homer Kommrad :
>>
>>> Thank you for the quick response. First of wall ,none of the servers
>>> have a firewall. And all the ignite instances are run as root, on machines
>>> that I solely run for ignite. So I really don't understand How they can
>>> have any connection permission problems at all.
>>>
>>> For now , I'll also provide the small java snippet I run for this test:
>>>
>>> IgniteConfiguration cfg = new IgniteConfiguration();
>>> cfg.setClientMode(true);
>>>
>>> cfg.setPeerClassLoadingEnabled(true);
>>>
>>> cfg.setWorkDirectory("/tmp/");
>>>
>>> TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
>>>
>>> ipFinder.setAddresses(Arrays.asList("51.15.203.48","51.15.88.216"));
>>>
>>> cfg.setDiscoverySpi(new
>>> TcpDiscoverySpi().setLocalAddress("68.183.91.137").setIpFinder(ipFinder));
>>>
>>> Ignite ignite=Ignition.start(cfg);
>>>
>>> On Tue, Aug 11, 2020 at 10:06 PM Evgenii Zhuravlev <
>>> e.zhuravlev...@gmail.com> wrote:
>>>
 Hi,

 It looks like the node can't establish a connection using the
 Communication channel to the remote node. I would recommend checking that
 all ports are opened and there is no firewall. Also, you can check that
 connection can be established using telnet or any other tool. If you're
 sure, that connection can be established between these 2 nodes using the
 communication port, please share full logs(not the console output) from all
 nodes.

 Evgenii

 вт, 11 авг. 2020 г. в 10:14, Homer Kommrad :

> Hello ,
>
> I have a cluster of 2 server nodes and I'm trying to connect them via
> a client node on another server. When I try to connect , my client node
> connects and I can see the topology changing , 1 Client up . But right
> after that , I keep getting the exception for socketTimeout .
>
> such is the log:
>
> Aug 11, 2020 5:03:47 PM java.util.logging.LogManager$RootLogger log
> WARNING: Failed to resolve default logging config file:
> config/java.util.logging.properties
> [17:03:48]__  
> [17:03:48]   /  _/ ___/ |/ /  _/_  __/ __/
> [17:03:48]  _/ // (7 7// /  / / / _/
> [17:03:48] /___/\___/_/|_/___/ /_/ /___/
> [17:03:48]
> [17:03:48] ver. 2.8.1#20200521-sha1:86422096
> [17:03:48] 2020 Copyright(C) Apache Software Foundation
> [17:03:48]
> [17:03:48] Ignite documentation: http://ignite.apache.org
> [17:03:48]
> [17:03:48] Quiet mode.
> [17:03:48]   ^-- Logging by 'JavaLogger [quiet=true, config=null]'
> [17:03:48]   ^-- To see **FULL** console log here add
> -DIGNITE_QUIET=false or "-v" to ignite.{sh|bat}
> [17:03:48]
> [17:03:48] OS: Linux 4.15.0-112-generic amd64
> [17:03:48] VM information: OpenJDK Runtime Environment
> 1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 Private Build OpenJDK 64-Bit Server
> VM 25.265-b01
> [17:03:48] Please set system property
> '-Djava.net.preferIPv4Stack=true' to avoid possible problems in mixed
> environments.
> [17:03:48] Initial heap size is 16MB (should be no less than 512MB,
> use -Xms512m -Xmx512m).
> [17:03:48] Configured plugins:
> [17:03:48]   ^-- None
> [17:03:48]
> [17:03:48] Configured failure handler:
> [hnd=StopNodeOrHaltFailureHandler [tryStop=false, 

Re: I have an exception while trying to send the "evt=NODE_JOINED" message

2020-08-14 Thread Homer Kommrad
Hello again,

my problem's root was using Scaleway instances as server nodes . Scaleway
developer instances do not know about their Public IPs , thus they were
sending local addresses in the addr and sockAddr arrays. When I set up my
server nodes on digital ocean instances , everything was solved.

example from scaleway , which is not working:
addrs=ArrayList [0:0:0:0:0:0:0:1%lo, 10.18.122.155, 127.0.0.1,
2001:bc8:1824:154d:0:0:0:1%ens2], sockAddrs=HashSet
[2001:bc8:1824:154d:0:0:0:1%ens2:47500, /10.18.122.155:47500,
/0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500]

example digital ocean node's arrays:
addrs=ArrayList [0:0:0:0:0:0:0:1%lo, 10.15.0.5, 127.0.0.1, 188.166.219.
XXX], sockAddrs=HashSet
[/0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500, /10.15.0.5:47500, /
188.166.219.XXX:47500 ]


How can we set scaleway server and client nodes' configurations , so that
they send correct public IPs for tcp communication  ?

On Wed, Aug 12, 2020 at 12:53 PM Ilya Kasnacheev 
wrote:

> Hello!
>
> I recommend setting the additional JVM arg -Djava.net.preferIPv4Stack=true on
> all nodes. You have IPv6 addresses there, which can cause issues in some
> cases.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
>
> вт, 11 авг. 2020 г. в 23:40, Homer Kommrad :
>
>> Thank you for the quick response. First of wall ,none of the servers have
>> a firewall. And all the ignite instances are run as root, on machines that
>> I solely run for ignite. So I really don't understand How they can have any
>> connection permission problems at all.
>>
>> For now , I'll also provide the small java snippet I run for this test:
>>
>> IgniteConfiguration cfg = new IgniteConfiguration();
>> cfg.setClientMode(true);
>>
>> cfg.setPeerClassLoadingEnabled(true);
>>
>> cfg.setWorkDirectory("/tmp/");
>>
>> TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
>>
>> ipFinder.setAddresses(Arrays.asList("51.15.203.48","51.15.88.216"));
>>
>> cfg.setDiscoverySpi(new
>> TcpDiscoverySpi().setLocalAddress("68.183.91.137").setIpFinder(ipFinder));
>>
>> Ignite ignite=Ignition.start(cfg);
>>
>> On Tue, Aug 11, 2020 at 10:06 PM Evgenii Zhuravlev <
>> e.zhuravlev...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> It looks like the node can't establish a connection using the
>>> Communication channel to the remote node. I would recommend checking that
>>> all ports are opened and there is no firewall. Also, you can check that
>>> connection can be established using telnet or any other tool. If you're
>>> sure, that connection can be established between these 2 nodes using the
>>> communication port, please share full logs(not the console output) from all
>>> nodes.
>>>
>>> Evgenii
>>>
>>> вт, 11 авг. 2020 г. в 10:14, Homer Kommrad :
>>>
 Hello ,

 I have a cluster of 2 server nodes and I'm trying to connect them via a
 client node on another server. When I try to connect , my client node
 connects and I can see the topology changing , 1 Client up . But right
 after that , I keep getting the exception for socketTimeout .

 such is the log:

 Aug 11, 2020 5:03:47 PM java.util.logging.LogManager$RootLogger log
 WARNING: Failed to resolve default logging config file:
 config/java.util.logging.properties
 [17:03:48]__  
 [17:03:48]   /  _/ ___/ |/ /  _/_  __/ __/
 [17:03:48]  _/ // (7 7// /  / / / _/
 [17:03:48] /___/\___/_/|_/___/ /_/ /___/
 [17:03:48]
 [17:03:48] ver. 2.8.1#20200521-sha1:86422096
 [17:03:48] 2020 Copyright(C) Apache Software Foundation
 [17:03:48]
 [17:03:48] Ignite documentation: http://ignite.apache.org
 [17:03:48]
 [17:03:48] Quiet mode.
 [17:03:48]   ^-- Logging by 'JavaLogger [quiet=true, config=null]'
 [17:03:48]   ^-- To see **FULL** console log here add
 -DIGNITE_QUIET=false or "-v" to ignite.{sh|bat}
 [17:03:48]
 [17:03:48] OS: Linux 4.15.0-112-generic amd64
 [17:03:48] VM information: OpenJDK Runtime Environment
 1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 Private Build OpenJDK 64-Bit Server
 VM 25.265-b01
 [17:03:48] Please set system property '-Djava.net.preferIPv4Stack=true'
 to avoid possible problems in mixed environments.
 [17:03:48] Initial heap size is 16MB (should be no less than 512MB, use
 -Xms512m -Xmx512m).
 [17:03:48] Configured plugins:
 [17:03:48]   ^-- None
 [17:03:48]
 [17:03:48] Configured failure handler:
 [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0,
 super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet
 [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITIC
 AL_OPERATION_TIMEOUT
 [17:03:49] Message queue limit is set to 0 which may lead to potential
 OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due
 to message queues growth on sender and receiver sides.
 [17:03:49] Security status 

Re: I have an exception while trying to send the "evt=NODE_JOINED" message

2020-08-12 Thread Ilya Kasnacheev
Hello!

I recommend setting the additional JVM arg -Djava.net.preferIPv4Stack=true on
all nodes. You have IPv6 addresses there, which can cause issues in some
cases.

Regards,
-- 
Ilya Kasnacheev



вт, 11 авг. 2020 г. в 23:40, Homer Kommrad :

> Thank you for the quick response. First of wall ,none of the servers have
> a firewall. And all the ignite instances are run as root, on machines that
> I solely run for ignite. So I really don't understand How they can have any
> connection permission problems at all.
>
> For now , I'll also provide the small java snippet I run for this test:
>
> IgniteConfiguration cfg = new IgniteConfiguration();
> cfg.setClientMode(true);
>
> cfg.setPeerClassLoadingEnabled(true);
>
> cfg.setWorkDirectory("/tmp/");
>
> TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
>
> ipFinder.setAddresses(Arrays.asList("51.15.203.48","51.15.88.216"));
>
> cfg.setDiscoverySpi(new
> TcpDiscoverySpi().setLocalAddress("68.183.91.137").setIpFinder(ipFinder));
>
> Ignite ignite=Ignition.start(cfg);
>
> On Tue, Aug 11, 2020 at 10:06 PM Evgenii Zhuravlev <
> e.zhuravlev...@gmail.com> wrote:
>
>> Hi,
>>
>> It looks like the node can't establish a connection using the
>> Communication channel to the remote node. I would recommend checking that
>> all ports are opened and there is no firewall. Also, you can check that
>> connection can be established using telnet or any other tool. If you're
>> sure, that connection can be established between these 2 nodes using the
>> communication port, please share full logs(not the console output) from all
>> nodes.
>>
>> Evgenii
>>
>> вт, 11 авг. 2020 г. в 10:14, Homer Kommrad :
>>
>>> Hello ,
>>>
>>> I have a cluster of 2 server nodes and I'm trying to connect them via a
>>> client node on another server. When I try to connect , my client node
>>> connects and I can see the topology changing , 1 Client up . But right
>>> after that , I keep getting the exception for socketTimeout .
>>>
>>> such is the log:
>>>
>>> Aug 11, 2020 5:03:47 PM java.util.logging.LogManager$RootLogger log
>>> WARNING: Failed to resolve default logging config file:
>>> config/java.util.logging.properties
>>> [17:03:48]__  
>>> [17:03:48]   /  _/ ___/ |/ /  _/_  __/ __/
>>> [17:03:48]  _/ // (7 7// /  / / / _/
>>> [17:03:48] /___/\___/_/|_/___/ /_/ /___/
>>> [17:03:48]
>>> [17:03:48] ver. 2.8.1#20200521-sha1:86422096
>>> [17:03:48] 2020 Copyright(C) Apache Software Foundation
>>> [17:03:48]
>>> [17:03:48] Ignite documentation: http://ignite.apache.org
>>> [17:03:48]
>>> [17:03:48] Quiet mode.
>>> [17:03:48]   ^-- Logging by 'JavaLogger [quiet=true, config=null]'
>>> [17:03:48]   ^-- To see **FULL** console log here add
>>> -DIGNITE_QUIET=false or "-v" to ignite.{sh|bat}
>>> [17:03:48]
>>> [17:03:48] OS: Linux 4.15.0-112-generic amd64
>>> [17:03:48] VM information: OpenJDK Runtime Environment
>>> 1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 Private Build OpenJDK 64-Bit Server
>>> VM 25.265-b01
>>> [17:03:48] Please set system property '-Djava.net.preferIPv4Stack=true'
>>> to avoid possible problems in mixed environments.
>>> [17:03:48] Initial heap size is 16MB (should be no less than 512MB, use
>>> -Xms512m -Xmx512m).
>>> [17:03:48] Configured plugins:
>>> [17:03:48]   ^-- None
>>> [17:03:48]
>>> [17:03:48] Configured failure handler: [hnd=StopNodeOrHaltFailureHandler
>>> [tryStop=false, timeout=0, super=AbstractFailureHandler
>>> [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITIC
>>> AL_OPERATION_TIMEOUT
>>> [17:03:49] Message queue limit is set to 0 which may lead to potential
>>> OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due
>>> to message queues growth on sender and receiver sides.
>>> [17:03:49] Security status [authentication=off, tls/ssl=off]
>>> [17:03:50] REST protocols do not start on client node. To start the
>>> protocols on client node set '-DIGNITE_REST_START_ON_CLIENT=true' system
>>> property.
>>> [17:03:52] Nodes started on local machine require more than 80% of
>>> physical RAM what can lead to significant slowdown due to swapping (please
>>> decrease JVM heap size, data region size or checkpoint buffer
>>> size) [required=239MB, available=985MB]
>>> Aug 11, 2020 5:04:02 PM org.apache.ignite.logger.java.JavaLogger error
>>> SEVERE: Failed to send message to remote node [node=TcpDiscoveryNode
>>> [id=7899fe2d-bd77-4077-8cce-a0550f1cab62, consistentId=22,
>>> addrs=ArrayList [0:0:0:0:0:0:0:1%lo, 10.69.44.105, 127.0.0.1, 2
>>> 001:bc8:628:1634:0:0:0:1%ens2], sockAddrs=HashSet [/10.69.44.105:47500,
>>> /0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500,
>>> 2001:bc8:628:1634:0:0:0:1%ens2:47500], discPort=47500, order=116,
>>> intOrder=59, lastExc
>>> hangeTime=1597165431145, loc=false, ver=2.8.1#20200521-sha1:86422096,
>>> isClient=false], msg=GridIoMessage [plc=2, topic=TOPIC_CACHE, topicOrd=8,
>>> ordered=false, timeout=0, 

Re: I have an exception while trying to send the "evt=NODE_JOINED" message

2020-08-11 Thread Homer Kommrad
Thank you for the quick response. First of wall ,none of the servers have a
firewall. And all the ignite instances are run as root, on machines that I
solely run for ignite. So I really don't understand How they can have any
connection permission problems at all.

For now , I'll also provide the small java snippet I run for this test:

IgniteConfiguration cfg = new IgniteConfiguration();
cfg.setClientMode(true);

cfg.setPeerClassLoadingEnabled(true);

cfg.setWorkDirectory("/tmp/");

TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();

ipFinder.setAddresses(Arrays.asList("51.15.203.48","51.15.88.216"));

cfg.setDiscoverySpi(new
TcpDiscoverySpi().setLocalAddress("68.183.91.137").setIpFinder(ipFinder));

Ignite ignite=Ignition.start(cfg);

On Tue, Aug 11, 2020 at 10:06 PM Evgenii Zhuravlev 
wrote:

> Hi,
>
> It looks like the node can't establish a connection using the
> Communication channel to the remote node. I would recommend checking that
> all ports are opened and there is no firewall. Also, you can check that
> connection can be established using telnet or any other tool. If you're
> sure, that connection can be established between these 2 nodes using the
> communication port, please share full logs(not the console output) from all
> nodes.
>
> Evgenii
>
> вт, 11 авг. 2020 г. в 10:14, Homer Kommrad :
>
>> Hello ,
>>
>> I have a cluster of 2 server nodes and I'm trying to connect them via a
>> client node on another server. When I try to connect , my client node
>> connects and I can see the topology changing , 1 Client up . But right
>> after that , I keep getting the exception for socketTimeout .
>>
>> such is the log:
>>
>> Aug 11, 2020 5:03:47 PM java.util.logging.LogManager$RootLogger log
>> WARNING: Failed to resolve default logging config file:
>> config/java.util.logging.properties
>> [17:03:48]__  
>> [17:03:48]   /  _/ ___/ |/ /  _/_  __/ __/
>> [17:03:48]  _/ // (7 7// /  / / / _/
>> [17:03:48] /___/\___/_/|_/___/ /_/ /___/
>> [17:03:48]
>> [17:03:48] ver. 2.8.1#20200521-sha1:86422096
>> [17:03:48] 2020 Copyright(C) Apache Software Foundation
>> [17:03:48]
>> [17:03:48] Ignite documentation: http://ignite.apache.org
>> [17:03:48]
>> [17:03:48] Quiet mode.
>> [17:03:48]   ^-- Logging by 'JavaLogger [quiet=true, config=null]'
>> [17:03:48]   ^-- To see **FULL** console log here add
>> -DIGNITE_QUIET=false or "-v" to ignite.{sh|bat}
>> [17:03:48]
>> [17:03:48] OS: Linux 4.15.0-112-generic amd64
>> [17:03:48] VM information: OpenJDK Runtime Environment
>> 1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 Private Build OpenJDK 64-Bit Server
>> VM 25.265-b01
>> [17:03:48] Please set system property '-Djava.net.preferIPv4Stack=true'
>> to avoid possible problems in mixed environments.
>> [17:03:48] Initial heap size is 16MB (should be no less than 512MB, use
>> -Xms512m -Xmx512m).
>> [17:03:48] Configured plugins:
>> [17:03:48]   ^-- None
>> [17:03:48]
>> [17:03:48] Configured failure handler: [hnd=StopNodeOrHaltFailureHandler
>> [tryStop=false, timeout=0, super=AbstractFailureHandler
>> [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITIC
>> AL_OPERATION_TIMEOUT
>> [17:03:49] Message queue limit is set to 0 which may lead to potential
>> OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due
>> to message queues growth on sender and receiver sides.
>> [17:03:49] Security status [authentication=off, tls/ssl=off]
>> [17:03:50] REST protocols do not start on client node. To start the
>> protocols on client node set '-DIGNITE_REST_START_ON_CLIENT=true' system
>> property.
>> [17:03:52] Nodes started on local machine require more than 80% of
>> physical RAM what can lead to significant slowdown due to swapping (please
>> decrease JVM heap size, data region size or checkpoint buffer
>> size) [required=239MB, available=985MB]
>> Aug 11, 2020 5:04:02 PM org.apache.ignite.logger.java.JavaLogger error
>> SEVERE: Failed to send message to remote node [node=TcpDiscoveryNode
>> [id=7899fe2d-bd77-4077-8cce-a0550f1cab62, consistentId=22,
>> addrs=ArrayList [0:0:0:0:0:0:0:1%lo, 10.69.44.105, 127.0.0.1, 2
>> 001:bc8:628:1634:0:0:0:1%ens2], sockAddrs=HashSet [/10.69.44.105:47500,
>> /0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500,
>> 2001:bc8:628:1634:0:0:0:1%ens2:47500], discPort=47500, order=116,
>> intOrder=59, lastExc
>> hangeTime=1597165431145, loc=false, ver=2.8.1#20200521-sha1:86422096,
>> isClient=false], msg=GridIoMessage [plc=2, topic=TOPIC_CACHE, topicOrd=8,
>> ordered=false, timeout=0, skipOnTimeout=false, msg=GridDhtP
>> artitionsSingleMessage [parts=null, partCntrs=null, partsSizes=null,
>> partHistCntrs=null, err=null, client=true, exchangeStartTime=4152061099919,
>> finishMsg=null, super=GridDhtPartitionsAbstractMessage [ex
>> chId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion
>> [topVer=201, minorTopVer=0], discoEvt=DiscoveryEvent
>> 

Re: I have an exception while trying to send the "evt=NODE_JOINED" message

2020-08-11 Thread Evgenii Zhuravlev
Hi,

It looks like the node can't establish a connection using the Communication
channel to the remote node. I would recommend checking that all ports are
opened and there is no firewall. Also, you can check that connection can be
established using telnet or any other tool. If you're sure, that connection
can be established between these 2 nodes using the communication port,
please share full logs(not the console output) from all nodes.

Evgenii

вт, 11 авг. 2020 г. в 10:14, Homer Kommrad :

> Hello ,
>
> I have a cluster of 2 server nodes and I'm trying to connect them via a
> client node on another server. When I try to connect , my client node
> connects and I can see the topology changing , 1 Client up . But right
> after that , I keep getting the exception for socketTimeout .
>
> such is the log:
>
> Aug 11, 2020 5:03:47 PM java.util.logging.LogManager$RootLogger log
> WARNING: Failed to resolve default logging config file:
> config/java.util.logging.properties
> [17:03:48]__  
> [17:03:48]   /  _/ ___/ |/ /  _/_  __/ __/
> [17:03:48]  _/ // (7 7// /  / / / _/
> [17:03:48] /___/\___/_/|_/___/ /_/ /___/
> [17:03:48]
> [17:03:48] ver. 2.8.1#20200521-sha1:86422096
> [17:03:48] 2020 Copyright(C) Apache Software Foundation
> [17:03:48]
> [17:03:48] Ignite documentation: http://ignite.apache.org
> [17:03:48]
> [17:03:48] Quiet mode.
> [17:03:48]   ^-- Logging by 'JavaLogger [quiet=true, config=null]'
> [17:03:48]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false
> or "-v" to ignite.{sh|bat}
> [17:03:48]
> [17:03:48] OS: Linux 4.15.0-112-generic amd64
> [17:03:48] VM information: OpenJDK Runtime Environment
> 1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 Private Build OpenJDK 64-Bit Server
> VM 25.265-b01
> [17:03:48] Please set system property '-Djava.net.preferIPv4Stack=true' to
> avoid possible problems in mixed environments.
> [17:03:48] Initial heap size is 16MB (should be no less than 512MB, use
> -Xms512m -Xmx512m).
> [17:03:48] Configured plugins:
> [17:03:48]   ^-- None
> [17:03:48]
> [17:03:48] Configured failure handler: [hnd=StopNodeOrHaltFailureHandler
> [tryStop=false, timeout=0, super=AbstractFailureHandler
> [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITIC
> AL_OPERATION_TIMEOUT
> [17:03:49] Message queue limit is set to 0 which may lead to potential
> OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due
> to message queues growth on sender and receiver sides.
> [17:03:49] Security status [authentication=off, tls/ssl=off]
> [17:03:50] REST protocols do not start on client node. To start the
> protocols on client node set '-DIGNITE_REST_START_ON_CLIENT=true' system
> property.
> [17:03:52] Nodes started on local machine require more than 80% of
> physical RAM what can lead to significant slowdown due to swapping (please
> decrease JVM heap size, data region size or checkpoint buffer
> size) [required=239MB, available=985MB]
> Aug 11, 2020 5:04:02 PM org.apache.ignite.logger.java.JavaLogger error
> SEVERE: Failed to send message to remote node [node=TcpDiscoveryNode
> [id=7899fe2d-bd77-4077-8cce-a0550f1cab62, consistentId=22,
> addrs=ArrayList [0:0:0:0:0:0:0:1%lo, 10.69.44.105, 127.0.0.1, 2
> 001:bc8:628:1634:0:0:0:1%ens2], sockAddrs=HashSet [/10.69.44.105:47500,
> /0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500,
> 2001:bc8:628:1634:0:0:0:1%ens2:47500], discPort=47500, order=116,
> intOrder=59, lastExc
> hangeTime=1597165431145, loc=false, ver=2.8.1#20200521-sha1:86422096,
> isClient=false], msg=GridIoMessage [plc=2, topic=TOPIC_CACHE, topicOrd=8,
> ordered=false, timeout=0, skipOnTimeout=false, msg=GridDhtP
> artitionsSingleMessage [parts=null, partCntrs=null, partsSizes=null,
> partHistCntrs=null, err=null, client=true, exchangeStartTime=4152061099919,
> finishMsg=null, super=GridDhtPartitionsAbstractMessage [ex
> chId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion
> [topVer=201, minorTopVer=0], discoEvt=DiscoveryEvent
> [evtNode=TcpDiscoveryNode [id=5aa5cfc5-8722-4e2f-bddf-35af5df73808,
> consistentId=5aa5c
> fc5-8722-4e2f-bddf-35af5df73808, addrs=ArrayList [68.183.91.137],
> sockAddrs=HashSet [/68.183.91.137:0], discPort=0, order=201, intOrder=0,
> lastExchangeTime=1597165430185, loc=true, ver=2.8.1#20200521-sha
> 1:86422096, isClient=true], topVer=201, nodeId8=5aa5cfc5, msg=null,
> type=NODE_JOINED, tstamp=1597165432178], nodeId=5aa5cfc5, evt=NODE_JOINED],
> lastVer=GridCacheVersion [topVer=0, order=1597165429407, no
> deOrder=0], super=GridCacheMessage [msgId=1, depInfo=null,
> lastAffChangedTopVer=AffinityTopologyVersion [topVer=-1, minorTopVer=0],
> err=null, skipPrepare=false]
> class org.apache.ignite.IgniteCheckedException: Failed to connect to node
> (is node still alive?). Make sure that each ComputeTask and cache
> Transaction has a timeout set in order to prevent parties from
> waiting forever in case of network issues
> 

I have an exception while trying to send the "evt=NODE_JOINED" message

2020-08-11 Thread Homer Kommrad
Hello ,

I have a cluster of 2 server nodes and I'm trying to connect them via a
client node on another server. When I try to connect , my client node
connects and I can see the topology changing , 1 Client up . But right
after that , I keep getting the exception for socketTimeout .

such is the log:

Aug 11, 2020 5:03:47 PM java.util.logging.LogManager$RootLogger log
WARNING: Failed to resolve default logging config file:
config/java.util.logging.properties
[17:03:48]__  
[17:03:48]   /  _/ ___/ |/ /  _/_  __/ __/
[17:03:48]  _/ // (7 7// /  / / / _/
[17:03:48] /___/\___/_/|_/___/ /_/ /___/
[17:03:48]
[17:03:48] ver. 2.8.1#20200521-sha1:86422096
[17:03:48] 2020 Copyright(C) Apache Software Foundation
[17:03:48]
[17:03:48] Ignite documentation: http://ignite.apache.org
[17:03:48]
[17:03:48] Quiet mode.
[17:03:48]   ^-- Logging by 'JavaLogger [quiet=true, config=null]'
[17:03:48]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false
or "-v" to ignite.{sh|bat}
[17:03:48]
[17:03:48] OS: Linux 4.15.0-112-generic amd64
[17:03:48] VM information: OpenJDK Runtime Environment
1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 Private Build OpenJDK 64-Bit Server
VM 25.265-b01
[17:03:48] Please set system property '-Djava.net.preferIPv4Stack=true' to
avoid possible problems in mixed environments.
[17:03:48] Initial heap size is 16MB (should be no less than 512MB, use
-Xms512m -Xmx512m).
[17:03:48] Configured plugins:
[17:03:48]   ^-- None
[17:03:48]
[17:03:48] Configured failure handler: [hnd=StopNodeOrHaltFailureHandler
[tryStop=false, timeout=0, super=AbstractFailureHandler
[ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITIC
AL_OPERATION_TIMEOUT
[17:03:49] Message queue limit is set to 0 which may lead to potential
OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due
to message queues growth on sender and receiver sides.
[17:03:49] Security status [authentication=off, tls/ssl=off]
[17:03:50] REST protocols do not start on client node. To start the
protocols on client node set '-DIGNITE_REST_START_ON_CLIENT=true' system
property.
[17:03:52] Nodes started on local machine require more than 80% of physical
RAM what can lead to significant slowdown due to swapping (please decrease
JVM heap size, data region size or checkpoint buffer
size) [required=239MB, available=985MB]
Aug 11, 2020 5:04:02 PM org.apache.ignite.logger.java.JavaLogger error
SEVERE: Failed to send message to remote node [node=TcpDiscoveryNode
[id=7899fe2d-bd77-4077-8cce-a0550f1cab62, consistentId=22,
addrs=ArrayList [0:0:0:0:0:0:0:1%lo, 10.69.44.105, 127.0.0.1, 2
001:bc8:628:1634:0:0:0:1%ens2], sockAddrs=HashSet [/10.69.44.105:47500,
/0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500,
2001:bc8:628:1634:0:0:0:1%ens2:47500], discPort=47500, order=116,
intOrder=59, lastExc
hangeTime=1597165431145, loc=false, ver=2.8.1#20200521-sha1:86422096,
isClient=false], msg=GridIoMessage [plc=2, topic=TOPIC_CACHE, topicOrd=8,
ordered=false, timeout=0, skipOnTimeout=false, msg=GridDhtP
artitionsSingleMessage [parts=null, partCntrs=null, partsSizes=null,
partHistCntrs=null, err=null, client=true, exchangeStartTime=4152061099919,
finishMsg=null, super=GridDhtPartitionsAbstractMessage [ex
chId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion
[topVer=201, minorTopVer=0], discoEvt=DiscoveryEvent
[evtNode=TcpDiscoveryNode [id=5aa5cfc5-8722-4e2f-bddf-35af5df73808,
consistentId=5aa5c
fc5-8722-4e2f-bddf-35af5df73808, addrs=ArrayList [68.183.91.137],
sockAddrs=HashSet [/68.183.91.137:0], discPort=0, order=201, intOrder=0,
lastExchangeTime=1597165430185, loc=true, ver=2.8.1#20200521-sha
1:86422096, isClient=true], topVer=201, nodeId8=5aa5cfc5, msg=null,
type=NODE_JOINED, tstamp=1597165432178], nodeId=5aa5cfc5, evt=NODE_JOINED],
lastVer=GridCacheVersion [topVer=0, order=1597165429407, no
deOrder=0], super=GridCacheMessage [msgId=1, depInfo=null,
lastAffChangedTopVer=AffinityTopologyVersion [topVer=-1, minorTopVer=0],
err=null, skipPrepare=false]
class org.apache.ignite.IgniteCheckedException: Failed to connect to node
(is node still alive?). Make sure that each ComputeTask and cache
Transaction has a timeout set in order to prevent parties from
waiting forever in case of network issues
[nodeId=7899fe2d-bd77-4077-8cce-a0550f1cab62, addrs=[/10.69.44.105:47100,
/0:0:0:0:0:0:0:1%lo:47100, /127.0.0.1:47100,
2001:bc8:628:1634:0:0:0:1%ens2:47100]]
   at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createNioSession(TcpCommunicationSpi.java:3738)

   at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3458)

   at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createCommunicationClient(TcpCommunicationSpi.java:3198)

   at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:3078)

   at