Re: Cluster can not let more than one client to do continuous queries

[dkarachentsev]

Hi,

What do you mean "another client can not join the cluster to do continuous
queries"? Is there some specific error? Could you please provide logs from
client and server?



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Cluster-can-not-let-more-than-one-client-to-do-continuous-queries-tp9124p9129.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cluster can not let more than one client to do continuous queries

[ght230]

When first client start, it is OK.

When one more client start, the client will pause at following place:

[08:52:08] Ignite documentation: http://ignite.apache.org
[08:52:08]
[08:52:08] Quiet mode.
[08:52:08]   ^-- Logging to file
'/home/ght/ignite-1.6.8/work/log/ignite-0127ad43.0.log'
[08:52:08]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false
or "-v" to ignite.{sh|bat}
[08:52:08]
[08:52:08] OS: Linux 2.6.32-573.el6.x86_64 amd64
[08:52:08] VM information: Java(TM) SE Runtime Environment 1.8.0_101-b13
Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 25.101-b13
[08:52:08] Initial heap size is 46MB (should be no less than 512MB, use
-Xms512m -Xmx512m).
[08:52:09] Configured plugins:
[08:52:09]   ^-- None
[08:52:09]
[08:52:10] Security status [authentication=off, tls/ssl=off]


there is no specific error in server, 

following is the log in the second client:

[10:31:28,555][INFO][main][IgniteKernal] Config URL:
jar:file:/home/ght/ignite-fabric-lgpl-distribution-by-ffcs-1.6.8/libs/Ignitest-0.0.2-SNAPSHOT-jar-with-dependencies.jar!/example-cache.xml
[10:31:28,559][INFO][main][IgniteKernal] Daemon mode: off
[10:31:28,560][INFO][main][IgniteKernal] OS: Linux 2.6.32-573.el6.x86_64
amd64
[10:31:28,562][INFO][main][IgniteKernal] OS user: ght
[10:31:28,567][INFO][main][IgniteKernal] Language runtime: Java Platform API
Specification ver. 1.8
[10:31:28,573][INFO][main][IgniteKernal] VM information: Java(TM) SE Runtime
Environment 1.8.0_101-b13 Oracle Corporation Java HotSpot(TM) 64-Bit Server
VM 25.101-b13
[10:31:28,578][INFO][main][IgniteKernal] VM total memory: 0.69GB
[10:31:28,579][INFO][main][IgniteKernal] Remote Management [restart: off,
REST: on, JMX (remote: off)]
[10:31:28,579][INFO][main][IgniteKernal] IGNITE_HOME=/home/ght/ignite-1.6.8
[10:31:28,583][INFO][main][IgniteKernal] VM arguments: []
[10:31:28,583][INFO][main][IgniteKernal] Configured caches
['ignite-marshaller-sys-cache', 'ignite-sys-cache',
'ignite-atomics-sys-cache', 'querycache']
[10:31:28,643][INFO][main][IgniteKernal] 3-rd party licenses can be found
at: /home/ght/ignite-1.6.8/libs/licenses
[10:31:28,663][WARNING][pub-#23%null%][GridDiagnostic] Initial heap size is
46MB (should be no less than 512MB, use -Xms512m -Xmx512m).
[10:31:29,059][INFO][main][IgniteKernal] Non-loopback local IPs:
192.168.137.32, fe80:0:0:0:a00:27ff:feaa:8032%eth1
[10:31:29,059][INFO][main][IgniteKernal] Enabled local MACs: 080027AA8032
[10:31:29,120][INFO][main][IgnitePluginProcessor] Configured plugins:
[10:31:29,120][INFO][main][IgnitePluginProcessor]   ^-- None
[10:31:29,122][INFO][main][IgnitePluginProcessor]
[10:31:30,280][INFO][main][TcpCommunicationSpi] IPC shared memory server
endpoint started [port=48100,
tokDir=/home/ght/ignite-1.6.8/work/ipc/shmem/9fda37c2-69a2-408d-9b07-017782235703-2344]
[10:31:30,281][INFO][main][TcpCommunicationSpi] Successfully bound shared
memory communication to TCP port [port=48100, locHost=0.0.0.0/0.0.0.0]
[10:31:30,391][INFO][main][TcpCommunicationSpi] Successfully bound to TCP
port [port=47100, locHost=0.0.0.0/0.0.0.0]
[10:31:30,629][WARNING][main][NoopCheckpointSpi] Checkpoints are disabled
(to enable configure any GridCheckpointSpi implementation)
[10:31:30,930][WARNING][main][GridCollisionManager] Collision resolution is
disabled (all jobs will be activated upon arrival).
[10:31:30,971][WARNING][main][NoopSwapSpaceSpi] Swap space is disabled. To
enable use FileSwapSpaceSpi.
[10:31:30,983][INFO][main][IgniteKernal] Security status
[authentication=off, tls/ssl=off]
[10:31:32,693][INFO][main][GridTcpRestProtocol] Command protocol
successfully started [name=TCP binary, host=0.0.0.0/0.0.0.0, port=11211]
[10:31:43,839][WARNING][tcp-client-disco-msg-worker-#4%null%][TcpDiscoverySpi]
Timed out waiting for message delivery receipt (most probably, the reason is
in long GC pauses on remote node; consider tuning GC and increasing
'ackTimeout' configuration property). Will retry to send message with
increased timeout. Current timeout: 1.
[10:31:49,788][INFO][tcp-client-disco-msg-worker-#4%null%][CacheContinuousQueryHandler]
handleClientDisconnectedException for cacheName[ignite-sys-cache]
allowReconnect=false, reconnectTimeout=30, allowHalt=false
[10:31:49,801][INFO][tcp-client-disco-msg-worker-#4%null%][CacheContinuousQueryHandler]
handleClientDisconnectedException for cacheName[ignite-marshaller-sys-cache]
allowReconnect=false, reconnectTimeout=30, allowHalt=false




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Cluster-can-not-let-more-than-one-client-to-do-continuous-queries-tp9124p9138.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cluster can not let more than one client to do continuous queries

[dkarachentsev]

Hi,

I'm not able to reproduce exact your issue, but found that there problem if
enabled peer class loading [1].
Could you please give me al your configurations and full logs? And it would
be very helpful if you create toy project which reproduces the problem.

Thanks!

[1]: https://issues.apache.org/jira/browse/IGNITE-4284



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Cluster-can-not-let-more-than-one-client-to-do-continuous-queries-tp9124p9148.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cluster can not let more than one client to do continuous queries

[ght230]

continueclient.jar
  
I have disabled peer class loading.
The attachment is the test project about it.

CacheContinuousAsyncQueryClient11 and CacheContinuousAsyncQueryClient12 are
RemoteFilterFactory with a parameter, the second client will pause.

CacheContinuousAsyncQueryClient21 and CacheContinuousAsyncQueryClient22 are
RemoteFilterFactory without parameter, the test result is OK.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Cluster-can-not-let-more-than-one-client-to-do-continuous-queries-tp9124p9171.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cluster can not let more than one client to do continuous queries

[dkarachentsev]

Hi,

Thanks for the reproducer, but it works for both tests. I think your issue
entails in too large timeouts, for example, you set 5 min timeout for join,
so it may take too long to connect. Is there any reason for such decision?



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Cluster-can-not-let-more-than-one-client-to-do-continuous-queries-tp9124p9182.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cluster can not let more than one client to do continuous queries

[ght230]

I think perhaps our test produres are Inconsistent.

My test produre as following:

Test RemoteFilterFactory without parameter.
1. In host 192.168.137.6, run ignite.sh -i(choose the example-cache.xml)
2. In host 192.168.137.31, run jar -cp continueclient.jar
com.test.CacheContinuousAsyncQueryClient21, query running normally.
3. In host 192.168.137.32, run jar -cp continueclient.jar
com.test.CacheContinuousAsyncQueryClient22, query running normally.

Test RemoteFilterFactory with a parameter.
1. In host 192.168.137.6, run ignite.sh -i(choose the example-cache.xml)
2. In host 192.168.137.31, run jar -cp continueclient.jar
com.test.CacheContinuousAsyncQueryClient11, query running normally.
3. In host 192.168.137.32, run jar -cp continueclient.jar
com.test.CacheContinuousAsyncQueryClient12, query can not be started.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Cluster-can-not-let-more-than-one-client-to-do-continuous-queries-tp9124p9188.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cluster can not let more than one client to do continuous queries

[dkarachentsev]

>From logs that you sent I see discovery issue: unsuccessful first connect and
long retry. Please do the following: in discovery config leave only server
address and reset timeouts to default value. Small timeouts will force
client fail fast and do not wait response from server.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Cluster-can-not-let-more-than-one-client-to-do-continuous-queries-tp9124p9196.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cluster can not let more than one client to do continuous queries

[ght230]

I had modified the ignite config as following, but it did not work.

http://www.springframework.org/schema/beans";
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
   xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd";>


















































 




192.168.37.103:47500..47520










Following is the log of the second client:
[14:34:28,565][WARN ][pub-#26%null%][GridDiagnostic] Initial heap size is
126MB (should be no less than 512MB, use -Xms512m -Xmx512m).
[14:34:30,308][INFO ][main][IgniteKernal] Non-loopback local IPs:
192.168.37.103, 2001:0:b499:3d88:3016:3975:3f57:da98,
fe80:0:0:0:3016:3975:3f57:da98%13, fe80:0:0:0:602e:de10:7049:3e4%14,
fe80:0:0:0:615f:2201:b3fc:7555%19, fe80:0:0:0:e805:e97b:b4ab:3310%17
[14:34:30,308][INFO ][main][IgniteKernal] Enabled local MACs:
00E0, 00FF3E61AAFB, 54EE758A7FE6, C8FF2861A4FB
[14:34:30,308][INFO ][main][IgniteKernal] Set
localHost@IgniteConfiguration='192.168.37.103' because of
localIpStartsWith='192.168.37' match.
[14:34:30,308][INFO ][main][IgniteKernal] Set
localAddress@TcpCommunicationSpi='192.168.37.103' because of
localIpStartsWith='192.168.37' match.
[14:34:30,308][INFO ][main][IgniteKernal] Set
localAddress@TcpDiscoverySpi='192.168.37.103' because of
localIpStartsWith='192.168.37' match.
[14:34:30,351][INFO ][main][IgnitePluginProcessor] Configured plugins:
[14:34:30,351][INFO ][main][IgnitePluginProcessor]   ^-- None
[14:34:30,351][INFO ][main][IgnitePluginProcessor] 
[14:34:30,392][WARN ][main][TcpCommunicationSpi] Failure detection timeout
will be ignored (one of SPI parameters has been set explicitly)
[14:34:32,074][INFO ][main][TcpCommunicationSpi] Successfully bound to TCP
port [port=47102, locHost=/192.168.37.103]
[14:34:32,120][WARN ][main][NoopCheckpointSpi] Checkpoints are disabled (to
enable configure any GridCheckpointSpi implementation)
[14:34:32,176][WARN ][main][GridCollisionManager] Collision resolution is
disabled (all jobs will be activated upon arrival).
[14:34:32,182][WARN ][main][NoopSwapSpaceSpi] Swap space is disabled. To
enable use FileSwapSpaceSpi.
[14:34:32,184][INFO ][main][IgniteKernal] Security status
[authentication=off, tls/ssl=off]
[14:34:32,814][INFO ][main][GridTcpRestProtocol] Command protocol
successfully started [name=TCP binary, host=/192.168.37.103, port=11213]
[14:34:32,861][WARN ][main][TcpDiscoverySpi] Failure detection timeout will
be ignored (one of SPI parameters has been set explicitly)
[14:35:10,391][INFO
][tcp-client-disco-msg-worker-#4%null%][CacheContinuousQueryHandler]
handleClientDisconnectedException for cacheName[ignite-marshaller-sys-cache]
allowReconnect=true, reconnectTimeout=18, allowHalt=true

At the same time of 14:35:10, the log of server as following:
[14:35:10,293][INFO
][tcp-disco-msg-worker-#2%null%][CacheContinuousQueryHandler]
handleClientDisconnectedException for cacheName[ignite-marshaller-sys-cache]
allowReconnect=true, reconnectTimeout=18, allowHalt=true
[14:35:10,293][INFO
][tcp-disco-msg-worker-#2%null%][CacheContinuousQueryHandler]
handleClientDisc

Re: Cluster can not let more than one client to do continuous queries

[Nikolai Tikhonov]

Hi,

You start grid with disabled peer class loading  . In this case you need to
deploy CacheEntryEventFilterFactory class on all nodes in topology. Please,
make sure that all nodes contain this jar with the factory in classpath or
set peerClassLoadingEnabled to true.



On Mon, Nov 28, 2016 at 10:47 AM, ght230  wrote:

> I had modified the ignite config as following, but it did not work.
>
> http://www.springframework.org/schema/beans";
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
>xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd";>
>  class="org.apache.ignite.configuration.IgniteConfiguration">
> 
> 
> 
> 
> 
>  value="querycache" />
> 
> 
> 
> 
> 
>
> 
>  class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
> 
>  />
>  value="false" />
>  value="5000" />
>  value="32" />
>  value="#{64 * 1024}" />
>  value="#{64 * 1024}" />
>  value="48100" />
>  value="256" />
>  value="1" />
>  />
>  value="2048" />
>  value="0" />
> 
> 
> 
>  class="org.apache.ignite.spi.eventstorage.memory.MemoryEventStorageSpi">
>  />
> 
> 
> 
> 
> 
>  value="20"/>
>  />
>
>  value="3" />
>  />
>  />
>  value="3" />
>  value="5" />
>  value="10" />
> 
>
>
>  class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.
> TcpDiscoveryVmIpFinder">
>
> 
> 
> 
> 192.168.37.103:47500..47520
> 
> 
> 
> 
> 
> 
> 
> 
>
>
> Following is the log of the second client:
> [14:34:28,565][WARN ][pub-#26%null%][GridDiagnostic] Initial heap size is
> 126MB (should be no less than 512MB, use -Xms512m -Xmx512m).
> [14:34:30,308][INFO ][main][IgniteKernal] Non-loopback local IPs:
> 192.168.37.103, 2001:0:b499:3d88:3016:3975:3f57:da98,
> fe80:0:0:0:3016:3975:3f57:da98%13, fe80:0:0:0:602e:de10:7049:3e4%14,
> fe80:0:0:0:615f:2201:b3fc:7555%19, fe80:0:0:0:e805:e97b:b4ab:3310%17
> [14:34:30,308][INFO ][main][IgniteKernal] Enabled local MACs:
> 00E0, 00FF3E61AAFB, 54EE758A7FE6, C8FF2861A4FB
> [14:34:30,308][INFO ][main][IgniteKernal] Set
> localHost@IgniteConfiguration='192.168.37.103' because of
> localIpStartsWith='192.168.37' match.
> [14:34:30,308][INFO ][main][IgniteKernal] Set
> localAddress@TcpCommunicationSpi='192.168.37.103' because of
> localIpStartsWith='192.168.37' match.
> [14:34:30,308][INFO ][main][IgniteKernal] Set
> localAddress@TcpDiscoverySpi='192.168.37.103' because of
> localIpStartsWith='192.168.37' match.
> [14:34:30,351][INFO ][main][IgnitePluginProcessor] Configured plugins:
> [14:34:30,351][INFO ][main][IgnitePluginProcessor]   ^-- None
> [14:34:30,351][INFO ][main][IgnitePluginProcessor]
> [14:34:30,392][WARN ][main][TcpCommunicationSpi] Failure detection timeout
> will be ignored (one of SPI parameters has been set explicitly)
> [14:34:32,074][INFO ][main][TcpCommunicationSpi] Successfully bound to TCP
> port [port=47102, locHost=/192.168.37.103]
> [14:34:32,120][WARN ][main][NoopCheckpointSpi] Checkpoints are disabled (to
> enable configure any GridCheckpointSpi implementation)
> [14:34:32,176][WARN ][main][GridCollisionManager] Collision resolution is
> disabled (all jobs will be activated upon arrival).
> [14:34:32,182][WARN ][main][NoopSwapSpaceSpi] Swap space is disabled. To
> enable use FileSwapSpaceSpi.
> [14:34:32,184][INFO ][main][IgniteKernal] Secur