Re: Ignite servers are not connecting to the cluster

[risomas]

I have already tried to put ports as you mentioned to the configuration and
also increase or decrease all timeouts add some other timeouts like
networkTimeout, ackTimeut, connectionTimeout but no change in the behavior.
Ports are exposed and they are reachable from all VM to any other VM.
We build docker image from centos8 image + official ignite installation (
due plugin licensing installation issue on official ignite image)

For me the most suspicious message is:
...Connection timed out...addr=/192.168.183.128:47100]
that ip is tunl0 ip on other VM and is not reachable




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

HashMap warning when using PutAll()

[Raymond Wilson]

We are using Ignite v2.8.0 and the C# client. Some of our operations use
PutAll() to save a collection of items in a single operation. This
operation is emitting the following warning into the log:

2020-06-10 15:04:14,199 [77] WRN [ImmutableClientServer]
 Unordered map java.util.HashMap is
used for putAll operation on cache Spatial-SubGridDirectory-Immutable. This
can lead to a distributed deadlock. Switch to a sorted map like TreeMap
instead.

Does this require a Jira ticket?

Thanks,
Raymond.


-- 

Raymond Wilson
Solution Architect, Civil Construction Software Systems (CCSS)
11 Birmingham Drive | Christchurch, New Zealand
+64-21-2013317 Mobile
raymond_wil...@trimble.com

Re: Ignite servers are not connecting to the cluster

[Denis Magda]

Hi, there are two things to check.

First, add 47500 port number to all the addresses of
your TcpDiscoveryVmIpFinder.addresses property. For instance, 10.0.2.4
needs to be changed to 10.0.2.4:47500.

Second, double-check that every Docker VM exposes the following port
numbers - 11211, 47100, 47500, 49112. Those should be exposed by default if
an official Ignite image is used. But I would still verify that.

-
Denis


On Thu, Jun 11, 2020 at 11:11 AM risomas  wrote:

> Hello,
> We have 12 identical azure servers in one network, each 32cpu and 64G ram.
> Ignite is running as docker container (identical including config). 8 our
> servers are working fine. But 4 are working only sometimes - they cant join
> and even worse they are blocking also others. I have recreated these VM
> (preserving NIC due licensing of some plugins) but it dint help. Sometimes
> these 4 nodes are working fine and they can connect togather, but sometimes
> they cant even connect if they are just 2. Identical behavior is while
> running as docker-compose or kubernetes. Once servers are connected there
> is
> no issue they are working fine.
> We are using ignite 2.8.1
>
> I am attaching logs and config where just 2 servers in the network:
>
> node8.node8
> 
> node7.node7
> 
> both logs are continuing in infinity error loop.
>
> 2.xml 
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Ignite concepts

[Denis Magda]

Eugene,

In such a case, I would go for the thin client for several reasons:

   - With the thick client (aka. standard client), you will come across
   several limitations if the servers are deployed on VMs, but the thick
   clients are running in a K8S environment. Those limitations are to be
   addressed soon [1].
   - With the partition-awareness feature [2], the thin client will be able
   to send requests to those server nodes that store the primary copy of a
   requested record. The functionality should be released in Ignite 2.9 and
   already exists in GridGain.

I don't think that you need to be bothered about a connection pool. Just
have a single thin client connection per service instead and enable the
partition-awareness. If you don't use the latter feature, then ensure that
the thin clients on those service instances are connected to random proxy
servers (you need to pass different IPs of the server nodes in the
connection string parameter [3]).

 One thing I am not clear on is in the scenario where we need to check 3
> different caches, for a single call to our service, is  this going to be an
> expensive operation?
>  Will we have to initiate a connection to Ignite for every call to the
> cache?


To help with this, I need to understand your business use case better.
There is a chance you can do all the checks with one network hop.

[1]
http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
[2]
https://www.gridgain.com/docs/latest/developers-guide/thin-clients/getting-started-with-thin-clients#partition-awareness
[3]
https://apacheignite.readme.io/docs/java-thin-client-initialization-and-configuration#initialization

-
Denis


On Thu, Jun 11, 2020 at 12:12 AM Eugene McGowan 
wrote:

> Thanks for getting back to me on this Dennis,
> The company's  Ignite Cluster is already up and running on VMs on our
> company network. The service my team is standing up will run in our Private
> cloud K8S environment.
>
> Regarding our usecase we will have say 2 or 3 caches, our service will be
> passed 3 parameters , if it gets a match to parameter 1 in cache 1 no
> further action is required , if not it needs to check cache 2 for parameter
> 2  ,again if it gets a match it uses data from cache 2 and no further
> action is required , otherwise it needs to check cache 3 . So cache 1 has
> most relevant data.
> We would have somewhere between 2 and 10 instances of our service running
> servicing a max of 3000+ request per second
>
> Our preference if possible would be to use a thin client. One thing I am
> not clear on is in the scenario where we need to check 3 different caches,
> for a single call to our service, is  this going to be an expensive
> operation?
>  Will we have to initiate a connection to Ignite for every call to the
> cache?  From reading  connection pooling does not come out of the box with
> key value operations over thin client, but its possible to implement your
> own connection pooling, is it therefore possible to have long lived
> connections to avoid the expense of constantly connecting and disconnecting
> from Ignite caches.
>
> Regarding the possibility of using the SQL API the background that is:
>   (1) We have a rarely used usecase where we want to get a combination
> of data from all 3 caches , so was thinking the SQL API would be one way of
> doing that.
>   (2) Looking at the docs it seems that connection pooling is possible
> via JDBC over thin client , so was wondering would that  be a more
> performant approach, if it's not possible to have a connection pool for key
> value operations over the thin client.
>
> Regards Eugene
>
> On Thu, Jun 11, 2020 at 12:28 AM Denis Magda  wrote:
>
>> Hi Eugene,
>>
>> Let me help you with that as much as I can. Please help me understand the
>> following:
>>
>>- Is the cluster supposed to be deployed outside of Kubernetes (on
>>VMs or bare-metal)? Is this a private cloud (OpenShift, PCF) or a public
>>environment (Azure, AWS, etc.)?
>>- If you use primary keys to access the records, do you really need
>>SQL for this type of operations? Or do you perform additional filtering of
>>records with SQL operands?
>>
>> In the meantime, please check this page to understand the pros and cons
>> of thick and thin clients:
>> https://www.gridgain.com/docs/latest/installation-guide/deployment-modes#thick-vs-thin-clients
>>
>> -
>> Denis
>>
>>
>> On Wed, Jun 10, 2020 at 1:07 PM Eugene McGowan 
>> wrote:
>>
>>> Hi Igniters,
>>> Our company has recently started using Ignite and our team are now to it.
>>> We would like to use Ignite as a Cache layer for an upcoming project
>>>
>>> Our use case is that we have an Ignite cluster that we want to interact
>>> with , our client service is Springboot based and  running in a Kubernetes
>>> Cluster  but not co located with our Ignite cluster , we anticipate our
>>> service will receive *3000+ request per 

Re: Non Distributed Join between tables

[Craig Gresbrink]

I am making a fair amount of assumptions:

  1.  aff_b means that table c has an affinity key that includes table b's id 
plus table c's id. Is that true?
  2.  I get that table c has a foreign key to table b.
  3.  You are doing puts to your caches, then use the sql query engine with 
distjoins=false hoping to join the 3 tables. Is that all correct?

We are doing this very thing. We need to differentiate the fk in your rdbms and 
your affinity key in ignite. In ignite, table/cache c will have:

  1.  An object attribute and column of bid (b.id or b_id if that makes more 
sense).
  2.  An affinity key of aid + cid  (table a's pk/id plus table c's pk/id).

The question becomes, on a put to table/cache c, how do we get a.id since it is 
not in table c's columns/objectGetters?

Answer:  We created a facade of the ignite thick client and we do puts via our 
facade. Our facade then runs a query like "select aid from b where b.id = ?" 
and we pass in the value of c.getbid() for ?. We use the return value from that 
query to generate the affinity key.

It does say that you need to populate or do puts to table/cache b before c 
(unless you have another way to get the value a.id when putting object c).

Hopefully this helps but might possibly cause more questions.

Cheers,
Craig


From: manueltg89 
Sent: Thursday, June 11, 2020 1:05 PM
To: user@ignite.apache.org 
Subject: Re: Non Distributed Join between tables

CAUTION: This email originated outside 24 Hour Fitness. Do not click links or 
open attachments unless you recognize the sender and know the content is safe.


Yes, table c is child of table b. But, It is redundant in my RDBMS, Would
have other solution whitout changing my schema?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Non Distributed Join between tables

[manueltg89]

Yes, table c is child of table b. But, It is redundant in my RDBMS, Would
have other solution whitout changing my schema?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Non Distributed Join between tables

[Craig Gresbrink]

Is table c, a child of table b?

You have:

tbl_a  tbl_b   tbl_c
---   -
 aff_a aff_b

You want:

tbl_a  tbl_b   tbl_c
---   -
 aff_a aff_a

You want a "cosmic parent" in this case it seems that tbl_c is grandchild of A 
and table C's affinity key should have table A's "id" so that all related data 
ends up on the same node as the id of table a.

Cheers,
Craig

From: manueltg89 
Sent: Thursday, June 11, 2020 12:39 PM
To: user@ignite.apache.org 
Subject: Re: Non Distributed Join between tables

CAUTION: This email originated outside 24 Hour Fitness. Do not click links or 
open attachments unless you recognize the sender and know the content is safe.


Sorry Ilya, in my first post had an error. My structure is the following:

tbl_a  tbl_b   tbl_c
---   -
 aff_a aff_b

When I make: select * from tbl_a INNER JOIN tbl_b ON tbl_b.id = tbl_a.fk_id
INNER JOIN tbl_c.fk_id = tbl_b.id -> Return empty results.

Could I to have three partitioned caches here?

Is there any way to make this with join collocated and whitout replicated
cache?

Thanks un advance.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Non Distributed Join between tables

[manueltg89]

Sorry Ilya, in my first post had an error. My structure is the following:

tbl_a  tbl_b   tbl_c
---   -
 aff_a aff_b

When I make: select * from tbl_a INNER JOIN tbl_b ON tbl_b.id = tbl_a.fk_id
INNER JOIN tbl_c.fk_id = tbl_b.id -> Return empty results.

Could I to have three partitioned caches here? 

Is there any way to make this with join collocated and whitout replicated
cache?

Thanks un advance.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Ignite servers are not connecting to the cluster

[risomas]

Hello,
We have 12 identical azure servers in one network, each 32cpu and 64G ram.
Ignite is running as docker container (identical including config). 8 our
servers are working fine. But 4 are working only sometimes - they cant join
and even worse they are blocking also others. I have recreated these VM
(preserving NIC due licensing of some plugins) but it dint help. Sometimes
these 4 nodes are working fine and they can connect togather, but sometimes
they cant even connect if they are just 2. Identical behavior is while
running as docker-compose or kubernetes. Once servers are connected there is
no issue they are working fine.
We are using ignite 2.8.1

I am attaching logs and config where just 2 servers in the network:
 
node8.node8
  
node7.node7
  
both logs are continuing in infinity error loop.

2.xml   



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: ignite work dir

[narges saleh]

Hi Ilya,
 IGNITE_HOME is /opt/ignite and in ignite-log4j, the log file set to
 
I don't have any issue on the server side.

This is the exception I get on the client side:

Caused by: class org.apache.ignite.IgniteCheckedException: Work directory
does not exist and cannot be created: /ignite/work
at
org.apache.ignite.internal.util.IgniteUtils.workDirectory(IgniteUtils.java:9440)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.initializeConfiguration(IgnitionEx.java:2181)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1697)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1117)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:615)
at
org.apache.ignite.internal.jdbc2.JdbcConnection.getIgnite(JdbcConnection.java:311)
at
org.apache.ignite.internal.jdbc2.JdbcConnection.(JdbcConnection.java:240)
... 39 more

On Thu, Jun 11, 2020 at 10:04 AM Ilya Kasnacheev 
wrote:

> Hello!
>
> I recommend setting the property workDirectory of IgniteConfiguration
> class to desired value. And leaving IGNITE_HOME and IGNITE_WORK_DIR not
> specified.
>
> This may still cause exception if your e.g. logging subsystem is
> configured to use /ignite/work. Need to see the exception.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> чт, 11 июн. 2020 г. в 17:55, narges saleh :
>
>> Hi All,
>>
>> What's the best way to set ignite's work dir on the client side,
>> considering that the client and server nodes run under different users.
>> I am setting both IGNITE_HOME and IGNITE_WORK_DIR system variables (on
>> client), but I  am still getting the exception "/ignite/work" cannot be
>> created.
>>
>> thanks.
>>
>

Re: NPE during printing failure information

[akorensh]

Hi,
  This looks like a communication problem.
GridWorker [name=grid-nio-worker-tcp-comm-3

   Make sure all nodes can see each other. Test it with a minimum number of
nodes.
   If it still doesn't work then describe your test and send the full logs
and Ignite condifg.
Thanks, Alex
   




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: ignite work dir

[Ilya Kasnacheev]

Hello!

I recommend setting the property workDirectory of IgniteConfiguration class
to desired value. And leaving IGNITE_HOME and IGNITE_WORK_DIR not specified.

This may still cause exception if your e.g. logging subsystem is configured
to use /ignite/work. Need to see the exception.

Regards,
-- 
Ilya Kasnacheev


чт, 11 июн. 2020 г. в 17:55, narges saleh :

> Hi All,
>
> What's the best way to set ignite's work dir on the client side,
> considering that the client and server nodes run under different users.
> I am setting both IGNITE_HOME and IGNITE_WORK_DIR system variables (on
> client), but I  am still getting the exception "/ignite/work" cannot be
> created.
>
> thanks.
>

ignite work dir

[narges saleh]

Hi All,

What's the best way to set ignite's work dir on the client side,
considering that the client and server nodes run under different users.
I am setting both IGNITE_HOME and IGNITE_WORK_DIR system variables (on
client), but I  am still getting the exception "/ignite/work" cannot be
created.

thanks.

Ignite persistence and activation

[steve.hostettler]

Hello.

I am trying to implement ignite persistence but I stumbled upon the
following problems/questions. It is required to activate the cluster, that
much is clear but I have bootstrap code that is using technical caches that
I do not want to persist and more problematic I need to use
ignite.atomicReference and that as part of the initialization of the node.

I assume that I need to create a another region that is not persisted for
the so called system caches but what do I do with  ignite.atomicReference?


Thanks in advance



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Need to sync two apache ignite cache DB using ip address

[Ilya Kasnacheev]

Hello!

Yes, you should specify both local and remote addresses in discovery SPI
section:
https://apacheignite.readme.io/docs/tcpip-discovery#static-ip-finder

Regards,
-- 
Ilya Kasnacheev


чт, 11 июн. 2020 г. в 15:07, rakshita04 :

> So are you saying apache ignite automatically does that.
> Do i need to configure the ip of the other node somewhere in xml file so
> that data keeps syncing?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Need to sync two apache ignite cache DB using ip address

[rakshita04]

So are you saying apache ignite automatically does that.
Do i need to configure the ip of the other node somewhere in xml file so
that data keeps syncing?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Need to sync two apache ignite cache DB using ip address

[Ilya Kasnacheev]

Hello!

There are ways to do that (affinity backup filter), but why would you want
to? You are much better off relying on Ignite's data model when using
Apache Ignite.

Regards,
-- 
Ilya Kasnacheev


чт, 11 июн. 2020 г. в 15:02, rakshita04 :

> Hi Ilya,
>
> I want one node to be master for all entries.
> How can i achieve this?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Need to sync two apache ignite cache DB using ip address

[rakshita04]

Hi Ilya,

I want one node to be master for all entries.
How can i achieve this? 



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Need to sync two apache ignite cache DB using ip address

[Ilya Kasnacheev]

Hello!

If you declare your caches as REPLICATED they will be kept in sync. One
node will be master of half entries and the other will be master of the
remaining half.

Regards,
-- 
Ilya Kasnacheev


чт, 11 июн. 2020 г. в 14:36, rakshita04 :

> Hi Team,
>
> We have 2 apache ignite DBs located at separate IPs and connected through
> TCP-ip connection.
> ne of this DB should work as master dB and on any change should sync to
> other apache ignite DB and update the entries accordingly.
> How can i achieve this ? what changes do i need to do in xml file?
> I am using C++ platform for apache ignite.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Need to sync two apache ignite cache DB using ip address

[rakshita04]

Hi Team,

We have 2 apache ignite DBs located at separate IPs and connected through
TCP-ip connection.
ne of this DB should work as master dB and on any change should sync to
other apache ignite DB and update the entries accordingly.
How can i achieve this ? what changes do i need to do in xml file?
I am using C++ platform for apache ignite.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Check service location possible ?

[Mikael]

Ah, of course, thanks.

Den 2020-06-11 kl. 10:41, skrev Alex Plehanov:

Hello,

IgniteServices.service(String name) method return service instance if 
it's deployed locally or null.


чт, 11 июн. 2020 г. в 11:16, Mikael >:


Hi!

Is there any good way to detect if a service is running local or
remote
? I guess I could check the instance of the returned proxy and see
if it
is the implementation class or not, but it feels a bit ugly, is there
some other nicer way to do this ?

Re: Check service location possible ?

[Alex Plehanov]

Hello,

IgniteServices.service(String name) method return service instance if it's
deployed locally or null.

чт, 11 июн. 2020 г. в 11:16, Mikael :

> Hi!
>
> Is there any good way to detect if a service is running local or remote
> ? I guess I could check the instance of the returned proxy and see if it
> is the implementation class or not, but it feels a bit ugly, is there
> some other nicer way to do this ?
>
>
>

NPE during printing failure information

[Andrey Davydov]

Hello, We start test our system on Ignite 2.8.1 and got very strange log. As I understand, It was error in system (on 2.7.6 this test works without any problem, I will investigate it later) and it was NPE during handling of this error:  [01:08:11] Configured failure handler: [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT[01:08:11] Message queue limit is set to 0 which may lead to potential OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender and receiver sides.2020-06-11 01:08:22,677 [tcp-disco-msg-worker-[crd]-#1225%TestNode-0%] WARN   :119 - Possible failure suppressed accordingly to a configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=grid-nio-worker-tcp-comm-3, igniteInstanceName=TestNode-0, finished=false, heartbeatTs=1591837692022]]]org.apache.ignite.IgniteException: GridWorker [name=grid-nio-worker-tcp-comm-3, igniteInstanceName=TestNode-0, finished=false, heartbeatTs=1591837692022] at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$3.apply(IgnitionEx.java:1810) ~[ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$3.apply(IgnitionEx.java:1805) ~[ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:234) ~[ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297) ~[ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2858) ~[ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7759) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2946) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7697) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:61) [ignite-core-2.8.1.jar:2.8.1][01:08:22] Possible failure suppressed accordingly to a configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=grid-nio-worker-tcp-comm-3, igniteInstanceName=TestNode-0, finished=false, heartbeatTs=1591837692022]]]2020-06-11 01:08:22,685 [tcp-disco-msg-worker-[crd]-#1225%TestNode-0%] WARN   :119 - Possible failure suppressed accordingly to a configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=grid-nio-worker-tcp-comm-0, igniteInstanceName=TestNode-0, finished=false, heartbeatTs=1591837692022]]]org.apache.ignite.IgniteException: GridWorker [name=grid-nio-worker-tcp-comm-0, igniteInstanceName=TestNode-0, finished=false, heartbeatTs=1591837692022] at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$3.apply(IgnitionEx.java:1810) ~[ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$3.apply(IgnitionEx.java:1805) ~[ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:234) ~[ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297) ~[ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2858) ~[ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7759) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2946) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) [ignite-core-2.8.1.jar:2.8.1] at 

Check service location possible ?

[Mikael]

Hi!

Is there any good way to detect if a service is running local or remote 
? I guess I could check the instance of the returned proxy and see if it 
is the implementation class or not, but it feels a bit ugly, is there 
some other nicer way to do this ?

RE: Re:RE: Gridgain8.7.14 in kubernetes problem

[Saxena, Ravi (CMS)]

This is just a workaround.
Delete your ignite cluster and all the related PV'sand later install with 
different service name
Or you can also try to re-deploy with persistence off

Regards,
Ravi Saxena
Stay home stay safe

From: s [mailto:13581902...@163.com]
Sent: Thursday, June 11, 2020 8:13 AM
To: user@ignite.apache.org; Saxena, Ravi (CMS) 
Subject: Re:RE: Gridgain8.7.14 in kubernetes problem


Could you be more specific



These three pods, I plan to form a cluster



Or, is there a successful configuration running in kubernetes? Can you provide 
it, including config, yaml and JVM parameters? Thank you







At 2020-06-10 17:56:55, "Saxena, Ravi (CMS)" 
mailto:ravi.sax...@hpe.com>> wrote:
Delete and redeploy with different name.

Regards,
Ravi Saxena
Stay home stay safe

From: Denis Magda [mailto:dma...@apache.org]
Sent: Wednesday, June 10, 2020 10:05 AM
To: user mailto:user@ignite.apache.org>>
Subject: Re: Gridgain8.7.14 in kubernetes problem

Hi,

This user list is intended for Ignite questions. Please direct all GridGain 
related questions to their forums or 
StackOverflow.

-
Denis


On Tue, Jun 9, 2020 at 6:59 PM s 
<13581902...@163.com> wrote:

Hello



When I used the official image of gridgain8.7.14 to deploy on kubernetes, I 
encountered the problem of slow cluster formation according to the official 
document instructions. In addition, with all the time default parameters, the 
third node cannot join the cluster. The third node reports an error as follows. 
See the attachment for more detailed logs.



Is there a similar problem? Do you need to configure special parameters?


Thank you for your attention.



class org.apache.ignite.IgniteCheckedException: Failed to start SPI: 
TcpDiscoverySpi [addrRslvr=null, sockTimeout=5000, ackTimeout=5000, 
marsh=JdkMarshaller 
[clsFilter=org.apache.ignite.marshaller.MarshallerUtils$1@1e287667], 
reconCnt=10, reconDelay=2000, maxAckTimeout=60, soLinger=5, 
forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null, 
skipAddrsRandomization=false]
at 
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:299)
at 
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:961)
at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1902)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1230)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2062)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1733)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1147)
at 
org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1065)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:951)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:850)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:720)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:689)
at org.apache.ignite.Ignition.start(Ignition.java:344)
at 
org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:299)
Caused by: class org.apache.ignite.spi.IgniteSpiException: Node with the same 
ID was found in node IDs history or existing node in topology has the same ID 
(fix configuration and restart local node) [localNode=TcpDiscoveryNode 
[id=0b1c3da6-7800-4efa-b468-6fd810491590, 
consistentId=12.111.9.34,127.0.0.1:47500, 
addrs=ArrayList [12.111.9.34, 127.0.0.1], sockAddrs=HashSet 
[ignite-cluster-57c7f744f9-kcph2/12.111.9.34:47500, 
/127.0.0.1:47500], discPort=47500, order=0, intOrder=3, 
lastExchangeTime=1591147553777, loc=true, ver=8.7.14#20200320-sha1:419ab454, 
isClient=false], existingNode=0b1c3da6-7800-4efa-b468-6fd810491590]
at 
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.duplicateIdError(TcpDiscoverySpi.java:1997)
at 
org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:1163)
at org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:444)
at 
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2121)
at 
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:296)
... 13 more
[01:25:53,949][SEVERE][main][IgniteKernal] Got exception while starting (will 
rollback startup routine).
class org.apache.ignite.IgniteCheckedException: Failed to start manager: 
GridManagerAdapter [enabled=true, 
name=org.apache.ignite.internal.managers.discovery.GridDiscoveryManager]
at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1907)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1230)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2062)
at 

Re: Ignite concepts

[Eugene McGowan]

Thanks for getting back to me on this Dennis,
The company's  Ignite Cluster is already up and running on VMs on our
company network. The service my team is standing up will run in our Private
cloud K8S environment.

Regarding our usecase we will have say 2 or 3 caches, our service will be
passed 3 parameters , if it gets a match to parameter 1 in cache 1 no
further action is required , if not it needs to check cache 2 for parameter
2  ,again if it gets a match it uses data from cache 2 and no further
action is required , otherwise it needs to check cache 3 . So cache 1 has
most relevant data.
We would have somewhere between 2 and 10 instances of our service running
servicing a max of 3000+ request per second

Our preference if possible would be to use a thin client. One thing I am
not clear on is in the scenario where we need to check 3 different caches,
for a single call to our service, is  this going to be an expensive
operation?
 Will we have to initiate a connection to Ignite for every call to the
cache?  From reading  connection pooling does not come out of the box with
key value operations over thin client, but its possible to implement your
own connection pooling, is it therefore possible to have long lived
connections to avoid the expense of constantly connecting and disconnecting
from Ignite caches.

Regarding the possibility of using the SQL API the background that is:
  (1) We have a rarely used usecase where we want to get a combination
of data from all 3 caches , so was thinking the SQL API would be one way of
doing that.
  (2) Looking at the docs it seems that connection pooling is possible
via JDBC over thin client , so was wondering would that  be a more
performant approach, if it's not possible to have a connection pool for key
value operations over the thin client.

Regards Eugene

On Thu, Jun 11, 2020 at 12:28 AM Denis Magda  wrote:

> Hi Eugene,
>
> Let me help you with that as much as I can. Please help me understand the
> following:
>
>- Is the cluster supposed to be deployed outside of Kubernetes (on VMs
>or bare-metal)? Is this a private cloud (OpenShift, PCF) or a public
>environment (Azure, AWS, etc.)?
>- If you use primary keys to access the records, do you really need
>SQL for this type of operations? Or do you perform additional filtering of
>records with SQL operands?
>
> In the meantime, please check this page to understand the pros and cons of
> thick and thin clients:
> https://www.gridgain.com/docs/latest/installation-guide/deployment-modes#thick-vs-thin-clients
>
> -
> Denis
>
>
> On Wed, Jun 10, 2020 at 1:07 PM Eugene McGowan 
> wrote:
>
>> Hi Igniters,
>> Our company has recently started using Ignite and our team are now to it.
>> We would like to use Ignite as a Cache layer for an upcoming project
>>
>> Our use case is that we have an Ignite cluster that we want to interact
>> with , our client service is Springboot based and  running in a Kubernetes
>> Cluster  but not co located with our Ignite cluster , we anticipate our
>> service will receive *3000+ request per second* , the service will scale
>> based on the volume of requests, and we will need to query Ignite for each
>> of these requests.
>> The data we need to query is fairly straight forward , a single row from
>>  either 1 or 2  caches based on primary key.
>> We are at the point where we are beginning to design the architecture.
>>
>> I have done some initial reading on Ignite and was wondering if someone
>> could
>> Verify my understanding is correct.
>>
>> I would also be interested to know  what architectures  other folks are
>> using for similar use cases
>>
>> *===*
>> *Thin Client*
>> 
>>   JDBC connection are not thread safe
>> https://apacheignite-sql.readme.io/docs/jdbc-driver
>>   To use this approach we need to ensure non cuncurrent use of
>> Connections, Statements, and ResultSet perhaps using a locking mechanism.
>>
>>   Key value operations or SQL Queries via the API over the thin client
>> are thread safe
>>   Connection Pooling is not provided via the API , but it is possible to
>> implement your own connection pool.
>>   If you implement your own connection pool you could keep connections
>> alive to reduce
>>   the cost of establishing a connection to Ignite and reusing
>>   the connections for various requests.
>>
>>
>> ==
>> *Thick Client *
>> ==
>>
>>   Does not have a concept of Connection pooling
>>
>>   When using either JDBC or the IGNITE API on the Thick Client
>>   It is thread safe and can handle multiple concurrent threads
>>
>>   Is part of the Ignite cluster
>>   Ignite Servers can iniatate TCP connection  needs to be reachable via a
>> TCP connection
>>   If the Client is behind a firewall,NAT or Load Balancer it makes using
>> ,thick client challenging
>>
>>
>>
>> =
>> *Question on GRPC*
>> =
>>
>> The Thin Client TCP socket connection uses a Binary protocol
>>