Ignite Backup doubts

[the_palakkaran]

Hi,

I have 3 nodes. I have my caches on heap enabled. Also I have persistence
enabled. I have configured 2 backups for each cache. 

As I understand, I can use RendezvousAffinityFunction to exclude neighbors,
so that backup of a cache in Node-1 on a Machine-1 will be stored in a
machine other than itself.

Now, because of this at any point of time, there can be at least a primary
of the same node and a backup of some other node on a single machine. 

My doubt is, 

(1) Will my cache.get(key) be run on the primary and backups on the machine? 
If so, can I limit this ? 

(2) Also, every time I do a get operation or query, will it be executed on
every node?

(3) Again is there an optimal backup configuration based on an N number of
nodes?

(4) When a node in the cluster is down, will another backup be created for
the primary and backups that were on that node on other available nodes?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite clustering doubts

[the_palakkaran]

Why do I want to map keys to partition ids? Can you tell me more, please? 

By keys, it means the keys used for the cache right? If so, particular keys
will always go to some node, is that so?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: How are SQL Queries executed on Ignite Cluster

[Sanjeev]

trying to understand this:

1) In case where no indexes are involved and you are doing a table scan, it
should automatically try to exploit available CPU cores and process each
partition on a separate thread/core. At least table scan queries should
entertain the idea dynamic parallelism through DML hints.

2) In case of indexes, what you are saying is that N trees are build for M
primary partitions on a node, where N being degree of parallelism. So each
tree is managing a certain number of partitions, M/N. As number of partition
on a nodes increase or decrease, the N trees are adjusted to reflect that. 

What I am wondering is in case if indexes were created, then could we always
create N trees. What are the performance implications of:
1) A single thread working on 1 large single index
2) A single thread working on 1 or few of the N small indexes based on the
query.
3) N cores working on N small indexes in parallel. 

3 should always perform well. Between 1 and 2, would one perform better or
worse.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Zookeeper discovery register all network interfaces address

[moon-duck]

Thanks! It is what I totally want.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Continuous query - Exactly once based event across multiple nodes..

[Raymond Wilson]

I another possibility to create a continuous query per node in your node
affinity set for the cache and have the continuous query return local
values, like this:

using (IContinuousQueryHandle>
queryHandle = queueCache.QueryContinuous
(qry: new ContinuousQuery< Key, Value >(new LocalListener())
{ Local = true },
 initialQry: new ScanQuery< Key, Value > { Local = true }))
(
// Perform the initial query to grab all existing elements
foreach (var item in queryHandle.GetInitialQueryCursor())
{
if (NodeIsPrimaryForThisKey(Key)) // Don’t let backups
get involved
   handler.Add(item.Key);
}

// move into steady state management of arriving elements...
)

-Original Message-
From: Николай Ижиков [mailto:nizhikov@gmail.com] On Behalf Of Nikolay
Izhikov
Sent: Monday, May 7, 2018 6:40 PM
To: user@ignite.apache.org
Cc: JP 
Subject: Re: Continuous query - Exactly once based event across multiple
nodes..

Hello, JP.

You should use target node in remote filter.

You should check "Is primary node for some record equal to target node?" in
your filter.
Please, see code below.
You can find related discussion and full example here [1].

@IgniteAsyncCallback
public static class RemoteFactory implements
Factory> {
private final ClusterNode node;

public RemoteFactory(ClusterNode node) {
this.node = node;
}

@Override
public CacheEntryEventFilter create() {
return new CacheEntryEventFilter() {
@IgniteInstanceResource
private Ignite ignite;

@Override
public boolean evaluate(CacheEntryEvent cacheEntryEvent) {
Affinity aff = ignite.affinity("myCache");

ClusterNode primary =
aff.mapKeyToNode(cacheEntryEvent.getKey());

return primary.id().equals(node.id());
}
};
}
}



[1] https://issues.apache.org/jira/browse/IGNITE-8035


В Вс, 06/05/2018 в 23:33 -0700, JP пишет:
> Using continuous query,
>
> How to achieve event trigger for cache exactly only once per key even
> if continuous query is listening in multiple nodes or multiple listener.
> example:
> 1. Scenario 1:
>  Node A: Start Continuous query
>  Node B: Start Continuous query
>  Node C: Insert or Update or Delete record ex: number from 1 to 100
>
> Expected Output should be as below
>  Node A - 1 ,2,3,4,550
>  Node B - 51, 52, 53, 54, ... 100
>  Above output is the expected output. Here, event per key should be
> triggered exactly once across nodes.
>
> Actual Output should be as below
>  Node A - 1 ,2,3,4,5,100
>  Node B - 1, 2, 3, 4,5 ... 100
>
> If this is not possible in Continuous query, then is there any way to
> achieve this.
>
> 2. Scenario 2:
> To achieve expected output,
>  I am using singleton service per Cluster.
> Ex: Cluster A
>   - Singleton service with Continuous query for cache
> Here problem is, service is running in only one instance.
> How to achieve above output with multiple instance of service?
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Zookeeper discovery register all network interfaces address

[vkulichenko]

By default Ignite binds to all interfaces (0.0.0.0) and therefore publish all
available IPs. If you want to bind to a specific address, you can set
IgniteConfiguration#localHost property. In this case only this address would
be published.

-Val



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Continuous query - Exactly once based event across multiple nodes..

[vkulichenko]

JP,

Do you have a solution for this? Do you need any more help?

-Val



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Using Java executor service inside Ignite transaction

[vkulichenko]

Prasad,

What is your business use case and what do you mean by processing? Please
describe in more detail. Are you sure you need to use the transaction here
in the first place?

-Val



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

[Article] Using Linear Regression with Apache Ignite

[Akmal Chaudhri]

Second article in a multi-part series:

https://www.gridgain.com/resources/blog/using-linear-regression-apacher-ignitetm

Feedback welcome.

Thank you.

Re: Ignite C# Geometry

[Jonathan Mayer]

Hi Andrew

Thanks for the reply.

" mock vividsolution classes" sounds interesting - is there an example of
how to mock a class with BinaryObjects?

Many Thanks

Jonathan



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite Spring Integration seems broken with 2.4 (upgrading from 2.3)

[Denis Magda]

The issue will be addressed once this ticket is resolved:
https://issues.apache.org/jira/browse/IGNITE-8740



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite 2.5 nodes do not rejoin the cluster after restart (works on 2.4)

[szj]

Hi

I'm afraid I wiped Ignite off my servers already as this behaviour was a
blocker to me. I only needed a key value store able to replicate across
several datacenters across the globe (my use case involves very few writes)
and I'm now evaluating another product already.

I strongly suggest you try to reproduce it with the exact steps I listed in
this thread if you didn't try it already. It's dead simple to me - there are
2 nodes running, both in the baseline, you connect ignitevisorcmd.sh then
shut down one node. Nothing else starts nor stops in the meantime, you just
try to start up the shut down node again. In my tests it was 100% clear that
ignitevisorcmd.sh being connected was the culprit of the 2.5 cluster being
confused. It did not happen in 2.4.

Good luck :-)



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: IgniteCache invoke CacheEntryProcessor, not throw EntryProcessorException

[Andrey Mashenkov]

Hi,

Yes, but no.
You will observe lower latency per operation on client and you will be able
to utilize more server resources with less number of threads.
But operation logic will be the same, no steps will be omit (backups will
be updated as well)... so, maximal performance that can be achieved will be
the same.


On Thu, Jun 7, 2018 at 7:05 PM, haotian.chen 
wrote:

> Awesome, that's super clear!
> One last question, will full_async have better performance over the other
> two usually?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov

Re: IgniteCache invoke CacheEntryProcessor, not throw EntryProcessorException

[haotian.chen]

Awesome, that's super clear! 
One last question, will full_async have better performance over the other
two usually?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: ignite peer class loading query

[Andrey Mashenkov]

1. Peer-classloading is designed for compute jobs. All other classes
(except may be user domain classes when BinaryObjects are used) must be in
classpath on all nodes.
2. Next is true for transactional cache. Client node must have a CacheStore
drivers as well as server nodes. This because a request-initiator node is
responsible to update cache store.
That means when you go put() from client, the client node will be
responsible for update underlying store.
May be this will be fixed at least for atomic caches in future versions,
but the requirement to have drivers on client doesn't look a serious issue.


On Thu, Jun 7, 2018 at 6:11 PM, David Harvey  wrote:

> Certain kinds of requests (e.g., remote calls) carry enough information
> for the receiver of the message to do peer class loading.   The main
> purpose of peer class loading is to avoid the need to restart the server to
> install software, and the peer class loading is therefore done by the
> server requesting the class from the client who sent the request.
>
> It is not targeted at what you are trying to do.
>
> On Thu, Jun 7, 2018 at 10:28 AM, vbm  wrote:
>
>> Hi,
>>
>> I have a Ignite server running with 3rd partyDB (MYSQL). I have the below
>> configuration in xml file:
>>
>> > class="org.springframework.jdbc.datasource.DriverManagerDataSource">
>>   
>>
>>   
>>   
>> 
>>
>> 
>> * *
>>   ...
>> 
>> 
>>
>> On the server the jar for mysql jdbc driver is present iand the servers
>> are
>> coming up successfully.
>> I have placed the mysql jdbc driver in a seperate folder and added it to
>> the
>> path using USER_LIBS env.
>>
>>
>> Now if I try to start a client, with the same the xml (only extra thing is
>> the client mode part), the client ignite node doesn't start and throws
>> below
>> error.
>>
>> * org.springframework.beans.MethodInvocationException: Property
>> 'driverClassName' threw exception; nested exception is
>> java.lang.IllegalStateException: Could not load JDBC driver class
>> [com.mysql.jdbc.Driver]*
>>
>> As peer class loading is enabled, I expect the client to get the class
>> from
>> the server. Why is this not happening ?
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>>
>
>
> *Disclaimer*
>
> The information contained in this communication from the sender is
> confidential. It is intended solely for use by the recipient and others
> authorized to receive it. If you are not the recipient, you are hereby
> notified that any disclosure, copying, distribution or taking action in
> relation of the contents of this information is strictly prohibited and may
> be unlawful.
>
> This email has been scanned for viruses and malware, and may have been
> automatically archived by *Mimecast Ltd*, an innovator in Software as a
> Service (SaaS) for business. Providing a *safer* and *more useful* place
> for your human generated data. Specializing in; Security, archiving and
> compliance. To find out more Click Here
> .
>



-- 
Best regards,
Andrey V. Mashenkov

Re: Cache getting cleareing automatically

[Andrey Mashenkov]

Ignite should support persistence format of older versions.
Also, please, take a look at Baseline topology feature that is available
from 2.4 version.


[1] https://apacheignite.readme.io/docs/baseline-topology

On Thu, Jun 7, 2018 at 5:35 PM, siva  wrote:

> If we switch to 2.5 version ,what about data ? I think there is no
> continuous
> upgrade in ignite
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov

Re: Ignite clustering doubts

[Ilya Kasnacheev]

Hello!

Affinity function determines mapping of keys to partition ids and of
partition ids to nodes. Usually you only override the first part since
changing the second one is a research problem.

Regards,

-- 
Ilya Kasnacheev

2018-06-07 18:36 GMT+03:00 the_palakkaran :

> Wow ! Thanks a ton. If there was an upvote thing here,I would have
> surely
> broke it.
>
> Can you tell me what is the need of AffinityFunction then? Also can I
> control this Rendezvous Hashing?
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Ignite clustering doubts

[the_palakkaran]

Wow ! Thanks a ton. If there was an upvote thing here,I would have surely
broke it.

Can you tell me what is the need of AffinityFunction then? Also can I
control this Rendezvous Hashing?




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: C++ Distribute Closure Failure

[Igor Sapego]

Depending on your configuration, job is either going
to be routed to another node for execution, or you will
get an exception. See [1] for details.

[1] - https://apacheignite-cpp.readme.io/docs/fault-tolerance

Best Regards,
Igor

On Thu, Jun 7, 2018 at 5:15 PM, F.D.  wrote:

> How can I know if I'm waiting because my job is complex and it takes much
> time, or because the node where it was running is crashed?
>
> Regards,
>F.D.
>
> On Thu, Jun 7, 2018 at 4:01 PM Igor Sapego  wrote:
>
>> What do you mean by "failed"?
>>
>> Best Regards,
>> Igor
>>
>> On Thu, Jun 7, 2018 at 2:33 PM, F.D.  wrote:
>>
>>> Hi,
>>> when I launch a distibute closure calling the Call method or by
>>> RunAsync, how can I check if the function is failed? Only by timeout?
>>>
>>> Thanks,
>>>F.D.
>>>
>>
>>

Re: Ignite user create/modify trouble

[engrdean]

Thanks Andrei, adding quotes and realizing that case matters made all of the
difference.

I did read the documentation around this but my impression was that a
connection via JDBC should be case insensitive and that does not appear to
be the case.  From the documentation:

/For instance, if test was set as a username then:

You can use Test, TEst, TEST and other combinations from JDBC and ODBC.
You have to use TEST as the username from Ignite's native SQL APIs designed
for Java, .NET and other programming languages./

I took that to mean that since I am connecting via JDBC that the username is
case insensitive.  Am I just misunderstanding?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite clustering doubts

[Ilya Kasnacheev]

Hello!

This might be helpful: https://apacheignite.readme.io/docs/data-grid

1. It will be done automatically, every node will have around 1 records.

2. They will be distributed according to Rendezvous hashing of partition
ids.

3. If you have sufficient backups, data will be re-balanced to ensure that
# of backups. If you don't have sufficient backups you will lose data.

4. Yes, you do. It depends on your setup and reliability considerations.

5. As far as I remember you had some pretty non-trivial considerations, so
your mileage may vary. Try it and see.

6. Yes, e.g. by using localPeek().

7. Why would you want to?

8. I'm afraid you might need to have models deployed universally to use
externalized classes.

Regards,

-- 
Ilya Kasnacheev

2018-06-07 15:04 GMT+03:00 the_palakkaran :

> Hi,
>
> The below is my scenario:
>
> I have a CustomerCache. I have externalizable keys and models. Customers
> ids
> range from 1 to 4. I have 4 nodes.
>
> I have few doubts on clustering in ignite:
>
> 1. How can I restrict 1 records on each node?
>
> 2. How does ignite distribute records between each node? On what basis? Is
> there any relevance of AffinityFunction here? Is there a demo of how to use
> that ?
>
> 3. How would fail over and re-balancing work when a node fails?
>
> 4. Do I need to configure number of backups? Is the number of backups
> dependent on number of nodes?
>
> 5. Will I have any problem to access data from any of these nodes or a
> client node since I have used externalization?
>
> 6. Is there any possible way to query data on a single node?
>
> 7. Leader election is based on age, as per documentation. Can I control
> this
> election?
>
> 8. Zero deployment means I do not need to have models deployed on every
> class right? Is this the same case for externalized models?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: ignite peer class loading query

[David Harvey]

Certain kinds of requests (e.g., remote calls) carry enough information for
the receiver of the message to do peer class loading.   The main purpose of
peer class loading is to avoid the need to restart the server to install
software, and the peer class loading is therefore done by the server
requesting the class from the client who sent the request.

It is not targeted at what you are trying to do.

On Thu, Jun 7, 2018 at 10:28 AM, vbm  wrote:

> Hi,
>
> I have a Ignite server running with 3rd partyDB (MYSQL). I have the below
> configuration in xml file:
>
>  class="org.springframework.jdbc.datasource.DriverManagerDataSource">
>   
>
>   
>   
> 
>
> 
> * *
>   ...
> 
> 
>
> On the server the jar for mysql jdbc driver is present iand the servers are
> coming up successfully.
> I have placed the mysql jdbc driver in a seperate folder and added it to
> the
> path using USER_LIBS env.
>
>
> Now if I try to start a client, with the same the xml (only extra thing is
> the client mode part), the client ignite node doesn't start and throws
> below
> error.
>
> * org.springframework.beans.MethodInvocationException: Property
> 'driverClassName' threw exception; nested exception is
> java.lang.IllegalStateException: Could not load JDBC driver class
> [com.mysql.jdbc.Driver]*
>
> As peer class loading is enabled, I expect the client to get the class from
> the server. Why is this not happening ?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
>

Disclaimer

The information contained in this communication from the sender is 
confidential. It is intended solely for use by the recipient and others 
authorized to receive it. If you are not the recipient, you are hereby notified 
that any disclosure, copying, distribution or taking action in relation of the 
contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been 
automatically archived by Mimecast Ltd, an innovator in Software as a Service 
(SaaS) for business. Providing a safer and more useful place for your human 
generated data. Specializing in; Security, archiving and compliance. To find 
out more visit the Mimecast website.

Re: IgniteSpringBean & Ignite SpringTransactionManager broken with 2.4?

[ilya.kasnacheev]

Hello!

I went ahead and filled a ticket:
https://issues.apache.org/jira/browse/IGNITE-8740

Regards,



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Cache getting cleareing automatically

[siva]

If we switch to 2.5 version ,what about data ? I think there is no continuous
upgrade in ignite



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

ignite peer class loading query

[vbm]

Hi,

I have a Ignite server running with 3rd partyDB (MYSQL). I have the below
configuration in xml file:


  
   
  
  



* *
  ...



On the server the jar for mysql jdbc driver is present iand the servers are
coming up successfully.
I have placed the mysql jdbc driver in a seperate folder and added it to the
path using USER_LIBS env.


Now if I try to start a client, with the same the xml (only extra thing is
the client mode part), the client ignite node doesn't start and throws below
error.

* org.springframework.beans.MethodInvocationException: Property
'driverClassName' threw exception; nested exception is
java.lang.IllegalStateException: Could not load JDBC driver class
[com.mysql.jdbc.Driver]*

As peer class loading is enabled, I expect the client to get the class from
the server. Why is this not happening ?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: C++ Distribute Closure Failure

[F.D.]

How can I know if I'm waiting because my job is complex and it takes much
time, or because the node where it was running is crashed?

Regards,
   F.D.

On Thu, Jun 7, 2018 at 4:01 PM Igor Sapego  wrote:

> What do you mean by "failed"?
>
> Best Regards,
> Igor
>
> On Thu, Jun 7, 2018 at 2:33 PM, F.D.  wrote:
>
>> Hi,
>> when I launch a distibute closure calling the Call method or by RunAsync,
>> how can I check if the function is failed? Only by timeout?
>>
>> Thanks,
>>F.D.
>>
>
>

Re: Cache getting cleareing automatically

[Andrey Mashenkov]

Hi,

Try to switch to the latest Ignite 2.5 version as it includes many bugfixes
related to cluster stability.

On Thu, Jun 7, 2018 at 4:57 PM, siva  wrote:

> 1.we are using ignite 2.3 version ,
> 2.We are using an Api.Client node is a spring boot rest application ,we
> have
> written an api to perform curd operations and as well as clearing and
> destroying cache.Bellow is the configuration
>
> cache mode initially *Partitioned * .Later changed to Replicated mode
>
> CacheConfiguration cacheCfg = new
> CacheConfiguration<>(cacheName);
> cacheCfg.setCacheMode(CacheMode.REPLICATED); // Default.
> cacheCfg.setBackups(1);
> cacheCfg.setIndexedTypes(String.class, Entity.class);
> cacheCfg.setAtomicityMode(atomic);
> // client node will wait for write or commit on all
> particpating nodes
>
> cacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.
> FULL_SYNC);
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov

Re: Does equals method have relevance in Custom Key Class

[Andrey Mashenkov]

Yes. there are scenarios you can't use simple data types. This relates to
not only Ignite.

As Ignite SQL layer is built over key-value storage, there always be some
overhead on SQL operations.

Key-value API allow you operate with one or more cache entries with known
keys.
If the keys of entries are unknown then the only way to retrieve entries
efficiently is to use some index: either SQL or implement your own custom
IndexingSPI,




On Wed, Jun 6, 2018 at 3:36 PM, the_palakkaran  wrote:

> But you cannot use simple datatypes all the time in a real scenario,
> right? I
> mean when you need to get results based on multiple fields/columns [unless
> I
> form a key or something based on them]
>
> As per different discussions here in the forum and from some demo stubs, I
> understand performance of ignite in the order high to low will be :
>
> 1. Using Binary Objects direct get
> 2. Using normal serialiazable objects
> 3. Using Externalizable direct get
> 4. Using SQLField query [ queries will always be the one taking most
> performance, from my experience]
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov

Re: C++ Distribute Closure Failure

[Igor Sapego]

What do you mean by "failed"?

Best Regards,
Igor

On Thu, Jun 7, 2018 at 2:33 PM, F.D.  wrote:

> Hi,
> when I launch a distibute closure calling the Call method or by RunAsync,
> how can I check if the function is failed? Only by timeout?
>
> Thanks,
>F.D.
>

Re: Cache getting cleareing automatically

[siva]

1.we are using ignite 2.3 version ,
2.We are using an Api.Client node is a spring boot rest application ,we have
written an api to perform curd operations and as well as clearing and
destroying cache.Bellow is the configuration 

cache mode initially *Partitioned * .Later changed to Replicated mode

CacheConfiguration cacheCfg = new
CacheConfiguration<>(cacheName);
cacheCfg.setCacheMode(CacheMode.REPLICATED); // Default.
cacheCfg.setBackups(1);
cacheCfg.setIndexedTypes(String.class, Entity.class);
cacheCfg.setAtomicityMode(atomic);
// client node will wait for write or commit on all 
particpating nodes

cacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Cache getting cleareing automatically

[Andrey Mashenkov]

Hi,

What Ignite version do you use? Looks like it was fixed in 2.5 [1].
How are you create and destroy a cache? via API or SQL?


[1] https://issues.apache.org/jira/browse/IGNITE-8021

On Thu, Jun 7, 2018 at 4:32 PM, siva  wrote:

>
> Hi,
> We have two ignite server nodes in single physicle machine,and one client
> node.Dont know what happend cache getting clear automatically.Initially We
> have created a cache name called "*c091e548-b45a-49b4-b8ec-2cb5e27c7af6*"
> with "*partitioned*" mode.Later we destroy the same cache and configured as
> "*Replicated*" mode and storing entities under the same cache.But after
> some
> days cache got cleared and cache changed to partitioned mode automatically
> and again & again same problem facing every 10 min.What might be the issue?
> I have gone through the logs  and
>
>  found that nodes were  disconnected and again connected .I am wondering
> why
> cache again changed to partitioned mode from Replicated mode?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov

Re: Ignite user create/modify trouble

[aealexsandrov]

Hi,

When you create the user in quotes ("test") using SQL as next:

CREATE USER "test" WITH PASSWORD 'test'

It will be created as it was set (in this case it will be test)

If you create the user without quotes (test) using SQL as next:

CREATE USER test WITH PASSWORD 'test'

then username will be stored in uppercase (TEST).

The same situation with other DDL commands. So if you are going to change
the password of the default user you should do next:

ALTER USER "ignite" WITH PASSWORD 
'test';

When you set the username in JDBC then there is no case transformation?
Could you please re-test your cases using quotes as well?

BR,
Andrei



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite 2.5 nodes do not rejoin the cluster after restart (works on 2.4)

[Andrey Mashenkov]

Hi,

What baseline topology does ./control.sh prints?
Is it possible, a node that out of baseline has started before baseline
node starts?

On Thu, Jun 7, 2018 at 9:54 AM, szj  wrote:

> Well, it definitely does work in 2.4. Please notice that there needs to be
> ignitevisorcmd.sh involved to trigger this bug (I didn't try with other
> clients though). Here's what is printed by Java on the console:
>
> [09:28:33]
> [09:28:33] To start Console Management & Monitoring run
> ignitevisorcmd.{sh|bat}
> [09:28:33]
> [09:28:33] Ignite node started OK (id=ae8697ad)
> [09:28:33] Topology snapshot [ver=33, servers=2, clients=0, CPUs=4,
> offheap=2.1GB, heap=2.0GB]
> [09:28:33]   ^-- Node [id=AE8697AD-6421-4C0C-96FE-FC29ED9B6DCA,
> clusterState=ACTIVE]
> [09:28:33]   ^-- Baseline [id=7, size=2, online=2, offline=0]
> [09:28:33] Data Regions Configured:
> [09:28:33]   ^-- default [initSize=256.0 MiB, maxSize=1.4 GiB,
> persistenceEnabled=true]
> [09:29:25] Ignite node stopped OK [uptime=00:00:51.837]
> [09:29:35]__  
> [09:29:35]   /  _/ ___/ |/ /  _/_  __/ __/
> [09:29:35]  _/ // (7 7// /  / / / _/
> [09:29:35] /___/\___/_/|_/___/ /_/ /___/
> [09:29:35]
> [09:29:35] ver. 2.5.0#20180523-sha1:86e110c7
> [09:29:35] 2018 Copyright(C) Apache Software Foundation
> [09:29:35]
> [09:29:35] Ignite documentation: http://ignite.apache.org
> [09:29:35]
> [09:29:35] Quiet mode.
> [09:29:35]   ^-- Logging to file
> '/usr/share/apache-ignite/work/log/ignite-d484e6c6.0.log'
> [09:29:35]   ^-- Logging by 'JavaLogger [quiet=true, config=null]'
> [09:29:35]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false
> or "-v" to ignite.{sh|bat}
> [09:29:35]
> [09:29:35] OS: Linux 2.6.32-696.18.7.el6.x86_64 amd64
> [09:29:35] VM information: OpenJDK Runtime Environment 1.8.0_121-b13 Oracle
> Corporation OpenJDK 64-Bit Server VM 25.121-b13
> [09:29:35] Configured plugins:
> [09:29:35]   ^-- None
> [09:29:35]
> [09:29:35] Configured failure handler: [hnd=StopNodeOrHaltFailureHandler
> [tryStop=false, timeout=0]]
> [09:29:35] Message queue limit is set to 0 which may lead to potential
> OOMEs
> when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to
> message queues growth on sender and receiver sides.
> [09:29:35] Security status [authentication=off, tls/ssl=off]
> [09:29:36,435][SEVERE][tcp-disco-msg-worker-#2][TcpDiscoverySpi]
> TcpDiscoverSpi's message worker thread failed abnormally. Stopping the node
> in order to prevent cluster wide instability.
> class org.apache.ignite.IgniteException: Node with BaselineTopology cannot
> join mixed cluster running in compatibility mode
> at
> org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor.
> onGridDataReceived(GridClusterStateProcessor.java:714)
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$5.
> onExchange(GridDiscoveryManager.java:883)
> at
> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.
> onExchange(TcpDiscoverySpi.java:1939)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.
> processNodeAddedMessage(ServerImpl.java:4354)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.
> processMessage(ServerImpl.java:2744)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.
> processMessage(ServerImpl.java:2536)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(
> ServerImpl.java:6775)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(
> ServerImpl.java:2621)
> at
> org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
> [09:29:36,437][SEVERE][tcp-disco-msg-worker-#2][] Critical system error
> detected. Will be handled accordingly to configured handler [hnd=class
> o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext
> [type=SYSTEM_WORKER_TERMINATION, err=class o.a.i.IgniteException: Node
> with
> BaselineTopology cannot join mixed cluster running in compatibility mode]]
> class org.apache.ignite.IgniteException: Node with BaselineTopology cannot
> join mixed cluster running in compatibility mode
> at
> org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor.
> onGridDataReceived(GridClusterStateProcessor.java:714)
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$5.
> onExchange(GridDiscoveryManager.java:883)
> at
> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.
> onExchange(TcpDiscoverySpi.java:1939)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.
> processNodeAddedMessage(ServerImpl.java:4354)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.
> processMessage(ServerImpl.java:2744)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.
> processMessage(ServerImpl.java:2536)
> at
> org.apache.ignite.spi.discov

Cache getting cleareing automatically

[siva]

Hi,
We have two ignite server nodes in single physicle machine,and one client
node.Dont know what happend cache getting clear automatically.Initially We
have created a cache name called "*c091e548-b45a-49b4-b8ec-2cb5e27c7af6*"
with "*partitioned*" mode.Later we destroy the same cache and configured as
"*Replicated*" mode and storing entities under the same cache.But after some
days cache got cleared and cache changed to partitioned mode automatically
and again & again same problem facing every 10 min.What might be the issue?
I have gone through the logs  and 

 found that nodes were  disconnected and again connected .I am wondering why 
cache again changed to partitioned mode from Replicated mode?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: How are SQL Queries executed on Ignite Cluster

[Evgenii Zhuravlev]

It can't work on the query level, because internally, it divides all
indexes into N trees instead of one(where N equals queryParallelism). You
can't redefine it after it was created since it will lead to the complete
rebuild of all indexes.

Evgenii

2018-06-06 20:55 GMT+03:00 Sanjeev :

> So it looks like that Query parallelism works at Cache level, but it would
> make more sense to do it at the Query level, more like a hint in a SQL
> query
> to control how much parallelism is needed. This way it will be very dynamic
> and users would have full control. Default could be 1, but OLAP queries
> could define it dynamically on the fly on a per query basis.
>
> Also, is CacheConfiguration.queryParallelism() call the only way to define
> it. Is there no way to define this, after the fact that cache has been
> created and loaded. There should be a way to turn this on-off, for now at
> Cache level (though would prefer at Query level), through SQL DDL or DML
> statements.
>
> Is this possible now in some way, or do we have to bring the cluster down
> and reload all the data?
>
> Thanks...
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Ignite C# Geometry

[Andrey Mashenkov]

Hi,

I don't think  JST classe are supported in Ignite binary protocol.
You can try to use BinaryObjects for mock vividsolution classes or inline
in SQL query string.

On Thu, Jun 7, 2018 at 10:34 AM, Jonathan Mayer  wrote:

> Hi Guys,
>
> I realise that 2.5 has just been released and that everyone is busy but...
>
> If I can't get an answer to the below question it is a bit of a show
> stopper.
>
> Do I have to implement my own Data Type mapper for a JTS Geometry within
> the
> Ignite .Net Platform code or is there an easier way to access it from
> within
> C#?
>
> Cheers,
>
> Jonathan
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov

Re: Ignite Node failure - Node out of topology (SEGMENTED)

[Andrey Mashenkov]

Hi,

Seems, there is a bug with IPv6 usage [1]. This has to be investigated.
Also, there is a discussion [2].

[1] https://issues.apache.org/jira/browse/IGNITE-6503
[2]
http://apache-ignite-developers.2346864.n4.nabble.com/Issues-if-Djava-net-preferIPv4Stack-true-is-not-set-td22372.html

On Wed, Jun 6, 2018 at 9:24 PM, naresh.goty  wrote:

> Thanks.
> We have enabled IPV4 JVM option in our non-production environment, found no
> issue reported on segmentation. Our main concern is, the issue is happening
> only in production, and we are very much interested in finding the real
> root
> cause (we can rule out - GC pauses, CPU spikes, network latencies as the
> cause is none of them).
>
> 1) please provide us with any useful tips in identifying the source of the
> problem, so that we can avoid the problem altogether instead of taking a
> remediation steps (of restarting JVM) if the issue happens.
>
> 2) do let us know if any timeout configurations should be increased to
> mitigate the problem?
>
> Regards
> Naresh
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov

Ignite clustering doubts

[the_palakkaran]

Hi,

The below is my scenario:

I have a CustomerCache. I have externalizable keys and models. Customers ids
range from 1 to 4. I have 4 nodes.

I have few doubts on clustering in ignite:

1. How can I restrict 1 records on each node?

2. How does ignite distribute records between each node? On what basis? Is
there any relevance of AffinityFunction here? Is there a demo of how to use
that ?

3. How would fail over and re-balancing work when a node fails?

4. Do I need to configure number of backups? Is the number of backups
dependent on number of nodes?

5. Will I have any problem to access data from any of these nodes or a
client node since I have used externalization?

6. Is there any possible way to query data on a single node? 

7. Leader election is based on age, as per documentation. Can I control this
election?

8. Zero deployment means I do not need to have models deployed on every
class right? Is this the same case for externalized models?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Spark + Ignite standalone mode on Kubernetes cluster.

[Andrey Mashenkov]

Hi,

It looks like a linkage error.
Please, check dependencies versions. E.g. Jackson or Scala version.

On Wed, Jun 6, 2018 at 6:47 AM, Premachandran, Mahesh (Nokia -
IN/Bangalore)  wrote:

> We are trying to integrate spark 2.2.0 with Ignite 2.4 on kubernetes, and
> getting the following exception.
>
> 2018-06-06 03:23:51 INFO  GridDiscoveryManager:495 - Topology snapshot
> [ver=3, servers=2, clients=1, CPUs=12, offheap=3.1GB, heap=3.0GB]
> Exception in thread "main" java.lang.NoSuchMethodError:
> scala.util.matching.Regex.unapplySeq(Ljava/lang/
> CharSequence;)Lscala/Option;
> at com.fasterxml.jackson.module.scala.JacksonModule$.version$
> lzycompute(JacksonModule.scala:30)
> at com.fasterxml.jackson.module.scala.JacksonModule$.version(
> JacksonModule.scala:26)
> at com.fasterxml.jackson.module.scala.JacksonModule$class.
> version(JacksonModule.scala:49)
> at com.fasterxml.jackson.module.scala.DefaultScalaModule.
> version(DefaultScalaModule.scala:19)
> at com.fasterxml.jackson.databind.ObjectMapper.
> registerModule(ObjectMapper.java:710)
> at org.apache.spark.rdd.RDDOperationScope$.(
> RDDOperationScope.scala:82)
> at org.apache.spark.rdd.RDDOperationScope$.(
> RDDOperationScope.scala)
> at org.apache.spark.SparkContext.withScope(SparkContext.scala:701)
> at org.apache.spark.SparkContext.parallelize(SparkContext.
> scala:718)
> at org.apache.spark.api.java.JavaSparkContext.parallelize(
> JavaSparkContext.scala:134)
> at org.apache.spark.api.java.JavaSparkContext.parallelize(
> JavaSparkContext.scala:146)
> at org.apache.ignite.examples.spark.SharedRDDExample.main(
> SharedRDDExample.java:81)
>
> Two spark executors connecting to ignite in standalone mode. We are using
> Ignite's sharedRDD example (https://github.com/apache/
> ignite/blob/2.4.0/examples/src/main/spark/org/apache/
> ignite/examples/spark/SharedRDDExample.java)
>
> The example jar and ignite was built with scala version 2.11. The ignite
> xml is attached.
>
>
>
> -Original Message-
> From: Ray [mailto:ray...@cisco.com]
> Sent: Friday, June 01, 2018 12:00 PM
> To: user@ignite.apache.org
> Subject: Re: Spark + Ignite standalone mode on Kubernetes cluster.
>
> Currently, Ignite supports Spark up to version 2.2.0.
> Please try with Spark 2.2.0 or wait until this ticket is resolved.
> https://issues.apache.org/jira/browse/IGNITE-8534
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov

C++ Distribute Closure Failure

[F.D.]

Hi,
when I launch a distibute closure calling the Call method or by RunAsync,
how can I check if the function is failed? Only by timeout?

Thanks,
   F.D.

Re: IgniteCache invoke CacheEntryProcessor, not throw EntryProcessorException

[Andrey Mashenkov]

Hi,

Looks like it is ok.

Ignite have next sync modes full_sync, primary_sync, full_Async.
You can detect failure from user thread if it happens on synchronous phase.

FULL_SYNC: cache operation return control to user thread after operation
has been applied on primary and all backups.
All exceptions occurs during operation will be propagated to user.

With primary_sync, cache operation return control to user thread after
operation has been applied on primary. Backups will be updated in async way.
Only exceptions which occurs during primary operation will be propagated to
user. Exceptions from backups will be ignored as nobody waits for backup
operation results.

With full_Ascync, cache operation return control to user thread immediately
after request has been sent to primary. All, Primary and backup will be
updates in async way.
Only exceptions, which occurs during sending request to primary node, will
be propagated to user. Others will be ignored as nobody waits for backup
operation results.



On Wed, Jun 6, 2018 at 7:24 PM, haotian.chen 
wrote:

> Hi Andrew,
>
> Thanks a lot for the response!
>
> While reproducing the case, I found that: If I set
> CacheWriteSynchronizatoinMode to FULL_ASYNC, EntryProcessorException is
> ignored. But if I set it PRIMARY_SYNC, then the EntryProcessorException
> will
> be thrown.
>
> Is it intentional that under FULL_ASYNC the exception will be ignored?
> Should it be that Exception will be thrown when the CacheEntryProcessor
> eventually ran?
>
> Here is the sample code:
>
>
> import org.apache.ignite.Ignite;
> import org.apache.ignite.IgniteCache;
> import org.apache.ignite.Ignition;
> import org.apache.ignite.cache.*;
> import org.apache.ignite.configuration.CacheConfiguration;
> import org.apache.ignite.configuration.IgniteConfiguration;
>
> import javax.cache.processor.EntryProcessorException;
> import javax.cache.processor.MutableEntry;
>
> public class Main {
> public static void main(String[] args) {
>
> class IncrementOddEntryProcessor implements
> CacheEntryProcessor {
>
> public Object process(MutableEntry entry,
> Object... arguments) throws EntryProcessorException {
> Integer value = entry.getValue();
> if (value % 2 == 0) {
> throw new EntryProcessorException("can't operates on
> even number");
> } else {
> entry.setValue(value + 1);
> return null;
> }
> }
>
> }
>
> IgniteConfiguration igniteCfg = new IgniteConfiguration()
> .setGridLogger(new Slf4jLogger())
> .setIgniteInstanceName("testinstance");
>
> Ignite ignite = Ignition.getOrStart(igniteCfg);
>
> try {
> CacheConfiguration cacheCfg = new
> CacheConfiguration()
> .setName("testcache")
> .setAtomicityMode(CacheAtomicityMode.ATOMIC)
> .setRebalanceMode(CacheRebalanceMode.ASYNC)
>
> .setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_ASYNC)
>
> .setPartitionLossPolicy(PartitionLossPolicy.READ_WRITE_SAFE);
>
> IgniteCache cache =
> ignite.createCache(cacheCfg);
>
> cache.put("1", 1);
> cache.put("2", 2);
>
> cache.invoke("1", new IncrementOddEntryProcessor());
> cache.invoke("2", new IncrementOddEntryProcessor());
>
> System.out.println(cache.get("1"));
> System.out.println(cache.get("2"));
> } finally {
> ignite.close();
> }
>
>
> }
> }
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov

Re: Ignite Cluster getting stuck when new node Join or release

[Andrey Mashenkov]

Hi,

It is ok if you kill client node. Grid will wait for
failureDetectionTimeout before drop failed node from topology.
All topology operations will stuck during that time as ignite nodes will
wait for answer from failed node until they detected failure.

On Thu, Jun 7, 2018 at 8:22 AM, Sambhaji Sawant 
wrote:

> An issue occurred when we abnormally stop Spark Java application which
> having Ignite client running inside that Spark context.So when we kill
> spark application its abnormally stop Ignite client and then when we
> restart our application and client try to connect with Ignite cluster then
> it getting stuck.
>
> On Mon, Jun 4, 2018 at 6:32 PM, dkarachentsev 
> wrote:
>
>> Hi,
>>
>> It's hard to get what's going wrong from your question.
>> Please attach full logs and thread dumps from all server nodes.
>>
>> Thanks!
>> -Dmitry
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>
>


-- 
Best regards,
Andrey V. Mashenkov

Re: Xgboost and LSTM support in Apache Ignite ML and DL

[dmitrievanthony]

Hi, 

As I've answered on  StackOverflow

 
, 

Depends on your data location and your goals. Does your data is already
stored in Apache Ignite? Do you use Apache Ignite in your ecosystem? 

Regarding XGBoost, Apache Ignite doesn't provide trainers for boosted
trees so far, but you can use imported XGBoost models. You can find an
example of such importer here. If you trains your model outside of the
Ignite and then use the model in Ignite to process data this approach will
be good enough for you. 

Regarding LSTM, Apache Ignite ML currently supports only MLP neural
networks, so you can't train your model inside Apache Ignite. Even so you
still can train the model outside of the Apache Ignite and then import
prepared model. 

Best regards, 
Anton Dmitriev



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: ignite c++ support service grid?

[Igor Sapego]

No, currently you can only call C++ closures on C++ nodes.

Best Regards,
Igor

On Thu, Jun 7, 2018 at 11:09 AM, wangsan  wrote:

> can i use c++ disturbted closure  on java ingite node
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: BinaryStringArrayWriter / BinaryStringArrayReader

[F.D.]

Perfect!

Thanks
   F.D.

On Mon, Jun 4, 2018 at 12:38 PM Igor Sapego  wrote:

> Hi,
>
> Yes, you can use BinaryStringArrayWriter / BinaryStringArrayReader.
>
> No, to get BinaryStringArrayReader, you should use method
> BinaryReader::ReadStringArray(const char* fieldName);
>
> Best Regards,
> Igor
>
> On Sun, Jun 3, 2018 at 1:31 AM, F.D.  wrote:
>
>> Hi,
>>
>> I'm trying to develop a distributed closure in C++. The return type of my
>> closure is a std::vector>, so I think that I have to implement
>> a BinaryType to serialize it. The question is: in the Write and Read
>> methods can I use the BinaryStringArrayWriter / BinaryStringArrayReader or
>> I have to use my own solution?
>>
>> And If I can, the first argument (impl::binary::BinaryReaderImpl *impl)
>> is the first argoment of my method Read (BinaryReader &reader)?
>>
>> Thanks,
>>F.D.
>>
>
>

Re: Ignite client mvn artifact's purpose?

[Muthu]

Thanks Andrey!

On Tue, Jun 5, 2018 at 9:59 AM, Andrey Mashenkov  wrote:

> Hi,
>
> Answered on SO, please take a look [1].
>
> [1] https://stackoverflow.com/questions/50691453/ignite-
> client-mvn-artifacts-purpose
>
> On Sun, Jun 3, 2018 at 2:26 PM, mlekshma 
> wrote:
>
>> Hi Folks,
>>
>> I notice there is a mvn artifact with artifact id "ignite-clients". What
>> is
>> its purpose?
>>
>> https://mvnrepository.com/artifact/org.apache.ignite/ignite-clients/2.4.0
>> > -clients/2.4.0>
>>
>> Thanks
>> Muthu
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>
>
>
> --
> Best regards,
> Andrey V. Mashenkov
>

Re: ignite c++ support service grid?

[wangsan]

can i use c++ disturbted closure  on java ingite node



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite C# Geometry

[Jonathan Mayer]

Hi Guys,

I realise that 2.5 has just been released and that everyone is busy but...

If I can't get an answer to the below question it is a bit of a show
stopper.

Do I have to implement my own Data Type mapper for a JTS Geometry within the
Ignite .Net Platform code or is there an easier way to access it from within
C#?

Cheers,

Jonathan



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/