RE: Information regarding Ignite Web Console

[Sriveena Mattaparthi]

Hi Denis,

Does ignite support oracle/RDBMS VIEWS without loading all the underlying 
tables that produced the view on to ignite cache?

Thanks & Regards,
Sriveena

From: Denis Mekhanikov [mailto:dmekhani...@gmail.com]
Sent: Wednesday, June 27, 2018 4:02 PM
To: user@ignite.apache.org
Subject: Re: Information regarding Ignite Web Console

Sriveena,

CacheStore
 extends the 
CacheWriter
 interface, which has 
delete
 and 
deleteAll
 methods, which will be called by Ignite, when you remove entries from Ignite 
caches.
write
 and 
writeAll
 methods will be called, when you put some data into cache.
It works the same way with MySQL or any other relational DB.
So, CacheStore gives you a possibility to implement synchronization with any 
external data source. You can do it for MongoDB, if you need.

Denis

ср, 27 июн. 2018 г. в 13:18, Sriveena Mattaparthi 
mailto:sriveena.mattapar...@ekaplus.com>>:
Thanks Denis for the pointers.

But the case explained in 
https://www.youtube.com/watch?v=XBtI2Z01kAA
 -  Using Ignite Web Console for Automatic RDBMS Integration with Apache Ignite 
- Part 3.
shows that any deletions happening on ignite cache are getting reflected in the 
mysql database.

Which I thought is ignite feature of autosyncing data to and from RDBMS 
databases.
Please correct my understanding.

Thanks & Regards,
Sriveena

From: Denis Mekhanikov 
[mailto:dmekhani...@gmail.com]
Sent: Wednesday, June 27, 2018 3:28 PM

To: user@ignite.apache.org
Subject: Re: Information regarding Ignite Web Console

Sriveena,

Web Console can generate schema in Ignite based on an existing schema in a 
relational database.
But you can do it yourself, without use of Web Console.
Web Console only makes it easier to wire-up stuff, i.e. generate required data 
classes and configuration.

Unfortunately, this feature is not available for NoSQL databases. But you still 
can configure everything yourself.
You will need to implement a 
CacheStore

RE: OptimizedMarshaller instead of BinaryMarshaller is used forScanQuery

[Calvin KL Wong, CLSA]

Hi Stan,

Thanks for your response.

>> How did you check which marshaller is used in each case?

i)from stack trace

ii)   step with a debugger

>> Can you share the code of your POJO?
Attached pojo.zip contains skeleton of the POJO.  The object that was passed 
between ignite nodes is called WorkingSetLedgerDelta.  It contains objects of 
other classes.

>> Can you also share the logs/check them for warnings? There might be a 
>> warning saying that there is a problem in using BinaryMarshaller.
I just checked them – did a grep of ‘-i marsh’, ‘-i warn’,  ‘-i except’, ‘-i 
error’; no marshaller related problem found.

Background:
A newer version of NewOrderRequest extends AbstractRequest is deployed on the 
service node, but not on the client node.  Client node sends a scan query to 
server node to get all WorkingSetLedgerDelta, which contains Request objects.

About the attached stack traces of the client node

1.   exception.when.using.pojo.with.scan.query.log

Cache: IgniteCache

Caught exception when trying to deserialize WorkingSetLedgerDelta



2.   
forced.exception.when.using.binary.object.with.scan.query.with.keep.binary.log

Cache: IgniteCache.withKeepBinary()

I forced an exception from eclipse when I saw that we successfully unmarshalled 
bytes into the BinaryObjectImpl of WorkingSetLedgerDelta.

The main difference between the two stack traces is at:

with POJO, the logic unmarshalls WorkingSetLedgerDelta using 
‘readSerializable’, which cannot handle versioning.
at 
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:601)
 ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at 
org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:927)
 ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at 
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
 ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at 
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:199)
 ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422) 
~[?:1.8.0_152]
at 
org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponseEntry.readExternal(GridCacheQueryResponseEntry.java:90)
 ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]

withKeepBinary, BinaryObjImpl object was received, the logic could unmarshall 
the BinaryObjImpl of WorkingSetLedgerDelta with no problem.
at 
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:558)
 ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at 
org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:917)
 ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at 
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
 ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at 
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:199)
 ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422) 
~[?:1.8.0_152]
at 
org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponseEntry.readExternal(GridCacheQueryResponseEntry.java:90)
 ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]


Thanks,
Calvin


From: Stanislav Lukyanov [mailto:stanlukya...@gmail.com]
Sent: Monday, July 09, 2018 9:04 PM
To: user@ignite.apache.org
Subject: RE: OptimizedMarshaller instead of BinaryMarshaller is used 
forScanQuery

Hi Calvin,

It should work the same for all queries. Ideally OptimizedMarshaller shouldn’t 
even be used (except for JDK classes).

How did you check which marshaller is used in each case?

Can you share the code of your POJO? Or perhaps a runnable reproducer?

Can you also share the logs/check them for warnings? There might be a warning 
saying that there is a problem in using BinaryMarshaller.

Thanks,
Stan

From: Calvin KL Wong, CLSA
Sent: 9 июля 2018 г. 13:40
To: user@ignite.apache.org
Subject: RE: OptimizedMarshaller instead of BinaryMarshaller is used 
forScanQuery

Ok, I found that I can use binaryObject.deserialize() to get back the POJO.  
So, the only remaining question is:

When a cache does not have ‘withKeepBinary’ set, why ContinuousQuery can handle 
class versioning fine whereas ScanQue

Re: Does Ignite support something like "database" from mysql?

[Denis Magda]

Multi-tenancy is provided as an extra feature for Ignite by GridGain. Talk
to them.

--
Denis

On Wed, Jul 4, 2018 at 4:23 AM Maxim Malygin 
wrote:

> Hi Guys,
>
> I 'd like to have isolated caches/tables for multiple tenants. In mysql I
> can do it by creating databases per tenant. Is it possible to do something
> like this in Ignite?
> I can create unique tables per tenant (for example
> -) but it's very uncomfortable in use.
>
> Also I'm trying to understand how Ignite caches maps to Ignite SQL tables.
> For example I create simple cache with the following configuration
>
>  class="org.apache.ignite.configuration.CacheConfiguration">
> 
> 
> 
>
> Is it possible to query this cache via SQL? What table name should be in
> query?
>
> Thanks,
> Maxim
>
>

Ignite has the most active dev and user communities

[Denis Magda]

Igniters,

Let me share some great news with you. ASF shared annual report [1] for
2018:

   - Top 5 Apache developer email lists: *Ignite (1st!)*, Kafka, Tomcat,
   Beam, and James
   - Top 5 Apache user email lists: Lucene/Solr, *Ignite (2nd!)*, Flink,
   Kafka, and Cassandra
   - Top 5 Apache repositories by commits: Hadoop, Ambari, Camel, *Ignite
   (4th!)*, and Beam



So, since the times of the previous report [2] we moved from 3rd to 2nd
position in the user lists rank and from 5th to 4th in the commits rank.
Ramping up!

[1]
http://globenewswire.com/news-release/2018/07/09/1534470/0/en/The-Apache-Software-Foundation-Announces-Annual-Report-for-2018-Fiscal-Year.html
[2] https://blogs.apache.org/foundation/entry/apache-in-2017-by-the

--
Denis

Re: apache ignite atomicLong.incrementAndGet() is causing starvation in striped pool

[Вячеслав Коптилин]

Hello Vadym,

The root cause of the behavior is that you are trying to use
`IgniteAtomicLong` instance within `ContinuousQuery` local listener.
In general, you should avoid using cache operations in that listener (
`IgniteAtomicLong` uses
IgniteCache under the hood),
because this callback is invoked from a sensitive part of implementation in
a synchronous way and therefore, it may lead to starvation and/or deadlocks.

In order to resolve this issue, you can use @IgniteAsyncCallback
annotation, it allows executing callback methods in the AsyncCallback
thread pool.
Please try the following approach:

@IgniteAsyncCallback
public class MyLocalListener implements
CacheEntryUpdatedListener {
@Override public void onUpdated(
Iterable>
events) throws CacheEntryListenerException {

...
}
}

continuousQuery.setLocalListener(new MyLocalListener());

Thanks,
Slava.


пн, 9 июл. 2018 г. в 22:19, Vadym Vasiuk :

> Hi All,
>
> I have Ignite cluster which consists of two nodes. Each node after start
> creates a continuous query which calls atomicLong.incrementAndGet() in
> "onUpdated" method:
>
> continuousQuery.setLocalListener(new
> CacheEntryUpdatedListener() {
> @Override public void onUpdated(Iterable extends Integer, ? extends String>> evts) {
> for (CacheEntryEvent
> e : evts){
> System.out.println("Incremented value: " +
> atomicLong.incrementAndGet());
> }
> }
> });
>
> This continuous query is listening on all events on "test" cache.
> I start both nodes and then start a client application which inserts 5
> entries into "test" cache. For the first two entries (which client inserts)
> I see below outputs only on one node:
> Incremented value: 1
> Incremented value: 2
>
> And after that I get below warn messages in logs on the node where
> "Incremented value" was printed (on the second node I see no messages) :
>
> 2018-07-09 21:56:57.993  WARN 1876 --- [eout-worker-#23]
> o.apache.ignite.internal.util.typedef.G  : >>> Possible starvation in
> striped pool.
> Thread name: sys-stripe-0-#1
> Queue: []
> Deadlock: false
> Completed: 1
> Thread [name="sys-stripe-0-#1", id=14, state=WAITING, blockCnt=0,
> waitCnt=5]
> at sun.misc.Unsafe.park(Native Method)
> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> at
> o.a.i.i.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
> at
> o.a.i.i.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
> at
> o.a.i.i.processors.cache.GridCacheAdapter$25.op(GridCacheAdapter.java:2492)
> at
> o.a.i.i.processors.cache.GridCacheAdapter$25.op(GridCacheAdapter.java:2478)
> at
> o.a.i.i.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4088)
> at
> o.a.i.i.processors.cache.GridCacheAdapter.invoke0(GridCacheAdapter.java:2478)
> at
> o.a.i.i.processors.cache.GridCacheAdapter.invoke(GridCacheAdapter.java:2456)
> at
> o.a.i.i.processors.cache.GridCacheProxyImpl.invoke(GridCacheProxyImpl.java:588)
> at
> o.a.i.i.processors.datastructures.GridCacheAtomicLongImpl.incrementAndGet(GridCacheAtomicLongImpl.java:98)
> at com.ignite.config.ServerConfig$2.onUpdated(ServerConfig.java:80)
> at
> o.a.i.i.processors.cache.query.continuous.CacheContinuousQueryHandler.onEntryUpdate(CacheContinuousQueryHandler.java:835)
> at
> o.a.i.i.processors.cache.query.continuous.CacheContinuousQueryHandler.access$700(CacheContinuousQueryHandler.java:82)
> at
> o.a.i.i.processors.cache.query.continuous.CacheContinuousQueryHandler$1$1.apply(CacheContinuousQueryHandler.java:420)
> at
> o.a.i.i.processors.cache.query.continuous.CacheContinuousQueryHandler$1$1.apply(CacheContinuousQueryHandler.java:415)
> at
> o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicAbstractUpdateFuture.onDone(GridDhtAtomicAbstractUpdateFuture.java:556)
> at
> o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicAbstractUpdateFuture.onDone(GridDhtAtomicAbstractUpdateFuture.java:61)
> at
> o.a.i.i.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:440)
> at
> o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicAbstractUpdateFuture.registerResponse(GridDhtAtomicAbstractUpdateFuture.java:367)
> at
> o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicAbstractUpdateFuture.onDhtResponse(GridDhtAtomicAbstractUpdateFuture.java:522)
> at
> o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.processDhtAtomicUpdateResponse(GridDhtAtomicCache.java:3472)
> at
> o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$700(GridDhtAtomicCache.java:130)
> at
> o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$8.apply(GridDhtAtomicCache.java:323)
> at
> o.a.i.i.processors.cache.distributed.dht

apache ignite atomicLong.incrementAndGet() is causing starvation in striped pool

[Vadym Vasiuk]

Hi All,

I have Ignite cluster which consists of two nodes. Each node after start
creates a continuous query which calls atomicLong.incrementAndGet() in
"onUpdated" method:

continuousQuery.setLocalListener(new
CacheEntryUpdatedListener() {
@Override public void onUpdated(Iterable> evts) {
for (CacheEntryEvent e
: evts){
System.out.println("Incremented value: " +
atomicLong.incrementAndGet());
}
}
});

This continuous query is listening on all events on "test" cache.
I start both nodes and then start a client application which inserts 5
entries into "test" cache. For the first two entries (which client inserts)
I see below outputs only on one node:
Incremented value: 1
Incremented value: 2

And after that I get below warn messages in logs on the node where
"Incremented value" was printed (on the second node I see no messages) :

2018-07-09 21:56:57.993  WARN 1876 --- [eout-worker-#23]
o.apache.ignite.internal.util.typedef.G  : >>> Possible starvation in
striped pool.
Thread name: sys-stripe-0-#1
Queue: []
Deadlock: false
Completed: 1
Thread [name="sys-stripe-0-#1", id=14, state=WAITING, blockCnt=0, waitCnt=5]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at
o.a.i.i.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
at
o.a.i.i.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
at
o.a.i.i.processors.cache.GridCacheAdapter$25.op(GridCacheAdapter.java:2492)
at
o.a.i.i.processors.cache.GridCacheAdapter$25.op(GridCacheAdapter.java:2478)
at
o.a.i.i.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4088)
at
o.a.i.i.processors.cache.GridCacheAdapter.invoke0(GridCacheAdapter.java:2478)
at
o.a.i.i.processors.cache.GridCacheAdapter.invoke(GridCacheAdapter.java:2456)
at
o.a.i.i.processors.cache.GridCacheProxyImpl.invoke(GridCacheProxyImpl.java:588)
at
o.a.i.i.processors.datastructures.GridCacheAtomicLongImpl.incrementAndGet(GridCacheAtomicLongImpl.java:98)
at com.ignite.config.ServerConfig$2.onUpdated(ServerConfig.java:80)
at
o.a.i.i.processors.cache.query.continuous.CacheContinuousQueryHandler.onEntryUpdate(CacheContinuousQueryHandler.java:835)
at
o.a.i.i.processors.cache.query.continuous.CacheContinuousQueryHandler.access$700(CacheContinuousQueryHandler.java:82)
at
o.a.i.i.processors.cache.query.continuous.CacheContinuousQueryHandler$1$1.apply(CacheContinuousQueryHandler.java:420)
at
o.a.i.i.processors.cache.query.continuous.CacheContinuousQueryHandler$1$1.apply(CacheContinuousQueryHandler.java:415)
at
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicAbstractUpdateFuture.onDone(GridDhtAtomicAbstractUpdateFuture.java:556)
at
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicAbstractUpdateFuture.onDone(GridDhtAtomicAbstractUpdateFuture.java:61)
at
o.a.i.i.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:440)
at
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicAbstractUpdateFuture.registerResponse(GridDhtAtomicAbstractUpdateFuture.java:367)
at
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicAbstractUpdateFuture.onDhtResponse(GridDhtAtomicAbstractUpdateFuture.java:522)
at
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.processDhtAtomicUpdateResponse(GridDhtAtomicCache.java:3472)
at
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$700(GridDhtAtomicCache.java:130)
at
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$8.apply(GridDhtAtomicCache.java:323)
at
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$8.apply(GridDhtAtomicCache.java:318)
at
o.a.i.i.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1060)
at
o.a.i.i.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:579)
at
o.a.i.i.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:378)
at
o.a.i.i.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:304)
at
o.a.i.i.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:99)
at
o.a.i.i.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:293)
at
o.a.i.i.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555)
at
o.a.i.i.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183)
at
o.a.i.i.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
at
o.a.i.i.managers.communication.GridIoManager$9.run(GridIoManager.java:1090)
at o.a.i.i.util.StripedExecutor$Stripe.run(StripedExecutor.java:505)
at java.lang.Thread.run(T

Re: Apache Ignite Install As Service error

[ApacheUser]

the service is running but cant access, full message below
[]# systemctl status apache-ign...@default-config.xml
● apache-ign...@default-config.xml.service - Apache Ignite In-Memory
Computing Platform Service
   Loaded: loaded (/etc/systemd/system/apache-ignite@.service; enabled;
vendor preset: disabled)
   Active: active (running) since Mon 2018-07-09 15:40:49 GMT; 2s ago
  Process: 16838 ExecStart=/usr/share/apache-ignite/bin/service.sh start %i
(code=exited, status=0/SUCCESS)
  Process: 16833 ExecStartPre=/usr/bin/env bash
/usr/share/apache-ignite/bin/service.sh set-firewall (code=exited,
status=0/SUCCESS)
  Process: 16830 ExecStartPre=/usr/bin/chown ignite:ignite
/var/run/apache-ignite (code=exited, status=0/SUCCESS)
  Process: 16828 ExecStartPre=/usr/bin/mkdir /var/run/apache-ignite
(code=exited, status=1/FAILURE)
 Main PID: 16840 (ignite.sh)
   CGroup:
/system.slice/system-apache\x2dignite.slice/apache-ign...@default-config.xml.service
   ├─16840 /bin/bash /usr/share/apache-ignite/bin/ignite.sh
/etc/apache-ignite/default-config.xml
   └─16929 /usr/bin/java -Xms1g -Xmx1g -server -XX:+AggressiveOpts
-XX:MaxMetaspaceSize=256m -DIGNITE_QUIET=true
-DIGNITE_SUCCESS_FILE=/usr/share/apache-ignite/work/ignite...

Jul 09 15:40:49 ccrc_spark_analytic_4 systemd[1]: Starting Apache Ignite
In-Memory Computing Platform Service...
Jul 09 15:40:49 ccrc_spark_analytic_4 systemd[1]: Started Apache Ignite
In-Memory Computing Platform Service.

Thanks





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Apache Ignite Install As Service error

[ApacheUser]

Hi Team,

I am trying to Install Apache Ignite as Service one node goes well but coupe
of nodes throw the below message and they don't allow connection.

Process: 16828 ExecStartPre=/usr/bin/mkdir /var/run/apache-ignite
(code=exited, status=1/FAILURE)


In Service mode is there anyway I can run ignitevisorcmd.sh  and see the
topology etc?

thanks



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: OOME causing caches to be removed

[akurbanov]

Hi,

How do you create USER cache?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Can we start an ignite node by passing ignite configuations in json file in case on Dot net Core

[slava.koptilin]

Hello,

Please take a look at Pavel's answer at stackoverflow:
https://stackoverflow.com/questions/51245698/can-we-start-an-ignite-node-by-passing-ignite-configurations-in-json-file-in-cas
It seems that it's the best option.

Thanks.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: When to use Ignite.Net Tansactions ?

[Ilya Kasnacheev]

Hello!

No, you don't need transactions with just one statement in them, as per
your example code.

If you have multiple statements, many things can happen between those 2
statements, such as entries updated and deleted, which might affect your
computations in undesirable way. It's up to you to decide whether atomicity
is sufficient for you.

In the scenario that you have provided, yes, there will be locking on a key
even in ATOMIC mode. Thread will have to wait to complete its operation.

Regards,

-- 
Ilya Kasnacheev

2018-07-09 13:06 GMT+03:00 Mahesh Talreja :

> Hi Team,
>Thanks for replying! I hope you might have got some bit of the
> background of the requirement from my previous mail. If not, below is the
> example of the class that will be consumed by a microservice (A console
> app/background service). And using docker there will be multiple instances
> of the same service hosted!
>
> In the example below, I have only single operations per function, so do I
> really need transactions? Thinking about multiple instance/process
> implementation I currently have them, But I am not really sure whether I
> really need them, Or having just atomic mode will be fine?
>
> Dose atomic mode guaranty prevention of dirty read and sequential
> execution?
>
> Scenario 1
> If thead1 trying to update tradeid 1 with version 1 started before thread
> 2
> Thread 2 trying to update tradeid 1 with version 2 started after thread 1
> (both trying to update the same key)
> if there is a race condition wherein trade 1 take time to complete its
> work,
> will thread 2 wait for it?  and commit in a sequential manner even in
> atomic
> mode?
>
>
> public class TradeCache
> {
> private IIgnite igniteClient { get; set; }
> private ICache cache { get; set; }
>
> public TradeCache(IgniteConfiguration configuration)
> {
> igniteClient = Ignition.Start(configuration);
> cache= igniteClient.GetCache TradeDetails>("TradeDetailsCache");
> }
>
> public async Task PutAsync(TradeId tradeId,TradeDetails
> tradeDetails)
> {
> using (ITransaction transaction =
> igniteClient.GetTransactions().TxStart(
>TransactionConcurrency.Pessimistic,
> TransactionIsolation.RepeatableRead,
>TimeSpan.FromMilliseconds(300), 0))
> {
> bool result = await cache.PutIfAbsentAsync(tradeId,
> tradeDetails);
>
> transaction.Commit();
>
> return result;
> }
> }
>
> public async Task ReplaceAsync(TradeId tradeId, TradeDetails
> tradeDetails)
> {
> using (ITransaction transaction =
> igniteClient.GetTransactions().TxStart(
>TransactionConcurrency.Pessimistic,
> TransactionIsolation.RepeatableRead,
>TimeSpan.FromMilliseconds(300), 0))
> {
> bool result = await cache.ReplaceAsync(tradeId,
> tradeDetails);
>
> transaction.Commit();
>
> return result;
> }
> }
>
> public async Task RemoveAsync(TradeId tradeId)
> {
> using (ITransaction transaction =
> igniteClient.GetTransactions().TxStart(
>TransactionConcurrency.Pessimistic,
> TransactionIsolation.RepeatableRead,
>TimeSpan.FromMilliseconds(300), 0))
> {
> bool result = await cache.RemoveAsync(tradeId);
>
> transaction.Commit();
>
> return result;
> }
> }
>
> }
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: cache configuration FULL_SYNC

[Evgenii Zhuravlev]

Ignite internally guarantees that write through to the 3rd party cache
store will be consistent with the update of the cache.

Evgenii

2018-07-09 17:50 GMT+03:00 Prasad Bhalerao :

> When you have multiple data sources (cache and oracle in this case) which
> should be updated in single transaction we use JTA. But with ignite
> transaction and ignite write through approach I don't have to use JTA.
> To keep the cache and oracle consistent one needs transaction.  In this
> case one update operation spans to multiple data sources.
>
> On Mon, Jul 9, 2018 at 8:05 PM Evgenii Zhuravlev 
> wrote:
>
>> No, you're not correct, each insert/update will be consistent between
>> cache and DB, transactions are not about consistency between cache and
>> oracle database here, it's about more than one operation, as in an
>> example I've shared in my previous message.
>>
>> Evgenii
>>
>> 2018-07-09 17:24 GMT+03:00 Prasad Bhalerao 
>> :
>>
>>> Thank you for the clarification.
>>>
>>> Ignite does not support transactions with jdbc updates. So if I use
>>> delete/update SQL to delete/update the multiple cache entries, ignite will
>>> not provide strong consistency between cache and oracle database.
>>> Am I correct?
>>>
>>> On Mon, Jul 9, 2018 at 7:43 PM Evgenii Zhuravlev <
>>> e.zhuravlev...@gmail.com> wrote:
>>>
 1. As this doc says: https://apacheignite.readme.
 io/docs/3rd-party-store: Ignite writes through results of its SQL
 INSERT, UPDATE and MERGE queries.
 2. Transaction in Ignite means absolutely the same as in another
 systems. If you want two or more changes to be in one transaction(i.e.,
 classic example with transfer between bank accounts), then, for sure, you
 need to use transactions.

 Evgenii

 2018-07-09 16:52 GMT+03:00 Prasad Bhalerao <
 prasadbhalerao1...@gmail.com>:

> Resending my last mail ...
>
> I understand that FULL_SYNC is required to keep the consistency
> between primary data node and backup data node and not between the cache
> and 3rd party data.
> Sorry for asking the incorrect question.
>
> 1) Does ignite supports write through updates to 3rd party store when
> using update or delete sql?
>
> 2) Is it mandatory to use ignite transaction to keep strong
> consistency between in memory cache and 3rd party data store?
> -Like you said, if I am updating more than one entry in cache then
> transaction is necessary to keep strong consistency between 3rd party db
> and ignite cache. Am I correct?
>
> Thanks,
> Prasad
>
> On Mon, Jul 9, 2018 at 6:38 PM Evgenii Zhuravlev <
> e.zhuravlev...@gmail.com> wrote:
>
>> Hi,
>>
>> 1. Cache write synchronization mode is not about 3rd party store,
>> it's about waiting for write replies from other nodes. But Ignite
>> with enabled 3rd party store guarantees consistency itself.
>>
>> 2. Ignite creates implicit transactions for atomic updates to
>> guarantee consistency between nodes. If you want to make more than one
>> update in transaction, then, of course, you should use explicit Ignite
>> transactions.
>>
>> Evgenii
>>
>> 2018-07-09 15:37 GMT+03:00 Prasad Bhalerao <
>> prasadbhalerao1...@gmail.com>:
>>
>>> Hi,
>>>
>>> I am using FULL_SYNC cache write synchronization mode.
>>> 1) Does FULL_SYNC mode guarantees 100% consistency between ignite in
>>> memory cache and 3rd party data store (oracle in my case)?
>>>
>>> 2) Does FULL_SYNC mode requires to do the cache updates in ignite
>>> transaction?( I am using write through approach.)
>>>
>>>
>>> Thanks,
>>> Prasad
>>>
>>
>>

>>

Re: cache configuration FULL_SYNC

[Prasad Bhalerao]

When you have multiple data sources (cache and oracle in this case) which
should be updated in single transaction we use JTA. But with ignite
transaction and ignite write through approach I don't have to use JTA.
To keep the cache and oracle consistent one needs transaction.  In this
case one update operation spans to multiple data sources.

On Mon, Jul 9, 2018 at 8:05 PM Evgenii Zhuravlev 
wrote:

> No, you're not correct, each insert/update will be consistent between
> cache and DB, transactions are not about consistency between cache and
> oracle database here, it's about more than one operation, as in an
> example I've shared in my previous message.
>
> Evgenii
>
> 2018-07-09 17:24 GMT+03:00 Prasad Bhalerao :
>
>> Thank you for the clarification.
>>
>> Ignite does not support transactions with jdbc updates. So if I use
>> delete/update SQL to delete/update the multiple cache entries, ignite will
>> not provide strong consistency between cache and oracle database.
>> Am I correct?
>>
>> On Mon, Jul 9, 2018 at 7:43 PM Evgenii Zhuravlev <
>> e.zhuravlev...@gmail.com> wrote:
>>
>>> 1. As this doc says: https://apacheignite.readme.io/docs/3rd-party-store:
>>> Ignite writes through results of its SQL INSERT, UPDATE and MERGE queries.
>>> 2. Transaction in Ignite means absolutely the same as in another
>>> systems. If you want two or more changes to be in one transaction(i.e.,
>>> classic example with transfer between bank accounts), then, for sure, you
>>> need to use transactions.
>>>
>>> Evgenii
>>>
>>> 2018-07-09 16:52 GMT+03:00 Prasad Bhalerao >> >:
>>>
 Resending my last mail ...

 I understand that FULL_SYNC is required to keep the consistency between
 primary data node and backup data node and not between the cache and 3rd
 party data.
 Sorry for asking the incorrect question.

 1) Does ignite supports write through updates to 3rd party store when
 using update or delete sql?

 2) Is it mandatory to use ignite transaction to keep strong consistency
 between in memory cache and 3rd party data store?
 -Like you said, if I am updating more than one entry in cache then
 transaction is necessary to keep strong consistency between 3rd party db
 and ignite cache. Am I correct?

 Thanks,
 Prasad

 On Mon, Jul 9, 2018 at 6:38 PM Evgenii Zhuravlev <
 e.zhuravlev...@gmail.com> wrote:

> Hi,
>
> 1. Cache write synchronization mode is not about 3rd party store, it's
> about waiting for write replies from other nodes. But Ignite with
> enabled 3rd party store guarantees consistency itself.
>
> 2. Ignite creates implicit transactions for atomic updates to
> guarantee consistency between nodes. If you want to make more than one
> update in transaction, then, of course, you should use explicit Ignite
> transactions.
>
> Evgenii
>
> 2018-07-09 15:37 GMT+03:00 Prasad Bhalerao <
> prasadbhalerao1...@gmail.com>:
>
>> Hi,
>>
>> I am using FULL_SYNC cache write synchronization mode.
>> 1) Does FULL_SYNC mode guarantees 100% consistency between ignite in
>> memory cache and 3rd party data store (oracle in my case)?
>>
>> 2) Does FULL_SYNC mode requires to do the cache updates in ignite
>> transaction?( I am using write through approach.)
>>
>>
>> Thanks,
>> Prasad
>>
>
>
>>>
>

Re: cache configuration FULL_SYNC

[Evgenii Zhuravlev]

No, you're not correct, each insert/update will be consistent between cache
and DB, transactions are not about consistency between cache and oracle
database here, it's about more than one operation, as in an example I've
shared in my previous message.

Evgenii

2018-07-09 17:24 GMT+03:00 Prasad Bhalerao :

> Thank you for the clarification.
>
> Ignite does not support transactions with jdbc updates. So if I use
> delete/update SQL to delete/update the multiple cache entries, ignite will
> not provide strong consistency between cache and oracle database.
> Am I correct?
>
> On Mon, Jul 9, 2018 at 7:43 PM Evgenii Zhuravlev 
> wrote:
>
>> 1. As this doc says: https://apacheignite.readme.io/docs/3rd-party-store:
>> Ignite writes through results of its SQL INSERT, UPDATE and MERGE queries.
>> 2. Transaction in Ignite means absolutely the same as in another systems.
>> If you want two or more changes to be in one transaction(i.e., classic
>> example with transfer between bank accounts), then, for sure, you need to
>> use transactions.
>>
>> Evgenii
>>
>> 2018-07-09 16:52 GMT+03:00 Prasad Bhalerao 
>> :
>>
>>> Resending my last mail ...
>>>
>>> I understand that FULL_SYNC is required to keep the consistency between
>>> primary data node and backup data node and not between the cache and 3rd
>>> party data.
>>> Sorry for asking the incorrect question.
>>>
>>> 1) Does ignite supports write through updates to 3rd party store when
>>> using update or delete sql?
>>>
>>> 2) Is it mandatory to use ignite transaction to keep strong consistency
>>> between in memory cache and 3rd party data store?
>>> -Like you said, if I am updating more than one entry in cache then
>>> transaction is necessary to keep strong consistency between 3rd party db
>>> and ignite cache. Am I correct?
>>>
>>> Thanks,
>>> Prasad
>>>
>>> On Mon, Jul 9, 2018 at 6:38 PM Evgenii Zhuravlev <
>>> e.zhuravlev...@gmail.com> wrote:
>>>
 Hi,

 1. Cache write synchronization mode is not about 3rd party store, it's
 about waiting for write replies from other nodes. But Ignite with
 enabled 3rd party store guarantees consistency itself.

 2. Ignite creates implicit transactions for atomic updates to guarantee
 consistency between nodes. If you want to make more than one update in
 transaction, then, of course, you should use explicit Ignite transactions.

 Evgenii

 2018-07-09 15:37 GMT+03:00 Prasad Bhalerao <
 prasadbhalerao1...@gmail.com>:

> Hi,
>
> I am using FULL_SYNC cache write synchronization mode.
> 1) Does FULL_SYNC mode guarantees 100% consistency between ignite in
> memory cache and 3rd party data store (oracle in my case)?
>
> 2) Does FULL_SYNC mode requires to do the cache updates in ignite
> transaction?( I am using write through approach.)
>
>
> Thanks,
> Prasad
>


>>

Re: How to create/ define a user defined cache while starting a standalone ignite node

[Ilya Kasnacheev]

Hello!

I'm afraid it's not possible to do that via app.config. You will have to
write Ignite XML config.

As per
https://apacheignite-net.readme.io/docs/configuration#section-spring-xml

Regards,

-- 
Ilya Kasnacheev

2018-07-09 11:36 GMT+03:00 Mahesh Talreja :

> Hi Team,
> I am trying to start an Ignite node in Standalone mode by just
> passing app.config file, I have set all ignite and ignite cache
> configuration in the file.
>
> but how to i specify a user defined type of cache to be created in
> app.config file in cache config section ?
>
> For example, I am trying to create a cache of 
>
> --
> *Thanking you,*
> *Regards,*
> *Mahesh Talreja*
> *Mob: +91 9769564242*
>

Register now for ApacheCon and save $250

[Rich Bowen]

Greetings, Apache software enthusiasts!

(You’re getting this because you’re on one or more dev@ or users@ lists 
for some Apache Software Foundation project.)


ApacheCon North America, in Montreal, is now just 80 days away, and 
early bird prices end in just two weeks - on July 21. Prices will be 
going up from $550 to $800 so register NOW to save $250, at 
http://apachecon.com/acna18


And don’t forget to reserve your hotel room. We have negotiated a 
special rate and the room block closes August 24. 
http://www.apachecon.com/acna18/venue.html


Our schedule includes over 100 talks and we’ll be featuring talks from 
dozens of ASF projects.,  We have inspiring keynotes from some of the 
brilliant members of our community and the wider tech space, including:


 * Myrle Krantz, PMC chair for Apache Fineract, and leader in the open 
source financing space
 * Cliff Schmidt, founder of Literacy Bridge (now Amplio) and creator 
of the Talking Book project

 * Bridget Kromhout, principal cloud developer advocate at Microsoft
 * Euan McLeod, Comcast engineer, and pioneer in streaming video

We’ll also be featuring tracks for Geospatial science, Tomcat, 
Cloudstack, and Big Data, as well as numerous other fields where Apache 
software is leading the way. See the full schedule at 
http://apachecon.com/acna18/schedule.html


As usual we’ll be running our Apache BarCamp, the traditional ApacheCon 
Hackathon, and the Wednesday evening Lighting Talks, too, so you’ll want 
to be there.


Register today at http://apachecon.com/acna18 and we’ll see you in Montreal!

--
Rich Bowen
VP, Conferences, The Apache Software Foundation
h...@apachecon.com
@ApacheCon

Re: cache configuration FULL_SYNC

[Prasad Bhalerao]

Thank you for the clarification.

Ignite does not support transactions with jdbc updates. So if I use
delete/update SQL to delete/update the multiple cache entries, ignite will
not provide strong consistency between cache and oracle database.
Am I correct?

On Mon, Jul 9, 2018 at 7:43 PM Evgenii Zhuravlev 
wrote:

> 1. As this doc says: https://apacheignite.readme.io/docs/3rd-party-store:
> Ignite writes through results of its SQL INSERT, UPDATE and MERGE queries.
> 2. Transaction in Ignite means absolutely the same as in another systems.
> If you want two or more changes to be in one transaction(i.e., classic
> example with transfer between bank accounts), then, for sure, you need to
> use transactions.
>
> Evgenii
>
> 2018-07-09 16:52 GMT+03:00 Prasad Bhalerao :
>
>> Resending my last mail ...
>>
>> I understand that FULL_SYNC is required to keep the consistency between
>> primary data node and backup data node and not between the cache and 3rd
>> party data.
>> Sorry for asking the incorrect question.
>>
>> 1) Does ignite supports write through updates to 3rd party store when
>> using update or delete sql?
>>
>> 2) Is it mandatory to use ignite transaction to keep strong consistency
>> between in memory cache and 3rd party data store?
>> -Like you said, if I am updating more than one entry in cache then
>> transaction is necessary to keep strong consistency between 3rd party db
>> and ignite cache. Am I correct?
>>
>> Thanks,
>> Prasad
>>
>> On Mon, Jul 9, 2018 at 6:38 PM Evgenii Zhuravlev <
>> e.zhuravlev...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> 1. Cache write synchronization mode is not about 3rd party store, it's
>>> about waiting for write replies from other nodes. But Ignite with
>>> enabled 3rd party store guarantees consistency itself.
>>>
>>> 2. Ignite creates implicit transactions for atomic updates to guarantee
>>> consistency between nodes. If you want to make more than one update in
>>> transaction, then, of course, you should use explicit Ignite transactions.
>>>
>>> Evgenii
>>>
>>> 2018-07-09 15:37 GMT+03:00 Prasad Bhalerao >> >:
>>>
 Hi,

 I am using FULL_SYNC cache write synchronization mode.
 1) Does FULL_SYNC mode guarantees 100% consistency between ignite in
 memory cache and 3rd party data store (oracle in my case)?

 2) Does FULL_SYNC mode requires to do the cache updates in ignite
 transaction?( I am using write through approach.)


 Thanks,
 Prasad

>>>
>>>
>

Re: cache configuration FULL_SYNC

[Evgenii Zhuravlev]

1. As this doc says: https://apacheignite.readme.io/docs/3rd-party-store:
Ignite writes through results of its SQL INSERT, UPDATE and MERGE queries.
2. Transaction in Ignite means absolutely the same as in another systems.
If you want two or more changes to be in one transaction(i.e., classic
example with transfer between bank accounts), then, for sure, you need to
use transactions.

Evgenii

2018-07-09 16:52 GMT+03:00 Prasad Bhalerao :

> Resending my last mail ...
>
> I understand that FULL_SYNC is required to keep the consistency between
> primary data node and backup data node and not between the cache and 3rd
> party data.
> Sorry for asking the incorrect question.
>
> 1) Does ignite supports write through updates to 3rd party store when
> using update or delete sql?
>
> 2) Is it mandatory to use ignite transaction to keep strong consistency
> between in memory cache and 3rd party data store?
> -Like you said, if I am updating more than one entry in cache then
> transaction is necessary to keep strong consistency between 3rd party db
> and ignite cache. Am I correct?
>
> Thanks,
> Prasad
>
> On Mon, Jul 9, 2018 at 6:38 PM Evgenii Zhuravlev 
> wrote:
>
>> Hi,
>>
>> 1. Cache write synchronization mode is not about 3rd party store, it's
>> about waiting for write replies from other nodes. But Ignite with
>> enabled 3rd party store guarantees consistency itself.
>>
>> 2. Ignite creates implicit transactions for atomic updates to guarantee
>> consistency between nodes. If you want to make more than one update in
>> transaction, then, of course, you should use explicit Ignite transactions.
>>
>> Evgenii
>>
>> 2018-07-09 15:37 GMT+03:00 Prasad Bhalerao 
>> :
>>
>>> Hi,
>>>
>>> I am using FULL_SYNC cache write synchronization mode.
>>> 1) Does FULL_SYNC mode guarantees 100% consistency between ignite in
>>> memory cache and 3rd party data store (oracle in my case)?
>>>
>>> 2) Does FULL_SYNC mode requires to do the cache updates in ignite
>>> transaction?( I am using write through approach.)
>>>
>>>
>>> Thanks,
>>> Prasad
>>>
>>
>>

Re: cache configuration FULL_SYNC

[Prasad Bhalerao]

Resending my last mail ...

I understand that FULL_SYNC is required to keep the consistency between
primary data node and backup data node and not between the cache and 3rd
party data.
Sorry for asking the incorrect question.

1) Does ignite supports write through updates to 3rd party store when using
update or delete sql?

2) Is it mandatory to use ignite transaction to keep strong consistency
between in memory cache and 3rd party data store?
-Like you said, if I am updating more than one entry in cache then
transaction is necessary to keep strong consistency between 3rd party db
and ignite cache. Am I correct?

Thanks,
Prasad

On Mon, Jul 9, 2018 at 6:38 PM Evgenii Zhuravlev 
wrote:

> Hi,
>
> 1. Cache write synchronization mode is not about 3rd party store, it's
> about waiting for write replies from other nodes. But Ignite with enabled 3rd
> party store guarantees consistency itself.
>
> 2. Ignite creates implicit transactions for atomic updates to guarantee
> consistency between nodes. If you want to make more than one update in
> transaction, then, of course, you should use explicit Ignite transactions.
>
> Evgenii
>
> 2018-07-09 15:37 GMT+03:00 Prasad Bhalerao :
>
>> Hi,
>>
>> I am using FULL_SYNC cache write synchronization mode.
>> 1) Does FULL_SYNC mode guarantees 100% consistency between ignite in
>> memory cache and 3rd party data store (oracle in my case)?
>>
>> 2) Does FULL_SYNC mode requires to do the cache updates in ignite
>> transaction?( I am using write through approach.)
>>
>>
>> Thanks,
>> Prasad
>>
>
>

Re: Can we start an ignite node by passing ignite configuations in json file in case on Dot net Core

[Вячеслав Коптилин]

Hello,

You can use Spring XML in order to configure Ignite instance.
Spring config file can be provided via Ignition.Start(string) method and
IgniteConfiguration.SpringConfigUrl property.
Please take a look at this page:
https://apacheignite-net.readme.io/docs/configuration#section-spring-xml

Thanks!

пн, 9 июл. 2018 г. в 11:15, Mahesh Talreja :

> Hi Team,
>  I am developing a project on DotNet Core , Since dot net core no longer
> supports app.config files, so is there a way where we can start an ignite
> node by passing ignite configurations in JSON file. If yes can you please
> share an example?
>
>
>
> --
> *Thanking you,*
> *Regards,*
> *Mahesh Talreja*
> *Mob: +91 9769564242*
>

Re: cache configuration FULL_SYNC

[Evgenii Zhuravlev]

Hi,

1. Cache write synchronization mode is not about 3rd party store, it's
about waiting for write replies from other nodes. But Ignite with enabled 3rd
party store guarantees consistency itself.

2. Ignite creates implicit transactions for atomic updates to guarantee
consistency between nodes. If you want to make more than one update in
transaction, then, of course, you should use explicit Ignite transactions.

Evgenii

2018-07-09 15:37 GMT+03:00 Prasad Bhalerao :

> Hi,
>
> I am using FULL_SYNC cache write synchronization mode.
> 1) Does FULL_SYNC mode guarantees 100% consistency between ignite in
> memory cache and 3rd party data store (oracle in my case)?
>
> 2) Does FULL_SYNC mode requires to do the cache updates in ignite
> transaction?( I am using write through approach.)
>
>
> Thanks,
> Prasad
>

RE: OptimizedMarshaller instead of BinaryMarshaller is used forScanQuery

[Stanislav Lukyanov]

Hi Calvin,

It should work the same for all queries. Ideally OptimizedMarshaller shouldn’t 
even be used (except for JDK classes). 

How did you check which marshaller is used in each case?

Can you share the code of your POJO? Or perhaps a runnable reproducer?

Can you also share the logs/check them for warnings? There might be a warning 
saying that there is a problem in using BinaryMarshaller.

Thanks,
Stan

From: Calvin KL Wong, CLSA
Sent: 9 июля 2018 г. 13:40
To: user@ignite.apache.org
Subject: RE: OptimizedMarshaller instead of BinaryMarshaller is used 
forScanQuery

Ok, I found that I can use binaryObject.deserialize() to get back the POJO.  
So, the only remaining question is:

When a cache does not have ‘withKeepBinary’ set, why ContinuousQuery can handle 
class versioning fine whereas ScanQuery cannot?  Is that an expected behaviour?

Thanks,
Calvin

From: Calvin KL Wong, CLSA [mailto:calvin.kl.w...@clsa.com] 
Sent: Monday, July 09, 2018 11:50 AM
To: user@ignite.apache.org
Subject: OptimizedMarshaller instead of BinaryMarshaller is used for ScanQuery

Hi,

I have a cache of .
I found that when I use a ScanQuery on that cache, result will be deserialized 
using OptimizedMarshaller; Ignite sends ‘ordered’ message using 
GridCacheQueryResonseEntry.  Whereas if I use ContinousQuery, result will be 
deserialized using BinaryMarshaller.

My problem is that ScanQuery won’t be able to handle class versioning.

Question:
1. I am using “keepBinary” and BinaryObject on the cache to avoid versioning 
related problem.  Is that the recommended approach?  Because it seems strange 
that “ScanQuery” and “ContinousQuery” have different behaviour.
2. What is the recommended way to convert BinaryObject into POJO?  Ignite 
website talks about getting field of each object using ‘BinaryObject.field()’.  
 However, Ignite uses ‘BinaryReaderExImpl implements BinaryReader’ internally.  
How can I get a reference to that object?

Thanks,
Calvin

Calvin KL Wong
Sr. Lead Engineer, Execution Services
D  +852 2600 7983  |  M  +852 9267 9471  |  T  +852 2600 
5/F, One Island East, 18 Westlands Road, Island East, Hong Kong



clsa.com
Insights. Liquidity. Capital. 



A CITIC Securities Company

The content of this communication is intended for the recipient and is subject 
to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon 
request.
Please consider before printing. CLSA is ISO14001 certified and committed to 
reducing its impact on the environment.
The content of this communication is intended for the recipient and is subject 
to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon 
request.
Please consider before printing. CLSA is ISO14001 certified and committed to 
reducing its impact on the environment.

Re: cache configuration FULL_SYNC

[Prasad Bhalerao]

I understand that FULL_SYNC is required to keep the consistency between
primary data node and backup data node and not between the cache and 3rd
party data.
Sorry for asking the incorrect question.

1) Does ignite supports write through updates to 3rd party store when using
update or delete sql?

2) Is it mandatory to use ignite transaction to keep strong consistency
between in memory cache and 3rd party data store?

Thanks,
Prasad


On Mon, Jul 9, 2018 at 6:07 PM Prasad Bhalerao 
wrote:

> Hi,
>
> I am using FULL_SYNC cache write synchronization mode.
> 1) Does FULL_SYNC mode guarantees 100% consistency between ignite in
> memory cache and 3rd party data store (oracle in my case)?
>
> 2) Does FULL_SYNC mode requires to do the cache updates in ignite
> transaction?( I am using write through approach.)
>
>
> Thanks,
> Prasad
>

Re: Affinity calls in stream receiver

[David Harvey]

We are testing whether  removing the nested classes helps things, and if
so, will create a reproducer.

   1. IGNITE-7905  is
   the issue where ignite.active(true) fails from a client if userVersion is
   non-zero, which seems to be due to nested classes.


On Tue, Jul 3, 2018, 4:20 AM Denis Mekhanikov  wrote:

> David,
>
> So, the problem is that the same class is loaded multiple times and it
> wastes the metaspace, right?
> Could you share a reproducer?
>
> Denis
>
> вт, 3 июл. 2018 г. в 0:58, David Harvey :
>
>> We have a custom stream receiver that makes affinity calls. This all
>> functions properly, but we see a very large number of the following
>> messages for the same two  classes.   We also just tripped a 2GB limit on
>> Metaspace size, which we came close to in the past.
>>
>> [18:41:50,365][INFO][pub-#6954%GridGainTrial%][GridDeploymentPerVersionStore]
>> Class was deployed in SHARED or CONTINUOUS mode: class
>> com.IgniteCallable
>>
>> So these affinity calls need to load classes that where loaded from
>> client nodes, which may be related to why this happening, but my primary
>> suspect is the fact that both classes are nested.  ( I had previously hit
>> an issue where setting the peer-class-loading "userVersion" would cause
>> ignite to thrown exceptions when the client node attempted to activate the
>> cluster.In that case, the Ignite call into the cluster was also using a
>> nested class. )
>>
>> We will try flattening these classes to see if the problem goes away.
>>
>>
>> *Disclaimer*
>>
>> The information contained in this communication from the sender is
>> confidential. It is intended solely for use by the recipient and others
>> authorized to receive it. If you are not the recipient, you are hereby
>> notified that any disclosure, copying, distribution or taking action in
>> relation of the contents of this information is strictly prohibited and may
>> be unlawful.
>>
>> This email has been scanned for viruses and malware, and may have been
>> automatically archived by *Mimecast Ltd*, an innovator in Software as a
>> Service (SaaS) for business. Providing a *safer* and *more useful* place
>> for your human generated data. Specializing in; Security, archiving and
>> compliance. To find out more Click Here
>> .
>>
>

Disclaimer

The information contained in this communication from the sender is 
confidential. It is intended solely for use by the recipient and others 
authorized to receive it. If you are not the recipient, you are hereby notified 
that any disclosure, copying, distribution or taking action in relation of the 
contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been 
automatically archived by Mimecast Ltd, an innovator in Software as a Service 
(SaaS) for business. Providing a safer and more useful place for your human 
generated data. Specializing in; Security, archiving and compliance. To find 
out more visit the Mimecast website.

cache configuration FULL_SYNC

[Prasad Bhalerao]

Hi,

I am using FULL_SYNC cache write synchronization mode.
1) Does FULL_SYNC mode guarantees 100% consistency between ignite in memory
cache and 3rd party data store (oracle in my case)?

2) Does FULL_SYNC mode requires to do the cache updates in ignite
transaction?( I am using write through approach.)


Thanks,
Prasad

Does ignite support Read-consistent queries?

[Prasad Bhalerao]

Read-consistent queries: Queries that produce consistent results with
respect to a point in time.

Please check this article. I am speaking w.r.t. this article.
https://asktom.oracle.com/pls/apex/f?p=100:11:0P11_QUESTION_ID:27330770500351

Oracle uses the information stored in rollback segments to provide this
read consistent view of information. Since the rollback segments are where
Oracle stores "old values" or before images of data during transaction
processing, it can use these old values to provide a query with the value
of data as it looked in the database when the query began. As a query
processes the blocks of data in a table, it will see if the data on that
block has changed since the query began. If it has Oracle will read the
information from the rollback segment to determine what the block looked
like when the query began, that is the view of the data the query will see.
This is how non-blocking reads are implemented as well - Oracle only looks
to see if the data changed, it does not care of the data is currently
locked (which implies that it has changed). It will simply retrieve the old
value from the rollback segment and proceed onto the next block of data.

Does ignite support this?


Thanks,
Prasad

OOME causing caches to be removed

[djax]

Ignite version: 2.5
This is on a 2 node cluster in AWS with 32GB RAM and 8 cores each.

When executing the following SQL query, the query fills the JVM heap of each
node (specified at 10 GB each node) and exits afterwards. On restart, the
USER cache is removed from persistence (the cache on which the SELECT
statement is being performed)

Any tips? Is this expected behavior?

SQL query:
INSERT INTO user_email
SELECT
email,max(sex),year(max(birthdate)),max(country),max(province),substr(email,position('@',email)+1)
FROM USER WHERE email IS NOT NULL GROUP BY email;

JVM options (excluding ignite libs and ignite home): -Xms18g -Xmx18g -server
-XX:+AggressiveOpts -XX:MaxMetaspaceSize=2048m
-Djava.net.preferIPv4Stack=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
-XX:+UseTLAB -XX:NewSize=128m -XX:MaxNewSize=128m -XX:MaxTenuringThreshold=0
-XX:SurvivorRatio=1024 -XX:+UseCMSInitiatingOccupancyOnly
-XX:CMSInitiatingOccupancyFraction=40 -XX:MaxGCPauseMillis=1000
-XX:InitiatingHeapOccupancyPercent=50 -XX:+UseCompressedOops
-XX:ParallelGCThreads=8 -XX:ConcGCThreads=8 -XX:+DisableExplicitGC
-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/usr/share/apache-ignite
-XX:+ExitOnOutOfMemoryError -DIGNITE_QUIET=false
-DIGNITE_SUCCESS_FILE=/usr/share/apache-ignite/work/ignite_success_ccedb5ad-5e37-4fee-aeae-19456d7da609
-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=49124
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false

Config (AWS credentials removed):


Logs:


Restart log:




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: OptimizedMarshaller instead of BinaryMarshaller is used for ScanQuery

[Calvin KL Wong, CLSA]

Ok, I found that I can use binaryObject.deserialize() to get back the POJO.  
So, the only remaining question is:

When a cache does not have 'withKeepBinary' set, why ContinuousQuery can handle 
class versioning fine whereas ScanQuery cannot?  Is that an expected behaviour?

Thanks,
Calvin

From: Calvin KL Wong, CLSA [mailto:calvin.kl.w...@clsa.com]
Sent: Monday, July 09, 2018 11:50 AM
To: user@ignite.apache.org
Subject: OptimizedMarshaller instead of BinaryMarshaller is used for ScanQuery

Hi,

I have a cache of .
I found that when I use a ScanQuery on that cache, result will be deserialized 
using OptimizedMarshaller; Ignite sends 'ordered' message using 
GridCacheQueryResonseEntry.  Whereas if I use ContinousQuery, result will be 
deserialized using BinaryMarshaller.

My problem is that ScanQuery won't be able to handle class versioning.

Question:

1.   I am using "keepBinary" and BinaryObject on the cache to avoid 
versioning related problem.  Is that the recommended approach?  Because it 
seems strange that "ScanQuery" and "ContinousQuery" have different behaviour.

2.   What is the recommended way to convert BinaryObject into POJO?  Ignite 
website talks about getting field of each object using 'BinaryObject.field()'.  
 However, Ignite uses 'BinaryReaderExImpl implements BinaryReader' internally.  
How can I get a reference to that object?

Thanks,
Calvin

Calvin KL Wong
Sr. Lead Engineer, Execution Services
D  +852 2600 7983  |  M  +852 9267 9471  |  T  +852 2600 
5/F, One Island East, 18 Westlands Road, Island East, Hong Kong

[:1. Social Media Icons:CLSA_Social Media 
Icons_linkedin.png][:1. Social Media 
Icons:CLSA_Social Media 
Icons_twitter.png][:1. Social Media 
Icons:CLSA_Social Media 
Icons_youtube.png][:1.
 Social Media Icons:CLSA_Social Media 
Icons_facebook.png]

clsa.com
Insights. Liquidity. Capital.

[CLSA_RGB]

A CITIC Securities Company


The content of this communication is intended for the recipient and is subject 
to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon 
request.
Please consider before printing. CLSA is ISO14001 certified and committed to 
reducing its impact on the environment.
The content of this communication is intended for the recipient and is subject 
to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon 
request.
Please consider before printing. CLSA is ISO14001 certified and committed to 
reducing its impact on the environment.

Re: When to use Ignite.Net Tansactions ?

[Mahesh Talreja]

Hi Team,
   Thanks for replying! I hope you might have got some bit of the
background of the requirement from my previous mail. If not, below is the
example of the class that will be consumed by a microservice (A console
app/background service). And using docker there will be multiple instances
of the same service hosted! 

In the example below, I have only single operations per function, so do I
really need transactions? Thinking about multiple instance/process
implementation I currently have them, But I am not really sure whether I
really need them, Or having just atomic mode will be fine?

Dose atomic mode guaranty prevention of dirty read and sequential execution?

Scenario 1
If thead1 trying to update tradeid 1 with version 1 started before thread 2 
Thread 2 trying to update tradeid 1 with version 2 started after thread 1
(both trying to update the same key)
if there is a race condition wherein trade 1 take time to complete its work,
will thread 2 wait for it?  and commit in a sequential manner even in atomic
mode?


public class TradeCache
{
private IIgnite igniteClient { get; set; }
private ICache cache { get; set; }

public TradeCache(IgniteConfiguration configuration)
{
igniteClient = Ignition.Start(configuration);
cache= igniteClient.GetCache("TradeDetailsCache");
}

public async Task PutAsync(TradeId tradeId,TradeDetails
tradeDetails)
{
using (ITransaction transaction =
igniteClient.GetTransactions().TxStart(
   TransactionConcurrency.Pessimistic,
TransactionIsolation.RepeatableRead,
   TimeSpan.FromMilliseconds(300), 0))
{
bool result = await cache.PutIfAbsentAsync(tradeId,
tradeDetails);

transaction.Commit();

return result;
}
}

public async Task ReplaceAsync(TradeId tradeId, TradeDetails
tradeDetails)
{
using (ITransaction transaction =
igniteClient.GetTransactions().TxStart(
   TransactionConcurrency.Pessimistic,
TransactionIsolation.RepeatableRead,
   TimeSpan.FromMilliseconds(300), 0))
{
bool result = await cache.ReplaceAsync(tradeId,
tradeDetails);

transaction.Commit();

return result;
}
}

public async Task RemoveAsync(TradeId tradeId)
{
using (ITransaction transaction =
igniteClient.GetTransactions().TxStart(
   TransactionConcurrency.Pessimistic,
TransactionIsolation.RepeatableRead,
   TimeSpan.FromMilliseconds(300), 0))
{
bool result = await cache.RemoveAsync(tradeId);

transaction.Commit();

return result;
}
}

}



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: When to use Ignite.Net Tansactions ?

[Jörn Franke]

Maybe you can elaborate more on your use case, because usually it is not a 
technical decision , but driven by user requirements.

> On 9. Jul 2018, at 10:01, Mahesh Talreja  wrote:
> 
> Hi Team,
> I am working on Dot Net project and trying to implement 
> Ignite.Net.
> Being new to the world of ignite, and after having a read above transactions 
> on ( https://apacheignite-net.readme.io/docs/transactions#atomicity-mode )
>  I am a bit confused with ignite transaction atomicity-mode's. 
> 
> To give you a bit of background, I am working on a project which is being 
> developed on microservice design/architecture, having plans to be hosted 
> using docker containers. So to summarise, I am expecting to have a 
> multiprocess/ multithreaded environment, wherein there can be multiple 
> instances of a microservice and each instance of a microservice will go ahead 
> a create an Ignite Node (Client Mode) and get connected to a cluster of 
> server nodes.
> 
> I am dealing with simple curd operations. Each user action resulting in only 
> a single curd operation at a time (Get/Put/Remove/Replace).  
> 
> So should I go ahead and use transactions (Pessimistic - Repeatable Read) 
> even when I have single operations in each transaction as it locks the 
> respective keys and provides sequential execution and is fully ACID compliant 
> in case of multiple processes,
> 
>  Or should I just go ahead with the Atomic mode which provides atomicity and 
> consistency but I am not sure about locks and sequential execution in case of 
> multiple processes?
>  

How to create/ define a user defined cache while starting a standalone ignite node

[Mahesh Talreja]

Hi Team,
I am trying to start an Ignite node in Standalone mode by just
passing app.config file, I have set all ignite and ignite cache
configuration in the file.

but how to i specify a user defined type of cache to be created in
app.config file in cache config section ?

For example, I am trying to create a cache of 

-- 
*Thanking you,*
*Regards,*
*Mahesh Talreja*
*Mob: +91 9769564242*

Can we start an ignite node by passing ignite configuations in json file in case on Dot net Core

[Mahesh Talreja]

Hi Team,
 I am developing a project on DotNet Core , Since dot net core no longer
supports app.config files, so is there a way where we can start an ignite
node by passing ignite configurations in JSON file. If yes can you please
share an example?



-- 
*Thanking you,*
*Regards,*
*Mahesh Talreja*
*Mob: +91 9769564242*

When to use Ignite.Net Tansactions ?

[Mahesh Talreja]

Hi Team,
I am working on Dot Net project and trying to implement
Ignite.Net.
Being new to the world of ignite, and after having a read above
transactions on (
https://apacheignite-net.readme.io/docs/transactions#atomicity-mode )
 I am a bit confused with ignite transaction atomicity-mode's.

To give you a bit of background, I am working on a project which is being
developed on microservice design/architecture, having plans to be hosted
using docker containers. So to summarise, I am expecting to have a
multiprocess/ multithreaded environment, wherein there *can be
multiple instances of a microservice and each instance of a microservice
will go ahead a create an Ignite Node (Client Mode) and get connected to a
cluster of server nodes.*

I am dealing with simple curd operations. Each user action resulting in
only a single curd operation at a time (Get/Put/Remove/Replace).

So should I go ahead and use transactions (Pessimistic - Repeatable Read)
even when I have single operations in each transaction as it locks the
respective keys and provides sequential execution and is fully ACID
compliant in case of multiple processes,

 Or should I just go ahead with the Atomic mode which provides atomicity
and consistency but I am not sure about locks and sequential execution in
case of multiple processes?