Re: Why is custom cacheStore.write() being called in clientMode?
Thanks to Ilya and Nikolai for your replies...Okay, as long as I can guarantee a consistent transaction should be fine...I will look into org.apache.ignite.lifecycle.LifecycleAware for my other problem. Thanks! -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Why-is-custom-cacheStore-write-being-called-in-clientMode-tp13309p13453.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Why is custom cacheStore.write() being called in clientMode?
Hi Rick,
Regarding your example: in Ignite Cache you can update keys 2 and 4 in the
same transaction without any problems, but it might be impossible to
perform a distributed transaction in DB behind CacheStore (it must
transactionally save key2 from node1 and key4 from node2). To solve this
problem, we put both keys into CacheStore from the same node - the one that
initiated transaction ("near node" in Ignite terminology).
Hope this helps.
On Tue, Jun 6, 2017 at 6:10 PM, Nikolai Tikhonov
wrote:
> Rick,
>
> > What you are saying is that I cannot update keys 2 and 4 in the same
> transaction, correct?
> No, it's will be update in the same transaction. I explained you why
> Ignite can't update store from dht nodes (nodes which own this data) and
> why Ignite propagates updates to store from client node.
>
> On Tue, Jun 6, 2017 at 5:42 PM, rick_tem wrote:
>
>> Hi Nikolai,
>>
>> Thanks for your reply. It is appreciated! Thanks for your answer to 2) I
>> will look into it. 3) and 4) are really the same issue I am trying to
>> understand how it works.
>>
>> With regards to 1) below, we aren't speaking about distributed databases,
>> but distributed caches that are java JVMs. But isn't that what a JTA
>> transaction manager is supposed to do? ie handle distributed
>> transactions?
>> if I enlist MQ and Jboss in the same transaction that is two seperate JVMs
>> and I believe should work with one atomic transaction...
>>
>> But regardless, I believe this is what you are saying here: Please
>> correct
>> me if I am wrong. Say I have keys 1, 2, 3 on node 1 and keys 4, 5, 6 on
>> node 2. What you are saying is that I cannot update keys 2 and 4 in the
>> same transaction, correct? This is because they live in two different
>> JVMs...If this is the case, that is a severe limitation as then I need to
>> know which node my data is on. What would your recommendation be here
>> then
>> for write-through cache? Have everything replicated? It is a requirement
>> that the transaction be rock solid in whatever model I implement. I
>> cannot
>> afford to lose writes or have half-committed data.
>>
>> Thanks,
>> Rick
>>
>>
>>
>> --
>> View this message in context: http://apache-ignite-users.705
>> 18.x6.nabble.com/Why-is-custom-cacheStore-write-being-called
>> -in-clientMode-tp13309p13424.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>
--
Best regards,
Ilya
Re: Why is custom cacheStore.write() being called in clientMode?
Rick, > What you are saying is that I cannot update keys 2 and 4 in the same transaction, correct? No, it's will be update in the same transaction. I explained you why Ignite can't update store from dht nodes (nodes which own this data) and why Ignite propagates updates to store from client node. On Tue, Jun 6, 2017 at 5:42 PM, rick_tem wrote: > Hi Nikolai, > > Thanks for your reply. It is appreciated! Thanks for your answer to 2) I > will look into it. 3) and 4) are really the same issue I am trying to > understand how it works. > > With regards to 1) below, we aren't speaking about distributed databases, > but distributed caches that are java JVMs. But isn't that what a JTA > transaction manager is supposed to do? ie handle distributed transactions? > if I enlist MQ and Jboss in the same transaction that is two seperate JVMs > and I believe should work with one atomic transaction... > > But regardless, I believe this is what you are saying here: Please correct > me if I am wrong. Say I have keys 1, 2, 3 on node 1 and keys 4, 5, 6 on > node 2. What you are saying is that I cannot update keys 2 and 4 in the > same transaction, correct? This is because they live in two different > JVMs...If this is the case, that is a severe limitation as then I need to > know which node my data is on. What would your recommendation be here then > for write-through cache? Have everything replicated? It is a requirement > that the transaction be rock solid in whatever model I implement. I cannot > afford to lose writes or have half-committed data. > > Thanks, > Rick > > > > -- > View this message in context: http://apache-ignite-users. > 70518.x6.nabble.com/Why-is-custom-cacheStore-write-being- > called-in-clientMode-tp13309p13424.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com. >
Re: Why is custom cacheStore.write() being called in clientMode?
Hi Nikolai, Thanks for your reply. It is appreciated! Thanks for your answer to 2) I will look into it. 3) and 4) are really the same issue I am trying to understand how it works. With regards to 1) below, we aren't speaking about distributed databases, but distributed caches that are java JVMs. But isn't that what a JTA transaction manager is supposed to do? ie handle distributed transactions? if I enlist MQ and Jboss in the same transaction that is two seperate JVMs and I believe should work with one atomic transaction... But regardless, I believe this is what you are saying here: Please correct me if I am wrong. Say I have keys 1, 2, 3 on node 1 and keys 4, 5, 6 on node 2. What you are saying is that I cannot update keys 2 and 4 in the same transaction, correct? This is because they live in two different JVMs...If this is the case, that is a severe limitation as then I need to know which node my data is on. What would your recommendation be here then for write-through cache? Have everything replicated? It is a requirement that the transaction be rock solid in whatever model I implement. I cannot afford to lose writes or have half-committed data. Thanks, Rick -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Why-is-custom-cacheStore-write-being-called-in-clientMode-tp13309p13424.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Why is custom cacheStore.write() being called in clientMode?
Hi, 1) Dht nodes is remote nodes which actually store data. Let's I explain it on simple example. You start transaction and update 10 keys. This keys mapped on dht 3 nodes (this entries phisycly will be stored on this nodes). You need to consisitency updates data on all nodes and persiste them CacheStore too. Ignite is not able to transactionally update store from 3 nodes. (It different processe, JVM and etc. Also how many db do you khow that support it? ;) ) By this reasons near node (which started transaction) propagate an updates to store. 2) As Yakov said about it you can do it by org.apache.ignite.lifecycle.LifecycleAware. Look at org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore as example. Also you could use store factory for initialization CacheConfiguration#setCacheStoreFactory 3) Are you sure that you invoked loadLoad() and got write() on client node? Could you share logs from both nodes: server and client? 4) To be honest this question not clear. Could you provide more context about what you mean? On Thu, Jun 1, 2017 at 4:42 PM, rick_tem wrote: > Hi Yakov, > > Thanks for you reply, it is appreciated, but I still have questions. > > When you say: 1) we don't have an option to write-through from dht nodes? > What is a dht node? Are you saying I can't write my own write-through > cache > store? > 2) I agree that initialization isn't the best in loadCache. But how do I > then do it? It is Ignite that is creating this class and the cache > returned > from ignite.getCache is not an instance of my class. > 3) Client mode seems inconsistent. It is not calling loadCache() as > expected but why would it call write()...It should be a proxy to the server > node, not acting as a server. > 4) Are you saying Ignite does not keep transactional state consistent > across > nodes for write-through? For any cache store? Even JDBC cache store? > > Thanks, > Rick > > > > > -- > View this message in context: http://apache-ignite-users. > 70518.x6.nabble.com/Why-is-custom-cacheStore-write-being- > called-in-clientMode-tp13309p13317.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com. >
Re: Why is custom cacheStore.write() being called in clientMode?
Hi Yakov, Thanks for you reply, it is appreciated, but I still have questions. When you say: 1) we don't have an option to write-through from dht nodes? What is a dht node? Are you saying I can't write my own write-through cache store? 2) I agree that initialization isn't the best in loadCache. But how do I then do it? It is Ignite that is creating this class and the cache returned from ignite.getCache is not an instance of my class. 3) Client mode seems inconsistent. It is not calling loadCache() as expected but why would it call write()...It should be a proxy to the server node, not acting as a server. 4) Are you saying Ignite does not keep transactional state consistent across nodes for write-through? For any cache store? Even JDBC cache store? Thanks, Rick -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Why-is-custom-cacheStore-write-being-called-in-clientMode-tp13309p13317.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Why is custom cacheStore.write() being called in clientMode?
In Ignite terminology near node is the node that coordinates transaction or cache update. As far as store initialization you can implement org.apache.ignite.lifecycle.LifecycleAware on your store and inject Ignite instance to your store with org.apache.ignite.resources.IgniteInstanceResource Initialization in loadCache seems incorrect to me. Imagine you start new nodes after loadCache is called with previous topology. --Yakov 2017-06-01 15:39 GMT+03:00 rick_tem : > I'm sorry, what you are saying doesn't make sense. I don't want a near > cache. I'm not sure why it is a near cache. I haven't specified that > anywhere in the config. It is showing up as a client in the topology. You > are saying it isn't possible to write your own write-through cacheStore? > > Thanks, > Rick > > > > -- > View this message in context: http://apache-ignite-users. > 70518.x6.nabble.com/Why-is-custom-cacheStore-write-being- > called-in-clientMode-tp13309p13314.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com. >
Re: Why is custom cacheStore.write() being called in clientMode?
I'm sorry, what you are saying doesn't make sense. I don't want a near cache. I'm not sure why it is a near cache. I haven't specified that anywhere in the config. It is showing up as a client in the topology. You are saying it isn't possible to write your own write-through cacheStore? Thanks, Rick -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Why-is-custom-cacheStore-write-being-called-in-clientMode-tp13309p13314.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Why is custom cacheStore.write() being called in clientMode?
In order to commit DB transaction consistently with in-memory one it is
committed from near node (which is client node in your case).
Currently we don't have an option to write-through from dht nodes, but for
now you can try write behind functionality -
https://apacheignite.readme.io/docs/persistent-store#write-behind-caching
--Yakov
2017-06-01 14:40 GMT+03:00 rick_tem :
> Hello,
>
> I have a written a cache store and started it up on one node and call
> loadCache() and the output is shown below.
>
> public class IgniteXMLClobCacheStoreimplements CacheStore
> {
>
> }
>
> IgniteXMLClobCacheStore::loadCache Loading: LAYOUT
> IgniteXMLClobCacheStore::loadCache Loading: RISK
> IgniteXMLClobCacheStore::loadCache Loading: TOP_ACCOUNT
> IgniteXMLClobCacheStore::loadCache finished loading. Now listening...
> [11:27:14] Topology snapshot [ver=2, servers=1, clients=1, CPUs=4,
> heap=1.3GB]
>
> I start another in client mode (see above topology snapshot), in this case
> loadCache() is not called (as expected), however write() is called. I
> eventually get a null pointer as something isn't initialized when I call
> loadCache. In the stack trace below it looks like it is treating it as a
> nearCache. Shouldn't write() be called on my server node and not my
> client?
> If I am doing something wrong, how then would I then initialize something
> as
> this object is created by Ignite and I can't pass it via loadCache() since
> it isn't called in client mode. ie,
>
>
> value="TablesXMLClobWriteThrough"
> />
>
>
> factory-method="factoryOf">
>
> value="com.temenos.tafj.cache.impl.IgniteXMLClobCacheStore">
>
>
>
>
>
> [ERROR] 2017-06-01 11:27:42,652 [sys-stripe-3-#4%TemenosGrid%] CACHE -
> IgniteXMLClobCacheStore::Error in write with Key: BLA*RANK
> java.lang.NullPointerException: null
> at
> com.temenos.tafj.cache.impl.IgniteXMLClobCacheStore.write(
> IgniteXMLClobCacheStore.java:451)
> [TAFJCache.jar:18.1.0-SNAPSHOT]
> at
> org.apache.ignite.internal.processors.cache.store.
> GridCacheStoreManagerAdapter.put(GridCacheStoreManagerAdapter.java:568)
> [ignite-core-2.0.0.jar:2.0.0]
> at
> org.apache.ignite.internal.processors.cache.store.
> GridCacheStoreManagerAdapter.putAll(GridCacheStoreManagerAdapter.java:609)
> [ignite-core-2.0.0.jar:2.0.0]
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.
> batchStoreCommit(IgniteTxAdapter.java:1326)
> [ignite-core-2.0.0.jar:2.0.0]
> at
> org.apache.ignite.internal.processors.cache.transactions.
> IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:500)
> [ignite-core-2.0.0.jar:2.0.0]
> at
> org.apache.ignite.internal.processors.cache.distributed.
> near.GridNearTxLocal.localFinish(GridNearTxLocal.java:3021)
> [ignite-core-2.0.0.jar:2.0.0]
> at
> org.apache.ignite.internal.processors.cache.distributed.
> near.GridNearTxFinishFuture.finish(GridNearTxFinishFuture.java:405)
> [ignite-core-2.0.0.jar:2.0.0]
> at
> org.apache.ignite.internal.processors.cache.distributed.
> near.GridNearTxLocal$16.apply(GridNearTxLocal.java:3171)
> [ignite-core-2.0.0.jar:2.0.0]
> at
> org.apache.ignite.internal.processors.cache.distributed.
> near.GridNearTxLocal$16.apply(GridNearTxLocal.java:3163)
> [ignite-core-2.0.0.jar:2.0.0]
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(
> GridFutureAdapter.java:382)
> [ignite-core-2.0.0.jar:2.0.0]
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.
> unblock(GridFutureAdapter.java:346)
> [ignite-core-2.0.0.jar:2.0.0]
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.
> unblockAll(GridFutureAdapter.java:334)
> [ignite-core-2.0.0.jar:2.0.0]
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.
> onDone(GridFutureAdapter.java:494)
> [ignite-core-2.0.0.jar:2.0.0]
> at
> org.apache.ignite.internal.processors.cache.GridCacheCompoundFuture.
> onDone(GridCacheCompoundFuture.java:56)
> [ignite-core-2.0.0.jar:2.0.0]
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.
> onDone(GridFutureAdapter.java:473)
> [ignite-core-2.0.0.jar:2.0.0]
> at
> org.apache.ignite.internal.processors.cache.distributed.near.
> GridNearOptimisticTxPrepareFuture.onComplete(
> GridNearOptimisticTxPrepareFuture.java:295)
> [ignite-core-2.0.0.jar:2.0.0]
> at
> org.apache.ignite.internal.processors.cache.distributed.near.
> GridNearOptimisticTxPrepareFuture.onDone(GridNearOptimisticTxPrepareFut
> ure.java:273)
> [ignite-core-2.0.0.jar:2.0.0]
> at
> org.apache.ignite.internal.processors.cache.distributed.near.
> GridNearOptimisticTxPrepar
Why is custom cacheStore.write() being called in clientMode?
Hello,
I have a written a cache store and started it up on one node and call
loadCache() and the output is shown below.
public class IgniteXMLClobCacheStoreimplements CacheStore {
}
IgniteXMLClobCacheStore::loadCache Loading: LAYOUT
IgniteXMLClobCacheStore::loadCache Loading: RISK
IgniteXMLClobCacheStore::loadCache Loading: TOP_ACCOUNT
IgniteXMLClobCacheStore::loadCache finished loading. Now listening...
[11:27:14] Topology snapshot [ver=2, servers=1, clients=1, CPUs=4,
heap=1.3GB]
I start another in client mode (see above topology snapshot), in this case
loadCache() is not called (as expected), however write() is called. I
eventually get a null pointer as something isn't initialized when I call
loadCache. In the stack trace below it looks like it is treating it as a
nearCache. Shouldn't write() be called on my server node and not my client?
If I am doing something wrong, how then would I then initialize something as
this object is created by Ignite and I can't pass it via loadCache() since
it isn't called in client mode. ie,
[ERROR] 2017-06-01 11:27:42,652 [sys-stripe-3-#4%TemenosGrid%] CACHE -
IgniteXMLClobCacheStore::Error in write with Key: BLA*RANK
java.lang.NullPointerException: null
at
com.temenos.tafj.cache.impl.IgniteXMLClobCacheStore.write(IgniteXMLClobCacheStore.java:451)
[TAFJCache.jar:18.1.0-SNAPSHOT]
at
org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.put(GridCacheStoreManagerAdapter.java:568)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.putAll(GridCacheStoreManagerAdapter.java:609)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.batchStoreCommit(IgniteTxAdapter.java:1326)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:500)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.localFinish(GridNearTxLocal.java:3021)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.finish(GridNearTxFinishFuture.java:405)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$16.apply(GridNearTxLocal.java:3171)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$16.apply(GridNearTxLocal.java:3163)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:382)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:346)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:334)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:494)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.processors.cache.GridCacheCompoundFuture.onDone(GridCacheCompoundFuture.java:56)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:473)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearOptimisticTxPrepareFuture.onComplete(GridNearOptimisticTxPrepareFuture.java:295)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearOptimisticTxPrepareFuture.onDone(GridNearOptimisticTxPrepareFuture.java:273)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearOptimisticTxPrepareFuture.onDone(GridNearOptimisticTxPrepareFuture.java:72)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:450)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.util.future.GridCompoundFuture.checkComplete(GridCompoundFuture.java:278)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.util.future.GridCompoundFuture.apply(GridCompoundFuture.java:148)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.util.future.GridCompoundFuture.apply(GridCompoundFuture.java:45)
[ignite-core-2.0.0.jar:2.0.0]
at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:382)
[ignite
