Re: Failed to execute the cache operation (all partition owners are left the grid partition data has been lost))
Seems you faced with https://issues.apache.org/jira/browse/IGNITE-14073 >Hi, >I have .Net Ignite client and server app Ignite v2.9.1. we have a 2 node >cluster, >recently we are upgraded ignite 2.7.6 to ignite 2.9.1 after using this >version we are facing issues frequently. > >[req=o.a.i.i.processors.platform.client.cache.ClientCacheGetRequest@bb5fb96] >javax.cache.CacheException: class >org.apache.ignite.internal.processors.cache.CacheInvalidStateException: >Failed to execute the cache operation (all partition owners have left the >grid, partition data has been lost) [cacheName=CREDENTIALS, partition=759, >key=UserKeyCacheObjectImpl [part=759, >val=SqlCredentialAdo-Althing-energyapp, hasValBytes=false]] >at >org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1270) >~[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2083) >~[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.get(IgniteCacheProxyImpl.java:1110) >~[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.get(GatewayProtectedCacheProxy.java:676) >~[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.processors.platform.client.cache.ClientCacheGetRequest.process(ClientCacheGetRequest.java:41) >~[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.processors.platform.client.ClientRequestHandler.handle(ClientRequestHandler.java:99) >~[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:202) >[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:56) >[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279) >[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109) >[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:97) >[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) >[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:70) >[ignite-core-2.9.1.jar:2.9.1] >at >java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >[?:1.8.0_181] >at >java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >[?:1.8.0_181] >at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181] >Caused by: >org.apache.ignite.internal.processors.cache.CacheInvalidStateException: >Failed to execute the cache operation (all partition owners have left the >grid, partition data has been lost) [cacheName=CREDENTIALS, partition=759, >key=UserKeyCacheObjectImpl [part=759, >val=SqlCredentialAdo-Althing-energyapp, hasValBytes=false]] >at >org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTopologyFutureAdapter.validateKey(GridDhtTopologyFutureAdapter.java:209) >~[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTopologyFutureAdapter.validateCache(GridDhtTopologyFutureAdapter.java:128) >~[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.validate(GridPartitionedSingleGetFuture.java:859) >~[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.map(GridPartitionedSingleGetFuture.java:277) >~[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.init(GridPartitionedSingleGetFuture.java:244) >~[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.getAsync0(GridDhtAtomicCache.java:1471) >~[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$1600(GridDhtAtomicCache.java:141) >~[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$16.apply(GridDhtAtomicCache.java:477) >~[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$16.apply(GridDhtAtomicCache.java:475) >~[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.asyncOp(GridDhtAtomicCache.java:779) >~[ignite-core-2.9.1.jar:2.9.1] >at >org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.getAsync(GridDhtAtomicCache.java:475) >~[ignite-cor
Re: Ignite Startup Warnings
Thanks for your replies and creating the issue. ilya.kasnacheev wrote > The second one is clearly also a bug. You can sidestep it by setting log > level to INFO. Shall I create an issue for the second one too? Regards, Paolo -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Failed to execute the cache operation (all partition owners are left the grid partition data has been lost))
Hi, I have .Net Ignite client and server app Ignite v2.9.1. we have a 2 node cluster, recently we are upgraded ignite 2.7.6 to ignite 2.9.1 after using this version we are facing issues frequently. [req=o.a.i.i.processors.platform.client.cache.ClientCacheGetRequest@bb5fb96] javax.cache.CacheException: class org.apache.ignite.internal.processors.cache.CacheInvalidStateException: Failed to execute the cache operation (all partition owners have left the grid, partition data has been lost) [cacheName=CREDENTIALS, partition=759, key=UserKeyCacheObjectImpl [part=759, val=SqlCredentialAdo-Althing-energyapp, hasValBytes=false]] at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1270) ~[ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2083) ~[ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.get(IgniteCacheProxyImpl.java:1110) ~[ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.get(GatewayProtectedCacheProxy.java:676) ~[ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.processors.platform.client.cache.ClientCacheGetRequest.process(ClientCacheGetRequest.java:41) ~[ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.processors.platform.client.ClientRequestHandler.handle(ClientRequestHandler.java:99) ~[ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:202) [ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:56) [ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279) [ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109) [ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:97) [ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) [ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:70) [ignite-core-2.9.1.jar:2.9.1] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181] Caused by: org.apache.ignite.internal.processors.cache.CacheInvalidStateException: Failed to execute the cache operation (all partition owners have left the grid, partition data has been lost) [cacheName=CREDENTIALS, partition=759, key=UserKeyCacheObjectImpl [part=759, val=SqlCredentialAdo-Althing-energyapp, hasValBytes=false]] at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTopologyFutureAdapter.validateKey(GridDhtTopologyFutureAdapter.java:209) ~[ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTopologyFutureAdapter.validateCache(GridDhtTopologyFutureAdapter.java:128) ~[ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.validate(GridPartitionedSingleGetFuture.java:859) ~[ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.map(GridPartitionedSingleGetFuture.java:277) ~[ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.init(GridPartitionedSingleGetFuture.java:244) ~[ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.getAsync0(GridDhtAtomicCache.java:1471) ~[ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$1600(GridDhtAtomicCache.java:141) ~[ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$16.apply(GridDhtAtomicCache.java:477) ~[ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$16.apply(GridDhtAtomicCache.java:475) ~[ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.asyncOp(GridDhtAtomicCache.java:779) ~[ignite-core-2.9.1.jar:2.9.1] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.getAsync(
Re: 2.8.1 : Continuous Query Initial query not returning any result sometimes and UnregisteredBinaryTypeExceptio
On further debugging, it looks like all the clients that had failed the count, were executing their query on the same server machinename002. When I looked at the server logs, it looks like that the records that were missing were not present on this server when the query was executed but were present on the other two servers. ( Kindly note that the cache is replicated one and all the records should ideally be present on all the 3 servers ). When I look at the exceptions logs, I see unregisteredbinarytypeexception for these records : Attempted to update binary metadata inside a critical synchronization block (will be automatically retried). This exception must not be wrapped to any other exception class. If you encounter this exception outside of EntryProcessor, please report to Apache Ignite dev-list. Typically we dont do anything when we encounter this exception as per the message. Please let me know if we need to do anything here . In this case, I suspect the reattempt of the entry processor did not work ? and left the machinename002 out of sync with the other two servers ? How do we know if the UnregisteredBinaryTypeException has resolved in the next attempt ? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ignite common code gitflow
Hi, guys! I am implementing an Ignite cluster with data node, service node and spring boot node (REST API) with embedded ignite client. My nodes have common POJO classes. Is there a nice approach to store all nodes at single repository (sharing common POJO) to implement common unit testing for cluster. Same time i need to separate cicd pipe for different nodes. 1st target: redeploy all nodes, when the changes in common POJO. 2nd target: redeploy service nodes only when buisness logic changes. My main target is cluster unit-testing and change safety for neighbor nodes. Also if it metters, we plan to put each node to a docker container. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
