Re: IgniteTxOptimisticCheckedException
Hello!
Optimistic transactions are supposed to be retry-able. Why won't you retry
such transactions upon getting the exception?
Regards,
--
Ilya Kasnacheev
пт, 31 июл. 2020 г. в 10:56, marble.zh...@coinflex.com <
marble.zh...@coinflex.com>:
> Hi Guru,
>
> We met below two kinds of Optimistic exceptions when doing the
> transactions,
> we use OPTIMISTIC and SERIALIZABLE isolation, do you have any suggestions,
> or if we change to PESSIMISTIC, any risks on that, such as performance or
> other exceptions? thanks.
>
> {"log":"Caused by:
> org.apache.ignite.internal.transactions.IgniteTxOptimisticCheckedException:
> Failed to prepare transaction, read/write conflict [key=92-USD,
> keyCls=java.lang.String, val=Balance(accountId=92,
> instrumentId=USD,
> quantity=2307.812920282500, reserved=0, lastUpdated=2020-07-31
> 03:10:36.889, lastUpdatedEvent=TRANSFER, tradeType=null),
> valCls=ignite.entity.Balance, cache=Balance, thread=IgniteThread
> [compositeRwLockIdx=7, stripe=4, plc=-1, holdsTopLock=false,
> forbiddenToRequestBinaryMetadata=false,
>
> name=sys-stripe-4-#5]]\n","stream":"stderr","time":"2020-07-31T03:10:36.953698541Z"}
>
>
> {"log":"Caused by:
> org.apache.ignite.internal.transactions.IgniteTxOptimisticCheckedException:
> Failed to prepare transaction (lock conflict): GridDhtTxLocal
> [nearNodeId=7e3589a0-c829-42a2-8fb4-115c57ba64f5,
> nearFutId=3077cc2a371-7364caca-357e-49f2-840e-a101e7c64fa8, nearMiniId=1,
> nearFinFutId=null, nearFinMiniId=0, nearXidVer=GridCacheVersion
> [topVer=207644034, order=1596164732765, nodeOrder=2], lb=null,
> super=GridDhtTxLocalAdapter [nearOnOriginatingNode=false,
> nearNodes=KeySetView [], dhtNodes=KeySetView [], explicitLock=false,
> super=IgniteTxLocalAdapter [completedBase=null, sndTransformedVals=false,
> depEnabled=false, txState=IgniteTxStateImpl [activeCacheIds=[1325467324],
> recovery=false, mvccEnabled=false, mvccCachingCacheIds=[], txMap=ArrayList
> [IgniteTxEntry [txKey=IgniteTxKey [key=KeyCacheObjectImpl [part=45,
> val=92-USD, hasValBytes=true], cacheId=1325467324], val=[op=UPDATE,
> val=ignite.entity.Balance [idHash=1055847466, hash=-1577210704,
> accountId=92, lastUpdated=2020-07-31 03:10:36.943,
> quantity=1204.259920282500, reserved=null, instrumentId=USD,
> tradeType=null, lastUpdatedEvent=TRANSFER]], prevVal=[op=NOOP, val=null],
> oldVal=[op=NOOP, val=null], entryProcessorsCol=null, ttl=-1,
> conflictExpireTime=-1, conflictVer=null, explicitVer=null, dhtVer=null,
> filters=CacheEntryPredicate[] [], filtersPassed=false, filtersSet=false,
> entry=GridDhtCacheEntry [rdrs=ReaderId[] [], part=45,
> super=GridDistributedCacheEntry [super=GridCacheMapEntry
> [key=KeyCacheObjectImpl [part=45, val=92-USD, hasValBytes=true],
> val=null, ver=GridCacheVersion [topVer=207644034, order=1596164732830,
> nodeOrder=1], hash=-148923086, extras=GridCacheObsoleteEntryExtras
> [obsoleteVer=GridCacheVersion [topVer=2147483647, order=0, nodeOrder=0]],
> flags=2]]], prepared=1, locked=false, nodeId=null, locMapped=false,
> expiryPlc=null, transferExpiryPlc=false, flags=0, partUpdateCntr=0,
> serReadVer=GridCacheVersion [topVer=207644034, order=1596164732670,
> nodeOrder=1], xidVer=null], IgniteTxEntry [txKey=IgniteTxKey
> [key=KeyCacheObjectImpl [part=504, val=1380978-USD, hasValBytes=true],
> cacheId=1325467324], val=[op=UPDATE, val=ignite.entity.Balance
> [idHash=577969755, hash=-2001853349, accountId=1380978,
> lastUpdated=2020-07-31 03:10:36.95, quantity=13614077.57624000,
> reserved=null, instrumentId=USD, tradeType=null,
> lastUpdatedEvent=TRANSFER]], prevVal=[op=NOOP, val=null], oldVal=[op=NOOP,
> val=null], entryProcessorsCol=null, ttl=-1, conflictExpireTime=-1,
> conflictVer=null, explicitVer=null, dhtVer=null,
> filters=CacheEntryPredicate[] [], filtersPassed=false, filtersSet=false,
> entry=GridDhtCacheEntry [rdrs=ReaderId[] [], part=504,
> super=GridDistributedCacheEntry [super=GridCacheMapEntry
> [key=KeyCacheObjectImpl [part=504, val=1380978-USD, hasValBytes=true],
> val=null, ver=GridCacheVersion [topVer=0, order=0, nodeOrder=0],
> hash=1004605465, extras=null, flags=0]]], prepared=0, locked=false,
> nodeId=null, locMapped=false, expiryPlc=null, transferExpiryPlc=false,
> flags=0, partUpdateCntr=0, serReadVer=GridCacheVersion [topVer=207644034,
> order=1596164732670, nodeOrder=1], xidVer=null]]], super=IgniteTxAdapter
> [xidVer=GridCacheVersion [topVer=207644034, order=1596164732835,
> nodeOrder=1], writeVer=null, implicit=false, loc=true, threadId=146,
> startTime=1596165036958, nodeId=990fbcf8-4105-4aee-b03f-bef9141f5533,
> startVer=GridCacheVersion [topVer=207644034, order=15961647
Re: question about collation with an ignite set
Hello!
Why don't you set cacheMode to REPLICATED?
Partitions in replicated caches should not be lost.
Regards,
--
Ilya Kasnacheev
пн, 3 авг. 2020 г. в 08:35, scottmf :
> hi, If I setup my *Ignite Set* as specified below what will happen when a
> node leaves the cluster topology forever? Will I simply lose the elements
> which are stored locally - via collation = true - or will I run into
> problems?
>
> Overall I want the all cluster nodes to be aware of all elements in the
> distributed set. If a node leaves the topology i want the elements
> associated with said node to simply disappear from the set.
>
> Will this configuration achieve that functionality?
>
> CollectionConfiguration cacheConfiguration = new
> CollectionConfiguration();
> cacheConfiguration.setCollocated(true);
> cacheConfiguration.setBackups(0);
> cacheConfiguration.setGroupName("grp");
> this.countQueriesSet = ignite.set("myset", cacheConfiguration);
>
>
> --
> Sent from the Apache Ignite Users mailing list archive
> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>
Re: What does all partition owners have left the grid, partition data has been lost mean?
Hello! What is your baseline topology at this moment? It means just that: you have lost enough nodes of your distributed grid that data is nowhere to be found now. Regards, -- Ilya Kasnacheev пн, 3 авг. 2020 г. в 19:12, John Smith : > I get the below exception on my client... > > #1 I rebooted the cache nodes error still continued. > #2 restarted the client node error went away. > #3 this seems to happen every few weeks. > #4 is there some sort of timeout values and retries I can put? > #5 cache operations seem to block when rebooting the nodes (I have 3 > nodes). Is there a way not to block? > > javax.cache.CacheException: class > org.apache.ignite.internal.processors.cache.CacheInvalidStateException: > Failed to execute cache operation (all partition owners have left the grid, > partition data has been lost) [cacheName=xx, part=273, key=16479796986] > at > org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1337) > at > org.apache.ignite.internal.processors.cache.IgniteCacheFutureImpl.convertException(IgniteCacheFutureImpl.java:62) > at > org.apache.ignite.internal.util.future.IgniteFutureImpl.get(IgniteFutureImpl.java:157) > at > com.xx.common.vertx.ext.data.impl.IgniteCacheRepository.lambda$executeAsync$394d953f$1(IgniteCacheRepository.java:59) > at > org.apache.ignite.internal.util.future.AsyncFutureListener$1.run(AsyncFutureListener.java:53) > at > com.xx.common.vertx.ext.data.impl.VertxIgniteExecutorAdapter.lambda$execute$0(VertxIgniteExecutorAdapter.java:18) > at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:369) > at > io.vertx.core.impl.EventLoopContext.lambda$executeAsync$0(EventLoopContext.java:38) > at > io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) > at > io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:497) > at > io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) > at > io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) > at > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > at java.lang.Thread.run(Thread.java:748) > Caused by: > org.apache.ignite.internal.processors.cache.CacheInvalidStateException: > Failed to execute cache operation (all partition owners have left the grid, > partition data has been lost) [cacheName=xx, part=273, key=16479796986] > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTopologyFutureAdapter.validatePartitionOperation(GridDhtTopologyFutureAdapter.java:161) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTopologyFutureAdapter.validateCache(GridDhtTopologyFutureAdapter.java:116) > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:417) > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:248) > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$26.apply(GridDhtAtomicCache.java:1146) > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$26.apply(GridDhtAtomicCache.java:1144) > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.asyncOp(GridDhtAtomicCache.java:761) > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update0(GridDhtAtomicCache.java:1144) > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.putAsync0(GridDhtAtomicCache.java:641) > at > org.apache.ignite.internal.processors.cache.GridCacheAdapter.putAsync(GridCacheAdapter.java:2828) > at > org.apache.ignite.internal.processors.cache.GridCacheAdapter.putAsync(GridCacheAdapter.java:2809) > at > org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.putAsync0(IgniteCacheProxyImpl.java:1125) > at > org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.putAsync(IgniteCacheProxyImpl.java:1114) > at > org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.putAsync(GatewayProtectedCacheProxy.java:832) > at > com.xx.common.vertx.ext.data.impl.IgniteCacheRepository.lambda$put$0(IgniteCacheRepository.java:27) > at > com.xx.common.vertx.ext.data.impl.IgniteCacheRepository.executeAsync(IgniteCacheRepository.java:55) > at > com.xx.common.vertx.ext.data.impl.IgniteCacheRepository.put(IgniteCacheRepository.ja
Re: Ignite client node hangs while IgniteAtomicLong is created
Hello!
Can you collect thread dumps from all nodes once you get them hanging?
Can you throw together a reproducer project?
Regards,
--
Ilya Kasnacheev
вт, 4 авг. 2020 г. в 12:51, Ilya Roublev :
> We are developing Jira cloud app using Apache Ignite both as data storage
> and as job scheduler. This is done via a standard Ignite client node. But
> we need to use Atlassian Connect Spring Boot to be able to communicate with
> Jira. In short, all is done exactly as in our article Boosting Jira Cloud
> app development with Apache Ignite
> <https://medium.com/alliedium/boosting-jira-cloud-app-development-with-apache-ignite-7eebc7bb3d48>.
> At first we used simple Ignite JDBC driver
> <https://apacheignite-sql.readme.io/docs/jdbc-driver> just for Atlassian
> Connect Spring Boot along with a separate Ignite client node for our own
> purposes. But this turned out to be very unstable being deployed in our
> local Kubernetes cluster (built via Kubespray) due to constant exceptions
>
> java.net.SocketException: Connection reset
>
> occuring from time to time (in fact, this revealed only in our local
> cluster, in AWS EKS all worked fine). To make all this more stable we tried
> to use Ignite JDBC Client driver
> <https://apacheignite-sql.readme.io/docs/jdbc-client-driver> exactly as
> described in the article mentioned above. Thus, now our backend uses two
> Ignite client nodes per single JVM: the first one for JDBC used by
> Atlassian Connect Spring Boot, the second one for our own purposes. This
> solution turned out to be good enough, because our app works now very
> stable both in our local cluster and in AWS EKS. But when we deploy our app
> in Docker for testing and developing purposes, our Ignite client nodes hang
> from time to time. After some investigation we were able to see that this
> occurs exactly at the instant when an object of IgniteAtomicLong is
> created. Below are logs both for successful initialization of our app and
> for the case when Ignite client node hanged. Logs when all is ok
> ignite-appclientnode-successful.log
> <http://apache-ignite-users.70518.x6.nabble.com/file/t2262/ignite-appclientnode-successful.log>
> ignite-jdbcclientnode-successful.log
> <http://apache-ignite-users.70518.x6.nabble.com/file/t2262/ignite-jdbcclientnode-successful.log>
> Logs
> when both client node hang ignite-appclientnode-failed.log
> <http://apache-ignite-users.70518.x6.nabble.com/file/t2262/ignite-appclientnode-failed.log>
> ignite-jdbcclientnode-failed.log
> <http://apache-ignite-users.70518.x6.nabble.com/file/t2262/ignite-jdbcclientnode-failed.log>
> Some
> analysis and questions From logs one can see that caches default,
> tenants, atlassian_host_audit, SQL_PUBLIC_ATLASSIAN_HOST are manipulated,
> in fact, default is given in client configuration: client.xml
> <http://apache-ignite-users.70518.x6.nabble.com/file/t2262/client.xml>,
> the cache SQL_PUBLIC_ATLASSIAN_HOST contains atlassian_host table mentioned
> in Boosting Jira Cloud app development with Apache Ignite
> <https://medium.com/alliedium/boosting-jira-cloud-app-development-with-apache-ignite-7eebc7bb3d48>
> and is created in advance even before the app starts. Further,
> atlassian_host_audit is a copy of atlassian_host, in any case it is not yet
> created when the app hangs. What concerns other entities processed by
> Ignite, they are created by the following code:
>
> CacheConfiguration tenantCacheCfg = new
> CacheConfiguration<>();
> tenantCacheCfg.setName("tenants");
> tenantCacheCfg.setSqlSchema("PROD");
> tenantCacheCfg.setIndexedTypes(Long.class, Tenant.class);
> tenantCacheCfg.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL);
> tenantCacheCfg.setEncryptionEnabled(true);
>
> IgniteCache tenantCache =
> ignite.getOrCreateCache(tenantCacheCfg);
> IgniteAtomicLong idGen = ignite.atomicLong("PROD_tenants_seq", 0,
> true);
>
> And from the logs of the app itself it is clear that the app hangs exactly
> on the last line. This is confirmed by the fact that the in
> ignite-jdbcclientnode-successful.log we have the following lines:
>
> [2020-07-31 09:52:12,237][INFO ][exchange-worker-#43][GridCacheProcessor]
> Started cache [name=ignite-sys-atomic-cache@default-ds-group, id=1481046058,
> group=default-ds-group, dataRegionName=null, mode=PARTITIONED,
> atomicity=TRANSACTIONAL, backups=1, mvcc=false]
> [2020-07-31 09:52:12,263][INFO ][exchange-worker-#43][time] Finished exchange
> init [topVer=AffinityTopologyVersion [topVer=6, minorTopVer=2], crd=false]
> [2020-07-31 09:52:12,631][INFO
> ][sys-#109%ignite-jdbc-driver-e1f56
Re: 2.8.1 - Loading Plugin Provider - Conflicting documentation
Hello! You do this with IgniteConfiguration.setPluginConfiguration(). Regards, -- Ilya Kasnacheev вт, 28 июл. 2020 г. в 13:04, VeenaMithare : > Okay. > > >> The recommended way is to use the service provider as detailed here: > > How do we pass configuration for the provider in this approach. > > regards, > Veena. > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: [External]Re: Node left from running cluster due to org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: Runtime failure on search row
Hello! You can't do that in runtime, but you can certainly remove index.bin as bulk operation. Regards, -- Ilya Kasnacheev вт, 28 июл. 2020 г. в 08:48, Kamlesh Joshi : > Thanks for the update Andrei. We upgraded from 2.6.0 to 2.7.6. > > We followed the same approach and it worked properly. > > Can we change INLINE SIZE for every cache at runtime ? > > Thanks and Regards, > Kamlesh Joshi > > > -Original Message- > From: aealexsandrov > Sent: 27 July 2020 19:35 > To: user@ignite.apache.org > Subject: [External]Re: Node left from running cluster due to > org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: > Runtime failure on search row > > The e-mail below is from an external source. Please do not open > attachments or click links from an unknown or suspicious origin. > > Hi, > > Can you please clarify the version from which you upgraded? > > I know that it's possible that if you used a pretty old version then you > should also rebuild your indexes during the upgrade because inline size > calculation logic was changed in last releases. > > It can be done by removing index.bin files. So the logic should be the > following: > > 1)Stop the Ignite nodes. > 2)Remove index.bin files from > $IGNITE_HOME/db///index.bin > 3)Start Ignite nodes one by one. For each node wait for the following > message: > > [2020-05-29 07:55:04,984][INFO > ][build-idx-runner-#61][GridCacheDatabaseSharedManager] Indexes rebuilding > completed for all caches. > > I guess that it can help you with your upgrade. Please test it before > applying it to your production because the reason can be different. > > BR, > Andrei > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > > "Confidentiality Warning: This message and any attachments are intended > only for the use of the intended recipient(s). > are confidential and may be privileged. If you are not the intended > recipient. you are hereby notified that any > review. re-transmission. conversion to hard copy. copying. circulation or > other use of this message and any attachments is > strictly prohibited. If you are not the intended recipient. please notify > the sender immediately by return email. > and delete this message and any attachments from your system. > > Virus Warning: Although the company has taken reasonable precautions to > ensure no viruses are present in this email. > The company cannot accept responsibility for any loss or damage arising > from the use of this email or attachment." > >
Re: Enabling swapPath causes invoking shutdown hook
Hello! >From the docs: To avoid this situation with the swapping capabilities, you need to : - Set maxSize = bigger_ than_RAM_size, in which case, the OS will take care of the swapping. - Enable swapping by setting the DataRegionConfiguration.swapPath property. I actually think these are either-or. You should either do the first (and configure OS swapping) or the second part. Having said that, I recommend setting proper Native Persistence instead. Regards, -- Ilya Kasnacheev сб, 25 июл. 2020 г. в 04:49, 38797715 <38797...@qq.com>: > Hi, > > https://apacheignite.readme.io/docs/swap-space > > According to the above document, if the physical memory is small, you can > solve this problem by opening the swap space,The specific method is to > configure maxSize to a larger value (i.e. larger than the physical memory), > and the swapPath property needs to be configured. > > But from the test results, the node is terminated. > > I think the correct result should be that even if the amount of data > exceeds the physical memory, the node should still be able to run normally, > but the data is exchanged to the disk. > > I want to know what parameters affect the behavior of this configuration? > *vm.swappiness* or others? > 在 2020/7/24 下午9:55, aealexsandrov 写道: > > Hi, > > Can you please clarify your expectations? You expected that JVM process will > be killed instead of gracefully stopping? What you are going to achieve? > > BR, > Andrei > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > >
Re: Block until partition map exchange is complete
Hello! It is supposed to be fixed in 2.8. Did you check that? Thanks. -- Ilya Kasnacheev ср, 22 июл. 2020 г. в 12:24, ssansoy : > Hi, could the behaviour I have observed be captured by this bug: > > https://issues.apache.org/jira/browse/IGNITE-9841 > > "Note, ScanQuery exhibits the same behavior - returns partial results when > some partitions are lost. Not sure if solution would be related or needs > to > be tracked and fixed under a separate ticket." > > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: apache-ignite compatibility with armhf(32-bit arm linux)
Hello! Our DEB packages are published with "all" architecture, which means they are not bound to any specific arch. Regards, -- Ilya Kasnacheev пт, 24 июл. 2020 г. в 17:31, Stanislav Lukyanov : > It should work fine. I ran Ignite on various platforms non-Intel platforms, > including arm. There were some issues in the past but modern versions work > well. But you do have to keep in mind that release testing for Ignite is > done on Intel platforms. Also arm can bring some surprises in terms of > performance. > > I haven't tried the debian package myself. On one hand, it should work - I > believe there is no platform-dependent code in the package. On the other > hand, I believe debian packages are bound to CPU architecture, aren't they? > So it probably won't allow you to install it. > > Stan > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: apache ignite compatibility with armhf(arm 32-bit)
Hello! Try it and see :) I don't see why it won't work. Maybe there will be some endianness issues, but you never know in advance. Regards, -- Ilya Kasnacheev чт, 23 июл. 2020 г. в 20:54, rakshita04 : > I need to use apache ignite for my Debian linux 32-bit version. > Is it compatible with armhf(32-bit arm linux)? > can i use apache-ignite debian package for the same? > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Remote Filter Execution
Hello! There's nothing we can do about 2.7.x right now, I suggest upgrading to 2.8.1. Regards, -- Ilya Kasnacheev пт, 17 июл. 2020 г. в 10:06, VeenaMithare : > I can also confirm that both these issues happen consistently on 2.7.6. All > the nodes are in baseline when both these issues happen. > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Block until partition map exchange is complete
Hello! I'm not actually sure. Do you have a reproducer where you see decreased count() result? What is PartitionLossPolicy, have you tried tweaking it? I can see a method in our tests for doing that, and it is very raw: it checks every cache to make sure that all partitions are OWNING. This is org.apache.ignite.testframework.junits.common.GridCommonAbstractTest#awaitPartitionMapExchange(boolean, boolean, java.util.Collection, boolean, java.util.Set) Regards, -- Ilya Kasnacheev ср, 15 июл. 2020 г. в 12:03, ssansoy : > By the way, just referring back to the original question - is there such a > callback that can be used to wait for the partition exchange to complete, > in > any version of ignite? We are using ignite 2.7.6 (which I acknowledge is > slightly behind - but we planning to upgrade) > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: How to get local node near cache metrics?
Hello! The only thing I can think of is cache.metrics().getHeapEntriesCount() Have you tried that? Regards, -- Ilya Kasnacheev вт, 14 июл. 2020 г. в 05:12, John Smith : > Hi I want to get how many entries are on the thick client's near cache. Is > there a way? >
Re: question about node segment and split brain
Hello! 1. if gc time > failureDetectionTimeout. 2. No, NODE_SEGMENTED is reserved for case where node is excluded from cluster, alone, and not able to function as a single-node cluster (explicitly removed from cluster). 3. I've never seen splitting into two clusters. 4. Perhaps, but see above. Regards, -- Ilya Kasnacheev чт, 2 июл. 2020 г. в 13:45, bbweb : > Hi, > > we are considering ignite for mission critical system and concerning > about node segmentation and split brain problem. I searched for current > documents and suggestions but still have some questions in the following, > please help and thanks in advance: > > > > 1、For long gc case , when a node has long gc and after gc is done, what's > the heartbeat check and rejoin sequence for this node and when will this > node be judged as segmented? > > > > 2、It seems sometimes EVT_NODE_SEGMENTED is reported from segmented node. > When will EVT_NODE_SEGMENTED event be fired and what's the further > action? SegmentationPolicy defines what happens when segmentation occurs. > The default value is STOP。Does this mean that even no segmentation plugin > is defined, behavior can still be controlled(like stop or restart) using > SegmentationPolicy for EVT_NODE_SEGMENTED ? > > > 3、If there is network issue like switch outage which cause one cluster be > splitted to 2 clusters, will node segmentation be reported and > EVT_NODE_SEGMENTED event be fired? Does this related to node number in the > cluster, e.g if this is only one node isolated then this event is possible > to be fired and if the splitted cluster has at least 2 nodes then no event > will be fired and this two splitted cluster can both provide service which > cause split brain? In out test, actually short time network outage can > generate even one node cluster and this node can still provide service, no > segmentation event is reported. > > > 4、Can ZooKeeper resolve all this kind of issues including long GC or > short-time network outage? > > > 5、As ZooKeeper is for big cluster, for small cluster can plugin resolve > all cases? I looked at some implentation including gridgain's paid verion, > different plugins checks for node access, port access and shared first > sytem but is seems it's difficult for some cases like GC as in this case > the host is still reacheable but ignite node is not reachable for > sometime, so what's the suggestions for plugin that can resolve all cases? > > Thanks! > > > > > > <https://a.app.qq.com/o/simple.jsp?pkgname=com.sohu.mail.client.cordova> > > > >
Re: Remote Filter Execution
Hello! I have just checked this behavior on 2.8.1, and I don't see any of these issues popping up. All 3 server nodes run remote listener (judging by log), and after restarting of nodes I can still see updates on 'id1' appearing in client npde's log. I didn't check it on 2.7.6. Can you confirm or deny that it works on 2.8.1 for you? Are you sure that all nodes are in the baseline? Regards, -- Ilya Kasnacheev ср, 15 июл. 2020 г. в 13:55, VeenaMithare : > Hi Ilya, > > Please find the reproducer project with a readme.txt. > > I have put in comments on how to reproduce both the issues mentioned in my > original mail. > > regards, > Veena. RemoteFilterIssueProject.zip > < > http://apache-ignite-users.70518.x6.nabble.com/file/t2757/RemoteFilterIssueProject.zip> > > Readme.txt > <http://apache-ignite-users.70518.x6.nabble.com/file/t2757/Readme.txt> > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: First 10 long running cache futures [total=1]
Hello! The printed cache future contains some of this information, but it's hard to decipher. In this case, you are waiting for a node 1e36906c-fae5-49a3-b67a-9eed6ada7e4e. It will make sense to check its thread dump. Regards, -- Ilya Kasnacheev вт, 14 июл. 2020 г. в 05:59, marble.zh...@coinflex.com < marble.zh...@coinflex.com>: > thanks, are there any hints that can find/locate the operations that impact > the server? > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: DataRegion sysMemPlc, TxLog
Hello! sysMemPlc is used to hold ignite-sys-cache and TxLog to hold SQL MVCC information. You may skip monitoring these. Regards, -- Ilya Kasnacheev ср, 15 июл. 2020 г. в 09:02, kay : > Hello > > We found sysMempPlc and Txlog DataRegion. What is each role? > Do I have to monitoring thoes regions?? > > Thank you so much > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: First 10 long running cache futures [total=1]
Hello! This means a cache operation takes longer to complete than usual, mostly due to locking, node network unavailability, checkpoint, etc... Regards, -- Ilya Kasnacheev пн, 13 июл. 2020 г. в 09:20, marble.zh...@coinflex.com < marble.zh...@coinflex.com>: > Hi Guru, > > we met an exception on below, it impacts system no responses, any > suggestions how to identify the case? > > 12:18:10,035 [grid-timeout-worker-#39] WARN > org.apache.ignite.internal.diagnostic - First 10 long running cache > futures > [total=1] > 12:18:10,035 [grid-timeout-worker-#39] WARN > org.apache.ignite.internal.diagnostic - >>> Future > [startTime=00:16:20.109, > curTime=00:18:10.030, fut=GridNearAtomicSingleUpdateFuture > [reqState=Primary > [id=06dc8f4f-b33a-45c1-8c0d-c6a62527fae1, opRes=true, expCnt=4, rcvdCnt=3, > primaryRes=false, done=false, > waitFor=[1e36906c-fae5-49a3-b67a-9eed6ada7e4e], > rcvd=[de1a8ef5-f0ca-41b1-87f5-dfe9f4517fab, > 48cc4679-c256-4666-a39d-76fbd915ac02, > f0f38fba-4963-42dd-a64e-774057a61044]], > super=GridNearAtomicAbstractUpdateFuture [remapCnt=100, > topVer=AffinityTopologyVersion [topVer=7, minorTopVer=0], remapTopVer=null, > err=null, futId=1, super=GridFutureAdapter [ignoreInterrupts=false, > state=INIT, res=null, hash=316835640 > > thanks a lot. > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: [External]Re: Ignite cluster became unresponsive
Hello! I recommend setting it somewhat lower, but longer than any of your expected GC pauses. 30s is OK. Regards, -- Ilya Kasnacheev вс, 12 июл. 2020 г. в 14:03, Kamlesh Joshi : > Thanks for the findings Ilya. > > > > So shall we set the same timeout value for *socketWriteTimeout* as that > of failure detection timeout on both client and server side? > > > > > > *Thanks and Regards,* > > *Kamlesh Joshi* > > > > *From:* Ilya Kasnacheev > *Sent:* 10 July 2020 19:48 > *To:* user@ignite.apache.org > *Subject:* Re: [External]Re: Ignite cluster became unresponsive > > > > The e-mail below is from an external source. Please do not open > attachments or click links from an unknown or suspicious origin. > > Hello! > > > > It seems that communication connections were closed after CG pause, then > you have got half-open connections. It is recommended to keep > socketWriteTimeout and failure detection timeout in relative sync. > > > > Default socketWriteTimeout on TcpConnectionSpi is very low while your > failure detection timeout is rather high, leading to such issue. > > > > It is also possible that client nodes can connect to a server node but not > vice versa, leading to failure of opening connections once they are closed: > > > > Thread [name="sys-stripe-12-#13%EDIFCustomerCC%", id=45, state=RUNNABLE, > blockCnt=851, waitCnt=27526057] > at sun.nio.ch.Net.poll(Native Method) > at sun.nio.ch.SocketChannelImpl.poll(SocketChannelImpl.java:954) > at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:110) > at > o.a.i.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3299) > at > o.a.i.spi.communication.tcp.TcpCommunicationSpi.createNioClient(TcpCommunicationSpi.java:2987) > at > o.a.i.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2870) > at > o.a.i.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2713) > at > o.a.i.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:2672) > > > > Regards, > > -- > > Ilya Kasnacheev > > > > > > пт, 10 июл. 2020 г. в 16:32, Kamlesh Joshi : > > Hi Ilya, > > > > PFA the entire node logs, which contains thread dump as well. Let us know > if any findings. > > > > *Thanks and Regards,* > > *Kamlesh Joshi* > > > > *From:* Ilya Kasnacheev > *Sent:* 10 July 2020 17:51 > *To:* user@ignite.apache.org > *Subject:* Re: [External]Re: Ignite cluster became unresponsive > > > > The e-mail below is from an external source. Please do not open > attachments or click links from an unknown or suspicious origin. > > Hello! > > > > Can you provide full thread dump (jstack) after you see these messages? > > > > Regards, > > -- > > Ilya Kasnacheev > > > > > > ср, 8 июл. 2020 г. в 15:57, Kamlesh Joshi : > > Hi Stephen/Team, > > > > Did you got any chance to look into this? > > > > *Thanks and Regards,* > > *Kamlesh Joshi* > > > > *From:* Kamlesh Joshi > *Sent:* 06 July 2020 14:50 > *To:* user@ignite.apache.org > *Subject:* RE: [External]Re: Ignite cluster became unresponsive > > > > Hi Stephen, > > > > We have started our node with below JVM parameters. Also, we have > increased these timeouts *failureDetectionTimeout*/ > *clientFailureDetectionTimeout*/*networkTimeout to 48*. > > > > *-XX:+AggressiveOpts -XX:+AlwaysPreTouch -XX:+UseG1GC > -XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC > -XX:+UnlockCommercialFeatures -Djava.net.preferIPv4Stack=true > -DIGNITE_LONG_OPERATIONS_DUMP_TIMEOUT=60 > -DIGNITE_THREAD_DUMP_ON_EXCHANGE_TIMEOUT=true -Dfile.encoding=UTF-8 > -DIGNITE_QUIET=false* > > > > Is there anything else that we have to tune ? > > > > And I think JVM pause is introduced as a result of the error that we > encountered right? Correct me if am wrong. > > > > *Thanks and Regards,* > > *Kamlesh Joshi* > > > > *From:* Stephen Darlington > *Sent:* 06 July 2020 14:09 > *To:* user > *Subject:* [External]Re: Ignite cluster became unresponsive > > > > The e-mail below is from an external source. Please do not open > attachments or click links from an unknown or suspicious origin. > > There are a few issues here — the blocked thread, the communication error > — but I possibly the key one is the JVM pause: > > > > *[2020-07-03T18:17:21,793][WARN > ][jvm-pause-detector-worker][IgniteKernal%CustomerCC] Possible t
Re: Remote Filter Execution
Hello! Maybe there's something with your filter implementation which prevents it from functioning. Can you throw together a small reproducer project to show this behavior? Regards, -- Ilya Kasnacheev пт, 10 июл. 2020 г. в 19:35, VeenaMithare : > Hi Ilya, > > Yes, So filter gets created but not executed on more than one node. > 'Entered > Remote Filter' happens only on one node. 'Filter created' log is printed in > the 'create()' method of the Factory. > 'Entered Remote Filter' is printed in the evaluate method of the > CacheEntryEventFilter. > > That is what I was saying in my earlier post - > > You will notice that this log is only on server 1 - "projectname LISTENS: > Entered Remote Filter ." > > You can also see that the filter has been created on all the three nodes. ( > LOG : projectname LISTENS: Filter created ) > > regards, > Veena. > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: DataStreamer. allowOverwrite(false) - Will it slow the writes?
Hello! With persistence, you will probably burn through the burst time (in-memory no-checkpoint) faster and hit the "slow" persistent epoch earlier. Then you will need to tune your persistence. Please see https://apacheignite.readme.io/docs/durable-memory-tuning Regards, -- Ilya Kasnacheev пт, 10 июл. 2020 г. в 19:07, krkumar24061...@gmail.com < krkumar24061...@gmail.com>: > Hi - Yes, Ignite native persistence is enabled. > > Thanx and Regards, > KR Kumar > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Using SORT BY and ORDER BY
Hello! By putting your affinity column in the public key of all tables and issuing WITH "affinity_key=" clause when creating table. Please also see https://apacheignite.readme.io/docs/affinity-collocation Regards, -- Ilya Kasnacheev пт, 10 июл. 2020 г. в 18:32, Surkov.Aleksandr : > There is the question. It turns out that I need to collocate the table, > because the field COL will have different values? > But how can i do this? > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Using SORT BY and ORDER BY
Hello!
1. I guess it is a honest mistake.
2. The idea here is that you can't expect that GROUP BY COL will return
anything relevant, if the table is not collocated by COL.
Regards,
--
Ilya Kasnacheev
пт, 10 июл. 2020 г. в 17:34, Surkov.Aleksandr :
> The org.apache.ignite.internal.processors.cache.query.CacheQuery interface
> has a comment:
>
> * {@code Group by} and {@code sort by} statements are applied
> separately
> * on each node, so result set will likely be incorrectly grouped
> or
> sorted
> * after results from multiple remote nodes are grouped together.
>
> As far as I understand:
> 1. {@code sort by} does not supported
> 2. ORDER BY returns a sorted list even if items are on different nodes
>
> It is right?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Remote Filter Execution
Hello!
I see the following lines on all 3 nodes:
2020-07-09 16:41:02,957 [disco-notifier-worker-#101] DEBUG
com.COMPANYNAME.prophet.configstore.common.remote.ConfigStoreTableRemoteFilterFactory
[] - projectname LISTENS: Filter creation started with
andPredicateMap:{NAME=(EQUAL,CENTRE)},orPredicateMap:null,
Remotefilterfactory:com.COMPANYNAME.prophet.configstore.common.remote.ConfigStoreTableRemoteFilterFactory@14bf552f,
Service:client Instance:null, HostName:null
2020-07-09 16:41:02,958 [disco-notifier-worker-#101] DEBUG
com.COMPANYNAME.prophet.configstore.common.remote.ConfigStoreTableRemoteFilterFactory
[] - projectname LISTENS: Filter created with
andPredicateMap{NAME=(EQUAL,CENTRE)},orPredicateMapnull,
Remotefilterfactory:com.COMPANYNAME.prophet.configstore.common.remote.ConfigStoreTableRemoteFilterFactory@14bf552f,
Is it relevant?
Regards,
--
Ilya Kasnacheev
пт, 10 июл. 2020 г. в 16:02, VeenaMithare :
> HI Ilya,
>
> Please find the attached logs.
>
> You will notice that this log is only on server 1 - "projectname LISTENS:
> Entered Remote Filter ."
>
> You can also see that the filter has been created on all the three nodes. (
> LOG : projectname LISTENS: Filter created )
>
> server1RemoteFilterExecuted.txt
> <
> http://apache-ignite-users.70518.x6.nabble.com/file/t2757/server1RemoteFilterExecuted.txt>
>
> server2.txt
> <http://apache-ignite-users.70518.x6.nabble.com/file/t2757/server2.txt>
> server3.txt
> <http://apache-ignite-users.70518.x6.nabble.com/file/t2757/server3.txt>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: [External]Re: Ignite cluster became unresponsive
Hello! It seems that communication connections were closed after CG pause, then you have got half-open connections. It is recommended to keep socketWriteTimeout and failure detection timeout in relative sync. Default socketWriteTimeout on TcpConnectionSpi is very low while your failure detection timeout is rather high, leading to such issue. It is also possible that client nodes can connect to a server node but not vice versa, leading to failure of opening connections once they are closed: Thread [name="sys-stripe-12-#13%EDIFCustomerCC%", id=45, state=RUNNABLE, blockCnt=851, waitCnt=27526057] at sun.nio.ch.Net.poll(Native Method) at sun.nio.ch.SocketChannelImpl.poll(SocketChannelImpl.java:954) at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:110) at o.a.i.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3299) at o.a.i.spi.communication.tcp.TcpCommunicationSpi.createNioClient(TcpCommunicationSpi.java:2987) at o.a.i.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2870) at o.a.i.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2713) at o.a.i.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:2672) Regards, -- Ilya Kasnacheev пт, 10 июл. 2020 г. в 16:32, Kamlesh Joshi : > Hi Ilya, > > > > PFA the entire node logs, which contains thread dump as well. Let us know > if any findings. > > > > *Thanks and Regards,* > > *Kamlesh Joshi* > > > > *From:* Ilya Kasnacheev > *Sent:* 10 July 2020 17:51 > *To:* user@ignite.apache.org > *Subject:* Re: [External]Re: Ignite cluster became unresponsive > > > > The e-mail below is from an external source. Please do not open > attachments or click links from an unknown or suspicious origin. > > Hello! > > > > Can you provide full thread dump (jstack) after you see these messages? > > > > Regards, > > -- > > Ilya Kasnacheev > > > > > > ср, 8 июл. 2020 г. в 15:57, Kamlesh Joshi : > > Hi Stephen/Team, > > > > Did you got any chance to look into this? > > > > *Thanks and Regards,* > > *Kamlesh Joshi* > > > > *From:* Kamlesh Joshi > *Sent:* 06 July 2020 14:50 > *To:* user@ignite.apache.org > *Subject:* RE: [External]Re: Ignite cluster became unresponsive > > > > Hi Stephen, > > > > We have started our node with below JVM parameters. Also, we have > increased these timeouts *failureDetectionTimeout*/ > *clientFailureDetectionTimeout*/*networkTimeout to 48*. > > > > *-XX:+AggressiveOpts -XX:+AlwaysPreTouch -XX:+UseG1GC > -XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC > -XX:+UnlockCommercialFeatures -Djava.net.preferIPv4Stack=true > -DIGNITE_LONG_OPERATIONS_DUMP_TIMEOUT=60 > -DIGNITE_THREAD_DUMP_ON_EXCHANGE_TIMEOUT=true -Dfile.encoding=UTF-8 > -DIGNITE_QUIET=false* > > > > Is there anything else that we have to tune ? > > > > And I think JVM pause is introduced as a result of the error that we > encountered right? Correct me if am wrong. > > > > *Thanks and Regards,* > > *Kamlesh Joshi* > > > > *From:* Stephen Darlington > *Sent:* 06 July 2020 14:09 > *To:* user > *Subject:* [External]Re: Ignite cluster became unresponsive > > > > The e-mail below is from an external source. Please do not open > attachments or click links from an unknown or suspicious origin. > > There are a few issues here — the blocked thread, the communication error > — but I possibly the key one is the JVM pause: > > > > *[2020-07-03T18:17:21,793][WARN > ][jvm-pause-detector-worker][IgniteKernal%CustomerCC] Possible too long JVM > pause: 10133 milliseconds.* > > > > This is usually due to garbage collection, but there are a number of other > possibilities such as slow I/O. Suggest you start with the recommendations > on the GC tuning documentation page: > https://apacheignite.readme.io/docs/jvm-and-system-tuning > > > > Regards, > > Stephen > > > > On 4 Jul 2020, at 12:44, Kamlesh Joshi wrote: > > > > Hi Team, > > > > We have encountered following defect in PROD environment. After which > entire traffic got halted for around 10 minutes, we recently upgraded our > cluster to Ignite 2.7.6 from 2.6.0. > > Is this related to any existing open defect in this version? Has anyone > observed the same defect earlier ? > > > > Any help or pointers around this will be appreciated. > > > > > > *[2020-07-03T18:17:11,613][ERROR][sys-stripe-36-#37%CustomerCC%][G] > Blocked system-critical thread has been detected. This can le
Re: Ignite Backup and Restore
Hello! I guess that Apache Ignite 2.9 will have snapshotting capabilities. Traditionally we rely more on backup nodes/partitions than on off-line backups. Regards, -- Ilya Kasnacheev ср, 8 июл. 2020 г. в 12:23, marble.zh...@coinflex.com < marble.zh...@coinflex.com>: > Hi, any suggestions? > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: ignite 2.8.1: Failed to resolve node topology
Hello! It seems that the node fell behind in switching to new topology. Can you provide full log of this node? I guess that something was blocking PME. Regards, -- Ilya Kasnacheev чт, 9 июл. 2020 г. в 09:44, Mahesh Renduchintala < mahesh.renduchint...@aline-consulting.com>: > Hi, > > We have a crash in our environment with the below error. > Any insight into what might have gone wrong? > > regards > Mahesh > > > ^-- Heap [used=17211MB, free=64.98%, comm=49152MB] > ^-- Off-heap [used=48448MB, free=26.41%, comm=65736MB] > ^-- sysMemPlc region [used=0MB, free=99.98%, comm=100MB] > ^-- default region [used=48447MB, free=26.07%, comm=65536MB] > ^-- metastoreMemPlc region [used=0MB, free=99.21%, comm=0MB] > ^-- TxLog region [used=0MB, free=100%, comm=100MB] > ^-- Ignite persistence [used=48647MB] > ^-- sysMemPlc region [used=0MB] > ^-- default region [used=48646MB] > ^-- metastoreMemPlc region [used=0MB] > ^-- TxLog region [used=0MB] > ^-- Outbound messages queue [size=0] > ^-- Public thread pool [active=0, idle=4, qSize=0] > ^-- System thread pool [active=11, idle=12, qSize=0] > [18:44:06,313][INFO][exchange-worker-#70][GridDhtPartitionsExchangeFuture] > Finish exchange future [startVer=AffinityTopologyVersion [topVer=83, > minorTopVer=0], resVer=AffinityTopologyVersion [topVer=83, minorTopVer=0], > err=null, rebalanced=true, wasRebalanced=true] > [18:44:06,316][WARNING][sys-stripe-21-#22][finish] Received finish request > for completed transaction (the message may be too late) > [txId=GridCacheVersion [topVer=205532475, order=1594223109470, > nodeOrder=51], dhtTxId=null, node=35c37b55-ec30-4457-b861-403bcfc20c12, > commit=false] > [18:44:06,317][WARNING][sys-stripe-18-#19][finish] Received finish request > for completed transaction (the message may be too late) > [txId=GridCacheVersion [topVer=205532475, order=1594223109273, > nodeOrder=51], dhtTxId=null, node=35c37b55-ec30-4457-b861-403bcfc20c12, > commit=false] > [18:44:06,317][WARNING][sys-stripe-23-#24][finish] Received finish request > for completed transaction (the message may be too late) > [txId=GridCacheVersion [topVer=205532475, order=1594223109436, > nodeOrder=51], dhtTxId=null, node=35c37b55-ec30-4457-b861-403bcfc20c12, > commit=false] > [18:44:06,321][WARNING][sys-stripe-21-#22][finish] Received finish request > for completed transaction (the message may be too late) > [txId=GridCacheVersion [topVer=205532475, order=1594223109502, > nodeOrder=51], dhtTxId=null, node=35c37b55-ec30-4457-b861-403bcfc20c12, > commit=false] > [18:44:06,327][INFO][exchange-worker-#70][GridDhtPartitionsExchangeFuture] > Completed partition exchange > [localNode=d38d1293-9dd5-4b4e-9934-97ba3fbafa62, > exchange=GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion > [topVer=83, minorTopVer=0], evt=NODE_FAILED, evtNode=TcpDiscoveryNode > [id=c19d4735-2b52-487f-9d6d-0574b8f15858, > consistentId=L1APISERVICE_7aa60a92371, addrs=ArrayList [10.244.0.93, > 127.0.0.1], sockAddrs=HashSet [/10.244.0.93:0, /127.0.0.1:0], discPort=0, > order=68, intOrder=39, lastExchangeTime=1594131247817, loc=false, > ver=2.8.1#20200521-sha1:86422096, isClient=true], done=true, > newCrdFut=null], topVer=AffinityTopologyVersion [topVer=83, minorTopVer=0]] > [18:44:06,327][INFO][exchange-worker-#70][GridDhtPartitionsExchangeFuture] > Exchange timings [startVer=AffinityTopologyVersion [topVer=83, > minorTopVer=0], resVer=AffinityTopologyVersion [topVer=83, minorTopVer=0], > stage="Waiting in exchange queue" (0 ms), stage="Exchange parameters > initialization" (0 ms), stage="Determine exchange type" (16 ms), > stage="Exchange done" (71053 ms), stage="Total time" (71069 ms)] > [18:44:06,327][INFO][exchange-worker-#70][GridDhtPartitionsExchangeFuture] > Exchange longest local stages [startVer=AffinityTopologyVersion [topVer=83, > minorTopVer=0], resVer=AffinityTopologyVersion [topVer=83, minorTopVer=0]] > [18:44:06,327][INFO][exchange-worker-#70][time] Finished exchange init > [topVer=AffinityTopologyVersion [topVer=83, minorTopVer=0], crd=false] > [18:44:06,345][INFO][db-checkpoint-thread-#112][GridCacheDatabaseSharedManager] > Checkpoint started [checkpointId=92718d25-2db4-4691-886c-ec26b8b6ecba, > startPtr=FileWALPointer [idx=389, fileOff=2151853, len=17162371], > checkpointBeforeLockTime=869ms, checkpointLockWait=86163ms, > checkpointListenersExecuteTime=784ms, checkpointLockHoldTime=859ms, > walCpRecordFsyncDuration=20ms, writeCheckpointEntryDuration=1ms, > splitAndSortCpPagesDuration=24ms, pages=28028, reason='timeout'] > [18:44:06,388][SEVERE][sys-stripe-3-#4][] Critical system error detected. > Will be handled accordingl
Re: [External]Re: Ignite cluster became unresponsive
Hello! Can you provide full thread dump (jstack) after you see these messages? Regards, -- Ilya Kasnacheev ср, 8 июл. 2020 г. в 15:57, Kamlesh Joshi : > Hi Stephen/Team, > > > > Did you got any chance to look into this? > > > > *Thanks and Regards,* > > *Kamlesh Joshi* > > > > *From:* Kamlesh Joshi > *Sent:* 06 July 2020 14:50 > *To:* user@ignite.apache.org > *Subject:* RE: [External]Re: Ignite cluster became unresponsive > > > > Hi Stephen, > > > > We have started our node with below JVM parameters. Also, we have > increased these timeouts *failureDetectionTimeout*/ > *clientFailureDetectionTimeout*/*networkTimeout to 48*. > > > > *-XX:+AggressiveOpts -XX:+AlwaysPreTouch -XX:+UseG1GC > -XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC > -XX:+UnlockCommercialFeatures -Djava.net.preferIPv4Stack=true > -DIGNITE_LONG_OPERATIONS_DUMP_TIMEOUT=60 > -DIGNITE_THREAD_DUMP_ON_EXCHANGE_TIMEOUT=true -Dfile.encoding=UTF-8 > -DIGNITE_QUIET=false* > > > > Is there anything else that we have to tune ? > > > > And I think JVM pause is introduced as a result of the error that we > encountered right? Correct me if am wrong. > > > > *Thanks and Regards,* > > *Kamlesh Joshi* > > > > *From:* Stephen Darlington > *Sent:* 06 July 2020 14:09 > *To:* user > *Subject:* [External]Re: Ignite cluster became unresponsive > > > > The e-mail below is from an external source. Please do not open > attachments or click links from an unknown or suspicious origin. > > There are a few issues here — the blocked thread, the communication error > — but I possibly the key one is the JVM pause: > > > > *[2020-07-03T18:17:21,793][WARN > ][jvm-pause-detector-worker][IgniteKernal%CustomerCC] Possible too long JVM > pause: 10133 milliseconds.* > > > > This is usually due to garbage collection, but there are a number of other > possibilities such as slow I/O. Suggest you start with the recommendations > on the GC tuning documentation page: > https://apacheignite.readme.io/docs/jvm-and-system-tuning > > > > Regards, > > Stephen > > > > On 4 Jul 2020, at 12:44, Kamlesh Joshi wrote: > > > > Hi Team, > > > > We have encountered following defect in PROD environment. After which > entire traffic got halted for around 10 minutes, we recently upgraded our > cluster to Ignite 2.7.6 from 2.6.0. > > Is this related to any existing open defect in this version? Has anyone > observed the same defect earlier ? > > > > Any help or pointers around this will be appreciated. > > > > > > *[2020-07-03T18:17:11,613][ERROR][sys-stripe-36-#37%CustomerCC%][G] > Blocked system-critical thread has been detected. This can lead to > cluster-wide undefined behaviour* > > *[threadName=partition-exchanger, blockedFor=480s]* > > *[2020-07-03T18:17:11,613][WARN ][sys-stripe-36-#37%CustomerCC%][G] Thread > [name="exchange-worker-#344%CustomerCC%", id=391, state=TIMED_WAITING, > blockCnt=1, waitCnt=2049782]* > > *Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6bf9f3a4, > ownerName=null, ownerId=-1]* > > > > *[2020-07-03T18:17:11,620][ERROR][sys-stripe-36-#37%CustomerCC%][] > Critical system error detected. Will be handled accordingly to configured > handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, > super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED, > SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext > [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker > [name=partition-exchanger, igniteInstanceName=CustomerCC, finished=false, > heartbeatTs=1593780431612]]]* > > *org.apache.ignite.IgniteException: GridWorker [name=partition-exchanger, > igniteInstanceName=CustomerCC, finished=false, heartbeatTs=1593780431612]* > > *at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831) > [ignite-core-2.7.6.jar:2.7.6]* > > *at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826) > [ignite-core-2.7.6.jar:2.7.6]* > > *at > org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233) > [ignite-core-2.7.6.jar:2.7.6]* > > *at > org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297) > [ignite-core-2.7.6.jar:2.7.6]* > > *at > org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:513) > [ignite-core-2.7.6.jar:2.7.6]* > > *at > org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) > [ignite-core-2.7.
Re: Block until partition map exchange is complete
Hello! Can you throw together a reproducer project which shows this behavior? I would check. Regards, -- Ilya Kasnacheev пт, 3 июл. 2020 г. в 13:14, ssansoy : > Thanks - the issue I have now is how can I confirm that the local listen > has > returned before executing my code? > e.g. in the local listen I can set a flag, and then the local listen > returns > - but the thread that detects this flag and runs the task could still be > scheduled to run before the local listen has returned. > Is there a callback I can register which is triggered after the local > listen > returns so I can guarantee I am executing in the correct order (e.g. after > whatever needs to be committed has been committed)? > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Ignite server of perNodeParallelOperatoins ?
Hello! Yes, there is Data Streamer Thread Pool Size on server, which corresponds to perNodeParallelOperations. Server will not run more parallel operations than thread pool size. Regards, -- Ilya Kasnacheev вт, 7 июл. 2020 г. в 23:18, Edward Chen : > Hello, > > In IgniteDataStreamer, there is a config: perNodeParallelOperatoins > (int), it is configured in the client side. in the Server side, does it > have similar configuration ? otherwise, if client has freedom to set > any number of perNodeParallelOperatoins they want, how server prevent > not crash ? > > Thanks. Ed > > >
Re: enum behavior in REST
Hello!
Can you provide an example of REST response? I don't understand how this
issue manifests.
Regards,
--
Ilya Kasnacheev
ср, 8 июл. 2020 г. в 01:25, Maxim Volkomorov <2201...@gmail.com>:
> I have "type":{"platformType":false}" in REST response for my
> enum object property.
>
> Object property:
> public class Organization {
> //...
> private OrganizationType type;
> //...
>public OrganizationType type() {
> return type;
> }
> //...
> }
>
> OrganizationType :
> public enum OrganizationType {
> /** Non-profit organization. */
> NON_PROFIT,
>
> /** Private organization. */
> PRIVATE,
>
> /** Government organization. */
> GOVERNMENT
> }
>
> I have correct deserializing at log:
>
> ... type=PRIVATE ...
>
> Should I make custom deserialization for REST requests? Could I make a
> custom REST method for retrieving some fields of object?
>
Re: DataStreamer. allowOverwrite(false) - Will it slow the writes?
Hello! I don't think so. It is supposed to speed it up. Do you have persistence? With persistence, you would expect a slow-down after an initial spike. Regards, -- Ilya Kasnacheev пт, 10 июл. 2020 г. в 15:09, krkumar24061...@gmail.com < krkumar24061...@gmail.com>: > Hi Guys - If I have my data streamer's allowOverwrite is set to false which > is default, will this cause my read IOPS go up as it need to check if the > key already exist? Bcoz the behavior that we see is that the write > performance goes down significantly after it has inserted few billion rows > and around the same time READ IOPS goes up. We are just curious if > allowOverwrite(false) is causing the read IOPS go up? > > > Thanx and Regards, > KR Kumar > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Ignite node top ram limit
Hello! I don't think there is any, if we are talking of off-heap size. Any problems in case of node failure are shared by persistent clusters which have more data than RAM. Regards, -- Ilya Kasnacheev ср, 8 июл. 2020 г. в 18:01, Maxim Volkomorov <2201...@gmail.com>: > Is there a RAM limit for single node (hardcoded or practical)? > > What practical top limit is comfortable in terms of node failures? >
Re: Request among nodes taking minimum of idleConnectionTimeout configuration in tcpcommunicationSpi
Hello! Maybe you have some firewall issue here, such as only one way connections are possible but not the other way around. We do not recommend geo-distributed clustering with Apache Ignite. Regards, -- Ilya Kasnacheev вт, 7 июл. 2020 г. в 06:59, trans : > Hi, can someone please suggest on above. > Why client is trying to use different port other than its for creating TCP > client and failing 30 times > > Thanks. > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Destroying a ignite cache when one of the worker nodes is down
Hello! As far as I know, there is no such way yet. Regards, -- Ilya Kasnacheev чт, 9 июл. 2020 г. в 17:22, krkumar24061...@gmail.com < krkumar24061...@gmail.com>: > Hi Guys - I have a five node cluster and all the nodes are part of the > baseline. Ignite native persistence is enabled. If one of the nodes in the > baseline is down and then we destroy a cache, it removes the cache on all > the remaining four nodes and cache is completely destroyed. But now when I > bring up the fifth node back into the cluster, it throws an exception: > > Caused by: class org.apache.ignite.IgniteCheckedException: Failed to start > SPI: TcpDiscoverySpi [addrRslvr=null, sockTimeout=5000, ackTimeout=5000, > marsh=JdkMarshaller > [clsFilter=org.apache.ignite.marshaller.MarshallerUtils$1@7d1cfe97], > reconCnt=10, reconDelay=2000, maxAckTimeout=60, soLinger=5, > forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null, > skipAddrsRandomization=false] > at > > org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:302) > at > > org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:943) > at > > org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1960) > ... 14 more > Caused by: class org.apache.ignite.spi.IgniteSpiException: Joining node has > caches with data which are not presented on cluster, it could mean that > they > were already destroyed, to add the node to cluster - remove directories > with > the caches[XX] > at > > org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.checkFailedError(TcpDiscoverySpi.java:1997) > at > > org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:1116) > at > > org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:427) > at > org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2099) > > > > Is there any automatic way for ignite to auto delete these stale cache > folders?? > > Thanx and Regards, > KR Kumar > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Remote Filter Execution
Hello! This is weird, I recommend checking logs for exceptions, maybe the filter cannot run on two other server nodes. Regards, -- Ilya Kasnacheev пт, 10 июл. 2020 г. в 10:25, VeenaMithare : > Hello, > > We have a 3 server cluster . The Caches on this cluster are configured as > PARTITIONED, with 1 BACKUP. > > We also have a client executing a simple continuous query awaiting updates > on record where NAME=AA. > > 1. When this record is updated, I see the remote filter being executed only > on one node( say : Server1 ) . Should this not be executed on atleast 2 > nodes since the Cache is configured with 1 Backup . > > From the documentation : > https://apacheignite.readme.io/docs/continuous-queries#remote-filter > > This filter is executed on primary and backup nodes for a given key and > evaluates whether an update should be propagated as an event to the query's > local listener. > > 2. If Server1 i.e. the server executing this remote filter is restarted ( > say for some new deployment or any other reason ), I see that the remote > filter is not deployed on this again. > > And now, changes in any other record on this table ( i.e. records other > than > NAME=AA ) are also passed to the client - since there is no remote filter > to > filter this record. > > Please let me know what I am missing. > > > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Equivalent Open Source Ignite Version to Gridgain Version
Hello! We can only recommend to use the latest release of Apache Ignite, which is currently 2.8.1. Regards, -- Ilya Kasnacheev пн, 6 июл. 2020 г. в 07:48, Ishan Gandhi : > Considering migrating from Gridgain Ignite version 8.5.8 or 8.7.13, So what > are the equivalent Apache Ignite versions with respect to them. > > Request to provide some insight with respect to equivalency of features and > such that the applications will not be impacted due to this migration. > > If possible, please provide Apache Ignite to Gridgain version mapping > information. > > Please note: Gridgain additional features are currently not being used in > our applications. > > > Regards, > Ishan > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: ScanQuery Transform: fail to deserialze! ignite 2.8.0, java 11
Hello!
There is now a ticket about this issue, which I indeed can see:
https://issues.apache.org/jira/browse/IGNITE-13212
While it is not fixed, you will have to make sure that filter code is
deployed to all nodes.
Regards,
--
Ilya Kasnacheev
пт, 3 июл. 2020 г. в 15:50, Rafael Troilo :
> Sorry I messed up the ScanQueryWithTransform.java a little bit.
> This is the clean version.
>
> Thank you.
>
>
> On 7/3/20 2:19 PM, Rafael Troilo wrote:
> > Hi Ilya, hey Denis,
> >
> > thank you both of you for taking a look into this problem.
> >
> > How would you like get the reproducing stand-alone runnable?
> >
> > I attached a single file which reproduces the problem.
> >
> > Do you like to have it in any other way?
> >
> >
> > Some comments:
> > In this issue, it does not matter if the Transformer Class is a static
> member class or a separate class it self. Both produce the exception.
> >
> > The Transformer (IgniteClosure) class works fine in a broadcast.
> >
> > // simple IgniteClosure broadcast
> > ignite.compute().broadcast(new Transformer<>(),
> "broadcast").forEach(System.out::println);
> >
> > and for a scanquery with transform from within remote job (like this
> broadcast)
> >
> > // broadcast which return the result the scanquery transform
> > ignite.compute().broadcast(() -> {
> > return cache.query(new ScanQuery<>(), new Transformer<>()).getAll();
> > }).stream().flatMap(List::stream).forEach(System.out::println);
> >
> > but does not work on the client side directly
> >
> > // does throw a ClassNotFound Exception about the Transformer Class.
> > cache.query(new ScanQuery<>(), new
> Transformer<>()).getAll().forEach(System.out::println);
> >
> >
> > Thank you.
> >
> > Let me know if I can do more to help you to find the problem.
> >
> > Best,
> > Rafael
> >
> >
> >
> >
> > On 7/2/20 5:39 PM, Ilya Kasnacheev wrote:
> >> Hello!
> >>
> >> I would like to start with a reproducer. Rafael, can you please throw
> >> together a runnable stand-alone reproducer of this issue?
> >>
> >> Thanks,
> >>
> >
>
> --
> Rafael Troilo, Dipl.-Inform. (FH)
>GIScience Research Group
>
>Heidelberg Institute for Geoinformation Technology
>
>rafael.tro...@uni-heidelberg.de
>http://giscience.uni-hd.de
>http://www.geog.uni-heidelberg.de/gis/heigit.html
>
>Berliner Str. 45 (Mathematikon), D-69120 Heidelberg, Germany
>fon: +49(0)6221 / 54 / 19704
>
Re: Block until partition map exchange is complete
Hello! Yes, you need to return from event listener as soon as you can. Regards, -- Ilya Kasnacheev пт, 3 июл. 2020 г. в 12:03, ssansoy : > Hi Ilya, thanks for the quick help! > Within the local listen, I am adding a task to an executor - so the cache > operations happen in a different thread. However, is the key thing here > that > the local listen handler metho needs to have returned? > E.g. the local listen may not have fully completed by the time the task on > the executor has been started - so perhaps there is a transaction still > open > somewhere by the time the cache operation occurs? > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Block until partition map exchange is complete
Hello!
Do you issue your cache operations from event listener thread? This might
be unsafe and also not return the expected results. Event listeners are
invoked from internal threads.
Consider issuing a task to public pool from event listener, and then
returning. I would expect that task will run when rebalance already takes
place.
Regards,
--
Ilya Kasnacheev
пт, 3 июл. 2020 г. в 11:17, ssansoy :
> Hi Ignite users,
>
> I have 3 nodes running, with a cache with the following configuration:
>
> cacheConfiguration.setCacheMode(CacheMode.PARTITIONED);
> cacheConfiguration.setBackups(1);
> cacheConfiguration.setRebalanceMode(CacheRebalanceMode.SYNC);
>
> cacheConfiguration.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
> cacheConfiguration.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL);
>
> E.g. a partitioned cache with 1 backup - so if one of the three nodes goes
> down, all the data is still available across the remaining 2 nodes.
>
> I also have some custom code that runs on the current "leader". E.g. the
> server code runs some tasks if it is the leader node - defined as being the
> "oldest node".
> The code running on each server registers a listener for
>
> {EventType.EVT_NODE_SEGMENTED,
> EventType.EVT_NODE_FAILED,EventType.EVT_NODE_LEFT}
>
> And if it discovers that it is now the new leader, the tasks restart on the
> new "oldest node".
>
> This works fine. The issue I am having is that one of these tasks that runs
> on the leader, needs to issue a cache query to do some work.
>
> I am finding, if one of my three nodes drops off, when one of the remaining
> two nodes becomes the leader and resumes the work, the records it gets back
> from the cache are incomplete. E.g. there may be 400 entries in the cache,
> but when node 1 drops off and node 2 takes over - it only sees 250, or some
> other number less than 400. A little later, this does correctly return to
> 400 - I expect because the exchange process has completed behind the scenes
> and the node now has all the data it needs.
>
> I am a little surprised by this however, because I am using
> CacheRebalanceMode.SYNC which suggests from the docs here:
> https://apacheignite.readme.io/docs/rebalancing that
>
> "This means that any call to cache public API will be blocked until
> rebalancing is finished."
>
> E.g. if I call ignite.cache("MYCACHE").size() (a public cache method) then
> this should not return an incomplete number, but rather block until the
> underlying rebalance has completes and then only return 400.
>
> Does anyone have any pointers to what I might be doing wrong here? Thanks!
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: KafkaStream marshaller error
Hello!
init() will be called before execute() and stop().
To be extra sure, you can check for null.
Regards,
--
Ilya Kasnacheev
чт, 2 июл. 2020 г. в 19:06, Maxim Volkomorov <2201...@gmail.com>:
> Ok, if i put an assignment to init(), how will i start it in execute()
> method, and stop in canсel()? This example was taken here
> http://apache-ignite-users.70518.x6.nabble.com/Kindly-tell-me-where-to-find-these-jar-files-td12649.html
>
>
> чт, 2 июл. 2020 г. в 18:57, Ilya Kasnacheev :
>
>> Hello!
>>
>> You should do the assignment in init() method.
>>
>> Don't create KafkaStreamer before service is sent over and is ready for
>> initialization.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> чт, 2 июл. 2020 г. в 18:54, Maxim Volkomorov <2201...@gmail.com>:
>>
>>> public class KafkaStreamerService implements Service {
>>>
>>> public static final String SERVICE_NAME = "KafkaStreamerService";
>>> private static final long serialVersionUID = 1L;
>>>
>>> @IgniteInstanceResource
>>> private Ignite ignite;
>>> private KafkaStreamer kafkaStreamer = new
>>> KafkaStreamer<>();
>>> private IgniteLogger log;
>>>
>>> чт, 2 июл. 2020 г. в 18:51, Ilya Kasnacheev :
>>>
>>>> Hello!
>>>>
>>>> Where do you assign kafkaStreamer field?
>>>>
>>>> Regards,
>>>> --
>>>> Ilya Kasnacheev
>>>>
>>>>
>>>> чт, 2 июл. 2020 г. в 18:46, Maxim Volkomorov <2201...@gmail.com>:
>>>>
>>>>> Now i disabled node Filtering. You mean i started KafkaStreamer in
>>>>> init()?
>>>>> I started KafkaStreamer like:
>>>>>
>>>>> @Override
>>>>> public void execute(ServiceContext ctx) throws Exception {
>>>>> log.info("KafkaStreamerService starting ...");
>>>>> kafkaStreamer.start();
>>>>> log.info("KafkaStreamerService started OK");
>>>>>
>>>>> In init() i only configure KafkaStreamer parameters.
>>>>>
>>>>> @Override
>>>>> public void init(ServiceContext ctx) throws Exception {
>>>>> log = ignite.log();
>>>>>
>>>>> IgniteDataStreamer stmr =
>>>>> ignite.dataStreamer("kafkaCache");
>>>>> stmr.allowOverwrite(true);
>>>>> stmr.autoFlushFrequency(1000);
>>>>>
>>>>> kafkaStreamer.setIgnite(ignite);
>>>>> kafkaStreamer.setStreamer(stmr);
>>>>> kafkaStreamer.setThreads(4);
>>>>> ...
>>>>>
>>>>>
>>>>> If i have to avoid fat instances, how i can share kafkaStreamer
>>>>> instance between init(), execute() and cancel()?
>>>>>
>>>>>
>>>>> чт, 2 июл. 2020 г. в 16:53, Ilya Kasnacheev >>>> >:
>>>>>
>>>>>> Hello!
>>>>>>
>>>>>> You should probably start KafkaStreamer on remote node when the
>>>>>> service is initialized (init()), instead of starting it in e.g.
>>>>>> constructor
>>>>>> and trying to send it to remote node.
>>>>>>
>>>>>> Avoid putting fat instances in the fields of
>>>>>> service/compute/predicate classes.
>>>>>>
>>>>>> Regards,
>>>>>> --
>>>>>> Ilya Kasnacheev
>>>>>>
>>>>>>
>>>>>> чт, 2 июл. 2020 г. в 10:42, Maxim Volkomorov <2201...@gmail.com>:
>>>>>>
>>>>>>> I have 1 DataNod and 1 Service with streaming. I have a filter for
>>>>>>> service:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> public boolean apply(ClusterNode node) {
>>>>>>> Boolean dataNode = node.attribute("kafkastreamer.service.node");
>>>>>>>
>>>>>>> return dataNode != null && dataNode;
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> I have a marshalling error java.io.NotSerializableException:
>>>>>>>
Re: KafkaStream marshaller error
Hello!
You should do the assignment in init() method.
Don't create KafkaStreamer before service is sent over and is ready for
initialization.
Regards,
--
Ilya Kasnacheev
чт, 2 июл. 2020 г. в 18:54, Maxim Volkomorov <2201...@gmail.com>:
> public class KafkaStreamerService implements Service {
>
> public static final String SERVICE_NAME = "KafkaStreamerService";
> private static final long serialVersionUID = 1L;
>
> @IgniteInstanceResource
> private Ignite ignite;
> private KafkaStreamer kafkaStreamer = new
> KafkaStreamer<>();
> private IgniteLogger log;
>
> чт, 2 июл. 2020 г. в 18:51, Ilya Kasnacheev :
>
>> Hello!
>>
>> Where do you assign kafkaStreamer field?
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> чт, 2 июл. 2020 г. в 18:46, Maxim Volkomorov <2201...@gmail.com>:
>>
>>> Now i disabled node Filtering. You mean i started KafkaStreamer in
>>> init()?
>>> I started KafkaStreamer like:
>>>
>>> @Override
>>> public void execute(ServiceContext ctx) throws Exception {
>>> log.info("KafkaStreamerService starting ...");
>>> kafkaStreamer.start();
>>> log.info("KafkaStreamerService started OK");
>>>
>>> In init() i only configure KafkaStreamer parameters.
>>>
>>> @Override
>>> public void init(ServiceContext ctx) throws Exception {
>>> log = ignite.log();
>>>
>>> IgniteDataStreamer stmr =
>>> ignite.dataStreamer("kafkaCache");
>>> stmr.allowOverwrite(true);
>>> stmr.autoFlushFrequency(1000);
>>>
>>> kafkaStreamer.setIgnite(ignite);
>>> kafkaStreamer.setStreamer(stmr);
>>> kafkaStreamer.setThreads(4);
>>> ...
>>>
>>>
>>> If i have to avoid fat instances, how i can share kafkaStreamer instance
>>> between init(), execute() and cancel()?
>>>
>>>
>>> чт, 2 июл. 2020 г. в 16:53, Ilya Kasnacheev :
>>>
>>>> Hello!
>>>>
>>>> You should probably start KafkaStreamer on remote node when the service
>>>> is initialized (init()), instead of starting it in e.g. constructor and
>>>> trying to send it to remote node.
>>>>
>>>> Avoid putting fat instances in the fields of service/compute/predicate
>>>> classes.
>>>>
>>>> Regards,
>>>> --
>>>> Ilya Kasnacheev
>>>>
>>>>
>>>> чт, 2 июл. 2020 г. в 10:42, Maxim Volkomorov <2201...@gmail.com>:
>>>>
>>>>> I have 1 DataNod and 1 Service with streaming. I have a filter for
>>>>> service:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> public boolean apply(ClusterNode node) {
>>>>> Boolean dataNode = node.attribute("kafkastreamer.service.node");
>>>>>
>>>>> return dataNode != null && dataNode;
>>>>> }
>>>>>
>>>>>
>>>>> I have a marshalling error java.io.NotSerializableException:
>>>>> org.apache.ignite.stream.kafka.KafkaStreamer using KafkaStreamer in my
>>>>> Service:
>>>>>
>>>>> private KafkaStreamer kafkaStreamer = new
>>>>> KafkaStreamer<>();
>>>>>
>>>>> I only can start service with:
>>>>>
>>>>> private static KafkaStreamer kafkaStreamer = new
>>>>> KafkaStreamer<>();
>>>>>
>>>>> Is it because Ignite trying data transfer KafkaStreamer instance
>>>>> between nodes?
>>>>>
>>>>> Log:
>>>>>
>>>>> [2020-07-02 10:27:47,552][ERROR][main][IgniteServiceProcessor] Failed
>>>>> to marshal service with configured marshaller [name=KafkaStreamerService,
>>>>> srvc=services.kafkastreamer.KafkaStreamerService@3dedb4a6,
>>>>> marsh=JdkMarshaller [clsFilter=null]]
>>>>> class org.apache.ignite.IgniteCheckedException: Failed to serialize
>>>>> object: services.kafkastreamer.KafkaStreamerService@3dedb4a6
>>>>> at
>>>>> org.apache.ignite.marshaller.jdk.JdkMarshaller.marshal0(JdkMarshaller.java:102)
>>>>> at
>>>>> org.apache.ignite.marshaller.jdk.JdkMarshaller.marshal0(JdkMa
Re: KafkaStream marshaller error
Hello!
Where do you assign kafkaStreamer field?
Regards,
--
Ilya Kasnacheev
чт, 2 июл. 2020 г. в 18:46, Maxim Volkomorov <2201...@gmail.com>:
> Now i disabled node Filtering. You mean i started KafkaStreamer in
> init()?
> I started KafkaStreamer like:
>
> @Override
> public void execute(ServiceContext ctx) throws Exception {
> log.info("KafkaStreamerService starting ...");
> kafkaStreamer.start();
> log.info("KafkaStreamerService started OK");
>
> In init() i only configure KafkaStreamer parameters.
>
> @Override
> public void init(ServiceContext ctx) throws Exception {
> log = ignite.log();
>
> IgniteDataStreamer stmr =
> ignite.dataStreamer("kafkaCache");
> stmr.allowOverwrite(true);
> stmr.autoFlushFrequency(1000);
>
> kafkaStreamer.setIgnite(ignite);
> kafkaStreamer.setStreamer(stmr);
> kafkaStreamer.setThreads(4);
> ...
>
>
> If i have to avoid fat instances, how i can share kafkaStreamer instance
> between init(), execute() and cancel()?
>
>
> чт, 2 июл. 2020 г. в 16:53, Ilya Kasnacheev :
>
>> Hello!
>>
>> You should probably start KafkaStreamer on remote node when the service
>> is initialized (init()), instead of starting it in e.g. constructor and
>> trying to send it to remote node.
>>
>> Avoid putting fat instances in the fields of service/compute/predicate
>> classes.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> чт, 2 июл. 2020 г. в 10:42, Maxim Volkomorov <2201...@gmail.com>:
>>
>>> I have 1 DataNod and 1 Service with streaming. I have a filter for
>>> service:
>>>
>>>
>>>
>>>
>>>
>>> public boolean apply(ClusterNode node) {
>>> Boolean dataNode = node.attribute("kafkastreamer.service.node");
>>>
>>> return dataNode != null && dataNode;
>>> }
>>>
>>>
>>> I have a marshalling error java.io.NotSerializableException:
>>> org.apache.ignite.stream.kafka.KafkaStreamer using KafkaStreamer in my
>>> Service:
>>>
>>> private KafkaStreamer kafkaStreamer = new
>>> KafkaStreamer<>();
>>>
>>> I only can start service with:
>>>
>>> private static KafkaStreamer kafkaStreamer = new
>>> KafkaStreamer<>();
>>>
>>> Is it because Ignite trying data transfer KafkaStreamer instance between
>>> nodes?
>>>
>>> Log:
>>>
>>> [2020-07-02 10:27:47,552][ERROR][main][IgniteServiceProcessor] Failed to
>>> marshal service with configured marshaller [name=KafkaStreamerService,
>>> srvc=services.kafkastreamer.KafkaStreamerService@3dedb4a6,
>>> marsh=JdkMarshaller [clsFilter=null]]
>>> class org.apache.ignite.IgniteCheckedException: Failed to serialize
>>> object: services.kafkastreamer.KafkaStreamerService@3dedb4a6
>>> at
>>> org.apache.ignite.marshaller.jdk.JdkMarshaller.marshal0(JdkMarshaller.java:102)
>>> at
>>> org.apache.ignite.marshaller.jdk.JdkMarshaller.marshal0(JdkMarshaller.java:109)
>>> at
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:57)
>>> at
>>> org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10386)
>>> at
>>> org.apache.ignite.internal.processors.service.IgniteServiceProcessor.prepareServiceConfigurations(IgniteServiceProcessor.java:583)
>>> at
>>> org.apache.ignite.internal.processors.service.IgniteServiceProcessor.staticallyConfiguredServices(IgniteServiceProcessor.java:1541)
>>> at
>>> org.apache.ignite.internal.processors.service.IgniteServiceProcessor.collectJoiningNodeData(IgniteServiceProcessor.java:354)
>>> at
>>> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$5.collect(GridDiscoveryManager.java:861)
>>> at
>>> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.collectExchangeData(TcpDiscoverySpi.java:2032)
>>> at
>>> org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:1029)
>>> at
>>> org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:427)
>>> at
>>> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2099)
>>> at
>>> org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:299)
>>> at
>>> org.apache.ignite.internal.managers.discovery.GridDiscoveryMana
Re: ScanQuery Transform: fail to deserialze! ignite 2.8.0, java 11
Hello!
I would like to start with a reproducer. Rafael, can you please throw
together a runnable stand-alone reproducer of this issue?
Thanks,
--
Ilya Kasnacheev
чт, 2 июл. 2020 г. в 17:28, Denis Magda :
> Ilya,
>
> Yes, Rafael confirmed that the peer-class-loading is set.
>
> Still does not work, this time complaining that the MyFilter class is not
> found.
> PeerClassLoading is active as far as I can tell. At least in both
> server/client ignite config
>
> Is this a known limitation or regression? Should we file an feature
> request or bug?
>
> Denis
>
> On Thursday, July 2, 2020, Ilya Kasnacheev
> wrote:
>
>> Hello!
>>
>> Did you enable peer class loading? You need to either enable peer class
>> loading, or put your filter class(es) to all server nodes.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> чт, 2 июл. 2020 г. в 15:09, Rafael Troilo :
>>
>>> Hi Denis,
>>>
>>> thank you for your quick response. Unfortunately even avoiding lambdas
>>> or anonymous classes didn't solve this problem.
>>> And was my first thought for this problem too.
>>>
>>> With:
>>>
>>> MyFilter.java
>>>
>>> import org.apache.ignite.lang.IgniteClosure;
>>>
>>> public class MyFilter implements IgniteClosure {
>>>
>>> private static final long serialVersionUID = 1L;
>>> private final long l;
>>>
>>> public MyFilter(long l) {
>>> this.l = l;
>>> }
>>>
>>> @Override
>>> public Long apply(T e) {
>>> return l;
>>> }
>>>
>>> }
>>>
>>>
>>> and
>>>
>>> try (Ignite ignite = Ignition.start("ignite.xml")) {
>>>IgniteCache cache =
>>> ignite.getOrCreateCache("ScanTransformTest");
>>> ...
>>>// 2 Running the ScanQuery on the remote server. Works!
>>> compute.broadcast(() -> {
>>> try (QueryCursor cursor = cache.query(new
>>> ScanQuery<>().setLocal(true), new MyFilter<>(234))) {
>>> for (Long row : cursor) {
>>> return row;
>>> }
>>> }
>>> return 0;
>>> })
>>>
>>> // 3 Still does not work.
>>> try (QueryCursor cursor = cache.query(new ScanQuery<>(), new
>>> MyFilter<>(345L))) {
>>> for (Long row : cursor) {
>>> System.out.println(row);
>>> break;
>>> }
>>> }
>>> }
>>>
>>> Still does not work, this time complaining that the MyFilter class is
>>> not found.
>>> PeerClassLoading is active as far as I can tell. At least in both
>>> server/client ignite config
>>> ...
>>>
>>> ...
>>> it is enabled.
>>>
>>> I'm also not sure why the same class (IgniteClosure) works in the other
>>> case but not in the Transformer or whats the differnt in case 2 to 3.
>>> I guess making the MyFilter class available on the server will solve
>>> this problem, but should not peerClassLoading work here?
>>>
>>> Is there any information I can provide or test I should do to help to
>>> solve this problem.
>>>
>>> Thank you very much.
>>>
>>>
>>> Stack Trace:
>>> Exception in thread "main" javax.cache.CacheException: class
>>> org.apache.ignite.IgniteCheckedException: MyFilter
>>> at
>>> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1317)
>>> at
>>> org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:173)
>>> at
>>> org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$5.onHasNext(GridCacheDistributedQueryManager.java:645)
>>> at
>>> org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53)
>>> at
>>> org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
>>> at
>>> org.apache.ignite.internal.processors.cache.AutoClosableCursorIterator.hasNext(AutoClosableCursorIterator.java:49)
>>> at ScanQueryTest.main(ScanQueryTest.java:57)
>>> Caused by: class org.apache.ignite.IgniteCheckedException: MyFilter
>>> at
>>> org.apache.ignite.internal.util.IgniteUtils
Re: KafkaStream marshaller error
Hello!
You should probably start KafkaStreamer on remote node when the service is
initialized (init()), instead of starting it in e.g. constructor and trying
to send it to remote node.
Avoid putting fat instances in the fields of service/compute/predicate
classes.
Regards,
--
Ilya Kasnacheev
чт, 2 июл. 2020 г. в 10:42, Maxim Volkomorov <2201...@gmail.com>:
> I have 1 DataNod and 1 Service with streaming. I have a filter for service:
>
>
>
>
>
> public boolean apply(ClusterNode node) {
> Boolean dataNode = node.attribute("kafkastreamer.service.node");
>
> return dataNode != null && dataNode;
> }
>
>
> I have a marshalling error java.io.NotSerializableException:
> org.apache.ignite.stream.kafka.KafkaStreamer using KafkaStreamer in my
> Service:
>
> private KafkaStreamer kafkaStreamer = new
> KafkaStreamer<>();
>
> I only can start service with:
>
> private static KafkaStreamer kafkaStreamer = new
> KafkaStreamer<>();
>
> Is it because Ignite trying data transfer KafkaStreamer instance between
> nodes?
>
> Log:
>
> [2020-07-02 10:27:47,552][ERROR][main][IgniteServiceProcessor] Failed to
> marshal service with configured marshaller [name=KafkaStreamerService,
> srvc=services.kafkastreamer.KafkaStreamerService@3dedb4a6,
> marsh=JdkMarshaller [clsFilter=null]]
> class org.apache.ignite.IgniteCheckedException: Failed to serialize
> object: services.kafkastreamer.KafkaStreamerService@3dedb4a6
> at
> org.apache.ignite.marshaller.jdk.JdkMarshaller.marshal0(JdkMarshaller.java:102)
> at
> org.apache.ignite.marshaller.jdk.JdkMarshaller.marshal0(JdkMarshaller.java:109)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:57)
> at
> org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10386)
> at
> org.apache.ignite.internal.processors.service.IgniteServiceProcessor.prepareServiceConfigurations(IgniteServiceProcessor.java:583)
> at
> org.apache.ignite.internal.processors.service.IgniteServiceProcessor.staticallyConfiguredServices(IgniteServiceProcessor.java:1541)
> at
> org.apache.ignite.internal.processors.service.IgniteServiceProcessor.collectJoiningNodeData(IgniteServiceProcessor.java:354)
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$5.collect(GridDiscoveryManager.java:861)
> at
> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.collectExchangeData(TcpDiscoverySpi.java:2032)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:1029)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:427)
> at
> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2099)
> at
> org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:299)
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:943)
> at
> org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1960)
> at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1276)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2045)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1703)
> at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1117)
> at
> org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1035)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:921)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:820)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:690)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:659)
> at org.apache.ignite.Ignition.start(Ignition.java:346)
> at
> app.KafkaStreamerServiceNodeStartup.main(KafkaStreamerServiceNodeStartup.java:40)
> Caused by: java.io.NotSerializableException:
> org.apache.ignite.stream.kafka.KafkaStreamer
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
> at
> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
> at
> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
> at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
> at
> org.apache.ignite.marshaller.jdk.JdkMarshaller.marshal0(JdkMarshaller.java:97)
> ... 25 more
>
Re: CusterGroup topology unexpectedly empty
Hello!
Please note that ClusterGroups usually do not recompute. This means, if you
had two nodes with _roleAttribute "True", and they all went away, the group
is now empty, even as two new nodes has joined with the same attribute.
You need to re-create ClusterGroup in order to refresh it.
Please check https://issues.apache.org/jira/browse/IGNITE-13050
Regards,
--
Ilya Kasnacheev
ср, 1 июл. 2020 г. в 11:26, Raymond Wilson :
> I've been researching a problem where we see a ClusterGroup topology
> becoming unexpectedly empty. I have not found anything that is similar to
> my case outlined below:
>
> We have some code using the C# client on Ignite 2.8.0 like this:
>
> var _group = _ignite?.GetCluster()?.ForAttribute(_roleAttribute, "True");
>
> There are nodes in the grid (deployed on AWS EKS) that have a local
> attribute with the _roleAttribute name set to the value "True", yet this
> command sometimes decides that there are no matching nodes. Once it does
> so, it does not seem to change requiring a restart of the grid.
>
> In this instance, the _ignite reference may be a long lived reference to
> the object returned by Ignition.GetIgnite("GridName") and cached for future
> reference. Is this a safe practice, or can the underlying Ingite object
> change leaving a long lived Ignite reference containing stale state
> information about the grid?
>
> Thanks,
> Raymond.
> .
>
> --
> <http://www.trimble.com/>
> Raymond Wilson
> Solution Architect, Civil Construction Software Systems (CCSS)
> 11 Birmingham Drive | Christchurch, New Zealand
> +64-21-2013317 Mobile
> raymond_wil...@trimble.com
>
>
> <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch>
>
Re: Re:
Hello! I've never seen this issue so I don't know. I would really appreciate a reproducer. Regards, -- Ilya Kasnacheev ср, 1 июл. 2020 г. в 08:16, BEELA GAYATRI : > > > Hi Kasnacheev, > > > >The below behavior is occurring randomly after stopping the node. Do we > have any resolution apart from deleting metastorage folder? > > > > > > Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for > Windows 10 > > > > *From: *Ilya Kasnacheev > *Sent: *Thursday, June 25, 2020 8:16 PM > *To: *user@ignite.apache.org > *Subject: *Re: > > > "External email. Open with Caution" > > Hello! > > > > This looks like a known issue: > https://issues.apache.org/jira/browse/IGNITE-12850 > > > > Do you have a reproducer for this behavior? > > > > Regards, > > -- > > Ilya Kasnacheev > > > > > > пн, 15 июн. 2020 г. в 10:49, BEELA GAYATRI : > > > > Hi Team, > > > > I am running 3 server nodes by setting baseline topology for all the 3 > servers with persistence enabled > > > > I have stopped one of the nodes and again started the node. I am getting > below error so I have remove metastorage folder in the persistence storage > path for that node. Now the node is up and running perfectly. > > > > *Please clarify ,what could be the reason behind the error and why the > node is up and rrunning after removing the metastorage folder in > persistence path of the node?* > > > > Exception during start processors, node will be stopped and close > connections > > class org.apache.ignite.IgniteCheckedException: Restoring of > BaselineTopology history has failed, expected history item not found for > id=3 > > at > org.apache.ignite.internal.processors.cluster.BaselineTopologyHistory.restoreHistory(BaselineTopologyHistory.java:54) > > at > org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor.onReadyForRead(GridClusterStateProcessor.java:223) > > at > org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetastorageReadyForRead(GridCacheDatabaseSharedManager.java:396) > > at > org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:662) > > at > org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead(GridCacheDatabaseSharedManager.java:4610) > > at > org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1048) > > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038) > > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730) > > at > org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158) > > at > org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1076) > > at > org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:962) > > at > org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:861) > > at > org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:731) > > at > org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:700) > > at org.apache.ignite.Ignition.start(Ignition.java:348 > > > > > > Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for > Windows 10 > > > > =-=-= > Notice: The information contained in this e-mail > message and/or attachments to it may contain > confidential or privileged information. If you are > not the intended recipient, any dissemination, use, > review, distribution, printing or copying of the > information contained in this e-mail message > and/or attachments to it are strictly prohibited. If > you have received this communication in error, > please notify us by reply e-mail or telephone and > immediately and permanently delete the message > and any attachments. Thank you > > >
Re: Ignite node log file setup
Hello!
"""
In Spring syntax it will be #{systemProperties['gridName']}:
"""
Regards,
--
Ilya Kasnacheev
ср, 1 июл. 2020 г. в 12:01, kay :
> Hello, I'm waiting for reply :)
>
> How can I set system property to use in ignite config file??
> I set and got a error so , attacted log file before..
>
> Thank you so much
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: ScanQuery Transform: fail to deserialze! ignite 2.8.0, java 11
Hello!
Did you enable peer class loading? You need to either enable peer class
loading, or put your filter class(es) to all server nodes.
Regards,
--
Ilya Kasnacheev
чт, 2 июл. 2020 г. в 15:09, Rafael Troilo :
> Hi Denis,
>
> thank you for your quick response. Unfortunately even avoiding lambdas or
> anonymous classes didn't solve this problem.
> And was my first thought for this problem too.
>
> With:
>
> MyFilter.java
>
> import org.apache.ignite.lang.IgniteClosure;
>
> public class MyFilter implements IgniteClosure {
>
> private static final long serialVersionUID = 1L;
> private final long l;
>
> public MyFilter(long l) {
> this.l = l;
> }
>
> @Override
> public Long apply(T e) {
> return l;
> }
>
> }
>
>
> and
>
> try (Ignite ignite = Ignition.start("ignite.xml")) {
>IgniteCache cache =
> ignite.getOrCreateCache("ScanTransformTest");
> ...
>// 2 Running the ScanQuery on the remote server. Works!
> compute.broadcast(() -> {
> try (QueryCursor cursor = cache.query(new
> ScanQuery<>().setLocal(true), new MyFilter<>(234))) {
> for (Long row : cursor) {
> return row;
> }
> }
> return 0;
> })
>
> // 3 Still does not work.
> try (QueryCursor cursor = cache.query(new ScanQuery<>(), new
> MyFilter<>(345L))) {
> for (Long row : cursor) {
> System.out.println(row);
> break;
> }
> }
> }
>
> Still does not work, this time complaining that the MyFilter class is not
> found.
> PeerClassLoading is active as far as I can tell. At least in both
> server/client ignite config
> ...
>
> ...
> it is enabled.
>
> I'm also not sure why the same class (IgniteClosure) works in the other
> case but not in the Transformer or whats the differnt in case 2 to 3.
> I guess making the MyFilter class available on the server will solve this
> problem, but should not peerClassLoading work here?
>
> Is there any information I can provide or test I should do to help to
> solve this problem.
>
> Thank you very much.
>
>
> Stack Trace:
> Exception in thread "main" javax.cache.CacheException: class
> org.apache.ignite.IgniteCheckedException: MyFilter
> at
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1317)
> at
> org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:173)
> at
> org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$5.onHasNext(GridCacheDistributedQueryManager.java:645)
> at
> org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53)
> at
> org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
> at
> org.apache.ignite.internal.processors.cache.AutoClosableCursorIterator.hasNext(AutoClosableCursorIterator.java:49)
> at ScanQueryTest.main(ScanQueryTest.java:57)
> Caused by: class org.apache.ignite.IgniteCheckedException: MyFilter
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10310)
> at
> org.apache.ignite.internal.processors.cache.query.GridCacheQueryRequest.finishUnmarshal(GridCacheQueryRequest.java:389)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1625)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:586)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1843)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1468)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$5200(GridIoManager.java:229)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1365)
> at
> java.base/java.util.concurrent
Re: Hot service upgrade without downtime
Hello! I don't see why you can't start the new service under new name, and then stop the old service when new service is already available. Regards, -- Ilya Kasnacheev чт, 2 июл. 2020 г. в 16:35, Sergey Antonov : > Hi, Aravind. > > > > Thank you for reply! I don’t think that kubernetes may help in my case. > The main problem is a requirement to cancel service during service upgrade > [1]. So, in that moment, the service will be stopped and other application > can’t use them. > > > > [1] > https://apacheignite.readme.io/docs/service-grid#service-updates-redeployment > > > > *From:* Aravind J [mailto:aravin...@gmail.com] > *Sent:* 02 July 2020 14:06 > *To:* user@ignite.apache.org > *Subject:* Re: Hot service upgrade without downtime > > > > Hi , > > > > If you are ready to port your cluster to kubernetes, these steps can be > handled in much more cleaner way , even though technically it does the same > steps mentioned above . > > > > With just "kubectil patch" command , you can achieve this . > > > > Regards > > Aravind > > > > On Thu, 2 Jul 2020 at 15:39, Sergey Antonov wrote: > > Hello, Igniters! > > > > I’d like to know, does Ignite have ability to upgrade user’s service in > service grid without downtime? > > > > Let’s imagine that I have grid with 2 nodes. Each node has deployed > instance of service. I’d like to upgrade service version without service’s > downtime. > > > > At the moment I have only one idea how to do it: start the same service on > different nodes with different names (service1 on node1 and service2 on > node2) and use node filter. > > · Stop one node > > · Upgrade service jar. > > · Return the node to cluster. > > > > Apply same steps to another node. > > > > Do you know simpler ways for the service upgrade? > > > > --- > Die Europäische Kommission hat unter http://ec.europa.eu/consumers/odr/ > eine Europäische Online-Streitbeilegungsplattform (OS-Plattform) errichtet. > Verbraucher können die OS-Plattform für die außergerichtliche Beilegung von > Streitigkeiten aus Online-Verträgen mit in der EU niedergelassenen > Unternehmen nutzen. > > Informationen (einschließlich Pflichtangaben) zu einzelnen, innerhalb der > EU tätigen Gesellschaften und Zweigniederlassungen des Konzerns Deutsche > Bank finden Sie unter https://www.deutsche-bank.de/Pflichtangaben. Diese > E-Mail enthält vertrauliche und/ oder rechtlich geschützte Informationen. > Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich > erhalten haben, informieren Sie bitte sofort den Absender und vernichten > Sie diese E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe > dieser E-Mail ist nicht gestattet. > > The European Commission has established a European online dispute > resolution platform (OS platform) under http://ec.europa.eu/consumers/odr/. > Consumers may use the OS platform to resolve disputes arising from online > contracts with providers established in the EU. > > Please refer to https://www.db.com/disclosures for information (including > mandatory corporate particulars) on selected Deutsche Bank branches and > group companies registered or incorporated in the European Union. This > e-mail may contain confidential and/or privileged information. If you are > not the intended recipient (or have received this e-mail in error) please > notify the sender immediately and delete this e-mail. Any unauthorized > copying, disclosure or distribution of the material in this e-mail is > strictly forbidden. > > > > --- > Die Europäische Kommission hat unter http://ec.europa.eu/consumers/odr/ > eine Europäische Online-Streitbeilegungsplattform (OS-Plattform) errichtet. > Verbraucher können die OS-Plattform für die außergerichtliche Beilegung von > Streitigkeiten aus Online-Verträgen mit in der EU niedergelassenen > Unternehmen nutzen. > > Informationen (einschließlich Pflichtangaben) zu einzelnen, innerhalb der > EU tätigen Gesellschaften und Zweigniederlassungen des Konzerns Deutsche > Bank finden Sie unter https://www.deutsche-bank.de/Pflichtangaben. Diese > E-Mail enthält vertrauliche und/ oder rechtlich geschützte Informationen. > Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich > erhalten haben, informieren Sie bitte sofort den Absender und vernichten > Sie diese E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe > dieser E-Mail ist nicht gestattet. > > The European Commission has established a European online dispute > resolution platform (OS platform) under http://ec.europa.eu/consumers/odr/. > Consumers may use the OS platform to res
Re: Question on Key choice for an Ignite cache
Hello! You can also use Binary Object (POJO) as a key, i.e., you can use an Object with String email and String phone fields. I don't recommend mangling strings further than concatenation. String key is no worse than numeric key. Regards, -- Ilya Kasnacheev вт, 30 июн. 2020 г. в 15:17, Eugene McGowan : > > We would like to create an Ignite key by concatenating data. This is a > standard distributed system pattern for key-value, and would allow the > reader and writer consistently access the cache. The data is a combination > of strings and integers. To simplify our use case, lets say its an email > address (f...@bar.com) and phone number (123444) we want to use to build > our key.Our key could therefore be:foo@bar.com_123444 The > advantage of this approach is the key can easily be read/debugged. Is there > a more optimum format for Ignite though? For regular RDBMS, it seems > integers were the default choice. We could convert f...@bar.com to an int, > e.g. f converts to 6, o to 15, etc. > This naïve first attempt of conversion would of-course lead to clashes, as > 111 could map to either aaa or ak. This could be worked around potentially, > so looking for an initial steer on what Ignite would prefer as a key (i.e. > strings or ints). Is hashing something that would be recommended either? In > terms of any partitioning type logic, we are guessing not, but just more > around creating deterministic, unique keys >
Re: How to do address resolution?
Hello! I can see the following in the thread dump: "main" #1 prio=5 os_prio=0 tid=0x7f02c400d800 nid=0x1e43 runnable [0x7f02cad1e000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.Net.poll(Native Method) at sun.nio.ch.SocketChannelImpl.poll(SocketChannelImpl.java:951) - locked <0xec066048> (a java.lang.Object) at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:121) - locked <0xec066038> (a java.lang.Object) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3299) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createNioClient(TcpCommunicationSpi.java:2987) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2870) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2713) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:2672) at org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1656) at org.apache.ignite.internal.managers.communication.GridIoManager.sendToGridTopic(GridIoManager.java:1731) at org.apache.ignite.internal.processors.task.GridTaskWorker.sendRequest(GridTaskWorker.java:1436) at org.apache.ignite.internal.processors.task.GridTaskWorker.processMappedJobs(GridTaskWorker.java:666) at org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:538) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at org.apache.ignite.internal.processors.task.GridTaskProcessor.startTask(GridTaskProcessor.java:764) at org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:392) at org.apache.ignite.internal.IgniteComputeImpl.executeAsync0(IgniteComputeImpl.java:528) at org.apache.ignite.internal.IgniteComputeImpl.execute(IgniteComputeImpl.java:498) at org.apache.ignite.visor.visor$.execute(visor.scala:1800) It seems that Visor is trying to connect to client node via Communication, and it fails because the network connection is filtered out. Regards, -- Ilya Kasnacheev пн, 29 июн. 2020 г. в 23:47, John Smith : > Ok. > > I am able to reproduce the "issue" unless we have a misunderstanding and > we are talking about the same thing... > > My thick client runs inside a container in a closed network NOT bridged > and NOT host. I added a flag to my application that allows it to add the > address resolver to the config. > > 1- If I disable address resolution and I connect with visor to the cluster > and try to print detailed statistics for that particular client, visor > freezes indefinitely at the Data Region Snapshot. > Control C doesn't kill the visor either. It just stuck. This also happens > when running the cache command. Just freezes indefinitely. > > I attached the jstack output to the email but it is also here: > https://www.dropbox.com/s/wujcee1gd87gk6o/jstack.out?dl=0 > > 2- If I enable address resolution for the thick client then all the > commands work ok. I also see an "Accepted incoming communication > connection" log in the client. > > > > > > > > > > On Mon, 29 Jun 2020 at 15:30, Ilya Kasnacheev > wrote: > >> Hello! >> >> The easiest way is jstack >> >> Regards, >> -- >> Ilya Kasnacheev >> >> >> пн, 29 июн. 2020 г. в 20:20, John Smith : >> >>> How? >>> >>> On Mon, 29 Jun 2020 at 12:03, Ilya Kasnacheev >>> wrote: >>> >>>> Hello! >>>> >>>> Try collecting thread dump from Visor as it freezes. >>>> >>>> Regards, >>>> -- >>>> Ilya Kasnacheev >>>> >>>> >>>> пн, 29 июн. 2020 г. в 18:11, John Smith : >>>> >>>>> How though? >>>>> >>>>> 1- Entered node command >>>>> 2- Got list of nodes, including thick clients >>>>> 3- Selected thick client >>>>> 4- Entered Y for detailed statistics >>>>> 5- Snapshot details displayed >>>>> 6- Data region stats frozen >>>>> >>>>> I think the address resolution is working for this as well. I need to >>>>> confirm. Because I fixed the resolver as per your solution and visor no >>>>> longer freezes on #6 above. >>>>> >>>>> On Mon, 29 Jun 2020 at 10:54, Ilya Kasnacheev < >>>>> ilya.kasnach...@gmail.com> wrote: >>>>> >>>>>> Hello! >>>>>> >>>>>> This usually means there's no connectivity between node and Visor. >>>>>&g
Re: Ignite Configuration file modify
Hello! You can implement this yourself if you really need it. Such as, you can watch an XML file, when it changes, do Ignition.loadSpringBin(file, bean name), and then call ignite.getOrCreateCache() on all cache configurations found in that bean. Regards. -- Ilya Kasnacheev вт, 30 июн. 2020 г. в 12:15, kay : > Hello :) > > Because If I add cache by code, the cache disappears when the node is > restarted > so I want to modify the configuration file directly. > > Thank you! > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: How to do address resolution?
Hello!
The easiest way is jstack
Regards,
--
Ilya Kasnacheev
пн, 29 июн. 2020 г. в 20:20, John Smith :
> How?
>
> On Mon, 29 Jun 2020 at 12:03, Ilya Kasnacheev
> wrote:
>
>> Hello!
>>
>> Try collecting thread dump from Visor as it freezes.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> пн, 29 июн. 2020 г. в 18:11, John Smith :
>>
>>> How though?
>>>
>>> 1- Entered node command
>>> 2- Got list of nodes, including thick clients
>>> 3- Selected thick client
>>> 4- Entered Y for detailed statistics
>>> 5- Snapshot details displayed
>>> 6- Data region stats frozen
>>>
>>> I think the address resolution is working for this as well. I need to
>>> confirm. Because I fixed the resolver as per your solution and visor no
>>> longer freezes on #6 above.
>>>
>>> On Mon, 29 Jun 2020 at 10:54, Ilya Kasnacheev
>>> wrote:
>>>
>>>> Hello!
>>>>
>>>> This usually means there's no connectivity between node and Visor.
>>>>
>>>> Regards,
>>>> --
>>>> Ilya Kasnacheev
>>>>
>>>>
>>>> пн, 29 июн. 2020 г. в 17:01, John Smith :
>>>>
>>>>> Also I think for Visor as well?
>>>>>
>>>>> When I do top or node commands, I can see the thick client. But when I
>>>>> look at detailed statistics for that particular thick client it freezes
>>>>> "indefinitely". Regular statistics it seems ok.
>>>>>
>>>>> On Mon, 29 Jun 2020 at 08:08, Ilya Kasnacheev <
>>>>> ilya.kasnach...@gmail.com> wrote:
>>>>>
>>>>>> Hello!
>>>>>>
>>>>>> For thick clients, you need both 47100 and 47500, both directions
>>>>>> (perhaps for 47500 only client -> server is sufficient, but for 47100,
>>>>>> both
>>>>>> are needed).
>>>>>>
>>>>>> For thin clients, 10800 is enough. For control.sh, 11211.
>>>>>>
>>>>>> Regards,
>>>>>> --
>>>>>> Ilya Kasnacheev
>>>>>>
>>>>>>
>>>>>> пт, 26 июн. 2020 г. в 22:06, John Smith :
>>>>>>
>>>>>>> I'm askin in separate question so people can search for it if they
>>>>>>> ever come across this...
>>>>>>>
>>>>>>> My server nodes are started as and I also connect the client as such.
>>>>>>>
>>>>>>> >>>>>> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>>>>>>>
>>>>>>>
>>>>>>> foo:47500
>>>>>>> ...
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> In my client code I used the basic address resolver
>>>>>>>
>>>>>>> And I put in the map
>>>>>>>
>>>>>>> "{internalHostIP}:47500", "{externalHostIp}:{externalPort}"
>>>>>>>
>>>>>>> igniteConfig.setAddressResolver(addrResolver);
>>>>>>>
>>>>>>>
>>>>>>> QUESTIONS
>>>>>>> ___
>>>>>>>
>>>>>>> 1- Port 47500 is used for discovery only?
>>>>>>> 2- Port 47100 is used for actual coms to the nodes?
>>>>>>> 3- In my container environment I have only mapped 47100, do I also
>>>>>>> need to map for 47500 for the Tcp Discovery SPI?
>>>>>>> 4- When I connect with Visor and I try to look at details for the
>>>>>>> client node it blocks. I'm assuming that's because visor cannot connect
>>>>>>> back to the client at 47100?
>>>>>>> Se logs below
>>>>>>>
>>>>>>> LOGS
>>>>>>> ___
>>>>>>>
>>>>>>> When I look at the client logs I get...
>>>>>>>
>>>>>>> IgniteConfiguration [
>>>>>>> igniteInstanceName=xx,
>>>>>>> ...
>>>>>>> discoSpi=TcpDiscoverySpi [
>>>>>>> addrRslvr=null, <--- Do I need to use BasicResolver or here???
>>>>>>> ...
>>>>>>> commSpi=TcpCommunicationSpi [
>>>>>>> ...
>>>>>>> locAddr=null,
>>>>>>> locHost=null,
>>>>>>> locPort=47100,
>>>>>>> addrRslvr=null, <--- Do I need to use BasicResolver or here???
>>>>>>> ...
>>>>>>> ],
>>>>>>> ...
>>>>>>> addrRslvr=BasicAddressResolver [
>>>>>>> inetAddrMap={},
>>>>>>> inetSockAddrMap={/internalIp:47100=/externalIp:2389} <
>>>>>>> ],
>>>>>>> ...
>>>>>>> clientMode=true,
>>>>>>> ...
>>>>>>>
>>>>>>>
>>>>>>>
Re: How to do address resolution?
Hello!
Try collecting thread dump from Visor as it freezes.
Regards,
--
Ilya Kasnacheev
пн, 29 июн. 2020 г. в 18:11, John Smith :
> How though?
>
> 1- Entered node command
> 2- Got list of nodes, including thick clients
> 3- Selected thick client
> 4- Entered Y for detailed statistics
> 5- Snapshot details displayed
> 6- Data region stats frozen
>
> I think the address resolution is working for this as well. I need to
> confirm. Because I fixed the resolver as per your solution and visor no
> longer freezes on #6 above.
>
> On Mon, 29 Jun 2020 at 10:54, Ilya Kasnacheev
> wrote:
>
>> Hello!
>>
>> This usually means there's no connectivity between node and Visor.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> пн, 29 июн. 2020 г. в 17:01, John Smith :
>>
>>> Also I think for Visor as well?
>>>
>>> When I do top or node commands, I can see the thick client. But when I
>>> look at detailed statistics for that particular thick client it freezes
>>> "indefinitely". Regular statistics it seems ok.
>>>
>>> On Mon, 29 Jun 2020 at 08:08, Ilya Kasnacheev
>>> wrote:
>>>
>>>> Hello!
>>>>
>>>> For thick clients, you need both 47100 and 47500, both directions
>>>> (perhaps for 47500 only client -> server is sufficient, but for 47100, both
>>>> are needed).
>>>>
>>>> For thin clients, 10800 is enough. For control.sh, 11211.
>>>>
>>>> Regards,
>>>> --
>>>> Ilya Kasnacheev
>>>>
>>>>
>>>> пт, 26 июн. 2020 г. в 22:06, John Smith :
>>>>
>>>>> I'm askin in separate question so people can search for it if they
>>>>> ever come across this...
>>>>>
>>>>> My server nodes are started as and I also connect the client as such.
>>>>>
>>>>> >>>> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>>>>>
>>>>>
>>>>> foo:47500
>>>>> ...
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> In my client code I used the basic address resolver
>>>>>
>>>>> And I put in the map
>>>>>
>>>>> "{internalHostIP}:47500", "{externalHostIp}:{externalPort}"
>>>>>
>>>>> igniteConfig.setAddressResolver(addrResolver);
>>>>>
>>>>>
>>>>> QUESTIONS
>>>>> ___
>>>>>
>>>>> 1- Port 47500 is used for discovery only?
>>>>> 2- Port 47100 is used for actual coms to the nodes?
>>>>> 3- In my container environment I have only mapped 47100, do I also
>>>>> need to map for 47500 for the Tcp Discovery SPI?
>>>>> 4- When I connect with Visor and I try to look at details for the
>>>>> client node it blocks. I'm assuming that's because visor cannot connect
>>>>> back to the client at 47100?
>>>>> Se logs below
>>>>>
>>>>> LOGS
>>>>> ___
>>>>>
>>>>> When I look at the client logs I get...
>>>>>
>>>>> IgniteConfiguration [
>>>>> igniteInstanceName=xx,
>>>>> ...
>>>>> discoSpi=TcpDiscoverySpi [
>>>>> addrRslvr=null, <--- Do I need to use BasicResolver or here???
>>>>> ...
>>>>> commSpi=TcpCommunicationSpi [
>>>>> ...
>>>>> locAddr=null,
>>>>> locHost=null,
>>>>> locPort=47100,
>>>>> addrRslvr=null, <--- Do I need to use BasicResolver or here???
>>>>> ...
>>>>> ],
>>>>> ...
>>>>> addrRslvr=BasicAddressResolver [
>>>>> inetAddrMap={},
>>>>> inetSockAddrMap={/internalIp:47100=/externalIp:2389} <
>>>>> ],
>>>>> ...
>>>>> clientMode=true,
>>>>> ...
>>>>>
>>>>>
>>>>>
Re: How to do address resolution?
Hello!
This usually means there's no connectivity between node and Visor.
Regards,
--
Ilya Kasnacheev
пн, 29 июн. 2020 г. в 17:01, John Smith :
> Also I think for Visor as well?
>
> When I do top or node commands, I can see the thick client. But when I
> look at detailed statistics for that particular thick client it freezes
> "indefinitely". Regular statistics it seems ok.
>
> On Mon, 29 Jun 2020 at 08:08, Ilya Kasnacheev
> wrote:
>
>> Hello!
>>
>> For thick clients, you need both 47100 and 47500, both directions
>> (perhaps for 47500 only client -> server is sufficient, but for 47100, both
>> are needed).
>>
>> For thin clients, 10800 is enough. For control.sh, 11211.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> пт, 26 июн. 2020 г. в 22:06, John Smith :
>>
>>> I'm askin in separate question so people can search for it if they ever
>>> come across this...
>>>
>>> My server nodes are started as and I also connect the client as such.
>>>
>>> >> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>>>
>>>
>>> foo:47500
>>> ...
>>>
>>>
>>>
>>>
>>> In my client code I used the basic address resolver
>>>
>>> And I put in the map
>>>
>>> "{internalHostIP}:47500", "{externalHostIp}:{externalPort}"
>>>
>>> igniteConfig.setAddressResolver(addrResolver);
>>>
>>>
>>> QUESTIONS
>>> ___
>>>
>>> 1- Port 47500 is used for discovery only?
>>> 2- Port 47100 is used for actual coms to the nodes?
>>> 3- In my container environment I have only mapped 47100, do I also need
>>> to map for 47500 for the Tcp Discovery SPI?
>>> 4- When I connect with Visor and I try to look at details for the client
>>> node it blocks. I'm assuming that's because visor cannot connect back to
>>> the client at 47100?
>>> Se logs below
>>>
>>> LOGS
>>> ___
>>>
>>> When I look at the client logs I get...
>>>
>>> IgniteConfiguration [
>>> igniteInstanceName=xx,
>>> ...
>>> discoSpi=TcpDiscoverySpi [
>>> addrRslvr=null, <--- Do I need to use BasicResolver or here???
>>> ...
>>> commSpi=TcpCommunicationSpi [
>>> ...
>>> locAddr=null,
>>> locHost=null,
>>> locPort=47100,
>>> addrRslvr=null, <--- Do I need to use BasicResolver or here???
>>> ...
>>> ],
>>> ...
>>> addrRslvr=BasicAddressResolver [
>>> inetAddrMap={},
>>> inetSockAddrMap={/internalIp:47100=/externalIp:2389} <
>>> ],
>>> ...
>>> clientMode=true,
>>> ...
>>>
>>>
>>>
Re: WriteBehind enabled problem FK constraint
Hello! I recommend disabling FKs for tables which are written by Cache Store. Regards, -- Ilya Kasnacheev пн, 29 июн. 2020 г. в 16:01, manueltg89 : > Hello! > > With writeBehind enabled I get many more errors that with writeBehind > disabled. What write state do you recommended with writes to external > database? > > Thanks in advance. > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: WriteBehind enabled problem FK constraint
Hello! I don't think so, writeBehind is managed in batches and sometimes a batch has to be re-tried, which will make it out-of-order. Other reasons may also cause writeBehind writes to be out of order. You may not get the "correct" order even for non-write-behind Cache Store since it may maintain its own ordering between different caches. Regards, -- Ilya Kasnacheev сб, 27 июн. 2020 г. в 18:59, manueltg89 : > Hello. > > In my app I have writeBehind to true, when I start to write in my Apache > Ignite instance sometimes throw an error with FK, is there anyway to > maintain order with writes to external database? > > Thanks. > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: How to do address resolution?
Hello!
For thick clients, you need both 47100 and 47500, both directions (perhaps
for 47500 only client -> server is sufficient, but for 47100, both are
needed).
For thin clients, 10800 is enough. For control.sh, 11211.
Regards,
--
Ilya Kasnacheev
пт, 26 июн. 2020 г. в 22:06, John Smith :
> I'm askin in separate question so people can search for it if they ever
> come across this...
>
> My server nodes are started as and I also connect the client as such.
>
>class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>
>
> foo:47500
> ...
>
>
>
>
> In my client code I used the basic address resolver
>
> And I put in the map
>
> "{internalHostIP}:47500", "{externalHostIp}:{externalPort}"
>
> igniteConfig.setAddressResolver(addrResolver);
>
>
> QUESTIONS
> ___
>
> 1- Port 47500 is used for discovery only?
> 2- Port 47100 is used for actual coms to the nodes?
> 3- In my container environment I have only mapped 47100, do I also need to
> map for 47500 for the Tcp Discovery SPI?
> 4- When I connect with Visor and I try to look at details for the client
> node it blocks. I'm assuming that's because visor cannot connect back to
> the client at 47100?
> Se logs below
>
> LOGS
> ___
>
> When I look at the client logs I get...
>
> IgniteConfiguration [
> igniteInstanceName=xx,
> ...
> discoSpi=TcpDiscoverySpi [
> addrRslvr=null, <--- Do I need to use BasicResolver or here???
> ...
> commSpi=TcpCommunicationSpi [
> ...
> locAddr=null,
> locHost=null,
> locPort=47100,
> addrRslvr=null, <--- Do I need to use BasicResolver or here???
> ...
> ],
> ...
> addrRslvr=BasicAddressResolver [
> inetAddrMap={},
> inetSockAddrMap={/internalIp:47100=/externalIp:2389} <
> ],
> ...
> clientMode=true,
> ...
>
>
>
Re: Non Distributed Join between tables
Hello! If both tbl_a and tbl_c use tbl_b.id == tbl_a.fk_id == tbl_c.fk_id as affinity key, then I assume it would. Regards, -- Ilya Kasnacheev чт, 25 июн. 2020 г. в 19:17, manueltg89 : > With the following structure, and partitioned cache in the three tables: > > tbl_a tbl_b tbl_c > --- - > aff_a aff_b > > When I make: select * from tbl_a INNER JOIN tbl_b ON tbl_b.id = > tbl_a.fk_id > INNER JOIN tbl_c.fk_id = tbl_b.id > > Should it work and return results always? > > The question is: If tbl_c is collocated with tbl_b and tbl_b is collocated > with tbl_a, should it be collocated tbl_a with tbl_c? > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Ignite authenticationEnabled queries
Hello! Make sure user's login is "in quotes", otherwise it will be case-insensitive, i.e. ALLCAPS login. Regards, -- Ilya Kasnacheev пт, 26 июн. 2020 г. в 05:02, marble.zh...@coinflex.com < marble.zh...@coinflex.com>: > thanks a lot, with statement works. > > Btw, created a new user , but cannot connect to the instance, anything else > need settle done? thanks > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Unsubscribe
Hello! Please write to user-unsubscr...@ignite.apache.org, etc, to unsubscribe from lists. Regards, -- Ilya Kasnacheev чт, 25 июн. 2020 г. в 22:16, Sriveena Mattaparthi < sriveena.mattapar...@ekaplus.com>: > > “Confidentiality Notice: The contents of this email message and any > attachments are intended solely for the addressee(s) and may contain > confidential and/or privileged information and may be legally protected > from disclosure. If you are not the intended recipient of this message or > their agent, or if this message has been addressed to you in error, please > immediately alert the sender by reply email and then delete this message > and any attachments. If you are not the intended recipient, you are hereby > notified that any use, dissemination, copying, or storage of this message > or its attachments is strictly prohibited.” >
Re: Ignite-cassandra with protobuf
Hello! You can put protobuf values as byte[] into our cache. SQL will not be able to index these. Regards, -- Ilya Kasnacheev чт, 25 июн. 2020 г. в 22:17, abhi2703 : > Hi All, > > I want to use protobuf end to end.. Java -> Ignite -> Cassandra. > > Is it possible? > > Please help. > > Regards, > vicky > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: CacheVersionIO.read and AbstractDataPageIO.readPayload are hot methods
Hello! In Apache Ignite, data is held in Durable Memory regardless whether you have persistence enabled or not. https://apacheignite.readme.io/docs/durable-memory-tuning I guess these classes are in the 'persistence' package for historical reasons. Regards, -- Ilya Kasnacheev пт, 26 июн. 2020 г. в 15:42, ashishb888 : > I am not able to understand why below methods are hot methods in my > application. And I do not use persistence for my application yet. > > - org.apache.ignite.internal.processors.cache.persistence.tree.io > - > > org.apache.ignite.internal.processors.cache.persistence.tree.io.CacheVersionIO.read(long, > boolean) > - > > org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO.readPayload(long, > int, int) > > BR, > Ashish > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re:
Hello! This looks like a known issue: https://issues.apache.org/jira/browse/IGNITE-12850 Do you have a reproducer for this behavior? Regards, -- Ilya Kasnacheev пн, 15 июн. 2020 г. в 10:49, BEELA GAYATRI : > > > Hi Team, > > > > I am running 3 server nodes by setting baseline topology for all the 3 > servers with persistence enabled > > > > I have stopped one of the nodes and again started the node. I am getting > below error so I have remove metastorage folder in the persistence storage > path for that node. Now the node is up and running perfectly. > > > > *Please clarify ,what could be the reason behind the error and why the > node is up and rrunning after removing the metastorage folder in > persistence path of the node?* > > > > Exception during start processors, node will be stopped and close > connections > > class org.apache.ignite.IgniteCheckedException: Restoring of > BaselineTopology history has failed, expected history item not found for > id=3 > > at > org.apache.ignite.internal.processors.cluster.BaselineTopologyHistory.restoreHistory(BaselineTopologyHistory.java:54) > > at > org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor.onReadyForRead(GridClusterStateProcessor.java:223) > > at > org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetastorageReadyForRead(GridCacheDatabaseSharedManager.java:396) > > at > org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:662) > > at > org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead(GridCacheDatabaseSharedManager.java:4610) > > at > org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1048) > > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038) > > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730) > > at > org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158) > > at > org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1076) > > at > org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:962) > > at > org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:861) > > at > org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:731) > > at > org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:700) > > at org.apache.ignite.Ignition.start(Ignition.java:348 > > > > > > Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for > Windows 10 > > > > =-=-= > Notice: The information contained in this e-mail > message and/or attachments to it may contain > confidential or privileged information. If you are > not the intended recipient, any dissemination, use, > review, distribution, printing or copying of the > information contained in this e-mail message > and/or attachments to it are strictly prohibited. If > you have received this communication in error, > please notify us by reply e-mail or telephone and > immediately and permanently delete the message > and any attachments. Thank you > >
Re: Non Distributed Join between tables
Hello! This seems to be a MySQL script. Can you please provide an Ignite SQL script to demonstrate the issue? Thanks, -- Ilya Kasnacheev пт, 19 июн. 2020 г. в 16:35, manueltg89 : > I attach required file SQL. dbaffinity.sql > <http://apache-ignite-users.70518.x6.nabble.com/file/t2878/dbaffinity.sql> > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Ignite authenticationEnabled queries
Hello! I have filed a ticket about this mismatch: https://issues.apache.org/jira/browse/IGNITE-13184 Regards, -- Ilya Kasnacheev чт, 25 июн. 2020 г. в 13:48, Ilya Kasnacheev : > Hello! > > I'm sorry for earlier advice, the correct syntax is: > ALTER USER "ignite" WITH PASSWORD 'abcdefg' ; > > Regards, > -- > Ilya Kasnacheev > > > чт, 25 июн. 2020 г. в 13:10, marble.zh...@coinflex.com < > marble.zh...@coinflex.com>: > >> not works, still show error, >> >> ALTER USER "ignite" SET password 'abcdefg' ; >> >> >> >> >> SQL Error [1001] [42000]: Failed to parse query. User "ignite" not found; >> SQL statement: >> ALTER USER "ignite" SET password 'abcdefg' [90032-197] >> >> >> >> -- >> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >> >
Re: Ignite authenticationEnabled queries
Hello! I'm sorry for earlier advice, the correct syntax is: ALTER USER "ignite" WITH PASSWORD 'abcdefg' ; Regards, -- Ilya Kasnacheev чт, 25 июн. 2020 г. в 13:10, marble.zh...@coinflex.com < marble.zh...@coinflex.com>: > not works, still show error, > > ALTER USER "ignite" SET password 'abcdefg' ; > > > > > SQL Error [1001] [42000]: Failed to parse query. User "ignite" not found; > SQL statement: > ALTER USER "ignite" SET password 'abcdefg' [90032-197] > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Unsubscribe
Hello! Please write to user-unsubscr...@ignite.apache.org, etc, to unsubscribe from lists. Regards, -- Ilya Kasnacheev чт, 25 июн. 2020 г. в 13:20, preeti nidgunde : > Unsubscribe . >
Re: Ignite Load Performance when memory exhausted
Hello! Until you got this message, you did not need to catch up with disk (except for WAL writing, maybe), but once you see it, good days are over and you need to do checkpoints with all pages which were updated recently. There are things to tune here, but don't expect miracles. https://apacheignite.readme.io/docs/durable-memory-tuning Regards, -- Ilya Kasnacheev вт, 23 июн. 2020 г. в 18:25, njcstreet : > Hi, > > I am evaluating Apache Ignite for a use case where I know we may not have > enough RAM to store the entire database. For example we might store 30 days > of data, but only have enough memory for 15, and in fact most of the > queries > will be against the last 5 days of data. I am doing some tests loading the > 30 days of data (roughly 2 billion objects) into a test environment. > > > > · The test environment has 6 servers each with 128GB RAM, which I > am > allocating roughly 70% to Ignite to allow the OS enough memory. > > · This is using Native Persistence in Ignite 2.8.1. > > · The disks I am using are not as fast as production would be (they > are not SSD) but they are still reasonably fast. > > · The object I am storing is reasonably simple, it has 25 > properties > which are integers (2 of these have indexes on) and 4 which are floats, but > there are a lot of them (roughly 2 billion). > > · The cache the data is being loaded into is partitioned. > > · Data is loaded using IDataStreamer – there is a single loader > process which has 10 threads reading files, then submitting into a single > DataStreamer. > > > > What I am observing is that: > > > > · Whilst the default region has RAM available, the inserts are > quite > quick – even though the data is being written to disk (the WAL is > enabled). > I get on average 150K objects inserted per second. > > · Once the default region runs out of RAM, I see an error message > > “WARN|org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl|Page > replacements started, pages will be rotated with disk, this will affect > storage performance (consider increasing > DataRegionConfiguration#setMaxSize).” > > · After this, inserts are much slower, going down to about 19K > objects per second. > > > > Now I kind of understand what is happening here is that for every new page > in memory that needs to be allocated, Ignite is having to purge another > page > from memory (this is implied by the error message). But what I don’t > understand is why the performance impact is so great (150K objects / sec > when memory is available vs 19K per sec when memory is exhausted). Before > and after the region ran out of RAM, the data was being written to disk. > After the region ran out of RAM, I would expect the only difference to be > that it needs to purge some data from RAM to make space for the data that > is > loading – I wouldn’t have thought there should be any more disk activity > since through Ignite Persistence the data was already being written to > disk. > Maybe I have something mis-configured, or I have misunderstood something? > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Ignite authenticationEnabled queries
Hello! Try ALTER USER "ignite" SET PASSWORD 'NewPassword'. WRT grants, we do not have granular object in JDBC authentication currently, except by using a 3rd party security plugin. Regards, -- Ilya Kasnacheev чт, 25 июн. 2020 г. в 06:11, marble.zh...@coinflex.com < marble.zh...@coinflex.com>: > Hi Guru, > > Enabled the authentication with below configuration, it is ok for jdbc thin > client with ignite/ignite to connect to server, but found cannot change > ignite's password, will say no this user. > > And created new user cannot access, and no where to grant priviledges, how > can we go? thanks a lot. > > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Can not finish proxy initialization in logs
Hello! This is INFO level, meaning, there's nothing for you to worry about (probably). If you dropped a cache and client node rejoins cluster, you will see this message. Regards, -- Ilya Kasnacheev чт, 25 июн. 2020 г. в 10:36, itsmeravikiran.c : > Hi Team, > > Recently i have upgraded ignite version 2.6 to 2.8 version. > > I am getting below INFO "Can not finish proxy initialization" > > 3:19:26.806 [sys-#273] INFO o.a.i.i.p.cache.GridCacheProcessor - Can not > finish proxy initialization because proxy does not exist, > cacheName=, localNodeId= > > But we are not getting this in 2.6 version. > > shall i need to do any configuration change not to get proxy > initialization. > > it is there any impact if we get the info in our logs. > > Could you please help me on this. > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Ignite Cache exists but table lost
Hello!
Can you perhaps provide some logs or error messages? It's hard to say what
happens here, it's unexpected.
Regards,
--
Ilya Kasnacheev
чт, 25 июн. 2020 г. в 11:26, marble.zh...@coinflex.com <
marble.zh...@coinflex.com>:
> Hi Guru,
>
> We found a problem that, caches exists, we can get/set successfully, and
> file also exists in the work folder,
> but cannot see the related table by jdbc thin access, the cache was created
> by java code, not by sql scripts.
>
> sometimes can see it, but sometimes not, really surprise, not sure if bug
> or
> not, thanks.
>
> We use below code to initialize the cache,
> CacheConfiguration cacheConfiguration = new
> CacheConfiguration<>(Market.class.getSimpleName());
> cacheConfiguration.setIndexedTypes(Long.class, Market.class);
> cacheConfiguration.setSqlIndexMaxInlineSize(100);
> cacheConfiguration.setSqlSchema("PUBLIC");
> cacheConfiguration.setAtomicityMode(CacheAtomicityMode.ATOMIC);
> cacheConfiguration.setCacheMode(CacheMode.REPLICATED);
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Setting Cache Size using Max Records
Hello! You can have a data region configured for this specific cache, with max size (in bytes) and page eviction on. Regards, -- Ilya Kasnacheev чт, 25 июн. 2020 г. в 07:16, Victor : > Hi Denis, > > For our product we are using an in-house solution where we expose the size > in records. Now we are looking at replacing it with Ignite. So we wanted to > make the experience seamless and let the end user's continue to set the > same > configuration they are familiar with, rather than adding a new learning > curve. > > Does any of the expiry policies except record size? > > Thanks, > Vic > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Respective XML Configuration for @QueryTextField in ignite-default.xml
Hello! I think in your QueryEntity you need to declare an index of type QueryIndexType.FULLTEXT. Regards, -- Ilya Kasnacheev вт, 23 июн. 2020 г. в 19:35, ram.gaurav.chhabra < ram.gaurav.chha...@gmail.com>: > Hello All, > > Currently working on one module which is using Apache Ignite and all > configuration is driven through XML(ignite-default.xml). > > Got one requirement where-in i need to implement full text search which can > be achieved by *@QueryTextField* > > I have done this change and annotated respective field in domain object but > as my actual configuration is in XML for this domain object which is being > saved to cache, *@QueryTextField* this is not working and I am not getting > any result. > > Just need help if anybody could please suggest respective XML configuration > for @QueryTextField so that it started working. I searched on internet but > not able to find relevant xml configuration. > > Thanks > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Problem with writeBehind Apache Ignite 2.8.1
Hello! I think there is indeed a problem with Cache Store here: https://issues.apache.org/jira/browse/IGNITE-13167 I hope it will get fixed, until it isn't I recommend switching that cache to ATOMIC mode, this will help sidestep the issue. Regards, -- Ilya Kasnacheev пт, 19 июн. 2020 г. в 16:22, manueltg89 : > You need to have MySQL Server with schema "bugdb.sql" imported, config your > mysql database in "secret.properties", run "ServerNodeCodeStartup" and > execute PHP file with inserts to Apache Ignite. > > Sorry, there was a problem to attach PHP file. I've attached also PHP file > now. > > bug.php <http://apache-ignite-users.70518.x6.nabble.com/file/t2878/bug.php> > > > > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Non Distributed Join between tables
Hello! Do you have a SQL script to reproduce this issue? Regards, -- Ilya Kasnacheev пт, 12 июн. 2020 г. в 09:48, manueltg89 : > I'm going to explain a bit more. I use Apache Ignite as front cache to my > RDBMS, I've done automatic RDBMS integration. Table A, Table B, Table C is > a example simplified for my real schema. My schema is much more complex. > > I answer the questions: > > 1. True. > > 2. True > > 3. I use thin client with sql queries and DISTRIBUTED_JOINS=false. > > I want to know if tbl_a <- tbl_b <- tbl_c, if I have collocated table_a > with > table_b and table_b with table_c, then, > Could I make a query with three tables? > > I understand that it is necessary to put affinity key from table_c to > table_a, so table_b and table_c have same affinity key. It is unique > solution > or put table_a as replicated and only table_c with affinity key to table_b. > It is true? > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: NPE during printing failure information
Hello! I suggest filing an issue against Apache Ignite JIRA. Ticket which added this line is IGNITE-11750 Regards, -- Ilya Kasnacheev сб, 13 июн. 2020 г. в 01:37, Andrey Davydov : > Yes, I think that initial problem was with communication and it need to be > tested more on my side. But during handling this error, Ignite throw NPE, I > think that NPE during communication exception handling is bug. > > > > [01:08:22] Possible failure suppressed accordingly to a configured handler > [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, > super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet > [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], > failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class > o.a.i.IgniteException: GridWorker [name=grid-nio-worker-tcp-comm-2, > igniteInstanceName=TestNode-0, finished=false, heartbeatTs=1591837692022]]] > > 2020-06-11 01:08:22,686 [nio-acceptor-tcp-comm-#16311%TestNode-1%] ERROR > :135 - Critical system error detected. Will be handled accordingly to > configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, > timeout=0, super=AbstractFailureHandler > [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, > SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext > [type=SYSTEM_WORKER_TERMINATION, err=java.lang.NullPointerException]] > > java.lang.NullPointerException: null > > at > org.apache.ignite.internal.processors.diagnostic.DiagnosticProcessor.onFailure(DiagnosticProcessor.java:109) > ~[ignite-core-2.8.1.jar:2.8.1] > > at > org.apache.ignite.internal.processors.failure.FailureProcessor.process(FailureProcessor.java:188) > ~[ignite-core-2.8.1.jar:2.8.1] > > at > org.apache.ignite.internal.processors.failure.FailureProcessor.process(FailureProcessor.java:146) > ~[ignite-core-2.8.1.jar:2.8.1] > > > > > > Andrey. > > > > *От: *akorensh > *Отправлено: *11 июня 2020 г. в 18:31 > *Кому: *user@ignite.apache.org > *Тема: *Re: NPE during printing failure information > > > > Hi, > > This looks like a communication problem. > > GridWorker [name=grid-nio-worker-tcp-comm-3 > > > >Make sure all nodes can see each other. Test it with a minimum number of > > nodes. > >If it still doesn't work then describe your test and send the full logs > > and Ignite condifg. > > Thanks, Alex > > > > > > > > > > > > -- > > Sent from: http://apache-ignite-users.70518.x6.abble.com/ > > >
Re: Problem with writeBehind Apache Ignite 2.8.1
Hello! What do I need to run to observe the problem? I run ServerNodeCodeStartup but it runs OK. If there should be any images in your message, they're not there. Regards, -- Ilya Kasnacheev пт, 19 июн. 2020 г. в 11:12, manueltg89 : > Hi. I attach example project. I explain a bit more. Details: > > - MySQL Server with max connections form example: 151 > - Insert with PHP PDO from PHP 7.2: > > > > It seems that Apache Ignite with writeBehind enabled open one connection by > insert and keeps open every connection over time, this bug-project.rar > <http://apache-ignite-users.70518.x6.nabble.com/file/t2878/bug-project.rar> > > that MySQL Server throws "Too many connections" error. > > In attached file you have "bugdb.sql" file with database schema for MySQL. > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Data Consistency Question
Hello! I'm not sure what you are trying to do and why you felt the need to over-complicate the solution. Regards, -- Ilya Kasnacheev пн, 15 июн. 2020 г. в 09:10, adipro : > Can you please suggest an efficient solution? > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: IgniteCheckedException
Hello! Such spring XML config cannot be started by ignite.sh, it should be started as part of a Spring application. Regards. -- Ilya Kasnacheev вт, 16 июн. 2020 г. в 10:43, kay bae : > Hello, I start node using command > > ' sh ignite.sh ./config/config.xml & ' > > It worked well before I added IgniteSpringBean. > > After I add this code, > > > > > class="org.apache.ignite.configuration.IgniteConfiguration"> > > > > > > > > 2020-06-16T16:20:38,429][INFO ][main][G] Node started : [stage="Configure > system pool" (53 ms),stage="Start managers" (187 ms),stage="Configure > binary metadata" (41 ms),stage="Start processors" (623 ms),stage="Init > metastore" (9 ms),stage="Finish recovery" (0 ms),stage="Join topology" (170 > ms),stage="Await transition" (14 ms),stage="Await exchange" (379 > ms),stage="Total time" (1476 ms)] > class org.apache.ignite.IgniteException: Failed to find configuration in: > file:./config/config.xml > at > org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1067) > at org.apache.ignite.Ignition.start(Ignition.java:349) > at > org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:300) > Caused by: class org.apache.ignite.IgniteCheckedException: Failed to find > configuration in: file:./config/config.xml > at > org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:116) > at > org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:98) > at > org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:710) > at > org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:911) > at > org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:820) > at > org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:690) > at > org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:659) > at org.apache.ignite.Ignition.start(Ignition.java:346) > ... 1 more > Failed to start grid: Failed to find configuration in: > file:./config/config.xml > > > > Node started and failed. > > Is there any set up to use IgniteSpringBean?? > > Thank you >
Re: primary keys
Hello! It seems that the index is not used. Please try creating an explicit index over these 3 fields. Regards, -- Ilya Kasnacheev пт, 19 июн. 2020 г. в 00:38, narges saleh : > Hello Ilya, > Here is the info. In this query only one table is involved. > There are about 4M records in the table and about 40,000 distinct accounts. > > thank you. > > > On Thu, Jun 18, 2020 at 9:50 AM Ilya Kasnacheev > wrote: > >> Hello! >> >> Please show DML for your tables as well as query plans. >> >> Regards, >> -- >> Ilya Kasnacheev >> >> >> чт, 18 июн. 2020 г. в 16:11, narges saleh : >> >>> Thanks Ilya. >>> Now I can see the complete plan, and it shows scanning the large tables >>> (but not the others). Increasing index size didn't help. >>> I only have primary keys on the caches and the fields in the primary >>> keys are the ones used in my where clause, so I am not sure >>> what's going on. >>> Currently, I am testing on one node only, so all the data should be in >>> one place. >>> >>> >>> On Thu, Jun 18, 2020 at 6:17 AM Ilya Kasnacheev < >>> ilya.kasnach...@gmail.com> wrote: >>> >>>> Hello! >>>> >>>> Please use !set outputFormat vertical to see complete execution plan. >>>> >>>> Index is created on primary key. There is no programmatic way to change >>>> its inline size other than specifying IGNITE_MAX_INDEX_PAYLOAD_SIZE >>>> system property or environment variable. >>>> >>>> If it is of complex type, some versions may not be able to search by >>>> its fields. >>>> >>>> Regards, >>>> -- >>>> Ilya Kasnacheev >>>> >>>> >>>> чт, 18 июн. 2020 г. в 13:13, narges saleh : >>>> >>>>> Hi All, >>>>> >>>>> Shouldn't primary keys result in indexes and if so, shouldn't I be >>>>> able to see them when I list indexes? >>>>> Does inline index size applicable to primary keys too? >>>>> Additionally, when I do explain plan on a query which involves tables >>>>> with primary keys, shouldn't I see the primary key/index being used? Or >>>>> lack of a scan statement imply that an index is being used? >>>>> ---+ >>>>> | PLAN | >>>>> ++ >>>>> | SELECT >>>>> ID, NAME,TIMESTAMP >>>>> FROM PUBLIC.table1 >>>>> /* PU | >>>>> >>>>> sql is: select timestamp from table1 where id = 50 and name = 'John'; >>>>> //primary key is on id + name >>>>> >>>>
Re: 2.7.6 to 2.8.1 migration issue
Hello! I think we restricted configuration of caches that share the same cache group. Previously, you could have e.g. caches with different atomicity mode in the same group, now you can't. If this is the case, you won't be able to upgrade without migration. You need to copy data to a new cache in a new group, then delete the old conflicting cache. Regards, -- Ilya Kasnacheev чт, 18 июн. 2020 г. в 18:47, Andrey Davydov : > > > Hello, > > > > We test migration from 2.7.6 to 2.8.1 in our DEV environment and got > problem: new code doesn’t start over old data. > > > > Some history: > > Initially we had following base cache configuration on 2.7.6: > > > > "org.apache.ignite.configuration.CacheConfiguration"> > > > > > > > > > > > > > > > > > > > > > > > > > > > > "org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction"> > > > > > > > > > > > > > > And create some caches using it. > > > > "org.apache.ignite.configuration.CacheConfiguration"> > > > > > > "org.apache.ignite.configuration.CacheConfiguration"> > > > > > > "org.apache.ignite.configuration.CacheConfiguration"> > > > > > > java.lang.String > > some.our.WellAnnotatedClass > > > > > > > > > > > > Some weeks later we update base cache configuration to use topology > validator. > > > > > > "org.apache.ignite.configuration.CacheConfiguration"> > > > > > > > > > > > > > > > > class="org.apache.ignite.configuration.CacheConfiguration"> > > > > > > > > > > > > > > > > > > > > > > > > > > > > "org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction"> > > > > > > > > > > > > > > > > We run new version over old data and everything starts ok. > > After that we update our system and create some more caches. > > > > > > "org.apache.ignite.configuration.CacheConfiguration"> > > > > > > "org.apache.ignite.configuration.CacheConfiguration"> > > > > > > > > > > And we also start new version of code and configuration over old data > directory and everything starts ok. > > Now we change ignite version from 2.7.6 to 2.8.1 and try start system over > old data. One node starts ok and two other fails with following error: > > > > 2020-06-18 15:37:12,598 [main] WARN o.a.i.i.p.c.GridLocalConfigManager - > Static configuration for the following caches will be ignored because a > persistent cache with the same name already exist (see > https://apacheignite.readme.io/docs/cache-configuration for more > information): [...] > > 2020-06-18 15:37:12,602 [main] INFO o.a.i.i.p.c.p.f.FilePageStoreManager > - Cleanup cache stores [total=1, left=0, cleanFiles=false] > > 2020-06-18 15:37:12,602 [main] ERROR o.a.i.i.IgniteKernal%dev_app - > Exception during start processors, node will be stopped and close > connections > > org.apache.ignite.IgniteCheckedException: Topology validator mismatch for > caches related to the same group [groupName=applicationGroup, > existingCache=cache1, existingTopologyValidator=null, startingCache=cache4, > startingTopologyValidator= our.custom.ValidatorClass] > > at > org.apache.ignite.internal.processors.cache.GridCacheUtils.validateCacheGroupsAttributesMismatch(GridCacheUtils.java:1032) > ~[ignite-core-2.8.1.jar:2.8.1] > > at > org.apache.ignite.internal.processors.cache.ClusterCachesInfo.validateCacheGroupConfiguration(ClusterCachesInfo.java:2305) > ~[ignite-core-2.8.1.jar:2.8.1] > > at > org.apache.ignite.internal.processors.cache.ClusterCachesInfo.onStart(ClusterCachesInfo.java:286) > ~[ignite-core-2.8.1.jar:2.8.1] > > > > In our development environment only the first node makes write operation > to cache1 (because we don’t use load balancer in dev and front-end send > request to the first node) and only this node starts without problem. Two > other nodes crashes. > > We rollback to 2.7.6 and all three nodes start ok. > > > > If there is way to start 2.8.1 over old data? > > > > > > Andrey. > > >
Re: primary keys
Hello! Please show DML for your tables as well as query plans. Regards, -- Ilya Kasnacheev чт, 18 июн. 2020 г. в 16:11, narges saleh : > Thanks Ilya. > Now I can see the complete plan, and it shows scanning the large tables > (but not the others). Increasing index size didn't help. > I only have primary keys on the caches and the fields in the primary keys > are the ones used in my where clause, so I am not sure > what's going on. > Currently, I am testing on one node only, so all the data should be in one > place. > > > On Thu, Jun 18, 2020 at 6:17 AM Ilya Kasnacheev > wrote: > >> Hello! >> >> Please use !set outputFormat vertical to see complete execution plan. >> >> Index is created on primary key. There is no programmatic way to change >> its inline size other than specifying IGNITE_MAX_INDEX_PAYLOAD_SIZE >> system property or environment variable. >> >> If it is of complex type, some versions may not be able to search by its >> fields. >> >> Regards, >> -- >> Ilya Kasnacheev >> >> >> чт, 18 июн. 2020 г. в 13:13, narges saleh : >> >>> Hi All, >>> >>> Shouldn't primary keys result in indexes and if so, shouldn't I be able >>> to see them when I list indexes? >>> Does inline index size applicable to primary keys too? >>> Additionally, when I do explain plan on a query which involves tables >>> with primary keys, shouldn't I see the primary key/index being used? Or >>> lack of a scan statement imply that an index is being used? >>> ---+ >>> | PLAN | >>> ++ >>> | SELECT >>> ID, NAME,TIMESTAMP >>> FROM PUBLIC.table1 >>> /* PU | >>> >>> sql is: select timestamp from table1 where id = 50 and name = 'John'; >>> //primary key is on id + name >>> >>
Re: Achieve uniform data distribution easily
Hello! Data distribution will only be uniform if affinity key distribution is uniform, and may still go +/- 10% between nodes. Regards, -- Ilya Kasnacheev чт, 18 июн. 2020 г. в 09:41, VincentCE : > Thank you for your answer. We are aware of that our approach is not the > clean > one as the one with explicitly defined affinity keys would be. But still > for > the sake of data distribution defining affinityKeys, in this example this > would be cacheName/key for a given key, should be equivalent to overriding > the partition method in the described way. Hence we would still get a > data-distribution far from being uniform. To me this is a bit surprising as > this would imply that even in the case of distinct affinity-keys one does > not achieve a balanced data-distribution with RendezvousAffinityFunction. > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re:
Hello!
I do not recommend relying on this order. It may change if you have more
than one node. It can also change without warning.
Regards,
--
Ilya Kasnacheev
пт, 12 июн. 2020 г. в 21:44, Prasad Bhalerao :
> I am executing following SQL in ignite. As you can see there is no order
> by clause in this SQL. Index is present on field name "id".
>
> SqlFieldsQuery sql = new SqlFieldsQuery("select p.id. p.name from Person
> p join table(id bigint = ?) i on p.id = i.id.") .setArgs(new
> Object[]{5,4,1,7,3});
>
>
> What I have observed is ignite always returns query result in an order as
> it is provided in input.
> E.g. I always get resultant rows in following order. This behaviour is
> consistent.
> p.id.p.name
> 5test5
> 4 Test4
> 1Test1
> 7Test7
> 3 Test3
>
> If I change input "id" sequence, the output changes as per the id sequence
> provided.
>
> Can someone please confirm this behaviour?
>
> Also, will it remain same with calcite integration?
>
>
>
Re: ignite web agent issue
Hello! I have 3.6.3 installed. Regards, -- Ilya Kasnacheev пт, 12 июн. 2020 г. в 09:40, itsmeravikiran.c : > Could you please let me know, what is the supported mongodb version. > Please let me know the mongodb version, which your using. > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: primary keys
Hello! Please use !set outputFormat vertical to see complete execution plan. Index is created on primary key. There is no programmatic way to change its inline size other than specifying IGNITE_MAX_INDEX_PAYLOAD_SIZE system property or environment variable. If it is of complex type, some versions may not be able to search by its fields. Regards, -- Ilya Kasnacheev чт, 18 июн. 2020 г. в 13:13, narges saleh : > Hi All, > > Shouldn't primary keys result in indexes and if so, shouldn't I be able to > see them when I list indexes? > Does inline index size applicable to primary keys too? > Additionally, when I do explain plan on a query which involves tables with > primary keys, shouldn't I see the primary key/index being used? Or lack of > a scan statement imply that an index is being used? > ---+ > | PLAN | > ++ > | SELECT > ID, NAME,TIMESTAMP > FROM PUBLIC.table1 > /* PU | > > sql is: select timestamp from table1 where id = 50 and name = 'John'; > //primary key is on id + name >
Re: HashMap warning when using PutAll()
Hello! If your Map is a singleton (only one pair) the warning should not be printed :) Regards, -- Ilya Kasnacheev ср, 17 июн. 2020 г. в 23:22, Raymond Wilson : > Hi Ilya, > > Thank you for the SortedDictionary pointer. > > I am not sure what you mean by 'size-1 collections'. Is there an issue > here that requires fixing in Ignite, or just the use of SortedDIctionary in > my client code? > > Thanks, > Raymond. > > On Thu, Jun 18, 2020 at 3:45 AM Ilya Kasnacheev > wrote: > >> Hello! >> >> The warning should not be issued for size-1 collections. >> In dotnet, you should use SortedDictionary with putAll. >> >> Regards, >> -- >> Ilya Kasnacheev >> >> >> сб, 13 июн. 2020 г. в 09:43, Raymond Wilson : >> >>> If the answer is that the consumer of the C# client should order the >>> items before calling PutAll(), how does the consumer of the C# client >>> ensure that is the case before calling PutAll()? >>> >>> On Sat, Jun 13, 2020 at 4:13 PM Raymond Wilson < >>> raymond_wil...@trimble.com> wrote: >>> >>>> Hi Evgenii, >>>> >>>> Thanks for the quick response and clarification :) >>>> >>>> Does this mean the C# client is looking at the collection provided to >>>> it and choosing treemap vs hashmap for an internal call to the Ignite Java >>>> layer based on whether the content of the collection provided is sorted? >>>> >>>> Thanks, >>>> Raymond. >>>> >>>> >>>> On Sat, Jun 13, 2020 at 2:23 AM Evgenii Zhuravlev < >>>> e.zhuravlev...@gmail.com> wrote: >>>> >>>>> The same works for other operations too(like removeAll). >>>>> >>>>> Evgenii >>>>> >>>>> пт, 12 июн. 2020 г. в 07:22, Evgenii Zhuravlev < >>>>> e.zhuravlev...@gmail.com>: >>>>> >>>>>> Raymond, >>>>>> >>>>>> Collections used in putAll should be sorted, because otherwise if >>>>>> they have the same entries but in a different order, it can lead to the >>>>>> classic deadlock. It is expected behavior. >>>>>> >>>>>> Best Regards, >>>>>> Evgenii >>>>>> >>>>>> чт, 11 июн. 2020 г. в 21:38, Raymond Wilson < >>>>>> raymond_wil...@trimble.com>: >>>>>> >>>>>>> We are using Ignite v2.8.0 and the C# client. Some of our operations >>>>>>> use PutAll() to save a collection of items in a single operation. This >>>>>>> operation is emitting the following warning into the log: >>>>>>> >>>>>>> 2020-06-10 15:04:14,199 [77] WRN [ImmutableClientServer] >>>>>>> Unordered map java.util.HashMap is >>>>>>> used for putAll operation on cache Spatial-SubGridDirectory-Immutable. >>>>>>> This >>>>>>> can lead to a distributed deadlock. Switch to a sorted map like TreeMap >>>>>>> instead. >>>>>>> >>>>>>> Does this require a Jira ticket? >>>>>>> >>>>>>> Thanks, >>>>>>> Raymond. >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> <http://www.trimble.com/> >>>>>>> Raymond Wilson >>>>>>> Solution Architect, Civil Construction Software Systems (CCSS) >>>>>>> 11 Birmingham Drive | Christchurch, New Zealand >>>>>>> +64-21-2013317 Mobile >>>>>>> raymond_wil...@trimble.com >>>>>>> >>>>>>> >>>>>>> <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch> >>>>>>> >>>>>> >>>> >>>> -- >>>> <http://www.trimble.com/> >>>> Raymond Wilson >>>> Solution Architect, Civil Construction Software Systems (CCSS) >>>> 11 Birmingham Drive | Christchurch, New Zealand >>>> +64-21-2013317 Mobile >>>> raymond_wil...@trimble.com >>>> >>>> >>>> <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch> >>>> >>> >>> >>> -- >>> <http://www.trimble.com/> >>> Raymond Wilson >>> Solution Architect, Civil Construction Software Systems (CCSS) >>> 11 Birmingham Drive | Christchurch, New Zealand >>> +64-21-2013317 Mobile >>> raymond_wil...@trimble.com >>> >>> >>> <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch> >>> >> > > -- > <http://www.trimble.com/> > Raymond Wilson > Solution Architect, Civil Construction Software Systems (CCSS) > 11 Birmingham Drive | Christchurch, New Zealand > +64-21-2013317 Mobile > raymond_wil...@trimble.com > > > <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch> >
Re: HashMap warning when using PutAll()
Hello! The warning should not be issued for size-1 collections. In dotnet, you should use SortedDictionary with putAll. Regards, -- Ilya Kasnacheev сб, 13 июн. 2020 г. в 09:43, Raymond Wilson : > If the answer is that the consumer of the C# client should order the items > before calling PutAll(), how does the consumer of the C# client ensure that > is the case before calling PutAll()? > > On Sat, Jun 13, 2020 at 4:13 PM Raymond Wilson > wrote: > >> Hi Evgenii, >> >> Thanks for the quick response and clarification :) >> >> Does this mean the C# client is looking at the collection provided to it >> and choosing treemap vs hashmap for an internal call to the Ignite Java >> layer based on whether the content of the collection provided is sorted? >> >> Thanks, >> Raymond. >> >> >> On Sat, Jun 13, 2020 at 2:23 AM Evgenii Zhuravlev < >> e.zhuravlev...@gmail.com> wrote: >> >>> The same works for other operations too(like removeAll). >>> >>> Evgenii >>> >>> пт, 12 июн. 2020 г. в 07:22, Evgenii Zhuravlev >> >: >>> >>>> Raymond, >>>> >>>> Collections used in putAll should be sorted, because otherwise if they >>>> have the same entries but in a different order, it can lead to the classic >>>> deadlock. It is expected behavior. >>>> >>>> Best Regards, >>>> Evgenii >>>> >>>> чт, 11 июн. 2020 г. в 21:38, Raymond Wilson >>> >: >>>> >>>>> We are using Ignite v2.8.0 and the C# client. Some of our operations >>>>> use PutAll() to save a collection of items in a single operation. This >>>>> operation is emitting the following warning into the log: >>>>> >>>>> 2020-06-10 15:04:14,199 [77] WRN [ImmutableClientServer] >>>>> Unordered map java.util.HashMap is >>>>> used for putAll operation on cache Spatial-SubGridDirectory-Immutable. >>>>> This >>>>> can lead to a distributed deadlock. Switch to a sorted map like TreeMap >>>>> instead. >>>>> >>>>> Does this require a Jira ticket? >>>>> >>>>> Thanks, >>>>> Raymond. >>>>> >>>>> >>>>> -- >>>>> <http://www.trimble.com/> >>>>> Raymond Wilson >>>>> Solution Architect, Civil Construction Software Systems (CCSS) >>>>> 11 Birmingham Drive | Christchurch, New Zealand >>>>> +64-21-2013317 Mobile >>>>> raymond_wil...@trimble.com >>>>> >>>>> >>>>> <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch> >>>>> >>>> >> >> -- >> <http://www.trimble.com/> >> Raymond Wilson >> Solution Architect, Civil Construction Software Systems (CCSS) >> 11 Birmingham Drive | Christchurch, New Zealand >> +64-21-2013317 Mobile >> raymond_wil...@trimble.com >> >> >> <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch> >> > > > -- > <http://www.trimble.com/> > Raymond Wilson > Solution Architect, Civil Construction Software Systems (CCSS) > 11 Birmingham Drive | Christchurch, New Zealand > +64-21-2013317 Mobile > raymond_wil...@trimble.com > > > <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch> >
Re: ignite work dir
Hello! Try setting IgniteConfiguration.workDirectory to some writable path. Regards, -- Ilya Kasnacheev чт, 11 июн. 2020 г. в 18:58, narges saleh : > Hi Ilya, > IGNITE_HOME is /opt/ignite and in ignite-log4j, the log file set to > > I don't have any issue on the server side. > > This is the exception I get on the client side: > > Caused by: class org.apache.ignite.IgniteCheckedException: Work directory > does not exist and cannot be created: /ignite/work > at > org.apache.ignite.internal.util.IgniteUtils.workDirectory(IgniteUtils.java:9440) > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.initializeConfiguration(IgnitionEx.java:2181) > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1697) > at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1117) > at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:615) > at > org.apache.ignite.internal.jdbc2.JdbcConnection.getIgnite(JdbcConnection.java:311) > at > org.apache.ignite.internal.jdbc2.JdbcConnection.(JdbcConnection.java:240) > ... 39 more > > On Thu, Jun 11, 2020 at 10:04 AM Ilya Kasnacheev < > ilya.kasnach...@gmail.com> wrote: > >> Hello! >> >> I recommend setting the property workDirectory of IgniteConfiguration >> class to desired value. And leaving IGNITE_HOME and IGNITE_WORK_DIR not >> specified. >> >> This may still cause exception if your e.g. logging subsystem is >> configured to use /ignite/work. Need to see the exception. >> >> Regards, >> -- >> Ilya Kasnacheev >> >> >> чт, 11 июн. 2020 г. в 17:55, narges saleh : >> >>> Hi All, >>> >>> What's the best way to set ignite's work dir on the client side, >>> considering that the client and server nodes run under different users. >>> I am setting both IGNITE_HOME and IGNITE_WORK_DIR system variables (on >>> client), but I am still getting the exception "/ignite/work" cannot be >>> created. >>> >>> thanks. >>> >>
Re: Apache Ignite with Spring framework
Hello! You have already sent a lot of e-mails with the same content. Please avoid sending more of these. As I have told you on Stack Overflow, Ignite is not a distributed Spring :) it will not register any spring components, but can re-use spring context and make beans from it available to local compute tasks, etc. Regards, -- Ilya Kasnacheev вт, 16 июн. 2020 г. в 16:11, kay : > Hello, > > Does the Apache Ignite operate on a spring framework basis? > > Can I register a spring controller in classpath at server remote node and > use it? > (using component scan, like @Controller) > > Thank you > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: ignite work dir
Hello! I recommend setting the property workDirectory of IgniteConfiguration class to desired value. And leaving IGNITE_HOME and IGNITE_WORK_DIR not specified. This may still cause exception if your e.g. logging subsystem is configured to use /ignite/work. Need to see the exception. Regards, -- Ilya Kasnacheev чт, 11 июн. 2020 г. в 17:55, narges saleh : > Hi All, > > What's the best way to set ignite's work dir on the client side, > considering that the client and server nodes run under different users. > I am setting both IGNITE_HOME and IGNITE_WORK_DIR system variables (on > client), but I am still getting the exception "/ignite/work" cannot be > created. > > thanks. >
Re: Need to sync two apache ignite cache DB using ip address
Hello! Yes, you should specify both local and remote addresses in discovery SPI section: https://apacheignite.readme.io/docs/tcpip-discovery#static-ip-finder Regards, -- Ilya Kasnacheev чт, 11 июн. 2020 г. в 15:07, rakshita04 : > So are you saying apache ignite automatically does that. > Do i need to configure the ip of the other node somewhere in xml file so > that data keeps syncing? > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Need to sync two apache ignite cache DB using ip address
Hello! There are ways to do that (affinity backup filter), but why would you want to? You are much better off relying on Ignite's data model when using Apache Ignite. Regards, -- Ilya Kasnacheev чт, 11 июн. 2020 г. в 15:02, rakshita04 : > Hi Ilya, > > I want one node to be master for all entries. > How can i achieve this? > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Need to sync two apache ignite cache DB using ip address
Hello! If you declare your caches as REPLICATED they will be kept in sync. One node will be master of half entries and the other will be master of the remaining half. Regards, -- Ilya Kasnacheev чт, 11 июн. 2020 г. в 14:36, rakshita04 : > Hi Team, > > We have 2 apache ignite DBs located at separate IPs and connected through > TCP-ip connection. > ne of this DB should work as master dB and on any change should sync to > other apache ignite DB and update the entries accordingly. > How can i achieve this ? what changes do i need to do in xml file? > I am using C++ platform for apache ignite. > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Ignite Spring Integration
Hello! I think that Uri Deployment has its own class loader, which can interfere with resources loading. Maybe you can wrap access to those via some intermediate class or read them from file. Regards, -- Ilya Kasnacheev ср, 10 июн. 2020 г. в 09:56, marble.zh...@coinflex.com < marble.zh...@coinflex.com>: > For more clear, i am using the bin/ignite.sh to start the ignite server, > and > using the uri hot deployment, my business logic is in the igniteSerivce, > and > need spring annotation/configuration support. > > case 1, we deploy our package jar into the ./libs folder, all works fine, > but this way cannot hot deploy; > case 2, we deploy our package jar into the uri deployment folder, it will > show cannot find the application configuration file. > > thanks. > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Questions on the mechanics of activated on-heap in ignite 2.x.x
Hello! 1. I'm not sure it will be immediately available on heap. 2. This sounds a more reasonable assumption. 3. I guess so, but the real issue here is that on-heap cache increases Full GC times. At some point it will become infeasible. 4. Yes, it would counteract the change to off-heap model. Regards, -- Ilya Kasnacheev сб, 6 июн. 2020 г. в 14:54, VincentCE : > In our project we are currently using ignite 2.7.6 with native persistence > disabled and java 11. At the moment we are not using the on-heap feature, > i.e. all our data lives in off-heap. However in order to gain performance > we > are thinking about activating on-heap. While there are already quite many > questions/answers on that topic in this forum we are still missing some > points. I would like to use the following scenario for our questions: Say > we > have one ignite-server-instance living on a *kubernetes-pod of 32 GiB > memory > request/limit size* with the following "hypothetical" configuration: > > - JVM options exactly as described here > https://apacheignite.readme.io/docs/jvm-and-system-tuning, i.e. in > particular *10 GB heap fixed*. > - Off-heap is limited to 15 GiB by adjusting the default region with > *initSize = maxSize = 15 GiB*. > No more data regions are defined. > > Before doing anything with our ignite-server-instance we *initially fill > its > off-heap with 10 GiB* of data and this will be the only data that it will > receive. > > What happens when we set > > *org.apache.ignite.configuration.CacheConfiguration.setOnheapCacheEnabled(true) > in each data configuration and for now use no eviction policies* in > particular during loading these 10 GB of data? > > More precisely: > 1. As it is emphasised several times in this forum the data will still be > be > loaded into off-heap. But will it immediately also be loaded into heap, > i.e. > during the loading procedure each data point gets replicated simultaneously > to heap resulting in two copies of the same data one in off-heap and one on > heap after the procedure is finished? > 2. ... Or will a given data point only be replicated to heap when ever it > is > being used, i.e. during computations? > 3. Lets furthermore assume that our overall configuration was stable before > switching to on-heap. In order to guarantee that it will do so afterwards > would we need to increase the heap size by roughly 10 GB to 20 GB and > therefore also our pod size to roughly 42 GiB? That would imply that using > on-heap always goes hand in hand with increasing memory resources. > 4. Obviously in this example we did not define any eviction-policy to > control the on-heap cache size. However this is indeed intended here > because > we would like each data point to be quickly available living also in heap. > Is this a useful approach (i.e. replicating the whole off-heap also in > heap) > in order to reach the overall goal namely better performance? It feels like > this approach would counteract the change to the off-heap model from ignite > 2.x.x onwards in terms of GC impacts and so on. Is this correct? > > Please let me know if you need more detailed informations about the > configurations/settings we use. > > Thanks in advance! > > Vincent > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: BinaryObjectException: Conflicting enum values
Hello!
Currently it is not implemented. Metadata will be preserver unless cluster
is restarted (in case persistence is enabled, unless it is cleared)
Regards,
--
Ilya Kasnacheev
вс, 7 июн. 2020 г. в 02:32, Andrew Munn :
> So once I insert an instance of a class in a Map I can't change the type
> of any existing member variable in the class, even if I clear the map?
> Seems like there should be some programatic way to clear any left-over
> class metadata/schema if simply clearing the cache won't do it. Right?
>
> On Sat, Jun 6, 2020 at 10:43 AM Denis Magda wrote:
>
>> You might have hit the following specificity ("Cluster Doesn’t Start
>> After Field Type Changes") that happens if you change a type of a field:
>> https://www.gridgain.com/docs/latest/perf-troubleshooting-guide/troubleshooting#cluster-doesnt-start-after-field-type-changes
>>
>> -
>> Denis
>>
>>
>> On Fri, Jun 5, 2020 at 10:53 AM Andrew Munn wrote:
>>
>>> I'm seeing the same issue as this one
>>> <http://apache-ignite-users.70518.x6.nabble.com/Help-needed-with-BinaryObjectException-td22938.html>
>>> .
>>>
>>> I had values in the cache. I cleared the cache, but did not shutdown
>>> the cluster node. I modified the enums in the class. Then I repopulated
>>> the cache with instances of the updated class. There must be some way to
>>> purge leftover metadata without bringing the cluster down, right? Can it
>>> be purged when the cache is cleared?
>>>
>>> Thanks
>>>
>>>
Re: ignite web agent issue
Hello! Yes, I did exactly that and it got built. I think I already had dependencies. Regards, -- Ilya Kasnacheev ср, 10 июн. 2020 г. в 17:44, itsmeravikiran.c : > thank you for the help. > > To Building Ignite Web Agent, shall we need MongoDB (version >=3.2.x > <=3.4.15) and NodeJS (version >=8.0.0) ??? > > Could you please let me know, the stop to building the web agent from > source's. > > As for the docs > > Building Ignite Web Agent > To build Ignite Web Agent from sources, run the following command from the > $IGNITE_HOME folder: > mvn clean package -pl :ignite-web-agent -am -P web-console -DskipTests=true > > Once the build process is over, you can find ignite-web-agent-x.x.x.zip in: > $IGNITE_HOME/modules/web-console/web-agent/target > > > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
