Re: Data loss validation. TopologyValidator
Appreciate if anyone has thoughts on how to solve the above issue -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-loss-validation-TopologyValidator-tp15147p15958.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Data loss validation. TopologyValidator
/Scenario : If there are multiple node failures occurred at the same time and ignite loses primary and as well as backup copies of partitions and how to know if this happened and data is lost? Once you detect the data loss, how to restrict/reject clients and because the cache is partial and not complete. Once the system loads lost partitions then ignite should allow the users to use the cache / I was trying to come up with a solution for the above scenario and tried to use "TopologyValidator" & "EVT_CACHE_REBALANCE_PART_DATA_LOST" as suggested above. But I ran into chicken and egg problem. I was able to capture data loss event and reload data on a data loss event. I implemented "TopologyValidator" to check the row count and block clients from read/writing the cache, but with this approach the cache loader is also NOT able to read/write the cache. How to allow cache loader to write to cache, but blocks other users from using the cache. Wonder if anyone has run into this situation and how you solved. Or is there any cleaner way to solve this issue. Appreciate your inputs. Raja -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-loss-validation-TopologyValidator-tp15147p15833.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Streaming data from concurrently from multiple nodes/data streamers - hangs
Is it right to use data streamers from multiple nodes concurrently to ingest into the same cache? If I ingest data from a single node and multiple threads it works just fine. But when I ingests from multiple nodes it hangs (I get Unable to exchange partitions error) Any thoughts? Thank you Raja -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Streaming-data-from-concurrently-from-multiple-nodes-data-streamers-hangs-tp15735.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Data loss validation. TopologyValidator
Thank you very much for the response. While implementing Topology Validator, I ran into the following issue. I wasn't able to get/inject ignite instance. Is there a way to inject/get hold of 'ignite' resource/instance within the Topology Validator? Thank you Raja -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-loss-validation-TopologyValidator-tp15147p15211.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Data loss validation. TopologyValidator
Hi, Whats the best way to check to see if the cache data is lost (partially) ? I was looking at 'Topology Validator' as an option. Scenario : If there any multiple node failures occurred at the same time and ignite loses primary and as well as backup copies of partitions and how to know if this happened and data is lost? Once you detect the data loss, how to restrict/reject clients and because the cache is partial and not complete. Appreciate your inputs how to handle such situations in Ignite. Thanking you Raja -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-loss-validation-TopologyValidator-tp15147.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: KeySize for each partition
Thank you very much Andrey -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/KeySize-for-each-partition-tp14783p14818.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
KeySize for each partition
Hi, Wonder if there is anyway in ignite to find what is the key-size for each partition? Thank you Raja -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/KeySize-for-each-partition-tp14783.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Updating a PART(one field) of the "value" object at scale
CacheConfiguration cfg = new CacheConfiguration(); cfg.setName(DATUM_CACHE); cfg.setIndexedTypes(Long.class, Datum.class); cfg.setCacheMode(CacheMode.PARTITIONED); cfg.setBackups(1); cfg.setAtomicityMode(CacheAtomicityMode.ATOMIC); cfg.setRebalanceMode(CacheRebalanceMode.ASYNC); cfg.setRebalanceBatchSize(2 * 1024 * 1024); cfg.setRebalanceThrottle(100); cfg.setRebalanceTimeout(20); *Problem Context:* --- The Datum object has 10 fields in it and cache contains 60 million entries. Every minute 1 field of Datum object gets updated. Instead of bringing the whole 'Datum" object and updating that one filed, wanted to SEND that update to the node which hosts that entry. The above cache configuration is used I'm doing that with the below code *IgniteCompute igniteCompute = ignite.compute().withAsync().withTimeout(1); igniteCompute.affinityRun(DATUM_CACHE, datumId, new DatumCacheUpdater(datumId, value));* It works, but slow. At 6 million per minute, the ignite cluster becomes slow. My question is there a better approach to the above problem and at scale we should be able to update a part of the object ? Appreciate your input into this. Thank you Raja -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Updating-a-PART-one-field-of-the-value-object-at-scale-tp13534.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Run Ignite on a separate Yarn Queue instead of default queue
Thank you -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Run-Ignite-on-a-separate-Yarn-Queue-instead-of-default-queue-tp12046p12315.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Run Ignite on a separate Yarn Queue instead of default queue
Hi, Is there any option/property to run Ignite on a separate yarn queue instead of a 'default' queue? This will allow YARN not to touch Ignite containers or else Yarn can preempt Ignite nodes if it runs short of resources. Thank you Raja -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Run-Ignite-on-a-separate-Yarn-Queue-instead-of-default-queue-tp12046.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Local node SEGMENTED. Data Lost on a Partitioned Cache with backups of 3
Yes we fill the cache manually after a cluster is started. We load data to Ignite for use it for ultra fast access and fast updates (millions of updates/minute) where traditional databases cannot handle. But we cannot afford to lose data on the cluster. if we lose a portion of data, the rest of the cluster is useless because of data inconsistency We would like to configure our cluster in such a way that its resilient to node failures. We cannot afford to lose data on the cluster. Appreciate if you can recommend the kind of configuration that needs to be used for such kind of resilient behavior. Thanking you Raja -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Local-node-SEGMENTED-Data-Lost-on-a-Partitioned-Cache-with-backups-of-3-tp11683p11726.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Local node SEGMENTED. Data Lost on a Partitioned Cache with backups of 3
Hi Andrew, Thanks for the reply. Here is the configuration that I use for the cluster. Because we have are maintaining 3 data copies in the cluster, if data loading from node is failed then it should be able to get the data from other node. isn't it? Thanks in advance for your support Raja http://www.springframework.org/schema/beans"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd";> list of static IPs -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Local-node-SEGMENTED-Data-Lost-on-a-Partitioned-Cache-with-backups-of-3-tp11683p11689.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Local node SEGMENTED. Data Lost on a Partitioned Cache with backups of 3
Hi, I have a Ignite cluster deployed on Hadoop/Yarn with cache configuration as follow Cache Mode : Partitioned Backups : 3 Default Segmentation behavior. Two nodes went down with "Local node SEGMENTED" error around same time at 7:00 AM and the another node failed with the same error around 15:00 PM. This resulted in data loss. We lost portion of data in cache. Technically we didn't had more than 2 node failures at a given time and we still lost data. *Is this possible to happen?* BTW, I found this warning in log...might be related to segmentation. Thought it will be useful in analysis. / *[main] WARN org.apache.ignite.spi.checkpoint.noop.NoopCheckpointSpi - Checkpoints are disabled (to enable configure any GridCheckpointSpi implementation)/* Appreciate any insight into this. Thanking you Raja -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Local-node-SEGMENTED-Data-Lost-on-a-Partitioned-Cache-with-backups-of-3-tp11683.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Ignite Client Auto Reconnect after a Ignite cluster restart
Hi, I'm creating a client connection (client node) from a web app (tomcat) as follows ignite = Ignition.start(IGNITE_CACHE_XML); Everything is great and it works forever without any issues. But, Once, I restart the ignite cluster, the client node is disconnected and stops serving any future requests. Is it possible for client node to auto connect and be available after cluster is started and available? Thank you Raja -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Client-Auto-Reconnect-after-a-Ignite-cluster-restart-tp11586.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Ignite Yarn Deployment
Hi, I have deployed Ignite on a Hadoop Yarn cluster and I wanted to pass a configuration file as part of the deployment so that each node picks that config file. say for example I want to pass a 'log4j.properties' How do I upload/deploy config files as part of Ignite/Yarn deployment Thanking you Raja -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Yarn-Deployment-tp11585.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite SQL Grid Load
Andrew, Figured out the issue. We created a user defined spatial function and in it we were using an object that's not thread safe. That created an issue. Thank you very much Raja -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-SQL-Grid-Load-tp11072p11086.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Ignite SQL Grid Load
Hi, I have a 12 node ignite cluster with node configuration of 8 core, 10g memory I am using the following JVM options as recommended by ignite docs IGNITE_JVM_OPTS=-server \ -Xms8g \ -Xmx8g \ -XX:+UseParNewGC \ -XX:+UseConcMarkSweepGC \ -XX:+UseTLAB \ -XX:NewSize=128m \ -XX:MaxNewSize=128m \ -XX:MaxTenuringThreshold=0 \ -XX:SurvivorRatio=1024 \ -XX:+UseCMSInitiatingOccupancyOnly \ -XX:CMSInitiatingOccupancyFraction=40 \ -XX:MaxGCPauseMillis=1000 \ -XX:InitiatingHeapOccupancyPercent=50 \ -XX:+UseCompressedOops \ -XX:ParallelGCThreads=8 \ -XX:ConcGCThreads=8 \ -XX:+DisableExplicitGC When I tried to load test ONE REST client, it just cannot handle more than 1 thread. If I hit it with 2 or more threads, some sql queries randomly fails. Even after increasing cluster to 40 nodes. No difference. Just if hit with more than 1 thread everything falls apart. Wondering if I'm missing something. We wanted to have cluster with a configuration where 50 or more concurrent users will be hitting Ignite. Is it possible with Ignite. BTW we are running geo-spatial sql queries on ignite. Appreciate any insights into this. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-SQL-Grid-Load-tp11072.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Ignite client connections hanged
IgniteCluster.jpg <http://apache-ignite-users.70518.x6.nabble.com/file/n10561/IgniteCluster.jpg> <http://apache-ignite-users.70518.x6.nabble.com/file/n10561/IgniteCluster.jpg> Hi, I have Ignite and Web applications deployment as shown in the attached image. I added couple of new functions to the ignite and restarted Ignite cluster with new functions added. I updated correspondingly in all other web applications except "WebApp1" (as shown in picture), but all web apps are deployed in the same tomcat container (same JVM). The WebApp1 throwed exceptions because of compatibility issues and the ignite connection is broken. Though the other web apps are having the latest code, but the connections seem to be hanged forever. Wondering if any of experienced this type of behavior and appreciate any insight into this. Thank you Raja -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-client-connections-hanged-tp10561.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Geo Spatial queries
Sergi, Thanks again. The approach you suggested worked like a charm. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Geo-Spatial-queries-tp10291p10364.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Geo Spatial queries
Sergi, Wonderful. Thanks for pointing me in the right direction. I wasn't aware of QuerySqlFunction. Thanks again. Will try this approach and will reply with my findings. Raja -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Geo-Spatial-queries-tp10291p10313.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Geo Spatial queries
Hi, I have been using Ignite to store Geo Spatial data, index it and query it. Till now my need was only to query the data by a bounding box and Ignite has been great for that type of need. Please see the link below for example of bounding box query https://www.gridgain.com/resources/blog/geospatial-queries-apache-ignite <https://www.gridgain.com/resources/blog/geospatial-queries-apache-ignite> Now I want to query the spatial data within a given radius of a geometry and many other ways Please see below for functions/methods PostGIS supports out of the box to query spatial data http://postgis.net/docs/reference.html <http://postgis.net/docs/reference.html> Wonder if ignite supports other ways of querying spatial data other than by bounding box. Appreciate if you can provide me a link to documentation or samples which helps to understand different ways to query spatial data. Thanking you Raja -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Geo-Spatial-queries-tp10291.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
