RE: 'Deploy' ignite cluster into a spark cluster

Hi Stan, 'executor die, that leads ignite cluster rebalance', I haven't expected this before. Our solution is strange, it's a bit like borrowing spark job scheduling API to deploy ignite and other platforms, so, how to persuade them work together is the major consideration, below is my plan, will

RE: 'Deploy' ignite cluster into a spark cluster

Hi Stan, Thank you for providing the information. Yes, it's exact 'embedded deployment', but it looks like to be deprecated because of performance issues. Stanislav Lukyanov wrote > Embedded Mode Deprecation > Embedded mode implies starting Ignite server nodes within Spark executors > which can

'Deploy' ignite cluster into a spark cluster

When I implement a Spark streaming project, I involuntarily found Ignite could be easily hacked into spark yarn cluster and work on the spark datanodes as an additional ignite cluster(use spark cluster resources, RAM,core without complex deployment), and all data process could be done with

Re: how to organize groups in cluster?

Hi Andrew, Thank you for the response. 1. yes, I will try this way to build sub caches 2. yes, make sense. 3. I tested EvictionPolicy, it seemed to be not available in ignite 2.0+, please refer to my earlier discuss (with denis)

Re: how to organize groups in cluster?

Hi Val, Here's the background: I'm thinking about creating message pipelines right in ignite cluster. it's a nightmare to save data in a centralized cache and run everything on it, so my whole idea is to guarantee *none-blocking process* and scalable consumers. 1. (producer) put real-time

Re: how to organize groups in cluster?

Hi Val, 1M per day in memory sales data, 100 parallel financial aggregations in each batch plus additional PUB/SUB requests for both instant query and server push. It has been confirmed to play group count distinct is way too slow, so we decided to play some of TOP N query in separate domain, for

how to organize groups in cluster?

assume we a ignite cluster of nodes A,B,C, is it possible to create partion cache only on node A and node B, and use topic message to replicate data to node C for real-time query? if not, is it possible to create 2 independant clusters, cluster1 for node A+node B, cluster2 for node C, and let

Re: Slow Group-By

hi mike, you can create another cache with key equals (idnumber + value), redirect your streamer to it, and then feel free to check cache size. it could waste your RAM, but fast enough for any parallel queries, because it's not map/reduce at all. we used this weird k/v trick in many 'ignite

Re: SortedEvictionPolicy doesn't work as expected

l just get > lower access time for some records, and higher memory consumption. > > And eviction policy is not determining, which entries are stored in the > cache, but the entries, which have lower access time from local nodes. > > Denis > > чт, 1 мар. 2018 г. в 1:36, mama

Re: SortedEvictionPolicy doesn't work as expected

Hi Denis, Thank you for the response, I appreciate it. Yes, I agree with you, it could be done by various solutions, 'REST', 'Event Listener' or any standalone instance. According to the new design you mentioned, ignite stop the support of '/setMemoryMode(CacheMemoryMode.ONHEAP_TIERED)/' use

Re: SortedEvictionPolicy doesn't work as expected

After reading valentin's example and a user discussion

Re: SortedEvictionPolicy doesn't work as expected

Hi Denis, Thank you for the explanation, so the cause is about Ignite use both of on-heap and off-heap to store the entries and unfortunately I read the everything from off-heap, so question #1 is: *how could I read the on-heap entries directly?* *My use case* is to get Top 100 entries by sorting

SortedEvictionPolicy doesn't work as expected

Here I have a five entries as below: (14,4); (21,1); (32,2); (113,3); (15,5); and I want to use SortedEvictionPolicy to keep below 3 entries (sort by values in descending order and get top 3 items): 15-->5 14-->4 113-->3 The actual output is: 21-->1 32-->2 113-->3 14-->4 15-->5 issue 1: The

Option meta schema in cache level?

I'm trying to use Binary Marshaller to replace old OptimizedMarshaller which is really a pain for deployment. But according to document of Type Metadata, it could be changed at runtime, that means if I access to a 3rd party cache which was created by someone else without explicit type settings,

Question about 'table' created by JDBC

Hi, I made a quick test to create a City table through DBeaver, and then I accessed it through java app successfully. But, when I loop the record, I found both of key and value were built by BinaryObject with strange type. Question 1: Is there any convenient API to get the type name from cache

Re: Invoke Cache.loadCache method in configuration file?

Yeah, LifeCycleBean is what exactly what I'm looking for, the solution is brilliant, thank you. package com.debug; public class CustomizedLifeCycleBean implements LifecycleBean{ @Override public void

Invoke Cache.loadCache method in configuration file?

Hi, I'm testing the persist store function, it works correctly, but I had to call the loadCache function manually in a Java app. can I realize this in configuration file? this way might save a lot of time. Thank you. http://www..>