Thanks Mikael. I did come across that link before, but I am not sure it addresses my concern. I want to see how I need I size my physical VMs based on affinity keys. How would I say for India affinity key use this super size VM and for others use the other smaller ones, so the data doesn't get shuffled around? Maybe, there is no way, and I just have to wait for ignite to rebalance the partitions and fit things where they should be based on the affinity key.
On Wed, Jan 2, 2019 at 8:32 AM Mikael <mikael-arons...@telia.com> wrote: > You can find some information about capacity planning here: > > https://apacheignite.readme.io/docs/capacity-planning > > About your India example you can use affinity keys to keep data together > in groups to avoid network traffic. > > https://apacheignite.readme.io/docs/affinity-collocation > > Mikael > Den 2019-01-02 kl. 14:44, skrev Clay Teahouse: > > Thanks Naveen. > > -- Cache Groups: When would I start considering cache groups, if my system > is growing, and sooner or later I will have to add to my caches and I need > to know 1) should I starting grouping now (I'd think yes), 2) if no, when, > what number of caches? > -- Capacity Planning: So, there is no guidelines on how to size the nodes > and the physical storage nodes reside on? How do I make sure all the > related data fit the same VM? It can't be the case that I have to come up > with 100s of super size VMs just because I have one instance with a huge > set of entries. For example, if I have millions of entries for India and > only a few for other countries, how do I make sure all the India related > data fits the same VM (to avoid the network) and have the data for all the > small countries fit on the same VM? > -- Pinning the data to cache: the data pinned to on-heap cache does not > get evicted from the memory? I want to see if there is something similar to > Oracle's memory pinning. > -- Read through: How do I know if something on cache or disk (using native > persistence)? > 5) Service chaining: Is there an example of service chaining that you can > point me to? > > 6) How do I implement service pipelining in apache ignite? Would > continuous query be the mechanism? Any examples? > > 7) Streaming: Are there examples on how to define watermarks, i.e., input > completeness with regard to the event timestamp? > > thank you > Clay > > On Tue, Jan 1, 2019 at 11:29 PM Naveen <naveen.band...@gmail.com> wrote: > >> Hello >> Couple of things I would like to with my experience >> >> 1. Cache Groups : Around 100 caches, I do not think we need to go for >> Cache >> groups, as you mentioned cache groups will have impact on you read/writes. >> However, changing the partition count to 128 from default 1024 would >> improve >> your cluster restart. >> >> 2. I doubt if Ignite has any settings we have for this. >> >> 3. The only I can think of is to keep the data in on-heap if the data size >> is not so huge. >> >> 4. Read through, with native persistence enabled, doing a read to the disk >> will load the cache. But the read is much slower compared with read from >> RAM, by default it does not pre-load the data. If you want to avoid this >> you >> can pre-load the data programatically and load Memory, good for even SQL >> SELECT as well. But with the 3rd party persistence, we need to pre-load >> the >> data to make your read work for SQL SELECT. >> >> Thanks >> Naveen >> >> >> >> -- >> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >> >
