Yep, that sounds right. The primary ways to load data into Ignite from another source are a custom cache store or a data streamer.
Data streamer will mostly outperform cache stores for initial/periodic data loading. Cache stores are good when you need to sync data between Ignite and backing DB (read-through/write-through/write-behind), and when you want to quickly start using Ignite on top of an existing DB with minimal code. WebConsole’s generated project isn’t supposed to be an optimal solution, it’s supposed to be an easy one that allows one to start quickly. When you start looking into what’s really going on inside the cache store, you can see that there is a lot of things how it could be optimized but it’s all depends on the exact model/data. For example, you need to somehow split your queries so that each node only loads a entries which it will be storing (i.e. use affinity key in the queries to the backing DB), and then split them more to allow loading with multiple threads per node – you can do this in a generic way. It’s probably possible to find out what’s exactly going on with the cache store in your case and tune it, but a completely custom cache store, or just a small client with data streamer running near the DB is likely to be perform better anyway. Stan From: wt Sent: 16 сентября 2018 г. 2:12 To: user@ignite.apache.org Subject: Re: webconsole generated project - cache load performance I think I know what I need to do. I need to override the loadcache method and implement my own based on the example on the site. I think if I use generics I could parse in any class and it will handle the load. What's your thoughts on that? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
