Re: What are typical load time for millions of records

2016-12-21 Thread steve.hostettler
Hi, I am using the partition capability with multiple threads to improve the data loading, but referring to my other post : I see that the performance are not constant with respect to the size of the cache. Regards -- View this message in context:

Re: What are typical load time for millions of records

2016-12-20 Thread vkulichenko
Hi, You're right, this heavily depends on a lot of factors and it's almost impossible to tell if these numbers are good or bad. For large data sets you can try partition aware data loading [1]. It can give performance improvement and also is pretty scalable. I.e. you can make it even better by

What are typical load time for millions of records

2016-12-20 Thread steve.hostettler
I'm currently trying to improve the load time. I would like to have some feedbacks from other Ignite users as for the actual load performances. In my setup, I have a DB in the same subnet with an Ignite node with 8 cores. It loads 20,000,000 records in 1h with the standard loadCache