Re: Slow data loading and high(very) memory usage issues

2016-05-06 Thread oskoblya
Thank's Denis it's very helpful and i hope really helps to solve data loading performance issues, but what about increased memory usage: ~6 GB became 27,4 GB in off-heap cache storage or all we can do in this case is to increase physical memory capacity? -- View this message in context:

Slow data loading and high(very) memory usage issues

2016-05-05 Thread oskoblya
Hi, I’m trying to load tSV file size ~six gigabytes (contains ~90 million records) into single Apache Ignite node with allocated heap size = 40 gigabytes. For loading used IgniteDataStreamer. Code and sample (10 records) of data is here: https://github.com/olegskoblya/traffic-loader.git Physical