> 295 million rows > 3 min 35 sec Agree with Ilya, DataStreamer should do this much faster, have you tried it?
> 3.7Gb I would not call this "big" by any means today, when even the cheapest laptops have 8GB of RAM. On Fri, Feb 19, 2021 at 1:33 PM Ilya Kasnacheev <[email protected]> wrote: > Hello! > > Is there a chance that you have tried enabling streaming (data streamer) > on the clients? > > Regards, > -- > Ilya Kasnacheev > > > пт, 19 февр. 2021 г. в 10:10, <[email protected]>: > >> Hi Denis, >> >> Data space is 3.7Gb according to MSSQL table properries >> >> Vladimir >> >> 9:47, 19 февраля 2021 г., Denis Magda <[email protected]>: >> >> Hello Vladimir, >> >> Good to hear from you! How much is that in gigabytes? >> >> - >> Denis >> >> >> On Thu, Feb 18, 2021 at 10:06 PM <[email protected]> wrote: >> >> Sep 2020 I've published the paper about Loading Large Datasets into >> Apache Ignite by Using a Key-Value API (English [1] and Russian [2] >> version). The approach described works in production, but shows >> inacceptable perfomance for very large tables. >> >> The story continues, and yesterday I've finished the proof of concept for >> very fast loading of very big table. The partitioned MSSQL table about 295 >> million rows was loaded by the 4-node Ignite cluster in 3 min 35 sec. Each >> node had executed its own SQL queries in parallel and then distributed the >> loaded values across the other cluster nodes. >> >> Probably that result will be of interest for the community. >> >> Regards, >> Vladimir Chernyi >> >> [1] >> https://www.gridgain.com/resources/blog/how-fast-load-large-datasets-apache-ignite-using-key-value-api >> [2] https://m.habr.com/ru/post/526708/ >> >> >> >> -- >> Отправлено из мобильного приложения Яндекс.Почты >> >
