Re: Data Load from Oracle to Ignite is very slow

2020-02-20 Thread nithin91
Hi When i executed on the server , Data is getting loaded very fast.Thanks for the inputs. With respect to Data Steamer, it would be really helpful if you can share any sample code other than the one provided in documentation. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Data Load from Oracle to Ignite is very slow

2020-02-19 Thread Ilya Kasnacheev
Hello! There's no way you will load 20,000 records in 25 minutes. That's 10 records per second. I just can't think of any reason why it might take such monumental amount of time. With regards to data streamer, as I have said I recommend partitioning your data and loading every segment from its ow

Re: Data Load from Oracle to Ignite is very slow

2020-02-18 Thread nithin91
Hi We are doing POC, as a result of which we are running it in local mode. Currently it is taking 25min to load 2 records with Cache JDBC POJO Store. Even i am giving the initial filter to reduce unnecessary records. ignite.cache("PieProductRiskCache").loadCache(null,"ignite.e

Re: Data Load from Oracle to Ignite is very slow

2020-02-18 Thread Ilya Kasnacheev
Hello! The code that you have provided will not have any edge over CacheStore. I'm not sure that running two nodes on the same machine is a sound approach for benchmarking data load speed. Are you sure that performance is not sufficient? What is actual run time and your expectations? Regards, --

Re: Data Load from Oracle to Ignite is very slow

2020-02-18 Thread nithin91
I am having two nodes running on local machine. Following is the logic i implemented to load data using Data Steamer.Can you please check whether the implementation is correct and also can you please share sample code on how to push entries to data streamer from multiple threads public class Per

Re: Data Load from Oracle to Ignite is very slow

2020-02-18 Thread Ilya Kasnacheev
Hello! Load Cache will pull all rows from result set on all nodes. How many nodes do you have? 10 million rows is actually a very modest number. I could understand you worrying if you had 10B rows. The best approach is to partition your table and push entries to data streamer from multiple threa

Data Load from Oracle to Ignite is very slow

2020-02-18 Thread nithin91
Hi, I have multiple oracle tables with more than 10 million rows. I want to load these tables into Ignite cache.To load the cache I am using Cache JDBC Pojo Store by getting the required project structure from Web Console. But Loading the data using cache JDBC POJO Store (i.e. ignite.cache("Cache