Since there are several approaches on solving this, I’m continue with your code, a couple of suggestions:
- Create streamer instance out of loop, and close it on finally, not within the loop. - stmr.autoflushfrequency(0) as you do it every 2000 elements… - Don’t forget remaining data (< 2000) from last iteration IgniteDataStreamer<Long, PieCountryAllocation> stmr = ignite.dataStreamer("PieCountryAllocationCache"); stmr.allowOverwrite(true); // disable auto flush - we’ll do it manually stmr.autoflushfrequency(0); try{ int j=0; for (Map.Entry<Long,PieCountryAllocation> entry : PieCountryAllocationobjs.entrySet()) { tempobjs.put(entry.getKey(), entry.getValue()); //For ever 2000 rows i am callling stmr.addData(tempobjs) and then stmr.flush and stmr.close(false) if(j++ == 2000 ){ System.out.println(j); stmr.addData(tempobjs); // do flush every 2000 items stmr.flush(); tempobjs.clear(); System.out.println(“Sent Ended"); System.out.println(j); j = 0; } } // stream remaining data if (!tempobjs.is <http://tempobjs.is/>Empty()){ stmr.addData(tempobjs); } } finally { stmr.flush(); stmr.close(false); } > El 18 may 2020, a las 12:36, nithin91 > <nithinbharadwaj.govindar...@franklintempleton.com> escribió: > > int j=0; > for (Map.Entry<Long,PieCountryAllocation> entry : > PieCountryAllocationobjs.entrySet()) { > > tempobjs.put(entry.getKey(), > entry.getValue()); > //For ever 2000 rows i am callling stmr.addData(tempobjs) and then > stmr.flush and stmr.close(false) > if((j%2000==0 && j!=0) || > > (PieCountryAllocationobjs.keySet().size() < 2000 && > j==PieCountryAllocationobjs.keySet().size()) > || > j==PieCountryAllocationobjs.keySet().size() > ){ > System.out.println(j); > IgniteDataStreamer<Long, > PieCountryAllocation> stmr = > ignite.dataStreamer("PieCountryAllocationCache"); > stmr.allowOverwrite(true); > stmr.addData(tempobjs); > stmr.flush(); > stmr.close(false); > tempobjs.clear(); > System.out.println("Stream > Ended"); > System.out.println(j); > > } > j++; > }