Re: Data not loading with Ignite DataStreamer
Hi, Hello, i tried putting allowOverwrite=true. And yet i'm getting the same exception as mentioned above. As for writeBehind and writeThrough, writeBehind doesn't work without writeThrough being enabled. Atleast when i tested it like that, any update in cache was not being reflected in the db(This was when i was using loadCache() to load data from db to cache). Still thanks for your suggestion. Do let me know if u come up with something else, because i'm totally stuck here. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-not-loading-with-Ignite-DataStreamer-tp14589p14754.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Data not loading with Ignite DataStreamer
Hi, answered you on SO: https://stackoverflow.com/questions/45029434/ignite-datastreamer-not-loading-data/45075192#45075192 Evgenii 2017-07-11 9:32 GMT+03:00 Aiden <1994hsekar1...@gmail.com>: > Hello Humphrey, > > I don't think that is the issue here. Still i checked with ur suggestion, > but the same exception is coming in the server logs(the exception is shown > in the previous mail). > > I'm just not able to understand what i'm doing wrong here. Still thanks for > ur suggestions. > > > Aiden > > > > > > -- > View this message in context: http://apache-ignite-users. > 70518.x6.nabble.com/Data-not-loading-with-Ignite- > DataStreamer-tp14589p14616.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com. >
Re: Data not loading with Ignite DataStreamer
Hello Humphrey, I don't think that is the issue here. Still i checked with ur suggestion, but the same exception is coming in the server logs(the exception is shown in the previous mail). I'm just not able to understand what i'm doing wrong here. Still thanks for ur suggestions. Aiden -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-not-loading-with-Ignite-DataStreamer-tp14589p14616.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Data not loading with Ignite DataStreamer
Have you tried to set the flush frequency? stmr.autoFlushFrequency(1000); Sets automatic flush frequency. Essentially, this is the time after which the streamer will make an attempt to submit all data added so far to remote nodes. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-not-loading-with-Ignite-DataStreamer-tp14589p14593.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Data not loading with Ignite DataStreamer
Hi, I am trying to load about 40M data from db to my ignite instances(2 in this case). I read about the Ignite DataStreamers from https://apacheignite.readme.io/v1.2/docs/data-streamers <https://apacheignite.readme.io/v1.2/docs/data-streamers> After downloading the project obtained from Ignite Web Console, i changed the loadCache() with IgniteDataStreamer.addData(). Upon execution, i notice in the web console that the metrics for on-heap entries gets incrementing(i.e., data is being loaded). But when i query the ignite cache, i get the result as empty. I understand it is something wrong with how i'm using the IgniteDataStreamer. Forwarding my LoadCache and my bean class: Persons.java <http://apache-ignite-users.70518.x6.nabble.com/file/n14589/Persons.java> LoadCaches.java <http://apache-ignite-users.70518.x6.nabble.com/file/n14589/LoadCaches.java> Here, are some screenshots depicting what i'm saying: <http://apache-ignite-users.70518.x6.nabble.com/file/n14589/1.png> <http://apache-ignite-users.70518.x6.nabble.com/file/n14589/2.png> Can someone please tell me what i'm doing wrong here? I would very much appreciate it. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-not-loading-with-Ignite-DataStreamer-tp14589.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Data not loading with Ignite DataStreamer
Hi, I forgot to mention that the following exception is shown in the ignite server instance logs: [19:43:20,443][SEVERE][flusher-0-#30%RemoteIgniteCluster%][GridCacheWriteBehindStore] Unable to update underlying store: CacheJdbcPojoStore [] javax.cache.CacheException: Failed to read property value from non binary object [class=class java.lang.String, property=class_no] at org.apache.ignite.cache.store.jdbc.CacheJdbcPojoStore.extractBinaryParameter(CacheJdbcPojoStore.java:122) at org.apache.ignite.cache.store.jdbc.CacheJdbcPojoStore.extractParameter(CacheJdbcPojoStore.java:69) at org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.fillValueParameters(CacheAbstractJdbcStore.java:1414) at org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.writeAll(CacheAbstractJdbcStore.java:1081) at org.apache.ignite.internal.processors.cache.store.GridCacheWriteBehindStore.updateStore(GridCacheWriteBehindStore.java:804) at org.apache.ignite.internal.processors.cache.store.GridCacheWriteBehindStore.applyBatch(GridCacheWriteBehindStore.java:720) at org.apache.ignite.internal.processors.cache.store.GridCacheWriteBehindStore.access$2400(GridCacheWriteBehindStore.java:75) at org.apache.ignite.internal.processors.cache.store.GridCacheWriteBehindStore$Flusher.flushCacheCoalescing(GridCacheWriteBehindStore.java:1108) at org.apache.ignite.internal.processors.cache.store.GridCacheWriteBehindStore$Flusher.body(GridCacheWriteBehindStore.java:1006) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Unknown Source) Also forwarding the server.xml: ser.xml <http://apache-ignite-users.70518.x6.nabble.com/file/n14591/ser.xml> Thanks, Aiden Aiden wrote > Hi, > > I am trying to load about 40M data from db to my ignite instances(2 in > this case). I read about the Ignite DataStreamers from > https://apacheignite.readme.io/v1.2/docs/data-streamers > <https://apacheignite.readme.io/v1.2/docs/data-streamers> > > After downloading the project obtained from Ignite Web Console, i changed > the loadCache() with IgniteDataStreamer.addData(). > > Upon execution, i notice in the web console that the metrics for on-heap > entries gets incrementing(i.e., data is being loaded). But when i query > the ignite cache, i get the result as empty. > > I understand it is something wrong with how i'm using the > IgniteDataStreamer. > > Forwarding my LoadCache and my bean class: > Persons.java > <http://apache-ignite-users.70518.x6.nabble.com/file/n14589/Persons.java> > > LoadCaches.java > <http://apache-ignite-users.70518.x6.nabble.com/file/n14589/LoadCaches.java> > > Here, are some screenshots depicting what i'm saying: <http://apache-ignite-users.70518.x6.nabble.com/file/n14589/1.png> > <http://apache-ignite-users.70518.x6.nabble.com/file/n14589/2.png> > > Can someone please tell me what i'm doing wrong here? I would very much > appreciate it. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-not-loading-with-Ignite-DataStreamer-tp14589p14591.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
