Re: Data not loading with Ignite DataStreamer

[Aiden]

Hi,

Hello, i tried putting allowOverwrite=true. And yet i'm getting the same
exception as mentioned above. As for writeBehind and writeThrough,
writeBehind doesn't work without writeThrough being enabled. Atleast when i
tested it like that, any update in cache was not being reflected in the
db(This was when i was using loadCache() to load data from db to cache).
Still thanks for your suggestion. Do let me know if u come up with something
else, because i'm totally stuck here.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Data-not-loading-with-Ignite-DataStreamer-tp14589p14754.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Data not loading with Ignite DataStreamer

[Aiden]

Hello Humphrey,

I don't think that is the issue here. Still i checked with ur suggestion,
but the same exception is coming in the server logs(the exception is shown
in the previous mail).

I'm just not able to understand what i'm doing wrong here. Still thanks for
ur suggestions.


Aiden





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Data-not-loading-with-Ignite-DataStreamer-tp14589p14616.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Data not loading with Ignite DataStreamer

[Aiden]

Hi,

I am trying to load about 40M data from db to my ignite instances(2 in this
case). I read about the Ignite DataStreamers from 
https://apacheignite.readme.io/v1.2/docs/data-streamers
  

After downloading the project obtained from Ignite Web Console, i changed
the loadCache() with IgniteDataStreamer.addData(). 

Upon execution, i notice in the web console that the metrics for on-heap
entries gets incrementing(i.e., data is being loaded). But when i query the
ignite cache, i get the result as empty. 

I understand it is something wrong with how i'm using the
IgniteDataStreamer.

Forwarding my LoadCache and my bean class:

Persons.java
  

LoadCaches.java
  

Here, are some screenshots depicting what i'm saying:

 

 

Can someone please tell me what i'm doing wrong here? I would very much
appreciate it.
  



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Data-not-loading-with-Ignite-DataStreamer-tp14589.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Data not loading with Ignite DataStreamer

[Aiden]

Hi,

I forgot to mention that the following exception is shown in the ignite
server instance logs:

[19:43:20,443][SEVERE][flusher-0-#30%RemoteIgniteCluster%][GridCacheWriteBehindStore]
Unable to update underlying store: CacheJdbcPojoStore []
javax.cache.CacheException: Failed to read property value from non binary
object [class=class java.lang.String, property=class_no]
at
org.apache.ignite.cache.store.jdbc.CacheJdbcPojoStore.extractBinaryParameter(CacheJdbcPojoStore.java:122)
at
org.apache.ignite.cache.store.jdbc.CacheJdbcPojoStore.extractParameter(CacheJdbcPojoStore.java:69)
at
org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.fillValueParameters(CacheAbstractJdbcStore.java:1414)
at
org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.writeAll(CacheAbstractJdbcStore.java:1081)
at
org.apache.ignite.internal.processors.cache.store.GridCacheWriteBehindStore.updateStore(GridCacheWriteBehindStore.java:804)
at
org.apache.ignite.internal.processors.cache.store.GridCacheWriteBehindStore.applyBatch(GridCacheWriteBehindStore.java:720)
at
org.apache.ignite.internal.processors.cache.store.GridCacheWriteBehindStore.access$2400(GridCacheWriteBehindStore.java:75)
at
org.apache.ignite.internal.processors.cache.store.GridCacheWriteBehindStore$Flusher.flushCacheCoalescing(GridCacheWriteBehindStore.java:1108)
at
org.apache.ignite.internal.processors.cache.store.GridCacheWriteBehindStore$Flusher.body(GridCacheWriteBehindStore.java:1006)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Unknown Source)


Also forwarding the server.xml:

ser.xml <http://apache-ignite-users.70518.x6.nabble.com/file/n14591/ser.xml>  

Thanks,

Aiden


Aiden wrote
> Hi,
> 
> I am trying to load about 40M data from db to my ignite instances(2 in
> this case). I read about the Ignite DataStreamers from 
> https://apacheignite.readme.io/v1.2/docs/data-streamers
> <https://apacheignite.readme.io/v1.2/docs/data-streamers>  
> 
> After downloading the project obtained from Ignite Web Console, i changed
> the loadCache() with IgniteDataStreamer.addData(). 
> 
> Upon execution, i notice in the web console that the metrics for on-heap
> entries gets incrementing(i.e., data is being loaded). But when i query
> the ignite cache, i get the result as empty. 
> 
> I understand it is something wrong with how i'm using the
> IgniteDataStreamer.
> 
> Forwarding my LoadCache and my bean class:
> Persons.java
> <http://apache-ignite-users.70518.x6.nabble.com/file/n14589/Persons.java>  
> 
> LoadCaches.java
> <http://apache-ignite-users.70518.x6.nabble.com/file/n14589/LoadCaches.java>  
> 
> Here, are some screenshots depicting what i'm saying:
<http://apache-ignite-users.70518.x6.nabble.com/file/n14589/1.png> 
> 
<http://apache-ignite-users.70518.x6.nabble.com/file/n14589/2.png> 
> 
> Can someone please tell me what i'm doing wrong here? I would very much
> appreciate it.





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Data-not-loading-with-Ignite-DataStreamer-tp14589p14591.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Loading similar data to specific instance and not all instances

[Aiden]

Hello,

The use-case is simply this: I'm trying to load data having similar
properties, in my case, records ending with same number, say 0, into a
single instance of ignite and not multiple instances. 

Basically, if similar data is stored in a specific instance, then when i'm
trying to query for that record, the query fetching time will be greatly
reduced.  

Let me give an example, say if following records are present:

 ROLL_NO
34560
12340
34439
45321
32359
43541
53250
34220

Say there are 3 instances A,B,C:

For instance, ALL Records ending with 0 should be loaded to either A or B or
C(irrespective). But not like 1 record ending with 0 in A another in B in
that manner. I believe this can be done using AffinityKeyMapping

Thanks,
Aiden 




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Loading-similar-data-to-specific-instance-and-not-all-instances-tp14387p14543.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Loading similar data to specific instance and not all instances

[Aiden]

Hello all,

I'm totally a newbie here, so just bear with me. I followed through this
link:
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/cache/affinity/AffinityKeyMapper.java

  

I tried to apply the same procedure as prescribed. Unfortunately due to some
error from my side i'm just not able to create this affinity key mapping.

I'm forwarding my BEAN Class and my server.xml(generated using ignite web
console).

Persons.java
  
RemoteIgniteCluster-server.xml

  

Age
*
50
60
70
80
90
100
10
20
30
40
110


As u can see, Age is 1 column in the database table that i'm trying to load
to the ignite cache partition. Every value in Age ends with 0. I have two
ignite instances A, B. When i'm loading the above table to ignite partition.
It should have to load to either A or B completely and NOT BOTH. 

I understand for this, AffinityKeyMapper can be used. But i'm not being able
to understand in what way. 

Can somebody please take time to tell me what all changes i have to do in my
in my bean class and my server.xml to bring about this change. I'd
appreciate it a lot.

Thanks in advance..



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Loading-similar-data-to-specific-instance-and-not-all-instances-tp14387p14471.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Loading similar data to specific instance and not all instances

[Aiden]

So i'm really new to apache ignite here. What i'm trying to do is load data
having similar properties to a single rather than it being loaded to random
instances. For example, say that some data of this form:

ROLL_NO
34569
12349
34439
45329
32359
43549
53259
34229

As u can see, the above data is all ending with 9. Say that i have two
ignite instances A and B currently running. **Is there any way i can load
these data ending with 9 to either of the instance A or B and NOT BOTH.**

Please let me know if this is possible and if so how to accomplish this.
Thanks in advance.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Loading-similar-data-to-specific-instance-and-not-all-instances-tp14387.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

How to create a dump of updated data in cache into local

[Aiden]

Hi,

I'm really new to ignite, so just bear with me here. I have integrated my
Mysql db with the ignite by making use of the ignite web console and the
ignite web agent. I am being able to perform sql update operations on the
data i've loaded to my ignite caches.

Now what i require is a way to create a backup of the entire data in the
cache into my local harddisk(possibly in a comma seperated file format).
OR
Even some kind of logger showcasing "this 'record' got updated to that
'record'" would even be fine.

What i basically require is some way to identify which all are the updated
records from the already existing records in the db.So some kind of a
snapshot of the updated records would be helpful for that.

Is there any configuration or strategy that can be done to attain this? All
help would be very much appreciated



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/How-to-create-a-dump-of-updated-data-in-cache-into-local-tp14268.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.