Hi Val, looking at the debug logs, I see that the query is executed on only one Ignite server. It seems that this happens when the where clause includes a partitioning column. Query " select * from test_11.fc_port_stats; " loads all objects. Query " select * from test_11.fc_port_stats where sid = 1d6dad9c-6e8d-4425-b3d0-a5e50ee0b845 allow filtering; " also loads all objects. SID has an index in Cassandra. Query " select * from test_11.fc_port_stats where hour = '2017-07-07 15:00:00' and bucket = 0 and date_time >= '2017-07-07 15:05:00' and date_time < '2017-07-07 15:10:00'; " only returns about 1/3 of the objects it should.
Server 2 log has: [17:41:39,532][FINE][mgmt-#77%null%][GridCacheProcessor] Getting cache for name: FcPortStatsCache [17:41:39,532][FINE][mgmt-#77%null%][CacheOsStoreManager] <FcPortStatsCache> Loading all values from store. [17:41:39,536][FINE][mgmt-#77%null%][CassandraCacheStore] Cache loaded from db: FcPortStatsCache [17:41:39,536][FINE][mgmt-#77%null%][CacheOsStoreManager] <FcPortStatsCache> Loaded all values from store. [17:41:39,537][FINE][mgmt-#77%null%][GridJobWorker] Job execution has successfully finished [job=C2 [c=LoadCacheJobV2 [keepBinary=false]], res=null] There are no logs between "Loading ..." and "Loaded ...". The same for server 3: [23:11:39,533][FINE][mgmt-#76%null%][GridCacheProcessor] Getting cache for name: FcPortStatsCache [23:11:39,533][FINE][mgmt-#76%null%][CacheOsStoreManager] <FcPortStatsCache> Loading all values from store. [23:11:39,536][FINE][mgmt-#76%null%][CassandraCacheStore] Cache loaded from db: FcPortStatsCache [23:11:39,537][FINE][mgmt-#76%null%][CacheOsStoreManager] <FcPortStatsCache> Loaded all values from store. [23:11:39,537][FINE][mgmt-#76%null%][GridJobWorker] Job execution has successfully finished [job=C2 [c=LoadCacheJobV2 [keepBinary=false]], res=null] Only server 1 shows actual loading: [17:41:39,539][FINE][mgmt-#78%null%][CacheOsStoreManager] <FcPortStatsCache> Loading all values from store. ... [17:41:39,648][FINE][pool-13-thread-1][GridQueryProcessor] Store [space=FcPortStatsCache, key=com.brocade.poc.icpoc.model.FcPortStatsKey [idHash=1898688509, hash=-1866764483, hour=Fri Jul 07 15:00:00 UTC 2017, bucket=0, dateTime=Fri Jul 07 15:19:00 UTC 2017, portId=002055f9-d8e6-4255-b3b4-b1b6cd6453f4], val=com.brocade.poc.icpoc.model.FcPortStats [idHash=1964258560, hash=-842901133, portId=002055f9-d8e6-4255-b3b4-b1b6cd6453f4, dateTime=Fri Jul 07 15:19:00 UTC 2017, portType=1, switchId=683277c5-da36-47cb-94e8-8e29a28bbc9c, rxUtil=79.01061, txUtil=70.75167, higherUtil=79.01061, lowerUtil=70.75167, rxRate=15127.0, txRate=19382.0, higherRate=19382.0, lowerRate=15127.0, c3Discards=0.0, crcErrors=0.0]] ... [17:41:39,649][FINE][pool-13-thread-1][GridDhtAtomicCache] <FcPortStatsCache> Remove will not be done for key (entry got replaced or removed): com.brocade.poc.icpoc.model.FcPortStatsKey [idHash=1898688509, hash=-1866764483, hour=Fri Jul 07 15:00:00 UTC 2017, bucket=0, dateTime=Fri Jul 07 15:19:00 UTC 2017, portId=002055f9-d8e6-4255-b3b4-b1b6cd6453f4] ... [17:41:39,649][FINE][pool-13-thread-1][GridDhtAtomicCache] <FcPortStatsCache> Ignoring entry for partition that does not belong [key=com.brocade.poc.icpoc.model.FcPortStatsKey [idHash=806790598, hash=984017435, hour=Fri Jul 07 15:00:00 UTC 2017, bucket=0, dateTime=Fri Jul 07 15:19:00 UTC 2017, portId=003d1a6f-d1c4-414c-8fc8-d0b54e127003], val=FcPortStats: 003d1a6f-d1c4-414c-8fc8-d0b54e127003, Fri Jul 07 15:19:00 UTC 2017, 2; bbd80298-fb88-4a1b-99b0-5e9974ed5f9d ..., err=class org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtInvalidPartitionException [part=189, msg=Creating partition which does not belong to local node (often may be caused by inconsistent 'key.hashCode()' implementation) [part=189, topVer=AffinityTopologyVersion [topVer=-1, minorTopVer=0], this.topVer=AffinityTopologyVersion [topVer=4, minorTopVer=1]]]] ... [17:41:40,712][FINE][mgmt-#78%null%][CacheOsStoreManager] <FcPortStatsCache> Loaded all values from store. At this time I don't know if this is related to the difference in the primary key between Cassandra and Ignite. I will try that next. Roger -----Original Message----- From: vkulichenko [mailto:valentin.kuliche...@gmail.com] Sent: Friday, July 28, 2017 11:46 AM To: user@ignite.apache.org Subject: RE: Missing object when loading from Cassandra with multiple queries Roger, Yeah, that's weird and hard to tell why this happens. Obviously there is some kind of race condition. In any case, I would do everything correctly first, and then check if issue persists or not. -Val -- View this message in context: https://urldefense.proofpoint.com/v2/url?u=http-3A__apache-2Dignite-2Dusers.70518.x6.nabble.com_Missing-2Dobject-2Dwhen-2Dloading-2Dfrom-2DCassandra-2Dwith-2Dmultiple-2Dqueries-2Dtp15644p15789.html&d=DwICAg&c=IL_XqQWOjubgfqINi2jTzg&r=1esZO0r0bYS90lcsaLA6N4AFxuNo6lzauhETGwdJQoQ&m=dl-eqK7P7J3D54eeGL5rjJ10bzaDUx2XL2vaGmM_uIM&s=i9qrKfjX2-aAJAu5beN73PXgRKyHFY6wZpJf5Wlf-zg&e= Sent from the Apache Ignite Users mailing list archive at Nabble.com.
