ignite read stale data from backup node

[Kent Deng]

Hi Ignites,

I ran into in-consistency data issues on version 2.8.1. I have three nodes run 
as a cluster and the cache configuration as:

        CacheConfiguration<String, Balance> cacheConfiguration = new 
CacheConfiguration<>(Balance.class.getSimpleName());
        cacheConfiguration.setIndexedTypes(String.class, Balance.class);
        cacheConfiguration.setSqlIndexMaxInlineSize(100);
        cacheConfiguration.setSqlSchema("PUBLIC");
        cacheConfiguration.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL);
        cacheConfiguration.setCacheMode(CacheMode.PARTITIONED);
        cacheConfiguration.setBackups(4);
        
cacheConfiguration.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);


Then I have very simple code to add balance in a loop:

for (int i = 0; i < 10000; i++) {
    balance = balanceDao.findByKey(accountId, "USD");
    balance.setQuantity(balance.getQuantity().add(BigDecimal.ONE));
    balanceDao.save(balance);
}

when I run above on the primary node, I always have balance increased 10000 
correctly, however when I run that on backup node, sometimes my balance 
increased around 8k, and sometimes 9k.

if setWriteSynchronizationMode was set to PRIMARY_SYNC and setReadFromBackup 
was set to false, I can get correct balance on all nodes.


is this a bug on 2.8.1 or anything wrong with my configuration?