Adding more ignite-clients slows down the first client

[Tobias Eriksson]

Hi
 I try to achieve very high inserts with Ignite-Cassandra
I have done some extensive testing, measuring all kinds of stuff
So long story short, here is my conclusion, I see the following when measuring 
INSERT rate per client
As you can see the first ignite-client has ~19,000 INSERTs / second
BUT when I add one more client to try to increased (scale horizontally) I 
clearly see that the INSERT speed decreases on the first client by a magnitude 
(dropping about 7,500 inserts/second from it’s 19,000 !!!)
Although the total speed increases to around ~25,000 INSERTs / second, this is 
still not good, and does not scale very well.
And as you can see adding a 3;rd and a 4;th client has the same affect.
Each client added steels overall performance from the other clients
I believe that Ignite is unable to handle this much inserts, can this be true…

Is this a known behavior ?
Is there a remedy ? (please say it is)

Details
Amazon C4.4XLarge  ==== 4 node cluster
Cassandra 3.3 ==== 4 Node cluster

Pojo Class with 50 attributes

Regards
Tobias


[cid:image001.png@01D35BF9.73D48C10]

<bean class="org.apache.ignite.configuration.CacheConfiguration">
                    <property name="name" value="cache_PojoProfile50"/>
                    <property name="atomicityMode" value="ATOMIC"/>
                    <property name="backups" value="1"/>
                    <property name="cacheMode" value="PARTITIONED"/>
                    <property name="indexedTypes">
                        <list>
                            <value>java.lang.Long</value>
                            
<value>com.qvantel.research.common.PojoProfile50</value>
                        </list>
                    </property>
                    <property name="readThrough" value="true"/>
                    <property name="writeThrough" value="true"/>

                    <property name="writeBehindEnabled" value="true"/>
                    <property name="writeBehindFlushFrequency" value="5000" />
                    <property name="writeBehindBatchSize" value="512" />
                    <property name="writeBehindFlushThreadCount" value="8" />
                    <property name="writeBehindFlushSize" value="0" />

                    <property name="cacheStoreFactory">
                        <bean 
class="org.apache.ignite.cache.store.cassandra.CassandraCacheStoreFactory">
                            <property name="dataSourceBean" 
value="cassandraDataSource"/>
                            <property name="persistenceSettingsBean" 
value="persistence_settings_PojoProfile50"/>
                        </bean>
                    </property>
                </bean>



<persistence keyspace="ignite" table="pojoprofile50" ttl="86400">
    <keyspaceOptions>
        REPLICATION = {'class' : 'SimpleStrategy', 'replication_factor' :2}
        AND DURABLE_WRITES = true
    </keyspaceOptions>
    <tableOption>
        comment = 'Cache test'
        AND read_repair_chance = 0.2
    </tableOption>
    <keyPersistence class="java.lang.Long" strategy="PRIMITIVE" 
column="profile_id"/>
    <valuePersistence class="com.qvantel.research.common.PojoProfile50" 
strategy="POJO"/>
</persistence>


--
Tobias Eriksson
Chief Architect Research – CTO Office
Qvantel Sweden AB
Tel; +46 768 832453
e-mail; tobias.eriks...@qvantel.com<mailto:tobias.eriks...@qvantel.com>