I'm currently trying to integrate ignite client within apache flink
taskmanagers(jvm) to fast lookup cache. Although my thoughput without ignite
is about 30k/s when i add ignite it drops to 2k/s.
    - Cache configuration was set with defaults expect for off-heap.
    - I have 3 different caches with one records each (testing purposes).
    - I've defined also near cache for clients. 
    - Both flink and ignite ara managed by yarn.

Is there any better and fastest way to access data? Am I doing something
wrong with the following configurations?

Thanks

------------------ Data access in client

this.nearCache.get(key);

------------------ Cache configuration

CacheConfiguration<String, Object[]> cacheConfig = new
CacheConfiguration<>();
cacheConfig.setName(table.getName());
cacheConfig.setCopyOnRead(false);
cacheConfig.setMemoryMode(CacheMemoryMode.OFFHEAP_TIERED);
cacheConfig.setOffHeapMaxMemory(0L);

------------------ Loading from client

IgniteDataStreamer<String, Object[]> streamer =
ignite.dataStreamer(cacheConfig.getName())

------------------ Near cache configuration

NearCacheConfiguration<String, Object[]> nearCacheConfig = new
NearCacheConfiguration<>();
nearCacheConfig.setNearEvictionPolicy(new LruEvictionPolicy(10000));
IgniteCache<String, Object[]> nearCache =
ignite.getOrCreateNearCache("near-" + cacheConfig.getName(),
nearCacheConfig);

------------------ Ignite configuration

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
        <property name="discoverySpi">
            <bean
class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                    <bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                        <property name="addresses">
                                <list>
                                       
<value>xx.xxx.xxx.111:47500..47510</value>
                                       
<value>xx.xxx.xxx.112:47500..47510</value>
                                </list>
                        </property>
                    </bean>
                </property>
                <property name="joinTimeout" value="60000"/>
            </bean>
        </property>
        <property name="communicationSpi">
                <bean
class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
                        <property name="sharedMemoryPort" value="-1"/>
                </bean>
        </property>
    </bean>



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Client-near-cache-with-Apache-Flink-tp11627.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Reply via email to