Hi ,
Version: 1.8
Java: 1.7.0_45-b18 [ we can't upgrade the java version to 1.8 due to some
internal restriction.]
We are facing that the ignite thread is hung when we are trying to insert
the data.
This is not happening that much frequently.
We don't see any exceptions in ignite except below that too the below
exception has come only on one server. Can you please check and suggest us.
*Exception in ignite server:*
[02:15:14,320][WARN
][grid-nio-worker-1-#10%CasinoApacheIgniteServices%][TcpCommunicationSpi]
Failed to process selector key (will close): GridSelectorNioSessionImpl
[selectorIdx=1, queueSize=0, writeBuf=java.nio.DirectByteBuffer[pos=0
lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768
cap=32768], recovery=GridNioRecoveryDescriptor [acked=2060041, resendCnt=0,
rcvCnt=2060037, sentCnt=2060041, reserved=true, lastAck=2060032,
nodeLeft=false, node=TcpDiscoveryNode
[id=62c088cc-f6a8-49b2-8eee-911f147d99b2, addrs=[10.166.161.111, 127.0.0.1],
sockAddrs=[/127.0.0.1:0, gi2p1xrcds001.gi02.bpty/10.166.161.111:0],
discPort=0, order=109, intOrder=61, lastExchangeTime=1539342673260,
loc=false, ver=1.8.0#20161205-sha1:9ca40dbe, isClient=true], connected=true,
connectCnt=0, queueLimit=100000, reserveCnt=68], super=GridNioSessionImpl
[locAddr=/10.166.186.158:9090, rmtAddr=/10.166.161.111:45006,
createTime=1539921738152, closeTime=0, bytesSent=6593820,
bytesRcvd=20615839, sndSchedTime=1539929665103, lastSndTime=1539921785427,
lastRcvTime=1539929665103, readsPaused=false,
filterChain=FilterChain[filters=[GridNioCodecFilter
[parser=o.a.i.i.util.nio.GridDirectParser@40e27086, directMode=true],
GridConnectionBytesVerifyFilter, SSL filter], accepted=true]]
[02:15:14,320][WARN
][grid-nio-worker-1-#10%CasinoApacheIgniteServices%][TcpCommunicationSpi]
Closing NIO session because of unhandled exception [cls=class
o.a.i.i.util.nio.GridNioException, msg=Connection timed out]
When we timed out the hung thread, we can see the below stack trace in
client service.
02:08:40,697 - ERROR - pool-45-thread-1 - CASINO_DATASYNC_LOGGER Exception
in Syncing the Data for LMTEMPLATECONFIG
javax.cache.CacheException: class
org.apache.ignite.IgniteInterruptedException: null
at
org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1440)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.cacheException(IgniteCacheProxy.java:2183)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.putAll(IgniteCacheProxy.java:1430)
at
com.pg.casino.service.datasync.DataSyncManager.syncDataIntoIgnite(DataSyncManager.java:835)
at
com.pg.casino.service.datasync.LMDataSyncManager.syncGenericLobbyTemplateConfig(LMDataSyncManager.java:327)
at
com.pg.casino.service.datasync.DataSyncCallable.call(DataSyncCallable.java:118)
at
com.pg.casino.service.datasync.DataSyncCallable.call(DataSyncCallable.java:22)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: class org.apache.ignite.IgniteInterruptedException: null
at
org.apache.ignite.internal.util.IgniteUtils$3.apply(IgniteUtils.java:766)
at
org.apache.ignite.internal.util.IgniteUtils$3.apply(IgniteUtils.java:764)
... 11 more
Caused by: java.lang.InterruptedException
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:160)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:118)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.putAll(GridDhtAtomicCache.java:704)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.putAll(IgniteCacheProxy.java:1423)
*cache info in Ignite xml :*
<bean id="lMTemplateIdVsJsonCacheIgnite"
class="org.apache.ignite.configuration.CacheConfiguration"
scope="singleton">
<property name="name"
value="lMTemplateIdVsJsonCacheIgnite"/>
<property name="cacheMode" value="REPLICATED"/>
<property name="memoryMode" value="ONHEAP_TIERED"/>
<property name="offHeapMaxMemory" value="#{1 * 20L *
1024L * 1024L}"/>
<property name="backups" value="1"/>
<property name="writeSynchronizationMode"
value="FULL_SYNC"/>
<property name="startSize" value="#{5 * 1024 * 1024}"/>
<property name="swapEnabled" value="false"/>
<property name="rebalanceMode" value="ASYNC"/>
<property name="rebalanceBatchSize" value="#{1024 *
1024}"/>
<property name="rebalanceThrottle" value="0"/>
<property name="rebalanceThreadPoolSize" value="4"/>
</bean>
*
Ignite config:*
<beans xmlns="http://www.springframework.org/schema/beans";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd";>
<bean id="ignite.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="gridName" value="CasinoApacheIgniteServices"/>
<property name="peerClassLoadingEnabled" value="false"/>
<property name="metricsUpdateFrequency" value="2000" />
<property name="discoverySpi">
<bean
class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
<property name="addresses">
<list>
<value>#$IGNITE_CASINO_SERVICE_1_IP$#:49500</value>
<value>#$IGNITE_CASINO_SERVICE_2_IP$#:49500</value>
<value>#$IGNITE_CASINO_SERVICE_3_IP$#:49500</value>
</list>
</property>
</bean>
</property>
<property name="localPort" value="49500"/>
<property name="heartbeatFrequency" value="2000"/>
<property name="maxMissedHeartbeats" value="5"/>
<property name="reconnectCount" value="100"/>
<property name="networkTimeout" value="10000"/>
<property name="socketTimeout" value="10000"/>
<property name="ackTimeout" value="10000"/>
<property name="statisticsPrintFrequency"
value="60000"/>
</bean>
</property>
<property name="communicationSpi">
<bean
class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
<property name="slowClientQueueLimit"
value="18000"/>
<property name="messageQueueLimit"
value="20000"/>
<property name="localPort" value="9090"/>
<property name="connectTimeout" value="5000"/>
<property name="idleConnectionTimeout"
value="75000"/>
<property name="sharedMemoryPort" value="-1"/>
</bean>
</property>
</bean>
</beans>
Thanks,
Praveen
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
