[ https://issues.apache.org/jira/browse/IGNITE-3401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676116#comment-15676116 ]
Ksenia Rybakova commented on IGNITE-3401: ----------------------------------------- Recently the issue is reproduced mostly during failover tests. And there are just several errors during one test. {noformat} [01:09:52,065][INFO ][grid-timeout-worker-#7%null%][IgniteKernal] Metrics for local node (to disable set 'metricsLogFrequency' to 0) ^-- Node [id=91d112f5, name=null, uptime=00:19:36:667] ^-- H/N/C [hosts=8, nodes=25, CPUs=128] ^-- CPU [cur=3.77%, avg=5.76%, GC=0%] ^-- Heap [used=787MB, free=80.77%, comm=4096MB] ^-- Non heap [used=45MB, free=64.37%, comm=49MB] ^-- Public thread pool [active=0, idle=0, qSize=0] ^-- System thread pool [active=0, idle=32, qSize=0] ^-- Outbound messages queue [size=0] [01:09:56,090][WARN ][grid-nio-worker-0-#9%null%][TcpCommunicationSpi] Failed to process selector key (will close): GridSelectorNioSessionImpl [selectorIdx=0, queueSize=0, writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], re [01:09:56,091][WARN ][grid-nio-worker-0-#9%null%][TcpCommunicationSpi] Closing NIO session because of unhandled exception [cls=class o.a.i.i.util.nio.GridNioException, msg=Connection reset by peer] [01:09:56,115][WARN ][tcp-comm-worker-#1%null%][TcpCommunicationSpi] Connect timed out (consider increasing 'failureDetectionTimeout' configuration property) [addr=/172.25.1.35:47100, failureDetectionTimeout=30000] [01:09:56,116][WARN ][tcp-comm-worker-#1%null%][TcpCommunicationSpi] Failed to connect to a remote node (make sure that destination node is alive and operating system firewall is disabled on local and remote hosts) [addrs=[/172.25.1.35:4 [01:09:56,173][ERROR][benchmark-worker-29][GridDhtColocatedCache] <tx> Failed to rollback transaction (cache may contain stale locks): GridNearTxLocal [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMapped=fal class org.apache.ignite.IgniteCheckedException: Failed to commit transaction: GridNearTxLocal[id=b8eba877851-056b03df-0000-0017-0000-01587456caf0, concurrency=OPTIMISTIC, isolation=READ_COMMITTED, state=ROLLED_BACK, invalidate=false, rol at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.finish(GridNearTxFinishFuture.java:433) at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.rollbackAsync(GridNearTxLocal.java:928) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter$14.applyx(IgniteTxLocalAdapter.java:3491) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter$14.applyx(IgniteTxLocalAdapter.java:3481) at org.apache.ignite.internal.util.lang.IgniteClosureX.apply(IgniteClosureX.java:38) at org.apache.ignite.internal.util.future.GridFutureChainListener.apply(GridFutureChainListener.java:54) at org.apache.ignite.internal.util.future.GridFutureChainListener.apply(GridFutureChainListener.java:28) at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:264) at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListeners(GridFutureAdapter.java:252) at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:382) at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:348) at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.onDone(GridNearTxFinishFuture.java:353) at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.onDone(GridNearTxFinishFuture.java:69) at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:325) at org.apache.ignite.internal.util.future.GridCompoundFuture.checkComplete(GridCompoundFuture.java:241) at org.apache.ignite.internal.util.future.GridCompoundFuture.apply(GridCompoundFuture.java:137) at org.apache.ignite.internal.util.future.GridCompoundFuture.apply(GridCompoundFuture.java:41) at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:264) at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListeners(GridFutureAdapter.java:252) at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:382) at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:348) at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:325) at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture$FinishMiniFuture.onNearFinishResponse(GridNearTxFinishFuture.java:1004) at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.onResult(GridNearTxFinishFuture.java:222) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishResponse(IgniteTxHandler.java:575) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$100(IgniteTxHandler.java:96) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$4.apply(IgniteTxHandler.java:158) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$4.apply(IgniteTxHandler.java:156) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:756) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:361) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:285) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:87) at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:230) at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1080) at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:708) at org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:101) at org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:671) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) [01:09:56,247][ERROR][sys-#171%null%][IgniteTxManager] Failed to send one phase commit ack to backup node [backup=14eb506b-b474-409f-84db-6391adc40582] class org.apache.ignite.internal.cluster.ClusterTopologyCheckedException: Node left grid while sending message to: 14eb506b-b474-409f-84db-6391adc40582 at org.apache.ignite.internal.processors.cache.GridCacheIoManager.send(GridCacheIoManager.java:834) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.send(GridCacheIoManager.java:977) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager$2.finish(IgniteTxManager.java:283) at org.apache.ignite.internal.processors.cache.GridDeferredAckMessageSender$DeferredAckMessageBuffer.finish0(GridDeferredAckMessageSender.java:214) at org.apache.ignite.internal.processors.cache.GridDeferredAckMessageSender$DeferredAckMessageBuffer.access$000(GridDeferredAckMessageSender.java:111) at org.apache.ignite.internal.processors.cache.GridDeferredAckMessageSender$DeferredAckMessageBuffer$1.run(GridDeferredAckMessageSender.java:159) at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6598) at org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:765) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: class org.apache.ignite.IgniteCheckedException: Failed to send message (node may have left the grid or TCP connection cannot be established due to firewall issues) [node=TcpDiscoveryNode [id=14eb506b-b474-409f-84db-6391adc4058 at org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1151) at org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1215) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.send(GridCacheIoManager.java:828) ... 11 more Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to send message to remote node: TcpDiscoveryNode [id=14eb506b-b474-409f-84db-6391adc40582, addrs=[127.0.0.1, 172.25.1.35], sockAddrs=[/172.25.1.35:47500, /127.0.0.1:47500] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2017) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:1955) at org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1146) ... 13 more Caused by: class org.apache.ignite.IgniteCheckedException: Failed to connect to node (is node still alive?). Make sure that each ComputeTask and GridCacheTransaction has a timeout set in order to prevent parties from waiting forever in c at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2521) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createNioClient(TcpCommunicationSpi.java:2161) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2055) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:1989) ... 15 more Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to connect to address: /172.25.1.35:47100 at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2526) ... 18 more Caused by: java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:117) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2386) ... 18 more Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to connect to address: /127.0.0.1:47100 at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2526) ... 18 more Caused by: class org.apache.ignite.IgniteCheckedException: Remote node ID is not as expected [expected=14eb506b-b474-409f-84db-6391adc40582, rcvd=a1293739-255a-476f-b6de-d421c8cc6c29] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.safeHandshake(TcpCommunicationSpi.java:2638) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2398) ... 18 more [01:09:56,806][WARN ][disco-event-worker-#20%null%][GridDiscoveryManager] Node FAILED: TcpDiscoveryNode [id=14eb506b-b474-409f-84db-6391adc40582, addrs=[127.0.0.1, 172.25.1.35], sockAddrs=[/172.25.1.35:47500, /127.0.0.1:47500], discPort= [01:09:56,807][INFO ][disco-event-worker-#20%null%][GridDiscoveryManager] Topology snapshot [ver=28, servers=19, clients=5, CPUs=128, heap=170.0GB] {noformat} > "Remote node ID is not as expected" errors at driver during load test > --------------------------------------------------------------------- > > Key: IGNITE-3401 > URL: https://issues.apache.org/jira/browse/IGNITE-3401 > Project: Ignite > Issue Type: Bug > Components: general > Reporter: Ksenia Rybakova > Fix For: 1.9 > > > A lot of errors at some drivers after several minutes after grid is up during > load test: > {noformat} > [14:33:42] (err) Failed to execute compound future reducer: Compound future > listener []class org.apache.ignite.IgniteCheckedException: Failed to send > message (node may have left the grid or TCP connection cannot be established > due to fir > at > org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1129) > at > org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1193) > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.submit(DataStreamerImpl.java:1378) > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.update(DataStreamerImpl.java:1173) > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.load0(DataStreamerImpl.java:756) > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addDataInternal(DataStreamerImpl.java:568) > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addData(DataStreamerImpl.java:604) > at > org.apache.ignite.yardstick.cache.load.IgniteCacheRandomOperationBenchmark$1.run(IgniteCacheRandomOperationBenchmark.java:383) > Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to send > message to remote node: TcpDiscoveryNode > [id=11da20f5-87c5-4321-8163-ffef85f2d54c, addrs=[10.20.0.215, 127.0.0.1], > sockAddrs=[/127.0.0.1:47503, fosters-215/10.20.0 > at > org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:1986) > at > org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:1926) > at > org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1124) > ... 7 more > Caused by: class org.apache.ignite.IgniteCheckedException: Failed to connect > to node (is node still alive?). Make sure that each GridComputeTask and > GridCacheTransaction has a timeout set in order to prevent parties from > waiting forever > at > org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2489) > at > org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createNioClient(TcpCommunicationSpi.java:2130) > at > org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2024) > at > org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:1960) > ... 9 more > Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to > connect to address: fosters-215/10.20.0.215:47103 > at > org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2494) > ... 12 more > Caused by: class org.apache.ignite.IgniteCheckedException: Failed to > read remote node recovery handshake (connection closed). > at > org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.safeHandshake(TcpCommunicationSpi.java:2699) > at > org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2361) > ... 12 more > Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to > connect to address: /127.0.0.1:47103 > at > org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2494) > ... 12 more > Caused by: class org.apache.ignite.IgniteCheckedException: Remote > node ID is not as expected [expected=11da20f5-87c5-4321-8163-ffef85f2d54c, > rcvd=03bc7d54-4fec-481d-af24-6f5408b32fe2] > at > org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.safeHandshake(TcpCommunicationSpi.java:2604) > at > org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2361) > ... 12 more > {noformat} > Tested version: 1.5.27 > Load config: > - 7 hosts with 4 servers and 3 drivers each (28 servers and 21 drivers in > total) > - benchmark: IgniteCacheRandomOperationBenchmark > - preload: 2M > - key range: 4M > - warmup: 30s > - duration: 10h > - operations: PUT PUT_ALL GET GET_ALL INVOKE INVOKE_ALL REMOVE REMOVE_ALL > PUT_IF_ABSENT REPLACE > -backup count: 1 > - GC: G1 > - heap size: 8Gb for servers, 4Gb for drivers > - caches: > {noformat} > <bean name="atomic" > class="org.apache.ignite.configuration.CacheConfiguration"> > <property name="name" value="atomic"/> > <property name="cacheMode" value="PARTITIONED"/> > <property name="atomicityMode" value="ATOMIC"/> > <property name="swapEnabled" value="false"/> > </bean> > <bean name="atomic-evict" > class="org.apache.ignite.configuration.CacheConfiguration"> > <property name="name" value="atomic-evict"/> > <property name="cacheMode" value="PARTITIONED"/> > <property name="atomicityMode" value="ATOMIC"/> > <property name="swapEnabled" value="false"/> > <property name="offHeapMaxMemory" value="#{1024L*1024L*1024L*6L}"/> > <property name="evictionPolicy"> > <bean > class="org.apache.ignite.cache.eviction.fifo.FifoEvictionPolicy"> > <property name="maxSize" value="750000"/> > </bean> > </property> > </bean> > <bean name="atomic-offheap" > class="org.apache.ignite.configuration.CacheConfiguration"> > <property name="name" value="atomic-offheap"/> > <property name="cacheMode" value="PARTITIONED"/> > <property name="atomicityMode" value="ATOMIC"/> > <property name="swapEnabled" value="false"/> > <property name="memoryMode" value="OFFHEAP_TIERED"/> > </bean> > <bean name="atomic-offheap-values" > class="org.apache.ignite.configuration.CacheConfiguration"> > <property name="name" value="atomic-offheap-values"/> > <property name="cacheMode" value="PARTITIONED"/> > <property name="atomicityMode" value="ATOMIC"/> > <property name="swapEnabled" value="false"/> > <property name="memoryMode" value="OFFHEAP_VALUES"/> > </bean> > <bean name="tx" > class="org.apache.ignite.configuration.CacheConfiguration"> > <property name="name" value="tx"/> > <property name="cacheMode" value="PARTITIONED"/> > <property name="atomicityMode" value="TRANSACTIONAL"/> > <property name="swapEnabled" value="false"/> > </bean> > <bean name="tx-evict" > class="org.apache.ignite.configuration.CacheConfiguration"> > <property name="name" value="tx-evict"/> > <property name="cacheMode" value="PARTITIONED"/> > <property name="atomicityMode" value="TRANSACTIONAL"/> > <property name="swapEnabled" value="false"/> > <property name="offHeapMaxMemory" value="#{1024L*1024L*1024L*6L}"/> > <property name="evictionPolicy"> > <bean > class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy"> > <property name="maxSize" value="750000"/> > </bean> > </property> > </bean> > <bean name="compute" > class="org.apache.ignite.configuration.CacheConfiguration"> > <property name="name" value="compute"/> > <property name="cacheMode" value="PARTITIONED"/> > <property name="atomicityMode" value="TRANSACTIONAL"/> > <property name="swapEnabled" value="false"/> > <property name="offHeapMaxMemory" value="#{1024L*1024L*1024L*6L}"/> > <property name="evictionPolicy"> > <bean > class="org.apache.ignite.cache.eviction.fifo.FifoEvictionPolicy"> > <property name="maxSize" value="750000"/> > </bean> > </property> > </bean> > <bean name="tx-offheap" > class="org.apache.ignite.configuration.CacheConfiguration"> > <property name="name" value="tx-offheap"/> > <property name="cacheMode" value="PARTITIONED"/> > <property name="atomicityMode" value="TRANSACTIONAL"/> > <property name="swapEnabled" value="false"/> > <property name="memoryMode" value="OFFHEAP_TIERED"/> > </bean> > <bean name="tx-offheap-values" > class="org.apache.ignite.configuration.CacheConfiguration"> > <property name="name" value="tx-offheap-values"/> > <property name="cacheMode" value="PARTITIONED"/> > <property name="atomicityMode" value="TRANSACTIONAL"/> > <property name="swapEnabled" value="false"/> > <property name="memoryMode" value="OFFHEAP_VALUES"/> > </bean> > <bean name="atomic-index" > class="org.apache.ignite.configuration.CacheConfiguration"> > <property name="name" value="atomic-index"/> > <property name="cacheMode" value="PARTITIONED"/> > <property name="atomicityMode" value="ATOMIC"/> > <property name="swapEnabled" value="false"/> > <property name="indexedTypes"> > <list> > <value>java.lang.Integer</value> > <value>org.apache.ignite.yardstick.cache.model.Person1</value> > </list> > </property> > </bean> > <bean name="query" > class="org.apache.ignite.configuration.CacheConfiguration"> > <property name="name" value="query"/> > <property name="cacheMode" value="PARTITIONED"/> > <property name="atomicityMode" value="ATOMIC"/> > <property name="swapEnabled" value="false"/> > <!-- Configure query entities --> > <property name="queryEntities"> > <list> > <bean class="org.apache.ignite.cache.QueryEntity"> > <property name="keyType" value="java.lang.Integer"/> > <property name="valueType" > value="org.apache.ignite.yardstick.cache.model.Person"/> > <property name="fields"> > <map> > <entry key="id" value="java.lang.Integer"/> > <entry key="orgId" value="java.lang.Integer"/> > <entry key="firstName" value="java.lang.String"/> > <entry key="lastName" value="java.lang.String"/> > <entry key="salary" value="java.lang.Double"/> > </map> > </property> > <property name="indexes"> > <list> > <bean class="org.apache.ignite.cache.QueryIndex"> > <constructor-arg value="id"/> > </bean> > <!-- Group index. --> > <bean class="org.apache.ignite.cache.QueryIndex"> > <constructor-arg> > <list> > <value>id</value> > <value>salary</value> > </list> > </constructor-arg> > <constructor-arg value="SORTED"/> > </bean> > </list> > </property> > </bean> > </list> > </property> > </bean> > <bean name="orgCache" > class="org.apache.ignite.configuration.CacheConfiguration"> > <property name="name" value="orgCache"/> > <property name="cacheMode" value="PARTITIONED"/> > <property name="atomicityMode" value="ATOMIC"/> > <property name="swapEnabled" value="false"/> > <!-- Configure query entities --> > <property name="queryEntities"> > <list> > <bean class="org.apache.ignite.cache.QueryEntity"> > <property name="keyType" value="java.lang.Integer"/> > <property name="valueType" > value="org.apache.ignite.yardstick.cache.model.Organization"/> > <property name="fields"> > <map> > <entry key="id" value="java.lang.Integer"/> > <entry key="name" value="java.lang.String"/> > </map> > </property> > <property name="indexes"> > <list> > <bean class="org.apache.ignite.cache.QueryIndex"> > <constructor-arg value="id"/> > </bean> > <bean class="org.apache.ignite.cache.QueryIndex"> > <constructor-arg value="name"/> > </bean> > </list> > </property> > </bean> > </list> > </property> > </bean> > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)