subject:"\[jira\] \[Updated\] \(IGNITE\-10589\) Multiple server node failure after a client node stopping"

[jira] [Updated] (IGNITE-10589) Multiple server node failure after a client node stopping

2018-12-07 Thread Alexey Goncharuk (JIRA)



 [ 
https://issues.apache.org/jira/browse/IGNITE-10589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Goncharuk updated IGNITE-10589:
--
Fix Version/s: 2.8

> Multiple server node failure after a client node stopping
> -
>
> Key: IGNITE-10589
> URL: https://issues.apache.org/jira/browse/IGNITE-10589
> Project: Ignite
>  Issue Type: Bug
>Reporter: Sergey Kosarev
>Assignee: Ilya Lantukh
>Priority: Critical
> Fix For: 2.8
>
> Attachments: 16_02.tar
>
>
> after stopping a client
> we see  topology change and pme finish on the coordinator, 
> and at soon on another nodes we still don't see new topology, but have 
> Critical error resulting nodes failure
> crd log
> {code}
> 2018-12-06 15:55:23.660 [WARN 
> ][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
>  Node FAILED: ZookeeperClusterNode [id=979f03db-f858-44f6-8646-12034dfd5c93, 
> addrs=[10.116.206.1], order=129, loc=false, client=true]
> 2018-12-06 15:55:23.660 [INFO 
> ][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
>  Topology snapshot [ver=162, servers=128, clients=0, CPUs=7168, 
> offheap=14.0GB, heap=4000.0GB]
> 2018-12-06 15:55:23.660 [INFO 
> ][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
>^-- Node [id=44D27930-80E5-4EB7-B377-8B07C02C2033, clusterState=ACTIVE]
> 2018-12-06 15:55:23.660 [INFO 
> ][zk-DPL_GRID%DplGridNodeName-EventThread][o.a.i.s.d.z.i.ZookeeperDiscoveryImpl]
>  Process alive nodes change [alives=128]
> 2018-12-06 15:55:23.661 [INFO 
> ][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
>^-- Baseline [id=0, size=128, online=128, offline=0]
> 2018-12-06 15:55:23.661 [INFO 
> ][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
>  Data Regions Configured:
> 2018-12-06 15:55:23.661 [INFO 
> ][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
>^-- dpl_mem_plc [initSize=256.0 MiB, maxSize=556.6 GiB, 
> persistenceEnabled=true]
> 2018-12-06 15:55:23.661 [INFO 
> ][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
>^-- not-persisted [initSize=256.0 MiB, maxSize=556.6 GiB, 
> persistenceEnabled=false]
> 2018-12-06 15:55:23.670 
> [DEBUG][sys-#564%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.d.d.p.l.ExchangeLatchManager]
>  Process node left 979f03db-f858-44f6-8646-12034dfd5c93
> 2018-12-06 15:55:23.670 [INFO 
> ][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.ignite.internal.exchange.time]
>  Started exchange init [topVer=AffinityTopologyVersion [topVer=162, 
> minorTopVer=0], crd=true, evt=NODE_FAILED, 
> evtNode=979f03db-f858-44f6-8646-12034dfd5c93, customEvt=null, allowMerge=true]
> 2018-12-06 15:55:23.712 [INFO 
> ][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.ignite.internal.exchange.time]
>  Finished exchange init [topVer=AffinityTopologyVersion [topVer=162, 
> minorTopVer=0], crd=true]
> 2018-12-06 15:55:23.699 [INFO 
> ][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture]
>  Finish exchange future [startVer=AffinityTopologyVersion [topVer=162, 
> minorTopVer=0], resVer=AffinityTopologyVersion [topVer=162, minorTopVer=0], 
> err=null]
> {code}
> on a node(1) we have critical error(1)
> {code}
> 2018-12-06 15:55:23.727 
> [ERROR][utility-#432%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
>  Failed processing message [senderId=1e17c56a-5213-4a1b-b94b-4575a95a2c81, 
> msg=GridDhtTxPrepareRequest [nearNodeId=44d27930-80e5-4eb7-b377-8b07c02c2033,
>  futId=1d225238761-05eea259-5c25-4a4b-8469-9dd8980e218c, miniId=105, 
> topVer=AffinityTopologyVersion [topVer=162, minorTopVer=0], 
> invalidateNearEntries={}, nearWrites=null, owned=null, 
> nearXidVer=GridCacheVersion [topVer=155571374, order=1545423626166, nodeOrd
> er=1], subjId=44d27930-80e5-4eb7-b377-8b07c02c2033, taskNameHash=0, 
> preloadKeys=null, skipCompletedVers=false, 
> super=GridDistributedTxPrepareRequest [threadId=1281, 
> concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, writeVer=GridCacheVersion 
> [topVer=15557137
> 4, order=1545423626614, nodeOrder=96], timeout=0, reads=null, 
> writes=ArrayList [IgniteTxEntry [key=KeyCacheObjectImpl [part=65, 
> val=GridServiceAssignmentsKey [name=DPLThreadManager_service], 
> hasValBytes=true], cacheId=-2100569601, txKey=IgniteTxKey [key=KeyCa
> cheObjectImpl [part=65, val=GridServiceAssignmentsKey 
> [name=DPLThreadManager_service], hasValBytes=true], cacheId=-2100569601], 
> val=CacheObjectImpl [val=GridServiceAssignments 
> [nodeId=426a4a51-1af3-4019-9769-4a58d8ece426, topVer=162, 
> cfg=LazyServiceConfigurat
> ion [srvcClsName=com.sbt.dpl.gridgain.thread.DPLT

[jira] [Updated] (IGNITE-10589) Multiple server node failure after a client node stopping

2018-12-07 Thread Alexey Goncharuk (JIRA)



 [ 
https://issues.apache.org/jira/browse/IGNITE-10589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Goncharuk updated IGNITE-10589:
--
Ignite Flags:   (was: Docs Required)

> Multiple server node failure after a client node stopping
> -
>
> Key: IGNITE-10589
> URL: https://issues.apache.org/jira/browse/IGNITE-10589
> Project: Ignite
>  Issue Type: Bug
>Reporter: Sergey Kosarev
>Assignee: Ilya Lantukh
>Priority: Critical
> Fix For: 2.8
>
> Attachments: 16_02.tar
>
>
> after stopping a client
> we see  topology change and pme finish on the coordinator, 
> and at soon on another nodes we still don't see new topology, but have 
> Critical error resulting nodes failure
> crd log
> {code}
> 2018-12-06 15:55:23.660 [WARN 
> ][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
>  Node FAILED: ZookeeperClusterNode [id=979f03db-f858-44f6-8646-12034dfd5c93, 
> addrs=[10.116.206.1], order=129, loc=false, client=true]
> 2018-12-06 15:55:23.660 [INFO 
> ][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
>  Topology snapshot [ver=162, servers=128, clients=0, CPUs=7168, 
> offheap=14.0GB, heap=4000.0GB]
> 2018-12-06 15:55:23.660 [INFO 
> ][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
>^-- Node [id=44D27930-80E5-4EB7-B377-8B07C02C2033, clusterState=ACTIVE]
> 2018-12-06 15:55:23.660 [INFO 
> ][zk-DPL_GRID%DplGridNodeName-EventThread][o.a.i.s.d.z.i.ZookeeperDiscoveryImpl]
>  Process alive nodes change [alives=128]
> 2018-12-06 15:55:23.661 [INFO 
> ][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
>^-- Baseline [id=0, size=128, online=128, offline=0]
> 2018-12-06 15:55:23.661 [INFO 
> ][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
>  Data Regions Configured:
> 2018-12-06 15:55:23.661 [INFO 
> ][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
>^-- dpl_mem_plc [initSize=256.0 MiB, maxSize=556.6 GiB, 
> persistenceEnabled=true]
> 2018-12-06 15:55:23.661 [INFO 
> ][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
>^-- not-persisted [initSize=256.0 MiB, maxSize=556.6 GiB, 
> persistenceEnabled=false]
> 2018-12-06 15:55:23.670 
> [DEBUG][sys-#564%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.d.d.p.l.ExchangeLatchManager]
>  Process node left 979f03db-f858-44f6-8646-12034dfd5c93
> 2018-12-06 15:55:23.670 [INFO 
> ][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.ignite.internal.exchange.time]
>  Started exchange init [topVer=AffinityTopologyVersion [topVer=162, 
> minorTopVer=0], crd=true, evt=NODE_FAILED, 
> evtNode=979f03db-f858-44f6-8646-12034dfd5c93, customEvt=null, allowMerge=true]
> 2018-12-06 15:55:23.712 [INFO 
> ][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.ignite.internal.exchange.time]
>  Finished exchange init [topVer=AffinityTopologyVersion [topVer=162, 
> minorTopVer=0], crd=true]
> 2018-12-06 15:55:23.699 [INFO 
> ][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture]
>  Finish exchange future [startVer=AffinityTopologyVersion [topVer=162, 
> minorTopVer=0], resVer=AffinityTopologyVersion [topVer=162, minorTopVer=0], 
> err=null]
> {code}
> on a node(1) we have critical error(1)
> {code}
> 2018-12-06 15:55:23.727 
> [ERROR][utility-#432%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
>  Failed processing message [senderId=1e17c56a-5213-4a1b-b94b-4575a95a2c81, 
> msg=GridDhtTxPrepareRequest [nearNodeId=44d27930-80e5-4eb7-b377-8b07c02c2033,
>  futId=1d225238761-05eea259-5c25-4a4b-8469-9dd8980e218c, miniId=105, 
> topVer=AffinityTopologyVersion [topVer=162, minorTopVer=0], 
> invalidateNearEntries={}, nearWrites=null, owned=null, 
> nearXidVer=GridCacheVersion [topVer=155571374, order=1545423626166, nodeOrd
> er=1], subjId=44d27930-80e5-4eb7-b377-8b07c02c2033, taskNameHash=0, 
> preloadKeys=null, skipCompletedVers=false, 
> super=GridDistributedTxPrepareRequest [threadId=1281, 
> concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, writeVer=GridCacheVersion 
> [topVer=15557137
> 4, order=1545423626614, nodeOrder=96], timeout=0, reads=null, 
> writes=ArrayList [IgniteTxEntry [key=KeyCacheObjectImpl [part=65, 
> val=GridServiceAssignmentsKey [name=DPLThreadManager_service], 
> hasValBytes=true], cacheId=-2100569601, txKey=IgniteTxKey [key=KeyCa
> cheObjectImpl [part=65, val=GridServiceAssignmentsKey 
> [name=DPLThreadManager_service], hasValBytes=true], cacheId=-2100569601], 
> val=CacheObjectImpl [val=GridServiceAssignments 
> [nodeId=426a4a51-1af3-4019-9769-4a58d8ece426, topVer=162, 
> cfg=LazyServiceConfigurat
> ion [srvcClsName=com.sbt.dpl.gr

[jira] [Updated] (IGNITE-10589) Multiple server node failure after a client node stopping

2018-12-07 Thread Sergey Kosarev (JIRA)



 [ 
https://issues.apache.org/jira/browse/IGNITE-10589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Kosarev updated IGNITE-10589:

Description: 
after stopping a client
we see  topology change and pme finish on the coordinator, 
and at soon on another nodes we still don't see new topology, but have 
Critical error resulting nodes failure
crd log
{code}
2018-12-06 15:55:23.660 [WARN 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
 Node FAILED: ZookeeperClusterNode [id=979f03db-f858-44f6-8646-12034dfd5c93, 
addrs=[10.116.206.1], order=129, loc=false, client=true]
2018-12-06 15:55:23.660 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
 Topology snapshot [ver=162, servers=128, clients=0, CPUs=7168, 
offheap=14.0GB, heap=4000.0GB]
2018-12-06 15:55:23.660 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
   ^-- Node [id=44D27930-80E5-4EB7-B377-8B07C02C2033, clusterState=ACTIVE]
2018-12-06 15:55:23.660 [INFO 
][zk-DPL_GRID%DplGridNodeName-EventThread][o.a.i.s.d.z.i.ZookeeperDiscoveryImpl]
 Process alive nodes change [alives=128]
2018-12-06 15:55:23.661 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
   ^-- Baseline [id=0, size=128, online=128, offline=0]
2018-12-06 15:55:23.661 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
 Data Regions Configured:
2018-12-06 15:55:23.661 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
   ^-- dpl_mem_plc [initSize=256.0 MiB, maxSize=556.6 GiB, 
persistenceEnabled=true]
2018-12-06 15:55:23.661 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
   ^-- not-persisted [initSize=256.0 MiB, maxSize=556.6 GiB, 
persistenceEnabled=false]
2018-12-06 15:55:23.670 
[DEBUG][sys-#564%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.d.d.p.l.ExchangeLatchManager]
 Process node left 979f03db-f858-44f6-8646-12034dfd5c93
2018-12-06 15:55:23.670 [INFO 
][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.ignite.internal.exchange.time]
 Started exchange init [topVer=AffinityTopologyVersion [topVer=162, 
minorTopVer=0], crd=true, evt=NODE_FAILED, 
evtNode=979f03db-f858-44f6-8646-12034dfd5c93, customEvt=null, allowMerge=true]
2018-12-06 15:55:23.712 [INFO 
][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.ignite.internal.exchange.time]
 Finished exchange init [topVer=AffinityTopologyVersion [topVer=162, 
minorTopVer=0], crd=true]
2018-12-06 15:55:23.699 [INFO 
][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture]
 Finish exchange future [startVer=AffinityTopologyVersion [topVer=162, 
minorTopVer=0], resVer=AffinityTopologyVersion [topVer=162, minorTopVer=0], 
err=null]
{code}

on a node(1) we have critical error(1)
{code}
2018-12-06 15:55:23.727 
[ERROR][utility-#432%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
 Failed processing message [senderId=1e17c56a-5213-4a1b-b94b-4575a95a2c81, 
msg=GridDhtTxPrepareRequest [nearNodeId=44d27930-80e5-4eb7-b377-8b07c02c2033,
 futId=1d225238761-05eea259-5c25-4a4b-8469-9dd8980e218c, miniId=105, 
topVer=AffinityTopologyVersion [topVer=162, minorTopVer=0], 
invalidateNearEntries={}, nearWrites=null, owned=null, 
nearXidVer=GridCacheVersion [topVer=155571374, order=1545423626166, nodeOrd
er=1], subjId=44d27930-80e5-4eb7-b377-8b07c02c2033, taskNameHash=0, 
preloadKeys=null, skipCompletedVers=false, 
super=GridDistributedTxPrepareRequest [threadId=1281, concurrency=PESSIMISTIC, 
isolation=REPEATABLE_READ, writeVer=GridCacheVersion [topVer=15557137
4, order=1545423626614, nodeOrder=96], timeout=0, reads=null, writes=ArrayList 
[IgniteTxEntry [key=KeyCacheObjectImpl [part=65, val=GridServiceAssignmentsKey 
[name=DPLThreadManager_service], hasValBytes=true], cacheId=-2100569601, 
txKey=IgniteTxKey [key=KeyCa
cheObjectImpl [part=65, val=GridServiceAssignmentsKey 
[name=DPLThreadManager_service], hasValBytes=true], cacheId=-2100569601], 
val=CacheObjectImpl [val=GridServiceAssignments 
[nodeId=426a4a51-1af3-4019-9769-4a58d8ece426, topVer=162, 
cfg=LazyServiceConfigurat
ion [srvcClsName=com.sbt.dpl.gridgain.thread.DPLThreadManager, svcCls=, 
nodeFilterCls=IgniteAllNodesPredicate], assigns=HashMap 
{74979bc7-e4c3-424a-8347-2f3a589aca3e=1, 
32afc0a1-156d-4998-9b64-2336b86fb1c2=1, 6b34e59e-924c-404b-9451-ddf4b8b935b5=1, 
c2af8947-1
a8d-48d5-93ca-165f97399519=1, 12f327e6-0f5f-44e2-be2c-f7bf99d22eec=1, 
5638cb8c-4abb-49e3-8eef-edb3d5ccad77=1, 20349aca-ecf9-4a5f-bce9-2640e54fbbb2=1, 
e83f5de6-deaf-4f60-af5c-5a13d4f251a7=1, 2b4261be-feb5-4d59-be2a-6c9fcbe2fa4a=1, 
c3f0bff0-08fc-4601-b9f9-33192
9065c0a=1, bac0156c-a56b-4ab8-aa7c-7d9878151e9f=1, 
ccc2d442-8df4-402e-8589-d8ec3c6ec243=1, e34256a2-1bb9-4a17-86d1-21532833dded=1, 
50bae4c5-16a5-

[jira] [Updated] (IGNITE-10589) Multiple server node failure after a client node stopping

2018-12-07 Thread Sergey Kosarev (JIRA)



 [ 
https://issues.apache.org/jira/browse/IGNITE-10589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Kosarev updated IGNITE-10589:

Description: 
after stopping a client
we see  topology change and pme finish on the coordinator, 
and at soon on another nodes we still don't see new topology, but have 
Critical error resulting nodes failure
crd log
{code}
2018-12-06 15:55:23.660 [WARN 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
 Node FAILED: ZookeeperClusterNode [id=979f03db-f858-44f6-8646-12034dfd5c93, 
addrs=[10.116.206.1], order=129, loc=false, client=true]
2018-12-06 15:55:23.660 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
 Topology snapshot [ver=162, servers=128, clients=0, CPUs=7168, 
offheap=14.0GB, heap=4000.0GB]
2018-12-06 15:55:23.660 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
   ^-- Node [id=44D27930-80E5-4EB7-B377-8B07C02C2033, clusterState=ACTIVE]
2018-12-06 15:55:23.660 [INFO 
][zk-DPL_GRID%DplGridNodeName-EventThread][o.a.i.s.d.z.i.ZookeeperDiscoveryImpl]
 Process alive nodes change [alives=128]
2018-12-06 15:55:23.661 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
   ^-- Baseline [id=0, size=128, online=128, offline=0]
2018-12-06 15:55:23.661 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
 Data Regions Configured:
2018-12-06 15:55:23.661 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
   ^-- dpl_mem_plc [initSize=256.0 MiB, maxSize=556.6 GiB, 
persistenceEnabled=true]
2018-12-06 15:55:23.661 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
   ^-- not-persisted [initSize=256.0 MiB, maxSize=556.6 GiB, 
persistenceEnabled=false]
2018-12-06 15:55:23.670 
[DEBUG][sys-#564%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.d.d.p.l.ExchangeLatchManager]
 Process node left 979f03db-f858-44f6-8646-12034dfd5c93
2018-12-06 15:55:23.670 [INFO 
][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.ignite.internal.exchange.time]
 Started exchange init [topVer=AffinityTopologyVersion [topVer=162, 
minorTopVer=0], crd=true, evt=NODE_FAILED, 
evtNode=979f03db-f858-44f6-8646-12034dfd5c93, customEvt=null, allowMerge=true]
2018-12-06 15:55:23.712 [INFO 
][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.ignite.internal.exchange.time]
 Finished exchange init [topVer=AffinityTopologyVersion [topVer=162, 
minorTopVer=0], crd=true]
2018-12-06 15:55:23.699 [INFO 
][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture]
 Finish exchange future [startVer=AffinityTopologyVersion [topVer=162, 
minorTopVer=0], resVer=AffinityTopologyVersion [topVer=162, minorTopVer=0], 
err=null]
{code}

on a node(1) we have critical error(1)
{code}
2018-12-06 15:55:23.727 
[ERROR][utility-#432%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
 Failed processing message [senderId=1e17c56a-5213-4a1b-b94b-4575a95a2c81, 
msg=GridDhtTxPrepareRequest [nearNodeId=44d27930-80e5-4eb7-b377-8b07c02c2033,
 futId=1d225238761-05eea259-5c25-4a4b-8469-9dd8980e218c, miniId=105, 
topVer=AffinityTopologyVersion [topVer=162, minorTopVer=0], 
invalidateNearEntries={}, nearWrites=null, owned=null, 
nearXidVer=GridCacheVersion [topVer=155571374, order=1545423626166, nodeOrd
er=1], subjId=44d27930-80e5-4eb7-b377-8b07c02c2033, taskNameHash=0, 
preloadKeys=null, skipCompletedVers=false, 
super=GridDistributedTxPrepareRequest [threadId=1281, concurrency=PESSIMISTIC, 
isolation=REPEATABLE_READ, writeVer=GridCacheVersion [topVer=15557137
4, order=1545423626614, nodeOrder=96], timeout=0, reads=null, writes=ArrayList 
[IgniteTxEntry [key=KeyCacheObjectImpl [part=65, val=GridServiceAssignmentsKey 
[name=DPLThreadManager_service], hasValBytes=true], cacheId=-2100569601, 
txKey=IgniteTxKey [key=KeyCa
cheObjectImpl [part=65, val=GridServiceAssignmentsKey 
[name=DPLThreadManager_service], hasValBytes=true], cacheId=-2100569601], 
val=CacheObjectImpl [val=GridServiceAssignments 
[nodeId=426a4a51-1af3-4019-9769-4a58d8ece426, topVer=162, 
cfg=LazyServiceConfigurat
ion [srvcClsName=com.sbt.dpl.gridgain.thread.DPLThreadManager, svcCls=, 
nodeFilterCls=IgniteAllNodesPredicate], assigns=HashMap 
{74979bc7-e4c3-424a-8347-2f3a589aca3e=1, 
32afc0a1-156d-4998-9b64-2336b86fb1c2=1, 6b34e59e-924c-404b-9451-ddf4b8b935b5=1, 
c2af8947-1
a8d-48d5-93ca-165f97399519=1, 12f327e6-0f5f-44e2-be2c-f7bf99d22eec=1, 
5638cb8c-4abb-49e3-8eef-edb3d5ccad77=1, 20349aca-ecf9-4a5f-bce9-2640e54fbbb2=1, 
e83f5de6-deaf-4f60-af5c-5a13d4f251a7=1, 2b4261be-feb5-4d59-be2a-6c9fcbe2fa4a=1, 
c3f0bff0-08fc-4601-b9f9-33192
9065c0a=1, bac0156c-a56b-4ab8-aa7c-7d9878151e9f=1, 
ccc2d442-8df4-402e-8589-d8ec3c6ec243=1, e34256a2-1bb9-4a17-86d1-21532833dded=1, 
50bae4c5-16a5-

[jira] [Updated] (IGNITE-10589) Multiple server node failure after a client node stopping

2018-12-07 Thread Sergey Kosarev (JIRA)



 [ 
https://issues.apache.org/jira/browse/IGNITE-10589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Kosarev updated IGNITE-10589:

Description: 
after stopping a client
we see  topology change and pme finish on the coordinator, 
and at soon on another nodes we still don't see new topology, but have 
Critical error resulting nodes failure
crd log
{code}
2018-12-06 15:55:23.660 [WARN 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
 Node FAILED: ZookeeperClusterNode [id=979f03db-f858-44f6-8646-12034dfd5c93, 
addrs=[10.116.206.1], order=129, loc=false, client=true]
2018-12-06 15:55:23.660 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
 Topology snapshot [ver=162, servers=128, clients=0, CPUs=7168, 
offheap=14.0GB, heap=4000.0GB]
2018-12-06 15:55:23.660 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
   ^-- Node [id=44D27930-80E5-4EB7-B377-8B07C02C2033, clusterState=ACTIVE]
2018-12-06 15:55:23.660 [INFO 
][zk-DPL_GRID%DplGridNodeName-EventThread][o.a.i.s.d.z.i.ZookeeperDiscoveryImpl]
 Process alive nodes change [alives=128]
2018-12-06 15:55:23.661 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
   ^-- Baseline [id=0, size=128, online=128, offline=0]
2018-12-06 15:55:23.661 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
 Data Regions Configured:
2018-12-06 15:55:23.661 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
   ^-- dpl_mem_plc [initSize=256.0 MiB, maxSize=556.6 GiB, 
persistenceEnabled=true]
2018-12-06 15:55:23.661 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
   ^-- not-persisted [initSize=256.0 MiB, maxSize=556.6 GiB, 
persistenceEnabled=false]
2018-12-06 15:55:23.670 
[DEBUG][sys-#564%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.d.d.p.l.ExchangeLatchManager]
 Process node left 979f03db-f858-44f6-8646-12034dfd5c93
2018-12-06 15:55:23.670 [INFO 
][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.ignite.internal.exchange.time]
 Started exchange init [topVer=AffinityTopologyVersion [topVer=162, 
minorTopVer=0], crd=true, evt=NODE_FAILED, 
evtNode=979f03db-f858-44f6-8646-12034dfd5c93, customEvt=null, allowMerge=true]
2018-12-06 15:55:23.712 [INFO 
][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.ignite.internal.exchange.time]
 Finished exchange init [topVer=AffinityTopologyVersion [topVer=162, 
minorTopVer=0], crd=true]
2018-12-06 15:55:23.699 [INFO 
][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture]
 Finish exchange future [startVer=AffinityTopologyVersion [topVer=162, 
minorTopVer=0], resVer=AffinityTopologyVersion [topVer=162, minorTopVer=0], 
err=null]
{code}

on a node(1) we have critical error(1)
{code}
2018-12-06 15:55:23.727 
[ERROR][utility-#432%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
 Failed processing message [senderId=1e17c56a-5213-4a1b-b94b-4575a95a2c81, 
msg=GridDhtTxPrepareRequest [nearNodeId=44d27930-80e5-4eb7-b377-8b07c02c2033,
 futId=1d225238761-05eea259-5c25-4a4b-8469-9dd8980e218c, miniId=105, 
topVer=AffinityTopologyVersion [topVer=162, minorTopVer=0], 
invalidateNearEntries={}, nearWrites=null, owned=null, 
nearXidVer=GridCacheVersion [topVer=155571374, order=1545423626166, nodeOrd
er=1], subjId=44d27930-80e5-4eb7-b377-8b07c02c2033, taskNameHash=0, 
preloadKeys=null, skipCompletedVers=false, 
super=GridDistributedTxPrepareRequest [threadId=1281, concurrency=PESSIMISTIC, 
isolation=REPEATABLE_READ, writeVer=GridCacheVersion [topVer=15557137
4, order=1545423626614, nodeOrder=96], timeout=0, reads=null, writes=ArrayList 
[IgniteTxEntry [key=KeyCacheObjectImpl [part=65, val=GridServiceAssignmentsKey 
[name=DPLThreadManager_service], hasValBytes=true], cacheId=-2100569601, 
txKey=IgniteTxKey [key=KeyCa
cheObjectImpl [part=65, val=GridServiceAssignmentsKey 
[name=DPLThreadManager_service], hasValBytes=true], cacheId=-2100569601], 
val=CacheObjectImpl [val=GridServiceAssignments 
[nodeId=426a4a51-1af3-4019-9769-4a58d8ece426, topVer=162, 
cfg=LazyServiceConfigurat
ion [srvcClsName=com.sbt.dpl.gridgain.thread.DPLThreadManager, svcCls=, 
nodeFilterCls=IgniteAllNodesPredicate], assigns=HashMap 
{74979bc7-e4c3-424a-8347-2f3a589aca3e=1, 
32afc0a1-156d-4998-9b64-2336b86fb1c2=1, 6b34e59e-924c-404b-9451-ddf4b8b935b5=1, 
c2af8947-1
a8d-48d5-93ca-165f97399519=1, 12f327e6-0f5f-44e2-be2c-f7bf99d22eec=1, 
5638cb8c-4abb-49e3-8eef-edb3d5ccad77=1, 20349aca-ecf9-4a5f-bce9-2640e54fbbb2=1, 
e83f5de6-deaf-4f60-af5c-5a13d4f251a7=1, 2b4261be-feb5-4d59-be2a-6c9fcbe2fa4a=1, 
c3f0bff0-08fc-4601-b9f9-33192
9065c0a=1, bac0156c-a56b-4ab8-aa7c-7d9878151e9f=1, 
ccc2d442-8df4-402e-8589-d8ec3c6ec243=1, e34256a2-1bb9-4a17-86d1-21532833dded=1, 
50bae4c5-16a5-

[jira] [Updated] (IGNITE-10589) Multiple server node failure after a client node stopping

2018-12-07 Thread Sergey Kosarev (JIRA)



 [ 
https://issues.apache.org/jira/browse/IGNITE-10589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Kosarev updated IGNITE-10589:

Description: 
after stopping a client
we see  topology change and pme finish on the coordinator, 
and at soon on another nodes we still don't see new topology, but have 
Critical error resulting nodes failure
crd log
{code}
2018-12-06 15:55:23.660 [WARN 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
 Node FAILED: ZookeeperClusterNode [id=979f03db-f858-44f6-8646-12034dfd5c93, 
addrs=[10.116.206.1], order=129, loc=false, client=true]
2018-12-06 15:55:23.660 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
 Topology snapshot [ver=162, servers=128, clients=0, CPUs=7168, 
offheap=14.0GB, heap=4000.0GB]
2018-12-06 15:55:23.660 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
   ^-- Node [id=44D27930-80E5-4EB7-B377-8B07C02C2033, clusterState=ACTIVE]
2018-12-06 15:55:23.660 [INFO 
][zk-DPL_GRID%DplGridNodeName-EventThread][o.a.i.s.d.z.i.ZookeeperDiscoveryImpl]
 Process alive nodes change [alives=128]
2018-12-06 15:55:23.661 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
   ^-- Baseline [id=0, size=128, online=128, offline=0]
2018-12-06 15:55:23.661 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
 Data Regions Configured:
2018-12-06 15:55:23.661 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
   ^-- dpl_mem_plc [initSize=256.0 MiB, maxSize=556.6 GiB, 
persistenceEnabled=true]
2018-12-06 15:55:23.661 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
   ^-- not-persisted [initSize=256.0 MiB, maxSize=556.6 GiB, 
persistenceEnabled=false]
2018-12-06 15:55:23.670 
[DEBUG][sys-#564%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.d.d.p.l.ExchangeLatchManager]
 Process node left 979f03db-f858-44f6-8646-12034dfd5c93
2018-12-06 15:55:23.670 [INFO 
][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.ignite.internal.exchange.time]
 Started exchange init [topVer=AffinityTopologyVersion [topVer=162, 
minorTopVer=0], crd=true, evt=NODE_FAILED, 
evtNode=979f03db-f858-44f6-8646-12034dfd5c93, customEvt=null, allowMerge=true]
2018-12-06 15:55:23.712 [INFO 
][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.ignite.internal.exchange.time]
 Finished exchange init [topVer=AffinityTopologyVersion [topVer=162, 
minorTopVer=0], crd=true]
2018-12-06 15:55:23.699 [INFO 
][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture]
 Finish exchange future [startVer=AffinityTopologyVersion [topVer=162, 
minorTopVer=0], resVer=AffinityTopologyVersion [topVer=162, minorTopVer=0], 
err=null]
{code}

on one we have critical error(1)
{code}
2018-12-06 15:55:23.727 
[ERROR][utility-#432%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
 Failed processing message [senderId=1e17c56a-5213-4a1b-b94b-4575a95a2c81, 
msg=GridDhtTxPrepareRequest [nearNodeId=44d27930-80e5-4eb7-b377-8b07c02c2033,
 futId=1d225238761-05eea259-5c25-4a4b-8469-9dd8980e218c, miniId=105, 
topVer=AffinityTopologyVersion [topVer=162, minorTopVer=0], 
invalidateNearEntries={}, nearWrites=null, owned=null, 
nearXidVer=GridCacheVersion [topVer=155571374, order=1545423626166, nodeOrd
er=1], subjId=44d27930-80e5-4eb7-b377-8b07c02c2033, taskNameHash=0, 
preloadKeys=null, skipCompletedVers=false, 
super=GridDistributedTxPrepareRequest [threadId=1281, concurrency=PESSIMISTIC, 
isolation=REPEATABLE_READ, writeVer=GridCacheVersion [topVer=15557137
4, order=1545423626614, nodeOrder=96], timeout=0, reads=null, writes=ArrayList 
[IgniteTxEntry [key=KeyCacheObjectImpl [part=65, val=GridServiceAssignmentsKey 
[name=DPLThreadManager_service], hasValBytes=true], cacheId=-2100569601, 
txKey=IgniteTxKey [key=KeyCa
cheObjectImpl [part=65, val=GridServiceAssignmentsKey 
[name=DPLThreadManager_service], hasValBytes=true], cacheId=-2100569601], 
val=CacheObjectImpl [val=GridServiceAssignments 
[nodeId=426a4a51-1af3-4019-9769-4a58d8ece426, topVer=162, 
cfg=LazyServiceConfigurat
ion [srvcClsName=com.sbt.dpl.gridgain.thread.DPLThreadManager, svcCls=, 
nodeFilterCls=IgniteAllNodesPredicate], assigns=HashMap 
{74979bc7-e4c3-424a-8347-2f3a589aca3e=1, 
32afc0a1-156d-4998-9b64-2336b86fb1c2=1, 6b34e59e-924c-404b-9451-ddf4b8b935b5=1, 
c2af8947-1
a8d-48d5-93ca-165f97399519=1, 12f327e6-0f5f-44e2-be2c-f7bf99d22eec=1, 
5638cb8c-4abb-49e3-8eef-edb3d5ccad77=1, 20349aca-ecf9-4a5f-bce9-2640e54fbbb2=1, 
e83f5de6-deaf-4f60-af5c-5a13d4f251a7=1, 2b4261be-feb5-4d59-be2a-6c9fcbe2fa4a=1, 
c3f0bff0-08fc-4601-b9f9-33192
9065c0a=1, bac0156c-a56b-4ab8-aa7c-7d9878151e9f=1, 
ccc2d442-8df4-402e-8589-d8ec3c6ec243=1, e34256a2-1bb9-4a17-86d1-21532833dded=1, 
50bae4c5-16a5-48bf-a

[jira] [Updated] (IGNITE-10589) Multiple server node failure after a client node stopping

2018-12-07 Thread Sergey Kosarev (JIRA)



 [ 
https://issues.apache.org/jira/browse/IGNITE-10589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Kosarev updated IGNITE-10589:

Attachment: 16_02.tar

> Multiple server node failure after a client node stopping
> -
>
> Key: IGNITE-10589
> URL: https://issues.apache.org/jira/browse/IGNITE-10589
> Project: Ignite
>  Issue Type: Bug
>Reporter: Sergey Kosarev
>Priority: Critical
> Attachments: 16_02.tar
>
>
> after stopping a client
> we see  topology change and pme finish on the coordinator, 
> and at soon on another nodes we still don't see new topology, but have 
> Critical error resulting nodes failure



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (IGNITE-10589) Multiple server node failure after a client node stopping

2018-12-07 Thread Sergey Kosarev (JIRA)



 [ 
https://issues.apache.org/jira/browse/IGNITE-10589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Kosarev updated IGNITE-10589:

Description: 
after stopping a client
we see  topology change and pme finish on the coordinator, 
and at soon on another nodes we still don't see new topology, but have 
Critical error resulting nodes failure
crd log
{code}
2018-12-06 15:55:23.660 [WARN 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
 Node FAILED: ZookeeperClusterNode [id=979f03db-f858-44f6-8646-12034dfd5c93, 
addrs=[10.116.206.1], order=129, loc=false, client=true]
2018-12-06 15:55:23.660 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
 Topology snapshot [ver=162, servers=128, clients=0, CPUs=7168, 
offheap=14.0GB, heap=4000.0GB]
2018-12-06 15:55:23.660 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
   ^-- Node [id=44D27930-80E5-4EB7-B377-8B07C02C2033, clusterState=ACTIVE]
2018-12-06 15:55:23.660 [INFO 
][zk-DPL_GRID%DplGridNodeName-EventThread][o.a.i.s.d.z.i.ZookeeperDiscoveryImpl]
 Process alive nodes change [alives=128]
2018-12-06 15:55:23.661 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
   ^-- Baseline [id=0, size=128, online=128, offline=0]
2018-12-06 15:55:23.661 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
 Data Regions Configured:
2018-12-06 15:55:23.661 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
   ^-- dpl_mem_plc [initSize=256.0 MiB, maxSize=556.6 GiB, 
persistenceEnabled=true]
2018-12-06 15:55:23.661 [INFO 
][disco-event-worker-#159%DPL_GRID%DplGridNodeName%][o.a.i.i.m.d.GridDiscoveryManager]
   ^-- not-persisted [initSize=256.0 MiB, maxSize=556.6 GiB, 
persistenceEnabled=false]
2018-12-06 15:55:23.670 
[DEBUG][sys-#564%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.d.d.p.l.ExchangeLatchManager]
 Process node left 979f03db-f858-44f6-8646-12034dfd5c93
2018-12-06 15:55:23.670 [INFO 
][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.ignite.internal.exchange.time]
 Started exchange init [topVer=AffinityTopologyVersion [topVer=162, 
minorTopVer=0], crd=true, evt=NODE_FAILED, 
evtNode=979f03db-f858-44f6-8646-12034dfd5c93, customEvt=null, allowMerge=true]
2018-12-06 15:55:23.712 [INFO 
][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.ignite.internal.exchange.time]
 Finished exchange init [topVer=AffinityTopologyVersion [topVer=162, 
minorTopVer=0], crd=true]
2018-12-06 15:55:23.699 [INFO 
][exchange-worker-#160%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture]
 Finish exchange future [startVer=AffinityTopologyVersion [topVer=162, 
minorTopVer=0], resVer=AffinityTopologyVersion [topVer=162, minorTopVer=0], 
err=null]
{code}

on one we have critical error(1)
{code}
2018-12-06 15:55:23.727 
[ERROR][utility-#432%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
 Failed processing message [senderId=1e17c56a-5213-4a1b-b94b-4575a95a2c81, 
msg=GridDhtTxPrepareRequest [nearNodeId=44d27930-80e5-4eb7-b377-8b07c02c2033,
 futId=1d225238761-05eea259-5c25-4a4b-8469-9dd8980e218c, miniId=105, 
topVer=AffinityTopologyVersion [topVer=162, minorTopVer=0], 
invalidateNearEntries={}, nearWrites=null, owned=null, 
nearXidVer=GridCacheVersion [topVer=155571374, order=1545423626166, nodeOrd
er=1], subjId=44d27930-80e5-4eb7-b377-8b07c02c2033, taskNameHash=0, 
preloadKeys=null, skipCompletedVers=false, 
super=GridDistributedTxPrepareRequest [threadId=1281, concurrency=PESSIMISTIC, 
isolation=REPEATABLE_READ, writeVer=GridCacheVersion [topVer=15557137
4, order=1545423626614, nodeOrder=96], timeout=0, reads=null, writes=ArrayList 
[IgniteTxEntry [key=KeyCacheObjectImpl [part=65, val=GridServiceAssignmentsKey 
[name=DPLThreadManager_service], hasValBytes=true], cacheId=-2100569601, 
txKey=IgniteTxKey [key=KeyCa
cheObjectImpl [part=65, val=GridServiceAssignmentsKey 
[name=DPLThreadManager_service], hasValBytes=true], cacheId=-2100569601], 
val=CacheObjectImpl [val=GridServiceAssignments 
[nodeId=426a4a51-1af3-4019-9769-4a58d8ece426, topVer=162, 
cfg=LazyServiceConfigurat
ion [srvcClsName=com.sbt.dpl.gridgain.thread.DPLThreadManager, svcCls=, 
nodeFilterCls=IgniteAllNodesPredicate], assigns=HashMap 
{74979bc7-e4c3-424a-8347-2f3a589aca3e=1, 
32afc0a1-156d-4998-9b64-2336b86fb1c2=1, 6b34e59e-924c-404b-9451-ddf4b8b935b5=1, 
c2af8947-1
a8d-48d5-93ca-165f97399519=1, 12f327e6-0f5f-44e2-be2c-f7bf99d22eec=1, 
5638cb8c-4abb-49e3-8eef-edb3d5ccad77=1, 20349aca-ecf9-4a5f-bce9-2640e54fbbb2=1, 
e83f5de6-deaf-4f60-af5c-5a13d4f251a7=1, 2b4261be-feb5-4d59-be2a-6c9fcbe2fa4a=1, 
c3f0bff0-08fc-4601-b9f9-33192
9065c0a=1, bac0156c-a56b-4ab8-aa7c-7d9878151e9f=1, 
ccc2d442-8df4-402e-8589-d8ec3c6ec243=1, e34256a2-1bb9-4a17-86d1-21532833dded=1, 
50bae4c5-16a5-48bf-a

[jira] [Updated] (IGNITE-10589) Multiple server node failure after a client node stopping

[jira] [Updated] (IGNITE-10589) Multiple server node failure after a client node stopping

[jira] [Updated] (IGNITE-10589) Multiple server node failure after a client node stopping

[jira] [Updated] (IGNITE-10589) Multiple server node failure after a client node stopping

[jira] [Updated] (IGNITE-10589) Multiple server node failure after a client node stopping

[jira] [Updated] (IGNITE-10589) Multiple server node failure after a client node stopping

[jira] [Updated] (IGNITE-10589) Multiple server node failure after a client node stopping

[jira] [Updated] (IGNITE-10589) Multiple server node failure after a client node stopping

8 matches

Site Navigation

Mail list logo

Footer information