[ https://issues.apache.org/jira/browse/IGNITE-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16666210#comment-16666210 ]
Pavel Pereslegin edited comment on IGNITE-9858 at 10/27/18 8:00 PM: -------------------------------------------------------------------- [~agoncharuk], check for client node is performed in {{TcpDiscoveryIpFinderAdapter#initializeLocalAddresses}} (I updated ticket description). We have one instance of IpFinder and following execution order: 1. Server injects ignite instance resource into IpFinder. {noformat} at org.apache.ignite.internal.processors.resource.GridResourceProcessor.inject(GridResourceProcessor.java:283) at org.apache.ignite.internal.processors.resource.GridResourceProcessor.inject(GridResourceProcessor.java:252) at org.apache.ignite.internal.processors.resource.GridResourceProcessor.injectGeneric(GridResourceProcessor.java:233) at org.apache.ignite.internal.managers.GridManagerAdapter.inject(GridManagerAdapter.java:173) at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:257) at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939) at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682) at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730) at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158) at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656) {noformat} 2. Client injects ignite instance resource into IpFinder (same stacktrace). 3. Client initializes SPI, checks for client discovery: {noformat} at org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.discoveryClientMode(TcpDiscoveryIpFinderAdapter.java:113) at org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.initializeLocalAddresses(TcpDiscoveryIpFinderAdapter.java:61) at org.apache.ignite.spi.discovery.tcp.TcpDiscoveryImpl.registerLocalNodeAddress(TcpDiscoveryImpl.java:338) at org.apache.ignite.spi.discovery.tcp.ClientImpl.spiStart(ClientImpl.java:298) at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2023) at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297) at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939) at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682) at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730) at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158) at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656) {noformat} 4. Server initializes SPI: {noformat} at org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.discoveryClientMode(TcpDiscoveryIpFinderAdapter.java:113) at org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.initializeLocalAddresses(TcpDiscoveryIpFinderAdapter.java:61) at org.apache.ignite.spi.discovery.tcp.TcpDiscoveryImpl.registerLocalNodeAddress(TcpDiscoveryImpl.java:338) at org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:374) at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2023) at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297) at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939) at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682) at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730) at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158) at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656) {noformat} As a result both nodes assume that the local node is a client and don't register local address. Usually in tests we start client node after starting the server, so this problem is not reproduced. was (Author: xtern): [~agoncharuk], check for client node is performed in {{TcpDiscoveryIpFinderAdapter#initializeLocalAddresses}} (I updated ticket description). We have one instance of IpFinder and following execution order: 1. Server injects ignite instance resource into IpFinder. {noformat} at org.apache.ignite.internal.processors.resource.GridResourceProcessor.inject(GridResourceProcessor.java:283) at org.apache.ignite.internal.processors.resource.GridResourceProcessor.inject(GridResourceProcessor.java:252) at org.apache.ignite.internal.processors.resource.GridResourceProcessor.injectGeneric(GridResourceProcessor.java:233) at org.apache.ignite.internal.managers.GridManagerAdapter.inject(GridManagerAdapter.java:173) at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:257) at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939) at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682) at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730) at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158) at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656) {noformat} 2. Client injects ignite instance resource into IpFinder (same stacktrace). 3. Client initializes SPI, checks for client discovery: {noformat} at org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.discoveryClientMode(TcpDiscoveryIpFinderAdapter.java:113) at org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.initializeLocalAddresses(TcpDiscoveryIpFinderAdapter.java:61) at org.apache.ignite.spi.discovery.tcp.TcpDiscoveryImpl.registerLocalNodeAddress(TcpDiscoveryImpl.java:338) at org.apache.ignite.spi.discovery.tcp.ClientImpl.spiStart(ClientImpl.java:298) at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2023) at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297) at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939) at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682) at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730) at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158) at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656) {noformat} 4. Server initializes SPI: {noformat} at org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.discoveryClientMode(TcpDiscoveryIpFinderAdapter.java:113) at org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.initializeLocalAddresses(TcpDiscoveryIpFinderAdapter.java:61) at org.apache.ignite.spi.discovery.tcp.TcpDiscoveryImpl.registerLocalNodeAddress(TcpDiscoveryImpl.java:338) at org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:374) at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2023) at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297) at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939) at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682) at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730) at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158) at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656) {noformat} As a result both nodes assume that the local node is a client and don't register local address. > [Test Failed] SystemCacheNotConfiguredTest#test flaky fails on TC (timeout). > ---------------------------------------------------------------------------- > > Key: IGNITE-9858 > URL: https://issues.apache.org/jira/browse/IGNITE-9858 > Project: Ignite > Issue Type: Bug > Affects Versions: 2.6 > Reporter: Pavel Pereslegin > Assignee: Pavel Pereslegin > Priority: Major > Labels: MakeTeamcityGreenAgain > Fix For: 2.8 > > > SystemCacheNotConfiguredTest hangs sometimes on TeamCity (timeout). > Example of such failures on master branch: > [https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&buildTypeId=&tab=testDetails&testNameId=-2762467041583095183&order=TEST_STATUS_DESC&itemsCount=50&branch_IgniteTests24Java8=%3Cdefault%3E] > When we using ip finder in shared mode each node should register self address > (except clients, obviously). > Check that the node is a client uses installed (via DI > @IgniteInstanceResource) Ignite instance (see > {{TcpDiscoveryIpFinderAdapter#initializeLocalAddresses}}). > So when client and server starts simultaneously, the following scenario is > possible - Ignite server injected at first, then the Ignite client injected > when the SPI is initialized ({{spiStart}}) both nodes assume that the local > node is a client and don't register local address. > {noformat} > [2018-10-11 18:03:49,794][WARN > ][tcp-client-disco-msg-worker-#57%client%][TcpDiscoverySpi] IP finder > returned empty addresses list. Please check IP finder configuration. Will > retry every 2000 ms. Change 'reconnectDelay' to configure the frequency of > retries.{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)