[
https://issues.apache.org/jira/browse/IGNITE-4111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15679867#comment-15679867
]
Alexandr Kuramshin commented on IGNITE-4111:
--------------------------------------------
I have try to get the simulation when the nodes in topology have received
TcpDiscoveryNodeAddFinishedMessage but the joining node still has not.
I've add the delay on processing TcpDiscoveryNodeAddFinishedMessage, decrease
the communication timeout, and start 10 nodes simultaneously.
Then I have got an error after the 10th node started:
Failed to send job request, caused by Failed to send message to remote node,
caused by Failed to connect to node (is node still alive?),
caused by Failed to perform handshake due to timeout (consider increasing
'connectionTimeout' configuration property).
But after some discovery I've found that the onFirstMessage method don't hang.
It successfully executes every time when onMessage is invoked.
> Communication fails to send message if target node did not finish join process
> ------------------------------------------------------------------------------
>
> Key: IGNITE-4111
> URL: https://issues.apache.org/jira/browse/IGNITE-4111
> Project: Ignite
> Issue Type: Bug
> Components: general
> Reporter: Semen Boikov
> Assignee: Alexandr Kuramshin
> Fix For: 2.0
>
> Attachments: test onFirstMessage hang.log
>
>
> Currently this scenario is possible:
> - joining node sent join request and waits for
> TcpDiscoveryNodeAddFinishedMessage inside ServerImpl.joinTopology
> - others nodes already see this node and can send messages to it (for example
> try to run compute job on this node)
> - joining node can not receive message: TcpCommunicationSpi will hang inside
> 'onFirstMessage' on 'getSpiContext' call, so sending node will get error
> trying to establish connection
> Possible fix: if in onFirstMessage() spi context is not available, then
> TcpCommunicationSpi should send special response which indicates that this
> node is not ready yet, and sender should retry after some time.
> Also need check internal code for places where message can be unnecessarily
> sent to node: one such place is
> GridCachePartitionExchangeManager.refreshPartitions - message is sent to all
> known nodes, but here we can filter by node order / finished exchage version.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
