errose28 opened a new pull request #2327: URL: https://github.com/apache/ozone/pull/2327
## What changes were proposed in this pull request? After merging master into the upgrade branch in HDDS-5321, an intermittent failure was noticed in TestSCMNodeManager#testLayoutOnHeartbeat. The issue occurs in SCMNodeManager#register, where the node is added to the nodeStateManager firing the NEW_NODE event, before the node report containing storage information for the new node is processed. The event triggers a one shot run on the background pipeline creator which will read the node's storage information to determine if it can hold a pipeline. If the storage report has not yet been processed when this happens, no pipeline will be created to use the new node when it is registered, because the node still appears to have no free space. In this fix, the NEW_NODE event is moved to not fire until all reports given to SCMNodeManager#register have been processed. The output of the existing test has also been modified for clearer output if a similar event occurs in the future. Although the bug is present on master as well, the fix was done on the upgrade branch because: 1. That is where the test to reproduce the issue resides. 2. The fix works better with SCMNodeManager modifications already done for upgrades. 3. Upgrade branch is expected to be merged into master soon. ## What is the link to the Apache JIRA HDDS-5336 ## How was this patch tested? Original failed multiple times on CI: - https://github.com/apache/ozone/runs/2787582345 - https://github.com/errose28/hadoop-ozone/runs/2748815362 Fix was tested with 100 runs on CI: - https://github.com/errose28/hadoop-ozone/runs/2797649712 I was unable to reproduce the original failure with 400 runs locally, so CI was used for testing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
