huangzhaobo99 commented on PR #8559: URL: https://github.com/apache/hadoop/pull/8559#issuecomment-4863142748
> Thanks @huangzhaobo99 for your works. Make sense to me. One concern, If the fsimage files may amass to large numbers because it can't upload to other namenodes on time when enable this configuration to end user by default? @Hexiaoqiao Thanks for review. I updated the concurrency policy in code as follows: 1. When parallel upload is enabled and maxThreads == 0 (default), poolSize is set to remoteNNAddresses.size(), preserving the original behavior. 2. When maxThreads > 0, poolSize is set to Math.min(remoteNNAddresses.size(), maxThreads), so concurrency is capped by the configured maximum. Therefore, when there are many Observer NN nodes, concurrency can be limited based on the actual deployment scenario, It will not easily lead to fsimage amass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
