smallzhongfeng commented on PR #168: URL: https://github.com/apache/incubator-uniffle/pull/168#issuecomment-1221921300
> It seems that we only call the method `removeResources` once, if `appIdToStorages` was removed because of method `removeResources`, we won't call the method `removeResources` any more. It seems that pr can't achieve our aim. Because we found the log `can't find HDFS storage for appid`, it means that `appIdToStorages` has removed the cache of this app. Because `removereSources` was called only once, but from the HDFS audit log, it can be seen that the shuffle file was deleted at 18:00:55, but was rewritten at 18:00:57, resulting in the loss of the appId of the cache of `appIdToStorages`, However, the cached appId of `shuffleTaskInfos` is still there and cannot be removed. Therefore, this exception occurs when `removeResources` is scheduled again. Therefore, this PR is mainly used to solve the problem that the path cannot be deleted after the cache of appid by appidtostorages expires. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
