Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/18905#discussion_r132901109 --- Diff: common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java --- @@ -333,33 +333,63 @@ protected Path getRecoveryPath(String fileName) { } /** + * Check the chosen DB file available or not. + */ + protected Boolean checkFileAvailable(File file) { + if (file.canWrite()){ + return true; + } + + return false; + } + + /** * Figure out the recovery path and handle moving the DB if YARN NM recovery gets enabled * when it previously was not. If YARN NM recovery is enabled it uses that path, otherwise * it will uses a YARN local dir. */ protected File initRecoveryDb(String dbName) { + Boolean bolRecoveryPathAvailable = true; + if (_recoveryPath != null) { File recoveryFile = new File(_recoveryPath.toUri().getPath(), dbName); - if (recoveryFile.exists()) { + + bolRecoveryPathAvailable = checkFileAvailable(recoveryFile); + logger.info("Recovery path {} ldb available: {}.", _recoveryPath, bolRecoveryPathAvailable); + if (recoveryFile.exists() && bolRecoveryPathAvailable) { return recoveryFile; } } + + // If recovery path unavailable, no use it any more. + if (!bolRecoveryPathAvailable) { --- End diff -- I think recovery path is set by user or use yarn default, user should make sure the availability of this directory, and yarn internally relies on it. It doesn't make sense to change to another disk if recovery path is unavailable.
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org