dongjoon-hyun opened a new pull request #24959: [SPARK-28157][CORE] Make SHS check Spark event log file permission changes URL: https://github.com/apache/spark/pull/24959 ## What changes were proposed in this pull request? At Spark 2.4.0/2.3.2/2.2.3, [SPARK-24948](https://issues.apache.org/jira/browse/SPARK-24948) delegated access permission checks to the file system, and maintains a permanent blacklist for all event log files failed once at reading. Although this reduces a lot of invalid accesses, there is no way to see this log files back after the permissions are recovered correctly. The only way has been restarting SHS. Apache Spark is unable to know the permission recovery. However, we had better give a second chances for those blacklisted files in a regular manner. ## How was this patch tested? Pass the Jenkins with the newly added updated test case.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
