XComp commented on a change in pull request #18987:
URL: https://github.com/apache/flink/pull/18987#discussion_r822596828
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/highavailability/FileSystemJobResultStore.java
##########
@@ -165,8 +171,16 @@ public boolean hasCleanJobResultEntryInternal(JobID jobId)
throws IOException {
@Override
public Set<JobResult> getDirtyResultsInternal() throws IOException {
+ final FileStatus[] statuses = fileSystem.listStatus(this.basePath);
+ if (statuses == null) {
+ LOG.warn(
+ "The JobResultStore directory '"
+ + basePath
+ + "' was deleted. No persisted JobResults could be
recovered.");
+ return Collections.emptySet();
Review comment:
This was added because I noticed that the `LocalFileSystem.listStatus`
is implemented in a way that it returns `null` if the folder doesn't exist. I
added this `null` check (instead of fixing it in `LocalFileSystem`) because the
contract of the method isn't clear enough and I didn't want to mingle with the
public interface `FileSystem`. `HadoopFileSystem.listStatus` would return an
empty array in that case (because of the contract of the internally [Hadoop
FileSystem](https://hadoop.apache.org/docs/stable/api/org/apache/hadoop/fs/FileSystem.html#listStatus-org.apache.hadoop.fs.Path-)).
Therefore, this guard is to cover cases where a `LocalFileSystem` or any
other `FileSystem` implementation is used that allows returning `null` here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
