lei w created HUDI-6029: --------------------------- Summary: Rollback may omit invalid files when commitMetadata is not completed for MOR Key: HUDI-6029 URL: https://issues.apache.org/jira/browse/HUDI-6029 Project: Apache Hudi Issue Type: Bug Reporter: lei w
Now ,Use listingBasedRollbackStrategy may omit invalid files when commitMetadata is not completed.The reason for this problem is due to use instantToRollback timestamp and the baseCommitTime of the logFile to judge whether the Logfiles is valid. {code:java} // commit is instant time which should be rollback // in most cases BaseCommitTime may not equals commit (path) -> { if (path.toString().endsWith(basefileExtension)) { String fileCommitTime = FSUtils.getCommitTime(path.getName()); return commit.equals(fileCommitTime); } else if (FSUtils.isLogFile(path)) { // Since the baseCommitTime is the only commit for new log files, it's okay here String fileCommitTime = FSUtils.getBaseCommitTimeFromLogPath(path); return commit.equals(fileCommitTime); } return false; }; {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)