Rajeev-01 commented on code in PR #13473:
URL: https://github.com/apache/hudi/pull/13473#discussion_r2279209679
##########
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java:
##########
@@ -504,6 +505,7 @@ public static void recover(HoodieStorage storage,
StoragePath metadataFolder) th
private static void modify(HoodieStorage storage, StoragePath
metadataFolder, Properties modifyProps, BiConsumer<Properties, Properties>
modifyFn) {
StoragePath cfgPath = new StoragePath(metadataFolder,
HOODIE_PROPERTIES_FILE);
StoragePath backupCfgPath = new StoragePath(metadataFolder,
HOODIE_PROPERTIES_FILE_BACKUP);
+ StoragePath tempCfgPath = new StoragePath(metadataFolder,
HOODIE_PROPERTIES_FILE + HOODIE_TEMP_FILE_SUFFIX);
Review Comment:
Yeah @danny0405 there are multiple issues that can happen right now due to
file creation and post copy. Let say in hdfs, our table path is running out of
quota and it reached it max. For now it reached it's limit in diskspace quota
and with the create and copy of actual file directly, will end up in creation
of 0 bytes properties file (Since there is no issue of namespace quota) and
while copying, it will fail due to quota issue. And that's how original file
will be present with 0 bytes. With this rename approach, all the intermittent
quota issues or namenode latency issues will be fallback to temp file and
original file will be populated once the temp file is populated correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
