shanrui commented on PR #8545: URL: https://github.com/apache/hadoop/pull/8545#issuecomment-4688174820
@ajfabbri Thanks for the feedback. There is a related historical Jira, HDFS-2551. While the root cause is different, the observed behavior is conceptually similar: if data is not properly flushed (especially when hdfs flush() not hdfs hflush() is invoked), HDFS cannot guarantee that newly written data will be immediately visible to readers. In this case, skipping hdfsHFlush() in dfs_flush() can lead to the same symptom: a reader opened after the write may observe an empty file or stale data until the writer closes the file. Regarding fi->flags, I found that as early as FUSE 2.4 the documentation did not guarantee that the original open flags would be reliably preserved across all callback invocations. Because of this, using fi->flags inside flush() to determine whether the file was opened for writing may not be reliable. In my environment, fi->flags was observed to be zero in dfs_flush(), which caused hdfsHFlush() to be skipped entirely. I'm working on creating a Jira and adding a test case as suggested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
