steveloughran commented on issue #1208: HADOOP-16423. S3Guard fsck: Check metadata consistency between S3 and metadatastore (log) URL: https://github.com/apache/hadoop/pull/1208#issuecomment-528883425 The testCLIFsckWithParam test works standalone. Looks to me like a race condition -the fsck is being performed on an active bucket and files have been deleted between listed and queued for scanning and the actual scan. 1. Test should only scan the test directory, or we handle FNFEs as something to ignore 2. Could the fsck code itself change here? Because on a stable bucket the FNFE could be a sign of a mismatch from DDB to store; you don't want them ignored. But: the failure could be delayed and reported as a missing file, rather than triggering a fast failure.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
