beyond1920 commented on PR #4913: URL: https://github.com/apache/hudi/pull/4913#issuecomment-1612808066
I cherrypick this PR to the internal HUDI. And find a `FileNotFoundException` during read latest snapshot of a mor table.  The exception would happen if enable spark speculative feature under the following case. 1. Job1 is writing to a MOR table and not finished yet. It enables spark speculative feature. 2. Job2 is reading the latest snapshot from the MOR table, when it call `getLatestMergedFileSlicesBeforeOrOn`, it might list the log files which are written by speculative attempt task in Job1. 3. Job1 is finished, deletes the log files which are written by slow speculative tasks. 4. Job2 throws the `FileNotFoundException` when it read the log file which is already deleted in step3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
