n3nash commented on pull request #2359: URL: https://github.com/apache/hudi/pull/2359#issuecomment-772301128
> The concern I had was the part 2 where, a committed write could have been archived and we may end up skipping it. Can you please clarify again how we guard that? By ensuring the archival will wait for the cleaner to log this block? @vinothchandar I have made 2 changes for multi-writer. 1. I changed the logic in `HoodieTimelineArchiveLog` to NOT archive anything after the oldest inflight. For long running jobs, we want to make sure that all instants that happened after it are present and not archived so conflict resolution can be done correctly. 2. I have added a check in log scanning code that log blocks instant should be either a) in the commit timeline or earlier that the earliest commit and b) not present in the inflight timeline. Not following your ask around committed writes and skipping, can you elaborate ? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
