nsivabalan commented on code in PR #11514: URL: https://github.com/apache/hudi/pull/11514#discussion_r1662991098
########## rfc/rfc-78/rfc-78.md: ########## @@ -145,7 +147,9 @@ This will be an automatic upgrade for users when they start using 1.x hudi libra - No changes to log reader. - Check custom payload class in table properties and switch to payload type. - Trigger compaction for latest file slices. We do not want a single file slice having a mix of log files from 0.x and log files from 1.x. So, we will trigger a full compaction -of the table to ensure all latest file slices has just the base files. +of the table to ensure all latest file slices has just the base files. + - Lets dissect and see what it needs to support not requiring the full compaction. In general, we plan to add a table config to track the commit time (more on this later in this doc) when the upgrade was done. + So, using the upgrade commit time, we should be able to use different log file comparator to order log files within a given file slice. Review Comment: 0.16.x reader is not going to order log files based on completion time and will only be ordering based on log version even for 1.x tables. which means, even for a single file slice, having a mix of 0.x log files and 1.x log files, we should be good here. pending: file slice determination: HoodieLogFile.getBaseInstantTime() has to work for both log files (0.x log files and 1.x log files). if we ensure this is intact, we should be good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
