vinothchandar commented on PR #17827: URL: https://github.com/apache/hudi/pull/17827#issuecomment-4676761064
@danny0405 Commenting on your high level thoughts. Overall, we definitely need a new table version, log format version for these changes. We should cleanly do these changes with backwards compatibility and upgrade/downgrade aspects factored in. Specifically, think about writers writing older table format versions (e.g 1.x writers writing v6) and being read by older readers (e.g 0.x readers reading v6) will work. Please let me know if/when the RFC captures these aspects clearly with detailed design decisions before we start implementing. I can review deeply then. > make the file visible instead of a hidden file; I don't think this is a big deal. But open to dropping the `.` at the beginning. > use the native file format suffix directly( .parquet .lance instead of .log.<version>_<write_token> how will we know something is a base file or log file? without opening the file. I feel strongly about keeping the `.log` in the name. may be `.log.parquet` or `.log.lance`. The log codifies that this is a change log relative to a base file. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
