rangareddy commented on issue #17270: URL: https://github.com/apache/hudi/issues/17270#issuecomment-5365809817
This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-8207). **Findings: needs a reproduction; the implicated code path has largely been replaced.** The report is specific and serious - unmerged (skip-merge) reads not merging the base file with the log files, while still merging log files among themselves, so a file group with two or more log files returns wrong results and a single-log-file group appears to work. Since it was filed, skip-merge handling moved into the file group reader. On `master` both `hudi-common/src/main/java/org/apache/hudi/common/table/read/HoodieFileGroupReader.java:206` and `.../buffer/DefaultFileGroupRecordBufferLoader.java:65` branch on `HoodieReaderConfig.REALTIME_SKIP_MERGE`, and the legacy Spark relation read paths named in the description have largely been removed or superseded (see the read-path removal in PR #14060 and the broader FG-reader migration). So the specific implementation this ticket blames may no longer exist - but "the code was replaced" is not the same as "the behaviour is correct", and asserting either would be a guess. The repro is cheap and worth running: a MOR table with two or more log files in a single file group, read with `hoodie.datasource.merge.type=skip_merge`, checking whether base-file records appear in the result. Worth doing on both the v1 and v2 relation paths if both are still reachable. Keeping this open. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
