cshuo opened a new issue, #19983: URL: https://github.com/apache/hudi/issues/19983
### Bug Description **What happened:** `HoodieLsmFileGroupReader` accepts `latestCommitTime` but does not use it to filter native data or delete logs. It only applies the optional `InstantRange`. A time-travel read can therefore merge updates or deletes from commits later than the requested query instant. Selecting a file slice before or on the query instant does not necessarily exclude these logs: `HoodieFileGroup.getLatestFileSliceBeforeOrOn` compares the slice's base instant, and the selected slice can contain logs from later completed commits. The classic log scanner explicitly skips data/delete blocks newer than `latestCommitTime`. **What you expected:** Only logs with a delta commit time less than or equal to `latestCommitTime` should participate in the read, intersected with `InstantRange` when present. A delete after the query instant must not remove a record from the earlier snapshot. **Steps to reproduce:** 1. Use one file slice with a base record `k=base` at instant `001` and a native data log updating it to `k=at-boundary` at instant `002`. 2. Include a native data log updating `k` at instant `003` and a native delete log deleting `k` at instant `004` in the same slice. 3. Read the slice with `HoodieLsmFileGroupReader`, `latestCommitTime=002`, and no `InstantRange`. 4. The reader applies the later logs and removes `k`. It should return `k=at-boundary`. The same problem occurs when an explicit `InstantRange` includes instants after the latest commit time. If all logs are after the query instant, the reader should retain its base-file-only path. ### Environment - Hudi version: `1.3.0-SNAPSHOT` / current development implementation. - Query engine: reproduced with the common LSM reader using `HoodieAvroReaderContext` and mocked file iterators; no engine-specific end-to-end reproduction was run. - Relevant settings: LSM storage layout with native Parquet data/delete logs; `EVENT_TIME_ORDERING` in the regression fixture, with increasing record ordering values. ### Logs and Stack Trace The reader regression test fails before the fix because the future delete removes a record that must remain visible at the query instant: ```text expected: [a:older-log, k:at-boundary] actual: [a:older-log] ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
