prashantwason commented on PR #19898: URL: https://github.com/apache/hudi/pull/19898#issuecomment-5998198145
Thanks @yihua for reproducing this, and sorry about the missing import — the branch was missing `import org.apache.hudi.core.read.BaseHoodieTableFileIndex`. Addressed in 4d63681: - Fixed the build (missing import). - Partition filters are evaluated again, against `modifiedPartitions` only: I extracted the predicate evaluation from `listMatchingPartitionPaths` into a shared helper and added `SparkHoodieTableFileIndex.filterPartitionPaths(paths, predicates)`, used by `listFileSplits` in V1 and V2. This keeps the O(modified partitions) listing without changing results. - `filterFileSlices` override now keeps the base contract (returns the `PartitionPath`, keeps `isPartitionPruneOnly`). The index also overrides `hasPredicatesPushedDown` instead of setting the flag in the constructor, since `refresh()` resets it. - `sizeInBytes`: handled by the companion #19912 (they should land together), noted in the description. - Added `TestIncrementalReadWithPartitionFilter`: a range touching two partitions, filtering on each, for COW and MOR. - Moved the description into the PR template so `validate-pr` passes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
