yihua opened a new pull request, #19133: URL: https://github.com/apache/hudi/pull/19133
### Describe the issue this Pull Request addresses The legacy (pre-file-group-reader) Spark read path has zero automated coverage: the per-Spark-version `SparkXXLegacyHoodieParquetFileFormat` copies (~245 lines each across the six version modules) and `BaseFileOnlyRelation`, `IncrementalRelationV1`, `IncrementalRelationV2`, `IncrementalRelationUtil` in `hudi-spark-common` all report 0% on Codecov. These classes still run in production: `DefaultSource#resolveBaseFileOnlyRelation` uses `BaseFileOnlyRelation` for metadata-table reads, and `HoodieStreamSourceV1/V2` pick `IncrementalRelationV1/V2` when `hoodie.file.group.reader.enabled` is false. A regression there currently ships silently. Note on routing: the batch `DefaultSource.createRelation` path no longer consults `hoodie.file.group.reader.enabled` for normal snapshot/incremental reads; the legacy relations are reachable via the streaming sources and metadata-table reads. The tests therefore construct the legacy relations directly with the flag disabled in their options, matching the streaming-source invocation. ### Summary and Changelog Add `TestLegacyParquetReadPath` (hudi-spark functional suite, COW, <=30 rows per table) asserting sorted row-level equality between the legacy path and the file-group-reader path on the same table, plus independently computed expected values so both paths cannot be equally wrong: - COW snapshot read after insert + upsert, through both `BaseFileOnlyRelation.buildScan` and its `toHadoopFsRelation` conversion (asserting the relation executes the legacy Hudi parquet file format). - The non-vectorized (row-based) branch via `spark.sql.parquet.enableVectorizedReader=false`. - The append-partition-values branch via hive-style partitions and `hoodie.datasource.read.extract.partition.values.from.path`. - Partition + data filter pushdown through both legacy scan paths. - Incremental query across 3 commits through `IncrementalRelationV1` (instant-time start) and `IncrementalRelationV2` (completion-time start, `OPEN_CLOSED`), compared against the file-group-reader incremental result. Since the suite lives in the shared `hudi-spark` module functional package, each CI matrix leg exercises its own Spark version's legacy file format copy. ### Impact Test-only; no production code change. Closes the coverage blind spot on the legacy read path (six version-module file format copies plus the legacy relations in `hudi-spark-common`). ### Risk Level low ### Documentation Update None. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
