skywalker0618 opened a new pull request, #20399: URL: https://github.com/apache/hudi/pull/20399
### Describe the issue this Pull Request addresses Follow-up to #20337, which fixed reading nested arrays of rows through the Flink COW Parquet reader in `hudi-flink1.18.x` only. `hudi-flink1.19.x`, `1.20.x`, `2.0.x`, `2.1.x` and `2.2.x` have the same reader code and the same bugs, so reading a column such as `ARRAY<ROW<name STRING, items ARRAY<ROW<id STRING>>>>` still fails there with a `NullPointerException` in `NestedColumnReader.readRow`. `2.2.x` is the default profile, so the default bundle is affected. ### Summary and Changelog Applies the #20337 change, unchanged, to the five other Flink modules: * `ParquetSplitReaderUtil`: the `ARRAY` branch of `createWritableColumnVector` unwraps the Parquet `LIST` element with the same backward-compatibility rules as `getArrayElementColumn`, which both share through `isThreeLevelListRepeatedType`. INT32/INT64-encoded decimals are allocated with int/long vectors. * `NestedColumnReader`: grows the null placeholders of fields absent from the file to the row's position count. * `NestedPositionUtil.calculateRowOffsets`: returns the row's own position count when it has no nulls. * `ParquetColumnarRowSplitReader`: passes the requested schema's name as the parent name of the top-level fields. * `TestParquetSplitReaderUtil` and `TestNestedPositionUtil` are added to each module. In `1.19.x`, `1.20.x` and `2.0.x`, these files were identical to `1.18.x` before #20337 and are identical to it after this PR. `2.1.x` and `2.2.x` also support VARIANT, which this PR leaves as is. ### Impact Fixes the same read failure for nested array-of-row schemas in Flink 1.19 and later, and arrays of rows now return their field values instead of nulls. No public API or config changes. ### Risk Level Low. The change is the one reviewed and merged in #20337. Verification: * With `-Dflink1.19`, `-Dflink1.20`, `-Dflink2.0`, `-Dflink2.1` and `-Dflink2.2`, all 55 tests of each module pass, including the 19 new ones, and `checkstyle` passes. * Without the main-code changes, 12 of the 19 new tests fail in `hudi-flink2.2.x`. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
