voonhous commented on issue #19506:
URL: https://github.com/apache/hudi/issues/19506#issuecomment-5439979766
Second instance of the same pairing, to fix alongside `orderFields`:
- [ ] `HoodieParquetInputFormat.createBootstrappingRecordReader`
(`HoodieParquetInputFormat.java:192-204` on master) zips `getReadColumnIDs`
(de-duplicated, `Integer.parseInt` on each id) with `getReadColumnNames`
(neither) by index, and only ever consumes the result via `isEmpty()`. A blank
id (`",2,5"`, the HIVE-22438 shape that `cleanProjectionColumnIds` scrubs on
the realtime path but not here) throws `NumberFormatException`; a duplicated
name misaligns the pair or throws `ArrayIndexOutOfBoundsException`. Deriving
the two booleans from names alone deletes the zip with no behaviour change:
```java
boolean anyHoodieCol =
Arrays.stream(rawColNames).anyMatch(HoodieRecord.HOODIE_META_COLUMNS::contains);
boolean anyExternalCol = Arrays.stream(rawColNames).anyMatch(n ->
!HoodieRecord.HOODIE_META_COLUMNS.contains(n) &&
!HoodieHiveUtils.VIRTUAL_COLUMN_NAMES.contains(n));
```
Raised on #19510
(https://github.com/apache/hudi/pull/19510#discussion_r3774251738); that PR
keeps the zip, so this issue is the home for it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
