nsivabalan opened a new issue, #20343:
URL: https://github.com/apache/hudi/issues/20343
## Describe the problem you faced
`HoodieIndexer` cannot rebuild a **partitioned** record-level index. When it
re-enables indexes for metadata partitions that already exist on storage, it
maps the on-storage partition name `record_index` to
`hoodie.metadata.global.record.level.index.enable` unconditionally, with no
branch for the partitioned variant.
Since the write happens *after* the user's `--props` are loaded, an
explicitly supplied `hoodie.metadata.record.level.index.enable=true` is
overwritten. There is no workaround via configuration.
`hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieIndexer.java:180-190`:
```java
Set<String> initializedMetadataPartitions =
getInflightAndCompletedMetadataPartitions(metaClient.getTableConfig());
initializedMetadataPartitions.forEach(p -> {
...
if (PARTITION_NAME_RECORD_INDEX.equals(p)) {
props.setProperty(GLOBAL_RECORD_LEVEL_INDEX_ENABLE_PROP.key(), "true");
// always global
}
});
```
`RECORD_LEVEL_INDEX_ENABLE_PROP`
(`hoodie.metadata.record.level.index.enable`) is not imported or referenced
anywhere in the class. Both configs are declared adjacently in
`HoodieMetadataConfig` (lines 362 and 370), and both index variants share the
same `record_index` partition name
(`HoodieTableMetadataUtil.PARTITION_NAME_RECORD_INDEX`), so the partition name
alone cannot distinguish them.
## To Reproduce
1. Create a table with a partitioned record-level index
(`hoodie.metadata.record.level.index.enable=true`).
2. Run `HoodieIndexer` against it in any mode that reaches `start()`,
passing the same key in `--props`.
3. The index is rebuilt as a **global** record-level index, with default
file-group sizing rather than the partitioned layout's.
## Expected behavior
The indexer should rebuild the index in the same shape it found.
Distinguishing the two likely needs the table config or the existing index
definition rather than the partition name — for example reading
`hoodie.table.metadata.partitions` alongside whichever enable flag the table
was built with, and setting the matching property.
## Environment Description
* Hudi version : 1.2.0 and master (1.3.0-SNAPSHOT)
* Running on Docker? : no
## Additional context
Introduced in 80381808a6c3 ("fix: Fixing record index related configs and
enums", #14180, 2025-11-05), which split the record-index enable flag into
global and partitioned variants. The indexer was updated to the global constant
only. Shipped in 1.2.0.
Found while reviewing apache/hudi#20153, which adds a record-index sizing
health check. That check's remedy suggests re-running `HoodieIndexer` to resize
an undersized index; on a partitioned index that advice cannot currently work,
so the remedy text there is being adjusted to say so until this is fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
