Davis-Zhang-Onehouse opened a new issue, #19716:
URL: https://github.com/apache/hudi/issues/19716

   **Describe the problem**
   
   When a table has two or more secondary index definitions whose metadata 
partitions are not initialized (for example after the metadata table was 
deleted and is being rebuilt while `.hoodie/.index_defs/index.json` survived), 
an indexing action for one of them builds nothing, yet the action completes and 
the partition is added to `hoodie.table.metadata.partitions`.
   
   **Why**
   
   `HoodieBackedTableMetadataWriter.buildMetadataPartitions` reduces the plan's 
partition paths to partition types. `SecondaryIndexer.buildInitialization` (and 
`ExpressionIndexer`) then pick the partition from every uninitialized 
definition of the type via `getSecondaryIndexPartitionsToInit`, and when that 
is not exactly one they log "Skipping secondary index initialization as only 
one secondary index bootstrap at a time is supported" and return nothing. 
`RunIndexActionExecutor` still calls `updateTableConfigAndTimeline` and marks 
the requested partition complete.
   
   **Impact**
   
   An empty index partition that readers trust for data skipping, on every 
table with more than one secondary (or expression) index whose metadata table 
is rebuilt index by index.
   
   **Reproduction**
   
   Register two secondary index definitions on a table with an initialized 
metadata table, then run `HoodieIndexer` (or 
`SparkRDDWriteClient.scheduleIndexing` + `index`) for one of them. 
`hoodie.table.metadata.partitions` lists the index; 
`HoodieTableMetadataUtil.metadataPartitionExists` is false and the partition 
has no file slices. A regression test is in the fixing PR.
   
   **Expected**
   
   The indexing action initializes exactly the partition its plan names 
whenever its definition exists, regardless of how many other definitions are 
uninitialized; a regular write keeps inferring from the uninitialized 
definitions.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to