ravjot28 opened a new pull request, #20613:
URL: https://github.com/apache/hudi/pull/20613

   ### Describe the issue this Pull Request addresses
   
   Closes #20607
   
   `BaseJavaCommitActionExecutor.execute` updated the index twice per write: 
`updateIndex` and then
   `updateIndexAndMaybeRunPreCommitValidations`, both calling 
`HoodieIndex#updateLocation` on the same write statuses. A
   user-defined index was updated twice, and the recorded index update duration 
covered only the second call.
   
   ### Summary and Changelog
   
   Writes through `BaseJavaCommitActionExecutor` update the index once, as 
`BaseSparkCommitActionExecutor` does.
   
   - `BaseJavaCommitActionExecutor`: remove the `updateIndex` call in `execute` 
and the now unused protected
     `updateIndex` method. `updateIndexAndMaybeRunPreCommitValidations` still 
updates the index, stores the returned
     statuses and the duration, sets the replaced file ids and runs the 
pre-commit validators.
   - 
`TestJavaCopyOnWriteActionExecutor#testIndexLocationsAreUpdatedOncePerWrite`: 
an index set through
     `hoodie.index.class` (a `HoodieBloomIndex` subclass) counts its 
`updateLocation` calls and returns the statuses in
     reverse order; insert, upsert, delete and insert overwrite each update it 
once and return the statuses it returned,
     and insert overwrite still reports its replaced file groups and the index 
update duration.
   
   Not covered: on a merge-on-read table, `upsertPreppedRecords` runs 
`JavaUpsertPreppedDeltaCommitActionExecutor`,
   which updates the index for new records in `JavaBulkInsertHelper` and again 
for all statuses at its end. That path
   does not use `BaseJavaCommitActionExecutor.execute` and is left for a 
separate change.
   
   ### Impact
   
   No config change. The protected method 
`BaseJavaCommitActionExecutor#updateIndex` is removed; it had no caller or
   override in the repository. A user-defined index sees one `updateLocation` 
call per write through this executor
   instead of two. For the built-in indexes the result is the same as before.
   
   ### Risk Level
   
   low. The removed call duplicated the one that remains; nothing read what it 
stored.
   
   **Backport:** please include this fix in the next 1.2.x patch release. The 
commit cherry-picks cleanly onto
   `branch-1.2.x`; its test uses only classes present on both lines.
   
   Tests, run on master with JDK 11 and checkstyle on for main and test sources:
   - Without the main code change: `testIndexLocationsAreUpdatedOncePerWrite` 
fails (2 index updates, expected 1).
   - With the change: `TestJavaCopyOnWriteActionExecutor`, 
`TestHoodieJavaClientOnCopyOnWriteStorage`,
     `TestHoodieJavaClientOnMergeOnReadStorage` and 
`TestJavaHoodieBackedMetadata` pass.
   
   ### Documentation Update
   
   none
   
   ### Contributor's checklist
   
   - [x] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [x] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to