ravjot28 opened a new pull request, #20613:
URL: https://github.com/apache/hudi/pull/20613
### Describe the issue this Pull Request addresses
Closes #20607
`BaseJavaCommitActionExecutor.execute` updated the index twice per write:
`updateIndex` and then
`updateIndexAndMaybeRunPreCommitValidations`, both calling
`HoodieIndex#updateLocation` on the same write statuses. A
user-defined index was updated twice, and the recorded index update duration
covered only the second call.
### Summary and Changelog
Writes through `BaseJavaCommitActionExecutor` update the index once, as
`BaseSparkCommitActionExecutor` does.
- `BaseJavaCommitActionExecutor`: remove the `updateIndex` call in `execute`
and the now unused protected
`updateIndex` method. `updateIndexAndMaybeRunPreCommitValidations` still
updates the index, stores the returned
statuses and the duration, sets the replaced file ids and runs the
pre-commit validators.
-
`TestJavaCopyOnWriteActionExecutor#testIndexLocationsAreUpdatedOncePerWrite`:
an index set through
`hoodie.index.class` (a `HoodieBloomIndex` subclass) counts its
`updateLocation` calls and returns the statuses in
reverse order; insert, upsert, delete and insert overwrite each update it
once and return the statuses it returned,
and insert overwrite still reports its replaced file groups and the index
update duration.
Not covered: on a merge-on-read table, `upsertPreppedRecords` runs
`JavaUpsertPreppedDeltaCommitActionExecutor`,
which updates the index for new records in `JavaBulkInsertHelper` and again
for all statuses at its end. That path
does not use `BaseJavaCommitActionExecutor.execute` and is left for a
separate change.
### Impact
No config change. The protected method
`BaseJavaCommitActionExecutor#updateIndex` is removed; it had no caller or
override in the repository. A user-defined index sees one `updateLocation`
call per write through this executor
instead of two. For the built-in indexes the result is the same as before.
### Risk Level
low. The removed call duplicated the one that remains; nothing read what it
stored.
**Backport:** please include this fix in the next 1.2.x patch release. The
commit cherry-picks cleanly onto
`branch-1.2.x`; its test uses only classes present on both lines.
Tests, run on master with JDK 11 and checkstyle on for main and test sources:
- Without the main code change: `testIndexLocationsAreUpdatedOncePerWrite`
fails (2 index updates, expected 1).
- With the change: `TestJavaCopyOnWriteActionExecutor`,
`TestHoodieJavaClientOnCopyOnWriteStorage`,
`TestHoodieJavaClientOnMergeOnReadStorage` and
`TestJavaHoodieBackedMetadata` pass.
### Documentation Update
none
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]