HuangZhenQiu opened a new pull request, #18790:
URL: https://github.com/apache/hudi/pull/18790
### Describe the issue this Pull Request addresses
Add shard assign metrics for bucket assign function. We have the
GlobalRecordIndexPartitioner that assigns potional of RLI shards to each bucket
assign task so that there is no cross task access to the same RLI shard, the
metrics will measure the the assignment of each task and we could detect skew
if the assignment is not that even.
Close #18732
### Summary and Changelog
1. GlobalRecordIndexPartitioner: Expose the file group count computation
as a static package-private method and add a computeNumShardsAssigned helper.
2. FlinkBucketAssignMetrics: Add a numShardsAssigned gauge backed by an
AtomicInteger.
3. BucketAssignFunction.open(): When global RLI is active, compute the
shard count for this task and push it to metrics.
### Impact
none
### Risk Level
none
### Documentation Update
none
### Contributor's checklist
- [ ] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Enough context is provided in the sections above
- [ ] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
