hudi-bot opened a new issue, #16912:
URL: https://github.com/apache/hudi/issues/16912
h3. Motivation
As of today for RLI we only set a fix number of file groups at all times.
Individual file group size will keep expanding as the size of the record count
increase. The size of individual file group will go unbounded which does not
scale.
To address this issue, we need to figure out a way to dynamically
repartition the existing file groups to a larger / smaller amount of file
groups as needed. * Minimum downtime / exclusive access blocking all other
operations while we scaling\
* basic correctness
* trigger of the scale up / down
* Integration with other operations on MDT
This should work for both RLI and secondary index on data columns.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-9163
- Type: Bug
- Fix version(s):
- 1.1.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
