Hi Abhijeet, Thanks for the KIP! This is an important feature for tiered storage.
Some comments: 1. Will we introduce new metrics for this tiered storage quotas? This is important because the admin can know the throttling status by checking the metrics while the remote write/read are slow, like the rate of uploading/reading byte rate, the throttled time for upload/read... etc. 2. Could you give some examples for the throttling algorithm in the KIP to explain it? That will make it much clearer. 3. To solve this problem, we can break down the RLMTask into two smaller tasks - one for segment upload and the other for handling expired segments. How do we handle the situation when a segment is still waiting for offloading while this segment is expired and eligible to be deleted? Maybe it'll be easier to not block the RLMTask when quota exceeded, and just check it each time the RLMTask runs? Thank you. Luke On Wed, Nov 22, 2023 at 6:27 PM Abhijeet Kumar <abhijeet.cse....@gmail.com> wrote: > Hi All, > > I have created KIP-956 for defining read and write quota for tiered > storage. > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-956+Tiered+Storage+Quotas > > Feedback and suggestions are welcome. > > Regards, > Abhijeet. >
