Thanks Xianming for the proposal, sorry for my late reply. I believe it is very necessary to extend congestion control to support multi-tenant control, which will further ensure the fairness of Shuffle for Celeborn's applications.
By the way, I have two questions: 1）How is the dynamic worker configuration synced/managed? Is there a DynamicConfigService in every worker? 2）From the design, I noticed that when high pressure occurs on some nodes, the client will switch locations between the initially allocated workers. Is this included in the CIP? I'm very interested in this, as those allocated workers may need to be replaced/add new resources in other scenarios. Thanks, Jiashu Xiong Xianming Lei <[email protected]> 于2024年7月16日周二 12:04写道： > Hi all, > I would like to start a discussion on CIP-11: Introduce Celeborn Traffic > Quota. > > > Celeborn Traffic Quota is mainly to solve the problem of individual > tenants on the cluster occupying a large amount of CPU and network card > resources. We will avoid this by limiting user traffic. > > > Looking forward to everyone's feedback and suggestions. Thank you! > > Doc Link: > https://docs.google.com/document/d/1Qe80fCS96TI_C-_cQiJ6dLUXmIOdsiQhZDaMpl3ssiM > Regards, Xianming Lei
