Hi, Thanks Nicholas for the comments!
I now got access to put the proposal in Confluence in the form of CIP, here <https://cwiki.apache.org/confluence/display/CELEBORN/CIP+7+-+Celeborn+CLI> it is. Regarding your questions: > 1. From a user's perspective, the CLI is more used for some maintenance operations such as online and offline of server, rescaling of cluster etc, not only based on the REST API. What CLI interfaces are there that the REST API doesn’t have for maintenance? This is highly dependent on what the user is leveraging to manage their cluster. For example, in k8s, you would be using k8s APIs to achieve this. We can probably add a generic interface API for it that provides basic operations that users can implement themselves for their cluster management logic based on what cluster managers they are using. Although, I think this will likely be a later evolution of the CLI, once basic REST API operations are implemented in the CLI. WDYT? > 2. There are same sub-commands between MASTER and WORKER. Why not these sub-commands belong to BOTH? Agreed - this was a formatting mistake. I fixed it now, thanks for pointing that out. > 3. Does the implementation of CLI invoke the REST API? IMO, the CLI works well no matter the server is alive. Yes, I agree. I think for this we would have to talk to the cluster manager, similar to my response to #1. We would have to query the specific cluster manager to get details if the Celeborn servers are dead, since the Celeborn REST API would not work then. We can add a generic API that users can implement based on their own environment. Thanks, Aravind On Wed, Jun 5, 2024 at 10:43 PM Nicholas Jiang <[email protected]> wrote: > Hi Aravind, > > Thanks for driving this CIP about Celeborn CLI. I have some comments on > this CIP: > > 1. From a user's perspective, the CLI is more used for some maintenance > operations such as online and offline of server, rescaling of cluster etc, > not only based on the REST API. What CLI interfaces are there that the REST > API doesn’t have for maintenance? > > 2. There are same sub-commands between MASTER and WORKER. Why not these > sub-commands belong to BOTH? > > 3. Does the implementation of CLI invoke the REST API? IMO, the CLI works > well no matter the server is alive. > > BTW, could this design doc of proposal follow the template of CIP[1]? > > [1] > https://cwiki.apache.org/confluence/display/CELEBORN/Celeborn+Improvement+Proposals > > Regards, > Nicholas Jiang > > On 2024/06/05 23:33:02 Aravind Patnam wrote: > > Hi all, > > > > I have written up a proposal about introducing a CLI for Celeborn. You > can > > find the proposal > > < > https://docs.google.com/document/d/1j9wKFSR_ychYDF0NU5YN67WCCtNAgYTbN5CN8V3SOnk/edit?usp=sharing > > > > here. > > Please let me know if you have any comments or questions. > > > > TLDR by introducing a CLI, it would complement the existing dashboard and > > would benefit us internally. We rely on CLI tools internally a lot for > > automation and other operations. > > > > FYI, I was not able to access the cwiki page to put this proposal there, > > there seems to be some permissions issue. Hope it is okay to just share > as > > a google doc here for now. > > > > -- > > Aravind K. Patnam > > > > Apache Celeborn CLI Proposal > > < > https://docs.google.com/document/d/1j9wKFSR_ychYDF0NU5YN67WCCtNAgYTbN5CN8V3SOnk/edit?usp=drive_web > > > > > -- Aravind K. Patnam
