Thanks to Gao for this proposal, it sounds good to me. This task plugin could increase the cloud native adaptability of DS. I also would like to help maintain this plugin.
Best Regards, Rick Cheng On 2024/06/07 09:43:53 Chufeng Gao wrote: > Hi team, > > I would like to propose supporting Aliyun EMR Serverless Spark Task: > > > * Purpose > > - Aliyun EMR Serverless Spark [1] is a cloud-native, fully-managed > serverless product designed specifically for large-scale data > processing and analysis. It provides enterprises with a one-stop data > platform service, including task development, debugging, scheduling, > and operations maintenance, which greatly simplifies the entire data > processing lifecycle. By using EMR Serverless Spark, enterprises can > focus more on data processing and analysis, thus improving work > efficiency. > - This proposal aims to provide the DolphinScheduler community users > with an out-of-box plugin which they could use to conveniently > interact with Aliyun EMR Serverless Spark. > > > * Environment for Manual Testing > > - Aliyun EMR Serverless Spark is currently in public beta and free > to use until the end of 25th, June, 2024. > - I have added detailed documents with examples in the related > issue [2] & PR [3]. Reviewers / Users interested in this plugin could > test it free of charge. > > > * Maintenance > > - I will fix any known bugs / security vulnerabilities directly > related to the Aliyun Serverless Spark task plugin. > - I will submit a test report for the Aliyun Serverless Spark task > plugin every time before a new DolphinScheduler release to make sure > the plugin functions well. > - The community could retire this plugin if the test report is not > received before release for three consecutive times. > > > [1] > https://help.aliyun.com/zh/emr/emr-serverless-spark/product-overview/what-is-emr-serverless-spark > [2] https://github.com/apache/dolphinscheduler/issues/16127 > [3] https://github.com/apache/dolphinscheduler/pull/16126 > > > Best Regards, > > Chufeng (Eric) Gao >
