Hi @Gang Li, I agree with you that the plug-in module should be a common module for all plug-in design. Currently I just create the SPI and plugin interface for alert plug-in. If need more type for future, it’s very easy to define other plugins.
Currently the alert service can detect the plug-in jars and load it. But I didn’t implement how to collaborate with front-end about the alert types API and how many properties a plugin should have yet. For example, if we add a webhook plugin, we should have the http method(form select), url(form text), etc… So the plugin interface may change to achieve this goal. Hi @leon bao, as mentioned above, I agree to make the plugin module as a common module for all plugin design, is that what you mean? > why don't we put the alert plug-in as an alert module? I’m not clear about it, do you mean we could just put alert plugin in alert module? I decide to put frequently used plugins in alert module, as for others like SMS(which may has different api), make it as a plugin jar file. Hi @Jun Gao, yes! I also use the SPI feature, it is quite convenient. Developers just need to implement the interfaces in SPI, which are in `dolphinscheduler-plugin-api` module. Hi all, thanks for your careful review! Here is some link. PR about plugin management. <https://github.com/apache/incubator-dolphinscheduler/pull/2572> PR about how to develop a simple plugin. <https://github.com/apache/incubator-dolphinscheduler/pull/2593> README about how to develop a simple plugin. <https://github.com/apache/incubator-dolphinscheduler/pull/2593/files#diff-0bdec77746c7a907bd4fca67469e0a07> Thanks in advance! Thanks, hgaol > On May 12, 2020, at 18:57, JUN GAO <[email protected]> wrote: > > we can refer to presto plug-in, etc. > 1. Design the SPI and define the interface that the plugin needs to implement > 2. All plugins create new models to implement the interface in SPI. > The benefits are as follows: > 1. For developers, they only need to care about the interface in SPI, and do > not need to care about other people's code, so it is easier to get started > developing plug-ins. > 2. For code reviewers, to improve the efficiency of review, independent > models will be more easily reviewed. If there is a problem with the code, it > is easy to roll back. > 3. For users, there are more flexible options. For example, some people only > want to use the A plug-in but not the B plug-in. > > Thank you ! > > [email protected] <mailto:[email protected]> > <[email protected] <mailto:[email protected]>> 于2020年5月12日周二 > 下午6:31写道： > Hi Gao, > > May be I didn't express clearly,I think the dolphinscheduler-plugin module is > a common module for all plugin design, > It's not neccessarily to put the implement of a plugin in the > dolphinscheduler-plugin module. > > I also agree with the following way your said,I see the pr you submited > didn't match it now,Will you give a deep design about the following way? > > > A better way is that user do not need to change the source code to support > > new alert method. Instead, they just need to implement some interface, and > > compliance with the convention, then package all(include dependencies) in a > > single jar file, and put the jar file in some specific place, such as > > $DS_HOME/plugins. After that, when start Alert service, the plugins can be > > recognized and loaded. So user can choose the alert method in the portal. > > > > DolphinScheduler(Incubator) PPMC > Gang Li 李岗 > > [email protected] > <mailto:[email protected]><mailto:[email protected] > <mailto:[email protected]>> > > From: hgaol > Date: 2020-05-11 19:00 > To: [email protected] <mailto:[email protected]> > CC: dev > Subject: Re: [PROPOSAL][Feature] Support plugin in Alert service > Hi Gang, > > I agree with that making examples as a pure folder, thanks for your > suggestion. > > For dolphinscheduler-plugin-api, do you mean it’s better to rename this > module? I’m not clear about ‘all plugin function need to be implemented > here’, do you mean we should put implemented plugin here? > > BTW, I’ve write a simple doc about how to implement an alert > plugin.https://github.com/apache/incubator-dolphinscheduler/pull/2593/files#diff-0bdec77746c7a907bd4fca67469e0a07 > > <https://github.com/apache/incubator-dolphinscheduler/pull/2593/files#diff-0bdec77746c7a907bd4fca67469e0a07> > > Also, I think for some frequently used plugins(email/webhook/...), we could > just implement in alert service. Here is the former PR about it. > https://github.com/apache/incubator-dolphinscheduler/pull/2572 > <https://github.com/apache/incubator-dolphinscheduler/pull/2572> > > Thanks for your careful review! > > Thanks, > hgaol > > On May 11, 2020, at 18:09, [email protected] > <mailto:[email protected]> wrote: > > Hi Gao, > > I agree with the following you said,If this is a plugin module,all the plugin > function need to be implemented here. > Now the name is dolphinscheduler-plugin-api,I think we can change the module > name, Such as `dolphinscheduler-plugin`. > > I think dolphinscheduler-plugin-api could not only support alert plugin, > > but also support java job task for future. That’s another topic. > > Yes,I think it just a folder,not a sub-module. > We can also look at other developers' comments on this. > >Do you mean the examples module should not be a sub-module in > >dolphinscheduler project pom file, just a pure folder? > > > > DolphinScheduler(Incubator) PPMC > Gang Li 李岗 > > [email protected] > <mailto:[email protected]><mailto:[email protected] > <mailto:[email protected]>> > > From: hgaol > Date: 2020-05-11 17:45 > To: dev; lgcareer2019 > Subject: Re: [PROPOSAL][Feature] Support plugin in Alert service > Hi Gang, > > Thanks for your suggestion. > 1.Whether the function of module dolphinscheduler-plugin-api can be puted in > the alert module. > [hgaol] The customer should depend on this module to implement their own > plugin. If put it into alert service, customer should put > `dolphinscheduler-alert` as dependence, which is too heavy for a plugin. > > Also, I think dolphinscheduler-plugin-api could not only support alert > plugin, but also support java job task for future. That’s another topic. > > 2. I think the examples module can be a folder. > [hgaol] Do you mean the examples module should not be a sub-module in > dolphinscheduler project pom file, just a pure folder? > > Thanks, > hgaol > > On May 11, 2020, at 11:51, [email protected] > <mailto:[email protected]> wrote: > > Hi hgaol, > Sorry for not replying in time. > I saw you added the module dolphinscheduler-plugin-api and examples > module,here are my two points want to discuss. > > 1、Whether the function of module dolphinscheduler-plugin-api can be puted in > the alert module. > 2、I think the examples module can be a folder. > > > > > > > DolphinScheduler(Incubator) PPMC > Gang Li 李岗 > > [email protected] > <mailto:[email protected]><mailto:[email protected] > <mailto:[email protected]>> > > From: hgaol > Date: 2020-04-29 23:37 > To: dev > Subject: Re: [PROPOSAL][Feature] Support plugin in Alert service > Hi folks, > > I’ve created a PR for this feature, here is the link > https://github.com/apache/incubator-dolphinscheduler/pull/2572 > <https://github.com/apache/incubator-dolphinscheduler/pull/2572> > <https://github.com/apache/incubator-dolphinscheduler/pull/2572 > <https://github.com/apache/incubator-dolphinscheduler/pull/2572>> > > Feel free to let me know if there is any suggestion. > > Thanks, > Han Gao > > On Apr 28, 2020, at 15:25, xingchun.chen <[email protected] > <mailto:[email protected]>> wrote: > > It sounds very good, looking forward to your contribution > > > best wish! > > > ------------------ 原始邮件 ------------------ > 发件人: "hgaol"<[email protected] <mailto:[email protected]>>; > 发送时间: 2020年4月28日(星期二) 下午3:11 > 收件人: "dev"<[email protected] > <mailto:[email protected]>>; > > 主题: [PROPOSAL][Feature] Support plugin in Alert service > > > > Hi folks, > > I think it will be useful for Alert service supporting the plugin management. > I’ve create an issue in Github and described about it, here > <https://github.com/apache/incubator-dolphinscheduler/issues/2553> > <https://github.com/apache/incubator-dolphinscheduler/issues/2553>>; is the > link. Copy here for convenience. > > > I think the plugin feature is needed by the customers, because different > user may have different environment. For example, I want to use SMS to send > alert information, but the SMS API is provided by some system we don't know > before, and our DS has already deployed. In this case, I just want to > implement the plugin code, and don't change the source code. > > > > Another benefits are developers can implement different plugin, like > slack, wechat, telegram, github issue, and etc... > > > > Describe the solution you'd like > > A clear and concise description of what you want to happen. > > > > A better way is that user do not need to change the source code to > support new alert method. Instead, they just need to implement some > interface, and compliance with the convention, then package all(include > dependencies) in a single jar file, and put the jar file in some specific > place, such as $DS_HOME/plugins. After that, when start Alert service, the > plugins can be recognized and loaded. So user can choose the alert method in > the portal. > > > > Currently, I've done some investigations, and if this feature is > accepted, I think maybe I can implement it. > > > > Firstly, I'd better to adapt the current function, which is send mail > and something like that. > > > > Then, we can add some more features, such as Alert handler instance, > which is created to describe the alert information(email/slack/wechat/..., to > whom, the title, content, and etc…). > > > > BTW, email and webhook are used frequently. I think they can be the > default plugin, and written in the source code, instead of a jar file. > > Thanks, > hgaol > > > > -- > DolphinScheduler(Incubator) PPMC > Jun Gao 高俊 > > [email protected] <mailto:[email protected]> >
