@[email protected] <[email protected]> @[email protected] <[email protected]> @GabryWu <[email protected]> @Xiaochun Liu <[email protected]> @Han Gao @许昌群 <[email protected]>
Thank you eveny one. I see that most of the friends who participate in this proposal discussion support it. Therefore, I will continue the work of DS plug-in design. Next, I applied to add the incubator-dolphinscheduler-maven-plugin repository to apache, and applied to create a branch on the incubator-dolphinscheduler repository for the next plug-in development and collaboration. And then I will open this todo list. leon bao <[email protected]> 于2020年6月10日周三 下午6:19写道： > As a developer, I think > 1. A good architecture allows a modification that will not affect the > operation of other modules, which is not possible in the current DS. For > example, adding new task types may affect the execution of other tasks > because they are all in one module. . > If good isolation is achieved, any modification to a module will not affect > the normal operation of other modules in the system. > 2. We need an architecture so that developers and the community stay in > sync, and each version of the community has the least impact on developers. > This requires a more flexible architecture and will attract more > developers. > > so i think the plug-in is suitable for DS. > > > [email protected] <[email protected]> 于2020年6月10日周三 > 下午4:48写道： > > > @jun gao > > So great to decouple the upper core logic of DS from the specific > > implementation. > > I think this design is more scalable and developers can cooperate in > > develop more easily.So I agree with it. > > > > @wu shaoj @changqun > > I saw you may be have other options that flexible plugin mechanism is > > complex and in a low priority. > > > > I think we can redesign the alert service in this time first,and I saw > jun > > gao have already done a lot of works. > > > > > > > > DolphinScheduler(Incubator) PPMC > > Gang Li 李岗 > > > > [email protected]<mailto:[email protected]> > > > > 发件人： wu shaoj > > 发送时间： 2020-06-04 17:44 > > 收件人： [email protected] > > 主题： Re: 答复: [Poprosal] DolphinScheduler Plugin > > Yes, we should forcus on the job with high priority . > > No need to satisfy all customer requirement which might be unreasonable > > > > On 2020/6/4, 17:41, "Hemin Wen" <[email protected]> wrote: > > > > Very detailed design. > > 1. About alert module, I agree with @许昌群. > > Feedback from the community on the alert module, the alert module > > function > > is relatively stable, > > and the current notification function plus webhook whether it can > meet > > the > > needs. > > > > 2.It is not recommended that the UI be modularized, this will > increase > > the > > complexity of module development and debugging, > > If there is more complicated UI interaction, it will cause trouble to > > the > > development, Not only need to understand UI development, > > but also understand modular configuration > > > > -------------------- > > DolphinScheduler(Incubator) Commtter > > Hemin Wen 温合民 > > [email protected] > > -------------------- > > > > > > Han Gao <[email protected]> 于2020年6月4日周四 下午4:42写道： > > > > > Hi folks, > > > > > > I prefer plugin way. @许昌群<mailto:[email protected]> provide > an > > > alert way like webhook(pls correct me if I'm wrong), which we can > > support > > > as an official plugin. > > > > > > The difference between webhook and plugin is you should start a > > server to > > > accept alert request in webhook, but in plugin way you don't need > to > > do > > > that. > > > > > > IMO, I prefer implement some official plugins in AlertServer > package > > > directly, as for others, we could put it in a plugin folder. I've > > > implemented a plugin class loader in DS common package, which used > > to load > > > plugin and avoid package conflict. Put official plugins directly in > > > AlertServer package can use the common packages, and will cost less > > memory > > > for class meta data. > > > > > > But put all plugin into a specific folder is fine too, because it > > looks > > > clean. > > > > > > BTW, I think there is no conflict to support both webhook and > plugin. > > > > > > Thanks, > > > Han Gao > > > > > > ________________________________ > > > From: wu shaoj <[email protected]> > > > Sent: Thursday, June 4, 2020 15:13 > > > To: [email protected] < > [email protected] > > >; > > > 许昌群 <[email protected]> > > > Subject: Re: 答复: [Poprosal] DolphinScheduler Plugin > > > > > > Yes , aggree with @许昌群 , flexible plugin mechanism is complex and > in > > a low > > > priority I think. > > > > > > On 2020/6/4, 15:04, "许昌群" <[email protected]> wrote: > > > > > > Hi all > > > > > > Regarding the alarm, I have some other ideas, please refer to > it. > > > > > > At present, there are many types of message notification > > components, > > > such as email/WeChat/SMS, or some notification components developed > > by the > > > company.If it is not realistic to connect all the components, can > we > > change > > > the way of thinking and flip this process? We provide an interface > > for > > > sending information, allowing users to interface with it. > > > > > > Described as follows: > > > 1. Define the sending interface of the message in DS, such as > > sendMsg > > > 2. Load the user-configured interface implementation, for > > example, > > > http implementation is http://192.168.1.1:8080/sendMsg/email > > > 3. Define the message sending format, {"tos":"[email protected] > ", > > > "content":"taskA failed."} > > > 4. Send http request. > > http://192.168.1.1:8080/sendMsg/email?data={ > > > "tos":"[email protected]", "content":"taskA failed."} > > > > > > If there are other alarm methods, add an http interface > > > > > > Finally, you only need to provide several default alarm > > functions in > > > DS to transfer other message notification functions to > > users.Although you > > > will lose some ease of use out of the box, it will increase > > flexibility. > > > You can implement your own system to ensure that the DS alarm logic > > is > > > simple and easy to use. > > > > > > > > > ________________________________ > > > 发件人: JUN GAO <[email protected]> > > > 发送时间: 2020年6月3日 12:13:21 > > > 收件人: dev > > > 主题: Re: [Poprosal] DolphinScheduler Plugin > > > > > > 1、Alert is very independent and common function, why should we > to > > > implement > > > it as plugin? > > > > > > I think Alert should be divided into two parts. One is that the > > server > > > collects various alarm information. The other is to pass these > > warning > > > contents through optional methods (such as email, WeChat). I > > think the > > > specific alarm method should not be implemented by the > > AlertServer > > > server. > > > It should be abstracted into an interface and implemented by a > > concrete > > > plug-in implementation layer. If someone understands and is > > familiar > > > with > > > other alarm methods, he can implement other alarm plug-ins > based > > on the > > > plug-in design. If he wants, he can contribute this plug-in to > > us. > > > Plug-inization is just a way to decouple projects. > > > > > > 2、The plug-in you mentioned is part of dolphinscheduler or a > > > thirdparty? If > > > it is a thirdparty, it may generally be managed by its own git, > > and it > > > will > > > not be placed in the dolphinscheduler related directory. > > > > > > Plug-inization is just a way to decouple projects. All Plugin > is > > part > > > of dolphinscheduler. > > > > > > And we can use jsonschema to solve the dynamic ui, for example: > > > alpacajs( > > > http://alpacajs.org/ <http://alpacajs.org/>), it is very > > powerful. > > > > > > @daolidong We can FYI this. > > > > > > Xiaochun Liu <[email protected]> 于2020年6月3日周三 上午11:35写道： > > > > > > > Good proposal, but I have some question as following: > > > > 1、Alert is very independent and common function, why should > we > > to > > > > implement it as plugin? > > > > 2、The plug-in you mentioned is part of dolphinscheduler or a > > > thirdparty? > > > > If it is a thirdparty, it may generally be managed by its own > > git, > > > and it > > > > will not be placed in the dolphinscheduler related directory. > > > > > > > > And we can use jsonschema to solve the dynamic ui, for > example: > > > alpacajs( > > > > http://alpacajs.org/ <http://alpacajs.org/>), it is very > > powerful. > > > > > > > > Best Regards > > > > --------------- > > > > DolphinScheduler(Incubator) Committer > > > > Xiaochun Liu 刘小春 > > > > [email protected] > > > > --------------- > > > > > > > > > > > > > > > > > 2020年6月2日 下午3:04，JUN GAO <[email protected]> 写道： > > > > > > > > > > > > https://github.com/gaojun2048/incubator-dolphinscheduler/pull/1 < > > > > > > https://github.com/gaojun2048/incubator-dolphinscheduler/pull/1> > > > > > > > > > > > > > -- > > > > > > DolphinScheduler(Incubator) PPMC > > > Jun Gao 高俊 > > > [email protected] > > > > > > > > -- > DolphinScheduler(Incubator) PPMC > BaoLiang 鲍亮 > [email protected] > -- DolphinScheduler(Incubator) PPMC Jun Gao 高俊 [email protected]
