I am saying from an observability system developer perspective. Master node as a trigger, is just the source of observability data. How and when to alert is totally another story. If you embed the alert inside it, you need to take care of the distributed coordination, again. In my mind, the alert function should an internal, scheduled by the master with the highest priority. It could read data from all master nodes, and calculate or do anything you need to do.
Sheng Wu 吴晟 Twitter, wusheng1108 guo jiwei <[email protected]> 于2019年12月11日周三 下午5:36写道： > Hi handsome mentor. > Master node is a common role in distribute system, but in DS, it's not very > meet. Master node is changing due to zk lock and is now responsible for > triggering task, spitting task and sending task command to worker . > so we put alert module into server module and triggered by master server is > suitable. > the important problem is the alert implementation. we will change it as > the architecture change. > > PS > Alert module is design to trigger task event change like timeout, fail or > success etc. and has an individual module. > > > On Wed, Dec 11, 2019 at 5:18 PM Sheng Wu <[email protected]> > wrote: > > > Ware a normal contributor hat, > > > > Merging alert into master seems a very strange design. I could get your > > point, but I haven't seen any distributed system asking the master nodes > to > > do business logic monitoring. > > > > Sheng Wu 吴晟 > > Twitter, wusheng1108 > > > > > > guo jiwei <[email protected]> 于2019年12月11日周三 下午5:00写道： > > > > > Hello everyone . > > > Simplify deployment is very nice to users. Currently, we have > discussed > > > combining or decreasing configuration files . How about deleting alert > > > module ? > > > We can move alert module into server module . MasterServer can take > the > > > responsibility for the task alert . > > > > > > General Implementation: > > > 1. Combine alert module into server module. Create a new package > for > > > alert function. > > > 2. MasterServer is responsible for alert. When starts MasterServer, > > > startup alert server. > > > 3. We can use zk lock to round-robin doing the alert around the > > > MasterServer. or just leader do(for it uses little resource) > > > > > > > > > If this proposal is taken into consideration , users do not to deploy > > alert > > > module anymore . > > > > > > On Tue, Dec 10, 2019 at 2:24 PM qiao zhanwei <[email protected]> > > > wrote: > > > > > > > > > > > Hello All , > > > > > > > > Now DolphinScheduler has so many Configuration files > > > > > > > > for example : > > > > > > > > dolphinscheduler-alert : > > > > alert.properties > > > > > > > > dolphinscheduler-api : > > > > application-api.properties > > > > application-combined.properties > > > > > > > > > > > > dolphinscheduler-common : > > > > hadoop.properties > > > > common.properties > > > > quartz.properties > > > > zookeeper.properties > > > > > > > > dolphinscheduler-dao : > > > > application-dao.properties > > > > > > > > dolphinscheduler-server : > > > > application-master.properties > > > > application-master.properties > > > > master.properties > > > > worker.properties > > > > > > > > .dolphinscheduler_env.sh > > > > > > > > Can we simplify deployment ? > > > > > > > > Main point : > > > > > > > > 1 configuration file simplification and merged configuration file > > > > 2 master server remove port > > > > 3 support offline installation，remove kazoo dependencies in install > and > > > > monitor > > > > 4 instll.sh script simplification > > > > > > > > > > > > ————————————— > > > > DolphinScheduler(Incubator) PPMC > > > > Zhanwei Qiao 乔占卫 > > > > > > > > [email protected] > > > > > > > > > >
