To mentor, yes, we will take care of these. On Wed, Dec 11, 2019 at 5:54 PM Sheng Wu <[email protected]> wrote:
> I am saying from an observability system developer perspective. > Master node as a trigger, is just the source of observability data. > How and when to alert is totally another story. If you embed the alert > inside it, you need to take care of the distributed coordination, again. > In my mind, the alert function should an internal, scheduled by the master > with the highest priority. It could read data from all master nodes, and > calculate or do anything you need to do. > > Sheng Wu 吴晟 > Twitter, wusheng1108 > > > guo jiwei <[email protected]> 于2019年12月11日周三 下午5:36写道： > > > Hi handsome mentor. > > Master node is a common role in distribute system, but in DS, it's not > very > > meet. Master node is changing due to zk lock and is now responsible for > > triggering task, spitting task and sending task command to worker . > > so we put alert module into server module and triggered by master server > is > > suitable. > > the important problem is the alert implementation. we will change it as > > the architecture change. > > > > PS > > Alert module is design to trigger task event change like timeout, fail or > > success etc. and has an individual module. > > > > > > On Wed, Dec 11, 2019 at 5:18 PM Sheng Wu <[email protected]> > > wrote: > > > > > Ware a normal contributor hat, > > > > > > Merging alert into master seems a very strange design. I could get your > > > point, but I haven't seen any distributed system asking the master > nodes > > to > > > do business logic monitoring. > > > > > > Sheng Wu 吴晟 > > > Twitter, wusheng1108 > > > > > > > > > guo jiwei <[email protected]> 于2019年12月11日周三 下午5:00写道： > > > > > > > Hello everyone . > > > > Simplify deployment is very nice to users. Currently, we have > > discussed > > > > combining or decreasing configuration files . How about deleting > alert > > > > module ? > > > > We can move alert module into server module . MasterServer can take > > the > > > > responsibility for the task alert . > > > > > > > > General Implementation: > > > > 1. Combine alert module into server module. Create a new package > > for > > > > alert function. > > > > 2. MasterServer is responsible for alert. When starts > MasterServer, > > > > startup alert server. > > > > 3. We can use zk lock to round-robin doing the alert around the > > > > MasterServer. or just leader do(for it uses little resource) > > > > > > > > > > > > If this proposal is taken into consideration , users do not to deploy > > > alert > > > > module anymore . > > > > > > > > On Tue, Dec 10, 2019 at 2:24 PM qiao zhanwei < > [email protected]> > > > > wrote: > > > > > > > > > > > > > > Hello All , > > > > > > > > > > Now DolphinScheduler has so many Configuration files > > > > > > > > > > for example : > > > > > > > > > > dolphinscheduler-alert : > > > > > alert.properties > > > > > > > > > > dolphinscheduler-api : > > > > > application-api.properties > > > > > application-combined.properties > > > > > > > > > > > > > > > dolphinscheduler-common : > > > > > hadoop.properties > > > > > common.properties > > > > > quartz.properties > > > > > zookeeper.properties > > > > > > > > > > dolphinscheduler-dao : > > > > > application-dao.properties > > > > > > > > > > dolphinscheduler-server : > > > > > application-master.properties > > > > > application-master.properties > > > > > master.properties > > > > > worker.properties > > > > > > > > > > .dolphinscheduler_env.sh > > > > > > > > > > Can we simplify deployment ? > > > > > > > > > > Main point : > > > > > > > > > > 1 configuration file simplification and merged configuration file > > > > > 2 master server remove port > > > > > 3 support offline installation，remove kazoo dependencies in install > > and > > > > > monitor > > > > > 4 instll.sh script simplification > > > > > > > > > > > > > > > ————————————— > > > > > DolphinScheduler(Incubator) PPMC > > > > > Zhanwei Qiao 乔占卫 > > > > > > > > > > [email protected] > > > > > > > > > > > > > > >
