I do not recommend remove DAG. I think, DS lacks support for batch tasks. DAG solves the expression of different types of tasks, there is no way to express batch tasks of the same type.
I don't know if what I understand is accurate, It is recommended to understand in depth the problem that users want to solve by using "List dependency”. -------------------- DolphinScheduler(Incubator) Commtter Hemin Wen 温合民 [email protected] -------------------- lidong dai <[email protected]> 于2020年5月20日周三 下午3:40写道： > I know what you said, there maybe need more people to discuss this topic, > I want to know other's opinion, how do they think this question > > > > Best Regards > --------------- > DolphinScheduler(Incubator) PPMC > Lidong Dai 代立冬 > [email protected] > --------------- > > > GabryWu <[email protected]> 于2020年5月19日周二 上午7:19写道： > > > not actually. I mean refactor backend code to remove DAG, where only JOB > > and DEPENDENCY remained > > > > ---Original--- > > *From:* "lidong dai"<[email protected]> > > *Date:* Mon, May 18, 2020 22:47 PM > > *To:* "GabryWu"<[email protected]>; > > *Cc:* "dev"<[email protected]>; > > *Subject:* Re: How do you think Task DAG dependency and List dependency？ > > > > I think your said is "List dependency", List dependency is also DAG, it's > > only different in use. when use List dependency, you can add your > upstream > > dependency for the task, this is convenient for hugely tasks in one > > workflow > > > > > > Best Regards > > --------------- > > DolphinScheduler(Incubator) PPMC > > Lidong Dai 代立冬 > > [email protected] > > --------------- > > > > > > GabryWu <[email protected]> 于2020年5月13日周三 上午11:08写道： > > > >> Until now ,DAG is one physical concept， which means that DAG is heavy > >> class , and introduces other classes and stored in a big json field. > >> On the bigdata platform , DAG is not a good concept for Schedulers, > which > >> can be abandoned. > >> If DAG was abandoned, JOB and DEPENDENCY will simply the architecture > >> and make DolphinScheduler stabilize and extended easily and easier to > >> search jobs > >> However abandoning DAG doesn't mean remove DAG graph in the front end, > >> which is also an important visualization way for JOBs > >> We can add one Job and Dependency manually ,and visualize them in DAG > >> graph automatically > >> > >> > >> > >> ------------------ 原始邮件 ------------------ > >> *发件人:* "lidong dai"<[email protected]>; > >> *发送时间:* 2020年5月12日(星期二) 下午5:26 > >> *收件人:* "dev"<[email protected]>; > >> *主题:* Re: How do you think Task DAG dependency and List dependency？ > >> > >> yes, your description is proper, thanks > >> > >> > >> Best Regards > >> --------------- > >> DolphinScheduler(Incubator) PPMC > >> Lidong Dai 代立冬 > >> [email protected] > >> --------------- > >> > >> > >> leon bao <[email protected]> 于2020年5月12日周二 下午5:06写道： > >> > >> > i think you want to show the DAG using list mode. > >> > DS show DAG using graphic mode, that will have some problems: > >> > > >> > 1. once the number of tasks is large, graphic mode will be confusing. > >> > 2. it is difficult to find the specified task in a complex DAG. > >> > > >> > lidong dai <[email protected]> 于2020年5月12日周二 下午4:31写道： > >> > > >> > > I want to say do we need to implement List dependency？ > >> > > > >> > > > >> > > > >> > > Best Regards > >> > > --------------- > >> > > DolphinScheduler(Incubator) PPMC > >> > > Lidong Dai 代立冬 > >> > > [email protected] > >> > > --------------- > >> > > > >> > > > >> > > JUN GAO <[email protected]> 于2020年5月12日周二 下午12:26写道： > >> > > > >> > > > Hi , @lidong dai > >> > > > Sorry , I don`t know your mean about this . > >> > > > You want to discuss how to implement Task DAG dependency and List > >> > > > dependency ? Or discuss what is Task DAG dependency and List > >> > dependency ? > >> > > > you can show us what`s your need and ideas , So that we can know > the > >> > > topic > >> > > > of discussion. > >> > > > > >> > > > Thank you ! > >> > > > > >> > > > lidong dai <[email protected]> 于2020年5月11日周一 下午10:34写道： > >> > > > > >> > > > > hi ， > >> > > > > > >> > > > > Task dependency between upstream and downstream called List > >> > > > dependency > >> > > > > do you have ideas about this topic? > >> > > > > > >> > > > > > >> > > > > > >> > > > > Best Regards > >> > > > > --------------- > >> > > > > DolphinScheduler(Incubator) PPMC > >> > > > > Lidong Dai 代立冬 > >> > > > > [email protected] > >> > > > > --------------- > >> > > > > > >> > > > > >> > > > > >> > > > -- > >> > > > > >> > > > DolphinScheduler(Incubator) PPMC > >> > > > Jun Gao 高俊 > >> > > > [email protected] > >> > > > > >> > > > >> > > >> > > >> > -- > >> > DolphinScheduler(Incubator) PPMC > >> > BaoLiang 鲍亮 > >> > [email protected] > >> > > >> > >> >
