zhuangchong commented on pull request #8584: URL: https://github.com/apache/dolphinscheduler/pull/8584#issuecomment-1060160299
API, master, worker, dao modules depend on the task plugin module The API mainly does task paramter verification and special scene processing (later optimization): such as quick import of SQL tasks, special processing of CONDITIONS/DEPENDENT/SUB_PROCESS/SWITCH DAO mainly does special task parameter conversion: such as CONDITIONS/DEPENDENT/SUB_PROCESS/SWITCH task parameter processing MASTER mainly does DAG generation, resource parameters supplement link information, task distribution (CONDITIONS/DEPENDENT/SUB_PROCESS/SWITCH/COMMON different processing) WORKER mainly does task parameter verification, task parameter processing, resource parameter processing, and task execution Later expectations: All modules only need to introduce the task-api module, and do not need to pay attention to a specific task. Change points: 1. Remove the TaskType enumeration class of the common module, the task plug-in cannot fix the task type 2. Move the TaskParameters parameter class of the common module to each task plugin 3. Change the master to the resource supplementary link information in the specific task parameters into a general method to provide whether it is necessary to supplement the resource method 4. Delete all specific task modules under the Worker module, create a new task-all module, and import all specific task modules. Other calling modules such as the Worker module only need to import task-api, task-all (task-all is because the current packaging The problem of the way, normally I suggest that you only need to introduce the task-api module). Finally, if possible, please merge as soon as possible, because there are many files involved, and conflicting files will be changed all the time --- API,master,worker,dao模块依赖 task插件模块 API 主要做task paramter 验证和特殊场景处理(后期在优化)：如快捷导入SQL任务，CONDITIONS/DEPENDENT/SUB_PROCESS/SWITCH的特殊处理 DAO 主要做特殊task参数转换：如CONDITIONS/DEPENDENT/SUB_PROCESS/SWITCH 任务参数处理 MASTER 主要做DAG生成，资源参数补充链接信息，任务分发（CONDITIONS/DEPENDENT/SUB_PROCESS/SWITCH/COMMON不同处理） WORKER 主要做任务参数验证，任务参数处理，资源参数处理，任务执行 后期期望：所有模块只需引入task-api模块，不需要关注某个具体任务。 改动点： 1.将common模块的TaskType枚举类移除，任务插件化不能将任务类型固定 2.将common模块的TaskParameters参数类移到各个任务插件下 3.将master对具体任务参数里的资源补充链接信息改成通用方式提供是否需要补充资源方法 4.将Worker模块下所有具体任务模块删除，新建task-all模块，将所有具体任务模块引入，像 Worker 模块等其他调用模块只需引入task-api,task-all（task-all是因为当前打包方式的问题，正常我建议只需要引入task-api模块）。 最后如果可以，请尽快合并，因为涉及文件较多，会一直在改冲突文件。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
