Hi, community

Issue: support other business systems to run in DS.
https://github.com/apache/dolphinscheduler/issues/16672
Issue background:
description
present situation: for enterprises that use Apache dolphinscheduler, Apache
dolphinscheduler is more often used as the core scheduling system for data
processing. however, there are often other business systems in the
enterprise, and most of these business systems have requirements such as
scheduled task job management. At the same time, the role of the business
system may also be applied to a certain aspect of data processing.

planned renovation:

add DS-plugin Java plug-in
(1) the business system is referenced through Maven. the business system
implements fixed interfaces and uses fixed annotations to complete the
creation of scheduled task executors. the plug-in includes registration of
Apache dolphinscheduler api-server, workflow, node creation, scheduled task
creation, etc. core functions.
(2) business system application processing can be referenced through the
parent-child process, making it a part of data flow.
2.Apache dolphinscheduler api-server and master-server add corresponding
processing logic
(1) api-server adds registration, task creation processor and logic.
(2) master-server adds task triggering and result processing logic
(3) add external system nodes

the purpose of SDK is to allow third-party systems to register in DS
through the integration of DS SDK plugins. the SDK includes third-party
system registration, task list functions, task parameter functions, and
task execution functions. third party systems can delegate tasks to DS
triggers by integrating SDKs and corresponding implementations. for
example, in a data quality system, if a company wants to use its own data
quality system to implement data quality audit tasks, and the data
processing tasks are in DS, the processing tasks of the two platforms
cannot be triggered by dependencies. the purpose of this feature is to
solve the integration problem between DS and third-party systems. the above
introduction is about the modification of DS, including the main process of
task definition development and task execution. the main implementation of
this DSIP is to provide a dependency (DS SDK) for third-party systems; 2.
add a new node (Other System node) to DS

the design scheme can be seen in the following figures:


[image: image.png]

Reply via email to