+1 This feature looks very interesting and I'm looking forward to it. simon <[email protected]> 于2021年1月12日周二 上午9:30写道：
> It's good,if you need help i can join > > > 原始邮件 > 发件人:Xingjie Wang(联通集团联通大数据有限公司本部)[email protected] > 收件人:[email protected][email protected] > 发送时间:2021年1月8日(周五) 10:55 > 主题:[DISCUSS]The global params that Task can change > > > Hi Dev Team > > The scheme of The global params that Task can change as blow。 > Here is the ISSUE about this DISCUSS。 > https://github.com/apache/incubator-dolphinscheduler/issues/3805 > > Discussion on the implementation scheme of global params (parameter > transfer between nodes) > > At present, we have implemented the global params function > 1. The params takes effect in the process instance, each node can accept > the variable, SQL and SHELL nodes can modify the params value, and > subsequent nodes use the params as the modified value. (requirement 1) > 2. The subprocess node passes in variables, and the subprocess can receive > params values (requirement 2) > > The detail of the scheme as follows > > a： requirement 1 > 1. When saving workflow, users will add global params definition (global > definition) in the pop-up box after clicking save button, and save the > defined params name and default value (if there is an initial value) to > t_ds_process_definition.global_Params field (using the existing function). > When saving, it will verify that the params names used by the node are all > from global definition. > 2. When the workflow is called up for instantiation, get the value of > globalparams from definition and initialize the params value to > t_ds_process_instance.global_Params field. (at this time, the initial value > of the workflow instance is the default value defined globally) > 3. About input parameters: when creating TaskInstance, take variables from > localparam, and set the value from t_ds_process_instance.global_params to > localparam. After storing the task instance, the task instance is saved in > the database and distributed to the worker, and the subsequent parameter > processing follows the original part. > 4. About output parameters: after the task execution of SQL node and SHELL > node is finished(SQL query results, only support one field one value, if > multiple fields or rows of data will be formatted as JSON;The SHELL uses > the output of the last line as the result), write the result of task > execution into TaskExecuteResponseCommand, return to master together with > the worker execution result information, and the master will update the > output t_ds_process_instance.global_params and update it to the database. > > b： requirement 2 > 1. About subprocess node: judge whether there is parameter passing when > generating subprocess command. If there is parameter passing, the parameter > will be written into the command table. > 2. When the subprocess is instantiated, judge that the command contains > the parameter passed down by the main process, and update the value to > t_ds_process_instance.global_params field. The subsequent part is > consistent with requirement 1 > To implement requirement 2, you need to add COLUMN son_Params in > t_ds_command table. > > c： requirement 3 > In addition, in order to realize the function of modifying parameters > easily when the workflow is executed manually > 1. When you click the [run] button, the window to modify the global params > value will pop up, and save it into command.son_Params . The follow-up > process is consistent with requirement 2 > > > The above three demand plans are submitted in two versions. The first > version completes demand 1, and the second version completes demand 2 and 3 > > 关于全局变量(节点间参数传递)实现方案的讨论： > 目前我们已经实现了全局变量功能，具体为： > 1.变量在流程实例内生效，每个节点可接受变量，sql和shell节点可修改变量值，后续节点在使用该变量时为修改后的值。（功能1） > 2.子流程节点传入变量，子流程可以接受到变量值（功能2） > 实现方案为： > 一：功能1 > 1. > 用户在保存工作流时，会在点击保存按钮后的弹出框内添加全局变量定义（全局定义），将定义的变量名和默认值（如果有初始值的话）保存到t_ds_process_definition.global_params字段（沿用已有功能），保存时会校验节点所使用的的变量名全部来自【全局定义】。 > 2. > 在工作流被调起实例化的时候，从definition中获取globalParams的值，并将变量值初始化到t_ds_process_instance.global_params字段中。（此时工作流实例的初始值为全局定义的默认值） > 3. > 关于入参：在创建taskInstance时，从localParam中取出变量，并将t_ds_process_instance.global_params中最新的值更新至localParam中。存入taskInstance，后续将taskInstance保存数据库，并分发给worker，后续参数处理沿用原有部分。 > 4. > 关于出参：在sql节点和shell节点任务执行结束后，将任务执行的结果（sql查询的结果，只支持一个字段一个值，如果是多个字段或多行数据会格式化为json；shell使用最后一行的输出作为结果）写入TaskExecuteResponseCommand，随worker执行结果信息一起返回master，master将出参更新t_ds_process_instance.global_params，并将其更新至数据库。 > 二：功能2 > 1. 关于子流程节点：生成子流程的command时判断是否有参数传递，如果有参数传递，会将参数写入到command表中。 > 2. > 在子流程实例化时判断command中包含主流程向下传递的参数，将该值更新至t_ds_process_instance.global_params字段中。后续部分与需求1一致 > 实现功能2需要在command中增加son_params字段。 > 三：另外为了实现在手动执行工作流时方便修改参数设计功能3： > 1． 在点击【运行】按钮时，弹出修改全局变量值的窗口，保存进command. son_params中。后续流程与需求2一致 > > > 以上三个需求计划两个版本提交，第一个版本完成需求1，第二个版本完成需求2、3 > > > 以下为该方案流程图： > > 附件为该流程图的Visio版。 > Welcome more discussion and suggestions. > > 如果您错误接收了该邮件，请通过电子邮件立即通知我们。请回复邮件到[email protected]，即可以退订此邮件。我们将立即将您的信息从我们的发送目录中删除。 > If you have received this email in error please notify us immediately by > e-mail. Please reply [email protected],you can unsubscribe from > this mail. We will immediately remove your information from send catalogue > of our. -- DolphinScheduler(Incubator) PPMC BaoLiang 鲍亮 [email protected]
