[GitHub] [dolphinscheduler] caishunfeng commented on a diff in pull request #10821: [fix][plugin]: fix spark sql use custom parma.

Mon, 11 Jul 2022 20:47:50 -0700 


caishunfeng commented on code in PR #10821:
URL: https://github.com/apache/dolphinscheduler/pull/10821#discussion_r918521722


##########
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/runner/WorkflowExecuteRunnable.java:
##########
@@ -1300,6 +1301,10 @@ public void getPreVarPool(TaskInstance taskInstance, 
Set<String> preTask) {
             }
             if (allProperty.size() > 0) {
                 
taskInstance.setVarPool(JSONUtils.toJsonString(allProperty.values()));
+                Map<String, Object> taskParams = 
JSONUtils.parseObject(taskInstance.getTaskParams(), new 
TypeReference<Map<String, Object>>() {
+                });
+                taskParams.put("varPool", 
JSONUtils.toJsonString(allProperty.values()));

Review Comment:
   I think this is a unified process for all task, so after you fix the spark 
ui, the custome params can pass to worker.
   BTW, the custome params for the current task is localParams.
   This is a shell task example, hope it can help you. 
   
![image](https://user-images.githubusercontent.com/11962619/178404246-ad166c76-1ebf-44e5-9b69-ac5c3be41a1c.png)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to