hillyuen commented on issue #4584:
URL:
https://github.com/apache/incubator-dolphinscheduler/issues/4584#issuecomment-768070737
Now I specify worker1 to run this process, the result is: the execution of
the two task nodes are executed in the server corresponding to worker1.
Post the log of the master node below:
-----------------------------------------------------------------------
现在我指定worker1来运行这个流程，结果是：两个任务节点执行都在worker1对应的服务器中执行。
下面贴出master节点的日志：
[INFO] 2021-01-27 14:19:43.073
org.apache.dolphinscheduler.server.master.runner.MasterSchedulerService:[132] -
find one command: id: 129, type: START_PROCESS
[INFO] 2021-01-27 14:19:43.084
org.apache.dolphinscheduler.server.master.runner.MasterSchedulerService:[140] -
start master exec thread , split DAG ...
[INFO] 2021-01-27 14:19:43.085
org.apache.dolphinscheduler.server.master.runner.MasterExecThread:[315] -
prepare process :110 end
[INFO] 2021-01-27 14:19:43.086
org.apache.dolphinscheduler.server.master.runner.MasterExecThread:[792] - add
task to stand by list: 测试3work执行shell
[INFO] 2021-01-27 14:19:43.087
org.apache.dolphinscheduler.server.master.runner.MasterExecThread:[805] -
remove task from stand by list: 测试3work执行shell
[INFO] 2021-01-27 14:19:43.089
org.apache.dolphinscheduler.service.process.ProcessService:[839] - start submit
task : 测试3work执行shell, instance id:110, state: RUNNING_EXEUTION
[INFO] 2021-01-27 14:19:43.091
org.apache.dolphinscheduler.service.process.ProcessService:[852] - end submit
task to db successfully:测试3work执行shell state:SUBMITTED_SUCCESS complete,
instance id:110 state: RUNNING_EXEUTION
[INFO] 2021-01-27 14:19:43.095
org.apache.dolphinscheduler.server.master.runner.MasterTaskExecThread:[216] -
task ready to submit: TaskInstance{id=139, name='测试3work执行shell',
taskType='SHELL', processDefinitionId=4, processInstanceId=110,
processInstanceName='null',
taskJson='{"conditionResult":"{\"successNode\":[\"\"],\"failedNode\":[\"\"]}","conditionsTask":false,"depList":[],"dependence":"{}","forbidden":false,"id":"tasks-56734","maxRetryTimes":0,"name":"测试3work执行shell","params":"{\"rawScript\":\"echo
'hello world'\\nhostname
-f\",\"localParams\":[],\"resourceList\":[]}","preTasks":"[]","retryInterval":1,"runFlag":"NORMAL","taskInstancePriority":"MEDIUM","taskTimeoutParameter":{"enable":false,"interval":0},"timeout":"{\"enable\":false,\"strategy\":\"\"}","type":"SHELL","workerGroup":"work1"}',
state=SUBMITTED_SUCCESS, submitTime=Wed Jan 27 14:19:43 CST 2021,
startTime=null, endTime=null, host='null', executePath='null', logPath='null',
retryTimes=0, alertFlag=NO
, processInstance=null, processDefine=null, pid=0, appLink='null', flag=YES,
dependency='null', duration=null, maxRetryTimes=0, retryInterval=1,
taskInstancePriority=MEDIUM, processInstancePriority=MEDIUM,
dependentResult='null', workerGroup='work1', executorId=1, executorName='null'}
[INFO] 2021-01-27 14:19:43.096
org.apache.dolphinscheduler.server.master.runner.MasterTaskExecThread:[227] -
master submit success, task : 测试3work执行shell
[INFO] 2021-01-27 14:19:43.097
org.apache.dolphinscheduler.server.master.runner.MasterTaskExecThread:[116] -
wait task: process id: 110, task id:139, task name:测试3work执行shell complete
[INFO] 2021-01-27 14:19:43.528
org.apache.dolphinscheduler.server.master.processor.TaskAckProcessor:[69] -
taskAckCommand : TaskExecuteAckCommand{taskInstanceId=139, startTime=Wed Jan 27
14:19:43 CST 2021, host='10.66.93.89:1234', status=1,
logPath='/mnt/bigdata/dolphinscheduler/dolphinscheduler/logs/4/110/139.log',
executePath='/tmp/dolphinscheduler/exec/process/1/4/110/139'}
[INFO] 2021-01-27 14:19:43.544
org.apache.dolphinscheduler.server.master.processor.TaskResponseProcessor:[71]
- received command : TaskExecuteResponseCommand{taskInstanceId=139, status=7,
endTime=Wed Jan 27 14:19:43 CST 2021, processId=24139, appIds=''}
[INFO] 2021-01-27 14:19:45.117
org.apache.dolphinscheduler.server.master.runner.MasterTaskExecThread:[102] -
task :测试3work执行shell id:139, process id:110, exec thread completed
[INFO] 2021-01-27 14:19:46.100
org.apache.dolphinscheduler.server.master.runner.MasterExecThread:[864] - task
:测试3work执行shell, id:139 complete, state is SUCCESS
[INFO] 2021-01-27 14:19:46.101
org.apache.dolphinscheduler.server.master.runner.MasterExecThread:[792] - add
task to stand by list: 测试4执行shell
[INFO] 2021-01-27 14:19:46.102
org.apache.dolphinscheduler.server.master.runner.MasterExecThread:[540] -
taskName: 测试4执行shell completeDependTaskList: [测试3work执行shell]
[INFO] 2021-01-27 14:19:46.102
org.apache.dolphinscheduler.server.master.runner.MasterExecThread:[805] -
remove task from stand by list: 测试4执行shell
[INFO] 2021-01-27 14:19:46.104
org.apache.dolphinscheduler.service.process.ProcessService:[839] - start submit
task : 测试4执行shell, instance id:110, state: RUNNING_EXEUTION
[INFO] 2021-01-27 14:19:46.107
org.apache.dolphinscheduler.service.process.ProcessService:[852] - end submit
task to db successfully:测试4执行shell state:SUBMITTED_SUCCESS complete, instance
id:110 state: RUNNING_EXEUTION
[INFO] 2021-01-27 14:19:46.114
org.apache.dolphinscheduler.server.master.runner.MasterTaskExecThread:[216] -
task ready to submit: TaskInstance{id=140, name='测试4执行shell', taskType='SHELL',
processDefinitionId=4, processInstanceId=110, processInstanceName='null',
taskJson='{"conditionResult":"{\"successNode\":[\"\"],\"failedNode\":[\"\"]}","conditionsTask":false,"depList":["测试3work执行shell"],"dependence":"{}","forbidden":false,"id":"tasks-34099","maxRetryTimes":0,"name":"测试4执行shell","params":"{\"rawScript\":\"echo
'hello world'\\nhostname
-f\",\"localParams\":[],\"resourceList\":[]}","preTasks":"[\"测试3work执行shell\"]","retryInterval":1,"runFlag":"NORMAL","taskInstancePriority":"MEDIUM","taskTimeoutParameter":{"enable":false,"interval":0},"timeout":"{\"enable\":false,\"strategy\":\"\"}","type":"SHELL","workerGroup":"default"}',
state=SUBMITTED_SUCCESS, submitTime=Wed Jan 27 14:19:46 CST 2021,
startTime=null, endTime=null, host='null', executePath='null'
, logPath='null', retryTimes=0, alertFlag=NO, processInstance=null,
processDefine=null, pid=0, appLink='null', flag=YES, dependency='null',
duration=null, maxRetryTimes=0, retryInterval=1, taskInstancePriority=MEDIUM,
processInstancePriority=MEDIUM, dependentResult='null', workerGroup='default',
executorId=1, executorName='null'}
[INFO] 2021-01-27 14:19:46.114
org.apache.dolphinscheduler.server.master.runner.MasterTaskExecThread:[227] -
master submit success, task : 测试4执行shell
[INFO] 2021-01-27 14:19:46.115
org.apache.dolphinscheduler.server.master.runner.MasterTaskExecThread:[116] -
wait task: process id: 110, task id:140, task name:测试4执行shell complete
[ERROR] 2021-01-27 14:19:46.544
org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[153]
- dispatch error
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException:
fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=5338,
bodyLen=1681] due to no suitable worker , current task need to default worker
group execute
at
org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:87)
at
org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:150)
at
org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:112)
[ERROR] 2021-01-27 14:19:48.556
org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[153]
- dispatch error
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException:
fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=5339,
bodyLen=1681] due to no suitable worker , current task need to default worker
group execute
at
org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:87)
at
org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:150)
at
org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:112)
[INFO] 2021-01-27 14:19:51.577
org.apache.dolphinscheduler.server.master.processor.TaskAckProcessor:[69] -
taskAckCommand : TaskExecuteAckCommand{taskInstanceId=140, startTime=Wed Jan 27
14:19:51 CST 2021, host='10.30.220.164:1234', status=1,
logPath='/mnt/bigdata/dolphinscheduler/dolphinscheduler/logs/4/110/140.log',
executePath='/tmp/dolphinscheduler/exec/process/1/4/110/140'}
[INFO] 2021-01-27 14:19:51.595
org.apache.dolphinscheduler.server.master.processor.TaskResponseProcessor:[71]
- received command : TaskExecuteResponseCommand{taskInstanceId=140, status=7,
endTime=Wed Jan 27 14:19:51 CST 2021, processId=13473, appIds=''}
[INFO] 2021-01-27 14:19:53.151
org.apache.dolphinscheduler.server.master.runner.MasterTaskExecThread:[102] -
task :测试4执行shell id:140, process id:110, exec thread completed
[INFO] 2021-01-27 14:19:54.135
org.apache.dolphinscheduler.server.master.runner.MasterExecThread:[864] - task
:测试4执行shell, id:140 complete, state is SUCCESS
[INFO] 2021-01-27 14:19:55.138
org.apache.dolphinscheduler.server.master.runner.MasterExecThread:[764] - work
flow process instance [id: 110, name:work执行测试-0-1611728383074], state change
from RUNNING_EXEUTION to SUCCESS, cmd type: START_PROCESS
[INFO] 2021-01-27 14:19:55.146
org.apache.dolphinscheduler.server.master.runner.MasterExecThread:[925] -
process:110 end, state :SUCCESS
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
