Is it possible to extend the data source and increase the ssh type. I look at the data structure of DB is ok.
-------------------- DolphinScheduler(Incubator) Commtter Hemin Wen 温合民 [email protected] -------------------- 裴龙武 <[email protected]> 于2020年5月27日周三 上午10:55写道： > Good idea! It's can solve the exceptions where the numbers of SSH > connections exceeds. > > > We also have another problem of how to config remote server easily when we > have hundreds of tasks. > > > Now I use the local param to config remote server's IP,username,password. > But every task needs to config. > > > > > > > ------------------ 原始邮件 ------------------ > 发件人: "lidong dai"<[email protected]>; > 发送时间: 2020年5月26日(星期二) 晚上6:38 > 收件人: "dev"<[email protected]>; > > 主题: Re: [Feature] Support SSH Task > > > > good idea, I think "Control the number of task's concurrent" may be have > some other methods, for now, we don't have this feature, but when I > communicate with the team of China Unicom， they provide a new solve > solution： > we can add a task concurrent page, you can set a variable value like > 'ETL_JOB' =10 and so on, then give the task type add a option for choosing > ETL_JOB or other variable, and there may be many task also choose ETL_JOB > variable. when these tasks begin to running at the same time, the > Master > Server will check whether a task begin to run or wait(if the running tasks > num = 10) > > > Best egards > --------------- > DolphinScheduler(Incubator) PPMC > Lidong Dai 代立冬 > [email protected] > --------------- > > > 裴龙武 <[email protected]> 于2020年5月26日周二 下午2:34写道： > > > There is only two ways that I can think to do about this. > > > > > > 1. Simple SSH connect pool > > 2. Control the number of task's concurrent > > > > > > ------------------&nbsp;原始邮件&nbsp;------------------ > > 发件人:&nbsp;"Hemin Wen"<[email protected]&gt;; > > 发送时间:&nbsp;2020年5月22日(星期五) 晚上7:29 > > 收件人:&nbsp;"dev"<[email protected]&gt;; > > > > 主题:&nbsp;Re: [Feature] Support SSH Task > > > > > > > > What other friends think about this discuss > > > > > > -------------------- > > DolphinScheduler(Incubator) Commtter > > Hemin Wen&nbsp; 温合民 > > [email protected] > > -------------------- > > > > > > 裴龙武 <[email protected]&gt; 于2020年5月22日周五 上午10:20写道： > > > > &gt; Your are right, it can be solved the problem by modifying the > > &gt; number of server ssh connections.&amp;nbsp; > > &gt; > > &gt; The real reason is that I want to control the&amp;nbsp; > number of SSH > > &gt; connections and reuse the SSH connection, Because the remote > server is > > &gt; managed by other team. > > &gt; > > &gt; Otherwise, I think that we can also control the number of > parallel > > tasks > > &gt; in a workflow. > > &gt; > > &gt; > > &gt; It is a good idea that we can use the encapsulated ssh > connection to > > get > > &gt; clear > > &gt; exception information&amp;nbsp; > > &gt; > > &gt; > > &gt; > > &gt; > > &gt; > > &gt; > > &gt; > > &gt; > ------------------&amp;nbsp;原始邮件&amp;nbsp;------------------ > > &gt; 发件人:&amp;nbsp;"Hemin Wen"<[email protected] > &amp;gt;; > > &gt; 发送时间:&amp;nbsp;2020年5月21日(星期四) 中午11:07 > > &gt; 收件人:&amp;nbsp;"dev"<[email protected] > &amp;gt;; > > &gt; > > &gt; 主题:&amp;nbsp;Re: [Feature] Support SSH Task > > &gt; > > &gt; > > &gt; > > &gt; Thanks. > > &gt; > > &gt; About ssh connections pool, > > &gt; 1. If the connection is not enough, Can it be solved by > modifying the > > &gt; number of server ssh connections. > > &gt; 2. I think we have to look at whether a server's parallel > ssh tasks > > will > > &gt; reach the bottleneck of server connections. > > &gt; > > &gt; I think it is better to use the encapsulated ssh connection > to get > > clear > > &gt; exception information. > > &gt; > > &gt; -------------------- > > &gt; DolphinScheduler(Incubator) Commtter > > &gt; Hemin Wen&amp;nbsp; 温合民 > > &gt; [email protected] > > &gt; -------------------- > > &gt; > > &gt; > > &gt; 裴龙武 <[email protected]&amp;gt; 于2020年5月20日周三 下午11:29写道： > > &gt; > > &gt; &amp;gt; Thanks for your advise and suggestions. It may > has some > > exceptions if > > &gt; I > > &gt; &amp;gt; implement SSH Task as you said. Now, let me > explain my ideas. > > &gt; &amp;gt; > > &gt; &amp;gt; > > &gt; &amp;gt; e.g > > &gt; &amp;gt; > > &gt; &amp;gt; > > &gt; &amp;gt; My workflow DAG has 100 or more task, it's all > ssh task. I > > &gt; have&amp;amp;nbsp; two > > &gt; &amp;gt; questions. > > &gt; &amp;gt; > > &gt; &amp;gt; > > &gt; &amp;gt; 1、We must limit SSH connection and reuse the > SSH connections, > > &gt; otherwise > > &gt; &amp;gt; the remote server will be refused. > > &gt; &amp;gt; > > &gt; &amp;gt; > > &gt; &amp;gt; 2、If we config all of these task, we have a lot > of repeat > > work to do. > > &gt; &amp;gt; > > &gt; &amp;gt; > > &gt; &amp;gt; By the way, the way I'm currently verifying is > I use local > > params to > > &gt; &amp;gt; config the remote server's host,user,password > > &gt; &amp;gt; > > &gt; &amp;gt; > > &gt; &amp;gt; > > > ------------------&amp;amp;nbsp;原始邮件&amp;amp;nbsp;------------------ > > &gt; &amp;gt; 发件人:&amp;amp;nbsp;"lidong dai"< > [email protected] > > &amp;amp;gt;; > > &gt; &amp;gt; 发送时间:&amp;amp;nbsp;2020年5月20日(星期三) 晚上10:56 > > &gt; &amp;gt; 收件人:&amp;amp;nbsp;"dev"< > [email protected] > > &amp;amp;gt;; > > &gt; &amp;gt; > > &gt; &amp;gt; 主题:&amp;amp;nbsp;Re: [Feature] Support SSH > Task > > &gt; &amp;gt; > > &gt; &amp;gt; > > &gt; &amp;gt; > > &gt; &amp;gt; hi, I see you will Jsch, does his license > compitable with > > apache v2 > > &gt; &amp;gt; license, I see the following words " > > &gt; &amp;gt; JSch 0.0.* was released under the GNU LGPL > license. Later, > > we have > > &gt; switched > > &gt; &amp;gt; over to a BSD-style license. > > &gt; &amp;gt; " on > https://github.com/is/jsch/blob/master/LICENSE.txt . > > you know, > > &gt; LGPL > > &gt; &amp;gt; license not compitable with apache license. but > BSD is ok , > > so the > > &gt; version > > &gt; &amp;gt; of Jsch is key point > > &gt; &amp;gt; > > &gt; &amp;gt; second, Connection Pool is heavy resource, > Please use > > carefully > > &gt; &amp;gt; > > &gt; &amp;gt; and I'm a little confused, why there must be > add a new > > &gt; Table,&amp;amp;nbsp; other > > &gt; &amp;gt; task > > &gt; &amp;gt; types also no seperate table,&amp;amp;nbsp; > could you > > consider task_type > > &gt; and > > &gt; &amp;gt; task_json > > &gt; &amp;gt; in t_ds_task_instance ? > > &gt; &amp;gt; > > &gt; &amp;gt; > > &gt; &amp;gt; > > &gt; &amp;gt; Best Regards > > &gt; &amp;gt; --------------- > > &gt; &amp;gt; DolphinScheduler(Incubator) PPMC > > &gt; &amp;gt; Lidong Dai 代立冬 > > &gt; &amp;gt; [email protected] > > &gt; &amp;gt; --------------- > > &gt; &amp;gt; > > &gt; &amp;gt; > > &gt; &amp;gt; 裴龙武 <[email protected]&amp;amp;gt; > 于2020年5月20日周三 下午9:57写道： > > &gt; &amp;gt; > > &gt; &amp;gt; &amp;amp;gt; I want to describe my idea. > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; First, we must design a host > table that's named > > &gt; t_ds_ssh_host, e.g > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; - id > > &gt; &amp;gt; &amp;amp;gt; - name > > &gt; &amp;gt; &amp;amp;gt; - ip / host > > &gt; &amp;gt; &amp;amp;gt; - user > > &gt; &amp;gt; &amp;amp;gt; - password > > &gt; &amp;gt; &amp;amp;gt; - max_connection > > &gt; &amp;gt; &amp;amp;gt; - create_time > > &gt; &amp;gt; &amp;amp;gt; - update_time > > &gt; &amp;gt; &amp;amp;gt; - user_id > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; Second, Shell can execute in > worker server or > > remote host. > > &gt; We can > > &gt; &amp;gt; choose a > > &gt; &amp;gt; &amp;amp;gt; host in host's list. > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; Third, when workflow running, > we must maintain > > the remote > > &gt; host's > > &gt; &amp;gt; &amp;amp;gt; connection pool. > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; Finally, when workflow finish, > we must release > > the > > &gt; connection pool. > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; Why do we have to maintain a > connection pool. > > Because it is > > &gt; very easy > > &gt; &amp;gt; to > > &gt; &amp;gt; &amp;amp;gt; get exceptions when it's has > many SSH task. > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; > > > ------------------&amp;amp;amp;nbsp;原始邮件&amp;amp;amp;nbsp;------------------ > > &gt; &amp;gt; &amp;amp;gt; > 发件人:&amp;amp;amp;nbsp;"lidong dai"< > > [email protected] > > &gt; &amp;amp;amp;gt;; > > &gt; &amp;gt; &amp;amp;gt; > 发送时间:&amp;amp;amp;nbsp;2020年5月20日(星期三) 晚上7:12 > > &gt; &amp;gt; &amp;amp;gt; > 收件人:&amp;amp;amp;nbsp;"dev"< > > [email protected] > > &gt; &amp;amp;amp;gt;; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; 主题:&amp;amp;amp;nbsp;Re: > [Feature] Support SSH > > Task > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; glad to hear that you will > implement this > > feature > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; Best Regards > > &gt; &amp;gt; &amp;amp;gt; --------------- > > &gt; &amp;gt; &amp;amp;gt; DolphinScheduler(Incubator) > PPMC > > &gt; &amp;gt; &amp;amp;gt; Lidong Dai 代立冬 > > &gt; &amp;gt; &amp;amp;gt; [email protected] > > &gt; &amp;gt; &amp;amp;gt; --------------- > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; 裴龙武 <[email protected] > &amp;amp;amp;gt; > > 于2020年5月20日周三 下午3:47写道： > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; My code > is not perfect yet. I > > will write a > > &gt; detailed design > > &gt; &amp;gt; document. > > &gt; &amp;gt; &amp;amp;gt; Then > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; I will > realize this feature > > about our > > &gt; discussion result. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > ------------------&amp;amp;amp;amp;nbsp;原始邮件&amp;amp;amp;amp;nbsp;------------------ > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > 发件人:&amp;amp;amp;amp;nbsp;"wenhemin"< > > &gt; [email protected]&amp;amp;amp;amp;gt;; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > 发送时间:&amp;amp;amp;amp;nbsp;2020年5月18日(星期一) 晚上7:50 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > 收件人:&amp;amp;amp;amp;nbsp;"裴龙武"<[email protected] > > &gt; &amp;amp;amp;amp;gt;;"dev"< > > &gt; &amp;gt; &amp;amp;gt; > [email protected] > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt;; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > 主题:&amp;amp;amp;amp;nbsp;Re: > > [Feature] Support SSH > > &gt; Task and Support > > &gt; &amp;gt; dummy task > > &gt; &amp;gt; &amp;amp;gt; like airflow > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; Thanks > for writing detailed > > documentation. I > > &gt; think this is > > &gt; &amp;gt; also a > > &gt; &amp;gt; &amp;amp;gt; missing > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; feature > of DS. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; About the > extension point: > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; 1.Can ssh > tasks be merged into > > shell tasks. > > &gt; Essentially, > > &gt; &amp;gt; they all > > &gt; &amp;gt; &amp;amp;gt; execute > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; shell > commands. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; 2.About > dummy task, DS has the > > function of > > &gt; disable nodes, I > > &gt; &amp;gt; do n’t > > &gt; &amp;gt; &amp;amp;gt; know if > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; this > requirement is met. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; The > script from AirFlow to > > Dolphin is great. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; 在 > > 2020年5月18日，09:28，裴龙武 < > > &gt; [email protected]&amp;amp;amp;amp;gt; > > &gt; &amp;gt; 写道： > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; OK, 3Q! > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; First, I > > will ensure that open > > &gt; source can use. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; Second, I > > think we must > > &gt; discuss deeply. I write > > &gt; &amp;gt; a more > > &gt; &amp;gt; &amp;amp;gt; detailed > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; document. > You can check the > > attachment. I also > > &gt; send the > > &gt; &amp;gt; document to > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; DaiLidong. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; > > Third,&amp;amp;amp;amp;nbsp; I'll > > &gt; give you the error of > > &gt; &amp;gt; not using SSH > > &gt; &amp;gt; &amp;amp;gt; connection pool. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; > > ------------------ 原始邮件 > > &gt; ------------------ > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; 发件人: > > "wenhemin"< > > &gt; [email protected]&amp;amp;amp;amp;gt;; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; 发送时间: > > 2020年5月14日(星期四) 晚上7:26 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; 收件人: > > "裴龙武"<[email protected] > > &gt; &amp;amp;amp;amp;gt;; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; 主题: Re: > > [Feature] Support SSH > > &gt; Task and Support > > &gt; &amp;gt; dummy task > > &gt; &amp;gt; &amp;amp;gt; like airflow > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; Great! > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; I think, > > Can ssh tasks be > > &gt; merged into shell > > &gt; &amp;gt; tasks,&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; &amp;amp;gt; execute > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; script > locally or remotely, > > Configure on the > > &gt; front end. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; About ssh > > connect pool, I did > > &gt; not find it > > &gt; &amp;gt; necessary to use > > &gt; &amp;gt; &amp;amp;gt; the > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > connection pool. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; BTW, Look > > at the code to > > &gt; introduce additional > > &gt; &amp;gt; jar packages, > > &gt; &amp;gt; &amp;amp;gt; You also > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; need to > ensure that open > > source can use the > > &gt; license of this > > &gt; &amp;gt; jar > > &gt; &amp;gt; &amp;amp;gt; package. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; 在 > > &gt; 2020年5月14日，16:20，裴龙武 < > > &gt; &amp;gt; [email protected] <mailto: > > &gt; &amp;gt; &amp;amp;gt; [email protected] > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; 写道： > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; 1. The > > &gt; priority between these tasks > > &gt; &amp;gt; is also depended > > &gt; &amp;gt; &amp;amp;gt; on the > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; dolphin > DAG define. When the > > front task is not > > &gt; finished, it > > &gt; &amp;gt; not > > &gt; &amp;gt; &amp;amp;gt; execute > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; next task. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; 2. I extend > > &gt; ssh task. I also use > > &gt; &amp;gt; local params to > > &gt; &amp;gt; &amp;amp;gt; config ssh host, > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; user and > password. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; E.g: > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; public static > > &gt; AbstractTask > > &gt; &amp;gt; &amp;amp;gt; newTask(TaskExecutionContext > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > taskExecutionContext, Logger > > logger) > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; throws > > &gt; &amp;gt; &amp;amp;gt; IllegalArgumentException { > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > Boolean > > &gt; &amp;gt; enable = > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > JSONUtils.parseObject(taskExecutionContext.getTaskParams()).getBoolean("enable"); > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > if > > &gt; &amp;gt; (enable != null > > &gt; &amp;gt; &amp;amp;gt; > &amp;amp;amp;amp;amp;&amp;amp;amp;amp;amp; > > enable == false ) { > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; return new > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > DummyTask(taskExecutionContext, logger); > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > } > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > switch > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > (EnumUtils.getEnum(TaskType.class,taskExecutionContext.getTaskType())) { > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; case SHELL: > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; &amp;amp;gt; return new > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > ShellTask(taskExecutionContext, logger); > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; case > > &gt; &amp;gt; &amp;amp;gt; PROCEDURE: > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; &amp;amp;gt; return new > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > ProcedureTask(taskExecutionContext, logger); > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; case SQL: > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; &amp;amp;gt; return new > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > SqlTask(taskExecutionContext, > > logger); > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; case MR: > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; &amp;amp;gt; return new > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > MapReduceTask(taskExecutionContext, logger); > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; case SPARK: > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; &amp;amp;gt; return new > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > SparkTask(taskExecutionContext, logger); > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; case FLINK: > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; &amp;amp;gt; return new > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > FlinkTask(taskExecutionContext, logger); > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; case PYTHON: > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; &amp;amp;gt; return new > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > PythonTask(taskExecutionContext, logger); > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; case HTTP: > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; &amp;amp;gt; return new > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > HttpTask(taskExecutionContext, > > logger); > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; case DATAX: > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; &amp;amp;gt; return new > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > DataxTask(taskExecutionContext, logger); > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; case SQOOP: > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; &amp;amp;gt; return new > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > SqoopTask(taskExecutionContext, logger); > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; case SSH: > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; &amp;amp;gt; return new > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > SSHTask(taskExecutionContext, > > logger); > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; default: > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; &amp;amp;gt; logger.error("unsupport task > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; type: {}", > > taskExecutionContext.getTaskType()); > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; &amp;amp;gt; throw new > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > IllegalArgumentException("not > > support task > > &gt; type"); > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > } > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; } > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; 3. I am not > > &gt; sure that it supports > > &gt; &amp;gt; window or not. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; ------------------ 原始邮件 > > &gt; &amp;gt; ------------------ > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; 发件人: > > &gt; "wenhemin"<[email protected] > > &gt; &amp;gt; <mailto: > > &gt; &amp;gt; &amp;amp;gt; [email protected] > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; 发送时间: > > &gt; 2020年5月14日(星期四) 下午3:46 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; 收件人: "裴龙武"< > > &gt; [email protected] > > &gt; &amp;gt; <mailto:[email protected] > > &gt; &amp;gt; &amp;amp;gt; > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; 主题: Re: > > &gt; [Feature] Support SSH Task > > &gt; &amp;gt; and Support dummy > > &gt; &amp;gt; &amp;amp;gt; task like > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; airflow > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; Sorry, My > > &gt; previous description is > > &gt; &amp;gt; not very clear. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; I want to ask > > &gt; some questions: > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; 1.How to > > &gt; control the priority > > &gt; &amp;gt; between ssh tasks? > > &gt; &amp;gt; &amp;amp;gt; There may be > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; some ssh > tasks that have been > > waiting for > > &gt; execution. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; 2.I understand > > &gt; what you want to > > &gt; &amp;gt; solve is the problem > > &gt; &amp;gt; &amp;amp;gt; of executing > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; remote > ssh scripts in batches. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > So, not > > &gt; &amp;gt; sure how to use this > > &gt; &amp;gt; &amp;amp;gt; function. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; 3.I don't know > > &gt; if this supports > > &gt; &amp;gt; windows system. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; 在 > > 2020年5月13日，20:56，裴龙武 < > > &gt; &amp;gt; [email protected] > > &gt; &amp;gt; &amp;amp;gt; <mailto: > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > [email protected] > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; 写道： > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; I > use > > spin lock. Here > > &gt; &amp;gt; is my code. Of course > > &gt; &amp;gt; &amp;amp;gt; , it's not > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; perfect. > I just do a test. To > > my surprise, it > > &gt; is the result > > &gt; &amp;gt; of the > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; execution > is the same as the > > AirFlow > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > 我通过模拟自选锁方式实现，附件中是我的代码，当然，这并不完善。我拿这个做了测试。令我惊喜的是，我得到了和 AirFlow > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; 相同的结果。 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > ------------------ 原始邮件 > > &gt; &amp;gt; ------------------ > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > 发件人: > > "whm_777"< > > &gt; &amp;gt; [email protected] <mailto: > > &gt; &amp;gt; &amp;amp;gt; [email protected] > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > 发送时间: > > 2020年5月13日(星期三) > > &gt; &amp;gt; 晚上7:21 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > 收件人: > > "裴龙武"< > > &gt; &amp;gt; [email protected] <mailto: > > &gt; &amp;gt; &amp;amp;gt; [email protected] > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > 主题: Re: > > [Feature] > > &gt; &amp;gt; Support SSH Task and > > &gt; &amp;gt; &amp;amp;gt; Support dummy task > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; like > airflow > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > You can > > modify the > > &gt; &amp;gt; maximum number of linux > > &gt; &amp;gt; &amp;amp;gt; ssh connections. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; If > use > > ssh connection > > &gt; &amp;gt; pool, How to control > > &gt; &amp;gt; &amp;amp;gt; the priority of > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; ssh? > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > 在 > > &gt; &amp;gt; 2020年5月13日，18:01，裴龙武 < > > &gt; &amp;gt; &amp;amp;gt; [email protected] <mailto: > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > [email protected] > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; 写道： > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > First 3Q， > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; I&amp;amp;amp;amp;nbsp; use more than 100 > task > > &gt; &amp;gt; &amp;amp;gt; node. But SSH connections > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; are > limited. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > 我是使用了100多个任务节点，但服务器SSH连接是有限制的，超过后，就会报错了。下面是我扩展SSH任务节点后的一张截图，另外这个DAG是我从AirFlow转换过来的。 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; <[email protected] > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; ------------------ 原始邮件 > > &gt; &amp;gt; &amp;amp;gt; ------------------ > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > 发件人: > > &gt; &amp;gt; "whm_777"<[email protected] > > &gt; &amp;gt; &amp;amp;gt; <mailto:[email protected] > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > 发送时间: > > &gt; &amp;gt; 2020年5月13日(星期三) 下午5:50 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > 收件人: "裴龙武"< > > &gt; &amp;gt; [email protected] <mailto: > > &gt; &amp;gt; &amp;amp;gt; [email protected] > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > 主题: Re: > > &gt; &amp;gt; [Feature] Support SSH Task > > &gt; &amp;gt; &amp;amp;gt; and Support dummy task > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; like > airflow > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > E.g. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; rtn_code=`ssh -o > > &gt; &amp;gt; &amp;amp;gt; ServerAliveInterval=60 -p xxxx > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > [email protected] <mailto: > > &gt; [email protected]&amp;amp;amp;amp;gt; > > &gt; &amp;gt; ‘shell > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > command&amp;amp;amp;amp;nbsp; > > &gt; &amp;amp;amp;amp;gt;/dev/null > > &gt; &amp;gt; > 2&amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;1; echo $?'` > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > if [ > > &gt; &amp;gt; "$rtn_code" -eq 0 ]; then > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; &amp;amp;gt; echo "成功" > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; &amp;amp;gt; exit 0 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > else > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; &amp;amp;gt; echo "失败" > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; &amp;amp;gt; exit 1 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > fi > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > Batch shell > > &gt; &amp;gt; command is not supported. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > Multiple > > &gt; &amp;gt; servers can be split into > > &gt; &amp;gt; &amp;amp;gt; multiple task nodes. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; 在 2020年5月13日，17:40，裴龙武 < > > &gt; &amp;gt; &amp;amp;gt; [email protected] <mailto: > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > [email protected] > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; 写道： > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; Could you give me a > > &gt; &amp;gt; &amp;amp;gt; example，3Q. 能否给我一个例子，谢谢！ > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; By the way, I have more than > > &gt; &amp;gt; &amp;amp;gt; 100 tasks in one DAG. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; These > tasks connect two other > > server to > > &gt; execute. So SSH > > &gt; &amp;gt; tasks must > > &gt; &amp;gt; &amp;amp;gt; have > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; pool to > manager. Now I use > > JSch and realize a > > &gt; simple pool. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; 顺带说一下，在我的实际场景中，我有100多个 SSH > > &gt; &amp;gt; &amp;amp;gt; 任务，这些任务连接两台任务服务器进行任务执行。所以 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; SSH > 任务进行连接时，必须使用连接池进行管理。当前我使用 > > JSch，并实现了一个简单的连接池。 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; ------------------ 原始邮件 > > &gt; &amp;gt; &amp;amp;gt; ------------------ > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; 发件人: "wenhemin"< > > &gt; &amp;gt; &amp;amp;gt; [email protected] <mailto: > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > [email protected] > > &gt; &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; 发送时间: 2020年5月13日(星期三) 下午5:24 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; 收件人: "dev"< > > &gt; &amp;gt; &amp;amp;gt; > [email protected] <mailto: > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > [email protected] > > &gt; &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; 主题: Re: [Feature] Support > > &gt; &amp;gt; &amp;amp;gt; SSH Task and Support dummy > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; task like > airflow > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; The shell node is supports > > &gt; &amp;gt; &amp;amp;gt; remote calling, and get > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; the > remote command result code. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; 在 > > &gt; &amp;gt; &amp;amp;gt; 2020年5月13日，15:16，裴龙武 < > [email protected] > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; <mailto: > [email protected] > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; 写道： > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; Dear ALL： > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; Support Linux SSH > > &gt; &amp;gt; &amp;amp;gt; Task 支持 Linux SSH 任务 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; 场景描述：当前项目中，工作流的任务的目标是执行不同服务器 > Shell 脚本，Shell > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > 脚本是保存在业务服务器的固定目录。当 Worker > > &gt; 调度执行时，需要通过固定用户登录这些服务器，然后执行 Shell > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > 脚本并获取这些任务执行的状态，其中服务器地址、用户名、密码可配置。 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; For example, in my > > &gt; &amp;gt; &amp;amp;gt; project, the workflow's tasks > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; want to > execute shell scripts > > where are in > > &gt; different server's > > &gt; &amp;gt; &amp;amp;gt; different > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > directory. When worker execute > > these shell > > &gt; scripts, it must > > &gt; &amp;gt; use the > > &gt; &amp;gt; &amp;amp;gt; same > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; user to > login these server. > > Also, the worker > > &gt; can get the > > &gt; &amp;gt; executing > > &gt; &amp;gt; &amp;amp;gt; state of > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; these > server. We can config > > these server 's > > &gt; host,user and > > &gt; &amp;gt; password. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; SSH Task is very > > &gt; &amp;gt; &amp;amp;gt; useful for most user SSH > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > 任务对大多数用户是非常有用的 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; 分布式调度任务所执行的 Shell > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > 脚本是处于不同的业务服务器，都有其固定的业务，这些业务服务器不是 Worker，只是需要 > > &gt; Worker > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > 调度执行，我们只需要传递不同的参数，让服务器执行任务脚本即可。 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; In > > &gt; &amp;gt; &amp;amp;gt; dolphinscheduler, the most > executing tasks > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; are in > different servers who > > are not workers. > > &gt; These servers > > &gt; &amp;gt; also have > > &gt; &amp;gt; &amp;amp;gt; their > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; different > fixed services. We > > just have to pass > > &gt; different > > &gt; &amp;gt; parameters to > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; schedule > these shell scripts > > to execute. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; Python has a module > > &gt; &amp;gt; &amp;amp;gt; to execute ssh script Python > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > 有固定的工具包，可执行这些SSH Shell 脚本 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; Python > > &gt; &amp;gt; &amp;amp;gt; 有一个可执行远程服务器SSH > Shell脚本的模块，其名字为：paramiko。 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; Python has a module > > &gt; &amp;gt; &amp;amp;gt; that can execute SSH Shell > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; script. > It's paramiko. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; Others 其他内容 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > 我发现之前的改进功能中也有关于这个的描述，不过相对简单。功能更新地址 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; I found this > > &gt; &amp;gt; &amp;amp;gt; described in previous feature, > but > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; it was > relatively simple. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; Feature URL > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; 另外，我通过 Shell Task > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > 方式去执行远程任务会非常不便，下面是我的脚本，不知道是否有更好的方式。 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; In addition, it is > > &gt; &amp;gt; &amp;amp;gt; very inconvenient for me to > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; perform > remote tasks through > > Shell Task. Here > > &gt; is my script. > > &gt; &amp;gt; I don't > > &gt; &amp;gt; &amp;amp;gt; know if > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; there's a > better way. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; sshpass -p > > &gt; &amp;gt; &amp;amp;gt; 'password' ssh user@host echo > 'ssh > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; success' > echo 'Hello World' > > &gt; -&amp;amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > /home/dolphinscheduler/test/hello.txt > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; echo 'end' > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; Support dummy task > > &gt; &amp;gt; &amp;amp;gt; like airflow 支持像 Airflow > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; 中的虚拟任务 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; 场景描述：项目中，有已经产品化的 > > &gt; &amp;gt; &amp;amp;gt; DAG 文件，DAG > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; > > 文件中包括不同的模块，这些模块之间的有些点是相互依赖的，有些不是，在用户购买不同模块时，需要把未购买模块且其他已购模块未依赖的点设置为 > > &gt; &amp;gt; &amp;amp;gt; Dummy > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > Task，这样实际这些任务就不会执行，这样设置的好处是产品统一性和图的完整性，在AirFlow中，这些是通过DummyOperator完成的。 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; For example, in my > > &gt; &amp;gt; &amp;amp;gt; project, it has a productized > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; DAG file. > The file contains > > different modules, > > &gt; some of which > > &gt; &amp;gt; are > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > interdependent and some of > > which are not. When > > &gt; customers > > &gt; &amp;gt; purchase > > &gt; &amp;gt; &amp;amp;gt; different > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; modules, > we need to set some > > tasks as dummy > > &gt; tasks, which > > &gt; &amp;gt; some modules > > &gt; &amp;gt; &amp;amp;gt; are > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; not > purchased and the > > purchased module is not > > &gt; dependent. > > &gt; &amp;gt; Because of > > &gt; &amp;gt; &amp;amp;gt; this > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; setting, > these dummy tasks are > > actually not > > &gt; executed. The > > &gt; &amp;gt; benefits of > > &gt; &amp;gt; &amp;amp;gt; this > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; setup are > product unity and > > diagram integrity. > > &gt; In airflow, > > &gt; &amp;gt; these task > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; execute > by dummy operator. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; ** Realize 实现方式** > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; Dummy Task > > &gt; &amp;gt; &amp;amp;gt; > 本身实现很简单，只是需要与其他任务配合使用，但任务执行方式设置为 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; dummy > 时，实际的任务不执行，执行 Dummy Task。 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; Dummy Task is easy > > &gt; &amp;gt; &amp;amp;gt; to realize, but it need to > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; use with > other different > > tasks. When the task's > > &gt; executed > > &gt; &amp;gt; type is set > > &gt; &amp;gt; &amp;amp;gt; to > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; dummy > type, the task are > > executed as a dummy > > &gt; task and the > > &gt; &amp;gt; real task > > &gt; &amp;gt; &amp;amp;gt; is not > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; executed. > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; 顺带说一下，因为项目着急测试使用，我Fork了开发版本，实现两种任务类型。在后续的版本中是否能够支持。 > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;amp;amp;gt; By the way，I > > &gt; &amp;gt; &amp;amp;gt; already realize these > > two&amp;amp;amp;amp;amp;nbsp; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; features > in my fork > > &gt; branch.&amp;amp;amp;amp;amp;nbsp;Whether the > > &gt; &amp;gt; follow-up > > &gt; &amp;gt; &amp;amp;gt; release can be > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; supported > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; > > &gt; &amp;gt; > > &gt; > > > <SSHClient.java&amp;amp;amp;amp;gt;<SSHPool.java&amp;amp;amp;amp;gt;<SSHTask.java&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > > &amp;amp;amp;amp;gt;&amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; > > &gt; &amp;gt; &amp;amp;gt; &amp;amp;amp;gt; > &amp;amp;amp;amp;gt; > > &gt; <项目场景中关于Dolphin的一些扩展点.pdf&amp;amp;amp;amp;gt;
