zhongjiajie commented on code in PR #10639: URL: https://github.com/apache/dolphinscheduler/pull/10639#discussion_r907973829
########## docs/docs/en/guide/task/conditions.md: ########## @@ -51,7 +51,7 @@ After you finish creating the workflow, you can run the workflow online. You can In the above figure, the task status marked with a green check mark is the successfully executed task node. -## Notice +## Note Review Comment: Hi @sneh-wha I find some of the task doc change `notice` to `note` but some of them do not. Could you change the other part to keep them consistency ########## docs/docs/en/guide/task/conditions.md: ########## @@ -4,25 +4,25 @@ Condition is a conditional node, that determines which downstream task should ru ## Create Task -- Click `Project -> Management-Project -> Name-Workflow Definition`, and click the "Create Workflow" button to enter the DAG editing page. +- Click `Project Management -> Project Name -> Workflow Definition`, and click the "`Create Workflow`" button to enter the DAG editing page. - Drag from the toolbar <img src="../../../../img/conditions.png" width="20"/> task node to canvas. -## Parameter - -- Node name: The node name in a workflow definition is unique. -- Run flag: Identifies whether this node schedules normally, if it does not need to execute, select the `prohibition execution`. -- Descriptive information: Describe the function of the node. -- Task priority: When the number of worker threads is insufficient, execute in the order of priority from high to low, and tasks with the same priority will execute in a first-in first-out order. -- Worker grouping: Assign tasks to the machines of the worker group to execute. If `Default` is selected, randomly select a worker machine for execution. -- Times of failed retry attempts: The number of times the task failed to resubmit. You can select from drop-down or fill-in a number. -- Failed retry interval: The time interval for resubmitting the task after a failed task. You can select from drop-down or fill-in a number. -- Timeout alarm: Check the timeout alarm and timeout failure. When the task runs exceed the "timeout", an alarm email will send and the task execution will fail. -- Downstream tasks selection: Depending on the status of the predecessor task, you can jump to the corresponding branch, currently two branches are supported: success, failure - - Success: When the upstream task runs successfully, run the success branch. - - Failure: When the upstream task runs failed, run the failure branch. -- Upstream condition selection: can select one or more upstream tasks for conditions. - - Add an upstream dependency: the first parameter is to choose a specified task name, and the second parameter is to choose the upstream task status to trigger conditions. - - Select upstream task relationship: use `and` and `or` operators to handle the complex relationship of upstream when there are multiple upstream tasks for conditions. +## Task Parameters + +| **Parameter** | **Description** | +| -------- | ---------- | +| Node Name | Set the name of the task. Node names within a workflow definition are unique. | +| Run flag | Indicates whether the node can be scheduled normally. If it is not necessary to execute, you can turn on the prohibiting execution switch. | +| Description | Describes the function of this node. | +| Task priority | When the number of worker threads is insufficient, they are executed in order from high to low according to the priority, and they are executed according to the first-in, first-out principle when the priority is the same. | +| Worker group | The task is assigned to the machines in the worker group for execution. If Default is selected, a worker machine will be randomly selected for execution. | +| Task group name | The group in Resources, if not configured, it will not be used. | +| Environment Name | Configure the environment in which to run the script. | +| Number of failed retries | The number of times the task is resubmitted after failure. It supports drop-down and manual filling. | +| Failure Retry Interval | The time interval for resubmitting the task if the task fails. It supports drop-down and manual filling. | +| Timeout alarm | Check Timeout Alarm and Timeout Failure. When the task exceeds the "timeout duration", an alarm email will be sent and the task execution will fail. | +| Downstream tasks selection | Depending on the status of the predecessor task, you can jump to the corresponding branch, currently two branches are supported: success, failure <ul><li style="list-style-type: none;"><ul><li>Success: When the upstream task runs successfully, run the success branch.</li><li>Failure: When the upstream task runs failed, run the failure branch.</li></ul></li></ul> | +| Upstream condition selection | Can select one or more upstream tasks for conditions.<ul><li style="list-style-type: none;"><ul><li>Add an upstream dependency: the first parameter is to choose a specified task name, and the second parameter is to choose the upstream task status to trigger conditions.</li><li>Select upstream task relationship: use<span> </span><code>and</code><span> </span>and<span> </span><code>or</code><span> </span>operators to handle the complex relationship of upstream when there are multiple upstream tasks for conditions.</li></ul></li></ul> | Review Comment: I find some of html syntax (`li`, `ul`, `span`) in this item, could you please remove them to markdown syntax? ########## docs/docs/en/guide/task/mlflow.md: ########## @@ -31,94 +31,67 @@ The MLflow plugin currently supports and will support the following: ## Create Task -- Click `Project -> Management-Project -> Name-Workflow Definition`, and click the "Create Workflow" button to enter the - DAG editing page. +- Click `Project Management -> Project Name -> Workflow Definition`, and click the `Create Workflow` button to enter the DAG editing page. - Drag from the toolbar <img src="../../../../img/tasks/icons/mlflow.png" width="15"/> task node to canvas. -## Task Example - -First, introduce some general parameters of DolphinScheduler: - -- **Node name**: The node name in a workflow definition is unique. -- **Run flag**: Identifies whether this node schedules normally, if it does not need to execute, select - the `prohibition execution`. -- **Descriptive information**: Describe the function of the node. -- **Task priority**: When the number of worker threads is insufficient, execute in the order of priority from high - to low, and tasks with the same priority will execute in a first-in first-out order. -- **Worker grouping**: Assign tasks to the machines of the worker group to execute. If `Default` is selected, - randomly select a worker machine for execution. -- **Environment Name**: Configure the environment name in which run the script. -- **Times of failed retry attempts**: The number of times the task failed to resubmit. -- **Failed retry interval**: The time interval (unit minute) for resubmitting the task after a failed task. -- **Delayed execution time**: The time (unit minute) that a task delays in execution. -- **Timeout alarm**: Check the timeout alarm and timeout failure. When the task runs exceed the "timeout", an alarm - email will send and the task execution will fail. -- **Predecessor task**: Selecting a predecessor task for the current task, will set the selected predecessor task as - upstream of the current task. - -Here are some specific parameters for the MLFlow component: - -- **MLflow Tracking Server URI**: MLflow Tracking Server URI, default http://localhost:5000. -- **Experiment Name**: Create the experiment where the task is running, if the experiment does not exist. If the name is empty, it is set to ` Default `, the same as MLflow. +## Task Parameters and Example + +| **Parameter** | **Description** | +| ------- | ---------- | +| Node Name | Set the name of the task. Node names within a workflow definition are unique. | +| Run flag | Indicates whether the node can be scheduled normally. If it is not necessary to execute, you can turn on the prohibiting execution switch. | +| Description | Describes the function of this node. | +| Task priority | When the number of worker threads is insufficient, they are executed in order from high to low according to the priority, and they are executed according to the first-in, first-out principle when the priority is the same. | +| Worker group | The task is assigned to the machines in the worker group for execution. If Default is selected, a worker machine will be randomly selected for execution. | +| Task group name | The group in Resources, if not configured, it will not be used. | +| Environment Name | Configure the environment in which to run the script. | +| Number of failed retries | The number of times the task is resubmitted after failure. It supports drop-down and manual filling. | +| Failure Retry Interval | The time interval for resubmitting the task if the task fails. It supports drop-down and manual filling. | +| Timeout alarm | Check Timeout Alarm and Timeout Failure. When the task exceeds the "timeout duration", an alarm email will be sent and the task execution will fail. | +| Predecessor task | Selecting the predecessor task of the current task will set the selected predecessor task as the upstream of the current task. | +| MLflow Tracking Server URI | MLflow Tracking Server URI, default http://localhost:5000. | +| Experiment Name | Create the experiment where the task is running, if the experiment does not exist. If the name is empty, it is set to ` Default `, the same as MLflow. | ### MLflow Projects #### BasicAlgorithm  -**Task Parameter** - -- **Register Model**: Register the model or not. If register is selected, the following parameters are expanded. - - **Model Name**: The registered model name is added to the original model version and registered as - Production. -- **Data Path**: The absolute path of the file or folder. Ends with .csv for file or contain train.csv and - test.csv for folder（In the suggested way, users should build their own test sets for model evaluation）. -- **Parameters**: Parameter when initializing the algorithm/AutoML model, which can be empty. For example - parameters `"time_budget=30;estimator_list=['lgbm']"` for flaml 。The convention will be passed with '; ' shards - each parameter, using the name before the equal sign as the parameter name, and using the name after the equal - sign to get the corresponding parameter value through `python eval()`. - - [Logistic Regression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression) - - [SVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html?highlight=svc#sklearn.svm.SVC) - - [lightgbm](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.LGBMClassifier.html#lightgbm.LGBMClassifier) - - [xgboost](https://xgboost.readthedocs.io/en/stable/python/python_api.html#xgboost.XGBClassifier) -- **Algorithm**：The selected algorithm currently supports `LR`, `SVM`, `LightGBM` and `XGboost` based - on [scikit-learn](https://scikit-learn.org/) form. -- **Parameter Search Space**: Parameter search space when running the corresponding algorithm, which can be - empty. For example, the parameter `max_depth=[5, 10];n_estimators=[100, 200]` for lightgbm 。The convention - will be passed with '; 'shards each parameter, using the name before the equal sign as the parameter name, - and using the name after the equal sign to get the corresponding parameter value through `python eval()`. +**Task Parameters** +| **Parameter** | **Description** | +| ------- | ---------- | +| Register Model | Register the model or not. If register is selected, the following parameters are expanded. | +| Model Name | The registered model name is added to the original model version and registered as Production. | +| Data Path | The absolute path of the file or folder. Ends with .csv for file or contain train.csv and test.csv for folder（In the suggested way, users should build their own test sets for model evaluation. | +| Parameters | Parameter when initializing the algorithm/AutoML model, which can be empty. For example, parameters `"time_budget=30;estimator_list=['lgbm']"` for flaml 。The convention will be passed with '; ' shards each parameter, using the name before the equal sign as the parameter name, and using the name after the equal sign to get the corresponding parameter value through `python eval()`. <ul><li>[Logistic Regression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression)</li><li>[SVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html?highlight=svc#sklearn.svm.SVC)</li><li>[lightgbm](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.LGBMClassifier.html#lightgbm.LGBMClassifier)</li><li>[xgboost](https://xgboost.readthedocs.io/en/stable/python/python_api.html#xgboost.XGBClassifier)</li></ul> | +| Algorithm |The selected algorithm currently supports `LR`, `SVM`, `LightGBM` and `XGboost` based + on [scikit-learn](https://scikit-learn.org/) form. | +| Parameter Search Space | Parameter search space when running the corresponding algorithm, which can be empty. For example, the parameter `max_depth=[5, 10];n_estimators=[100, 200]` for lightgbm 。The convention will be passed with '; 'shards each parameter, using the name before the equal sign as the parameter name, and using the name after the equal sign to get the corresponding parameter value through `python eval()`. | #### AutoML  **Task Parameter** - -- **Register Model**: Register the model or not. If register is selected, the following parameters are expanded. - - **model name**: The registered model name is added to the original model version and registered as - Production. -- **Data Path**: The absolute path of the file or folder. Ends with .csv for file or contain train.csv and - test.csv for folder(In the suggested way, users should build their own test sets for model evaluation). -- **Parameters**: Parameter when initializing the algorithm/AutoML model, which can be empty. For example - parameters `n_estimators=200;learning_rate=0.2` for flaml. The convention will be passed with '; 'shards - each parameter, using the name before the equal sign as the parameter name, and using the name after the equal - sign to get the corresponding parameter value through `python eval()`. The detailed parameter list is as follows: - - [flaml](https://microsoft.github.io/FLAML/docs/reference/automl#automl-objects) - - [autosklearn](https://automl.github.io/auto-sklearn/master/api.html) -- **AutoML tool**: The AutoML tool used, currently - supports [autosklearn](https://github.com/automl/auto-sklearn) - and [flaml](https://github.com/microsoft/FLAML). +| **Parameter** | **Description** | +| ------- | ---------- | +| Register Model | Register the model or not. If register is selected, the following parameters are expanded. | +| model name | The registered model name is added to the original model version and registered as Production. | +| Data Path | The absolute path of the file or folder. Ends with .csv for file or contain train.csv and test.csv for folder(In the suggested way, users should build their own test sets for model evaluation). | +| Parameters | Parameter when initializing the algorithm/AutoML model, which can be empty. For example, parameters `n_estimators=200;learning_rate=0.2` for flaml. The convention will be passed with '; 'shards each parameter, using the name before the equal sign as the parameter name, and using the name after the equal sign to get the corresponding parameter value through `python eval()`. The detailed parameter list is as follows: <ul><li>[flaml](https://microsoft.github.io/FLAML/docs/reference/automl#automl-objects)</li><li>[autosklearn](https://automl.github.io/auto-sklearn/master/api.html)</li></ul> | Review Comment: exists HTML syntax `li` and `ul` here ########## docs/docs/en/guide/task/mlflow.md: ########## @@ -31,94 +31,67 @@ The MLflow plugin currently supports and will support the following: ## Create Task -- Click `Project -> Management-Project -> Name-Workflow Definition`, and click the "Create Workflow" button to enter the - DAG editing page. +- Click `Project Management -> Project Name -> Workflow Definition`, and click the `Create Workflow` button to enter the DAG editing page. - Drag from the toolbar <img src="../../../../img/tasks/icons/mlflow.png" width="15"/> task node to canvas. -## Task Example - -First, introduce some general parameters of DolphinScheduler: - -- **Node name**: The node name in a workflow definition is unique. -- **Run flag**: Identifies whether this node schedules normally, if it does not need to execute, select - the `prohibition execution`. -- **Descriptive information**: Describe the function of the node. -- **Task priority**: When the number of worker threads is insufficient, execute in the order of priority from high - to low, and tasks with the same priority will execute in a first-in first-out order. -- **Worker grouping**: Assign tasks to the machines of the worker group to execute. If `Default` is selected, - randomly select a worker machine for execution. -- **Environment Name**: Configure the environment name in which run the script. -- **Times of failed retry attempts**: The number of times the task failed to resubmit. -- **Failed retry interval**: The time interval (unit minute) for resubmitting the task after a failed task. -- **Delayed execution time**: The time (unit minute) that a task delays in execution. -- **Timeout alarm**: Check the timeout alarm and timeout failure. When the task runs exceed the "timeout", an alarm - email will send and the task execution will fail. -- **Predecessor task**: Selecting a predecessor task for the current task, will set the selected predecessor task as - upstream of the current task. - -Here are some specific parameters for the MLFlow component: - -- **MLflow Tracking Server URI**: MLflow Tracking Server URI, default http://localhost:5000. -- **Experiment Name**: Create the experiment where the task is running, if the experiment does not exist. If the name is empty, it is set to ` Default `, the same as MLflow. +## Task Parameters and Example + +| **Parameter** | **Description** | +| ------- | ---------- | +| Node Name | Set the name of the task. Node names within a workflow definition are unique. | +| Run flag | Indicates whether the node can be scheduled normally. If it is not necessary to execute, you can turn on the prohibiting execution switch. | +| Description | Describes the function of this node. | +| Task priority | When the number of worker threads is insufficient, they are executed in order from high to low according to the priority, and they are executed according to the first-in, first-out principle when the priority is the same. | +| Worker group | The task is assigned to the machines in the worker group for execution. If Default is selected, a worker machine will be randomly selected for execution. | +| Task group name | The group in Resources, if not configured, it will not be used. | +| Environment Name | Configure the environment in which to run the script. | +| Number of failed retries | The number of times the task is resubmitted after failure. It supports drop-down and manual filling. | +| Failure Retry Interval | The time interval for resubmitting the task if the task fails. It supports drop-down and manual filling. | +| Timeout alarm | Check Timeout Alarm and Timeout Failure. When the task exceeds the "timeout duration", an alarm email will be sent and the task execution will fail. | +| Predecessor task | Selecting the predecessor task of the current task will set the selected predecessor task as the upstream of the current task. | +| MLflow Tracking Server URI | MLflow Tracking Server URI, default http://localhost:5000. | +| Experiment Name | Create the experiment where the task is running, if the experiment does not exist. If the name is empty, it is set to ` Default `, the same as MLflow. | ### MLflow Projects #### BasicAlgorithm  -**Task Parameter** - -- **Register Model**: Register the model or not. If register is selected, the following parameters are expanded. - - **Model Name**: The registered model name is added to the original model version and registered as - Production. -- **Data Path**: The absolute path of the file or folder. Ends with .csv for file or contain train.csv and - test.csv for folder（In the suggested way, users should build their own test sets for model evaluation）. -- **Parameters**: Parameter when initializing the algorithm/AutoML model, which can be empty. For example - parameters `"time_budget=30;estimator_list=['lgbm']"` for flaml 。The convention will be passed with '; ' shards - each parameter, using the name before the equal sign as the parameter name, and using the name after the equal - sign to get the corresponding parameter value through `python eval()`. - - [Logistic Regression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression) - - [SVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html?highlight=svc#sklearn.svm.SVC) - - [lightgbm](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.LGBMClassifier.html#lightgbm.LGBMClassifier) - - [xgboost](https://xgboost.readthedocs.io/en/stable/python/python_api.html#xgboost.XGBClassifier) -- **Algorithm**：The selected algorithm currently supports `LR`, `SVM`, `LightGBM` and `XGboost` based - on [scikit-learn](https://scikit-learn.org/) form. -- **Parameter Search Space**: Parameter search space when running the corresponding algorithm, which can be - empty. For example, the parameter `max_depth=[5, 10];n_estimators=[100, 200]` for lightgbm 。The convention - will be passed with '; 'shards each parameter, using the name before the equal sign as the parameter name, - and using the name after the equal sign to get the corresponding parameter value through `python eval()`. +**Task Parameters** +| **Parameter** | **Description** | +| ------- | ---------- | +| Register Model | Register the model or not. If register is selected, the following parameters are expanded. | +| Model Name | The registered model name is added to the original model version and registered as Production. | +| Data Path | The absolute path of the file or folder. Ends with .csv for file or contain train.csv and test.csv for folder（In the suggested way, users should build their own test sets for model evaluation. | +| Parameters | Parameter when initializing the algorithm/AutoML model, which can be empty. For example, parameters `"time_budget=30;estimator_list=['lgbm']"` for flaml 。The convention will be passed with '; ' shards each parameter, using the name before the equal sign as the parameter name, and using the name after the equal sign to get the corresponding parameter value through `python eval()`. <ul><li>[Logistic Regression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression)</li><li>[SVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html?highlight=svc#sklearn.svm.SVC)</li><li>[lightgbm](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.LGBMClassifier.html#lightgbm.LGBMClassifier)</li><li>[xgboost](https://xgboost.readthedocs.io/en/stable/python/python_api.html#xgboost.XGBClassifier)</li></ul> | Review Comment: exists HTML syntax `<li>` and `ul` here ########## docs/docs/en/guide/task/map-reduce.md: ########## @@ -6,45 +6,41 @@ MapReduce(MR) task type used for executing MapReduce programs. For MapReduce nod ## Create Task -- Click `Project -> Management-Project -> Name-Workflow Definition`, and click the `Create Workflow` button to enter the DAG editing page. +- Click `Project Management -> Project Name -> Workflow Definition`, and click the `Create Workflow` button to enter the DAG editing page. - Drag from the toolbar <img src="../../../../img/tasks/icons/mr.png" width="15"/> to the canvas. -## Task Parameter - -- **Node name**: The node name in a workflow definition is unique. -- **Run flag**: Identifies whether this node schedules normally, if it does not need to execute, select the `prohibition execution`. -- **Descriptive information**: Describe the function of the node. -- **Task priority**: When the number of worker threads is insufficient, execute in the order of priority from high to low, and tasks with the same priority will execute in a first-in first-out order. -- **Worker grouping**: Assign tasks to the machines of the worker group to execute. If `Default` is selected, randomly select a worker machine for execution. -- **Environment Name**: Configure the environment name in which run the script. -- **Times of failed retry attempts**: The number of times the task failed to resubmit. -- **Failed retry interval**: The time interval (unit minute) for resubmitting the task after a failed task. -- **Delayed execution time**: The time (unit minute) that a task delays in execution. -- **Timeout alarm**: Check the timeout alarm and timeout failure. When the task runs exceed the "timeout", an alarm email will send and the task execution will fail. -- **Resource**: Refers to the list of resource files that called in the script, and upload or create files by the Resource Center file management. -- **Custom parameters**: It is a local user-defined parameter for MapReduce, and will replace the content with `${variable}` in the script. -- **Predecessor task**: Selecting a predecessor task for the current task, will set the selected predecessor task as upstream of the current task. - -### JAVA or SCALA Program - -- **Program type**: Select JAVA or SCALA program. -- **The class of the main function**: The **full path** of Main Class, the entry point of the MapReduce program. -- **Main jar package**: The jar package of the MapReduce program. -- **Task name** (optional): MapReduce task name. -- **Command line parameters**: Set the input parameters of the MapReduce program and support the substitution of custom parameter variables. -- **Other parameters**: support `-D`, `-files`, `-libjars`, `-archives` format. -- **Resource**: Appoint resource files in the `Resource` if parameters refer to them. -- **User-defined parameter**: It is a local user-defined parameter for MapReduce, and will replace the content with `${variable}` in the script. - -## Python Program - -- **Program type**: Select Python language. -- **Main jar package**: The Python jar package for running MapReduce. -- **Other parameters**: support `-D`, `-mapper`, `-reducer,` `-input` `-output` format, and you can set the input of user-defined parameters, such as: -- `-mapper "mapper.py 1"` `-file mapper.py` `-reducer reducer.py` `-file reducer.py` `–input /journey/words.txt` `-output /journey/out/mr/\${currentTimeMillis}` -- The `mapper.py 1` after `-mapper` is two parameters, the first parameter is `mapper.py`, and the second parameter is `1`. -- **Resource**: Appoint resource files in the `Resource` if parameters refer to them. -- **User-defined parameter**: It is a local user-defined parameter for MapReduce, and will replace the content with `${variable}` in the script. +## Task Parameters + +| **Parameter** | **Description** | +| ------- | ---------- | +| Node Name | Set the name of the task. Node names within a workflow definition are unique. | +| Run flag | Indicates whether the node can be scheduled normally. If it is not necessary to execute, you can turn on the prohibiting execution switch. | +| Description | Describes the function of this node. | +| Task priority | When the number of worker threads is insufficient, they are executed in order from high to low according to the priority, and they are executed according to the first-in, first-out principle when the priority is the same. | +| Worker group | The task is assigned to the machines in the worker group for execution. If Default is selected, a worker machine will be randomly selected for execution. | +| Task group name | The group in Resources, if not configured, it will not be used. | +| Environment Name | Configure the environment in which to run the script. | +| Number of failed retries | The number of times the task is resubmitted after failure. It supports drop-down and manual filling. | +| Failure Retry Interval | The time interval for resubmitting the task if the task fails. It supports drop-down and manual filling. | +| Timeout alarm | Check Timeout Alarm and Timeout Failure. When the task exceeds the "timeout duration", an alarm email will be sent and the task execution will fail. | +| Resource | Refers to the list of resource files that called in the script, and upload or create files by the Resource Center file management. | +| Custom parameters | It is a local user-defined parameter for MapReduce, and will replace the content with `${variable}` in the script. | +| Predecessor task | Selecting a predecessor task for the current task, will set the selected predecessor task as upstream of the current task. | +| **JAVA or SCALA Program** | +| Program type | Select JAVA or SCALA program. | +| The class of the main function | The **full path** of Main Class, the entry point of the MapReduce program. | +| Main jar package | The jar package of the MapReduce program. | +| Task name | MapReduce task name. | +| Command line parameters | Set the input parameters of the MapReduce program and support the substitution of custom parameter variables. | +| Other parameters | Support `-D`, `-files`, `-libjars`, `-archives` format. | +| Resource | Appoint resource files in the `Resource` if parameters refer to them. | +| User-defined parameter | It is a local user-defined parameter for MapReduce, and will replace the content with `${variable}` in the script. | +| **Python Program** | +| Program type | Select Python language. | +| Main jar package | The Python jar package for running MapReduce. | +| Other parameters | Support `-D`, `-mapper`, `-reducer,` `-input` `-output` format, and you can set the input of user-defined parameters, such as:<ul><li>`-mapper "mapper.py 1"` `-file mapper.py` `-reducer reducer.py` `-file reducer.py` `–input /journey/words.txt` `-output /journey/out/mr/\${currentTimeMillis}`</li><li>The `mapper.py 1` after `-mapper` is two parameters, the first parameter is `mapper.py`, and the second parameter is `1`. </li></ul> | Review Comment: There are also the HTML syntax in this item too ########## docs/docs/en/guide/task/sql.md: ########## @@ -10,24 +10,21 @@ Refer to [DataSource](../datasource/introduction.md) ## Create Task -- Click `Project -> Management-Project -> Name-Workflow Definition`, and click the "Create Workflow" button to enter the DAG editing page. +- Click `Project Management -> Project Name -> Workflow Definition`, and click the `Create Workflow` button to enter the DAG editing page. - Drag from the toolbar <img src="../../../../img/tasks/icons/sql.png" width="25"/> to the canvas. ## Task Parameter -- Data source: Select the corresponding DataSource. -- SQL type: Supports query and non-query. - - Query: supports `DML select` type commands, which return a result set. You can specify three templates for email notification as form, attachment or form attachment; - - Non-query: support `DDL` all commands and `DML update, delete, insert` three types of commands; - - Segmented execution symbol: When the data source does not support executing multiple SQL statements at a time, the symbol for splitting SQL statements is provided to call the data source execution method multiple times. - Example: 1. When the Hive data source is selected as the data source, this parameter does not need to be filled in. Because the Hive data source itself supports executing multiple SQL statements at one time; - 2. When the MySQL data source is selected as the data source, and multi-segment SQL statements are to be executed, this parameter needs to be filled in with a semicolon `;`. Because the MySQL data source does not support executing multiple SQL statements at one time; -- SQL parameter: The input parameter format is `key1=value1;key2=value2...`. -- SQL statement: SQL statement. -- UDF function: For Hive DataSources, you can refer to UDF functions created in the resource center, but other DataSource do not support UDF functions. -- Custom parameters: SQL task type, and stored procedure is a custom parameter order, to set customized parameter type and data type for the method is the same as the stored procedure task type. The difference is that the custom parameter of the SQL task type replaces the `${variable}` in the SQL statement. -- Pre-SQL: Pre-SQL executes before the SQL statement. -- Post-SQL: Post-SQL executes after the SQL statement. +| **Parameter** | **Description** | +| ------- | ---------- | +| Data source | Select the corresponding DataSource. | +| SQL type | Supports query and non-query. <ul><li>Query: supports<span> </span><code>DML select</code><span> </span>type commands, which return a result set. You can specify three templates for email notification as form, attachment or form attachment;</li><li>Non-query: support<span> </span><code>DDL</code><span> </span>all commands and<span> </span><code>DML update, delete, insert</code><span> </span>three types of commands;<ul><li>Segmented execution symbol: When the data source does not support executing multiple SQL statements at a time, the symbol for splitting SQL statements is provided to call the data source execution method multiple times. Example: 1. When the Hive data source is selected as the data source, this parameter does not need to be filled in. Because the Hive data source itself supports executing multiple SQL statements at one time; 2. When the MySQL data source is selected as the data source, and multi-segment SQL statements are t o be executed, this parameter needs to be filled in with a semicolon<span> </span><code>;</code>. Because the MySQL data source does not support executing multiple SQL statements at one time.</li></ul></li></ul> | Review Comment: exists HTML syntax `li` and `ul` here -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
