[dolphinscheduler-website] branch master updated: [Imporvement][Doc] /development/architecture-design.html (#415)

Mon, 02 Aug 2021 19:08:16 -0700 

This is an automated email from the ASF dual-hosted git repository.

kirs pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler-website.git


The following commit(s) were added to refs/heads/master by this push:
     new 975ae80  [Imporvement][Doc] /development/architecture-design.html 
(#415)
975ae80 is described below

commit 975ae801273c86cd8eec52480cd3f600fcdda1ff
Author: Tandoy <[email protected]>
AuthorDate: Tue Aug 3 10:08:04 2021 +0800

    [Imporvement][Doc] /development/architecture-design.html (#415)
    
    * [Imporvement][Doc] /development/architecture-design.html
    ·Modify the zk node error in the framework design module 
distributed_lock.png of the developer guide
    ·There are some Chinese words in the picture of the modified English 
document
    ·Adjust the size of related pictures to make them look clearer
    
    * [Imporvement][Doc] /development/architecture-design.html
    ·Modify the zk node error in the framework design module 
distributed_lock.png of the developer guide
    ·There are some Chinese words in the picture of the modified English 
document
    ·Adjust the size of related pictures to make them look clearer
    
    * [Imporvement][Doc] /development/architecture-design.html
    ·Modify the zk node error in the framework design module 
distributed_lock.png of the developer guide
    ·There are some Chinese words in the picture of the modified English 
document
    ·Adjust the size of related pictures to make them look clearer
---
 development/en-us/architecture-design.md           |  24 ++++++++++-----------
 development/zh-cn/architecture-design.md           |  18 ++++++++--------
 img/architecture-design/dag_examples_ch_zn.png     | Bin 0 -> 89203 bytes
 img/architecture-design/dag_examples_en_us.png     | Bin 0 -> 89184 bytes
 img/architecture-design/distributed_lock_ch_zn.png | Bin 0 -> 179233 bytes
 img/architecture-design/distributed_lock_en_us.png | Bin 0 -> 188964 bytes
 .../distributed_lock_procss_ch_zn.png              | Bin 0 -> 345394 bytes
 .../distributed_lock_procss_en_us.png              | Bin 0 -> 363294 bytes
 img/architecture-design/fault-tolerant_ch_zn.png   | Bin 0 -> 177202 bytes
 img/architecture-design/fault-tolerant_en_us.png   | Bin 0 -> 193511 bytes
 .../fault-tolerant_master_ch_zn.png                | Bin 0 -> 156134 bytes
 .../fault-tolerant_master_en_us.png                | Bin 0 -> 169131 bytes
 .../fault-tolerant_worker_ch_zn.png                | Bin 0 -> 156743 bytes
 .../fault-tolerant_worker_en_us.png                | Bin 0 -> 164622 bytes
 img/architecture-design/grpc_en_us.png             | Bin 0 -> 68933 bytes
 img/architecture-design/lack_thread_ch_zn.png      | Bin 0 -> 173229 bytes
 img/architecture-design/lack_thread_en_us.png      | Bin 0 -> 195509 bytes
 img/architecture-design/process_priority_en_us.png | Bin 0 -> 49112 bytes
 img/architecture-design/task_priority_en_us.png    | Bin 0 -> 30544 bytes
 19 files changed, 21 insertions(+), 21 deletions(-)

diff --git a/development/en-us/architecture-design.md 
b/development/en-us/architecture-design.md
index 94abd81..ed8d27d 100644
--- a/development/en-us/architecture-design.md
+++ b/development/en-us/architecture-design.md
@@ -6,7 +6,7 @@ Before explaining the architecture of the schedule system, let 
us first understa
 **DAG：** Full name Directed Acyclic Graph，referred to as DAG。Tasks in the 
workflow are assembled in the form of directed acyclic graphs, which are 
topologically traversed from nodes with zero indegrees of ingress until there 
are no successor nodes. For example, the following picture:
 
 <p align="center">
-  <img src="/img/dag_examples_cn.jpg" alt="dag示例"  width="60%" />
+  <img src="/img/architecture-design/dag_examples_en_us.png" alt="dag示例"  
width="80%" />
   <p align="center">
         <em>dag example</em>
   </p>
@@ -40,7 +40,7 @@ Before explaining the architecture of the schedule system, 
let us first understa
 
 #### 2.1 System Architecture Diagram
 <p align="center">
-  <img src="/img/architecture.jpg" alt="System Architecture Diagram"  />
+  <img src="/img/architecture-en.jpg" alt="System Architecture Diagram"  />
   <p align="center">
         <em>System Architecture Diagram</em>
   </p>
@@ -141,13 +141,13 @@ DolphinScheduler uses ZooKeeper distributed locks to 
implement only one Master t
 1. The core process algorithm for obtaining distributed locks is as follows
 
  <p align="center">
-   <img 
src="https://analysys.github.io/easyscheduler_docs_cn/images/distributed_lock.png";
 alt="Get Distributed Lock Process" width="50%" />
+   <img src="/img/architecture-design/distributed_lock_en_us.png" alt="Get 
Distributed Lock Process" width="70%" />
  </p>
 
 2. Scheduler thread distributed lock implementation flow chart in 
DolphinScheduler:
 
  <p align="center">
-   <img src="/img/distributed_lock_procss.png" alt="Get Distributed Lock 
Process" width="50%" />
+   <img src="/img/architecture-design/distributed_lock_procss_en_us.png" 
alt="Get Distributed Lock Process" />
  </p>
 
 ##### Third, the thread is insufficient loop waiting problem
@@ -156,7 +156,7 @@ DolphinScheduler uses ZooKeeper distributed locks to 
implement only one Master t
 - If a large number of sub-processes are nested in a large DAG, the following 
figure will result in a "dead" state:
 
  <p align="center">
-   <img 
src="https://analysys.github.io/easyscheduler_docs_cn/images/lack_thread.png"; 
alt="Thread is not enough to wait for loop" width="50%" />
+   <img src="/img/architecture-design/lack_thread_en_us.png" alt="Thread is 
not enough to wait for loop" width="70%" />
  </p>
 
 In the above figure, MainFlowThread waits for SubFlowThread1 to end, 
SubFlowThread1 waits for SubFlowThread2 to end, SubFlowThread2 waits for 
SubFlowThread3 to end, and SubFlowThread3 waits for a new thread in the thread 
pool, then the entire DAG process cannot end, and thus the thread cannot be 
released. This forms the state of the child parent process loop waiting. At 
this point, the scheduling cluster will no longer be available unless a new 
Master is started to add threads to break s [...]
@@ -180,7 +180,7 @@ Fault tolerance is divided into service fault tolerance and 
task retry. Service
 Service fault tolerance design relies on ZooKeeper's Watcher mechanism. The 
implementation principle is as follows:
 
  <p align="center">
-   <img 
src="https://analysys.github.io/easyscheduler_docs_cn/images/fault-tolerant.png";
 alt="DolphinScheduler Fault Tolerant Design" width="40%" />
+   <img src="/img/architecture-design/fault-tolerant_en_us.png" 
alt="DolphinScheduler Fault Tolerant Design" width="70%" />
  </p>
 
 The Master monitors the directories of other Masters and Workers. If the 
remove event is detected, the process instance is fault-tolerant or the task 
instance is fault-tolerant according to the specific business logic.
@@ -190,7 +190,7 @@ The Master monitors the directories of other Masters and 
Workers. If the remove
 - Master fault tolerance flow chart:
 
  <p align="center">
-   <img 
src="https://analysys.github.io/easyscheduler_docs_cn/images/fault-tolerant_master.png";
 alt="Master Fault Tolerance Flowchart" width="40%" />
+   <img src="/img/architecture-design/fault-tolerant_master_en_us.png" 
alt="Master Fault Tolerance Flowchart" width="70%" />
  </p>
 
 After the ZooKeeper Master is fault-tolerant, it is rescheduled by the 
Scheduler thread in DolphinScheduler. It traverses the DAG to find the 
"Running" and "Submit Successful" tasks, and monitors the status of its task 
instance for the "Running" task. You need to determine whether the Task Queue 
already exists. If it exists, monitor the status of the task instance. If it 
does not exist, resubmit the task instance.
@@ -200,7 +200,7 @@ After the ZooKeeper Master is fault-tolerant, it is 
rescheduled by the Scheduler
 - Worker fault tolerance flow chart:
 
  <p align="center">
-   <img 
src="https://analysys.github.io/easyscheduler_docs_cn/images/fault-tolerant_worker.png";
 alt="Worker Fault Tolerance Flowchart" width="40%" />
+   <img src="/img/architecture-design/fault-tolerant_worker_en_us.png" 
alt="Worker Fault Tolerance Flowchart" width="70%" />
  </p>
 
 Once the Master Scheduler thread finds the task instance as "need to be fault 
tolerant", it takes over the task and resubmits.
@@ -239,13 +239,13 @@ In the early scheduling design, if there is no priority 
design and fair scheduli
     - The priority of the process definition is that some processes need to be 
processed before other processes. This can be configured at the start of the 
process or at the time of scheduled start. There are 5 levels, followed by 
HIGHEST, HIGH, MEDIUM, LOW, and LOWEST. As shown below
 
       <p align="center">
-         <img 
src="https://analysys.github.io/easyscheduler_docs_cn/images/process_priority.png";
 alt="Process Priority Configuration" width="40%" />
+         <img src="/img/architecture-design/process_priority_en_us.png" 
alt="Process Priority Configuration" width="40%" />
        </p>
 
     - The priority of the task is also divided into 5 levels, followed by 
HIGHEST, HIGH, MEDIUM, LOW, and LOWEST. As shown below
 
-      <p align="center">
-         <img 
src="https://analysys.github.io/easyscheduler_docs_cn/images/task_priority.png"; 
alt="task priority configuration" width="35%" />
+      <p align="center">`
+         <img src="/img/architecture-design/task_priority_en_us.png" alt="task 
priority configuration" width="35%" />
        </p>
 
 ##### VI. Logback and gRPC implement log access
@@ -256,7 +256,7 @@ In the early scheduling design, if there is no priority 
design and fair scheduli
 - Considering the lightweightness of DolphinScheduler as much as possible, 
gRPC was chosen to implement remote access log information.
 
  <p align="center">
-   <img src="https://analysys.github.io/easyscheduler_docs_cn/images/grpc.png"; 
alt="grpc remote access" width="50%" />
+   <img src="/img/architecture-design/grpc_en_us.png" alt="grpc remote access" 
width="50%" />
  </p>
 
 - We use a custom Logback FileAppender and Filter function to generate a log 
file for each task instance.
diff --git a/development/zh-cn/architecture-design.md 
b/development/zh-cn/architecture-design.md
index 12f1633..67c3399 100644
--- a/development/zh-cn/architecture-design.md
+++ b/development/zh-cn/architecture-design.md
@@ -5,7 +5,7 @@
 **DAG：** 全称Directed Acyclic 
Graph，简称DAG。工作流中的Task任务以有向无环图的形式组装起来，从入度为零的节点进行拓扑遍历，直到无后继节点为止。举例如下图：
 
 <p align="center">
-  <img src="/img/dag_examples_cn.jpg" alt="dag示例"  width="60%" />
+  <img src="/img/architecture-design/dag_examples_ch_zn.png" alt="dag示例"  
width="80%" />
   <p align="center">
         <em>dag示例</em>
   </p>
@@ -37,7 +37,7 @@
 
 #### 2.1 系统架构图
 <p align="center">
-  <img src="/img/architecture.jpg" alt="系统架构图"  width="70%" />
+  <img src="/img/architecture.jpg" alt="系统架构图"  />
   <p align="center">
         <em>系统架构图</em>
   </p>
@@ -131,12 +131,12 @@
 
DolphinScheduler使用ZooKeeper分布式锁来实现同一时刻只有一台Master执行Scheduler，或者只有一台Worker执行任务的提交。
 1. 获取分布式锁的核心流程算法如下
  <p align="center">
-   <img 
src="https://analysys.github.io/easyscheduler_docs_cn/images/distributed_lock.png";
 alt="获取分布式锁流程"  width="50%" />
+   <img src="/img/architecture-design/distributed_lock_ch_zn.png" 
alt="获取分布式锁流程"  width="70%" />
  </p>
 
 2. DolphinScheduler中Scheduler线程分布式锁实现流程图：
  <p align="center">
-   <img src="/img/distributed_lock_procss.png" alt="获取分布式锁流程"  width="50%" />
+   <img src="/img/architecture-design/distributed_lock_procss_ch_zn.png" 
alt="获取分布式锁流程" />
  </p>
 
 
@@ -146,7 +146,7 @@ DolphinScheduler使用ZooKeeper分布式锁来实现同一时刻只有一台Mast
 -  如果一个大的DAG中嵌套了很多子流程，如下图则会产生“死等”状态：
 
  <p align="center">
-   <img 
src="https://analysys.github.io/easyscheduler_docs_cn/images/lack_thread.png"; 
alt="线程不足循环等待问题"  width="50%" />
+   <img src="/img/architecture-design/lack_thread_ch_zn.png" alt="线程不足循环等待问题"  
width="70%" />
  </p>
 上图中MainFlowThread等待SubFlowThread1结束，SubFlowThread1等待SubFlowThread2结束， 
SubFlowThread2等待SubFlowThread3结束，而SubFlowThread3等待线程池有新线程，则整个DAG流程不能结束，从而其中的线程也不能释放。这样就形成的子父流程循环等待的状态。此时除非启动新的Master来增加线程来打破这样的”僵局”，否则调度集群将不能再使用。
 
@@ -169,7 +169,7 @@ DolphinScheduler使用ZooKeeper分布式锁来实现同一时刻只有一台Mast
 服务容错设计依赖于ZooKeeper的Watcher机制，实现原理如图：
 
  <p align="center">
-   <img 
src="https://analysys.github.io/easyscheduler_docs_cn/images/fault-tolerant.png";
 alt="DolphinScheduler容错设计"  width="40%" />
+   <img src="/img/architecture-design/fault-tolerant_ch_zn.png" 
alt="DolphinScheduler容错设计"  width="70%" />
  </p>
 其中Master监控其他Master和Worker的目录，如果监听到remove事件，则会根据具体的业务逻辑进行流程实例容错或者任务实例容错。
 
@@ -178,7 +178,7 @@ DolphinScheduler使用ZooKeeper分布式锁来实现同一时刻只有一台Mast
 - Master容错流程图：
 
  <p align="center">
-   <img 
src="https://analysys.github.io/easyscheduler_docs_cn/images/fault-tolerant_master.png";
 alt="Master容错流程图"  width="40%" />
+   <img src="/img/architecture-design/fault-tolerant_master_ch_zn.png" 
alt="Master容错流程图"  width="70%" />
  </p>
 ZooKeeper Master容错完成之后则重新由DolphinScheduler中Scheduler线程调度，遍历 DAG 
找到”正在运行”和“提交成功”的任务，对”正在运行”的任务监控其任务实例的状态，对”提交成功”的任务需要判断Task 
Queue中是否已经存在，如果存在则同样监控任务实例的状态，如果不存在则重新提交任务实例。
 
@@ -187,7 +187,7 @@ ZooKeeper Master容错完成之后则重新由DolphinScheduler中Scheduler线程
 - Worker容错流程图：
 
  <p align="center">
-   <img 
src="https://analysys.github.io/easyscheduler_docs_cn/images/fault-tolerant_worker.png";
 alt="Worker容错流程图"  width="40%" />
+   <img src="/img/architecture-design/fault-tolerant_worker_ch_zn.png" 
alt="Worker容错流程图"  width="70%" />
  </p>
 
 Master Scheduler线程一旦发现任务实例为” 需要容错”状态，则接管任务并进行重新提交。
@@ -242,7 +242,7 @@ Master Scheduler线程一旦发现任务实例为” 需要容错”状态，则
 -  介于考虑到尽可能的DolphinScheduler的轻量级性，所以选择了gRPC实现远程访问日志信息。
 
  <p align="center">
-   <img src="https://analysys.github.io/easyscheduler_docs_cn/images/grpc.png"; 
alt="grpc远程访问"  width="50%" />
+   <img src="https://analysys.github.io/easyscheduler_docs_cn/images/grpc.png"; 
alt="grpc远程访问"  width="60%" />
  </p>
 
 
diff --git a/img/architecture-design/dag_examples_ch_zn.png 
b/img/architecture-design/dag_examples_ch_zn.png
new file mode 100644
index 0000000..95134b3
Binary files /dev/null and b/img/architecture-design/dag_examples_ch_zn.png 
differ
diff --git a/img/architecture-design/dag_examples_en_us.png 
b/img/architecture-design/dag_examples_en_us.png
new file mode 100644
index 0000000..15848da
Binary files /dev/null and b/img/architecture-design/dag_examples_en_us.png 
differ
diff --git a/img/architecture-design/distributed_lock_ch_zn.png 
b/img/architecture-design/distributed_lock_ch_zn.png
new file mode 100644
index 0000000..f2e2337
Binary files /dev/null and b/img/architecture-design/distributed_lock_ch_zn.png 
differ
diff --git a/img/architecture-design/distributed_lock_en_us.png 
b/img/architecture-design/distributed_lock_en_us.png
new file mode 100644
index 0000000..5c34fc4
Binary files /dev/null and b/img/architecture-design/distributed_lock_en_us.png 
differ
diff --git a/img/architecture-design/distributed_lock_procss_ch_zn.png 
b/img/architecture-design/distributed_lock_procss_ch_zn.png
new file mode 100644
index 0000000..78b87df
Binary files /dev/null and 
b/img/architecture-design/distributed_lock_procss_ch_zn.png differ
diff --git a/img/architecture-design/distributed_lock_procss_en_us.png 
b/img/architecture-design/distributed_lock_procss_en_us.png
new file mode 100644
index 0000000..469128b
Binary files /dev/null and 
b/img/architecture-design/distributed_lock_procss_en_us.png differ
diff --git a/img/architecture-design/fault-tolerant_ch_zn.png 
b/img/architecture-design/fault-tolerant_ch_zn.png
new file mode 100644
index 0000000..5da473c
Binary files /dev/null and b/img/architecture-design/fault-tolerant_ch_zn.png 
differ
diff --git a/img/architecture-design/fault-tolerant_en_us.png 
b/img/architecture-design/fault-tolerant_en_us.png
new file mode 100644
index 0000000..45dadf7
Binary files /dev/null and b/img/architecture-design/fault-tolerant_en_us.png 
differ
diff --git a/img/architecture-design/fault-tolerant_master_ch_zn.png 
b/img/architecture-design/fault-tolerant_master_ch_zn.png
new file mode 100644
index 0000000..56c3500
Binary files /dev/null and 
b/img/architecture-design/fault-tolerant_master_ch_zn.png differ
diff --git a/img/architecture-design/fault-tolerant_master_en_us.png 
b/img/architecture-design/fault-tolerant_master_en_us.png
new file mode 100644
index 0000000..a9901ce
Binary files /dev/null and 
b/img/architecture-design/fault-tolerant_master_en_us.png differ
diff --git a/img/architecture-design/fault-tolerant_worker_ch_zn.png 
b/img/architecture-design/fault-tolerant_worker_ch_zn.png
new file mode 100644
index 0000000..c8b5cd5
Binary files /dev/null and 
b/img/architecture-design/fault-tolerant_worker_ch_zn.png differ
diff --git a/img/architecture-design/fault-tolerant_worker_en_us.png 
b/img/architecture-design/fault-tolerant_worker_en_us.png
new file mode 100644
index 0000000..e7f379d
Binary files /dev/null and 
b/img/architecture-design/fault-tolerant_worker_en_us.png differ
diff --git a/img/architecture-design/grpc_en_us.png 
b/img/architecture-design/grpc_en_us.png
new file mode 100644
index 0000000..633b837
Binary files /dev/null and b/img/architecture-design/grpc_en_us.png differ
diff --git a/img/architecture-design/lack_thread_ch_zn.png 
b/img/architecture-design/lack_thread_ch_zn.png
new file mode 100644
index 0000000..31263cd
Binary files /dev/null and b/img/architecture-design/lack_thread_ch_zn.png 
differ
diff --git a/img/architecture-design/lack_thread_en_us.png 
b/img/architecture-design/lack_thread_en_us.png
new file mode 100644
index 0000000..0dc5a7b
Binary files /dev/null and b/img/architecture-design/lack_thread_en_us.png 
differ
diff --git a/img/architecture-design/process_priority_en_us.png 
b/img/architecture-design/process_priority_en_us.png
new file mode 100644
index 0000000..c6cd600
Binary files /dev/null and b/img/architecture-design/process_priority_en_us.png 
differ
diff --git a/img/architecture-design/task_priority_en_us.png 
b/img/architecture-design/task_priority_en_us.png
new file mode 100644
index 0000000..3470260
Binary files /dev/null and b/img/architecture-design/task_priority_en_us.png 
differ

Reply via email to