This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 66959ac9f Automated deployment:
a7d0de2b86d26e21479d52cb2b94d34fc8d5c7be
66959ac9f is described below
commit 66959ac9f2fd708eef44c412013d400273cbe602
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Tue Jun 21 05:13:52 2022 +0000
Automated deployment: a7d0de2b86d26e21479d52cb2b94d34fc8d5c7be
---
.../dev/user_doc/architecture/configuration.html | 25 +++++++++++
.../dev/user_doc/architecture/configuration.json | 2 +-
en-us/docs/dev/user_doc/guide/task/jupyter.html | 36 +++++++++++++++-
en-us/docs/dev/user_doc/guide/task/jupyter.json | 2 +-
.../dev/user_doc/architecture/configuration.html | 25 +++++++++++
.../dev/user_doc/architecture/configuration.json | 2 +-
zh-cn/docs/dev/user_doc/guide/task/jupyter.html | 48 ++++++++++++++++++----
zh-cn/docs/dev/user_doc/guide/task/jupyter.json | 2 +-
8 files changed, 130 insertions(+), 12 deletions(-)
diff --git a/en-us/docs/dev/user_doc/architecture/configuration.html
b/en-us/docs/dev/user_doc/architecture/configuration.html
index 9c951c35c..3200546b4 100644
--- a/en-us/docs/dev/user_doc/architecture/configuration.html
+++ b/en-us/docs/dev/user_doc/architecture/configuration.html
@@ -566,6 +566,31 @@ Currently, DS just makes a basic config, remember to
config further JVM options
<td>CREATE</td>
<td>action when LDAP user is not exist. Default CREATE: automatically create
user when user not exist, DENY: deny log-in when user not exist</td>
</tr>
+<tr>
+<td>traffic.control.global.switch</td>
+<td>false</td>
+<td>traffic control global switch</td>
+</tr>
+<tr>
+<td>traffic.control.max-global-qps-rate</td>
+<td>300</td>
+<td>global max request number per second</td>
+</tr>
+<tr>
+<td>traffic.control.tenant-switch</td>
+<td>false</td>
+<td>traffic control tenant switch</td>
+</tr>
+<tr>
+<td>traffic.control.default-tenant-qps-rate</td>
+<td>10</td>
+<td>default tenant max request number per second</td>
+</tr>
+<tr>
+<td>traffic.control.customize-tenant-qps-rate</td>
+<td></td>
+<td>customize tenant max request number per second</td>
+</tr>
</tbody>
</table>
<h3>master.properties [master-service log config]</h3>
diff --git a/en-us/docs/dev/user_doc/architecture/configuration.json
b/en-us/docs/dev/user_doc/architecture/configuration.json
index 5a15ab196..f033c23ca 100644
--- a/en-us/docs/dev/user_doc/architecture/configuration.json
+++ b/en-us/docs/dev/user_doc/architecture/configuration.json
@@ -1,6 +1,6 @@
{
"filename": "configuration.md",
- "__html": "<!-- markdown-link-check-disable
-->\n<h1>Configuration</h1>\n<h2>Preface</h2>\n<p>This document explains the
DolphinScheduler application configurations according to DolphinScheduler-1.3.x
versions.</p>\n<h2>Directory Structure</h2>\n<p>Currently, all the
configuration files are under [conf ] directory.\nCheck the following
simplified DolphinScheduler installation directories to have a direct view
about the position of [conf] directory and configuration files it has.\nThis
[...]
+ "__html": "<!-- markdown-link-check-disable
-->\n<h1>Configuration</h1>\n<h2>Preface</h2>\n<p>This document explains the
DolphinScheduler application configurations according to DolphinScheduler-1.3.x
versions.</p>\n<h2>Directory Structure</h2>\n<p>Currently, all the
configuration files are under [conf ] directory.\nCheck the following
simplified DolphinScheduler installation directories to have a direct view
about the position of [conf] directory and configuration files it has.\nThis
[...]
"link": "/dist/en-us/docs/dev/user_doc/architecture/configuration.html",
"meta": {}
}
\ No newline at end of file
diff --git a/en-us/docs/dev/user_doc/guide/task/jupyter.html
b/en-us/docs/dev/user_doc/guide/task/jupyter.html
index 74d6878ce..0400a396a 100644
--- a/en-us/docs/dev/user_doc/guide/task/jupyter.html
+++ b/en-us/docs/dev/user_doc/guide/task/jupyter.html
@@ -20,6 +20,40 @@ it will use <code>papermill</code> to evaluate jupyter
notes. Click <a href="htt
Click <a href="https://docs.conda.io/en/latest/";>here</a> for more information
about <code>conda</code>.</li>
<li><code>conda.path</code> is set to
<code>/opt/anaconda3/etc/profile.d/conda.sh</code> by default. If you have no
idea where your <code>conda</code> is, simply run <code>conda info | grep -i
'base environment'</code>.</li>
</ul>
+<blockquote>
+<p>NOTICE: <code>Jupyter Task Plugin</code> uses <code>source</code> command
to activate conda environment.
+If your tenant does not have permission to use <code>source</code>,
<code>Jupyter Task Plugin</code> will not function.</p>
+</blockquote>
+<h2>Python Dependency Management</h2>
+<h3>Use Pre-Installed Conda Environment</h3>
+<ol>
+<li>Create a conda environment manually or using <code>shell task</code> on
your target worker.</li>
+<li>In your <code>jupyter task</code>, set <code>condaEnvName</code> as the
name of the conda environment you just created.</li>
+</ol>
+<h3>Use Packed Conda Environment</h3>
+<ol>
+<li>Use <a href="https://conda.github.io/conda-pack/";>Conda-Pack</a> to pack
your conda environment into <code>tarball</code>.</li>
+<li>Upload packed conda environment to <code>resource center</code>.</li>
+<li>Select your packed conda environment as <code>resource</code> in your
<code>jupyter task</code>, e.g. <code>jupyter_env.tar.gz</code>.</li>
+</ol>
+<blockquote>
+<p><strong><em>Note:</em></strong> Make sure you follow the <a
href="https://conda.github.io/conda-pack/";>Conda-Pack</a> official instructions.
+If you unpack your packed conda environment, the directory structure should be
the same as below:</p>
+</blockquote>
+<pre><code>.
+├── bin
+├── conda-meta
+├── etc
+├── include
+├── lib
+├── share
+└── ssl
+</code></pre>
+<blockquote>
+<p>NOTICE: Please follow the <code>conda pack</code> instructions above
strictly, and DO NOT modify <code>bin/activate</code>.
+<code>Jupyter Task Plugin</code> uses <code>source</code> command to activate
your packed conda environment.
+If you are concerned about using <code>source</code>, choose other options to
manage your python dependency.</p>
+</blockquote>
<h2>Create Task</h2>
<ul>
<li>Click Project Management-Project Name-Workflow Definition, and click the
"Create Workflow" button to enter the DAG editing page.</li>
@@ -37,7 +71,7 @@ Click <a href="https://docs.conda.io/en/latest/";>here</a> for
more information a
<li>Cpu quota: Assign the specified CPU time quota to the task executed. Takes
a percentage value. Default -1 means unlimited. For example, the full CPU load
of one core is 100%,and that of 16 cores is 1600%. This function is controlled
by <a
href="../../architecture/configuration.md">task.resource.limit.state</a></li>
<li>Max memory：Assign the specified max memory to the task executed. Exceeding
this limit will trigger oom to be killed and will not automatically retry.
Takes an MB value. Default -1 means unlimited. This function is controlled by
<a href="../../architecture/configuration.md">task.resource.limit.state</a></li>
<li>Timeout alarm: Check the timeout alarm and timeout failure. When the task
exceeds the "timeout period", an alarm email will send and the task
execution will fail.</li>
-<li>Conda Env Name: Name of conda environment.</li>
+<li>Conda Env Name: Name of conda environment or packed conda environment
tarball.</li>
<li>Input Note Path: Path of input jupyter note template.</li>
<li>Out Note Path: Path of output note.</li>
<li>Jupyter Parameters: Parameters in json format used for jupyter note
parameterization.</li>
diff --git a/en-us/docs/dev/user_doc/guide/task/jupyter.json
b/en-us/docs/dev/user_doc/guide/task/jupyter.json
index b970793f2..ba2cc8eec 100644
--- a/en-us/docs/dev/user_doc/guide/task/jupyter.json
+++ b/en-us/docs/dev/user_doc/guide/task/jupyter.json
@@ -1,6 +1,6 @@
{
"filename": "jupyter.md",
- "__html": "<h1>Jupyter</h1>\n<h2>Overview</h2>\n<p>Use <code>Jupyter
Task</code> to create a jupyter-type task and execute jupyter notes. When the
worker executes <code>Jupyter Task</code>,\nit will use <code>papermill</code>
to evaluate jupyter notes. Click <a
href=\"https://papermill.readthedocs.io/en/latest/\";>here</a> for details about
<code>papermill</code>.</p>\n<h2>Conda Configuration</h2>\n<ul>\n<li>Config
<code>conda.path</code> in <code>common.properties</code> to the path of [...]
+ "__html": "<h1>Jupyter</h1>\n<h2>Overview</h2>\n<p>Use <code>Jupyter
Task</code> to create a jupyter-type task and execute jupyter notes. When the
worker executes <code>Jupyter Task</code>,\nit will use <code>papermill</code>
to evaluate jupyter notes. Click <a
href=\"https://papermill.readthedocs.io/en/latest/\";>here</a> for details about
<code>papermill</code>.</p>\n<h2>Conda Configuration</h2>\n<ul>\n<li>Config
<code>conda.path</code> in <code>common.properties</code> to the path of [...]
"link": "/dist/en-us/docs/dev/user_doc/guide/task/jupyter.html",
"meta": {}
}
\ No newline at end of file
diff --git a/zh-cn/docs/dev/user_doc/architecture/configuration.html
b/zh-cn/docs/dev/user_doc/architecture/configuration.html
index 9b1812c91..5145d9800 100644
--- a/zh-cn/docs/dev/user_doc/architecture/configuration.html
+++ b/zh-cn/docs/dev/user_doc/architecture/configuration.html
@@ -565,6 +565,31 @@
<td>CREATE</td>
<td>当LDAP用户不存在时执行的操作。CREATE：当用户不存在时自动新建用户, DENY：当用户不存在时拒绝登陆</td>
</tr>
+<tr>
+<td>traffic.control.global.switch</td>
+<td>false</td>
+<td>流量控制全局开关</td>
+</tr>
+<tr>
+<td>traffic.control.max-global-qps-rate</td>
+<td>300</td>
+<td>全局最大请求数/秒</td>
+</tr>
+<tr>
+<td>traffic.control.tenant-switch</td>
+<td>false</td>
+<td>流量控制租户开关</td>
+</tr>
+<tr>
+<td>traffic.control.default-tenant-qps-rate</td>
+<td>10</td>
+<td>默认租户最大请求数/秒限制</td>
+</tr>
+<tr>
+<td>traffic.control.customize-tenant-qps-rate</td>
+<td></td>
+<td>自定义租户最大请求数/秒限制</td>
+</tr>
</tbody>
</table>
<h2>6.master.properties [Master服务配置]</h2>
diff --git a/zh-cn/docs/dev/user_doc/architecture/configuration.json
b/zh-cn/docs/dev/user_doc/architecture/configuration.json
index dd6889ec2..76e220ae5 100644
--- a/zh-cn/docs/dev/user_doc/architecture/configuration.json
+++ b/zh-cn/docs/dev/user_doc/architecture/configuration.json
@@ -1,6 +1,6 @@
{
"filename": "configuration.md",
- "__html": "<!-- markdown-link-check-disable
-->\n<h1>前言</h1>\n<p>本文档为dolphinscheduler配置文件说明文档,针对版本为 dolphinscheduler-1.3.x
版本.</p>\n<h1>目录结构</h1>\n<p>目前dolphinscheduler 所有的配置文件都在 [conf ]
目录中.\n为了更直观的了解[conf]目录所在的位置以及包含的配置文件,请查看下面dolphinscheduler安装目录的简化说明.\n本文主要讲述dolphinscheduler的配置文件.其他部分先不做赘述.</p>\n<p>[注:以下
dolphinscheduler 简称为DS.]</p>\n<pre><code>├── LICENSE\n│\n├── NOTICE\n│\n├──
licenses licenses存放目录\n│\n├── bin
[...]
+ "__html": "<!-- markdown-link-check-disable
-->\n<h1>前言</h1>\n<p>本文档为dolphinscheduler配置文件说明文档,针对版本为 dolphinscheduler-1.3.x
版本.</p>\n<h1>目录结构</h1>\n<p>目前dolphinscheduler 所有的配置文件都在 [conf ]
目录中.\n为了更直观的了解[conf]目录所在的位置以及包含的配置文件,请查看下面dolphinscheduler安装目录的简化说明.\n本文主要讲述dolphinscheduler的配置文件.其他部分先不做赘述.</p>\n<p>[注:以下
dolphinscheduler 简称为DS.]</p>\n<pre><code>├── LICENSE\n│\n├── NOTICE\n│\n├──
licenses licenses存放目录\n│\n├── bin
[...]
"link": "/dist/zh-cn/docs/dev/user_doc/architecture/configuration.html",
"meta": {}
}
\ No newline at end of file
diff --git a/zh-cn/docs/dev/user_doc/guide/task/jupyter.html
b/zh-cn/docs/dev/user_doc/guide/task/jupyter.html
index 5a465b8ac..0dda88f72 100644
--- a/zh-cn/docs/dev/user_doc/guide/task/jupyter.html
+++ b/zh-cn/docs/dev/user_doc/guide/task/jupyter.html
@@ -11,21 +11,55 @@
</head>
<body>
<div id="root"><div class="md2html docs-page" data-reactroot=""><header
class="header-container header-container-dark"><div class="banner-tips"><div>🤔
有关于 Apache DolphinScheduler 的疑问，加入 Slack 频道来讨论他们 <a class="link-tips"
href="https://s.apache.org/dolphinscheduler-slack";>join #dolphinscheduler
channel</a>! 🌟</div></div><div class="header-body"><span class="mobile-menu-btn
mobile-menu-btn-dark"></span><a href="/zh-cn/index.html"><img class="logo"
src="/img/hlogo_white.svg"/></a><div cla [...]
-<h2>Overview</h2>
+<h2>综述</h2>
<p><code>Jupyter</code>任务类型，用于创建并执行<code>Jupyter</code>类型任务。worker
执行该任务的时候，会通过<code>papermill</code>执行<code>jupyter note</code>。
点击<a href="https://papermill.readthedocs.io/en/latest/";>这里</a>
获取更多关于<code>papermill</code>的信息。</p>
-<h2>Conda Configuration</h2>
+<h2>Conda虚拟环境配置</h2>
<ul>
<li>在<code>common.properties</code>配置<code>conda.path</code>，将其指向您的<code>conda.sh</code>。这里的<code>conda</code>应该是您用来管理您的
<code>papermill</code>和<code>jupyter</code>所在python环境的相同<code>conda</code>。
点击 <a href="https://docs.conda.io/en/latest/";>这里</a>
获取更多关于<code>conda</code>的信息.</li>
<li><code>conda.path</code>默认设置为<code>/opt/anaconda3/etc/profile.d/conda.sh</code>。
如果您不清楚您的<code>conda</code>环境在哪里，只需要在命令行执行<code>conda info | grep -i 'base
environment'</code>即可获得。</li>
</ul>
-<h2>Create Task</h2>
+<blockquote>
+<p>注意：<code>Jupyter任务插件</code>使用<code>source</code>命令激活conda环境，
+如果您的租户没有<code>source</code>命令使用权限，<code>Jupyter任务插件</code>将无法使用。</p>
+</blockquote>
+<h2>Python依赖管理</h2>
+<h3>使用预装好的Conda环境</h3>
+<ol>
+<li>手动或使用<code>shell任务</code>在您的目标机器上创建conda环境。</li>
+<li>在您的<code>jupyter任务</code>中，将<code>condaEnvName</code>设置为您在上一步创建的conda环境名。</li>
+</ol>
+<h3>使用打包的Conda环境</h3>
+<ol>
+<li>使用 <a href="https://conda.github.io/conda-pack/";>Conda-Pack</a>
将您的conda环境打包成<code>tarball</code>.</li>
+<li>将您打包好的conda环境上传到<code>资源中心</code>.</li>
+<li>在您的<code>jupyter任务</code>资源设置中，添加您在上一步中上传的conda环境包，如<code>jupyter_env.tar.gz</code>.</li>
+</ol>
+<blockquote>
+<p><strong><em>提示:</em></strong> 请您按照 <a
href="https://conda.github.io/conda-pack/";>Conda-Pack</a> 官方指导打包conda环境，
+正确打包出的conda环境包解压后文件目录结构应和下图完全一致：</p>
+</blockquote>
+<pre><code>.
+├── bin
+├── conda-meta
+├── etc
+├── include
+├── lib
+├── share
+└── ssl
+</code></pre>
+<blockquote>
+<p>注意: 请严格按照上述<code>conda pack</code>指示操作，并且不要随意修改<code>bin/activate</code>。
+<code>Jupyter任务插件</code>使用<code>source</code>命令激活您打包的conda环境。
+若您对使用<code>source</code>命令有安全性上的担忧，请使用其他方法管理您的python依赖。</p>
+</blockquote>
+<h2>创建任务</h2>
<ul>
<li>点击项目管理-项目名称-工作流定义，点击"创建工作流"按钮，进入DAG编辑页面。</li>
<li>工具栏中拖动 <img src="/img/tasks/icons/jupyter.png" width="15"/>
到画板中，即可完成创建。</li>
</ul>
-<h2>Task Parameter</h2>
+<h2>任务参数</h2>
<ul>
<li>任务名称：设置任务的名称。一个工作流定义中的节点名称是唯一的。</li>
<li>运行标志：标识这个节点是否能正常调度,如果不需要执行，可以打开禁止执行开关。</li>
@@ -38,7 +72,7 @@
<li>最大内存：为执行的任务分配指定的内存大小，超过会触发OOM被Kill同时不会进行自动重试，单位MB，默认-1代表不限制。这个功能由 <a
href="../../architecture/configuration.md">task.resource.limit.state</a> 控制</li>
<li>超时告警：勾选超时告警、超时失败，当任务超过"超时时长"后，会发送告警邮件并且任务执行失败.这个功能由 <a
href="../../architecture/configuration.md">task.resource.limit.state</a> 控制</li>
<li>前置任务：选择当前任务的前置任务，会将被选择的前置任务设置为当前任务的上游。</li>
-<li>Conda Env Name: Conda环境名称。</li>
+<li>Conda Env Name: Conda环境或打包的Conda环境包名称</li>
<li>Input Note Path: 输入的jupyter note模板路径。</li>
<li>Out Note Path: 输出的jupyter note路径。</li>
<li>Jupyter Parameters: 用于对接jupyter note参数化的JSON格式参数。</li>
@@ -48,8 +82,8 @@
<li>Jupyter Start Timeout: 对于jupyter notebook kernel设定的启动超时时间。</li>
<li>Others: 传入papermill命令的其他参数。</li>
</ul>
-<h2>Task Example</h2>
-<h3>Jupyter Task Example</h3>
+<h2>任务样例</h2>
+<h3>简单的Jupyter任务样例</h3>
<p>这个示例展示了如何创建Jupyter任务节点：</p>
<p><img src="/img/tasks/demo/jupyter.png" alt="demo-jupyter-simple"></p>
</div></section><footer class="footer-container"><div
class="footer-body"><div><h3>联系我们</h3><h4>有问题需要反馈？请通过以下方式联系我们。</h4></div><div
class="contact-container"><ul><li><a
href="/zh-cn/community/development/subscribe.html"><img class="img-base"
src="/img/emailgray.png"/><img class="img-change"
src="/img/emailblue.png"/><p>邮件列表</p></a></li><li><a
href="https://twitter.com/dolphinschedule";><img class="img-base"
src="/img/twittergray.png"/><img class="img-change"
src="/img/twitterblue.png"/><p [...]
diff --git a/zh-cn/docs/dev/user_doc/guide/task/jupyter.json
b/zh-cn/docs/dev/user_doc/guide/task/jupyter.json
index 418bf1f85..6a583c489 100644
--- a/zh-cn/docs/dev/user_doc/guide/task/jupyter.json
+++ b/zh-cn/docs/dev/user_doc/guide/task/jupyter.json
@@ -1,6 +1,6 @@
{
"filename": "jupyter.md",
- "__html":
"<h1>Jupyter</h1>\n<h2>Overview</h2>\n<p><code>Jupyter</code>任务类型，用于创建并执行<code>Jupyter</code>类型任务。worker
执行该任务的时候，会通过<code>papermill</code>执行<code>jupyter note</code>。\n点击<a
href=\"https://papermill.readthedocs.io/en/latest/\";>这里</a>
获取更多关于<code>papermill</code>的信息。</p>\n<h2>Conda
Configuration</h2>\n<ul>\n<li>在<code>common.properties</code>配置<code>conda.path</code>，将其指向您的<code>conda.sh</code>。这里的<code>conda</code>应该是您用来管理您的
<code>papermill</code>和<code>jupyter</code>所在python [...]
+ "__html":
"<h1>Jupyter</h1>\n<h2>综述</h2>\n<p><code>Jupyter</code>任务类型，用于创建并执行<code>Jupyter</code>类型任务。worker
执行该任务的时候，会通过<code>papermill</code>执行<code>jupyter note</code>。\n点击<a
href=\"https://papermill.readthedocs.io/en/latest/\";>这里</a>
获取更多关于<code>papermill</code>的信息。</p>\n<h2>Conda虚拟环境配置</h2>\n<ul>\n<li>在<code>common.properties</code>配置<code>conda.path</code>，将其指向您的<code>conda.sh</code>。这里的<code>conda</code>应该是您用来管理您的
<code>papermill</code>和<code>jupyter</code>所在python环境的相同<code>con [...]
"link": "/dist/zh-cn/docs/dev/user_doc/guide/task/jupyter.html",
"meta": {}
}
\ No newline at end of file
