This is an automated email from the ASF dual-hosted git repository. trohrmann pushed a commit to branch release-1.12 in repository https://gitbox.apache.org/repos/asf/flink.git
commit a319f5a77e4780b2cdd6e31fc631041a23278536 Author: Till Rohrmann <[email protected]> AuthorDate: Wed Nov 25 14:00:28 2020 +0100 [FLINK-20342][docs] Group standalone resource providers under resource-providers/standalone --- docs/concepts/flink-architecture.md | 2 +- docs/concepts/flink-architecture.zh.md | 2 +- docs/deployment/config.md | 4 ++-- docs/deployment/config.zh.md | 4 ++-- docs/deployment/index.md | 8 ++++---- docs/deployment/index.zh.md | 8 ++++---- docs/deployment/jobmanager_high_availability.md | 6 +++--- docs/deployment/jobmanager_high_availability.zh.md | 6 +++--- docs/deployment/memory/mem_migration.md | 12 ++++++------ docs/deployment/memory/mem_migration.zh.md | 8 ++++---- docs/deployment/memory/mem_setup.md | 4 ++-- docs/deployment/memory/mem_setup.zh.md | 4 ++-- docs/deployment/memory/mem_tuning.md | 4 ++-- docs/deployment/memory/mem_tuning.zh.md | 4 ++-- docs/deployment/repls/python_shell.md | 2 +- docs/deployment/repls/python_shell.zh.md | 2 +- docs/deployment/resource-providers/native_kubernetes.md | 6 +++--- .../resource-providers/native_kubernetes.zh.md | 4 ++-- .../resource-providers/{ => standalone}/docker.md | 10 +++++----- .../resource-providers/{ => standalone}/docker.zh.md | 10 +++++----- .../{cluster_setup.md => standalone/index.md} | 4 +++- .../{cluster_setup.zh.md => standalone/index.zh.md} | 4 +++- .../resource-providers/{ => standalone}/kubernetes.md | 16 ++++++++-------- .../resource-providers/{ => standalone}/kubernetes.zh.md | 16 ++++++++-------- .../resource-providers/{ => standalone}/local.md | 2 +- .../resource-providers/{ => standalone}/local.zh.md | 2 +- docs/dev/table/sqlClient.md | 2 +- docs/dev/table/sqlClient.zh.md | 2 +- docs/ops/upgrading.md | 2 +- docs/ops/upgrading.zh.md | 2 +- docs/redirects/local_setup_tutorial.md | 2 +- docs/redirects/setup_quickstart.md | 2 +- docs/redirects/windows_local_setup.md | 2 +- docs/release-notes/flink-1.11.md | 10 +++++----- docs/release-notes/flink-1.11.zh.md | 10 +++++----- 35 files changed, 96 insertions(+), 92 deletions(-) diff --git a/docs/concepts/flink-architecture.md b/docs/concepts/flink-architecture.md index 73a3431..3b31ae5 100644 --- a/docs/concepts/flink-architecture.md +++ b/docs/concepts/flink-architecture.md @@ -53,7 +53,7 @@ that triggers the execution, or in the command line process `./bin/flink run The JobManager and TaskManagers can be started in various ways: directly on the machines as a [standalone cluster]({% link -deployment/resource-providers/cluster_setup.md %}), in containers, or managed by resource +deployment/resource-providers/standalone/index.md %}), in containers, or managed by resource frameworks like [YARN]({% link deployment/resource-providers/yarn_setup.md %}) or [Mesos]({% link deployment/resource-providers/mesos.md %}). TaskManagers connect to JobManagers, announcing themselves as available, and diff --git a/docs/concepts/flink-architecture.zh.md b/docs/concepts/flink-architecture.zh.md index f6335fe..82386a2 100644 --- a/docs/concepts/flink-architecture.zh.md +++ b/docs/concepts/flink-architecture.zh.md @@ -53,7 +53,7 @@ that triggers the execution, or in the command line process `./bin/flink run The JobManager and TaskManagers can be started in various ways: directly on the machines as a [standalone cluster]({% link -deployment/resource-providers/cluster_setup.zh.md %}), in containers, or managed by resource +deployment/resource-providers/standalone/index.zh.md %}), in containers, or managed by resource frameworks like [YARN]({% link deployment/resource-providers/yarn_setup.zh.md %}) or [Mesos]({% link deployment/resource-providers/mesos.zh.md %}). TaskManagers connect to JobManagers, announcing themselves as available, and diff --git a/docs/deployment/config.md b/docs/deployment/config.md index 2cd4e80a..2d0bc59 100644 --- a/docs/deployment/config.md +++ b/docs/deployment/config.md @@ -39,7 +39,7 @@ The options in this section are the ones most commonly needed for a basic distri **Hostnames / Ports** -These options are only necessary for *standalone* application- or session deployments ([simple standalone]({% link deployment/resource-providers/cluster_setup.md %}) or [Kubernetes]({% link deployment/resource-providers/kubernetes.md %})). +These options are only necessary for *standalone* application- or session deployments ([simple standalone]({% link deployment/resource-providers/standalone/index.md %}) or [Kubernetes]({% link deployment/resource-providers/standalone/kubernetes.md %})). If you use Flink with [Yarn]({% link deployment/resource-providers/yarn_setup.md %}), [Mesos]({% link deployment/resource-providers/mesos.md %}), or the [*active* Kubernetes integration]({% link deployment/resource-providers/native_kubernetes.md %}), the hostnames and ports are automatically discovered. @@ -202,7 +202,7 @@ Please refer to the [Flink and Kerberos Docs]({% link deployment/security/securi This section contains options related to integrating Flink with resource orchestration frameworks, like Kubernetes, Yarn, Mesos, etc. Note that is not always necessary to integrate Flink with the resource orchestration framework. -For example, you can easily deploy Flink applications on Kubernetes without Flink knowing that it runs on Kubernetes (and without specifying any of the Kubernetes config options here.) See [this setup guide]({% link deployment/resource-providers/kubernetes.md %}) for an example. +For example, you can easily deploy Flink applications on Kubernetes without Flink knowing that it runs on Kubernetes (and without specifying any of the Kubernetes config options here.) See [this setup guide]({% link deployment/resource-providers/standalone/kubernetes.md %}) for an example. The options in this section are necessary for setups where Flink itself actively requests and releases resources from the orchestrators. diff --git a/docs/deployment/config.zh.md b/docs/deployment/config.zh.md index 79abd26..bb22de9 100644 --- a/docs/deployment/config.zh.md +++ b/docs/deployment/config.zh.md @@ -39,7 +39,7 @@ The options in this section are the ones most commonly needed for a basic distri **Hostnames / Ports** -These options are only necessary for *standalone* application- or session deployments ([simple standalone]({% link deployment/resource-providers/cluster_setup.zh.md %}) or [Kubernetes]({% link deployment/resource-providers/kubernetes.zh.md %})). +These options are only necessary for *standalone* application- or session deployments ([simple standalone]({% link deployment/resource-providers/standalone/index.zh.md %}) or [Kubernetes]({% link deployment/resource-providers/standalone/kubernetes.zh.md %})). If you use Flink with [Yarn]({% link deployment/resource-providers/yarn_setup.zh.md %}), [Mesos]({% link deployment/resource-providers/mesos.zh.md %}), or the [*active* Kubernetes integration]({% link deployment/resource-providers/native_kubernetes.zh.md %}), the hostnames and ports are automatically discovered. @@ -202,7 +202,7 @@ Please refer to the [Flink and Kerberos Docs]({% link deployment/security/securi This section contains options related to integrating Flink with resource orchestration frameworks, like Kubernetes, Yarn, Mesos, etc. Note that is not always necessary to integrate Flink with the resource orchestration framework. -For example, you can easily deploy Flink applications on Kubernetes without Flink knowing that it runs on Kubernetes (and without specifying any of the Kubernetes config options here.) See [this setup guide]({% link deployment/resource-providers/kubernetes.zh.md %}) for an example. +For example, you can easily deploy Flink applications on Kubernetes without Flink knowing that it runs on Kubernetes (and without specifying any of the Kubernetes config options here.) See [this setup guide]({% link deployment/resource-providers/standalone/kubernetes.zh.md %}) for an example. The options in this section are necessary for setups where Flink itself actively requests and releases resources from the orchestrators. diff --git a/docs/deployment/index.md b/docs/deployment/index.md index 77c5e66..34d3fb7 100644 --- a/docs/deployment/index.md +++ b/docs/deployment/index.md @@ -120,7 +120,7 @@ Apache Flink ships with first class support for a number of common deployment ta </div> <div class="panel-body"> Run Flink locally for basic testing and experimentation - <br><a href="{% link deployment/resource-providers/local.md %}">Learn more</a> + <br><a href="{% link deployment/resource-providers/standalone/local.md %}">Learn more</a> </div> </div> </div> @@ -131,7 +131,7 @@ Apache Flink ships with first class support for a number of common deployment ta </div> <div class="panel-body"> A simple solution for running Flink on bare metal or VM's - <br><a href="{% link deployment/resource-providers/cluster_setup.md %}">Learn more</a> + <br><a href="{% link deployment/resource-providers/standalone/index.md %}">Learn more</a> </div> </div> </div> @@ -166,7 +166,7 @@ Apache Flink ships with first class support for a number of common deployment ta </div> <div class="panel-body"> A popular solution for running Flink within a containerized environment - <br><a href="{% link deployment/resource-providers/docker.md %}">Learn more</a> + <br><a href="{% link deployment/resource-providers/standalone/docker.md %}">Learn more</a> </div> </div> </div> @@ -177,7 +177,7 @@ Apache Flink ships with first class support for a number of common deployment ta </div> <div class="panel-body"> An automated system for deploying containerized applications - <br><a href="{% link deployment/resource-providers/kubernetes.md %}">Learn more</a> + <br><a href="{% link deployment/resource-providers/standalone/kubernetes.md %}">Learn more</a> </div> </div> </div> diff --git a/docs/deployment/index.zh.md b/docs/deployment/index.zh.md index 955abe5..2cdd7b4 100644 --- a/docs/deployment/index.zh.md +++ b/docs/deployment/index.zh.md @@ -120,7 +120,7 @@ Apache Flink ships with first class support for a number of common deployment ta </div> <div class="panel-body"> Run Flink locally for basic testing and experimentation - <br><a href="{% link deployment/resource-providers/local.zh.md %}">Learn more</a> + <br><a href="{% link deployment/resource-providers/standalone/local.zh.md %}">Learn more</a> </div> </div> </div> @@ -131,7 +131,7 @@ Apache Flink ships with first class support for a number of common deployment ta </div> <div class="panel-body"> A simple solution for running Flink on bare metal or VM's - <br><a href="{% link deployment/resource-providers/cluster_setup.zh.md %}">Learn more</a> + <br><a href="{% link deployment/resource-providers/standalone/index.zh.md %}">Learn more</a> </div> </div> </div> @@ -166,7 +166,7 @@ Apache Flink ships with first class support for a number of common deployment ta </div> <div class="panel-body"> A popular solution for running Flink within a containerized environment - <br><a href="{% link deployment/resource-providers/docker.zh.md %}">Learn more</a> + <br><a href="{% link deployment/resource-providers/standalone/docker.zh.md %}">Learn more</a> </div> </div> </div> @@ -177,7 +177,7 @@ Apache Flink ships with first class support for a number of common deployment ta </div> <div class="panel-body"> An automated system for deploying containerized applications - <br><a href="{% link deployment/resource-providers/kubernetes.zh.md %}">Learn more</a> + <br><a href="{% link deployment/resource-providers/standalone/kubernetes.zh.md %}">Learn more</a> </div> </div> </div> diff --git a/docs/deployment/jobmanager_high_availability.md b/docs/deployment/jobmanager_high_availability.md index d6ad735..91c95cf 100644 --- a/docs/deployment/jobmanager_high_availability.md +++ b/docs/deployment/jobmanager_high_availability.md @@ -216,7 +216,7 @@ Starting zookeeper daemon on host localhost.</pre> $ bin/yarn-session.sh -n 2</pre> ## Kubernetes Cluster High Availability -Kubernetes high availability service could support both [standalone Flink on Kubernetes]({{ site.baseurl}}/deployment/resource-providers/kubernetes.html) and [native Kubernetes integration]({{ site.baseurl}}/deployment/resource-providers/native_kubernetes.html). +Kubernetes high availability service could support both [standalone Flink on Kubernetes]({% link deployment/resource-providers/standalone/kubernetes.md %}) and [native Kubernetes integration]({% link deployment/resource-providers/native_kubernetes.md %}). When running Flink JobManager as a Kubernetes deployment, the replica count should be configured to 1 or greater. * The value `1` means that a new JobManager will be launched to take over leadership if the current one terminates exceptionally. @@ -230,9 +230,9 @@ high-availability.storageDir: hdfs:///flink/recovery {% endhighlight %} #### Example: Highly Available Standalone Flink Cluster on Kubernetes -Both session and job/application clusters support using the Kubernetes high availability service. Users just need to add the following Flink config options to [flink-configuration-configmap.yaml]({{ site.baseurl}}/deployment/resource-providers/kubernetes.html#common-cluster-resource-definitions). All other yamls do not need to be updated. +Both session and job/application clusters support using the Kubernetes high availability service. Users just need to add the following Flink config options to [flink-configuration-configmap.yaml]({% link deployment/resource-providers/standalone/kubernetes.md %}#common-cluster-resource-definitions). All other yamls do not need to be updated. -<span class="label label-info">Note</span> The filesystem which corresponds to the scheme of your configured HA storage directory must be available to the runtime. Refer to [custom Flink image]({{ site.baseurl}}/deployment/resource-providers/docker.html#customize-flink-image) and [enable plugins]({{ site.baseurl}}/deployment/resource-providers/docker.html#using-plugins) for more information. +<span class="label label-info">Note</span> The filesystem which corresponds to the scheme of your configured HA storage directory must be available to the runtime. Refer to [custom Flink image]({% link deployment/resource-providers/standalone/docker.md %}#customize-flink-image) and [enable plugins]({% link deployment/resource-providers/standalone/docker.md %}#using-plugins) for more information. {% highlight yaml %} apiVersion: v1 diff --git a/docs/deployment/jobmanager_high_availability.zh.md b/docs/deployment/jobmanager_high_availability.zh.md index 0bac266..2da40f2 100644 --- a/docs/deployment/jobmanager_high_availability.zh.md +++ b/docs/deployment/jobmanager_high_availability.zh.md @@ -216,7 +216,7 @@ Starting zookeeper daemon on host localhost.</pre> $ bin/yarn-session.sh -n 2</pre> ## Kubernetes Cluster High Availability -Kubernetes high availability service could support both [standalone Flink on Kubernetes]({{ site.baseurl}}/zh/deployment/resource-providers/kubernetes.html) and [native Kubernetes integration]({{ site.baseurl}}/zh/deployment/resource-providers/native_kubernetes.html). +Kubernetes high availability service could support both [standalone Flink on Kubernetes]({%link deployment/resource-providers/standalone/kubernetes.zh.md %}) and [native Kubernetes integration]({% link deployment/resource-providers/native_kubernetes.zh.md %}). When running Flink JobManager as a Kubernetes deployment, the replica count should be configured to 1 or greater. * The value `1` means that a new JobManager will be launched to take over leadership if the current one terminates exceptionally. @@ -230,9 +230,9 @@ high-availability.storageDir: hdfs:///flink/recovery {% endhighlight %} #### Example: Highly Available Standalone Flink Cluster on Kubernetes -Both session and job/application clusters support using the Kubernetes high availability service. Users just need to add the following Flink config options to [flink-configuration-configmap.yaml]({{ site.baseurl}}/zh/deployment/resource-providers/kubernetes.html#common-cluster-resource-definitions). All other yamls do not need to be updated. +Both session and job/application clusters support using the Kubernetes high availability service. Users just need to add the following Flink config options to [flink-configuration-configmap.yaml]({% link deployment/resource-providers/standalone/kubernetes.zh.md %}#common-cluster-resource-definitions). All other yamls do not need to be updated. -<span class="label label-info">Note</span> The filesystem which corresponds to the scheme of your configured HA storage directory must be available to the runtime. Refer to [custom Flink image]({{ site.baseurl}}/zh/deployment/resource-providers/docker.html#customize-flink-image) and [enable plugins]({{ site.baseurl}}/zh/deployment/resource-providers/docker.html#using-plugins) for more information. +<span class="label label-info">Note</span> The filesystem which corresponds to the scheme of your configured HA storage directory must be available to the runtime. Refer to [custom Flink image]({% link deployment/resource-providers/standalone/docker.zh.md %}#customize-flink-image) and [enable plugins]({% link deployment/resource-providers/standalone/docker.zh.md %}#using-plugins) for more information. {% highlight yaml %} apiVersion: v1 diff --git a/docs/deployment/memory/mem_migration.md b/docs/deployment/memory/mem_migration.md index 05f0001..a2aad1f 100644 --- a/docs/deployment/memory/mem_migration.md +++ b/docs/deployment/memory/mem_migration.md @@ -109,7 +109,7 @@ The following options are deprecated but if they are still used they will be int <td><h5>taskmanager.heap.size</h5></td> <td> <ul> - <li><a href="{% link deployment/config.md %}#taskmanager-memory-flink-size">taskmanager.memory.flink.size</a> for <a href="{% link deployment/resource-providers/cluster_setup.md %}">standalone deployment</a></li> + <li><a href="{% link deployment/config.md %}#taskmanager-memory-flink-size">taskmanager.memory.flink.size</a> for <a href="{% link deployment/resource-providers/standalone/index.md %}">standalone deployment</a></li> <li><a href="{% link deployment/config.md %}#taskmanager-memory-process-size">taskmanager.memory.process.size</a> for containerized deployments</li> </ul> See also <a href="#total-memory-previously-heap-memory">how to migrate total memory</a>. @@ -218,19 +218,19 @@ Previously, there were options responsible for setting the *JVM Heap* size of th * `jobmanager.heap.size` * `jobmanager.heap.mb` -Despite their naming, they represented the *JVM Heap* only for [standalone deployments]({% link deployment/resource-providers/cluster_setup.md %}). -For the containerized deployments ([Kubernetes]({% link deployment/resource-providers/kubernetes.md %}) and [Yarn]({% link deployment/resource-providers/yarn_setup.md %})), +Despite their naming, they represented the *JVM Heap* only for [standalone deployments]({% link deployment/resource-providers/standalone/index.md %}). +For the containerized deployments ([Kubernetes]({% link deployment/resource-providers/standalone/kubernetes.md %}) and [Yarn]({% link deployment/resource-providers/yarn_setup.md %})), they also included other off-heap memory consumption. The size of *JVM Heap* was additionally reduced by the container cut-off which has been completely removed after *1.11*. The [Mesos]({% link deployment/resource-providers/mesos.md %}) integration did not take into account the mentioned legacy memory options. The scripts provided in Flink to start the Mesos JobManager process did not set any memory JVM arguments. After the *1.11* release, -they are set the same way as it is done by the [standalone deployment]({% link deployment/resource-providers/cluster_setup.md %}) scripts. +they are set the same way as it is done by the [standalone deployment]({% link deployment/resource-providers/standalone/index.md %}) scripts. The mentioned legacy options have been deprecated. If they are used without specifying the corresponding new options, they will be directly translated into the following new options: -* JVM Heap ([`jobmanager.memory.heap.size`]({% link deployment/config.md %}#jobmanager-memory-heap-size)) for [standalone]({% link deployment/resource-providers/cluster_setup.md %}) and [Mesos]({% link deployment/resource-providers/mesos.md %}) deployments -* Total process memory ([`jobmanager.memory.process.size`]({% link deployment/config.md %}#jobmanager-memory-process-size)) for containerized deployments ([Kubernetes]({% link deployment/resource-providers/kubernetes.md %}) and [Yarn]({% link deployment/resource-providers/yarn_setup.md %})) +* JVM Heap ([`jobmanager.memory.heap.size`]({% link deployment/config.md %}#jobmanager-memory-heap-size)) for [standalone]({% link deployment/resource-providers/standalone/index.md %}) and [Mesos]({% link deployment/resource-providers/mesos.md %}) deployments +* Total process memory ([`jobmanager.memory.process.size`]({% link deployment/config.md %}#jobmanager-memory-process-size)) for containerized deployments ([Kubernetes]({% link deployment/resource-providers/standalone/kubernetes.md %}) and [Yarn]({% link deployment/resource-providers/yarn_setup.md %})) It is also recommended using these new options instead of the legacy ones as they might be completely removed in the following releases. diff --git a/docs/deployment/memory/mem_migration.zh.md b/docs/deployment/memory/mem_migration.zh.md index 35c970c..f66a1d6 100644 --- a/docs/deployment/memory/mem_migration.zh.md +++ b/docs/deployment/memory/mem_migration.zh.md @@ -105,7 +105,7 @@ Flink 自带的[默认 flink-conf.yaml](#default-configuration-in-flink-confyaml <td><h5>taskmanager.heap.size</h5></td> <td> <ul> - <li><a href="{%link deployment/resource-providers/cluster_setup.zh.md %}">独立部署模式（Standalone Deployment）</a>下：<a href="{%link deployment/config.zh.md %}#taskmanager-memory-flink-size">taskmanager.memory.flink.size</a></li> + <li><a href="{%link deployment/resource-providers/standalone/index.zh.md %}">独立部署模式（Standalone Deployment）</a>下：<a href="{%link deployment/config.zh.md %}#taskmanager-memory-flink-size">taskmanager.memory.flink.size</a></li> <li>容器化部署模式（Containerized Deployement）下：<a href="{%link deployment/config.zh.md %}#taskmanager-memory-process-size">taskmanager.memory.process.size</a></li> </ul> 请参考<a href="#total-memory-previously-heap-memory">如何升级总内存</a>。 @@ -221,14 +221,14 @@ Flink 现在总是会预留一部分 JVM 堆内存供框架使用（[`taskmanage * `jobmanager.heap.size` * `jobmanager.heap.mb` -尽管这两个参数以“堆（Heap）”命名，在此之前它们实际上只有在[独立部署模式]({% link deployment/resource-providers/cluster_setup.zh.md %})才完全对应于 *JVM 堆内存*。 -在容器化部署模式下（[Kubernetes]({% link deployment/resource-providers/kubernetes.zh.md %}) 和 [Yarn]({% link deployment/resource-providers/yarn_setup.zh.md %})），它们指定的内存还包含了其他堆外内存部分。 +尽管这两个参数以“堆（Heap）”命名，在此之前它们实际上只有在[独立部署模式]({% link deployment/resource-providers/standalone/index.zh.md %})才完全对应于 *JVM 堆内存*。 +在容器化部署模式下（[Kubernetes]({% link deployment/resource-providers/standalone/kubernetes.zh.md %}) 和 [Yarn]({% link deployment/resource-providers/yarn_setup.zh.md %})），它们指定的内存还包含了其他堆外内存部分。 *JVM 堆空间*的实际大小，是参数指定的大小减去容器切除（Cut-Off）内存后剩余的部分。 容器切除内存在 *1.11* 及以上版本中已被彻底移除。 上述两个参数此前对 [Mesos]({% link deployment/resource-providers/mesos.zh.md %}) 部署模式并不生效。 Flink 在 Mesos 上启动 JobManager 进程时并未设置任何 JVM 内存参数。 -从 *1.11* 版本开始，Flink 将采用与[独立部署模式]({% link deployment/resource-providers/cluster_setup.zh.md %})相同的方式设置这些参数。 +从 *1.11* 版本开始，Flink 将采用与[独立部署模式]({% link deployment/resource-providers/standalone/index.zh.md %})相同的方式设置这些参数。 这两个配置参数目前已被弃用。 如果配置了上述弃用的参数，同时又没有配置与之对应的新配置参数，那它们将按如下规则对应到新的配置参数。 diff --git a/docs/deployment/memory/mem_setup.md b/docs/deployment/memory/mem_setup.md index 6dfdad3..10fc6e9 100644 --- a/docs/deployment/memory/mem_setup.md +++ b/docs/deployment/memory/mem_setup.md @@ -59,7 +59,7 @@ The simplest way to setup memory in Flink is to configure either of the two foll The rest of the memory components will be adjusted automatically, based on default values or additionally configured options. See also how to set up other components for [TaskManager]({% link deployment/memory/mem_setup_tm.md %}) and [JobManager]({% link deployment/memory/mem_setup_jobmanager.md %}) memory. -Configuring *total Flink memory* is better suited for [standalone deployments]({% link deployment/resource-providers/cluster_setup.md %}) +Configuring *total Flink memory* is better suited for [standalone deployments]({% link deployment/resource-providers/standalone/index.md %}) where you want to declare how much memory is given to Flink itself. The *total Flink memory* splits up into *JVM Heap* and *Off-heap* memory. See also [how to configure memory for standalone deployments]({% link deployment/memory/mem_tuning.md %}#configure-memory-for-standalone-deployment). @@ -67,7 +67,7 @@ See also [how to configure memory for standalone deployments]({% link deployment If you configure *total process memory* you declare how much memory in total should be assigned to the Flink *JVM process*. For the containerized deployments it corresponds to the size of the requested container, see also [how to configure memory for containers]({% link deployment/memory/mem_tuning.md %}#configure-memory-for-containers) -([Kubernetes]({% link deployment/resource-providers/kubernetes.md %}), [Yarn]({% link deployment/resource-providers/yarn_setup.md %}) or [Mesos]({% link deployment/resource-providers/mesos.md %})). +([Kubernetes]({% link deployment/resource-providers/standalone/kubernetes.md %}), [Yarn]({% link deployment/resource-providers/yarn_setup.md %}) or [Mesos]({% link deployment/resource-providers/mesos.md %})). Another way to set up the memory is to configure the required internal components of the *total Flink memory* which are specific to the concrete Flink process. Check how to configure them for [TaskManager]({% link deployment/memory/mem_setup_tm.md %}#configure-heap-and-managed-memory) diff --git a/docs/deployment/memory/mem_setup.zh.md b/docs/deployment/memory/mem_setup.zh.md index f39a108..8a037b6 100644 --- a/docs/deployment/memory/mem_setup.zh.md +++ b/docs/deployment/memory/mem_setup.zh.md @@ -60,12 +60,12 @@ Flink JVM 进程的*进程总内存（Total Process Memory）*包含了由 Flink Flink 会根据默认值或其他配置参数自动调整剩余内存部分的大小。 关于各内存部分的更多细节，请分别参考 [TaskManager]({% link deployment/memory/mem_setup_tm.zh.md %}) 和 [JobManager]({% link deployment/memory/mem_setup_jobmanager.zh.md %}) 的相关文档。 -对于[独立部署模式（Standalone Deployment）]({% link deployment/resource-providers/cluster_setup.zh.md %})，如果你希望指定由 Flink 应用本身使用的内存大小，最好选择配置 *Flink 总内存*。 +对于[独立部署模式（Standalone Deployment）]({% link deployment/resource-providers/standalone/index.zh.md %})，如果你希望指定由 Flink 应用本身使用的内存大小，最好选择配置 *Flink 总内存*。 *Flink 总内存*会进一步划分为 *JVM 堆内存*和*堆外内存*。 更多详情请参考[如何为独立部署模式配置内存]({% link deployment/memory/mem_tuning.zh.md %}#configure-memory-for-standalone-deployment)。 通过配置*进程总内存*可以指定由 Flink *JVM 进程*使用的总内存大小。 -对于容器化部署模式（Containerized Deployment），这相当于申请的容器（Container）大小，详情请参考[如何配置容器内存]({% link deployment/memory/mem_tuning.zh.md %}#configure-memory-for-containers)（[Kubernetes]({% link deployment/resource-providers/kubernetes.zh.md %})、[Yarn]({% link deployment/resource-providers/yarn_setup.zh.md %}) 或 [Mesos]({% link deployment/resource-providers/mesos.zh.md %})）。 +对于容器化部署模式（Containerized Deployment），这相当于申请的容器（Container）大小，详情请参考[如何配置容器内存]({% link deployment/memory/mem_tuning.zh.md %}#configure-memory-for-containers)（[Kubernetes]({% link deployment/resource-providers/standalone/kubernetes.zh.md %})、[Yarn]({% link deployment/resource-providers/yarn_setup.zh.md %}) 或 [Mesos]({% link deployment/resource-providers/mesos.zh.md %})）。 此外，还可以通过设置 *Flink 总内存*的特定内部组成部分的方式来进行内存配置。 不同进程需要设置的内存组成部分是不一样的。 diff --git a/docs/deployment/memory/mem_tuning.md b/docs/deployment/memory/mem_tuning.md index 3880954..1d9e761 100644 --- a/docs/deployment/memory/mem_tuning.md +++ b/docs/deployment/memory/mem_tuning.md @@ -32,7 +32,7 @@ depending on the use case and which options are important for each case. It is recommended to configure [total Flink memory]({% link deployment/memory/mem_setup.md %}#configure-total-memory) ([`taskmanager.memory.flink.size`]({% link deployment/config.md %}#taskmanager-memory-flink-size) or [`jobmanager.memory.flink.size`]({% link deployment/config.md %}#jobmanager-memory-flink-size)) -or its components for [standalone deployment]({% link deployment/resource-providers/cluster_setup.md %}) where you want to declare how much memory +or its components for [standalone deployment]({% link deployment/resource-providers/standalone/index.md %}) where you want to declare how much memory is given to Flink itself. Additionally, you can adjust *JVM metaspace* if it causes [problems]({% link deployment/memory/mem_trouble.md %}#outofmemoryerror-metaspace). The *total Process memory* is not relevant because *JVM overhead* is not controlled by Flink or the deployment environment, @@ -42,7 +42,7 @@ only physical resources of the executing machine matter in this case. It is recommended to configure [total process memory]({% link deployment/memory/mem_setup.md %}#configure-total-memory) ([`taskmanager.memory.process.size`]({% link deployment/config.md %}#taskmanager-memory-process-size) or [`jobmanager.memory.process.size`]({% link deployment/config.md %}#jobmanager-memory-process-size)) -for the containerized deployments ([Kubernetes]({% link deployment/resource-providers/kubernetes.md %}), [Yarn]({% link deployment/resource-providers/yarn_setup.md %}) or [Mesos]({% link deployment/resource-providers/mesos.md %})). +for the containerized deployments ([Kubernetes]({% link deployment/resource-providers/standalone/kubernetes.md %}), [Yarn]({% link deployment/resource-providers/yarn_setup.md %}) or [Mesos]({% link deployment/resource-providers/mesos.md %})). It declares how much memory in total should be assigned to the Flink *JVM process* and corresponds to the size of the requested container. <span class="label label-info">Note</span> If you configure the *total Flink memory* Flink will implicitly add JVM memory components diff --git a/docs/deployment/memory/mem_tuning.zh.md b/docs/deployment/memory/mem_tuning.zh.md index d85578b..f8c71ac 100644 --- a/docs/deployment/memory/mem_tuning.zh.md +++ b/docs/deployment/memory/mem_tuning.zh.md @@ -31,7 +31,7 @@ under the License. ## 独立部署模式（Standalone Deployment）下的内存配置 -[独立部署模式]({% link deployment/resource-providers/cluster_setup.zh.md %})下，我们通常更关注 Flink 应用本身使用的内存大小。 +[独立部署模式]({% link deployment/resource-providers/standalone/index.zh.md %})下，我们通常更关注 Flink 应用本身使用的内存大小。 建议配置 [Flink 总内存]({% link deployment/memory/mem_setup.zh.md %}#configure-total-memory)（[`taskmanager.memory.flink.size`]({% link deployment/config.zh.md %}#taskmanager-memory-flink-size) 或者 [`jobmanager.memory.flink.size`]({% link deployment/config.zh.md %}#jobmanager-memory-flink-size.zh.md %})）或其组成部分。 此外，如果出现 [Metaspace 不足的问题]({% link deployment/memory/mem_trouble.zh.md %}#outofmemoryerror-metaspace)，可以调整 *JVM Metaspace* 的大小。 @@ -41,7 +41,7 @@ under the License. ## 容器（Container）的内存配置 -在容器化部署模式（Containerized Deployment）下（[Kubernetes]({% link deployment/resource-providers/kubernetes.zh.md %})、[Yarn]({% link deployment/resource-providers/yarn_setup.zh.md %}) 或 [Mesos]({% link deployment/resource-providers/mesos.zh.md %})），建议配置[进程总内存]({% link deployment/memory/mem_setup.zh.md %}#configure-total-memory)（[`taskmanager.memory.process.size`]({% link deployment/config.zh.md %}#taskmanager-memory-process-size) 或者 [`jobmanager.memory.process.size`]({% link deployment/config.zh.m [...] +在容器化部署模式（Containerized Deployment）下（[Kubernetes]({% link deployment/resource-providers/standalone/kubernetes.zh.md %})、[Yarn]({% link deployment/resource-providers/yarn_setup.zh.md %}) 或 [Mesos]({% link deployment/resource-providers/mesos.zh.md %})），建议配置[进程总内存]({% link deployment/memory/mem_setup.zh.md %}#configure-total-memory)（[`taskmanager.memory.process.size`]({% link deployment/config.zh.md %}#taskmanager-memory-process-size) 或者 [`jobmanager.memory.process.size`]({% link deployment/ [...] 该配置参数用于指定分配给 Flink *JVM 进程*的总内存，也就是需要申请的容器大小。 <span class="label label-info">提示</span> diff --git a/docs/deployment/repls/python_shell.md b/docs/deployment/repls/python_shell.md index ffabdd1..ddc7bc5 100644 --- a/docs/deployment/repls/python_shell.md +++ b/docs/deployment/repls/python_shell.md @@ -24,7 +24,7 @@ under the License. Flink comes with an integrated interactive Python Shell. It can be used in a local setup as well as in a cluster setup. -See the [local setup page]({% link deployment/resource-providers/local.md %}) for more information about how to setup a local Flink. +See the [local setup page]({% link deployment/resource-providers/standalone/local.md %}) for more information about how to setup a local Flink. You can also [build a local setup from source]({% link flinkDev/building.md %}). <span class="label label-info">Note</span> The Python Shell will run the command “python”. Please refer to the Python Table API [installation guide]({% link dev/python/installation.md %}) on how to set up the Python execution environments. diff --git a/docs/deployment/repls/python_shell.zh.md b/docs/deployment/repls/python_shell.zh.md index 929a88c..72b79b5 100644 --- a/docs/deployment/repls/python_shell.zh.md +++ b/docs/deployment/repls/python_shell.zh.md @@ -24,7 +24,7 @@ under the License. Flink附带了一个集成的交互式Python Shell。 它既能够运行在本地启动的local模式，也能够运行在集群启动的cluster模式下。 -本地安装Flink，请看[本地安装]({% link deployment/resource-providers/local.zh.md %})页面。 +本地安装Flink，请看[本地安装]({% link deployment/resource-providers/standalone/local.zh.md %})页面。 您也可以从源码安装Flink，请看[从源码构建 Flink]({% link flinkDev/building.zh.md %})页面。 <span class="label label-info">注意</span> Python Shell会调用“python”命令。关于Python执行环境的要求，请参考Python Table API[环境安装]({% link dev/python/installation.zh.md %})。 diff --git a/docs/deployment/resource-providers/native_kubernetes.md b/docs/deployment/resource-providers/native_kubernetes.md index 967d433..4f66b21 100644 --- a/docs/deployment/resource-providers/native_kubernetes.md +++ b/docs/deployment/resource-providers/native_kubernetes.md @@ -83,8 +83,8 @@ Please refer to the following [section](#custom-flink-docker-image). <div class="codetabs" markdown="1"> <div data-lang="java" markdown="1"> -If you want to use a custom Docker image to deploy Flink containers, check [the Flink Docker image documentation]({% link deployment/resource-providers/docker.md %}), -[its tags]({% link deployment/resource-providers/docker.md %}#image-tags), [how to customize the Flink Docker image]({% link deployment/resource-providers/docker.md %}#customize-flink-image) and [enable plugins]({% link deployment/resource-providers/docker.md %}#using-plugins). +If you want to use a custom Docker image to deploy Flink containers, check [the Flink Docker image documentation]({% link deployment/resource-providers/standalone/docker.md %}), +[its tags]({% link deployment/resource-providers/standalone/docker.md %}#image-tags), [how to customize the Flink Docker image]({% link deployment/resource-providers/standalone/docker.md %}#customize-flink-image) and [enable plugins]({% link deployment/resource-providers/standalone/docker.md %}#using-plugins). If you created a custom Docker image you can provide it by setting the [`kubernetes.container.image`]({% link deployment/config.md %}#kubernetes-container-image) configuration option: {% highlight bash %} @@ -208,7 +208,7 @@ $ kubectl delete deployment/<ClusterID> ### Start Flink Application <div class="codetabs" markdown="1"> -Application mode allows users to create a single image containing their Job and the Flink runtime, which will automatically create and destroy cluster components as needed. The Flink community provides base docker images [customized]({% link deployment/resource-providers/docker.md %}#customize-flink-image) for any use case. +Application mode allows users to create a single image containing their Job and the Flink runtime, which will automatically create and destroy cluster components as needed. The Flink community provides base docker images [customized]({% link deployment/resource-providers/standalone/docker.md %}#customize-flink-image) for any use case. <div data-lang="java" markdown="1"> {% highlight dockerfile %} FROM flink diff --git a/docs/deployment/resource-providers/native_kubernetes.zh.md b/docs/deployment/resource-providers/native_kubernetes.zh.md index 3afb5f3..9353886 100644 --- a/docs/deployment/resource-providers/native_kubernetes.zh.md +++ b/docs/deployment/resource-providers/native_kubernetes.zh.md @@ -81,7 +81,7 @@ $ ./bin/kubernetes-session.sh \ <div class="codetabs" markdown="1"> <div data-lang="java" markdown="1"> -如果要使用自定义的 Docker 镜像部署 Flink 容器，请查看 [Flink Docker 镜像文档]({% link deployment/resource-providers/docker.zh.md %})、[镜像 tags]({% link deployment/resource-providers/docker.zh.md %}#image-tags)、[如何自定义 Flink Docker 镜像]({% link deployment/resource-providers/docker.zh.md %}#customize-flink-image)和[启用插件]({% link deployment/resource-providers/docker.zh.md %}#using-plugins)。 +如果要使用自定义的 Docker 镜像部署 Flink 容器，请查看 [Flink Docker 镜像文档]({% link deployment/resource-providers/standalone/docker.zh.md %})、[镜像 tags]({% link deployment/resource-providers/standalone/docker.zh.md %}#image-tags)、[如何自定义 Flink Docker 镜像]({% link deployment/resource-providers/standalone/docker.zh.md %}#customize-flink-image)和[启用插件]({% link deployment/resource-providers/standalone/docker.zh.md %}#using-plugins)。 如果创建了自定义的 Docker 镜像，则可以通过设置 [`kubernetes.container.image`]({% link deployment/config.zh.md %}#kubernetes-container-image) 配置项来指定它： {% highlight bash %} @@ -205,7 +205,7 @@ $ kubectl delete deployment/<ClusterID> ### 启动 Flink Application <div class="codetabs" markdown="1"> -Application 模式允许用户创建单个镜像，其中包含他们的作业和 Flink 运行时，该镜像将按需自动创建和销毁集群组件。Flink 社区提供了可以构建[多用途自定义镜像]({% link deployment/resource-providers/docker.zh.md %}#customize-flink-image)的基础镜像。 +Application 模式允许用户创建单个镜像，其中包含他们的作业和 Flink 运行时，该镜像将按需自动创建和销毁集群组件。Flink 社区提供了可以构建[多用途自定义镜像]({% link deployment/resource-providers/standalone/docker.zh.md %}#customize-flink-image)的基础镜像。 <div data-lang="java" markdown="1"> {% highlight dockerfile %} diff --git a/docs/deployment/resource-providers/docker.md b/docs/deployment/resource-providers/standalone/docker.md similarity index 97% rename from docs/deployment/resource-providers/docker.md rename to docs/deployment/resource-providers/standalone/docker.md index d0191c1..8ee9d4d 100644 --- a/docs/deployment/resource-providers/docker.md +++ b/docs/deployment/resource-providers/standalone/docker.md @@ -1,8 +1,8 @@ --- title: "Docker Setup" nav-title: Docker -nav-parent_id: resource_providers -nav-pos: 6 +nav-parent_id: standalone +nav-pos: 3 --- <!-- Licensed to the Apache Software Foundation (ASF) under one @@ -26,7 +26,7 @@ under the License. [Docker](https://www.docker.com) is a popular container runtime. There are Docker images for Apache Flink available [on Docker Hub](https://hub.docker.com/_/flink). You can use the docker images to deploy a *Session* or *Job cluster* in a containerized environment, e.g., -[standalone Kubernetes]({% link deployment/resource-providers/kubernetes.md %}) or [native Kubernetes]({% link deployment/resource-providers/native_kubernetes.md %}). +[standalone Kubernetes]({% link deployment/resource-providers/standalone/kubernetes.md %}) or [native Kubernetes]({% link deployment/resource-providers/native_kubernetes.md %}). * This will be replaced by the TOC {:toc} @@ -67,7 +67,7 @@ You can run its entry point in the following modes: This allows you to deploy a standalone cluster (Session or Job) in any containerised environment, for example: * manually in a local Docker setup, -* [in a Kubernetes cluster]({% link deployment/resource-providers/kubernetes.md %}), +* [in a Kubernetes cluster]({% link deployment/resource-providers/standalone/kubernetes.md %}), * [with Docker Compose](#flink-with-docker-compose), * [with Docker swarm](#flink-with-docker-swarm). @@ -78,7 +78,7 @@ The next chapters describe how to start a single Flink Docker container for vari Once you've started Flink on Docker, you can access the Flink Webfrontend on [localhost:8081](http://localhost:8081/#/overview) or submit jobs like this `./bin/flink run ./examples/streaming/TopSpeedWindowing.jar`. -We recommend using [Docker Compose]({% link deployment/resource-providers/docker.md %}#session-cluster-with-docker-compose) or [Docker Swarm]({% link deployment/resource-providers/docker.md %}#session-cluster-with-docker-swarm) for deploying Flink as a Session Cluster to ease system configuration. +We recommend using [Docker Compose]({% link deployment/resource-providers/standalone/docker.md %}#session-cluster-with-docker-compose) or [Docker Swarm]({% link deployment/resource-providers/standalone/docker.md %}#session-cluster-with-docker-swarm) for deploying Flink as a Session Cluster to ease system configuration. ### Start a Session Cluster diff --git a/docs/deployment/resource-providers/docker.zh.md b/docs/deployment/resource-providers/standalone/docker.zh.md similarity index 97% rename from docs/deployment/resource-providers/docker.zh.md rename to docs/deployment/resource-providers/standalone/docker.zh.md index 478ebe8..91eab55 100644 --- a/docs/deployment/resource-providers/docker.zh.md +++ b/docs/deployment/resource-providers/standalone/docker.zh.md @@ -1,8 +1,8 @@ --- title: "Docker 设置" nav-title: Docker -nav-parent_id: resource_providers -nav-pos: 6 +nav-parent_id: standalone +nav-pos: 3 --- <!-- Licensed to the Apache Software Foundation (ASF) under one @@ -26,7 +26,7 @@ under the License. [Docker](https://www.docker.com) is a popular container runtime. There are Docker images for Apache Flink available [on Docker Hub](https://hub.docker.com/_/flink). You can use the docker images to deploy a *Session* or *Job cluster* in a containerized environment, e.g., -[standalone Kubernetes]({% link deployment/resource-providers/kubernetes.zh.md %}) or [native Kubernetes]({% link deployment/resource-providers/native_kubernetes.zh.md %}). +[standalone Kubernetes]({% link deployment/resource-providers/standalone/kubernetes.zh.md %}) or [native Kubernetes]({% link deployment/resource-providers/native_kubernetes.zh.md %}). * This will be replaced by the TOC {:toc} @@ -67,7 +67,7 @@ You can run its entry point in the following modes: This allows you to deploy a standalone cluster (Session or Job) in any containerised environment, for example: * manually in a local Docker setup, -* [in a Kubernetes cluster]({% link deployment/resource-providers/kubernetes.zh.md %}), +* [in a Kubernetes cluster]({% link deployment/resource-providers/standalone/kubernetes.zh.md %}), * [with Docker Compose](#flink-with-docker-compose), * [with Docker swarm](#flink-with-docker-swarm). @@ -78,7 +78,7 @@ The next chapters describe how to start a single Flink Docker container for vari Once you've started Flink on Docker, you can access the Flink Webfrontend on [localhost:8081](http://localhost:8081/#/overview) or submit jobs like this `./bin/flink run ./examples/streaming/TopSpeedWindowing.jar`. -We recommend using [Docker Compose]({% link deployment/resource-providers/docker.zh.md %}#session-cluster-with-docker-compose) or [Docker Swarm]({% link deployment/resource-providers/docker.zh.md %}#session-cluster-with-docker-swarm) for deploying Flink as a Session Cluster to ease system configuration. +We recommend using [Docker Compose]({% link deployment/resource-providers/standalone/docker.zh.md %}#session-cluster-with-docker-compose) or [Docker Swarm]({% link deployment/resource-providers/standalone/docker.zh.md %}#session-cluster-with-docker-swarm) for deploying Flink as a Session Cluster to ease system configuration. ### Start a Session Cluster diff --git a/docs/deployment/resource-providers/cluster_setup.md b/docs/deployment/resource-providers/standalone/index.md similarity index 98% rename from docs/deployment/resource-providers/cluster_setup.md rename to docs/deployment/resource-providers/standalone/index.md index a9c568e..b8da461 100644 --- a/docs/deployment/resource-providers/cluster_setup.md +++ b/docs/deployment/resource-providers/standalone/index.md @@ -1,7 +1,9 @@ --- -title: "Standalone Cluster" +title: "Standalone" +nav-id: standalone nav-parent_id: resource_providers nav-pos: 3 +nav-show_overview: true --- <!-- Licensed to the Apache Software Foundation (ASF) under one diff --git a/docs/deployment/resource-providers/cluster_setup.zh.md b/docs/deployment/resource-providers/standalone/index.zh.md similarity index 98% rename from docs/deployment/resource-providers/cluster_setup.zh.md rename to docs/deployment/resource-providers/standalone/index.zh.md index 0ea905a..21310f0 100644 --- a/docs/deployment/resource-providers/cluster_setup.zh.md +++ b/docs/deployment/resource-providers/standalone/index.zh.md @@ -1,7 +1,9 @@ --- -title: "独立集群" +title: "Standalone" +nav-id: standalone nav-parent_id: resource_providers nav-pos: 3 +nav-show_overview: true --- <!-- Licensed to the Apache Software Foundation (ASF) under one diff --git a/docs/deployment/resource-providers/kubernetes.md b/docs/deployment/resource-providers/standalone/kubernetes.md similarity index 96% rename from docs/deployment/resource-providers/kubernetes.md rename to docs/deployment/resource-providers/standalone/kubernetes.md index 46b26e6..bd1b9a3 100644 --- a/docs/deployment/resource-providers/kubernetes.md +++ b/docs/deployment/resource-providers/standalone/kubernetes.md @@ -1,8 +1,8 @@ --- title: "Kubernetes Setup" nav-title: Kubernetes -nav-parent_id: resource_providers -nav-pos: 7 +nav-parent_id: standalone +nav-pos: 4 --- <!-- Licensed to the Apache Software Foundation (ASF) under one @@ -28,7 +28,7 @@ This page describes how to deploy a *Flink Job* and *Session cluster* on [Kubern * This will be replaced by the TOC {:toc} -{% info %} This page describes deploying a [standalone]({% link deployment/resource-providers/cluster_setup.md %}) Flink cluster on top of Kubernetes. +{% info %} This page describes deploying a [standalone]({% link deployment/resource-providers/standalone/index.md %}) Flink cluster on top of Kubernetes. You can find more information on native Kubernetes deployments [here]({% link deployment/resource-providers/native_kubernetes.md %}). ## Setup Kubernetes @@ -43,9 +43,9 @@ If you want to run Kubernetes locally, we recommend using [MiniKube](https://kub ## Flink Docker image -Before deploying the Flink Kubernetes components, please read [the Flink Docker image documentation]({% link deployment/resource-providers/docker.md %}), -[its tags]({% link deployment/resource-providers/docker.md %}#image-tags), [how to customize the Flink Docker image]({% link deployment/resource-providers/docker.md %}#customize-flink-image) and -[enable plugins]({% link deployment/resource-providers/docker.md %}#using-plugins) to use the image in the Kubernetes definition files. +Before deploying the Flink Kubernetes components, please read [the Flink Docker image documentation]({% link deployment/resource-providers/standalone/docker.md %}), +[its tags]({% link deployment/resource-providers/standalone/docker.md %}#image-tags), [how to customize the Flink Docker image]({% link deployment/resource-providers/standalone/docker.md %}#customize-flink-image) and +[enable plugins]({% link deployment/resource-providers/standalone/docker.md %}#using-plugins) to use the image in the Kubernetes definition files. ## Deploy Flink cluster on Kubernetes @@ -138,13 +138,13 @@ A basic *Flink Job cluster* deployment in Kubernetes has three components: Check [the Job cluster specific resource definitions](#job-cluster-resource-definitions) and adjust them accordingly. The `args` attribute in the `jobmanager-job.yaml` has to specify the main class of the user job. -See also [how to specify the JobManager arguments]({% link deployment/resource-providers/docker.md %}#jobmanager-additional-command-line-arguments) to understand +See also [how to specify the JobManager arguments]({% link deployment/resource-providers/standalone/docker.md %}#jobmanager-additional-command-line-arguments) to understand how to pass other `args` to the Flink image in the `jobmanager-job.yaml`. The *job artifacts* should be available from the `job-artifacts-volume` in [the resource definition examples](#job-cluster-resource-definitions). The definition examples mount the volume as a local directory of the host assuming that you create the components in a minikube cluster. If you do not use a minikube cluster, you can use any other type of volume, available in your Kubernetes cluster, to supply the *job artifacts*. -Alternatively, you can build [a custom image]({% link deployment/resource-providers/docker.md %}#start-a-job-cluster) which already contains the artifacts instead. +Alternatively, you can build [a custom image]({% link deployment/resource-providers/standalone/docker.md %}#start-a-job-cluster) which already contains the artifacts instead. After creating [the common cluster components](#deploy-flink-cluster-on-kubernetes), use [the Job cluster specific resource definitions](#job-cluster-resource-definitions) to launch the cluster with the `kubectl` command: diff --git a/docs/deployment/resource-providers/kubernetes.zh.md b/docs/deployment/resource-providers/standalone/kubernetes.zh.md similarity index 96% rename from docs/deployment/resource-providers/kubernetes.zh.md rename to docs/deployment/resource-providers/standalone/kubernetes.zh.md index 0787c4a..70d4219 100644 --- a/docs/deployment/resource-providers/kubernetes.zh.md +++ b/docs/deployment/resource-providers/standalone/kubernetes.zh.md @@ -1,8 +1,8 @@ --- title: "Kubernetes 设置" nav-title: Kubernetes -nav-parent_id: resource_providers -nav-pos: 7 +nav-parent_id: standalone +nav-pos: 4 --- <!-- Licensed to the Apache Software Foundation (ASF) under one @@ -28,7 +28,7 @@ This page describes how to deploy a *Flink Job* and *Session cluster* on [Kubern * This will be replaced by the TOC {:toc} -{% info %} This page describes deploying a [standalone]({% link deployment/resource-providers/cluster_setup.zh.md %}) Flink cluster on top of Kubernetes. +{% info %} This page describes deploying a [standalone]({% link deployment/resource-providers/standalone/index.zh.md %}) Flink cluster on top of Kubernetes. You can find more information on native Kubernetes deployments [here]({% link deployment/resource-providers/native_kubernetes.zh.md %}). ## Setup Kubernetes @@ -43,9 +43,9 @@ If you want to run Kubernetes locally, we recommend using [MiniKube](https://kub ## Flink Docker image -Before deploying the Flink Kubernetes components, please read [the Flink Docker image documentation]({% link deployment/resource-providers/docker.zh.md %}), -[its tags]({% link deployment/resource-providers/docker.zh.md %}#image-tags), [how to customize the Flink Docker image]({% link deployment/resource-providers/docker.zh.md %}#customize-flink-image) and -[enable plugins]({% link deployment/resource-providers/docker.zh.md %}#using-plugins) to use the image in the Kubernetes definition files. +Before deploying the Flink Kubernetes components, please read [the Flink Docker image documentation]({% link deployment/resource-providers/standalone/docker.zh.md %}), +[its tags]({% link deployment/resource-providers/standalone/docker.zh.md %}#image-tags), [how to customize the Flink Docker image]({% link deployment/resource-providers/standalone/docker.zh.md %}#customize-flink-image) and +[enable plugins]({% link deployment/resource-providers/standalone/docker.zh.md %}#using-plugins) to use the image in the Kubernetes definition files. ## Deploy Flink cluster on Kubernetes @@ -138,13 +138,13 @@ A basic *Flink Job cluster* deployment in Kubernetes has three components: Check [the Job cluster specific resource definitions](#job-cluster-resource-definitions) and adjust them accordingly. The `args` attribute in the `jobmanager-job.yaml` has to specify the main class of the user job. -See also [how to specify the JobManager arguments]({% link deployment/resource-providers/docker.zh.md %}#jobmanager-additional-command-line-arguments) to understand +See also [how to specify the JobManager arguments]({% link deployment/resource-providers/standalone/docker.zh.md %}#jobmanager-additional-command-line-arguments) to understand how to pass other `args` to the Flink image in the `jobmanager-job.yaml`. The *job artifacts* should be available from the `job-artifacts-volume` in [the resource definition examples](#job-cluster-resource-definitions). The definition examples mount the volume as a local directory of the host assuming that you create the components in a minikube cluster. If you do not use a minikube cluster, you can use any other type of volume, available in your Kubernetes cluster, to supply the *job artifacts*. -Alternatively, you can build [a custom image]({% link deployment/resource-providers/docker.zh.md %}#start-a-job-cluster) which already contains the artifacts instead. +Alternatively, you can build [a custom image]({% link deployment/resource-providers/standalone/docker.zh.md %}#start-a-job-cluster) which already contains the artifacts instead. After creating [the common cluster components](#deploy-flink-cluster-on-kubernetes), use [the Job cluster specific resource definitions](#job-cluster-resource-definitions) to launch the cluster with the `kubectl` command: diff --git a/docs/deployment/resource-providers/local.md b/docs/deployment/resource-providers/standalone/local.md similarity index 99% rename from docs/deployment/resource-providers/local.md rename to docs/deployment/resource-providers/standalone/local.md index 567a4ff..1d7f579 100644 --- a/docs/deployment/resource-providers/local.md +++ b/docs/deployment/resource-providers/standalone/local.md @@ -1,7 +1,7 @@ --- title: "Local Cluster" nav-title: 'Local Cluster' -nav-parent_id: resource_providers +nav-parent_id: standalone nav-pos: 2 --- <!-- diff --git a/docs/deployment/resource-providers/local.zh.md b/docs/deployment/resource-providers/standalone/local.zh.md similarity index 99% rename from docs/deployment/resource-providers/local.zh.md rename to docs/deployment/resource-providers/standalone/local.zh.md index 7d5d0de..4a4c6fe 100755 --- a/docs/deployment/resource-providers/local.zh.md +++ b/docs/deployment/resource-providers/standalone/local.zh.md @@ -1,7 +1,7 @@ --- title: "本地集群" nav-title: '本地集群' -nav-parent_id: resource_providers +nav-parent_id: standalone nav-pos: 2 --- <!-- diff --git a/docs/dev/table/sqlClient.md b/docs/dev/table/sqlClient.md index a4dc9f4..debad54 100644 --- a/docs/dev/table/sqlClient.md +++ b/docs/dev/table/sqlClient.md @@ -37,7 +37,7 @@ Getting Started This section describes how to setup and run your first Flink SQL program from the command-line. -The SQL Client is bundled in the regular Flink distribution and thus runnable out-of-the-box. It requires only a running Flink cluster where table programs can be executed. For more information about setting up a Flink cluster see the [Cluster & Deployment]({{ site.baseurl }}/deployment/resource-providers/cluster_setup.html) part. If you simply want to try out the SQL Client, you can also start a local cluster with one worker using the following command: +The SQL Client is bundled in the regular Flink distribution and thus runnable out-of-the-box. It requires only a running Flink cluster where table programs can be executed. For more information about setting up a Flink cluster see the [Cluster & Deployment]({{ site.baseurl }}/deployment/resource-providers/standalone/index.html) part. If you simply want to try out the SQL Client, you can also start a local cluster with one worker using the following command: {% highlight bash %} ./bin/start-cluster.sh diff --git a/docs/dev/table/sqlClient.zh.md b/docs/dev/table/sqlClient.zh.md index cd53edd..dee8686 100644 --- a/docs/dev/table/sqlClient.zh.md +++ b/docs/dev/table/sqlClient.zh.md @@ -36,7 +36,7 @@ Flink 的 Table & SQL API 可以处理 SQL 语言编写的查询语句，但是 本节介绍如何在命令行里启动（setup）和运行你的第一个 Flink SQL 程序。 -SQL 客户端捆绑在常规 Flink 发行版中，因此可以直接运行。它仅需要一个正在运行的 Flink 集群就可以在其中执行表程序。有关设置 Flink 群集的更多信息，请参见[集群和部署]({{ site.baseurl }}/zh/deployment/resource-providers/cluster_setup.html)部分。如果仅想试用 SQL 客户端，也可以使用以下命令启动本地集群： +SQL 客户端捆绑在常规 Flink 发行版中，因此可以直接运行。它仅需要一个正在运行的 Flink 集群就可以在其中执行表程序。有关设置 Flink 群集的更多信息，请参见[集群和部署]({{ site.baseurl }}/zh/deployment/resource-providers/standalone/index.html)部分。如果仅想试用 SQL 客户端，也可以使用以下命令启动本地集群： {% highlight bash %} ./bin/start-cluster.sh diff --git a/docs/ops/upgrading.md b/docs/ops/upgrading.md index aa471da..93a0fac 100644 --- a/docs/ops/upgrading.md +++ b/docs/ops/upgrading.md @@ -185,7 +185,7 @@ In this step, we update the framework version of the cluster. What this basicall the Flink installation with the new version. This step can depend on how you are running Flink in your cluster (e.g. standalone, on Mesos, ...). -If you are unfamiliar with installing Flink in your cluster, please read the [deployment and cluster setup documentation]({% link deployment/resource-providers/cluster_setup.md %}). +If you are unfamiliar with installing Flink in your cluster, please read the [deployment and cluster setup documentation]({% link deployment/resource-providers/standalone/index.md %}). ### STEP 3: Resume the job under the new Flink version from savepoint. diff --git a/docs/ops/upgrading.zh.md b/docs/ops/upgrading.zh.md index 8ca8708..27814bd 100644 --- a/docs/ops/upgrading.zh.md +++ b/docs/ops/upgrading.zh.md @@ -183,7 +183,7 @@ In this step, we update the framework version of the cluster. What this basicall the Flink installation with the new version. This step can depend on how you are running Flink in your cluster (e.g. standalone, on Mesos, ...). -If you are unfamiliar with installing Flink in your cluster, please read the [deployment and cluster setup documentation]({% link deployment/resource-providers/cluster_setup.zh.md %}). +If you are unfamiliar with installing Flink in your cluster, please read the [deployment and cluster setup documentation]({% link deployment/resource-providers/standalone/index.zh.md %}). ### STEP 3: Resume the job under the new Flink version from savepoint. diff --git a/docs/redirects/local_setup_tutorial.md b/docs/redirects/local_setup_tutorial.md index 91356a1..7ca7f8a 100644 --- a/docs/redirects/local_setup_tutorial.md +++ b/docs/redirects/local_setup_tutorial.md @@ -1,7 +1,7 @@ --- title: "Local Setup Tutorial" layout: redirect -redirect: /deployment/resource-providers/local.html +redirect: /deployment/resource-providers/standalone/local.html permalink: /getting-started/tutorials/local_setup.html --- <!-- diff --git a/docs/redirects/setup_quickstart.md b/docs/redirects/setup_quickstart.md index e8af998..f3f13d5 100644 --- a/docs/redirects/setup_quickstart.md +++ b/docs/redirects/setup_quickstart.md @@ -1,7 +1,7 @@ --- title: "Local Setup Tutorial" layout: redirect -redirect: /deployment/resource-providers/local.html +redirect: /deployment/resource-providers/standalone/local.html permalink: /quickstart/setup_quickstart.html --- <!-- diff --git a/docs/redirects/windows_local_setup.md b/docs/redirects/windows_local_setup.md index b5b5081..ce6611c 100644 --- a/docs/redirects/windows_local_setup.md +++ b/docs/redirects/windows_local_setup.md @@ -1,7 +1,7 @@ --- title: "Running Flink on Windows" layout: redirect -redirect: /deployment/resource-providers/local.html +redirect: /deployment/resource-providers/standalone/local.html permalink: /getting-started/tutorials/flink_on_windows.html --- <!-- diff --git a/docs/release-notes/flink-1.11.md b/docs/release-notes/flink-1.11.md index c805f08..166a545 100644 --- a/docs/release-notes/flink-1.11.md +++ b/docs/release-notes/flink-1.11.md @@ -68,11 +68,11 @@ The examples of `Dockerfiles` and docker image `build.sh` scripts have been remo - `flink-container/docker` - `flink-container/kubernetes` -Check the updated user documentation for [Flink Docker integration](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/docker.html) instead. It now describes in detail how to [use](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/docker.html#how-to-run-a-flink-image) and [customize](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/docker.html#customize-flink-image) [the Flink officia [...] -- [docker run](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/docker.html#how-to-run-flink-image) -- [docker compose](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/docker.html#flink-with-docker-compose) -- [docker swarm](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/docker.html#flink-with-docker-swarm) -- [standalone Kubernetes](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/kubernetes.html) +Check the updated user documentation for [Flink Docker integration](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/docker.html) instead. It now describes in detail how to [use](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/docker.html#how-to-run-a-flink-image) and [customize](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/docker.html#customiz [...] +- [docker run](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/docker.html#how-to-run-flink-image) +- [docker compose](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/docker.html#flink-with-docker-compose) +- [docker swarm](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/docker.html#flink-with-docker-swarm) +- [standalone Kubernetes](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/kubernetes.html) ### Memory Management #### New JobManager Memory Model diff --git a/docs/release-notes/flink-1.11.zh.md b/docs/release-notes/flink-1.11.zh.md index 8c92d15..4bfe98c 100644 --- a/docs/release-notes/flink-1.11.zh.md +++ b/docs/release-notes/flink-1.11.zh.md @@ -68,11 +68,11 @@ The examples of `Dockerfiles` and docker image `build.sh` scripts have been remo - `flink-container/docker` - `flink-container/kubernetes` -Check the updated user documentation for [Flink Docker integration](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/docker.html) instead. It now describes in detail how to [use](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/docker.html#how-to-run-a-flink-image) and [customize](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/docker.html#customize-flink-image) [the Flink officia [...] -- [docker run](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/docker.html#how-to-run-flink-image) -- [docker compose](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/docker.html#flink-with-docker-compose) -- [docker swarm](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/docker.html#flink-with-docker-swarm) -- [standalone Kubernetes](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/kubernetes.html) +Check the updated user documentation for [Flink Docker integration](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/docker.html) instead. It now describes in detail how to [use](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/docker.html#how-to-run-a-flink-image) and [customize](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/docker.html#customiz [...] +- [docker run](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/docker.html#how-to-run-flink-image) +- [docker compose](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/docker.html#flink-with-docker-compose) +- [docker swarm](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/docker.html#flink-with-docker-swarm) +- [standalone Kubernetes](https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/kubernetes.html) ### Memory Management #### New JobManager Memory Model
