tillrohrmann commented on a change in pull request #14305: URL: https://github.com/apache/flink/pull/14305#discussion_r537386611
########## File path: docs/deployment/resource-providers/native_kubernetes.md ########## @@ -24,394 +23,244 @@ specific language governing permissions and limitations under the License. --> -This page describes how to deploy a Flink session cluster natively on [Kubernetes](https://kubernetes.io). +This page describes how to deploy Flink natively on [Kubernetes](https://kubernetes.io). * This will be replaced by the TOC {:toc} -<div class="alert alert-warning"> -Flink's native Kubernetes integration is still experimental. There may be changes in the configuration and CLI flags in later versions. -</div> +## Getting Started -## Requirements +This *Getting Started* section guides you through setting up a fully functional Flink Cluster on Kubernetes. -- Kubernetes 1.9 or above. -- KubeConfig, which has access to list, create, delete pods and services, configurable via `~/.kube/config`. You can verify permissions by running `kubectl auth can-i <list|create|edit|delete> pods`. -- Kubernetes DNS enabled. -- A service Account with [RBAC](#rbac) permissions to create, delete pods. - -## Flink Kubernetes Session +### Introduction -### Start Flink Session +Kubernetes is a popular container-orchestration system for automating computer application deployment, scaling, and management. +Flink's native Kubernetes integration allows to directly deploy Flink on a running Kubernetes cluster. +Moreover, Flink is able to dynamically allocate and de-allocate TaskManagers depending on the required resources because it can directly talk to Kubernetes. -Follow these instructions to start a Flink Session within your Kubernetes cluster. +### Preparation -A session will start all required Flink services (JobManager and TaskManagers) so that you can submit programs to the cluster. -Note that you can run multiple programs per session. +This *Getting Started* section assumes a running Kubernetes cluster fulfilling the following requirements: -{% highlight bash %} -$ ./bin/kubernetes-session.sh -{% endhighlight %} +- Kubernetes >= 1.9. +- KubeConfig, which has access to list, create, delete pods and services, configurable via `~/.kube/config`. You can verify permissions by running `kubectl auth can-i <list|create|edit|delete> pods`. +- Enabled Kubernetes DNS. +- `default` service account with [RBAC](#rbac) permissions to create, delete pods. -All the Kubernetes configuration options can be found in our [configuration guide]({% link deployment/config.md %}#kubernetes). +If you have problems setting up a Kubernetes cluster, then take a look at [how to setup a Kubernetes cluster](https://kubernetes.io/docs/setup/). -**Example**: Issue the following command to start a session cluster with 4 GB of memory and 2 CPUs with 4 slots per TaskManager: +### Starting a Flink Session on Kubernetes -In this example we override the `resourcemanager.taskmanager-timeout` setting to make -the pods with task managers remain for a longer period than the default of 30 seconds. -Although this setting may cause more cloud cost it has the effect that starting new jobs is in some scenarios -faster and during development you have more time to inspect the logfiles of your job. +Once you have your Kubernetes cluster running and configured your `kubectl` to point to it, you can launch a Flink session cluster via {% highlight bash %} -$ ./bin/kubernetes-session.sh \ - -Dkubernetes.cluster-id=<ClusterId> \ - -Dtaskmanager.memory.process.size=4096m \ - -Dkubernetes.taskmanager.cpu=2 \ - -Dtaskmanager.numberOfTaskSlots=4 \ - -Dresourcemanager.taskmanager-timeout=3600000 -{% endhighlight %} +# (1) Start Kubernetes session +$ ./bin/kubernetes-session.sh -Dkubernetes.cluster-id=my-first-flink-cluster -The system will use the configuration in `conf/flink-conf.yaml`. -Please follow our [configuration guide]({% link deployment/config.md %}) if you want to change something. +# (2) Submit example job +$ ./bin/flink run --target kubernetes-session -Dkubernetes.cluster-id=my-first-flink-cluster ./examples/streaming/TopSpeedWindowing.jar -If you do not specify a particular name for your session by `kubernetes.cluster-id`, the Flink client will generate a UUID name. +# (3) Stop Kubernetes session by deleting cluster deployment +$ kubectl delete deployment/my-first-flink-cluster Review comment: Alright. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
