This is an automated email from the ASF dual-hosted git repository. zykkk pushed a commit to branch revert-27049-operator-doc-en in repository https://gitbox.apache.org/repos/asf/doris.git
commit 17bf3eecfd9675a2cd772938aa2d618b9b417026 Author: zy-kkk <[email protected]> AuthorDate: Thu Nov 16 18:41:03 2023 +0800 Revert "[doc](fix) en docs for k8s operator (#27049)" This reverts commit 0ac3984d4b0a578f803e5e3e5311e1b2d66e4a71. --- docs/en/docs/install/k8s-deploy.md | 162 ++++++++++++++-------------------- docs/zh-CN/docs/install/k8s-deploy.md | 4 +- 2 files changed, 67 insertions(+), 99 deletions(-) diff --git a/docs/en/docs/install/k8s-deploy.md b/docs/en/docs/install/k8s-deploy.md index 27ad14ec7bd..408de23158c 100644 --- a/docs/en/docs/install/k8s-deploy.md +++ b/docs/en/docs/install/k8s-deploy.md @@ -1,7 +1,7 @@ --- { - "title": "Deploy Doris on Kubernetes", - "language": "en" +"title": "Kubernetes Deployment", +"language": "en" } --- @@ -24,98 +24,66 @@ specific language governing permissions and limitations under the License. --> -Doris-Operator is software extension to Kubernetes that make use of custom resource to manage Doris and it components. It provides [DorisCluster](https://github.com/selectdb/doris-operator/blob/master/config/crd/bases/doris.selectdb.com_dorisclusters.yaml) a Kubernetes [CustomResourceDefinition](https://kubernetes.io/docs/reference/kubernetes-api/extend-resources/custom-resource-definition-v1/) for user to custom resource. -## Deploy Doris on Kubernetes - -### Start Kubernetes -Having a Kubernetes environment is the premise to deploy Doris on Kubernetes. If you already have it, please ignore this step. -Hosted Kubernetes on cloud platform or set-up by yourself are all good choice. - -**Hosted EKS** -1. Check that the following [command-line](https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html) tools are installed in your environment: -- Install and configure AWS command-line tool AWS CLI. -- Install EKS cluster command-line tool eksctl. -- Install Kubernetes cluster command-line tool kubectl. - -2. Use one of the following methods to create an EKS cluster: -- [Use eksctl to quickly create an EKS cluster](https://docs.aws.amazon.com/eks/latest/userguide/getting-started-eksctl.html). -- [Manually create an EKS cluster with the AWS console and AWS CLI](https://docs.aws.amazon.com/eks/latest/userguide/getting-started-console.html). - -**Hosted GKE** -Complete all the [prerequisites](https://cloud.google.com/kubernetes-engine/docs/deploy-app-cluster#before-you-begin) when [create a GKE cluster](https://cloud.google.com/kubernetes-engine/docs/deploy-app-cluster#create_cluster). - -**Create as Kubernetes recommend** -Kubernetes official documents recommends some ways to set up Kubernetes, as [minikube](https://minikube.sigs.k8s.io/docs/start/),[kOps](https://kubernetes.io/zh-cn/docs/setup/production-environment/tools/kops/). - -### Deploy Doris-Operator on Kubernetes -**1. Apply the [Custom Resource Definition(CRD)](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/#customresourcedefinitions)** -```shell -kubectl apply -f https://raw.githubusercontent.com/selectdb/doris-operator/master/config/crd/bases/doris.selectdb.com_dorisclusters.yaml -``` -**2. Install Doris-Operator** -If you want to use the defaults operator resource: - ```shell - kubectl apply -f https://raw.githubusercontent.com/selectdb/doris-operator/master/config/operator/operator.yaml - ``` -The user defined deployment in github repo are simply: -Instead of using the command below, apply your local version of the Operator manifest to the cluster when you custom operator resource. - ```shell - kubectl apply -f operator.yaml - ``` -**3. Validate The Operator is Running** -Using the command `kubectl -n {namespace} get pods` get the status of deployed operator. -```shell - kubectl -n doris get pods - NAME READY STATUS RESTARTS AGE - doris-operator-5b9f7f57bf-tsvjz 1/1 Running 66 (164m ago) 6d22h -``` -Expected result, the Pod `STATUS` is `Running` and all containers in Pod are all `READY`. - -### Start Doris on Kubernetes -**1. Initialize Doris Cluster** -User can directly deploy Doris by [examples](https://github.com/selectdb/doris-operator/tree/master/doc/examples) provided by Doris-Operator. Below is the command: -```shell -kubectl apply -f https://raw.githubusercontent.com/selectdb/doris-operator/master/doc/examples/doriscluster-sample.yaml -``` -Or download [doriscluster-sample](https://github.com/selectdb/doris-operator/master/doc/examples/doriscluster-sample.yaml) a custom resource that tells the Operator how to configure the Kubernetes cluster, and custom resource as [api.md](https://github.com/selectdb/doris-operator/blob/master/doc/api.md) and -[how_to_use](https://github.com/selectdb/doris-operator/tree/master/doc/how_to_use.md) docs. Instead of using the command below, apply the customized resource. -```shell -kubeectl apply -f doriscluster-sample.yaml -``` -**2. Validate Doris Cluster Status** -Using the command `kubectl -n {namespace} get pods` check pods status. -```shell -kubectl get pods - NAME READY STATUS RESTARTS AGE - doriscluster-sample-fe-0 1/1 Running 0 20m - doriscluster-sample-be-0 1/1 Running 0 19m -``` -All Pods created by DorisCluster resource should be in `Running` STATUS, and each pod's containers should be `RREADY`. -### Use Doris Cluster -On kubernetes Doris-Operator provide `Service` a resource build in kubernetes for access to Doris. - -The command `kubectl -n {namespace} get svc -l "app.doris.ownerreference/name={dorisCluster.Name}"` used to get `service` created by Doris-Operator. `dorisCluster.Nmae` is the name of DorisCluster resource deployed by step 1. -```shell -kubectl -n default get svc -l "app.doris.ownerreference/name=doriscluster-sample" -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -doriscluster-sample-fe-internal ClusterIP None <none> 9030/TCP 30m -doriscluster-sample-fe-service ClusterIP 10.152.183.37 a7509284bf3784983a596c6eec7fc212-618xxxxxx.com 8030/TCP,9020/TCP,9030/TCP,9010/TCP 30m -doriscluster-sample-be-internal ClusterIP None <none> 9050/TCP 29m -doriscluster-sample-be-service ClusterIP 10.152.183.141 <none> 9060/TCP,8040/TCP,9050/TCP,8060/TCP 29m -``` -**Use SQL Client for Access** -Service created by Doris-Operator have two types, suffix is `-internal` or `-service`. Service have the `-internal` suffix for communicating in Doris components, Service have `-service` suffix for user to access. - -- In Kubernetes -In kubernetes, Using `CLUSTER-IP` is recommended. For example, the fe service's `CLUSTER-IP` is `10.152.183.37` that displayed by above command. Using below command to access fe service. - - ```shell - mysql -h 10.152.183.37 -uroot -P9030 - ``` - -- Out Kubernetes -Using `EXTERNAL-IP` to access fe from Kubernetes external. In default, Doris-Operator not provided `EXTERNAL-IP` mode. If you want to use `EXTERNAL-IP`, should custom resource `Service` field, reference the doc [api.md](https://github.com/selectdb/doris-operator/blob/master/doc/api.md) to deploy. - -:::tip -If the doc not cover your requirements, Pleaser reference the docs [Doris-Operator](https://github.com/selectdb/doris-operator/tree/master/doc/how_to_use.md) and the api document to custom [DorisCluster](https://github.com/selectdb/doris-operator/blob/master/doc/api.md) resource to deploy. -::: \ No newline at end of file +# K8s Deployment Doris + +<version since="dev"></version> + +## Environmental Preparation + +- Installation k8s +- Build or download a Doris image + - Building an image [Build Docker Image](./construct-docker/construct-docker-image) + - Download Image https://hub.docker.com/r/apache/doris/tags +- Create or download the yml file for Doris on k8s + - https://github.com/apache/doris/blob/master/docker/runtime/k8s/doris_follower.yml + - https://github.com/apache/doris/blob/master/docker/runtime/k8s/doris_be.yml + - https://github.com/apache/doris/blob/master/docker/runtime/k8s/doris_cn.yml + +## Starting A Cluster +Start FE (role type is Follower):`kubectl create -f doris_follower.yml` + +Start BE:`kubectl create -f doris_be.yml` + +Start the BE (role type is Compute Node):`kubectl create -f doris_cn.yml` + +## Expansion and Contraction Capacity + +- FE + - Currently, scaling is not supported. It is recommended to initialize 1 or 3 nodes as needed +- BE + - Command:`kubectl scale statefulset doris-be-cluster1 --replicas=4` +- BE (role type is Compute Node) + - Command:`kubectl scale statefulset doris-cn-cluster1 --replicas=4` + +## Test and Verify + +Connect to the FE using mysql-client and perform operations such as' show backends' and 'show frontends' to view the status of each node + +## K8s Simple Operation Command + +- Executing the yml file for the first time `kubectl create -f xxx.yml` +- Execute after modifying the yml file `kubectl apply -f xxx.yml` +- Delete all resources defined by yml `kubectl delete -f xxx.yml` +- View the pod list `kubectl get pods` +- Entering the container `kubectl exec -it xxx（podName） -- /bin/sh` +- view log `kubectl logs xxx（podName）` +- View IP and port information `kubectl get ep` +- [More knowledge of k8s](https://kubernetes.io) + +## Common Problem + +- How is data persistent? + + Users need to mount PVC on their own to persist metadata information, data information, or log information +- How to safely shrink the BE node? + + BE:User manual execution is required before current resizing[ALTER-SYSTEM-DECOMMISSION-BACKEND](../../sql-manual/sql-reference/Cluster-Management-Statements/ALTER-SYSTEM-DECOMMISSION-BACKEND) + + BE(The role type is Compute Node): Do not store data files and can directly shrink，[About Computing Nodes](../../advanced/compute_node) +- FE startup error "failed to init statefulSetName" + + doris_ The environment variables statefulSetName and serviceName for follower. yml must appear in pairs, such as CN configured_ SERVICE, CN must be configured_ STATEFULSET + + + + diff --git a/docs/zh-CN/docs/install/k8s-deploy.md b/docs/zh-CN/docs/install/k8s-deploy.md index da32b83cb72..06b34efa6da 100644 --- a/docs/zh-CN/docs/install/k8s-deploy.md +++ b/docs/zh-CN/docs/install/k8s-deploy.md @@ -24,10 +24,10 @@ specific language governing permissions and limitations under the License. --> -Doris-Operator 是按照 Kubernetes 原则构建的在 Kubernetes 平台之上管理运维 Doris 集群的管理软件，允许用户按照资源定义的方式在 Kubernetes 平台之上部署管理 Doris 服务。Doris-Operator 能够管理 Doris 的所有部署形态，能够实现 Doris 大规模形态下智能化和并行化管理。 - ## Kubernetes 上部署 Doris 集群 +<version since="dev"></version> +Doris-Operator 是按照 Kubernetes 原则构建的在 Kubernetes 平台之上管理运维 Doris 集群的管理软件，允许用户按照资源定义的方式在 Kubernetes 平台之上部署管理 Doris 服务。Doris-Operator 能够管理 Doris 的所有部署形态，能够实现 Doris 大规模形态下智能化和并行化管理。 ### 环境准备 使用 Doris-Operator 部署 Doris 前提需要一个 Kubernetes (简称 K8S)集群，如果已拥有可直接跳过环境准备阶段。 --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
