This is an automated email from the ASF dual-hosted git repository. wusheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/skywalking-swck.git
The following commit(s) were added to refs/heads/master by this push: new 954b36e Add a getting started doc for swck (#100) 954b36e is described below commit 954b36e470dadfe5902d7fea5ccccbb832ba3979 Author: Ye Cao <dashan...@gmail.com> AuthorDate: Sun Mar 10 20:14:51 2024 +0800 Add a getting started doc for swck (#100) --- CHANGES.md | 3 + README.md | 4 +- docs/getting-started.md | 371 +++++++++++++++++++++ .../controllers/operator/javaagent_controller.go | 7 + .../manifests/injector/templates/javaagent.yaml | 2 + test/e2e/skywalking-components.yaml | 1 - 6 files changed, 384 insertions(+), 4 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index 1f67f65..ac97a96 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -5,6 +5,9 @@ Release Notes. 0.9.0 ------------------ +#### Features +- Add a getting started document about how to deploy swck on the kubernetes cluster. + #### Bugs - Fix the bug that the java agent is duplicated injected when update the pod. diff --git a/README.md b/README.md index df38cb9..87bdf48 100644 --- a/README.md +++ b/README.md @@ -21,9 +21,7 @@ SWCK is a platform for the SkyWalking user that provisions, upgrades, maintains # Quick Start -There are two ways to install swck. -* Go to the [download page](https://skywalking.apache.org/downloads/#SkyWalkingCloudonKubernetes) to download the latest release binary, `skywalking-swck-<SWCK_VERSION>-bin.tgz`. Unarchive the package to a folder named `skywalking-swck-<SWCK_VERSION>-bin` -* Apply the kustomization directory from github. +You can follow the [Getting Started](docs/getting-started.md) to deploy swck on a testing Kubernetes cluster quickly and try out the skywalking components end to end. ## Java Agent Injector diff --git a/docs/getting-started.md b/docs/getting-started.md new file mode 100644 index 0000000..a8f2ce7 --- /dev/null +++ b/docs/getting-started.md @@ -0,0 +1,371 @@ +## Getting Started + +This document introduces how to create a kubernetes cluster locally using kind and how to deploy the basic skywalking components to the cluster. + +### Prerequisites +- [docker](https://docs.docker.com/get-docker/) >= v20.10.6 +- [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) >= v1.21.0 +- [kind](https://kind.sigs.k8s.io/docs/user/quick-start/#installation) >= v0.20.0 +- [swctl](https://github.com/apache/skywalking-cli?tab=readme-ov-file#install) >= v0.10.0 + +### Step1: Create a kubernetes cluster locally using kind + +> Note: If you have a kubernetes cluster (> v1.21.10) already, you can skip this step. + +Here we create a kubernetes cluster with 1 control-plane node and 1 worker nodes. + +```shell +$ cat <<EOF | kind create cluster --config=- +kind: Cluster +apiVersion: kind.x-k8s.io/v1alpha4 +nodes: +- role: control-plane + image: kindest/node:v1.21.10 +- role: worker + image: kindest/node:v1.21.10 +EOF +``` + +<details> + <summary>Expected output</summary> + +```shell +Creating cluster "kind" ... + â Ensuring node image (kindest/node:v1.21.10) đŧ + â Preparing nodes đĻ đĻ + â Writing configuration đ + â Starting control-plane đšī¸ + â Installing CNI đ + â Installing StorageClass đž + â Joining worker nodes đ +Set kubectl context to "kind-kind" +You can now use your cluster with: + +kubectl cluster-info --context kind-kind + +Not sure what to do next? đ Check out https://kind.sigs.k8s.io/docs/user/quick-start/ +``` +</details> + +Check all pods in the cluster. + +```shell +$ kubectl get pods -A +``` + +<details> + <summary>Expected output</summary> + +```shell +NAMESPACE NAME READY STATUS RESTARTS AGE +kube-system coredns-558bd4d5db-h5gxt 1/1 Running 0 106s +kube-system coredns-558bd4d5db-lhnvz 1/1 Running 0 106s +kube-system etcd-kind-control-plane 1/1 Running 0 116s +kube-system kindnet-fxlkm 1/1 Running 0 106s +kube-system kindnet-vmcvl 1/1 Running 0 91s +kube-system kube-apiserver-kind-control-plane 1/1 Running 0 116s +kube-system kube-controller-manager-kind-control-plane 1/1 Running 0 116s +kube-system kube-proxy-nr4f4 1/1 Running 0 91s +kube-system kube-proxy-zl4h2 1/1 Running 0 106s +kube-system kube-scheduler-kind-control-plane 1/1 Running 0 116s +local-path-storage local-path-provisioner-74567d47b4-kmtjh 1/1 Running 0 106s +``` +</details> + +### Step2: Build the operator image + +Check into the root directory of SWCK and build the operator image as follows. + +```shell +$ cd operator +# Build the operator image +$ make docker-build +``` + +You will get the operator image `controller:latest` as follows. + +```shell +$ docker images +REPOSITORY TAG IMAGE ID CREATED SIZE +controller latest 84da7509092a 22 seconds ago 53.6MB +``` + +Load the operator image into the kind cluster or push the image to a registry that +your kubernetes cluster can access. + +```shell +$ kind load docker-image controller +``` +or +```shell +$ docker push $(YOUR_REGISTRY)/controller +``` + +### Step3: Deploy operator on the kubernetes cluster + +Install the CRDs as follows. + +```shell +$ make install +``` + +Check the CRDs are installed successfully. + +<details> + <summary>Expected output</summary> + +```shell +kubectl get crd | grep skywalking +banyandbs.operator.skywalking.apache.org 2023-11-05T03:30:43Z +fetchers.operator.skywalking.apache.org 2023-11-05T03:30:43Z +javaagents.operator.skywalking.apache.org 2023-11-05T03:30:43Z +oapserverconfigs.operator.skywalking.apache.org 2023-11-05T03:30:43Z +oapserverdynamicconfigs.operator.skywalking.apache.org 2023-11-05T03:30:43Z +oapservers.operator.skywalking.apache.org 2023-11-05T03:30:43Z +satellites.operator.skywalking.apache.org 2023-11-05T03:30:43Z +storages.operator.skywalking.apache.org 2023-11-05T03:30:43Z +swagents.operator.skywalking.apache.org 2023-11-05T03:30:43Z +uis.operator.skywalking.apache.org 2023-11-05T03:30:43Z +``` +</details> + +Deploy the SWCK operator to the cluster. + +```shell +$ make deploy +``` + +Or deploy the SWCK operator to the cluster with your own image. + +```shell +$ make deploy OPERATOR_IMG=$(YOUR_REGISTRY)/controller +``` + +Get the status of the SWCK operator pod. + +```shell +$ kubectl get pod -n skywalking-swck-system +NAME READY STATUS RESTARTS AGE +skywalking-swck-controller-manager-5f5bbd4fd-9wdw6 2/2 Running 0 34s +``` + +### Step4: Deploy skywalking componentes on the kubernetes cluster + +Create the `skywalking-system` namespace. + +```shell +$ kubectl create namespace skywalking-system +``` + +Deploy the skywalking components to the cluster. + +```shell +$ cat <<EOF | kubectl apply -f - +apiVersion: operator.skywalking.apache.org/v1alpha1 +kind: OAPServer +metadata: + name: skywalking-system + namespace: skywalking-system +spec: + version: 9.5.0 + instances: 1 + image: apache/skywalking-oap-server:9.5.0 + service: + template: + type: ClusterIP +--- +apiVersion: operator.skywalking.apache.org/v1alpha1 +kind: UI +metadata: + name: skywalking-system + namespace: skywalking-system +spec: + version: 9.5.0 + instances: 1 + image: apache/skywalking-ui:9.5.0 + OAPServerAddress: http://skywalking-system-oap.skywalking-system:12800 + service: + template: + type: ClusterIP + ingress: + host: demo.ui.skywalking +EOF +``` + +Check the status of the skywalking components. + +```shell +$ kubectl get pod -n skywalking-system +NAME READY STATUS RESTARTS AGE +skywalking-system-oap-68bd877f57-fhzdz 1/1 Running 0 6m23s +skywalking-system-ui-6db8579b47-rphtl 1/1 Running 0 6m23s +``` + +### Step5: Use the java agent injector to inject the java agent into the application pod + +Label the namespace where the application pod is located with `swck-injection=enabled`. + +```shell +$ kubectl label namespace skywalking-system swck-injection=enabled +``` + +Create the application pod. + +> Note: The application pod must be labeled with `swck-java-agent-injected=true` and the `agent.skywalking.apache.org/collector.backend_service` annotation must be set to the address of the OAP server. For more configurations, please refer to the [guide](./java-agent-injector.md#use-annotations-to-overlay-default-agent-configuration). + +```shell +$ cat <<EOF | kubectl apply -f - +apiVersion: apps/v1 +kind: Deployment +metadata: + name: demo + namespace: skywalking-system +spec: + selector: + matchLabels: + app: demo + template: + metadata: + labels: + # enable the java agent injector + swck-java-agent-injected: "true" + app: demo + annotations: + agent.skywalking.apache.org/collector.backend_service: "skywalking-system-oap.skywalking-system:11800" + spec: + containers: + - name: demo1 + imagePullPolicy: IfNotPresent + image: ghcr.io/apache/skywalking-swck-spring-demo:v0.0.1 + command: ["java"] + args: ["-jar","/app.jar"] + ports: + - containerPort: 8085 + readinessProbe: + httpGet: + path: /hello + port: 8085 + initialDelaySeconds: 3 + periodSeconds: 3 + failureThreshold: 10 +--- +apiVersion: v1 +kind: Service +metadata: + name: demo + namespace: skywalking-system +spec: + type: ClusterIP + ports: + - name: 8085-tcp + port: 8085 + protocol: TCP + targetPort: 8085 + selector: + app: demo +EOF +``` + +Check the status of the application pod and make +sure the java agent is injected into the application pod. + + +```shell +$ kubectl get pod -n skywalking-system -l app=demo -ojsonpath='{.items[0].spec.initContainers[0]}' +``` + +<details> + <summary>Expected output</summary> + +```shell +{"args":["-c","mkdir -p /sky/agent \u0026\u0026 cp -r /skywalking/agent/* /sky/agent"],"command":["sh"],"image":"apache/skywalking-java-agent:8.16.0-java8","imagePullPolicy":"IfNotPresent","name":"inject-skywalking-agent","resources":{},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","volumeMounts":[{"mountPath":"/sky/agent","name":"sky-agent"},{"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount","name":"kube-api-access-4qk26","readOnly":true}]} +``` +</details> + +Also, you could check the final java agent configurations with the following command. + +```shell +$ kubectl get javaagent -n skywalking-system -l app=demo -oyaml +``` + +<details> + <summary>Expected output</summary> + +```shell +apiVersion: v1 +items: +- apiVersion: operator.skywalking.apache.org/v1alpha1 + kind: JavaAgent + metadata: + creationTimestamp: "2023-11-19T05:34:03Z" + generation: 1 + labels: + app: demo + name: app-demo-javaagent + namespace: skywalking-system + ownerReferences: + - apiVersion: apps/v1 + blockOwnerDeletion: true + controller: true + kind: ReplicaSet + name: demo-75d8d995cc + uid: 8cb64abc-9b50-4f67-9304-2e09de476168 + resourceVersion: "21515" + uid: 6cbafb3d-9f43-4448-95e8-bda1f7c72bc3 + spec: + agentConfiguration: + collector.backend_service: skywalking-system-oap.skywalking-system:11800 + optional-plugin: webflux|cloud-gateway-2.1.x + backendService: skywalking-system-oap.skywalking-system:11800 + podSelector: app=demo + serviceName: Your_ApplicationName + status: + creationTime: "2023-11-19T05:34:03Z" + expectedInjectiedNum: 1 + lastUpdateTime: "2023-11-19T05:34:46Z" + realInjectedNum: 1 +kind: List +metadata: + resourceVersion: "" + selfLink: "" +``` +</details> + +If you want to check the logs of the java agent, you can run the following command. + +```shell +$ kubectl logs -f -n skywalking-system -l app=demo -c inject-skywalking-agent +``` + + +### Step6: Check the application metrics in the skywalking UI + +First, port-forward the demo service to your local machine. + +```shell +$ kubectl port-forward svc/demo 8085:8085 -n skywalking-system +``` + +Then, trigger the application to generate some metrics. + +```shell +$ for i in {1..10}; do curl http://127.0.0.1:8085/hello && echo ""; done +``` + +After that, you can port-forward the skywalking UI to your local machine. + +```shell +$ kubectl port-forward svc/skywalking-system-ui 8080:80 -n skywalking-system +``` + +Open the skywalking UI in your browser and navigate to `http://127.0.0.1:8080` to check the application metrics. + +<details> + <summary>Expected output</summary> + +![ui](https://skywalking.apache.org/doc-graph/swck/demo-ui.png) +</details> + + +Also, if you want to expose the external metrics to the kubernetes HPA, you can follow the [guide](./custom-metrics-adapter.md) to deploy the custom metrics adapter and you may get some inspiration from the +[e2e test](../test/e2e/oap-agent-adapter-hpa/e2e.yaml). \ No newline at end of file diff --git a/operator/controllers/operator/javaagent_controller.go b/operator/controllers/operator/javaagent_controller.go index b4a937e..7fbd43d 100644 --- a/operator/controllers/operator/javaagent_controller.go +++ b/operator/controllers/operator/javaagent_controller.go @@ -98,6 +98,7 @@ func (r *JavaAgentReconciler) Reconcile(ctx context.Context, req ctrl.Request) ( } config := map[string]string{} r.injectConfigBySwAgent(lastMatchedSwAgent, config) + injector.GetInjectedAgentConfig(&pod.Annotations, &config) // only get the first selector label from labels as podselector labels := pod.Labels @@ -124,6 +125,12 @@ func (r *JavaAgentReconciler) Reconcile(ctx context.Context, req ctrl.Request) ( "config": func() map[string]string { return config }, + "labelKey": func() string { + return keys[0] + }, + "labelValue": func() string { + return labels[keys[0]] + }, "ownerReference": func() metav1.OwnerReference { return ownerReference }, diff --git a/operator/pkg/operator/manifests/injector/templates/javaagent.yaml b/operator/pkg/operator/manifests/injector/templates/javaagent.yaml index db8b455..b516a27 100644 --- a/operator/pkg/operator/manifests/injector/templates/javaagent.yaml +++ b/operator/pkg/operator/manifests/injector/templates/javaagent.yaml @@ -21,6 +21,8 @@ kind: JavaAgent metadata: name: {{SelectorName}}-javaagent namespace: {{Namespace}} + labels: + {{labelKey}}: {{labelValue}} ownerReferences: - apiVersion: {{ownerReference.APIVersion}} blockOwnerDeletion: {{ownerReference.BlockOwnerDeletion}} diff --git a/test/e2e/skywalking-components.yaml b/test/e2e/skywalking-components.yaml index 6021ecf..2e9b1d0 100644 --- a/test/e2e/skywalking-components.yaml +++ b/test/e2e/skywalking-components.yaml @@ -27,7 +27,6 @@ spec: service: template: type: ClusterIP - --- apiVersion: operator.skywalking.apache.org/v1alpha1 kind: UI