This is an automated email from the ASF dual-hosted git repository. dashanji pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/skywalking-swck.git
The following commit(s) were added to refs/heads/master by this push: new bc0e7cc Support inject java agent bootstrap-plugins. (#91) bc0e7cc is described below commit bc0e7cc64f444972de58fe31d173d3c6b76e8f43 Author: weixiang1862 <652048...@qq.com> AuthorDate: Thu Aug 31 16:03:14 2023 +0800 Support inject java agent bootstrap-plugins. (#91) * Support inject java agent bootstrap-plugins. * Add bootstrap-plugin inject e2e case. Co-authored-by: weixiang1862 <weixiang1...@gmail.com> --- docs/java-agent-injector.md | 27 +++++++++++++------ operator/apis/operator/v1alpha1/swagent_types.go | 4 +++ .../operator/v1alpha1/zz_generated.deepcopy.go | 5 ++++ .../operator.skywalking.apache.org_swagents.yaml | 6 +++++ operator/pkg/operator/injector/injector.go | 31 ++++++++++++++++------ test/e2e/demo.yaml | 2 ++ test/e2e/oap-ui-agent/e2e.yaml | 8 ++++++ test/e2e/oap-ui-swagent/e2e.yaml | 4 +++ test/e2e/skywalking-components-with-swagent.yaml | 8 +++++- test/e2e/verify/bootstrap-plugin.yaml | 19 +++++++++++++ 10 files changed, 97 insertions(+), 17 deletions(-) diff --git a/docs/java-agent-injector.md b/docs/java-agent-injector.md index 35b8aed..282e0a4 100644 --- a/docs/java-agent-injector.md +++ b/docs/java-agent-injector.md @@ -144,6 +144,8 @@ spec: optionalPlugins: - "webflux" - "cloud-gateway-2.1.x" + bootstrapPlugins: + - "jdk-threading" ``` There are three kind of configs in SwAgent CR. @@ -162,14 +164,15 @@ label selector and container matcher decides which pod and container should be i injection configuration will affect on agent injection behaviour -| key path | description | default value | -|------------------------|-------------------------------------------------------------------------------------------------------------------|------------------------------------------| -| javaSidecar | javaSidecar is the configs for init container, which holds agent sdk and take agent sdk to the target containers. | | -| javaSidecar.name | the name of the init container. | inject-skywalking-agent | -| javaSidecar.image | the image of the init container. | apache/skywalking-java-agent:8.16.0-java8 | -| SharedVolumeName | SharedVolume is the name of an empty volume which shared by initContainer and target containers. | sky-agent | -| OptionalPlugins | Select the optional plugin which needs to be moved to the directory(/plugins). Such as `trace`,`webflux`,`cloud-gateway-2.1.x`. | no default value | -| OptionalReporterPlugins | Select the optional reporter plugin which needs to be moved to the directory(/plugins). such as `kafka`. | no default value | +| key path | description | default value | +|-------------------------|---------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------| +| javaSidecar | javaSidecar is the configs for init container, which holds agent sdk and take agent sdk to the target containers. | | +| javaSidecar.name | the name of the init container. | inject-skywalking-agent | +| javaSidecar.image | the image of the init container. | apache/skywalking-java-agent:8.16.0-java8 | +| SharedVolumeName | SharedVolume is the name of an empty volume which shared by initContainer and target containers. | sky-agent | +| OptionalPlugins | Select the optional plugin which needs to be moved to the directory(/plugins). Such as `trace`,`webflux`,`cloud-gateway-2.1.x`. | no default value | +| OptionalReporterPlugins | Select the optional reporter plugin which needs to be moved to the directory(/plugins). such as `kafka`. | no default value | +| BootstrapPlugins | Select the bootstrap plugin which needs to be moved to the directory(/plugins). such as `jdk-threading`. | no default value | #### 3. skywalking agent configuration @@ -216,6 +219,14 @@ The optional reporter plugin configuration is the annotation as below. | ----------------------------------------- | ------------------------------------------------------------ | ---------------- | | `optional-reporter.skywalking.apache.org` | Select the optional reporter plugin which needs to be moved to the directory(/plugins). Users can select several optional reporter plugins by separating from `|`, such as `kafka`. | not set | +#### 6. bootstrap plugin configuration + +The bootstrap plugin configuration is the annotation as below. + +| Annotation key | Description | Annotation value | +|-----------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------| +| `bootstrap.skywalking.apache.org` | Select the bootstrap plugin which needs to be moved to the directory(/plugins). Users can select several bootstrap plugins by separating from `|`, such as `jdk-threading`. | not set | + ## Configure sidecar The injector can recognize the following annotations to configure the sidecar: diff --git a/operator/apis/operator/v1alpha1/swagent_types.go b/operator/apis/operator/v1alpha1/swagent_types.go index 01d4c3e..ed3917c 100644 --- a/operator/apis/operator/v1alpha1/swagent_types.go +++ b/operator/apis/operator/v1alpha1/swagent_types.go @@ -47,6 +47,10 @@ type SwAgentSpec struct { // Select the optional reporter plugin which needs to be moved to the directory(/plugins). such as `kafka`. // +kubebuilder:validation:Optional OptionalReporterPlugins []string `json:"optionalReporterPlugins,omitempty"` + + // Select the bootstrap plugin which needs to be moved to the directory(/plugins). such as `jdk-threading`. + // +kubebuilder:validation:Optional + BootstrapPlugins []string `json:"bootstrapPlugins,omitempty"` } // Java defines Java agent special configs. diff --git a/operator/apis/operator/v1alpha1/zz_generated.deepcopy.go b/operator/apis/operator/v1alpha1/zz_generated.deepcopy.go index 09e6019..627c68a 100644 --- a/operator/apis/operator/v1alpha1/zz_generated.deepcopy.go +++ b/operator/apis/operator/v1alpha1/zz_generated.deepcopy.go @@ -1166,6 +1166,11 @@ func (in *SwAgentSpec) DeepCopyInto(out *SwAgentSpec) { *out = make([]string, len(*in)) copy(*out, *in) } + if in.BootstrapPlugins != nil { + in, out := &in.BootstrapPlugins, &out.BootstrapPlugins + *out = make([]string, len(*in)) + copy(*out, *in) + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SwAgentSpec. diff --git a/operator/config/crd/bases/operator.skywalking.apache.org_swagents.yaml b/operator/config/crd/bases/operator.skywalking.apache.org_swagents.yaml index 2f682aa..a061a4a 100644 --- a/operator/config/crd/bases/operator.skywalking.apache.org_swagents.yaml +++ b/operator/config/crd/bases/operator.skywalking.apache.org_swagents.yaml @@ -52,6 +52,12 @@ spec: spec: description: SwAgentSpec defines the desired state of SwAgent properties: + bootstrapPlugins: + description: Select the bootstrap plugin which needs to be moved to + the directory(/plugins). such as `jdk-threading`. + items: + type: string + type: array containerMatcher: description: ContainerMatcher is the regular expression to match pods which is to be injected. diff --git a/operator/pkg/operator/injector/injector.go b/operator/pkg/operator/injector/injector.go index be17c3f..e2823cd 100644 --- a/operator/pkg/operator/injector/injector.go +++ b/operator/pkg/operator/injector/injector.go @@ -49,12 +49,15 @@ const ( pluginsAnnotationPrefix = "plugins.skywalking.apache.org/" // If user want to use optional-plugins , the annotation must match a optinal plugin // such as optional.skywalking.apache.org: "trace|webflux|cloud-gateway-2.1.x" - // Notice , If the injected container's image don't has the optional plugin , + // Notice , If the injected container's image don't hava the optional plugin , // the container will panic optionsAnnotation = "optional.skywalking.apache.org" - // If user want to use optional-reporter-plugins , the annotation must match a optinal-reporter plugin - // such as optional-exporter.skywalking.apache.org: "kafka" + // If user want to use optional-reporter-plugins , the annotation must match an optional-reporter plugin + // such as optional-reporter.skywalking.apache.org: "kafka" optionsReporterAnnotation = "optional-reporter.skywalking.apache.org" + // If user want to use bootstrap-plugins , the annotation must match a bootstrap plugin + // such as bootstrap.skywalking.apache.org: "jdk-threading" + bootstrapAnnotation = "bootstrap.skywalking.apache.org" // the mount path of empty volume, which shared by init container and target container. mountPath = "/sky/agent" ) @@ -377,11 +380,13 @@ func (s *SidecarInjectField) OverlayAgent(a Annotations, ao *AnnotationOverlay, func (s *SidecarInjectField) OverlayOptional(swAgentL []v1alpha1.SwAgent, annotation *map[string]string) { sourceOptionalPath := strings.Join([]string{s.SidecarVolumeMount.MountPath, "optional-plugins/"}, "/") sourceOptionalReporterPath := strings.Join([]string{s.SidecarVolumeMount.MountPath, "optional-reporter-plugins/"}, "/") + sourceBootstrapPath := strings.Join([]string{s.SidecarVolumeMount.MountPath, "bootstrap-plugins/"}, "/") targetPath := strings.Join([]string{s.SidecarVolumeMount.MountPath, "plugins/"}, "/") command := "" optionalPlugin := "" - optinalReporterPlugins := "" + optionalReporterPlugins := "" + bootstrapPlugins := "" // chose the last matched SwAgent if len(swAgentL) > 0 { @@ -391,7 +396,10 @@ func (s *SidecarInjectField) OverlayOptional(swAgentL []v1alpha1.SwAgent, annota optionalPlugin = strings.Join(swAgent.Spec.OptionalPlugins, "|") } if len(swAgent.Spec.OptionalReporterPlugins) > 0 { - optinalReporterPlugins = strings.Join(swAgent.Spec.OptionalReporterPlugins, "|") + optionalReporterPlugins = strings.Join(swAgent.Spec.OptionalReporterPlugins, "|") + } + if len(swAgent.Spec.BootstrapPlugins) > 0 { + bootstrapPlugins = strings.Join(swAgent.Spec.BootstrapPlugins, "|") } } @@ -399,7 +407,9 @@ func (s *SidecarInjectField) OverlayOptional(swAgentL []v1alpha1.SwAgent, annota if strings.EqualFold(k, optionsAnnotation) { optionalPlugin = v } else if strings.EqualFold(k, optionsReporterAnnotation) { - optinalReporterPlugins = v + optionalReporterPlugins = v + } else if strings.EqualFold(k, bootstrapAnnotation) { + bootstrapPlugins = v } if command != "" { s.Initcontainer.Args[1] = strings.Join([]string{s.Initcontainer.Args[1], command}, " && ") @@ -411,8 +421,13 @@ func (s *SidecarInjectField) OverlayOptional(swAgentL []v1alpha1.SwAgent, annota s.Initcontainer.Args[1] = strings.Join([]string{s.Initcontainer.Args[1], command}, " && ") } - if len(optinalReporterPlugins) > 0 { - command = "cd " + sourceOptionalReporterPath + " && ls | grep -E \"" + optinalReporterPlugins + "\" | xargs -i cp {} " + targetPath + if len(optionalReporterPlugins) > 0 { + command = "cd " + sourceOptionalReporterPath + " && ls | grep -E \"" + optionalReporterPlugins + "\" | xargs -i cp {} " + targetPath + s.Initcontainer.Args[1] = strings.Join([]string{s.Initcontainer.Args[1], command}, " && ") + } + + if len(bootstrapPlugins) > 0 { + command = "cd " + sourceBootstrapPath + " && ls | grep -E \"" + bootstrapPlugins + "\" | xargs -i cp {} " + targetPath s.Initcontainer.Args[1] = strings.Join([]string{s.Initcontainer.Args[1], command}, " && ") } diff --git a/test/e2e/demo.yaml b/test/e2e/demo.yaml index 52b82cb..37cf1ce 100644 --- a/test/e2e/demo.yaml +++ b/test/e2e/demo.yaml @@ -31,6 +31,8 @@ spec: sidecar.skywalking.apache.org/initcontainer.resources.limits: '{"memory": "100Mi","cpu": "100m"}' sidecar.skywalking.apache.org/initcontainer.resources.requests: '{"memory": "100Mi","cpu": "100m"}' agent.skywalking.apache.org/collector.backend_service: "oap-service:11800" + optional.skywalking.apache.org: "webflux|cloud-gateway-2.1.x" + bootstrap.skywalking.apache.org: "jdk-http|jdk-threading" spec: containers: - name: demo1 diff --git a/test/e2e/oap-ui-agent/e2e.yaml b/test/e2e/oap-ui-agent/e2e.yaml index fe489c2..8eaee57 100644 --- a/test/e2e/oap-ui-agent/e2e.yaml +++ b/test/e2e/oap-ui-agent/e2e.yaml @@ -99,5 +99,13 @@ verify: expected: ../verify/endpoint.yaml - query: swctl --display yaml --base-url=http://${service_skywalking_system_oap_host}:${service_skywalking_system_oap_12800}/graphql metrics linear --name endpoint_cpm --endpoint-name GET:/hello --service-name Your_ApplicationName | yq e 'to_entries' - expected: ../verify/metrics.yaml + - query: 'kubectl logs -l app=demo -n skywalking-system --tail=-1 | grep -Eo "AgentClassLoader.*webflux.*\.jar loaded" | grep -Eo "/sky/agent/plugins/.*webflux.*\.jar" | jq -R -s -c "split(\"\n\")[:-1]" | yq e ''to_entries''' + expected: ../verify/optional-plugin.yaml + - query: 'kubectl logs -l app=demo -n skywalking-system --tail=-1 | grep -Eo "AgentClassLoader.*cloud-gateway-2\.1\.x.*\.jar loaded" | grep -Eo "/sky/agent/plugins/.*cloud-gateway-2\.1\.x.*\.jar" | jq -R -s -c "split(\"\n\")[:-1]" | yq e ''to_entries''' + expected: ../verify/optional-plugin.yaml + - query: 'kubectl logs -l app=demo -n skywalking-system --tail=-1 | grep -Eo "AgentClassLoader.*jdk-http.*\.jar loaded" | grep -Eo "/sky/agent/plugins/.*jdk-http.*\.jar" | jq -R -s -c "split(\"\n\")[:-1]" | yq e ''to_entries''' + expected: ../verify/bootstrap-plugin.yaml + - query: 'kubectl logs -l app=demo -n skywalking-system --tail=-1 | grep -Eo "AgentClassLoader.*jdk-threading.*\.jar loaded" | grep -Eo "/sky/agent/plugins/.*jdk-threading.*\.jar" | jq -R -s -c "split(\"\n\")[:-1]" | yq e ''to_entries''' + expected: ../verify/bootstrap-plugin.yaml - includes: - ../ui-cases.yaml \ No newline at end of file diff --git a/test/e2e/oap-ui-swagent/e2e.yaml b/test/e2e/oap-ui-swagent/e2e.yaml index b9a492f..7bc1802 100644 --- a/test/e2e/oap-ui-swagent/e2e.yaml +++ b/test/e2e/oap-ui-swagent/e2e.yaml @@ -109,6 +109,10 @@ verify: expected: ../verify/optional-plugin.yaml - query: 'kubectl logs -l app=demo -n skywalking-system --tail=-1 | grep -Eo "AgentClassLoader.*cloud-gateway-2\.1\.x.*\.jar loaded" | grep -Eo "/sky/agent/plugins/.*cloud-gateway-2\.1\.x.*\.jar" | jq -R -s -c "split(\"\n\")[:-1]" | yq e ''to_entries''' expected: ../verify/optional-plugin.yaml + - query: 'kubectl logs -l app=demo -n skywalking-system --tail=-1 | grep -Eo "AgentClassLoader.*jdk-http.*\.jar loaded" | grep -Eo "/sky/agent/plugins/.*jdk-http.*\.jar" | jq -R -s -c "split(\"\n\")[:-1]" | yq e ''to_entries''' + expected: ../verify/bootstrap-plugin.yaml + - query: 'kubectl logs -l app=demo -n skywalking-system --tail=-1 | grep -Eo "AgentClassLoader.*jdk-threading.*\.jar loaded" | grep -Eo "/sky/agent/plugins/.*jdk-threading.*\.jar" | jq -R -s -c "split(\"\n\")[:-1]" | yq e ''to_entries''' + expected: ../verify/bootstrap-plugin.yaml - query: swctl --display yaml --base-url=http://${service_skywalking_system_ui_host}:${service_skywalking_system_ui_80}/graphql service ls expected: ../verify/service.yaml - query: swctl --display yaml --base-url=http://${service_skywalking_system_ui_host}:${service_skywalking_system_ui_80}/graphql endpoint list --keyword=hello --service-name Your_ApplicationName diff --git a/test/e2e/skywalking-components-with-swagent.yaml b/test/e2e/skywalking-components-with-swagent.yaml index e3f3337..0f9aeef 100644 --- a/test/e2e/skywalking-components-with-swagent.yaml +++ b/test/e2e/skywalking-components-with-swagent.yaml @@ -66,6 +66,9 @@ spec: optionalPlugins: - "webflux" - "cloud-gateway-2.1.x" + bootstrapPlugins: + - "jdk-http" + - "jdk-threading" --- apiVersion: operator.skywalking.apache.org/v1alpha1 @@ -87,4 +90,7 @@ spec: sharedVolumeName: "sky-agent-test-002" optionalPlugins: - "webflux" - - "cloud-gateway-2.1.x" \ No newline at end of file + - "cloud-gateway-2.1.x" + bootstrapPlugins: + - "jdk-http" + - "jdk-threading" \ No newline at end of file diff --git a/test/e2e/verify/bootstrap-plugin.yaml b/test/e2e/verify/bootstrap-plugin.yaml new file mode 100644 index 0000000..d23ccee --- /dev/null +++ b/test/e2e/verify/bootstrap-plugin.yaml @@ -0,0 +1,19 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +{{- contains . }} +- key: {{ ge .key 0 }} + value: {{ notEmpty .value }} +{{- end }} \ No newline at end of file