Re: [I] [Bug] skywalking-showcase docker deployment has some incorrect parts [skywalking]
kezhenxu94 commented on issue #12077: URL: https://github.com/apache/skywalking/issues/12077#issuecomment-2041994460 > 1. rocketmq-feature seems incorrect: > > https://private-user-images.githubusercontent.com/74546965/320372769-453f26fe-2486-4fa0-92f3-efe43e21599d.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTI1NTkyNTksIm5iZiI6MTcxMjU1ODk1OSwicGF0aCI6Ii83NDU0Njk2NS8zMjAzNzI3NjktNDUzZjI2ZmUtMjQ4Ni00ZmEwLTkyZjMtZWZlNDNlMjE1OTlkLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA0MDglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNDA4VDA2NDkxOVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTVjM2IzOWJjNmFhZWUyYWRlN2E4YTJlZmM1YmQ0NDU0MGQ2OTE4NGRhMjk4Y2I3NzQxMzQwNzJhMTg0MjdlM2QmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.lquFeeCdN7A5JyD69XwqxEIW-BC6ZnF-uNx7Ukeo_Oc";> I believe that a feature should utilize a common OAP(such as single-node) instead of spinning up a new one. Additionally, there are issues with the configuration of this OAP. Nice catch! I think we can fix this by simply removing the OAP section in this docker compose file. > 2. whether we need to expose this port? > > https://private-user-images.githubusercontent.com/74546965/320374139-9090f8f7-aeda-4b8e-9cec-87f22c302de3.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTI1NTkyNTksIm5iZiI6MTcxMjU1ODk1OSwicGF0aCI6Ii83NDU0Njk2NS8zMjAzNzQxMzktOTA5MGY4ZjctYWVkYS00YjhlLTljZWMtODdmMjJjMzAyZGUzLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA0MDglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNDA4VDA2NDkxOVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTkwN2FjYjNhNDkxOGJkM2RiMjQ0ZmU5MjEyOThiM2M3NDg1ZmQ2NjdkZDYzYThmYjFjNWU3ZDE4ZTM0OWEyZTQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.L4B8mNfInUIeT3AXJJJAjXCPflqEzUvEVw4sTdEBDvM";> This is not mandatory as we have a loadgen to trigger traffic, however, if users want to manually trigger traffic, they can open this port in their browser. > 3. oap dependency > > https://private-user-images.githubusercontent.com/74546965/320374564-974c244e-d217-434f-b9f0-bf8a3b1c5915.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTI1NTkyNTksIm5iZiI6MTcxMjU1ODk1OSwicGF0aCI6Ii83NDU0Njk2NS8zMjAzNzQ1NjQtOTc0YzI0NGUtZDIxNy00MzRmLWI5ZjAtYmY4YTNiMWM1OTE1LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA0MDglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNDA4VDA2NDkxOVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWUyYmM5YjMxODliMDdlZWMwNjFiNmYwZDFkYzU4YmZmNTRjZGQwNjAyZGNlY2MzY2E1ZjIyMTc0OTk0OWEzN2ImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.sMEVeL07fpF7ged9QCTAQbV84xJDdtCoQwBFioGvtwI";> Some feature depend on oap. However, in some cases, we only need these services and do not require the built-in OAP. Can you elaborate? OAP is required to be up and running for those components that need to talk to OAP, in your screenshot, the vm exporter needs to send data to OAP so it needs to wait for OAP's readiness. > and also, some config files use oap as host: https://private-user-images.githubusercontent.com/74546965/320376116-8cd4c623-9394-482b-8755-3ddbc130c116.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTI1NTkyNTksIm5iZiI6MTcxMjU1ODk1OSwicGF0aCI6Ii83NDU0Njk2NS8zMjAzNzYxMTYtOGNkNGM2MjMtOTM5NC00ODJiLTg3NTUtM2RkYmMxMzBjMTE2LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA0MDglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNDA4VDA2NDkxOVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTY4MDM2NzY0N2JjMzc5ODkxNTE1YWNjMGNmYjI2NmMxYjRhZDM2OWM4YTY5NDlkYjhjMmEyZjhjZTZmOWIyMjcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.WsRYZXCM1anjGoGiBEoJNxbksBO11I3hEqhWz3KnZcI";> so we can't easily edit Makefile to change oap address. Good catch, this can be replace with the env var `BACKEND_SERVICE`. Will you file a pull request @yswdqz ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] skywalking-showcase docker deployment has some incorrect parts [skywalking]
wu-sheng commented on issue #12077: URL: https://github.com/apache/skywalking/issues/12077#issuecomment-2041975023 @kezhenxu94 Could you guide @yswdqz about this? We may need to polish showcase. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Bug] skywalking-showcase docker deployment has some incorrect parts [skywalking]
yswdqz opened a new issue, #12077: URL: https://github.com/apache/skywalking/issues/12077 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/skywalking/issues?q=is%3Aissue) and found no similar issues. ### Apache SkyWalking Component Showcase (apache/skywalking-showcase) ### What happened 1. rocketmq-feature seems incorrect: https://github.com/apache/skywalking/assets/74546965/453f26fe-2486-4fa0-92f3-efe43e21599d";> I believe that a feature should utilize a common OAP(such as single-node) instead of spinning up a new one. Additionally, there are issues with the configuration of this OAP. 2. whether we need to expose this port? https://github.com/apache/skywalking/assets/74546965/9090f8f7-aeda-4b8e-9cec-87f22c302de3";> 3. oap dependency https://github.com/apache/skywalking/assets/74546965/974c244e-d217-434f-b9f0-bf8a3b1c5915";> Some feature depend on oap. However, in some cases, we only need these services and do not require the built-in OAP. and also, some config files use oap as host: https://github.com/apache/skywalking/assets/74546965/8cd4c623-9394-482b-8755-3ddbc130c116";> so we can't easily edit Makefile to change oap address. https://github.com/apache/skywalking/assets/74546965/2ad3db87-bf2d-4b04-9b9e-5defcad7cf68";> ### What you expected to happen We can easily edit Makefile to change OAP localhost, fix rocketmq feature. ### How to reproduce None ### Anything else _No response_ ### Are you willing to submit a pull request to fix on your own? - [ ] Yes I am willing to submit a pull request on my own! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] build(deps): bump @adobe/css-tools from 4.3.1 to 4.3.2 [skywalking-grafana-plugins]
dependabot[bot] commented on PR #31: URL: https://github.com/apache/skywalking-grafana-plugins/pull/31#issuecomment-2041769153 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting `@dependabot ignore this major version` or `@dependabot ignore this minor version`. If you change your mind, just re-open this PR and I'll resolve any conflicts on it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(skywalking-grafana-plugins) branch dependabot/npm_and_yarn/adobe/css-tools-4.3.2 deleted (was 10f751e)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/npm_and_yarn/adobe/css-tools-4.3.2 in repository https://gitbox.apache.org/repos/asf/skywalking-grafana-plugins.git was 10f751e build(deps): bump @adobe/css-tools from 4.3.1 to 4.3.2 The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
Re: [PR] build(deps): bump @adobe/css-tools from 4.3.1 to 4.3.2 [skywalking-grafana-plugins]
wu-sheng closed pull request #31: build(deps): bump @adobe/css-tools from 4.3.1 to 4.3.2 URL: https://github.com/apache/skywalking-grafana-plugins/pull/31 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(skywalking-banyandb) branch clean-up deleted (was 46fff5f6)
This is an automated email from the ASF dual-hosted git repository. wusheng pushed a change to branch clean-up in repository https://gitbox.apache.org/repos/asf/skywalking-banyandb.git was 46fff5f6 Remove wal and fix e2e The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
Re: [PR] Clean up redundant API [skywalking-banyandb]
wu-sheng merged PR #429: URL: https://github.com/apache/skywalking-banyandb/pull/429 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(skywalking-banyandb) branch clean-up updated (ba8bf559 -> 46fff5f6)
This is an automated email from the ASF dual-hosted git repository. hanahmily pushed a change to branch clean-up in repository https://gitbox.apache.org/repos/asf/skywalking-banyandb.git omit ba8bf559 Remove wal and fix e2e add 46fff5f6 Remove wal and fix e2e This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (ba8bf559) \ N -- N -- N refs/heads/clean-up (46fff5f6) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: test/stress/classic/env | 2 +- test/stress/classic/env.dev | 2 +- ui/src/components/Aside/index.vue | 24 3 files changed, 2 insertions(+), 26 deletions(-)
[PR] Clean up redundant API [skywalking-banyandb]
hanahmily opened a new pull request, #429: URL: https://github.com/apache/skywalking-banyandb/pull/429 - [x] Update the documentation to include this new feature. - [x] Tests(including UT, IT, E2E) are added to verify the new feature. - [x] If it's UI related, attach the screenshots below. - [x] Update the [`CHANGES` log](https://github.com/apache/skywalking-banyandb/blob/main/CHANGES.md). ![WX20240408-085127](https://github.com/apache/skywalking-banyandb/assets/1960537/f68a8706-069f-4dac-8d17-6946e8eb0915) ![1712537560597](https://github.com/apache/skywalking-banyandb/assets/1960537/0cc47df2-912e-45da-99d5-d8e31008e41d) ![WX20240408-085343](https://github.com/apache/skywalking-banyandb/assets/1960537/628cdc5c-0de7-4a28-b88b-72e9b3aaaf05) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Add Apache Airflow Monitoring [skywalking]
wu-sheng commented on code in PR #12072: URL: https://github.com/apache/skywalking/pull/12072#discussion_r1555122490 ## test/e2e-v2/cases/airflow/airflow-exporter/otel-collector-config.yaml: ## @@ -0,0 +1,69 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +receivers: + otlp: +protocols: + http: +endpoint: "0.0.0.0:8889" # Receiver configuration using the OTLP protocol over HTTP, listening Airflow on port 8889 + +exporters: + otlp: +endpoint: "oap:11800" # Exporter configuration, sending data to the specified endpoint address +tls: + insecure: true # Disables TLS security or TLS certificate verification in the testing environment + +processors: + transform: +metric_statements: + - context: resource +statements: [] + - context: metric +statements: + - set(name, ConvertCase(name, "snake")) # Transform metric names to snake case + + metricstransform: +transforms: + - include: "^airflow.*" # Select metrics starting with 'airflow' for transformation +match_type: regexp # Use regular expression for matching metric names +action: update +operations: + - action: add_label +new_label: job_name # Add a new label 'job_name' to the selected metrics +new_value: "airflow-monitoring" + - include: "^airflow.*" +match_type: regexp +action: update +operations: + - action: add_label +new_label: service_instance_id # Add 'service_instance_id' label to metrics +new_value: "instance1" + - include: "^airflow.*" +match_type: regexp +action: update +operations: + - action: add_label +new_label: host_name # Add 'host_name' label indicating the host name +new_value: "airflow-webserver.airflow" Review Comment: If this is the service name, and hard coded instance name, I doubt the service and instance hierarchy would work. I saw your screenshot, but that seems not match your e2e. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Add Apache Airflow Monitoring [skywalking]
wu-sheng commented on code in PR #12072: URL: https://github.com/apache/skywalking/pull/12072#discussion_r1555121756 ## test/e2e-v2/cases/airflow/airflow-main/e2e.yaml: ## @@ -0,0 +1,38 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# This file is used to show how to write configuration files and can be used to test. + +setup: + env: compose Review Comment: Please make sure whether compose is a good way to run metadata clear deployment. If it isn't, airflow instance + otel collector as sidecar deployment on k8s may be a better way. e2e is not just for testing, you could see on the docs(you wrote), all configurations here are a kind of guidance and reference for users to deploy this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(skywalking-banyandb) 02/03: Update non-go files
This is an automated email from the ASF dual-hosted git repository. hanahmily pushed a commit to branch clean-up in repository https://gitbox.apache.org/repos/asf/skywalking-banyandb.git commit 343e42eb359c030631f0c32ff86f6cde348665fa Author: Gao Hongtao AuthorDate: Sun Apr 7 05:38:50 2024 + Update non-go files Signed-off-by: Gao Hongtao --- banyand/metadata/schema/testdata/group.json| 4 - .../schema/testdata/index_rules/db.instance.json | 1 - .../schema/testdata/index_rules/db.type.json | 1 - .../schema/testdata/index_rules/duration.json | 3 +- .../schema/testdata/index_rules/endpoint_id.json | 1 - .../schema/testdata/index_rules/http.method.json | 1 - .../schema/testdata/index_rules/mq.broker.json | 1 - .../schema/testdata/index_rules/mq.queue.json | 1 - .../schema/testdata/index_rules/mq.topic.json | 1 - .../schema/testdata/index_rules/status_code.json | 1 - .../schema/testdata/index_rules/trace_id.json | 3 +- bydbctl/internal/cmd/group_test.go | 9 - bydbctl/internal/cmd/index_rule_binding_test.go| 3 - bydbctl/internal/cmd/index_rule_test.go| 12 +- bydbctl/internal/cmd/measure_test.go | 3 - bydbctl/internal/cmd/stream_test.go| 3 - dist/LICENSE | 7 +- .../license-github.com-SkyAPM-badger-v3.txt| 176 -- .../license-github.com-dgraph-io-ristretto.txt | 176 -- dist/licenses/license-github.com-golang-glog.txt | 191 --- .../license-github.com-golang-groupcache.txt | 191 --- .../license-github.com-google-flatbuffers.txt | 202 - .../license-github.com-spaolacci-murmur3.txt | 24 +++ dist/licenses/license-go.opencensus.io.txt | 202 - docs/concept/data-model.md | 10 +- docs/crud/group.md | 6 - docs/crud/index_rule.md| 9 +- docs/crud/index_rule_binding.md| 3 - docs/crud/measure/schema.md| 3 - docs/crud/stream/schema.md | 3 - pkg/test/measure/testdata/groups/sw_metric.json| 4 - pkg/test/measure/testdata/index_rules/id.json | 1 - pkg/test/measure/testdata/index_rules/layer.json | 1 - .../testdata/index_rules/searchable_name.json | 1 - .../measure/testdata/index_rules/service_id.json | 1 - pkg/test/stream/testdata/group.json| 4 - .../stream/testdata/index_rules/db.instance.json | 1 - pkg/test/stream/testdata/index_rules/db.type.json | 1 - pkg/test/stream/testdata/index_rules/duration.json | 3 +- .../stream/testdata/index_rules/endpoint_id.json | 1 - .../stream/testdata/index_rules/extended_tags.json | 1 - .../stream/testdata/index_rules/http.method.json | 1 - .../stream/testdata/index_rules/mq.broker.json | 1 - pkg/test/stream/testdata/index_rules/mq.queue.json | 1 - pkg/test/stream/testdata/index_rules/mq.topic.json | 1 - .../stream/testdata/index_rules/status_code.json | 1 - pkg/test/stream/testdata/index_rules/trace_id.json | 3 +- .../index-rules/measure-default-index-rule.yaml| 30 +-- .../index-rules/measure-minute-index-rule.yaml | 13 +- ui/src/components/IndexRule/Editor.vue | 32 ui/src/components/IndexRule/index.vue | 3 - 51 files changed, 38 insertions(+), 1318 deletions(-) diff --git a/banyand/metadata/schema/testdata/group.json b/banyand/metadata/schema/testdata/group.json index e200d011..ac044783 100644 --- a/banyand/metadata/schema/testdata/group.json +++ b/banyand/metadata/schema/testdata/group.json @@ -5,10 +5,6 @@ "catalog": "CATALOG_STREAM", "resource_opts": { "shard_num": 2, -"block_interval": { - "unit": "UNIT_HOUR", - "num": 2 -}, "segment_interval": { "unit": "UNIT_DAY", "num": 1 diff --git a/banyand/metadata/schema/testdata/index_rules/db.instance.json b/banyand/metadata/schema/testdata/index_rules/db.instance.json index d6bcbb10..8f2068a4 100644 --- a/banyand/metadata/schema/testdata/index_rules/db.instance.json +++ b/banyand/metadata/schema/testdata/index_rules/db.instance.json @@ -8,6 +8,5 @@ "db.instance" ], "type": "TYPE_INVERTED", - "location": "LOCATION_SERIES", "updated_at": "2021-04-15T01:30:15.01Z" } diff --git a/banyand/metadata/schema/testdata/index_rules/db.type.json b/banyand/metadata/schema/testdata/index_rules/db.type.json index b13adedc..f23fd03f 100644 --- a/banyand/metadata/schema/testdata/index_rules/db.type.json +++ b/banyand/metadata/schema/testdata/index_rules/db.type.json @@ -8,6 +8,5 @@ "db.type" ], "type": "TYPE_INVERTED", - "location": "LOCATION_SERIES", "updated_at": "2021-04-15T01:30:15.01Z" } diff --git a/banyand/m
(skywalking-banyandb) branch clean-up created (now ba8bf559)
This is an automated email from the ASF dual-hosted git repository. hanahmily pushed a change to branch clean-up in repository https://gitbox.apache.org/repos/asf/skywalking-banyandb.git at ba8bf559 Remove wal and fix e2e This branch includes the following new commits: new cf770259 Clean deprecated modules: new 343e42eb Update non-go files new ba8bf559 Remove wal and fix e2e The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(skywalking-banyandb) 03/03: Remove wal and fix e2e
This is an automated email from the ASF dual-hosted git repository. hanahmily pushed a commit to branch clean-up in repository https://gitbox.apache.org/repos/asf/skywalking-banyandb.git commit ba8bf559de68892b87fde961bd9776a51ea5de51 Author: Gao Hongtao AuthorDate: Mon Apr 8 00:14:01 2024 + Remove wal and fix e2e Signed-off-by: Gao Hongtao --- CHANGES.md |3 + dist/LICENSE |1 - dist/licenses/license-github.com-golang-snappy.txt | 27 - docs/concept/persistence-storage.md|6 +- docs/concept/wal.md| 47 - go.mod |1 - go.sum |2 - pkg/wal/README.md | 147 --- pkg/wal/wal.go | 1081 pkg/wal/wal_benchmark_test.go | 450 pkg/wal/wal_test.go| 242 - test/docker/base-compose.yml |3 +- test/e2e-v2/script/env |2 +- test/stress/istio/testdata/groups/group.yaml | 21 - test/stress/trace/docker-compose.yaml |3 +- test/stress/trace/env | 15 +- ui/src/components/Aside/index.vue | 10 - 17 files changed, 12 insertions(+), 2049 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index 7dcb06f3..6f2b85d8 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -22,6 +22,9 @@ Release Notes. - Merge memory data and disk data. - Add HTTP services to TopNAggregation operations. - Add preload for the TopN query of index. +- Remove "TREE" index type. The "TREE" index type is merged into "INVERTED" index type. +- Remove "Location" field on IndexRule. Currently, the location of index is in a segment. +- Remove "BlockInterval" from Group. The block size is determined by the part. ### Bugs diff --git a/dist/LICENSE b/dist/LICENSE index 3d7db32b..66b9ff1d 100644 --- a/dist/LICENSE +++ b/dist/LICENSE @@ -266,7 +266,6 @@ BSD-3-Clause licenses github.com/fsnotify/fsnotify v1.7.0 BSD-3-Clause github.com/gogo/protobuf v1.3.2 BSD-3-Clause github.com/golang/protobuf v1.5.3 BSD-3-Clause -github.com/golang/snappy v0.0.4 BSD-3-Clause github.com/google/go-cmp v0.6.0 BSD-3-Clause github.com/google/uuid v1.4.0 BSD-3-Clause github.com/gorilla/websocket v1.5.1 BSD-3-Clause diff --git a/dist/licenses/license-github.com-golang-snappy.txt b/dist/licenses/license-github.com-golang-snappy.txt deleted file mode 100644 index 6050c10f.. --- a/dist/licenses/license-github.com-golang-snappy.txt +++ /dev/null @@ -1,27 +0,0 @@ -Copyright (c) 2011 The Snappy-Go Authors. All rights reserved. - -Redistribution and use in source and binary forms, with or without -modification, are permitted provided that the following conditions are -met: - - * Redistributions of source code must retain the above copyright -notice, this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above -copyright notice, this list of conditions and the following disclaimer -in the documentation and/or other materials provided with the -distribution. - * Neither the name of Google Inc. nor the names of its -contributors may be used to endorse or promote products derived from -this software without specific prior written permission. - -THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS -"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT -LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR -A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT -OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, -SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT -LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, -DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY -THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT -(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/docs/concept/persistence-storage.md b/docs/concept/persistence-storage.md index 48199985..dd057ed2 100644 --- a/docs/concept/persistence-storage.md +++ b/docs/concept/persistence-storage.md @@ -1,11 +1,11 @@ # Persistence Storage -Persistence storage is used for unifying data of BanyanDB persistence, including write-ahead logging(WAL), index, and data collected from skywalking and other observability platforms or APM systems. It provides various implementations and IO modes to satisfy the need of different components. +Persistence storage is used for unifying data of BanyanDB persistence, including index, and data collected from skywalking and oth
Re: [PR] Add Apache Airflow Monitoring [skywalking]
wu-sheng commented on code in PR #12072: URL: https://github.com/apache/skywalking/pull/12072#discussion_r1555120308 ## test/e2e-v2/cases/airflow/airflow-exporter/otel-collector-config.yaml: ## @@ -0,0 +1,69 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +receivers: + otlp: +protocols: + http: +endpoint: "0.0.0.0:8889" # Receiver configuration using the OTLP protocol over HTTP, listening Airflow on port 8889 + +exporters: + otlp: +endpoint: "oap:11800" # Exporter configuration, sending data to the specified endpoint address +tls: + insecure: true # Disables TLS security or TLS certificate verification in the testing environment + +processors: + transform: +metric_statements: + - context: resource +statements: [] + - context: metric +statements: + - set(name, ConvertCase(name, "snake")) # Transform metric names to snake case + + metricstransform: +transforms: + - include: "^airflow.*" # Select metrics starting with 'airflow' for transformation +match_type: regexp # Use regular expression for matching metric names +action: update +operations: + - action: add_label +new_label: job_name # Add a new label 'job_name' to the selected metrics +new_value: "airflow-monitoring" + - include: "^airflow.*" +match_type: regexp +action: update +operations: + - action: add_label +new_label: service_instance_id # Add 'service_instance_id' label to metrics +new_value: "instance1" Review Comment: If this is impossible(if there is no value to have metadata in OTLP push), I think we should recommend using K8s sidecar as recommended deployment to add this label. In there, we could use pod name as the instance name, and k8s service name as the service name. Meanwhile, is `host_name` representing the service name? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(skywalking-banyandb-java-client) 01/01: Bump api
This is an automated email from the ASF dual-hosted git repository. hanahmily pushed a commit to branch bump-api in repository https://gitbox.apache.org/repos/asf/skywalking-banyandb-java-client.git commit 0dcb1e0a7acf04f14054d2dad49a7130d56754b1 Author: Gao Hongtao AuthorDate: Mon Apr 8 07:25:09 2024 +0800 Bump api Signed-off-by: Gao Hongtao --- .../banyandb/v1/client/metadata/Group.java | 19 ++ .../banyandb/v1/client/metadata/IndexRule.java | 47 +++--- src/main/proto/banyandb/v1/banyandb-common.proto | 11 ++-- src/main/proto/banyandb/v1/banyandb-database.proto | 72 ++ .../v1/client/BanyanDBClientMeasureQueryTest.java | 2 +- .../v1/client/BanyanDBClientMeasureWriteTest.java | 2 +- .../v1/client/BanyanDBClientStreamQueryTest.java | 2 +- .../v1/client/BanyanDBClientStreamWriteTest.java | 2 +- .../banyandb/v1/client/BanyanDBClientTestCI.java | 4 +- .../v1/client/ITBanyanDBMeasureQueryTests.java | 4 +- .../v1/client/ITBanyanDBStreamQueryTests.java | 4 +- .../metadata/IndexRuleMetadataRegistryTest.java| 8 +-- .../metadata/MeasureMetadataRegistryTest.java | 6 +- .../metadata/StreamMetadataRegistryTest.java | 6 +- 14 files changed, 71 insertions(+), 118 deletions(-) diff --git a/src/main/java/org/apache/skywalking/banyandb/v1/client/metadata/Group.java b/src/main/java/org/apache/skywalking/banyandb/v1/client/metadata/Group.java index 4fcca7d..0d7cb09 100644 --- a/src/main/java/org/apache/skywalking/banyandb/v1/client/metadata/Group.java +++ b/src/main/java/org/apache/skywalking/banyandb/v1/client/metadata/Group.java @@ -38,37 +38,32 @@ public abstract class Group extends NamedSchema { */ abstract int shardNum(); -@Nullable -abstract IntervalRule blockInterval(); - @Nullable abstract IntervalRule segmentInterval(); @Nullable abstract IntervalRule ttl(); -public static Group create(String name, Catalog catalog, int shardNum, IntervalRule blockInterval, IntervalRule segmentInterval, IntervalRule ttl) { +public static Group create(String name, Catalog catalog, int shardNum, IntervalRule segmentInterval, IntervalRule ttl) { Preconditions.checkArgument(shardNum > 0, "shardNum should more than 0"); -Preconditions.checkNotNull(blockInterval, "blockInterval is null"); Preconditions.checkNotNull(segmentInterval, "segmentInterval is null"); Preconditions.checkNotNull(ttl, "ttl is null"); -return new AutoValue_Group(null, name, null, catalog, shardNum, blockInterval, segmentInterval, ttl); +return new AutoValue_Group(null, name, null, catalog, shardNum, segmentInterval, ttl); } -public static Group create(String name, Catalog catalog, int shardNum, IntervalRule blockInterval, IntervalRule segmentInterval, IntervalRule ttl, ZonedDateTime updatedAt) { +public static Group create(String name, Catalog catalog, int shardNum, IntervalRule segmentInterval, IntervalRule ttl, ZonedDateTime updatedAt) { Preconditions.checkArgument(shardNum > 0, "shardNum should more than 0"); -Preconditions.checkNotNull(blockInterval, "blockInterval is null"); Preconditions.checkNotNull(segmentInterval, "segmentInterval is null"); Preconditions.checkNotNull(ttl, "ttl is null"); -return new AutoValue_Group(null, name, updatedAt, catalog, shardNum, blockInterval, segmentInterval, ttl); +return new AutoValue_Group(null, name, updatedAt, catalog, shardNum, segmentInterval, ttl); } public static Group create(String name) { -return new AutoValue_Group(null, name, null, Catalog.UNSPECIFIED, 0, null, null, null); +return new AutoValue_Group(null, name, null, Catalog.UNSPECIFIED, 0, null, null); } public static Group create(String name, ZonedDateTime updatedAt) { -return new AutoValue_Group(null, name, updatedAt, Catalog.UNSPECIFIED, 0, null, null, null); +return new AutoValue_Group(null, name, updatedAt, Catalog.UNSPECIFIED, 0, null, null); } @Override @@ -80,7 +75,6 @@ public abstract class Group extends NamedSchema { if (shardNum() > 0) { builder.setResourceOpts(BanyandbCommon.ResourceOpts.newBuilder() .setShardNum(shardNum()) -.setBlockInterval(blockInterval().serialize()) .setSegmentInterval(segmentInterval().serialize()) .setTtl(ttl().serialize()) .build()); @@ -104,7 +98,6 @@ public abstract class Group extends NamedSchema { TimeUtils.parseTimestamp(group.getUpdatedAt()), catalog, opts == null ? 0 : opts.getShardNum(), -opts == null ? null : IntervalRule.fromProtobuf(opts.getBlockInterval()), opts == null ? null : IntervalRule.fromProtobu
(skywalking-banyandb-java-client) branch bump-api created (now 0dcb1e0)
This is an automated email from the ASF dual-hosted git repository. hanahmily pushed a change to branch bump-api in repository https://gitbox.apache.org/repos/asf/skywalking-banyandb-java-client.git at 0dcb1e0 Bump api This branch includes the following new commits: new 0dcb1e0 Bump api The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(skywalking) branch bump-banyandb-api created (now cec72540e7)
This is an automated email from the ASF dual-hosted git repository. hanahmily pushed a change to branch bump-banyandb-api in repository https://gitbox.apache.org/repos/asf/skywalking.git at cec72540e7 Bump banyandb api This branch includes the following new commits: new cec72540e7 Bump banyandb api The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(skywalking) 01/01: Bump banyandb api
This is an automated email from the ASF dual-hosted git repository. hanahmily pushed a commit to branch bump-banyandb-api in repository https://gitbox.apache.org/repos/asf/skywalking.git commit cec72540e712e0c9a56c18171fb3e98897b27f11 Author: Gao Hongtao AuthorDate: Mon Apr 8 07:21:41 2024 +0800 Bump banyandb api Signed-off-by: Gao Hongtao --- oap-server-bom/pom.xml | 2 +- .../analysis/manual/log/AbstractLogRecord.java | 2 -- .../analysis/manual/segment/SegmentRecord.java | 1 - .../manual/spanattach/SpanAttachedEventRecord.java | 1 - .../server/core/storage/annotation/BanyanDB.java | 15 .../core/storage/model/BanyanDBExtension.java | 8 - .../server/core/storage/model/StorageModels.java | 3 -- .../plugin/banyandb/BanyanDBStorageConfig.java | 14 .../plugin/banyandb/BanyanDBStorageProvider.java | 3 -- .../storage/plugin/banyandb/MetadataRegistry.java | 40 +++--- skywalking-ui | 2 +- 11 files changed, 7 insertions(+), 84 deletions(-) diff --git a/oap-server-bom/pom.xml b/oap-server-bom/pom.xml index f58d52b157..b386c5f78e 100644 --- a/oap-server-bom/pom.xml +++ b/oap-server-bom/pom.xml @@ -72,7 +72,7 @@ 3.0.0 4.4.13 1.21 -0.5.0 + 0.6.0-SNAPSHOT 2.8.1 2.4.6.RELEASE 1.5.3 diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/manual/log/AbstractLogRecord.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/manual/log/AbstractLogRecord.java index e3dd57f616..a7efde7f16 100644 --- a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/manual/log/AbstractLogRecord.java +++ b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/manual/log/AbstractLogRecord.java @@ -68,12 +68,10 @@ public abstract class AbstractLogRecord extends Record { @Setter @Getter @Column(name = TRACE_ID, length = 150) -@BanyanDB.GlobalIndex private String traceId; @Setter @Getter @Column(name = TRACE_SEGMENT_ID, length = 150) -@BanyanDB.GlobalIndex private String traceSegmentId; @Setter @Getter diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/manual/segment/SegmentRecord.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/manual/segment/SegmentRecord.java index d07f6842d6..4b8966a6ea 100644 --- a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/manual/segment/SegmentRecord.java +++ b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/manual/segment/SegmentRecord.java @@ -65,7 +65,6 @@ public class SegmentRecord extends Record { @Setter @Getter @Column(name = TRACE_ID, length = 150) -@BanyanDB.GlobalIndex @ElasticSearch.Routing private String traceId; @Setter diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/manual/spanattach/SpanAttachedEventRecord.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/manual/spanattach/SpanAttachedEventRecord.java index 0e4c368659..fb681be8bc 100644 --- a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/manual/spanattach/SpanAttachedEventRecord.java +++ b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/manual/spanattach/SpanAttachedEventRecord.java @@ -67,7 +67,6 @@ public class SpanAttachedEventRecord extends Record { @Column(name = TRACE_REF_TYPE) private int traceRefType; @Column(name = RELATED_TRACE_ID) -@BanyanDB.GlobalIndex private String relatedTraceId; @Column(name = TRACE_SEGMENT_ID) private String traceSegmentId; diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/annotation/BanyanDB.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/annotation/BanyanDB.java index cb9ad293b8..68a1e0fcc4 100644 --- a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/annotation/BanyanDB.java +++ b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/annotation/BanyanDB.java @@ -32,21 +32,6 @@ import org.apache.skywalking.oap.server.core.analysis.record.Record; * @since 9.1.0 */ public @interface BanyanDB { -/** - * GlobalIndex declares advanced global index, which are only available in BanyanDB. - * - * Global index should only be considered if a column value has a huge value candidates, but we will need a direct - * equal - * query without timestamp. - * The typical global index is designed for huge candidate of indexed
(skywalking-showcase) branch main updated: Fix docker configurations and bump up images (#167)
This is an automated email from the ASF dual-hosted git repository. wusheng pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/skywalking-showcase.git The following commit(s) were added to refs/heads/main by this push: new bbe2e5d Fix docker configurations and bump up images (#167) bbe2e5d is described below commit bbe2e5d3b17c3d68094aafc16ae0d5f1e3328448 Author: Starry AuthorDate: Sun Apr 7 23:33:55 2024 +0800 Fix docker configurations and bump up images (#167) --- Makefile.in | 4 ++-- deploy/platform/docker/docker-compose.mongodb-monitor.yaml | 2 +- deploy/platform/docker/docker-compose.pulsar-monitor.yaml | 2 +- deploy/platform/docker/docker-compose.rocketmq-monitor.yaml | 4 ++-- docs/readme.md | 1 + 5 files changed, 7 insertions(+), 6 deletions(-) diff --git a/Makefile.in b/Makefile.in index fd1de2d..566e680 100644 --- a/Makefile.in +++ b/Makefile.in @@ -29,10 +29,10 @@ ES_IMAGE ?= docker.elastic.co/elasticsearch/elasticsearch-oss ES_IMAGE_TAG ?= 7.10.2 SW_OAP_IMAGE ?= ghcr.io/apache/skywalking/oap -SW_OAP_IMAGE_TAG ?= 8fad0cc8a822a3b266e528b30295e1376b956f1d +SW_OAP_IMAGE_TAG ?= 48c96a233103ba869143062923d4033678b91f73 SW_UI_IMAGE ?= ghcr.io/apache/skywalking/ui -SW_UI_IMAGE_TAG ?= 8fad0cc8a822a3b266e528b30295e1376b956f1d +SW_UI_IMAGE_TAG ?= 8bc67614681b37623fa64233f8389d6754e64cf2 SW_CLI_IMAGE ?= ghcr.io/apache/skywalking-cli/skywalking-cli:0883266bfaa36612927b69e35781b64ea181758d SW_EVENT_EXPORTER_IMAGE ?= ghcr.io/apache/skywalking-kubernetes-event-exporter/skywalking-kubernetes-event-exporter:8a012a3f968cb139f817189afb9b3748841bba22 diff --git a/deploy/platform/docker/docker-compose.mongodb-monitor.yaml b/deploy/platform/docker/docker-compose.mongodb-monitor.yaml index 640c9f3..b311f34 100644 --- a/deploy/platform/docker/docker-compose.mongodb-monitor.yaml +++ b/deploy/platform/docker/docker-compose.mongodb-monitor.yaml @@ -77,7 +77,7 @@ services: - '--collector.diagnosticdata' - '--collector.replicasetstatus' otel-collector: -image: otel/opentelemetry-collector:0.72.0 +image: ${OTEL_COLLECTOR_IMAGE}:${OTEL_COLLECTOR_IMAGE_TAG} networks: - sw command: [ "--config=/etc/otel-collector-config.yaml" ] diff --git a/deploy/platform/docker/docker-compose.pulsar-monitor.yaml b/deploy/platform/docker/docker-compose.pulsar-monitor.yaml index 6fbdcf5..7357b3b 100644 --- a/deploy/platform/docker/docker-compose.pulsar-monitor.yaml +++ b/deploy/platform/docker/docker-compose.pulsar-monitor.yaml @@ -120,7 +120,7 @@ services: command: bash -c "bin/pulsar-perf produce -m 1 sw-topic" otel-collector: -image: otel/opentelemetry-collector:${OTEL_COLLECTOR_VERSION} +image: ${OTEL_COLLECTOR_IMAGE}:${OTEL_COLLECTOR_IMAGE_TAG} networks: - sw command: [ "--config=/etc/otel-collector-config.yaml" ] diff --git a/deploy/platform/docker/docker-compose.rocketmq-monitor.yaml b/deploy/platform/docker/docker-compose.rocketmq-monitor.yaml index 2a5fc52..bf57f08 100644 --- a/deploy/platform/docker/docker-compose.rocketmq-monitor.yaml +++ b/deploy/platform/docker/docker-compose.rocketmq-monitor.yaml @@ -101,10 +101,10 @@ services: - e2e otel-collector: -image: otel/opentelemetry-collector:${OTEL_COLLECTOR_VERSION} +image: ${OTEL_COLLECTOR_IMAGE}:${OTEL_COLLECTOR_IMAGE_TAG} networks: - e2e -command: [ "--config=/etc/otel-collector-rocketmq-config.yaml" ] +command: [ "--config=/etc/otel-collector-config.yaml" ] volumes: - ./config/rocketmq/otel-collector-rocketmq-config.yaml:/etc/otel-collector-config.yaml expose: diff --git a/docs/readme.md b/docs/readme.md index 40f1c76..c5131ef 100644 --- a/docs/readme.md +++ b/docs/readme.md @@ -113,6 +113,7 @@ Currently, the features supported are: | `r3`| Deploy [R3](https://github.com/SkyAPM/R3) as RESTful URL recognition service. | | | `rocketmq-monitor` | Deploy OpenTelemetry and export RocketMQ monitoring metrics to SkyWalking for analysis and display on UI. | | | `pulsar-monitor`| Deploy OpenTelemetry and export Pulsar monitoring metrics to SkyWalking for analysis and display on UI. | | +| `rabbitmq-
Re: [PR] Fix docker configurations and bump up images [skywalking-showcase]
wu-sheng merged PR #167: URL: https://github.com/apache/skywalking-showcase/pull/167 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Add Apache Airflow Monitoring [skywalking]
wankai123 commented on code in PR #12072: URL: https://github.com/apache/skywalking/pull/12072#discussion_r1554996072 ## test/e2e-v2/cases/airflow/airflow-exporter/otel-collector-config.yaml: ## @@ -0,0 +1,69 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +receivers: + otlp: +protocols: + http: +endpoint: "0.0.0.0:8889" # Receiver configuration using the OTLP protocol over HTTP, listening Airflow on port 8889 + +exporters: + otlp: +endpoint: "oap:11800" # Exporter configuration, sending data to the specified endpoint address +tls: + insecure: true # Disables TLS security or TLS certificate verification in the testing environment + +processors: + transform: +metric_statements: + - context: resource +statements: [] + - context: metric +statements: + - set(name, ConvertCase(name, "snake")) # Transform metric names to snake case + + metricstransform: +transforms: + - include: "^airflow.*" # Select metrics starting with 'airflow' for transformation +match_type: regexp # Use regular expression for matching metric names +action: update +operations: + - action: add_label +new_label: job_name # Add a new label 'job_name' to the selected metrics +new_value: "airflow-monitoring" + - include: "^airflow.*" +match_type: regexp +action: update +operations: + - action: add_label +new_label: service_instance_id # Add 'service_instance_id' label to metrics +new_value: "instance1" Review Comment: When airflow metrics use the push mode, do the metrics have any labels to identify which service and instance? You add labels manually here, I'm not sure when 1 otel-collector facing 2 airflow instances, how to transform the metrics and add labels. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Add Apache Airflow Monitoring [skywalking]
wankai123 commented on code in PR #12072: URL: https://github.com/apache/skywalking/pull/12072#discussion_r1554974594 ## test/e2e-v2/cases/airflow/airflow-main/docker-compose.yaml: ## @@ -0,0 +1,139 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +version: '2.1' + +services: + # SkyWalking OAP server + oap: +extends: + file: ../../../script/docker-compose/base-compose.yml + service: oap +ports: + - 12800:12800 +networks: + - e2e +environment: + - TZ=Asia/Shanghai + - SW_CORE_TOPN_REPORT_PERIOD=2 + - SW_STORAGE=h2 + # OpenTelemetry collector 1 + otel-collector: +image: otel/opentelemetry-collector-contrib:0.96.0 Review Comment: ```suggestion image: otel/opentelemetry-collector:${OTEL_COLLECTOR_VERSION} ``` ## oap-server/server-starter/src/main/resources/ui-initialized-templates/airflow/airflow-service.json: ## @@ -0,0 +1,365 @@ +[ + { +"id": "Airflow-Service", +"configuration": { + "children": [ +{ + "x": 0, + "y": 0, + "w": 24, + "h": 33, + "i": "0", + "type": "Tab", + "children": [ +{ + "name": "Inspections", + "children": [ +{ + "x": 0, + "y": 0, + "w": 6, + "h": 13, + "i": "1", + "type": "Widget", + "graph": { +"type": "Card", +"fontSize": 124, +"textAlign": "center", +"showUnit": true + }, + "widget": { +"title": "Airflow Job", +"name": "FileOpen", +"tips": "This count includes the number of all instances" + }, + "metricMode": "Expression", + "metricConfig": [ + { + "label": "job Start Nums" +} + ], + "expressions": [ +"latest(aggregate_labels(meter_airflow_job_start,sum))" + ], + "typesOfMQE": [ +"SINGLE_VALUE" + ] +}, +{ + "x": 6, + "y": 0, + "w": 6, + "h": 13, + "i": "2", + "type": "Widget", + "graph": { +"type": "Line", +"step": false, +"smooth": false, +"showSymbol": false, +"showXAxis": true, +"showYAxis": true + }, + "widget": { +"title": "Tasks Related", +"tips": "This count includes the number of all instances" + }, + "metricConfig": [ +{ + "label": "Tasks Executable" +}, +{ + "label": "Tasks Cleared" +}, +{ + "label": "Tasks Adopted" +}, +{ + "label": "Queued Tasks" +} + ], + "metricMode": "Expression", + "expressions": [ + "aggregate_labels(meter_airflow_scheduler_tasks_executable,sum)", + "aggregate_labels(meter_airflow_scheduler_orphaned_tasks_cleared,sum)", + "aggregate_labels(meter_airflow_scheduler_orphaned_tasks_adopted,sum)", +"aggregate_labels(meter_airflow_executor_queued_tasks,sum)" + ], + "typesOfMQE": [ +"TIME_SERIES_VALUES", +"TIME_SERIES_VALUES", +"TIME_SERIES_VALUES", +"TIME_SERIES_VALUES" + ] +}, +{ + "x": 12, + "y": 0, + "
Re: [PR] Add Apache Airflow Monitoring [skywalking]
wu-sheng commented on PR #12072: URL: https://github.com/apache/skywalking/pull/12072#issuecomment-2041486644 Please recheck UI submodule, you are changing that unexpectedly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Add Apache Airflow Monitoring [skywalking]
wu-sheng commented on code in PR #12072: URL: https://github.com/apache/skywalking/pull/12072#discussion_r1554985214 ## docs/en/changes/changes.md: ## @@ -103,7 +106,9 @@ * Enhance VNode logic and support multiple Trace IDs in span's ref. * Add the layers filed and associate layers dashboards for the service topology nodes. * Fix `Nginx-Instance` metrics to instance level. -* Update tabs of the Kubernetes service page. +* Update tabs of the Kubernetes service page. +* Support Airflow monitoring. +* Update tabs of the Kubernetes service page. Review Comment: ```suggestion * Update tabs of the Kubernetes service page. * Support Airflow monitoring. ``` Duplicated -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Add Apache Airflow Monitoring [skywalking]
wu-sheng commented on PR #12072: URL: https://github.com/apache/skywalking/pull/12072#issuecomment-2041409947 Don't close a PR, as you only update things. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Add Apache Airflow Monitoring [skywalking]
songzhendong closed pull request #12072: Add Apache Airflow Monitoring URL: https://github.com/apache/skywalking/pull/12072 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(skywalking-banyandb) branch dependabot/npm_and_yarn/ui/vite-3.2.10 deleted (was 074f797a)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/npm_and_yarn/ui/vite-3.2.10 in repository https://gitbox.apache.org/repos/asf/skywalking-banyandb.git was 074f797a Bump vite from 3.2.7 to 3.2.10 in /ui The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
Re: [PR] Bump vite from 3.2.7 to 3.2.10 in /ui [skywalking-banyandb]
dependabot[bot] commented on PR #427: URL: https://github.com/apache/skywalking-banyandb/pull/427#issuecomment-2041353142 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting `@dependabot ignore this major version` or `@dependabot ignore this minor version`. If you change your mind, just re-open this PR and I'll resolve any conflicts on it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Bump vite from 3.2.7 to 3.2.10 in /ui [skywalking-banyandb]
wu-sheng closed pull request #427: Bump vite from 3.2.7 to 3.2.10 in /ui URL: https://github.com/apache/skywalking-banyandb/pull/427 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(skywalking-banyandb) branch idx-gc deleted (was b3728d5b)
This is an automated email from the ASF dual-hosted git repository. wusheng pushed a change to branch idx-gc in repository https://gitbox.apache.org/repos/asf/skywalking-banyandb.git was b3728d5b Merge remote-tracking branch 'origin/main' into idx-gc The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
Re: [PR] Add Apache Airflow Monitoring [skywalking]
wu-sheng commented on code in PR #12072: URL: https://github.com/apache/skywalking/pull/12072#discussion_r1554831337 ## docs/en/swip/readme.md: ## @@ -71,7 +71,7 @@ All accepted and proposed SWIPs can be found in [here](https://github.com/apache Next SWIP Number: 6 Review Comment: ```suggestion Next SWIP Number: 8 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org