Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
wu-sheng closed issue #11982: [Bug] Metrics are not aggregated properly as OTEL collector targeting per node URL: https://github.com/apache/skywalking/issues/11982 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
innerpeacez commented on issue #11982: URL: https://github.com/apache/skywalking/issues/11982#issuecomment-2066206843 Rabbitmq server metrics are accurate（instance level used node tag）, thx for your help @wankai123 again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
wu-sheng closed issue #11982: [Bug] Metrics are not aggregated properly as OTEL collector targeting per node URL: https://github.com/apache/skywalking/issues/11982 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
wu-sheng closed issue #11982: [Bug] Metrics are not aggregated properly as OTEL collector targeting per node URL: https://github.com/apache/skywalking/issues/11982 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
wu-sheng commented on issue #11982: URL: https://github.com/apache/skywalking/issues/11982#issuecomment-2047362729 @weixiang1862 @pg-yang @innerpeacez We have the new multi-label support merged. You should be good to fix this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
wu-sheng commented on issue #11982: URL: https://github.com/apache/skywalking/issues/11982#issuecomment-2016697181 > Yes, it's incorrect. I'm waiting for `aggregate_labels ` with label parameters Got it. That will need another one or two weeks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
pg-yang commented on issue #11982: URL: https://github.com/apache/skywalking/issues/11982#issuecomment-2016675897 Yes, it's incorrect. I'm waiting for `aggregate_labels ` with label parameters -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
wu-sheng commented on issue #11982: URL: https://github.com/apache/skywalking/issues/11982#issuecomment-2016652944 @pg-yang How is the APISIX? The metrics on the dashboard seem not correctly because they have cross-instance aggregation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
wu-sheng commented on issue #11982: URL: https://github.com/apache/skywalking/issues/11982#issuecomment-1988781280 > > > Pulsar monitoring is to aggregate the cluster and node(broker) metrics together now, it should be fine. > > > > > > The question is how to aggregate the metrics of the cluster? > > The prometheus-exporter collects from a single node, the collection of the cluster requires collecting and summarizing each node. OK. Then it is fine for now. But you could start a new issue about polishing the Pulsar monitoring as we discussed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
CodePrometheus commented on issue #11982: URL: https://github.com/apache/skywalking/issues/11982#issuecomment-1988769602 > > Pulsar monitoring is to aggregate the cluster and node(broker) metrics together now, it should be fine. > > The question is how to aggregate the metrics of the cluster? The prometheus-exporter collects from a single node, the collection of the cluster requires collecting and summarizing each node. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
wu-sheng commented on issue #11982: URL: https://github.com/apache/skywalking/issues/11982#issuecomment-1987288065 > Pulsar monitoring is to aggregate the cluster and node(broker) metrics together now, it should be fine. The question is how to aggregate the metrics of the cluster? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
CodePrometheus commented on issue #11982: URL: https://github.com/apache/skywalking/issues/11982#issuecomment-1987276192 Pulsar monitoring is to aggregate the cluster and node(broker) metrics together now, it should be fine. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
CodePrometheus commented on issue #11982: URL: https://github.com/apache/skywalking/issues/11982#issuecomment-1985163909 @wu-sheng I'll try to check pulsar first. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
wu-sheng commented on issue #11982: URL: https://github.com/apache/skywalking/issues/11982#issuecomment-1985158286 > @wu-sheng I would be willing, of course! I'll do recheck on this weekend. Which one will you check? Let's avoid duplicate works with others. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
CodePrometheus commented on issue #11982: URL: https://github.com/apache/skywalking/issues/11982#issuecomment-1985151340 @wu-sheng I would be willing, of course! I'll do recheck on this weekend. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
wu-sheng commented on issue #11982: URL: https://github.com/apache/skywalking/issues/11982#issuecomment-1980821173 @weixiang1862 As you asked about, we posted https://github.com/apache/skywalking/issues/11992 for v10. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
wu-sheng commented on issue #11982:
URL: https://github.com/apache/skywalking/issues/11982#issuecomment-1980252064
In your case, the data in the DB should be `http_status_401{192.168.0.1=5,
192.168.0.3=1, 192.168.0.2=2}`. Please verify.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
wu-sheng commented on issue #11982:
URL: https://github.com/apache/skywalking/issues/11982#issuecomment-1980250350
I don't think we support multiple labels after you have done the persistent.
```
http_status{host=192.168.0.1, status=401} 5
http_status{host=192.168.0.3, status=401} 1
http_status{host=192.168.0.2, status=401} 2
```
This is not a proper example, because OAP doesn't persistent data like this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
pg-yang commented on issue #11982:
URL: https://github.com/apache/skywalking/issues/11982#issuecomment-1980247042
To build service level metrics with instance/node level metrics through
`AggregateLabels Operation`, I think `AggregateLabels` should support
aggregation through specified labels. Like the scenario below:
instance/node HTTP status metrics
```
# The number means the occurred times
http_status{host=192.168.0.1, status=401} 5
http_status{host=192.168.0.3, status=401} 1
http_status{host=192.168.0.2, status=401} 2
```
The MQE express to aggregate service level metrics should be
```
aggregate_labels(http_status,'status',sum) # aggregate by status label
```
The result will be
```
http_status{status=401} 8
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
wankai123 commented on issue #11982: URL: https://github.com/apache/skywalking/issues/11982#issuecomment-1979901855 Kafka monitoring service's MAL aggregates the metrics with `cluster and broker`, so it's good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
innerpeacez commented on issue #11982: URL: https://github.com/apache/skywalking/issues/11982#issuecomment-1978109337 ES checked, the metrics data from one cluster-level API. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
wu-sheng commented on issue #11982: URL: https://github.com/apache/skywalking/issues/11982#issuecomment-1977919068 For all the owners, this list doesn't mean all monitoring at service level is inaccurate due to the way od scraping from OTEL. Please recheck first and only fix if it fits the bug descriptions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
wu-sheng commented on issue #11982: URL: https://github.com/apache/skywalking/issues/11982#issuecomment-1977891270 @CodePrometheus If you could help this, please let me know. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Bug] Metrics are not aggregated properly as OTEL collector targeting per node [skywalking]
wu-sheng opened a new issue, #11982: URL: https://github.com/apache/skywalking/issues/11982 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/skywalking/issues?q=is%3Aissue) and found no similar issues. ### Apache SkyWalking Component OAP server (apache/skywalking) ### What happened This issue was found during Clickhouse PR review, https://github.com/apache/skywalking/pull/11966#pullrequestreview-1912778007 We noticed this is a common bug case in previous monitoring. We should check them and update the status here. - [ ] MongoDB @weixiang1862 - [ ] MySQL @wankai123 - [ ] MariaDB - [ ] PostgreSQL - [ ] Redis ### What you expected to happen Service metrics should be labeled values, and only use MQE(query stage aggregation) to merge the data to build metrics accurately. ### How to reproduce Metrics from nodeA and nodeB will overlap with each other. Due to nodeA and nodeB will report data separately. ### Anything else _No response_ ### Are you willing to submit a pull request to fix on your own? - [ ] Yes I am willing to submit a pull request on my own! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
