Like a have a cluster A which has the entire monitoring stack: Prometheus,
Grafana, Node Exporter and cAdvisor.
And I have 2 other clusters B and C which has other services.
I want to retrieve the node_id and the node_name of these 2 cluster's nodes.
This is why I don't understand how can I retrieve metrics from both node's
cluster into my cluster A which has the Prom.
In the previous way I did that:
- job_name: 'clusterB-node-exporter'
scrape_interval: 5s
dns_sd_configs:
- names:
- 'node-01'
- 'node-02'
- 'node-03'
- 'node-04'
type: 'A'
port: 9100
- job_name: 'clusterC-node-exporter'
scrape_interval: 5s
dns_sd_configs:
- names:
- 'node-01'
- 'node-02'
- 'node-03'
- 'node-04'
type: 'A'
port: 9100
Do you understand what I want to do now?
On Wednesday, 29 July 2020 20:20:05 UTC+2, Julien Pivotto wrote:
>
> On 29 Jul 11:05, Tom Kun wrote:
> > Hi Julien,
> >
> > No error messages in my Prometheus service.
> >
> > OK, but if I need the Swarm nodes (like the node_id and the node_name)
> > informations from the other Swarm clusters?
> >
> > I read that part of the Docker's documentation about exposing service
> port
> > to be able to retrieve containers metrics.
>
> Can you point me to the relevant docs? What do you mean by other swarm
> clusters?
>
> >
> > On Wednesday, 29 July 2020 19:50:06 UTC+2, Julien Pivotto wrote:
> > >
> > > On 29 Jul 10:29, Tom Kun wrote:
> > > > Hello folks,
> > > >
> > > > I'm a beginner with Swarm and Prometheus and I wanted to know if
> with
> > > the
> > > > new Prometheus release
> > > > <
> > >
> https://prometheus.io/docs/guides/dockerswarm/#docker-swarm-service-discovery-architecture>
>
>
> > >
> > > > (v.2.20.0) and the docker_sd_configs feature.is it possible to get
> > > metrics
> > > > from different cluster to my Prometheus Swarm cluster.
> > > >
> > > > I already follow the documentation's example and I try to propagate
> the
> > > > /etc/docker/daemon.json changes on each Swarm nodes but in Prom' I
> still
> > > > only can see metrics from the Prometheus...
> > > >
> > > > Did I miss something to be able to retrieve metrics from other
> cluster?
> > > >
> > > > This is a bunch of configuration : prometheus.yml-
> > > >
> > > > global:
> > > > scrape_interval: 15s
> > > > scrape_timeout: 10s
> > > > evaluation_interval: 15s
> > > > external_labels:
> > > > monitor: monitoring
> > > >
> > > > rule_files:
> > > > - rules/alerts.yml
> > > > - rules/node.yml
> > > > - rules/stack.yml
> > > >
> > > > alerting:
> > > > alertmanagers:
> > > > - static_configs:
> > > > - targets:
> > > > - alertmanager:9093
> > > >
> > > > scrape_configs:
> > > > - job_name: prometheus
> > > > static_configs:
> > > > - targets:
> > > > - prometheus:9090
> > > >
> > > > - job_name: 'docker-daemon'
> > > > dockerswarm_sd_configs:
> > > > - host: unix:///var/run/docker.sock
> > > > role: nodes
> > > > relabel_configs:
> > > > # Fetch metrics on port 9323.
> > > > - source_labels: [__meta_dockerswarm_node_address]
> > > > target_label: __address__
> > > > replacement: $1:9323
> > > > - source_labels: [__meta_dockerswarm_node_hostname]
> > > > target_label: instance
> > > >
> > > > - job_name: 'docker-swarm'
> > > > dockerswarm_sd_configs:
> > > > - host: unix:///var/run/docker.sock
> > > > role: tasks
> > > > relabel_configs:
> > > > - source_labels: [__meta_dockerswarm_task_desired_state]
> > > > regex: running
> > > > action: keep
> > > > - source_labels:
> [__meta_dockerswarm_service_label_prometheus_job]
> > > > target_label: job
> > > > - source_labels: [__meta_dockerswarm_node_hostname]
> > > > target_label: node_name
> > > > - source_labels: [__meta_dockerswarm_node_id]
> > > > target_label: node_id
> > > >
> > > > Thank you in advance for your time.
> > >
> > > Do you have any error message?
> > >
> > > Also, the implementation in v2.20.0 only discover tasks with published
> > > ports
> > > (
> > >
> https://docs.docker.com/engine/swarm/services/#publish-a-services-ports-directly-on-the-swarm-node
>
> > > )
> > > which is something that will change in the next release.
> > >
> > > >
> > > > Regards,
> > > > Thomas
> > > >
> > > > --
> > > > You received this message because you are subscribed to the Google
> > > Groups "Prometheus Users" group.
> > > > To unsubscribe from this group and stop receiving emails from it,
> send
> > > an email to [email protected] <javascript:>.
> > > > To view this discussion on the web visit
> > >
> https://groups.google.com/d/msgid/prometheus-users/b028ee8c-c557-494b-bce4-3d0809ac08bao%40googlegroups.com.
>
>
> > >
> > >
> > >
> > > --
> > > Julien Pivotto
> > > @roidelapluie
> > >
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "Prometheus Users" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to [email protected] <javascript:>.
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-users/f126e45f-288a-440e-8680-481c6ee8763ao%40googlegroups.com.
>
>
>
>
> --
> Julien Pivotto
> @roidelapluie
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/779e65a9-7352-460f-8a9c-2c1be4b810b6o%40googlegroups.com.
