Ok Antonin... this is what I'm trying: https://github.com/prometheus-operator/prometheus-operator/issues/4428
But they are telling me to change the PodMonitor "selector", but this definition is created by camel-k ... the only thing I can change is the label of the PodMonitor resource according to the doc https://camel.apache.org/camel-k/1.7.x/observability/monitoring/integration.html#_discovery. I also can disable the PodMonitor with the trait -t prometheus.pod-monitor=false... but this is not an option to me.... I don't want to manage the PodMonitors each time I run an integration. I'm running out of options... On Tue, Nov 30, 2021 at 10:42 AM Antonin Stefanutti <anto...@stefanutti.fr.invalid> wrote: > > As you've pointed it, the problem is that the Prometheus operator does not > update the prometheus configuration. > > That is consistent with why restarting fixes the issue, as it forces an > update. > > There may be some configuration parameters to update in the Prometheus > resource. It's strictly related to the Prometheus operator, so you'll have > better answer if you ask the Prometheus operator team directly. > > On 30 Nov 2021, at 14:34, Roberto Camelk > <betonetotbo.cam...@gmail.com<mailto:betonetotbo.cam...@gmail.com>> wrote: > > I created a simples integration and enabled the prometheus > integration, as explained here: > https://camel.apache.org/camel-k/1.4.x/observability/integration.html#_discovery > > Before that, I also configured my Prometheus operator resource, as > explained here: > https://camel.apache.org/camel-k/1.7.x/observability/monitoring.html#kubernetes > > The problem is... > > When I start my integration (platform/micro-profile-metrics2), and > RESTART my prometheus operator it discovers my integration PodMonitor, > and regenerate the prometheus config and sync it. This the the > prometheus operator log: > > level=debug ts=2021-11-30T13:11:50.399141448Z caller=operator.go:1853 > component=prometheusoperator msg="selected PodMonitors" > podmonitors=platform/micro-profile-metrics2,platform/create-role-route > namespace=cattle-prometheus prometheus=cluster-monitoring > level=debug ts=2021-11-30T13:11:50.42285181Z caller=operator.go:1677 > component=prometheusoperator msg="updating Prometheus configuration > secret skipped, no configuration change" > > But, when I don't restart the prometheus operator and run a new > integration in camel-k, the only thing in prometheus operator log is: > > level=debug ts=2021-11-30T13:16:08.094103259Z caller=operator.go:748 > component=prometheusoperator msg="PodMonitor updated" > > And even after a while, the prometheus operator never fire the > prometheus config regeneration, and the prometheus never start to > scrape the metrics of this new pod. Only if I restart the prometheus > operator it starts to scrape the pod. >
