[prometheus-users] too old sample - Error appending remote write

Hi All, We are using prometheus version 2.48.0 and enabled out_of_order_time_window to 1d. Promethues A and Prometheus B are in same version Prometheus A normally remote writes to Prometheus B. Prometheus A receives data from a fleet of Prometheus Agents . Prometheus A was done for 45 minut

[prometheus-users] Re: Drop metrics based on a different metric

abel_replace function instead of the static label implementation . Is this the right way to do it or is there any other better options that you can suggest us with. Thank You On Friday, 1 December 2023 at 17:33:22 UTC+8 rksakt...@gmail.com wrote: > Hi All, > > We want to drop set of met

[prometheus-users] Drop metrics based on a different metric

Hi All, We want to drop set of metrics where the pod is created cronjobs and jobs (shortlived pods) kube_pod_info has a label created_by_kind with value as "". I have another set of metrics such a kube_pod_status_phase , kube_pod_status_reason etc. Now I want to drop all the three metrics but

[prometheus-users] Clarification on Prometheus built-in remote receive endpoint

Hi All, We have enabled Prometheus remote receive endpoint "--web.enable-remote-write-receiver" in our Prometheus instance and trying OTEL Prometheus remote write exporter to push the data to our Prometheus instance. At OTEL/Prometheus end we are not seeing any error even with debug log enab

[prometheus-users] Metric Relabeling

Hi All, Our Java applications are exposing exposing application metrics using micrometer framework and we have the http_server_requests_bucket getting created based on the response time of the requests. Since the number of microservices are in 1000's we are already having more than 1 million t

[prometheus-users] Remote Write Kuberenetes Mixin Recorded Rules

Hi All, I'm trying to remote write kubernetes Mixin Recorded K8's rules and additional cAdvisor metrics exposed by prometheus operator. The config reload happens properly and I can see the below metric names in prometheus. The prometheus logs says Done replaying WAL but i don't see the metric o