Re: [prometheus-users] left join queries with different labels

Hi, I guess you would need to rename the namespace label of the second query to exported_namespace. or vice versa. try this example thanks On Mon, Sep 28, 2020 at 10:54 PM nesa...@gmail.com wrote: > Hi > I would like to join two queries, one of them

Re: [prometheus-users] Prometheus not restoring Alert state on restart.

Thanks for the quick response. I get your point On Fri, Jun 26, 2020 at 1:57 PM Brian Brazil < brian.bra...@robustperception.io> wrote: > On Fri, 26 Jun 2020 at 09:40, soha...@gmail.com > wrote: > >> Say an alert is in firing state then after a few minutes, I restart the >> Prometheus. After the

Re: [prometheus-users] External labels overwritten in alerts

> > or are they lost in the expression evaluation, and then replaced by the > labels from the Prometheus that evaluates and sends the alert? I guess this is what's happening probably. Is there any way to keep the original external labels in the final alert I guess your best bet is to either chan

Re: [prometheus-users] Prometheus memory usage keeps increasing over time.

Thanks for the quick response, Julien. I Will update to 2.17.2 and report back. Cheers! On Friday, 22 May 2020 16:04:12 UTC+5, Julien Pivotto wrote: > > On 22 May 03:38, Sohaib Omar wrote: > > Hi all, I am running Prometheus *2.17.1 *as a stateful set inside the > > Kubernet

[prometheus-users] Prometheus memory usage keeps increasing over time.

Hi all, I am running Prometheus *2.17.1 *as a stateful set inside the Kubernetes cluster. I want to debug what's causing Prometheu's memory usage to increase over time even when cardinality or unique time series are constant over time. How can I make sure that Prometheus is not leaking memory?

[prometheus-users] node_exporter not able to discover expected filesystem devices

Hi all, node_exporter is not able to return *filesystem_bytes *except devices that are mounted on "*/" *below are the filesystem devices on my *azure *instance. Filesystem Type 1K-blocksUsed Available Use% Mounted on udev devtmpfs 8167128 0 8167128

Re: [prometheus-users] remote_read: inconsistency between remote storage(influxDB) and Prometheus results.

there is a command-line parameter called *lookback-delta *which controls the loopback. read more about it here: https://prometheus.io/docs/prometheus/latest/migration/#flags On Monday, 11 May 2020 12:26:26 UTC+5, Sohaib Omar wrote: > > Thanks, Stuart for the reply, is the loo

Re: [prometheus-users] remote_read: inconsistency between remote storage(influxDB) and Prometheus results.

Thanks, Stuart for the reply, is the look back configurable? I need my Prometheus queries result to be in sync with my remote storage. On Monday, 11 May 2020 12:11:06 UTC+5, Stuart Clark wrote: > > On 11/05/2020 08:06, Sohaib Omar wrote: > > > For example, even with the *read_rece

Re: [prometheus-users] Re: Prometheus remote write drop.

if *rate(prometheus_remote_storage_samples_in_total[10m])* and *rate(prometheus_remote_storage_succeeded_samples_total[10m]) *does not match, does this means that some of the series fail to be writtent into remote storage. For example in my case the above queries result is. rate(prometheus_remot

Re: [prometheus-users] remote_write option for specific job

You may use write_relabel config at remote write time. https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write for example: remote_write: - url: write_relabel_configs: - source_labels: ["__name__"] regex: "|" replacement: "$1"

[prometheus-users] Prometheus remote write drop.

Hey all, I am using Prometheus remote write to write metrics into remote storage(InfluxDB). I am trying to benchmark the Prometheus remote_writer, for that purpose is there any way to achieve below: 1. Calculate the ingestion rate/second into the remote storage. 2. the drop rate of seri

Re: [prometheus-users] Node Exporter have wrong nodename in node_uname_info.

Got it now, thanks once again, Julius. On Friday, 24 April 2020 18:47:55 UTC+5, Julius Volz wrote: > > On Fri, Apr 24, 2020 at 2:37 PM Sohaib Omar > wrote: > >> Hi all, >> I am stuck in some weird situation. in metric *node_uname_info *I get >> the same va

[prometheus-users] Node Exporter have wrong nodename in node_uname_info.

Hi all, I am stuck in some weird situation. in metric *node_uname_info *I get the same value for nodename and pod label. node_uname_info{domainname="(none)",endpoint="metrics",instance= "x.x.x.x:9100",job="node-exporter",machine="x86_64",namespace="monitoring", nodename="node-exporter-ljhlp",po

Re: [prometheus-users] Finding number of Unique Label Pairs across all time series.

> count by() ({=~".+"}) > > You could write a small script / program that could do this via the API I > guess. > > On Fri, Apr 17, 2020 at 10:41 AM Sohaib Omar wrote: > >> Hi all, I was going through this blog : >> https://www.robustperception.io/how-much-

[prometheus-users] Finding number of Unique Label Pairs across all time series.

Hi all, I was going through this blog : https://www.robustperception.io/how-much-ram-does-prometheus-2-x-need-for-cardinality-and-ingestion to calculate Ram capacity for our Prometheus setup. I am bit confused about how to find *unique label pairs across all time-series *and *Avg byte per labe

[prometheus-users] External Labels not accessible in alert rules.

Hi, I have set external labels as below in my Prometheus configuration. externalLabels: deployment_env: dev org_id: myOrganization ... and in my rules.yaml I am trying to access external labels like below. - alert: ClusterCapacityWarning annotations: summary: "Available capacity of cluste

[prometheus-users] Prometheus remote read and Influx retention policies.

Hi all, I have InfluxDB set up for remote write/read storage for Prometheus. I lack a bit of an understanding of how does Prometheus decides which retention policy to use? Or does it even use one? Let's say below is my remote read config apiVersion: monitoring.coreos.com/v1 kind: Prometheus ..