Re: [prometheus-users] Re: Filter metrics based on timestamp

2021-10-13 Thread Harald Koch
On Wed, Oct 13, 2021, at 13:41, Brian Candler wrote: > Googling for "argos workflow.creationTimestamp" finds > https://argoproj.github.io/argo-workflows/variables/ > > *workflow.creationTimestamp > Workflow creation timestamp formatted in RFC 3339 (e.g. 2018-08-23T05:42:49Z)* > ** > *workflow.crea

Re: [prometheus-users] How to add a static_config to the promethes deployed by helm?

2021-09-17 Thread Harald Koch
On Fri, Sep 17, 2021, at 06:42, kazakaze wrote: > My prometheus is deployed via helm-charts > > > > [...] > > 2. How to automatically modify scrape_configs Since I just pieced th

Re: [prometheus-users] monitoring files created date in a directory

2021-09-15 Thread Harald Koch
On Wed, Sep 15, 2021, at 06:19, ishu...@gmail.com wrote: > I have been looking if there is a way prometheus can monitor file > modification/creation time. I have some crons generating few files everyday > and i want to monitor if these are created every day. If prom can monitor a > directory and

Re: [prometheus-users] Why not to load balance traffic between Prometheus and its Alertmanagers?

2021-06-08 Thread Harald Koch
On Tue, Jun 8, 2021, at 02:25, Julien Pivotto wrote: > On 07 Jun 20:24, 赵金鑫 wrote: > > Thanks, so if i push an alert to two alertmanagers(ex. A and B), and stop > > push alert to A, then i will receive a 'resolved' notification sent from A. > > > > Is that right? > > > Yes. That sounds like a

Re: [prometheus-users] Best way to record system state?

2021-06-08 Thread Harald Koch
On Tue, Jun 8, 2021, at 06:35, John Dexter wrote: > Our software has a single state value indicating if the system is running, > starting, stopping, or stopped (but the software is still actually running). > I would like to record this as a metric and I can't decide if recording a > mapping to ke

Re: [prometheus-users] Reg: Error on ingesting out-of-order samples

2021-04-12 Thread Harald Koch
On Thu, Apr 8, 2021, at 22:36, RAJESH DASARI wrote: > is there any way to recover from this issue, without any manual action > (like restarting prometheus). Could some please help me with this ? Upgrade to 2.25.2 or later - most of these out-of-order issues were fixed. (There's still an issue wit

Re: [prometheus-users] bind_exporter flags in Docker Compose

2021-04-09 Thread Harald Koch
On Fri, Apr 9, 2021, at 10:05, Mohamed Boukhlif wrote: > But the container keeps restarting, and I have this error in the logs : > "bind-exporter_1 | bind_exporter: error: unknown short flag '-b', try --help" The most common thing to try whenever you see this error is to use two dashes (which

Re: [prometheus-users] rpm repo link in node_exporter readme

2021-03-23 Thread Harald Koch
On Tue, Mar 23, 2021, at 08:33, Ben Kochie wrote: > Hrm, good question. It's not maintained by Prometheus officially, someone > else supports this. > > It looks like there's also https://github.com/lest/prometheus-rpm, which > provides packages via https://packagecloud.io/prometheus-rpm/release

Re: [prometheus-users] Support Forums for Prometheus

2021-03-17 Thread Harald Koch
On Tue, Mar 16, 2021, at 12:18, Greg Beyer wrote: > I’d like to suggest Prometheus considered setting up a forum for support and > community conversation, Somethinng like phpBB or SimpleMachines. This debate (forum vs. mailling list vs. newsgroup) has been a constant since the dawn of the inter

Re: [prometheus-users] Monitoring NFS service

2020-12-15 Thread Harald Koch
On Tue, Dec 15, 2020, at 09:59, Srikanth Reddy K wrote: > could you please give me those startup flags! https://github.com/prometheus/node_exporter#collectors -- Harald -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from t

Re: [prometheus-users] The possibility of multiple push gateways

2020-11-20 Thread Harald Koch
On Fri, Nov 20, 2020, at 07:54, Stuart Clark wrote: > A "normal" push gateway setup, with auto-restart in case of failure > (e.g. Kubernetes, systemd, auto scaling group), optionally with state > storage on disk is usually adequate to handle failures (which should be > pretty rare). If there was

Re: [prometheus-users] suspending a VM that is running prometheus?

2020-11-07 Thread Harald Koch
On Thu, Nov 5, 2020, at 03:54, Ben Kochie wrote: > For working hypervisor combinations, running NTP on the guest VM is > unnecessary. For example, with KVM/QEMU, the guest can sync directly with the > hypervisor. Thanks for your reply! I agree, but the the state of the art continues to be that

[prometheus-users] suspending a VM that is running prometheus?

2020-11-04 Thread Harald Koch
I'm running prometheus at home, to monitor a few VMs on my homelab, as well as my Linodes. My homelab hypervisor runs Arch Linux and libvirtd. Sometimes I need to patch the OS on the hypervisor and reboot it. The process up until now has been to suspend the guests, reboot the server, and restore

Re: [prometheus-users] Re: disable http method OPTIONS

2020-10-30 Thread Harald Koch
On Fri, Oct 30, 2020, at 05:45, Vincent Pek wrote: > agree on that.. but my company policy states that even for info/low I need to > seek waiver to close it off.. So ... seek a waiver? Or better - get your security team to disable this particular check, since it's both useless (attackers can jus

Re: [prometheus-users] Re: blackbox exporter's probe_ssl_earliest_cert_expiry giving negative values

2020-10-20 Thread Harald Koch
On Tue, Oct 20, 2020, at 08:52, Nemanja Delic wrote: > In this case the cert has expired on Oct 13 22:12:47 2020 GMT. Since it's not > last one in chain it's not breaking things. I can set the filter accordingly > ( with smth like > -7), just thought there might be some other solution. PKIX pat

Re: [prometheus-users] Monitoring NFS service

2020-10-19 Thread Harald Koch
On Mon, Oct 19, 2020, at 09:32, Srikanth Reddy K wrote: > Guys do anyone know how to monitor nfs service through pometheus There are collectors for 'nfs' and 'nfsd' built-in to the node_exporter, but IIRC they're not enabled by default; you just need to change the startup flags for the exporter

Re: [prometheus-users] Active alertnames from Prometheus alertmanager

2020-09-12 Thread Harald Koch
On Fri, Sep 11, 2020, at 08:56, neel patel wrote: > > But is there any way we can display all the active alerts with names in the > grafana dashboard in tabular format, how to fetch those data from the alert > manager ? I use this: https://github.com/camptocamp/grafana-prometheus-alertmanager-

Re: [prometheus-users] custom http get api query for rabbitmq queue length

2020-08-28 Thread Harald Koch
On Fri, Aug 28, 2020, at 09:38, ishu...@gmail.com wrote: > I have rabbitmq running with aggregated metrics. I want to monitor queue > level metrics to alert when a certain queue's threshold is breached. Enabling > detailed logging of rabbitmq is resource intensive while I just need queue > leng

Re: [prometheus-users] Remove Selective Labels : Slack Alerts

2020-06-26 Thread Harald Koch
The Go template 'eq' function has a special feature that lets you list multiple arguments, so you can change your if statement to: {{-if (eq .Name "name1" "name2" "name3" }}{{ else -}} • *{{- .Name }}:* `{{- .Value }}` {{ end -}} As you add more exclusions this gets more concise and easier to

Re: [prometheus-users] Re: Can I disable SSL validation for prometheus-nats-exporter

2020-06-10 Thread Harald Koch
On Wed, Jun 10, 2020, at 15:29, Mark Leone wrote: > Well, looking at the docker build, there is only a single executable in the > image, so the CAs are apparently baked in. I tried, just in case, using the > arg -tlscacert to point to my ca file, but that's advertised as being for > client conne

Re: [prometheus-users] Is it safe to grab metrics via JMX in production?

2020-06-04 Thread Harald Koch
On Thu, Jun 4, 2020, at 20:19, Cameron Kerr wrote: > > True, the default configuration is unhelpful (harmful even) in that it grabs > everything, and the provided examples should do a better job of showing > something more production-ready. The whitelistObjectNames and > blacklistObjectNames ar

Re: [prometheus-users] Is it safe to grab metrics via JMX in production?

2020-06-04 Thread Harald Koch
On Thu, Jun 4, 2020, at 08:10, Cameron Kerr wrote: > You should be sure to use the whitelist and blacklist features; browse the > available mbean objects using jconsole or similar to find just the bits you > will be interested in, and then query only those. But that's my point - the JMX exporter

Re: [prometheus-users] Alertmanager: how to split alerts in email notifications

2020-06-04 Thread Harald Koch
On Thu, Jun 4, 2020, at 11:09, anya...@gmail.com wrote: > Hi, colleagues! > I setup and configured Alertmanager for getting email notifications, but all > of my alerts about different environments are gathered in one email and > sorted by alphabetical order by job_name. In test environment there

Re: [prometheus-users] Error shutting down ActiveMQ with JMX Exporter

2020-05-19 Thread Harald Koch
On Tue, May 19, 2020, at 18:00, Brad Pridgeon wrote: > I'm testing this JMX Exporter > with ActiveMQ to get metrics. Per instructions in this post, I setup the jar > as a Java agent. >

Re: [prometheus-users] "Sectigo AddTrust External CA Root" and the "probe_ssl_earliest_cert_expiry"

2020-05-12 Thread Harald Koch
On Tue, May 12, 2020, at 18:34, Julian van den Berkmortel wrote: > It's in regards to the "probe_ssl_earliest_cert_expiry" metric which uses the > date of the earliest expiring certificate in the chain as its value. > Its value at the moment is the 30th of May because the root certificate is > th

Re: [prometheus-users] PromQL: comparison to custom metric and/or static threshold

2020-05-11 Thread Harald Koch
On Mon, May 11, 2020, at 11:47, Justin W wrote: > Hi all, > > I'm trying to compare the calculated filesystem used to either a static > threshold, or a custom threshold if a certain metric is present. What I have > so far is: > > ((avg(node_filesystem_used_bytes{mountpoint=~".*foo.*"}) by > (s

Re: [prometheus-users] Re: SSL Expiry panel in the blackbox exporter dashboard shows "YES" and no dates of when SSL will expire.

2020-05-10 Thread Harald Koch
On Sun, May 10, 2020, at 06:25, Pooja Chauhan wrote: > Hi , > I am using the query as mentioned by you : > (probe_ssl_earliest_cert_expiry{job="blackbox"} - time())/86400 but it gives > me value in minutes and not in days in grafana. > > Whereas the output of prometheus is different .Pls suggest

Re: [prometheus-users] Re: [prometheus-developers] Looking for a maintainer for the collectd_exporter

2020-05-09 Thread Harald Koch
On Sat, May 9, 2020, at 04:23, Ben Kochie wrote: > On Fri, May 8, 2020, 16:10 Julius Volz wrote: >> By some historic accident I became the maintainer of the collectd exporter >> (https://github.com/prometheus/collectd_exporter), although I have never >> used collectd myself and know very little

Re: [prometheus-users] One Prometheus per Observed System vs. One Prometheus for Everything

2020-05-06 Thread Harald Koch
On Wed, May 6, 2020, at 03:46, Stuart Clark wrote: > The other big reason to split into multiple servers is organisational rather > than technical. We run Prometheus Alertmanager Grafana sets on the load balancers for each of our environments (DEV/TEST/PROD). This allows us to, for example, re

Re: [prometheus-users] collectd vs node-exporter

2020-04-29 Thread Harald Koch
On Thu, Apr 23, 2020, at 16:19, Brian Candler wrote: > FWIW, there's a node_exporter textfile collector script for SMART stats > including drive temperature: > https://github.com/prometheus-community/node-exporter-textfile-collector-scripts/blob/master/smartmon.sh > > Also there are a couple of d

Re: [prometheus-users] collectd vs node-exporter

2020-04-23 Thread Harald Koch
On Wed, Apr 22, 2020, at 19:17, Matt Zagrabelny wrote: > Greetings, > > I'm a brand new Prometheus user. Please excuse the question if it is foolish. > > How much of a difference is there between collectd (and its exporter) and the > node-exporter? > > Do folks use both on their systems or do b

Re: [prometheus-users] API/script to ADD/MODIFY/DELETE the routes/receivers in the alertmanager.yaml

2020-04-17 Thread Harald Koch
On Fri, Apr 17, 2020, at 13:04, Aniv Maskay wrote: > > Can you please share how have you achieved this level of automation? To manage configuration change history and automatically deploy them into our infrastructure, We use: * bitbucket * jenkins * ansible There are a million tools in this

Re: [prometheus-users] Giving different Dynamic Thresholds for the same alert.

2020-03-14 Thread Harald Koch
iated. -- Harald Koch c...@pobox.com -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discus

Re: [prometheus-users] Giving dynamic thresholds in alertmanager.

2020-03-11 Thread Harald Koch
On Wed, Mar 11, 2020, at 13:00, Yagyansh S. Kumar wrote: > Hi. I have configured alert for CPU Load for my servers and my current > threshold is 8 for warning and 10 for critical. > I want to make this threshold dynamic i.e I want the critical alert when the > CPU Load becomes greater than the

Re: [prometheus-users] Newbie having trouble with regex escaping in blackbox exporter

2020-03-07 Thread Harald Koch
On Sat, Mar 7, 2020, at 12:48, TommyG wrote: > That was exactly what it was. > > Many thanks for the answer and the explanation. Your timing was perfect - I've spent the last couple of weeks replacing the ACK generator in our engine, so it's all fresh in my brain! -- Harald -- You received

Re: [prometheus-users] Newbie having trouble with regex escaping in blackbox exporter

2020-03-07 Thread Harald Koch
On Sat, Mar 7, 2020, at 11:49, TommyG wrote: > > fail_if_body_not_matches_regexp: > - .*MSA|*AE*|123456$ This matches anything containing "MSA" or "AE" or "123456", which is why it's matching the AA response also. > > fail_if_body_not_matches_regexp: > - .*MSA\|AE\|123456$ My first suspicion

Re: [prometheus-users] Prometheus HA strategies

2020-02-27 Thread Harald Koch
On Thu, Feb 27, 2020, at 07:36, Riyan Shaik wrote: > > I believe the downside or a corner case with the prometheus instances behind > LB is, if one of the instances goes down, then you may have gaps in your > graphs and you can't backfill (not supported by prom) the data once that > instance co

Re: [prometheus-users] Prometheus HA strategies

2020-02-26 Thread Harald Koch
On Wed, Feb 26, 2020, at 14:01, Riyan Shaik wrote: > a) With a HA pair, the prometheus data will be local to both the prometheus > instances. Is it a good idea to have these 2 prometheus instances write to > some sort of a network mounted filesystem like NFS / GlusterFS filesystem,so > that the