Re: [prometheus-users] AM Executor on Kubernetes Cluster

[Test Kumar]

Is there any kubernetes configuration for the same.

On Tuesday, July 5, 2022 at 5:32:40 PM UTC+5:30 Test Kumar wrote:

> yes this same one
>
> On Tuesday, July 5, 2022 at 2:45:12 PM UTC+5:30 Brian Candler wrote:
>
>> Are you talking about this third-party package?
>> https://github.com/imgix/prometheus-am-executor
>>
>> If so you'll need to run prometheus alertmanager in one container, and 
>> this tool in another.  If this tool hasn't already been dockerized, then 
>> you'll need to dockerize it yourself.
>>
>> On Tuesday, 5 July 2022 at 09:44:02 UTC+1 test1...@gmail.com wrote:
>>
>>> Yes, Alert Manager Executor which runs custom script for any alert.
>>> SO how the docker image will work for me??
>>>
>>> On Tuesday, July 5, 2022 at 1:17:32 PM UTC+5:30 Stuart Clark wrote:
>>>
 What are you meaning by "AM executor"? Are you meaning the Alertmanager 
 application?

 If so, there are both community Helm charts and a Docker image 
 available, so running within Kubernetes should be no problem. 

 On 5 July 2022 08:03:45 BST, Test Kumar  wrote:
>
> Hi Team,
>
> I installed the AM executor as a stand-alone application.
> But I want to install it as a Kubernetes pod service same as 
> Kubernetes Prometheus.
> So is there a way to install AM executor on Kubernetes?
>
> Thanks & Regards,
> Test Kumar 
>
> -- 
 Sent from my Android device with K-9 Mail. Please excuse my brevity.

>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/7b928663-b369-4958-a47c-4a9b61de1b64n%40googlegroups.com.

Re: [prometheus-users] AM Executor on Kubernetes Cluster

[Test Kumar]

yes this same one

On Tuesday, July 5, 2022 at 2:45:12 PM UTC+5:30 Brian Candler wrote:

> Are you talking about this third-party package?
> https://github.com/imgix/prometheus-am-executor
>
> If so you'll need to run prometheus alertmanager in one container, and 
> this tool in another.  If this tool hasn't already been dockerized, then 
> you'll need to dockerize it yourself.
>
> On Tuesday, 5 July 2022 at 09:44:02 UTC+1 test1...@gmail.com wrote:
>
>> Yes, Alert Manager Executor which runs custom script for any alert.
>> SO how the docker image will work for me??
>>
>> On Tuesday, July 5, 2022 at 1:17:32 PM UTC+5:30 Stuart Clark wrote:
>>
>>> What are you meaning by "AM executor"? Are you meaning the Alertmanager 
>>> application?
>>>
>>> If so, there are both community Helm charts and a Docker image 
>>> available, so running within Kubernetes should be no problem. 
>>>
>>> On 5 July 2022 08:03:45 BST, Test Kumar  wrote:

 Hi Team,

 I installed the AM executor as a stand-alone application.
 But I want to install it as a Kubernetes pod service same as Kubernetes 
 Prometheus.
 So is there a way to install AM executor on Kubernetes?

 Thanks & Regards,
 Test Kumar 

 -- 
>>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/af98c343-7bb7-4be2-9be3-48cb2845d561n%40googlegroups.com.

Re: [prometheus-users] implement smtp connection with blackbox exporter

[nina guo]

Thank you Brian for your great help.

On Tuesday, July 5, 2022 at 5:10:24 PM UTC+8 Brian Candler wrote:

> > My blackbox is running as a pod in a k8s cluster. Do you have good 
> suggestion to store the cert file?
>
> A configMap is how I'd do it.
>
> On Tuesday, 5 July 2022 at 09:21:23 UTC+1 ninag...@gmail.com wrote:
>
>> Added as below:
>>
>> smtp_starttls:
>> prober: tcp
>> timeout: 20s
>> tcp:
>>   tls_config:  # add here?
>>   ca_file: "/certs/my_cert.crt" # add here?
>>
>>   query_response:
>> - expect: "^220 ([^ ]+) ESMTP"
>>   send: "EHLO prober\r"
>> - expect: "^250-STARTTLS"
>> - expect: "^250 .*$"
>>   send: "STARTTLS\r"
>> - expect: "^220"
>>   starttls: true
>> - send: "EHLO prober\r"
>> - expect: "^250 .*$"
>>   send: "QUIT\r"
>>
>> My blackbox is running as a pod in a k8s cluster. Do you have good 
>> suggestion to store the cert file?
>> On Tuesday, July 5, 2022 at 4:09:08 PM UTC+8 nina guo wrote:
>>
>>> Thank you very much.
>>>
>>> I changed to connect with DNS name. So according to the error msg, 
>>> following your kind suggestion, I need to go with [^2], am I right?
>>>
>>> ts=2022-07-05T07:32:17.495596017Z caller=main.go:130 
>>> module=smtp_starttls target=mailserver:25 level=error msg="TLS Handshake 
>>> (client) failed" err="x509: certificate signed by unknown authority"
>>>
>>> On Tuesday, July 5, 2022 at 3:58:09 PM UTC+8 Brian Candler wrote:
>>>
 Problem 1: timeout / regexp mismatch

 > ts=2022-07-05T02:29:41.609572861Z caller=main.go:130 
 module=smtp_starttls target=1.1.1.1:25 level=debug msg="Read line" 
 line="220 mail.com ESMTP"
 > ts=2022-07-05T02:29:46.607798497Z caller=main.go:130 
 module=smtp_starttls target=11.1.1.1:25 level=error msg="Error reading 
 from connection" err="read tcp4 0.0.0.0:57074->1.1.1.1:25: i/o timeout"

 You asked it to match "^220 ([^ ]+) ESMTP (.+)$" which means:

 - "220"
 - 
 - 1 or more 
 - 
 - "ESMTP"
 - 
 - 1 or more 
 - end of line

 But in the banner you got back from the target host, there's nothing 
 after the word "ESMTP": the line ends there immediately.  Hence you were 
 right to change the regexp.

 "I'm not sure if the change is acceptable from blackbox's view"  - 
 Blackbox will do whatever you ask it to do.  If you want to match both 
 cases (i.e. mail servers which do and don't put any text after "ESMTP"), 
 then use a regexp which does that:

 "^220 ([^ ]+) ESMTP( .+)?$"

 or

 "^220 ([^ ]+) ESMTP.*$"

 or even just

 "^220 ([^ ]+) ESMTP"

 (no need to match all the way to the end of the line)

 Problem 2: TLS handshake failed

 * You asked Blackbox Exporter to connect to a host by IP address 
 (1.1.1.1), instead of using a DNS name (smtp.example.com)
 * Therefore, Blackbox Exporter cannot verify the certificate unless the 
 certificate itself contains an IP address

 It's the same as if you put https://1.1.1.1/ in your web browser.  
 There's no way the browser can validate the certificate, unless the 
 certificate itself contains IP address "1.1.1.1".  And that is a very 
 uncommon sort of certificate.

 Please, try *exactly* the command I gave you before:
 *curl -g 
 'localhost:9115/probe?module=smtp_starttls=aspmx.l.google.com:25=true'*

 No changes. Copy-paste exactly that, with the hostname "
 aspmx.l.google.com" in it.  You should find that it verifies 
 correctly, as long as the ESMTP regexp matches OK.

 Then to get it to work with your own mail server, you will need to do 
 exactly the same:
 * Refer to your mail server by its DNS name, not IP address, when 
 asking blackbox_exporter to test it
 * The mail server must have a certificate which contains this name [^1]
 * The mail server's certificate must be signed by a certificate 
 authority that blackbox_exporter trusts [^2]
 * The mail server's certificate must not have expired, i.e. the current 
 time is within its notBefore and notAfter times

 [^1] If you still want to connect to the server by IP address but check 
 for a different name in the certificate, then use the "server_name" 
 setting 
 in tls_config: see 
 https://prometheus.io/docs/prometheus/latest/configuration/configuration/#tls_config

 [^2] If the certificate authority that signed your mail server's 
 certificate is not in the system trust store then use the "ca_file" 
 setting 
 in tls_config: see 
 https://github.com/prometheus/blackbox_exporter/blob/v0.21.1/example.yml#L53-L54

 Or you can give up, and use "insecure_skip_verify: true" to disable 
 certificate validation altogether: 
 

Re: [prometheus-users] AM Executor on Kubernetes Cluster

[Brian Candler]

Are you talking about this third-party package?
https://github.com/imgix/prometheus-am-executor

If so you'll need to run prometheus alertmanager in one container, and this 
tool in another.  If this tool hasn't already been dockerized, then you'll 
need to dockerize it yourself.

On Tuesday, 5 July 2022 at 09:44:02 UTC+1 test1...@gmail.com wrote:

> Yes, Alert Manager Executor which runs custom script for any alert.
> SO how the docker image will work for me??
>
> On Tuesday, July 5, 2022 at 1:17:32 PM UTC+5:30 Stuart Clark wrote:
>
>> What are you meaning by "AM executor"? Are you meaning the Alertmanager 
>> application?
>>
>> If so, there are both community Helm charts and a Docker image available, 
>> so running within Kubernetes should be no problem. 
>>
>> On 5 July 2022 08:03:45 BST, Test Kumar  wrote:
>>>
>>> Hi Team,
>>>
>>> I installed the AM executor as a stand-alone application.
>>> But I want to install it as a Kubernetes pod service same as Kubernetes 
>>> Prometheus.
>>> So is there a way to install AM executor on Kubernetes?
>>>
>>> Thanks & Regards,
>>> Test Kumar 
>>>
>>> -- 
>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/dbdf0ba7-43d3-48d2-8941-b17799bde85en%40googlegroups.com.

Re: [prometheus-users] implement smtp connection with blackbox exporter

[Brian Candler]

> My blackbox is running as a pod in a k8s cluster. Do you have good 
suggestion to store the cert file?

A configMap is how I'd do it.

On Tuesday, 5 July 2022 at 09:21:23 UTC+1 ninag...@gmail.com wrote:

> Added as below:
>
> smtp_starttls:
> prober: tcp
> timeout: 20s
> tcp:
>   tls_config:  # add here?
>   ca_file: "/certs/my_cert.crt" # add here?
>
>   query_response:
> - expect: "^220 ([^ ]+) ESMTP"
>   send: "EHLO prober\r"
> - expect: "^250-STARTTLS"
> - expect: "^250 .*$"
>   send: "STARTTLS\r"
> - expect: "^220"
>   starttls: true
> - send: "EHLO prober\r"
> - expect: "^250 .*$"
>   send: "QUIT\r"
>
> My blackbox is running as a pod in a k8s cluster. Do you have good 
> suggestion to store the cert file?
> On Tuesday, July 5, 2022 at 4:09:08 PM UTC+8 nina guo wrote:
>
>> Thank you very much.
>>
>> I changed to connect with DNS name. So according to the error msg, 
>> following your kind suggestion, I need to go with [^2], am I right?
>>
>> ts=2022-07-05T07:32:17.495596017Z caller=main.go:130 module=smtp_starttls 
>> target=mailserver:25 level=error msg="TLS Handshake (client) failed" 
>> err="x509: certificate signed by unknown authority"
>>
>> On Tuesday, July 5, 2022 at 3:58:09 PM UTC+8 Brian Candler wrote:
>>
>>> Problem 1: timeout / regexp mismatch
>>>
>>> > ts=2022-07-05T02:29:41.609572861Z caller=main.go:130 
>>> module=smtp_starttls target=1.1.1.1:25 level=debug msg="Read line" 
>>> line="220 mail.com ESMTP"
>>> > ts=2022-07-05T02:29:46.607798497Z caller=main.go:130 
>>> module=smtp_starttls target=11.1.1.1:25 level=error msg="Error reading 
>>> from connection" err="read tcp4 0.0.0.0:57074->1.1.1.1:25: i/o timeout"
>>>
>>> You asked it to match "^220 ([^ ]+) ESMTP (.+)$" which means:
>>>
>>> - "220"
>>> - 
>>> - 1 or more 
>>> - 
>>> - "ESMTP"
>>> - 
>>> - 1 or more 
>>> - end of line
>>>
>>> But in the banner you got back from the target host, there's nothing 
>>> after the word "ESMTP": the line ends there immediately.  Hence you were 
>>> right to change the regexp.
>>>
>>> "I'm not sure if the change is acceptable from blackbox's view"  - 
>>> Blackbox will do whatever you ask it to do.  If you want to match both 
>>> cases (i.e. mail servers which do and don't put any text after "ESMTP"), 
>>> then use a regexp which does that:
>>>
>>> "^220 ([^ ]+) ESMTP( .+)?$"
>>>
>>> or
>>>
>>> "^220 ([^ ]+) ESMTP.*$"
>>>
>>> or even just
>>>
>>> "^220 ([^ ]+) ESMTP"
>>>
>>> (no need to match all the way to the end of the line)
>>>
>>> Problem 2: TLS handshake failed
>>>
>>> * You asked Blackbox Exporter to connect to a host by IP address 
>>> (1.1.1.1), instead of using a DNS name (smtp.example.com)
>>> * Therefore, Blackbox Exporter cannot verify the certificate unless the 
>>> certificate itself contains an IP address
>>>
>>> It's the same as if you put https://1.1.1.1/ in your web browser.  
>>> There's no way the browser can validate the certificate, unless the 
>>> certificate itself contains IP address "1.1.1.1".  And that is a very 
>>> uncommon sort of certificate.
>>>
>>> Please, try *exactly* the command I gave you before:
>>> *curl -g 
>>> 'localhost:9115/probe?module=smtp_starttls=aspmx.l.google.com:25=true'*
>>>
>>> No changes. Copy-paste exactly that, with the hostname "
>>> aspmx.l.google.com" in it.  You should find that it verifies correctly, 
>>> as long as the ESMTP regexp matches OK.
>>>
>>> Then to get it to work with your own mail server, you will need to do 
>>> exactly the same:
>>> * Refer to your mail server by its DNS name, not IP address, when asking 
>>> blackbox_exporter to test it
>>> * The mail server must have a certificate which contains this name [^1]
>>> * The mail server's certificate must be signed by a certificate 
>>> authority that blackbox_exporter trusts [^2]
>>> * The mail server's certificate must not have expired, i.e. the current 
>>> time is within its notBefore and notAfter times
>>>
>>> [^1] If you still want to connect to the server by IP address but check 
>>> for a different name in the certificate, then use the "server_name" setting 
>>> in tls_config: see 
>>> https://prometheus.io/docs/prometheus/latest/configuration/configuration/#tls_config
>>>
>>> [^2] If the certificate authority that signed your mail server's 
>>> certificate is not in the system trust store then use the "ca_file" setting 
>>> in tls_config: see 
>>> https://github.com/prometheus/blackbox_exporter/blob/v0.21.1/example.yml#L53-L54
>>>
>>> Or you can give up, and use "insecure_skip_verify: true" to disable 
>>> certificate validation altogether: 
>>> https://github.com/prometheus/blackbox_exporter/blob/v0.21.1/example.yml#L27-L28
>>> (although then you lose valuable tests, because if your certificate was 
>>> valid and becomes invalid you won't know.  I also note that if you can't 
>>> get 

Re: [prometheus-users] AM Executor on Kubernetes Cluster

[Test Kumar]

Yes, Alert Manager Executor which runs custom script for any alert.
SO how the docker image will work for me??

On Tuesday, July 5, 2022 at 1:17:32 PM UTC+5:30 Stuart Clark wrote:

> What are you meaning by "AM executor"? Are you meaning the Alertmanager 
> application?
>
> If so, there are both community Helm charts and a Docker image available, 
> so running within Kubernetes should be no problem. 
>
> On 5 July 2022 08:03:45 BST, Test Kumar  wrote:
>>
>> Hi Team,
>>
>> I installed the AM executor as a stand-alone application.
>> But I want to install it as a Kubernetes pod service same as Kubernetes 
>> Prometheus.
>> So is there a way to install AM executor on Kubernetes?
>>
>> Thanks & Regards,
>> Test Kumar 
>>
>> -- 
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/970071e8-c1e3-4191-b0fe-3cf4100054can%40googlegroups.com.

Re: [prometheus-users] Re: Alerts are getting auto resolved automatically

[Venkatraman Natarajan]

Hi Stuart,

Yes I can see both cluster peers and showing information like the cluster
is ready.

[image: image.png]

Thanks,
Venkatraman N

On Tue, Jul 5, 2022 at 1:19 PM Stuart Clark 
wrote:

> Two alerts suggests that the two instances aren't talking to each other.
> How have you configured them? Does the UI show the "other" instance?
>
> On 5 July 2022 08:34:45 BST, Venkatraman Natarajan 
> wrote:
>>
>> Thanks Brian. I have used last_over_time query in our expression instead
>> of turning off auto-resolved.
>>
>> Also, we have two alert managers in our environment. Both are up and
>> running. But Nowadays, we are getting two alerts from two alert managers.
>> Could you please help me to sort this issue as well.?
>>
>> Please find the alert manager configuration.
>>
>>   alertmanager0:
>> image: prom/alertmanager
>> container_name: alertmanager0
>> user: rootuser
>> volumes:
>>   - ../data:/data
>>   - ../config/alertmanager.yml:/etc/alertmanager/alertmanager.yml
>> command:
>>   - '--config.file=/etc/alertmanager/alertmanager.yml'
>>   - '--storage.path=/data/alert0'
>>   - '--cluster.listen-address=0.0.0.0:6783'
>>   - '--cluster.peer={{ IP Address }}:6783'
>>   - '--cluster.peer={{ IP Address }}:6783'
>> restart: unless-stopped
>> logging:
>>   driver: "json-file"
>>   options:
>> max-size: "10m"
>> max-file: "2"
>> ports:
>>   - 9093:9093
>>   - 6783:6783
>> networks:
>>   - network
>>
>> Regards,
>> Venkatraman N
>>
>>
>>
>> On Sat, Jun 25, 2022 at 9:05 PM Brian Candler 
>> wrote:
>>
>>> If probe_success becomes non-zero, even for a single evaluation
>>> interval, then the alert will be immediately resolved.  There is no delay
>>> on resolving, like there is for pending->firing ("for: 5m").
>>>
>>> I suggest you enter the alerting expression, e.g. "probe_success == 0",
>>> into the PromQL web interface (query browser), and switch to Graph view,
>>> and zoom in.  If you see any gaps in the graph, then the alert was resolved
>>> at that instant.
>>>
>>> Conversely, use the query
>>> probe_success{instance="xxx"} != 0
>>> to look at a particular timeseries, as identified by the label9s), and
>>> see if there are any dots shown where the label is non-zero.
>>>
>>> To make your alerts more robust you may need to use queries with range
>>> vectors, e.g. min_over_time(foo[5m]) or max_over_time(foo[5m]) or whatever.
>>>
>>> As a general rule though: you should consider carefully whether you want
>>> to send *any* notification for resolved alerts.  Personally, I have
>>> switched to send_resolved = false.  There are some good explanations here:
>>>
>>> https://www.robustperception.io/running-into-burning-buildings-because-the-fire-alarm-stopped
>>>
>>> https://docs.google.com/document/d/199PqyG3UsyXlwieHaqbGiWVa8eMWi8zzAn0YfcApr8Q/
>>>
>>> You don't want to build a culture where people ignore alerts because the
>>> alert cleared itself - or is expected to clear itself.
>>>
>>> You want the alert condition to trigger a *process*, which is an
>>> investigation of *why* the alert happened, *what* caused it, whether the
>>> underlying cause has been fixed, and whether the alerting rule itself was
>>> wrong.  When all that has been investigated, manually close the ticket.
>>> The fact that the alert has gone below threshold doesn't mean that this
>>> work no longer needs to be done.
>>>
>>> On Saturday, 25 June 2022 at 13:27:22 UTC+1 v.ra...@gmail.com wrote:
>>>
 Hi Team,

 We are having two prometheus and two alert managers in separate VMs as
 containers.

 Alerts are getting auto resolved even though the issues are there as
 per threshold.

 For example, if we have an alert rule called probe_success == 0 means
 it is triggering an alert but after sometime the alert gets auto-resolved
 because we have enabled send_resolved = true. But probe_success == 0 still
 there so we don't want to auto resolve the alerts.

 Could you please help us on this.?

 Thanks,
 Venkatraman N

>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Prometheus Users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to prometheus-users+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/prometheus-users/68bff458-ee79-42ce-bafb-facd239e26aen%40googlegroups.com
>>> 
>>> .
>>>
>> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To 

Re: [prometheus-users] implement smtp connection with blackbox exporter

[nina guo]

Added as below:

smtp_starttls:
prober: tcp
timeout: 20s
tcp:
  tls_config:  # add here?
  ca_file: "/certs/my_cert.crt" # add here?
  query_response:
- expect: "^220 ([^ ]+) ESMTP"
  send: "EHLO prober\r"
- expect: "^250-STARTTLS"
- expect: "^250 .*$"
  send: "STARTTLS\r"
- expect: "^220"
  starttls: true
- send: "EHLO prober\r"
- expect: "^250 .*$"
  send: "QUIT\r"

My blackbox is running as a pod in a k8s cluster. Do you have good 
suggestion to store the cert file?
On Tuesday, July 5, 2022 at 4:09:08 PM UTC+8 nina guo wrote:

> Thank you very much.
>
> I changed to connect with DNS name. So according to the error msg, 
> following your kind suggestion, I need to go with [^2], am I right?
>
> ts=2022-07-05T07:32:17.495596017Z caller=main.go:130 module=smtp_starttls 
> target=mailserver:25 level=error msg="TLS Handshake (client) failed" 
> err="x509: certificate signed by unknown authority"
>
> On Tuesday, July 5, 2022 at 3:58:09 PM UTC+8 Brian Candler wrote:
>
>> Problem 1: timeout / regexp mismatch
>>
>> > ts=2022-07-05T02:29:41.609572861Z caller=main.go:130 
>> module=smtp_starttls target=1.1.1.1:25 level=debug msg="Read line" 
>> line="220 mail.com ESMTP"
>> > ts=2022-07-05T02:29:46.607798497Z caller=main.go:130 
>> module=smtp_starttls target=11.1.1.1:25 level=error msg="Error reading 
>> from connection" err="read tcp4 0.0.0.0:57074->1.1.1.1:25: i/o timeout"
>>
>> You asked it to match "^220 ([^ ]+) ESMTP (.+)$" which means:
>>
>> - "220"
>> - 
>> - 1 or more 
>> - 
>> - "ESMTP"
>> - 
>> - 1 or more 
>> - end of line
>>
>> But in the banner you got back from the target host, there's nothing 
>> after the word "ESMTP": the line ends there immediately.  Hence you were 
>> right to change the regexp.
>>
>> "I'm not sure if the change is acceptable from blackbox's view"  - 
>> Blackbox will do whatever you ask it to do.  If you want to match both 
>> cases (i.e. mail servers which do and don't put any text after "ESMTP"), 
>> then use a regexp which does that:
>>
>> "^220 ([^ ]+) ESMTP( .+)?$"
>>
>> or
>>
>> "^220 ([^ ]+) ESMTP.*$"
>>
>> or even just
>>
>> "^220 ([^ ]+) ESMTP"
>>
>> (no need to match all the way to the end of the line)
>>
>> Problem 2: TLS handshake failed
>>
>> * You asked Blackbox Exporter to connect to a host by IP address 
>> (1.1.1.1), instead of using a DNS name (smtp.example.com)
>> * Therefore, Blackbox Exporter cannot verify the certificate unless the 
>> certificate itself contains an IP address
>>
>> It's the same as if you put https://1.1.1.1/ in your web browser.  
>> There's no way the browser can validate the certificate, unless the 
>> certificate itself contains IP address "1.1.1.1".  And that is a very 
>> uncommon sort of certificate.
>>
>> Please, try *exactly* the command I gave you before:
>> *curl -g 
>> 'localhost:9115/probe?module=smtp_starttls=aspmx.l.google.com:25=true'*
>>
>> No changes. Copy-paste exactly that, with the hostname "
>> aspmx.l.google.com" in it.  You should find that it verifies correctly, 
>> as long as the ESMTP regexp matches OK.
>>
>> Then to get it to work with your own mail server, you will need to do 
>> exactly the same:
>> * Refer to your mail server by its DNS name, not IP address, when asking 
>> blackbox_exporter to test it
>> * The mail server must have a certificate which contains this name [^1]
>> * The mail server's certificate must be signed by a certificate authority 
>> that blackbox_exporter trusts [^2]
>> * The mail server's certificate must not have expired, i.e. the current 
>> time is within its notBefore and notAfter times
>>
>> [^1] If you still want to connect to the server by IP address but check 
>> for a different name in the certificate, then use the "server_name" setting 
>> in tls_config: see 
>> https://prometheus.io/docs/prometheus/latest/configuration/configuration/#tls_config
>>
>> [^2] If the certificate authority that signed your mail server's 
>> certificate is not in the system trust store then use the "ca_file" setting 
>> in tls_config: see 
>> https://github.com/prometheus/blackbox_exporter/blob/v0.21.1/example.yml#L53-L54
>>
>> Or you can give up, and use "insecure_skip_verify: true" to disable 
>> certificate validation altogether: 
>> https://github.com/prometheus/blackbox_exporter/blob/v0.21.1/example.yml#L27-L28
>> (although then you lose valuable tests, because if your certificate was 
>> valid and becomes invalid you won't know.  I also note that if you can't 
>> get blackbox_exporter to validate the certificate, then your clients 
>> probably won't be able to validate it either)
>>
>> At this point, it isn't really an issue with blackbox_exporter any more: 
>> it's a case of understanding how certificates and certificate validation 
>> work.  
>>
>> On Tuesday, 5 July 2022 at 05:53:46 UTC+1 ninag...@gmail.com wrote:
>>

Re: [prometheus-users] implement smtp connection with blackbox exporter

[nina guo]

Thank you very much.

I changed to connect with DNS name. So according to the error msg, 
following your kind suggestion, I need to go with [^2], am I right?

ts=2022-07-05T07:32:17.495596017Z caller=main.go:130 module=smtp_starttls 
target=mailserver:25 level=error msg="TLS Handshake (client) failed" 
err="x509: certificate signed by unknown authority"

On Tuesday, July 5, 2022 at 3:58:09 PM UTC+8 Brian Candler wrote:

> Problem 1: timeout / regexp mismatch
>
> > ts=2022-07-05T02:29:41.609572861Z caller=main.go:130 
> module=smtp_starttls target=1.1.1.1:25 level=debug msg="Read line" 
> line="220 mail.com ESMTP"
> > ts=2022-07-05T02:29:46.607798497Z caller=main.go:130 
> module=smtp_starttls target=11.1.1.1:25 level=error msg="Error reading 
> from connection" err="read tcp4 0.0.0.0:57074->1.1.1.1:25: i/o timeout"
>
> You asked it to match "^220 ([^ ]+) ESMTP (.+)$" which means:
>
> - "220"
> - 
> - 1 or more 
> - 
> - "ESMTP"
> - 
> - 1 or more 
> - end of line
>
> But in the banner you got back from the target host, there's nothing after 
> the word "ESMTP": the line ends there immediately.  Hence you were right to 
> change the regexp.
>
> "I'm not sure if the change is acceptable from blackbox's view"  - 
> Blackbox will do whatever you ask it to do.  If you want to match both 
> cases (i.e. mail servers which do and don't put any text after "ESMTP"), 
> then use a regexp which does that:
>
> "^220 ([^ ]+) ESMTP( .+)?$"
>
> or
>
> "^220 ([^ ]+) ESMTP.*$"
>
> or even just
>
> "^220 ([^ ]+) ESMTP"
>
> (no need to match all the way to the end of the line)
>
> Problem 2: TLS handshake failed
>
> * You asked Blackbox Exporter to connect to a host by IP address 
> (1.1.1.1), instead of using a DNS name (smtp.example.com)
> * Therefore, Blackbox Exporter cannot verify the certificate unless the 
> certificate itself contains an IP address
>
> It's the same as if you put https://1.1.1.1/ in your web browser.  
> There's no way the browser can validate the certificate, unless the 
> certificate itself contains IP address "1.1.1.1".  And that is a very 
> uncommon sort of certificate.
>
> Please, try *exactly* the command I gave you before:
> *curl -g 
> 'localhost:9115/probe?module=smtp_starttls=aspmx.l.google.com:25=true'*
>
> No changes. Copy-paste exactly that, with the hostname "aspmx.l.google.com" 
> in it.  You should find that it verifies correctly, as long as the ESMTP 
> regexp matches OK.
>
> Then to get it to work with your own mail server, you will need to do 
> exactly the same:
> * Refer to your mail server by its DNS name, not IP address, when asking 
> blackbox_exporter to test it
> * The mail server must have a certificate which contains this name [^1]
> * The mail server's certificate must be signed by a certificate authority 
> that blackbox_exporter trusts [^2]
> * The mail server's certificate must not have expired, i.e. the current 
> time is within its notBefore and notAfter times
>
> [^1] If you still want to connect to the server by IP address but check 
> for a different name in the certificate, then use the "server_name" setting 
> in tls_config: see 
> https://prometheus.io/docs/prometheus/latest/configuration/configuration/#tls_config
>
> [^2] If the certificate authority that signed your mail server's 
> certificate is not in the system trust store then use the "ca_file" setting 
> in tls_config: see 
> https://github.com/prometheus/blackbox_exporter/blob/v0.21.1/example.yml#L53-L54
>
> Or you can give up, and use "insecure_skip_verify: true" to disable 
> certificate validation altogether: 
> https://github.com/prometheus/blackbox_exporter/blob/v0.21.1/example.yml#L27-L28
> (although then you lose valuable tests, because if your certificate was 
> valid and becomes invalid you won't know.  I also note that if you can't 
> get blackbox_exporter to validate the certificate, then your clients 
> probably won't be able to validate it either)
>
> At this point, it isn't really an issue with blackbox_exporter any more: 
> it's a case of understanding how certificates and certificate validation 
> work.  
>
> On Tuesday, 5 July 2022 at 05:53:46 UTC+1 ninag...@gmail.com wrote:
>
>> I tried to change the first expect to ^220 ([^ ]+) ESMTP$. But I'm not 
>> sure if the change is acceptable from blackbox's view.
>>
>> smtp_starttls:
>> prober: tcp
>> timeout: 20s
>> tcp:
>>   query_response:
>> #- expect: "^220 ([^ ]+) ESMTP (.+)$"
>> - expect: "^220 ([^ ]+) ESMTP$"
>>
>>   send: "EHLO prober\r"
>> - expect: "^250-STARTTLS"
>> - expect: "^250 .*$"
>>   send: "STARTTLS\r"
>> - expect: "^220"
>>   starttls: true
>> - send: "EHLO prober\r"
>> - expect: "^250 .*$"
>>   send: "QUIT\r"
>>
>>
>> Curled and got the following error. Does it mean the mail server doesn't 
>> contain any IP SANs? And on mail server certificate cannot be 

Re: [prometheus-users] implement smtp connection with blackbox exporter

[Brian Candler]

Problem 1: timeout / regexp mismatch

> ts=2022-07-05T02:29:41.609572861Z caller=main.go:130 module=smtp_starttls 
target=1.1.1.1:25 level=debug msg="Read line" line="220 mail.com ESMTP"
> ts=2022-07-05T02:29:46.607798497Z caller=main.go:130 module=smtp_starttls 
target=11.1.1.1:25 level=error msg="Error reading from connection" 
err="read tcp4 0.0.0.0:57074->1.1.1.1:25: i/o timeout"

You asked it to match "^220 ([^ ]+) ESMTP (.+)$" which means:

- "220"
- 
- 1 or more 
- 
- "ESMTP"
- 
- 1 or more 
- end of line

But in the banner you got back from the target host, there's nothing after 
the word "ESMTP": the line ends there immediately.  Hence you were right to 
change the regexp.

"I'm not sure if the change is acceptable from blackbox's view"  - Blackbox 
will do whatever you ask it to do.  If you want to match both cases (i.e. 
mail servers which do and don't put any text after "ESMTP"), then use a 
regexp which does that:

"^220 ([^ ]+) ESMTP( .+)?$"

or

"^220 ([^ ]+) ESMTP.*$"

or even just

"^220 ([^ ]+) ESMTP"

(no need to match all the way to the end of the line)

Problem 2: TLS handshake failed

* You asked Blackbox Exporter to connect to a host by IP address (1.1.1.1), 
instead of using a DNS name (smtp.example.com)
* Therefore, Blackbox Exporter cannot verify the certificate unless the 
certificate itself contains an IP address

It's the same as if you put https://1.1.1.1/ in your web browser.  There's 
no way the browser can validate the certificate, unless the certificate 
itself contains IP address "1.1.1.1".  And that is a very uncommon sort of 
certificate.

Please, try *exactly* the command I gave you before:
*curl -g 
'localhost:9115/probe?module=smtp_starttls=aspmx.l.google.com:25=true'*

No changes. Copy-paste exactly that, with the hostname "aspmx.l.google.com" 
in it.  You should find that it verifies correctly, as long as the ESMTP 
regexp matches OK.

Then to get it to work with your own mail server, you will need to do 
exactly the same:
* Refer to your mail server by its DNS name, not IP address, when asking 
blackbox_exporter to test it
* The mail server must have a certificate which contains this name [^1]
* The mail server's certificate must be signed by a certificate authority 
that blackbox_exporter trusts [^2]
* The mail server's certificate must not have expired, i.e. the current 
time is within its notBefore and notAfter times

[^1] If you still want to connect to the server by IP address but check for 
a different name in the certificate, then use the "server_name" setting in 
tls_config: 
see 
https://prometheus.io/docs/prometheus/latest/configuration/configuration/#tls_config

[^2] If the certificate authority that signed your mail server's 
certificate is not in the system trust store then use the "ca_file" setting 
in tls_config: 
see 
https://github.com/prometheus/blackbox_exporter/blob/v0.21.1/example.yml#L53-L54

Or you can give up, and use "insecure_skip_verify: true" to disable 
certificate validation 
altogether: 
https://github.com/prometheus/blackbox_exporter/blob/v0.21.1/example.yml#L27-L28
(although then you lose valuable tests, because if your certificate was 
valid and becomes invalid you won't know.  I also note that if you can't 
get blackbox_exporter to validate the certificate, then your clients 
probably won't be able to validate it either)

At this point, it isn't really an issue with blackbox_exporter any more: 
it's a case of understanding how certificates and certificate validation 
work.  

On Tuesday, 5 July 2022 at 05:53:46 UTC+1 ninag...@gmail.com wrote:

> I tried to change the first expect to ^220 ([^ ]+) ESMTP$. But I'm not 
> sure if the change is acceptable from blackbox's view.
>
> smtp_starttls:
> prober: tcp
> timeout: 20s
> tcp:
>   query_response:
> #- expect: "^220 ([^ ]+) ESMTP (.+)$"
> - expect: "^220 ([^ ]+) ESMTP$"
>
>   send: "EHLO prober\r"
> - expect: "^250-STARTTLS"
> - expect: "^250 .*$"
>   send: "STARTTLS\r"
> - expect: "^220"
>   starttls: true
> - send: "EHLO prober\r"
> - expect: "^250 .*$"
>   send: "QUIT\r"
>
>
> Curled and got the following error. Does it mean the mail server doesn't 
> contain any IP SANs? And on mail server certificate cannot be validated?
> ts=2022-07-05T04:18:10.539326118Z caller=main.go:130 module=smtp_starttls 
> target=1.1.1.1:25 level=error msg="TLS Handshake (client) failed" 
> err="x509: cannot validate certificate for 1.1.1.1 because it doesn't 
> contain any IP SANs"
>
> curl -g '
> http://0.0.0.0:9115/probe?module=smtp_starttls=1.1.1.1:25=true
> '
> Logs for the probe:
> ts=2022-07-05T04:18:10.527958378Z caller=main.go:320 module=smtp_starttls 
> target=1.1.1.1:25 level=info msg="Beginning probe" probe=tcp 
> timeout_seconds=20
> ts=2022-07-05T04:18:10.528040568Z caller=tcp.go:40 module=smtp_starttls 
> target=1.1.1.1:25 

Re: [prometheus-users] Re: Alerts are getting auto resolved automatically

[Stuart Clark]

Two alerts suggests that the two instances aren't talking to each other. How 
have you configured them? Does the UI show the "other" instance? 

On 5 July 2022 08:34:45 BST, Venkatraman Natarajan  wrote:
>Thanks Brian. I have used last_over_time query in our expression instead of
>turning off auto-resolved.
>
>Also, we have two alert managers in our environment. Both are up and
>running. But Nowadays, we are getting two alerts from two alert managers.
>Could you please help me to sort this issue as well.?
>
>Please find the alert manager configuration.
>
>  alertmanager0:
>image: prom/alertmanager
>container_name: alertmanager0
>user: rootuser
>volumes:
>  - ../data:/data
>  - ../config/alertmanager.yml:/etc/alertmanager/alertmanager.yml
>command:
>  - '--config.file=/etc/alertmanager/alertmanager.yml'
>  - '--storage.path=/data/alert0'
>  - '--cluster.listen-address=0.0.0.0:6783'
>  - '--cluster.peer={{ IP Address }}:6783'
>  - '--cluster.peer={{ IP Address }}:6783'
>restart: unless-stopped
>logging:
>  driver: "json-file"
>  options:
>max-size: "10m"
>max-file: "2"
>ports:
>  - 9093:9093
>  - 6783:6783
>networks:
>  - network
>
>Regards,
>Venkatraman N
>
>
>
>On Sat, Jun 25, 2022 at 9:05 PM Brian Candler  wrote:
>
>> If probe_success becomes non-zero, even for a single evaluation interval,
>> then the alert will be immediately resolved.  There is no delay on
>> resolving, like there is for pending->firing ("for: 5m").
>>
>> I suggest you enter the alerting expression, e.g. "probe_success == 0",
>> into the PromQL web interface (query browser), and switch to Graph view,
>> and zoom in.  If you see any gaps in the graph, then the alert was resolved
>> at that instant.
>>
>> Conversely, use the query
>> probe_success{instance="xxx"} != 0
>> to look at a particular timeseries, as identified by the label9s), and see
>> if there are any dots shown where the label is non-zero.
>>
>> To make your alerts more robust you may need to use queries with range
>> vectors, e.g. min_over_time(foo[5m]) or max_over_time(foo[5m]) or whatever.
>>
>> As a general rule though: you should consider carefully whether you want
>> to send *any* notification for resolved alerts.  Personally, I have
>> switched to send_resolved = false.  There are some good explanations here:
>>
>> https://www.robustperception.io/running-into-burning-buildings-because-the-fire-alarm-stopped
>>
>> https://docs.google.com/document/d/199PqyG3UsyXlwieHaqbGiWVa8eMWi8zzAn0YfcApr8Q/
>>
>> You don't want to build a culture where people ignore alerts because the
>> alert cleared itself - or is expected to clear itself.
>>
>> You want the alert condition to trigger a *process*, which is an
>> investigation of *why* the alert happened, *what* caused it, whether the
>> underlying cause has been fixed, and whether the alerting rule itself was
>> wrong.  When all that has been investigated, manually close the ticket.
>> The fact that the alert has gone below threshold doesn't mean that this
>> work no longer needs to be done.
>>
>> On Saturday, 25 June 2022 at 13:27:22 UTC+1 v.ra...@gmail.com wrote:
>>
>>> Hi Team,
>>>
>>> We are having two prometheus and two alert managers in separate VMs as
>>> containers.
>>>
>>> Alerts are getting auto resolved even though the issues are there as per
>>> threshold.
>>>
>>> For example, if we have an alert rule called probe_success == 0 means it
>>> is triggering an alert but after sometime the alert gets auto-resolved
>>> because we have enabled send_resolved = true. But probe_success == 0 still
>>> there so we don't want to auto resolve the alerts.
>>>
>>> Could you please help us on this.?
>>>
>>> Thanks,
>>> Venkatraman N
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Prometheus Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to prometheus-users+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/prometheus-users/68bff458-ee79-42ce-bafb-facd239e26aen%40googlegroups.com
>> 
>> .
>>
>
>-- 
>You received this message because you are subscribed to the Google Groups 
>"Prometheus Users" group.
>To unsubscribe from this group and stop receiving emails from it, send an 
>email to prometheus-users+unsubscr...@googlegroups.com.
>To view this discussion on the web visit 
>https://groups.google.com/d/msgid/prometheus-users/CANSgTEbTrr7Jjf_XwD0J8wgMAdiLg9g_MmWDK%3DpgkTjwMA5YZA%40mail.gmail.com.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an 

Re: [prometheus-users] AM Executor on Kubernetes Cluster

[Stuart Clark]

What are you meaning by "AM executor"? Are you meaning the Alertmanager 
application?

If so, there are both community Helm charts and a Docker image available, so 
running within Kubernetes should be no problem. 

On 5 July 2022 08:03:45 BST, Test Kumar  wrote:
>Hi Team,
>
>I installed the AM executor as a stand-alone application.
>But I want to install it as a Kubernetes pod service same as Kubernetes 
>Prometheus.
>So is there a way to install AM executor on Kubernetes?
>
>Thanks & Regards,
>Test Kumar 
>
>-- 
>You received this message because you are subscribed to the Google Groups 
>"Prometheus Users" group.
>To unsubscribe from this group and stop receiving emails from it, send an 
>email to prometheus-users+unsubscr...@googlegroups.com.
>To view this discussion on the web visit 
>https://groups.google.com/d/msgid/prometheus-users/a8b46c22-160d-4c37-b600-7511b7433849n%40googlegroups.com.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/26D5B237-5E46-42AD-ACEE-414B71ACFF25%40Jahingo.com.

Re: [prometheus-users] Re: Alerts are getting auto resolved automatically

[Venkatraman Natarajan]

Thanks Brian. I have used last_over_time query in our expression instead of
turning off auto-resolved.

Also, we have two alert managers in our environment. Both are up and
running. But Nowadays, we are getting two alerts from two alert managers.
Could you please help me to sort this issue as well.?

Please find the alert manager configuration.

  alertmanager0:
image: prom/alertmanager
container_name: alertmanager0
user: rootuser
volumes:
  - ../data:/data
  - ../config/alertmanager.yml:/etc/alertmanager/alertmanager.yml
command:
  - '--config.file=/etc/alertmanager/alertmanager.yml'
  - '--storage.path=/data/alert0'
  - '--cluster.listen-address=0.0.0.0:6783'
  - '--cluster.peer={{ IP Address }}:6783'
  - '--cluster.peer={{ IP Address }}:6783'
restart: unless-stopped
logging:
  driver: "json-file"
  options:
max-size: "10m"
max-file: "2"
ports:
  - 9093:9093
  - 6783:6783
networks:
  - network

Regards,
Venkatraman N



On Sat, Jun 25, 2022 at 9:05 PM Brian Candler  wrote:

> If probe_success becomes non-zero, even for a single evaluation interval,
> then the alert will be immediately resolved.  There is no delay on
> resolving, like there is for pending->firing ("for: 5m").
>
> I suggest you enter the alerting expression, e.g. "probe_success == 0",
> into the PromQL web interface (query browser), and switch to Graph view,
> and zoom in.  If you see any gaps in the graph, then the alert was resolved
> at that instant.
>
> Conversely, use the query
> probe_success{instance="xxx"} != 0
> to look at a particular timeseries, as identified by the label9s), and see
> if there are any dots shown where the label is non-zero.
>
> To make your alerts more robust you may need to use queries with range
> vectors, e.g. min_over_time(foo[5m]) or max_over_time(foo[5m]) or whatever.
>
> As a general rule though: you should consider carefully whether you want
> to send *any* notification for resolved alerts.  Personally, I have
> switched to send_resolved = false.  There are some good explanations here:
>
> https://www.robustperception.io/running-into-burning-buildings-because-the-fire-alarm-stopped
>
> https://docs.google.com/document/d/199PqyG3UsyXlwieHaqbGiWVa8eMWi8zzAn0YfcApr8Q/
>
> You don't want to build a culture where people ignore alerts because the
> alert cleared itself - or is expected to clear itself.
>
> You want the alert condition to trigger a *process*, which is an
> investigation of *why* the alert happened, *what* caused it, whether the
> underlying cause has been fixed, and whether the alerting rule itself was
> wrong.  When all that has been investigated, manually close the ticket.
> The fact that the alert has gone below threshold doesn't mean that this
> work no longer needs to be done.
>
> On Saturday, 25 June 2022 at 13:27:22 UTC+1 v.ra...@gmail.com wrote:
>
>> Hi Team,
>>
>> We are having two prometheus and two alert managers in separate VMs as
>> containers.
>>
>> Alerts are getting auto resolved even though the issues are there as per
>> threshold.
>>
>> For example, if we have an alert rule called probe_success == 0 means it
>> is triggering an alert but after sometime the alert gets auto-resolved
>> because we have enabled send_resolved = true. But probe_success == 0 still
>> there so we don't want to auto resolve the alerts.
>>
>> Could you please help us on this.?
>>
>> Thanks,
>> Venkatraman N
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-users/68bff458-ee79-42ce-bafb-facd239e26aen%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/CANSgTEbTrr7Jjf_XwD0J8wgMAdiLg9g_MmWDK%3DpgkTjwMA5YZA%40mail.gmail.com.

[prometheus-users] AM Executor on Kubernetes Cluster

[Test Kumar]

Hi Team,

I installed the AM executor as a stand-alone application.
But I want to install it as a Kubernetes pod service same as Kubernetes 
Prometheus.
So is there a way to install AM executor on Kubernetes?

Thanks & Regards,
Test Kumar 

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/a8b46c22-160d-4c37-b600-7511b7433849n%40googlegroups.com.

Re: [prometheus-users] implement smtp connection with blackbox exporter

[nina guo]

I found a module [smtp_banner], but cannot find it  
blackbox_exporter/example.yml 
at master · prometheus/blackbox_exporter (github.com) 


May I ask if [smtp_banner] still valid?
On Tuesday, July 5, 2022 at 12:53:46 PM UTC+8 nina guo wrote:

> I tried to change the first expect to ^220 ([^ ]+) ESMTP$. But I'm not 
> sure if the change is acceptable from blackbox's view.
>
> smtp_starttls:
> prober: tcp
> timeout: 20s
> tcp:
>   query_response:
> #- expect: "^220 ([^ ]+) ESMTP (.+)$"
> - expect: "^220 ([^ ]+) ESMTP$"
>
>   send: "EHLO prober\r"
> - expect: "^250-STARTTLS"
> - expect: "^250 .*$"
>   send: "STARTTLS\r"
> - expect: "^220"
>   starttls: true
> - send: "EHLO prober\r"
> - expect: "^250 .*$"
>   send: "QUIT\r"
>
>
> Curled and got the following error. Does it mean the mail server doesn't 
> contain any IP SANs? And on mail server certificate cannot be validated?
> ts=2022-07-05T04:18:10.539326118Z caller=main.go:130 module=smtp_starttls 
> target=1.1.1.1:25 level=error msg="TLS Handshake (client) failed" 
> err="x509: cannot validate certificate for 1.1.1.1 because it doesn't 
> contain any IP SANs"
>
> curl -g '
> http://0.0.0.0:9115/probe?module=smtp_starttls=1.1.1.1:25=true
> '
> Logs for the probe:
> ts=2022-07-05T04:18:10.527958378Z caller=main.go:320 module=smtp_starttls 
> target=1.1.1.1:25 level=info msg="Beginning probe" probe=tcp 
> timeout_seconds=20
> ts=2022-07-05T04:18:10.528040568Z caller=tcp.go:40 module=smtp_starttls 
> target=1.1.1.1:25 level=info msg="Resolving target address" 
> ip_protocol=ip6
> ts=2022-07-05T04:18:10.528063021Z caller=tcp.go:40 module=smtp_starttls 
> target=1.1.1.1:25 level=info msg="Resolving target address" 
> ip_protocol=ip4
> ts=2022-07-05T04:18:10.528076698Z caller=tcp.go:40 module=smtp_starttls 
> target=1.1.1.1:25 level=info msg="Resolved target address" ip=1.1.1.1
> ts=2022-07-05T04:18:10.528115109Z caller=tcp.go:121 module=smtp_starttls 
> target=1.1.1.1:25 level=info msg="Dialing TCP without TLS"
> ts=2022-07-05T04:18:10.529223463Z caller=main.go:130 module=smtp_starttls 
> target=1.1.1.1:25 level=info msg="Successfully dialed"
> ts=2022-07-05T04:18:10.529258037Z caller=main.go:130 module=smtp_starttls 
> target=1.1.1.1:25 level=info msg="Processing query response entry" 
> entry_number=0
> ts=2022-07-05T04:18:10.530236839Z caller=main.go:130 module=smtp_starttls 
> target=1.1.1.1:25 level=debug msg="Read line" line="220 mail.com ESMTP"
> ts=2022-07-05T04:18:10.530278493Z caller=main.go:130 module=smtp_starttls 
> target=1.1.1.1:25 level=info msg="Regexp matched" regexp="^220 ([^ ]+) 
> ESMTP$" line="220 mail.com ESMTP"
> ts=2022-07-05T04:18:10.53030557Z caller=main.go:130 module=smtp_starttls 
> target=1.1.1.1:25 level=debug msg="Sending line" line="EHLO prober\r"
> ts=2022-07-05T04:18:10.530348868Z caller=main.go:130 module=smtp_starttls 
> target=1.1.1.1:25 level=info msg="Processing query response entry" 
> entry_number=1
> ts=2022-07-05T04:18:10.530919204Z caller=main.go:130 module=smtp_starttls 
> target=1.1.1.1:25 level=debug msg="Read line" line=250-mail.com
> ts=2022-07-05T04:18:10.530955761Z caller=main.go:130 module=smtp_starttls 
> target=1.1.1.1:25 level=debug msg="Read line" line=250-PIPELINING
> ts=2022-07-05T04:18:10.530971243Z caller=main.go:130 module=smtp_starttls 
> target=1.1.1.1:25 level=debug msg="Read line" line="250-SIZE 1024"
> ts=2022-07-05T04:18:10.530985701Z caller=main.go:130 module=smtp_starttls 
> target=1.1.1.1:25 level=debug msg="Read line" line=250-ETRN
> ts=2022-07-05T04:18:10.531006196Z caller=main.go:130 module=smtp_starttls 
> target=1.1.1.1:25 level=debug msg="Read line" line=250-STARTTLS
> ts=2022-07-05T04:18:10.531025088Z caller=main.go:130 module=smtp_starttls 
> target=1.1.1.1:25 level=info msg="Regexp matched" regexp=^250-STARTTLS 
> line=250-STARTTLS
> ts=2022-07-05T04:18:10.531045174Z caller=main.go:130 module=smtp_starttls 
> target=1.1.1.1:25 level=info msg="Processing query response entry" 
> entry_number=2
> ts=2022-07-05T04:18:10.531064549Z caller=main.go:130 module=smtp_starttls 
> target=1.1.1.1:25 level=debug msg="Read line" line="250-AUTH PLAIN LOGIN"
> ts=2022-07-05T04:18:10.531077325Z caller=main.go:130 module=smtp_starttls 
> target=1.1.1.1:25 level=debug msg="Read line" line=250-ENHANCEDSTATUSCODES
> ts=2022-07-05T04:18:10.531088466Z caller=main.go:130 module=smtp_starttls 
> target=1.1.1.1:25 level=debug msg="Read line" line=250-8BITMIME
> ts=2022-07-05T04:18:10.53110099Z caller=main.go:130 module=smtp_starttls 
> target=1.1.1.1:25 level=debug msg="Read line" line="250 DSN"
> ts=2022-07-05T04:18:10.531134286Z caller=main.go:130 module=smtp_starttls 
> target=1.1.1.1:25 level=info msg="Regexp matched" regexp="^250 .*$" 
> line="250 DSN"
>