Hi This is my first time asking a question in the group, and I may not be very clear on some of the rules, but I will do my best to describe my question clearly. Below is the description of my issue:
- I deployed* Prometheus, Node Exporter, Grafana, Alertmanager, Blackbox Exporter, *and* Webhook-WeChat *on the server with IP *10.1.1.161* using Docker, and all the ports are set to their default values. These services are running normally. I want to use *Blackbox Exporter* to monitor whether the *Tableau *service installed on a Windows system with IP *10.101.1.112* returns a status code of 200. I am monitoring port 8850 (the Tableau port).I used* curl https://10.101.1.112:8850* on 10.1.1.161 and received the following response: ------------------------------------------------------------------------------------------------------------------------------------------ curl: (60) Peer's certificate issuer has been marked as not trusted by the user. More details here: http://curl.haxx.se/docs/sslcerts.html curl performs SSL certificate verification by default, using a "bundle" of Certificate Authority (CA) public keys (CA certs). If the default bundle file isn't adequate, you can specify an alternate file using the --cacert option. ------------------------------------------------------------------------------------------------------------------------------------------- - when I use "*curl -k https://10.101.1.112:8850*";, I can successfully retrieve the response body. Therefore, I added insecure_skip_verify: true in the *'config/blackbox.yml*' file to skip certificate verification. - I restarted Prometheus, Blackbox Exporter, and the Blackbox container, but after the restart, the Webhook-WeChat service triggered an alert: ' *https://10.101.1.112:8850* responded with code 0' instead of 200. However, when I access *https://10.101.1.112:8850* in the browser, it works, though it warns that the connection is not secure. At the same time, when I use the browser to access *http://10.1.1.161:9115/probemodule=http_2xx&target=https://10.101.1.112:8850*, it returns a status in a format like this, which seems to indicate that the request was unsuccessful(for example：probe_success 0,probe_http_status_code 0): ------------------------------------------------------------------------------------------------------------------------------------------- # HELP probe_dns_lookup_time_seconds Returns the time taken for probe dns lookup in seconds # TYPE probe_dns_lookup_time_seconds gauge probe_dns_lookup_time_seconds 1.6356e-05 # HELP probe_duration_seconds Returns how long the probe took to complete in seconds # TYPE probe_duration_seconds gauge probe_duration_seconds 0.022416452 # HELP probe_failed_due_to_regex Indicates if probe failed due to regex # TYPE probe_failed_due_to_regex gauge probe_failed_due_to_regex 0 # HELP probe_http_content_length Length of http content response # TYPE probe_http_content_length gauge probe_http_content_length 0 # HELP probe_http_duration_seconds Duration of http request by phase, summed over all redirects # TYPE probe_http_duration_seconds gauge probe_http_duration_seconds{phase="connect"} 0 probe_http_duration_seconds{phase="processing"} 0 probe_http_duration_seconds{phase="resolve"} 1.6356e-05 probe_http_duration_seconds{phase="tls"} 0 probe_http_duration_seconds{phase="transfer"} 0 # HELP probe_http_redirects The number of redirects # TYPE probe_http_redirects gauge probe_http_redirects 0 # HELP probe_http_ssl Indicates if SSL was used for the final redirect # TYPE probe_http_ssl gauge probe_http_ssl 0 # HELP probe_http_status_code Response HTTP status code # TYPE probe_http_status_code gauge probe_http_status_code 0 # HELP probe_http_uncompressed_body_length Length of uncompressed response body # TYPE probe_http_uncompressed_body_length gauge probe_http_uncompressed_body_length 0 # HELP probe_http_version Returns the version of HTTP of the probe response # TYPE probe_http_version gauge probe_http_version 0 # HELP probe_ip_addr_hash Specifies the hash of IP address. It's useful to detect if the IP address changes. # TYPE probe_ip_addr_hash gauge probe_ip_addr_hash 3.959139489e+09 # HELP probe_ip_protocol Specifies whether probe ip protocol is IP4 or IP6 # TYPE probe_ip_protocol gauge probe_ip_protocol 4 # HELP probe_success Displays whether or not the probe was a success # TYPE probe_success gauge probe_success 0 ------------------------------------------------------------------------------------------------------------------------------------------ Here is my Prometheus prometheus.yml configuration: --- global: scrape_interval: 15s evaluation_interval: 15s # config alertmanagers alerting: alertmanagers: - static_configs: - targets: - 10.1.1.161:9093 rule_files: - "/etc/prometheus/rules/*.yml" scrape_configs: # Prometheus port 9090 - job_name: 'prometheus' static_configs: - targets: ['10.1.1.161:9090'] labels: instance: 'prometheus' - job_name: 'tableau_blackbox' metrics_path: /probe params: module: [http_2xx] static_configs: - targets: - https://10.101.1.112:8850 relabel_configs: - source_labels: [__address__] target_label: __param_target - source_labels: [__param_target] target_label: instance - target_label: __address__ # This is Blackbox Exporter replacement: 10.1.1.161:9115 - source_labels: [__param_target] target_label: service replacement: 'new pro environment' # https://10.101.1.112:8850 regex: 'https://10\.101\.1\.112:8850' --- Here is the Prometheus rule configuration: --- groups: - name: tableau_alerts rules: # 1. HTTP/HTTPS - alert: Tableau service problem expr: probe_http_status_code{job="tableau_blackbox",service="new pro environment"} != 200 for: 10s labels: severity: critical annotations: summary: "tableau code exception" description: "Target service {{ $labels.instance }} response code {{ $value }}，should be 200!" --- Here is the Blackbox Exporter blackbox.yml configuration: --- modules: http_2xx: prober: http timeout: 5s http: valid_http_versions: ["HTTP/1.1", "HTTP/2.0"] method: GET preferred_ip_protocol: "ip4" ip_protocol_fallback: false no_follow_redirects: false fail_if_ssl: false fail_if_not_ssl: false tls_config: insecure_skip_verify: true --- Here is my Docker startup command: *Prometheus：* *docker run -d --name prometheus -p 9090:9090 -v /data/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml \ -v /data/prometheus/rules/:/etc/prometheus/rules/ prom/prometheus* *blackbox explorter:* *docker run -d -p 9115:9115 -v /data/blackbox_exporter/config/:/config/ --name blackbox_exporter quay.io/prometheus/blackbox-exporter * -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/prometheus-users/e64ca74d-169c-4454-bcd7-87b67a5f3272n%40googlegroups.com.
