I found an issue (https://github.com/prometheus/blackbox_exporter/issues/579) that characterizes a similar change in time for the rest of the phases, except for "processing" - the metric returns the value 0, but this phase, in principle, did not start processing at all.
вторник, 25 января 2022 г. в 14:07:58 UTC+3, Houston Heat: > There are several phases of HTTP (github) > <https://github.com/prometheus/blackbox_exporter/blob/e34a36be40264b1ceda1ba8ddbd769c4a62eac16/prober/http.go#L310> > in > Prometheus metric labels while we are probing data via BlackBox exporter: > > > 1. resolve > 2. connect > 3. tls > 4. processing > 5. transfer > > What does "processing" stand for in the context of an exporter? What kind > of processing exactly (js/html/json render waiting) if the response results > time are coming from "transfer" phase? > > Context: We found a sudden increase in latency and availability on one of > our sites when checking with BlackBox exporter, but the total latency > measured by request tracing and APM (NewRelic) did not change. This is > supported by additional geo-distributed blackboxes and synthetic monitoring > tools. We are convinced that in general the work of the site has not > changed, however, network/hardware/software problems may be observed in the > environment of this particular exporter, and therefore we would like to > understand how the "processing" stage reflects the picture of the request. > > It is quite obvious to me that if the site is unavailable, the check time > of *probe_duration_seconds* will tend to the timeout value, however, a > significant difference in the time of the > *probe_http_duration_seconds{phase="processing"}* makes us think not only > about network problems, but also about the performance of the exporter > itself, for example in case there are a large number of competitive checks. > > [image: overall-phases-shift.png] [image: > phases-shift-without-processing.png] > > [image: probe_http_duration-shift.png] [image: probe-failures.png] > > Single scrape example: > > *# HELP probe_duration_seconds Returns how long the probe took to complete > in seconds# TYPE probe_duration_seconds gauge* > probe_duration_seconds 1.6482566360000002 > > *# HELP probe_http_content_length Length of http content response# TYPE > probe_http_content_length gauge* > probe_http_content_length -1 > *# HELP probe_http_duration_seconds Duration of http request by phase, > summed over all redirects* > *# TYPE probe_http_duration_seconds gauge* > probe_http_duration_seconds{phase="connect"} 0.00674702 > probe_http_duration_seconds{phase="processing"} 1.57984579 > probe_http_duration_seconds{phase="resolve"} 0.008215475 > probe_http_duration_seconds{phase="tls"} 0.024449581 > probe_http_duration_seconds{phase="transfer"} 0.028283079 > > *# HELP probe_http_redirects The number of redirects# TYPE > probe_http_redirects gauge* > probe_http_redirects 1 > > Thank you. > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/b3953f59-8324-43ef-aac2-ce9b7d0e4f7cn%40googlegroups.com.
