On Tuesday 19 December 2023 at 06:03:25 UTC Kesavanand Chavali wrote: We have a customer exporter written in GoLang that scrapes metrics from windows exporter and corrects the time to current UTC Time. This custom exporter is scrapped by Prometheus for every 4 minutes.
Still makes no sense to me. Can you show some examples of the actual scrapes, e.g. tested using curl? - windows_exporter should not be adding timestamps to metrics (although I don't have a running instance to test with) so there should be nothing to change - your custom exporter should not be adding timestamps to metrics - prometheus by default records the scrape time, not the metric timestamp (Aside: there may be some metrics whose value *is* a timestamp, like node_boot_time_seconds in node_exporter. But a metric is just a number, so whether it's "in the future" or "in the past" makes no difference for that kind of metric) The custom exporter is written in GoLang and uses NewMetricWithTimestamp to add the correct timestamp Why are you doing this? Why not just use [Must]NewConstMetric? And why are you proxying through a custom exporter, rather than just having the agent scrape windows_exporter directly? If we scrape windows exporter directly from Prometheus and remote write to Thanos, Thanos accepts the metrics as Out of order ingestion is allowed. If we scrape our custom exporter then we see metrics too old or too future error in Prometheus logs Do you have "honor_timestamps: true" in Prometheus agent? If so, why? To me, it seems like you're swimming against the current here. Just do what Prometheus does by default, which is to set the timestamp of every scrape as the time when it was scraped. The state of the clock on the scrape target is irrelevant. -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/89d0fb28-bc8d-43c0-9166-99e660cd9b7fn%40googlegroups.com.
