Thanks for the response Stuart.. To explain you more.. I am load testing an application through Gatling scripts (similar to jmeter).
Now I want to have a real time monitoring of this load test. For this, Gatling supports graphite writer protocol(it can't directly talk with prometheus hence I have used graphite-exporter in between) Now Promotheus will collect these metrics sent by Gatling and provide to Grafana to plot the graphs. Now the problem is I am getting graphs but even after my load test is finished, I see the last value graph repeating for 5 minutes. Which is the known issue of prometheus... Hence I am confused on how to resolve this issue? Any configuration need to be added to prometheus.yml file? Please let me know if you need any further details.. On Tue, Apr 19, 2022, 4:44 PM Stuart Clark <stuart.cl...@jahingo.com> wrote: > On 2022-04-19 08:58, Aniket Kulkarni wrote: > > Hi, > > > > I have referred below links: > > > > I understand this was a problem with 1.x > > https://github.com/prometheus/prometheus/issues/398 > > > > I also got this link as a solution > > https://promcon.io/2017-munich/talks/staleness-in-prometheus-2-0/ > > > > No doubt it's a great session. But I am still not clear as to what > > change I have to make and where? > > > > I also couldn't find the prometheus docs useful for this. > > > > I am using following tech stack: > > Gatling -> graphite-exporter -> prometheus-> grafana. > > > > I am still facing staleness issue. Please guide me on the solution or > > any extra configuration needed? > > > > I am using the default storage system by prometheus and not any > > external one. > > > > Could you describe a bit more of the problem you are seeing and what you > are wanting to do? > > All time series will be marked as stale if they have not been scraped > for a while, which causes data to stop being returned by queries, which > is important as things like labels will change over time (especially for > things like Kubernetes which include pod names). It is expected that > targets will be regularly scraped, so things shouldn't otherwise > disapear (unless there is an error, which should be visible via > something like the "up" metric). > > As the standard staleness interval is 5 minutes it is recommended that > the maximum scrape period should be no more that 2 minutes (to allow for > a failed scrape without the time series being marked as stale). > > -- > Stuart Clark > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAPYU55fLc7rnq41B6aFdjc-GyNFq6em6rpw-xeYuE7LbZnhHbA%40mail.gmail.com.
