Re: [prometheus-users] Re: Prometheus file format
On 08/08/2023 20:31, Matt Doughty wrote: So you are trying to get discreet metrics for every run of the batch job? That sounds like an unbounded cardinality problem as you would end up with a timeseries for every run of the batch job. Am I misunderstanding or is this accurate? You're right I don't need the exact time when the metric is fetched. I only need it to differentiate between iterations within the batch job. Then is creating a separate metric the best way to go? If that is the case then Prometheus isn't the right tool. Having distinctly detectable groups of data for a particular job run indicates you are talking about events which are quite different to metrics. For events you'd want to be looking at tools such as Elasticsearch, Loki or a standard SQL database. Events and metrics can (and often are) used in parallel. For example Prometheus would tell you that the average job runtime is 5 minutes over the past 3 hours, but you'd then use the events system to find the exact durations for each run (or the number of events processed, or the error message returned, etc.). -- Stuart Clark -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/579c5062-cc5a-0d7b-7353-61ed436e25b6%40Jahingo.com.
Re: [prometheus-users] Re: Prometheus file format
So you are trying to get discreet metrics for every run of the batch job? That sounds like an unbounded cardinality problem as you would end up with a timeseries for every run of the batch job. Am I misunderstanding or is this accurate? --Matt On Tue, Aug 8, 2023 at 3:22 PM Moe wrote: > > You're right I don't need the exact time when the metric is fetched. I only > need it to differentiate between iterations within the batch job. Then is > creating a separate metric the best way to go? > > On Tuesday, August 8, 2023 at 4:09:24 AM UTC+5:30 Matt Doughty wrote: >> >> I just wanted to chime in to say that you almost never need to know >> exactly when a metric is updated. >> Before you start exporting additional timestamp metrics, you should >> really ask yourself whether you actually need them. What problem do >> they actually solve? >> >> --Matt >> >> On Mon, Aug 7, 2023 at 5:54 PM Stuart Clark wrote: >> > >> > On 07/08/2023 21:00, Moe wrote: >> > > Thanks Brian that was really helpful, >> > > >> > > 2. The use case I want this for doesn't need continuous ingestion. In >> > > that case is there a way for me to add time stamp to MetricFamilySample? >> > >> > That isn't how Prometheus works. It will scrape that metric every 10 >> > seconds to 2 minutes. If you need to know when that set of metrics was >> > created a common pattern is to include a metric who's value is the >> > timestamp. >> > >> > -- >> > Stuart Clark >> > >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "Prometheus Users" group. >> > To unsubscribe from this group and stop receiving emails from it, send an >> > email to prometheus-use...@googlegroups.com. >> > To view this discussion on the web visit >> > https://groups.google.com/d/msgid/prometheus-users/d401af4d-8583-d2d6-dbbe-89853000e421%40Jahingo.com. >> >> >> >> -- >> --Matt > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to prometheus-users+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/bf01a491-2b12-4f10-b7db-fbf865647496n%40googlegroups.com. -- --Matt -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAGyBzczT0Qvw%3DB%2BD8v4_BhhHFNtsp_fysXXVfPwvMB7RnzNyqQ%40mail.gmail.com.
Re: [prometheus-users] Re: Prometheus file format
You're right I don't need the exact time when the metric is fetched. I only need it to differentiate between iterations within the batch job. Then is creating a separate metric the best way to go? On Tuesday, August 8, 2023 at 4:09:24 AM UTC+5:30 Matt Doughty wrote: > I just wanted to chime in to say that you almost never need to know > exactly when a metric is updated. > Before you start exporting additional timestamp metrics, you should > really ask yourself whether you actually need them. What problem do > they actually solve? > > --Matt > > On Mon, Aug 7, 2023 at 5:54 PM Stuart Clark wrote: > > > > On 07/08/2023 21:00, Moe wrote: > > > Thanks Brian that was really helpful, > > > > > > 2. The use case I want this for doesn't need continuous ingestion. In > > > that case is there a way for me to add time stamp to > MetricFamilySample? > > > > That isn't how Prometheus works. It will scrape that metric every 10 > > seconds to 2 minutes. If you need to know when that set of metrics was > > created a common pattern is to include a metric who's value is the > > timestamp. > > > > -- > > Stuart Clark > > > > -- > > You received this message because you are subscribed to the Google > Groups "Prometheus Users" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to prometheus-use...@googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/d401af4d-8583-d2d6-dbbe-89853000e421%40Jahingo.com > . > > > > -- > --Matt > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/bf01a491-2b12-4f10-b7db-fbf865647496n%40googlegroups.com.
