Re: [prometheus-developers] Feature request: Setting a custom lookback delta for a query

2021-12-06 Thread Vilius Pranckaitis 
Made a draft PR: https://github.com/prometheus/prometheus/pull/9946

On Tuesday, November 23, 2021 at 6:56:18 AM UTC+2 death...@gmail.com wrote:

> (slightly tangential)
>
> > We want richer metadata anyway,
> +1, richer metadata about the data in tsdb will be required for the query 
> optimizer too. IMO without the metadata, the optimizer cannot choose the 
> best query plan (order of evaluating the query). 
>
> On Monday, 22 November 2021 at 17:28:36 UTC+5:30 bjo...@rabenste.in wrote:
>
>> As a short- to mid-term remedy, I like the idea of a lookback-delta 
>> per query. 
>>
>> Long-term, I would prefer if we could get rid of the lookback delta 
>> altogether. We want richer metadata anyway, and part of it could be 
>> when a series starts and ends and what the configured scrape interval 
>> was. Once we know that, we can create query strategies so that we can 
>> always return the most recent valid sample (if there is any) without 
>> generally killing query performance. 
>>
>> -- 
>> Björn Rabenstein 
>> [PGP-ID] 0x851C3DA17D748D03 
>> [email] bjo...@rabenste.in 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/c8df5559-441c-405b-8b9d-84295737b7e0n%40googlegroups.com.

Re: [prometheus-developers] Feature request: Setting a custom lookback delta for a query

2021-11-22 Thread Darshan Chaudhary 
(slightly tangential)

> We want richer metadata anyway,
+1, richer metadata about the data in tsdb will be required for the query 
optimizer too. IMO without the metadata, the optimizer cannot choose the 
best query plan (order of evaluating the query). 

On Monday, 22 November 2021 at 17:28:36 UTC+5:30 bjo...@rabenste.in wrote:

> As a short- to mid-term remedy, I like the idea of a lookback-delta
> per query.
>
> Long-term, I would prefer if we could get rid of the lookback delta
> altogether. We want richer metadata anyway, and part of it could be
> when a series starts and ends and what the configured scrape interval
> was. Once we know that, we can create query strategies so that we can
> always return the most recent valid sample (if there is any) without
> generally killing query performance.
>
> -- 
> Björn Rabenstein
> [PGP-ID] 0x851C3DA17D748D03
> [email] bjo...@rabenste.in
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/1a2124f9-bd10-4af4-8cc7-e4868da58e96n%40googlegroups.com.

Re: [prometheus-developers] Feature request: Setting a custom lookback delta for a query

2021-11-22 Thread Bjoern Rabenstein 
As a short- to mid-term remedy, I like the idea of a lookback-delta
per query.

Long-term, I would prefer if we could get rid of the lookback delta
altogether. We want richer metadata anyway, and part of it could be
when a series starts and ends and what the configured scrape interval
was. Once we know that, we can create query strategies so that we can
always return the most recent valid sample (if there is any) without
generally killing query performance.

-- 
Björn Rabenstein
[PGP-ID] 0x851C3DA17D748D03
[email] bjo...@rabenste.in

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/20211122115833.GS3660%40jahnn.

Re: [prometheus-developers] Feature request: Setting a custom lookback delta for a query

2021-11-19 Thread Vilius Pranckaitis 
> Thanos automatically deals with downsample lookback with no changes.

That's true. We've at M3 have done the same. However, the resulting 
solutions are not very clean and involve trade-offs (Thanos seem to 
initialize up to three engine with increasing lookbacks 
;
 
in M3 we pre-initialize a few engines based on configured resolutions 

 
and in any other case create them on the fly 
).
 


>From what we've discussed among M3 maintainers and from TODOs in Thanos 
codebase 
,
 
both seem to agree that it would be more convenient if dynamic lookback was 
supported by Prometheus.

> We are willing to have downsampling in Prometheus too in the future.
>
> I would merge that pull request.

I'll look into submitting a draft PR, just so it would be easier to see 
what changes would be needed.


On Monday, November 15, 2021 at 9:12:44 PM UTC+2 sup...@gmail.com wrote:

> Thanos automatically deals with downsample lookback with no changes. This 
> was fixed at least 2 years ago.
>
> On Mon, Nov 15, 2021, 18:54 Julien Pivotto  
> wrote:
>
>> We are willing to have downsampling in Prometheus too in the future.
>>
>> I would merge that pull request.
>>
>> Le lun. 15 nov. 2021, 15:19, Vilius Pranckaitis  
>> a écrit :
>>
>>> *TL;DR*: ability to set a custom lookback delta for a query would be 
>>> useful when time series have drastically different resolutions.
>>>
>>> ---
>>>
>>> Currently, Prometheus allows configuring lookback delta through 
>>> configuration flag. This works well if the resolutions/scrape intervals of 
>>> time series are similar, but is inconvenient if they differ a lot. E.g. if 
>>> we had two time series, one with a datapoint each hour and another one with 
>>> a datapoint each minute, a lookback of 2h would work for the first one, but 
>>> would be way too big for the second one.
>>>
>>> This impacts looking at downsampled data greater than 5 minutes (the 
>>> default lookback period) which both Thanos and M3 frequently keep 
>>> (depending on user configuration). Both support downsampling data to user 
>>> configured resolutions, thus lookback delta should be selected according to 
>>> what resolution is being queried when querying downsampled data. Both 
>>> Thanos and M3 depend on Prometheus and use `promql.Engine` for executing 
>>> queries, which doesn't allow setting custom lookback for a particular query.
>>>
>>> While I understand that this might be more relevant for M3 and Thanos 
>>> than for Prometheus, just wanted to check what the maintainers of 
>>> Prometheus think about this. If this seems useful, someone from M3 should 
>>> be able to submit a PR (we have already looked at the code and it doesn't 
>>> look like it needs huge changes).
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Prometheus Developers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to prometheus-devel...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/prometheus-developers/84c03b13-25d7-479c-b678-44cd8625f2c0n%40googlegroups.com
>>>  
>>> 
>>> .
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Prometheus Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to prometheus-devel...@googlegroups.com.
>>
> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/prometheus-developers/CAFJ6V0p-_k_Yjy_gUdvHaogxZz%3D7DeNEMM7zz7uBYD%2BYBA%2B6bw%40mail.gmail.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/d1fa16b6-d91c-4fd6-93ef-694af3482e05n%40googlegroups.com.

Re: [prometheus-developers] Feature request: Setting a custom lookback delta for a query

2021-11-15 Thread Ben Kochie 
Thanos automatically deals with downsample lookback with no changes. This
was fixed at least 2 years ago.

On Mon, Nov 15, 2021, 18:54 Julien Pivotto 
wrote:

> We are willing to have downsampling in Prometheus too in the future.
>
> I would merge that pull request.
>
> Le lun. 15 nov. 2021, 15:19, Vilius Pranckaitis 
> a écrit :
>
>> *TL;DR*: ability to set a custom lookback delta for a query would be
>> useful when time series have drastically different resolutions.
>>
>> ---
>>
>> Currently, Prometheus allows configuring lookback delta through
>> configuration flag. This works well if the resolutions/scrape intervals of
>> time series are similar, but is inconvenient if they differ a lot. E.g. if
>> we had two time series, one with a datapoint each hour and another one with
>> a datapoint each minute, a lookback of 2h would work for the first one, but
>> would be way too big for the second one.
>>
>> This impacts looking at downsampled data greater than 5 minutes (the
>> default lookback period) which both Thanos and M3 frequently keep
>> (depending on user configuration). Both support downsampling data to user
>> configured resolutions, thus lookback delta should be selected according to
>> what resolution is being queried when querying downsampled data. Both
>> Thanos and M3 depend on Prometheus and use `promql.Engine` for executing
>> queries, which doesn't allow setting custom lookback for a particular query.
>>
>> While I understand that this might be more relevant for M3 and Thanos
>> than for Prometheus, just wanted to check what the maintainers of
>> Prometheus think about this. If this seems useful, someone from M3 should
>> be able to submit a PR (we have already looked at the code and it doesn't
>> look like it needs huge changes).
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Prometheus Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to prometheus-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/prometheus-developers/84c03b13-25d7-479c-b678-44cd8625f2c0n%40googlegroups.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-developers/CAFJ6V0p-_k_Yjy_gUdvHaogxZz%3D7DeNEMM7zz7uBYD%2BYBA%2B6bw%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/CABbyFmrjPhnrWFm441MnH86qVy1mxhE4U8F6ZDPt8Hg-oRjEsA%40mail.gmail.com.

Re: [prometheus-developers] Feature request: Setting a custom lookback delta for a query

2021-11-15 Thread Julien Pivotto 
We are willing to have downsampling in Prometheus too in the future.

I would merge that pull request.

Le lun. 15 nov. 2021, 15:19, Vilius Pranckaitis  a
écrit :

> *TL;DR*: ability to set a custom lookback delta for a query would be
> useful when time series have drastically different resolutions.
>
> ---
>
> Currently, Prometheus allows configuring lookback delta through
> configuration flag. This works well if the resolutions/scrape intervals of
> time series are similar, but is inconvenient if they differ a lot. E.g. if
> we had two time series, one with a datapoint each hour and another one with
> a datapoint each minute, a lookback of 2h would work for the first one, but
> would be way too big for the second one.
>
> This impacts looking at downsampled data greater than 5 minutes (the
> default lookback period) which both Thanos and M3 frequently keep
> (depending on user configuration). Both support downsampling data to user
> configured resolutions, thus lookback delta should be selected according to
> what resolution is being queried when querying downsampled data. Both
> Thanos and M3 depend on Prometheus and use `promql.Engine` for executing
> queries, which doesn't allow setting custom lookback for a particular query.
>
> While I understand that this might be more relevant for M3 and Thanos than
> for Prometheus, just wanted to check what the maintainers of Prometheus
> think about this. If this seems useful, someone from M3 should be able to
> submit a PR (we have already looked at the code and it doesn't look like it
> needs huge changes).
>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-developers/84c03b13-25d7-479c-b678-44cd8625f2c0n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/CAFJ6V0p-_k_Yjy_gUdvHaogxZz%3D7DeNEMM7zz7uBYD%2BYBA%2B6bw%40mail.gmail.com.

[prometheus-developers] Feature request: Setting a custom lookback delta for a query

2021-11-15 Thread Vilius Pranckaitis 
*TL;DR*: ability to set a custom lookback delta for a query would be useful 
when time series have drastically different resolutions.

---

Currently, Prometheus allows configuring lookback delta through 
configuration flag. This works well if the resolutions/scrape intervals of 
time series are similar, but is inconvenient if they differ a lot. E.g. if 
we had two time series, one with a datapoint each hour and another one with 
a datapoint each minute, a lookback of 2h would work for the first one, but 
would be way too big for the second one.

This impacts looking at downsampled data greater than 5 minutes (the 
default lookback period) which both Thanos and M3 frequently keep 
(depending on user configuration). Both support downsampling data to user 
configured resolutions, thus lookback delta should be selected according to 
what resolution is being queried when querying downsampled data. Both 
Thanos and M3 depend on Prometheus and use `promql.Engine` for executing 
queries, which doesn't allow setting custom lookback for a particular query.

While I understand that this might be more relevant for M3 and Thanos than 
for Prometheus, just wanted to check what the maintainers of Prometheus 
think about this. If this seems useful, someone from M3 should be able to 
submit a PR (we have already looked at the code and it doesn't look like it 
needs huge changes).

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/84c03b13-25d7-479c-b678-44cd8625f2c0n%40googlegroups.com.