Re: Partial aggregates pushdown

2022-03-22 Thread Tomas Vondra
On 3/22/22 01:49, Andres Freund wrote: > On 2022-01-17 15:27:53 +0300, Alexander Pyhalov wrote: >> Alexander Pyhalov писал 2022-01-17 15:26: >>> Updated patch. >> >> Sorry, missed attachment. > > Needs another update: http://cfbot.cputube.org/patch_37_3369.log > > Marked as waiting on author. >

Re: Partial aggregates pushdown

2022-03-21 Thread Andres Freund
On 2022-01-17 15:27:53 +0300, Alexander Pyhalov wrote: > Alexander Pyhalov писал 2022-01-17 15:26: > > Updated patch. > > Sorry, missed attachment. Needs another update: http://cfbot.cputube.org/patch_37_3369.log Marked as waiting on author. - Andres

Re: Partial aggregates pushdown

2022-01-17 Thread Alexander Pyhalov
Alexander Pyhalov писал 2022-01-17 15:26: Zhihong Yu писал 2022-01-17 11:43: Hi, + FdwScanPrivateConvertors + * Generate attinmeta if there are some converters: I think it would be better if converter is spelled the same way across the patch. For build_conv_list(): + if

Re: Partial aggregates pushdown

2022-01-17 Thread Alexander Pyhalov
Zhihong Yu писал 2022-01-17 11:43: Hi, + FdwScanPrivateConvertors + * Generate attinmeta if there are some converters: I think it would be better if converter is spelled the same way across the patch. For build_conv_list(): + if (IS_UPPER_REL(foreignrel)) You can return NIL for

Re: Partial aggregates pushdown

2022-01-17 Thread Zhihong Yu
On Sun, Jan 16, 2022 at 11:47 PM Alexander Pyhalov wrote: > Julien Rouhaud писал 2022-01-14 15:16: > > Hi, > > > > On Mon, Nov 15, 2021 at 04:01:51PM +0300, Alexander Pyhalov wrote: > >> > >> I've updated patch - removed catversion dump. > > > > This version of the patchset doesn't apply

Re: Partial aggregates pushdown

2022-01-16 Thread Alexander Pyhalov
Julien Rouhaud писал 2022-01-14 15:16: Hi, On Mon, Nov 15, 2021 at 04:01:51PM +0300, Alexander Pyhalov wrote: I've updated patch - removed catversion dump. This version of the patchset doesn't apply anymore: http://cfbot.cputube.org/patch_36_3369.log === Applying patches on top of

Re: Partial aggregates pushdown

2022-01-14 Thread Julien Rouhaud
Hi, On Mon, Nov 15, 2021 at 04:01:51PM +0300, Alexander Pyhalov wrote: > > I've updated patch - removed catversion dump. This version of the patchset doesn't apply anymore: http://cfbot.cputube.org/patch_36_3369.log === Applying patches on top of PostgreSQL commit ID

Re: Partial aggregates pushdown

2021-11-15 Thread Alexander Pyhalov
Daniel Gustafsson писал 2021-11-15 13:16: On 3 Nov 2021, at 15:50, Alexander Pyhalov wrote: Daniel Gustafsson писал 2021-11-03 16:45: On 2 Nov 2021, at 10:12, Alexander Pyhalov wrote: Updated and rebased patch. + state = (Int128AggState *) palloc0(sizeof(Int128AggState)); +

Re: Partial aggregates pushdown

2021-11-15 Thread Daniel Gustafsson
> On 3 Nov 2021, at 15:50, Alexander Pyhalov wrote: > > Daniel Gustafsson писал 2021-11-03 16:45: >>> On 2 Nov 2021, at 10:12, Alexander Pyhalov wrote: >>> Updated and rebased patch. >> +state = (Int128AggState *) palloc0(sizeof(Int128AggState)); >> +state->calcSumX2 = false; >> + >> +

Re: Partial aggregates pushdown

2021-11-03 Thread Alexander Pyhalov
Daniel Gustafsson писал 2021-11-03 16:45: On 2 Nov 2021, at 10:12, Alexander Pyhalov wrote: Updated and rebased patch. + state = (Int128AggState *) palloc0(sizeof(Int128AggState)); + state->calcSumX2 = false; + + if (!PG_ARGISNULL(0)) + { +#ifdef HAVE_INT128 +

Re: Partial aggregates pushdown

2021-11-03 Thread Daniel Gustafsson
> On 2 Nov 2021, at 10:12, Alexander Pyhalov wrote: > Updated and rebased patch. + state = (Int128AggState *) palloc0(sizeof(Int128AggState)); + state->calcSumX2 = false; + + if (!PG_ARGISNULL(0)) + { +#ifdef HAVE_INT128 + do_int128_accum(state, (int128)

Re: Partial aggregates pushdown

2021-11-02 Thread Alexander Pyhalov
Hi. Updated and rebased patch. Ilya Gladyshev писал 2021-11-02 00:31: Hi, On 21.10.2021 13:55, Alexander Pyhalov wrote: Hi. Updated patch. Now aggregates with internal states can be pushed down, if they are marked as pushdown safe (this flag is set to true for min/max/sum), have internal

Re: Partial aggregates pushdown

2021-11-01 Thread Tomas Vondra
On 11/1/21 22:53, Ilya Gladyshev wrote: On 01.11.2021 13:30, Alexander Pyhalov wrote: Peter Eisentraut писал 2021-11-01 12:47: On 21.10.21 12:55, Alexander Pyhalov wrote: Now aggregates with internal states can be pushed down, if they are marked as pushdown safe (this flag is set to true

Re: Partial aggregates pushdown

2021-11-01 Thread Tomas Vondra
On 11/1/21 22:31, Ilya Gladyshev wrote: Hi, On 21.10.2021 13:55, Alexander Pyhalov wrote: Hi. Updated patch. Now aggregates with internal states can be pushed down, if they are marked as pushdown safe (this flag is set to true for min/max/sum), have internal states and associated

Re: Partial aggregates pushdown

2021-11-01 Thread Ilya Gladyshev
On 01.11.2021 13:30, Alexander Pyhalov wrote: Peter Eisentraut писал 2021-11-01 12:47: On 21.10.21 12:55, Alexander Pyhalov wrote: Now aggregates with internal states can be pushed down, if they are marked as pushdown safe (this flag is set to true for min/max/sum), have internal states and

Re: Partial aggregates pushdown

2021-11-01 Thread Ilya Gladyshev
Hi, On 21.10.2021 13:55, Alexander Pyhalov wrote: Hi. Updated patch. Now aggregates with internal states can be pushed down, if they are marked as pushdown safe (this flag is set to true for min/max/sum), have internal states and associated converters. I don't quite understand why this is

Re: Partial aggregates pushdown

2021-11-01 Thread Alexander Pyhalov
Peter Eisentraut писал 2021-11-01 12:47: On 21.10.21 12:55, Alexander Pyhalov wrote: Now aggregates with internal states can be pushed down, if they are marked as pushdown safe (this flag is set to true for min/max/sum), have internal states and associated converters. Converters are called

Re: Partial aggregates pushdown

2021-11-01 Thread Peter Eisentraut
On 21.10.21 12:55, Alexander Pyhalov wrote: Now aggregates with internal states can be pushed down, if they are marked as pushdown safe (this flag is set to true for min/max/sum), have internal states and associated converters. Converters are called locally, they transform aggregate result to

Re: Partial aggregates pushdown

2021-10-22 Thread Alexander Pyhalov
Zhihong Yu писал 2021-10-22 00:43: Hi, w.r.t. 0001-Partial-aggregates-push-down-v03.patch Hi. For partial_agg_ok(), + if (agg->aggdistinct || agg->aggvariadic || agg->aggkind != AGGKIND_NORMAL || agg->aggorder != NIL) + ok = false; Since SearchSysCache1() is not called yet, you

Re: Partial aggregates pushdown

2021-10-21 Thread Zhihong Yu
Hi, w.r.t. 0001-Partial-aggregates-push-down-v03.patch For partial_agg_ok(), + if (agg->aggdistinct || agg->aggvariadic || agg->aggkind != AGGKIND_NORMAL || agg->aggorder != NIL) + ok = false; Since SearchSysCache1() is not called yet, you can return false directly. + if

Re: Partial aggregates pushdown

2021-10-21 Thread Alexander Pyhalov
Tomas Vondra писал 2021-10-19 16:25: On 10/19/21 08:56, Alexander Pyhalov wrote: Hi. Tomas Vondra писал 2021-10-15 17:56: As for the proposed approach, it's probably good enough for the first version to restrict this to aggregates where the aggregate result is sufficient, i.e. we don't need

Re: Partial aggregates pushdown

2021-10-19 Thread Tomas Vondra
On 10/19/21 08:56, Alexander Pyhalov wrote: Hi. Tomas Vondra писал 2021-10-15 17:56: As for the proposed approach, it's probably good enough for the first version to restrict this to aggregates where the aggregate result is sufficient, i.e. we don't need any new export/import procedures. But

Re: Partial aggregates pushdown

2021-10-19 Thread Alexander Pyhalov
Hi. Tomas Vondra писал 2021-10-15 17:56: As for the proposed approach, it's probably good enough for the first version to restrict this to aggregates where the aggregate result is sufficient, i.e. we don't need any new export/import procedures. But it's very unlikely we'd want to restrict it

Re: Partial aggregates pushdown

2021-10-15 Thread Tomas Vondra
On 10/15/21 21:31, Stephen Frost wrote: Greetings, * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: On 10/15/21 17:05, Alexander Pyhalov wrote: Tomas Vondra писал 2021-10-15 17:56: And then we should extend this for aggregates with more complex internal states (e.g. avg), by supporting

Re: Partial aggregates pushdown

2021-10-15 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > On 10/15/21 17:05, Alexander Pyhalov wrote: > >Tomas Vondra писал 2021-10-15 17:56: > >>And then we should extend this for aggregates with more complex > >>internal states (e.g. avg), by supporting a function that "exports" >

Re: Partial aggregates pushdown

2021-10-15 Thread Tomas Vondra
On 10/15/21 17:05, Alexander Pyhalov wrote: Tomas Vondra писал 2021-10-15 17:56: Hi Alexander, Hi. And then we should extend this for aggregates with more complex internal states (e.g. avg), by supporting a function that "exports" the aggregate state - similar to serial/deserial functions,

Re: Partial aggregates pushdown

2021-10-15 Thread Alexander Pyhalov
Tomas Vondra писал 2021-10-15 17:56: Hi Alexander, Hi. And then we should extend this for aggregates with more complex internal states (e.g. avg), by supporting a function that "exports" the aggregate state - similar to serial/deserial functions, but needs to be portable. I think the

Re: Partial aggregates pushdown

2021-10-15 Thread Tomas Vondra
Hi Alexander, On 10/15/21 15:15, Alexander Pyhalov wrote: Hi. One of the issues when we try to use sharding in PostgreSQL is absence of partial aggregates pushdown. I see several opportunities to alleviate this issue. If we look at Citus, it implements aggregate, calculating internal state

Partial aggregates pushdown

2021-10-15 Thread Alexander Pyhalov
Hi. One of the issues when we try to use sharding in PostgreSQL is absence of partial aggregates pushdown. I see several opportunities to alleviate this issue. If we look at Citus, it implements aggregate, calculating internal state of an arbitrary agregate function and exporting it as text

<    1   2