Re: Yet another vectorized engine

2020-02-27 Thread Konstantin Knizhnik
On 27.02.2020 11:09, Hubert Zhang wrote: Hi Konstantin, I also vimdiff nodeAgg.c in your PG13 branch with nodeAgg.c in pg's main repo. Many functions has changed from PG96 to PG13, e.g. 'advance_aggregates', 'lookup_hash_entry' The vectorized nodeAgg seems still follow the PG96 way of

Re: Yet another vectorized engine

2020-02-27 Thread Hubert Zhang
Hi Konstantin, I also vimdiff nodeAgg.c in your PG13 branch with nodeAgg.c in pg's main repo. Many functions has changed from PG96 to PG13, e.g. 'advance_aggregates', 'lookup_hash_entry' The vectorized nodeAgg seems still follow the PG96 way of implementing these functions. In general, I think

Re: Yet another vectorized engine

2020-02-26 Thread Hubert Zhang
On Wed, Feb 26, 2020 at 7:59 PM Konstantin Knizhnik < k.knizh...@postgrespro.ru> wrote: > > > On 26.02.2020 13:11, Hubert Zhang wrote: > > > >> and with JIT: >> >> 13.88% postgres postgres [.] tts_buffer_heap_getsomeattrs >>7.15% postgres vectorize_engine.so [.]

Re: Yet another vectorized engine

2020-02-26 Thread Konstantin Knizhnik
On 26.02.2020 13:11, Hubert Zhang wrote: and with JIT:  13.88%  postgres  postgres [.] tts_buffer_heap_getsomeattrs    7.15%  postgres  vectorize_engine.so  [.] vfloat8_accum    6.03%  postgres  postgres [.] HeapTupleSatisfiesVisibility   

Re: Yet another vectorized engine

2020-02-26 Thread Hubert Zhang
Hi Konstantin, On Tue, Feb 25, 2020 at 6:44 PM Konstantin Knizhnik < k.knizh...@postgrespro.ru> wrote: > > > On 25.02.2020 11:06, Hubert Zhang wrote: > > Hi Konstantin, > > I checkout your branch pg13 in repo > https://github.com/zhangh43/vectorize_engine > After I fixed some compile error, I

Re: Yet another vectorized engine

2020-02-26 Thread Konstantin Knizhnik
On 25.02.2020 19:40, Konstantin Knizhnik wrote: I have ported vectorize_engine  for zedstore (vertical table AM). Results of TPCH-10G/Q1 are the following: par.workers PG9_6 vectorize=off PG9_6 vectorize=on master vectorize=off jit=on master vectorize=off

Re: Yet another vectorized engine

2020-02-25 Thread Konstantin Knizhnik
I have ported vectorize_engine  for zedstore (vertical table AM). Results of TPCH-10G/Q1 are the following: par.workers PG9_6 vectorize=off PG9_6 vectorize=on master vectorize=off jit=on master vectorize=off jit=off master vectorize=on jit=on master

Re: Yet another vectorized engine

2020-02-25 Thread Konstantin Knizhnik
On 25.02.2020 11:06, Hubert Zhang wrote: Hi Konstantin, I checkout your branch pg13 in repo https://github.com/zhangh43/vectorize_engine After I fixed some compile error, I tested Q1 on TPCH-10G The result is different from yours and vectorize version is too slow. Note that I disable

Re: Yet another vectorized engine

2020-02-25 Thread Hubert Zhang
Hi Konstantin, I checkout your branch pg13 in repo https://github.com/zhangh43/vectorize_engine After I fixed some compile error, I tested Q1 on TPCH-10G The result is different from yours and vectorize version is too slow. Note that I disable parallel worker by default. no JIT no Vectorize: 36

Re: Yet another vectorized engine

2020-02-24 Thread Hubert Zhang
Hi Konstantin, I have added you as a collaborator on github. Please accepted and try again. I think non collaborator could also open pull requests. On Mon, Feb 24, 2020 at 8:02 PM Konstantin Knizhnik < k.knizh...@postgrespro.ru> wrote: > > > On 24.02.2020 05:08, Hubert Zhang wrote: > > Hi > > On

Re: Yet another vectorized engine

2020-02-24 Thread Konstantin Knizhnik
On 24.02.2020 05:08, Hubert Zhang wrote: Hi On Sat, Feb 22, 2020 at 12:58 AM Konstantin Knizhnik mailto:k.knizh...@postgrespro.ru>> wrote: On 12.02.2020 13:12, Hubert Zhang wrote: On Tue, Feb 11, 2020 at 1:20 AM Konstantin Knizhnik mailto:k.knizh...@postgrespro.ru>> wrote:

Re: Yet another vectorized engine

2020-02-23 Thread Hubert Zhang
Hi On Sat, Feb 22, 2020 at 12:58 AM Konstantin Knizhnik < k.knizh...@postgrespro.ru> wrote: > > > On 12.02.2020 13:12, Hubert Zhang wrote: > > On Tue, Feb 11, 2020 at 1:20 AM Konstantin Knizhnik < > k.knizh...@postgrespro.ru> wrote: > >> >> So looks like PG-13 provides significant advantages in

Re: Yet another vectorized engine

2020-02-21 Thread Konstantin Knizhnik
On 12.02.2020 13:12, Hubert Zhang wrote: On Tue, Feb 11, 2020 at 1:20 AM Konstantin Knizhnik mailto:k.knizh...@postgrespro.ru>> wrote: So looks like PG-13 provides significant advantages in OLAP queries comparing with 9.6! Definitely it doesn't mean that vectorized executor is

Re: Yet another vectorized engine

2020-02-12 Thread Hubert Zhang
On Tue, Feb 11, 2020 at 1:20 AM Konstantin Knizhnik < k.knizh...@postgrespro.ru> wrote: > > So looks like PG-13 provides significant advantages in OLAP queries > comparing with 9.6! > Definitely it doesn't mean that vectorized executor is not needed for new > version of Postgres. > Once been

Re: Yet another vectorized engine

2020-02-10 Thread Pavel Stehule
po 10. 2. 2020 v 18:20 odesílatel Konstantin Knizhnik < k.knizh...@postgrespro.ru> napsal: > I have done some performance comparisons. > First of all I failed to run vectorized version of Q1 with master branch > of your repository and > PG9_6_STABLE branch of Postgres: > > NOTICE: query can't be

Re: Yet another vectorized engine

2020-02-10 Thread Konstantin Knizhnik
I have done some performance comparisons. First of all I failed to run vectorized version of Q1 with master branch of your repository and PG9_6_STABLE branch of Postgres: NOTICE:  query can't be vectorized DETAIL:  Non plain agg is not supported I have to switch to pg96 branch. Results

Re: Yet another vectorized engine

2019-12-08 Thread Hubert Zhang
Thanks Konstantin, Your suggestions are very helpful. I have added them into issues of vectorize_engine repo https://github.com/zhangh43/vectorize_engine/issues On Wed, Dec 4, 2019 at 10:08 PM Konstantin Knizhnik < k.knizh...@postgrespro.ru> wrote: > > > On 04.12.2019 12:13, Hubert Zhang wrote:

Re: Yet another vectorized engine

2019-12-04 Thread Konstantin Knizhnik
On 04.12.2019 12:13, Hubert Zhang wrote: 3. Why you have to implement your own plan_tree_mutator and not using expression_tree_mutator? I also want to replace plan node, e.g. Agg->CustomScan(with VectorAgg implementation). expression_tree_mutator cannot be used to mutate plan node such as

Re: Yet another vectorized engine

2019-12-04 Thread Hubert Zhang
Thanks Konstantin for your detailed review! On Tue, Dec 3, 2019 at 5:58 PM Konstantin Knizhnik < k.knizh...@postgrespro.ru> wrote: > > > On 02.12.2019 4:15, Hubert Zhang wrote: > > > The prototype extension is at https://github.com/zhangh43/vectorize_engine > > > I am very sorry, that I have no

Re: Yet another vectorized engine

2019-12-03 Thread Konstantin Knizhnik
On 02.12.2019 4:15, Hubert Zhang wrote: The prototype extension is at https://github.com/zhangh43/vectorize_engine I am very sorry, that I have no followed this link. Few questions concerning your design decisions: 1. Will it be more efficient to use native arrays in vtype instead of

Re: Yet another vectorized engine

2019-12-01 Thread Hubert Zhang
On Sun, Dec 1, 2019 at 10:05 AM Michael Paquier wrote: > On Thu, Nov 28, 2019 at 05:23:59PM +0800, Hubert Zhang wrote: > > Note that the vectorized executor engine is based on PG9.6 now, but it > > could be ported to master / zedstore with some effort. We would > appreciate > > some feedback

Re: Yet another vectorized engine

2019-12-01 Thread Hubert Zhang
Hi Konstantin, Thanks for your reply. On Fri, Nov 29, 2019 at 12:09 AM Konstantin Knizhnik < k.knizh...@postgrespro.ru> wrote: > On 28.11.2019 12:23, Hubert Zhang wrote: > > We just want to introduce another POC for vectorized execution engine > https://github.com/zhangh43/vectorize_engine and

Re: Yet another vectorized engine

2019-11-30 Thread Michael Paquier
On Thu, Nov 28, 2019 at 05:23:59PM +0800, Hubert Zhang wrote: > Note that the vectorized executor engine is based on PG9.6 now, but it > could be ported to master / zedstore with some effort. We would appreciate > some feedback before moving further in that direction. There has been no feedback

Re: Yet another vectorized engine

2019-11-28 Thread Konstantin Knizhnik
On 28.11.2019 12:23, Hubert Zhang wrote: Hi hackers, We just want to introduce another POC for vectorized execution engine https://github.com/zhangh43/vectorize_engine and want to get some feedback on the idea. The basic idea is to extend the TupleTableSlot and introduce

Yet another vectorized engine

2019-11-28 Thread Hubert Zhang
Hi hackers, We just want to introduce another POC for vectorized execution engine https://github.com/zhangh43/vectorize_engine and want to get some feedback on the idea. The basic idea is to extend the TupleTableSlot and introduce VectorTupleTableSlot, which is an array of datums organized by