Re: Batching page logging during B-tree build

2020-10-12 Thread Dmitry Dolgov
> On Fri, Oct 09, 2020 at 11:08:42PM +0500, Andrey Borodin wrote: > > > 23 сент. 2020 г., в 23:19, Peter Geoghegan написал(а): > > > > On Fri, Sep 18, 2020 at 8:39 AM Andrey M. Borodin > > wrote: > >> Here is PoC with porting that same routine to B-tree. It allows to build > >> B-trees ~10% fas

Re: Batching page logging during B-tree build

2020-10-10 Thread Andrey Borodin
> 23 сент. 2020 г., в 23:29, Andres Freund написал(а): > > Hi, > > On 2020-09-23 11:19:18 -0700, Peter Geoghegan wrote: >> On Fri, Sep 18, 2020 at 8:39 AM Andrey M. Borodin >> wrote: >>> Here is PoC with porting that same routine to B-tree. It allows to build >>> B-trees ~10% faster on my

Re: Batching page logging during B-tree build

2020-10-09 Thread Andrey Borodin
> 23 сент. 2020 г., в 23:19, Peter Geoghegan написал(а): > > On Fri, Sep 18, 2020 at 8:39 AM Andrey M. Borodin > wrote: >> Here is PoC with porting that same routine to B-tree. It allows to build >> B-trees ~10% faster on my machine. > > It doesn't seem to make any difference on my machine

Re: Batching page logging during B-tree build

2020-09-23 Thread Andres Freund
Hi, On 2020-09-23 12:02:42 -0700, Peter Geoghegan wrote: > On Wed, Sep 23, 2020 at 11:29 AM Andres Freund wrote: > > Really should replace WAL compression with lz4 (or possibly zstd). > > Yeah. WAL compression is generally a good idea, and we should probably > find a way to enable it by default

Re: Batching page logging during B-tree build

2020-09-23 Thread Peter Geoghegan
On Wed, Sep 23, 2020 at 11:29 AM Andres Freund wrote: > I wonder what the effect of logging WAL records as huge as this (~256kb) > is on concurrent sessions. I think it's possible that logging 32 pages > at once would cause latency increases for concurrent OLTP-ish > writes. And that a smaller bat

Re: Batching page logging during B-tree build

2020-09-23 Thread Andres Freund
Hi, On 2020-09-23 11:19:18 -0700, Peter Geoghegan wrote: > On Fri, Sep 18, 2020 at 8:39 AM Andrey M. Borodin > wrote: > > Here is PoC with porting that same routine to B-tree. It allows to build > > B-trees ~10% faster on my machine. I wonder what the effect of logging WAL records as huge as t

Re: Batching page logging during B-tree build

2020-09-23 Thread Peter Geoghegan
On Fri, Sep 18, 2020 at 8:39 AM Andrey M. Borodin wrote: > Here is PoC with porting that same routine to B-tree. It allows to build > B-trees ~10% faster on my machine. It doesn't seem to make any difference on my machine, which has an NVME SSD (a Samsung 970 Pro). This is quite a fast SSD, thou

Batching page logging during B-tree build

2020-09-18 Thread Andrey M. Borodin
Hi! There's a thread about GiST build [0]. And we've found out there that logging newly created index with batches of 32 pages is slightly faster. Heikki implemented new logging routine log_newpages() for GiST build and it makes code for this batching nice and clean. Here is PoC with porting th