Re: WIP: parallel GiST index builds

2024-07-26 Thread Andrey M. Borodin
> On 26 Jul 2024, at 14:30, Andreas Karlsson wrote: > > I feel the tricky part about doing that is that we need to make sure the fake > LSNs are all less than the current real LSN when the index build completes > and while that normally should be the case we will have a almost never >

Re: WIP: parallel GiST index builds

2024-07-26 Thread Andreas Karlsson
On 7/22/24 2:08 PM, Andrey M. Borodin wrote: During inserting tuples we need NSN on page. For NSN we can use just a counter, generated by gistGetFakeLSN() which in turn will call GetFakeLSNForUnloggedRel(). Or any other shared counter. After inserting tuples we call log_newpage_range() to

Re: WIP: parallel GiST index builds

2024-07-22 Thread Andrey M. Borodin
> On 22 Jul 2024, at 14:53, Tomas Vondra wrote: > > > > On 7/22/24 13:08, Andrey M. Borodin wrote: >> >> >>> On 22 Jul 2024, at 12:26, Tomas Vondra >>> wrote: >>> >>> I don't understand how would that solve the problem, can you >>> elaborate? Which of the values are you suggesting

Re: WIP: parallel GiST index builds

2024-07-22 Thread Tomas Vondra
On 7/22/24 13:08, Andrey M. Borodin wrote: > > >> On 22 Jul 2024, at 12:26, Tomas Vondra >> wrote: >> >> I don't understand how would that solve the problem, can you >> elaborate? Which of the values are you suggesting should be >> replaced with the shared counter? lastlsn? > > I think

Re: WIP: parallel GiST index builds

2024-07-22 Thread Andrey M. Borodin
> On 22 Jul 2024, at 12:26, Tomas Vondra wrote: > > I don't understand how would that solve the problem, can you elaborate? > Which of the values are you suggesting should be replaced with the > shared counter? lastlsn? I think during build we should consider index unlogged and always use

Re: WIP: parallel GiST index builds

2024-07-22 Thread Tomas Vondra
On 7/22/24 11:00, Andrey M. Borodin wrote: > > >> On 21 Jul 2024, at 23:42, Tomas Vondra wrote: >> >>> >>> 1. Do I get it right that is_parallel argument for gistGetFakeLSN() >>> is only needed for assertion? And this assertion can be ensured just >>> by inspecting code. Is it really necessary?

Re: WIP: parallel GiST index builds

2024-07-22 Thread Andrey M. Borodin
> On 21 Jul 2024, at 23:42, Tomas Vondra wrote: > >> >> 1. Do I get it right that is_parallel argument for gistGetFakeLSN() >> is only needed for assertion? And this assertion can be ensured just >> by inspecting code. Is it really necessary? > > Yes, in the patch it's only used for an

Re: WIP: parallel GiST index builds

2024-07-21 Thread Tomas Vondra
On 7/21/24 21:31, Andrey M. Borodin wrote: > Hi Tomas! > >> On 7 Jun 2024, at 20:41, Tomas Vondra >> wrote: >> >> After looking into parallel builds for BRIN and GIN indexes, I was >> wondering if there's a way to do parallel builds for GiST too. I >> knew next to nothing about how GiST works,

Re: WIP: parallel GiST index builds

2024-07-21 Thread Andrey M. Borodin
Hi Tomas! > On 7 Jun 2024, at 20:41, Tomas Vondra wrote: > > After looking into parallel builds for BRIN and GIN indexes, I was > wondering if there's a way to do parallel builds for GiST too. I knew > next to nothing about how GiST works, but I gave it a shot and here's > what I have - the

Re: WIP: parallel GiST index builds

2024-07-01 Thread Tomas Vondra
Hi, I've done a number of experiments with the GiST parallel builds, both with the sorted and unsorted cases, so let me share some of the results and conclusions from that. In the first post I did some benchmarks using btree_gist, but that seemed not very realistic - there certainly are much

WIP: parallel GiST index builds

2024-06-07 Thread Tomas Vondra
Hi, After looking into parallel builds for BRIN and GIN indexes, I was wondering if there's a way to do parallel builds for GiST too. I knew next to nothing about how GiST works, but I gave it a shot and here's what I have - the attached patch allows parallel GiST builds for the "unsorted" case