Re: Table AM Interface Enhancements

2024-06-22 Thread Alexander Korotkov
On Fri, Jun 21, 2024 at 7:37 PM Matthias van de Meent wrote: > On Tue, 16 Apr 2024 at 12:34, Alexander Korotkov wrote: > > > > You're right. No sense trying to fix this. Reverted. > > I just noticed that this revert (commit 6377e12a) seems to have > introduced two comment blocks atop TableAmRou

Re: Table AM Interface Enhancements

2024-06-21 Thread Matthias van de Meent
Hi, On Tue, 16 Apr 2024 at 12:34, Alexander Korotkov wrote: > > You're right. No sense trying to fix this. Reverted. I just noticed that this revert (commit 6377e12a) seems to have introduced two comment blocks atop TableAmRoutine's scan_analyze_next_block, and I can't find a clear reason why

Re: Table AM Interface Enhancements

2024-05-23 Thread Mats Kindahl
On Thu, Nov 23, 2023 at 1:43 PM Alexander Korotkov wrote: > Hello PostgreSQL Hackers, > > I am pleased to submit a series of patches related to the Table Access > Method (AM) interface, which I initially announced during my talk at > PGCon 2023 [1]. These patches are primarily designed to support

Re: Table AM Interface Enhancements

2024-04-16 Thread Andres Freund
Hi, On 2024-04-16 08:31:24 -0400, Robert Haas wrote: > On Tue, Apr 16, 2024 at 6:52 AM Alexander Korotkov > wrote: > > Taking a closer look at acquire_sample_rows(), I think it would be > > good if table AM implementation would care about block-level (or > > whatever-level) sampling. So that ac

Re: Table AM Interface Enhancements

2024-04-16 Thread Andres Freund
On 2024-04-16 13:33:53 +0300, Alexander Korotkov wrote: > Reverted. Thanks!

Re: Table AM Interface Enhancements

2024-04-16 Thread Robert Haas
On Tue, Apr 16, 2024 at 6:52 AM Alexander Korotkov wrote: > Taking a closer look at acquire_sample_rows(), I think it would be > good if table AM implementation would care about block-level (or > whatever-level) sampling. So that acquire_sample_rows() just fetches > tuples one-by-one from table A

Re: Table AM Interface Enhancements

2024-04-16 Thread Pavel Borisov
On Tue, 16 Apr 2024 at 14:52, Alexander Korotkov wrote: > On Mon, Apr 15, 2024 at 11:17 PM Andres Freund wrote: > > On 2024-04-15 16:02:00 -0400, Robert Haas wrote: > > > Do you want that patch applied, not applied, or applied with some set > of > > > modifications? > > > > I think we should app

Re: Table AM Interface Enhancements

2024-04-16 Thread Alexander Korotkov
On Mon, Apr 15, 2024 at 11:17 PM Andres Freund wrote: > On 2024-04-15 16:02:00 -0400, Robert Haas wrote: > > Do you want that patch applied, not applied, or applied with some set of > > modifications? > > I think we should apply Alexander's proposed revert and then separately > discuss what we sho

Re: Table AM Interface Enhancements

2024-04-16 Thread Alexander Korotkov
On Mon, Apr 15, 2024 at 11:11 PM Andres Freund wrote: > On 2024-04-12 01:04:03 +0300, Alexander Korotkov wrote: > > 1) If we just apply my revert patch and leave c6fc50cb4028 and > > 041b96802ef in the tree, then we get our table AM API narrowed. As > > you expressed the current API requires bloc

Re: Table AM Interface Enhancements

2024-04-15 Thread Andres Freund
Hi, On 2024-04-15 16:02:00 -0400, Robert Haas wrote: > On Mon, Apr 15, 2024 at 3:47 PM Andres Freund wrote: > > That said, I don't like the state after applying > > https://postgr.es/m/CAPpHfdvuT6DnguzaV-M1UQ2whYGDojaNU%3D-%3DiHc0A7qo9HBEJw%40mail.gmail.com > > because there's too much coupling.

Re: Table AM Interface Enhancements

2024-04-15 Thread Andres Freund
Hi, On 2024-04-12 01:04:03 +0300, Alexander Korotkov wrote: > 1) If we just apply my revert patch and leave c6fc50cb4028 and > 041b96802ef in the tree, then we get our table AM API narrowed. As > you expressed the current API requires block numbers to be 1:1 with > the actual physical on-disk loc

Re: Table AM Interface Enhancements

2024-04-15 Thread Robert Haas
On Mon, Apr 15, 2024 at 3:47 PM Andres Freund wrote: > That said, I don't like the state after applying > https://postgr.es/m/CAPpHfdvuT6DnguzaV-M1UQ2whYGDojaNU%3D-%3DiHc0A7qo9HBEJw%40mail.gmail.com > because there's too much coupling. Hence talking about needing to iterate on > the interface in s

Re: Table AM Interface Enhancements

2024-04-15 Thread Andres Freund
Hi, On 2024-04-15 23:14:01 +0400, Pavel Borisov wrote: > Why it makes a difference looks a little bit unclear to me, I can't comment > on this. I noticed that before 041b96802ef we had a block number and block > sampler state that tied acquire_sample_rows() to the actual block > structure. That,

Re: Table AM Interface Enhancements

2024-04-15 Thread Pavel Borisov
On Mon, 15 Apr 2024 at 22:09, Robert Haas wrote: > On Mon, Apr 15, 2024 at 12:37 PM Pavel Borisov > wrote: > > In my understanding, the downside of 041b96802ef is bringing > read_stream* things from being heap-only-related up to the level of > acquire_sample_rows() that is not supposed to be tie

Re: Table AM Interface Enhancements

2024-04-15 Thread Robert Haas
On Mon, Apr 15, 2024 at 12:41 PM Nazir Bilal Yavuz wrote: > I agree with you but I did not understand one thing. If out-of-core > AMs are used, does not all block sampling logic (BlockSampler_Init(), > BlockSampler_Next() etc.) need to be edited as well since these > functions assume block numbers

Re: Table AM Interface Enhancements

2024-04-15 Thread Robert Haas
On Mon, Apr 15, 2024 at 12:37 PM Pavel Borisov wrote: > In my understanding, the downside of 041b96802ef is bringing read_stream* > things from being heap-only-related up to the level of acquire_sample_rows() > that is not supposed to be tied to heap. And changing *_analyze_next_block() > funct

Re: Table AM Interface Enhancements

2024-04-15 Thread Nazir Bilal Yavuz
Hi, On Mon, 15 Apr 2024 at 18:36, Robert Haas wrote: > > On Sat, Apr 13, 2024 at 5:28 AM Alexander Korotkov > wrote: > > Yes, I think so. Table AM API deals with TIDs and block numbers, but > > doesn't force on what they actually mean. For example, in ZedStore > > [1], data is stored on per-c

Re: Table AM Interface Enhancements

2024-04-15 Thread Pavel Borisov
On Mon, 15 Apr 2024 at 19:36, Robert Haas wrote: > On Sat, Apr 13, 2024 at 5:28 AM Alexander Korotkov > wrote: > > Yes, I think so. Table AM API deals with TIDs and block numbers, but > > doesn't force on what they actually mean. For example, in ZedStore > > [1], data is stored on per-column B

Re: Table AM Interface Enhancements

2024-04-15 Thread Robert Haas
On Sat, Apr 13, 2024 at 5:28 AM Alexander Korotkov wrote: > Yes, I think so. Table AM API deals with TIDs and block numbers, but > doesn't force on what they actually mean. For example, in ZedStore > [1], data is stored on per-column B-trees, where TID used in table AM > is just a logical key of

Re: Table AM Interface Enhancements

2024-04-13 Thread Alexander Korotkov
Hi, Melanie! On Fri, Apr 12, 2024 at 8:48 PM Melanie Plageman wrote: > > On Thu, Apr 11, 2024 at 6:04 PM Alexander Korotkov > wrote: > > > > On Fri, Apr 12, 2024 at 12:04 AM Andres Freund wrote: > > > On 2024-04-11 20:46:02 +0300, Alexander Korotkov wrote: > > > > I hope this work is targeting

Re: Table AM Interface Enhancements

2024-04-12 Thread Melanie Plageman
On Thu, Apr 11, 2024 at 6:04 PM Alexander Korotkov wrote: > > On Fri, Apr 12, 2024 at 12:04 AM Andres Freund wrote: > > On 2024-04-11 20:46:02 +0300, Alexander Korotkov wrote: > > > I hope this work is targeting pg18. > > > > I think anything of the scope discussed by Melanie would be very clearl

Re: Table AM Interface Enhancements

2024-04-11 Thread Alexander Korotkov
On Thu, Apr 11, 2024 at 11:27 PM Robert Haas wrote: > On Thu, Apr 11, 2024 at 1:46 PM Alexander Korotkov > wrote: > > I understand that I'm the bad guy of this release, not sure if my > > opinion counts. > > > > But what is going on here? I hope this work is targeting pg18. > > Otherwise, do I

Re: Table AM Interface Enhancements

2024-04-11 Thread Alexander Korotkov
On Fri, Apr 12, 2024 at 12:04 AM Andres Freund wrote: > On 2024-04-11 20:46:02 +0300, Alexander Korotkov wrote: > > I hope this work is targeting pg18. > > I think anything of the scope discussed by Melanie would be very clearly > targeting 18. For 17, I don't know yet whether we should revert the

Re: Table AM Interface Enhancements

2024-04-11 Thread Andres Freund
Hi, On 2024-04-11 20:46:02 +0300, Alexander Korotkov wrote: > I hope this work is targeting pg18. I think anything of the scope discussed by Melanie would be very clearly targeting 18. For 17, I don't know yet whether we should revert the the ANALYZE streaming read user (041b96802ef), just do a b

Re: Table AM Interface Enhancements

2024-04-11 Thread Robert Haas
On Thu, Apr 11, 2024 at 1:46 PM Alexander Korotkov wrote: > I understand that I'm the bad guy of this release, not sure if my > opinion counts. > > But what is going on here? I hope this work is targeting pg18. > Otherwise, do I get this right that this post feature-freeze works on > designing a

Re: Table AM Interface Enhancements

2024-04-11 Thread Melanie Plageman
On Thu, Apr 11, 2024 at 12:19 PM Melanie Plageman wrote: > > On Wed, Apr 10, 2024 at 5:21 PM Andres Freund wrote: > > > > Hi, > > > > On 2024-04-10 16:50:44 -0400, Melanie Plageman wrote: > > > This brings up a question about the prefetching. We never had to have > > > this discussion for sequent

Re: Table AM Interface Enhancements

2024-04-11 Thread Alexander Korotkov
Hi! On Thu, Apr 11, 2024 at 7:19 PM Melanie Plageman wrote: > On Wed, Apr 10, 2024 at 5:21 PM Andres Freund wrote: > > On 2024-04-10 16:50:44 -0400, Melanie Plageman wrote: > > > This brings up a question about the prefetching. We never had to have > > > this discussion for sequential scan strea

Re: Table AM Interface Enhancements

2024-04-11 Thread Alexander Korotkov
On Thu, Apr 11, 2024 at 8:11 PM Jeff Davis wrote: > On Wed, 2024-04-10 at 15:19 +0300, Alexander Korotkov wrote: > > 1) 9bd99f4c26 comprises the reworked patch after working with notes > > from Jeff Davis. I agree it would be better to wait for him to > > express explicit agreement. Before rever

Re: Table AM Interface Enhancements

2024-04-11 Thread Jeff Davis
On Wed, 2024-04-10 at 15:19 +0300, Alexander Korotkov wrote: > 1) 9bd99f4c26 comprises the reworked patch after working with notes > from Jeff Davis.  I agree it would be better to wait for him to > express explicit agreement.  Before reverting this, I would prefer to > hear his opinion. On this p

Re: Table AM Interface Enhancements

2024-04-11 Thread Melanie Plageman
On Wed, Apr 10, 2024 at 5:21 PM Andres Freund wrote: > > Hi, > > On 2024-04-10 16:50:44 -0400, Melanie Plageman wrote: > > This brings up a question about the prefetching. We never had to have > > this discussion for sequential scan streaming read because it didn't > > (and still doesn't) do prefe

Re: Table AM Interface Enhancements

2024-04-11 Thread Alexander Korotkov
Hi Andres, On Wed, Apr 10, 2024 at 7:52 PM Andres Freund wrote: > On 2024-04-08 14:54:46 -0400, Robert Haas wrote: > > Exactly how much is getting reverted here? I see these, all since March > > 23rd: > > IMO: > > > > dd1f6b0c17 Provide a way block-level table AMs could re-use > > acquire_sample

Re: Table AM Interface Enhancements

2024-04-10 Thread Andres Freund
Hi, On 2024-04-10 16:50:44 -0400, Melanie Plageman wrote: > This brings up a question about the prefetching. We never had to have > this discussion for sequential scan streaming read because it didn't > (and still doesn't) do prefetching. But, if we push the streaming read > code down into the hea

Re: Table AM Interface Enhancements

2024-04-10 Thread Melanie Plageman
On Wed, Apr 10, 2024 at 4:33 PM Andres Freund wrote: > > Hi, > > On 2024-04-10 16:24:40 -0400, Melanie Plageman wrote: > > This thread has been moving pretty fast, so could someone point out > > which version of the patch has the modifications to > > acquire_sample_rows() that would be relevant fo

Re: Table AM Interface Enhancements

2024-04-10 Thread Andres Freund
Hi, On 2024-04-10 16:24:40 -0400, Melanie Plageman wrote: > This thread has been moving pretty fast, so could someone point out > which version of the patch has the modifications to > acquire_sample_rows() that would be relevant for Bilal (and others > involved in analyze streaming read) to review

Re: Table AM Interface Enhancements

2024-04-10 Thread Melanie Plageman
On Wed, Apr 10, 2024 at 4:03 PM Andres Freund wrote: > > Hi, > > On 2024-04-10 15:19:47 +0300, Alexander Korotkov wrote: > > On Mon, Apr 8, 2024 at 9:54 PM Robert Haas wrote: > > > On Mon, Apr 8, 2024 at 12:33 PM Alexander Korotkov > > > wrote: > > > > Yes, it was my mistake. I got rushing tryi

Re: Table AM Interface Enhancements

2024-04-10 Thread Andres Freund
Hi, On 2024-04-10 15:19:47 +0300, Alexander Korotkov wrote: > On Mon, Apr 8, 2024 at 9:54 PM Robert Haas wrote: > > On Mon, Apr 8, 2024 at 12:33 PM Alexander Korotkov > > wrote: > > > Yes, it was my mistake. I got rushing trying to fit this to FF, even > > > doing significant changes just befo

Re: Table AM Interface Enhancements

2024-04-10 Thread Bruce Momjian
On Wed, Apr 10, 2024 at 05:42:51PM +0400, Pavel Borisov wrote: > Hi, Alexander! > In my view, the actual list of what has raised discussion is: > dd1f6b0c17 Provide a way block-level table AMs could re-use > acquire_sample_rows > () > 27bc1772fc Generalize relation analyze in table AM interface >

Re: Table AM Interface Enhancements

2024-04-10 Thread Peter Geoghegan
On Wed, Apr 10, 2024 at 1:25 PM Robert Haas wrote: > That is somewhat fair, but it is also a lot of work. There are > multiple people asking for you to revert things on multiple threads, > and figuring out all of the revert requests and trying to come to some > consensus about what should be done

Re: Table AM Interface Enhancements

2024-04-10 Thread Robert Haas
On Wed, Apr 10, 2024 at 12:36 PM Alexander Korotkov wrote: > But I have to mention that even that I've committed table AM stuff > close to the FF, there has been quite amount of depended work > committed. So, revert of these patches is promising to be not > something immediate and easy, which req

Re: Table AM Interface Enhancements

2024-04-10 Thread Andres Freund
Hi, On 2024-04-08 14:54:46 -0400, Robert Haas wrote: > Exactly how much is getting reverted here? I see these, all since March 23rd: IMO: > dd1f6b0c17 Provide a way block-level table AMs could re-use > acquire_sample_rows() Should be reverted. > 9bd99f4c26 Custom reloptions for table AM Hm.

Re: Table AM Interface Enhancements

2024-04-10 Thread Alexander Korotkov
On Wed, Apr 10, 2024 at 4:19 PM Robert Haas wrote: > > On Wed, Apr 10, 2024 at 8:20 AM Alexander Korotkov > wrote: > > I agree with the facts. But I have a different interpretation on > > this. The patch was committed as 11470f544e on March 23, 2023, then > > reverted on April 3. I've propose

Re: Table AM Interface Enhancements

2024-04-10 Thread Joe Conway
On 4/10/24 09:19, Robert Haas wrote: When you commit a patch and another committer writes a post-commit review saying that your patch has so many serious problems that he gave up on reviewing before enumerating all of them, that's a really bad sign. That should be a strong signal to you to step b

Re: Table AM Interface Enhancements

2024-04-10 Thread Pavel Borisov
Hi, Alexander! On Wed, 10 Apr 2024 at 16:20, Alexander Korotkov wrote: > On Mon, Apr 8, 2024 at 9:54 PM Robert Haas wrote: > > On Mon, Apr 8, 2024 at 12:33 PM Alexander Korotkov > wrote: > > > Yes, it was my mistake. I got rushing trying to fit this to FF, even > doing significant changes just

Re: Table AM Interface Enhancements

2024-04-10 Thread Robert Haas
On Wed, Apr 10, 2024 at 8:20 AM Alexander Korotkov wrote: > I agree with the facts. But I have a different interpretation on > this. The patch was committed as 11470f544e on March 23, 2023, then > reverted on April 3. I've proposed the revised version, but Andres > complained that this is the n

Re: Table AM Interface Enhancements

2024-04-10 Thread Alexander Korotkov
On Mon, Apr 8, 2024 at 9:54 PM Robert Haas wrote: > On Mon, Apr 8, 2024 at 12:33 PM Alexander Korotkov > wrote: > > Yes, it was my mistake. I got rushing trying to fit this to FF, even doing > > significant changes just before commit. > > I'll revert this later today. The patch to revert is at

Re: Table AM Interface Enhancements

2024-04-08 Thread Alexander Korotkov
On Mon, Apr 8, 2024 at 9:54 PM Robert Haas wrote: > On Mon, Apr 8, 2024 at 12:33 PM Alexander Korotkov > wrote: > > Yes, it was my mistake. I got rushing trying to fit this to FF, even doing > > significant changes just before commit. > > I'll revert this later today. It appears to be a non-tr

Re: Table AM Interface Enhancements

2024-04-08 Thread Robert Haas
On Mon, Apr 8, 2024 at 12:33 PM Alexander Korotkov wrote: > Yes, it was my mistake. I got rushing trying to fit this to FF, even doing > significant changes just before commit. > I'll revert this later today. Alexander, Exactly how much is getting reverted here? I see these, all since March 23r

Re: Table AM Interface Enhancements

2024-04-08 Thread Alexander Korotkov
On Mon, Apr 8, 2024, 19:08 Andres Freund wrote: > On 2024-04-08 08:37:44 -0700, Andres Freund wrote: > > On 2024-04-08 11:17:51 +0400, Pavel Borisov wrote: > > > On Mon, 8 Apr 2024 at 03:25, Alexander Korotkov > > > > I was under the impression there are not so many out-of-core table > > > > AMs

Re: Table AM Interface Enhancements

2024-04-08 Thread Andres Freund
On 2024-04-08 08:37:44 -0700, Andres Freund wrote: > On 2024-04-08 11:17:51 +0400, Pavel Borisov wrote: > > On Mon, 8 Apr 2024 at 03:25, Alexander Korotkov > > > I was under the impression there are not so many out-of-core table > > > AMs, which have non-dummy analysis implementations. And even i

Re: Table AM Interface Enhancements

2024-04-08 Thread Andres Freund
Hi, On 2024-04-08 11:17:51 +0400, Pavel Borisov wrote: > On Mon, 8 Apr 2024 at 03:25, Alexander Korotkov > > I was under the impression there are not so many out-of-core table > > AMs, which have non-dummy analysis implementations. And even if there > > are some, duplicating acquire_sample_rows(

Re: Table AM Interface Enhancements

2024-04-08 Thread Pavel Borisov
Hi, Alexander On Mon, 8 Apr 2024 at 13:59, Pavel Borisov wrote: > > > On Mon, 8 Apr 2024 at 13:34, Alexander Korotkov > wrote: > >> On Mon, Apr 8, 2024 at 10:18 AM Pavel Borisov >> wrote: >> > On Mon, 8 Apr 2024 at 03:25, Alexander Korotkov >> wrote: >> >> >> >> Hi, >> >> >> >> On Mon, Apr 8,

Re: Table AM Interface Enhancements

2024-04-08 Thread Pavel Borisov
On Mon, 8 Apr 2024 at 13:34, Alexander Korotkov wrote: > On Mon, Apr 8, 2024 at 10:18 AM Pavel Borisov > wrote: > > On Mon, 8 Apr 2024 at 03:25, Alexander Korotkov > wrote: > >> > >> Hi, > >> > >> On Mon, Apr 8, 2024 at 12:40 AM Andres Freund > wrote: > >> > On 2024-03-30 23:33:04 +0200, Alexa

Re: Table AM Interface Enhancements

2024-04-08 Thread Alexander Korotkov
On Mon, Apr 8, 2024 at 10:18 AM Pavel Borisov wrote: > On Mon, 8 Apr 2024 at 03:25, Alexander Korotkov wrote: >> >> Hi, >> >> On Mon, Apr 8, 2024 at 12:40 AM Andres Freund wrote: >> > On 2024-03-30 23:33:04 +0200, Alexander Korotkov wrote: >> > > I've pushed 0001, 0002 and 0006. >> > >> > I brie

Re: Table AM Interface Enhancements

2024-04-08 Thread Pavel Borisov
Hi, Alexander and Andres! On Mon, 8 Apr 2024 at 03:25, Alexander Korotkov wrote: > Hi, > > On Mon, Apr 8, 2024 at 12:40 AM Andres Freund wrote: > > On 2024-03-30 23:33:04 +0200, Alexander Korotkov wrote: > > > I've pushed 0001, 0002 and 0006. > > > > I briefly looked at 27bc1772fc81 and I don't

Re: Table AM Interface Enhancements

2024-04-07 Thread Andres Freund
Hi, On 2024-04-08 02:25:17 +0300, Alexander Korotkov wrote: > On Mon, Apr 8, 2024 at 12:40 AM Andres Freund wrote: > > On 2024-03-30 23:33:04 +0200, Alexander Korotkov wrote: > > > I've pushed 0001, 0002 and 0006. > > > > I briefly looked at 27bc1772fc81 and I don't think the state post this > >

Re: Table AM Interface Enhancements

2024-04-07 Thread Alexander Korotkov
On Mon, Apr 8, 2024 at 2:25 AM Alexander Korotkov wrote: > On Mon, Apr 8, 2024 at 12:40 AM Andres Freund wrote: > > On 2024-03-30 23:33:04 +0200, Alexander Korotkov wrote: > > > I've pushed 0001, 0002 and 0006. > > > > I briefly looked at 27bc1772fc81 and I don't think the state post this > > co

Re: Table AM Interface Enhancements

2024-04-07 Thread Alexander Korotkov
Hi, On Mon, Apr 8, 2024 at 12:40 AM Andres Freund wrote: > On 2024-03-30 23:33:04 +0200, Alexander Korotkov wrote: > > I've pushed 0001, 0002 and 0006. > > I briefly looked at 27bc1772fc81 and I don't think the state post this commit > makes sense. Before this commit another block based AM could

Re: Table AM Interface Enhancements

2024-04-07 Thread Andres Freund
Hi, On 2024-03-30 23:33:04 +0200, Alexander Korotkov wrote: > I've pushed 0001, 0002 and 0006. I briefly looked at 27bc1772fc81 and I don't think the state post this commit makes sense. Before this commit another block based AM could implement analyze without much code duplication. Now a large po

Re: Table AM Interface Enhancements

2024-04-07 Thread Pavel Borisov
Hi, Alexander! On Sun, 7 Apr 2024 at 12:34, Pavel Borisov wrote: > Hi, Alexander! > > On Sun, 7 Apr 2024 at 07:33, Alexander Korotkov > wrote: > >> Hi, Pavel! >> >> On Fri, Apr 5, 2024 at 6:58 PM Pavel Borisov >> wrote: >> > On Tue, 2 Apr 2024 at 19:17, Jeff Davis wrote: >> >> >> >> On Tue, 2

Re: Table AM Interface Enhancements

2024-04-07 Thread Pavel Borisov
Hi, Alexander! On Sun, 7 Apr 2024 at 07:33, Alexander Korotkov wrote: > Hi, Pavel! > > On Fri, Apr 5, 2024 at 6:58 PM Pavel Borisov > wrote: > > On Tue, 2 Apr 2024 at 19:17, Jeff Davis wrote: > >> > >> On Tue, 2024-04-02 at 11:49 +0300, Alexander Korotkov wrote: > >> > I don't like the idea th

Re: Table AM Interface Enhancements

2024-04-06 Thread Alexander Korotkov
Hi, Pavel! On Fri, Apr 5, 2024 at 6:58 PM Pavel Borisov wrote: > On Tue, 2 Apr 2024 at 19:17, Jeff Davis wrote: >> >> On Tue, 2024-04-02 at 11:49 +0300, Alexander Korotkov wrote: >> > I don't like the idea that every custom table AM reltoptions should >> > begin with StdRdOptions. I would rathe

Re: Table AM Interface Enhancements

2024-04-05 Thread Pavel Borisov
Hi, hackers! On Tue, 2 Apr 2024 at 19:17, Jeff Davis wrote: > On Tue, 2024-04-02 at 11:49 +0300, Alexander Korotkov wrote: > > I don't like the idea that every custom table AM reltoptions should > > begin with StdRdOptions. I would rather introduce the new data > > structure with table options,

Re: Table AM Interface Enhancements

2024-04-02 Thread Jeff Davis
On Tue, 2024-04-02 at 11:49 +0300, Alexander Korotkov wrote: > I don't like the idea that every custom table AM reltoptions should > begin with StdRdOptions.  I would rather introduce the new data > structure with table options, which need to be accessed outside of > table AM.  Then reloptions will

Re: Table AM Interface Enhancements

2024-04-02 Thread Alexander Korotkov
Hi, Jeff! On Tue, Apr 2, 2024 at 8:19 AM Jeff Davis wrote: > On Sat, 2024-03-30 at 23:33 +0200, Alexander Korotkov wrote: > > I've pushed 0001, 0002 and 0006. > > Sorry to jump in to this discussion so late. I had worked on something > like the custom reloptions (0002) in the past, and there were

Re: Table AM Interface Enhancements

2024-04-01 Thread Jeff Davis
On Sat, 2024-03-30 at 23:33 +0200, Alexander Korotkov wrote: > I've pushed 0001, 0002 and 0006. Sorry to jump in to this discussion so late. I had worked on something like the custom reloptions (0002) in the past, and there were some complications that don't seem to be addressed in commit c95c25f9

Re: Table AM Interface Enhancements

2024-04-01 Thread Japin Li
On Tue, 02 Apr 2024 at 07:59, Alexander Korotkov wrote: > On Mon, Apr 1, 2024 at 7:36 PM Tom Lane wrote: >> Coverity complained about what you did in RelationParseRelOptions >> in c95c25f9a: >> >> *** CID 1595992: Null pointer dereferences (FORWARD_NULL) >> /srv/coverity/git/pgsql-git/postgre

Re: Table AM Interface Enhancements

2024-04-01 Thread Alexander Korotkov
On Mon, Apr 1, 2024 at 7:36 PM Tom Lane wrote: > Coverity complained about what you did in RelationParseRelOptions > in c95c25f9a: > > *** CID 1595992: Null pointer dereferences (FORWARD_NULL) > /srv/coverity/git/pgsql-git/postgresql/src/backend/utils/cache/relcache.c: > 499 in RelationParseRel

Re: Table AM Interface Enhancements

2024-04-01 Thread Alexander Korotkov
On Mon, Apr 1, 2024 at 7:36 PM Tom Lane wrote: > > Coverity complained about what you did in RelationParseRelOptions > in c95c25f9a: > > *** CID 1595992: Null pointer dereferences (FORWARD_NULL) > /srv/coverity/git/pgsql-git/postgresql/src/backend/utils/cache/relcache.c: > 499 in RelationParseR

Re: Table AM Interface Enhancements

2024-04-01 Thread Tom Lane
Coverity complained about what you did in RelationParseRelOptions in c95c25f9a: *** CID 1595992: Null pointer dereferences (FORWARD_NULL) /srv/coverity/git/pgsql-git/postgresql/src/backend/utils/cache/relcache.c: 499 in RelationParseRelOptions() 493 494 /* 495 * Fe

Re: Table AM Interface Enhancements

2024-04-01 Thread Andrei Lepikhov
On 31/3/2024 00:33, Alexander Korotkov wrote: I think the way forward might be to introduce the new API, which would isolate executor details from table AM. We may introduce a new data structure InsertWithArbiterContext which would contain EState and a set of callbacks which would avoid table AM

Re: Table AM Interface Enhancements

2024-03-30 Thread Alexander Korotkov
Hi, Pavel! I've pushed 0001, 0002 and 0006. On Fri, Mar 29, 2024 at 5:23 PM Pavel Borisov wrote: >> >> I think for better code look this could be removed: >> >vlock: >> > CHECK_FOR_INTERRUPTS(); >> together with CHECK_FOR_INTERRUPTS(); in heapam_tuple_insert_with_arbiter

Re: Table AM Interface Enhancements

2024-03-29 Thread Pavel Borisov
> > I think for better code look this could be removed: > >vlock: > > CHECK_FOR_INTERRUPTS(); > together with CHECK_FOR_INTERRUPTS(); in heapam_tuple_insert_with_arbiter > placed in the beginning of while loop. > To clarify things, this I wrote only about CHECK_FOR_INTERRUPTS

Re: Table AM Interface Enhancements

2024-03-29 Thread Pavel Borisov
I've looked at patch 0003. Generally, it does a similar thing as 0001 - it exposes a more generalized method tuple_insert_with_arbiter that encapsulates tuple_insert_speculative/tuple_complete_speculative and at the same time allows extensibility of this i.e. different implementation for custom ta

Re: Table AM Interface Enhancements

2024-03-28 Thread Japin Li
On Thu, 28 Mar 2024 at 21:26, Alexander Korotkov wrote: > Hi Pavel! > > Revised patchset is attached. > > On Thu, Mar 28, 2024 at 3:12 PM Pavel Borisov wrote: >> The other extensibility that seems quite clear and uncontroversial to me is >> 0006. >> >> It simply shifts the decision on whether

Re: Table AM Interface Enhancements

2024-03-28 Thread Pavel Borisov
Hi, Alexander! The other extensibility that seems quite clear and uncontroversial to me is 0006. It simply shifts the decision on whether tuple inserts should invoke inserts to the related indices to the table am level. It doesn't change the current heap insert behavior so it's safe for the exist

Re: Table AM Interface Enhancements

2024-03-28 Thread Pavel Borisov
Hi, Alexander! Thank you for working on these patches. On Thu, 28 Mar 2024 at 02:14, Alexander Korotkov wrote: > Hi, Pavel! > > Thank you for your feedback. The revised patch set is attached. > > I found that vacuum.c has a lot of heap-specific code. Thus, it > should be OK for analyze.c to kee

Re: Table AM Interface Enhancements

2024-03-27 Thread Pavel Borisov
> > This seems not needed, it's already inited to InvalidOid before. > +else > +accessMethod = default_table_access_method; > > + accessMethodId = InvalidOid; > > This code came from 374c7a22904. I don't insist on this simplification in > a patch 0002. > A correction of the code quote for th

Re: Table AM Interface Enhancements

2024-03-27 Thread Pavel Borisov
Hi, Alexander! The revised rest of the patchset is attached. > 0001 (was 0006) – I prefer the definition of AcquireSampleRowsFunc to > stay in vacuum.h. If we move it to sampling.h then we would have to > add there includes to define Relation, HeapTuple etc. I'd like to > avoid this kind of chan

Re: Table AM Interface Enhancements

2024-03-21 Thread Pavel Borisov
On Fri, 22 Mar 2024 at 08:51, Pavel Borisov wrote: > Hi, Alexander! > > Thank you for working on this patchset and pushing some of these patches! > > I tried to write comments for tts_minimal_is_current_xact_tuple() > and tts_minimal_getsomeattrs() for them to be the same as for the same > functi

Re: Table AM Interface Enhancements

2024-03-21 Thread Pavel Borisov
Hi, Alexander! Thank you for working on this patchset and pushing some of these patches! I tried to write comments for tts_minimal_is_current_xact_tuple() and tts_minimal_getsomeattrs() for them to be the same as for the same functions for heap and virtual tuple slots, as I proposed above in the

Re: Table AM Interface Enhancements

2024-03-20 Thread Pavel Borisov
Hi, Alexander! On Wed, 20 Mar 2024 at 09:22, Pavel Borisov wrote: > Hi, Alexander! > > For 0007: > > Code inside > > +heapam_reloptions(char relkind, Datum reloptions, bool validate) > +{ > + if (relkind == RELKIND_RELATION || > + relkind == RELKIND_TOASTVALUE || > + relkind == RE

Re: Table AM Interface Enhancements

2024-03-19 Thread Pavel Borisov
Hi, Alexander! For 0007: Code inside +heapam_reloptions(char relkind, Datum reloptions, bool validate) +{ + if (relkind == RELKIND_RELATION || + relkind == RELKIND_TOASTVALUE || + relkind == RELKIND_MATVIEW) + return heap_reloptions(relkind, reloptions, validate); + + retur

Re: Table AM Interface Enhancements

2024-03-19 Thread Japin Li
On Tue, 19 Mar 2024 at 21:05, Alexander Korotkov wrote: > Hi, Pavel! > > On Tue, Mar 19, 2024 at 11:34 AM Pavel Borisov wrote: >> On Tue, 19 Mar 2024 at 03:34, Alexander Korotkov >> wrote: >>> >>> On Sun, Mar 3, 2024 at 1:50 PM Alexander Korotkov >>> wrote: >>> > On Mon, Nov 27, 2023 at 10:

Re: Table AM Interface Enhancements

2024-03-19 Thread Pavel Borisov
Hi, Alexander! On Tue, 19 Mar 2024 at 03:34, Alexander Korotkov wrote: > On Sun, Mar 3, 2024 at 1:50 PM Alexander Korotkov > wrote: > > On Mon, Nov 27, 2023 at 10:18 PM Mark Dilger > > wrote: > > > > > > > On Nov 25, 2023, at 9:47 AM, Alexander Korotkov < > aekorot...@gmail.com> wrote: > > > >

Re: Table AM Interface Enhancements

2024-03-03 Thread Alexander Korotkov
On Mon, Nov 27, 2023 at 10:18 PM Mark Dilger wrote: > > > On Nov 25, 2023, at 9:47 AM, Alexander Korotkov > > wrote: > > > >> Should the patch at least document which parts of the EState are expected > >> to be in which states, and which parts should be viewed as undefined? If > >> the implem

Re: Table AM Interface Enhancements

2024-03-03 Thread Alexander Korotkov
Hi, Matthias! On Fri, Nov 24, 2023 at 1:07 AM Matthias van de Meent wrote: > On Thu, 23 Nov 2023 at 13:43, Alexander Korotkov wrote: > > > > Hello PostgreSQL Hackers, > > > > I am pleased to submit a series of patches related to the Table Access > > Method (AM) interface, which I initially annou

Re: Table AM Interface Enhancements

2023-12-20 Thread Pavel Borisov
> > > Additionally changes in 0007 looks dependent from 0005. Does replacement > of slot inside ExecInsert, that is already used in the code below the call > of > > >/* insert the tuple normally */ > >- table_tuple_insert(resultRelationDesc, slot, > >- estate->es_output_cid, > >- 0, NULL); > >

Re: Table AM Interface Enhancements

2023-12-20 Thread Pavel Borisov
Hi, Alexander! I've reviewed patch 0004. It's clear enough and I think does what it's supposed. One thing, in function signature +bool (*tuple_is_current) (Relation rel, TupleTableSlot *slot); there is a Relation agrument, which is unused in both existing heapam method. Also it's unused in OrioleD

Re: Table AM Interface Enhancements

2023-11-29 Thread Pavel Borisov
Hi, Nikita! On Wed, 29 Nov 2023 at 18:27, Nikita Malakhov wrote: > Hi, > > Pavel, as far as I understand Alexander's idea assertion and especially > ereport > here does not make any sense - this method is not considered to report > error, it > silently calls if there is underlying [free] functio

Re: Table AM Interface Enhancements

2023-11-29 Thread Nikita Malakhov
Hi, Pavel, as far as I understand Alexander's idea assertion and especially ereport here does not make any sense - this method is not considered to report error, it silently calls if there is underlying [free] function and simply falls through otherwise, also, take into account that it could be lo

Re: Table AM Interface Enhancements

2023-11-29 Thread Pavel Borisov
Hi, Alexander! > I'm planning to review some of the other patches from the current patchset > soon. > I've looked into the patch 0003. The patch looks in good shape and is uncontroversial to me. Making memory structures to be dynamically allocated is simple enough and it allows to store complex d

Re: Table AM Interface Enhancements

2023-11-28 Thread Pavel Borisov
Hi, Alexander! I think table AM extensibility is a very good idea generally, not only in the scope of APIs that are needed in OrioleDB. Thanks for your proposals! For patches > 0001-Allow-locking-updated-tuples-in-tuple_update-and--v1.patch 0002-Add-EvalPlanQual-delete-returning-isolation-test-

Re: Table AM Interface Enhancements

2023-11-27 Thread Mark Dilger
> On Nov 25, 2023, at 9:47 AM, Alexander Korotkov wrote: > >> Should the patch at least document which parts of the EState are expected to >> be in which states, and which parts should be viewed as undefined? If the >> implementors of table AMs rely on any/all aspects of EState, doesn't tha

Re: Table AM Interface Enhancements

2023-11-25 Thread Alexander Korotkov
On Fri, Nov 24, 2023 at 5:18 PM Mark Dilger wrote: > > On Nov 23, 2023, at 4:42 AM, Alexander Korotkov > > wrote: > > > > 0006-Generalize-table-AM-API-for-INSERT-.-ON-CONFLICT-v1.patch > > > > Provides a new table AM API method to encapsulate the whole INSERT ... > > ON CONFLICT ... algorithm ra

Re: Table AM Interface Enhancements

2023-11-24 Thread Mark Dilger
> On Nov 23, 2023, at 4:42 AM, Alexander Korotkov wrote: > 0006-Generalize-table-AM-API-for-INSERT-.-ON-CONFLICT-v1.patch > > Provides a new table AM API method to encapsulate the whole INSERT ... > ON CONFLICT ... algorithm rather than just implementation of > speculative tokens. I *think*

Re: Table AM Interface Enhancements

2023-11-23 Thread Matthias van de Meent
Hi, On Thu, 23 Nov 2023 at 13:43, Alexander Korotkov wrote: > > Hello PostgreSQL Hackers, > > I am pleased to submit a series of patches related to the Table Access > Method (AM) interface, which I initially announced during my talk at > PGCon 2023 [1]. These patches are primarily designed to sup