Re: Introduce new multi insert Table AM and improve performance of various SQL commands with it for Heap AM

2024-06-05 Thread Bharath Rupireddy
Hi, On Thu, May 16, 2024 at 5:01 AM Jeff Davis wrote: > > The flushing behavior is entirely controlled by the table AM. The heap > can use the same flushing logic that it did before, which is to hold > 1000 tuples. > > I like that it's accounting for memory, too, but it doesn't need to be >

Re: Introduce new multi insert Table AM and improve performance of various SQL commands with it for Heap AM

2024-05-16 Thread Jeff Davis
On Wed, 2024-05-15 at 16:31 -0700, Jeff Davis wrote: > Even better would be if we could take into account partitioning. That > might be out of scope for your current work, but it would be very > useful. We could have a couple new GUCs like modify_table_buffer and >

Re: Introduce new multi insert Table AM and improve performance of various SQL commands with it for Heap AM

2024-05-15 Thread Jeff Davis
On Wed, 2024-05-15 at 12:56 +0530, Bharath Rupireddy wrote: > Because of this, the > buffers get flushed sooner than that of the existing COPY with > table_multi_insert AM causing regression in pgbench which uses COPY > extensively. The flushing behavior is entirely controlled by the table AM.

Re: Introduce new multi insert Table AM and improve performance of various SQL commands with it for Heap AM

2024-05-15 Thread Michael Paquier
On Wed, May 15, 2024 at 11:14:14AM +0200, Alvaro Herrera wrote: > We don't use IAM anywhere, for example (it's always "index AM"), and I > don't think we'd turn "sequence AM" into SAM either, would we? SAM is not a term I've seen used for sequence AMs in the past, I don't intend to use it. TAM

Re: Introduce new multi insert Table AM and improve performance of various SQL commands with it for Heap AM

2024-05-15 Thread Andres Freund
Hi, On 2024-05-15 11:14:14 +0200, Alvaro Herrera wrote: > On 2024-May-15, Bharath Rupireddy wrote: > > > It looks like with the use of the new multi insert table access method > > (TAM) for COPY (v20-0005), pgbench regressed about 35% [1]. > > Where does this acronym "TAM" comes from for "table

Re: Introduce new multi insert Table AM and improve performance of various SQL commands with it for Heap AM

2024-05-15 Thread Bharath Rupireddy
On Wed, May 15, 2024 at 2:44 PM Alvaro Herrera wrote: > > > It looks like with the use of the new multi insert table access method > > (TAM) for COPY (v20-0005), pgbench regressed about 35% [1]. > > Where does this acronym "TAM" comes from for "table access method"? Thanks for pointing it out. I

Re: Introduce new multi insert Table AM and improve performance of various SQL commands with it for Heap AM

2024-05-15 Thread Alvaro Herrera
Sorry to interject, but -- On 2024-May-15, Bharath Rupireddy wrote: > It looks like with the use of the new multi insert table access method > (TAM) for COPY (v20-0005), pgbench regressed about 35% [1]. Where does this acronym "TAM" comes from for "table access method"? I find it thoroughly

Re: Introduce new multi insert Table AM and improve performance of various SQL commands with it for Heap AM

2024-05-15 Thread Bharath Rupireddy
On Mon, Apr 29, 2024 at 11:36 AM Bharath Rupireddy wrote: > > Please see the attached v20 patch set. It looks like with the use of the new multi insert table access method (TAM) for COPY (v20-0005), pgbench regressed about 35% [1]. The reason is that the memory-based flushing decision the new

Re: Introduce new multi insert Table AM and improve performance of various SQL commands with it for Heap AM

2024-04-29 Thread Bharath Rupireddy
On Thu, Apr 25, 2024 at 10:11 PM Jeff Davis wrote: > > On Wed, 2024-04-24 at 18:19 +0530, Bharath Rupireddy wrote: > > I added a flush callback named TableModifyBufferFlushCallback; when > > provided by callers invoked after tuples are flushed to disk from the > > buffers but before the AM frees

Re: Introduce new multi insert Table AM and improve performance of various SQL commands with it for Heap AM

2024-04-25 Thread Jeff Davis
On Wed, 2024-04-24 at 18:19 +0530, Bharath Rupireddy wrote: > I added a flush callback named TableModifyBufferFlushCallback; when > provided by callers invoked after tuples are flushed to disk from the > buffers but before the AM frees them up. Index insertions and AFTER > ROW INSERT triggers can

Re: Introduce new multi insert Table AM and improve performance of various SQL commands with it for Heap AM

2024-04-24 Thread Pavel Stehule
st 24. 4. 2024 v 14:50 odesílatel Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> napsal: > On Wed, Apr 3, 2024 at 1:10 AM Jeff Davis wrote: > > > > Here's where I think this API should go: > > > > 1. Have table_modify_begin/end and table_modify_buffer_insert, like > > those that are

Introduce new multi insert Table AM and improve performance of various SQL commands with it for Heap AM

2024-04-24 Thread Bharath Rupireddy
On Wed, Apr 3, 2024 at 1:10 AM Jeff Davis wrote: > > Here's where I think this API should go: > > 1. Have table_modify_begin/end and table_modify_buffer_insert, like > those that are implemented in your patch. I added table_modify_begin, table_modify_buffer_insert, table_modify_buffer_flush and