Re: Switch to multi-inserts for pg_depend

2020-09-05 Thread Michael Paquier
On Fri, Sep 04, 2020 at 10:15:57AM +0900, Michael Paquier wrote: > Thanks, fixed. With the two comment fixes included, I have looked at both patches again today, and applied them. -- Michael signature.asc Description: PGP signature

Re: Switch to multi-inserts for pg_depend

2020-09-03 Thread Michael Paquier
On Thu, Sep 03, 2020 at 10:50:49AM -0400, Alvaro Herrera wrote: > I'm not sure you need the second sentence in this comment; keeping the > "delay initialization until ..." part seems sufficient. If you really > want to highlight that initialization is costly, maybe just say "delay > costly

Re: Switch to multi-inserts for pg_depend

2020-09-03 Thread Alvaro Herrera
I agree, this version looks much better, thanks. Two very minor things: On 2020-Sep-03, Michael Paquier wrote: > @@ -76,11 +77,23 @@ recordMultipleDependencies(const ObjectAddress *depender, > > dependDesc = table_open(DependRelationId, RowExclusiveLock); > > + /* > + *

Re: Switch to multi-inserts for pg_depend

2020-09-03 Thread Daniel Gustafsson
> On 3 Sep 2020, at 12:19, Michael Paquier wrote: > > On Thu, Sep 03, 2020 at 09:47:07AM +0200, Daniel Gustafsson wrote: >> I think this version is a clear improvement. Nothing more sticks out from a >> read-through. > > Thanks for taking the time to look at it, Daniel. We of course could >

Re: Switch to multi-inserts for pg_depend

2020-09-03 Thread Michael Paquier
On Thu, Sep 03, 2020 at 09:47:07AM +0200, Daniel Gustafsson wrote: > I think this version is a clear improvement. Nothing more sticks out from a > read-through. Thanks for taking the time to look at it, Daniel. We of course could still try to figure out how we could group all dependencies

Re: Switch to multi-inserts for pg_depend

2020-09-03 Thread Daniel Gustafsson
> On 3 Sep 2020, at 07:35, Michael Paquier wrote: > > On Tue, Sep 01, 2020 at 11:53:36AM +0200, Daniel Gustafsson wrote: >> On 14 Aug 2020, at 20:23, Alvaro Herrera wrote: >> >>> The logic to keep track number of used slots used is baroque, though -- that >>> could use a lot of simplification.

Re: Switch to multi-inserts for pg_depend

2020-09-02 Thread Michael Paquier
On Tue, Sep 01, 2020 at 11:53:36AM +0200, Daniel Gustafsson wrote: > On 14 Aug 2020, at 20:23, Alvaro Herrera wrote: > >> The logic to keep track number of used slots used is baroque, though -- that >> could use a lot of simplification. > > What if slot_init was an integer which increments

Re: Switch to multi-inserts for pg_depend

2020-09-01 Thread Daniel Gustafsson
> On 14 Aug 2020, at 20:23, Alvaro Herrera wrote: > The logic to keep track number of used slots used is baroque, though -- that > could use a lot of simplification. What if slot_init was an integer which increments together with the loop variable until max_slots is reached? If so, maybe it

Re: Switch to multi-inserts for pg_depend

2020-08-31 Thread Michael Paquier
On Sat, Aug 15, 2020 at 10:50:37AM +0900, Michael Paquier wrote: > What are you suggesting here? A new API layer to manage a set of > slots? It has been a couple of weeks, and I am not really sure what is the suggestion here. So I would like to move on with this patch set as the changes are

Re: Switch to multi-inserts for pg_depend

2020-08-14 Thread Michael Paquier
On Fri, Aug 14, 2020 at 02:23:16PM -0400, Alvaro Herrera wrote: > It seems a bit silly to worry about allocating just the exact amount > needed; the current approach looked fine to me. Okay, thanks. > The logic to keep track > number of used slots used is baroque, though -- that could use a lot

Re: Switch to multi-inserts for pg_depend

2020-08-14 Thread Alvaro Herrera
On 2020-Aug-14, Michael Paquier wrote: > Regarding the maximum number of slots allocated. Do people like the > current approach taken by the patch to do a single loop of the > dependency entries at the cost of more allocating perhaps too much for > the array holding the set of TupleTableSlots

Re: Switch to multi-inserts for pg_depend

2020-08-14 Thread Michael Paquier
On Thu, Aug 13, 2020 at 11:45:52AM -0400, Alvaro Herrera wrote: > MAX_CATALOG_INSERT_BYTES sounds decent to me. I mentioned dependency.h > because I was uncaffeinatedly thinking that this was used with API > defined there -- but in reality it's used with indexing.h functions, and > it seems to me

Re: Switch to multi-inserts for pg_depend

2020-08-13 Thread Alvaro Herrera
On 2020-Aug-13, Michael Paquier wrote: > On Thu, Aug 13, 2020 at 05:35:14AM -0400, Alvaro Herrera wrote: > > Next to the API definition I guess, is that dependency.h? > > We need something more central, see also MAX_PGATTRIBUTE_INSERT_BYTES > and MAX_PGSHDEPEND_INSERT_BYTES. And the definition

Re: Switch to multi-inserts for pg_depend

2020-08-13 Thread Michael Paquier
On Thu, Aug 13, 2020 at 05:35:14AM -0400, Alvaro Herrera wrote: > Next to the API definition I guess, is that dependency.h? We need something more central, see also MAX_PGATTRIBUTE_INSERT_BYTES and MAX_PGSHDEPEND_INSERT_BYTES. And the definition should be named something like

Re: Switch to multi-inserts for pg_depend

2020-08-13 Thread Alvaro Herrera
On 2020-Aug-13, Michael Paquier wrote: > Okay. Would src/include/catalog/catalog.h be a suited location for > this flag or somebody has a better idea? Next to the API definition I guess, is that dependency.h? -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL

Re: Switch to multi-inserts for pg_depend

2020-08-12 Thread Michael Paquier
On Wed, Aug 12, 2020 at 07:52:42PM -0400, Alvaro Herrera wrote: > Yeah. As I understand, the only reason to have this number is to avoid > an arbitrarily large number of entries created as a single multi-insert > WAL record ... but does that really ever happen? I guess if you create > a table

Re: Switch to multi-inserts for pg_depend

2020-08-12 Thread Alvaro Herrera
On 2020-Aug-11, Robert Haas wrote: > On Tue, Aug 11, 2020 at 1:59 AM Michael Paquier wrote: > > On Mon, Aug 10, 2020 at 05:32:21PM -0700, Andres Freund wrote: > > > Do we really want to end up with several separate defines for different > > > type of catalog batch inserts? That doesn't seem like

Re: Switch to multi-inserts for pg_depend

2020-08-11 Thread Robert Haas
On Tue, Aug 11, 2020 at 1:59 AM Michael Paquier wrote: > On Mon, Aug 10, 2020 at 05:32:21PM -0700, Andres Freund wrote: > > Do we really want to end up with several separate defines for different > > type of catalog batch inserts? That doesn't seem like a good > > thing. Think there should be a

Re: Switch to multi-inserts for pg_depend

2020-08-11 Thread Michael Paquier
On Mon, Aug 10, 2020 at 05:32:21PM -0700, Andres Freund wrote: > Do we really want to end up with several separate defines for different > type of catalog batch inserts? That doesn't seem like a good > thing. Think there should be a single define for all catalog bulk > inserts. Unlikely so, but I

Re: Switch to multi-inserts for pg_depend

2020-08-07 Thread Michael Paquier
On Fri, Aug 07, 2020 at 03:16:19PM +0900, Michael Paquier wrote: > I am adding this thread to the next commit fest. Thoughts are > welcome. Forgot to mention that this is based on some initial work from Daniel, and that we have discussed the issue before I worked on it. -- Michael

Switch to multi-inserts for pg_depend

2020-08-07 Thread Michael Paquier
Hi all, This is a continuation of the work that has been previously discussed here, resulting mainly in e3931d0 for pg_attribute and pg_shdepend: https://www.postgresql.org/message-id/20190213182737.mxn6hkdxwrzgx...@alap3.anarazel.de I have been looking at the amount of work that could be done