Re: Auxiliary Processes and MyAuxProc

2020-09-16 Thread Michael Paquier
On Thu, Jul 09, 2020 at 04:17:59PM -0400, Mike Palmiotto wrote: > Thanks for the reviews. I'm hoping to get to this next week (hopefully > sooner). It was on my TODO list to use this approach (from the last > round of reviews), so I'll make sure to do it first. Nothing has happened here in two

Re: Auxiliary Processes and MyAuxProc

2020-07-09 Thread Mike Palmiotto
On Thu, Jul 2, 2020 at 11:11 AM Alvaro Herrera wrote: > > On 2020-Mar-26, Mike Palmiotto wrote: > > Regarding 0001: > > > diff --git a/src/backend/postmaster/subprocess.c > > b/src/backend/postmaster/subprocess.c > > new file mode 100644 > > index 00..3e7a45bf10 > > --- /dev/null > > +++

Re: Auxiliary Processes and MyAuxProc

2020-07-02 Thread Alvaro Herrera
On 2020-Mar-26, Mike Palmiotto wrote: Regarding 0001: > diff --git a/src/backend/postmaster/subprocess.c > b/src/backend/postmaster/subprocess.c > new file mode 100644 > index 00..3e7a45bf10 > --- /dev/null > +++ b/src/backend/postmaster/subprocess.c > @@ -0,0 +1,62 @@ >

Re: Auxiliary Processes and MyAuxProc

2020-07-02 Thread Daniel Gustafsson
> On 2 Jul 2020, at 17:03, Alvaro Herrera wrote: > > On 2020-Jul-02, Daniel Gustafsson wrote: > >>> On 27 Mar 2020, at 00:30, Mike Palmiotto >>> wrote: >> >>> Are these pieces required to make this patchset committable? Is there >>> anything else needed at this point to make it committable?

Re: Auxiliary Processes and MyAuxProc

2020-07-02 Thread Alvaro Herrera
On 2020-Jul-02, Daniel Gustafsson wrote: > > On 27 Mar 2020, at 00:30, Mike Palmiotto > > wrote: > > > Are these pieces required to make this patchset committable? Is there > > anything else needed at this point to make it committable? > > The submitted version no longer applies, the 0009

Re: Auxiliary Processes and MyAuxProc

2020-07-02 Thread Daniel Gustafsson
> On 27 Mar 2020, at 00:30, Mike Palmiotto > wrote: > Are these pieces required to make this patchset committable? Is there > anything else needed at this point to make it committable? The submitted version no longer applies, the 0009 patch has conflicts in postmaster.c. Can you please submit

Re: Auxiliary Processes and MyAuxProc

2020-03-21 Thread Peter Eisentraut
On 2020-03-19 14:29, Mike Palmiotto wrote: More specifically, I don't agree with the wholesale renaming of auxiliary process to subprocess. Besides the massive code churn, the I'm not sure I understand the argument here. Where do you see wholesale renaming of AuxProc to Subprocess? Subprocess

Re: Auxiliary Processes and MyAuxProc

2020-03-19 Thread Andres Freund
Hi, On 2020-03-19 11:35:41 +0100, Peter Eisentraut wrote: > On 2020-03-18 17:07, Mike Palmiotto wrote: > > On Wed, Mar 18, 2020 at 11:26 AM Mike Palmiotto > > wrote: > > > > > > On Wed, Mar 18, 2020 at 10:17 AM Justin Pryzby > > > wrote: > > > > Also, I saw this was failing tests both before

Re: Auxiliary Processes and MyAuxProc

2020-03-19 Thread Mike Palmiotto
On Thu, Mar 19, 2020 at 6:35 AM Peter Eisentraut wrote: > > While working on (My)BackendType, I was attempting to get rid of > (My)AuxProcType altogether. This would mostly work except that it's > sort of wired into the pgstats subsystem (see NumBackendStatSlots). > This can probably be

Re: Auxiliary Processes and MyAuxProc

2020-03-19 Thread Peter Eisentraut
On 2020-03-18 17:07, Mike Palmiotto wrote: On Wed, Mar 18, 2020 at 11:26 AM Mike Palmiotto wrote: On Wed, Mar 18, 2020 at 10:17 AM Justin Pryzby wrote: Also, I saw this was failing tests both before and after my rebase. http://cfbot.cputube.org/

Re: Auxiliary Processes and MyAuxProc

2020-03-18 Thread Mike Palmiotto
On Wed, Mar 18, 2020 at 12:54 PM Alvaro Herrera wrote: > > On 2020-Mar-18, Mike Palmiotto wrote: > > > On Wed, Mar 18, 2020 at 11:26 AM Mike Palmiotto > > wrote: > > > > > > On Wed, Mar 18, 2020 at 10:17 AM Justin Pryzby > > > wrote: > > > > Also, I saw this was failing tests both before and

Re: Auxiliary Processes and MyAuxProc

2020-03-18 Thread Alvaro Herrera
On 2020-Mar-18, Mike Palmiotto wrote: > On Wed, Mar 18, 2020 at 11:26 AM Mike Palmiotto > wrote: > > > > On Wed, Mar 18, 2020 at 10:17 AM Justin Pryzby wrote: > > > Also, I saw this was failing tests both before and after my rebase. > > > > > > http://cfbot.cputube.org/ > > >

Re: Auxiliary Processes and MyAuxProc

2020-03-18 Thread Mike Palmiotto
On Wed, Mar 18, 2020 at 10:17 AM Justin Pryzby wrote: > > On Wed, Mar 18, 2020 at 09:22:58AM -0400, Mike Palmiotto wrote: > > On Tue, Mar 17, 2020 at 9:04 PM Alvaro Herrera > > wrote: > > > > > > On 2020-Mar-17, Justin Pryzby wrote: > > > > > > > +static PgSubprocess process_types[] = { > > > >

Re: Auxiliary Processes and MyAuxProc

2020-03-18 Thread Justin Pryzby
On Wed, Mar 18, 2020 at 09:22:58AM -0400, Mike Palmiotto wrote: > On Tue, Mar 17, 2020 at 9:04 PM Alvaro Herrera > wrote: > > > > On 2020-Mar-17, Justin Pryzby wrote: > > > > > +static PgSubprocess process_types[] = { > > > + { > > > + .desc = "checker", > > > +

Re: Auxiliary Processes and MyAuxProc

2020-03-18 Thread Mike Palmiotto
On Tue, Mar 17, 2020 at 9:04 PM Alvaro Herrera wrote: > > On 2020-Mar-17, Justin Pryzby wrote: > > > +static PgSubprocess process_types[] = { > > + { > > + .desc = "checker", > > + .entrypoint = CheckerModeMain > > + }, > > + { > > + .desc =

Re: Auxiliary Processes and MyAuxProc

2020-03-17 Thread Alvaro Herrera
On 2020-Mar-17, Justin Pryzby wrote: > +static PgSubprocess process_types[] = { > + { > + .desc = "checker", > + .entrypoint = CheckerModeMain > + }, > + { > + .desc = "bootstrap", > + .entrypoint = BootstrapModeMain > + }, Maybe

Re: Auxiliary Processes and MyAuxProc

2020-03-17 Thread Justin Pryzby
On Tue, Mar 17, 2020 at 02:50:19PM -0400, Mike Palmiotto wrote: > The patchset is now split out. I've just noticed that Peter Eisentraut > included some changes for a generic MyBackendType, which I should have > been aware of. I was unable to rebase due to these changes, but can > fold these

Re: Auxiliary Processes and MyAuxProc

2020-03-02 Thread Mike Palmiotto
On Sat, Feb 29, 2020 at 9:51 PM Mike Palmiotto wrote: > > On Sat, Nov 30, 2019 at 9:15 PM Michael Paquier wrote: > > > > On Thu, Oct 03, 2019 at 11:39:37AM -0700, Andres Freund wrote: > > > Color me unconvinced. > > > > The latest comments of the thread have not been addressed yet. so I am > >

Re: Auxiliary Processes and MyAuxProc

2020-02-29 Thread Mike Palmiotto
On Sat, Nov 30, 2019 at 9:15 PM Michael Paquier wrote: > > On Thu, Oct 03, 2019 at 11:39:37AM -0700, Andres Freund wrote: > > Color me unconvinced. > > The latest comments of the thread have not been addressed yet. so I am > marking the patch as returned with feedback. If you think that's not >

Re: Auxiliary Processes and MyAuxProc

2019-11-30 Thread Michael Paquier
On Thu, Oct 03, 2019 at 11:39:37AM -0700, Andres Freund wrote: > Color me unconvinced. The latest comments of the thread have not been addressed yet. so I am marking the patch as returned with feedback. If you think that's not correct, please feel free to update the status of the patch. If you

Re: Auxiliary Processes and MyAuxProc

2019-10-03 Thread Andres Freund
Hi, On 2019-10-03 14:33:26 -0400, Mike Palmiotto wrote: > > > +#ifdef EXEC_BACKEND > > > + aux_fork->av[ac++] = pstrdup("--forkboot"); > > > + aux_fork->av[ac++] = NULL; /* filled in by > > > postmaster_forkexec */ > > > +#endif > > > > What's the point of

Re: Auxiliary Processes and MyAuxProc

2019-10-03 Thread Mike Palmiotto
On Thu, Oct 3, 2019 at 1:31 PM Andres Freund wrote: > > Hi, > > On 2019-09-30 15:13:18 -0400, Mike Palmiotto wrote: > > Attached is the reworked and rebased patch set. I put the hook on top > > and a separate commit for each process type. Note that avworker and > > avlauncher were intentionally

Re: Auxiliary Processes and MyAuxProc

2019-10-03 Thread Andres Freund
Hi, On 2019-09-30 15:13:18 -0400, Mike Palmiotto wrote: > Attached is the reworked and rebased patch set. I put the hook on top > and a separate commit for each process type. Note that avworker and > avlauncher were intentionally left together. Let me know if you think > those should be split out

Re: Auxiliary Processes and MyAuxProc

2019-09-30 Thread Mike Palmiotto
On Thu, Sep 26, 2019 at 6:03 PM Mike Palmiotto wrote: > > On Thu, Sep 26, 2019 at 10:56 AM Alvaro Herrera > wrote: > > > > > Well, I think it would be easier to manage as split patches, yeah. > > I think it'd be infrastructure that needs to be carefully reviewed, > > while the other ones are

Re: Auxiliary Processes and MyAuxProc

2019-09-26 Thread Mike Palmiotto
On Thu, Sep 26, 2019 at 10:56 AM Alvaro Herrera wrote: > > On 2019-Sep-26, Mike Palmiotto wrote: > > > On Thu, Sep 26, 2019 at 9:49 AM Alvaro Herrera > > wrote: > > > > > > 0002 seems way too large (and it doesn't currently apply). Is there > > > something we can do to make it more manageable?

Re: Auxiliary Processes and MyAuxProc

2019-09-26 Thread Alvaro Herrera
On 2019-Sep-26, Mike Palmiotto wrote: > On Thu, Sep 26, 2019 at 9:49 AM Alvaro Herrera > wrote: > > > > 0002 seems way too large (and it doesn't currently apply). Is there > > something we can do to make it more manageable? > > Initially we were thinking of submitting one patch for the >

Re: Auxiliary Processes and MyAuxProc

2019-09-26 Thread Mike Palmiotto
On Thu, Sep 26, 2019 at 9:49 AM Alvaro Herrera wrote: > > 0002 seems way too large (and it doesn't currently apply). Is there > something we can do to make it more manageable? Initially we were thinking of submitting one patch for the centralization work and then separate patches per backend

Re: Auxiliary Processes and MyAuxProc

2019-09-26 Thread Alvaro Herrera
0002 seems way too large (and it doesn't currently apply). Is there something we can do to make it more manageable? I think it would be better to put your 0001 in second place rather than first, since your other patch doesn't use it AFAICS and it adds functionality that has not yet received

Re: Auxiliary Processes and MyAuxProc

2019-08-14 Thread Yuli Khodorkovskiy
On Mon, Feb 25, 2019 at 5:25 PM Mike Palmiotto wrote: > > On Mon, Feb 25, 2019 at 1:41 PM Mike Palmiotto > wrote: > > > > > > > > > > If memory serves, StartChildProcess already was an attempt to unify > > > the treatment of postmaster children. It's possible that another > > > round of

Re: Auxiliary Processes and MyAuxProc

2019-02-25 Thread Mike Palmiotto
On Mon, Feb 25, 2019 at 1:41 PM Mike Palmiotto wrote: > > > > > > If memory serves, StartChildProcess already was an attempt to unify > > the treatment of postmaster children. It's possible that another > > round of unification would be productive, but I think you'll find > > that there are

Re: Auxiliary Processes and MyAuxProc

2019-02-25 Thread Mike Palmiotto
On Mon, Feb 25, 2019 at 1:29 PM Tom Lane wrote: > > Mike Palmiotto writes: > > > > For some context, I'm trying to come up with a patch to set the > > process identifier (MyAuxProc/am_autovacuumworker/launcher, > > am_archiver, etc.) immediately after forking, > > Don't we do that already?

Re: Auxiliary Processes and MyAuxProc

2019-02-25 Thread Tom Lane
Mike Palmiotto writes: > I've been looking through the startup code for postmaster forks and > saw a couple of mechanisms used. Most forks seem to be using > StartChildProcess with a MyAuxProc emumerator, but then some > (autovacuum launcher/worker, syslogger, bgworker, archiver) are forked >

Auxiliary Processes and MyAuxProc

2019-02-25 Thread Mike Palmiotto
Greetings, I've been looking through the startup code for postmaster forks and saw a couple of mechanisms used. Most forks seem to be using StartChildProcess with a MyAuxProc emumerator, but then some (autovacuum launcher/worker, syslogger, bgworker, archiver) are forked through their own start