Re: [PATCH 1/r47] maint: help2man targets should rely on the binaries they call.
On 23 Sep 2010, at 03:40, Ralf Wildenhues wrote: > Hello Gary, Hallo Ralf, Thanks for the swift reviews again. > * Gary V. Vaughan wrote on Wed, Sep 22, 2010 at 10:29:44PM CEST: >> On 23 Sep 2010, at 00:35, Ralf Wildenhues wrote: >>> * Gary V. Vaughan wrote on Wed, Sep 22, 2010 at 07:05:48PM CEST: * Makefile.am (doc/libtool.1, doc/libtoolize.1): Don't rely on the intermediate files, since they might have changed without giving make the opportunity to update the actual binaries that help2man calls in time. >>> >>> No, because 'libtool' is created in the build tree, and the manpages are >>> distributed. Distributed files may not depend on undistributed files, >>> as that breaks building from a read-only source tree. Moreover, >>> help2man is something the user is expected to not have to install prior >>> to building Libtool. >> >> Yuck. Another reason to always start afresh after making changes >> rather than relying on make to DTRT :( >> >> In my case, ltmain.sh was corrupted, but even though I fixed it, >> rerunning make ended up leaving the empty manpages generated by >> a libtool script that had no --version output, and *then* it >> proceeded to rebuild ltmain.sh. > > I can try to debug it, if you can show me how to reliably reproduce the > failure. Thanks. Rather than me chasing my tail trying to tickle that bug again, let's skip this particular patch for now. I pushed it to the front of my queue because it was straight forward and I didn't think it relied on any of my other patches... but quite possibly, the fault is introduced later in one of the Makefile.am changes further down my queue, in which case we'll trip over the bug again soon enough, and can work out a proper fix for it at that time. >> Is there no way to make sure help2man doesn't run until the >> programs it wants to call have been rebuilt, rather than building >> (and potentially distributing) manpages documenting options from the >> previous script? > > I outlined four separate possible approaches for this in another mail in > this thread already. Except that you ruled out two of them, one involves giving up the advantages of non-recursive make, and the last is already in use! :-p Cheers, -- Gary V. Vaughan (g...@gnu.org) PGP.sig Description: This is a digitally signed message part
Re: [PATCH 1/r47] maint: help2man targets should rely on the binaries they call.
On 23 Sep 2010, at 01:22, Ralf Wildenhues wrote: > * Eric Blake wrote on Wed, Sep 22, 2010 at 08:19:28PM CEST: >> On 09/22/2010 12:13 PM, Ralf Wildenhues wrote: Is it acceptable instead to use a nested $(MAKE) invocation prior to running help2man to ensure the binary is up-to-date? >>> >>> Can you show a patch so I can see what you mean? >> >> diff --git i/Makefile.am w/Makefile.am >> index 6e29a29..f74708c 100644 >> --- i/Makefile.am >> +++ w/Makefile.am >> @@ -327,8 +327,10 @@ update_mans = \ >> PATH=.$(PATH_SEPARATOR)$$PATH; export PATH; \ >> $(HELP2MAN) --output=$@ >> $(srcdir)/doc/libtool.1: $(srcdir)/$(auxdir)/ltmain.sh >> +$(MAKE) libtool >> $(update_mans) --help-option=--help-all libtool > > When -jN has been passed, the two makes may both try to update 'libtool' > at the same time, leading to a race. > >> $(srcdir)/doc/libtoolize.1: $(srcdir)/libtoolize.in >> +$(MAKE) libtoolize >> $(update_mans) libtoolize > > Likewise here. How about a putting the shell code for libtoolize.in -> libtoolize transformation and writing: $(srcdir)/doc/libtoolize.1: $(srcdir)/libtoolize.in $(generate_libtoolize) $(update_mans) libtoolize Cheers, -- Gary V. Vaughan (g...@gnu.org) PGP.sig Description: This is a digitally signed message part
Re: [PATCH 1/r47] maint: help2man targets should rely on the binaries they call.
* Gary V. Vaughan wrote on Wed, Sep 22, 2010 at 10:36:01PM CEST: > On 23 Sep 2010, at 01:22, Ralf Wildenhues wrote: > > * Eric Blake wrote on Wed, Sep 22, 2010 at 08:19:28PM CEST: > >> On 09/22/2010 12:13 PM, Ralf Wildenhues wrote: > Is it acceptable instead to use a nested $(MAKE) invocation prior to > running help2man to ensure the binary is up-to-date? > >>> > >>> Can you show a patch so I can see what you mean? > >> > >> $(srcdir)/doc/libtool.1: $(srcdir)/$(auxdir)/ltmain.sh > >> + $(MAKE) libtool > >>$(update_mans) --help-option=--help-all libtool > > > > When -jN has been passed, the two makes may both try to update 'libtool' > > at the same time, leading to a race. > > > >> $(srcdir)/doc/libtoolize.1: $(srcdir)/libtoolize.in > >> + $(MAKE) libtoolize > >>$(update_mans) libtoolize > > > > Likewise here. > > How about a putting the shell code for libtoolize.in -> libtoolize > transformation and writing: > > $(srcdir)/doc/libtoolize.1: $(srcdir)/libtoolize.in > $(generate_libtoolize) > $(update_mans) libtoolize That will not help, because 'make' may still run it in parallel with the commands from the 'libtoolize' rule. Cheers, Ralf
Re: [PATCH 1/r47] maint: help2man targets should rely on the binaries they call.
Hello Gary, * Gary V. Vaughan wrote on Wed, Sep 22, 2010 at 10:29:44PM CEST: > On 23 Sep 2010, at 00:35, Ralf Wildenhues wrote: > > * Gary V. Vaughan wrote on Wed, Sep 22, 2010 at 07:05:48PM CEST: > >> * Makefile.am (doc/libtool.1, doc/libtoolize.1): Don't rely on > >> the intermediate files, since they might have changed without > >> giving make the opportunity to update the actual binaries that > >> help2man calls in time. > > > > No, because 'libtool' is created in the build tree, and the manpages are > > distributed. Distributed files may not depend on undistributed files, > > as that breaks building from a read-only source tree. Moreover, > > help2man is something the user is expected to not have to install prior > > to building Libtool. > > Yuck. Another reason to always start afresh after making changes > rather than relying on make to DTRT :( > > In my case, ltmain.sh was corrupted, but even though I fixed it, > rerunning make ended up leaving the empty manpages generated by > a libtool script that had no --version output, and *then* it > proceeded to rebuild ltmain.sh. I can try to debug it, if you can show me how to reliably reproduce the failure. > Is there no way to make sure help2man doesn't run until the > programs it wants to call have been rebuilt, rather than building > (and potentially distributing) manpages documenting options from the > previous script? I outlined four separate possible approaches for this in another mail in this thread already. Cheers, Ralf
Re: [PATCH 1/r47] maint: help2man targets should rely on the binaries they call.
On 23 Sep 2010, at 00:35, Ralf Wildenhues wrote: > Hello Gary, Hallo Ralf, > * Gary V. Vaughan wrote on Wed, Sep 22, 2010 at 07:05:48PM CEST: >> The start of my post-release patch queue... okay to push? >> >> * Makefile.am (doc/libtool.1, doc/libtoolize.1): Don't rely on >> the intermediate files, since they might have changed without >> giving make the opportunity to update the actual binaries that >> help2man calls in time. > > No, because 'libtool' is created in the build tree, and the manpages are > distributed. Distributed files may not depend on undistributed files, > as that breaks building from a read-only source tree. Moreover, > help2man is something the user is expected to not have to install prior > to building Libtool. Yuck. Another reason to always start afresh after making changes rather than relying on make to DTRT :( In my case, ltmain.sh was corrupted, but even though I fixed it, rerunning make ended up leaving the empty manpages generated by a libtool script that had no --version output, and *then* it proceeded to rebuild ltmain.sh. Is there no way to make sure help2man doesn't run until the programs it wants to call have been rebuilt, rather than building (and potentially distributing) manpages documenting options from the previous script? Cheers, -- Gary V. Vaughan (g...@gnu.org) PGP.sig Description: This is a digitally signed message part
Re: [PATCH 1/r47] maint: help2man targets should rely on the binaries they call.
* Eric Blake wrote on Wed, Sep 22, 2010 at 08:30:08PM CEST: > On 09/22/2010 12:22 PM, Ralf Wildenhues wrote: > >* Eric Blake wrote on Wed, Sep 22, 2010 at 08:19:28PM CEST: > >> $(srcdir)/doc/libtool.1: $(srcdir)/$(auxdir)/ltmain.sh > >>+ $(MAKE) libtool > >>$(update_mans) --help-option=--help-all libtool > > > >When -jN has been passed, the two makes may both try to update 'libtool' > >at the same time, leading to a race. > > Any ideas on how to avoid such a race? 1) .NOTPARALLEL: Is not a good idea because it will make 'make -j3 check' slow again. :-/ And if you require non-parallel builds, then you can again rely on the fact that Posix make will update prerequisites in the order listed, so that all you need to ensure is that all-am depends on 'libtool' earlier than it depends on '$(srcdir)/doc/libtool.1'. 2) Use a subdir Makefile.am in doc, as SUBDIRS is a serializer. Update the manpage in doc/Makefile.am, and have 'SUBDIRS = . doc' in toplevel so that it is updated first. 3) GNU make could use order-only prerequisites, but that is far too unportable, as many practical GNU make installations are too old still. 4) BUILT_SOURCES = libtool libtoolize Wait ... we are using (4) already. Why does it not work? IOW, I don't see the failure mode which this patch is supposed to fix. Please show a verbose make log exhibiting the failure. I'm guessing there is a different actual reason for the failure. Thanks, ralf
Re: [PATCH 1/r47] maint: help2man targets should rely on the binaries they call.
On 09/22/2010 12:22 PM, Ralf Wildenhues wrote: * Eric Blake wrote on Wed, Sep 22, 2010 at 08:19:28PM CEST: $(srcdir)/doc/libtool.1: $(srcdir)/$(auxdir)/ltmain.sh + $(MAKE) libtool $(update_mans) --help-option=--help-all libtool When -jN has been passed, the two makes may both try to update 'libtool' at the same time, leading to a race. Any ideas on how to avoid such a race? -- Eric Blake ebl...@redhat.com+1-801-349-2682 Libvirt virtualization library http://libvirt.org
Re: [PATCH 1/r47] maint: help2man targets should rely on the binaries they call.
* Eric Blake wrote on Wed, Sep 22, 2010 at 08:19:28PM CEST: > On 09/22/2010 12:13 PM, Ralf Wildenhues wrote: > >>Is it acceptable instead to use a nested $(MAKE) invocation prior to > >>running help2man to ensure the binary is up-to-date? > > > >Can you show a patch so I can see what you mean? > > diff --git i/Makefile.am w/Makefile.am > index 6e29a29..f74708c 100644 > --- i/Makefile.am > +++ w/Makefile.am > @@ -327,8 +327,10 @@ update_mans = \ >PATH=.$(PATH_SEPARATOR)$$PATH; export PATH; \ >$(HELP2MAN) --output=$@ > $(srcdir)/doc/libtool.1: $(srcdir)/$(auxdir)/ltmain.sh > + $(MAKE) libtool > $(update_mans) --help-option=--help-all libtool When -jN has been passed, the two makes may both try to update 'libtool' at the same time, leading to a race. > $(srcdir)/doc/libtoolize.1: $(srcdir)/libtoolize.in > + $(MAKE) libtoolize > $(update_mans) libtoolize Likewise here. Cheers, Ralf
Re: [PATCH 1/r47] maint: help2man targets should rely on the binaries they call.
On 09/22/2010 12:13 PM, Ralf Wildenhues wrote: Is it acceptable instead to use a nested $(MAKE) invocation prior to running help2man to ensure the binary is up-to-date? Can you show a patch so I can see what you mean? diff --git i/Makefile.am w/Makefile.am index 6e29a29..f74708c 100644 --- i/Makefile.am +++ w/Makefile.am @@ -327,8 +327,10 @@ update_mans = \ PATH=.$(PATH_SEPARATOR)$$PATH; export PATH; \ $(HELP2MAN) --output=$@ $(srcdir)/doc/libtool.1: $(srcdir)/$(auxdir)/ltmain.sh + $(MAKE) libtool $(update_mans) --help-option=--help-all libtool $(srcdir)/doc/libtoolize.1: $(srcdir)/libtoolize.in + $(MAKE) libtoolize $(update_mans) libtoolize -- Eric Blake ebl...@redhat.com+1-801-349-2682 Libvirt virtualization library http://libvirt.org
Re: [PATCH 1/r47] maint: help2man targets should rely on the binaries they call.
Hi Eric, * Eric Blake wrote on Wed, Sep 22, 2010 at 07:37:58PM CEST: > On 09/22/2010 11:35 AM, Ralf Wildenhues wrote: > >* Gary V. Vaughan wrote on Wed, Sep 22, 2010 at 07:05:48PM CEST: > >>* Makefile.am (doc/libtool.1, doc/libtoolize.1): Don't rely on > >>the intermediate files, since they might have changed without > >>giving make the opportunity to update the actual binaries that > >>help2man calls in time. > > > >No, because 'libtool' is created in the build tree, and the manpages are > >distributed. Distributed files may not depend on undistributed files, > >as that breaks building from a read-only source tree. Moreover, > >help2man is something the user is expected to not have to install prior > >to building Libtool. > > Is it acceptable instead to use a nested $(MAKE) invocation prior to > running help2man to ensure the binary is up-to-date? Can you show a patch so I can see what you mean? There are differences in semantics between GNU and some non-GNU make implementations in the way that some of the latter may always consider some prerequisite updated if they invoked the rule for updating the prerequisite. I'm hoping these makes die out, but we aren't quite there yet. Cheers, Ralf
Re: [PATCH 1/r47] maint: help2man targets should rely on the binaries they call.
On 09/22/2010 11:35 AM, Ralf Wildenhues wrote: Hello Gary, * Gary V. Vaughan wrote on Wed, Sep 22, 2010 at 07:05:48PM CEST: The start of my post-release patch queue... okay to push? * Makefile.am (doc/libtool.1, doc/libtoolize.1): Don't rely on the intermediate files, since they might have changed without giving make the opportunity to update the actual binaries that help2man calls in time. No, because 'libtool' is created in the build tree, and the manpages are distributed. Distributed files may not depend on undistributed files, as that breaks building from a read-only source tree. Moreover, help2man is something the user is expected to not have to install prior to building Libtool. Is it acceptable instead to use a nested $(MAKE) invocation prior to running help2man to ensure the binary is up-to-date? -- Eric Blake ebl...@redhat.com+1-801-349-2682 Libvirt virtualization library http://libvirt.org
Re: [PATCH 1/r47] maint: help2man targets should rely on the binaries they call.
Hello Gary, * Gary V. Vaughan wrote on Wed, Sep 22, 2010 at 07:05:48PM CEST: > The start of my post-release patch queue... okay to push? > > * Makefile.am (doc/libtool.1, doc/libtoolize.1): Don't rely on > the intermediate files, since they might have changed without > giving make the opportunity to update the actual binaries that > help2man calls in time. No, because 'libtool' is created in the build tree, and the manpages are distributed. Distributed files may not depend on undistributed files, as that breaks building from a read-only source tree. Moreover, help2man is something the user is expected to not have to install prior to building Libtool. Cheers, Ralf > --- a/Makefile.am > +++ b/Makefile.am > @@ -326,9 +326,9 @@ MAINTAINERCLEANFILES += $(dist_man1_MANS) > update_mans = \ >PATH=.$(PATH_SEPARATOR)$$PATH; export PATH; \ >$(HELP2MAN) --output=$@ > -$(srcdir)/doc/libtool.1: $(srcdir)/$(auxdir)/ltmain.sh > +$(srcdir)/doc/libtool.1: libtool > $(update_mans) --help-option=--help-all libtool > -$(srcdir)/doc/libtoolize.1: $(srcdir)/libtoolize.in > +$(srcdir)/doc/libtoolize.1: libtoolize > $(update_mans) libtoolize
