Re: [bug] Sibling submakes: wait for other submakes before failing

[Philip Guenther]

On Sun, Jun 13, 2021 at 6:29 PM David A. Wheeler 
wrote:

> > On Jun 13, 2021, at 5:44 PM, Alejandro Colomar (man-pages) <
> alx.manpa...@gmail.com> wrote:
> >
> > Let's say there's a project with a submake for each module.
>
> It’s counter-intuitive, but it is usually *MUCH* faster & more accurate to
> have a single make process
> figure things out in most cases than to use recursive makes.
>
> For more info, see “Recursive make considered harmful” by Peter Miller:
>
> https://web.archive.org/web/20200209034547/http://aegis.sourceforge.net/auug97.pdf


+1000

In October 2001 I applied the logic of that paper to a 1/2 million line
work project to cut its build time by a factor of 20 and make it reliably
build the Right Stuff.  Took less than a month to win back the entire time
I had spent reading the paper and working out the details of the new
Makefile setup.


Philip Guenther

Re: [bug] Sibling submakes: wait for other submakes before failing

[David A. Wheeler]

> On Jun 13, 2021, at 5:44 PM, Alejandro Colomar (man-pages) 
>  wrote:
> 
> Let's say there's a project with a submake for each module.

It’s counter-intuitive, but it is usually *MUCH* faster & more accurate to have 
a single make process
figure things out in most cases than to use recursive makes.

For more info, see “Recursive make considered harmful” by Peter Miller:
https://web.archive.org/web/20200209034547/http://aegis.sourceforge.net/auug97.pdf

--- David A. Wheeler

[bug] Sibling submakes: wait for other submakes before failing

[Alejandro Colomar (man-pages)]

Let's say there's a project with a submake for each module.

If a module depends on a file of another module, it's the other module
that will have the recipe for it.  For example, the shared library of a
module links to the shared library of another module.  Each module will
have the rules for building their shared libraries, and I can state that
the shared library of one module depends on the other, but I can't tell
it how to build it.  I can't even tell it to run $(MAKE) on the other
module, because if I run $(MAKE) on all modules from the top-level
Makefile, and then submakes also call make on other modules, then there
are duplicate makes, which if running `make -j` can fail, as I may be
compiling the same file at the same time from different sibling
submakes, which is something not wanted.

Also, the other solution would be to have a huge Makefile that controls
everything.  However, except for a bunch (5~10) of lines, all of the
modules build exactly the same (it was hard to generalize it, but I did
it), so I have a module.inc.mk that can build almost every module, and
then each Mkaefile for each module has a bunch of specific lines, and an
include of the generic Makefile.  Then the top-level Makefile calls the
same commands but on different dirs, to build each module.  If I had to
move all that to the top-level Makefile, I'd need to copy-paste a lot of
code, because that generic module.inc.mk can't work on a single invocation.

Anyway, here's the bug report/feature request:
See the following reduced example:



~/src/test$ find * -type f
a/Makefile
b/Makefile
Makefile


~/src/test$ cat Makefile
.PHONY: all aaa bbb

all: bbb aaa

aaa:
$(MAKE) -C $(CURDIR)/a

bbb:
$(MAKE) -C $(CURDIR)/b


~/src/test$ cat a/Makefile
.PHONY: all
all: $(CURDIR)/t

$(CURDIR)/t:
sleep 10
touch t


~/src/test$ cat b/Makefile
.PHONY: all
all: $(CURDIR)/s

$(CURDIR)/s: $(abspath $(CURDIR)/../a/t)
touch s



module b depends on a file produced by module a, but given that it will
execute later (in a single-process make, because of the order or the
prerequisites of the main Makefile, and in parallel make, because of the
sleep), it won't have the file available, nor will it know how to
produce it, and therefore it will fail.

A solution for this would be that when a make doesn't know how to
produce a file, and it is part of a "super"make (i.e., it's a submake,
there are other make processes running), it could wait for those other
processes before failing, and maybe those other processes will resolve
the dependency.

Is this something that could be fixed in the current make
implementation?  Or the intercommunication would have to change too much?


Thanks,

Alex



-- 
Alejandro Colomar
Linux man-pages comaintainer; https://www.kernel.org/doc/man-pages/
http://www.alejandro-colomar.es/

[bug #60777] Command line switch --trace disables -d.

[Dmitry Goncharov]

Additional Item Attachment, bug #60777 (project make):

File name: sv_60777_fix.diff  Size:0 KB


File name: sv_60777_test.diff Size:0 KB




___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[bug #60777] Command line switch --trace disables -d.

[Dmitry Goncharov]

URL:
  

 Summary: Command line switch --trace disables -d.
 Project: make
Submitted by: dgoncharov
Submitted on: Sun 13 Jun 2021 09:16:10 PM UTC
Severity: 3 - Normal
  Item Group: Bug
  Status: None
 Privacy: Public
 Assigned to: None
 Open/Closed: Open
 Discussion Lock: Any
   Component Version: SCM
Operating System: None
   Fixed Release: None
   Triage Status: None

___

Details:


Commit 7044e0c93 introduced a change which breaks -d when --trace is
specified.

$ cat makefile 
all: ; :
$ ~/src/gmake/make/l64/make -rd --trace
makefile:1: update target 'all' due to: target does not exist
:
$

There is no -d output.




___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/