[bug #62200] Make tries to execute directories name as commands instead of using shell PATH resulting failure with permission denied

[Martin Dorey]

Follow-up Comment #5, bug #62200 (project make):

> typed the recipe command line into your shell prompt
I see the point but I'm not sure I'd put it *quite* that way.  Didn't bash win
the Shell Wars?


martind@sirius:~/tmp/make-62200$ PATH=$(pwd):$PATH /bin/sh -c asdf
/bin/sh: 1: asdf: Permission denied
martind@sirius:~/tmp/make-62200$ PATH=$(pwd):$PATH /bin/bash -c asdf
/bin/bash: asdf: command not found
martind@sirius:~/tmp/make-62200$ 


Perhaps someone else thought the message was worth customizing, probably with
their own localization, as the capitalization, for one thing, suggests it's
not an errno.  Huh, I see we have some commented-out localization for "Command
not found", which goes back in the Gnu Make source to 1992's:


rol...@redhat.com   d2e5ab8c57d613ce8955a08fce5c0e453b3038e6:
  if (!search_path (argv[0], path, program))
error ("%s: Command not found", argv[0]);


> If you rebuild your GNU make 4.3 with the latest gnulib
That would intimidate me and I've submitted patches.  README.git has been
tractable in recent years.  Paul would hate to suggest that you use an earlier
version, because he's more aware than anyone of all the bugs fixed since,
but...

>> I don't have in my PATH the $(pwd) folder
Sorry, I should have been explicit that I didn't want asdf in /usr/local/bin
(and like to promote the semicolon recipe separator), so had changed my test
case to:


martind@sirius:~/tmp/make-62200$ mkdir asdf
martind@sirius:~/tmp/make-62200$ echo 'all:; asdf' > Makefile
martind@sirius:~/tmp/make-62200$ PATH=$(pwd):$PATH ~/download/make/make 
asdf
make: asdf: Permission denied
make: *** [Makefile:1: all] Error 127
martind@sirius:~/tmp/make-62200$ 


I don't think that's materially different from your original asdf case,
though, as Paul says, the cp case *is* likely to be different - that
already-fixed, but after 4.3, bug #57962.


___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[bug #62200] Make tries to execute directories name as commands instead of using shell PATH resulting failure with permission denied

[Lahfa Samy]

Follow-up Comment #4, bug #62200 (project make):

Thanks for the explanation this made things very clearer to me, I fixed my
issue with building gcc that had a 'cp' directory by building the latest
version of make from git, when is the next release scheduled because since
January 2020 there has been no new release so far.


___

Reply to this item at:

  

___
  Message posté via Savannah
  https://savannah.gnu.org/

[bug #62200] Make tries to execute directories name as commands instead of using shell PATH resulting failure with permission denied

[Paul D. Smith]

Update of bug #62200 (project make):

  Status:None => Duplicate  
 Open/Closed:Open => Closed 

___

Follow-up Comment #3:

It's important to understand that the issue you originally described, where
there's a "cp" directory, and the issue you actually provided a test case for,
and for which Martin replied, where there's some random directory, are very
very different.

In the original issue, with a "cp" directory, GNU make used to get the right
behavior (that is ignore the "cp" directory and continue searching the PATH
and find the right "cp" program) up until GNU make 4.3.  In GNU make 4.3 we
started using a gnulib method to find programs on PATH, and there was a bug in
it.  That bug has been fixed in gnulib.  If you rebuild your GNU make 4.3 with
the latest gnulib, you won't see this problem.

The issue that you actually provided, where make invokes a random directory
name that is on the PATH, is very different: in this situation make will
always report the "permission denied" error as Martin suggests and this is the
correct behavior (or at least the behavior required by POSIX).  That's because
make will continue to search the path, discover there is actually no program
with the random name, so it will assume you really wanted to run the directory
it found and report that error.

Basically, make should always behave "as if" you typed the recipe command line
into your shell prompt.  In your example, if you do that, you'll see:


$ /bin/sh -c 'asdf'
/bin/sh: 1: asdf: Permission denied


and that's exactly what make tells you as well.

For more details see bug #57962


___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[bug #62200] Make tries to execute directories name as commands instead of using shell PATH resulting failure with permission denied

[Lahfa Samy]

Follow-up Comment #2, bug #62200 (project make):

So what should be done to fix this problem for a build such as gcc ?

This bug is stated as fixed : https://savannah.gnu.org/bugs/index.php?57962

So why am I encountering it again ? What should be done to mitigate the bug,
also I don't have in my PATH the $(pwd) folder, hence I don't understand why
fails for that.



___

Reply to this item at:

  

___
  Message posté via Savannah
  https://savannah.gnu.org/

[bug #62200] Make tries to execute directories name as commands instead of using shell PATH resulting failure with permission denied

[Martin Dorey]

Follow-up Comment #1, bug #62200 (project make):

Oh no, not error 127 again.  The error message here is the strerror form of
EACCES, which currently seems to come (on my build on reasonably modern Linux,
with findprog-in and posix_spawn) from this change:

https://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=commitdiff;h=6e6abd0cdfe4bb96f6412aebc511f10bf254a820

... which was from this email discussion:

https://lists.gnu.org/archive/html/bug-gnulib/2020-05/msg00248.html

... which was a continuation of a discussion inspired by a similar-sounding
Gnu Make issue:

https://savannah.gnu.org/bugs/index.php?57962

... when a directory called perl stopped us from finding the executable later
in the PATH.

The behavior, though, is much older than the current implementation, going
back at least 15 years:


(ia32)martind@sirius:~/tmp/make-62200$ PATH=$(pwd):$PATH make 
asdf
make: execvp: asdf: Permission denied
make: *** [all] Error 127
(ia32)martind@sirius:~/tmp/make-62200$ make --version
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i486-pc-linux-gnu
(ia32)martind@sirius:~/tmp/make-62200$ 


All that's different with today's git version is that the "execvp: " piece has
gone.


martind@sirius:~/tmp/make-62200$ PATH=$(pwd):$PATH ~/download/make/make
asdf
make: asdf: Permission denied
make: *** [Makefile:1: all] Error 127
martind@sirius:~/tmp/make-62200$ ~/download/make/make --version
GNU Make 4.3.90
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
martind@sirius:~/tmp/make-62200$ 


That's arguably mandated by:

https://pubs.opengroup.org/onlinepubs/9699919799/functions/exec.html

... which sayeth:

> [EACCES]
> The new process image file is not a regular file and the implementation does
not support execution of files of its type.

So I fear this is destined for not-a-bug status, sorry.


___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

Re: Bug report made anonymously should be assigned to my username on Savannah for receiving updates on it

[Paul Smith]

On Fri, 2022-03-18 at 21:12 +, Lahfa Samy wrote:
> I've reported this bug anonymously : 
> https://savannah.gnu.org/bugs/index.php?62200 and would like to
> receive updates/comments on it by mail on my Savannah account, I
> don't know if the bug report could be assigned to me or the "posted
> by" field could be updated to my username on Savannah (AkechiShiro
> ).

Unfortunately I'm not aware of an way to reset the submitter of the
bug.

However, you can add yourself to the CC list to receive any updates
made to it: look at the bottom of the bug to find the Mail Notification
Carbon-Copy List and add yourself.

Bug report made anonymously should be assigned to my username on Savannah for receiving updates on it

[Lahfa Samy]

Hi,

I've reported this bug anonymously : 
https://savannah.gnu.org/bugs/index.php?62200 and would like to receive 
updates/comments on it by mail on my Savannah account, I don't know if 
the bug report could be assigned to me or the "posted by" field could be 
updated to my username on Savannah (AkechiShiro ).


Thanks for any help regarding this matter.

--
Kind regards,

Lahfa Samy

[bug #62200] Make tries to execute directories name as commands instead of using shell PATH resulting failure with permission denied

[anonyme]

URL:
  

 Summary: Make tries to execute directories name as commands
instead of using shell PATH resulting failure with permission denied
 Project: make
Submitted by: None
Submitted on: ven. 18 mars 2022 21:00:15 UTC
Severity: 3 - Normal
  Item Group: Bug
  Status: None
 Privacy: Public
 Assigned to: None
 Open/Closed: Open
 Discussion Lock: Any
   Component Version: 4.3
Operating System: POSIX-Based
   Fixed Release: None
   Triage Status: None

___

Details:

I've been trying to build a big project and exchanging back and forth with the
community, we have created a minimal Makefile for what we think is a bug and
not an intended feature.

If a binary is named 'cp' and a directory is named 'cp', then the make build
fails with "cp: permission denied", if the directory is renamed, then the
build will go on, I met this issue whilst trying to build gcc from source.


Reproduce the bug with a minimal Makefile:

sudo mkdir /usr/local/bin/asdf
echo -e 'all:\n\tasdf' > Makefile
make


Output:

asdf
make: asdf: Permission denied
make: *** [Makefile:2: all] Error 127


I don't think this should be happening, if it should, this should be
documented and maybe the error message should be more helpful, the current
error leads someone to think there are permissions issues when it is not a
permission issue.






___

Reply to this item at:

  

___
  Message posté via Savannah
  https://savannah.gnu.org/

Re: GNU Make bug report: broken dry-run functionality with sub-make invocations

[David A. Wheeler]

> On Mar 18, 2022, at 2:03 PM, Paul Smith  wrote:
> 
> On Fri, 2022-03-18 at 17:48 +, Martin Dorey wrote:
>> Maybe putting it in the form of a patch on the latest git source will
>> help it over the finish line:
> 
> I'm OK with adding some short text about this into the man page.  As
> David mentions it may be important enough to do that since command
> being run by make even when the user gives "-n" could give unexpected
> or even unpleasant consequences.

It's also a potential security problem. Someone who runs "make -n" and
expects *nothing* to be run (other than make) will NOT get what they expect.
Hinting, in the man page, that some statements *are* run seems appropriate...
and if we can be specific without a lot of text, even better.

> However, in general I want to be clear that the man page is not, and is
> not intended to be, user documentation.  It's a reference page.

I agree. But adding 1-2 sentences seems reasonable.

--- David A. Wheeler

Re: GNU Make bug report: broken dry-run functionality with sub-make invocations

[Gisle Vanem]

Paul Smith wrote:


I'm OK with adding some short text about this into the man page.  As
David mentions it may be important enough to do that since command
being run by make even when the user gives "-n" could give unexpected
or even unpleasant consequences.


The most unpleasant consequences of using 'make -n' and
pressing Ctrl-C, is for me that 'make' hangs real good.
Seemingly waiting for nothing (?); a "dead-lock".
It's this 5 year old bug I believe:
  https://savannah.gnu.org/bugs/?51237

Using his version:
  https://github.com/mbuilov/gnumake-windows/blob/master/gnumake-4.3.exe

as 'make -n ...' on a large complex Makefile, I've seldom seen a
complete hang. But could take some time to stop.

Just my 0.02 €.

--
--gv

Re: GNU Make bug report: broken dry-run functionality with sub-make invocations

[Paul Smith]

On Fri, 2022-03-18 at 17:48 +, Martin Dorey wrote:
> Maybe putting it in the form of a patch on the latest git source will
> help it over the finish line:

I'm OK with adding some short text about this into the man page.  As
David mentions it may be important enough to do that since command
being run by make even when the user gives "-n" could give unexpected
or even unpleasant consequences.

However, in general I want to be clear that the man page is not, and is
not intended to be, user documentation.  It's a reference page.  The
man page is intended for people who ALREADY KNOW how to run make and
write makefiles, but need to quickly remind themselves of some syntax
that might have slipped their mind, and they don't want to have to
search the manual for it.

make and makefiles are entirely too complex a topic to be addressed in
a man page.

If you want to learn about how something works, not just remind
yourself how to use it, you need to be reading the user manual not the
man page.  Everything should be documented in full and understandable
detail in the user manual.

https://www.gnu.org/software/make/manual/html_node/index.html

Re: GNU Make bug report: broken dry-run functionality with sub-make invocations

[Martin Dorey]

Maybe putting it in the form of a patch on the latest git source will help it 
over the finish line:

diff --git a/doc/make.1 b/doc/make.1
index ec6f8a3b..4c258d68 100644
--- a/doc/make.1
+++ b/doc/make.1
@@ -222,8 +222,12 @@ With no argument, removes a previous load limit.
 Use the latest mtime between symlinks and target.
 .TP 0.5i
 \fB\-n\fR, \fB\-\-just\-print\fR, \fB\-\-dry\-run\fR, \fB\-\-recon\fR
-Print the commands that would be executed, but do not execute them (except in
-certain circumstances).
+Print the commands that would be executed, but do not execute them in
+most cases.
+If the line contains a call to \fB$(MAKE)\fR, the entire line will still be
+executed, with the \fB\-n\fR option passed to the sub-make instance that
+is run.
+Be prepared for side effects of output redirection.
 .TP 0.5i
 \fB\-o\fR \fIfile\fR, \fB\-\-old\-file\fR=\fIfile\fR, 
\fB\-\-assume\-old\fR=\fIfile\fR
 Do not remake the file


From: David A. Wheeler 
Sent: Friday, March 18, 2022 09:08
To: Ambrus Sumegi 
Cc: psm...@gnu.org ; Martin Dorey 
; bug-make@gnu.org 
Subject: Re: GNU Make bug report: broken dry-run functionality with sub-make 
invocations

* EXTERNAL EMAIL *

> On Mar 18, 2022, at 3:19 AM, Ambrus Sumegi  wrote:
>
> Thanks a lot for your suggestions, Martin and Paul. I understand the 
> reasoning behind why Make cannot improve this behavior, and the conditional 
> execution of tee that you both proposed looks like a concise and elegant 
> solution to my problem. My only remaining concern is about the man page, 
> which currently has this rather vague description of the switch in question:
>
>   -n, --just-print, --dry-run, --recon
>   Print the commands that would be executed, but do not execute 
> them (except in certain circumstances).
>
> Perhaps the "(except in certain circumstances)" could be expanded to 
> something like "If the line contains a call to $(MAKE), the entire line will 
> still be executed, with the -n option passed to the sub-make instance. Be 
> prepared for side effects of output redirection."

+1 to this. Proposal:

 Print the commands that would be executed, but do not execute them in most 
cases.
Note: If the line contains a call to $(MAKE), the entire line will still be 
executed, with the -n option passed to the sub-make instance that is run.
 Also, be prepared for side effects of output redirection.

(If someone has a better way to write that last sentence, please do so.)

It's a pretty important exception, so it should be explicitly described in the 
summary.

--- David A. Wheeler

Re: GNU Make bug report: broken dry-run functionality with sub-make invocations

[David A. Wheeler]

> On Mar 18, 2022, at 3:19 AM, Ambrus Sumegi  wrote:
> 
> Thanks a lot for your suggestions, Martin and Paul. I understand the 
> reasoning behind why Make cannot improve this behavior, and the conditional 
> execution of tee that you both proposed looks like a concise and elegant 
> solution to my problem. My only remaining concern is about the man page, 
> which currently has this rather vague description of the switch in question:
> 
>   -n, --just-print, --dry-run, --recon
>   Print the commands that would be executed, but do not execute 
> them (except in certain circumstances).
> 
> Perhaps the "(except in certain circumstances)" could be expanded to 
> something like "If the line contains a call to $(MAKE), the entire line will 
> still be executed, with the -n option passed to the sub-make instance. Be 
> prepared for side effects of output redirection."

+1 to this. Proposal:

 Print the commands that would be executed, but do not execute them in most 
cases.
Note: If the line contains a call to $(MAKE), the entire line will still be 
executed, with the -n option passed to the sub-make instance that is run.
 Also, be prepared for side effects of output redirection.

(If someone has a better way to write that last sentence, please do so.)

It's a pretty important exception, so it should be explicitly described in the 
summary.

--- David A. Wheeler

RE: GNU Make bug report: broken dry-run functionality with sub-make invocations

[Ambrus Sumegi]

Thanks a lot for your suggestions, Martin and Paul. I understand the reasoning 
behind why Make cannot improve this behavior, and the conditional execution of 
tee that you both proposed looks like a concise and elegant solution to my 
problem. My only remaining concern is about the man page, which currently has 
this rather vague description of the switch in question:

-n, --just-print, --dry-run, --recon
Print the commands that would be executed, but do not execute 
them (except in certain circumstances).

Perhaps the "(except in certain circumstances)" could be expanded to something 
like "If the line contains a call to $(MAKE), the entire line will still be 
executed, with the -n option passed to the sub-make instance. Be prepared for 
side effects of output redirection."

-Original Message-
From: Paul Smith 
Sent: 17 March 2022 20:08
To: Ambrus Sumegi 
Cc: bug-make@gnu.org
Subject: Re: GNU Make bug report: broken dry-run functionality with sub-make 
invocations

On Thu, 2022-03-17 at 18:27 +, Martin Dorey wrote:
> That coped with -nj --no-print-directory on the one version of Make
> that I tested it with, but I don't know how portable that would prove.

Modern versions of make guarantee a canonical format of MAKEFLAGS such that you 
can parse them relatively easily, and with confidence.

The details are in the manual I believe but the short is:

 * All options that have "short" variants with no arguments (for
   example --dry-run -> -n) are converted to the short variants and put
   into the first word, with no leading "-"
 * All other options are added after that.
 * Short options have a single dash prefix and if they have an argument
   it's attached to the option
 * Long options have a double-dash prefix (obviously) and if they have
   an argument it's attached to the option with an "="
 * Options that have both short and long forms, prefer the short form.

So, for -n, you can use:

$(findstring n,$(word 1,$(MAKEFLAGS)))

and it will expand to "n" if dry run is enabled (regardless of which form the 
option was given in), or empty if not.

So, the OP could use something like this:

DRYRUN = $(findstring n,$(word 1,$(MAKEFLAGS)))

 main_target: create_logdirs
 $(MAKE) external_target $(if $(DRYRUN),,| tee logs/external_task.log)

There are other alternatives. For example, you could add "+" as a prefix to the 
recipe in the create_logdirs so that the directories are created even when "-n" 
is given.


But ultimately Martin's comment is correct: this is not a bug in make and 
there's no possible way that make could do anything "better" than what it does. 
At some level, especially if you're writing recursive makefile environments, 
your recipes have to be written to be resilient to the possibility that make 
was invoked with "-n".
IMPORTANT NOTICE: The contents of this email and any attachments are 
confidential and may also be privileged. If you are not the intended recipient, 
please notify the sender immediately and do not disclose the contents to any 
other person, use it for any purpose, or store or copy the information in any 
medium. Thank you.