Re: False positive "doesn't match the target pattern" error

[Ingo Schwarze]

Hi,

Masahiro Yamada wrote in his original bug report:

> ./foo.x: %/foo.x: %/foo.z
>   cp $< $@

This is not make(1) syntax.  POSIX does not define "%", nor multiple
colons on the same line, and i fail to see right now how any similar
problem might occur with POSIX make(1) - of course, i might be overlooking
something.  I'm not a specialist for make(1), just an ordinary OpenBSD
userland and GNU roff developer and an ordinary user of make(1).
I'm answering because Alejandro directly asked for my opinion.


On Aug 22, 2022, at 12:02 AM, Alejandro Colomar wrote:

> Do other make(1) implementations trim this leading ./?

That question is imprecisely worded.  In which context?

When actually creating the file, for example via "touch $@" or something
similar, obviously yes.  Then again, you might argue that in that case,
it isn't really make(1) resolving the "./" but rather the kernel on
behalf of the open(2) system call.

So, in which context?  As a target?  As a dependency?  Or even in yet
some other context, like variable expansion, conditional testing,
or substitution?


Alejandro Colomar wrote on Mon, Aug 22, 2022 at 08:08:13PM +0200:

> I tried on OpenBSD, and it seems to do that and more; it seems to be 
> resolving symlinks and .. (physically, although I'm not yet sure of what 
> it does to .. after segments of the path that don't exist).  It doesn't 
> seem to be documented, and also doesn't seem to be obvious in the source 
> code (I can't find any realpath(3) calls, nor '..').

You don't say what exactly you tested.  I did this quick test
right now:

   $ uname -a
  OpenBSD isnote.usta.de 7.2 GENERIC.MP#662 amd64
   $ cat Makefile 
  foo:
@echo no leading dot
  ./foo:
@echo leading dot
  bar: foo
  baz: ./foo
   $ make foo
  no leading dot
   $ make ./foo
  leading dot
   $ make bar
  no leading dot
   $ make baz
  leading dot

So, at least in simple situations, it seems to me that OpenBSD make(1)
treats both target and destination names as strings and does not attempt
to resolve "./".

Then again, i do not doubt that, if you look hard enough, you may well
be able to find edge cases that may surprise you - in OpenBSD make 
just like in GNU make.

I believe it may well be intentional that the OpenBSD make(1) manual,
https://man.openbsd.org/make.1 , does not discuss whether targets
"foo:" and "./foo:" are the same or distinct, because you should
not rely on it.  Instead, it says:

  "For maximum portability, target names should only consist of
   periods, underscores, digits and alphabetic characters."

> Ingo, do you know how your make(1) behaves with respect to '..', '//', 
> '.', and symlinks?  Is that documented anywhere?

I don't see how exactly "..", "//", and symlinks are related to
the orignal bug report, and unless the question is worded with
significantly more precision, i don't feel like testing all over
the place.

> On 8/22/22 16:46, David A. Wheeler wrote:

>> No idea. However, I took a quick look at the POSIX make specification:
>> https://pubs.opengroup.org/onlinepubs/9699919799/utilities/make.html
>> 
>> The POSIX specification for make is notoriously underpowered.

Little doubt about that; unfortunately, improving it is very difficult
given how much different implementations of make(1) have diverged.
That can easily be witnessed when watching the Austin list for
proposals to improve the specification of POSIX make(1).

>> It doesn't say anything about whether or not ./FOO is the same as FOO.

It does!  Quoting from the above URI:

  "Applications shall select target names from the set of characters
   consisting solely of periods, underscores, digits, and alphabetics
   from the portable character set [...]"

Consequently, "/" is invalid in a target name, and in particular, "./foo"
is not a valid target name.

>> It refers to files by PRECEDING them with "./", e.g.
>> "By default, the following files shall be tried in sequence:
>> ./makefile and ./Makefile. If neither ./makefile or ./Makefile are
>> found, other implementation-defined files may also be tried.
>> [XSI] On XSI-conformant systems, the additional files ./s.makefile,
>> SCCS/s.makefile, ./s.Makefile, and SCCS/s.Makefile shall also be tried."
>> I interpret that as an expectation that adding "./" is expected to have
>> the same result if the filename starts with an alphabetic character,
>> but that's just my reading and not directly supported.

> That refers to the input file, and not to targets within the DAG, AFAICS.

Indeed, i agree that is completely unrelated to the interpretation
of "./foo" in *any* context inside a Makefile.  It is merely related
to the rules how programs *in general* treat "./" when *opening*
files - and i don't believe anyone could doubt, for example, that
the following two commands behave identically:

  make -f GNUmakefile
  make -f ./GNUmakefile

To summarize, i believe it is not a good idea for Makefile authors
to expect that t

Re: False positive "doesn't match the target pattern" error

[Alejandro Colomar]

Hi David, Ingo,

For others starting to read this thread now, it started here:


On 8/22/22 16:46, David A. Wheeler wrote:




On Aug 22, 2022, at 12:02 AM, Alejandro Colomar  wrote:

Hi David,

On 8/22/22 04:45, David A. Wheeler wrote:

On Aug 20, 2022, at 11:35 AM, Alejandro Colomar  wrote:
I'd say there is:  make(1) treats file names as text strings, not really file 
names, for most of its operations.  As an example, foo/ and foo/. are different 
targets.  I don't see why ./bar and bar should be the same.  Consistency is 
essential; otherwise, what to expect?  Why does make(1) need to special-case a 
leading ./ ?

Because treating "./foo" and "foo" as different files is likely to lead to 
endless footguns.
Consistency is nice, but making something easy to use *correctly* is more important. I have no 
problem with special-casing "./XXX" as a synonym for "XXX"; anything else would 
be *unexpected* (if for no other reason than backwards compatibility).
As far as "foo/" vs. "foo/.", it's *much* less common to have directories as 
prerequisites or targets, so not handling them with special conveniences seems quite reasonable.


Is it so *much* less common?  I never really knew that make(1) treats ./foo and 
foo as the same thing until this bug report.  make(1) is (almost) very 
consistent in that it handles text, and not really file names.  But _all_ 
non-phony targets should declare a dependency like '|$$(@D)/', unless you can 
guarantee that it already exists by the time you'll try to create the file 
(e.g., when the file depends on another file in the same dir).



My Makefiles are plagued with that dependency, but a correctly-written Makefile 
shouldn't even notice that ./ is trimmed.


I also try to do that, but I think it'd backwards-incompatible if GNU make 
changed its current behavior.
I suspect a lot of makefiles would quietly work incorrectly if that change was 
made.


Yeah, I guess the bug is resolving ./, and also that we will live 
forever with that bug.





Do other make(1) implementations trim this leading ./?


I tried on OpenBSD, and it seems to do that and more; it seems to be 
resolving symlinks and .. (physically, although I'm not yet sure of what 
it does to .. after segments of the path that don't exist).  It doesn't 
seem to be documented, and also doesn't seem to be obvious in the source 
code (I can't find any realpath(3) calls, nor '..').


I wrote a program that behaves similar to how I think BSD make(1) 
behaves, after testing it a little bit:


$ cat canonicalizepath.c
#include 
#include 
#include 
#include 
#include 
#include 


char *canonicalizepath(const char *path);


int
main(int argc, char *argv[])
{
int   ret;
char *p;

if (argc < 2)
errx(EXIT_FAILURE, "missing operand");

ret = EXIT_SUCCESS;

for (ptrdiff_t i = 1; i < argc; i++) {
p = canonicalizepath(argv[i]);
if (!p) {
ret = EXIT_FAILURE;
warn("%s", argv[1]);
} else {
puts(p);
free(p);
}
}

exit(ret);
}


char *
canonicalizepath(const char *path)
{
char *cpy, *end, *sep, *p, *ret;
size_t len;

cpy = strdup(path);
if (!cpy)
return NULL;

end = cpy + strlen(cpy);
sep = end;

do {
*sep = '\0';

p = realpath(cpy, NULL);
if (p)
goto found;

sep = strrchr(cpy, '/');
} while (sep);

sep = cpy;
found:
len = end - sep + 1;
if (p)
len += strlen(p);

ret = malloc(len);
if (!ret)
goto fail;

ret[0] = '\0';

if (p)
strlcat(ret, p, len);
for (char *q = sep; q < end; q++) {
if (*q == '\0')
*q = '/';
}
strlcat(ret, sep, len);

fail:
free(cpy);
return ret;
}


alx@asus5775:~/tmp$ ./canonicalizepath .
/home/alx/tmp
alx@asus5775:~/tmp$ ./canonicalizepath ./foo/../asd
/home/alx/tmp/foo/../asd
alx@asus5775:~/tmp$ ./canonicalizepath ./asd
/home/alx/tmp/asd
alx@asus5775:~/tmp$ ./canonicalizepath asd
asd
alx@asus5775:~/tmp$ ./canonicalizepath canonicalizepath
/home/alx/tmp/canonicalizepath
alx@asus5775:~/tmp$ ./canonicalizepath /home/../tmp/../../etc/
/etc


I don't think me want GNU make(1) to behave like that.


Ingo, do you know how your make(1) behaves with respect to '..', '//', 
'.', and symlinks?  Is that documented anywhere?





No idea. However, I took a quick look at the POSIX make specification:
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/make.html

The POSIX specification for make is notoriously underpowered.
It doesn't say anything about whether or not ./FOO is the same a

Re: False positive "doesn't match the target pattern" error

[David A. Wheeler]

> On Aug 22, 2022, at 12:02 AM, Alejandro Colomar  
> wrote:
> 
> Hi David,
> 
> On 8/22/22 04:45, David A. Wheeler wrote:
>>> On Aug 20, 2022, at 11:35 AM, Alejandro Colomar  
>>> wrote:
>>> I'd say there is:  make(1) treats file names as text strings, not really 
>>> file names, for most of its operations.  As an example, foo/ and foo/. are 
>>> different targets.  I don't see why ./bar and bar should be the same.  
>>> Consistency is essential; otherwise, what to expect?  Why does make(1) need 
>>> to special-case a leading ./ ?
>> Because treating "./foo" and "foo" as different files is likely to lead to 
>> endless footguns.
>> Consistency is nice, but making something easy to use *correctly* is more 
>> important. I have no problem with special-casing "./XXX" as a synonym for 
>> "XXX"; anything else would be *unexpected* (if for no other reason than 
>> backwards compatibility).
>> As far as "foo/" vs. "foo/.", it's *much* less common to have directories as 
>> prerequisites or targets, so not handling them with special conveniences 
>> seems quite reasonable.
> 
> Is it so *much* less common?  I never really knew that make(1) treats ./foo 
> and foo as the same thing until this bug report.  make(1) is (almost) very 
> consistent in that it handles text, and not really file names.  But _all_ 
> non-phony targets should declare a dependency like '|$$(@D)/', unless you can 
> guarantee that it already exists by the time you'll try to create the file 
> (e.g., when the file depends on another file in the same dir).

> My Makefiles are plagued with that dependency, but a correctly-written 
> Makefile shouldn't even notice that ./ is trimmed.

I also try to do that, but I think it'd backwards-incompatible if GNU make 
changed its current behavior.
I suspect a lot of makefiles would quietly work incorrectly if that change was 
made.

> Do other make(1) implementations trim this leading ./?

No idea. However, I took a quick look at the POSIX make specification:
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/make.html

The POSIX specification for make is notoriously underpowered.
It doesn't say anything about whether or not ./FOO is the same as FOO.
It refers to files by PRECEDING them with "./", e.g.
"By default, the following files shall be tried in sequence: ./makefile and 
./Makefile. If neither ./makefile or ./Makefile are found, other 
implementation-defined files may also be tried. [XSI]   On XSI-conformant 
systems, the additional files ./s.makefile, SCCS/s.makefile, ./s.Makefile, and 
SCCS/s.Makefile shall also be tried."
I interpret that as an expectation that adding "./" is expected to have the same
result if the filename starts with an alphabetic character, but that's
just my reading and not directly supported.

--- David A. Wheeler

Re: False positive "doesn't match the target pattern" error

[Alejandro Colomar]

Hi David,

On 8/22/22 04:45, David A. Wheeler wrote:




On Aug 20, 2022, at 11:35 AM, Alejandro Colomar  wrote:
I'd say there is:  make(1) treats file names as text strings, not really file 
names, for most of its operations.  As an example, foo/ and foo/. are different 
targets.  I don't see why ./bar and bar should be the same.  Consistency is 
essential; otherwise, what to expect?  Why does make(1) need to special-case a 
leading ./ ?


Because treating "./foo" and "foo" as different files is likely to lead to 
endless footguns.

Consistency is nice, but making something easy to use *correctly* is more important. I have no 
problem with special-casing "./XXX" as a synonym for "XXX"; anything else would 
be *unexpected* (if for no other reason than backwards compatibility).

As far as "foo/" vs. "foo/.", it's *much* less common to have directories as 
prerequisites or targets, so not handling them with special conveniences seems quite reasonable.


Is it so *much* less common?  I never really knew that make(1) treats 
./foo and foo as the same thing until this bug report.  make(1) is 
(almost) very consistent in that it handles text, and not really file 
names.  But _all_ non-phony targets should declare a dependency like 
'|$$(@D)/', unless you can guarantee that it already exists by the time 
you'll try to create the file (e.g., when the file depends on another 
file in the same dir).


My Makefiles are plagued with that dependency, but a correctly-written 
Makefile shouldn't even notice that ./ is trimmed.


Do other make(1) implementations trim this leading ./?

Cheers,

Alex



--- David A. Wheeler




--
Alejandro Colomar



OpenPGP_signature
Description: OpenPGP digital signature

Re: False positive "doesn't match the target pattern" error

[David A. Wheeler]

> On Aug 20, 2022, at 11:35 AM, Alejandro Colomar  
> wrote:
> I'd say there is:  make(1) treats file names as text strings, not really file 
> names, for most of its operations.  As an example, foo/ and foo/. are 
> different targets.  I don't see why ./bar and bar should be the same.  
> Consistency is essential; otherwise, what to expect?  Why does make(1) need 
> to special-case a leading ./ ?

Because treating "./foo" and "foo" as different files is likely to lead to 
endless footguns.

Consistency is nice, but making something easy to use *correctly* is more 
important. I have no problem with special-casing "./XXX" as a synonym for 
"XXX"; anything else would be *unexpected* (if for no other reason than 
backwards compatibility).

As far as "foo/" vs. "foo/.", it's *much* less common to have directories as 
prerequisites or targets, so not handling them with special conveniences seems 
quite reasonable.

--- David A. Wheeler

Re: False positive "doesn't match the target pattern" error

[Alejandro Colomar]

Hi Dmitry

On 8/21/22 05:52, Dmitry Goncharov wrote:

On Sat, Aug 20, 2022 at 8:28 PM Masahiro Yamada  wrote:

build-dirs := . drivers sound net virt arch/x86/pci arch/x86/power lib
arch/x86/lib
subdir-modorder := $(addsuffix /.modules.order, $(build-dirs))
$(sort $(subdir-modorder)): %/.modules.order: %


Can you remove . from build-dirs and add a dedicated rule for .module.order?
E.g.
build-dirs := drivers sound net virt arch/x86/pci arch/x86/power lib
arch/x86/lib
subdir-modorder := $(addsuffix /.modules.order, $(build-dirs))
$(sort $(subdir-modorder)): %/.modules.order: %; @:
.modules.order:; @:



Make considers 'foo/bar' and 'foo/./bar'
as different targets.


This is a deficiency in make. i opened
https://savannah.gnu.org/bugs/index.php?62929 and attached a fix.
Thanks for your report.


I'm not sure it's a bug/deficiency.  I use that as a feature (I'm don't 
even remember the bug that made me workaround it like that, but I'd like 
to check again; maybe I don't need such workaround really).


Anyway, it'll be a hard bug to fix:

Should make normalize . by just removing them?

Then, should it go further and collapse // ?

Then, should make go further and normalize ..?

But if so, should it go even further and resolve symlinks (.. is just a 
link to the parent directory)?


And if symlinks are in the way, how to deal with ..?  Logically, or 
physically?  See realpath(1).



Of course symlinks and physically resolving .. are out of the table, 
because since make(1) deals with possibly-non-existent pathnames, 
realpath(1) can't be used on them.  So, are we sure everyone is happy 
with make(1) resolving .. logically?  Clearly not, so .. can't be resolved.


Then, why should make(1) resolve . or // if it can't resolve .. or 
symlinks?  Let's be consistent and treat everything as text strings, and 
let users make sure they use the correct string.  And if that bug may be 
useful to workaround some weird situation by having two rules foo and 
foo/.,  let's call it a feature! =)


Cheers,

Alex

--
Alejandro Colomar



OpenPGP_signature
Description: OpenPGP digital signature

Re: False positive "doesn't match the target pattern" error

[Dmitry Goncharov]

On Sat, Aug 20, 2022 at 8:28 PM Masahiro Yamada  wrote:
> build-dirs := . drivers sound net virt arch/x86/pci arch/x86/power lib
> arch/x86/lib
> subdir-modorder := $(addsuffix /.modules.order, $(build-dirs))
> $(sort $(subdir-modorder)): %/.modules.order: %

Can you remove . from build-dirs and add a dedicated rule for .module.order?
E.g.
build-dirs := drivers sound net virt arch/x86/pci arch/x86/power lib
arch/x86/lib
subdir-modorder := $(addsuffix /.modules.order, $(build-dirs))
$(sort $(subdir-modorder)): %/.modules.order: %; @:
.modules.order:; @:


> Make considers 'foo/bar' and 'foo/./bar'
> as different targets.

This is a deficiency in make. i opened
https://savannah.gnu.org/bugs/index.php?62929 and attached a fix.
Thanks for your report.

regards, Dmitry

Re: False positive "doesn't match the target pattern" error

[Masahiro Yamada]

On Sun, Aug 21, 2022 at 12:48 AM Dmitry Goncharov
 wrote:
>
> On Sat, Aug 20, 2022 at 11:36 AM Alejandro Colomar
>  wrote:
> >  Why does make(1) need to special-case a leading ./ ?
>
> If your makefile has a rule like
> foo.x: foo.z; cp $< $@
> then make foo.x and make ./foo.x both produce the same file.
>
> regards, Dmitry




This happens only for leading './'.


'foo' and '././././foo' are the same.




Make considers 'foo/bar' and 'foo/./bar'
as different targets.





-- 
Best Regards
Masahiro Yamada

Re: False positive "doesn't match the target pattern" error

[Masahiro Yamada]

On Sun, Aug 21, 2022 at 12:32 AM Dmitry Goncharov
 wrote:
>
> On Sat, Aug 20, 2022 at 5:52 AM Masahiro Yamada  wrote:
> > I appreciate GNU Make normalize the path
> > by removing "./"
> >
> > This is helpful in some cases, but I think it is a bad side-effect
> > in this case.
>
> Is there a reason to treat './foo.x' as different from 'foo.x'?


I just provided a simple test case.





My real use-case is the Linux kernel build system.
The code (with a bit simplification) looks like this:





build-dirs := . drivers sound net virt arch/x86/pci arch/x86/power lib
arch/x86/lib


subdir-modorder := $(addsuffix /.modules.order, $(build-dirs))

$(sort $(subdir-modorder)): %/.modules.order: %
@:








>
> > If this is a bug, I can file for it.
> > Or, any workaround exists?
>
> It is not clear what you need to achieve.
> Can you use an explicit rule like
> foo.x: foo.z
> ?
> Do you need make to perform a directory search for foo.x and foo.z in
> various directories? In this case i'd look for vpath.
>
> regards, Dmitry
--
Best Regards
Masahiro Yamada

Re: False positive "doesn't match the target pattern" error

[Dmitry Goncharov]

On Sat, Aug 20, 2022 at 11:36 AM Alejandro Colomar
 wrote:
>  Why does make(1) need to special-case a leading ./ ?

If your makefile has a rule like
foo.x: foo.z; cp $< $@
then make foo.x and make ./foo.x both produce the same file.

regards, Dmitry

Re: False positive "doesn't match the target pattern" error

[Alejandro Colomar]

Hi Dmitry,

On 8/20/22 17:32, Dmitry Goncharov wrote:

On Sat, Aug 20, 2022 at 5:52 AM Masahiro Yamada  wrote:

I appreciate GNU Make normalize the path
by removing "./"

This is helpful in some cases, but I think it is a bad side-effect
in this case.


Is there a reason to treat './foo.x' as different from 'foo.x'?


I'd say there is:  make(1) treats file names as text strings, not really 
file names, for most of its operations.  As an example, foo/ and foo/. 
are different targets.  I don't see why ./bar and bar should be the 
same.  Consistency is essential; otherwise, what to expect?  Why does 
make(1) need to special-case a leading ./ ?






If this is a bug, I can file for it.
Or, any workaround exists?


It is not clear what you need to achieve.
Can you use an explicit rule like
foo.x: foo.z
?
Do you need make to perform a directory search for foo.x and foo.z in
various directories? In this case i'd look for vpath.

regards, Dmitry



Cheers,

Alex

--
Alejandro Colomar



OpenPGP_signature
Description: OpenPGP digital signature

Re: False positive "doesn't match the target pattern" error

[Dmitry Goncharov]

On Sat, Aug 20, 2022 at 5:52 AM Masahiro Yamada  wrote:
> I appreciate GNU Make normalize the path
> by removing "./"
>
> This is helpful in some cases, but I think it is a bad side-effect
> in this case.

Is there a reason to treat './foo.x' as different from 'foo.x'?


> If this is a bug, I can file for it.
> Or, any workaround exists?

It is not clear what you need to achieve.
Can you use an explicit rule like
foo.x: foo.z
?
Do you need make to perform a directory search for foo.x and foo.z in
various directories? In this case i'd look for vpath.

regards, Dmitry