bug#55025: Automake should allow one to enable POSIX make behavior

[Mike Frysinger]

On 14 Jan 2023 01:30, Nick Bowler wrote:
> On 2023-01-13, Mike Frysinger  wrote:
> > On 13 Jan 2023 16:01, Karl Berry wrote:
> >> I am doubtful about blithely defining .POSIX unconditionally. I feel
> >> sure that will break existing Makefiles. I don't think we should do that.
> >>
> >> Detecting .POSIX in an existing Makefile.am and moving it to the front
> >> sounds desirable, since that is clearly what the developer intended.
> >>
> >> Another (not mutually exclusive) possibility is to add an Automake
> >> option (say, "make-posix") that outputs the .POSIX line. --thanks, karl.
> >
> > i'd be amenable to a `no-make-posix` option, but imo forcing people to
> > opt-in to the right behavior is the wrong mental model.
> 
> Does adding .POSIX: to a Makefile actually solve any real-world
> portability problem with Automake generated makefiles, or is all of
> this just hypothetical?
> 
> Because we shouldn't go changing default behaviour to solve imaginary
> problems ...
> 
> > i'm also a bit skeptical of the idea that we're breaking makefiles.
> 
> ... and adding this to a Makefile really does change the behaviour
> significantly on at least one popular make implementation (GNU make).
> 
> I imagine a lot of people who are unfamiliar with traditional UNIX
> implementations would be very surprised to see gmake suddenly start
> running their rules using /bin/sh -e, which is probably the most
> obvious effect that the .POSIX special target has on this particular
> implementation.
> 
> And it doesn't even seem to work to get GNU make to follow the POSIX
> behaviour, as POSIX says -e is NOT used whenever errors are being
> suppressed (that is, when running commands that start with a -, or
> if the .IGNORE: special target is present in the makefile, or if
> make is run with the -i option).  GNU make appears to just always
> run the shell with -e if you include the .POSIX special target.

if this is the case, then it sounds like a bug.  at a glance, i don't see a
report in the upstream tracker, so can you throw one up there if you have a
test case that shows the problem ?

> Even then, portable makefiles can't rely on the shell being run
> with -e even normally, as NetBSD make does not do this.
> 
> I tried several other implementations and they follow the POSIX
> behaviour by default, adding -e only when errors are not suppressed.

this is exactly my point.  if i'm developing a project with automake and i'm
using gnu make, i can easily produce bad code that is not portable.  but the
insidious part is that it doesn't fail for me, it fails for my users who are
not using gnu make, but using a POSIX-compliant implementation that respects
the `set -e` behavior.

the reason i use automake in my projects in the first place is to produce a
build env that is reasonably portable, and if i have to write custom hooks,
to help steer me away from non-POSIX (i.e. things likely to not be portable)
constructs.
-mike


signature.asc
Description: PGP signature

bug#55025: Automake should allow one to enable POSIX make behavior

[Nick Bowler]

On 2023-01-13, Mike Frysinger  wrote:
> On 13 Jan 2023 16:01, Karl Berry wrote:
>> I am doubtful about blithely defining .POSIX unconditionally. I feel
>> sure that will break existing Makefiles. I don't think we should do that.
>>
>> Detecting .POSIX in an existing Makefile.am and moving it to the front
>> sounds desirable, since that is clearly what the developer intended.
>>
>> Another (not mutually exclusive) possibility is to add an Automake
>> option (say, "make-posix") that outputs the .POSIX line. --thanks, karl.
>
> i'd be amenable to a `no-make-posix` option, but imo forcing people to
> opt-in to the right behavior is the wrong mental model.

Does adding .POSIX: to a Makefile actually solve any real-world
portability problem with Automake generated makefiles, or is all of
this just hypothetical?

Because we shouldn't go changing default behaviour to solve imaginary
problems ...

> i'm also a bit skeptical of the idea that we're breaking makefiles.

... and adding this to a Makefile really does change the behaviour
significantly on at least one popular make implementation (GNU make).

I imagine a lot of people who are unfamiliar with traditional UNIX
implementations would be very surprised to see gmake suddenly start
running their rules using /bin/sh -e, which is probably the most
obvious effect that the .POSIX special target has on this particular
implementation.

And it doesn't even seem to work to get GNU make to follow the POSIX
behaviour, as POSIX says -e is NOT used whenever errors are being
suppressed (that is, when running commands that start with a -, or
if the .IGNORE: special target is present in the makefile, or if
make is run with the -i option).  GNU make appears to just always
run the shell with -e if you include the .POSIX special target.

Even then, portable makefiles can't rely on the shell being run
with -e even normally, as NetBSD make does not do this.

I tried several other implementations and they follow the POSIX
behaviour by default, adding -e only when errors are not suppressed.

> if an implementation is POSIX compliant, then it already behaves as
> POSIX defines make.  as it stands now, we, and our users, are
> generating files that target an undefined standard that no one agrees
> on, cannot be tested or asserted, and maybe they happen to work on the
> developer's desktop, but not on random users of their own.  by
> defining .POSIX, we turn the state from "usually works, but sometimes
> fails" to "either works or fails, but it's the same everywhere".

Unfortunately, reality is messy.  GNU's behaviour means that adding
.POSIX: actually means that with this specific behaviour (whether or
not the shell is run with "-e"), you now have to consider three
possibilities instead of just two.

Consider these two Makefiles, identical except for the presence of
the .POSIX special target:

  % cat >noposix.mk <<'EOF'
  craziness:
@-false; echo hello
@false; echo hello
EOF

  % cat >posix.mk <<'EOF'
  .POSIX:
  craziness:
@-false; echo hello
@false; echo hello
EOF

GNU:

  % gmake -f noposix.mk
  hello
  hello

  % gmake -f posix.mk
  gmake: [posix.mk:3: craziness] Error 1 (ignored)
  gmake: *** [posix.mk:4: craziness] Error 1

NetBSD:

  % make -f noposix.mk
  hello
  hello

  % make -f posix.mk
  hello
  hello

HP-UX 11:

  (this is the actual POSIX-specified behaviour which I also observed on
  multiple other systems):

  % make -f noposix.mk
  hello
  *** Error exit code 1

  Stop.

  % make -f posix.mk
  hello
  *** Error exit code 1

Cheers,
  Nick

bug#54421: AM_PROG_AR and LT_INIT don't work together when using lib.exe

[Mike Frysinger]

On 13 Jan 2023 13:52, Tim Ruffing wrote:
> On Fri, 2023-01-13 at 05:52 +, Mike Frysinger wrote:
> > i think the expectation is that, if you're using libtool, then you
> > use libtool.  
> 
> Well, I *use* libtool. libtool needs some AR. I choose to use
> libtool.exe with the ar-lib wrapper. 

you wondered why no one has reported/noticed hits before.  i gave you
an explanation.  what you're doing is unusual, and so far, i haven't
seen an explanation as to why you're doing something unusual.  so the
(implicit) question still stands: why are you using Automake's archive
(ar-lib) logic when libtool already provides support for doing the same
thing -- creating static archives.

should we fix it ?  probably.  is it worth the effort ?  maybe not.
would we accept patches from someone who wants to spend the time ?
depends on the patch.
-mike


signature.asc
Description: PGP signature

bug#55025: Automake should allow one to enable POSIX make behavior

[Mike Frysinger]

On 13 Jan 2023 16:01, Karl Berry wrote:
> I am doubtful about blithely defining .POSIX unconditionally. I feel
> sure that will break existing Makefiles. I don't think we should do that.
> 
> Detecting .POSIX in an existing Makefile.am and moving it to the front
> sounds desirable, since that is clearly what the developer intended.
> 
> Another (not mutually exclusive) possibility is to add an Automake
> option (say, "make-posix") that outputs the .POSIX line. --thanks, karl.

i'd be amenable to a `no-make-posix` option, but imo forcing people to
opt-in to the right behavior is the wrong mental model.

i'm also a bit skeptical of the idea that we're breaking makefiles.  if
an implementation is POSIX compliant, then it already behaves as POSIX
defines make.  as it stands now, we, and our users, are generating files
that target an undefined standard that no one agrees on, cannot be tested
or asserted, and maybe they happen to work on the developer's desktop, but
not on random users of their own.  by defining .POSIX, we turn the state
from "usually works, but sometimes fails" to "either works or fails, but
it's the same everywhere".

i grok that not everyone is POSIX-compliant, and we sometimes try to add
support for such systems where reasonable, but that's a case-by-case, and
we can't keep bending over backwards to support decades old dead software
that no one uses.
-mike


signature.asc
Description: PGP signature

bug#55025: Automake should allow one to enable POSIX make behavior

[Karl Berry]

I am doubtful about blithely defining .POSIX unconditionally. I feel
sure that will break existing Makefiles. I don't think we should do that.

Detecting .POSIX in an existing Makefile.am and moving it to the front
sounds desirable, since that is clearly what the developer intended.

Another (not mutually exclusive) possibility is to add an Automake
option (say, "make-posix") that outputs the .POSIX line. --thanks, karl.

bug#54421: AM_PROG_AR and LT_INIT don't work together when using lib.exe

[Tim Ruffing]

On Fri, 2023-01-13 at 05:52 +, Mike Frysinger wrote:
> i think the expectation is that, if you're using libtool, then you
> use libtool.  

Well, I *use* libtool. libtool needs some AR. I choose to use
libtool.exe with the ar-lib wrapper. 

bug#55025: Automake should allow one to enable POSIX make behavior

[Vincent Lefevre]

On 2023-01-13 01:19:39 -0500, Mike Frysinger wrote:
> any reason we don't just define it ourselves unconditionally ?  seems
> like the whole point of Automake is for devs to not worry about these
> kind of nitty details.

Probably. Note, however, that .POSIX affects GNU "make" on a few
points (the default is obviously not portable, but some developers
might require GNU "make", though probably for other reasons):

   If the '.POSIX' special target is defined then backslash/newline
handling is modified slightly to conform to POSIX.2: first, whitespace
preceding a backslash is not removed and second, consecutive
backslash/newlines are not condensed.

 In particular, if this target is mentioned then recipes will be
 invoked as if the shell had been passed the '-e' flag: the first
 failing command in a recipe will cause the recipe to fail
 immediately.

'warning: ignoring prerequisites on suffix rule definition'
 According to POSIX, a suffix rule cannot contain prerequisites.  If
 a rule that could be a suffix rule has prerequisites it is
 interpreted as a simple explicit rule, with an odd target name.
 This requirement is obeyed when POSIX-conforming mode is enabled
 (the '.POSIX' target is defined).  In versions of GNU 'make' prior
 to 4.3, no warning was emitted and a suffix rule was created,
 however all prerequisites were ignored and were not part of the
 suffix rule.  Starting with GNU 'make' 4.3 the behavior is the
 same, and in addition this warning is generated.  In a future
 version the POSIX-conforming behavior will be the only behavior: no
 rule with a prerequisite can be suffix rule and this warning will
 be removed.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

bug#54390: YACC rules don't check DESTDIR existence for VPATH builds

[Sam James]

> On 13 Jan 2023, at 08:43, Mike Frysinger  wrote:
> 
> On 13 Jan 2023 07:07, Sam James wrote:
>> $ /tmp/libaacs/configure YACC=bison LEX=flex
> 
> the problem is you're forcing `YACC=bison`.  Automake defaults to `bison -y`
> which tells bison to operate in POSIX-yacc-compatible mode because Automake's
> rules target POSIX.  the end result is that you've forced a non-POSIX mode,
> and bison is writing files to the cwd which is not what Automake expects.
> it isn't that the src/file/ directory doesn't exist -- you can see from your
> log that files have been generated in src/file/ before the failure.

thanks, duh. That makes sense.


signature.asc
Description: Message signed with OpenPGP

bug#54390: YACC rules don't check DESTDIR existence for VPATH builds

[Mike Frysinger]

On 13 Jan 2023 07:07, Sam James wrote:
> $ /tmp/libaacs/configure YACC=bison LEX=flex

the problem is you're forcing `YACC=bison`.  Automake defaults to `bison -y`
which tells bison to operate in POSIX-yacc-compatible mode because Automake's
rules target POSIX.  the end result is that you've forced a non-POSIX mode,
and bison is writing files to the cwd which is not what Automake expects.
it isn't that the src/file/ directory doesn't exist -- you can see from your
log that files have been generated in src/file/ before the failure.
-mike


signature.asc
Description: PGP signature