Re: Alpine: useless-if-before-free: Exec format error

[Tim Rühsen]

On 6/23/19 6:32 PM, Jim Meyering wrote:
> On Wed, Jun 19, 2019 at 3:12 AM Bruno Haible  wrote:
> ...
>> Done like this:
>>
>> 2019-06-19  Bruno Haible  
>>
>> Reorder pieces of header in perl scripts.
>> The desired order is
>> - Prologue part 1 (2 lines with #!)
>> - Program short description
>> - Copyright and license notice
>> - Written-by notice
>> - Program short description (optional)
>> - Program long description (optional)
>> - Prologue part 2
>> - Time stamp
>> - Code
>> Reported by Paul Eggert.
>> * build-aux/announce-gen: Reorder header.
>> * build-aux/gitlog-to-changelog: Likewise.
>> * build-aux/useless-if-before-free: Likewise.
>> * build-aux/prefix-gnulib-mk: Add copyright notice and short
>> description.
>> * build-aux/update-copyright: Likewise. Add short description. Bump
>> time-stamp-line-limit to 200.
> 
> Thank you all for improving those scripts!
> 

I didn't expect this to escalate so much - and can only repeat what Jim
says, thank you for doing all this work !

Regards, Tim



signature.asc
Description: OpenPGP digital signature

Re: Alpine: useless-if-before-free: Exec format error

[Jim Meyering]

On Wed, Jun 19, 2019 at 3:12 AM Bruno Haible  wrote:
...
> Done like this:
>
> 2019-06-19  Bruno Haible  
>
> Reorder pieces of header in perl scripts.
> The desired order is
> - Prologue part 1 (2 lines with #!)
> - Program short description
> - Copyright and license notice
> - Written-by notice
> - Program short description (optional)
> - Program long description (optional)
> - Prologue part 2
> - Time stamp
> - Code
> Reported by Paul Eggert.
> * build-aux/announce-gen: Reorder header.
> * build-aux/gitlog-to-changelog: Likewise.
> * build-aux/useless-if-before-free: Likewise.
> * build-aux/prefix-gnulib-mk: Add copyright notice and short
> description.
> * build-aux/update-copyright: Likewise. Add short description. Bump
> time-stamp-line-limit to 200.

Thank you all for improving those scripts!

Re: Alpine: useless-if-before-free: Exec format error

[Bruno Haible]

Paul Eggert wrote:
> >  Prologue part 1 (2 lines with #!)
> >  Program short description
> >  Copyright and license notice
> >  Written-by notice
> >  Program short description (optional)
> >  Program long description
> >  Prologue part 2
> >  Time stamp
> >  Code
> >
> > Would that be OK with you?
> 
> Yes, that sounds OK, thanks.

Done like this:


2019-06-19  Bruno Haible  

Reorder pieces of header in perl scripts.
The desired order is
- Prologue part 1 (2 lines with #!)
- Program short description
- Copyright and license notice
- Written-by notice
- Program short description (optional)
- Program long description (optional)
- Prologue part 2
- Time stamp
- Code
Reported by Paul Eggert.
* build-aux/announce-gen: Reorder header.
* build-aux/gitlog-to-changelog: Likewise.
* build-aux/useless-if-before-free: Likewise.
* build-aux/prefix-gnulib-mk: Add copyright notice and short
description.
* build-aux/update-copyright: Likewise. Add short description. Bump
time-stamp-line-limit to 200.

diff --git a/build-aux/announce-gen b/build-aux/announce-gen
index 5921dcc..8fe0cc8 100755
--- a/build-aux/announce-gen
+++ b/build-aux/announce-gen
@@ -1,5 +1,25 @@
 #!/bin/sh
 #! -*-perl-*-
+
+# Generate a release announcement message.
+
+# Copyright (C) 2002-2019 Free Software Foundation, Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+# Written by Jim Meyering
+
 # This is a prologue that allows to run a perl script as an executable
 # on systems that are compliant to a POSIX version before POSIX:2017.
 # On such systems, the usual invocation of an executable through execlp()
@@ -15,33 +35,13 @@
 eval 'exec perl -wSx "$0" "$@"'
  if 0;
 
-# Generate a release announcement message.
-
 my $VERSION = '2018-03-07 03:46'; # UTC
 # The definition above must lie within the first 8 lines in order
 # for the Emacs time-stamp write hook (at end) to update it.
 # If you change this file with Emacs, please let the write hook
 # do its job.  Otherwise, update this string manually.
 
-# Copyright (C) 2002-2019 Free Software Foundation, Inc.
-
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see .
-
-# Written by Jim Meyering
-
 use strict;
-
 use Getopt::Long;
 use POSIX qw(strftime);
 
diff --git a/build-aux/gitlog-to-changelog b/build-aux/gitlog-to-changelog
index 40b839e..3acfa8b 100755
--- a/build-aux/gitlog-to-changelog
+++ b/build-aux/gitlog-to-changelog
@@ -1,5 +1,25 @@
 #!/bin/sh
 #! -*-perl-*-
+
+# Convert git log output to ChangeLog format.
+
+# Copyright (C) 2008-2019 Free Software Foundation, Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+# Written by Jim Meyering
+
 # This is a prologue that allows to run a perl script as an executable
 # on systems that are compliant to a POSIX version before POSIX:2017.
 # On such systems, the usual invocation of an executable through execlp()
@@ -15,31 +35,12 @@
 eval 'exec perl -wSx "$0" "$@"'
  if 0;
 
-# Convert git log output to ChangeLog format.
-
 my $VERSION = '2018-03-07 03:47'; # UTC
 # The definition 

Re: Alpine: useless-if-before-free: Exec format error

[Paul Eggert]

On 6/17/19 6:32 PM, Bruno Haible wrote:

In some cases, it will be useful to repeat the Program short description, then
(for those who start reading after the license notice). Like this:

 Prologue part 1 (2 lines with #!)
 Program short description
 Copyright and license notice
 Written-by notice
 Program short description (optional)
 Program long description
 Prologue part 2
 Time stamp
 Code

Would that be OK with you?


Yes, that sounds OK, thanks.

Re: Alpine: useless-if-before-free: Exec format error

[Bruno Haible]

Paul Eggert wrote:
> > How about if we reorder it to:
> >Prologue part 1 (2 lines with #!)
> >Copyright and license notice
> >Written-by notice
> >Program short description
> >Program long description
> >Prologue part 2
> >Time stamp
> >Code
> 
> That would be better, but I'd still prefer the program short description 
> (I assume it's one line or so) to be as early as possible, which looks 
> like it'd be line 3 since the first two lines are immovable.

In some cases, it will be useful to repeat the Program short description, then
(for those who start reading after the license notice). Like this:

Prologue part 1 (2 lines with #!)
Program short description
Copyright and license notice
Written-by notice
Program short description (optional)
Program long description
Prologue part 2
Time stamp
Code

Would that be OK with you?

Bruno

Re: Alpine: useless-if-before-free: Exec format error

[Paul Eggert]

On 6/16/19 6:12 AM, Bruno Haible wrote:

How about if we reorder it to:
   Prologue part 1 (2 lines with #!)
   Copyright and license notice
   Written-by notice
   Program short description
   Program long description
   Prologue part 2
   Time stamp
   Code


That would be better, but I'd still prefer the program short description 
(I assume it's one line or so) to be as early as possible, which looks 
like it'd be line 3 since the first two lines are immovable.

Re: Alpine: useless-if-before-free: Exec format error

[Bruno Haible]

Paul Eggert wrote:
> Your suggestion to use a followup 
> comment with fancy highlighting characters is less useful to my eyes; 
> I'm used to the more-standard convention of a short comment first.

So, what can we do?

Moving the prologue's comment to another file would be bad. Comments
should be very close to the code, otherwise they are overlooked.

Would it help to reorder the file's header? Currently it has
  Prologue part 1 (2 lines with #!)
  Prologue part 2
  Program short description
  Time stamp
  Copyright and license notice
  Written-by notice (needed for upholding copyright, at least in German courts 
[1])
  Program long description
  Code

How about if we reorder it to:
  Prologue part 1 (2 lines with #!)
  Copyright and license notice
  Written-by notice
  Program short description
  Program long description
  Prologue part 2
  Time stamp
  Code

Bruno

[1] I'm referring to the Hellwig vs. VMware judgement.

Re: Alpine: useless-if-before-free: Exec format error

[Paul Eggert]

On 6/15/19 1:14 PM, Bruno Haible wrote:

I found that they got in the way of my understanding the code

Do you mean, we should highlight the comment about the program itself?


Partly that - the first comment of a script should be very brief 
(typically one line) and say what the script does; it shouldn't jump 
into gory implementation details. Your suggestion to use a followup 
comment with fancy highlighting characters is less useful to my eyes; 
I'm used to the more-standard convention of a short comment first.


But mostly because it's a big blob of text that just gets in the way of 
likely readers. A brief reference to perlrun will be better both for 
readers who already know this stuff (and don't need the details, just a 
reminder) and for readers who don't (for which the comment itself is a 
big and barely-understandable blob and will require reading perlrun and 
other stuff anyway). At least, that's how the comment strikes this 
particular reader.


But I won't fight about it.

Re: Alpine: useless-if-before-free: Exec format error

[Bruno Haible]

Hi Paul,

> One quibble: those long comments are 
> probably more trouble than they're worth.

I heavily disagree: This piece of code needs to consider
  - the distinction between POSIX:2017 and earlier versions,
  - execlp/execvp vs. execl/execv,
  - GuixSD vs. traditional file system layouts,
  - perl documentation,
  - perl source code (toke.c).

> I suggest we just cite perlrun's man page

No, that man page represents only a small part of the entire picture.
Future maintainers should not have to wade through a mail thread of 12 mails
in order to understand a piece of code.

> I found that at least one important detail 
> was missing: the "#! -*-perl-*-" line is not just there for Emacs, but is 
> also 
> there for Perl.

Oh, indeed! So, both this line and the '-x' option are essential for perl.
I'm updating the comment:

--- a/build-aux/useless-if-before-free
+++ b/build-aux/useless-if-before-free
@@ -6,12 +6,12 @@
 # or execvp() fails with ENOEXEC if it is a script that does not start
 # with a #! line.  The script interpreter mentioned in the #! line has
 # to be /bin/sh, because on GuixSD systems that is the only program that
-# has a fixed file name.  The second line is for editing this file in
-# Emacs.  The next two lines below are valid code in both sh and perl.
-# When executed by sh, they re-execute the script through the perl
-# program found in $PATH.  The '-x' option is essential; without it,
-# perl would re-execute the script through /bin/sh.  When executed by
-# perl, the next two lines are a no-op.
+# has a fixed file name.  The second line is essential for perl and is
+# also useful for editing this file in Emacs.  The next two lines below
+# are valid code in both sh and perl.  When executed by sh, they re-execute
+# the script through the perl program found in $PATH.  The '-x' option
+# is essential as well; without it, perl would re-execute the script
+# through /bin/sh.  When executed by  perl, the next two lines are a no-op.
 eval 'exec perl -wSx "$0" "$@"'
  if 0;
 

> I found that they got in the way of my understanding the code

Do you mean, we should highlight the comment about the program itself?
Like this:

#!/bin/sh
#! -*-perl-*-
# This is a prologue that allows to run a perl script as an executable
# on systems that are compliant to a POSIX version before POSIX:2017.
# On such systems, the usual invocation of an executable through execlp()
# or execvp() fails with ENOEXEC if it is a script that does not start
# with a #! line.  The script interpreter mentioned in the #! line has
# to be /bin/sh, because on GuixSD systems that is the only program that
# has a fixed file name.  The second line is essential for perl and is
# also useful for editing this file in Emacs.  The next two lines below
# are valid code in both sh and perl.  When executed by sh, they re-execute
# the script through the perl program found in $PATH.  The '-x' option
# is essential as well; without it, perl would re-execute the script
# through /bin/sh.  When executed by  perl, the next two lines are a no-op.
eval 'exec perl -wSx "$0" "$@"'
 if 0;

# ╭───╮
# │ Detect instances of "if (p) free (p);".   │
# │ Likewise "if (p != 0)", "if (0 != p)", or with NULL; and with braces. │
# ╰───╯

my $VERSION = '2018-03-07 03:47'; # UTC
...

Bruno

Re: Alpine: useless-if-before-free: Exec format error

[Paul Eggert]

Thanks for looking into and fixing that. One quibble: those long comments are 
probably more trouble than they're worth. I found that they got in the way of my 
understanding the code, as my eyes glazed over when reading them. And when I 
finally forced myself to read one, I found that at least one important detail 
was missing: the "#! -*-perl-*-" line is not just there for Emacs, but is also 
there for Perl.


Instead of adding even more length to a comment that is already a bit of an 
obstacle to understanding the code, I suggest we just cite perlrun's man page as 
in the attached proposed patch. This should be good enough for anyone who's not 
already versed in perl+sh.
>From 7969fe73da89ac372c681dca7839b880b59767f1 Mon Sep 17 00:00:00 2001
From: Paul Eggert 
Date: Sat, 15 Jun 2019 11:27:52 -0700
Subject: [PATCH] Use shorter perlrun comment

* build-aux/announce-gen, build-aux/gitlog-to-changelog:
* build-aux/prefix-gnulib-mk, build-aux/update-copyright:
* build-aux/useless-if-before-free, tests/test-update-copyright.sh:
Just cite perlrun(1) in the comment.
---
 ChangeLog|  8 
 build-aux/announce-gen   | 13 +
 build-aux/gitlog-to-changelog| 13 +
 build-aux/prefix-gnulib-mk   | 13 +
 build-aux/update-copyright   | 13 +
 build-aux/useless-if-before-free | 13 +
 tests/test-update-copyright.sh   | 13 +
 7 files changed, 14 insertions(+), 72 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index a0d2f415f..77e66c0e9 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,11 @@
+2019-06-15  Paul Eggert  
+
+	Use shorter perlrun comment
+	* build-aux/announce-gen, build-aux/gitlog-to-changelog:
+	* build-aux/prefix-gnulib-mk, build-aux/update-copyright:
+	* build-aux/useless-if-before-free, tests/test-update-copyright.sh:
+	Just cite perlrun(1) in the comment.
+
 2019-06-15  Bruno Haible  
 
 	Fix scripts to have valid executable format on Alpine Linux.
diff --git a/build-aux/announce-gen b/build-aux/announce-gen
index b5728338c..1e2648d0e 100755
--- a/build-aux/announce-gen
+++ b/build-aux/announce-gen
@@ -1,17 +1,6 @@
 #!/bin/sh
 #! -*-perl-*-
-# This is a prologue that allows to run a perl script as an executable
-# on systems that are compliant to a POSIX version before POSIX:2017.
-# On such systems, the usual invocation of an executable through execlp()
-# or execvp() fails with ENOEXEC if it is a script that does not start
-# with a #! line.  The script interpreter mentioned in the #! line has
-# to be /bin/sh, because on GuixSD systems that is the only program that
-# has a fixed file name.  The second line is for editing this file in
-# Emacs.  The next two lines below are valid code in both sh and perl.
-# When executed by sh, they re-execute the script through the perl
-# program found in $PATH.  The '-x' option is essential; without it,
-# perl would re-execute the script through /bin/sh.  When executed by
-# perl, the next two lines are a no-op.
+# Either perl or sh can run this file; see perlrun(1).
 eval 'exec perl -wSx "$0" "$@"'
  if 0;
 
diff --git a/build-aux/gitlog-to-changelog b/build-aux/gitlog-to-changelog
index a616b8234..68722bacb 100755
--- a/build-aux/gitlog-to-changelog
+++ b/build-aux/gitlog-to-changelog
@@ -1,17 +1,6 @@
 #!/bin/sh
 #! -*-perl-*-
-# This is a prologue that allows to run a perl script as an executable
-# on systems that are compliant to a POSIX version before POSIX:2017.
-# On such systems, the usual invocation of an executable through execlp()
-# or execvp() fails with ENOEXEC if it is a script that does not start
-# with a #! line.  The script interpreter mentioned in the #! line has
-# to be /bin/sh, because on GuixSD systems that is the only program that
-# has a fixed file name.  The second line is for editing this file in
-# Emacs.  The next two lines below are valid code in both sh and perl.
-# When executed by sh, they re-execute the script through the perl
-# program found in $PATH.  The '-x' option is essential; without it,
-# perl would re-execute the script through /bin/sh.  When executed by
-# perl, the next two lines are a no-op.
+# Either perl or sh can run this file; see perlrun(1).
 eval 'exec perl -wSx "$0" "$@"'
  if 0;
 
diff --git a/build-aux/prefix-gnulib-mk b/build-aux/prefix-gnulib-mk
index 66e138b46..10871a286 100755
--- a/build-aux/prefix-gnulib-mk
+++ b/build-aux/prefix-gnulib-mk
@@ -1,17 +1,6 @@
 #!/bin/sh
 #! -*-perl-*-
-# This is a prologue that allows to run a perl script as an executable
-# on systems that are compliant to a POSIX version before POSIX:2017.
-# On such systems, the usual invocation of an executable through execlp()
-# or execvp() fails with ENOEXEC if it is a script that does not start
-# with a #! line.  The script interpreter mentioned in the #! line has
-# to be /bin/sh, because on GuixSD systems that is the only program that
-# has a fixed file name.  The second line is for editing this 

Re: Alpine: useless-if-before-free: Exec format error

[Darshit Shah]

* Bruno Haible  [190615 13:07]:

> 2019-06-15  Bruno Haible  
> 
>   Fix scripts to have valid executable format on Alpine Linux.
>   Reported by Tim Rühsen .
>   Idea by Paul Eggert.
>   * build-aux/useless-if-before-free: Use a prologue that starts with
>   '#!/bin/sh'.
>   * build-aux/announce-gen: Likewise.
>   * build-aux/gitlog-to-changelog: Likewise.
>   * build-aux/prefix-gnulib-mk: Likewise.
>   * build-aux/update-copyright: Likewise.
>   * tests/test-update-copyright.sh: Update test program accordingly.
> 
> diff --git a/build-aux/announce-gen b/build-aux/announce-gen
> index 0174f5c..b572833 100755
> --- a/build-aux/announce-gen
> +++ b/build-aux/announce-gen
> @@ -1,6 +1,20 @@
> -eval '(exit $?0)' && eval 'exec perl -wS "$0" "$@"'
> -  & eval 'exec perl -wS "$0" $argv:q'
> -if 0;
> +#!/bin/sh
> +#! -*-perl-*-
> +# This is a prologue that allows to run a perl script as an executable
> +# on systems that are compliant to a POSIX version before POSIX:2017.
> +# On such systems, the usual invocation of an executable through execlp()
> +# or execvp() fails with ENOEXEC if it is a script that does not start
> +# with a #! line.  The script interpreter mentioned in the #! line has
> +# to be /bin/sh, because on GuixSD systems that is the only program that
> +# has a fixed file name.  The second line is for editing this file in
> +# Emacs.  The next two lines below are valid code in both sh and perl.
> +# When executed by sh, they re-execute the script through the perl
> +# program found in $PATH.  The '-x' option is essential; without it,
> +# perl would re-execute the script through /bin/sh.  When executed by
> +# perl, the next two lines are a no-op.
> +eval 'exec perl -wSx "$0" "$@"'
> + if 0;
> +
>  # Generate a release announcement message.
>  
>  my $VERSION = '2018-03-07 03:46'; # UTC

Won't this break the emacs timestamp hook which apparently needs to be within
the first eight lines of the file? Maybe either reduce the text, or move the
hooks around.


-- 
Thanking You,
Darshit Shah
PGP Fingerprint: 7845 120B 07CB D8D6 ECE5 FF2B 2A17 43ED A91A 35B6


signature.asc
Description: PGP signature

Re: Alpine: useless-if-before-free: Exec format error

[Bruno Haible]

Paul Eggert wrote:
>  suggests something like the 
> following instead. This would avoid having separate shell scripts and .pl 
> files, 
> which would be a plus:
> 
> #!/bin/sh
> #! −*−perl−*−
> eval 'exec perl −x −wS $0 ${1+"$@"}'
>  if 0;

This does not work for me (with perl 5.22.1), unfortunately:

$ build-aux/useless-if-before-free
Can't open perl script "−x": No such file or directory

Ah, it's because it contains a non-ASCII '−' character! This one works:

#!/bin/sh
#! -*-perl-*-
eval 'exec perl -x -wS $0 ${1+"$@"}'
 if 0;

This one as well:

#!/bin/sh
#! -*-perl-*-
eval 'exec perl -wS -x $0 ${1+"$@"}'
 if 0;

Or this one:

#!/bin/sh
#! -*-perl-*-
eval 'exec perl -wSx $0 ${1+"$@"}'
 if 0;

${1+"$@"} is the same as "$@". Haven't seen a platform that needs
${1+"$@"} in 20 years. See also the discussion in

 .

And $0 needs to be double-quoted (in case the file name contains a space).

So, I'll use this prologue:

#!/bin/sh
#! -*-perl-*-
eval 'exec perl -wSx "$0" "$@"'
 if 0;


2019-06-15  Bruno Haible  

Fix scripts to have valid executable format on Alpine Linux.
Reported by Tim Rühsen .
Idea by Paul Eggert.
* build-aux/useless-if-before-free: Use a prologue that starts with
'#!/bin/sh'.
* build-aux/announce-gen: Likewise.
* build-aux/gitlog-to-changelog: Likewise.
* build-aux/prefix-gnulib-mk: Likewise.
* build-aux/update-copyright: Likewise.
* tests/test-update-copyright.sh: Update test program accordingly.

diff --git a/build-aux/announce-gen b/build-aux/announce-gen
index 0174f5c..b572833 100755
--- a/build-aux/announce-gen
+++ b/build-aux/announce-gen
@@ -1,6 +1,20 @@
-eval '(exit $?0)' && eval 'exec perl -wS "$0" "$@"'
-  & eval 'exec perl -wS "$0" $argv:q'
-if 0;
+#!/bin/sh
+#! -*-perl-*-
+# This is a prologue that allows to run a perl script as an executable
+# on systems that are compliant to a POSIX version before POSIX:2017.
+# On such systems, the usual invocation of an executable through execlp()
+# or execvp() fails with ENOEXEC if it is a script that does not start
+# with a #! line.  The script interpreter mentioned in the #! line has
+# to be /bin/sh, because on GuixSD systems that is the only program that
+# has a fixed file name.  The second line is for editing this file in
+# Emacs.  The next two lines below are valid code in both sh and perl.
+# When executed by sh, they re-execute the script through the perl
+# program found in $PATH.  The '-x' option is essential; without it,
+# perl would re-execute the script through /bin/sh.  When executed by
+# perl, the next two lines are a no-op.
+eval 'exec perl -wSx "$0" "$@"'
+ if 0;
+
 # Generate a release announcement message.
 
 my $VERSION = '2018-03-07 03:46'; # UTC
diff --git a/build-aux/gitlog-to-changelog b/build-aux/gitlog-to-changelog
index deddef2..a616b82 100755
--- a/build-aux/gitlog-to-changelog
+++ b/build-aux/gitlog-to-changelog
@@ -1,6 +1,20 @@
-eval '(exit $?0)' && eval 'exec perl -wS "$0" "$@"'
-  & eval 'exec perl -wS "$0" $argv:q'
-if 0;
+#!/bin/sh
+#! -*-perl-*-
+# This is a prologue that allows to run a perl script as an executable
+# on systems that are compliant to a POSIX version before POSIX:2017.
+# On such systems, the usual invocation of an executable through execlp()
+# or execvp() fails with ENOEXEC if it is a script that does not start
+# with a #! line.  The script interpreter mentioned in the #! line has
+# to be /bin/sh, because on GuixSD systems that is the only program that
+# has a fixed file name.  The second line is for editing this file in
+# Emacs.  The next two lines below are valid code in both sh and perl.
+# When executed by sh, they re-execute the script through the perl
+# program found in $PATH.  The '-x' option is essential; without it,
+# perl would re-execute the script through /bin/sh.  When executed by
+# perl, the next two lines are a no-op.
+eval 'exec perl -wSx "$0" "$@"'
+ if 0;
+
 # Convert git log output to ChangeLog format.
 
 my $VERSION = '2018-03-07 03:47'; # UTC
diff --git a/build-aux/prefix-gnulib-mk b/build-aux/prefix-gnulib-mk
index bef726f..66e138b 100755
--- a/build-aux/prefix-gnulib-mk
+++ b/build-aux/prefix-gnulib-mk
@@ -1,6 +1,19 @@
-eval '(exit $?0)' && eval 'exec perl -wS "$0" "$@"'
-  & eval 'exec perl -wS "$0" $argv:q'
-if 0;
+#!/bin/sh
+#! -*-perl-*-
+# This is a prologue that allows to run a perl script as an executable
+# on systems that are compliant to a POSIX version before POSIX:2017.
+# On such systems, the usual invocation of an executable through execlp()
+# or execvp() fails with ENOEXEC if it is a script that does not start
+# with a #! line.  The script interpreter mentioned in the #! line has
+# to be /bin/sh, because on GuixSD systems that is the only program that
+# has a fixed file 

Re: Alpine: useless-if-before-free: Exec format error

[Bruno Haible]

I wrote:
> So, the ENOEXEC error is a bug in musl. But POSIX does not specify that the
> command interpreter for scripts without shebang is /bin/sh; therefore IMHO
> it would be good not to make this assumption.

Correction:
 says
  "In the cases where the other members of the exec family of functions
   would fail and set errno to [ENOEXEC], the execlp() and execvp() functions
   shall execute a command interpreter and the environment of the executed
   command shall be as if the process invoked the sh utility using execl()
   as follows:

 execl(, arg0, file, arg1, ..., (char *)0);

   where  is an unspecified pathname for the sh utility ..."

So, the command interpreter must be sh-compatible; only its file name is
unspecified.

=> We need to change 'useless-if-before-free' not because of POSIX,
but only because of the musl bug and/or for systems that are compliant
with older versions of POSIX (older than POSIX:2017).

Bruno

Re: Alpine: useless-if-before-free: Exec format error

[Paul Eggert]

Bruno Haible wrote:

   - Add a shell script useless-if-before-free that merely does this:

  #!/bin/sh
  exec perl "$0".pl "$@"
 suggests something like the 
following instead. This would avoid having separate shell scripts and .pl files, 
which would be a plus:


#!/bin/sh
#! −*−perl−*−
eval 'exec perl −x −wS $0 ${1+"$@"}'
if 0;

Re: Alpine: useless-if-before-free: Exec format error

[Bruno Haible]

Paul Eggert wrote:
> Perhaps we 
> should just prepend "#!/bin/sh" or "#!/usr/bin/env perl" (not sure 
> which) and let people on weird systems fix things by hand.

"#!/usr/bin/env perl" does not work on GuixSD (where the only program
that has a hardcoded file name is /bin/sh; there is no /usr and no
/bin/env on this distro).

Naively prepending "#!/bin/sh" leads to this (with perl 5.22.1):

$ build-aux/useless-if-before-free
/bin/sh: 0: Illegal option -w

$ strace -f build-aux/useless-if-before-free 2>&1 | grep ^exec
execve("build-aux/useless-if-before-free", 
["build-aux/useless-if-before-free"], [/* 79 vars */]) = 0
execve("/home/bruno/bin/perl", ["perl", "-wST", 
"build-aux/useless-if-before-free"], [/* 79 vars */]) = -1 ENOENT (No such file 
or directory)
execve("/home/bruno/local/bin/perl", ["perl", "-wST", 
"build-aux/useless-if-before-free"], [/* 79 vars */]) = -1 ENOENT (No such file 
or directory)
execve("/darch/x86_64-linux-gnu/gnu/bin/perl", ["perl", "-wST", 
"build-aux/useless-if-before-free"], [/* 79 vars */]) = -1 ENOENT (No such file 
or directory)
execve("/arch/local/x86_64-linux-gnu/bin/perl", ["perl", "-wST", 
"build-aux/useless-if-before-free"], [/* 79 vars */]) = -1 ENOENT (No such file 
or directory)
execve("/arch/local/x86_64-linux/bin/perl", ["perl", "-wST", 
"build-aux/useless-if-before-free"], [/* 79 vars */]) = -1 ENOENT (No such file 
or directory)
execve("/usr/local/sbin/perl", ["perl", "-wST", 
"build-aux/useless-if-before-free"], [/* 79 vars */]) = -1 ENOENT (No such file 
or directory)
execve("/usr/local/bin/perl", ["perl", "-wST", 
"build-aux/useless-if-before-free"], [/* 79 vars */]) = -1 ENOENT (No such file 
or directory)
execve("/usr/sbin/perl", ["perl", "-wST", "build-aux/useless-if-before-free"], 
[/* 79 vars */]) = -1 ENOENT (No such file or directory)
execve("/usr/bin/perl", ["perl", "-wST", "build-aux/useless-if-before-free"], 
[/* 79 vars */]) = 0
execve("/bin/sh", ["/bin/sh", "-wST", "build-aux/useless-if-before-free"], [/* 
79 vars */]) = 0

So, what you can see is that the redirect from /bin/sh to perl works right.
But perl then sees the first line and redirects back to /bin/sh. In other
words, there is logic in perl that amounts to "I refuse to be executed
through scripts that start with '#!/bin/sh'".

This finding is confirmed by the following comment in the perl source code:
/* ALTERNATE_SHEBANG:
 *  This symbol, if defined, contains a "magic" string which may be used
 *  as the first line of a Perl program designed to be executed directly
 *  by name, instead of the standard Unix #!.  If ALTERNATE_SHEBANG
 *  begins with a character other then #, then Perl will only treat
 *  it as a command line if it finds the string "perl" in the first
 *  word; otherwise it's treated as the first line of code in the script.
 *  (IOW, Perl won't hand off to another interpreter via an alternate
 *  shebang sequence that might be legal Perl code.)
 */

I propose this solution:

  - Rename 'useless-if-before-free' to 'useless-if-before-free.pl',
removing the first 3 lines, and removing the execute permission.
Also update the reference to ME in the --help output.

  - Add a shell script useless-if-before-free that merely does this:

 #!/bin/sh
 exec perl "$0".pl "$@"

  - Update the module description, to list both
build-aux/useless-if-before-free and build-aux/useless-if-before-free.pl.

And likewise for the other programs:

  build-aux/gitlog-to-changelog
  build-aux/update-copyright
  build-aux/announce-gen
  build-aux/prefix-gnulib-mk
  tests/test-update-copyright.sh

Bruno

Re: Alpine: useless-if-before-free: Exec format error

[Paul Eggert]

On 6/14/19 4:51 PM, Bruno Haible wrote:

But POSIX does not specify that the
command interpreter for scripts without shebang is /bin/sh; therefore IMHO
it would be good not to make this assumption.


Perhaps the code in those scripts was taken from an earlier version of 
the output of "perldoc perlrun". The current version 
 gives several alternative script 
headers (including at least one that appears to be a joke) and say that 
none work everywhere. It is a bit of a portability mess. Perhaps we 
should just prepend "#!/bin/sh" or "#!/usr/bin/env perl" (not sure 
which) and let people on weird systems fix things by hand.

Re: Alpine: useless-if-before-free: Exec format error

[Bruno Haible]

>Since xargs happens to use execvp(), the executable gets executed by "a
>known command interpreter". This command interpreter might be sh, python,
>emacs, or whatever.
> 
> So, the *omission* of #! renders a script non-portable, when that script is
> invoked like an executable.

Anyway, there seems to be an agreement that execvp() needs to execute scripts
through a command interpreter, unlike execv().

  * The glibc source code does so, see glibc/posix/execvpe.c.
Also the description of this commit:

https://sourceware.org/git/?p=glibc.git;a=commit;h=283d98512272a12cb84e7798c23edbdf1adb287d

  * The musl author agrees as well:
https://www.openwall.com/lists/musl/2018/03/09/1
https://www.openwall.com/lists/musl/2018/03/09/2
https://www.openwall.com/lists/musl/2018/03/11/1

So, the ENOEXEC error is a bug in musl. But POSIX does not specify that the
command interpreter for scripts without shebang is /bin/sh; therefore IMHO
it would be good not to make this assumption.

Bruno

Re: Alpine: useless-if-before-free: Exec format error

[Bruno Haible]

Eric Blake wrote:
> Conversely, POSIX requires that execution by 'sh' is the expected
> behavior when #! is missing, and that use of #! renders a script
> non-portable:
> 
> https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_01

This paragraph specifies what happens when you run
  sh some-file

Here, maint.mk invokes xargs, which invokes 'useless-if-before-free'.
xargs is not a shell. Therefore the portions of POSIX that matter here are

1) The description of "Executable File" in section 3.154 of
   https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html
   "The internal format of an executable file is unspecified"

2) The description of the exec() system call:
   https://pubs.opengroup.org/onlinepubs/9699919799/functions/exec.html
   "Historically, there have been two ways that implementations can exec shell
scripts.
One common historical implementation is that the execl(), execv(), execle(),
and execve() functions return an [ENOEXEC] error for any file not
recognizable as executable, including a shell script. When the execlp() and
execvp() functions encounter such a file, they assume the file to be a shell
script and invoke a known command interpreter to interpret such files. This
is now required by POSIX.1-2017. ..."

   Since xargs happens to use execvp(), the executable gets executed by "a
   known command interpreter". This command interpreter might be sh, python,
   emacs, or whatever.

So, the *omission* of #! renders a script non-portable, when that script is
invoked like an executable.

Bruno

Re: Alpine: useless-if-before-free: Exec format error

[Eric Blake]

On 6/14/19 4:28 PM, Bruno Haible wrote:
> Tim Rühsen wrote:
>> Do you think it makes more sense to open a bug at busybox then ?
> 
> I don't think so. Execution of scripts without shebang is considered
> legacy. Quoting  :
> 
>   "Note that, even in systems with full kernel support for the #! magic 
> number,
>some scripts lacking interpreter directives (although usually still
>requiring execute permission) are still runnable by virtue of the legacy
>script handling of the Bourne shell, still present in many of its modern
>descendants. Scripts are then interpreted by the user's default shell."
> 

Conversely, POSIX requires that execution by 'sh' is the expected
behavior when #! is missing, and that use of #! renders a script
non-portable:

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_01
"The shell reads its input from a file (see sh), from the -c option or
from the system() and popen() functions defined in the System Interfaces
volume of POSIX.1-2017. If the first line of a file of shell commands
starts with the characters "#!", the results are unspecified."

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: Alpine: useless-if-before-free: Exec format error

[Bruno Haible]

Tim Rühsen wrote:
> Do you think it makes more sense to open a bug at busybox then ?

I don't think so. Execution of scripts without shebang is considered
legacy. Quoting  :

  "Note that, even in systems with full kernel support for the #! magic number,
   some scripts lacking interpreter directives (although usually still
   requiring execute permission) are still runnable by virtue of the legacy
   script handling of the Bourne shell, still present in many of its modern
   descendants. Scripts are then interpreted by the user's default shell."

Bruno

Re: Alpine: useless-if-before-free: Exec format error

[Tim Rühsen]

On 14.06.19 19:50, Paul Eggert wrote:
> On 6/14/19 10:04 AM, Tim Rühsen wrote:
>> Adding a shebang to build-aux/useless-if-before-free fixes the issue.
> 
> What shebang should that be, exactly? The current style is used in
> several scripts (build-aux/announce-gen, build-aux/gitlog-to-changelog,
> build-aux/prefix-gnulib-mk, build-aux/update-copyright,
> build-aux/useless-if-before-free) and presumably works in a wide variety
> of systems. Adding a shebang might break one of those systems unless
> it's done carefully.

I am not an expert in this area, just mentioned the shebang to give a
hint. Alpine / busybox is a pretty common system and it would be nice to
see 'make syntax-check' work there as well.
Do you think it makes more sense to open a bug at busybox then ?

Regards, Tim



signature.asc
Description: OpenPGP digital signature

Re: Alpine: useless-if-before-free: Exec format error

[Paul Eggert]

On 6/14/19 10:04 AM, Tim Rühsen wrote:

Adding a shebang to build-aux/useless-if-before-free fixes the issue.


What shebang should that be, exactly? The current style is used in 
several scripts (build-aux/announce-gen, build-aux/gitlog-to-changelog, 
build-aux/prefix-gnulib-mk, build-aux/update-copyright, 
build-aux/useless-if-before-free) and presumably works in a wide variety 
of systems. Adding a shebang might break one of those systems unless 
it's done carefully.

Alpine: useless-if-before-free: Exec format error

[Tim Rühsen]

Hi,

on Alpine, with pretty much latest gnulib commit
5905d8ca9945f0d60ff40eb6cfa42afc0199ab8f, 'make syntax-check' throws out

xargs: ./build-aux/useless-if-before-free: Exec format error

Adding a shebang to build-aux/useless-if-before-free fixes the issue.

Regards, Tim



signature.asc
Description: OpenPGP digital signature