Bug#580492: bug#6124: basename/dirname are not POSIX compliant on arg starting with a dash

[Eric Blake]

On 05/06/2010 09:38 AM, Vincent Lefevre wrote:
>>> there's a test for "basename --" first,
>>
>> Can you find an actual example of this?  I think you read it
>> incorrectly, as really, autoconf first tests for 'basename -- /', but if
>> you indeed found an instance of autoconf testing for 'basename --', that
>> would be a bug in autoconf that needs to be reported to that list.
> 
> Sorry, I meant it was testing basename with "--" as the first
> argument while in this case it is useless. So, I wondered whether
> the goal of this test was to test whether this form was supported.

Yes, the goal of that autoconf test is to determine whether basename(1)
supports leading --, as required by POSIX.

>> Or, more likely, perhaps they are independently obeying what POSIX says,
>> without reference to either coreutils or busybox.
> 
> So, I think I should reassign the bug to busybox?

Yes.

-- 
Eric Blake   ebl...@redhat.com+1-801-349-2682
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Bug#580492: bug#6124: basename/dirname are not POSIX compliant on arg starting with a dash

[Vincent Lefevre]

On 2010-05-06 08:02:04 -0600, Eric Blake wrote:
> On 05/06/2010 07:23 AM, Vincent Lefevre wrote:
> > According to
> > 
> >   grep 'basename --' /usr/bin/*
> >   grep 'basename \$' /usr/bin/*
> > 
> > on my Debian machine, almost all scripts use basename without "--".
> > The only exceptions are autoconf (but I'm not sure this counts, as
> > there's a test for "basename --" first,
> 
> Can you find an actual example of this?  I think you read it
> incorrectly, as really, autoconf first tests for 'basename -- /', but if
> you indeed found an instance of autoconf testing for 'basename --', that
> would be a bug in autoconf that needs to be reported to that list.

Sorry, I meant it was testing basename with "--" as the first
argument while in this case it is useless. So, I wondered whether
the goal of this test was to test whether this form was supported.

> >> Note solaris behaves like busybox and openbsd behaves like coreutils.
> > 
> > Perhaps they're copying coreutils without looking at what POSIX says.
> 
> Or, more likely, perhaps they are independently obeying what POSIX says,
> without reference to either coreutils or busybox.

So, I think I should reassign the bug to busybox?

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#580492: bug#6124: basename/dirname are not POSIX compliant on arg starting with a dash

[Michael Stone]

On Thu, May 06, 2010 at 04:24:10PM +0200, you wrote:

However, if extensions are allowed, this would mean that many scripts
would have to be fixed.


Frankly, there are vast numbers of scripts which don't use -- at all, 
even for things like rm. I think you overstate both the amount of work 
and the likelihood that many scripts will bother.


Mike Stone



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#580492: bug#6124: basename/dirname are not POSIX compliant on arg starting with a dash

[Eric Blake]

On 05/06/2010 08:24 AM, Vincent Lefevre wrote:
>> Default Behavior: When this section is listed as "None.", it means
>> that the implementation need not support any options. Standard utilities
>> that do not accept options, but that do accept operands, shall recognize
>> "--" as a first argument to be discarded.
> 
> OK, an alternative way would be that an exception is added to POSIX,
> just like for "echo".

Please no.  echo has an exception precisely because it has historical
baggage, and that is what makes echo so pitiful that POSIX itself states
that printf(1) is more portable than echo(1).  Relatively newer
applications, like basename and dirname, that were first written in an
age when consistency was more important, should not be burdened with the
historical baggage of ancient echo(1).  And changing it now is too late;
just as you argue about the number of broken scripts that are not robust
because they failed to use --, you have to also consider the number of
correct scripts that have been written with the correct assumption that
basename and dirname accept --: it has been 9 years since POSIX 2001
first codified that behavior.

Besides, we would much prefer to keep 'basename --help' as a GNU
extension.  Not having 'POSIXLY_CORRECT=1 /bin/echo --help' give useful
help is an annoyance that we would rather avoid repeating.

> 
> However, if extensions are allowed, this would mean that many scripts
> would have to be fixed.

Yes, many broken scripts need to be fixed.  But that is the fault of
those broken scripts, not coreutils.

-- 
Eric Blake   ebl...@redhat.com+1-801-349-2682
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Bug#580492: bug#6124: basename/dirname are not POSIX compliant on arg starting with a dash

[Vincent Lefevre]

On 2010-05-06 07:41:59 -0600, Eric Blake wrote:
> No, you're missing one other key point of POSIX:
> 
> http://www.opengroup.org/onlinepubs/9699919799/utilities/V3_chap01.html
> Section 1.4, OPTIONS:
> 
> Default Behavior: When this section is listed as "None.", it means
> that the implementation need not support any options. Standard utilities
> that do not accept options, but that do accept operands, shall recognize
> "--" as a first argument to be discarded.

OK, an alternative way would be that an exception is added to POSIX,
just like for "echo".

> The requirement for recognizing "--" is because conforming
> applications need a way to shield their operands from any arbitrary
> options that the implementation may provide as an extension. For
> example, if the standard utility foo is listed as taking no options, and
> the application needed to give it a pathname with a leading , it
> could safely do it as:
> 
> foo -- -myfile
> 
> and avoid any problems with -m used as an extension.

However, if extensions are allowed, this would mean that many scripts
would have to be fixed.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#580492: bug#6124: basename/dirname are not POSIX compliant on arg starting with a dash

[Eric Blake]

On 05/06/2010 07:23 AM, Vincent Lefevre wrote:
> If one follows POSIX, the correct way to do it is:
> 
>   base=$(basename "$path")
> 
> and this is robust because the first argument is necessarily the path
> (it cannot be an option, even "--").

POSIX does not forbid applications from adding extension options.  And
BSD basename has done just that:

SYNOPSIS
 basename string [suffix]
 basename [-a] [-s suffix] string [...]
 dirname string [...]

And since POSIX allows FreeBSD's extensions to exist (although a
portable script will not use them), it must also require that a portable
app be able to guarantee a way to bypass those extensions (ergo
base=$(basename -- "$path") is a POSIX requirement).

> According to
> 
>   grep 'basename --' /usr/bin/*
>   grep 'basename \$' /usr/bin/*
> 
> on my Debian machine, almost all scripts use basename without "--".
> The only exceptions are autoconf (but I'm not sure this counts, as
> there's a test for "basename --" first,

Can you find an actual example of this?  I think you read it
incorrectly, as really, autoconf first tests for 'basename -- /', but if
you indeed found an instance of autoconf testing for 'basename --', that
would be a bug in autoconf that needs to be reported to that list.

>> Note solaris behaves like busybox and openbsd behaves like coreutils.
> 
> Perhaps they're copying coreutils without looking at what POSIX says.

Or, more likely, perhaps they are independently obeying what POSIX says,
without reference to either coreutils or busybox.

-- 
Eric Blake   ebl...@redhat.com+1-801-349-2682
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Bug#580492: bug#6124: basename/dirname are not POSIX compliant on arg starting with a dash

[Eric Blake]

On 05/06/2010 07:41 AM, Eric Blake wrote:
>> According to POSIX[*], basename and dirname take no options (examples
>> in POSIX also assume that), so that this would make BusyBox's behavior
>> correct.
> 
> No, you're missing one other key point of POSIX:
> 
> http://www.opengroup.org/onlinepubs/9699919799/utilities/V3_chap01.html
> Section 1.4, OPTIONS:

If you want more evidence from POSIX that 'basename -- foo' is mandated
behavior, read up on this correction to POSIX:
http://austingroupbugs.net/view.php?id=192

-- 
Eric Blake   ebl...@redhat.com+1-801-349-2682
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Bug#580492: bug#6124: basename/dirname are not POSIX compliant on arg starting with a dash

[Eric Blake]

On 05/06/2010 05:15 AM, Vincent Lefevre wrote:
> Hi,
> 
> I've reported the following bug in the Debian BTS:
> 
>   http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=580492

That bug should be reassigned to busybox.

> 
> With the basename and dirname utilities from GNU Coreutils, one has:
> 
> ypig% POSIXLY_CORRECT=1 basename --
> basename: missing operand

Correct.

> 
> ypig% busybox basename --
> --

Buggy.

> According to POSIX[*], basename and dirname take no options (examples
> in POSIX also assume that), so that this would make BusyBox's behavior
> correct.

No, you're missing one other key point of POSIX:

http://www.opengroup.org/onlinepubs/9699919799/utilities/V3_chap01.html
Section 1.4, OPTIONS:

Default Behavior: When this section is listed as "None.", it means
that the implementation need not support any options. Standard utilities
that do not accept options, but that do accept operands, shall recognize
"--" as a first argument to be discarded.

The requirement for recognizing "--" is because conforming
applications need a way to shield their operands from any arbitrary
options that the implementation may provide as an extension. For
example, if the standard utility foo is listed as taking no options, and
the application needed to give it a pathname with a leading , it
could safely do it as:

foo -- -myfile

and avoid any problems with -m used as an extension.

-- 
Eric Blake   ebl...@redhat.com+1-801-349-2682
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Bug#580492: bug#6124: basename/dirname are not POSIX compliant on arg starting with a dash

[Vincent Lefevre]

On 2010-05-06 13:50:27 +0100, Pádraig Brady wrote:
> The above busybox behavior implies that
> if you write a robust script like this for coreutils,
> that it is not portable to busybox:
> 
> base=$(basename -- "$path")

If one follows POSIX, the correct way to do it is:

  base=$(basename "$path")

and this is robust because the first argument is necessarily the path
(it cannot be an option, even "--").

> That also implies that we can't change the behavior.

According to

  grep 'basename --' /usr/bin/*
  grep 'basename \$' /usr/bin/*

on my Debian machine, almost all scripts use basename without "--".
The only exceptions are autoconf (but I'm not sure this counts, as
there's a test for "basename --" first, and I don't know what it
really expects) and savelog (from the debianutils package, thus
specific to Debian, but savelog also uses `basename $0`, so that
it is already inconsistent). Therefore I assume that changing
coreutils would be easier.

> Because there is an optional suffix parameter, one can't
> know which behavior is required for the above example.
> I suppose one could choose based on POSIXLY_CORRECT.

I think that it should really behave correctly if POSIXLY_CORRECT
is set, but also if it isn't set, given what appears to be the
current practice.

> Note solaris behaves like busybox and openbsd behaves like coreutils.

Perhaps they're copying coreutils without looking at what POSIX says.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)



--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#580492: bug#6124: basename/dirname are not POSIX compliant on arg starting with a dash

[Michael Stone]

On Thu, May 06, 2010 at 01:50:27PM +0100, Pádraig Brady wrote:

The above busybox behavior implies that
if you write a robust script like this for coreutils,
that it is not portable to busybox:

base=$(basename -- "$path")


Yup, I'd suggest busybox should change. :-)


Note solaris behaves like busybox and openbsd behaves like coreutils.


freebsd & OSX also use the existing coreutils behavior. So that's most 
of the deployed unixbase at this point.


Mike Stone




--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#580492: bug#6124: basename/dirname are not POSIX compliant on arg starting with a dash

[Pádraig Brady]

On 06/05/10 12:15, Vincent Lefevre wrote:
> Hi,
> 
> I've reported the following bug in the Debian BTS:
> 
>   http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=580492
> 
> With the basename and dirname utilities from GNU Coreutils, one has:
> 
> ypig% POSIXLY_CORRECT=1 basename --
> basename: missing operand
> Try `basename --help' for more information.
> ypig% POSIXLY_CORRECT=1 dirname --
> dirname: missing operand
> Try `dirname --help' for more information.
> 
> instead of:
> 
> ypig% busybox basename --
> --
> ypig% busybox dirname --
> .
> 
> According to POSIX[*], basename and dirname take no options (examples
> in POSIX also assume that), so that this would make BusyBox's behavior
> correct.
> 
> [*] http://www.opengroup.org/onlinepubs/9699919799/utilities/basename.html
> http://www.opengroup.org/onlinepubs/9699919799/utilities/dirname.html
> 

The above busybox behavior implies that
if you write a robust script like this for coreutils,
that it is not portable to busybox:

base=$(basename -- "$path")

That also implies that we can't change the behavior.
Because there is an optional suffix parameter, one can't
know which behavior is required for the above example.
I suppose one could choose based on POSIXLY_CORRECT.

Note solaris behaves like busybox and openbsd behaves like coreutils.

cheers,
Pádraig.



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org