bug#24730: rmdir/mkdir error(s) and/or not working "reciprocally" w/each other

[Assaf Gordon]

Hello,

> On Oct 18, 2016, at 17:35, Linda Walsh  wrote:
> 
> Assaf Gordon wrote:
>>> rmdir -p ./a/b/c   # get error msg, but a,b,c removed.
>   I see... so in ".a/b/c", a,b,c are removed, but the error
> comes in "."?

Yes.

>   Ok, but is "-p" a posix switch in mkdir or rmdir?

Yes, it is defined in posix for both mkdir(1) and rmdir(1):

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/rmdir.html
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/mkdir.html


>> However, by definition, when 'rmdir' traverses the directories on the given 
>> path,
>> the directory 'a/..' is not empty (it contains 'a') - so this must fail.
> 
>   with "-p"?  Am talking about the case where you create a dir with 
> "mkdir -p "$dirname"
> and later, rmdir -p "$dirname" (where dirname is passed as a parameter, but
> is not user-input).

Yes. By posix definition, For "rmdir -p DIR" where DIR contains multiple path 
components,
rmdir should behave as if run with "rmdir -p $(dirname DIR)".

And for example:
  $ dirname "a/../b"
  a/..

And trying to call rmdir(2) on "a/.." will fail (definitely fails  on Linux 
kernel, though I suspect on other systems as well).


>   Right, but am trying to get rmdir to be a useful
> "opposite" to a "mkdir" -- at least w/r/t "-p"...

"useful" is somewhat subjective.
For the more common case of directories without ".." or "." - this works quite 
well,
and that is very likely the intended goal:

 mkdir -p a/b/c
 rmdir -p a/b/c


As the current behavior is mandated by POSIX, If you want to suggest a 
different behavior,
the best place to suggest this is to the posix group ( 
https://www.opengroup.org/austin/ ).
If the standard for 'rmdir(1)' is modified or adjusted, GNU coreutils will 
surely follow suit.

> But is -p's behavior in mkdir and rmdir mandated by POSIX?

Yes, see above links.

regards,
 - assaf

bug#24730: rmdir/mkdir error(s) and/or not working "reciprocally" w/each other

[Linda Walsh]

Assaf Gordon wrote:

Hello,

Before deciding on the wording, it's worth nothing that the errors and 
reasons for the errors are different between mkdir and rmdir, and 
between the two cases.


On 10/18/2016 03:49 PM, L. A. Walsh wrote:


mkdir -p ./a/b/c   # no error
rmdir -p ./a/b/c   # get error msg, but a,b,c removed.


The error in this case (at least on Linux) is "Invalid Argument",
because 'rmdir .' is invalid and rejected by the kernel (EINVAL)
while 'mkdir .' returns EEXISTS - and '-p' specifically instruct it to 
silently ignore EEXIST.


I see... so in ".a/b/c", a,b,c are removed, but the error
comes in "."?




$ strace -e rmdir rmdir .
rmdir(".")  = -1 EINVAL (Invalid argument)
rmdir: failed to remove '.': Invalid argument

This is also mandated by posix:
   http://pubs.opengroup.org/onlinepubs/9699919799/functions/rmdir.html
   "If the path argument refers to a path whose final component is 
either dot or dot-dot, rmdir() shall fail."

---
Ok, but is "-p" a posix switch in mkdir or rmdir?
If not, wouldn't the behavior be undefined?




mkdir -p a/../b  # no error
rmdir -p a/../b  # error, but a & b removed


At least on my system (Linux kernel 3.13), 'a' is not removed in the 
above example,

and the error is due to non-empty directory:

---
Right..., my bad, only checked 'b'. (*duh*)


Note that coreutils' mkdir contains an optimization not to try and make 
'..' as it is

guaranteed to exist (implemented in gnulib's mkancesdirs.c).

---
Right, same when rmdir traverses, when it hits "..", that
must exist (even in '/').  



However, by definition, when 'rmdir' traverses the directories on the 
given path,

the directory 'a/..' is not empty (it contains 'a') - so this must fail.


	with "-p"?  Am talking about the case where you 
create a dir with "mkdir -p "$dirname"

and later, rmdir -p "$dirname" (where dirname is passed as a parameter, but
is not user-input).






If you want to 'rmdir' to silently ignore non-empty directories,
there's a gnu extension option of 'rmdir --ignore-fail-on-non-empty':

$ strace -e rmdir rmdir --ignore-fail-on-non-empty -p a/../b
rmdir("a/../b") = 0
rmdir("a/..")   = -1 ENOTEMPTY (Directory 
not empty)

+++ exited with 0 +++

But note that this causes 'rmdir' to stop upon first failure, and 'a' is 
still not removed.



==> seems to be best wording & solution:

"mkdir -p", it seems should really be restated to:

   follow given path and make directories as possible"

then "rmdir -p" could be

   "follow given path and delete directories if empty"


This does not accurately reflect how 'rmdir' is currently implemented.
A more accurate description is "follow given path and delete 
directories, until the first encountered failure".


Right, but am trying to get rmdir to be a useful
"opposite" to a "mkdir" -- at least w/r/t "-p"...



If you want a behavior where 'rmdir' will continue beyond the first failure
and try to delete all directories in the given path (e.g. a/b/c in the 
example above),
then it sounds like a new feature request, and a non-standard behavior 
which will be a gnu extension (and also quite confusing behavior, IMHO).

---
Yeah -- new feature, but confusing?

I think the fact that you can mkdir -p XXX and later  canNOT
rmdir -p XXX, is quite confusing behavior...

But is -p's behavior in mkdir and rmdir mandated by POSIX?

bug#24730: rmdir/mkdir error(s) and/or not working "reciprocally" w/each other

[Reuti]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


Am 18.10.2016 um 22:33 schrieb Eric Blake:

> tag 24730 notabug
> thanks
> 
> On 10/18/2016 02:49 PM, L. A. Walsh wrote:
>> 
>> It doesn't seem rmdir and mkdir are behaving "reciprocally"...
>> 
>> If I type
>> 
>> mkdir -p ./a/b/c   # no error
> 
> . already exists, so mkdir silently does nothing,
> ./a needs to be created,
> ./a/b needs to be created,
> ./a/b/c needs to be created
> 
>> rmdir -p ./a/b/c   # get error msg, but a,b,c removed.
> 
> ./a/b/c needs to be removed,
> ./a/b needs to be removed,
> ./a needs to be removed,
> . needs to be removed, but you can't do that, hence the error
> 
> The apparent asymmetry is due to the POSIX rules on how '.' is treated;
> but we can't change the behavior of either of these commands, because
> they are both following the POSIX rules.
> 
> If you want symmetry, omit the leading './', as in:
> 
> $ mkdir -p a/b/c
> $ rmdir -p a/b/c
> 
>> 
>> 1) thinking either rmdir shouldn't generate an error or mkdir should
>> 
>> mkdir -p a/../b  # no error
> 
> a needs to be created,
> a/.. already exists, so it silently does nothing,
> a/../b needs to be created
> 
>> rmdir -p a/../b  # error, but a & b removed
> 
> a/../b needs to be removed,
> a/.. needs to be removed, but you can't do that,
> at this point, POSIX is fuzzy whether to attempt to remove 'a', or to
> give up since 'a/..' was already an error; but obviously coreutils
> removes 'a'

What version of core-utils shows this behavior. In the latest one it's not 
removed AFAICS.

- -- Reuti


> 
>> 
>> 2) similar comment to above -- leading to:
>> 
>> for rmdir, if "-p" is used, then as similar to "mkdir -p":
>>  (no error if existing, make parent directories as needed)
>> 
>> rmdir -p should be
>>  "no error if dir not empty, but directories are followed
>> and deleted as possible".
> 
> Sadly, while that might be a nicer definition for 'rmdir -p', it doesn't
> match the POSIX requirements nor the historical behavior, so we can't
> really change it now.
> 
> So I'm marking this as not a bug, as there is nothing to change.
> 
> -- 
> Eric Blake   eblake redhat com+1-919-301-3266
> Libvirt virtualization library http://libvirt.org
> 

-BEGIN PGP SIGNATURE-
Comment: GPGTools - https://gpgtools.org

iEYEARECAAYFAlgGizIACgkQo/GbGkBRnRrB9ACgpbqqSW0ZF27H5nSKF97zrDth
kLMAnRDKMmGxZ9p2SAqp/g+8AIkinqtU
=ykPn
-END PGP SIGNATURE-

bug#24730: rmdir/mkdir error(s) and/or not working "reciprocally" w/each other

[Assaf Gordon]

Hello,

Before deciding on the wording, it's worth nothing that the errors and reasons 
for the errors are different between mkdir and rmdir, and between the two cases.

On 10/18/2016 03:49 PM, L. A. Walsh wrote:


mkdir -p ./a/b/c   # no error
rmdir -p ./a/b/c   # get error msg, but a,b,c removed.


The error in this case (at least on Linux) is "Invalid Argument",
because 'rmdir .' is invalid and rejected by the kernel (EINVAL)
while 'mkdir .' returns EEXISTS - and '-p' specifically instruct it to silently 
ignore EEXIST.

Demonstrated another way:

'mkdir' without -p:

$ strace -e mkdir mkdir .
mkdir(".", 0777)= -1 EEXIST (File exists)
mkdir: cannot create directory ‘.’: File exists
+++ exited with 1 +++

'mkdir' with -p:

$ strace -e mkdir mkdir -p .
mkdir(".", 0777)= -1 EEXIST (File exists)
+++ exited with 0 +++

but 'rmdir' gives an unrecoverable error because of the kernel,
not because of coreutils' code:

$ strace -e rmdir rmdir .
rmdir(".")  = -1 EINVAL (Invalid argument)
rmdir: failed to remove '.': Invalid argument

This is also mandated by posix:
   http://pubs.opengroup.org/onlinepubs/9699919799/functions/rmdir.html
   "If the path argument refers to a path whose final component is either dot or 
dot-dot, rmdir() shall fail."


However, if there is no dot component in the path, they do behave similarly (or 
reciprocally, as you've said).


mkdir -p a/../b  # no error
rmdir -p a/../b  # error, but a & b removed


At least on my system (Linux kernel 3.13), 'a' is not removed in the above 
example,
and the error is due to non-empty directory:

$ strace -e mkdir mkdir -p a/../b
mkdir("a", 0777)= 0
mkdir("b", 0777)= 0
+++ exited with 0 +++

$ strace -e rmdir rmdir -p a/../b
rmdir("a/../b") = 0
rmdir("a/..")   = -1 ENOTEMPTY (Directory not empty)
rmdir: failed to remove directory 'a/..': Directory not empty
+++ exited with 1 +++


So two cases these are slightly different (EINVAL vs ENOTEMPTY).

Note that coreutils' mkdir contains an optimization not to try and make '..' as 
it is
guaranteed to exist (implemented in gnulib's mkancesdirs.c).

However, by definition, when 'rmdir' traverses the directories on the given 
path,
the directory 'a/..' is not empty (it contains 'a') - so this must fail.


If you want to 'rmdir' to silently ignore non-empty directories,
there's a gnu extension option of 'rmdir --ignore-fail-on-non-empty':

$ strace -e rmdir rmdir --ignore-fail-on-non-empty -p a/../b
rmdir("a/../b") = 0
rmdir("a/..")   = -1 ENOTEMPTY (Directory not empty)
+++ exited with 0 +++

But note that this causes 'rmdir' to stop upon first failure, and 'a' is still 
not removed.


==> seems to be best wording & solution:

"mkdir -p", it seems should really be restated to:

   follow given path and make directories as possible"

then "rmdir -p" could be

   "follow given path and delete directories if empty"


This does not accurately reflect how 'rmdir' is currently implemented.
A more accurate description is "follow given path and delete directories, until the 
first encountered failure".


$ mkdir -p a/../b/../c/../d

$ strace -e rmdir rmdir -p a/../b/../c/../d
rmdir("a/../b/../c/../d")   = 0
rmdir("a/../b/../c/..") = -1 ENOTEMPTY (Directory not empty)
rmdir: failed to remove directory 'a/../b/../c/..': Directory not empty
+++ exited with 1 +++

$ ls
a  b  c

If you want a behavior where 'rmdir' will continue beyond the first failure
and try to delete all directories in the given path (e.g. a/b/c in the example 
above),
then it sounds like a new feature request, and a non-standard behavior which 
will be a gnu extension (and also quite confusing behavior, IMHO).



regards,
 - assaf

bug#24730: rmdir/mkdir error(s) and/or not working "reciprocally" w/each other

[Eric Blake]

On 10/18/2016 03:50 PM, Reuti wrote:

>>>
>>> 1) thinking either rmdir shouldn't generate an error or mkdir should
>>>
>>> mkdir -p a/../b  # no error
> 
>> a needs to be created,
>> a/.. already exists, so it silently does nothing,
>> a/../b needs to be created
> 
>>> rmdir -p a/../b  # error, but a & b removed
> 
>> a/../b needs to be removed,
>> a/.. needs to be removed, but you can't do that,
>> at this point, POSIX is fuzzy whether to attempt to remove 'a', or to
>> give up since 'a/..' was already an error; but obviously coreutils
>> removes 'a'
> 
> What version of core-utils shows this behavior. In the latest one it's not 
> removed AFAICS.

Hmm, you're right. I was going off the (incorrect) comment in the text
above, rather than actually testing it; so it looks like coreutils gives
up on the first error, rather than trying to remove a.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

bug#24730: rmdir/mkdir error(s) and/or not working "reciprocally" w/each other

[Eric Blake]

tag 24730 notabug
thanks

On 10/18/2016 02:49 PM, L. A. Walsh wrote:
> 
> It doesn't seem rmdir and mkdir are behaving "reciprocally"...
> 
> If I type
> 
> mkdir -p ./a/b/c   # no error

. already exists, so mkdir silently does nothing,
./a needs to be created,
./a/b needs to be created,
./a/b/c needs to be created

> rmdir -p ./a/b/c   # get error msg, but a,b,c removed.

./a/b/c needs to be removed,
./a/b needs to be removed,
./a needs to be removed,
. needs to be removed, but you can't do that, hence the error

The apparent asymmetry is due to the POSIX rules on how '.' is treated;
but we can't change the behavior of either of these commands, because
they are both following the POSIX rules.

If you want symmetry, omit the leading './', as in:

$ mkdir -p a/b/c
$ rmdir -p a/b/c

> 
> 1) thinking either rmdir shouldn't generate an error or mkdir should
> 
> mkdir -p a/../b  # no error

a needs to be created,
a/.. already exists, so it silently does nothing,
a/../b needs to be created

> rmdir -p a/../b  # error, but a & b removed

a/../b needs to be removed,
a/.. needs to be removed, but you can't do that,
at this point, POSIX is fuzzy whether to attempt to remove 'a', or to
give up since 'a/..' was already an error; but obviously coreutils
removes 'a'

> 
> 2) similar comment to above -- leading to:
> 
> for rmdir, if "-p" is used, then as similar to "mkdir -p":
>   (no error if existing, make parent directories as needed)
> 
> rmdir -p should be
>   "no error if dir not empty, but directories are followed
> and deleted as possible".

Sadly, while that might be a nicer definition for 'rmdir -p', it doesn't
match the POSIX requirements nor the historical behavior, so we can't
really change it now.

So I'm marking this as not a bug, as there is nothing to change.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

bug#24730: rmdir/mkdir error(s) and/or not working "reciprocally" w/each other

[L. A. Walsh]

It doesn't seem rmdir and mkdir are behaving "reciprocally"...

If I type

mkdir -p ./a/b/c   # no error
rmdir -p ./a/b/c   # get error msg, but a,b,c removed.

1) thinking either rmdir shouldn't generate an error or mkdir should

mkdir -p a/../b  # no error
rmdir -p a/../b  # error, but a & b removed

2) similar comment to above -- leading to:

for rmdir, if "-p" is used, then as similar to "mkdir -p":
  (no error if existing, make parent directories as needed)

rmdir -p should be
  "no error if dir not empty, but directories are followed
and deleted as possible".

==> seems to be best wording & solution:

"mkdir -p", it seems should really be restated to:

   follow given path and make directories as possible"

then "rmdir -p" could be

   "follow given path and delete directories if empty"


Does that look reasonable?

linda w.