bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Assaf Gordon]

close 13516
stop

(triaging old bugS)

Hello,

On 03/02/13 10:33 PM, Paul Eggert wrote:

On 02/01/2013 05:40 PM, Global Odey wrote:
I'm afraid I'm still puzzled.  Still, the patch improves things, since it fixes
what appears to be a serious bug [...]

>

http://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=ca369c8d661a10d8a0ae3d712e7dfa91664921f1


With no further follow-ups in 5 years,
and Mac OS X 10.8 being out of date,
I'm closing this bug.

-assaf

bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Paul Eggert]

On 02/01/2013 05:40 PM, Global Odey wrote:
> Okay that certainly got me on the right track. I tried a lot of different 
> things without finding
> the exact cause of the error so I'll pass on a few things that stood out to 
> me and maybe that will
> help make more sense out of things on your end.

I'm afraid I'm still puzzled.  Still, the patch improves things, since it fixes
what appears to be a serious bug (recursive runaway) while introducing a 
relatively
minor problem (getcwd doesn't work in unreadable directories).  Overall, that's
a win, even if we can improve it further later, so I pushed the fix into gnulib
here:

http://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=ca369c8d661a10d8a0ae3d712e7dfa91664921f1

bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Global Odey]

On 1/30/13 7:19 PM, Paul Eggert wrote:

What happens if you put a breakpoint on rpl_getcwd instead?  Use "b rpl_getcwd".
That should get Gnulib's getcwd instead of libc's.
Okay that certainly got me on the right track. I tried a lot of 
different things without finding the exact cause of the error so I'll 
pass on a few things that stood out to me and maybe that will help make 
more sense out of things on your end.


Ultimately, I used a breakpoint at line 196 of test-getcwd.c (see below) 
quite a bit.


   192 if (dotdot_max <= cwd_len - initial_cwd_len)
   193{
   194  if (dotdot_max + DIR_NAME_SIZE < cwd_len - initial_cwd_len)
   195break;
-> 196  c = getcwd (buf, cwd_len + 1);
   197  if (!c)
   198{
   199  if (! (errno == ERANGE || errno == ENOENT

The main reason was that I very quickly got lost trying follow getcwd 
(in getcwd.c) so I tried to stop things right before getcwd was called 
and that's when a few interesting things showed up. To start with, the 
value of buf was always 1023 characters long despite the array having a 
size of 4125. I don't know if it's related but it stands out to me that 
PATH_MAX is set to 1024. The way the path was made to always be 1023 
characters was interesting, too. Rather than truncating the very end of 
the path, it would start by truncating the name of the first directory 
and then go on to the next one if the name of the first directory wasn't 
long enough. So what should have been 
"/reallylongdirectoryname/confdir3.../confdir3" would end up as 
"/reallylongdire/confdir3.../confdir3" and 
"/a/b/c/d/e/f/g/h/i/j/confdir3.../confdir3" became 
"//d/e/f/g/h/i/j/confdir3.../confdir3".


Looking at some of the other variables only brought more confusion. 
cwd_len (getcwd's second argument on line 196) turned out to have the 
same value as initial_cwd_len so I don't know what line 160 was doing. I 
was expecting cwd_len to be at least 1023 to match buf. Trying to figure 
out how the program even got to this point, I checked the value of 
dotdot_max (used on line 192) which should have been 3072 and instead it 
was 0. The values of dotdot_max and cwd_len both depend on 
DIR_NAME_SIZE. Is this being unset by another function? It's definitely 
defined above (line 116) and successfully used in setting the array size 
of buf (line 141).



On another note, I've actually gotten to the bottom of a few, ancillary 
things:


First, I was wrong on my assumptions of what Darwin is and is not. Even 
though eg. Google's autocomplete seems to think "Darwin kernel" is a 
legitimate phrase, according to Wikipedia, Darwin is the Unix-based OS 
that OS X (also an OS) is built upon. The kernel is called XNU and is a 
combination of the Mach and FreeBSD kernels which is why you'll often 
see it referenced as simply the Mach kernel. To me, this makes OS X look 
more like a distro where instead of being built on top of GNU/Linux with 
a Linux kernel, it's built on top of Darwin with an XNU kernel. My 
points on versioning between OS X and Darwin and the ridiculousness of 
that FAQ still stand, though.


Paul, earlier in this thread you asked why Apple hadn't yet implemented 
fdopendir. My best guess is that Apple will have it implemented the day 
The Open Group settles on a UNIX 1x specification. Apple claims (and is 
certified for) 100% conformance to UNIX 03 which is based on SUS 3 / 
POSIX.1-2001. Even though there is a SUS 4 / POSIX.1-2008, there doesn't 
seem to be any corresponding UNIX branding so, hopefully, as soon as The 
Open Group figures out how they want to certify compliance (and how much 
they want to charge), there will be an fdopendir in the following 
release of OS X. Also, if you don't already know about it, 
opensource.apple.com has a ton of information on mostly Darwin including 
all of the (open) source code for every version of OS X.


Assaf, you might give LLDB (the debugger for LLVM/Clang) a try. Apple's 
version of GDB is about 8 years old now. I tried the latest version of 
GDB (even with the latest GCC) and, suffice to say, it is utterly broken 
on OS X. Still, wanting something newer, I tried LLDB using this guide 
here  for reference and it seemed to work 
pretty well (but then again, I don't really know what I'm doing so ymmv).


Global Odey

bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Paul Eggert]

On 01/30/2013 03:59 PM, Global Odey wrote:
> On 1/30/13 1:25 AM, Paul Eggert wrote:
>> Can you use GDB to debug the situation?
> No. That is, I'm willing but apparently not able.
>> Try running something like this:
>>
>> gdb test-getcwd
>> b getcwd
>> r
>> fin
>> p errno

> GDB didn't seem to offer up much. It was able to find getcwd (after loading 
> shared libraries)
> and set the breakpoint but it would exit out (still with error code 7) before 
> reaching the break.

What happens if you put a breakpoint on rpl_getcwd instead?  Use "b rpl_getcwd".
That should get Gnulib's getcwd instead of libc's.

bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Global Odey]

On 1/30/13 1:25 AM, Paul Eggert wrote:

Can you use GDB to debug the situation?

No. That is, I'm willing but apparently not able.

Try running something like this:

gdb test-getcwd
b getcwd
r
fin
p errno
GDB didn't seem to offer up much. It was able to find getcwd (after 
loading shared libraries) and set the breakpoint but it would exit out 
(still with error code 7) before reaching the break. After looking 
through test-getcwd.c, though, I don't know how that's possible. I can 
say for sure that when GDB was setting the breakpoint, it was finding 
libc's getcwd, not GNUlib's (based on its address).


That sort of thing.
Hopefully, this qualifies: Because I was unable to get anything else out 
of GDB (that was literally my first time ever using the program), I 
resorted to directly editing test-getcwd.c a few times and determined 
that errno gets set to ENOENT before triggering the failure.


Global Odey

bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Paul Eggert]

On 01/29/2013 09:11 AM, Global Odey wrote:
> + test-getcwd
> + Exit 7

Evidently the test-getcwd program is encountering
the failure marked "fail = 7;" in that program.
Can you use GDB to debug the situation?
Try running something like this:

gdb test-getcwd
b getcwd
r
fin
p errno

That sort of thing.

bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Global Odey]

On 1/28/13 6:47 PM, Paul Eggert wrote:

OK, how about the attached patch instead?
After correcting a minor issue (*/tests/test-getcwd.c needed to be 
changed to */gnulib-tests/test-getcwd.c in the last patch), 
tests/rm/unread3.sh continues to pass while gnulib-tests/test-getcwd.sh 
still fails.


Thanks

Global Odey
++ initial_cwd_=/var/ac/bld/coreutils-8.20.104-f2551/gnulib-tests
++ fail=0
+++ testdir_prefix_
+++ printf gt
++ pfx_=gt
+++ mktempd_ /var/ac/bld/coreutils-8.20.104-f2551/gnulib-tests 
gt-test-getcwd.sh.
+++ case $# in
+++ destdir_=/var/ac/bld/coreutils-8.20.104-f2551/gnulib-tests
+++ template_=gt-test-getcwd.sh.
+++ MAX_TRIES_=4
+++ case $destdir_ in
+++ case $template_ in
 unset TMPDIR
+++ d='/tmp/-p.lC7r3rZe
gt-test-getcwd.sh.DvFF'
+++ fail=1
+++ case $d in
+++ fail=1
+++ test -d '/tmp/-p.lC7r3rZe
gt-test-getcwd.sh.DvFF'
+++ fail=1
 ls -dgo '/tmp/-p.lC7r3rZe
gt-test-getcwd.sh.DvFF'
 tr S -
+++ perms=
+++ case $perms in
+++ fail=1
+++ test 1 = 0
 echo gt-test-getcwd.sh.
 sed 's/XX*$//'
+++ base_template_=gt-test-getcwd.sh.
 echo gt-test-getcwd.sh.
 wc -c
+++ template_length_='  23'
 echo gt-test-getcwd.sh.
 wc -c
+++ nx_='  19'
 expr 23 - 19
+++ nx_=4
+++ err_=
+++ i_=1
+++ :
 rand_bytes_ 4
 n_=4
 chars_=abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789
 dev_rand_=/dev/urandom
 test -r /dev/urandom
 dd ibs=4 count=1 if=/dev/urandom
 LC_ALL=C
 tr -c abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789 
01234567abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789
 return
+++ X_=l0KN
+++ 
candidate_dir_=/var/ac/bld/coreutils-8.20.104-f2551/gnulib-tests/gt-test-getcwd.sh.l0KN
 mkdir -m 0700 
/var/ac/bld/coreutils-8.20.104-f2551/gnulib-tests/gt-test-getcwd.sh.l0KN
+++ err_=
+++ echo 
/var/ac/bld/coreutils-8.20.104-f2551/gnulib-tests/gt-test-getcwd.sh.l0KN
+++ return
++ 
test_dir_=/var/ac/bld/coreutils-8.20.104-f2551/gnulib-tests/gt-test-getcwd.sh.l0KN
++ cd /var/ac/bld/coreutils-8.20.104-f2551/gnulib-tests/gt-test-getcwd.sh.l0KN
++ gl_init_sh_nl_='
'
++ IFS='
'
++ for sig_ in 1 2 3 13 15
+++ expr 1 + 128
++ eval 'trap '\''Exit 129'\'' 1'
+++ trap 'Exit 129' 1
++ for sig_ in 1 2 3 13 15
+++ expr 2 + 128
++ eval 'trap '\''Exit 130'\'' 2'
+++ trap 'Exit 130' 2
++ for sig_ in 1 2 3 13 15
+++ expr 3 + 128
++ eval 'trap '\''Exit 131'\'' 3'
+++ trap 'Exit 131' 3
++ for sig_ in 1 2 3 13 15
+++ expr 13 + 128
++ eval 'trap '\''Exit 141'\'' 13'
+++ trap 'Exit 141' 13
++ for sig_ in 1 2 3 13 15
+++ expr 15 + 128
++ eval 'trap '\''Exit 143'\'' 15'
+++ trap 'Exit 143' 15
++ trap remove_tmp_ 0
+ path_prepend_ .
+ test 1 '!=' 0
+ path_dir_=.
+ case $path_dir_ in
+ abs_path_dir_=/var/ac/bld/coreutils-8.20.104-f2551/gnulib-tests/.
+ case $abs_path_dir_ in
+ 
PATH=/var/ac/bld/coreutils-8.20.104-f2551/gnulib-tests/.:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin
+ create_exe_shims_ /var/ac/bld/coreutils-8.20.104-f2551/gnulib-tests/.
+ case $EXEEXT in
+ return 0
+ shift
+ test 0 '!=' 0
+ export PATH
+ test-getcwd
+ Exit 7
+ set +e
+ exit 7
+ exit 7
+ remove_tmp_
+ __st=7
+ cleanup_
+ :
+ cd /var/ac/bld/coreutils-8.20.104-f2551/gnulib-tests
+ chmod -R u+rwx 
/var/ac/bld/coreutils-8.20.104-f2551/gnulib-tests/gt-test-getcwd.sh.l0KN
+ rm -rf 
/var/ac/bld/coreutils-8.20.104-f2551/gnulib-tests/gt-test-getcwd.sh.l0KN
+ exit 7

bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Paul Eggert]

On 01/28/2013 09:55 AM, Global Odey wrote:
> it causes test-getcwd.sh to fail now.

OK, how about the attached patch instead?
Unless you have developer tools such as autoconf, please
patch just lib/getcwd.c and gnulib-tests/test-getcwd.c;
don't apply the patch to getcwd-abort-bug.m4.
diff --git a/ChangeLog b/ChangeLog
index bb7a142..a8eb014 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,19 @@
+2013-01-28  Paul Eggert  
+
+   getcwd: break a recursive loop between fdopendir and save_cwd
+   Reported for Mac OS X 10.6.8 by Assaf Gordon in
+   .
+   * lib/getcwd.c (HAVE_OPENAT_SUPPORT): Do not define if
+   !HAVE_OPENAT && !HAVE_FDOPENDIR.
+   * m4/getcwd-abort-bug.m4: Reformat to match test-getcwd.c
+   so that they can be kept in sync more easily.  Avoid PATH_MAX
+   test on the Hurd.  Sync from test-getcwd.c for errno tests after
+   mkdir or chdir failure.
+   * tests/test-getcwd.c (HAVE_OPENAT_SUPPORT): New macro, from
+   lib/getcwd.c.
+   (test_abort_bug): Do not test for the deep directory bug unless we
+   have openat support.  Avoid PATH_MAX test on the Hurd.
+
 2013-01-27  Paul Eggert  
 
futimens-tests, utimens-tests: Depend on gettext.
diff --git a/lib/getcwd.c b/lib/getcwd.c
index a4cbe01..4b78138 100644
--- a/lib/getcwd.c
+++ b/lib/getcwd.c
@@ -28,9 +28,9 @@
 #include  /* For AT_FDCWD on Solaris 9.  */
 
 /* If this host provides the openat function or if we're using the
-   gnulib replacement function, then enable code below to make getcwd
-   more efficient and robust.  */
-#if defined HAVE_OPENAT || defined GNULIB_OPENAT
+   gnulib replacement function with a native fdopendir, then enable
+   code below to make getcwd more efficient and robust.  */
+#if defined HAVE_OPENAT || (defined GNULIB_OPENAT && defined HAVE_FDOPENDIR)
 # define HAVE_OPENAT_SUPPORT 1
 #else
 # define HAVE_OPENAT_SUPPORT 0
diff --git a/m4/getcwd-abort-bug.m4 b/m4/getcwd-abort-bug.m4
index a6ee6cd..9b3b563 100644
--- a/m4/getcwd-abort-bug.m4
+++ b/m4/getcwd-abort-bug.m4
@@ -58,16 +58,18 @@ AC_DEFUN([gl_FUNC_GETCWD_ABORT_BUG],
 int
 main ()
 {
-  char const *dir_name = "confdir-14B---";
   char *cwd;
   size_t initial_cwd_len;
   int fail = 0;
-  size_t desired_depth;
-  size_t d;
 
   /* The bug is triggered when PATH_MAX < getpagesize (), so skip
  this relatively expensive and invasive test if that's not true.  */
-  if (getpagesize () <= PATH_MAX)
+#ifdef PATH_MAX
+  int bug_possible = PATH_MAX < getpagesize ();
+#else
+  int bug_possible = 0;
+#endif
+  if (! bug_possible)
 return 0;
 
   cwd = getcwd (NULL, 0);
@@ -76,35 +78,43 @@ main ()
 
   initial_cwd_len = strlen (cwd);
   free (cwd);
-  desired_depth = ((TARGET_LEN - 1 - initial_cwd_len)
-   / (1 + strlen (dir_name)));
-  for (d = 0; d < desired_depth; d++)
+
+  if (1)
 {
-  if (mkdir (dir_name, S_IRWXU) < 0 || chdir (dir_name) < 0)
+  static char const dir_name[] = "confdir-14B---";
+  size_t desired_depth = ((TARGET_LEN - 1 - initial_cwd_len)
+  / sizeof dir_name);
+  size_t d;
+  for (d = 0; d < desired_depth; d++)
 {
-  fail = 3; /* Unable to construct deep hierarchy.  */
-  break;
+  if (mkdir (dir_name, S_IRWXU) < 0 || chdir (dir_name) < 0)
+{
+  if (! (errno == ERANGE || errno == ENAMETOOLONG
+ || errno == ENOENT))
+fail = 3; /* Unable to construct deep hierarchy.  */
+  break;
+}
 }
-}
 
-  /* If libc has the bug in question, this invocation of getcwd
- results in a failed assertion.  */
-  cwd = getcwd (NULL, 0);
-  if (cwd == NULL)
-fail = 4; /* getcwd failed: it refuses to return a string longer
- than PATH_MAX.  */
-  free (cwd);
+  /* If libc has the bug in question, this invocation of getcwd
+ results in a failed assertion.  */
+  cwd = getcwd (NULL, 0);
+  if (cwd == NULL)
+fail = 4; /* getcwd didn't assert, but it failed for a long name
+ where the answer could have been learned.  */
+  free (cwd);
 
-  /* Call rmdir first, in case the above chdir failed.  */
-  rmdir (dir_name);
-  while (0 < d--)
-{
-  if (chdir ("..") < 0)
+  /* Call rmdir first, in case the above chdir failed.  */
+  rmdir (dir_name);
+  while (0 < d--)
 {
-  fail = 5;
-  break;
+  if (chdir ("..") < 0)
+{
+  fail = 5;
+  break;
+}
+  rmdir (dir_name);
 }
-  rmdir (dir_name);
 }
 
   return fail;
diff --git a/tests/test-getcwd.c b/tests/test-getcwd.c
index 4b951e0..810b476 100644
--- a/tests/test-getcwd.c
+++ b/tests/test-getcwd.c
@@ -38,23 +38,29 @@
trigger a bug in glibc's getcwd implementation before 2.4.90-10.  */
 #define TARGET_LEN (5 * 1024)
 
+#if defined HAVE_OPENAT || (define

bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Assaf Gordon]

Global Odey wrote, On 01/28/2013 12:55 PM:
> On 1/27/13 8:34 PM, Paul Eggert wrote:
>> Thanks, can you please try this patch instead? It's a bit more drastic, but 
>> I hope it fixes the loop without introducing that other bug.
> 
> Thank you. Your patch does the trick on OS X 10.8.2, however, it causes 
> test-getcwd.sh to fail now.
> 

Same for me, on Mac OS 10.6.8 (Darwin 10.8):
tests/rm/unread3 succeeds, but gnulib-tests/test-getcwd.sh fails with exit code 
47 .

bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Global Odey]

On 1/27/13 8:34 PM, Paul Eggert wrote:
Thanks, can you please try this patch instead? It's a bit more 
drastic, but I hope it fixes the loop without introducing that other bug.


Thank you. Your patch does the trick on OS X 10.8.2, however, it causes 
test-getcwd.sh to fail now.


Global Odey
FAIL: test-getcwd.sh


++ initial_cwd_=/var/ac/bld/coreutils-8.20.103-bb116/gnulib-tests
++ fail=0
+++ testdir_prefix_
+++ printf gt
++ pfx_=gt
+++ mktempd_ /var/ac/bld/coreutils-8.20.103-bb116/gnulib-tests 
gt-test-getcwd.sh.
+++ case $# in
+++ destdir_=/var/ac/bld/coreutils-8.20.103-bb116/gnulib-tests
+++ template_=gt-test-getcwd.sh.
+++ MAX_TRIES_=4
+++ case $destdir_ in
+++ case $template_ in
 unset TMPDIR
+++ d='/tmp/-p.sDffDhsB
gt-test-getcwd.sh.DI26'
+++ fail=1
+++ case $d in
+++ fail=1
+++ test -d '/tmp/-p.sDffDhsB
gt-test-getcwd.sh.DI26'
+++ fail=1
 ls -dgo '/tmp/-p.sDffDhsB
gt-test-getcwd.sh.DI26'
 tr S -
+++ perms=
+++ case $perms in
+++ fail=1
+++ test 1 = 0
 echo gt-test-getcwd.sh.
 sed 's/XX*$//'
+++ base_template_=gt-test-getcwd.sh.
 echo gt-test-getcwd.sh.
 wc -c
+++ template_length_='  23'
 echo gt-test-getcwd.sh.
 wc -c
+++ nx_='  19'
 expr 23 - 19
+++ nx_=4
+++ err_=
+++ i_=1
+++ :
 rand_bytes_ 4
 n_=4
 chars_=abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789
 dev_rand_=/dev/urandom
 test -r /dev/urandom
 dd ibs=4 count=1 if=/dev/urandom
 LC_ALL=C
 tr -c abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789 
01234567abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789
 return
+++ X_=Bv7A
+++ 
candidate_dir_=/var/ac/bld/coreutils-8.20.103-bb116/gnulib-tests/gt-test-getcwd.sh.Bv7A
 mkdir -m 0700 
/var/ac/bld/coreutils-8.20.103-bb116/gnulib-tests/gt-test-getcwd.sh.Bv7A
+++ err_=
+++ echo 
/var/ac/bld/coreutils-8.20.103-bb116/gnulib-tests/gt-test-getcwd.sh.Bv7A
+++ return
++ 
test_dir_=/var/ac/bld/coreutils-8.20.103-bb116/gnulib-tests/gt-test-getcwd.sh.Bv7A
++ cd /var/ac/bld/coreutils-8.20.103-bb116/gnulib-tests/gt-test-getcwd.sh.Bv7A
++ gl_init_sh_nl_='
'
++ IFS='
'
++ for sig_ in 1 2 3 13 15
+++ expr 1 + 128
++ eval 'trap '\''Exit 129'\'' 1'
+++ trap 'Exit 129' 1
++ for sig_ in 1 2 3 13 15
+++ expr 2 + 128
++ eval 'trap '\''Exit 130'\'' 2'
+++ trap 'Exit 130' 2
++ for sig_ in 1 2 3 13 15
+++ expr 3 + 128
++ eval 'trap '\''Exit 131'\'' 3'
+++ trap 'Exit 131' 3
++ for sig_ in 1 2 3 13 15
+++ expr 13 + 128
++ eval 'trap '\''Exit 141'\'' 13'
+++ trap 'Exit 141' 13
++ for sig_ in 1 2 3 13 15
+++ expr 15 + 128
++ eval 'trap '\''Exit 143'\'' 15'
+++ trap 'Exit 143' 15
++ trap remove_tmp_ 0
+ path_prepend_ .
+ test 1 '!=' 0
+ path_dir_=.
+ case $path_dir_ in
+ abs_path_dir_=/var/ac/bld/coreutils-8.20.103-bb116/gnulib-tests/.
+ case $abs_path_dir_ in
+ 
PATH=/var/ac/bld/coreutils-8.20.103-bb116/gnulib-tests/.:/usr/local/llvm-3.2/bin:/usr/local/bin:/usr/local/stage1/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin
+ create_exe_shims_ /var/ac/bld/coreutils-8.20.103-bb116/gnulib-tests/.
+ case $EXEEXT in
+ return 0
+ shift
+ test 0 '!=' 0
+ export PATH
+ test-getcwd
+ Exit 47
+ set +e
+ exit 47
+ exit 47
+ remove_tmp_
+ __st=47
+ cleanup_
+ :
+ cd /var/ac/bld/coreutils-8.20.103-bb116/gnulib-tests
+ chmod -R u+rwx 
/var/ac/bld/coreutils-8.20.103-bb116/gnulib-tests/gt-test-getcwd.sh.Bv7A
+ rm -rf 
/var/ac/bld/coreutils-8.20.103-bb116/gnulib-tests/gt-test-getcwd.sh.Bv7A
+ exit 47


Testsuite summary for GNU coreutils 8.20.103-bb116

# TOTAL: 1
# PASS:  0
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

See gnulib-tests/test-suite.log
Please report to bug-coreutils@gnu.org

bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Paul Eggert]

On 01/24/2013 12:35 PM, Assaf Gordon wrote:

> With this patch, the recursive loop is gone, but it leads to another bug: 
> descending into the directory (due to "-r") doesn't work.

Thanks, can you please try this patch instead?
It's a bit more drastic, but I hope it fixes the loop
without introducing that other bug.
This is a gnulib patch, so it fixes a 'ChangeLog'
file that you don't have -- please don't worry about that part.

---
 ChangeLog| 6 ++
 lib/getcwd.c | 6 +++---
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index bb7a142..6db18b4 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,11 @@
 2013-01-27  Paul Eggert  
 
+   getcwd: break a recursive loop between fdopendir and save_cwd
+   Reported for Mac OS X 10.6.8 by Assaf Gordon in
+   .
+   * lib/getcwd.c (HAVE_OPENAT_SUPPORT): Do not define if
+   !HAVE_OPENAT && !HAVE_FDOPENDIR.
+
futimens-tests, utimens-tests: Depend on gettext.
This works around a problem introduced in my 2013-01-12 patch,
which added @LIBINTL@ to these modules.
diff --git a/lib/getcwd.c b/lib/getcwd.c
index a4cbe01..4b78138 100644
--- a/lib/getcwd.c
+++ b/lib/getcwd.c
@@ -28,9 +28,9 @@
 #include  /* For AT_FDCWD on Solaris 9.  */
 
 /* If this host provides the openat function or if we're using the
-   gnulib replacement function, then enable code below to make getcwd
-   more efficient and robust.  */
-#if defined HAVE_OPENAT || defined GNULIB_OPENAT
+   gnulib replacement function with a native fdopendir, then enable
+   code below to make getcwd more efficient and robust.  */
+#if defined HAVE_OPENAT || (defined GNULIB_OPENAT && defined HAVE_FDOPENDIR)
 # define HAVE_OPENAT_SUPPORT 1
 #else
 # define HAVE_OPENAT_SUPPORT 0
-- 
1.7.11.7

bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Assaf Gordon]

Hello,

Paul Eggert wrote, On 01/24/2013 12:03 PM:
> Thanks, could you please try the following patch?
> This is a gnulib patch, so it also fixes modules/fdopendir --
> you needn't worry about that part of the patch,
> as it doesn't apply to coreutils.
> I'm CC'ing this possible patch to bug-gnulib to give
> the Gnulib people a heads-up.

With this patch, the recursive loop is gone, but it leads to another bug: 
descending into the directory (due to "-r") doesn't work.

Example:
==
##
## The 'setup'
##
$ echo $D
/Users/admin/Projects/coreutils-8.20.90-e27bfd/src
$ pwd
/Users/admin/Projects/coreutils-8.20.90-e27bfd/src/c
$ ls -ld .
d-wx--xr-x  2 admin  staff  68 Jan 24 14:27 .
$ ls -lh
ls: .: Permission denied

## target directory is empty - works OK
$ mkdir $D/a
$ $D/rm -r $D/a

## target directory is not empty (and current directory unreadable) - fails
$ mkdir -p $D/a/1
$ $D/rm -r $D/a
/Users/admin/Projects/coreutils-8.20.90-e27bfd/src/rm: cannot remove 
'/Users/admin/Projects/coreutils-8.20.90-e27bfd/src/a': Permission denied

## Trying again from a readable directory - works.
$ cd $D
$ $D/rm -r $D/a
==

The immediate error happens in "unlinkat()", but I'm guessing the deeper cause 
is that somehow "a" is being deleted before "a/1" ?
===
$ pwd
/Users/admin/Projects/coreutils-8.20.90-e27bfd/src/c
$ ls -lh
ls: .: Permission denied
$ ls -ld $D/a
drwxr-xr-x  3 admin  staff  102 Jan 24 15:15 
/Users/admin/Projects/coreutils-8.20.90-e27bfd/src/a
$ ls -ld $D/a/1
drwxr-xr-x  2 admin  staff  68 Jan 24 15:15 
/Users/admin/Projects/coreutils-8.20.90-e27bfd/src/a/1
$ gdb /Users/admin/Projects/coreutils-8.20.90-e27bfd/src/rm
(gdb) b unlinkat
(gdb) r -r /Users/admin/Projects/coreutils-8.20.90-e27bfd/src/a

##
## Just before the call to "unlinkat"
##
Breakpoint 1, excise (fts=0x100100080, ent=0x100100210, x=0x7fff5fbff9f0, 
is_dir=true) at remove.c:369
369   if (unlinkat (fts->fts_cwd_fd, ent->fts_accpath, flag) == 0)
(gdb) bt
#0  excise (fts=0x100100080, ent=0x100100210, x=0x7fff5fbff9f0, is_dir=true) at 
remove.c:369
#1  0x00013244 in rm_fts (fts=0x100100080, ent=0x100100210, 
x=0x7fff5fbff9f0) at remove.c:509
#2  0x00013487 in rm (file=0x7fff5fbffa70, x=0x7fff5fbff9f0) at 
remove.c:568
#3  0x00011b10 in main (argc=3, argv=0x7fff5fbffa60) at rm.c:349
(gdb) p *fts
$1 = {
  fts_cur = 0x100100210, 
  fts_child = 0x0, 
  fts_array = 0x0, 
  fts_dev = 234881026, 
  fts_path = 0x10080 
"/Users/admin/Projects/coreutils-8.20.90-e27bfd/src/a", 
  fts_rfd = 0, 
  fts_cwd_fd = -3041965, 
  fts_pathlen = 1280, 
  fts_nitems = 0, 
  fts_compar = 0, 
  fts_options = 28, 
  fts_leaf_optimization_works_ht = 0x0, 
  fts_cycle = {
ht = 0x100100460, 
state = 0x100100460
  }, 
  fts_fd_ring = {
ir_data = {-1, -1, -1, -1}, 
ir_default_val = -1, 
ir_front = 0, 
ir_back = 0, 
ir_empty = true
  }
}
(gdb) p *ent
$2 = {
  fts_cycle = 0x0, 
  fts_parent = 0x100100100, 
  fts_link = 0x0, 
  fts_dirp = 0x0, 
  fts_number = 0, 
  fts_pointer = 0x0, 
  fts_accpath = 0x10080 
"/Users/admin/Projects/coreutils-8.20.90-e27bfd/src/a", 
  fts_path = 0x10080 
"/Users/admin/Projects/coreutils-8.20.90-e27bfd/src/a", 
  fts_errno = 13, 
  fts_symfd = 0, 
  fts_pathlen = 52, 
  fts_fts = 0x100100080, 
  fts_level = 0, 
  fts_namelen = 1, 
  fts_n_dirs_remaining = 1, 
  fts_info = 4, 
  fts_flags = 0, 
  fts_instr = 3, 
  fts_statp = {{
  st_dev = 234881026, 
  st_mode = 16877, 
  st_nlink = 3, 
  st_ino = 4157629, 
  st_uid = 502, 
  st_gid = 20, 
  st_rdev = 0, 
  st_atimespec = {
tv_sec = 1359058544, 
tv_nsec = 0
  }, 
  st_mtimespec = {
tv_sec = 1359058544, 
tv_nsec = 0
  }, 
  st_ctimespec = {
tv_sec = 1359058544, 
tv_nsec = 0
  }, 
  st_birthtimespec = {
tv_sec = 1359058544, 
tv_nsec = 0
  }, 
  st_size = 102, 
  st_blocks = 0, 
  st_blksize = 4096, 
  st_flags = 0, 
  st_gen = 0, 
  st_lspare = 0, 
  st_qspare = {0, 0}
}}, 
  fts_name = "a"
}
(gdb) p flag
$3 = 1

##
## Inside "unlinkat()", (assuming "CALL_FUNC" calls MacOS's "unlink(2)" ?) - 
this fails.
##
(gdb) n
Breakpoint 2, unlinkat (fd=-3041965, file=0x10080 
"/Users/admin/Projects/coreutils-8.20.90-e27bfd/src/a", flag=1) at at-func.c:72
72VALIDATE_FLAG (flag);
(gdb) n
74if (fd == AT_FDCWD || IS_ABSOLUTE_FILE_NAME (file))
(gdb) n
75  return CALL_FUNC (file);
(gdb) n
72VALIDATE_FLAG (flag);
(gdb) n

##
## The rest is error handling in remove.c/rm.c
##
excise (fts=0x100100080, ent=0x100100210, x=0x7fff5fbff9f0, is_dir=true) at 
remove.c:384
384   if (errno == EROFS)
(gdb) n
392   if (ignorable_missing (x, errno))
(gdb) n
400   if (ent->fts_info == FTS_DNR
(gdb) n
404 errno = ent->fts_errno;
(gdb) n
405   error (0, errno, _("cannot remove %s"), quote (ent->fts_path));
(gdb) n
/Users/admin/Projects/coreuti

bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Paul Eggert]

Thanks, could you please try the following patch?
This is a gnulib patch, so it also fixes modules/fdopendir --
you needn't worry about that part of the patch,
as it doesn't apply to coreutils.
I'm CC'ing this possible patch to bug-gnulib to give
the Gnulib people a heads-up.

By the way, do you happen to know why OS X doesn't
yet support fdopendir?  It's been in the standard for ages.
Do the OSX maintainers plan to implement it eventually?



Break a recursive loop between fdopendir and save_cwd.
Reported for OS X 10.8 by Assaf Gordon in .
* lib/fdopendir.c [!HAVE_FDOPENDIR]: Don't include save-cwd.h.
(fdopendir): Don't try to invoke getcwd via save_cwd,
since it invokes us.  Instead, merely open the working directory
and return a failure indication if that fails.
(fdopendir_with_dup): Second arg is now a fd (or -1), not a
struct saved_cwd * (or 0).  All uses changed.
* modules/fdopendir (Depends-on): Remove save-cwd.
diff --git a/lib/fdopendir.c b/lib/fdopendir.c
index 63e06b9..25e984b 100644
--- a/lib/fdopendir.c
+++ b/lib/fdopendir.c
@@ -27,7 +27,6 @@

 # include "openat.h"
 # include "openat-priv.h"
-# include "save-cwd.h"

 # if GNULIB_DIRENT_SAFER
 #  include "dirent--.h"
@@ -37,15 +36,15 @@
 #  define REPLACE_FCHDIR 0
 # endif

-static DIR *fdopendir_with_dup (int, int, struct saved_cwd const *);
-static DIR *fd_clone_opendir (int, struct saved_cwd const *);
+static DIR *fdopendir_with_dup (int, int, int);
+static DIR *fd_clone_opendir (int, int);

 /* Replacement for POSIX fdopendir.

First, try to simulate it via opendir ("/proc/self/fd/...").  Failing
that, simulate it by using fchdir metadata, or by doing
-   save_cwd/fchdir/opendir(".")/restore_cwd.
-   If either the save_cwd or the restore_cwd fails (relatively unlikely),
+   open(".")/fchdir/opendir(".")/fchdir.
+   If the final fchdir fails (relatively unlikely),
then give a diagnostic and exit nonzero.

If successful, the resulting stream is based on FD in
@@ -65,19 +64,19 @@ static DIR *fd_clone_opendir (int, struct saved_cwd const 
*);
 DIR *
 fdopendir (int fd)
 {
-  DIR *dir = fdopendir_with_dup (fd, -1, NULL);
+  DIR *dir = fdopendir_with_dup (fd, -1, -1);

   if (! REPLACE_FCHDIR && ! dir)
 {
   int saved_errno = errno;
   if (EXPECTED_ERRNO (saved_errno))
 {
-  struct saved_cwd cwd;
-  if (save_cwd (&cwd) != 0)
-openat_save_fail (errno);
-  dir = fdopendir_with_dup (fd, -1, &cwd);
+  int cwd = open (".", O_SEARCH);
+  if (cwd < 0)
+return 0;
+  dir = fdopendir_with_dup (fd, -1, cwd);
   saved_errno = errno;
-  free_cwd (&cwd);
+  close (cwd);
   errno = saved_errno;
 }
 }
@@ -93,11 +92,11 @@ fdopendir (int fd)
That way, barring race conditions, fd_clone_opendir returns a
stream whose file descriptor is FD.

-   If REPLACE_CHDIR or CWD is null, use opendir ("/proc/self/fd/...",
+   If REPLACE_CHDIR or CWD < 0, use opendir ("/proc/self/fd/...",
falling back on fchdir metadata.  Otherwise, CWD is a saved version
-   of the working directory; use fchdir/opendir(".")/restore_cwd(CWD).  */
+   of the working directory; use fchdir/opendir(".")/fchdir(CWD).  */
 static DIR *
-fdopendir_with_dup (int fd, int older_dupfd, struct saved_cwd const *cwd)
+fdopendir_with_dup (int fd, int older_dupfd, int cwd)
 {
   int dupfd = dup (fd);
   if (dupfd < 0 && errno == EMFILE)
@@ -137,9 +136,9 @@ fdopendir_with_dup (int fd, int older_dupfd, struct 
saved_cwd const *cwd)
the caller's responsibility both to close FD and (if the result is
not null) to closedir the result.  */
 static DIR *
-fd_clone_opendir (int fd, struct saved_cwd const *cwd)
+fd_clone_opendir (int fd, int cwd)
 {
-  if (REPLACE_FCHDIR || ! cwd)
+  if (REPLACE_FCHDIR || cwd < 0)
 {
   DIR *dir = NULL;
   int saved_errno = EOPNOTSUPP;
@@ -170,7 +169,7 @@ fd_clone_opendir (int fd, struct saved_cwd const *cwd)
 {
   DIR *dir = opendir (".");
   int saved_errno = errno;
-  if (restore_cwd (cwd) != 0)
+  if (fchdir (cwd) != 0)
 openat_restore_fail (errno);
   errno = saved_errno;
   return dir;
diff --git a/modules/fdopendir b/modules/fdopendir
index ee9a875..3d99472 100644
--- a/modules/fdopendir
+++ b/modules/fdopendir
@@ -15,7 +15,6 @@ fchdir  [test $HAVE_FDOPENDIR = 0]
 fstat   [test $HAVE_FDOPENDIR = 0]
 openat-die  [test $HAVE_FDOPENDIR = 0]
 opendir [test $HAVE_FDOPENDIR = 0]
-save-cwd[test $HAVE_FDOPENDIR = 0]

 configure.ac:
 gl_FUNC_FDOPENDIR

bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Assaf Gordon]

Paul Eggert wrote, On 01/23/2013 02:19 AM:
> It looks like there's no way to get the equivalent of strace, then.
> That's too bad.  I guess you'll have to use GDB, then.

Here's what I get with GDB.

Seems like some weird case of endless recursion, that leads to file-descriptor 
exhaustion (if I understand the situation correctly).
(note: "cwd" is unreadable to the current user).

The beginning is normal:
#1  main() src/rm.c:349
#2  rm(file=0x7fff5fbffa00, x=0x7fff5fbff998) at src/remove.c:568
#3  rm_fts (fts=0x100100080, ent=0x100100210, x=0x7fff5fbff998) at 
src/remove.c:462
#4  prompt (fts=0x100100080, ent=0x100100210, [...]) at src/remove.c:196
#5  is_empty_dir (fd_cwd=-3041965, dir=0x10080 
"/Users/admin/Projects/coreutils-8.20.90-e27bfd/src/a") at system.h:301
#6  fdopendir (fd=5) at fdopendir.c:76

Then it loops in:
#7  fdopendir (fd=6) at fdopendir.c:76
#8  save_cwd (cwd=0x7fff5fbff6a8) at save-cwd.c:72
#9  rpl_getcwd (buf=0x0, size=0) at getcwd.c:251

(rpl_getcwd() calls fdopendir() if HAVE_OPENAT_SUPPORT in getcwd.c:80)


Until too many file descriptors are open, then:
#10  fdopendir (fd=254) at fdopendir.c:77
#11  openat_save_fail (errnum=24) at openat-die.c:37
#12  error (status=1, errnum=24, message=0x10001d4b1 "unable to record current 
working directory") at error.c:318
#13  error_tail (status=1, errnum=24, message=0x10001d4b1 "unable to record 
current working directory", args=0x7fff5fbdf4b0) at error.c:281
#14  exit ()


This is the complete stack-trace:

$ pwd
/Users/admin/Projects/coreutils-8.20.90-e27bfd/src/c
$ ls -lh
ls: .: Permission denied
$ gdb ../rm
GNU gdb 6.3.50-20050815 (Apple version gdb-1822) (Sun Aug  5 03:00:42 UTC 2012)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-apple-darwin"...Reading symbols for shared 
libraries .. done

(gdb) b exit
Breakpoint 1 at 0x20c49ba5c85283
(gdb) r -r /Users/admin/Projects/coreutils-8.20.90-e27bfd/src/a 
/Users/admin/Projects/coreutils-8.20.90-e27bfd/src/b
Starting program: /Users/admin/Projects/coreutils-8.20.90-e27bfd/src/rm -r 
/Users/admin/Projects/coreutils-8.20.90-e27bfd/src/a 
/Users/admin/Projects/coreutils-8.20.90-e27bfd/src/b
Reading symbols for shared libraries +. done
/Users/admin/Projects/coreutils-8.20.90-e27bfd/src/rm: unable to record current 
working directory: Too many open files

Breakpoint 1, 0x7fff805c2283 in exit ()
(gdb) bt
#0  0x7fff805c2283 in exit ()
#1  0x00019b5f in error_tail (status=1, errnum=24, message=0x10001d4b1 
"unable to record current working directory", args=0x7fff5fbdf2a0) at 
error.c:281
#2  0x00019a41 in error (status=1, errnum=24, message=0x10001d4b1 
"unable to record current working directory") at error.c:318
#3  0x00015ea5 in openat_save_fail (errnum=24) at openat-die.c:37
#4  0x0001a5cc in fdopendir (fd=254) at fdopendir.c:77
#5  0x0001f3a9 in rpl_getcwd (buf=0x0, size=0) at getcwd.c:251
#6  0x00018419 in save_cwd (cwd=0x7fff5fbdf518) at save-cwd.c:72
#7  0x0001a5b5 in fdopendir (fd=253) at fdopendir.c:76
#8  0x0001f3a9 in rpl_getcwd (buf=0x0, size=0) at getcwd.c:251
#9  0x00018419 in save_cwd (cwd=0x7fff5fbdf728) at save-cwd.c:72
#10 0x0001a5b5 in fdopendir (fd=252) at fdopendir.c:76
#11 0x0001f3a9 in rpl_getcwd (buf=0x0, size=0) at getcwd.c:251
#12 0x00018419 in save_cwd (cwd=0x7fff5fbdf938) at save-cwd.c:72
#13 0x0001a5b5 in fdopendir (fd=251) at fdopendir.c:76
#14 0x0001f3a9 in rpl_getcwd (buf=0x0, size=0) at getcwd.c:251
#15 0x00018419 in save_cwd (cwd=0x7fff5fbdfb48) at save-cwd.c:72
#16 0x0001a5b5 in fdopendir (fd=250) at fdopendir.c:76
#17 0x0001f3a9 in rpl_getcwd (buf=0x0, size=0) at getcwd.c:251
#18 0x00018419 in save_cwd (cwd=0x7fff5fbdfd58) at save-cwd.c:72
#19 0x0001a5b5 in fdopendir (fd=249) at fdopendir.c:76
#20 0x0001f3a9 in rpl_getcwd (buf=0x0, size=0) at getcwd.c:251
#21 0x00018419 in save_cwd (cwd=0x7fff5fbdff68) at save-cwd.c:72
#22 0x0001a5b5 in fdopendir (fd=248) at fdopendir.c:76
#23 0x0001f3a9 in rpl_getcwd (buf=0x0, size=0) at getcwd.c:251
#24 0x00018419 in save_cwd (cwd=0x7fff5fbe0178) at save-cwd.c:72
#25 0x0001a5b5 in fdopendir (fd=247) at fdopendir.c:76
#26 0x0001f3a9 in rpl_getcwd (buf=0x0, size=0) at getcwd.c:251
#27 0x00018419 in save_cwd (cwd=0x7fff5fbe0388) at save-cwd.c:72
#28 0x0001a5b5 in fdopendir (fd=246) at fdopendir.c:76
#29 0x0001f3a9 in rpl_getcwd (buf=0x0, size=0) at getcwd.c:251
#30 0x00018419 in save_cwd (cwd=0x7fff5fbe0598) at save-cwd.c:72
#31 0x0001a5b5 in fdopendir (fd=245) at fdopendir.c

bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Paul Eggert]

It looks like there's no way to get the equivalent of strace, then.
That's too bad.  I guess you'll have to use GDB, then.

bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Assaf Gordon]

Paul Eggert wrote, On 01/22/2013 03:21 PM:
> On 01/22/13 11:22, Assaf Gordon wrote:
>> 3. Running with 'sudo', 'rm' doesn't fail:
>> $ sudo dtruss $D/rm -r $D/a $D/b 2>/tmp/rm_trace.txt
> 
> You need a sudo inside dtruss, to, right?
> Something like this
> 
>sudo dtruss sudo -u $USER $D/rm -r $D/a $D/b 2>/tmp/rm_trace.txt
> 

Sorry, but I can't get sudo+dtruss+sudo to work as expected.
It has some side effects that I do not understand, most likely because dtruss 
isn't following forks.

Simple example of the problem:

1. no sudo/dtruss - works:
$ $D/rm foo 2>&1 | grep foo
/Users/admin/Projects/coreutils-8.20.90-e27bfd/src/rm: cannot remove 'foo': 
No such file or directory

2. sudo+dtruss - work (but not good enough to reproduce this bug):
$ sudo dtruss $D/rm foo 2>&1 | grep foo
/Users/admin/Projects/coreutils-8.20.90-e27bfd/src/rm: cannot remove 'foo': 
No such file or directory
lstat64("foo\0", 0x100100280, 0x100100280)   = -1 Err#2
unlink("foo\0", 0x10080, 0x10080)= -1 Err#2
write_nocancel(0x2, "cannot remove 'foo'\0", 0x13)   = 19 0

3. sudo+dtruss+sudo - doesn't trace "rm", sometimes shows the error message, 
sometimes doesn't. Using "-f" has no effect:

$ sudo dtruss sudo -u $USER $D/rm foo 2>&1 | grep foo
/Users/admin/Projects/coreutils-8.20.90-e27bfd/src/rm: cannot remove 'foo': 
No such file or directory
$ sudo dtruss sudo -u $USER $D/rm foo 2>&1 | grep foo
$ sudo dtruss -f sudo -u $USER $D/rm foo 2>&1 | grep foo
$ sudo dtruss -f sudo -u $USER $D/rm foo 2>&1 | grep foo


Any help is appreciated.

 -gordon

bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Paul Eggert]

On 01/22/13 11:22, Assaf Gordon wrote:
> 3. Running with 'sudo', 'rm' doesn't fail:
> $ sudo dtruss $D/rm -r $D/a $D/b 2>/tmp/rm_trace.txt

You need a sudo inside dtruss, to, right?
Something like this

   sudo dtruss sudo -u $USER $D/rm -r $D/a $D/b 2>/tmp/rm_trace.txt

bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Assaf Gordon]

Paul Eggert wrote, On 01/22/2013 02:01 AM:
> Thanks, to help debug that can you please try the following
> shell commands?  The idea is to run the newly-built "rm" in
> a directory that is not readable.
> 
> cd coreutils-8.20/src
> D=$(pwd)
> mkdir -p a/1 b c d/2 e/3
> cd c
> chmod u=x,go= .
> ktrace -o /tmp/tr $D/rm -r $D/a $D/b
> kdump /tmp/tr
> 
> The "ktrace" should get a trace of the system calls executed
> by the failing "rm", and the "kdump" should generate a printable
> output of that.

ktrace/kdump are not available any more on Mac OS X (since 10.5).
I found "dtruss" - but got into a catch-22:

1. Running 'rm' as-is gives error:
$ $D/rm -r $D/a $D/b 
/Users/admin/Projects/coreutils-8.20.90-e27bfd/src/rm: unable to record 
current working directory: Too many open files

2. Running with 'dtruss' requires root:
$ dtruss $D/rm -r $D/a $D/b 
dtrace: failed to initialize dtrace: DTrace requires additional privileges

3. Running with 'sudo', 'rm' doesn't fail:
$ sudo dtruss $D/rm -r $D/a $D/b 2>/tmp/rm_trace.txt
[ $D/a and $D/b are deleted ]

Attached is the "rm_trace.txt" file, but it won't show failure...

I'm not familiar with MacOS, so any suggestions are welcomed.


Pádraig:
running "make check" in gnulib-test directory failed 8 tests (attached).
I can try instrumentations and other tests, but please tell me how to run them.



-gordon



SYSCALL(args)= return
madvise(0x7FFF8AC18000, 0x2000, 0x5) = 0 0
open("/dev/dtracehelper\0", 0x2, 0x7FFF5FC45200) = 3 0
ioctl(0x3, 0x80086804, 0x7FFF5FBFD4F0)   = 0 0
close(0x3)   = 0 0
stat64("/usr/lib/libstdc++.6.dylib\0", 0x7FFF5FBFC900, 0x7FFF5FBFCF40)  
 = 0 0
open("/dev/dtracehelper\0", 0x2, 0x7FFF5FC452C8) = 3 0
ioctl(0x3, 0x80086804, 0x7FFF5FBFD4F0)   = 0 0
close(0x3)   = 0 0
__sysctl(0x7FFF5FBFD3E0, 0x2, 0x7FFF5FBFD3D0)= 0 0
bsdthread_register(0x7FFF805E8E7C, 0x7FFF805C9A98, 0x2000)   = 0 0
thread_selfid(0x7FFF805E8E7C, 0x7FFF805C9A98, 0x0)   = 629779 0
open_nocancel("/dev/urandom\0", 0x0, 0x7FFF700F8C00) = 3 0
read_nocancel(0x3, 
"\25422\365p\2578e\222\374~\241\020\001\215\265?\036JM\377\357\276\263\346^J\272\303\250\343\211\247\325VdZ\320F\373\346\326\364\356?\374\315\366\313\376\035-\025\332\271g\210\272\2551!&\271\305\0",
 0x40)  = 64 0
close_nocancel(0x3)  = 0 0
mmap(0x0, 0x3000, 0x3, 0x1002, 0x100, 0x7FFF0001)= 
0x25000 0
__sysctl(0x7FFF5FBFD260, 0x2, 0x7FFF5FBFD220)= 0 0
__sysctl(0x7FFF5FBFD220, 0x2, 0x7FFF5FBFD2B8)= 0 0
getpid(0x7FFF5FBFD1B0, 0x7FE00050, 0x0)  = 36891 0
open_nocancel("/dev/urandom\0", 0x0, 0x0)= 3 0
read_nocancel(0x3, ".^\3241ql^\322\331\246d\243\361/S 
\200/\223\006\204\254g\357AZ\326|\246\236Ui\031\273\203\220;\204|\344\367mO`e\213\245^\335\240\251yEh$\264\247\335P\325G\264Q\364\340\207\330S-\226g\363\223@S\213\310v\306>\251I\265\030\361\2540\331V\264\374\254\333b\315d\241\277\236\271\240m\267\336+\016\314n\0",
 0x6C) = 108 0
close_nocancel(0x3)  = 0 0
__sysctl(0x7FFF5FBFD260, 0x2, 0x7FFF5FBFD28C)= 0 0
mmap(0x0, 0x17000, 0x3, 0x1002, 0x100, 0x7FFF0001)   = 
0x28000 0
mmap(0x0, 0x17000, 0x3, 0x1002, 0x100, 0x7FFF0001)   = 
0x3F000 0
mmap(0x0, 0x1000, 0x3, 0x1002, 0x100, 0x7FFF0001)= 
0x56000 0
mmap(0x0, 0x20, 0x3, 0x1002, 0x700, 0x7FFF0001)  = 
0x57000 0
munmap(0x100057000, 0xA9000) = 0 0
munmap(0x10020, 0x57000) = 0 0
__sysctl(0x7FFF5FBFD3C0, 0x2, 0x7FFF5FBFD380)= 0 0
__sysctl(0x7FFF5FBFD380, 0x2, 0x7FFF701057A0)= 0 0
__sysctl(0x7FFF5FBFD3C0, 0x2, 0x7FFF5FBFD380)= 0 0
__sysctl(0x7FFF5FBFD380, 0x2, 0x7FFF701057A4)= 0 0
__sysctl(0x7FFF5FBFD3C0, 0x2, 0x7FFF5FBFD380)= 0 0
__sysctl(0x7FFF5FBFD380, 0x2, 0x7FFF701057A8)= 0 0
mmap(0x0, 0x3000, 0x3, 0x1002, 0x100, 0x7FFF0001)= 
0x57000 0
__sysctl(0x7FFF5FBFD390, 0x2, 0x7FFF5FBFD350)= 0 0
__sysctl(0x7FFF5FBFD350, 0x2, 0x7FFF5FBFD3E8)= 0 0
__sysctl(0x7FFF5FBFD390, 0x2, 0x7FFF5FBFD3BC)= 0 0
mmap(0x0, 0x17000, 0x3, 0x1002, 0x100, 0x7FFF0001)   = 
0x5A000 0
mmap(0x0, 0x17000, 0x3, 0x1002, 0x100, 0x7FFF0001)   = 
0x71000 0
mmap(0x0, 0x20, 0x3, 0x1002, 0x700, 0x7FFF0001)  = 
0x20 0
munmap(0x10030, 0x10)= 0 0
ioctl(0x0, 0x4004667A, 0x7FFF5FBFF6FC)   = 0 0
lstat64("/\0", 0x7FFF5FBFF6C8, 0x0)  = 0 0
open(".\0", 0x0, 0x0)= 3 0
close(0x3)   = 0 0
mmap(0x0, 0x100, 0x3, 0x1002, 0x200, 0x7FFF) = 
0x30 0
munmap(0x10030, 0x50)= 0 0
munmap(0x10100, 0x30

bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Paul Eggert]

Thanks, to help debug that can you please try the following
shell commands?  The idea is to run the newly-built "rm" in
a directory that is not readable.

cd coreutils-8.20/src
D=$(pwd)
mkdir -p a/1 b c d/2 e/3
cd c
chmod u=x,go= .
ktrace -o /tmp/tr $D/rm -r $D/a $D/b
kdump /tmp/tr

The "ktrace" should get a trace of the system calls executed
by the failing "rm", and the "kdump" should generate a printable
output of that.

bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Pádraig Brady]

On 01/21/2013 07:51 PM, Assaf Gordon wrote:

Hello,

Testing the latest coreutils on Mac OS X, "tests/rm/unread3" fails.
The immediate cause is:
"rm: unable to record current working directory: Too many open files"





Attached is the relevant part from the log file.

The programs were compiled using:
===
$ ./configure CC=clang

## - ##
## Platform. ##
## - ##

hostname = minimacc.local
uname -m = i386
uname -r = 10.8.0
uname -s = Darwin
uname -v = Darwin Kernel Version 10.8.0: Tue Jun  7 16:33:36 PDT 2011; 
root:xnu-1504.15.3~1/RELEASE_I386

/usr/bin/uname -p = i386
/bin/uname -X = unknown

/bin/arch  = unknown
/usr/bin/arch -k   = unknown
/usr/convex/getsysinfo = unknown
/usr/bin/hostinfo  = Mach kernel version:
  Darwin Kernel Version 10.8.0: Tue Jun  7 16:33:36 PDT 2011; 
root:xnu-1504.15.3~1/RELEASE_I386
Kernel configured for up to 2 processors.
2 processors are physically available.
2 processors are logically available.
Processor type: i486 (Intel 80486)
Processors active: 0 1
Primary memory available: 2.00 gigabytes
Default processor set: 71 tasks, 267 threads, 2 processors
Load average: 0.19, Mach factor: 1.80
/bin/machine   = unknown
/usr/bin/oslevel   = unknown
/bin/universe  = unknown
[...]
configure:4946: clang --version >&5
Apple clang version 4.1 (tags/Apple/clang-421.11.66) (based on LLVM 3.1svn)
Target: x86_64-apple-darwin10.8.0
Thread model: posix
configure:4957: $? = 0
configure:4946: clang -v >&5
Apple clang version 4.1 (tags/Apple/clang-421.11.66) (based on LLVM 3.1svn)
Target: x86_64-apple-darwin10.8.0
Thread model: posix
===

-gordon

P.S.
To compile it on Mac, Paul Eggert's fix is required ( 
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=13495 ).



Interesting. That's coming from openat_save_fail() in gnulib.
There was a similar issue reported on AIX 5.1 recently:
http://lists.gnu.org/archive/html/coreutils/2012-01/msg00059.html
Do the gnulib tests pass?
Perhaps you could add some instrumentation to the
gnulib code in that's being called in lib/
to see why that's happening?

thanks,
Pádraig.

bug#13516: tests/rm/unread3 fails on Mac OS X 10.8

[Assaf Gordon]

Hello,

Testing the latest coreutils on Mac OS X, "tests/rm/unread3" fails.
The immediate cause is:
"rm: unable to record current working directory: Too many open files"

Attached is the relevant part from the log file.

The programs were compiled using:
===
$ ./configure CC=clang

## - ##
## Platform. ##
## - ##

hostname = minimacc.local
uname -m = i386
uname -r = 10.8.0
uname -s = Darwin
uname -v = Darwin Kernel Version 10.8.0: Tue Jun  7 16:33:36 PDT 2011; 
root:xnu-1504.15.3~1/RELEASE_I386

/usr/bin/uname -p = i386
/bin/uname -X = unknown

/bin/arch  = unknown
/usr/bin/arch -k   = unknown
/usr/convex/getsysinfo = unknown
/usr/bin/hostinfo  = Mach kernel version:
 Darwin Kernel Version 10.8.0: Tue Jun  7 16:33:36 PDT 2011; 
root:xnu-1504.15.3~1/RELEASE_I386
Kernel configured for up to 2 processors.
2 processors are physically available.
2 processors are logically available.
Processor type: i486 (Intel 80486)
Processors active: 0 1
Primary memory available: 2.00 gigabytes
Default processor set: 71 tasks, 267 threads, 2 processors
Load average: 0.19, Mach factor: 1.80
/bin/machine   = unknown
/usr/bin/oslevel   = unknown
/bin/universe  = unknown
[...]
configure:4946: clang --version >&5
Apple clang version 4.1 (tags/Apple/clang-421.11.66) (based on LLVM 3.1svn)
Target: x86_64-apple-darwin10.8.0
Thread model: posix
configure:4957: $? = 0
configure:4946: clang -v >&5
Apple clang version 4.1 (tags/Apple/clang-421.11.66) (based on LLVM 3.1svn)
Target: x86_64-apple-darwin10.8.0
Thread model: posix
===

-gordon

P.S.
To compile it on Mac, Paul Eggert's fix is required ( 
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=13495 ).
FAIL: tests/rm/unread3
==

++ initial_cwd_=/Users/admin/Projects/coreutils-8.20.90-e27bfd
++ fail=0
+++ testdir_prefix_
+++ printf gt
++ pfx_=gt
+++ mktempd_ /Users/admin/Projects/coreutils-8.20.90-e27bfd gt-unread3.sh.
+++ case $# in
+++ destdir_=/Users/admin/Projects/coreutils-8.20.90-e27bfd
+++ template_=gt-unread3.sh.
+++ MAX_TRIES_=4
+++ case $destdir_ in
+++ case $template_ in
 unset TMPDIR
+++ d=/Users/admin/Projects/coreutils-8.20.90-e27bfd/gt-unread3.sh.8WPN
+++ case $d in
+++ test -d /Users/admin/Projects/coreutils-8.20.90-e27bfd/gt-unread3.sh.8WPN
 ls -dgo /Users/admin/Projects/coreutils-8.20.90-e27bfd/gt-unread3.sh.8WPN
 tr S -
+++ perms='drwx-- 2 68 Jan 21 14:27 /Users/admin/Projects/coreutils-8.20.90-e27bfd/gt-unread3.sh.8WPN'
+++ case $perms in
+++ test 0 = 0
+++ echo /Users/admin/Projects/coreutils-8.20.90-e27bfd/gt-unread3.sh.8WPN
+++ return
++ test_dir_=/Users/admin/Projects/coreutils-8.20.90-e27bfd/gt-unread3.sh.8WPN
++ cd /Users/admin/Projects/coreutils-8.20.90-e27bfd/gt-unread3.sh.8WPN
++ gl_init_sh_nl_='
'
++ IFS=' 	
'
++ for sig_ in 1 2 3 13 15
+++ expr 1 + 128
++ eval 'trap '\''Exit 129'\'' 1'
+++ trap 'Exit 129' 1
++ for sig_ in 1 2 3 13 15
+++ expr 2 + 128
++ eval 'trap '\''Exit 130'\'' 2'
+++ trap 'Exit 130' 2
++ for sig_ in 1 2 3 13 15
+++ expr 3 + 128
++ eval 'trap '\''Exit 131'\'' 3'
+++ trap 'Exit 131' 3
++ for sig_ in 1 2 3 13 15
+++ expr 13 + 128
++ eval 'trap '\''Exit 141'\'' 13'
+++ trap 'Exit 141' 13
++ for sig_ in 1 2 3 13 15
+++ expr 15 + 128
++ eval 'trap '\''Exit 143'\'' 15'
+++ trap 'Exit 143' 15
++ trap remove_tmp_ 0
+ path_prepend_ ./src
+ test 1 '!=' 0
+ path_dir_=./src
+ case $path_dir_ in
+ abs_path_dir_=/Users/admin/Projects/coreutils-8.20.90-e27bfd/./src
+ case $abs_path_dir_ in
+ PATH=/Users/admin/Projects/coreutils-8.20.90-e27bfd/./src:/Users/admin/Projects/coreutils-8.20.90-e27bfd/src:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin
+ create_exe_shims_ /Users/admin/Projects/coreutils-8.20.90-e27bfd/./src
+ case $EXEEXT in
+ return 0
+ shift
+ test 0 '!=' 0
+ export PATH
+ print_ver_ rm
+ test yes = yes
+ local i
+ for i in '$*'
+ env rm --version
rm (GNU coreutils) 8.20.90-e27bfd
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later .
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Paul Rubin, David MacKenzie, Richard M. Stallman,
and Jim Meyering.
+ skip_if_root_
+ uid_is_privileged_
++ id -u
+ my_uid=502
+ case $my_uid in
+ return 1
+ mkdir -p a/1 b c d/2 e/3
++ pwd
+ t=/Users/admin/Projects/coreutils-8.20.90-e27bfd/gt-unread3.sh.8WPN
+ cd c
+ chmod u=x,go= .
+ rm -r /Users/admin/Projects/coreutils-8.20.90-e27bfd/gt-unread3.sh.8WPN/a /Users/admin/Projects/coreutils-8.20.90-e27bfd/gt-unread3.sh.8WPN/b
rm: unable to record current working directory: Too many open files
+ fail=1
+ rm -r /Users/admin/Projects/coreutils-8.20.90-e27bfd/gt-unread3.sh.8WPN/d /Users/admin/Projects/coreutils-8.20.90-e27bfd/gt-unread3.sh.8WPN/e
rm: unable to record current working directory: Too many open files
+ fail=1
+ test -d /Users/admin/Projects/coreutils-8.20.90-e27bfd/gt-unread3.sh.8