infinite loop in rm
I saw some mention of this problem here: http://www.cygwin.com/ml/cygwin/2002-07/msg00147.html Is there a fix for this that works, or will be incorporated into a future version of cygwin? I looked in the FAQ and saw nothing about it. I have some nightly scripts that clean some directories, and if I leave a shell open in one of the directories, the scripts just run forever trying to remove the directory. -Bill -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: infinite loop in rm
William A. Hoffman wrote:
I saw some mention of this problem here:
http://www.cygwin.com/ml/cygwin/2002-07/msg00147.html
Is there a fix for this that works, or will be incorporated
into a future version of cygwin? I looked in the FAQ and
saw nothing about it. I have some nightly scripts that clean
some directories, and if I leave a shell open in one of the
directories, the scripts just run forever trying to remove
the directory.
I am using the following patch to unlink() on a version of Cygwin that
is several months old. It does nothing unless there is a directory
named .cygdel at the root of the drive holding the file to be deleted.
In that case, it moves the file there before trying to delete it.
Make sure you make the directory mode 777 when you create it...
Works for me -- I think I posted this to the list but may be wrong.
Joe Buehler
Index: src/winsup/cygwin/syscalls.cc
===
RCS file: /cvs/src/src/winsup/cygwin/syscalls.cc,v
retrieving revision 1.214
diff -u -r1.214 syscalls.cc
--- src/winsup/cygwin/syscalls.cc 2 Jul 2002 03:06:32 - 1.214
+++ src/winsup/cygwin/syscalls.cc 8 Aug 2002 16:38:12 -
@@ -142,6 +142,23 @@
SetFileAttributes (win32_name, (DWORD) win32_name ~FILE_ATTRIBUTE_READONLY);
}
+ // attempt to rename before deleting
+ char *basename;
+ basename = strrchr(win32_name, '\\');
+ if (basename *++basename) {
+const char *rootdir = win32_name.root_dir();
+if (rootdir) {
+ const char *s = strrchr(rootdir, '\\');
+ if (s !s[1]) {
+ char newname[MAX_PATH + 12];
+ __small_sprintf(newname, %s.cygdel\\%s, rootdir, basename);
+ if (MoveFile(win32_name.get_win32(), newname)) {
+ win32_name.check(newname, PC_SYM_NOFOLLOW | PC_FULL);
+ }
+ }
+}
+ }
+
DWORD lasterr;
lasterr = 0;
for (int i = 0; i 2; i++)
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Bug reporting: http://cygwin.com/bugs.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
Infinite Loop In rm -fr When Busy File Encountered
Greetings, I have encountered a problem in rm when used with the -f and -r options and a busy and hence unremovable file is encountered. When rm -fr encounters a busy and hence unremovable file, it goes into an infinite loop consuming as much CPU time as it can get until it is forcibly terminated. The error is completely repeatable as long as the file remains busy by virtue of being open in another program. The file system is NTFS and the mount is binmode. The system is Windows 2000 Pro with SP2 and security patches installed. This is an up-to-date Cygwin installation that is exhibiting no other anomalous behavior. In reference to the attached cygcheck -rs output, this happened on C: /cygdrive/c (reached via a symlink /c - /cygdrive/c). % rm --version rm (fileutils) 4.1 Written by Paul Rubin, David MacKenzie, Richard Stallman, and Jim Meyering. Copyright (C) 2001 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. My hunch, unencumbered by the source code, is that the directory access libraries as implemented in Cygwin / Windows is, in some sense, dynamic (as opposed to a statically built snapshot that is acquired once and kept internally) and hence keeps presenting rm with the same unsuccessfully removed and unremovable file over and over again. Randall Schulz Mountain View, CA USA Cygwin Win95/NT Configuration Diagnostics Current System Time: Sat Apr 06 08:49:25 2002 Windows 2000 Professional Ver 5.0 Build 2195 Service Pack 2 Path: . D:\cygwin\home\RSchulz\bin D:\cygwin\usr\sbin D:\cygwin\usr\X11R6\bin c:\j2sdk1.4.0\bin D:\cygwin\usr\local\bin D:\cygwin\bin D:\cygwin\bin d:\Apps\texmf\miktex\bin d:\Perl\bin\ d:\Tcl\bin\ d:\WINNT\system32 d:\WINNT d:\WINNT\system32\WBEM d:\Apps\XEmacs\XEmacs-21.1.14\i586-pc-win32 d:\Apps\GNUEmacs\emacs-20.7\bin D:\cygwin\bin D:\cygwin\usr\X11R6\bin . d:\WN16\BIN SysDir: D:\WINNT\System32 WinDir: D:\WINNT CYGWIN = `binmode ntsec' HOME = `D:\cygwin\home\RSchulz' MAKE_MODE = `unix' PWD = `/home/RSchulz' TCL_LIBRARY = `D:\WN16\LIB\TCL7.6' USER = `RSchulz' HKEY_CURRENT_USER\Software\Cygnus Solutions HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin (default) = 0x0200 HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\mounts v2 (default) = `/cygdrive' cygdrive flags = 0x0022 HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\mounts v2\/dev/fd0 (default) = `\\.\A:' flags = 0x0002 HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\Program Options HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin (default) = 0x0200 HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2 HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/ (default) = `D:/cygwin' flags = 0x000a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/usr/bin (default) = `D:/cygwin/bin' flags = 0x000a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/usr/lib (default) = `D:/cygwin/lib' flags = 0x000a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\Program Options a: fd N/AN/A c: hd NTFS 10001Mb 34% CP CS UN PA FC WIN2K d: hd NTFS 11664Mb 49% CP CS UN PA FC Win2K2niW e: hd NTFS4027Mb 94% CP CS UN PA FC Stash f: cd N/AN/A g: cd CDFS 647Mb 100%CS UN BOOKSHELF2K h: hd FAT32649Mb 1% CPUN BURNME \\.\A: /dev/fd0 userbinmode . /cygdrive userbinmode,noumount D:/cygwin / system binmode D:/cygwin/bin /usr/bin system binmode D:/cygwin/lib /usr/lib system binmode Found: D:\cygwin\bin\bash.exe Found: D:\cygwin\bin\cat.exe Found: D:\cygwin\bin\cpp.exe Found: D:\cygwin\bin\find.exe Found: D:\cygwin\bin\gcc.exe Found: D:\cygwin\bin\gdb.exe Found: D:\cygwin\bin\ld.exe Found: D:\cygwin\bin\ls.exe Found: D:\cygwin\bin\make.exe Found: D:\cygwin\bin\sh.exe 56k 2000/12/03 D:\cygwin\bin\cygbz21.0.dll 621k 2002/01/16 D:\cygwin\bin\cygcrypto.dll 107k 2002/03/07 D:\cygwin\bin\cygcurl-2.dll 50k 2002/03/17 D:\cygwin\bin\cygexslt-0.dll 45k 2001/04/25 D:\cygwin\bin\cygform5.dll 35k 2002/01/09 D:\cygwin\bin\cygform6.dll 19k 2002/02/20 D:\cygwin\bin\cyggdbm.dll 17k 2001/06/28 D:\cygwin\bin\cyghistory4.dll 20k 2002/01/13 D:\cygwin\bin\cyghistory5.dll 22k 2001/12/13 D:\cygwin\bin\cygintl-1.dll 21k 2001/06/20 D:\cygwin\bin\cygintl.dll 81k 2000/12/05 D:\cygwin\bin\cygitcl30.dll 35k 2000/12/05 D:\cygwin\bin\cygitk30.dll 45k 2002/02/08 D:\cygwin\bin\cygjbig1.dll 119k 2002/02/09 D:\cygwin\bin\cygjpeg6b.dll 25k 2002/03/17 D:\cygwin\bin\cygltdl-3.dll 26k 2001/04/25
Re: Infinite Loop In rm -fr When Busy File Encountered
Greetings, I have encountered a problem in rm when used with the -f and -r options and a busy and hence unremovable file is encountered. When rm -fr encounters a busy and hence unremovable file, it goes into an infinite loop consuming as much CPU time as it can get until it is forcibly terminated. The error is completely repeatable as long as the file remains busy by virtue of being open in another program. The file system is NTFS and the mount is binmode. The system is Windows 2000 Pro with SP2 and security patches installed. This is an up-to-date Cygwin installation that is exhibiting no other anomalous behavior. In reference to the attached cygcheck -rs output, this happened on C: /cygdrive/c (reached via a symlink /c - /cygdrive/c). I tried exhaustively to reproduce this (cygwin1.dll v1.3.10), but failed. Chris -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Infinite Loop In rm -fr When Busy File Encountered
Chris, At 09:26 2002-04-06, Chris January wrote: Greetings, I have encountered a problem in rm when used with the -f and -r options and a busy and hence unremovable file is encountered. When rm -fr encounters a busy and hence unremovable file, it goes into an infinite loop consuming as much CPU time as it can get until it is forcibly terminated. The error is completely repeatable as long as the file remains busy by virtue of being open in another program. The file system is NTFS and the mount is binmode. The system is Windows 2000 Pro with SP2 and security patches installed. This is an up-to-date Cygwin installation that is exhibiting no other anomalous behavior. In reference to the attached cygcheck -rs output, this happened on C: /cygdrive/c (reached via a symlink /c - /cygdrive/c). I tried exhaustively to reproduce this (cygwin1.dll v1.3.10), but failed. Exhaustively? In such little time? Please tell me what you tried. I believe when it happened to me the file triggering the problem was open for writing. Randall Schulz Chris -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Infinite Loop In rm -fr When Busy File Encountered
Greetings, I have encountered a problem in rm when used with the -f and -r options and a busy and hence unremovable file is encountered. When rm -fr encounters a busy and hence unremovable file, it goes into an infinite loop consuming as much CPU time as it can get until it is forcibly terminated. The error is completely repeatable as long as the file remains busy by virtue of being open in another program. The file system is NTFS and the mount is binmode. The system is Windows 2000 Pro with SP2 and security patches installed. This is an up-to-date Cygwin installation that is exhibiting no other anomalous behavior. In reference to the attached cygcheck -rs output, this happened on C: /cygdrive/c (reached via a symlink /c - /cygdrive/c). I tried exhaustively to reproduce this (cygwin1.dll v1.3.10), but failed. Exhaustively? In such little time? Oops - exhaustively was the wrong word to use. My apologies! Please tell me what you tried. I believe when it happened to me the file triggering the problem was open for writing. From memory: mkdir test cp test.c test (where test.c is a program that just sleeps in a loop) cd test gcc test.c -otest ./test cd .. rm -rf test mkdir test2 ln -s $PWD/test test2 rm -rf test2 mkdir test2 ln -s $PWD/test test2 rm -rf test2/* mkdir test/test3 cp test/test test/test3 test/test3/test rm -rf test mkdir test2 ln -s $PWD/test test2 rm -rf test2/test3 mkdir /cygdrive/c/test cp test/test /cygdrive/c/test /cygdrive/c/test/test rm -rf /cygdrive/c/test ln -s /cygdrive/c /c rm -rf /c/test And maybe some more, I forget. If it is indeed a bug causing the problem you specified, it probably lies in delqueue.cc somewhere. If you can reproduce the problem, try running rm under strace: strace rm -fr directory Randall Schulz -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Infinite Loop In rm -fr When Busy File Encountered
Chris,
OK. Here's more information. Possibly even interesting information.
I tried this as a test:
% mkdir tstdir
% cd tstdir
% cat tstdir
[1]+ Stopped cat target
% rm target
# No complaint!
% ls -l
ls: target: Permission denied
% rm target
rm: cannot remove `target': Permission denied
That was not quite what I expected, and, ignorant as I am about the Windows APIs, I
cannot explain it.
Now as it turns out, the case where I encountered this problem was when an errant
non-Cygwin program (specifically, a Java program) was the culprit keeping the file
open. I happened, presumably, due to incomplete clean-up by a tool plug-in executing
within an IDE (jEdit).
So, I wrote a little Java program to open a file for writing and then just pause,
keeping the file open:
import java.io.*;
public
class Writer
{
public
static
void
main(String args[])
{
if (args.length 2) {
System.err.println(Usage: Writer outputFile string [ ... ]);
System.exit(1);
}
PrintWriter pw = null;
try {
pw = new PrintWriter(new FileWriter(args[0]));
}
catch (IOException ioEx) {
System.err.println(Error opening \ + args[0] + \);
System.exit(2);
}
for (int argN = 1; argN args.length; argN++) {
pw.print(args[argN]);
}
Thread thisThread = Thread.currentThread();
for (;;) {
try {
thisThread.sleep(100 * 1000);
}
catch (InterruptedException iEx) {
}
}
}
}
Now I do this:
% mkdir subdir
% java Writer subdir/target Now is the time
In a separate Cygwin window:
% cd tstdir
% rm -fr subdir
And the symptom is manifest: rm sits there, churning away, awaiting its death at the
hands of a merciless signal.
Randall Schulz
Mountain View, CA USA
At 09:26 2002-04-06, Chris January wrote:
Greetings,
I have encountered a problem in rm when used with the -f and -r
options and a busy and hence unremovable file is encountered.
When rm -fr encounters a busy and hence unremovable file, it goes into an
infinite loop consuming as much CPU time as it can get until it is forcibly
terminated. The error is completely repeatable as long as the file remains
busy by virtue of being open in another program.
The file system is NTFS and the mount is binmode. The system is Windows
2000 Pro with SP2 and security patches installed.
This is an up-to-date Cygwin installation that is exhibiting no other
anomalous behavior.
In reference to the attached cygcheck -rs output, this happened on C:
/cygdrive/c (reached via a symlink /c - /cygdrive/c).
I tried exhaustively to reproduce this (cygwin1.dll v1.3.10), but failed.
Chris
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Bug reporting: http://cygwin.com/bugs.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
Re: Infinite Loop In rm -fr When Busy File Encountered
Now I do this: % mkdir subdir % java Writer subdir/target Now is the time In a separate Cygwin window: % cd tstdir % rm -fr subdir And the symptom is manifest: rm sits there, churning away, awaiting its death at the hands of a merciless signal. What output do you get with: strace -fr subdir ? -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Infinite Loop In rm -fr When Busy File Encountered
Chris, OK. Here's more information. Possibly even interesting information. Another question - which shell are you using? Can you reproduce the problem using bash? Chris -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Infinite Loop In rm -fr When Busy File Encountered
Chris, I'm using BASH. I'll let you know what the strace experiment turns up when I have some time to give that a try. Randall At 10:25 2002-04-06, you wrote: Chris, OK. Here's more information. Possibly even interesting information. Another question - which shell are you using? Can you reproduce the problem using bash? Chris -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Infinite Loop In rm -fr When Busy File Encountered
Chris, OK. I ran the strace test as you requested. Here's the invocation of strace I used in place of the unadorned rm -fr subdir invocation from my earlier report: % strace -d -n -S 1 -w -o rm-fr-strace-out rm -fr subdir I waited a few seconds then closed the sub-window. A Bzip2-compressed form of rm-fr-strace-out is attached (19.77 : 1 compression!). A 170-column-wide window will eliminate line wrapping. If you seach for lstat: entering you'll see the infinitely repeating sequence of operations. I'm not entirely sure, but that appears to be the entry point of the loop. There appear to be 270 iterations present in the strace log. If you're a Vim-er, this macro will help you see the pattern: :map ^[N Nzz (^[ is an escape, naturally, which requires a prefix CTRL-V if entered manually--mine originates in my .vimrc). Prime the search patter given above and then enter repeated ALT-N and you'll see that only the timestamps along the left change. One last piece of information, after running strace and killing the waiting Java program, subdir/target is gone and a simple rmdir subdir succeeds. Let me know if I can help further. Randall Schulz Mountain View, CA USA At 10:24 2002-04-06, Chris January wrote: Now I do this: % mkdir subdir % java Writer subdir/target Now is the time In a separate Cygwin window: % cd tstdir % rm -fr subdir And the symptom is manifest: rm sits there, churning away, awaiting its death at the hands of a merciless signal. What output do you get with: strace -fr subdir ? rm-fr-strace-out.bz2 Description: Binary data -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Infinite Loop In rm -fr When Busy File Encountered
Chris, OK. I ran the strace test as you requested. Here's the invocation of strace I used in place of the unadorned rm -fr subdir invocation from my earlier report: % strace -d -n -S 1 -w -o rm-fr-strace-out rm -fr subdir I waited a few seconds then closed the sub-window. A Bzip2-compressed form of rm-fr-strace-out is attached (19.77 : 1 compression!). A 170-column-wide window will eliminate line wrapping. If you seach for lstat: entering you'll see the infinitely repeating sequence of operations. I'm not entirely sure, but that appears to be the entry point of the loop. There appear to be 270 iterations present in the strace log. If you're a Vim-er, this macro will help you see the pattern: :map ^[N Nzz (^[ is an escape, naturally, which requires a prefix CTRL-V if entered manually--mine originates in my .vimrc). Prime the search patter given above and then enter repeated ALT-N and you'll see that only the timestamps along the left change. One last piece of information, after running strace and killing the waiting Java program, subdir/target is gone and a simple rmdir subdir succeeds. Let me know if I can help further. I've just looked over the sources to _unlink in Cygwin and the rm command. The problem as far as I can see is that if there is a sharing violation then the file being deleted is added to a queue of files to be deleted later. However if you list the contents of the directory the file was originally in, it will still show up. The second problem is this: 174347 [main] rm 2052 fhandler_disk_file::readdir: 0x0 = readdir (0xA012470) 174849 [main] rm 2052 fhandler_disk_file::readdir: 0xA012474 = readdir (0xA012470) (.) This shouldn't happen. The remove_cwd_entries function in rm's remove.c should return when readdir returns NULL. I can only presume this is a result of HAVE_WORKING_READDIR being undefined in fileutil's config.h. On the other hand, I've just configured fileutils now and HAVE_WORKING_READDIR is defined as 1. Maybe something changed in the Cygwin DLL between the last distribution of fileutils and now. The result of the second problem is that rm calls unlink to delete the file 'target', unlink suceeds, then rm starts reading the directory again. But because the file wasn't really removed, the first problem manifests itself: readdir returns an entry for 'target' again and then rm tries to delete it again, ad infinitum. Perhaps the maintainer of fileutils could contribute his or her opinion on this. Regards Chris -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
