[bug #26075] $(wildcard) function holds parent directories open preventing deletes
URL: http://savannah.gnu.org/bugs/?26075 Summary: $(wildcard) function holds parent directories open preventing deletes Project: make Submitted by: centraspike Submitted on: Fri 03 Apr 2009 09:33:42 GMT Severity: 3 - Normal Item Group: Bug Status: None Privacy: Public Assigned to: None Open/Closed: Open Discussion Lock: Any Component Version: 3.81 Operating System: MS Windows Fixed Release: None ___ Details: Apologies for the windows stuff but the following makefile clean target does not work: all: $(filter-out $(wildcard build\release), build\release) build\release: mkdir build\release clean: rmdir /Q /S build First do gmake all then try gmake clean. The following error occurs: rmdir /Q /S build The process cannot access the file because it is being used by another process. gmake: *** [clean] Error 32 I have managed to trace the cause to the fact that the wildcard function opens a handle to the build directory in order to check if the release directory exists and then does not release the handle till gmake exits ___ File Attachments: --- Date: Fri 03 Apr 2009 09:33:42 GMT Name: makefile Size: 131B By: centraspike Example makefile http://savannah.gnu.org/bugs/download.php?file_id=17857 ___ Reply to this item at: http://savannah.gnu.org/bugs/?26075 ___ Message sent via/by Savannah http://savannah.gnu.org/ ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
[bug #26075] $(wildcard) function holds parent directories open preventing deletes
Follow-up Comment #1, bug #26075 (project make): I've just tried the same construct on Ubuntu where it works just fine so it does seems to be windows specific. I'm currently digging through the source to see if i've forgotten to set some flag at compile time or if i can see the error but so far i haven't found anything. ___ Reply to this item at: http://savannah.gnu.org/bugs/?26075 ___ Message sent via/by Savannah http://savannah.gnu.org/ ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
[bug #26075] $(wildcard) function holds parent directories open preventing deletes
Follow-up Comment #2, bug #26075 (project make): I think I have a fix for this issue (at least it's working for me with my makefiles). I have edited dir.c to limit the maximum number of open directories under windows to just 1 (normally it is 10) this results in directories being closed straight away after caching contents. The change is to replace the following line: #define MAX_OPEN_DIRECTORIES 1 with: #ifdef WINDOWS32 /* Under windows holding directories open at all can cause problems as it prevents them being deleted. For instance in this makefile example the clean target will fail as the build directory will be locked by the make process as a result of the wildcard test: all: $(filter-out $(wildcard buildrelease), buildrelease) buildrelease: mkdir buildrelease clean: rmdir /Q /S build This may be an NTFS specific problem but I have not yet checked this on a Fat file system. */ # define MAX_OPEN_DIRECTORIES 1 #else # define MAX_OPEN_DIRECTORIES 10 #endif I have also attached my version of dir.c (file #17859) ___ Additional Item Attachment: File name: dir.c Size:30 KB ___ Reply to this item at: http://savannah.gnu.org/bugs/?26075 ___ Message sent via/by Savannah http://savannah.gnu.org/ ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #26075] $(wildcard) function holds parent directories open preventing deletes
From: Peter Halliday invalid.nore...@gnu.org Date: Fri, 03 Apr 2009 13:28:36 + Cc: I think I have a fix for this issue (at least it's working for me with my makefiles). I have edited dir.c to limit the maximum number of open directories under windows to just 1 (normally it is 10) this results in directories being closed straight away after caching contents. Thanks, but I hope there is a less Draconian solution to this bug. I will have a look when I have time. ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
[bug #26075] $(wildcard) function holds parent directories open preventing deletes
Follow-up Comment #3, bug #26075 (project make): If you use strace you will see the same problem on Ubuntu - but you won't see rm throw an error because the Linux file system you are using allows directories to be unlinked even if they are in use. This is still a bug, though, even though it isn't obvious on some systems. ___ Reply to this item at: http://savannah.gnu.org/bugs/?26075 ___ Message sent via/by Savannah http://savannah.gnu.org/ ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #26075] $(wildcard) function holds parent directories open preventing deletes
I didn't verify this but the comments in the code suggest that when the open directory limit is reached all the contents of the directory are cached before the handle is released so it may be slightly less draconian to just release the handle without completing the caching and reopening it if another search is required - might be quite a bit more complex though. Also assuming that the wildcard has been evaluated before the clean target is processed (as it must be to run in to the issue) is there a point where the variables have been expanded and the handles can be released (as they would no longer be needed) before the targets are processed - it seems odd that the handles are held until make exits. As a side note i think the only reason i need to do this sort of thing at all is due to the mtime of directories being updated when their contents change thus i get odd behaviour when building something a second time that hasn't changed (early build outputs end up with an earlier mtime than the directory they are in and depend on) - Again from the comments in the code it seems this might be specific to NTFS. However i was wondering if there was a better way to specify a dependency on a directory without using the filter-out / wildcard construct. Or even if it should be considered a bug that make checks mtimes on directories (at least under NTFS). I guess if the directory doesn't exist then make doesn't know it's a directory but then it won't be checking the mtime and if it does exist then it can tell it's a directory and who cares what the mtime is. Of course i may be completely wrong about why i have to construct directory dependencies like this or the validity of directory mtimes and their importance in other use cases. Anyway the solution i have reduces any sense of urgency for me but i think my boss would be happier if i hadn't just made our build environment dependent on a special version of make :) 2009/4/3 Eli Zaretskii e...@gnu.org: From: Peter Halliday invalid.nore...@gnu.org Date: Fri, 03 Apr 2009 13:28:36 + Cc: I think I have a fix for this issue (at least it's working for me with my makefiles). I have edited dir.c to limit the maximum number of open directories under windows to just 1 (normally it is 10) this results in directories being closed straight away after caching contents. Thanks, but I hope there is a less Draconian solution to this bug. I will have a look when I have time. ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
[bug #26075] $(wildcard) function holds parent directories open preventing deletes
Follow-up Comment #4, bug #26075 (project make): Actually that directory cache is just crap. Not only that it causes several bad sideeffects (like new files not being noticed) but actually is slower on average than no cache at all, in cases slowser by magnitudes. ___ Reply to this item at: http://savannah.gnu.org/bugs/?26075 ___ Message sent via/by Savannah http://savannah.gnu.org/ ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
