Parallel build fails with 3.82
Hi, I'm doing builds of WebKit with make -j8 because it reduces the build time from hours to 20 minutes on my build machine, but with GNU Make 3.82 there are occasional build failures due to what appears to be a problem in 3.82 (not 3.81). The WebKit bug is https://bugs.webkit.org/show_bug.cgi?id=79498, and people discovered that the Make bug http://savannah.gnu.org/bugs/?30653 not only describes the problem, but has two independent patches for it. This bug and the patches are now two years old - it would be good if a patch could be integrated or at least reviewed and rejected with a reason. Cheers, Ross ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
[bug #36925] Memory corruption or use after free
URL: http://savannah.gnu.org/bugs/?36925 Summary: Memory corruption or use after free Project: make Submitted by: None Submitted on: Mon 23 Jul 2012 18:14:40 UTC Severity: 3 - Normal Item Group: Bug Status: None Privacy: Public Assigned to: None Open/Closed: Open Discussion Lock: Any Component Version: CVS Operating System: POSIX-Based Fixed Release: None Triage Status: None ___ Details: I'm seeing memory problems in GNU Make CVS, and I think the 3.82 release is also affected. I tested the CVS version with Valgrind (using Ubuntu 12.04 as the host OS), and managed to create a couple of small makefiles to demonstrate the problem. The path of the current directory is significant to the problem. To reproduce the problem the attached bug.tar.gz archive needs to be extracted to: /local/gnumake_test/1234567890123456789012345678901234567890 This will create a subdirectory called bug. In there you will find a couple of makefiles, Makefile and foo/bar/barf/barf.mak. There is also a log valgrind.txt of the results I get from Valgrind. To see the problem, change into the bug directory and execute the following command (where cvsmake is the make executable): env -i valgrind --malloc-fill=0x54 --free-fill=0x55 --tool=memcheck cvsmake -Rr The problem shows itself with this error from GNU Make: make: *** No rule to make target 'UU...', needed by 'foo/bar/barf/barf.i'. Stop. The UU... target is not mentioned in any makefile and is a result of GNU Make accessing freed memory (from --free-fill=0x55 valgrind option). If instead you tell GNU Make to build /dev/null: env -i valgrind --malloc-fill=0x54 --free-fill=0x55 --tool=memcheck cvsmake -Rr /dev/null the GNU Make error is not reported because there is no attempt to built the barf.i target, however Valgrind still shows something is going wrong with memory handling. If the path to the current directory is reduced in length or the paths mentioned in the makefiles are changed then the problem seems to go away. I hope you're able to reproduce what I'm seeing. regards, Rob. ___ File Attachments: --- Date: Mon 23 Jul 2012 18:14:40 UTC Name: bug.tar.gz Size: 1kB By: None http://savannah.gnu.org/bugs/download.php?file_id=26248 ___ Reply to this item at: http://savannah.gnu.org/bugs/?36925 ___ Message sent via/by Savannah http://savannah.gnu.org/ ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: sh embedding
On Mon, 2012-07-23 at 11:59 -0700, icegood wrote: .PHONEY: all all: if [ \( $$(ls *.lock 2/dev/null) == \) ]; then \ touch $@.lock; \ if [ \( ! -e $@ \) -o \( ../$(tag_fn) -nt $@ \) ]; then \ echo $@ done; \ else \ touch $@; \ fi; \ rm -f $@.lock; \ else \ sleep 1; \ fi; The code as you've shown it looks fine to me (except you misspelled .PHONY). When I run it, it works as expected; no errors. So obviously there's something different about your environment that you haven't shown us. For example, you don't show what the value of the tag_fn variable is here. If it contains special characters that could explain it. You probably want to quote it (along with all other variables like $@, etc., just to be safe). What does make print out to the screen when you run the makefile? If you examine that carefully you should be able to see the problem; that's what make sends to the shell and the shell is the one that's complaining. ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: sh embedding
On Mon, Jul 23, 2012 at 11:59 AM, icegood icegood1...@gmail.com wrote: from newer version of gnu make (3.81 under kubuntu 12.04) .PHONEY: all all: if [ \( $$(ls *.lock 2/dev/null) == \) ]; then \ The '==' operator is a bash extension that is supported by many but not all shells. Perhaps the shell seen by make on your kubuntu system is one that doesn't support it? The portable (30+ years!) equality operator is =. (The '==' operator was an extension that improved usability but added no new functionality and instead reduced portability; thanks, bash maintainers, for screwing everyone by making that tradeoff!) touch $@.lock; \ This check for lock file, if it doesn't exist then create it operation is *not* 100% safe! If two makes are run at almost the same time, they may both see the file doesn't exist and then both touch the lock file and continue, so the lock file is not actually guaranteeing that only one copy is running. You should see if there's a real atomic lock file program on your system that you can use for this. For example, I see a flock program on a Redhat system that could be used, those the usage is a bit different. if [ \( ! -e $@ \) -o \( ../$(tag_fn) -nt $@ \) ]; then \ echo $@ done; \ else \ touch $@; \ fi; \ Isn't this the same as making $(tag_fn) a dependency of 'all' and having 'all' *not* be PHONY? rm -f $@.lock; \ else \ sleep 1; \ fi; If the lock cannot be obtained, you just sleep a second and then return success? Why not at least exit 1 there so that the caller can tell that the make failed in a consistent fashion? Philip Guenther ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: sh embedding
Philip Guenther-2 wrote: The '==' operator is a bash extension... Yes! Indeed! I checked. The problem was in '=='. I simply use 'c' much more than bash, that why i used it. Since main question is closed we can now flud about others :) So, about locks. You understand idea of locks in right way. And they indeed was created to prevent dublication of making. And, of course, it's not atomic, i know. Nothing actually atomic in make. Only rule chains are well synchronized. All i have. Full template lib code is: http://old.nabble.com/file/p34202354/common_lib.mak common_lib.mak where 1) real library has own folder, make inside that looks like this: library:=... src:=... # template classes are not included into output library templ_class_src:= sublibraries:=other libs with same make and same common template include ../common_lib.mak and maybe some inner library dependencies: foo: bar 2) other variables (like compiler) are defined in root make So. idea of parallel making was not only prevent double make but also allow other make process to continue building process without waiting sibling one. That's why i don't use any 'exit 1' cases. I hasn't found better way to allow continue building of sibling process without building all files. But at least it works. In nonparallel case everything works, of course, too, but it's not interest case. -- View this message in context: http://old.nabble.com/sh-embedding-tp34201839p34202354.html Sent from the Gnu - Make - Bugs mailing list archive at Nabble.com. ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make