On 23 Jun 2003, Paul D. Smith wrote: > %% Ted Stern <[EMAIL PROTECTED]> writes: > > ts> My version of the "make: *** virtual memory exhausted. Stop." bug > ts> occurs from trying to include 1000's of files. It doesn't matter > ts> how large the files are --- what is important is the length of the > ts> files' pathnames. If I allow 128 mega words (1GB) for the qsub > ts> job, the error occurs with about 5000 80 character filenames. If > ts> I allow 16MW or 128MB for the job, the error occurs with about > ts> 1500 included files of about 80-character pathnames each. > > ts> In earlier correspondence with Paul Smith, he indicated that he > ts> thought there may be a memory leak in read.c. > > ts> I have verified that the bug has not been fixed in current CVS. > > ts> Sure would be nice if somebody with access to purify could run it > ts> on GNU make ... > > I have run the GNU make test suite with Purify on Solaris, and valgrind > on Linux, and neither have reported any memory leaks with the latest CVS > code. > > If you have a test case that can show this problem pls. let me know. > Thanks.
I am attaching three files that should give an idea of how to re-create the problem. "touchempty.ksh" creates a hierarchy of 1000 directories, each containing 10 *.mk files. The Makefile tries to include all the 10K makefiles. The qsub script runs the make job in a batch environment with total memory for all processes limited by the QSUB "-lM" option. To duplicate this outside of a batch environment you may have to play with your shell's "limit" command. The test directory in which I set this up was named "/ptmp/stern/testmake/", 21 leading characters, so to really duplicate this you should give your top directory an absolute pathname of 21 characters or more. Ted -- Ted Stern Applications Group Cray Inc. office: 206-701-2182 411 First Avenue South, Suite 600 cell: 206-383-1049 Seattle, WA 98104-2860 FAX: 206-701-2500 Frango ut patefaciam -- I break that I may reveal (The Paleontological Society motto, equally apropos for debugging)
#!/bin/ksh i=0 while [[ $i -lt 10 ]] ; do j=0 while [[ $j -lt 10 ]] ; do k=0 while [[ $k -lt 10 ]] ; do mkdir -p AAAA_$i/BBBB_$j/CCCC_$k m=0 while [[ $m -lt 10 ]] ; do touch AAAA_$i/BBBB_$j/CCCC_$k/DDDD_$m.mk m=$((m+1)) done k=$((k+1)) done j=$((j+1)) done i=$((i+1)) done
SHELL := /bin/ksh ALL_INCLUDES := $(shell find `/bin/pwd`/AAAA* -name '*.mk' -print) include $(ALL_INCLUDES) .PHONY: all all: echo "completed successfully"
#!/bin/ksh # QSUB -lM 16Mw -lT 86400 -s /bin/ksh # QSUB -eo -J m -me # cd $QSUB_WORKDIR num_backup testmake.stdout exec 1> testmake.stdout 2>&1 . $MODULESHOME/init/ksh module load modules PrgEnv open open34 export PATH=$HOME/local/bin:$PATH /ptmp/stern/make/make -R -k -d all
_______________________________________________ Bug-make mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/bug-make