Thanks all -- ok, so the symbolic link issue is a distinct possibility, but fundamentally doesn't solve the issue since most users will have symbolic links on their machines SOMEPLACE, so a full drive scan will run into these issues -- is list.files calling find, or is it using a different algorithm? This seems like a shortcoming in the list.files algorithm -- is there a better solution (short of a System call, which I'm still not sure will work on Macs without Xcode -- a colleague of mine did NOT have Xcode, and reported not being able to run find from the command line) -- perhaps a different package?
--j On Fri, Sep 27, 2013 at 3:08 PM, William Dunlap <wdun...@tibco.com> wrote: > Toss a couple of extra files in there and you will see the output grow > exponentially. > > % touch dir/IMPORTANT_1 dir/subdir/IMPORTANT_2 > > and in R those two new files cause 82 more strings to appear in list.file's > output: > >> nchar(list.files("dir", recursive=TRUE)) > [1] 11 18 33 40 55 62 77 84 99 106 121 128 143 150 165 172 187 194 > 209 > [20] 216 231 238 253 260 275 282 297 304 319 326 341 348 363 370 385 392 407 > 414 > [39] 429 436 451 458 473 480 495 502 517 524 539 546 561 568 583 590 605 612 > 627 > [58] 634 649 656 671 678 693 700 715 722 737 744 759 766 781 788 803 810 825 > 832 > [77] 847 854 869 876 891 898 901 > > 'find', by default, does not following symbolic links. > > % find dir > dir > dir/subdir > dir/subdir/IMPORTANT_2 > dir/subdir/linkToUpperDir > dir/IMPORTANT_1 > > The -L option makes it follow them, but it won't follow loops: > > % find -L dir > dir > dir/subdir > dir/subdir/IMPORTANT_2 > find: File system loop detected; `dir/subdir/linkToUpperDir' is part of the > same file system loop as `dir'. > dir/IMPORTANT_1 > > Bill Dunlap > Spotfire, TIBCO Software > wdunlap tibco.com > > >> -----Original Message----- >> From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On >> Behalf >> Of William Dunlap >> Sent: Friday, September 27, 2013 12:56 PM >> To: Jonathan Greenberg; r-help >> Subject: Re: [R] Error: C stack usage is too close to the limit when using >> list.files() >> >> Do you have some symbolic links that make loops in your file system? >> list.files() has problems with such loops and find does not. E.g., on a >> Linux box: >> >> % cd /tmp >> % mkdir dir dir/subdir >> % cd dir/subdir >> % ln -s ../../dir linkToUpperDir >> % cd /tmp >> % R --quiet >> > list.files("dir", recursive=TRUE, full=TRUE) >> [1] >> "dir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToU >> pperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkT >> oUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/li >> nkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdi >> r/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/su >> bdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir >> /subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpper >> Dir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUp >> perDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkTo >> UpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/lin >> kToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir" >> > system("find dir") >> dir >> dir/subdir >> dir/subdir/linkToUpperDir >> >> Bill Dunlap >> Spotfire, TIBCO Software >> wdunlap tibco.com >> >> >> > -----Original Message----- >> > From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] >> > On Behalf >> > Of Jonathan Greenberg >> > Sent: Friday, September 27, 2013 12:13 PM >> > To: r-help >> > Subject: [R] Error: C stack usage is too close to the limit when using >> > list.files() >> > >> > R-helpers: >> > >> > I'm running a file search on my entire drive (Mac OS X) using: >> > >> > files_found <- >> > list.files(dir="/",pattern=somepattern,recursive=TRUE,full.names=TRUE) >> > where somepattern is a search pattern (which I have confirmed via a >> > unix "find / -name somepattern" only returns ~ 3 results). >> > >> > I keep getting an error: >> > >> > Error: C stack usage is too close to the limit >> > >> > when running this command. Any ideas on 1) how to fix this or 2) if >> > there is an alternative to using list.files() to accomplish this >> > search without resorting to an external package? >> > >> > Cheers! >> > >> > --jonathan >> > >> > >> > -- >> > Jonathan A. Greenberg, PhD >> > Assistant Professor >> > Global Environmental Analysis and Remote Sensing (GEARS) Laboratory >> > Department of Geography and Geographic Information Science >> > University of Illinois at Urbana-Champaign >> > 259 Computing Applications Building, MC-150 >> > 605 East Springfield Avenue >> > Champaign, IL 61820-6371 >> > Phone: 217-300-1924 >> > http://www.geog.illinois.edu/~jgrn/ >> > AIM: jgrn307, MSN: jgrn...@hotmail.com, Gchat: jgrn307, Skype: jgrn3007 >> > >> > ______________________________________________ >> > R-help@r-project.org mailing list >> > https://stat.ethz.ch/mailman/listinfo/r-help >> > PLEASE do read the posting guide >> > http://www.R-project.org/posting-guide.html >> > and provide commented, minimal, self-contained, reproducible code. >> >> ______________________________________________ >> R-help@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. -- Jonathan A. Greenberg, PhD Assistant Professor Global Environmental Analysis and Remote Sensing (GEARS) Laboratory Department of Geography and Geographic Information Science University of Illinois at Urbana-Champaign 259 Computing Applications Building, MC-150 605 East Springfield Avenue Champaign, IL 61820-6371 Phone: 217-300-1924 http://www.geog.illinois.edu/~jgrn/ AIM: jgrn307, MSN: jgrn...@hotmail.com, Gchat: jgrn307, Skype: jgrn3007 ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.