Re: [R] Error: C stack usage is too close to the limit when using list.files()
FYI, in R.utils (= 1.28.4) you can use listDirectory() to control how deep the recursion goes, which would give you protection against your problem, e.g. R.utils::listDirectory(dir, recursive=5L) where recursive=0L is equivalent to recursive=FALSE. Using recursive=TRUE corresponds to recursive=+Inf, that is infinite depth. listDirectory() accepts similar arguments that list.files() does. /Henrik On Sat, Sep 28, 2013 at 5:06 PM, William Dunlap wdun...@tibco.com wrote: The issue is not symbolic links per se, but ones that form loops. Note that you can detect such loops by running 'find -L ...' and looking for the error messages. (find by default does not follow any symbolic links, which can be a problem also.) It is a shortcoming of the current version of list.files(). Bill Dunlap Spotfire, TIBCO Software wdunlap tibco.com -Original Message- From: jgrn...@gmail.com [mailto:jgrn...@gmail.com] On Behalf Of Jonathan Greenberg Sent: Saturday, September 28, 2013 10:51 AM To: William Dunlap Cc: r-help Subject: Re: [R] Error: C stack usage is too close to the limit when using list.files() Thanks all -- ok, so the symbolic link issue is a distinct possibility, but fundamentally doesn't solve the issue since most users will have symbolic links on their machines SOMEPLACE, so a full drive scan will run into these issues -- is list.files calling find, or is it using a different algorithm? This seems like a shortcoming in the list.files algorithm -- is there a better solution (short of a System call, which I'm still not sure will work on Macs without Xcode -- a colleague of mine did NOT have Xcode, and reported not being able to run find from the command line) -- perhaps a different package? --j On Fri, Sep 27, 2013 at 3:08 PM, William Dunlap wdun...@tibco.com wrote: Toss a couple of extra files in there and you will see the output grow exponentially. % touch dir/IMPORTANT_1 dir/subdir/IMPORTANT_2 and in R those two new files cause 82 more strings to appear in list.file's output: nchar(list.files(dir, recursive=TRUE)) [1] 11 18 33 40 55 62 77 84 99 106 121 128 143 150 165 172 187 194 209 [20] 216 231 238 253 260 275 282 297 304 319 326 341 348 363 370 385 392 407 414 [39] 429 436 451 458 473 480 495 502 517 524 539 546 561 568 583 590 605 612 627 [58] 634 649 656 671 678 693 700 715 722 737 744 759 766 781 788 803 810 825 832 [77] 847 854 869 876 891 898 901 'find', by default, does not following symbolic links. % find dir dir dir/subdir dir/subdir/IMPORTANT_2 dir/subdir/linkToUpperDir dir/IMPORTANT_1 The -L option makes it follow them, but it won't follow loops: % find -L dir dir dir/subdir dir/subdir/IMPORTANT_2 find: File system loop detected; `dir/subdir/linkToUpperDir' is part of the same file system loop as `dir'. dir/IMPORTANT_1 Bill Dunlap Spotfire, TIBCO Software wdunlap tibco.com -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of William Dunlap Sent: Friday, September 27, 2013 12:56 PM To: Jonathan Greenberg; r-help Subject: Re: [R] Error: C stack usage is too close to the limit when using list.files() Do you have some symbolic links that make loops in your file system? list.files() has problems with such loops and find does not. E.g., on a Linux box: % cd /tmp % mkdir dir dir/subdir % cd dir/subdir % ln -s ../../dir linkToUpperDir % cd /tmp % R --quiet list.files(dir, recursive=TRUE, full=TRUE) [1] dir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToU pperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkT oUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/li nkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdi r/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/su bdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir /subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpper Dir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUp perDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkTo UpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/lin kToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir system(find dir) dir dir/subdir dir/subdir/linkToUpperDir Bill Dunlap Spotfire, TIBCO Software wdunlap tibco.com -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Jonathan Greenberg Sent: Friday, September 27, 2013 12:13 PM To: r-help Subject: [R] Error: C stack usage is too close to the limit when using
Re: [R] Error: C stack usage is too close to the limit when using list.files()
Thanks all -- ok, so the symbolic link issue is a distinct possibility, but fundamentally doesn't solve the issue since most users will have symbolic links on their machines SOMEPLACE, so a full drive scan will run into these issues -- is list.files calling find, or is it using a different algorithm? This seems like a shortcoming in the list.files algorithm -- is there a better solution (short of a System call, which I'm still not sure will work on Macs without Xcode -- a colleague of mine did NOT have Xcode, and reported not being able to run find from the command line) -- perhaps a different package? --j On Fri, Sep 27, 2013 at 3:08 PM, William Dunlap wdun...@tibco.com wrote: Toss a couple of extra files in there and you will see the output grow exponentially. % touch dir/IMPORTANT_1 dir/subdir/IMPORTANT_2 and in R those two new files cause 82 more strings to appear in list.file's output: nchar(list.files(dir, recursive=TRUE)) [1] 11 18 33 40 55 62 77 84 99 106 121 128 143 150 165 172 187 194 209 [20] 216 231 238 253 260 275 282 297 304 319 326 341 348 363 370 385 392 407 414 [39] 429 436 451 458 473 480 495 502 517 524 539 546 561 568 583 590 605 612 627 [58] 634 649 656 671 678 693 700 715 722 737 744 759 766 781 788 803 810 825 832 [77] 847 854 869 876 891 898 901 'find', by default, does not following symbolic links. % find dir dir dir/subdir dir/subdir/IMPORTANT_2 dir/subdir/linkToUpperDir dir/IMPORTANT_1 The -L option makes it follow them, but it won't follow loops: % find -L dir dir dir/subdir dir/subdir/IMPORTANT_2 find: File system loop detected; `dir/subdir/linkToUpperDir' is part of the same file system loop as `dir'. dir/IMPORTANT_1 Bill Dunlap Spotfire, TIBCO Software wdunlap tibco.com -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of William Dunlap Sent: Friday, September 27, 2013 12:56 PM To: Jonathan Greenberg; r-help Subject: Re: [R] Error: C stack usage is too close to the limit when using list.files() Do you have some symbolic links that make loops in your file system? list.files() has problems with such loops and find does not. E.g., on a Linux box: % cd /tmp % mkdir dir dir/subdir % cd dir/subdir % ln -s ../../dir linkToUpperDir % cd /tmp % R --quiet list.files(dir, recursive=TRUE, full=TRUE) [1] dir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToU pperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkT oUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/li nkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdi r/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/su bdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir /subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpper Dir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUp perDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkTo UpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/lin kToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir system(find dir) dir dir/subdir dir/subdir/linkToUpperDir Bill Dunlap Spotfire, TIBCO Software wdunlap tibco.com -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Jonathan Greenberg Sent: Friday, September 27, 2013 12:13 PM To: r-help Subject: [R] Error: C stack usage is too close to the limit when using list.files() R-helpers: I'm running a file search on my entire drive (Mac OS X) using: files_found - list.files(dir=/,pattern=somepattern,recursive=TRUE,full.names=TRUE) where somepattern is a search pattern (which I have confirmed via a unix find / -name somepattern only returns ~ 3 results). I keep getting an error: Error: C stack usage is too close to the limit when running this command. Any ideas on 1) how to fix this or 2) if there is an alternative to using list.files() to accomplish this search without resorting to an external package? Cheers! --jonathan -- Jonathan A. Greenberg, PhD Assistant Professor Global Environmental Analysis and Remote Sensing (GEARS) Laboratory Department of Geography and Geographic Information Science University of Illinois at Urbana-Champaign 259 Computing Applications Building, MC-150 605 East Springfield Avenue Champaign, IL 61820-6371 Phone: 217-300-1924 http://www.geog.illinois.edu/~jgrn/ AIM: jgrn307, MSN: jgrn...@hotmail.com, Gchat: jgrn307, Skype: jgrn3007 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide
Re: [R] Error: C stack usage is too close to the limit when using list.files()
On 28-09-2013, at 19:51, Jonathan Greenberg j...@illinois.edu wrote: Thanks all -- ok, so the symbolic link issue is a distinct possibility, but fundamentally doesn't solve the issue since most users will have symbolic links on their machines SOMEPLACE, so a full drive scan will run into these issues -- is list.files calling find, or is it using a different algorithm? This seems like a shortcoming in the list.files algorithm -- is there a better solution (short of a System call, which I'm still not sure will work on Macs without Xcode -- a colleague of mine did NOT have Xcode, and reported not being able to run find from the command line) -- perhaps a different package? Since Mac OS X Tiger at least there is a find utility. By doing a search here http://www.opensource.apple.com you can see that there is a find utility since 10.0 (to be found here http://www.opensource.apple.com/source/shell_cmds/shell_cmds-17.1/ ). There must be something wrong with your colleague's setup. Berend __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Error: C stack usage is too close to the limit when using list.files()
The issue is not symbolic links per se, but ones that form loops. Note that you can detect such loops by running 'find -L ...' and looking for the error messages. (find by default does not follow any symbolic links, which can be a problem also.) It is a shortcoming of the current version of list.files(). Bill Dunlap Spotfire, TIBCO Software wdunlap tibco.com -Original Message- From: jgrn...@gmail.com [mailto:jgrn...@gmail.com] On Behalf Of Jonathan Greenberg Sent: Saturday, September 28, 2013 10:51 AM To: William Dunlap Cc: r-help Subject: Re: [R] Error: C stack usage is too close to the limit when using list.files() Thanks all -- ok, so the symbolic link issue is a distinct possibility, but fundamentally doesn't solve the issue since most users will have symbolic links on their machines SOMEPLACE, so a full drive scan will run into these issues -- is list.files calling find, or is it using a different algorithm? This seems like a shortcoming in the list.files algorithm -- is there a better solution (short of a System call, which I'm still not sure will work on Macs without Xcode -- a colleague of mine did NOT have Xcode, and reported not being able to run find from the command line) -- perhaps a different package? --j On Fri, Sep 27, 2013 at 3:08 PM, William Dunlap wdun...@tibco.com wrote: Toss a couple of extra files in there and you will see the output grow exponentially. % touch dir/IMPORTANT_1 dir/subdir/IMPORTANT_2 and in R those two new files cause 82 more strings to appear in list.file's output: nchar(list.files(dir, recursive=TRUE)) [1] 11 18 33 40 55 62 77 84 99 106 121 128 143 150 165 172 187 194 209 [20] 216 231 238 253 260 275 282 297 304 319 326 341 348 363 370 385 392 407 414 [39] 429 436 451 458 473 480 495 502 517 524 539 546 561 568 583 590 605 612 627 [58] 634 649 656 671 678 693 700 715 722 737 744 759 766 781 788 803 810 825 832 [77] 847 854 869 876 891 898 901 'find', by default, does not following symbolic links. % find dir dir dir/subdir dir/subdir/IMPORTANT_2 dir/subdir/linkToUpperDir dir/IMPORTANT_1 The -L option makes it follow them, but it won't follow loops: % find -L dir dir dir/subdir dir/subdir/IMPORTANT_2 find: File system loop detected; `dir/subdir/linkToUpperDir' is part of the same file system loop as `dir'. dir/IMPORTANT_1 Bill Dunlap Spotfire, TIBCO Software wdunlap tibco.com -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of William Dunlap Sent: Friday, September 27, 2013 12:56 PM To: Jonathan Greenberg; r-help Subject: Re: [R] Error: C stack usage is too close to the limit when using list.files() Do you have some symbolic links that make loops in your file system? list.files() has problems with such loops and find does not. E.g., on a Linux box: % cd /tmp % mkdir dir dir/subdir % cd dir/subdir % ln -s ../../dir linkToUpperDir % cd /tmp % R --quiet list.files(dir, recursive=TRUE, full=TRUE) [1] dir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToU pperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkT oUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/li nkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdi r/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/su bdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir /subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpper Dir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUp perDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkTo UpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/lin kToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir system(find dir) dir dir/subdir dir/subdir/linkToUpperDir Bill Dunlap Spotfire, TIBCO Software wdunlap tibco.com -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Jonathan Greenberg Sent: Friday, September 27, 2013 12:13 PM To: r-help Subject: [R] Error: C stack usage is too close to the limit when using list.files() R-helpers: I'm running a file search on my entire drive (Mac OS X) using: files_found - list.files(dir=/,pattern=somepattern,recursive=TRUE,full.names=TRUE) where somepattern is a search pattern (which I have confirmed via a unix find / -name somepattern only returns ~ 3 results). I keep getting an error: Error: C stack usage is too close to the limit when running this command. Any ideas on 1) how to fix this or 2
[R] Error: C stack usage is too close to the limit when using list.files()
R-helpers: I'm running a file search on my entire drive (Mac OS X) using: files_found - list.files(dir=/,pattern=somepattern,recursive=TRUE,full.names=TRUE) where somepattern is a search pattern (which I have confirmed via a unix find / -name somepattern only returns ~ 3 results). I keep getting an error: Error: C stack usage is too close to the limit when running this command. Any ideas on 1) how to fix this or 2) if there is an alternative to using list.files() to accomplish this search without resorting to an external package? Cheers! --jonathan -- Jonathan A. Greenberg, PhD Assistant Professor Global Environmental Analysis and Remote Sensing (GEARS) Laboratory Department of Geography and Geographic Information Science University of Illinois at Urbana-Champaign 259 Computing Applications Building, MC-150 605 East Springfield Avenue Champaign, IL 61820-6371 Phone: 217-300-1924 http://www.geog.illinois.edu/~jgrn/ AIM: jgrn307, MSN: jgrn...@hotmail.com, Gchat: jgrn307, Skype: jgrn3007 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Error: C stack usage is too close to the limit when using list.files()
Jonathan Greenberg jgrn at illinois.edu writes: R-helpers: I'm running a file search on my entire drive (Mac OS X) using: files_found - list.files(dir=/,pattern=somepattern,recursive=TRUE,full.names=TRUE) where somepattern is a search pattern (which I have confirmed via a unix find / -name somepattern only returns ~ 3 results). I keep getting an error: Error: C stack usage is too close to the limit when running this command. Any ideas on 1) how to fix this or 2) if there is an alternative to using list.files() to accomplish this search without resorting to an external package? I assuming that using system(find / -name somepattern) (possibly with intern=TRUE) isn't allowed? (I don't know what you're trying to do, but if you don't need it to work on Windows-without-cygwin, this should work across most Unix variants (although a -print might be required) __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Error: C stack usage is too close to the limit when using list.files()
Ben: I'd like to avoid using that (previous version of my code solved it in that way) -- I would like cross-platform compatibility and I am pretty sure, along with Windows, vanilla Macs don't come with find either unless XCode has been installed. Is the list.files() code itself recursive when using recursive=TRUE (so it has one recursion per bottom-folder)? --j P.S. I recognized that in my initial post I indicated using dir as the parameter -- it should have been path (the error occurred through the correct usage of list.files(path=/,...) That'll teach me not to copy/paste from my code... On Fri, Sep 27, 2013 at 2:36 PM, Ben Bolker bbol...@gmail.com wrote: Jonathan Greenberg jgrn at illinois.edu writes: R-helpers: I'm running a file search on my entire drive (Mac OS X) using: files_found - list.files(dir=/,pattern=somepattern,recursive=TRUE,full.names=TRUE) where somepattern is a search pattern (which I have confirmed via a unix find / -name somepattern only returns ~ 3 results). I keep getting an error: Error: C stack usage is too close to the limit when running this command. Any ideas on 1) how to fix this or 2) if there is an alternative to using list.files() to accomplish this search without resorting to an external package? I assuming that using system(find / -name somepattern) (possibly with intern=TRUE) isn't allowed? (I don't know what you're trying to do, but if you don't need it to work on Windows-without-cygwin, this should work across most Unix variants (although a -print might be required) __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Jonathan A. Greenberg, PhD Assistant Professor Global Environmental Analysis and Remote Sensing (GEARS) Laboratory Department of Geography and Geographic Information Science University of Illinois at Urbana-Champaign 259 Computing Applications Building, MC-150 605 East Springfield Avenue Champaign, IL 61820-6371 Phone: 217-300-1924 http://www.geog.illinois.edu/~jgrn/ AIM: jgrn307, MSN: jgrn...@hotmail.com, Gchat: jgrn307, Skype: jgrn3007 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Error: C stack usage is too close to the limit when using list.files()
Do you have some symbolic links that make loops in your file system? list.files() has problems with such loops and find does not. E.g., on a Linux box: % cd /tmp % mkdir dir dir/subdir % cd dir/subdir % ln -s ../../dir linkToUpperDir % cd /tmp % R --quiet list.files(dir, recursive=TRUE, full=TRUE) [1] dir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir system(find dir) dir dir/subdir dir/subdir/linkToUpperDir Bill Dunlap Spotfire, TIBCO Software wdunlap tibco.com -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Jonathan Greenberg Sent: Friday, September 27, 2013 12:13 PM To: r-help Subject: [R] Error: C stack usage is too close to the limit when using list.files() R-helpers: I'm running a file search on my entire drive (Mac OS X) using: files_found - list.files(dir=/,pattern=somepattern,recursive=TRUE,full.names=TRUE) where somepattern is a search pattern (which I have confirmed via a unix find / -name somepattern only returns ~ 3 results). I keep getting an error: Error: C stack usage is too close to the limit when running this command. Any ideas on 1) how to fix this or 2) if there is an alternative to using list.files() to accomplish this search without resorting to an external package? Cheers! --jonathan -- Jonathan A. Greenberg, PhD Assistant Professor Global Environmental Analysis and Remote Sensing (GEARS) Laboratory Department of Geography and Geographic Information Science University of Illinois at Urbana-Champaign 259 Computing Applications Building, MC-150 605 East Springfield Avenue Champaign, IL 61820-6371 Phone: 217-300-1924 http://www.geog.illinois.edu/~jgrn/ AIM: jgrn307, MSN: jgrn...@hotmail.com, Gchat: jgrn307, Skype: jgrn3007 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Error: C stack usage is too close to the limit when using list.files()
On 27-09-2013, at 21:50, Jonathan Greenberg j...@illinois.edu wrote: Ben: I'd like to avoid using that (previous version of my code solved it in that way) -- I would like cross-platform compatibility and I am pretty sure, along with Windows, vanilla Macs don't come with find either unless XCode has been installed. As far as I can tell Mac OS X provides a find utility (the Unix version). It is not part of Xcode. Berend Is the list.files() code itself recursive when using recursive=TRUE (so it has one recursion per bottom-folder)? --j P.S. I recognized that in my initial post I indicated using dir as the parameter -- it should have been path (the error occurred through the correct usage of list.files(path=/,...) That'll teach me not to copy/paste from my code... On Fri, Sep 27, 2013 at 2:36 PM, Ben Bolker bbol...@gmail.com wrote: Jonathan Greenberg jgrn at illinois.edu writes: R-helpers: I'm running a file search on my entire drive (Mac OS X) using: files_found - list.files(dir=/,pattern=somepattern,recursive=TRUE,full.names=TRUE) where somepattern is a search pattern (which I have confirmed via a unix find / -name somepattern only returns ~ 3 results). I keep getting an error: Error: C stack usage is too close to the limit when running this command. Any ideas on 1) how to fix this or 2) if there is an alternative to using list.files() to accomplish this search without resorting to an external package? I assuming that using system(find / -name somepattern) (possibly with intern=TRUE) isn't allowed? (I don't know what you're trying to do, but if you don't need it to work on Windows-without-cygwin, this should work across most Unix variants (although a -print might be required) __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Jonathan A. Greenberg, PhD Assistant Professor Global Environmental Analysis and Remote Sensing (GEARS) Laboratory Department of Geography and Geographic Information Science University of Illinois at Urbana-Champaign 259 Computing Applications Building, MC-150 605 East Springfield Avenue Champaign, IL 61820-6371 Phone: 217-300-1924 http://www.geog.illinois.edu/~jgrn/ AIM: jgrn307, MSN: jgrn...@hotmail.com, Gchat: jgrn307, Skype: jgrn3007 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Error: C stack usage is too close to the limit when using list.files()
Toss a couple of extra files in there and you will see the output grow exponentially. % touch dir/IMPORTANT_1 dir/subdir/IMPORTANT_2 and in R those two new files cause 82 more strings to appear in list.file's output: nchar(list.files(dir, recursive=TRUE)) [1] 11 18 33 40 55 62 77 84 99 106 121 128 143 150 165 172 187 194 209 [20] 216 231 238 253 260 275 282 297 304 319 326 341 348 363 370 385 392 407 414 [39] 429 436 451 458 473 480 495 502 517 524 539 546 561 568 583 590 605 612 627 [58] 634 649 656 671 678 693 700 715 722 737 744 759 766 781 788 803 810 825 832 [77] 847 854 869 876 891 898 901 'find', by default, does not following symbolic links. % find dir dir dir/subdir dir/subdir/IMPORTANT_2 dir/subdir/linkToUpperDir dir/IMPORTANT_1 The -L option makes it follow them, but it won't follow loops: % find -L dir dir dir/subdir dir/subdir/IMPORTANT_2 find: File system loop detected; `dir/subdir/linkToUpperDir' is part of the same file system loop as `dir'. dir/IMPORTANT_1 Bill Dunlap Spotfire, TIBCO Software wdunlap tibco.com -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of William Dunlap Sent: Friday, September 27, 2013 12:56 PM To: Jonathan Greenberg; r-help Subject: Re: [R] Error: C stack usage is too close to the limit when using list.files() Do you have some symbolic links that make loops in your file system? list.files() has problems with such loops and find does not. E.g., on a Linux box: % cd /tmp % mkdir dir dir/subdir % cd dir/subdir % ln -s ../../dir linkToUpperDir % cd /tmp % R --quiet list.files(dir, recursive=TRUE, full=TRUE) [1] dir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToU pperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkT oUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/li nkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdi r/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/su bdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir /subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpper Dir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUp perDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkTo UpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/lin kToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir system(find dir) dir dir/subdir dir/subdir/linkToUpperDir Bill Dunlap Spotfire, TIBCO Software wdunlap tibco.com -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Jonathan Greenberg Sent: Friday, September 27, 2013 12:13 PM To: r-help Subject: [R] Error: C stack usage is too close to the limit when using list.files() R-helpers: I'm running a file search on my entire drive (Mac OS X) using: files_found - list.files(dir=/,pattern=somepattern,recursive=TRUE,full.names=TRUE) where somepattern is a search pattern (which I have confirmed via a unix find / -name somepattern only returns ~ 3 results). I keep getting an error: Error: C stack usage is too close to the limit when running this command. Any ideas on 1) how to fix this or 2) if there is an alternative to using list.files() to accomplish this search without resorting to an external package? Cheers! --jonathan -- Jonathan A. Greenberg, PhD Assistant Professor Global Environmental Analysis and Remote Sensing (GEARS) Laboratory Department of Geography and Geographic Information Science University of Illinois at Urbana-Champaign 259 Computing Applications Building, MC-150 605 East Springfield Avenue Champaign, IL 61820-6371 Phone: 217-300-1924 http://www.geog.illinois.edu/~jgrn/ AIM: jgrn307, MSN: jgrn...@hotmail.com, Gchat: jgrn307, Skype: jgrn3007 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
