Stackoverflow on a function listing files in a directory recursively

Ramesh Mon, 20 May 2013 13:12:48 -0700

Hi all,

I have the following function to list all the files recursively under a
directory.


(defn list-all-files2 [path]
(let [basepath (java.io.File. path)]
 (loop [origlist [basepath] finallist []]
(if-let [cur (first origlist)]
;This line is not required (if (.isDirectory cur)
 (recur (concat (rest origlist) (.listFiles cur)) (conj finallist
(.getCanonicalPath cur)))
finallist))))


It runs fine, when the number of files are quite less, 10-20.

But when I recurse through even moderately large amount of files, it fails
with stackoverflow! The total number of files are not that much (50114). So
even if each of the filename is 100 chars long, it would roughly take (5M
bytes, which is 5MB). But I get a stack overflow error, and I have no clue.
I think I am doing something wrong with the lazy functions.

Here is the stacktrace:
Exception in thread "main" java.lang.RuntimeException:
java.lang.StackOverflowError
        at clojure.lang.Util.runtimeException(Util.java:165)
        at clojure.lang.Compiler.eval(Compiler.java:6476)
        at clojure.lang.Compiler.eval(Compiler.java:6455)
        at clojure.lang.Compiler.eval(Compiler.java:6431)
        at clojure.core$eval.invoke(core.clj:2795)
        at clojure.main$eval_opt.invoke(main.clj:296)
        at clojure.main$initialize.invoke(main.clj:315)
        at clojure.main$null_opt.invoke(main.clj:348)
        at clojure.main$main.doInvoke(main.clj:426)
        at clojure.lang.RestFn.invoke(RestFn.java:421)
        at clojure.lang.Var.invoke(Var.java:405)
        at clojure.lang.AFn.applyToHelper(AFn.java:163)
        at clojure.lang.Var.applyTo(Var.java:518)
        at clojure.main.main(main.java:37)
Caused by: java.lang.StackOverflowError
        at clojure.core$concat$fn__3528.invoke(core.clj:661)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:60)
        at clojure.lang.RT.seq(RT.java:466)
        at clojure.core$seq.invoke(core.clj:133)
        at clojure.core$concat$fn__3528.invoke(core.clj:661)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:60)
        at clojure.lang.RT.seq(RT.java:466)
        at clojure.core$seq.invoke(core.clj:133)
        at clojure.core$concat$fn__3528.invoke(core.clj:661)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:60)
        at clojure.lang.RT.seq(RT.java:466)
        at clojure.core$seq.invoke(core.clj:133)
        at clojure.core$concat$fn__3528.invoke(core.clj:661)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:60)
        at clojure.lang.RT.seq(RT.java:466)
        at clojure.core$seq.invoke(core.clj:133)
        at clojure.core$concat$fn__3528.invoke(core.clj:661)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:60)
        at clojure.lang.RT.seq(RT.java:466)
        at clojure.core$seq.invoke(core.clj:133)
        at clojure.core$concat$fn__3528.invoke(core.clj:661)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:60)
        at clojure.lang.RT.seq(RT.java:466)
        at clojure.core$seq.invoke(core.clj:133)
        at clojure.core$concat$fn__3528.invoke(core.clj:661)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:60)
        at clojure.lang.RT.seq(RT.java:466)
        at clojure.core$seq.invoke(core.clj:133)
        at clojure.core$concat$fn__3528.invoke(core.clj:661)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:60)
        at clojure.lang.RT.seq(RT.java:466)
        at clojure.core$seq.invoke(core.clj:133)
        at clojure.core$concat$fn__3528.invoke(core.clj:661)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:60)
        at clojure.lang.RT.seq(RT.java:466)
        at clojure.core$seq.invoke(core.clj:133)
        at clojure.core$concat$fn__3528.invoke(core.clj:661)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:60)
        at clojure.lang.RT.seq(RT.java:466)
        at clojure.core$seq.invoke(core.clj:133)
        at clojure.core$concat$fn__3528.invoke(core.clj:661)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:60)
        at clojure.lang.RT.seq(RT.java:466)
        at clojure.core$seq.invoke(core.clj:133)
        at clojure.core$concat$fn__3528.invoke(core.clj:661)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:60)
        at clojure.lang.RT.seq(RT.java:466)
        at clojure.core$seq.invoke(core.clj:133)
        at clojure.core$concat$fn__3528.invoke(core.clj:661)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:60)
        at clojure.lang.RT.seq(RT.java:466)
        at clojure.core$seq.invoke(core.clj:133)
        at clojure.core$concat$fn__3528.invoke(core.clj:661)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:60)
        at clojure.lang.RT.seq(RT.java:466)
        at clojure.core$seq.invoke(core.clj:133)
        at clojure.core$concat$fn__3528.invoke(core.clj:661)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:60)
        at clojure.lang.RT.seq(RT.java:466)
        at clojure.core$seq.invoke(core.clj:133)
        at clojure.core$concat$fn__3528.invoke(core.clj:661)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:60)
        at clojure.lang.RT.seq(RT.java:466)
        at clojure.core$seq.invoke(core.clj:133)
        at clojure.core$concat$fn__3528.invoke(core.clj:661)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:60)
        at clojure.lang.RT.seq(RT.java:466)
        at clojure.core$seq.invoke(core.clj:133)
        at clojure.core$concat$fn__3528.invoke(core.clj:661)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:60)
        at clojure.lang.RT.seq(RT.java:466)
        at clojure.core$seq.invoke(core.clj:133)
        at clojure.core$concat$fn__3528.invoke(core.clj:661)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:60)
        at clojure.lang.RT.seq(RT.java


Thanks,
ramesh

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Stackoverflow on a function listing files in a directory recursively

Reply via email to