Processing elements in a lazy seq in parallel

2009-08-10 Thread Tom Emerson
Hello Clojurians, I want to process approximately 74K XML files that are stored on disk in a series of nested directories, each of which contains upto 1000 files. For example, rootdir 0 file1.xml file2.xml 1 file3.xml file4.xml and so on. file-seq gives

Re: Processing elements in a lazy seq in parallel

2009-08-10 Thread tmountain
You can use agents in combination with the send function which will operate on a fixed size thread pool. I'm sure there are other ways as well, but I've found agents very easy to work with. Travis On Aug 10, 2:18 pm, Tom Emerson tremer...@gmail.com wrote: Hello Clojurians, I want to process

Re: Processing elements in a lazy seq in parallel

2009-08-10 Thread Laurent PETIT
Hi, 2009/8/10 Tom Emerson tremer...@gmail.com Hello Clojurians, I want to process approximately 74K XML files that are stored on disk in a series of nested directories, each of which contains upto 1000 files. For example, rootdir 0 file1.xml file2.xml 1