2010/1/28 free_variation <cane.c...@gmail.com>: > I have an infinite sequence. I'd like to have the sequence be a > source for N parallel worker threads that now and then will show up to > grab a few elements from the sequence, then go off and crunch on the > data. The sequence should be traversed only once per execution of the > software.
Who's in charge here? :) The other way to look at this problem is an executor handing out the tasks (as opposed to threads asking for work). That way there is no need to coordinate between workers. eg: if you had a file which you read as a lazy sequence you could create worker tasks like this: (doseq [d data-seq] (.submit clojure.lang.Agent/pooledExecutor (cast Callable #(foo d)))) pooledExecutor is just a standard java fixed size thread pool based upon the number of processors available, so it will only create X threads at a time. However I believe that submitted jobs are queued so if your seq processing can get too far ahead you would end up with a very full queue. I'm sure there must be a way to limit the submission rate but can't think of it right now maybe someone else will chime in :) Regards, Tim. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
