First of al let me say that pmap is NOT useless at all...I've
encountered cases where I've had to accumulate a collection of 'pmap'-ed
elements (say Strings) before the final task (say 'spit'-ing them) where
having laziness is a big win in terms of memory footprint...On the other
hand there are cases where pmap is not ideal and an eager, more
disciplined approach is appropriate... I wrote this a while back and
I've been using it ever since with no problems...However, I've recently
noticed a patch on jira for changing pmap and implementing it on top of
Executors. I'd like to know what people think about:
1. entirely changing pmap in core (
http://dev.clojure.org/jira/browse/CLJ-862)
2. my pool-map (critique/corrections are of course welcome)...would you
use it?
3. pmap usefulness as it currently stands
I know we're sort of recycling what has already been discussed but my
experiences contradict the dominant opinion that pmap is useless...If
people are curious then I can share a specific use-case where pmap
really shines whereas pool-map needs ridiculous amount of memory to even
finish and start spitting to the disk (pmap will spit as it goes along)!
(defn pool-map
"A saner, more disciplined version of pmap. Not lazy at all.
Don't use if original ordering of 'coll' matters!"
[f coll]
(let [cpus (.. Runtime getRuntime availableProcessors)
exec (Executors/newFixedThreadPool cpus)
pool (ExecutorCompletionService. exec)
futures (for [x coll] (.submit pool #(f x)))]
(try
(doall (for [e futures] (.. pool take get)))
(finally (.shutdown exec)))))
thanks a lot for your time... :)
Jim
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
