On OSX (.availableProcessors (Runtime/getRuntime)) returns 2 in my case.
I've also run into the following thread:
https://groups.google.com/forum/#!topic/clojure/AtA-0LKYe9A
There are two other implementations of CPU bound pmap: pmapall and
pmap-pool. The latter one (by j-g-faustus) actually solved my problem on
OSX.
Attaching complete example for reference:
(import '(java.util.concurrent Executors))
(defn pmap-pool [f coll]
(let [queue (ref coll) ;; shared queue of work units
nthreads (.availableProcessors (Runtime/getRuntime))
pool (Executors/newFixedThreadPool nthreads)
tasks (map (fn [_]
(fn [] ; one task per thread
(let [local-res (atom [])] ;; collect results per
thread to minimize synchronization
(while (seq @queue)
;; queue may be emptied between 'while'
;; and 'dosync'.
(when-let [wu (dosync
;; grab work unit, update
queue
(when-let [w (first @queue)]
(alter queue next)
w))]
(swap! local-res conj (f wu))))
local-res)))
(range nthreads))
results (doall (map #(deref (.get %)) ;; blocks until completion
(.invokeAll pool tasks))) ;; start all tasks
results (reduce concat results)]
(.shutdown pool)
;; sanity check
(when-not (and (empty? @queue)
(= (count results) (count coll))
(every? #(= % :done) results))
(println "ERROR: queue " (count @queue) " #results" (count results)))
results))
(defn myFunc [input]
(doseq [n (take 10000000 (range))]) ;; some slow dummy process
(println (str "---" input))
)
(time (last (pmap-pool #(myFunc %) (take 6 (range)))))
--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
