Thank you very much! I suspected that I using the agents in wrong way. I
ran your code and it worked great.
Also I little changed your example to a worker keep only the last task and
run it on very large number of tasks (I had problem only on huge
collections):
(def tasks (ref (into [] (range 1 10000000))))
(defn get-task
[tasks]
(dosync
(let [task (first @tasks)]
(alter tasks rest)
task)))
(defn worker-loop
[]
(loop [last-task nil]
(if-let [task (get-task tasks)]
(recur task)
last-task)))
(defn create-workers
[n & options]
(vec (repeatedly n (fn []
(future
(worker-loop))))))
(def workers
(create-workers 100))
(apply max (map deref workers))
This code worked as expected and returned 9999999.
But when I replace
(def tasks (ref (into [] (range 1 10000000))))
to
(def tasks (ref (range 1 10000000)))
the problem arose again and this code returnined 1734656.
So I think the problem is the lazy collection in the ref.
вторник, 23 августа 2016 г., 20:32:37 UTC+7 пользователь
[email protected] написал:
>
> Here is working example code demonstrating how you might do this without
> agents:
>
> (in-ns 'user)
>
> (def tasks (ref (into clojure.lang.PersistentQueue/EMPTY (range 1 1000))))
>
> (defn get-task
> [tasks]
> (dosync
> (let [task (first @tasks)]
> (alter tasks pop)
> task)))
>
> (defn worker-loop
> []
> (loop [completed-tasks []]
> (if-let [task (get-task tasks)]
> (recur (conj completed-tasks task))
> completed-tasks)))
>
> (defn create-workers
> [n & options]
> (vec (repeatedly n (fn []
> (future
> (worker-loop))))))
>
> (def workers
> (create-workers 100))
>
> (defn worker-test
> [xs]
> (= (set (mapcat deref workers))
> (set xs)))
>
> (worker-test (range 1 1000))
>
> On Tuesday, August 23, 2016 at 9:09:31 AM UTC-4, [email protected]
> wrote:
>>
>> I haven't run your code yet, but it's bad form to use Clojure's reference
>> types inside other reference types. They should store persistent data
>> structures to really make any sense in a concurrent context.
>>
>> On Tuesday, August 23, 2016 at 8:22:00 AM UTC-4, Sergei Koledov wrote:
>>>
>>> Hello,
>>>
>>> I had a problem when I run the following code:
>>>
>>> (defn get-task [tasks]
>>> (dosync
>>> (let [task (first @tasks)]
>>> (alter tasks rest)
>>> task)))
>>>
>>> (defn worker [& {:keys [tasks]}]
>>> (agent {:tasks tasks}))
>>>
>>> (defn worker-loop [{:keys [tasks] :as state}]
>>> (loop [last-task nil]
>>> (if-let [task (get-task tasks)]
>>> (recur task)
>>> (locking :out (println "Last task: " last-task))))
>>> state)
>>>
>>> (defn create-workers [count & options]
>>> (->> (range 0 count)
>>> (map (fn [_] (apply worker options)))
>>> (into [])))
>>>
>>> (defn start-workers [workers]
>>> (doseq [worker workers] (send-off worker worker-loop)))
>>>
>>> (def tasks (ref (range 1 10000000)))
>>>
>>> (def workers (create-workers 100 :tasks tasks))
>>>
>>> (start-workers workers)
>>> (apply await workers)
>>>
>>> Description: I have several agents (100 in my case). Each agent running
>>> in a separate thread. All agents share the one ref with the collection of
>>> tasks (range of longs in my case). Each agent get tasks from the collection
>>> (in transaction) one by one until the collection becomes empty and then
>>> prints the last task which it handle. However, when I run this code it
>>> looks like the collection of tasks suddenly becomes empty and workers
>>> handle only portion of all tasks (average 25-40% of all number).
>>>
>>> This code behave as I expected, when I create only one agent or use
>>> explicit locking in get-task function:
>>>
>>> (defn get-task [tasks]
>>> (locking :lock
>>> (dosync
>>> (let [task (first @tasks)]
>>> (alter tasks rest)
>>> task))))
>>>
>>> I run this code on the Clojure 1.8.0
>>> java version "1.8.0_91"
>>> Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
>>> Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)
>>>
>>> Can anyone tell me, what am I doing wrong, or it really looks like a bug
>>> in the clojure STM?
>>> I already asked this question on stackoverflow.com (
>>> http://stackoverflow.com/questions/39054911/strange-behavior-of-clojure-ref),
>>>
>>> but so far nobody has been able to help me.
>>>
>>> P.S. Sorry for my english skill.
>>>
>>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
