Re: determining whether state has changed in a thread safe manner
Hi Nick, On 25 Jul 2011, at 23:55, cassiel wrote: Not very practical, but if you want a safe transaction-free operation on an atom which returns whether it was changed, you can perhaps hack it by embedding the change state into the atom itself: (def a (atom {:value 45 :changed? false})) (defn update-and-check-whether-modified? [update-fn] (:changed? (swap! a (fn [{v :value _ :changed?}] (let [new-v (update-fn v)] (if (== v new-v) {:value new-v :changed? false} {:value new-v :changed? true})) (update-and-check-whether-modified? (fn [x] (+ x 1))) (update-and-check-whether-modified? (fn [x] 70)) Great thinking. I suppose an even more general version of this is to replace the key :changed? with :old-value in the atom: (def a (atom {:value 45 :old-val nil})) (defn update-and-check-whether-modified? [update-fn] (let [updated (swap! a (fn [{v :value o :old-val}] (let [new-v (update-fn v)] {:value new-v :old-val o})))] (= (:old-val updated) (:value updated However, it seems remarkably kludgy to encode the old value into the contract of the atom and *all* update fns when really this could be achieved in a much cleaner fashion with a version of swap! that returned a vec of new and old vals: (defn update-and-check-whether-modified? [update-fn] (let [[new old] (swap-and-also-return-old! (fn [v] (update-fn v)))] (= new old))) Sam --- http://sam.aaron.name -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: determining whether state has changed in a thread safe manner
On Tue, Jul 26, 2011 at 4:00 AM, Sam Aaron samaa...@gmail.com wrote: However, it seems remarkably kludgy to encode the old value into the contract of the atom and *all* update fns when really this could be achieved in a much cleaner fashion with a version of swap! that returned a vec of new and old vals: (defn update-and-check-whether-modified? [update-fn] (let [[new old] (swap-and-also-return-old! (fn [v] (update-fn v)))] (= new old))) This seems to have been left out: (defn swap-and-also-return-old! [^clojure.lang.Atom a f] (loop [] (let [v @a nv (f v)] (if (compare-and-set! a v nv) [nv v] (recur) :) -- Protege: What is this seething mass of parentheses?! Master: Your father's Lisp REPL. This is the language of a true hacker. Not as clumsy or random as C++; a language for a more civilized age. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: determining whether state has changed in a thread safe manner
Hey Ken, On 26 Jul 2011, at 09:45, Ken Wesson wrote: This seems to have been left out: (defn swap-and-also-return-old! [^clojure.lang.Atom a f] (loop [] (let [v @a nv (f v)] (if (compare-and-set! a v nv) [nv v] (recur) :) Thanks for this :-) Actually I was just looking at compare-and-set! just now. This solution seems nicer than Nick's 'place changed/old-val in atom' but still not particularly clean as you have to enter a tight spin loop. With both solutions you have to write a certain amount of machinery - when to my mind this is something a library/language should do for you as boiler-plate machinery coding can be error-prone (and not everyone will code it the same leading to slightly different versions with potentially different efficiencies etc.) I've also read other people's code that doesn't even use either of these approaches, but implements something synonymous with my naive solution I posted at the start of this thread. This leads me to believe that not everyone is aware that you have to write a certain amount of machinery to make comparing an atom's old and new vals thread safe. The additional 'return old and new vals' versions of swap! and reset! that I propose place this machinery at the language level where I believe it belongs. However, I'm also super interested to hear other people's thoughts on this matter... Sam --- http://sam.aaron.name -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: determining whether state has changed in a thread safe manner
On Tue, Jul 26, 2011 at 4:56 AM, Sam Aaron samaa...@gmail.com wrote: Hey Ken, Thanks for this :-) You're welcome. Actually I was just looking at compare-and-set! just now. This solution seems nicer than Nick's 'place changed/old-val in atom' but still not particularly clean as you have to enter a tight spin loop. Actually, the swap! function has the same tight spin loop: https://github.com/clojure/clojure/blob/f704853751d02faf72bd53be599ee0be6c1da63e/src/clj/clojure/core.clj#L2099 https://github.com/clojure/clojure/blob/f704853751d02faf72bd53be599ee0be6c1da63e/src/jvm/clojure/lang/Atom.java#L33 The additional 'return old and new vals' versions of swap! and reset! that I propose place this machinery at the language level where I believe it belongs. However, I'm also super interested to hear other people's thoughts on this matter... Complicating the return value for common functions might be problematic. I wonder if Clojure could benefit from a secondary value return concept like in Common Lisp: the secondary value is ignored unless captured with a special binding form like (let [a/b (foo)] (println a b)), which doesn't clash with any existing syntax (currently a/b as a let symbol is plain invalid since only unqualified names are allowed). Returning it would be a matter of returning the output of some special form, say (ret2 a b) instead of just a. (Note that this would have to knock a (and b) out of tail position for recur; sorry.) Implementing this under the hood is slightly trickier. Returning a pair object with a usually-nil right cell for every function return under the hood would be expensive and wasteful. Ideally existing functions have unchanged calling convention. Instead, I'd propose adding PairObj, PairLong, and PairDouble classes in clojure.lang with public final fields left and right and a constructor of two arguments to set them -- right is always Object and left is Object, long, or double, respectively, along with the first constructor argument's type. AFn is augmented with another 20 invoke methods, pairInvoke rather than just invoke, which are implemented to each call the corresponding regular invoke method and wrap its return value as follows: new Pair(invoke(...),null);. IFn also grows to specify these added methods. There's also a pairApplyTo method, implemented in AFn to do what applyTo does only ending up calling a pairInvoke method rather than an invoke method. The compiler emits only the same invoke methods it currently does for normal functions. Functions with a (ret2 ...) form inside, on the other hand, get overrides for the pairInvoke methods corresponding to their arities. The invoke implementations are compiled with all instances of (ret2 a b) in tail position (ret2, like recur, being disallowed outside of tail position) replaced by a. The pairInvoke implementations are corresponding ones with all instances of (ret2 a b) turned into (Pair. a b), and all outermost tail-position forms x that aren't (ret2 ...) forms and don't have nested (ret2 ...) forms wrapped in (Pair. x nil). The special destructuring binding form is handled by the compiler (so is available in let* and loop*) to call a pairInvoke method of the function call form that is the right hand side of the assignment (and it must be a function call form) and then destructure the pair. Only the primary return value can be primitive. Note: ret2 is only a provisional name; someone may well be able to think of a better one and/or one that's less likely to collide with other code. Returning more than 2 things should probably not get extra support; just use a vector and destructure the second thing. (That suggests allowing (let [a/[b c d] (foo quux)]) binding forms. That would require reader support. Or another syntax than a/b for binding auxiliary return values.) Note 2: binding auxiliary return values in parameter lists is problematic (e.g. (defn foo [a/b c] ...) ... (foo (bar-which-uses-ret2) 42)). The problem is that when foo is called, the compiler won't know whether to use invoke or pairInvoke on bar-which-uses-ret2, since it won't in general know whether foo's first argument is going to be destructured in this way. Defensively always using pairInvoke and dealing with that in parameter binding would be inefficient for nearly all function calls. This should therefore not be attempted; (defn foo [a b c] ...) ... (let [a/b (bar-which-uses-ret2)] (foo a b 42)) will have to be good enough. -- Protege: What is this seething mass of parentheses?! Master: Your father's Lisp REPL. This is the language of a true hacker. Not as clumsy or random as C++; a language for a more civilized age. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to
Re: determining whether state has changed in a thread safe manner
Hi Sam, an another hardcore solution: (defn update! [a f args] (let [updated? (promise) watch (fn [k a o n] (remove-watch a k) (deliver updated? (not= o n)))] (add-watch a (Object.) watch) (apply swap! a f args) @updated?)) But... Am Dienstag, 26. Juli 2011 10:56:25 UTC+2 schrieb Sam Aaron: Actually I was just looking at compare-and-set! just now. This solution seems nicer than Nick's 'place changed/old-val in atom' but still not particularly clean as you have to enter a tight spin loop. That's what swap! does anyway. With both solutions you have to write a certain amount of machinery - when to my mind this is something a library/language should do for you as boiler-plate machinery coding can be error-prone (and not everyone will code it the same leading to slightly different versions with potentially different efficiencies etc.) I've also read other people's code that doesn't even use either of these approaches, but implements something synonymous with my naive solution I posted at the start of this thread. This leads me to believe that not everyone is aware that you have to write a certain amount of machinery to make comparing an atom's old and new vals thread safe. The additional 'return old and new vals' versions of swap! and reset! that I propose place this machinery at the language level where I believe it belongs. However, I'm also super interested to hear other people's thoughts on this matter... I'm still a bit unsure of why you need that. Maybe the fact, that you have to know the previous value (or at least whether the previous value was the same as the current one) is really part of your state? And should go into your domain value explicitly? The fact that you need special implementations for the update function for the different reference types, because the handling is intermingled with the actual implementation details of the reference types, somehow rings a bell with me. I can't explain this. Just some bad feeling in my stomach. ... Or maybe this is just reflux. YMMV. Sincerely Meikel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Aw: Re: determining whether state has changed in a thread safe manner
Hi again, Am Dienstag, 26. Juli 2011 11:35:08 UTC+2 schrieb Meikel Brandmeyer: (defn update! [a f args] (let [updated? (promise) watch (fn [k a o n] (remove-watch a k) (deliver updated? (not= o n)))] (add-watch a (Object.) watch) (apply swap! a f args) @updated?)) This should probably be: (defn update! [a f args] (let [result (promise) watch (fn [k a o n] (remove-watch a k) (deliver result [n (not= o n)]))] (add-watch a (Object.) watch) (apply swap! a f args) @result)) Anyway... Sincerely Meikel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: determining whether state has changed in a thread safe manner
Hi, Am 25.07.2011 um 23:58 schrieb Sam Aaron: Sorry, I should have been more specific. The callback-based watchers are cool, but I don't believe they specifically address my problem (which I don't believe I sufficiently explained from the outset). Hopefully this is a more succinct and specific description. Is there a way to write the following in a thread safe manner without using a transaction: (def a (ref 1)) (defn update-and-check-whether-modified? [update-fn] (let [changed? (dosync (let [old-a @a new-a (alter a update-fn)] (not= old-a new-a)))] changed?)) This will work for a ref. For an atom, you'll have to roll your own swap! variant. (defn swap-and-check! [a f args] (loop [] (let [x @a v (apply f x args)] (if (compare-and-set! a x v) (not= x v) (recur) Sincerely Meikel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: determining whether state has changed in a thread safe manner
On 25 Jul 2011, at 21:45, Sam Aaron wrote: (defn update [] (let [changed? (dosync (let [old-a @a new-a (ref-set a (new-val))] (= old-a new-a)))] (when changed? (changed-fn Clearly I meant (not (= old-a new-a)) :-) Sam --- http://sam.aaron.name -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: determining whether state has changed in a thread safe manner
Hi, you want a watch. (def a (atom 0)) (add-watch a ::your-id (fn [_your-id _a old-val new-val] (when (not= old-val new-val) (println New value: new-val (swap! a inc) (reset! a 1) (swap! a inc) Sincerely Meikel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: determining whether state has changed in a thread safe manner
Hi Meikel, On 25 Jul 2011, at 21:51, Meikel Brandmeyer wrote: you want a watch. (def a (atom 0)) (add-watch a ::your-id (fn [_your-id _a old-val new-val] (when (not= old-val new-val) (println New value: new-val (swap! a inc) (reset! a 1) (swap! a inc) That's cool to know - thanks. So what if I would like #'update to return a boolean representing whether the state has changed rather than call #'changed-fn internally? (which is actually what i'm trying to do, I just refactored the example to be a lot simpler and hopefully clearer). Sam --- http://sma.aaron.name -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: determining whether state has changed in a thread safe manner
Hi, Am 25.07.2011 um 23:12 schrieb Sam Aaron: So what if I would like #'update to return a boolean representing whether the state has changed rather than call #'changed-fn internally? (which is actually what i'm trying to do, I just refactored the example to be a lot simpler and hopefully clearer). Since this is callback based, you can't return a value. Do you want more something like a polling solution? Then you'll have to roll your own with an atom and a clojure.lang.PersistentQueue. Sincerely Meikel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: determining whether state has changed in a thread safe manner
Hi Meikel, On 25 Jul 2011, at 22:46, Meikel Brandmeyer wrote: Am 25.07.2011 um 23:12 schrieb Sam Aaron: Since this is callback based, you can't return a value. Do you want more something like a polling solution? Then you'll have to roll your own with an atom and a clojure.lang.PersistentQueue. Sorry, I should have been more specific. The callback-based watchers are cool, but I don't believe they specifically address my problem (which I don't believe I sufficiently explained from the outset). Hopefully this is a more succinct and specific description. Is there a way to write the following in a thread safe manner without using a transaction: (def a (ref 1)) (defn update-and-check-whether-modified? [update-fn] (let [changed? (dosync (let [old-a @a new-a (alter a update-fn)] (not= old-a new-a)))] changed?)) Sam --- http://sam.aaron.name -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: determining whether state has changed in a thread safe manner
Hi Sam, A nice late night exercise... Not very practical, but if you want a safe transaction-free operation on an atom which returns whether it was changed, you can perhaps hack it by embedding the change state into the atom itself: (def a (atom {:value 45 :changed? false})) (defn update-and-check-whether-modified? [update-fn] (:changed? (swap! a (fn [{v :value _ :changed?}] (let [new-v (update-fn v)] (if (== v new-v) {:value new-v :changed? false} {:value new-v :changed? true})) (update-and-check-whether-modified? (fn [x] (+ x 1))) (update-and-check-whether-modified? (fn [x] 70)) -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: determining whether state has changed in a thread safe manner
Of course, once posted, I realised the conditional could be eliminated: (defn update-and-check-whether-modified? [update-fn] (:changed? (swap! a (fn [{v :value _ :changed?}] (let [new-v (update-fn v)] {:value new-v :changed? (not= v new-v)}) -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en