Re: determining whether state has changed in a thread safe manner

[Sam Aaron]

Hi Nick,

On 25 Jul 2011, at 23:55, cassiel wrote:
 
 Not very practical, but if you want a safe transaction-free operation
 on an atom which returns whether it was changed, you can perhaps hack
 it by embedding the change state into the atom itself:
 
 (def a (atom {:value 45 :changed? false}))
 
 (defn update-and-check-whether-modified?
  [update-fn]
  (:changed?
   (swap! a (fn [{v :value _ :changed?}]
  (let [new-v (update-fn v)]
(if (== v new-v)
  {:value new-v :changed? false}
  {:value new-v :changed? true}))
 
 (update-and-check-whether-modified? (fn [x] (+ x 1)))
 (update-and-check-whether-modified? (fn [x] 70))

Great thinking. 

I suppose an even more general version of this is to replace the key :changed? 
with :old-value in the atom:

(def a (atom {:value 45 :old-val nil}))

(defn update-and-check-whether-modified?
 [update-fn]
 (let [updated (swap! a (fn [{v :value o :old-val}]
  (let [new-v (update-fn v)]
{:value new-v :old-val o})))]
   (= (:old-val updated) (:value updated


However, it seems remarkably kludgy to encode the old value into the contract 
of the atom and *all* update fns when really this could be achieved in a much 
cleaner fashion with a version of swap! that returned a vec of new and old vals:

(defn update-and-check-whether-modified?
  [update-fn]
  (let [[new old] (swap-and-also-return-old! (fn [v] (update-fn v)))]
(= new old)))

Sam

---
http://sam.aaron.name


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: determining whether state has changed in a thread safe manner

[Ken Wesson]

On Tue, Jul 26, 2011 at 4:00 AM, Sam Aaron samaa...@gmail.com wrote:
 However, it seems remarkably kludgy to encode the old value into the contract 
 of the atom and *all* update fns when really this could be achieved in a much 
 cleaner fashion with a version of swap! that returned a vec of new and old 
 vals:

 (defn update-and-check-whether-modified?
  [update-fn]
  (let [[new old] (swap-and-also-return-old! (fn [v] (update-fn v)))]
    (= new old)))

This seems to have been left out:

(defn swap-and-also-return-old! [^clojure.lang.Atom a f]
  (loop []
(let [v @a
  nv (f v)]
  (if (compare-and-set! a v nv)
[nv v]
(recur)

:)

-- 
Protege: What is this seething mass of parentheses?!
Master: Your father's Lisp REPL. This is the language of a true
hacker. Not as clumsy or random as C++; a language for a more
civilized age.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: determining whether state has changed in a thread safe manner

[Sam Aaron]

Hey Ken,

On 26 Jul 2011, at 09:45, Ken Wesson wrote:
 
 This seems to have been left out:
 
 (defn swap-and-also-return-old! [^clojure.lang.Atom a f]
  (loop []
(let [v @a
  nv (f v)]
  (if (compare-and-set! a v nv)
[nv v]
(recur)
 
 :)

Thanks for this :-)

Actually I was just looking at compare-and-set! just now. This solution seems 
nicer than Nick's 'place changed/old-val in atom' but still not particularly 
clean as you have to enter a tight spin loop. With both solutions you have to 
write a certain amount of machinery - when to my mind this is something a 
library/language should do for you as boiler-plate machinery coding can be 
error-prone (and not everyone will code it the same leading to slightly 
different versions with potentially different efficiencies etc.) 

I've also read other people's code that doesn't even use either of these 
approaches, but implements something synonymous with my naive solution I posted 
at the start of this thread. This leads me to believe that not everyone is 
aware that you have to write a certain amount of machinery to make comparing an 
atom's old and new vals thread safe.

The additional 'return old and new vals' versions of swap! and reset! that I 
propose place this machinery at the language level where I believe it belongs. 
However, I'm also super interested to hear other people's thoughts on this 
matter...

Sam

---
http://sam.aaron.name


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: determining whether state has changed in a thread safe manner

[Ken Wesson]

On Tue, Jul 26, 2011 at 4:56 AM, Sam Aaron samaa...@gmail.com wrote:
 Hey Ken,

 Thanks for this :-)

You're welcome.

 Actually I was just looking at compare-and-set! just now. This solution seems 
 nicer than Nick's 'place changed/old-val in atom' but still not particularly 
 clean as you have to enter a tight spin loop.

Actually, the swap! function has the same tight spin loop:

https://github.com/clojure/clojure/blob/f704853751d02faf72bd53be599ee0be6c1da63e/src/clj/clojure/core.clj#L2099
https://github.com/clojure/clojure/blob/f704853751d02faf72bd53be599ee0be6c1da63e/src/jvm/clojure/lang/Atom.java#L33

 The additional 'return old and new vals' versions of swap! and reset! that I 
 propose place this machinery at the language level where I believe it 
 belongs. However, I'm also super interested to hear other people's thoughts 
 on this matter...

Complicating the return value for common functions might be problematic.

I wonder if Clojure could benefit from a secondary value return
concept like in Common Lisp: the secondary value is ignored unless
captured with a special binding form like (let [a/b (foo)] (println a
b)), which doesn't clash with any existing syntax (currently a/b as a
let symbol is plain invalid since only unqualified names are allowed).
Returning it would be a matter of returning the output of some special
form, say (ret2 a b) instead of just a. (Note that this would have to
knock a (and b) out of tail position for recur; sorry.)

Implementing this under the hood is slightly trickier. Returning a
pair object with a usually-nil right cell for every function return
under the hood would be expensive and wasteful. Ideally existing
functions have unchanged calling convention.

Instead, I'd propose adding PairObj, PairLong, and PairDouble classes
in clojure.lang with public final fields left and right and a
constructor of two arguments to set them -- right is always Object and
left is Object, long, or double, respectively, along with the first
constructor argument's type. AFn is augmented with another 20 invoke
methods, pairInvoke rather than just invoke, which are implemented to
each call the corresponding regular invoke method and wrap its return
value as follows: new Pair(invoke(...),null);. IFn also grows to
specify these added methods. There's also a pairApplyTo method,
implemented in AFn to do what applyTo does only ending up calling a
pairInvoke method rather than an invoke method.

The compiler emits only the same invoke methods it currently does for
normal functions. Functions with a (ret2 ...) form inside, on the
other hand, get overrides for the pairInvoke methods corresponding to
their arities. The invoke implementations are compiled with all
instances of (ret2 a b) in tail position (ret2, like recur, being
disallowed outside of tail position) replaced by a. The pairInvoke
implementations are corresponding ones with all instances of (ret2 a
b) turned into (Pair. a b), and all outermost tail-position forms x
that aren't (ret2 ...) forms and don't have nested (ret2 ...) forms
wrapped in (Pair. x nil).

The special destructuring binding form is handled by the compiler (so
is available in let* and loop*) to call a pairInvoke method of the
function call form that is the right hand side of the assignment (and
it must be a function call form) and then destructure the pair.

Only the primary return value can be primitive.

Note: ret2 is only a provisional name; someone may well be able to
think of a better one and/or one that's less likely to collide with
other code. Returning more than 2 things should probably not get extra
support; just use a vector and destructure the second thing. (That
suggests allowing (let [a/[b c  d] (foo quux)]) binding forms. That
would require reader support. Or another syntax than a/b for binding
auxiliary return values.)

Note 2: binding auxiliary return values in parameter lists is
problematic (e.g. (defn foo [a/b c] ...) ... (foo
(bar-which-uses-ret2) 42)). The problem is that when foo is called,
the compiler won't know whether to use invoke or pairInvoke on
bar-which-uses-ret2, since it won't in general know whether foo's
first argument is going to be destructured in this way. Defensively
always using pairInvoke and dealing with that in parameter binding
would be inefficient for nearly all function calls. This should
therefore not be attempted; (defn foo [a b c] ...) ... (let [a/b
(bar-which-uses-ret2)] (foo a b 42)) will have to be good enough.

-- 
Protege: What is this seething mass of parentheses?!
Master: Your father's Lisp REPL. This is the language of a true
hacker. Not as clumsy or random as C++; a language for a more
civilized age.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to

Re: determining whether state has changed in a thread safe manner

[Meikel Brandmeyer]

Hi Sam,

an another hardcore solution:

(defn update!
  [a f  args]
  (let [updated? (promise)
watch (fn [k a o n] (remove-watch a k) (deliver updated? (not= o 
n)))]
(add-watch a (Object.) watch)
(apply swap! a f args)
@updated?))

But...

Am Dienstag, 26. Juli 2011 10:56:25 UTC+2 schrieb Sam Aaron:

Actually I was just looking at compare-and-set! just now. This solution 
 seems nicer than Nick's 'place changed/old-val in atom' but still not 
 particularly clean as you have to enter a tight spin loop.


That's what swap! does anyway.
 

 With both solutions you have to write a certain amount of machinery - when 
 to my mind this is something a library/language should do for you as 
 boiler-plate machinery coding can be error-prone (and not everyone will code 
 it the same leading to slightly different versions with potentially 
 different efficiencies etc.) 

 I've also read other people's code that doesn't even use either of these 
 approaches, but implements something synonymous with my naive solution I 
 posted at the start of this thread. This leads me to believe that not 
 everyone is aware that you have to write a certain amount of machinery to 
 make comparing an atom's old and new vals thread safe.

 The additional 'return old and new vals' versions of swap! and reset! that 
 I propose place this machinery at the language level where I believe it 
 belongs. However, I'm also super interested to hear other people's thoughts 
 on this matter...

I'm still a bit unsure of why you need that. Maybe the fact, that you have 
to know the previous value (or at least whether the previous value was the 
same as the current one) is really part of your state? And should go into 
your domain value explicitly? The fact that you need special implementations 
for the update function for the different reference types, because the 
handling is intermingled with the actual implementation details of the 
reference types, somehow rings a bell with me. I can't explain this. Just 
some bad feeling in my stomach. ... Or maybe this is just reflux. YMMV.

Sincerely
Meikel

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Aw: Re: determining whether state has changed in a thread safe manner

[Meikel Brandmeyer]

Hi again,

Am Dienstag, 26. Juli 2011 11:35:08 UTC+2 schrieb Meikel Brandmeyer:

 (defn update!
   [a f  args]
   (let [updated? (promise)
 watch (fn [k a o n] (remove-watch a k) (deliver updated? (not= o 
 n)))]
 (add-watch a (Object.) watch)
 (apply swap! a f args)
 @updated?))


This should probably be:

(defn update!
  [a f  args]
  (let [result (promise)
watch (fn [k a o n] (remove-watch a k) (deliver result [n (not= o 
n)]))]
(add-watch a (Object.) watch)
(apply swap! a f args)
@result))

Anyway...

Sincerely
Meikel

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: determining whether state has changed in a thread safe manner

[Meikel Brandmeyer]

Hi,

Am 25.07.2011 um 23:58 schrieb Sam Aaron:

 Sorry, I should have been more specific. The callback-based watchers are 
 cool, but I don't believe they specifically address my problem (which I don't 
 believe I sufficiently explained from the outset).
 
 Hopefully this is a more succinct and specific description. Is there a way to 
 write the following in a thread safe manner without using a transaction:
 
 (def a (ref 1))
 
 (defn update-and-check-whether-modified?
 [update-fn]
 (let [changed? (dosync
 (let [old-a @a
   new-a (alter a update-fn)]
   (not= old-a new-a)))]
   changed?))

This will work for a ref. For an atom, you'll have to roll your own swap! 
variant.

(defn swap-and-check!
 [a f  args]
 (loop []
   (let [x @a
 v (apply f x args)]
 (if (compare-and-set! a x v)
   (not= x v)
   (recur)

Sincerely
Meikel

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: determining whether state has changed in a thread safe manner

[Sam Aaron]

On 25 Jul 2011, at 21:45, Sam Aaron wrote:
 
 (defn update
  []
  (let [changed? (dosync
  (let [old-a @a
new-a (ref-set a (new-val))]
(= old-a new-a)))]
(when changed? (changed-fn

Clearly I meant (not (= old-a new-a)) :-)

Sam

---
http://sam.aaron.name

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: determining whether state has changed in a thread safe manner

[Meikel Brandmeyer]

Hi,

you want a watch.

(def a (atom 0))
(add-watch a ::your-id (fn [_your-id _a old-val new-val] (when (not= old-val 
new-val) (println New value: new-val
(swap! a inc)
(reset! a 1)
(swap! a inc)

Sincerely
Meikel

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: determining whether state has changed in a thread safe manner

[Sam Aaron]

Hi Meikel,

On 25 Jul 2011, at 21:51, Meikel Brandmeyer wrote:
 
 you want a watch.
 
 (def a (atom 0))
 (add-watch a ::your-id (fn [_your-id _a old-val new-val] (when (not= old-val 
 new-val) (println New value: new-val
 (swap! a inc)
 (reset! a 1)
 (swap! a inc)
 

That's cool to know - thanks.

So what if I would like #'update to return a boolean representing whether the 
state has changed rather than call #'changed-fn internally? (which is actually 
what i'm trying to do, I just refactored the example to be a lot simpler and 
hopefully clearer).

Sam

---
http://sma.aaron.name

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: determining whether state has changed in a thread safe manner

[Meikel Brandmeyer]

Hi,

Am 25.07.2011 um 23:12 schrieb Sam Aaron:

 So what if I would like #'update to return a boolean representing whether the 
 state has changed rather than call #'changed-fn internally? (which is 
 actually what i'm trying to do, I just refactored the example to be a lot 
 simpler and hopefully clearer).

Since this is callback based, you can't return a value. Do you want more 
something like a polling solution? Then you'll have to roll your own with an 
atom and a clojure.lang.PersistentQueue.

Sincerely
Meikel

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: determining whether state has changed in a thread safe manner

[Sam Aaron]

Hi Meikel,

On 25 Jul 2011, at 22:46, Meikel Brandmeyer wrote:

 Am 25.07.2011 um 23:12 schrieb Sam Aaron:
 
 Since this is callback based, you can't return a value. Do you want more 
 something like a polling solution? Then you'll have to roll your own with an 
 atom and a clojure.lang.PersistentQueue.

Sorry, I should have been more specific. The callback-based watchers are cool, 
but I don't believe they specifically address my problem (which I don't believe 
I sufficiently explained from the outset).

Hopefully this is a more succinct and specific description. Is there a way to 
write the following in a thread safe manner without using a transaction:

(def a (ref 1))

(defn update-and-check-whether-modified?
  [update-fn]
  (let [changed? (dosync
  (let [old-a @a
new-a (alter a update-fn)]
(not= old-a new-a)))]
changed?))


Sam

---
http://sam.aaron.name


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: determining whether state has changed in a thread safe manner

[cassiel]

Hi Sam,

A nice late night exercise...

Not very practical, but if you want a safe transaction-free operation
on an atom which returns whether it was changed, you can perhaps hack
it by embedding the change state into the atom itself:

(def a (atom {:value 45 :changed? false}))

(defn update-and-check-whether-modified?
  [update-fn]
  (:changed?
   (swap! a (fn [{v :value _ :changed?}]
  (let [new-v (update-fn v)]
(if (== v new-v)
  {:value new-v :changed? false}
  {:value new-v :changed? true}))

(update-and-check-whether-modified? (fn [x] (+ x 1)))
(update-and-check-whether-modified? (fn [x] 70))

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: determining whether state has changed in a thread safe manner

[cassiel]

Of course, once posted, I realised the conditional could be
eliminated:

(defn update-and-check-whether-modified?
  [update-fn]
  (:changed?
   (swap! a (fn [{v :value _ :changed?}]
  (let [new-v (update-fn v)]
{:value new-v :changed? (not= v new-v)})

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en