Hi Cecil,
On Apr 12, 2014, at 3:18 PM, Cecil Westerhof <[email protected]> wrote:
>
> I just started playing with Clojure a few days ago, so I am a tabula rasa. I
> attached what I have until now. If it can be improved, I like to know it.
I had a look at your code and it's not clear to me what you are trying to
experiment with. Possibly benchmark using futures to run the do-sequential
function concurrently. I'll assume that for the moment and make a couple of
observations...
You are using atoms to store two values (max-diff and max-factor) that are used
and changed in main-function. Did you really intend to share these across all
the calculations? The code you've provided will have concurrency errors as a
result. You might consider writing this as something like (the *completely*
untested and so only good as a sketch):
(def max-diff (ref 0.0))
(def max-factor (ref 0.0))
(defn init [type]
(give-message (format "Start %s" type))
(dosync
(ref-set max-diff 0.0)
(ref-set max-factor 0.0)))
(defn main-function [i diff]
(dosync
(ensure max-diff)
(ensure max-factor)
(when (> diff @max-diff)
(give-message (format "Different for %12d (%e, %e)" i diff (/ diff i)))
(alter max-diff max diff)
(when (> (/ diff i) @max-factor)
(alter max-factor max (/ diff i)))))
The 'ensures' are there because max-diff and max-factor aren't really
independent and some interaction between concurrent threads in the outer when
could lead to some strange situations (like testing for max-diff (or
max-factor) then finding that some other thread ran and changed max-diff (or
max-factor) out from under you.
Your concurrency will likely be restricted to (some subset) of your cores since
there's no IO in there (you're dumping the actual logging off to an agent).
This isn't a bug, just mentioning it, but there's something to think about
below.
There's a problem in your calculation of diff... it's always 0.0 which means
that there's nothing being done in main-function. I changed it to something
arbitrary (and *completely* untested and so only good as a sketch):
(let [v (int (Math/sqrt i))
diff (Math/abs (- (Math/pow v 2) i))]
No idea if that's reasonable, but at least main-function does something now.
With the hacked up calculation of diff, I was able to demonstrate the
concurrency problem in main-function by simply running (the *completely*
untested and so only good as a sketch):
(defn main-function [i diff]
(swap! iterations0 inc)
(when (> diff @max-diff)
(swap! iterations1 inc)
#_(give-message (format "Different for %12d (%e, %e)" i diff (/ diff i)))
(swap! max-diff max diff)
(when (> (/ diff i) @max-factor)
(swap! iterations2 inc)
(swap! max-factor max (/ diff i)))))
and printing the values of iterations* -- with your swap algorithm you get
different values for iterations1 and sometimes iterations2 for successive run.
If you didn't intend to share max-diff and max-factor, then you'd likely be
better off if do-sequential passed local values into main-function, something
like (the *completely* untested and so only good as a sketch):
(defn main-function [[max-diff max-factor] i]
(let [v (int (Math/sqrt i))
diff (Math/abs (- (Math/pow v 2) i))]
(if (> diff max-diff)
(do
#_(give-message (format "Different for %12d (%e, %e)" i diff (/ diff
i)))
[(max max-diff diff) (max max-factor (/ diff i))])
[max-diff max-factor])))
(defn do-sequential [start stop step]
(reduce main-function [0.0 0.0] (range start stop step)))
You've decided to use futures for concurrency. Maybe that's the whole point,
but if you're just trying to get a feel for concurrency in Clojure then there
are some options. I find I directly use futures and promises, at most, rarely
(I can't remember ever using them directly, maybe in the very early days of
Clojure). What I find handy for this sort of thing is something like pmap, and
if you really want this asynchronous then have a look at core.async. Anyway,
with pmap (the *completely* untested and so only good as a sketch):
(defn check-concurrent [number]
(doall
(pmap #(do-sequential % check-until number) (range 1 (inc number)))))
The shutdown-agents call is likely not what you want, you don't need it.
You're using agents for the logging. This gets the IO out of your code being
tested, but there's still a lot of formatting work happening there (not much of
a problem). But you don't know the impact of the agent thread pool on your
benchmark. It's possible that it acts as a throttle. You can avoid this easily
by not logging anything in main-function (I've commented it out in the example
above).
Benchmarking is tricky to do. It's a lot easier to use something like
criterium, kind of like this (I added the *verbose* dynamic variable for fun to
the *completely* untested and so only good as a sketch):
(def ^:dynamic *verbose* true)
(defn give-message [message]
(when *verbose*
(send logger
(fn [_ & [msg]] (println (format "%s: %s"
(. time-format format (new
java.util.Date (now))) msg)))
message)))
(defn run-bench [n]
(println "\n\n\nBench:: threads:" n)
(binding [*verbose* false]
(bench (check-concurrent n bench-until))))
bench-until should be much smaller than your check-until. All you need is for a
real test to be run and criterium will do the rest (so maybe 1001 will work,
but if it's too small and you'll benchmark the wrong thing and introduce
variations in timing you don't want)
Criterium will warm up the JVM, call check-concurrent as many times as
necessary to satisfy some criteria it decides while pre-running the benchmark
(pre-running isn't what they call it... but anyway, if a run takes too long,
criterium will take a long time -- as it is, it takes a minute or two for each
benchmark). When done Criterium will report mean execution time, standard
deviation and lower & upper quintiles. It's a little slow, but you'll not be
wasting your time dealing with strange JVM behaviour, and your benchmarks have
a good chance of being representative.
Anyway, have fun!
Cheers,
Bob
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
