On Jul 14, 12:24 pm, Howard Lewis Ship <[email protected]> wrote:
> Seems like every time I do a Clojure talk I get questions I can't
> answer. I did an informal session last night for Portland's
> Functional Programming Study Group.
>
> Two questions came up:
>
> First, a Haskell coder made the broad claim that other attempts at STM
> did not use MVCC because it was "too slow". I responded to the effect
> that Clojure HashMap's are really fast and fast to mutate (he
> countered with Haskell being able to modify an unmodifiable map in
> place if there's only one reference to it). I also emphasized that
> Rich is very concerned with performance and that if Clojure STM would
> be implemented some other way if MVCC wasn't fast enough.
>
> In retrospect, the right answer may have been more to the effect that
> Clojure STM limits the number of mutable "slots" (the refs) to keep it
> efficient. What's the correct response here?
>
The correct response is that most general statements about STM are not
very useful. There are tremendous variations in approach and
implementation details etc. Clojure's STM does no read tracking, and
no transaction tracking. Clojure's MVCC history is adaptive. Were all
these true for the others?
Clojure does encourage placing immutable values in refs, and thus uses
a coarser granularity than STMs that wrap every field access. Nothing
about this in in conflict with how you might use an STM in Haskell
btw, but is pretty different from most other STMs you'll see for Java,
i.e. that let you continue to mutate subfields of objects. In those
STMs there is no way to see a consistent object value except inside a
transaction and through the STM layer. In Clojure, objects are always
values and refs are just time-management tools. Ditto coarse-grained
use of STM in Haskell. Really, the bigger contrast is between FP+STM
and OO+STM.
I'm extremely skeptical of "modify an unmodifiable map in place if
there's only one reference to it" working well at all in a true
multithreaded context with any potential persistent use. Normal
Haskell data structures used shared structure just like Clojure's and
provide the same benefits, both inside and out of STM.
You'll note I am not criticizing Haskell's STM - it is a very nice
design and a good fit for Haskell. Anyone who's happy with Haskell
should stick with it. Haskell and Clojure users should just be happy
they have STMs and persistent data structures at hand.
> Secondly, I told the story about the big simulation on the Azul, and
> the 20 gig of GC/sec, etc. So if you've coded your simulation really
> well, do 600 cores give you a 600x speedup over a single core? (Of
> course not). Given that 10% of the cores are doing GC, how much
> benefit to you get from spreading across multiple cores? And how much
> better (or worse) is Clojure at this than a typical, non-naive, Java
> approach?
>
There are few general answers to such questions that hold across
anything other than a single scenario. Every application has differing
degrees of transaction footprint overlap, and transaction durations.
Applications with minimal conflicts will get the most benefit from
multicore, applications with a lot of overlap will get the least. This
is true no matter what conflict avoidance strategy you use (locks,
STM, actors). Share less and mutate less, scale more. People also need
to be cognizant of the difference between using multicore for
parallelization (get one job done faster), and for concurrency (get
more jobs done). Both ants demos are designed to demonstrate how to do
something potentially hard (many jobs, with arbitrarily overlapping
resource usage), correctly and easily.
There are higher constant-factor costs for accessing an STM-managed
reference than a field, so a single lock covering a lot of field
accesses has a better per field overhead story. Clojure sits in a good
spot with its coarser granularity - you only pay once to pull the
whole immutable object out of the ref, then you can have at it without
any STM code in the path.
Everyone who is genuinely interested in STM should try one on their
own problems and provide feedback. These theoretical discussions are
just that, theoretical. I could argue all day about why I think MVCC
STM with adaptive history, plus immutable persistent data structures,
used in a coarse-grained manner, is a good approach. Instead, I built
it, and everyone can try it.
Rich
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~----------~----~----~----~------~----~------~--~---
