I'm low on time but I will throw a few rocks in the pool here.
Composition of futures / asynchronous tasks leads you towards dataflow
concurrency (a trend in the making). People are dipping their toes in
this all over right now but it needs further pushing. imho this is
going to be big in 3-5 years.
For me the core characteristics / benefits of dataflow are:
- you only run tasks in cpu-hot threads when they are ready to run
- you never block in those threads
- if you need to block for data, you swap out (some form of
continuation or paused task is required for this)
- a pool of IO threads either sits to block for data or better is
notified by an event that there is work to do
- a scheduler must understand a) when a task's inputs are fulfilled
and it can be run and b) schedule tasks to run in optimal ways
See:
- Scala Akka's dataflow support - built over the actor core
http://vasilrem.com/blog/software-development/scatter-gather-with-akka-dataflow/
- GPars in Groovy
http://www.gpars.org/guide/guide/7.%20Dataflow%20Concurrency.html
http://www.jroller.com/vaclav/entry/add_one_function_to_my
- Clojure core support for promises
http://clojuredocs.org/clojure_core/clojure.core/promise
- Zach Tellman's Clojure work in Lamina with async programming
https://github.com/ztellman/lamina/wiki/Async-Programming
- Alexey Radul and Gerald Sussman's work on Propagators in Scheme
http://web.mit.edu/~axch/www/art.pdf
- we've built our own version with some special considerations at
Revelytix, which David McNeil will be talking about at the conj and I
will be talking about at Devoxx. We use Clojure of course and
leverage fork/join for compute threads.
I think Clojure is uniquely poised to take advantage of these ideas
due to the flexibility of its syntax wrt control structures and
delayed computation and the availability of high-class portable
concurrency libraries in Java. Having a great scheduler is a hard and
tricky problem.
Alex
On Sep 11, 11:18 pm, Kevin Downey <redc...@gmail.com> wrote:
> Clojure's concurrency primitives are built on the functionality
> provided by java.util.concurrent and I think solutions for
> asynchronous composition should also be built on java.util.concurrent.
> Agents are identities over a series of results from asynchronous
> function application. Nothing about composable async tasks there.
>
> I don't think java has a nice solution we can just use, but all the
> bits we need are there. die-geister is sort of a rough sketch of what
> could be built, it needs better control over which thread pool tasks
> are schedule on, and most of the custom types it uses should just be
> Futures.
>
> http://channel9.msdn.com/events/PDC/PDC10/FT09
>
> http://medianetwork.oracle.com/media/show/17013(this is my favorite
> video, because it is the only one I've seen that talks at least a
> little about how the new Task<>s are scheduled)
>
> http://tomasp.net/academic/joinads/joinads.pdf
>
>
>
>
>
>
>
>
>
> On Sun, Sep 11, 2011 at 2:22 PM, Sean Corfield <seancorfi...@gmail.com> wrote:
> > On Sun, Sep 11, 2011 at 1:29 AM, Laurent PETIT <laurent.pe...@gmail.com>
> > wrote:
> >> now that I've stepped back a little bit and acknowledged my ignorance
> >> (while
> >> working on it : currently reading "Java concurrency in practice" :-) )
>
> > Kevin pulled me up on this too (in IRC) and pointed me at the Java
> > solution. As it happened I decided to avoid an async solution in the
> > end (the speed up wasn't worth the complexity of adding concurrency
> > in my situation - which I didn't know for sure until I'd tried both
> > approaches).
>
> >> This remains me of the "viscosity" smell : if it's way easier to do the
> >> wrong thing than it is to do the right thing, expect people to do the wrong
> >> one.
>
> > Yes, it seems that this is a common enough need that it would be nice
> > to have an idiomatic Clojure library to implement this, perhaps as
> > part of new contrib?
>
> > Kevin, I'd be particularly interested in your thoughts on that since I
> > understand you feel the Java solution is already pretty clean and
> > simple?
> > --
> > Sean A Corfield -- (904) 302-SEAN
> > An Architect's View --http://corfield.org/
> > World Singles, LLC. --http://worldsingles.com/
> > Railo Technologies, Inc. --http://www.getrailo.com/
>
> > "Perfection is the enemy of the good."
> > -- Gustave Flaubert, French realist novelist (1821-1880)
>
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Clojure" group.
> > To post to this group, send email to clojure@googlegroups.com
> > Note that posts from new members are moderated - please be patient with
> > your first post.
> > To unsubscribe from this group, send email to
> > clojure+unsubscr...@googlegroups.com
> > For more options, visit this group at
> >http://groups.google.com/group/clojure?hl=en
>
> --
> And what is good, Phaedrus,
> And what is not good—
> Need we ask anyone to tell us these things?
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
