If you are just visiting every node as a tree, and you have core.async,
then there's
(let [c (chan)]
(go
(doseq [n (tree-seq ... (depends on tree implementation) ...)]
(>! c n)))
(dotimes [_ (.availableProcessors (Runtime/getRuntime))]
(go
(loop []
(when-let [n (<! c)]
(expensive-visitation n)
(recur))))))
unless there's some trickiness with core.async that I've not "gotten" here.
Of course, if expensive-visitation returns something instead of having side
effects, you'll need to accumulate some sort of a result in each loop, and
combine these afterward (the dotimes likely becoming a for).
If the visitation is really expensive, then converting the output of
tree-seq into a vector and using reducers or pmap might also parallelize
things significantly.
If you need to do more than just visit each node -- if the tree structure
around the node and not just some value in the node needs to be considered
-- then it gets complicated. Something that traverses the tree, but spawns
new threads at each branching (up to some limit perhaps), seems indicated.
On Thu, Nov 7, 2013 at 6:23 PM, Timothy Washington <[email protected]>wrote:
> Umm I can't think of parallel dependency tree walking algos, off the top
> of my head. Sorry :/
>
> But niiice. Yeah, we're definitely thinking along the same lines. But
> you're way ahead of me. Now just to be able to isolate that browser repl on
> a per project basis.
>
>
> Sweet
>
> Tim Washington
> Interruptsoftware.ca <http://interruptsoftware.ca/> /
> Bkeeping.com<http://bkeeping.com/>
>
>
>
> On Wed, Nov 6, 2013 at 8:46 PM, Malcolm Sparks <[email protected]> wrote:
>
>> Hi Tim,
>>
>> It's interesting you're thinking about browser-connected repls. Today I
>> added clojurescript support to Jig, it's pushed to master but I haven't
>> documented the config options yet, but below is an example. Nothing fancy
>> like crossovers yet, it's basically a thin wrapper over ClojureScript's
>> compiler, the config options map directly to the options given to the
>> compiler.
>>
>> :cljs-builder
>> {:jig/component jig.cljs/Builder
>> :jig/project "../my-proj/project.clj"
>> :source-path "../my-proj/src-cljs"
>> :output-dir "../my-proj/target/js"
>> :output-to "../my-proj/target/js/main.js"
>> :source-map "../my-proj/target/js/main.js.map"
>> :optimizations :simple
>> :pretty-print true
>> ;; :clean-build true
>> }
>>
>> Another built-in component can figure out, given the dependencies, what
>> files to serve and on which server to serve it from.
>>
>> :cljs-server
>> {:jig/component jig.cljs/FileServer
>> :jig/dependencies [:cljs-builder :web]
>> :jig.web/context "/js"
>> }
>>
>> which depends on some Ring or Pedestal routing and Jetty or other
>> webserver, but you're likely to have that configured already :-
>>
>>
>> :server
>> {:jig/component jig.web.server/Component
>> :io.pedestal.service.http/port 8000
>> :io.pedestal.service.http/type :jetty
>> }
>>
>> :web
>>
>> {:jig/component jig.web.app/Component
>> :jig/dependencies [:server]
>> :jig/scheme :http
>> :jig/hostname "localhost"
>> :jig.web/server :server
>> }
>>
>> That's it really. I think Jig makes an ideal platform for building and
>> serving clojurescript, in comparison to Leiningen plugins. Firstly, there's
>> no JVM start-up cost, the point of Jig is to keep the JVM running all the
>> time, so there's no need for the once/auto service that lein-cljsbuild
>> offers. Secondly, serving the resulting Javascript files is straight
>> forward, and built in, so no 'lein ring server' requirement either.. With
>> Leiningen, you either have to build routes to the javascript into your own
>> app, or run lein ring server, but then Leiningen doesn't make it easy to
>> run multiple services concurrently. Thirdly, there's no file-system
>> polling, and services that depend on the clojurescript compilation can be
>> started as soon as the compilation is complete.
>>
>> One thing that lein-cljsbuild does really well is multiple build configs.
>> I've decided to use a single build configuration, to keep implementation
>> easier and a direct mapping to the clojurescript compiler, but of course
>> you can add different builds by just adding (differently configured)
>> components to the Jig config. This would benefit from being able to
>> initialize and start components in parallel, where possible. Unlike
>> cljsbuild, Jig components are started serially, in reverse dependency
>> order. Does anyone know of existing algorithms that can walk a dependency
>> tree in parallel?
>>
>> Feel free to take Jig in different directions though, I'm just letting
>> you know my current thoughts. The single REPL to multiple projects idea
>> might have to be rethought - it seems the prevailing wind is in the
>> direction of multiple REPL connections per editor/IDE.
>>
>>
>> On Thursday, 7 November 2013 01:09:53 UTC, frye wrote:
>>
>>> Too right.
>>>
>>> Yes, wrt the multi-project / classpath thing, running isolated test for
>>> a particular project is only one aspect. I also have an eye to running a i)
>>> browser-connected repl and ii) debugger for a particular project. So those
>>> things, along with iii) running tests, make very high, the attractiveness
>>> of having that isolation.
>>>
>>> I'll try to put those in github issues. And also pick at the problem
>>> myself when I get some cycles. I think it would add a powerful feature to
>>> the tool. Anyways... :)
>>>
>>>
>>> Tim Washington
>>> Interruptsoftware.ca <http://interruptsoftware.ca/> /
>>> Bkeeping.com<http://bkeeping.com/>
>>>
>>>
>> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to [email protected]
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> [email protected]
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/groups/opt_out.
>
--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
