Hi Jason, That's roughly what I figured, thanks for the helpful reply.
Thanks for taking the time release this lib; its really great Edmund On Saturday, 9 February 2013 01:28:09 UTC, Jason Wolfe wrote: > > > > On Friday, February 8, 2013 1:56:27 AM UTC-8, Edmund wrote: >> >> Hey Jason, >> >> Going slightly OT here to describe my motivations, but you did ask >> for more details ! :) >> >> I wrote a little blog post + lib last month using Stuart Sierra's Flow >> (much like >> Graph) to implement the steps in doing a very simple discrete inference >> task. Its >> demonstrating the same thing you talk about in your post about >> declarative computation - here the point is that you can efficiently >> update your >> inference based new data on account of having the >> graph structure to work over. Here's the post >> http://boss-level.com/?p=160 and a >> literate version of the code http://ejackson.github.com/inflow/ >> This is my original motivation. >> > > I think I understand what you're trying to do from the example, thanks! > > >> What I'm doing there is 'building-up' the computation over time, >> so I start with a graph with just the input data, and end up with a >> graph with the input and output. But the 'building up' is not the >> principal hangup I >> have: I'm happy to have an in- and out- map. The problem is, when >> dealing with a very nested graph, getting a fine grained syntax for >> describing where to put the outputs. Ie how do I tell Graph to put just >> the output x into >> {:a {:b 1 :c x}} of the graph, without replacing the entire map of the :a >> key ? >> > > Like I said earlier, it's a requirement of Graph that node names must be > unique, and distinct from top-level input keys. > > Thus, there cannot be a way to 'put x into the map under :a at key :c' , > because there is no way to in-place modify the inputs or other node values > of the graph. The output is just a map of all the node results, nothing > more and nothing less. > > That said, you can use a hierarchical graph to specify different node > functions for the different sub-keys: > > ;; As a hierarchical graph > user> ((eager-compile {:a2 {:b (fnk [[:a1 b]] b) > :c (fnk [[:a1 b]] (inc b))}}) > {:a1 {:b 1}}) > {:a2 {:b 1, :c 2}} > > So for an example like the one you gave in the above link, you can > represent your nested stats directly as a nested graph without resorting to > key flattening. And if you want to compute just a subset of the graph for > updated downstream values and then merge this into your previous map, you > can do this in two steps: > > 1. Pick out the subset of leaf functions that may have changed, using the > same strategy you currently use with Flow, and construct a Graph with only > those sub-keys. For example: > > user> ((eager-compile {:a2 {:b (fnk [[:a1 b]] b)}}) > {:a1 {:b 2}}) > {:a2 {:b 2}} > > , then > > 2. deep-merge these results with the original result map. > > i..e, (deep-merge-with (fn [x y] y) original-results updated-results) > where deep-merge-with is > http://clojuredocs.org/clojure_contrib/clojure.contrib.map-utils/deep-merge-with > > This logic could be wrapped up into a single higher-order function that > takes a graph and knows how to incrementally update the results given > incremental changes in input. Does that make sense / seem like it will > satisfy your need? > > Cheers, > Jason > > > > > >> That's obviously no good if I need to have lots of things changing >> beneath :a in a single graph flow due to uniqueness.[ >> >> With Flow what I did is flatten the nested graph such that the keys are >> vectors of the path to each node, run the flow on that graph, and then >> re-nest the the graph afterwards (the flow-> and ->flow functions). >> So the map {:a {:b 1 :c x}} becomes {[:a :b] 1, [:a :c] x}, and I can >> refer to >> nested nodes this way. >> >> In Graph such a scheme would be possible too, but that Graph enforces >> keywords as the keys for the graph. Having vectors for the keys still >> allows you to >> have uniqueness, topological ordering etc. So the example I gave in my >> last email: >> >> {:a {:b 1}} -> {:a {:b 1, :c 2}} >> >> could, hypothetically, be written for just the outputs as >> >> ((graph/eager-compile {[:a :c] (fnk [[:a b]] (inc b))}) >> {:a {:b 1}}) >> >> >> {:a {:c 2}} >> >> So my question is: how do I refer to nested nodes in the output >> selector using Graph? Conditionally, on that being impossible, would >> there be any merit in the vector scheme that I have suggested ? >> >> Sorry for the long post, I hope I haven't obscured my question. I also >> get the feeling that I must be missing something very obvious in your >> code for how to do this ! >> >> Edmund >> >> >> On Thursday, 7 February 2013 19:27:13 UTC, Jason Wolfe wrote: >>> >>> On Thu, Feb 7, 2013 at 10:54 AM, AtKaaZ <atk...@gmail.com> wrote: >>> > Hello. >>> > >>> https://github.com/Prismatic/plumbing/blob/master/test/plumbing/graph_examples_test.clj#L148 >>> >>> > Why do they return in a map instead of maybe a set ? do we ever get >>> {:key >>> > false} ? >>> > Thanks. >>> >>> The leaf value for output schemata is always 'true'. It's a bit odd, >>> but to support specifications of functions/graphs that return nested >>> maps, the outer layers need to be maps. It's true that the innermost >>> layers could be represented as sets, but we chose to keep things >>> uniform instead. >>> >>> > >>> > >>> > On Thu, Feb 7, 2013 at 7:22 PM, Jason Wolfe <ja...@w01fe.com> wrote: >>> >> >>> >> We've just posted a blog post with more high-level context on what >>> we're >>> >> trying to accomplish with Graph (plus more examples!) >>> >> >>> >> >>> >> >>> http://blog.getprismatic.com/blog/2013/2/1/graph-abstractions-for-structured-computation >>> >>> >> >>> >> We're also answering questions and reading comments in the Hacker >>> News >>> >> thread, if that's your thing: >>> >> >>> >> http://news.ycombinator.com/item?id=5183236 >>> >> >>> >> Cheers, Jason >>> >> >>> >> >>> >> On Tuesday, January 29, 2013 10:46:54 AM UTC-8, Aria Haghighi wrote: >>> >>> >>> >>> Hey all, >>> >>> >>> >>> Prismatic has open-sourced our Plumbing and Graph library on >>> github. >>> >>> Jason Wolfe gave a talk about how we use graph for systems >>> composition at >>> >>> Strange loop last year. Please give the library >>> >>> a whirl and let us know if you're using it and if you find any >>> issues or >>> >>> feature requests. We use this library very heavily throughout our >>> code and >>> >>> hope others find it useful as well. >>> >>> >>> >>> Best, Aria >>> >> >>> >> -- >>> >> -- >>> >> You received this message because you are subscribed to the Google >>> >> Groups "Clojure" group. >>> >> To post to this group, send email to clo...@googlegroups.com >>> >> Note that posts from new members are moderated - please be patient >>> with >>> >> your first post. >>> >> To unsubscribe from this group, send email to >>> >> clojure+u...@googlegroups.com >>> >> For more options, visit this group at >>> >> http://groups.google.com/group/clojure?hl=en >>> >> --- >>> >> You received this message because you are subscribed to the Google >>> Groups >>> >> "Clojure" group. >>> >> To unsubscribe from this group and stop receiving emails from it, >>> send an >>> >> email to clojure+u...@googlegroups.com. >>> >> For more options, visit https://groups.google.com/groups/opt_out. >>> >> >>> >> >>> > >>> > >>> > >>> > >>> > -- >>> > Please correct me if I'm wrong or incomplete, >>> > even if you think I'll subconsciously hate it. >>> > >>> > -- >>> > -- >>> > You received this message because you are subscribed to the Google >>> > Groups "Clojure" group. >>> > To post to this group, send email to clo...@googlegroups.com >>> > Note that posts from new members are moderated - please be patient >>> with your >>> > first post. >>> > To unsubscribe from this group, send email to >>> > clojure+u...@googlegroups.com >>> > For more options, visit this group at >>> > http://groups.google.com/group/clojure?hl=en >>> > --- >>> > You received this message because you are subscribed to the Google >>> Groups >>> > "Clojure" group. >>> > To unsubscribe from this group and stop receiving emails from it, send >>> an >>> > email to clojure+u...@googlegroups.com. >>> > For more options, visit https://groups.google.com/groups/opt_out. >>> > >>> > >>> >> -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
