---------- Forwarded message ---------- From: Moshe Looks <[email protected]> Date: Mon, Jun 22, 2009 at 9:39 PM Subject: Re: [opencog-soc] MOSES/RNN combo issue (restricted DAGs in combo) To: Ben Goertzel <[email protected]> Cc: [email protected], Nil Geisweiller <[email protected]>
> I think we need feedback from Nil and Moshe on this, but my own > intuition is that Option 1 is better. Reduct is important and making > tweaks to the data structures that will make Reduct's life difficult, > doesn't seem a good idea to me... > Well, the "natural" solution from my (non-NN-oriented) perspective is neither of these, but rather to use variable binding to emulate DAG structure. Consider a network with inputs A, B, C, an intermediate layer with nodes X, Y, and outputs O, P, where A->X, B->X, B->Y, C->Y, X->O, Y->O, X->P, Y->P. This would be encoded as : function (a, b, c) let x = sigmoid (w1* a, w2 * b), y = sigmoid (w3 * b, w4 * c) return list(sigmoid(w5 * x, w6 * y), sigmoid(w7 * x, w8 * y)) This would use the existing tree structure, and the overhead would not be *too* horrendous, I don't think (and any way, you could easily compile to some efficient representation for evaluation if needed). I don't know what reductions you are planning to implement, however, and whether this would make them easier or harder to express. I would recommend that if you do decide to implement your own dag class, you just make a quick wrapper around an stl list of lists or something - redoing tree.hh or treetree (http://code.google.com/p/treetree) as a dag would be too time-consuming to allow for a successful SoC... Let me know if you have a questions/comments on this! - Moshe > ben g > > On Tue, Jun 23, 2009 at 3:17 AM, jal278<[email protected]> wrote: >> >> To be able to fully represent feed-forward NNs >> in combo (at least the current approach that >> was decided on earlier) requires that certain >> tree-like DAGs be able to fit into the combo_tree >> class. (Basically some nodes may be pointed to >> by two or more parent nodes as long as they >> are higher up in the tree). >> >> I thought that maybe this would be doable without >> modifying opencog::tree, as I assumed that a tree_node >> would have access to all of its children. However, >> there are optimizations in the tree class so that pointers >> to all children are not maintained. That is, pointers >> to the first and last child are explicitly maintained, >> while the rest are inferred by looking at chain of >> sibling nodes of its first child. This makes representing >> DAGs impossible although it is certainly a good >> optimization for trees. >> >> I'm not exactly sure how to proceed here. I can >> imagine it would not be good to modify opencog::tree >> from which combo_tree derives since it is used in >> other areas. >> >> I currently can think of two options; >> >> 1) I could make a subclass of opencog::tree that >> would support the tree-like DAGs that MOSES/RNN >> will need, and then redefine combo_tree as an >> instance of that. >> >> 2) We could represent nodes in combo with >> id numbers so that it is understood that ANN-Node with >> id #3 is equivalent to any other ANN-Node vertex with the >> id #3. In this way, we can avoid having to explicitly link >> one node to more than one parent. I believe someone >> raised the issue that this would not work well with the >> current Reduct system (because it would not be able >> to equate the two references to the same node). >> >> With option #2, we could directly evolve RNNs as well >> as feed-forward ANNs. Although this may not be a good >> way to evolve RNNs, it would offer a control to gauge >> how effective our generalized elman network approach >> is. >> --~--~---------~--~----~------------~-------~--~----~ >> You received this message because you are subscribed to the Google Groups >> "OpenCog Summer of Code Announcements & Discussion List" group. >> To post to this group, send email to [email protected] >> To unsubscribe from this group, send email to >> [email protected] >> For more options, visit this group at >> http://groups.google.com/group/opencog-soc?hl=en >> -~----------~----~----~----~------~----~------~--~--- >> >> > > > > -- > Ben Goertzel, PhD > CEO, Novamente LLC and Biomind LLC > Director of Research, SIAI > [email protected] > > "Truth is a pathless land" -- Jiddu Krishnamurti > _______________________________________________ Mailing list: https://launchpad.net/~opencog-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~opencog-dev More help : https://help.launchpad.net/ListHelp
