From: "Ben Goertzel" <[EMAIL PROTECTED]> >> (1) A dynamical ANN activates the next state according to >> its current state, so there exists an "objective function" >> for all states such that V(x) <= V(y) if state x is at >> least as preferable as state y. > >I don't fully undertsand this. Is your function V supposed to be the >"energy function" from a symmetric Hopfield ANN? > >But no energy function exists for an asymmetric Hopfield net >(which can have chaotic states, etc., and is hence not easily >modeled as an energy-minimizer).
Yes V is also called the energy or potential function. You're also right that the energy cannot be defined for asymmetric Hopfield nets (however they can still be analysed using Lyapunov functions, which also map the states to a real number in a non-increasing way). Then all ANNs can be viewed as minimizing an objective function (either the energy function or the Lyapunov function, and they're both denoted by V in the book). >> (2) A relational system defines a measure over a set of >> events. For example the relation "<=" over the set of >> events with different subjective probabilities. The >> probability density function p(x) is one such measure. >> >> So there is an isomorphism between (1) and (2), given >> some constraints. This is very formal and I'm not sure >> if it's althogether very useful when applied to complex >> ANN systems. > >??? Finding a maximum a posteriori (MAP) estimate means selecting the most *probable* element of the sample space, and thus is an optimization problem. Therefore any ANN can be viewed as making some rational inference based on certain *implicit* probability models. The difficult part is to work backwards from a given ANN to reverse-engineer the probabilistic model that it implicitly contains; ie given the formula for V(x) and a set of synaptic weights W how to derive the formula for p(x). It turns out that p(x) = exp(-V(x))/Z where Z is a normalization constant equal to Sum over y: exp(-V(y)); Which you may recognize is the Gibbs distribution! Pretty interesting, and I don't fully understand it, the complete derivation involves Markov random fields which is an extension of Markov chains; the Markov random field theorem which helps to factorize propability densities and thus shows that the random field is equivalent to a Gibbs distribution. Sorry that my original explanation was too brief. I was just reading a lot of books to see what kind of tools are available and not trying to understand deeply unless I find the stuff useful... =) YKY PS Let's not forget the original point of this thread was to find some equivalence relations between various machine learning approaches such as Bayesian, symbolic, ANN, etc. It would be very helpful *if* some approaches are found to be interconvertable. ____________________________________________________________ Get advanced SPAM filtering on Webmail or POP Mail ... Get Lycos Mail! http://login.mail.lycos.com/r/referral?aid=27005 ------- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
