> > In my book the first thing you do is sort out your definitions. Then you > do the math. Not the other way around.
Historically, in the evolution of math, definitions and theorems have generally evolved together. The right definitions are the ones that lead to the interesting theorems. > > > Sadly, the very existence of the mathematical apparatus that Goertzel > proposes will serve to disguise the fact that all of our attention, right > now, should be directed at the insecure foundations. In the literature as a > whole, the concept of a "goal" is bandied about as if everyone understood > that this was a moderately well-defined concept. In fact it is anything > but. It is all well and good to have philosophical discussions in which we > take a kind of Turing-esque, hands-off approach and say that a goal is just > what a "reasonably smart guy" would judge to be a goal, but this kind of > philosophical handwaving is not going to cut the mustard if real systems > need to be designed to do real intelligent things. > I don't think that this kind of foundational theorizing about goals and self-modification is necessary in any way for AGI design. However, it may ultimately be useful for studying or statistically predicting the behavior of AGI's, tuning their goal systems, etc. > > We still await a calculus of goals and motivation that is founded on basic > concepts that are not defined in terms of subjective observers or homunculi. I could get rid of the explicit subjectivity in that definition if it bothers you ... Just define an **implicit goal** of a system S over a time period T as a function f so that: the proposition "S approximately optimizes f" is a pattern in the system S's trajectory over time period T. There is still subjectivity here behind the scenes though, because the definition of "what is a pattern" relies on some measure of simplicity ... which basically means relying on some "reference universal Turing machine" as in classical CS theory. Maybe it's more elegant to talk about reference UTM's than observers, but ultimately it's the same story ... I just chose to write that paragraph of that essay w/out referring to any particular formalism.. If you don't accept definitions involving reference UTM's, you've got to throw out most of theoretical CS ... which might please you, but not me ;-) ben g ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=111637683-c8fa51 Powered by Listbox: http://www.listbox.com
