Philip, On 12/24/08, Philip Hunt <cabala...@googlemail.com> wrote: > > 2008/12/24 Steve Richfield <steve.richfi...@gmail.com>: > > > > Clearly, it would seem that no AGI researcher can program a level of > > self-awareness that they themselves have not reached, tried and failed to > > reach, etc. > > This is not at all clear to me. It is certainly prossible for > programmers to program computer to do tasks better than they can (e.g. > play chess)
Yes, but these programmers already know how to play chess. They (probably) can't program a game in which they themselves don't have any skill at all. In the case of higher forms of self-awareness, programmers in effect don't even know the rules of the "game" to be programmed, yet the "game" will have a vast overall effect on everything the AGI "thinks". To illustrate, much human thought goes into dispute resolution - a field rich with advanced concepts that are generally unknown to the general population and AGI programmers. Since this has to much to do with the subtleties of common errors in human thinking, there is no practical way for an AGI to figure this out for itself short of participating in thousands of disputes - that humans would simply not tolerate. Once these concepts are understood, the very act of thinking is changed forever. Someone who is highly trained and experienced in dispute resolution thinks quite differently than you probably do, and probably regards your thinking as immature and generally low-level. In short, their idea of self-awareness is quite different than yours. Regardless of tools, I don't see how such a thing could be programmed except by someone who is already able to think at that level. Then, how about the NEXT level, whatever that might be? > and I see no reason why it shouldn't be possible for self > awareness. My point is that lower-level self-awareness is MUCH simpler to contemplate than is higher-level, and further, that different people (and AGI researchers) function at various levels. Indeed it would be rather trivial to give an AGI access to > its source code. Why should it be any better at modifying its source code than we would be at writing it? The problem of levels still remains. Steve Richfield ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=123753653-47f84b Powered by Listbox: http://www.listbox.com
