Ben, Since your paper is on "preservation of AI goal systems under repeated self-modification", I wonder whether you should address the following related issues:
(1) Whether "goal drift" (I call "task alienation" in http://www.springer.com/west/home/computer/artificial?SGWID=4-147-22-173659733-0) is always undesired --- your paper treats it as obviously bad. (2) Whether it is possible to completely avoid it in a truly intelligent system --- you suggests one way to avoid it, without saying how much of the problem can be handled by this solution. I don't have the time to write a detailed paper on this topic, but I've written here or there (include the book) that (1) This phenomenon is a root of many valuable properties, including originality, creativity, and flexibility, and it explained many things, including art appreciation, aimless playing, even scientific exploration. Without it, human beings would just be like other animals, driving only by their built-in biological goals. (2) It is impossible to completely avoid this phenomenon in a truly intelligent system, whether we like it or not. Your solution won't change the big picture, even though it may help in some special cases. Pei On Sat, Aug 30, 2008 at 4:39 PM, Ben Goertzel <[EMAIL PROTECTED]> wrote: > > Hi > > All who interested in such topics and are willing to endure some raw > speculative trains of thought, > may be interested in an essay I recently posted on goal-preservation in > strongly self-modifying > systems, which is linked to from this blog post > > http://multiverseaccordingtoben.blogspot.com/2008/08/on-preservation-of-goals-in-self.html > > (I've got another, shorter post on AGI systems with superhuman empathy > coming up ... but probably > not this weekend; I've got some real work to get through first ;-) > > -- Ben > > > -- > Ben Goertzel, PhD > CEO, Novamente LLC and Biomind LLC > Director of Research, SIAI > [EMAIL PROTECTED] > > "Nothing will ever be attempted if all possible objections must be first > overcome " - Dr Samuel Johnson > > > ________________________________ > agi | Archives | Modify Your Subscription ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=111637683-c8fa51 Powered by Listbox: http://www.listbox.com
