On Thu, Jul 3, 2008 at 4:05 PM, William Pearson <[EMAIL PROTECTED]> wrote: > 2008/7/3 Vladimir Nesov <[EMAIL PROTECTED]>: >> On Thu, Jul 3, 2008 at 10:45 AM, William Pearson <[EMAIL PROTECTED]> wrote: >>> >>> Nope. I don't include B in A because if A' is faulty it can cause >>> problems to whatever is in the same vmprogram as it, by overwriting >>> memory locations. A' being a separate vmprogram means it is insulated >>> from the B and A, and can only have limited impact on them. >> >> Why does it need to be THIS faulty? If there is a known method to >> prevent such faultiness, it can be reliably implemented in A, so that >> all its descendants keep it, unless they are fairly sure it's not >> needed anymore or there is a better alternative. > > Because it is dealing with powerful stuff, when it gets it wrong it > goes wrong powerfully. You could lock the experimental code away in a > sand box inside A, but then it would be a separate program just one > inside A, but it might not be able to interact with programs in a way > that it can do its job. > > There are two grades of faultiness. frequency and severity. You cannot > predict the severity of faults of arbitrary programs (and accepting > arbitrary programs from the outside world is something I want the > system to be able to do, after vetting etc). >
You can't prove any interesting thing about an arbitrary program. It can behave like a Friendly AI before February 25, 2317, and like a Giant Cheesecake AI after that. -- Vladimir Nesov [EMAIL PROTECTED] http://causalityrelay.wordpress.com/ ------------------------------------------- agi Archives: http://www.listbox.com/member/archive/303/=now RSS Feed: http://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: http://www.listbox.com/member/?member_id=8660244&id_secret=106510220-47b225 Powered by Listbox: http://www.listbox.com
