--- Stathis Papaioannou <[EMAIL PROTECTED]> wrote: > On 01/07/07, Tom McCabe <[EMAIL PROTECTED]> > wrote: > > > > But Deep Blue wouldn't try to poison Kasparov in > > > order to win the > > > game. This isn't because it isn't intelligent > enough > > > to figure out > > > that disabling your opponent would be helpful, > it's > > > because the > > > problem it is applying its intelligence to is > > > winning according to the > > > formal rules of chess. > > > > Exactly. The formal rules of chess say stuff about > > where to put pawns and knights; they're analogous > to > > the laws of physics. They don't say anything about > > poisoning the opposing player. If you try to build > in > > a rule about poisoning the player, the chess > program > > will shoot him; if you build in a rule against > killing > > him, the chess program will give him a > hallucinogen; > > if you build in a rule against giving him drugs, > the > > chess program will hijack the room wall and turn > it > > into a realistic 3D display of what would happen > if a > > truck smashed into the room by accident. This > approach > > will never work- you're pitting your intelligence > at > > designing rules against the program's intelligence > at > > evading them, and it's smarter than you are. > > Why do you assume that "win at any cost" is the > default around which > you need to work?
Because it corresponds to the behavior of the vast, vast majority of possible AGI systems. Is there a single AGI design now in existence which wouldn't wipe us all out in order to achieve some goal? > > > -- > Stathis Papaioannou > > ----- > This list is sponsored by AGIRI: > http://www.agiri.org/email > To unsubscribe or change your options, please go to: > http://v2.listbox.com/member/?&; > - Tom ____________________________________________________________________________________ Take the Internet to Go: Yahoo!Go puts the Internet in your pocket: mail, news, photos & more. http://mobile.yahoo.com/go?refer=1GNXIC ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=4007604&user_secret=7d7fb4d8
