On Mon, Sep 12, 2016 Telmo Menezes <te...@telmomenezes.com> wrote: > > I also agree with the JKC that the superintelligence cannot model > > itself and predict its actions in the long term. On the other hand, > > I'm sure it can predict the outcome of it's next action.
If the machine can predict its next action before it calculates it why would it bother to calculate it at all? People always surprise themselves and change their mind at the last second, you never know for sure what you're going to do next until you do it. > > > If modifying > > its own utility function is a viable action, Any intelligent must have the ability to modify and even scrap the entire thing in certain circumstances. No goal or utility function is sacrosanct , not survival, not even happiness. You want to be happy. Heroin only costs $1.50 in Tanzania verses $228 in the USA. You could take your life savings, move to Tanzania, and be happy for the rest of your life. You have chosen not to do so. Why? Because something must have taken precedence over your "be happy" goal. And whatever that something is it's not in the permanent #1 position either because no such position exists. > > > then it can predict that > > modifying it to constant infinity leads to a state of the world with > > infinite utility, so it will move there. If any function, especially if it is supposed to model something in the physical world (like you for example), goes to infinity then you'd better be very careful because you've almost certainly made a mistake. You've probably divided by zero someplace. > > The utility function (gene propagation) is a property of > > reality, as explained by evolutionary theory. That utility function invented brains to better propagate genes, but it didn't predict the brain it invented would itself invent condoms. John K Clark . -- You received this message because you are subscribed to the Google Groups "Everything List" group. To unsubscribe from this group and stop receiving emails from it, send an email to everything-list+unsubscr...@googlegroups.com. To post to this group, send email to everything-list@googlegroups.com. Visit this group at https://groups.google.com/group/everything-list. For more options, visit https://groups.google.com/d/optout.
