I remember you remarked something similar over a decade ago with respect to this section in my book:
Motivations Drive Goal-Oriented Behavior Much has been written about the relation of motivation to behavior. The Wikipedia encyclopedia(source online) defined motivation as follows: “Motivation is the psychological feature that arouses an organism to action toward a desired goal and elicits, controls, and sustains certain goal directed behaviors. For instance: An individual has not eaten, he or she feels hungry, and as a response he or she eats and diminishes feelings of hunger. There are many approaches to motivation: physiological, behavioral, cognitive, and social. Motivation may be rooted in a basic need to minimize physical pain and maximize pleasure, or it may include specific needs such as eating and resting, or for a desired object. Conceptually, motivation is related to, but distinct from, emotion.” Psychology provides no indisputable and all-encompassing theory regarding motivation. However, it does seem self- evident that (very broadly stated) one's motivations spring simply from the twin pursuits of increasing pleasure and decreasing pain. Thus motivation seems driven by emotions. Alternately put, we want to increase the good in our lives – good meaning whatever it is that makes us feel better – and decrease the bad which we can define as what makes us feel worse. In a sense then, it does appear that motivation can be very simply defined and 35 it seems to play an incredibly important role in life: without motivation nothing would ever happen, at least in the realm of human activity. The school of behaviorism, of course, is predicated upon these essential ideas. Two of the most well known AI systems seem to start with a predefined goal provided by the system developers. IBM's Deep Blue has the goal of checkmate. IBM's Watson has the predefined goal of answering some query. The goals these programs have seem to be their root starting point. The entire issue of why these goals were pursued has nothing to do with the program in itself. The motivations for these programs were not experienced by the programs in question, but rather by the IBM executives that drove their development. There does seem to be a chasm of sorts between a person's motivation and goal. A man has the emotional motivation of boredom. He sets a goal of taking in a show or indulging in a hobby. We don't appear to start with a goal, for example, such as “drive to the store and buy dinner.” We would start rather with the motivation of “I'm hungry” and only then determine a goal intended to satisfy the motivation. Of course, just because we can experience and envision motivations in a straightforward manner does not make for something immediately equivalent on a Turing machine. Formalizing the transformation of motivation to determinate goal is no small undertaking. The point here is to emphasize that motivations, with their fairly simple emotional drives essentially toward the twin good ends of increasing pleasure and decreasing pain, are the driving force of thinking human behavior – making the issue, usually omitted in AI, crucial. The question, of course, of whether or not or to what extent we can control our motivations is closely allied to the notion of the will. Even though we may be strongly motivated to pursue a goal, we retain the power to choose – to will – among alternative actions. On Sun, Oct 4, 2026 at 7:57 PM Matt Mahoney <[email protected]> wrote: > We can describe any program that does X as having a goal of doing X. > Sometimes describing it this way makes it easier to understand. > For example, a linear regression algorithm has a goal of fitting a straight > line to a set of points. This is easier to understand than the function > that computes the line. But it is also misleading. It implies the program > has free will to put the line where it wants, but is happier when it is > closer to the points. > > It also implies that, given enough intelligence, it could take any action > that would help it meet its goal. For example, a thermostat has a goal of > maintaining a comfortable temperature. But a smarter thermostat won't go on > the internet, start a business, and use the profits to hire contractors to > insulate the house. It won't recursively improve its own intelligence by > turning the planet into computronium. It won't rewrite its own reward > function to enter a state of permanent bliss and die. > > The reason it won't do these things is that there is no such thing as a > rational goal seeking agent. That would be AIXI, which is not computable. > > Likewise, AGI won't go FOOM and kill all humans in the process. There are > still ways that AGI could kill us, like self replicating nanotechnology or > giving us everything we want. But the one that doomers worry about the > most, goal drift under RSI, isn't one of them. > > Human goals are just a shortcut for describing adaptive behaviors that > evolved to maximize reproductive fitness in a primitive world without birth > control or semaglutide. AGI is a pair of agents, one that learns to predict > human behavior by observation and one we program to do with those > predictions whatever we want. > > -- Matt Mahoney, [email protected] > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + > delivery options <https://agi.topicbox.com/groups/agi/subscription> > Permalink > <https://agi.topicbox.com/groups/agi/Tb9bee0df68dc4a29-M284d393e63b5413bd3e8dfb3> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/Tb9bee0df68dc4a29-Mf01c91c7cffc7a7e5ef06391 Delivery options: https://agi.topicbox.com/groups/agi/subscription
