[agi] Reward versus Punishment? .... Motivational system
What about punishment?
Dan Goe
>From : William Pearson <[EMAIL PROTECTED]>
To : agi@v2.listbox.com
Subject : Re: [agi] Motivational system
Date : Fri, 9 Jun 2006 23:48:07 +0100
> On 09/06/06, Dennis Gorelik <[EMAIL PROTECTED]> wrote:
> > William,
> >
> > > It is very simple and I wouldn't apply it to everything that
> > > behaviourists would (we don't get direct rewards for solving
crossword
> > > puzzles).
> >
> > How do you know that we don't get direct rewards on solving crossword
> > puzzles (or any other mental task)?
>
> I don't know, I only make hypotheses. As far as my model is concerned
> the structures that give direct reward have to be pretty much in-built
> otherwise for a selectionist system allowing a selected for behaviour
> to give direct reward would quickly lead to behaviour that gives
> itself direct reward and doesn't actually do anything.
>
> > Chances are that under certain mental condition ("achievement state"),
> > brain produces some form of pleasure signal.
> > If there is no such reward, then what's your explanation why people
> > like to solve crossword puzzles?
> >
> Why? By indirect rewards! If you will allow me to slip into my
> economics metaphor, I shall try to explain my view of things. The
> consumer is the direct reward giver, something that attempts to mold
> the system to produce certain products, it doesn't say what is wants
> just what is good, by giving money ( direct reward).
>
> In humans this role played by the genome constructing structures that
> says nice food and sex is good, along with respect from your peers
> (probably the Hypothalamus and amygdala).
>
> The role of raw materials is played by the information coming from the
> environment. It can be converted to products or tools.
>
> You have retail outlets that interact directly with the consumer,
> being closest to the outputs they get directly the money that allows
> their survival. However they have to pass some of the money onto the
> companies that produced the products they passed onto the consumer.
> This network of money passing will have to carefully controlled so
> that more money isn't produced in one company than was given
> (currently I think of the network of dopaminergic neurons being this
> part).
>
> Now with this sort of system you can make a million just so stories
> about why one program would be selected that passes reward to another,
> that is give indirect reward. This is where the complexity kicks in.
> In terms of crossword solving one possibility is that a program closer
> to the output and with lots of reward has selected for rewarding
> logical problem solving because in general it is useful for getting
> reward and so passes reward on to a program that has proven its
> ability to logical problem solve, possibly entering into a deal of
> some sort.
>
> This is all very subconcious, as it is needed to be to be able to
> encompass and explain low level learning such as neural plasticity,
> which is very subconcious itself.
>
> Will Pearson
>
> ---
> To unsubscribe, change your address, or temporarily deactivate your
subscription,
> please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
---
To unsubscribe, change your address, or temporarily deactivate your
subscription,
please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Reward versus Punishment? .... Motivational system
On Fri, 09 Jun 2006 19:13:19 -500, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: What about punishment? Currently I see it as the programs in control of outputting (and hence the ones to get reward), losing the control and the chance to get reinforcement. However experiment or better theory would be needed to determine whether this is sufficient or negative reward would be needed. Will --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Reward versus Punishment? .... Motivational system
Will, Right now I would think that a negative reward would be usable for this aspect. I am using the positive negative reward system right now for motivational/planning aspects for the AGI. So if sitting at a desk considering a plan of action that might hurt himself or another, the plan would have a negative rating, where another safer plan may have a higher rating. One possible thing here as well is to have asmall random value added, so that even though a plan has a suboptimal value, it woul dbe possible to take that route instead. (maybe adding in the value of having a new experience here as well)One important thing we will need here, is an entire KR that will represent all the AGI's past actions, and a way to look back over the actions and compare them to thier expected outcomes, and see why something is different. (Reflection)IE If the robot proposes to cross the road at a point, sees it as a good plan, and does it, but nearly gets hit by a car, it needs to be able to look back over its actions, and determine that something was missing from his equation, and try to add it back in, or ask a human for assistance, so in the future, he can better handle this activity.James RatcliffOn Fri, 09 Jun 2006 19:13:19 -500, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:>> What about punishment?Currently I see it as the programs in control of outputting (and hencethe ones to get reward), losing the control and the chance to getreinforcement. However experiment or better theory would be needed todetermine whether this is sufficient or negative reward would beneeded. Will---To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]Thank YouJames Ratcliffhttp://FallsTown.com - Local Wichita Falls Community Websitehttp://Falazar.com - Personal WebsiteHosting Starting at $9.95Dialups Accounts - $8.95 __Do You Yahoo!?Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Reward versus Punishment? .... Motivational system
On 12/06/06, James Ratcliff <[EMAIL PROTECTED]> wrote: Will, Right now I would think that a negative reward would be usable for this aspect. I agree it is usable. But I am not sure it necessary, you can just normalise the reward value. Let say for most states you normally give 0 for a satiated entity, the best state is 100 and the worst and the worst -100. You can just transform that to 0 for the worst state 100 for everyday satiated state and 200 for the best state, without affecting the choices that most reinforcement systems would make. So pain would be a below baseline reward. I am using the positive negative reward system right now for motivational/planning aspects for the AGI. So if sitting at a desk considering a plan of action that might hurt himself or another, the plan would have a negative rating, where another safer plan may have a higher rating. Heh. Well I expect an AI system that worked like a human would have a very tenuous link between the motivation and planning systems. The tenuous link is ably shown by my own actions. I have stated that I think the plausible genetically specified positive motivations are to do with food, sex and positive social interaction. Yet I tend to plan how to create interesting computer systems, which isn't the best route to any of the above More later... Will Pearson --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
