The impression that I get from the neuroscience literature is that there are two basic types of learning in the brain. The first type could be called "model learning", it is what the cortex specializes in, and it's about learning hierarchical spatio-temporal models of input, from both external sensors and from other brain areas, representing the outside world, the body, and other internal states, and how they change over time. The second type is reinforcement learning, which uses built-in "reward" and "punishment" signals (such as pain or the taste of sugar) to learn what cortical patterns should be activated within a particular context of the activity of other cortical patterns, so as to maximize reward and minimize punishment. In the brain, reinforcement learning takes place in the basal ganglia, but uses input from many different areas of the cortex, and affects the activation of patterns within prefrontal and motor cortex to result in the control of attention, working memory and movement.
For a more detailed discussion, see e.g. chapter 7 here: http://grey.colorado.edu/mediawiki/sites/CompCogNeuro/images/8/89/ccnbook_01_09_2012.pdf It's this dichotomy that I think the BECCA system is getting at, with their distinction between a "feature creator" and a "reinforcement learner". All cortical regions contribute in some way to motor output, if only by providing contextual information to the basal ganglia or to other subcortical structures involved in shaping motor output, such as the cerebellum or superior colliculus. But the final output to the spinal cord that actually produces movement comes mostly from the motor areas. CLA strikes me as being potentially a major advance in simulating the cortex and its spatio-temporal "model learning", but I think the addition of reinforcement learning will be necessary in order to approach the problems of action selection, attention, working memory and cognition in a brain-like way. -Mike _____________ Michael Ferrier Department of Cognitive, Linguistic and Psychological Sciences, Brown University [email protected] On Tue, Aug 20, 2013 at 11:34 AM, Thompson, Jeff <[email protected]>wrote: > Hello SeH, > > While I appreciate your pointing out this open source project of which I > was not aware, it seems to go against my question. I started paying > attention to work on the CLA (again after many years) when I heard Jeff > Hawkins speaking as he does below that "There are no pure “sensory” regions > and no pure “motor” regions". It gave me hope that this work might avoid > the pitfall of the classic "input-processing-output" loop of classic AI, > which BECCA clearly seems to follow (see the attached diagram). > > We now know that there are just as many feedback connections going to back > down to the "input" regions, and that action is not so different from > perception (in that it uses similar machinery of prediction), and that > "input" is intimately tied to the actions active during the input (instead > of having "action" on the other side of world from "input", as in the BECCA > diagram). > > I'm hopeful to see a diagram soon of many CLA modules for action and > perception connected in a hierarchy which shows how action comes from > similar prediction machinery as perception and how to avoid the pitfall > of "input on one end, output on the other end." > > Thank you, > - Jeff T > ------------------------------ > *From:* nupic [[email protected]] on behalf of SeH [ > [email protected]] > *Sent:* Monday, August 19, 2013 6:26 PM > *To:* NuPIC general mailing list.; [email protected] > > *Subject:* Re: [nupic-dev] Mentioned presentation on action with CLA? > > i imagined that something like OpenBECCA http://openbecca.org could be > integrated with NuPIC for a motor control system > > https://github.com/brohrer/becca<https://github.com/brohrer/becca/blob/master/doc/becca_0.4.5_users_guide.pdf> > > https://github.com/brohrer/becca/blob/master/doc/becca_0.4.5_users_guide.pdf > > from the opposite direction: part of BECCA's perceptual components may > be enhanced (or replaced) with NuPIC > > https://github.com/brohrer/becca/blob/master/core/agent.py > > > On Mon, Aug 19, 2013 at 8:25 PM, Thompson, Jeff <[email protected]>wrote: > >> Thank you for the quick reply. I'm in Los Angeles, so I hope someone >> does record your presentation at NASA. >> >> A similar question arose when I read "Thinking, predicting, and doing >> are all part of the same unfolding of sequences moving down the cortical >> hierarchy." (On Intelligence, p. 158.) I'm sure this is a FAQ, but do you >> have some writings or presentations about how a CLA would receive feedback >> signals coming down the hierarchy? >> >> Thank you, >> - Jeff >> >> >> From: Jeff Hawkins <[email protected]> >> Reply-To: "NuPIC general mailing list." <[email protected]> >> Date: Sunday, August 18, 2013 11:35 AM >> To: "'NuPIC general mailing list.'" <[email protected]> >> Subject: Re: [nupic-dev] Mentioned presentation on action with CLA? >> >> Jeff,**** >> >> I wrote this presentation a couple years ago for a workshop on sensory >> motor integration. That workshop was held at the Santa Fe institute and I >> don’t believe it was recorded. The genesis of the workshop was a paper >> written by Murray Sherman and Raymond Guillery where they point out that >> every region of the neocortex (as far as they have looked) has cells in >> layer 5 that have a motor function. The big idea is that every region of >> neocortex does sensory inference and generates behavior. There are no pure >> “sensory” regions and no pure “motor” regions. It is one of those >> beautiful results that make you slap your head and say “of course!”**** >> >> **** >> >> I have always envisioned the CLA as modeling a section layer 3 in a >> region of the neocortex. Layer 3 is the primary input layer and is >> therefore doing inference on the input to that cortical region. Layer 5 is >> driven by layer 3 and has the cells that innervate muscles, or more often >> project to some sub-cortical area that generates behavior. I see how two >> CLAs, one for layer 3 and the other for layer 5 can work together to learn >> a sensory motor model of the world where today’s CLA is purely sensory. >> There is a lot I don’t understand but there is enough that I think we can >> make progress. **** >> >> **** >> >> I gave this presentation again earlier this year at Numenta. It wasn’t >> recorded. It looks like I might give it again this fall at NASA Ames here >> in Silicon Valley as there are a few roboticists there interested in it.* >> *** >> >> **** >> >> I don’t mind recording it if someone could take care of the logistics.*** >> * >> >> Jeff **** >> >> **** >> >> *From:* nupic >> [mailto:[email protected]<[email protected]>] >> *On Behalf Of *Thompson, Jeff >> *Sent:* Saturday, August 17, 2013 4:57 PM >> *To:* NuPIC general mailing list. >> *Subject:* [nupic-dev] Mentioned presentation on action with CLA?**** >> >> **** >> >> Hello. **** >> >> **** >> >> In the introduction for the NuPIC Hackathon Kickoff, Jeff Hawkins talks >> briefly about the need for CLA integration with action. In response to a >> question, he says "We haven't done experiments with motor interaction. I >> have a presentation, I think about it." Is the presentation about motor >> interaction with CLA available?**** >> >> >> http://www.youtube.com/watch?feature=player_detailpage&v=yShNQvJEP6A&t=2188 >> **** >> >> **** >> >> Thank you,**** >> >> - Jeff Thompson **** >> >> _______________________________________________ >> nupic mailing list >> [email protected] >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
