Re: Bogus Neuroscience [WAS Re: [agi] Human memory and number of synapses]

[Richard Loosemore]

Edward W. Porter wrote:
Richard,
I am aware of the type-token distinction, and I think the distinction 
between the class of Diet Coke cans and the particular physical object 
can_1 I discussed in my prior email is, in fact, an example of just such 
a distinction.

If not, please to explain to me why it is not.

I will explain, as you request.

The "type-token" distinction you refer to is one aspect of how types and 
tokens manifest themselves, but it is not the sense that is relevant to 
the discussion we were having ... and it is because you did not perceive 
the way it was relevant, that I dismissed your last post so abruptly and 
suggested further reading.

The issue in the case of Granger's model of cognition is that there are 
some types of system in which the arrival of a percept (e.g. image of a 
coke can) causes the [coke can] node to become activated.

So far so good, but what if two coke cans are seen?  In these primitive 
types of system, two coke cans simply cause the ONE [coke can] to fire 
some more.  How does the system represent the appearance of two things 
that belong to the same category?  Simple answer:  in Granger's system 
and in other simple systems of this sort, there is NO provision for 
representing two coke cans, because all that can happen is that the one 
[coke can] node fires when there is evidence for any number of coke cans 
in the visual field.

Ideally, we would like the system to have spare nodes available (this is 
the Good Old fashioned AI way of doing things), so the system can assign 
[node_32718] to the first coke can, and [node_32719] to the second coke 
can, and then make links back to the generic [coke can] node (with the 
links probably labelled with the IS-A marker).

Trouble with that way of doing things is this:  well, in fact, there 
isn't anything particularly wrong with this way of doing things, EXCEPT 
that Granger's naive idea of a neural representation scheme makes 
absolutely no provision for temporary nodes that can be used in this way 
to represent instances.  His system cannot do this. The neural machinery 
to do this, if it existed in his model of cognition, would be utterly 
crucial to the operation of the system, but it is not there.  He leaves 
it out.  He seems unaware that some such machinery is needed.

One of the main reasons to ask about this issue of how temporary nodes 
are deployed to represent instances is that IF such temporary nodes 
exist, THEN more than likely they are not simply fixed clusters of 
neurons.  If they were just fixed clusters, how would they be recruited, 
how would they be transiently connected to the generic [coke can] nodes, 
how could they later develop and become more permanent, or how could 
they be recycled for use as other temporary concepts?  A whole rat's 
nest of issues is raised by the choice about how to represent instances. 
 And the worst part of this rat's nest of issues is that whichever way 
you choose to resolve the problem, the simple idea of {one concept 
equals one fixed cluster of neurons} will probably have to be abandoned.

Does Granger deal with this?  Does he show any sign that he understands 
this?  No.  Instead, he naively assumes that clusters of neurons 
represent concepts like [coke can], and he talks as if the only 
important thing is to see that the [coke can] cluster has its activation 
increased when a coke can is seen (or thought about).

So, to summarize.  If Granger means us to take his model as it stands, 
then there are single, fixed clusters of neurons encoding concepts, and 
the system cannot represent multiple instances 9and as a model of 
cognition, therefore, it is Dead On Arrival).  If, on the other hand, 
someone modifies or extends Granger's model so that it can deal with 
multiple instances, the modifications would have to be so extensive that 
there is a very high probability that everything he said in his paper 
would have to be thrown out, and he would have to start all over again.

That is the "type/token" problem I was talking about.  I was not making 
any reference to any of the other type/token issues that you were 
alluding to.

If you look back at Ben Goertzel's responses to myself and Mark Waser, 
you will see that he concedes that the cognitive psychology component of 
Granger's paper is "largely BS".  It is issues such as this one that, I 
believe, he had in mind when he said that.  Ben actually went further 
and suggested that some of Susan Greenfield's proposals might be used to 
resolve the problem ... but he then adds that, of course, this was not 
what Granger was proposing.


*****

I have gone back and re-read your previous post in which you explain in 
detail your thoughts about the differences between coke cans seen in 
different contexts.  The problem (and I hope you can see this now) is 
that none of it makes any difference because that was not the issue I 
raised:  I talked only about the ability of Granger's model to encode 
any of the distinctions you talked about.  My beef with Granger is that 
he implicitly uses a model that cannot handle the representation of 
multiple instances of the same type of object.

*****

Now, please believe me, but when I first wrote about how bad I thought 
Granger's paper was, I did not have the slightest intention of 
criticising you:  I was not rude to you, made no personal attack on you, 
nor implied you were stupid.  I simply launched an attack on the content 
of his paper.  I was shocked when you came back and seemed to take it 
personally that I had attacked his paper, because I did not put any 
blame on you at all.  Since then I have tried to keep emotion out of the 
discussion: if you read our sequence of exhanges you will see that I 
have been brief and to the point, and I have disagreed, but the only way 
you can see rudeness is if you choose to find rudeness in what was 
actually an attempt to deal with complex issues in a brief span of 
words.  I am certainly annoyed at the general problem caused by papers 
such as Granger's, but that annoyance should not be mistaken for a 
personal attack on you.

The only point at which I became testy was in that last message when you 
really did start to make personal remarks.  I mean, come on:  all those 
sarcastic references to the word "DEEP"?  Those implications that I was 
simply being closed minded?  Did I engage in such sarcasm?

(When I used the emphasized word "DEEP", by the way, I was meaning "This 
is a heck of a big issue to get into, and I dread the thought of having 
to explain the whole thing in posts to this list, because then we will 
be here all week."  That was all.)

Please review the sequence if you are not convinced of this:  I intended 
no personal rudeness, and other than a little impatience with the amount 
of time I was spending on the matter, I do not believe I showed any.

And then, I did indeed suggest that you should do some more reading.

Do you not think, in the light of the explanation I have just given 
above, that it would be fair to say that the real issue I was addressing 
(Grangers' model's inability to handle particular types of encodings) 
was not actually in your mind when you replied?  Is it not true that you 
did indeed miss the point completely?  And if a person misses the point 
in such a significant way, several posts in a row, AND starts to become 
angry and tell me that I should stop being so closed-minded, would it 
not be fair, under those circumstances, that I run out of energy to 
write an entire textbook to explain, and instead just say that you 
really should do some more reading?

My suggestion about the reading was a last resort.

Finally, do not drag my interactions with other people into this 
discussion, please.  Some people here have known me for a long time, and 
some have axes that they like to get out and grind.  As I have said 
before, when people mount unprovoked attacks, I tend to respond.  Simple 
as that.  If you want to know more about the reason why, email me in 
private.


Regards,

 (... and nevertheless, with respect,)


Richard Loosemore



With regard you P.S., I agree very much with its general thrust. I have 
normally attempted to avoid attacking people themselves.  In fact, 
except for issue were I feel it is important to fight hard for a 
paradigm shift, such as in fighting the small machine mindset, I 
normally try to be relatively tempered even in my critiques.

BUT RICHARD HAVE YOU ANY IDEA HOW INSULTING AND DISMISSIVE THE TONE OF 
MANY OF YOUR OWN POSTS HAVE BEEN -- NOT ONLY TO ME -- BUT TO OTHERS?  I 
would guess at least 1/3 of your many posts in this thread have either 
explicitly or implicitly been more insulting than my language you 
complained of below. 

For example, in the below post you implied I am too dumb to know what 
the type-token distinction is and need to do some reading to understand 
it, with the implication you know much more on the subject than I.  This 
is when my response, if you actually took the time to read it, indicates 
I was not only aware of the distinction, but directly addressing it 
(although, in the interest of space and time, perhaps not every possible 
ramification of it).  

In sum, such an implication is quite an insult. 

I emphasized the word “DEEP” with implied reverb in the post before not 
because it was incorrect to label the issue as deep, in fact, I thought 
it was deep, but because of the implication that it was too “DEEP” for 
me, but not you, with our current respective levels of knowledge, to 
understand. 

Are you capable of understanding how that might be considered insulting?

But that is nothing compared to some of the dismissive language I have 
read in some of your responses to others.

So, Richard, I was only trying to give you a hint -- so a note like this 
would not be necessary -- to treat others a little more the way you 
apparently want them to treat you. 

If we could all do that, perhaps we could have a little more light and a 
little less darkness on this list.

Ed Porter


-----Original Message-----
From: Richard Loosemore [_mailto:[EMAIL PROTECTED]
Sent: Monday, October 22, 2007 8:21 PM
To: agi@v2.listbox.com
Subject: Re: Bogus Neuroscience [WAS Re: [agi] Human memory and number 
of synapses]


Edward W. Porter wrote:
 Richard,

 I will only respond to the below copied one of the questions in your
 last message because of lack of time.   I pick this example because it
 was so “DEEP” (to be heard in your mind with max reverb).  I hoped
 that
 if I could give a halfway reasonable answer to it and if, just maybe,
 you could open your mind (and that is one of the main issue in this
 thread), you might actually also try to think how your other questions
 could be answered.

 In response to this “DEEP” question, I ask "How do you, Richard
 Loosemore, normally distinguish different instances of a given type."

Okay, I have to stop you right there.

I pointed out the question of type-token distinctions because it has
been a serious issue for a long time (decades) and anyone who wants to
understand AI systems or models of cognition at all has to know what it
is and what its ramifications are.

By saying that it is a "DEEP" issue I was inviting you to do some
reading, not to make a reverb happen inside your head.

You can find a good summary of it in many places, but one is in the
second volume of the Parallel Distributed Processing set (McClelland and
Rumelhart, 1986), chapter 26 (a chapter by Donald Norman).

Granger's proposal makes no mention of how to handle multiple instances,
and IMPLICITLY refers to a type of system that is known to be incapable
of handling multiple instances.


Richard Loosemore

P.S.  Why is it necessary to personalize this issue by comments such as
"... if, just maybe, you could open your mind (and that is one of the
main issue in this thread)..."?






 By distinguishing characteristics?  (This would include things like
 little dings on your car or the junk in its back seat that distinquish
 it from a similar make and model of the same years and color. )

 If so, that is handled by Granger’s system in the manner described in
 my
 response to the question copied below.

 Now when you are dealing with objects that have an identical
 appearance,
 such as Diet Coke cans (the example I normally use when I think of this
 problem), often the only thing you can distinguish them by is – again –
 their distinguishing characteristics.  But in this case the
 distinguishing characteristics would be things like their location,
 orientation, or perhaps relationship to other objects.  It would also
 include implications that can properly be drawn from or about such
 characteristics for the type of thing involved.

 For example, if you leave a Diet Coke can (can_1) downstairs in your
 kitchen and go up to you bedroom and see an identical looking coke can
 next to your bed, you would normally assume the can next to your bed was
 not can_1, unless you had some explanation for how can_1 was moved next
 to your bed.   (For purposes of dealing with the hardest part of the
 problem we will assume all coke cans have been opened and have the same
 amount of coke with roughly the same level of carbonation.)  If you go
 back down stairs and see a Diet Coke can exactly where you left can_1,
 you will assume it is can_1, itself, barring some reason to believe the
 can might have been replaced with another, such as if you know someone
 was in your kitchen during your absence.

 All these types of inferences are based on generalities, often
 important
 broad generalities like the persistence of objects, that take the
 learning of even more basic or more primiative generalities (such as
 those needed for object recognition, understanding the concept of
 physical objects,  the ability to see similarities and dissimilarities
 between objects, and spatial and temporal models), all of which take
 millions of trillions of machine opps and weeks or months of experience
 to learn.  So I hope you will forgive me and Granger if we don’t explain
 them in detail.  (Goertzel in "Hidden Pattern", I think it is, actually
 gives an example of how an AGI could learn object persistence.)

 However, the whole notion of AGI is built on the premise that such
 things can be learned by a machine architecture having certain
 generalized capabilities and having something like the physical world to
 interact in and with.  Those of us who are bullish on AGI think we
 already have a pretty good ideas how to make system that can have the
 required capabilities to learn such broad generalities, or at least get
 us much closer to such a system, so we can get a much better
 understanding of what more is needed, and then try to add it.

 With such ideas of how to make an AGI, it become much easier to map
 the
 various aspects of it into known, or hypothesized, operations in the
 brain.  The features described in Granger’s paper, when combined with
 other previous ideas on how the brain could function as an AGI, would
 seem to describe a system having roughly the general capability to learn
 and properly inference from all of the basic generalizations of the type
 I described above, such as the persistence of objects, and what types of
 objects move on their own, and with what probabilities under what
 circumstances. For example, Granger's article explains how to learn
 patterns, generalizations of pattersn, patterns of generalizations of
 patterns, and with something like a hippocampus it could learn episodes,
 and then patterns from episodes, and generalizations from patterns from
 episodes, and patterns of generalazations from episodes, etc.

 Yes, the Granger article, itself, does not describe all of the
 features
 necessary for the brain to act as a general AGI, but when interpreted in
 the context of enlightened AGI models, such as Novamente, and the
 current knowledge and leading hypotheses in brain science, it is easy to
 imagine how what he describes could play a very important role in
 solving even mental problems as “DEEP” (again with reverb) as that of
 determining whether the Diet Coke can on the table is the one you have
 been drinking from, or someone else’s.

 Has there been a little hand waving in the above explanation?  Yes,
 but
 if you have a good understanding of AGI and its brain equivalent, you
 will understand the amount of hand waving is actually rather limited.

 Ed Porter


 ============= from prior post ====================

>  “RICHARD>> “How does it cope with the instance/generic distinction?”
>
>             I assume after the most general cluster, or the cluster
>             having the most activation from the current feature set,
>             spreads its activation through the matrix loop, then the
>             cluster most activated by the remaining features spreads
>             activation through the matrix loop.  This sequence can
>             continue to presumably any desired level of detail supported
>             by the current set of observed, remembered, or imagined
>             features to be communicated in the brain.  The added detail
>             from such a sequence of descriptions would distinguish an
>             instance from a generic description reprsented by just one
>             such description..

 A misunnderstanding:  the question is how it can represent multiple
 copies of a concept that occur in a situation without getting confused
 about which is which.  If the appearance of one chair in a scene
 causes the [chair] neuron (or neurons, if they are a cluster) to fire,
 then what happens when you walk into a chair factory?  What happens
 when you try to understand a sentence in which there are several
 nouns:  does the [noun] node fire more than before, and if it does,
 how does this help you parse the sentence?

 This is a DEEP issue:  you cannot just say that this will be handled
 by other neural machinery on top of the basic (neural-cluster =
 representation of generic thing) idea, because that "other machinery
 is nontrivial, and potentially it will require the original
 (neural-cluster = representation of generic thing) idea to be
 abandoned completely.

 -----Original Message-----
 From: Richard Loosemore [_mailto:[EMAIL PROTECTED]
 Sent: Monday, October 22, 2007 2:55 PM
 To: agi@v2.listbox.com
 Subject: Re: Bogus Neuroscience [WAS Re: [agi] Human memory and number
 of synapses]


 Edward W. Porter wrote:
>  Dear Readers of the RE: Bogus Neuroscience Thread,
>
>  Because I am the one responsible for bringing to the attention of
> this  list the Granger article (“Engines of the brain: The
> computational  instruction set of human cognition”, by Richard
> Granger) that has caused  the recent  kerfuffle, this morning I took
> the time to do a reasonably  careful re-read of it.
>
>  [snip]
>
>  In his Sun 10/21/2007 2:12 PM post Richard Loosemore cited failure
> to  answer the following questions as indications of the paper’s
 worthlessness.
>
>  “RICHARD>> “How does it cope with the instance/generic distinction?”
>
>             I assume after the most general cluster, or the cluster
>             having the most activation from the current feature set,
>             spreads its activation through the matrix loop, then the
>             cluster most activated by the remaining features spreads
>             activation through the matrix loop.  This sequence can
>             continue to presumably any desired level of detail supported
>             by the current set of observed, remembered, or imagined
>             features to be communicated in the brain.  The added detail
>             from such a sequence of descriptions would distinguish an
>             instance from a generic description reprsented by just one
>             such description..

 A misunnderstanding:  the question is how it can represent multiple
 copies of a concept that occur in a situation without getting confused
 about which is which.  If the appearance of one chair in a scene
 causes the [chair] neuron (or neurons, if they are a cluster) to fire,
 then what happens when you walk into a chair factory?  What happens
 when you try to understand a sentence in which there are several
 nouns:  does the [noun] node fire more than before, and if it does,
 how does this help you parse the sentence?

 This is a DEEP issue:  you cannot just say that this will be handled
 by other neural machinery on top of the basic (neural-cluster =
 representation of generic thing) idea, because that "other machinery
 is nontrivial, and potentially it will require the original
 (neural-cluster = representation of generic thing) idea to be
 abandoned completely.

>
>  “RICHARD>> “How does it allow top-down processes to operate in the 
> recognition process?”
>
>             I don’t think there was anything said about this, but the
>             need for, and presence in the brain of, both top-down and
>             bottom-up processes is so well know as to have properly been
>             assumed.

 Granted, but in a system in which the final state is determined by
 expectations as well as by incoming input, the dynamics of the system
 are potentially completely different, and all of Granger's assertions
 about the roles played by various neural structures may have to be
 completely abandoned in order to make allowance for that new dynamic.


>  “RICHARD>> “How are relationships between instances encoded?” ”
>
>             I assume the readers will understand how it handles temporal
>             relationships (if you add the time dilation and compression
>             mentioned above).  Spatial relationships would come from the
>             topology of V1 (but sensed spatial relationships can also be
>             build via a kohonen net SOM with temporal difference of
>             activiation time as the SOM’s similarity metric).
>             Similarly, other higher order relationships can be built
>             from patterns in the space of hierarchical gen/comp pats
>             networks derived from inputs in these two basic dimensions
>             of space and time plus in the dimensions defined by other
>             sensory, emotional, and motor inputs.  [I consider motor
>             outputs as a type of input].

 Again, no:  relationships are extremely dynamic:  any two concepts can
 be linked by a relationship at any moment, so the specific question
 is, if "things" are represented as clusters of neurons, how does the
 system set up a temporary connection between those clusters, given
 that there is not, in general, a direct link between any two neurons 
in the brain?
  You cannot simply "strengthen" the link between your "artichoke"
 neuron and your "basilisk" neuron in order to form the relationship
 caused by my mention of both of them in the same sentence, because, in
 general, there may not be any axons going from one to the other.


>  “RICHARD>> “How are relationships abstracted?”
>
>             By shared features.  He addresses how clusters tend to form
>             automatically.  These clusters are abstractions.

 These are only clusters of "things".  He has to address this issue
 separately for "relationships" which are connections or links between
 things.  The question is about "types" of links, and about how there
 are potentially an infinite number of different types of such links: 
 how are those different types represented and built and used?  Again,
 a simple neural connection is not good enough, because there would
 only be one possible type of relationship in your thoughts.


>  “RICHARD>> “How does position-independent recognition occur?”
>
>             He deals with this.  His nodes are nodes in a hierarchical
>             memory that provides degrees of position and shape
>             invariance, or the type mentioned by Hawkins and the Serre
>             paper I have cited so many times.  Granger’s figures 6 and 7
>             indicates exactly this type of invariance.

 I have not looked in detail at this, but how does his position
 invariance scale up?  For example, if I learn the new concept of "floo
 powder", do I now have to build an entire set of neural machinery for
 the all the possible positions on my retina where I might see "floo
 powder"?  If the answer is yes, the mechanism is bankrupt, as I am
 sure you realise:  we do not have that much neural machinery to
 dedicate to it.


>  “RICHARD>> “What about the main issue that usually devastates any 
> behaviorist-type proposal:  patterns to be associated with other 
> patterns are first extracted from the input by some (invisible,
>  unacknowledged) preprocessor, but when the nature of this
> preprocessor  is examined carefully, it turns out that its job is
> far, far more  intelligent than the supposed association engine to
> which it delivers  its goods?
>
>             What he feeds to his system are things like the output of
>             Gabor filters.  I don’t think a Gabor filter is something
>             that is “far, far, more intelligent than the supposed
>             association engine to which it delivers its goods.”

 He has to show that the system is capable, by itself, of picking up
 objects like the "letter A" in a scene without the programmer of the
 simulation giving it some hint.  The fact that he uses Gabor filters
 does not bear on the issue, as far as I can see.

 This issue is more subtle than the others.  Too much for me to go into
 in great detail, due to time constraints.  Suffice it to say that you
 do not really address the issue I had in mind.


>  This is just an example of how a serious attempt to understand what
> is  good in Granger’s paper, and to expand on those good features,
> overcomes  a significant number of the objections raised by those
> whose major  motivation seems to be to dismiss it.

 I think I have shown that none of my objections were overcome, alas.


>  Wikipedia, that font of undisputed truth, defines Cognitive science
> as
>
>             “Cognitive science is most simply defined as the scientific
>             study either of mind or of intelligence (e.g. Luger 1994).
>             It is an interdisciplinary study drawing from relevant
>             fields including psychology, philosophy, neuroscience,
>             linguistics, anthropology, computer science, biology, and
>             physics”
>
>  Based on this definition I would say the cognitive science aspect of 
> Granger’s paper, although speculative and far from fully fleshed out,
> is  actually quite good.

 Cognitive science is more than just saying a few things that seem to
 come from a selction of these fields.

 I would welcome further discussion of these issues, but it might be
 better for me to point to some references in which they are discussed
 properly, rather than for me to try to do the whole job here.

-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?member_id=8660244&id_secret=56876873-9ed808