[agi] Visual Reasoning Part 1 The Scene
RE: [agi] A 1st Step To Using Your Image-inationEd & Co, Rather than answer
your objections directly, I propose to :
1) In this post, demonstrate that visual reasoning while still regarded by our
culture generally (& not just AI/AGI) as a minimal and peripheral part of our
thinking actually plays a massive and fairly continuous part in our life - & we
are extremely unaware of it
2) In an accompanying post, not only provide some more dramatic examples of
visual reasoning, but provide proof that visual processing cannot be handled by
symbolic processing (or not to any serious, practical extent)
If you still object not unreasonably, given current attitudes, that this is all
peripheral to AGI, I will then in a day or two:
3) demonstrate that all this lies at the dead centre of AGI and most, if not
all of its unsolved problems..
{Please start downloading this file:
http://www.mediafire.com/?2wxyn5rjdyq
Don't open yet, but if it doesn't work, post immediately!!- the rest is
pointless without it]
I think we can agree that our culture regards visual reasoning as a pretty
peripheral part of thinking generally and our life. For example, it is fairly
standard in psychology textbooks to ask whether thinking and language are not
identical/interdependent. The main point here is that while people know we
don't only think in language (and symbols), they have a generally hard time
talking about other forms of thinking, or instancing them in any detail. Even
I, who have enormous sympathy with my own opinions, have had a hard time
explaining the importance of visual and common sense (literally
all-the-senses-together) thinking - and didn't even realise till a recent
exchange with Pei, how massively important observation-as-reasoning, (incl.
visual reasoning) is.
It is not uncommon for even a highly educated psychologist to say something
like: "I only think in language; I never think in visuals."
So I would like you to engage in some visual reasoning - and I think you'll
find that you won't be able to help it - it happens automatically.
I'd like you in a minute to look at the slideshow of visuals in that file., and
as you do,observe yourself as best you can. What I think you'll find is that
you don't look at any photo as a "shot" but rather as a "scene" - a story in
pictures - with a before and after. And it's quite remarkable how much you do
infer about each photo - how you can and do:
-predict to some extent what subjects are likely to do next
-detect what subjects may have done just before
-identify where the scene is taking place
and could, if asked, fill in a whole story around the photos.
Please look at the whole file now!!
And when you've looked, you might start asking yourself more detailed questions
about how you came to work out all you did about those photos.
How do you know where people and animals are likely to move, objects are likely
to move/splash, whether a figure is threatening to reach or actually reaching
for his gun, considering shooting or about to shoot a rifle, what those girls
on the sofa are trying to do, what those four feet mean, what that man by the
sea is looking at and even what mood he might be in, how that woman dancing is
talking to the man and how he is reacting, why that lovers' embrace is
particularly hot, why that man is a drunk,how a child or the cat will play that
piano and even react and what noises she may make, what those people in the
dark are looking at, and so on ...?
One thing's for sure: you are doing a lot of visual reasoning.
And in fact, you are doing visual reasoning all day long - reasoning -
composing stories-in-pictures about what has just happened and is about to
happen in front of you - where objects are going to move, or how they've just
moved, (fallen on the floor), how the people around you are about to move, how
fast they will approach you and whether that car might hit you, what their
expressions mean, and whether they are likely to be friendly or come on or be
angry, and how fast that blood may coagulate, whether that light indicates
someone is in a room, whether the clouds indicate rain, whether those people
are grouping together in friendship or to fight, whether that shop attendant is
going to take too long etc etc.
And all day long you are in effect doing tacit physics, chemistry, biology,
psychology, sociology about the world around you. But almost none of it
involves formal reasoning that any of those disciplines could explain. They
couldn't begin to tell you for example how you work out visually how things and
animals and people are likely to behave - how you read the emotional
complexities of a face - how someone is straining that smile too hard. There
are no formulae that can tell you just by looking whether that suitcase is
likely to be too heavy.
All of this is visual and common-sense reasoning, most of which you'd be v.
hard put to explain verbally let alone mathematica
Re: [agi] Visual Reasoning Part 1 The Scene
It seems fairly obvious to me that a large component of "thinking" is based
upon visually derived concepts. In evolutionary terms language arrived late
to the party, or only existed in primitive low bandwidth forms of
communication.
If you think only in terms of linguistics many common techniques used in the
movies will pass you by, and silent movies will be totally
incomprehensible. Even worse, your ability to navigate around and recognise
common situations may be severely impaired.
This doesn't mean that language is unimportant. The addition of language
effectively supercharges the evolutionary earlier modes of thought,
permitting advanced features such as being able to count large numbers of
things. One of my favourite books "ape, primitive man and child" deals with
this transition from pre-linguistic to linguistically augmented
intelligence.
On 16/02/2008, Mike Tintner <[EMAIL PROTECTED]> wrote:
>
> Ed & Co, Rather than answer your objections directly, I propose to :
>
> 1) In this post, demonstrate that visual reasoning while still regarded by
> our culture generally (& not just AI/AGI) as a minimal and peripheral part
> of our thinking actually plays a massive and fairly continuous part in our
> life - & we are extremely unaware of it
>
> 2) In an accompanying post, not only provide some more dramatic examples
> of visual reasoning, but provide proof that visual processing cannot be
> handled by symbolic processing (or not to any serious, practical extent)
>
> If you still object not unreasonably, given current attitudes, that this
> is all peripheral to AGI, I will then in a day or two:
>
> 3) demonstrate that all this lies at the dead centre of AGI and most, if
> not all of its unsolved problems..
>
> {Please start downloading this file:
>
> http://www.mediafire.com/?2wxyn5rjdyq
>
> Don't open yet, but if it doesn't work, post immediately!!- the rest is
> pointless without it]
>
> I think we can agree that our culture regards visual reasoning as a pretty
> peripheral part of thinking generally and our life. For example, it is
> fairly standard in psychology textbooks to ask whether thinking and language
> are not identical/interdependent. The main point here is that while people
> know we don't only think in language (and symbols), they have a generally
> hard time talking about other forms of thinking, or instancing them in any
> detail. Even I, who have enormous sympathy with my own opinions, have had a
> hard time explaining the importance of visual and common sense (literally
> all-the-senses-together) thinking - and didn't even realise till a recent
> exchange with Pei, how massively important observation-as-reasoning, (incl.
> visual reasoning) is.
>
> It is not uncommon for even a highly educated psychologist to say
> something like: "I only think in language; I never think in visuals."
>
> So I would like you to engage in some visual reasoning - and I think
> you'll find that you won't be able to help it - it happens automatically.
>
> I'd like you in a minute to look at the slideshow of visuals in that
> file., and as you do,observe yourself as best you can. What I think you'll
> find is that you don't look at any photo as a "shot" but rather as a "scene"
> - a story in pictures - with a before and after. And it's quite remarkable
> how much you do infer about each photo - how you can and do:
>
> -predict to some extent what subjects are likely to do next
> -detect what subjects may have done just before
> -identify where the scene is taking place
>
> and could, if asked, fill in a whole story around the photos.
>
> Please look at the whole file now!!
>
> And when you've looked, you might start asking yourself more detailed
> questions about how you came to work out all you did about those photos.
>
> How do you know where people and animals are likely to move, objects are
> likely to move/splash, whether a figure is threatening to reach or
> actually reaching for his gun, considering shooting or about to shoot a
> rifle, what those girls on the sofa are trying to do, what those four feet
> mean, what that man by the sea is looking at and even what mood he might be
> in, how that woman dancing is talking to the man and how he is reacting, why
> that lovers' embrace is particularly hot, why that man is a drunk,how a
> child or the cat will play that piano and even react and what noises she may
> make, what those people in the dark are looking at, and so on ...?
>
> One thing's for sure: you are doing a lot of visual reasoning.
>
> And in fact, you are doing visual reasoning all day long - reasoning -
> composing stories-in-pictures about what has just happened and is about to
> happen in front of you - where objects are going to move, or how they've
> just moved, (fallen on the floor), how the people around you are about to
> move, how fast they will approach you and whether that car might hit you,
> what their expressions mean, and whether they are likely to be friendly
Re: [agi] Visual Reasoning Part 1 The Scene
Bob:It seems fairly obvious to me that a large component of "thinking" is based upon visually derived concepts. In evolutionary terms language arrived late to the party, or only existed in primitive low bandwidth forms of communication. Hi, Bob, I'm particularly interested in your response here. Yes, to the first part - and concepts are actually labels for complex clusters of images. But what interests me especially is how you respond to what I think is probably my most important assertion re you - and that is : we don't actually see "shots" in any photo or any real scene period - i.e. we don't see pure spatial arrangements of objects. We see every shot as a "still" - i.e. a slice of a movie sequence. Movies and not photos are actually the primary medium of conscious thought - and we delight in photos precisely because they stop - and "capture" - parts of the flow. If that's true, doesn't it have enormous consequences for Visual Object Recognition - and which if any people are thinking along those lines? (Remember Hawkins' insistence that we have to see an object in *motion* to understand it - which may in some cases means that the motion comes from the observer). No question re the evolutionary thing - formal symbols are a last-minute-before-midnight arrival to the party - which makes the insistence that symbols can do the whole job utterly absurd. Evolution ain't stupid. If it took billions of years perfecting the movie/ image parts of consciousness first, that's because that's the way it had to be, incredibly hard as symbol-ists and literate minds may at first find it to believe. --- agi Archives: http://www.listbox.com/member/archive/303/=now RSS Feed: http://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: http://www.listbox.com/member/?member_id=8660244&id_secret=95818715-a78a9b Powered by Listbox: http://www.listbox.com
Re: [agi] Visual Reasoning Part 1 The Scene
On 16/02/2008, Mike Tintner <[EMAIL PROTECTED]> wrote: > > you respond to what I think is probably my most important assertion re > you - and that is : we don't actually see "shots" in any photo or any real > scene period - i.e. we don't see pure spatial arrangements of objects. We > see every shot as a "still" - i.e. a slice of a movie sequence. Movies > and not photos are actually the primary medium of conscious thought - and we > delight in photos precisely because they stop - and "capture" - parts of the > flow. If that's true, doesn't it have enormous consequences for Visual > Object Recognition - and which if any people are thinking along those lines? > (Remember Hawkins' insistence that we have to see an object in *motion* to > understand it - which may in some cases means that the motion comes from the > observer). > What I think these pundits are hinting at is that in order to understand still images you need to have initially had embodied experience moving about in some environment. Embodied experience allows us to tune the probability distributions for certain types of visual feature, correlating them with 3D shape. Structure form motion allows us to estimate 3D shape, and this can be associated with the features we can see in 2D (such as lines or corners) so that we can later use constellations of these to do a quick lookup or a best guess. There are obvious evolutionary reasons for this kind of shape lookup, especially if you're being chased by a predator and need to make hasty decisions. --- agi Archives: http://www.listbox.com/member/archive/303/=now RSS Feed: http://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: http://www.listbox.com/member/?member_id=8660244&id_secret=95818715-a78a9b Powered by Listbox: http://www.listbox.com
Re: [agi] Visual Reasoning Part 1 The Scene
Mike,
It is well recognized that as well as declarative and procedural
memory, the human
brain contains a substantial "episodic memory" aspect, which stores some sort of
abstracted "movies" of a mind's history. Clearly, matching of
abstracted-movie-subsets against
others is important, and variation manipulation processes on these
abstracted-movie-subsets
And clearly, humans possess the ability to use this component metaphorically
and imaginatively, beyond our actual experience.
That much is well-recognized among pretty all cognitive psychologists
What you're claiming seems to be that
1)
In the human brain, this episodic/visual faculty
is not just one component among many important ones,
but rather the most central component, which needs to be understood in
order for the others
to make any sense. This is a stronger claim which I don't really agree with.
2)
AGI's need to emulate the human brain in including internal visual
episodic memory and associated abstracted manipulations in a critical
role. This I am almost certain is wrong.
Then you go on to allude that
3)
Somehow all this episodic, visual processing cannot be done by
"programs." Here you totally lose me.
Regarding why the visual/episodic component of cognition has received
relatively little attention in the AI field, I guess there are two
main reasons, neither of which are that people find this aspect
uninteresting;
1)
Computer vision, which is an extremely active area, has not yet
succeeded fully enough to let us really successfully abstract visual
forms from images, in the context of real-world data
For instance, identifying the objects in a complex visual scene
remains a hard problem for computer vision system.
There is loads of research $$ going into this, though, far more than
into AGI, so it's hardly a neglected area
2)
Manipulating and storing large databases of movies is expensive and
irritating using current technology
I would guess that as hard drive and processor become cheaper, we will
see more experimentation with the episodic/visual aspects of
intelligence in AI and AGI.
-- Ben G
On Feb 16, 2008 12:20 AM, Mike Tintner <[EMAIL PROTECTED]> wrote:
>
>
> Ed & Co, Rather than answer your objections directly, I propose to :
>
> 1) In this post, demonstrate that visual reasoning while still regarded by
> our culture generally (& not just AI/AGI) as a minimal and peripheral part
> of our thinking actually plays a massive and fairly continuous part in our
> life - & we are extremely unaware of it
>
> 2) In an accompanying post, not only provide some more dramatic examples of
> visual reasoning, but provide proof that visual processing cannot be handled
> by symbolic processing (or not to any serious, practical extent)
>
> If you still object not unreasonably, given current attitudes, that this is
> all peripheral to AGI, I will then in a day or two:
>
> 3) demonstrate that all this lies at the dead centre of AGI and most, if not
> all of its unsolved problems..
>
> {Please start downloading this file:
>
> http://www.mediafire.com/?2wxyn5rjdyq
>
> Don't open yet, but if it doesn't work, post immediately!!- the rest is
> pointless without it]
>
> I think we can agree that our culture regards visual reasoning as a pretty
> peripheral part of thinking generally and our life. For example, it is
> fairly standard in psychology textbooks to ask whether thinking and language
> are not identical/interdependent. The main point here is that while people
> know we don't only think in language (and symbols), they have a generally
> hard time talking about other forms of thinking, or instancing them in any
> detail. Even I, who have enormous sympathy with my own opinions, have had a
> hard time explaining the importance of visual and common sense (literally
> all-the-senses-together) thinking - and didn't even realise till a recent
> exchange with Pei, how massively important observation-as-reasoning, (incl.
> visual reasoning) is.
>
> It is not uncommon for even a highly educated psychologist to say something
> like: "I only think in language; I never think in visuals."
>
> So I would like you to engage in some visual reasoning - and I think you'll
> find that you won't be able to help it - it happens automatically.
>
> I'd like you in a minute to look at the slideshow of visuals in that file.,
> and as you do,observe yourself as best you can. What I think you'll find is
> that you don't look at any photo as a "shot" but rather as a "scene" - a
> story in pictures - with a before and after. And it's quite remarkable how
> much you do infer about each photo - how you can and do:
>
> -predict to some extent what subjects are likely to do next
> -detect what subjects may have done just before
> -identify where the scene is taking place
>
> and could, if asked, fill in a whole story around the photos.
>
> Please look at the whole file now!!
>
> And when you've looked, you might start asking yourself more detailed
> questions about h
Re: [agi] Visual Reasoning Part 1 The Scene
On 16/02/2008, Ben Goertzel <[EMAIL PROTECTED]> wrote: > It is well recognized that as well as declarative and procedural > memory, the human > brain contains a substantial "episodic memory" aspect, which stores some sort > of > abstracted "movies" of a mind's history. Clearly, matching of > abstracted-movie-subsets against > others is important, and variation manipulation processes on these > abstracted-movie-subsets The way I imagine that such experiences might be stored in the brain is as a three dimensional fourier space, with neuron firing rates representing the constituent phases. A few people believe that fourier analysis occurs at the earliest stages of vision (http://www.ghuth.com/) and recent research on stereo vision (http://www.staff.ncl.ac.uk/j.c.a.read/publications/ReadCumming07.pdf) would be consistent with this view. > And clearly, humans possess the ability to use this component metaphorically > and imaginatively, beyond our actual experience. This is why I think different people are able to read a book and understand its contents, because they have a similar back catalogue of linguistic and pre-linguistic experience. The missing pre-linguistic component for automatic text understanding systems suggests a reason why computers cannot yet read and understand even the simple stories read by young children. --- agi Archives: http://www.listbox.com/member/archive/303/=now RSS Feed: http://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: http://www.listbox.com/member/?member_id=8660244&id_secret=95818715-a78a9b Powered by Listbox: http://www.listbox.com
Re: [agi] Visual Reasoning Part 1 The Scene
Ben, Yes I'm making the strong claim re consciousness/thought are movies-based. Yes, I don't begin to try and explain it re programs - I will do a bit more towards that soon - I just wanted to give first a sense how it could be possible, and I hope you agree that that is/has been v. difficult to even give a sense of. What I would suggest - simply by way of opening your mind is: think long and hard about why the brain only dreams in movies. That is such an important fact. (I think it's fairly generally agreed that some kind of important reprocessing and reorganizing of past experiences, as well as preparation for future experiences, is going on). It would, you might think, be so much more efficient to think in just words or other symbols as we do consciously for such long periods. Also - and I have just begun to think about this - we understand those photos, I believe - by "animating" them unconsciously - animating the objects and people in the scene. And if that's true, (which is presumably experimentally testable), then a) we can do it for anything, (as in - every language sentence may be unconsciously processed as a movie scene - and actually, I believe, is ), and b) I'm suggesting we *have* to do it. Difficult as it is for us, still in general largely movemaking-undeducated, (though that will change v. shortly), to believe, you can't understand or think about the world at all if you can't run movies of it. P.S. My v. broad suggestion re the computer hardware/program aspect of all this is simply this: the only way, v. broadly, that our brain can achieve the things I'm claiming, is by being a true *image processor*, by being able to a) overlay and compare images whole, and b) manipulate them with something like the freedom of a sculptor/artist and movie editor. I've no real idea whether digital processors might not be rethought to achieve that, or whether it will require an add-on analog processor or something else entirely. And I wish people would start having ideas about this side of things - one person here actually did respond positively (offline) to my math test, and was stimulated to start thinking in hardware terms how a computer might doodle (presumably, although I can't remember) without coordinates. P.P.S. Bear in mind that we have just (almost literally this year) entered the age of "interactive video" - as well as interactive media generally - and that is an even more important and culturally earthshaking development than that of the printed book from the manuscript. (Interactive video is still almost literally BTW in its birth pangs. It will become vastly more sophisticated). Mike, It is well recognized that as well as declarative and procedural memory, the human brain contains a substantial "episodic memory" aspect, which stores some sort of abstracted "movies" of a mind's history. Clearly, matching of abstracted-movie-subsets against others is important, and variation manipulation processes on these abstracted-movie-subsets And clearly, humans possess the ability to use this component metaphorically and imaginatively, beyond our actual experience. That much is well-recognized among pretty all cognitive psychologists What you're claiming seems to be that 1) In the human brain, this episodic/visual faculty is not just one component among many important ones, but rather the most central component, which needs to be understood in order for the others to make any sense. This is a stronger claim which I don't really agree with. 2) AGI's need to emulate the human brain in including internal visual episodic memory and associated abstracted manipulations in a critical role. This I am almost certain is wrong. Then you go on to allude that 3) Somehow all this episodic, visual processing cannot be done by "programs." Here you totally lose me. Regarding why the visual/episodic component of cognition has received relatively little attention in the AI field, I guess there are two main reasons, neither of which are that people find this aspect uninteresting; 1) Computer vision, which is an extremely active area, has not yet succeeded fully enough to let us really successfully abstract visual forms from images, in the context of real-world data For instance, identifying the objects in a complex visual scene remains a hard problem for computer vision system. There is loads of research $$ going into this, though, far more than into AGI, so it's hardly a neglected area 2) Manipulating and storing large databases of movies is expensive and irritating using current technology I would guess that as hard drive and processor become cheaper, we will see more experimentation with the episodic/visual aspects of intelligence in AI and AGI. -- Ben G On Feb 16, 2008 12:20 AM, Mike Tintner <[EMAIL PROTECTED]> wrote: Ed & Co, Rather than answer your objections directly, I propose to : 1) In this post, demonstrate that visual reasoning whi
Re: [agi] Visual Reasoning Part 1 The Scene
Well, hardware & software development is going pretty fast in this direction due to the video game and movie-viewing market, so I have every expectation that within 5-10 years, commodity hardware will be DAMN powerful at video manipulation... which will be a real boon to this kind of AI ... Right now, even, you can get very nice rack-mounted NVidia GPU's that can do a lot of image and video processing really fast ... ben On Feb 16, 2008 10:48 AM, Mike Tintner <[EMAIL PROTECTED]> wrote: > Ben, > > Yes I'm making the strong claim re consciousness/thought are movies-based. > Yes, I don't begin to try and explain it re programs - I will do a bit more > towards that soon - I just wanted to give first a sense how it could be > possible, and I hope you agree that that is/has been v. difficult to even > give a sense of. What I would suggest - simply by way of opening your mind > is: think long and hard about why the brain only dreams in movies. That is > such an important fact. (I think it's fairly generally agreed that some > kind of important reprocessing and reorganizing of past experiences, as well > as preparation for future experiences, is going on). It would, you might > think, be so much more efficient to think in just words or other symbols as > we do consciously for such long periods. > > Also - and I have just begun to think about this - we understand those > photos, I believe - by "animating" them unconsciously - animating the > objects and people in the scene. > > And if that's true, (which is presumably experimentally testable), then a) > we can do it for anything, (as in - every language sentence may be > unconsciously processed as a movie scene - and actually, I believe, is ), > and b) I'm suggesting we *have* to do it. Difficult as it is for us, still > in general largely movemaking-undeducated, (though that will change v. > shortly), to believe, you can't understand or think about the world at all > if you can't run movies of it. > > P.S. My v. broad suggestion re the computer hardware/program aspect of all > this is simply this: the only way, v. broadly, that our brain can achieve > the things I'm claiming, is by being a true *image processor*, by being able > to a) overlay and compare images whole, and b) manipulate them with > something like the freedom of a sculptor/artist and movie editor. I've no > real idea whether digital processors might not be rethought to achieve that, > or whether it will require an add-on analog processor or something else > entirely. And I wish people would start having ideas about this side of > things - one person here actually did respond positively (offline) to my > math test, and was stimulated to start thinking in hardware terms how a > computer might doodle (presumably, although I can't remember) without > coordinates. > > P.P.S. Bear in mind that we have just (almost literally this year) entered > the age of "interactive video" - as well as interactive media generally - > and that is an even more important and culturally earthshaking development > than that of the printed book from the manuscript. (Interactive video is > still almost literally BTW in its birth pangs. It will become vastly more > sophisticated). > > > > > Mike, > > > > It is well recognized that as well as declarative and procedural > > memory, the human > > brain contains a substantial "episodic memory" aspect, which stores some > > sort of > > abstracted "movies" of a mind's history. Clearly, matching of > > abstracted-movie-subsets against > > others is important, and variation manipulation processes on these > > abstracted-movie-subsets > > > > And clearly, humans possess the ability to use this component > > metaphorically > > and imaginatively, beyond our actual experience. > > > > That much is well-recognized among pretty all cognitive psychologists > > > > What you're claiming seems to be that > > > > 1) > > In the human brain, this episodic/visual faculty > > is not just one component among many important ones, > > but rather the most central component, which needs to be understood in > > order for the others > > to make any sense. This is a stronger claim which I don't really agree > > with. > > > > 2) > > AGI's need to emulate the human brain in including internal visual > > episodic memory and associated abstracted manipulations in a critical > > role. This I am almost certain is wrong. > > > > Then you go on to allude that > > > > 3) > > Somehow all this episodic, visual processing cannot be done by > > "programs." Here you totally lose me. > > > > > > Regarding why the visual/episodic component of cognition has received > > relatively little attention in the AI field, I guess there are two > > main reasons, neither of which are that people find this aspect > > uninteresting; > > > > 1) > > Computer vision, which is an extremely active area, has not yet > > succeeded fully enough to let us really successfully abstract visual > > forms from images, in the context of r
RE: [agi] Visual Reasoning Part 1 The Scene
>Mike Tintner ###> Ed & Co, Rather than answer your objections directly,
I propose to :
1) In this post, demonstrate that visual reasoning while still regarded by
our culture generally (& not just AI/AGI) as a minimal and peripheral part
of our thinking actually plays a massive and fairly continuous part in our
life - & we are extremely unaware of it
Ed Porter ###> Some say roughly 1/3 of the human cortex is dedicated to
visual processing, so I think a lot of AI/Cognitive science types place a
lot of emphasis on visual processing.
Jeff Hawkins AGI approach is starting with modeling vision. In his system
of hierarchical memory, individual nodes learn temporal sequences of
activations of lower or other layer elements in a compositional hierarchy.
At the lower level these these temporal sequences correspond to picture
elements such as edges and movies of them. Higher up the hierarchy the
nodes increasingly become abstracted from specific patterns of pixel
activation. At the higher levels where much of the implication and visual
pun making you make so much of are taking place, the representations are
sufficiently abstract that it is not clear that they are best described as
"images". Such higher level visual processing is much more complicated that
just images, it involves a compositional and generalization hierarchy of
patterns with both temporal, spatial, and semantic aspects. It is not just
about playing movies in our minds, it is about much more, and a lot of that
"more" involves representations that are not directly "images".
And all of the types of representations that are needed -- from pixel level
to the abstract levels that model objects, including people, and their
behavior - can be represented and computed with the states, changes in
states, and communication between the states, of complex structures of
symbols.
So I think your view of the AGI community is based more on GOFAI than the
more modern approaches.
>Mike Tintner ###> 2) In an accompanying post, not only provide some
more dramatic examples of visual reasoning, but provide proof that visual
processing cannot be handled by symbolic processing (or not to any serious,
practical extent)
Ed Porter ###> No such proof was provided.
You did show, as does common sense, that important parts of human thinking
are performed with complex computations, which have many more levels of
complexity than those associated with elementary symbol processing. Human
biology has many more levels of complexity than those described individual
chemical reactions, yet almost all of it can be described by a complex
combination of individual chemical reactions. Similarly just as the human
mind has much more complexity than can be described by individual
computations between symbols, nevertheless, almost all of that complexity
can be described by complex interactions between symbols.
>Mike Tintner ###> 3) demonstrate that all this lies at the dead centre
of AGI and most, if not all of its unsolved problems..
Ed Porter ###> No such demonstration was provided. Clearly an AGI that
was to model human behavior would have to have sophisticated visually
processing, but that could be supplied with complex symbolic processing (as
much surprisingly sophisticated visual processing already been). But AGI
need not be limited to human-like behaviors, and it seems likely the
powerful and useful AGI could be created without visual processing, although
any such system would lack certain desirable features.
>Mike Tintner ###> {Please start downloading this file:
http://www.mediafire.com/?2wxyn5rjdyq
Ed Porter ###> Mike! Now we know why you think visual images are so
important. (I hope you are also getting some Haptic input to.) I just hope
you haven't caused my computer to catch any nasty-porn viruses or other
forms of computer VD.
Ed Porter
-Original Message-
From: Mike Tintner [mailto:[EMAIL PROTECTED]
Sent: Saturday, February 16, 2008 12:21 AM
To: agi@v2.listbox.com
Subject: [agi] Visual Reasoning Part 1 The Scene
Ed & Co, Rather than answer your objections directly, I propose to :
1) In this post, demonstrate that visual reasoning while still regarded by
our culture generally (& not just AI/AGI) as a minimal and peripheral part
of our thinking actually plays a massive and fairly continuous part in our
life - & we are extremely unaware of it
2) In an accompanying post, not only provide some more dramatic examples of
visual reasoning, but provide proof that visual processing cannot be handled
by symbolic processing (or not to any serious, practical extent)
If you still object not unreasonably, given current attitudes, that this is
all peripheral to AGI, I will then in a day or two:
3) demonstrate that all this lies at the dead centre of AGI an
Re: [agi] Visual Reasoning Part 1 The Scene
On 16/02/2008, Mike Tintner <[EMAIL PROTECTED]> wrote: > Pretty at sea here, but could this in any way explain why the image on the > retina is so distorted compared with the actual image we perceive? Yes. From Huth's paper: "This geometrically defined pattern on the retina might then be seen as the actual physical embodiment of a Fourier transform - the translation of frequency/wavelength information into a spatial design" --- agi Archives: http://www.listbox.com/member/archive/303/=now RSS Feed: http://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: http://www.listbox.com/member/?member_id=8660244&id_secret=95818715-a78a9b Powered by Listbox: http://www.listbox.com
Re: [agi] Visual Reasoning Part 1 The Scene
Bob: > The way I imagine that such experiences might be stored in the brain is as a three dimensional fourier space, with neuron firing rates representing the constituent phases. A few people believe that fourier analysis occurs at the earliest stages of vision (http://www.ghuth.com/) and recent research on stereo vision (http://www.staff.ncl.ac.uk/j.c.a.read/publications/ReadCumming07.pdf) would be consistent with this view. Bob, From Huth: "Combined with the equally well understood principles of self-organized molecular lipid bilayers, one can visualize formation of the primordial eye - as the result of light shining through a drop of water?" Pretty at sea here, but could this in any way explain why the image on the retina is so distorted compared with the actual image we perceive? (See Hawkins On Intelligence illustration of beach photo). Or do you have or know of any explanation of that discrepancy? --- agi Archives: http://www.listbox.com/member/archive/303/=now RSS Feed: http://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: http://www.listbox.com/member/?member_id=8660244&id_secret=95818715-a78a9b Powered by Listbox: http://www.listbox.com
Re: [agi] Visual Reasoning Part 1 The Scene
On Feb 16, 2008 11:11 PM, Mike Tintner <[EMAIL PROTECTED]> wrote: > > From Huth: > "Combined with the equally well understood principles of self-organized > molecular lipid bilayers, one can visualize formation of the primordial > eye - as the result of light shining through a drop of water?" > > Pretty at sea here, but could this in any way explain why the image on the > retina is so distorted compared with the actual image we perceive? (See > Hawkins On Intelligence illustration of beach photo). Or do you have or know > of any explanation of that discrepancy? > Mike, This is a tutorial talk I saw recently (see Section A): http://nips.cc/Conferences/2007/Program/event.php?ID=577 Michael Lewicki Sensory Coding and Hierarchical Representations It describes how early stages of visual and auditory processing are optimally tuned to preserve information about outside world under biological and mechanical constraints. Retina owes its form to distortion of image from outside created by the eye before it reaches the retina. What we perceive is a reconstruction of outside world, and as such it much more obviously appears undistorted than image on the retina where you need to employ information theory to appreciate its undistortedness. -- Vladimir Nesov [EMAIL PROTECTED] --- agi Archives: http://www.listbox.com/member/archive/303/=now RSS Feed: http://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: http://www.listbox.com/member/?member_id=8660244&id_secret=95818715-a78a9b Powered by Listbox: http://www.listbox.com
Re: [agi] Visual Reasoning Part 1 The Scene
Vlad: This is a tutorial talk I saw recently (see Section A): http://nips.cc/Conferences/2007/Program/event.php?ID=577 Michael Lewicki Sensory Coding and Hierarchical Representations An outstanding presentation. Thankyou v. much --- agi Archives: http://www.listbox.com/member/archive/303/=now RSS Feed: http://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: http://www.listbox.com/member/?member_id=8660244&id_secret=95818715-a78a9b Powered by Listbox: http://www.listbox.com
Re: [agi] Visual Reasoning Part 1 The Scene
On 17/02/2008, Vladimir Nesov <[EMAIL PROTECTED]> wrote: > This is a tutorial talk I saw recently (see Section A): > http://nips.cc/Conferences/2007/Program/event.php?ID=577 > Michael Lewicki > Sensory Coding and Hierarchical Representations I like this guy's approach, which covers many familiar topics. Particularly I liked the way he treated both vision and auditory systems as essentially the same problem. --- agi Archives: http://www.listbox.com/member/archive/303/=now RSS Feed: http://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: http://www.listbox.com/member/?member_id=8660244&id_secret=95818715-a78a9b Powered by Listbox: http://www.listbox.com
Re: [agi] Visual Reasoning Part 1 The Scene
Yes, great presentation and summary of the work done by these guys at CMU. It reinforces my belief that sensory processing should be explained in terms of finding efficient representations, in other words: dimensionality reduction. One nice aspect is that there are machine learning techniques that do exactly this in a general way, like(Restricted) Boltzmann Machines or adapted ICA. The wonderful thing about this kind of architecture is that at the same time it explains biological phenomena, and performs very well on computer vision tasks like classification and clustering. I'm wondering how much further one could extend such architecture to coding of spatiotemporal (video) patterns, multimodal patterns (video + audio) and eventually coding of 3D objects. They are all 'just' extensions of such a model, 'just' about finding efficient ways of learning the joint probability distributions :) however I imagine that finding efficient ways of training such models (e.g. finding compact representations) should become increasingly hard. The 'ultimate' general computer vision architecture, in my mind, would be series of (stereo) images as input, learns such a hierarchical model (using dimensionality reduction) and uses this to interpret the images, and in parallel uses this 2D/2.5D interpretation (plus disparity information) to create a full dynamical 3D mapping of its environment, which in term could be subject to further interpretation (dimensionality reduction). A huge task :) But, increasingly realistic. This, then, of course, could be used by e.g. Novamente to step out of the virtual realm and perform inference in the real world. And that's where the fun would start, don't you all agree? On Feb 17, 2008 3:16 PM, Bob Mottram <[EMAIL PROTECTED]> wrote: > On 17/02/2008, Vladimir Nesov <[EMAIL PROTECTED]> wrote: > > This is a tutorial talk I saw recently (see Section A): > > http://nips.cc/Conferences/2007/Program/event.php?ID=577 > > Michael Lewicki > > Sensory Coding and Hierarchical Representations > > I like this guy's approach, which covers many familiar topics. > Particularly I liked the way he treated both vision and auditory > systems as essentially the same problem. > > > --- > agi > Archives: http://www.listbox.com/member/archive/303/=now > RSS Feed: http://www.listbox.com/member/archive/rss/303/ > Modify Your Subscription: http://www.listbox.com/member/?&; > Powered by Listbox: http://www.listbox.com > --- agi Archives: http://www.listbox.com/member/archive/303/=now RSS Feed: http://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: http://www.listbox.com/member/?member_id=8660244&id_secret=95818715-a78a9b Powered by Listbox: http://www.listbox.com
Re: [agi] Visual Reasoning Part 1 The Scene
On 17/02/2008, Kingma, D.P. <[EMAIL PROTECTED]> wrote: > I'm wondering how > much further one could extend such architecture to coding of > spatiotemporal (video) patterns, multimodal patterns (video + audio) > and eventually coding of 3D objects. They are all 'just' extensions of > such a model, 'just' about finding efficient ways of learning the > joint probability distributions :) however I imagine that finding > efficient ways of training such models (e.g. finding compact > representations) should become increasingly hard. This is true. In principle reconstructing a 3D model based upon observations from one or two images over time is just the reverse of the ray tracing problem. By finding correspondences, either in structure from motion or by stereo correspondence (basically these two things are the same problem) you can then try to probabilistically model the ray of light which traveled to the image pixel from the object. There's no doubt that this is a hard problem, but I think it's one which is solvable. The next logical step in that fellow's research is as you say to extend the approach to matching features over time in video sequences. This involves not only detecting the features themselves but also making a forward prediction about where the feature will be next and iteratively modeling the position uncertainty and local surface orientation. Andrew Davison's group is doing just this kind of thing, applying information theory to vision with some success. --- agi Archives: http://www.listbox.com/member/archive/303/=now RSS Feed: http://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: http://www.listbox.com/member/?member_id=8660244&id_secret=95818715-a78a9b Powered by Listbox: http://www.listbox.com
Re: [agi] Visual Reasoning Part 1 The Scene
On Feb 17, 2008 11:56 PM, Bob Mottram <[EMAIL PROTECTED]> wrote: > On 17/02/2008, Kingma, D.P. <[EMAIL PROTECTED]> wrote: > > I'm wondering how > > much further one could extend such architecture to coding of > > spatiotemporal (video) patterns, multimodal patterns (video + audio) > > and eventually coding of 3D objects. They are all 'just' extensions of > > such a model, 'just' about finding efficient ways of learning the > > joint probability distributions :) however I imagine that finding > > efficient ways of training such models (e.g. finding compact > > representations) should become increasingly hard. > > > This is true. In principle reconstructing a 3D model based upon > observations from one or two images over time is just the reverse of > the ray tracing problem. By finding correspondences, either in > structure from motion or by stereo correspondence (basically these two > things are the same problem) you can then try to probabilistically > model the ray of light which traveled to the image pixel from the > object. There's no doubt that this is a hard problem, but I think > it's one which is solvable. > > The next logical step in that fellow's research is as you say to > extend the approach to matching features over time in video sequences. > This involves not only detecting the features themselves but also > making a forward prediction about where the feature will be next and > iteratively modeling the position uncertainty and local surface > orientation. Andrew Davison's group is doing just this kind of thing, > applying information theory to vision with some success. Yes, stereo correspondence and structure from motion are very similar, although in the second case (like SLAM) there's the extra task of determining the relative camera position. Anyway, I think that SLAM is a very useful technology to determine camera location and motion and I imagine it as being one of the first steps when analyzing video, as you know better then me, for example to ensure epipolarity for disparity analysis. However, as you will agree with me, the features used (SIFT-like i believe) are very sparse and not directly useful for actual scene reconstruction. However, stereo correspondence becomes more robust when abstract and invariant features are used (like taking the max over Gabor-like responses, as you describe on your web page), so I imagine that some features higher in the Michael Lewicki' architecture would be quite useful for disparity matching. I have no idea which abstraction levesl our brain uses to guide stereopsis, though. Could be that distance of far-away objects is done by calculating the disparity of the whole objects (vs. its pixels). > > --- > agi > Archives: http://www.listbox.com/member/archive/303/=now > RSS Feed: http://www.listbox.com/member/archive/rss/303/ > Modify Your Subscription: http://www.listbox.com/member/?&; > Powered by Listbox: http://www.listbox.com > --- agi Archives: http://www.listbox.com/member/archive/303/=now RSS Feed: http://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: http://www.listbox.com/member/?member_id=8660244&id_secret=95818715-a78a9b Powered by Listbox: http://www.listbox.com
