subject:"Re\: \[agi\] Understanding Natural Language"

On Wednesday 29 November 2006 17:23, Philip Goetz wrote:

> What is a pointer-and-tag record structure, and what's it got to do
> with n-dim vectors?

I was using the phrase to cover the typical datastructures representing 
"objects" or "frames" in standard AI ( and much of mainstream programming) 
practice. Nothing special.

> I still don't know why you talk about using different numbers of
> dimensions simultaneously.  Seems to me that you can capture these
> invariants in whatever dimensionality you choose, so no need to talk
> about fractal representations.

At the lowest levels I don't get to choose, since the dimensionality is fixed 
by my input hardware. In that forced representation, the structure of the 
real world generates fractal shapes. At higher levels, I do want to choose -- 
and I have to work out which subspaces / transforms of the lower spaces to 
map into the higher ones. In fact, I can't just do this myself at design time 
-- the system itself needs to do it in learning new concepts. 

--Josh

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Re: [agi] Understanding Natural Language

On 11/29/06, J. Storrs Hall, PhD. <[EMAIL PROTECTED]> wrote:

On Wednesday 29 November 2006 16:04, Philip Goetz wrote:
> On 11/29/06, J. Storrs Hall, PhD. <[EMAIL PROTECTED]> wrote:
> > There will be many occurances of the smaller subregions, corresponding to
> > all different sizes and positions of Tom's face in the raster. In other
> > words, the Tom's face region is fractal.
>
> Are you saying that a hierarchy of categories is just a linear chain
> of resolutions?

A *linear* chain of resolutions would be just one root-to-leaf path in an
abstraction tree (root=lo-res, leaves = all the hi-res pix that would map
into that lo-res one).  The whole tree would be a hierarchy of categories.

I meant that a linear chain of resolutions would create a tree,
because at finer resolutions, you would have more categories.

At the raster level, you can brighten or dim any one pixel without
substantially changing whose face it is. At higher levels of abstraction you
can move the vector along dimensions of lighting, orientation, and size
without changing whose face it is. These invariants can be captured by
transformations or projections in the space -- they're the kind of regularity
that I'm trying to capture implicitly by using n-spaces, rather than having
to represent explicitly in pointer-and-tag record structures.

What is a pointer-and-tag record structure, and what's it got to do
with n-dim vectors?

I still don't know why you talk about using different numbers of
dimensions simultaneously.  Seems to me that you can capture these
invariants in whatever dimensionality you choose, so no need to talk
about fractal representations.

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Re: [agi] Understanding Natural Language

On Wednesday 29 November 2006 16:04, Philip Goetz wrote:
> On 11/29/06, J. Storrs Hall, PhD. <[EMAIL PROTECTED]> wrote:
> > There will be many occurances of the smaller subregions, corresponding to
> > all different sizes and positions of Tom's face in the raster. In other
> > words, the Tom's face region is fractal.
>
> Are you saying that a hierarchy of categories is just a linear chain
> of resolutions?

A *linear* chain of resolutions would be just one root-to-leaf path in an 
abstraction tree (root=lo-res, leaves = all the hi-res pix that would map 
into that lo-res one).  The whole tree would be a hierarchy of categories.

> I don't see why you need to work with multiple dimensionalities - at
> least, when identifying Tom's face, you need only deal with one
> dimensionality, although you might use fewer dimensions when looking
> for any old human face.

At the raster level, you can brighten or dim any one pixel without 
substantially changing whose face it is. At higher levels of abstraction you 
can move the vector along dimensions of lighting, orientation, and size 
without changing whose face it is. These invariants can be captured by 
transformations or projections in the space -- they're the kind of regularity 
that I'm trying to capture implicitly by using n-spaces, rather than having 
to represent explicitly in pointer-and-tag record structures.

--Josh

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Re: [agi] Understanding Natural Language


On 11/29/06, J. Storrs Hall, PhD. <[EMAIL PROTECTED]> wrote:


There will be many occurances of the smaller subregions, corresponding to all
different sizes and positions of Tom's face in the raster. In other words,
the Tom's face region is fractal.

So, of course, is the Dick's face region, but note that at they lower limits
of resolution they begin to overlap; after a while you're just able to
recognize a human face but not say whose.

So even if you started out saying "only 16K-D space, fixed resolution," you
wind up having to work with the other dimensionalities anyway.


Are you saying that a hierarchy of categories is just a linear chain
of resolutions?

I don't see why you need to work with multiple dimensionalities - at
least, when identifying Tom's face, you need only deal with one
dimensionality, although you might use fewer dimensions when looking
for any old human face.

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Re: [agi] Understanding Natural Language


On 11/29/06, Philip Goetz <[EMAIL PROTECTED]> wrote:


Either that, or I wouldn't do a purely syntactic parse.  It doesn't
work very well to try to handle syntax first, then semantics.


Bother.  I've made some contradictory statements.  I started out by
saying that you could parse English into predicates, without resolving
the semantics, and feed those predicates into whatever process you
like to "understand" the sentence.

What I actually do, as opposed to what I say, is to attack syntax and
semantics at the same time.  The more you commit to a particular
semantic interpretation, the more elaborate you can make your parse,
and the more predications you can extract.  Understanding is a large
part of parsing.

This is complicated by the fact that the ambiguities that are easy to
think of (e.g., does "bank" mean a river bank or a place to put money)
are also easy to resolve, whereas subtler ambiguities that are very
difficult to resolve (say, what qualities is the speaker focusing on,
and what qualities are they ignoring, when they say someone is
"admirable") generally have little impact on the syntax.

I can at least say that, supposing you can figure out what the
sentence means, predicates can be a good way of representing that
meaning.

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Re: [agi] Understanding Natural Language

On Wednesday 29 November 2006 13:56, Matt Mahoney wrote:
> How is a raster scan (16K vector) of an image useful?  The difference
> between two images of faces is the RMS of the differences of the images
> obtained by subtracting pixels.  Given an image of Tom, how do you compute
> the set of all images that look like Tom?
>
> Humans perceive images by reducing them to a small set of complex features,
> which can be compared in a space with much fewer dimensions.

Certainly. The n-space representation at that early stage is primarily an 
abstraction for thinking about the process mathematically. In humans, the 
retina starts with more like a 16M raster but reduces it precipitously before 
even sending it down the optic nerve. On the other hand, visual cortex has 
not one but many 2-d maps where various functions of images are manipulated 
in a very straightforward way -- the raster form is still used quite a bit 
before the higher abstractions are generated.

One way of computing the set of images that look like Tom would be to follow 
from your one image along a trajectory that you knew maintained invariants in 
pose, orientation, lighting, etc, until you hit the hypersurface that was the 
images of Dick, and then shift the whole Dick region the opposite offset. I 
doubt it actually happens at this level in vision, but I bet the basic 
mechanism, at higher levels, is *extremely* common in carrying expectations 
from an experience to a new situation.

--Josh

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Re: [agi] Understanding Natural Language

2006-11-29 Thread Matt Mahoney

How is a raster scan (16K vector) of an image useful?  The difference between 
two images of faces is the RMS of the differences of the images obtained by 
subtracting pixels.  Given an image of Tom, how do you compute the set of all 
images that look like Tom?

Humans perceive images by reducing them to a small set of complex features, 
which can be compared in a space with much fewer dimensions.

-- Matt Mahoney, [EMAIL PROTECTED]

- Original Message 
From: "J. Storrs Hall, PhD." <[EMAIL PROTECTED]>
To: agi@v2.listbox.com
Sent: Wednesday, November 29, 2006 10:50:51 AM
Subject: Re: [agi] Understanding Natural Language

On Tuesday 28 November 2006 17:50, Philip Goetz wrote:

> I see that a raster is a vector.  I see that you can have rasters at
> different resolutions.  I don't see what you mean by "map the regions
> that represent the same face between higher and lower-dimensional
> spaces", or what you are taking the limit of as resolution goes to
> infinity, or why you don't just stick with one particular resolution.

Take rasters representing the faces of Tom and Dick. Just for concreteness 
let's assume they're 16K numbers long. Each one represents a point in a 
16K-dimensional space. If we think of all the points in the space that are 
pictures of Tom, they form a (probably connected) region in the space.

All the pictures that represent Dick form a similar shape, offset from the Tom 
region. There's a larger region that contains both of them that is the union 
of all men's faces, and a larger one yet that's all human faces, and so 
forth.

Now imagine the picture of Tom being shrunk until it's only 1/4 of the 
original raster. It's still Tom's face so it's still part of the Tom region 
of the 16K-D space, but 12K of those dimensions can change however they like 
and not affect the Tom-ness of the picture. A 4K-D slice across this subspace 
would look like the original space would if it only had 4K dimensions in the 
first place -- and it will also resemble a diagonal slice across the big part 
of the Tom region.

There will be many occurances of the smaller subregions, corresponding to all 
different sizes and positions of Tom's face in the raster. In other words, 
the Tom's face region is fractal.

So, of course, is the Dick's face region, but note that at they lower limits 
of resolution they begin to overlap; after a while you're just able to 
recognize a human face but not say whose.

So even if you started out saying "only 16K-D space, fixed resolution," you 
wind up having to work with the other dimensionalities anyway.

Cheers,

--Josh

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Re: [agi] Understanding Natural Language


On 11/29/06, J. Storrs Hall, PhD. <[EMAIL PROTECTED]> wrote:


Presumably you would produce multiple parses for syntactically ambiguous
sentences:

flies(time,like(arrow))
like(time(flies),arrow)

?


Either that, or I wouldn't do a purely syntactic parse.  It doesn't
work very well to try to handle syntax first, then semantics.

To a statistical learner, syntax and semantics aren't even different things.

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Re: [agi] Understanding Natural Language

On Wednesday 29 November 2006 12:28, Philip Goetz wrote:
> Oops - looking back at my earlier post, I said that "English sentences
> translate neatly into predicate logic statements".  I should have left
> out "logic".  I like using predicates to organize sentences.  I made
> that post because Josh was pointing out some of the problems with
> logic, but then making the false conclusion that predications are a
> bad representation.  I wanted to say that you can use a predicate
> representation, but use something other than FOPL to process it.

I don't think there is a basic disagreement here. I prefaced my earlier 
remarks with the observation, "... it is clearly straightforward to translate 
a sentence into a predicate expression in a syntactic way ..." And it doesn't 
look like you're trying to claim a lot more than that.

Presumably you would produce multiple parses for syntactically ambiguous 
sentences:

flies(time,like(arrow))
like(time(flies),arrow)

?

--Josh

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Re: [agi] Understanding Natural Language


Oops - looking back at my earlier post, I said that "English sentences
translate neatly into predicate logic statements".  I should have left
out "logic".  I like using predicates to organize sentences.  I made
that post because Josh was pointing out some of the problems with
logic, but then making the false conclusion that predications are a
bad representation.  I wanted to say that you can use a predicate
representation, but use something other than FOPL to process it.

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Re: [agi] Understanding Natural Language


On 11/28/06, Matt Mahoney <[EMAIL PROTECTED]> wrote:

First order logic (FOL) is good for expressing simple facts like "all birds have wings" or "no 
bird has hair", but not for statements like "most birds can fly".  To do that you have to at 
least extend it with fuzzy logic (probability and confidence).


Quantification is a logic problem.  I am not talking about logic, but
using predications for representation.  I can represent "most birds
can fly" as something like

[S [NP (mod most) (head birds)] [VP (mod can) (head fly)]]

No quantification involved.


A second problem is, how do you ground the terms?  If you have "for all X, bird(X) => has(X, wings)", where 
does "bird", "wings", "has" get their meanings?  The terms do not map 1-1 to English words, 
even though we may use the same notation.


They DO map 1-1 to English words, because I simply use the English
words in my predicates.  I don't care if "wing" has multiple meanings.
It is the business of whatever process works with those predicates to
sort that out.

I've said this twice already:  I am not talking about using logic, in
which you assign semantics to the terms, and assume that every
instance of a particular predicate has the same semantics.  I am just
talking about using predicates to organize the English terms in a
sentence.  Predicates are a nice representation, even if you are not
going to use FOPL.

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Re: [agi] Understanding Natural Language

On Tuesday 28 November 2006 17:50, Philip Goetz wrote:

> I see that a raster is a vector.  I see that you can have rasters at
> different resolutions.  I don't see what you mean by "map the regions
> that represent the same face between higher and lower-dimensional
> spaces", or what you are taking the limit of as resolution goes to
> infinity, or why you don't just stick with one particular resolution.

Take rasters representing the faces of Tom and Dick. Just for concreteness 
let's assume they're 16K numbers long. Each one represents a point in a 
16K-dimensional space. If we think of all the points in the space that are 
pictures of Tom, they form a (probably connected) region in the space.

All the pictures that represent Dick form a similar shape, offset from the Tom 
region. There's a larger region that contains both of them that is the union 
of all men's faces, and a larger one yet that's all human faces, and so 
forth.

Now imagine the picture of Tom being shrunk until it's only 1/4 of the 
original raster. It's still Tom's face so it's still part of the Tom region 
of the 16K-D space, but 12K of those dimensions can change however they like 
and not affect the Tom-ness of the picture. A 4K-D slice across this subspace 
would look like the original space would if it only had 4K dimensions in the 
first place -- and it will also resemble a diagonal slice across the big part 
of the Tom region.

There will be many occurances of the smaller subregions, corresponding to all 
different sizes and positions of Tom's face in the raster. In other words, 
the Tom's face region is fractal.

So, of course, is the Dick's face region, but note that at they lower limits 
of resolution they begin to overlap; after a while you're just able to 
recognize a human face but not say whose.

So even if you started out saying "only 16K-D space, fixed resolution," you 
wind up having to work with the other dimensionalities anyway.

Cheers,

--Josh

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Re: [agi] Understanding Natural Language

2006-11-28 Thread Matt Mahoney

First order logic (FOL) is good for expressing simple facts like "all birds 
have wings" or "no bird has hair", but not for statements like "most birds can 
fly".  To do that you have to at least extend it with fuzzy logic (probability 
and confidence).

A second problem is, how do you ground the terms?  If you have "for all X, 
bird(X) => has(X, wings)", where does "bird", "wings", "has" get their 
meanings?  The terms do not map 1-1 to English words, even though we may use 
the same notation.  For example, you can talk about the wings of a building, or 
the idiom "wing it".  Most words in the dictionary list several definitions 
that depend on context.  Also, words gradually change their meaning over time.

I think FOL represents complex ideas poorly.  Try translating what you just 
wrote into FOL and you will see what I mean.
 
-- Matt Mahoney, [EMAIL PROTECTED]

- Original Message 
From: Philip Goetz <[EMAIL PROTECTED]>
To: agi@v2.listbox.com
Sent: Tuesday, November 28, 2006 5:45:51 PM
Subject: Re: [agi] Understanding Natural Language

Oops, Matt actually is making a different objection than Josh.

> Now it seems to me that you need to understand sentences before you can 
> translate them into FOL, not the other way around. Before you can translate 
> to FOL you have to parse the sentence, and before you can parse it you have 
> to understand it, e.g.
>
> I ate pizza with pepperoni.
> I ate pizza with a fork.
>
> Using my definition of understanding, you have to recognize that "ate with a 
> fork" and "pizza with pepperoni" rank higher than "ate with pepperoni" and 
> "pizza with a fork".  A parser needs to know millions of rules like this.

Yes, this is true.  When I said "neatly", I didn't mean "easily".  I
mean that the correct representation in predicate logic is very
similar to the English, and doesn't lose much meaning.  It was
misleading of me to say that it's a good starting point, though, since
you do have to do a lot to get those predicates.

A predicate representation can be very useful.  This doesn't mean that
you have to represent all of the predications that could be extracted
from a sentence.  The NLP system I'm working on does not, in fact, use
a parse tree, for essentially the reasons Matt just gave.  It doesn't
want to make commitments about grammatical structure, so instead it
just groups things into phrases, without deciding what the
dependencies are between those phrases, and then has a bunch of
different demons that scan those phrases looking for particular
predications.  As you find predications in the text, you can eliminate
certain choices of lexical or semantic category for words, and
eliminate arguments so that they can't be re-used in other
predications.  You never actually find the correct parse in our
system, but you could if you wanted to.  It's just that, we've already
extracted the meaning that we're interested in by the time we have
enough information to get the right parse, so the parse tree isn't of
much use.  We get the predicates that we're interested in, for the
purposes at hand.  We might never have to figure out whether pepperoni
is a part or an instrument, because we don't care.

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303



-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Re: [agi] Understanding Natural Language


On 11/28/06, J. Storrs Hall, PhD. <[EMAIL PROTECTED]> wrote:

Sorry -- should have been clearer. Constructive Solid Geometry. Manipulating
shapes in high- (possibly infinite-) dimensional spaces.

Suppose I want to represent a face as a point in a space. First, represent it
as a raster. That is in turn a series of numbers that can be a vector in the
space. Same face, higher resolution: more numbers, higher dimensionality
space, but you can map the regions that represent the same face between
higher and lower-dimensional spaces. Do it again, again, etc: take the limit
as the resolution and dimensionality go to infinity. You can no more
represent this explicitly than you can a real number, but you can use it as
an abstraction, as a theory to tell you how well your approximations are
working.


I see that a raster is a vector.  I see that you can have rasters at
different resolutions.  I don't see what you mean by "map the regions
that represent the same face between higher and lower-dimensional
spaces", or what you are taking the limit of as resolution goes to
infinity, or why you don't just stick with one particular resolution.

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Re: [agi] Understanding Natural Language

Oops, Matt actually is making a different objection than Josh.

Now it seems to me that you need to understand sentences before you can
translate them into FOL, not the other way around. Before you can translate to
FOL you have to parse the sentence, and before you can parse it you have to
understand it, e.g.

I ate pizza with pepperoni.
I ate pizza with a fork.

Using my definition of understanding, you have to recognize that "ate with a fork" and "pizza with
pepperoni" rank higher than "ate with pepperoni" and "pizza with a fork". A parser needs to
know millions of rules like this.

Yes, this is true. When I said "neatly", I didn't mean "easily". I
mean that the correct representation in predicate logic is very
similar to the English, and doesn't lose much meaning. It was
misleading of me to say that it's a good starting point, though, since
you do have to do a lot to get those predicates.

A predicate representation can be very useful. This doesn't mean that
you have to represent all of the predications that could be extracted
from a sentence. The NLP system I'm working on does not, in fact, use
a parse tree, for essentially the reasons Matt just gave. It doesn't
want to make commitments about grammatical structure, so instead it
just groups things into phrases, without deciding what the
dependencies are between those phrases, and then has a bunch of
different demons that scan those phrases looking for particular
predications. As you find predications in the text, you can eliminate
certain choices of lexical or semantic category for words, and
eliminate arguments so that they can't be re-used in other
predications. You never actually find the correct parse in our
system, but you could if you wanted to. It's just that, we've already
extracted the meaning that we're interested in by the time we have
enough information to get the right parse, so the parse tree isn't of
much use. We get the predicates that we're interested in, for the
purposes at hand. We might never have to figure out whether pepperoni
is a part or an instrument, because we don't care.

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Re: [agi] Understanding Natural Language

I think that Matt and Josh are both misunderstanding what I said in
the same way.  Really, you're both attacking the use of logic on the
predicates, not the predicates themselves as a representation, and so
ignoring the distinction I was trying to create.  I am not saying that
rewriting English as predicates magically provides semantics.

On 11/28/06, J. Storrs Hall, PhD. <[EMAIL PROTECTED]> wrote:

On Tuesday 28 November 2006 14:47, Philip Goetz wrote:
> The use of predicates for representation, and the use of logic for
> reasoning, are separate issues.  I think it's pretty clear that
> English sentences translate neatly into predicate logic statements,
> and that such a transformation is likely a useful first step for any
> sentence-understanding process.  Whether those predicates are then
> used to draw conclusions according to a standard logic system, or are
> used as inputs to a completely different process, is a different
> matter.

I would beg to differ. While it is clearly straightforward to translate a
sencence into a predicate expression in a syntactic way, the resulting
structure has no coherent semantics.

Translating into a predicate expression doesn't give you any semantics.
But it doesn't take any away, either.  It just gives you the sentence
in a neater form, with the hierarchies and dependencies spelled out.

Consider the following sentences. Could you translate them all using the
single predicate on(A,B)? If not, the translation gets messier:

On the table is an apple.
On Lake Ontario is Toronto.
On Hadamard's theory transubstantiation is ineffable.
On Comet, on Cupid, on Prancer and Vixen.
On Christmas we open presents.
On time is better than late.
On budget expenditures are dwarfed by Social Security.
On and on the list goes...

You used the same word "on" in English for each of them.
I thus get to use the same word "on" in a predicate representation for
each of them.
I don't claim that each instance of the predicate "on" means the same thing!
The application of a logic rule that matched any instance of "on(A,B)"
would be making such a claim, but, as I tried to explicitly point out,
that is a problem with logic, not with predicates as a representation.

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Re: [agi] Understanding Natural Language

2006-11-28 Thread J. Storrs Hall, PhD.

On Tuesday 28 November 2006 14:47, Philip Goetz wrote:
> The use of predicates for representation, and the use of logic for
> reasoning, are separate issues.  I think it's pretty clear that
> English sentences translate neatly into predicate logic statements,
> and that such a transformation is likely a useful first step for any
> sentence-understanding process.  Whether those predicates are then
> used to draw conclusions according to a standard logic system, or are
> used as inputs to a completely different process, is a different
> matter.

I would beg to differ. While it is clearly straightforward to translate a 
sencence into a predicate expression in a syntactic way, the resulting 
structure has no coherent semantics. 

Consider how much harder it is to translate a sentence of English into a 
sentence of Chinese. Even then you won't have uncovered the meat of the 
semantics, since in both languages you can rely on a lot of knowledge the 
hearer already knows. 

But when you put the sentence into predicate form, you've moved into a 
formalism where there is no such semantics behind the representation. In 
order to provide them, you have to do the equivalent of writing a Prolog 
program that could make the same predictions, explanations, or replies that a 
human speaker could to the original English sentence.

Consider the following sentences. Could you translate them all using the 
single predicate on(A,B)? If not, the translation gets messier:

On the table is an apple.
On Lake Ontario is Toronto.
On Hadamard's theory transubstantiation is ineffable.
On Comet, on Cupid, on Prancer and Vixen.
On Christmas we open presents.
On time is better than late.
On budget expenditures are dwarfed by Social Security.
On and on the list goes...

> > The open questions are representation -- I'm leaning towards CSG in
> > Hilbert spaces at the moment, but that may be too computationally
> > demanding -- and how to form abstractions.
>
> Does CSG = context-sensitive grammar in this case?  How would you use
> Hilbert spaces?

Sorry -- should have been clearer. Constructive Solid Geometry. Manipulating 
shapes in high- (possibly infinite-) dimensional spaces.

Suppose I want to represent a face as a point in a space. First, represent it 
as a raster. That is in turn a series of numbers that can be a vector in the 
space. Same face, higher resolution: more numbers, higher dimensionality 
space, but you can map the regions that represent the same face between 
higher and lower-dimensional spaces. Do it again, again, etc: take the limit 
as the resolution and dimensionality go to infinity. You can no more 
represent this explicitly than you can a real number, but you can use it as 
an abstraction, as a theory to tell you how well your approximations are 
working.

--Josh

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Re: [agi] Understanding Natural Language

2006-11-28 Thread Matt Mahoney

Philip Goetz <[EMAIL PROTECTED]> wrote:
>The use of predicates for representation, and the use of logic for
>reasoning, are separate issues.  I think it's pretty clear that
>English sentences translate neatly into predicate logic statements,
>and that such a transformation is likely a useful first step for any
>sentence-understanding process.  

I don't think it is clear at all.  Try translating some poetry.  Even for 
sentences that do have a clear representation in first order logic, the 
translation from English is not straightforward at all.  It is an unsolved 
problem.

I also dispute that it is even useful for sentence understanding.  Google 
understands simple questions, and its model is just a bag of words.  Attempts 
to apply parsing or reasoning to information retrieval have generally been a 
failure.

It would help to define what "sentence-understanding" means.  I say a computer 
"understands" English if it can correctly assign probabilities to long strings, 
where "correct" means ranked in the same order as judged by humans.  So a 
program that recognizes the error in the string "the cat caught a moose" could 
be said to understand English.  Thus, the grammar checker in Microsoft Word 
would have more understanding of a text document than a simple spell checker, 
but less understanding than most humans.  Maybe you have a different 
definition.  A reasonable definition for AI should be close to the conventional 
meaning and also be testable without making any assumption about the internals 
of the machine.

Now it seems to me that you need to understand sentences before you can 
translate them into FOL, not the other way around. Before you can translate to 
FOL you have to parse the sentence, and before you can parse it you have to 
understand it, e.g.

I ate pizza with pepperoni.
I ate pizza with a fork.

Using my definition of understanding, you have to recognize that "ate with a 
fork" and "pizza with pepperoni" rank higher than "ate with pepperoni" and 
"pizza with a fork".  A parser needs to know millions of rules like this.

-- Matt Mahoney, [EMAIL PROTECTED]

- Original Message 
From: Philip Goetz <[EMAIL PROTECTED]>
To: agi@v2.listbox.com
Sent: Tuesday, November 28, 2006 2:47:41 PM
Subject: Re: [agi] Understanding Natural Language

On 11/24/06, J. Storrs Hall, PhD. <[EMAIL PROTECTED]> wrote:
> On Friday 24 November 2006 06:03, YKY (Yan King Yin) wrote:
> > You talked mainly about how sentences require vast amounts of external
> > knowledge to interpret, but it does not imply that those sentences cannot
> > be represented in (predicate) logical form.
>
> Substitute "bit string" for "predicate logic" and you'll have a sentence that
> is just as true and not a lot less useful.
>
> > I think there should be a
> > working memory in which sentences under attention would "bring up" other
> > sentences by association.  For example if "a person is being kicked" is in
> > working memory, that fact would bring up other facts such as "being kicked
> > causes a person to feel pain and possibly to get angry", etc.  All this is
> > orthogonal to *how* the facts are represented.
>
> Oh, I think the representation is quite important. In particular, logic lets
> you in for gazillions of inferences that are totally inapropos and no good
> way to say which is better. Logic also has the enormous disadvantage that you
> tend to have frozen the terms and levels of abstraction. Actual word meanings
> are a lot more plastic, and I'd bet internal representations are damn near
> fluid.

The use of predicates for representation, and the use of logic for
reasoning, are separate issues.  I think it's pretty clear that
English sentences translate neatly into predicate logic statements,
and that such a transformation is likely a useful first step for any
sentence-understanding process.  Whether those predicates are then
used to draw conclusions according to a standard logic system, or are
used as inputs to a completely different process, is a different
matter.

> The open questions are representation -- I'm leaning towards CSG in Hilbert
> spaces at the moment, but that may be too computationally demanding -- and
> how to form abstractions.

Does CSG = context-sensitive grammar in this case?  How would you use
Hilbert spaces?

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Re: [agi] Understanding Natural Language


On 11/26/06, Pei Wang <[EMAIL PROTECTED]> wrote:

Therefore, the problem of using an n-space representation for AGI is
not its theoretical possibility (it is possible), but its practical
feasibility. I have no doubt that for many limited application,
n-space representation is the most natural and efficient choice.
However, for a general purpose system, the situation is very
different. I'm afraid for AGI we may have to need millions (if not
more) dimensions, and it won't be easy to decide in advance what
dimensions are necessary.


I see evidence of dimensionality reduction by humans in the fact that
adopting a viewpoint has such a strong effect on the kind of
information a person is able to absorb.  In conversations about
politics or religion, I often find ideas that to me seem simple, that
I cannot communicate to someone of a different viewpoint.  We both
start with the same input - some English sentences, say - but I think
we compress them in different, yet internally consistent, ways.  Their
viewpoint is based on a compression scheme that simply compresses out
what I am trying to communicate.

It may be that psychological repression is the result of compressing
out dimensions, or data, that had low utility.  Someone who is
repeatedly exposed to a trauma which they are unable to do anything
about may calculate, subconsciously, that the awareness of that trauma
is simply useless information.

Trying to suggest a PCA-like dimensionality reduction of concepts by
humans has the difficulty that a human should then remember, or be
aware of, those implications of a sentence which have had the most
variance, or the most impact, in their experiences.  In fact, we often
find people make the greatest compression along dimensions that have
the highest importance to them, compressing a whole set of important
distinctions into the binary "good-evil" dimension.  It may be that
our motivational system can handle only a small number of dimensions -
say, five - and that "good-evil" is one of the principle components
whose impact is so large we are actually aware of it.

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Re: Re: [agi] Understanding Natural Language


On 11/27/06, Ben Goertzel <[EMAIL PROTECTED]> wrote:

An issue with Hopfield content-addressable memories is that their
memory capability gets worse and worse as the networks get sparser and
sparser.   I did some experiments on this in 1997, though I never
bothered to publish the results ... some of them are at:

http://www.goertzel.org/papers/ANNPaper.html


I found just the opposite - Hopfield network memory capability gets
much better as the networks get sparser, down to very low levels of
sparseness.  However, I was measuring performance as a function of
storage space and computation.  A fully-connected Hopfield network of
100 neurons has about 10,000 connections.  A Hopfield network of 100
neurons that has only 10 connections per neuron performs has one-tenth
as many connections, and can recall more than one-tenth as many
patterns.

Furthermore, if you selectively eliminate the weak connections and
save the strong connections, you can make Hopfield networks very
sparse that perform almost as well as the fully-connected ones.

BTW, the "canonical" results about Hopfield network capacity in the
McEliece 1987 paper are wrong - I can't find the flaw, so I don't know
why they're wrong, but I know that the paper

a) makes the mistake of comparing recall errors of a fixed number of
bits between networks of different sizes, which means that it counts a
1-bit error in recalling a 1000-node pattern as equivalent to a 1-bit
error in recalling a 10-node pattern, and

b) the paper claims that recall of n-bit patterns, starting from a
presented pattern that differs in n/2 bits, is quite good.  This is
impossible, since differing in n/2 bits means the input pattern is a
RANDOM pattern wrt the target, and half of all the targets should be
closer to the input pattern.

-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Re: [agi] Understanding Natural Language