Bah! I hate it when I rush and get stupid. This is why I'm not a
politician (and why I think the Republican tirade/crusade against
"flip-flopping" is so damaging/dangerous). But, it does serve to illustrate
a number of useful points so I'll just go with it . . . .
I'm writing this e-mail without any additional external information than
I had last night (though I expect to shortly be reading *several* e-mails
from the list telling me what an idiot I am :-). However, my subconscious
knowledge-retrieval processes have finally seen fit to provide me with a
number of "You know . . . . "s. I think that observations of this type are
very important to make when considering building an AI. Not all
observations will be compiled into knowledge and not all knowledge will be
immediately accessible to a system even if the system has what it needs to
retrieve/derive the knowledge. Designs that assume total knowledge
integrity and retrieval are exactly as bad as designs that assume infinite
processing power and memory.
Clearly, Philip is referring to the analogies questions of the SAT (not
the synonym questions that I got stuck on last night). Clearly, vectors
have direction in addition to distance. And, clearly, Philip is referring
to the fact that the directions/vectors that the system generates are not in
human-readable form . . . . (though I would argue that they are easily
human-comprehensible if you write a translator).
<I'm tempted to make a digression into how much common
knowledge/world-modeling we assume/rely upon -- knowledge that my brain was
not coming up with last night and replacing with a poor substitute instead>
So let me extend and refine my stupid answer (because the core *is*
still fundamentally correct) . . . .
Training SVDs on a given corpus produces a database that is always
fundamentally isomorphic to pairs of word-pairs and their similarity
distances (normally expressed as the number and frequency of
dimensions/common-usages they have in common) through a very simple
algorithm that compares how they are used in sentences. There are, of
course, also various representations that appear more vector-like but the
fundamental isomorphism remains.
With the simplest SVD algorithms and most obvious cases, these
directions can often be easily translated into human terms. For example,
hat/head and hands/gloves both have dimensionalities of wore and wear.
(Note, however, that if you wrote the SAT test specifically to confuse this
type of system without messing with humans, you could have examples like
yarmulke/temple (dimensions wear-in and wear-to) include possibly
system-acceptable answers like hole/sock to distract from tuxedo/dance).
With many SVD systems, however, the representation is more vector-like
and *not* conducive to easy translation to human terms. I have two answers
to these cases. Answer 1 is that it is still easy for a human to look at
the closest matches to a particular word pair and figure out what they have
in common. Answer 2 is that I still contend that this is a major design
flaw (which can also be rectified by taking the time to write a translator).
You really, really, *really* don't want to create an intelligence that may
be both smarter/faster than you and seriously flawed -- and statistical
knowledge is very, very shallow; very prone to certain types of error; and
*not* particularly conducive to being built upon (unless, of course, you use
it merely as a subsystem and you're packing up it's results and sending them
to an entirely different type of system). You clearly do *not* want a
system of this type at the core of your AI's reasoning processes --
particularly since, I contend, this type of system is frequently (and in
some classes of systems which are well behaved, always) isomorphic to a
system that *is* easily human-comprehensible. (Note that neural networks,
in particular, are a class of system that are *not* well behaved because the
internal data structures formed by the neural network algorithms that we
know most frequently do not correspond to the real-world simplest
explanation unless you get really, really lucky in choosing your number of
nodes and your connections. Nature has clearly found a way around this
problem but we do not know this solution yet.)
Mark (going off to be plastered by replies to last night's message)
----- Original Message -----
From: "Mark Waser" <[EMAIL PROTECTED]>
To: <agi@v2.listbox.com>
Sent: Wednesday, November 29, 2006 6:21 PM
Subject: Re: [agi] A question on the symbol-system hypothesis
Yes, it was insulting. I am sorry. However, I don't think this
conversation is going anywhere. There are many, many examples just of
the use of SVD and PCI that I think meet your criteria. The one I
mentioned earlier, to you, that uses SVD on word-pair similarities,
and scores at human-level on the SAT, is an example. There are
thousands of examples.
Hmmm. We're definitely in very different realms and are currently talking
past each other. I guess that I'm having trouble seeing how you would
think that SVD is at all human-incomprehensible. Training SVD on a given
corpus produces a set of word pairs and their similarity distances through
a very simple algorithm. SVD taking the SAT simply involves four or five
database lookups (assuming that there are four or five answers) for each
question and taking the answer with the smallest distance. What is
incomprehensible about that? Why *can't* I debug a wrong answer (assuming
that I have access to the training corpus)?
----- Original Message -----
From: "Philip Goetz" <[EMAIL PROTECTED]>
To: <agi@v2.listbox.com>
Sent: Wednesday, November 29, 2006 5:17 PM
Subject: Re: [agi] A question on the symbol-system hypothesis
On 11/29/06, Mark Waser <[EMAIL PROTECTED]> wrote:
> If you look into the literature of the past 20 years, you will easily
> find several thousand examples.
I'm sorry but either you didn't understand my point or you don't
know
what you are talking about (and the constant terseness of your replies
gives
me absolutely no traction on assisting you). If you would provide just
one
example and state why you believe it refutes my point, then you'll give
me
something to answer -- as it is, you're making a meaningless assertion
of no
value that I can't even begin to respond to (not to mention the point
that
contending/assuming that I've overlooked several thousand examples is
pretty
insulting).
Yes, it was insulting. I am sorry. However, I don't think this
conversation is going anywhere. There are many, many examples just of
the use of SVD and PCI that I think meet your criteria. The one I
mentioned earlier, to you, that uses SVD on word-pair similarities,
and scores at human-level on the SAT, is an example. There are
thousands of examples.
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303
