Well, it really depends on what you mean by "too complex for a human
to understand." Do you mean
-- too complex for a single human expert to understand within 1 week of
effort
-- too complex for a team of human experts to understand within 1 year of
effort
-- fundamentally too complex for humans to understand, ever
Actually, I'm willing to stake my claim to "too complex for a single human
expert to understand within 1 week of effort".
My main point in this regard is that a machine learning algorithm can
find a complex predictive pattern, in a few seconds or minutes of
learning, that is apparently inscrutable to humans -- and that remains
inscrutable to an educated human after hours or days of scrutiny.
Take that complex predictive pattern. Assume that it is in or that can be
translated to it's simplest correct yet complete form. Assume that it is
translated to the most human-friendly representation possible . . . .
I would contend that *all* complex predictive patterns that human-level and
even near-super-human AGIs are likely to be able to extract/generate are
reducible to maps which partition an n-space into areas where the
predictions are constants or reasonably simple formulas -- and that humans
can easily handle any prediction likely to be made by a human or even
near-superhuman AI. In day-to-day life, our world is not controlled enough
and regular enough that we (or any near-human system) can collect enough
data to *correctly* extract formulas with a large enough number of
inextricably interlinked variables that we can't understand it.
It is possible that, eventually, a true super-human level AI will take a ton
of data with a large number of irreducibly interacting variables that
interact differently in a tremendous number of partitions -- but, I'm not at
all convinced that our world is regular enough/would provide enough
controlled data where it's also the case that the interactions are so
interlinked that the problem can't be decomposed -- and I certainly don't
expect to see it anytime in the near future or see it as a *requirement* for
AGI.
(And, yes, I will acknowledge that I cheated tremendously with my "Take that
complex predictive pattern" paragraph since doing those things requires
human-level intelligence).
So, what if we make a prediction about the price of Dell stock tomorrow by
<snip>
Then we are certainly not just using nearest-neighbor or CBR or
anything remotely like that.
But the behavior across a phase change is going to be just as incorrect.
Yet, can a human understand why the system made the prediction it did?
Not readily....
Again, I've got two answers. First, in part, this is because the system is
not expressing (or even deriving) the rules in the simplest correct yet
complete form. Second, the human "understands" the prediction as well as
the system does (i.e. as a collection of unrelated rules derived from
previous data with weights added together). I would contend that knowledge
and understanding are measured by predictive power -- particularly under
novel circumstances (to separate them from simple pattern-matching). The
system is doing what it does faster than a human can but it really isn't
doing anything that a human can't (and certainly not anything that the human
can't understand).
I would also note that, on a big enough empirical dataset, an
algorithmic approach like SVM or the ensemble method described above
definitely COULD produce predictive rules that were "fundamentally
incomprehensible to humans" --- in the sense of having an algorithmic
information content greater than that of the human brain. This is
quite a feasible possibility. But I don't claim that this is the case
with these algorithms as applied in the present day, in fact I doubt
it.
:-) I missed this paragraph the first time through. It sounds like my
argument except I have more skepticism about the world being regular enough
and the myriad of other variables being controlled enough that the *data*
for SVM to do this is going to be collected any time soon. (Note to mention
that, by the time it happens, I fully expect that the algorithmic
information capacity of the human brain will be severely augmented :-) (and
even so, it's really just more of the same except that it's run across the
phase change where we poor limited humans have run out of capacity -- not
the complete change in understanding that you see between us and the lower
animals).
----- Original Message -----
From: "Ben Goertzel" <[EMAIL PROTECTED]>
To: <agi@v2.listbox.com>
Sent: Thursday, November 30, 2006 9:30 AM
Subject: Re: Re: [agi] A question on the symbol-system hypothesis
Would you argue that any of your examples produce good results that
are
not comprehensible by humans? I know that you sometimes will argue that
the
systems can find patterns that are both the real-world simplest
explanation
and still too complex for a human to understand -- but I don't believe
that
such patterns exist in the real world (I'd ask you to provide me with an
example of such a pattern to disprove this belief -- but I wouldn't
understand it :-).
Well, it really depends on what you mean by "too complex for a human
to understand."
Do you mean
-- too complex for a single human expert to understand within 1 week of
effort
-- too complex for a team of human experts to understand within 1 year of
effort
etc.
-- fundamentally too complex for humans to understand, ever
??
My main point in this regard is that a machine learning algorithm can
find a complex predictive pattern, in a few seconds or minutes of
learning, that is apparently inscrutable to humans -- and that remains
inscrutable to an educated human after hours or days of scrutiny.
This doesn't mean the pattern is **fundamentally impossible** for
humans to understand, of course... though in some cases it might
conceivably be (more on that later)
As an example consider ensemble-based prediction algorithms. In this
approach, you make a prediction by learning say 1000 or 10,000
predictive rules (by one or another machine learning algorithm), each
of which may make a prediction that is just barely statistically
significant. Then, you use some sort of voting or estimate-merging
mechanism (and there are some subtle ones as well as simple ones,
e.g. ranging from simple voting to an approach that tries to find a
minimum-entropy prob. distribution for the underlying reality
explaining the variety of individal predictions)
So, what if we make a prediction about the price of Dell stock tomorrow by
-- learning (based on analysis of historical price data) 10K weak
predictive rules, each of which is barely meaningful, and each of
which combines a few dozen relevant factors
-- merging the predictions of these models using an
entropy-minimization estimate-merging algorithm
Then we are certainly not just using nearest-neighbor or CBR or
anything remotely like that.
Yet, can a human understand why the system made the prediction it did?
Not readily....
Maybe, after months of study -- statistically analyzing the 10K models
in various ways, etc. -- a human could puzzle out this system's one
prediction. But the predictive system may make similar predictions
for a whole bunch of stocks, every day....
There is plenty of evidence in the literature that ensemble methods
like this outperform individual-predictive-model methods. And there
is plenty of evidence suggesting that the brain uses ensemble methods
(i.e. it combines together multiple unreliable estimates to get a
single reliable one) in simple contexts, so maybe it does in complex
contexts too...
I would also note that, on a big enough empirical dataset, an
algorithmic approach like SVM or the ensemble method described above
definitely COULD produce predictive rules that were "fundamentally
incomprehensible to humans" --- in the sense of having an algorithmic
information content greater than that of the human brain. This is
quite a feasible possibility. But I don't claim that this is the case
with these algorithms as applied in the present day, in fact I doubt
it.
-- Ben G
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303
