On 11/29/06, Philip Goetz <[EMAIL PROTECTED]> wrote:
On 11/29/06, Mark Waser <[EMAIL PROTECTED]> wrote:
> I defy you to show me *any* black-box method that has predictive power
> outside the bounds of it's training set. All that the black-box methods are
> doing is curve-fitting. If you give them enough variables they can brute
> force solutions through what is effectively case-based/nearest-neighbor
> reasoning but that is *not* intelligence. You and they can't build upon
> that.
If you look into the literature of the past 20 years, you will easily
find several thousand examples.
Mark, I believe your point is overstated, although probably based on a
correct intuition....
Plenty of "black box" methods can extrapolate successfully beyond
their training sets, and using approaches not fairly describable as
"case based" or "nearest neighbor."
CBR and nearest neighbor are very far from optimally-performing as far
as prediction/categorization algorithms go.
As examples of learning algorithms that
-- successfully extrapolate beyond their training sets using
pattern-recognition much more complex than CBR/nearest-neighbor
-- do so by learning predictive rules that are opaque to humans, in practice
I could cite a bunch, e.g.
-- SVM's
-- genetic programming
-- MOSES (www.metacog.org), a probabilistic evolutionary method used
in Novamente
-- Eric Baum's Hayek system
-- recurrent neural nets trained using recurrent backprop or marker-based GA's
-- etc. etc. etc.
These methods are definitely not human-level AGI. But, they
definitely do extrapolate beyond their training set, via recognizing
complex patterns in their training sets far beyond
CBR/nearest-neighbor.
What these methods do not do, yet at least, is to extrapolate to data
of a **radically different type** from their training set.
For instance, suppose you train an SVM algorithm to recognize gene
expression patterns indicative of lung cancer, by exposing it to data
from 50 lung cancer patients and 50 controls. Then, the SVM can
generalize to predict whether a new person has lung cancer or not --
whether or not this person particularly resembles **any** of the 100
people on whose data the SVM was trained. It can do so by paying
attention to a complex nonlinear combination of features, whose
meaning may well not be comprehensible to any human within a
reasonable amount of effort. This is not CBR or nearest-neighbor. It
is a more fundamental form of learning, displaying much greater
compression and pattern-recognition and hence greater generalization.
On the other hand, if you want to apply the SVM to breast cancer, you
have to run it all over again, on different data. And if you want to
apply it to "cancer in general" you need to specifically feed it
training data regarding a variety of cancers. You can't feed it
training data regarding breast, lung and liver cancer separately, have
it learn predictive rules for each of these and then have it
generalize these predictive rules into a rule for cancer in
general....
In a sense, SVM is just doing curve-fitting, sure....
But in a similar sense, Marcus Hutter's AIXItl theorems show that
given vast computational resources, an arbitrarily powerful level of
intelligence can be achieved via "curve-fitting."
Human-level AGI represents curve-fitting at a level of generality
somewhere between that of the SVM and that of AIXItl. But
curve-fitting at the human level of generality, given the humanly
feasible amount of computational resources, does seem to involve many
properties not characteristic either of
-- curve-fitting algorithms as narrow as SVM, GP, etc.
-- curve-fitting algorithms as broad (but computationally infeasible) as AIXItl
Do you disagree with any of this?
-- Ben G
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303
