I'm working on the assumption that a basic, simple, universal, FAST capability
to do analogical quadrature in each module (read: the chunk of brain that
owns each concept) working all at once and all together is what can make this
possible.
I basically agree with this, actually.
In Novamente what we do is as follows.
We have a probabilistic network of nodes and links, which is an explicit
declarative data store
We then have code that projects the nodes/links in this network into a
dimensional embedding space (50 dimensional in our experiments with this
subsystem, but the dimensionality hasn't been tuned).
The advantage of the embedding space is, it can be used to very quickly
answer questions of the form "Given X, find me all Y so that d(X,Y)< e"
where e is a specified threshold, and d is a distance metric in the
embedding space.
So, in Novamente what you call "rapid analogical quadrature" would be
done as follows:
-- We want to evaluate f(X) for some f and some X described in the
node/link network
-- We find all Y sufficiently close to X in the embedding space
-- We average f(Y) for all these Y ... or, if we have more effort
available for this task, we combine f(Y) for these Y via probabilistic
inference rules (which take into account dependencies that may exist and
are ignored by the averaging operator)
All this sounds pretty similar to your approach.
Furthermore, we have not experimented with this AT ALL yet, but we have
the notion in the design of "context-specific embedding spaces."
Given a context C, one can create an embedding space in which the vector
for X corresponds to a dimensional embedding of "the restriction of X to
the context C." One then do what you might call "rapid,
context-specific analogical quadrature."
What is different, it seems, is that we don't currently do arithmetic
and combinatory operations and projections and such on the dimensionally
embedded vectors (at least, we don't right now -- we **could**, within
the NM system design but we don't currently see a reason to). Instead
we do operations on the node/link network, and then mirror the results
in the embedding space.
For instance, what you call projections from a big dimensional space to
a littler one ---- we call the decision to create an embedding space
corresponding to a ConceptNode that seems to serve as an important
context. But note that this is in general going to be a NONLINEAR
projection, defined by the restriction of nodes/links to a particular
context defining the embedding space that is the range of the "projection."
Using nonlinear projections allows one to work with a dimensional space
that has relatively few coordinates. If one wants to use linear
projections and have full generality one needs to use exponentially
large dimensional spaces and tensorial linearization matrices. Of
course, effective use of nonlinear projections requires that one have
some appropriately restricted class of nonlinear projections to pay
attention to -- and in our approach this is provided by the nonlinear
projections that are naturally constructed in terms of
probabilistic-logic-based contextual restrictions on the level of the
node/link network.
The question I have regarding your approach, then, is whether there is
really a strong reason to do operations other than rapid proximity
search in the dimensional space rather than the node/link network.
If I represent concepts as probabilistic-logical combinations of the
salient attributes, that IS my numerical representation.
But, prob-logic combinations are not necessarily LINEAR combinations
So, to represent them using vectors and matrix multiplication, you need
to use exponentially
huge vectors whose components involve arbitrarily large conjunctions of
elementary terms; and then
you need to use matrices that represent tensorial linearizations of
Boolean functions.
Yuck! ;-)
If I wanted any mathematical claim to completeness or to represent arbitrary
functions, yes; but I'm betting I can get away with discriminatory machinery
roughly equivalent to SVM or simply lists of hyperspheres.
Well, maybe for basic perception, but not for cognition, IMNSHO...
Also, SVM is not at all rapid, as you must know. To get decent SVM
performance on nontrivial problems requires a lot of attention to
feature selection (e.g. recursive feature selection), and systematic
search of parameter space (e.g. kernel parameters).
Remember that one thing I'm assuming is that everything is done in parallel.
there's a processor for each concept (or possibly of each remembered instance
of recognizing a concept (you have something like 10 MIPS per second of
waking life to work with). Going to discrete symbols is a drastic
optimization that fits very well into von Neumann computers, but it's a
little too daring for me (I've seen it fail too many times).
I am confused about whether you are proposing a brain model or an AGI
design.
If you are assuming a processor for each concept, then you won't be able
to build an AGI based on your assumptions for quite a long time, since
massively parallel computing development is sorta stalled these days...
My own discussion has been on the level of AGI rather than brain
modeling, except where explicitly stated otherwise.
I don't buy the idea that brute force search in this matrix space is
gonna be good enough...
To convince me, you need to explain what clever trick is used to do
pruning in the search space,
in a way that is contrived to prune in the right way to make search work
effectively FOR THE
PARTICULAR SORTS OF PROBLEMS THAT WILL HABITUALLY BE CONFRONTED
BY THE SYSTEM.
???
prunesearch space S =
foreach processor P ;; 1e7 procs @ 1e9 IPS
do hillclimbing search around random point in S
after 100 ms
report success or give up
i.e. you have a fixed search capacity; throw it at the problem and live or
die. The space is pruned randomly.
OK, but this just pushes the search problem up a level to the problem of
"searching for search spaces" ...
And anyway, I don't believe that brute force search is going to be
nearly powerful enough....
-- Ben
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303
