RE: [agi] Universal intelligence test benchmark

[Matt Mahoney]

--- On Sat, 12/27/08, John G. Rose  wrote:

> > > How does consciousness fit into your compression
> > > intelligence modeling?
> > 
> > It doesn't. Why is consciousness important?
> > 
> 
> I was just prodding you on this. Many people on this list talk about the
> requirements of consciousness for AGI and I was imagining some sort of
> consciousness in one of your command line compressors :) 
> I've yet to grasp
> the relationship between intelligence and consciousness though lately I
> think consciousness may be more of an evolutionary social thing. Home grown
> digital intelligence, since it is a loner, may not require "much"
> consciousness IMO..

What we commonly call consciousness is a large collection of features that 
distinguish living human brains from dead human brains: ability to think, 
communicate, perceive, make decisions, learn, move, talk, see, etc. We only 
attach significance to it because we evolved, like all animals, to fear a large 
set of things that can kill us.

> > > Max compression implies hacks, kludges and a
> large decompressor.
> > 
> > As I discovered with the large text benchmark.
> > 
> 
> Yep and the behavior of the metrics near max theoretical
> compression is erratic I think?

It shouldn't be. There is a well defined (but possibly not computable) limit 
for each of the well defined universal Turing machines that the benchmark 
accepts (x86, C, C++, etc).

I was hoping to discover an elegant theory for AI. It didn't quite work that 
way. It seems to be a kind of genetic algorithm: make random changes to the 
code and keep the ones that improve compression.

-- Matt Mahoney, matmaho...@yahoo.com



---
agi
Archives: https://www.listbox.com/member/archive/303/=now
RSS Feed: https://www.listbox.com/member/archive/rss/303/
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=8660244&id_secret=123753653-47f84b
Powered by Listbox: http://www.listbox.com

RE: [agi] Universal intelligence test benchmark

[John G. Rose]

> From: Matt Mahoney [mailto:matmaho...@yahoo.com]
> 
> --- On Sat, 12/27/08, John G. Rose  wrote:
> 
> > > > How does consciousness fit into your compression
> > > > intelligence modeling?
> > >
> > > It doesn't. Why is consciousness important?
> > >
> >
> > I was just prodding you on this. Many people on this list talk about
> the
> > requirements of consciousness for AGI and I was imagining some sort
> of
> > consciousness in one of your command line compressors :)
> > I've yet to grasp
> > the relationship between intelligence and consciousness though lately
> I
> > think consciousness may be more of an evolutionary social thing. Home
> grown
> > digital intelligence, since it is a loner, may not require "much"
> > consciousness IMO..
> 
> What we commonly call consciousness is a large collection of features
> that distinguish living human brains from dead human brains: ability to
> think, communicate, perceive, make decisions, learn, move, talk, see,
> etc. We only attach significance to it because we evolved, like all
> animals, to fear a large set of things that can kill us.
> 


Well I think consciousness must be some sort of out of band intelligence
that bolsters an entity in terms of survival. Intelligence probably
stratifies or optimizes in zonal regions of similar environmental
complexity, consciousness being one or an overriding out-of-band one...

> 
> I was hoping to discover an elegant theory for AI. It didn't quite work
> that way. It seems to be a kind of genetic algorithm: make random
> changes to the code and keep the ones that improve compression.
> 

Is this true for most data? For example would PI digit compression attempts
result in genetic emergences the same as say compressing environmental
noise? I'm just speculating that genetically originated data would require
compression avenues of similar algorithmic complexity descriptors, for
example PI digit data does not originate genetically so compression attempts
would not show genetic emergences as "chained" as say environmental
noise basically I'm asking if you can tell the difference from data that
has a genetic origination ingredient verses all non-genetic...

John








---
agi
Archives: https://www.listbox.com/member/archive/303/=now
RSS Feed: https://www.listbox.com/member/archive/rss/303/
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=8660244&id_secret=123753653-47f84b
Powered by Listbox: http://www.listbox.com

Re: Real-world vs. universal prior (was Re: [agi] Universal intelligence test benchmark)

[David Hart]

On Sat, Dec 27, 2008 at 5:25 PM, Ben Goertzel  wrote:

>
> I wrote down my thoughts on this in a little more detail here (with some
> pastings from these emails plus some new info):
>
>
> http://multiverseaccordingtoben.blogspot.com/2008/12/subtle-structure-of-physical-world.html
>

I really liked this essay. I'm curious about the clarity of terms 'real
world' and 'physical world' in some places. It seems that, to make its
point, the essay requires 'real world' and 'physical world' mean only
'practical' or 'familiar physical reality', depending on context. Whereas,
if 'real world' is reserved for a very broad definition of realities
including physical realities (including classical, quantum mechanical and
relativistic time and distance scales), peculiar human cultural realities,
and other definable realities, it will be easier in follow-up essays to
discuss AGI systems that can natively think simultaneously about any
multitude of interrelated realities (a trick that humans are really bad at).
I hope this makes sense...

-dave



---
agi
Archives: https://www.listbox.com/member/archive/303/=now
RSS Feed: https://www.listbox.com/member/archive/rss/303/
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=8660244&id_secret=123753653-47f84b
Powered by Listbox: http://www.listbox.com

Re: Real-world vs. universal prior (was Re: [agi] Universal intelligence test benchmark)

[Ben Goertzel]

David,

Good point... I'll revise the essay to account for it...

The truth is, we just don't know -- but in taking the "virtual world"
approach to AGI, we're very much **hoping** that a subset of "human everyday
physical reality" is good enough. ..

ben

On Sat, Dec 27, 2008 at 6:46 AM, David Hart  wrote:

> On Sat, Dec 27, 2008 at 5:25 PM, Ben Goertzel  wrote:
>
>>
>> I wrote down my thoughts on this in a little more detail here (with some
>> pastings from these emails plus some new info):
>>
>>
>> http://multiverseaccordingtoben.blogspot.com/2008/12/subtle-structure-of-physical-world.html
>>
>
> I really liked this essay. I'm curious about the clarity of terms 'real
> world' and 'physical world' in some places. It seems that, to make its
> point, the essay requires 'real world' and 'physical world' mean only
> 'practical' or 'familiar physical reality', depending on context. Whereas,
> if 'real world' is reserved for a very broad definition of realities
> including physical realities (including classical, quantum mechanical and
> relativistic time and distance scales), peculiar human cultural realities,
> and other definable realities, it will be easier in follow-up essays to
> discuss AGI systems that can natively think simultaneously about any
> multitude of interrelated realities (a trick that humans are really bad at).
> I hope this makes sense...
>
> -dave
>
>
>  --
>   *agi* | Archives 
>  | 
> ModifyYour Subscription
> 
>



-- 
Ben Goertzel, PhD
CEO, Novamente LLC and Biomind LLC
Director of Research, SIAI
b...@goertzel.org

"I intend to live forever, or die trying."
-- Groucho Marx



---
agi
Archives: https://www.listbox.com/member/archive/303/=now
RSS Feed: https://www.listbox.com/member/archive/rss/303/
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=8660244&id_secret=123753653-47f84b
Powered by Listbox: http://www.listbox.com

Re: Real-world vs. universal prior (was Re: [agi] Universal intelligence test benchmark)

[Ben Goertzel]

Dave --

See mildly revised version, where I replaced "real world" with "everyday
world" (and defined the latter term explicitly), and added a final section
relevant to the distinctions between the everyday world, simulated everyday
worlds, and other portions of the physical world.

http://multiverseaccordingtoben.blogspot.com/2008/12/subtle-structure-of-physical-world.html

-- Ben


On Sat, Dec 27, 2008 at 8:28 AM, Ben Goertzel  wrote:

>
> David,
>
> Good point... I'll revise the essay to account for it...
>
> The truth is, we just don't know -- but in taking the "virtual world"
> approach to AGI, we're very much **hoping** that a subset of "human everyday
> physical reality" is good enough. ..
>
> ben
>
>
> On Sat, Dec 27, 2008 at 6:46 AM, David Hart  wrote:
>
>> On Sat, Dec 27, 2008 at 5:25 PM, Ben Goertzel  wrote:
>>
>>>
>>> I wrote down my thoughts on this in a little more detail here (with some
>>> pastings from these emails plus some new info):
>>>
>>>
>>> http://multiverseaccordingtoben.blogspot.com/2008/12/subtle-structure-of-physical-world.html
>>>
>>
>> I really liked this essay. I'm curious about the clarity of terms 'real
>> world' and 'physical world' in some places. It seems that, to make its
>> point, the essay requires 'real world' and 'physical world' mean only
>> 'practical' or 'familiar physical reality', depending on context. Whereas,
>> if 'real world' is reserved for a very broad definition of realities
>> including physical realities (including classical, quantum mechanical and
>> relativistic time and distance scales), peculiar human cultural realities,
>> and other definable realities, it will be easier in follow-up essays to
>> discuss AGI systems that can natively think simultaneously about any
>> multitude of interrelated realities (a trick that humans are really bad at).
>> I hope this makes sense...
>>
>> -dave
>>
>>
>>  --
>>   *agi* | Archives 
>>  | 
>> ModifyYour Subscription
>> 
>>
>
>
>
> --
> Ben Goertzel, PhD
> CEO, Novamente LLC and Biomind LLC
> Director of Research, SIAI
> b...@goertzel.org
>
> "I intend to live forever, or die trying."
> -- Groucho Marx
>
>


-- 
Ben Goertzel, PhD
CEO, Novamente LLC and Biomind LLC
Director of Research, SIAI
b...@goertzel.org

"I intend to live forever, or die trying."
-- Groucho Marx



---
agi
Archives: https://www.listbox.com/member/archive/303/=now
RSS Feed: https://www.listbox.com/member/archive/rss/303/
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=8660244&id_secret=123753653-47f84b
Powered by Listbox: http://www.listbox.com

Re: Real-world vs. universal prior (was Re: [agi] Universal intelligence test benchmark)

[Mike Tintner]

Ben: in taking the "virtual world" approach to AGI, we're very much **hoping** 
that a subset of "human everyday physical reality" is good enough. ..

Ben,

Which subset(s)?

The idea that you can virtually recreate any part or processes of reality seems 
horribly flawed - and unexamined.

Take the development of intelligence. You seem (from recent exchanges) to 
accept that there is very roughly some natural order to the development of 
intelligence. So for example, you can't learn about planets & universes, if you 
haven't first learned about simple objects like stones and balls - nor about 
politics, governments and international relations if you haven't first learned 
about language, speech/conversation, emotions, other minds & much more.  Now we 
- science - have some ideas about this natural order - about how we have to 
develop from understanding simple to complex things. But overall our picture is 
pathetic and hugely gapped.  For science to produce an extensive picture of 
development here would - at a guess - take at least hundreds of thousands, if 
not millions of scientists, and many thousands (or millions) of discoveries, 
and many changes of competing paradigms.

What are the chances then of an individual like you, or team of individuals, 
being able to design a coherent, practical order of intellectual development 
for an artificial, virtual agent straight off in a few years ?

The same applies to any part of reality. We - science - may have a detailed 
picture of how some pieces of objects, like stones and water, work. But again 
our overall ability to model how all those particles, atoms and molecules 
interrelate in any given object, and how the object as a whole behaves, is 
still very limited. We still have all kinds of gaps in our picture of water. 
Scientific models are always far from the real thing.

Again, to come anywhere near completing those models will take new armies of 
scientists.

What are the chances then of a few individuals being able to correctly model 
the behaviour of any objects in the real world on a flat screen?

IOW the "short cut" you hope for is probably the longest way round you could 
possibly choose. Robotics - forgetting altogether about formally modelling the 
world - and just interacting with it directly,   is actually shorter by far. So 
I doubt whether you have ever seriously examined how you would recreate a 
*particular* "subset of reality".in any detail  - as simple even, say, as a 
ball -  as opposed to the general idea. Have you?  

[Nb We're talking here about composite models of objects - so it's easy enough 
to create a reasonable picture of a ball bouncing on a hard surface, but what 
happens when your agent sits on it, or rubs it on his shirt, or bounces it on 
water,  or sand, or throws it at another ball in mid-air, or (as we've partly 
discussed) plays with it like an infant ?] 


---
agi
Archives: https://www.listbox.com/member/archive/303/=now
RSS Feed: https://www.listbox.com/member/archive/rss/303/
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=8660244&id_secret=123753653-47f84b
Powered by Listbox: http://www.listbox.com

Re: Real-world vs. universal prior (was Re: [agi] Universal intelligence test benchmark)

[Ben Goertzel]

The question is how much detail about the world needs to be captured in a
simulation in order to support humanlike cognitive development.

As a single example, Piagetan conservation of volume experiments are often
done with water, which would suggest you need to have fluid dynamics in your
simulation to support that kind of experiment.  But you don't necessarily,
because you can do those same experiments with fairly large beads, via using
Newtonian mechanics to simulate the rolling-around of the beads.  So it's
not clear whether fluidics is needed in the sim world to enable humanlike
cognitive development, versus whether beads rolling around is good enough
(at the moment I suspect the latter)

As I'm planning to write a paper on this stuff, I don't want to diver time
to writing a long email about it.

As for "which subset" of a physical reality: my specific idea is to simulate
a real-world preschool, with enough fidelity that AIs can carry out the same
learning tasks that human kids carry out in a real preschool.


On Sat, Dec 27, 2008 at 9:56 AM, Mike Tintner wrote:

>  Ben: in taking the "virtual world" approach to AGI, we're very much
> **hoping** that a subset of "human everyday physical reality" is good
> enough. ..
>
> Ben,
>
> Which subset(s)?
>
> The idea that you can virtually recreate any part or processes of reality
> seems horribly flawed - and unexamined.
>
> Take the development of intelligence. You seem (from recent exchanges) to
> accept that there is very roughly some natural order to the development of
> intelligence. So for example, you can't learn about planets & universes, if
> you haven't first learned about simple objects like stones and balls - nor
> about politics, governments and international relations if you haven't first
> learned about language, speech/conversation, emotions, other minds & much
> more.  Now we - science - have some ideas about this natural order - about
> how we have to develop from understanding simple to complex things. But
> overall our picture is pathetic and hugely gapped.  For science to produce
> an extensive picture of development here would - at a guess - take at least
> hundreds of thousands, if not millions of scientists, and many thousands (or
> millions) of discoveries, and many changes of competing paradigms.
>
> What are the chances then of an individual like you, or team of
> individuals, being able to design a coherent, practical order of
> intellectual development for an artificial, virtual agent straight off in a
> few years ?
>
> The same applies to any part of reality. We - science - may have a detailed
> picture of how some pieces of objects, like stones and water, work. But
> again our overall ability to model how all those particles, atoms and
> molecules interrelate in any given object, and how the object as a whole
> behaves, is still very limited. We still have all kinds of gaps in our
> picture of water. Scientific models are always far from the real thing.
>
> Again, to come anywhere near completing those models will take new armies
> of scientists.
>
> What are the chances then of a few individuals being able to correctly
> model the behaviour of any objects in the real world on a flat screen?
>
> IOW the "short cut" you hope for is probably the longest way round you
> could possibly choose. Robotics - forgetting altogether about formally
> modelling the world - and just interacting with it directly,   is actually
> shorter by far. So I doubt whether you have ever seriously examined how you
> would recreate a *particular* "subset of reality".in any detail  - as simple
> even, say, as a ball -  as opposed to the general idea. Have you?
>
> [Nb We're talking here about composite models of objects - so it's easy
> enough to create a reasonable picture of a ball bouncing on a hard surface,
> but what happens when your agent sits on it, or rubs it on his shirt, or
> bounces it on water,  or sand, or throws it at another ball in mid-air, or
> (as we've partly discussed) plays with it like an infant ?]
> --
>   *agi* | Archives 
>  | 
> ModifyYour Subscription
> 
>



-- 
Ben Goertzel, PhD
CEO, Novamente LLC and Biomind LLC
Director of Research, SIAI
b...@goertzel.org

"I intend to live forever, or die trying."
-- Groucho Marx



---
agi
Archives: https://www.listbox.com/member/archive/303/=now
RSS Feed: https://www.listbox.com/member/archive/rss/303/
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=8660244&id_secret=123753653-47f84b
Powered by Listbox: http://www.listbox.com

RE: [agi] Universal intelligence test benchmark

[Matt Mahoney]

--- On Sat, 12/27/08, John G. Rose  wrote:

> Well I think consciousness must be some sort of out of band intelligence
> that bolsters an entity in terms of survival. Intelligence probably
> stratifies or optimizes in zonal regions of similar environmental
> complexity, consciousness being one or an overriding out-of-band one...

No, consciousness only seems mysterious because human brains are programmed 
that way. For example, I should logically be able to convince you that "pain" 
is just a signal that reduces the probability of you repeating whatever actions 
immediately preceded it. I can't do that because emotionally you are convinced 
that "pain is real". Emotions can't be learned the way logical facts can, so 
emotions always win. If you could accept the logical consequences of your brain 
being just a computer, then you would not pass on your DNA. That's why you 
can't.

BTW the best I can do is believe both that consciousness exists and 
consciousness does not exist. I realize these positions are inconsistent, and I 
leave it at that.

> > I was hoping to discover an elegant theory for AI. It didn't quite work
> > that way. It seems to be a kind of genetic algorithm: make random
> > changes to the code and keep the ones that improve compression.
> > 
> 
> Is this true for most data? For example would PI digit compression attempts
> result in genetic emergences the same as say compressing environmental
> noise? I'm just speculating that genetically originated data would require
> compression avenues of similar algorithmic complexity descriptors, for
> example PI digit data does not originate genetically so compression attempts
> would not show genetic emergences as "chained" as say environmental
> noise basically I'm asking if you can tell the difference from data that
> has a genetic origination ingredient verses all non-genetic...

No, pi can be compressed to a simple program whose size is dominated by the log 
of the number of digits you want.

For text, I suppose I should be satisfied that a genetic algorithm compresses 
it, except for the fact that so far the algorithm requires a human in the loop, 
so it doesn't solve the AI problem.

-- Matt Mahoney, matmaho...@yahoo.com



---
agi
Archives: https://www.listbox.com/member/archive/303/=now
RSS Feed: https://www.listbox.com/member/archive/rss/303/
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=8660244&id_secret=123753653-47f84b
Powered by Listbox: http://www.listbox.com

Re: [agi] Universal intelligence test benchmark

[J. Andrew Rogers]

On Dec 26, 2008, at 6:18 PM, Ben Goertzel wrote:
Most compression tests are like defining intelligence as the  
ability to catch mice. They measure the ability of compressors to  
compress specific files. This tends to lead to hacks that are tuned  
to the benchmarks. For the generic intelligence test, all you know  
about the source is that it has a Solomonoff distribution (for a  
particular machine). I don't know how you could make the test any  
more generic.


IMO the test is *too* generic  ... I don't think real-world AGI is  
mainly about being able to recognize totally general patterns in  
totally general datasets.   I suspect that to do that, the best  
approach is ultimately going to be some AIXItl variant ... meaning  
it's a problem that's not really solvable using a real-world amount  
of resources.  I suspect that all the AGI system one can really  
build are SO BAD at this general problem, that it's better to  
characterize AGI systems



An interesting question is which pattern subset if ignored would make  
the problem tractable.


J. Andrew Rogers



---
agi
Archives: https://www.listbox.com/member/archive/303/=now
RSS Feed: https://www.listbox.com/member/archive/rss/303/
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=8660244&id_secret=123753653-47f84b
Powered by Listbox: http://www.listbox.com

Re: [agi] Introducing Steve's "Theory of Everything" in cognition.

[Abram Demski]

Russel,

There is a somewhat brief section in this article:

http://plato.stanford.edu/entries/goedel/#SpeUpThe

The section gives 2 forms of the theorem, the 2nd of which is the more
interesting ("theorem 6").

I came across this subject in the book "logic, logic, and logic" by
Boolos. Boolos describes 1st-order logic as "practically incomplete"
(complete, but astronomically slow in some cases). He also discusses
astronomical differences between different first-order deduction
systems. A fascinating topic.

--Abram

On Sat, Dec 27, 2008 at 2:55 AM, Russell Wallace
 wrote:
> On Fri, Dec 26, 2008 at 11:56 PM, Abram Demski  wrote:
>> That's not to say that I don't think some representations are
>> fundamentally more useful than others-- for example, I know that some
>> proofs are astronomically larger in 1st-order logic as compared to
>> 2nd-order logic, even in domains where 1st-order logic is
>> representationally sufficient.
>
> Do you have any online references handy for these? One of the things
> I'm still trying to figure out is to just what extent it is necessary
> to go to higher-order logic to make interesting statements about
> program code, and this sounds like useful data.
>
>
> ---
> agi
> Archives: https://www.listbox.com/member/archive/303/=now
> RSS Feed: https://www.listbox.com/member/archive/rss/303/
> Modify Your Subscription: https://www.listbox.com/member/?&;
> Powered by Listbox: http://www.listbox.com
>



-- 
Abram Demski
Public address: abram-dem...@googlegroups.com
Public archive: http://groups.google.com/group/abram-demski
Private address: abramdem...@gmail.com


---
agi
Archives: https://www.listbox.com/member/archive/303/=now
RSS Feed: https://www.listbox.com/member/archive/rss/303/
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=8660244&id_secret=123753653-47f84b
Powered by Listbox: http://www.listbox.com

Re: [agi] Introducing Steve's "Theory of Everything" in cognition.

[Abram Demski]

Steve,

My thinking in the "significant figures" issue is that the purpose of
unsupervised learning is to find a probabilistic model of the data
(whereas the purpose of supervised learning is to find a probabilistic
model of *one* variable *conditioned on* all the others). When you
talk about the insufficiency of standard PCA, do you think the
problems you refer to relate to

(1) PCA finding a suboptimal model, or
(2) the optimal model being not quite what you are after?

--Abram

On Sat, Dec 27, 2008 at 3:05 AM, Steve Richfield
 wrote:
> Abram,
>
> On 12/26/08, Abram Demski  wrote:
>>
>> Steve,
>>
>> When I made the statement about Fourier I was thinking of JPEG
>> encoding. A little digging found this book, which presents a unified
>> approach to (low-level) computer vision based on the Fourier
>> transform:
>>
>>
>> http://books.google.com/books?id=1wJuTMbNT0MC&dq=fourier+vision&printsec=frontcover&source=bl&ots=3ogSJ2i5uW&sig=ZdvvWvu82q8UX1c5Abq6hWvgZCY&hl=en&sa=X&oi=book_result&resnum=2&ct=result#PPA4,M
>
>
> Interesting, but seems far removed from wet neuronal functionality,
> unsupervised learning, etc.
>>
>> >> But that is beside the present point. :)
>> >
>> >
>> > Probably so. I noticed that you recently graduated, so I thought that I
>> > would drop that thought to make (or unmake) your day.
>>
>> :) I should really update that. It's been a while now.
>>
>> >> generally, any transform that makes the data more sparse, or simpler,
>> >> seems good
>> >
>> >
>> > Certainly if it results in extracting some useful of merit.
>> >
>> >>
>> >> -- which is of course what PCA does,
>> >
>> >
>> > Sometimes yes, and sometimes no. I am looking at incremental PCA
>> > approaches
>> > that reliably extract separate figures of merit rather than
>> > smushed-together
>> > figures of merit as PCA often does.
>>
>> How do you define "figures of merit"? Sounds like an ill-defined
>> problem to me. We don't know which features we *really* want to
>> extract from an image until we know the utility function of the
>> environment, and so know what information will help us achieve our
>> goals.
>
>
> There are several views of this, e.g.
> 1.  Pick something to recognize and see if back-propagation says that it is
> useful.
>  In practice this has problems, because once a downstream neuron makes
> any tentative use,
>  then changing an upstream neuron's functionality scrambles the
> downstream neuron's output.
> 2.  Pick one of the most consistent, easily-recognizable, most
> information-containing things to recognize ala PCA,
>  and expect downstream neurons to combine inputs to extract whatever
> they need through Bayesian logic.
>  This may suffer from too many prospective things to recognize, most of
> which are not needed.
>
> These two methods look like they could fix each other's shortcomings,
> because good initial choices
> of figures of merit should then result in neurons either keeping their
> functionality or abandoning
> it, and thereby avoid the problems of changing functionality scrambling
> downstream neurons.
> This way, back-propagation could be used to select which upstream neurons
> need to find something
> else to do, but would have little/no impact on incremental "learning" as
> reward/punishment systems now do.
>
> My BIG challenge is that without something like dp/dt, unsupervised learning
> doesn't work.
> Now, with dp/dt it is a whole new game, and I have no idea where the
> threshold of real-world functionality lies.
> Hence, I seem to be forced into "pessimization"- making things as good as
> possible,
> even though I may be well past that threshold.
>
> Eddie's NN platform is able to tie into other applications, like flight
> simulator, web cams, etc.
> Hence, there is the whole Internet full of cameras to learn with, and it
> might be interesting to see if
> such a NN would be able to figure out how to fly a plane, maybe like
> Skinner's pidgeons .
>
> Thanks for your continuing thoughts.
>
> Steve Richfield
> 
>>
>> On Sat, Dec 27, 2008 at 12:01 AM, Steve Richfield
>>  wrote:
>> > Abram,
>> >
>> > On 12/26/08, Abram Demski  wrote:
>> >>
>> >> Steve,
>> >>
>> >> It is strange to claim that prior PhDs will be worthless when what you
>> >> are suggesting is that we apply the standard methods to a different
>> >> representation.
>> >
>> >
>> > Much of AI and pretty much all of AGI is built on the proposition that
>> > we
>> > humans must code knowledge because the stupid machines can't efficiently
>> > learn it on their own, in short, that UNsupervised learning is
>> > difficult.
>> > Note that in nature, UNsupervised learning handily outperforms
>> > supervised
>> > learning. What good is supervised NN technology when UNsupervised NNs
>> > will
>> > perform MUCH better? What good are a few hand-coded AGI rules and the
>> > engine
>> > that runs them, when an UNsupervised AGI can learn them orders of
>> > magnitude
>> > faster than cities full of programmer

Re: [agi] Universal intelligence test benchmark

[J. Andrew Rogers]

On Dec 26, 2008, at 7:24 PM, Philip Hunt wrote:

2008/12/27 J. Andrew Rogers :


I think many people greatly underestimate how many gaping algorithm  
holes
there are in computer science for even the most important and  
mundane tasks.

The algorithm coverage of computer science is woefully incomplete,


Is it? In all my time as a programmer, it's never occurred to me to
think "I wish there was an algorithm to do X". mybe that's just me.
And there are vast numbers of useful algorithms that people use every
day.



Computers are general, so there always exists an obvious algorithm for  
doing any particular task. Whether or not that obvious algorithm is  
efficient is quite another thing, since the real costs of various  
algorithms are far from equivalent even if their functionality is.


The Sieve of Eratosthenes will allow you to factor any integer in  
theory, but for non-trivial integers you will want to use a number  
field sieve. The limitations of many types of software are  
fundamentally based in the complexity class of the of the attributes  
of the algorithms they use.  We frequently improperly conflate  
"theoretically impossible" and "no tractable algorithm currently  
exists".




I wonder (thinking out loud here) are there any statistics for this?
For example if you plot the number of such algorithms that've been
found over time, what sort of curve would you get? (Of course, you'd
have to define "general, elegant algorithm for basic problem", which
might be tricky)



I am still surprised often enough that it is obvious that there is  
considerable amounts of innovation still being done.  It both amuses  
and annoys me no end that some common algorithms have design  
characteristics that reflect long-forgotten assumptions that do not  
even make sense in the context they are used e.g. compulsive tree  
balancing behavior of intrinsically unbalanced data structures.




In short, we have no idea what important and fundamental
algorithms will be discovered from one year to the next that change  
the

boundaries of what is practically possible with computer science.


Is this true? It doesn't seem right to me. AIUI the current state of
the art in operating systems, compilers, garbage collectors, etc is
only slightly more efficient than it was 10 or 20 years ago. (In fact,
most practical programs are a good deal less efficient, because faster
processors mean they don't have to be).



It is easy to forget how many basic algorithms we use ubiquitously are  
relatively recent.  The concurrent B-tree algorithm that is  
pervasively used in databases, file systems, and just about everything  
else was published in the 1980s.  In fact, most of the algorithms that  
make up a modern SQL database as we understand them were developed in  
the 1980s, even though the relational model goes back to the 1960s.




I don't think I understand you. To me "indexing" means what the Google
search engine or an SQL database does -- but you're using the word
with a different meaning aren't you?



I mean it exactly like you understand it.  Indexed access methods and  
representations.




Sorry, you've lost me again -- I've never heard of the term
"hyper-rectangles" in relation to relational databases.



Most people haven't, because there are no hyper-rectangles in  
relational database *implementations* seeing as how there are no  
useful algorithms for representing them.  Nonetheless, the underlying  
model describes operations using hyper-rectangles in high-dimensional  
spaces.


In an ideal relational implementation there are never external  
indexes, only data organized in its native high-dimensionality logical  
space, since external indexes are a de-normalization.




It is not because it is theoretically impossible, but
because it is only possible if someone discovers a general  
algorithm for

indexing hyper-rectangles -- faking it is not distributable.


How do we know that there is such an algorithm?



We don't unless someone publishes one, but there is a lot of evidence  
that seems to imply otherwise and which proves that much of the  
research that has been done was misdirected.  Aesthetically, the  
current algorithms for doing this are nasty ugly hacks, and that lack  
of elegance is often an indicator that a better way exists.


In the specific case of indexing hyper-rectangles, the first basic  
algorithm was published in 1971 (IIRC), but was supplanted by a  
completely different family of algorithm in 1981. Virtually all  
research has been based on derivatives of the 1981 algorithm, since it  
appeared to have better properties.  Unfortunately, we can now prove  
that this algorithm class can never yield a general solution and that  
a solution must look like a variant of the original 1971 algorithm  
family that has been ignored for a quarter century. Interestingly, the  
proof of this comes by way of the recent explosion in the research on  
massively concurrent data structures due to the 

Re: Real-world vs. universal prior (was Re: [agi] Universal intelligence test benchmark)

[David Hart]

'On Sun, Dec 28, 2008 at 1:02 AM, Ben Goertzel  wrote:

>
> See mildly revised version, where I replaced "real world" with "everyday
> world" (and defined the latter term explicitly), and added a final section
> relevant to the distinctions between the everyday world, simulated everyday
> worlds, and other portions of the physical world.


I think that's much more clear, and the additions help to frame the meaning
of 'everyday world'.

Another important open question, that's really a generalization of 'how much
detail does the virtual world need to have?', is can we create practical
progressions of simulations of the everyday world, such that the first (and
more crude) simulations are very useful to early attempts at teaching
proto-AGIs, and the development of progressively more sophisticated
simulations roughly tracks the development of progress in AGI design and
development.

I also see the kernel of a formally defined science of discovery of the
general properties of everyday intelligence; if presented in ways that
cognitive scientists appreciate, it could really catch on!

-dave



---
agi
Archives: https://www.listbox.com/member/archive/303/=now
RSS Feed: https://www.listbox.com/member/archive/rss/303/
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=8660244&id_secret=123753653-47f84b
Powered by Listbox: http://www.listbox.com

Re: [agi] Universal intelligence test benchmark

[Matt Mahoney]

--- On Sat, 12/27/08, J. Andrew Rogers  wrote:

> An interesting question is which pattern subset if ignored
> would make the problem tractable.

We don't want to make the problem tractable. We want to discover new, efficient 
general purpose learning algorithms. AIXI^tl is intractable, yet we have lots 
of fast algorithms for important subsets: linear regression, decision trees, 
neural networks, clustering, SVM, etc. If we took out all the problems we 
couldn't already solve quickly, then what is the point?

Here is some sample output of the generic compression benchmark data. It 
consists of NUL terminated strings packed 8 bits per byte with the low bits of 
the last byte padded with zero bits. I sorted the data by decreasing frequency 
of occurrence in a sample of 1 million strings. The data is binary, but 
displayed here in hex.

The top 20 string are 5 bits or less in length. The most frequent string is all 
zero bits, which has an algorithmic complexity of about -log2(47161/100) = 
4.4 bits in the chosen instruction set.

47161 00
26352 8000
14290 C000
14137 4000
 7323 A000
 7220 E000
 7122 2000
 7084 6000
 3658 3000
 3651 5000
 3616 7000
 3588 9000
 3588 1000
 3549 D000
 3523 B000
 3451 F000
 1819 A800
 1799 7800
 1797 B800
 1787 6800
 1786 8800

Later we start seeing strings of 1 bits of various length, sometimes with a 
leading 0 bit, and patterns of alternating 0 and 1 bits (...). The string 
format constraint does not allow the obvious case of long strings of 0 bits.

  393 1200
  392 F000
  392 AE00
  391 B600
  390 BE00
  388 D600
  386 8600
  385 5E00
  384 BA00
  384 4E00
  379 7A00
  377 FA00
  375 F600
  374 6A00
  373 8A00
  373 3A00
  371 7600
  370 D200
  370 9600
  369 8E00
  368 FFFE00
  367 9E00
  366 1600
  364 7E00
  363 9A00
  351 FFE000
  344 F800
  341 F800
  325 FFF000
  308 C000
  289 F000
  243 7FE000
  242 555400
  241 FE00
  240 F000
  236 FFF800
  230 FF8000
  230 E500
  229 FFC000
  224 FF00
  224 7800
  224 0D00
  222 9900
  219 5500
  218 0500
  216 8100
  216 7FFFE000
  215 0100
  213 4700
  211 FFFE00
  211 4100
  210 AD00
  209 0300
  208 8900
  207 1500

Here is a sample from the large set that occur exactly twice, which implies 
about 19 bits of algorithmic complexity (probability 2/10^6). A typical 
sequence has a few leading bits that occur once, followed by a repeating bit 
sequence of length 3-5 or occasionally longer. A hex sequence like 249249249... 
is actually the bit sequence 001001001001...

 2 E4E4E4E4E4E4E4E4E4E4E400
 2 E4E4E4E4E4E4E4E4E4E4C000
 2 E4E4E4E4E4E4E4E4E4E000
 2 E4E4E4E4E400
 2 E4DFFE00
 2 E4DC00
 2 E4DB6DB6DB6DB600
 2 E4D400
 2 E4D0A000
 2 E400
 2 E4CC00
 2 E400
 2 E4CC00
 2 E400
 2 E4C993264C993264C993264C99326400
 2 E4C993264C993264C99300
 2 E4C993264C993264C99000
 2 E4C993264C993264C98000
 2 E4C993264C99324000
 2 E4C993264C993000
 2 E4C800
 2 E4C400
 2 E4BC9792F25E4BC97900
 2 E4B700
 2 E4AE00
 2 E48000
 2 E4AAA000
 2 E4AAA800
 2 E4A49249249249249000
 2 E4A400
 2 E492492492492492492492492492492492492000
 2 E4924924924924924924924924924924924900
 2 E492492492492492492492492492492400
 2 E492492492492492492492492492492000
 2 E49249249249249249249249249200
 2 E49249249249249249249249248000
 2 E48A00
 2 E4892248922489224892248800
 2 E48800
 2 E484422110884422110800
 2 E48120481204812000
 2 E47FFE00

Among strings that occur once (which is most of the data), we see many strings 
that follow the same type of patterns, but with more unique leading bits and 
longer repetition cycles. However you occasionally come across strings that 
have no obvious pattern. THOSE are the interesting problems.

1 FC514514514000
1 FC51255125512551255100
1 FC5100
1 FC50F143C50F143C50F143C400
1 FC50D50D50D50D50D50D50D500
1 FC50AB8A15714200
1 FC508000
1 FC507941E507941E507941E500
1 FC5028140A05028000
1 FC4FB7776000
1 FC4FDC4FDC4FDC00
1 FC4FB6DB6DB6DB6DB6DB6800
1 FC4F62F727C5EE5F00
1 FC4EC9D93B2764EC9D93B27000
1 FC4E66739CE739CC00
1 FC4DC1B89B83713700
1 FC4DB4924924924800
1 FC4D89B13626C4D89B136000
1 FC4D89B13626C4D800
1 FC4D4C4D4C4D4C4D4C00
1 FC4D1C8000
1 FC4D09A1342684D09A13424000
1 FC4CF8933E24CF8000
1 FC4CC400
1 FC4C7C4C7C4C7C4C7C00
1 FC4C4C4C4C4C4C4C4C4C00
1 FC4C4C4C4C4C4C00
1 FC4C1194A32946528000
1 FC4C00
1 FC4BD24924924924924000
1 FC4B89712E25C4B897128000
1 FC4B575B96AEB72D5D6E4000
1 FC4B48D2348D2348D22000
1 FC4B0800
1 FC4A7E253F129F894FC4A78000
1 FC4A5294A52000
1 F

Re: [agi] Indexing

[J. Andrew Rogers]

On Dec 26, 2008, at 7:40 PM, Jim Bromer wrote:

I noticed that neither linked lists nor arrays were particularly
efficient for general operations that would include insertions,
deletions and searches, which, when you think about it, are pretty
much the norm.  How often do you need a large data index that only
rarely needs to be searched.  The irony is that you cannot combine the
two forms in a simple manner so that you can have a linked list for
fast insertion and deletion and an array for fast searches.



There are data structures and algorithms that offer fast insert/delete  
and fast search, approximately constant computational complexity for  
both even.  It does require slightly more cleverness than a linked  
list though since your glorified lookup table will require a space- 
preserving representation.  It is much more common and usually simpler  
to merely use order-preserving representations like the common B+tree  
variants unless you have vast quantities of data.  Brute-force can be  
exceedingly efficient in small doses.




And with indirect indexes (using a handle or an index to an location
entry) the data requires frequent compression (to squeeze out the gaps
in the data area) if there is a heavy insertion and deletion.



An old, solved problem.  Well, "solved" in the sense that the  
tradeoffs and methods for managing this are well-understood.




I believe the problem is directly related to agi because data relevant
to some particular situation will tend to be distributed in a file so
that a lot of relational indexing is needed.



Perhaps the most relevant application to AGI is that it would very  
significantly improve the computational complexity of of representing  
and manipulating high-dimensionality relationships, particularly in  
distributed systems.  In conventional data-mining and pattern  
discovery analytics, the lack of scalability of high-dimensionality  
representations has long been major limitation on what one could do.


But for AI, consider algorithms like SIFT, which turn massive  
aggregates of 2-dimensional representations of 3-dimensional space  
(i.e. "photos") into a virtual model of the 3D space represented.  A  
neat algorithm, but limited by the fact that the algorithm represents  
the data in a 128-dimensional space before reducing it to 3- 
dimensional space, limiting the amount of data you could apply as a  
practical matter.




Since a lot of data can
be described as being analogously similar to other kinds of data and
since many variations in some particular kind of data might already
exist in a database, a great many complicated modifications of
concepts could, hypothetically, be done by modifying the indexes
alone.



In an ideal system, the database relation *is* the index.  External  
indexes are largely a software engineering artifact of only being able  
to represent one dimension per relation in a scalable manner.


J. Andrew Rogers



---
agi
Archives: https://www.listbox.com/member/archive/303/=now
RSS Feed: https://www.listbox.com/member/archive/rss/303/
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=8660244&id_secret=123753653-47f84b
Powered by Listbox: http://www.listbox.com