[agi] Re: Languages for AGI
I think choosing an architecture for AGI is a much more important problem than choosing a language. But there are some things we already know about AGI. First, AGI requires a vast amount of knowledge, and therefore a vast amount of computation. Therefore, at least part of the AGI will have to be implemented in a fast (perhaps parallel) language. Second, if you plan on having a team of programmers do the work (rather than all by yourself) then you will have to choose a widely known language. Early work in AI used languages like Lisp or Prolog to directly express knowledge. Now we all know (except at Cycorp) that this does not work. There is too much knowledge to code directly. You will need a learning algorithm and training and test data. The minimum requirement for AGI is a language model, which requires about 10^9 bits of information (based on estimates by Turing and Landauer, and the amount of language processed by adulthood). When you add vision, speech, robotics, etc., it will be more. We don't know how much, but if we use the human brain as a model, then one estimate is the number of synapses (about 10^13) multiplied by the access rate (10 Hz) = 10^14 operations per second. But these numbers are really just guesses. Perhaps they are high, but people have been working on computational shortcuts for the last 50 years without success. My work is in data compression, which I believe is an AI problem. (You might disagree, but first see my argument at http://cs.fit.edu/~mmahoney/compression/rationale.html ). Whether or not you agree, compression, like AGI, requires a great deal of memory and CPU. Many of the top compressors ranked in my benchmark are open source, and of those, the top languages are C++ followed by C and assembler. I don't know of any written in Java, C#, Python, or any interpreted languages, or any that use relational databases. AGI is amenable to parallel computation. Language, vision, speech, and robotics all involve combining thousands of soft constraints. This requires vector operations. The fastest way to do this on a PC is to use the parallel MMX and SSE2 instructions (or a GPU) that are not accessible in high level languages. The 16-bit vector dot product that I implemented in MMX as part of the neural network used in the PAQ compressor is 6 times faster than optimized C. Fortunately you do not need a lot of assembler, maybe a couple hundred lines of code to do most of the work. AGI is still an area of research. Not only do you need fast implementations so your experiments finish in reasonable time, but you will need to change your code many times. Train, test, modify, repeat. Your code has to be both optimized and structured so that it can be easily changed in ways you can't predict. This is hard, but unfortunately we do not know yet what will work. -- Matt Mahoney, [EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Re: Languages for AGI
J. Storrs Hall, PhD. wrote: On Sunday 18 February 2007 19:22, Ricardo Barreira wrote: You can spend all the time you want sharpening your axes, it'll do you no good if you don't know what you'll use it for... True enough. However, as I've also mentioned in this venue before, I want to be able to do general associative retrieval, interpolation, and extrapolation of time-varying trajectories of manifolds in n-dimensional spaces, and constructive solid geometry between them. BTW, if you do make efficient tools for this, we could certainly use them within Novamente -- though perhaps for a more limited purpose than what you envision. I wouldn't try to get NM to represent general knowledge in this way, but, for representing knowledge about the physical environment and things observed and projected therein, having such operations to act on 3D manifolds would be quite valuable ben I'm guessing that's about halfway to AI -- i.e. the amount of coding needed, with that as a primitive, needed for AI is about as much as required to get that from current programming tools. Josh - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303 - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Re: Languages for AGI
On Sunday 18 February 2007 19:22, Ricardo Barreira wrote: > You can spend all the time you want sharpening your axes, it'll do you > no good if you don't know what you'll use it for... True enough. However, as I've also mentioned in this venue before, I want to be able to do general associative retrieval, interpolation, and extrapolation of time-varying trajectories of manifolds in n-dimensional spaces, and constructive solid geometry between them. I'm guessing that's about halfway to AI -- i.e. the amount of coding needed, with that as a primitive, needed for AI is about as much as required to get that from current programming tools. Josh - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Re: Languages for AGI
In Abraham Lincoln's case I think it makes sense, since he already knows how he'll use the axe. I doubt that most people who are worrying about which language they'll use actually have a good idea of how to actually design an AGI... You can spend all the time you want sharpening your axes, it'll do you no good if you don't know what you'll use it for... Ricardo Well put, Ricardo. In the case of Novamente, we have the AGI design in hand; and in this context it's clear that different programming languages would have their different plusses and minuses for implementing Novamente ... but ultimately it is not going to be the programming language that's going to be the bottleneck. It's going to be tuning and tweaking the details of the numerous component algorithms that's going to be the bottleneck. And making **this** process tractable, within the context of the Novamente design, is much more dependent on how the code is structured and how thoughtfully the detailed design is done, than on what programming language is chosen. It's true that some languages more strongly encourage well-structured code than others do, but this is not really such a major point. We are currently restructuring some of our older Novamente code to use a more modern C++ idiom, heavier on templates and Boost. But the most critical aspect of this restructuring is the greater insight we've achieved recently into our AGI design itself, which has told us what kind of abstract interfaces our Novamente core system really needs, for interacting with the various AI modules. Ultimately the nature of these abstract interfaces would not be so different, no matter what the programming language (so long as the programming language was reasonably expressive and supplied modern programming constructs, i.e. no COBOL or FORTRAN...) -- Ben G - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Re: Languages for AGI
BTW: I really loved Haskell when I used it in the 90's, and if there were a rip-roaring fast SMP Haskell implementation with an effective customizable garbage collector, Novamente would probably be written in Haskell. But, there is not, and so Novamente is written in C++ ... but Novamente's "internal procedures" are written (i.e. learned by the system) in a language called Combo that has more in common with LISP or Haskell than C++ (it's purely functional, for one thing). So, the "codic modality" of the system refers to this internal Combo language, not to the underlying C++ or assembly language layers. -- Ben Mark Waser wrote: One reason for picking a language more powerful than the run-of-the-mill imperative ones (of which virtually all the ones mentioned so far are just different flavors) is that the can give you access to different paradigms that will enhance your view of how an AGI should work internally. Very true. Arguably, before choosing a language, an AGI researcher should know an ordinary imperative language (Pascal/Java/C++/C#), some flavor of LISP, Prolog, and some flavor of ML -- just to know what the real choices are . . . . The differences between most of the (ordinary imperative) languages that have been cited in this debate, from Python to Ruby to Java to C++ to C#, are merely syntax and the supporting infrastructure. Python is a little looser so that it is faster to develop in but it is more of a b*tch to debug. Ruby has a lot of infrastructure that makes web development really fast and easy but doesn't have a lot beyond that realm. Java and C# are virtually the same language -- except C# has a *lot* more infrastructure. And (in my opinion), C++ needs to be retired (get over it). Personally, I've got a bias against Perl and Python because I've had far too much experience that shows that quick to develop turns into difficult to maintain and expand past a given point. That's not what you want to see in something like AGI. I also have a bias against lower-level languages. Writing in machine language is for compilers, not human beings. Writing in assembly language is only justified when doing heavy duty algorithms on specialized floating point processors (and only if someone else hasn't done it first). Writing in C is just plain dumb (these days). I've done all three of these things *when it was appropriate* but it just isn't appropriate any longer. The successful developer is the one who uses the existing tools and infrastructure as the foundation for serious progress with clean, elegant architecture (and builds more layers of tools/infrastructure in their own personal toolbox). The programmers who are ending up out of work are the ones who keep re-inventing the wheel over and over again. As I've pointed out before in this venue, AGI is a hard enough task that it makes sense to do some serious work on tools-to-build-the-tools. As Abraham Lincoln put it, "If I had 8 hours to chop down a tree, I'd spend 6 sharpening my axe." Amen. - Original Message - From: "J. Storrs Hall, PhD." <[EMAIL PROTECTED]> To: Sent: Sunday, February 18, 2007 3:29 PM Subject: [agi] Re: Languages for AGI One reason for picking a language more powerful than the run-of-the-mill imperative ones (of which virtually all the ones mentioned so far are just different flavors) is that the can give you access to different paradigms that will enhance your view of how an AGI should work internally. A classic example is Prolog (which I use for most my day-to-day programming). I suggest reading "Clause and Effect" by Clocksin, a slim, high-level volume, to get a "For God's sake, why didn't they ever mention this in school" reaction when you see how 5 lines of Prolog do more than 100 lines of C for a wide range of AI-like problems. (Not the actual AI itself, mind you, but all the sort of thing that forms the infrastructure of a system). Surveys of languages in common use very often show that O'Caml leads the pack in a combination of conciseness of code and fast execution time, for what it's worth. http://shootout.alioth.debian.org/sandbox/index.php For the past month or two, I've been delving into a new paradigm that promises to have a deep effect on the way programming in general is done, but is especially germane to AI. It's reactive programming (see this discussion at http://lambda-the-ultimate.org/node/2068 ) The idea is a language that looks a lot more like the signals-and-systems mindset of cybernetics than the logic-based one of McCarthy and early AI. As I've pointed out before in this venue, AGI is a hard enough task that it makes sense to do some serious work on tools-to-build-the-tools. As Abraham Lincoln put it, "If I had 8 hours to chop down
Re: [agi] Re: Languages for AGI
On 2/18/07, Mark Waser <[EMAIL PROTECTED]> wrote: personal toolbox). The programmers who are ending up out of work are the ones who keep re-inventing the wheel over and over again. Thinking about the amount of redundant (wasted) effort involved with starting from scratch on an AI project, I considered an old adage and modified it: If you are not standing on the shoulders of giants, you are likely to be trampled by them. .. though I guess in the case of AGI, even giants have only taken a few tentative steps - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Re: Languages for AGI
> The idea is a language that looks a lot more like the signals-and-systems > mindset of cybernetics than the logic-based one of McCarthy and early AI. > > As I've pointed out before in this venue, AGI is a hard enough task that > it > makes sense to do some serious work on tools-to-build-the-tools. As > Abraham > Lincoln put it, "If I had 8 hours to chop down a tree, I'd spend 6 > sharpening > my axe." In Abraham Lincoln's case I think it makes sense, since he already knows how he'll use the axe. I doubt that most people who are worrying about which language they'll use actually have a good idea of how to actually design an AGI... You can spend all the time you want sharpening your axes, it'll do you no good if you don't know what you'll use it for... Ricardo - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Re: Languages for AGI
One reason for picking a language more powerful than the run-of-the-mill imperative ones (of which virtually all the ones mentioned so far are just different flavors) is that the can give you access to different paradigms that will enhance your view of how an AGI should work internally. Very true. Arguably, before choosing a language, an AGI researcher should know an ordinary imperative language (Pascal/Java/C++/C#), some flavor of LISP, Prolog, and some flavor of ML -- just to know what the real choices are . . . . The differences between most of the (ordinary imperative) languages that have been cited in this debate, from Python to Ruby to Java to C++ to C#, are merely syntax and the supporting infrastructure. Python is a little looser so that it is faster to develop in but it is more of a b*tch to debug. Ruby has a lot of infrastructure that makes web development really fast and easy but doesn't have a lot beyond that realm. Java and C# are virtually the same language -- except C# has a *lot* more infrastructure. And (in my opinion), C++ needs to be retired (get over it). Personally, I've got a bias against Perl and Python because I've had far too much experience that shows that quick to develop turns into difficult to maintain and expand past a given point. That's not what you want to see in something like AGI. I also have a bias against lower-level languages. Writing in machine language is for compilers, not human beings. Writing in assembly language is only justified when doing heavy duty algorithms on specialized floating point processors (and only if someone else hasn't done it first). Writing in C is just plain dumb (these days). I've done all three of these things *when it was appropriate* but it just isn't appropriate any longer. The successful developer is the one who uses the existing tools and infrastructure as the foundation for serious progress with clean, elegant architecture (and builds more layers of tools/infrastructure in their own personal toolbox). The programmers who are ending up out of work are the ones who keep re-inventing the wheel over and over again. As I've pointed out before in this venue, AGI is a hard enough task that it makes sense to do some serious work on tools-to-build-the-tools. As Abraham Lincoln put it, "If I had 8 hours to chop down a tree, I'd spend 6 sharpening my axe." Amen. - Original Message - From: "J. Storrs Hall, PhD." <[EMAIL PROTECTED]> To: Sent: Sunday, February 18, 2007 3:29 PM Subject: [agi] Re: Languages for AGI One reason for picking a language more powerful than the run-of-the-mill imperative ones (of which virtually all the ones mentioned so far are just different flavors) is that the can give you access to different paradigms that will enhance your view of how an AGI should work internally. A classic example is Prolog (which I use for most my day-to-day programming). I suggest reading "Clause and Effect" by Clocksin, a slim, high-level volume, to get a "For God's sake, why didn't they ever mention this in school" reaction when you see how 5 lines of Prolog do more than 100 lines of C for a wide range of AI-like problems. (Not the actual AI itself, mind you, but all the sort of thing that forms the infrastructure of a system). Surveys of languages in common use very often show that O'Caml leads the pack in a combination of conciseness of code and fast execution time, for what it's worth. http://shootout.alioth.debian.org/sandbox/index.php For the past month or two, I've been delving into a new paradigm that promises to have a deep effect on the way programming in general is done, but is especially germane to AI. It's reactive programming (see this discussion at http://lambda-the-ultimate.org/node/2068 ) The idea is a language that looks a lot more like the signals-and-systems mindset of cybernetics than the logic-based one of McCarthy and early AI. As I've pointed out before in this venue, AGI is a hard enough task that it makes sense to do some serious work on tools-to-build-the-tools. As Abraham Lincoln put it, "If I had 8 hours to chop down a tree, I'd spend 6 sharpening my axe." Josh - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303 - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
[agi] Re: Languages for AGI
One reason for picking a language more powerful than the run-of-the-mill imperative ones (of which virtually all the ones mentioned so far are just different flavors) is that the can give you access to different paradigms that will enhance your view of how an AGI should work internally. A classic example is Prolog (which I use for most my day-to-day programming). I suggest reading "Clause and Effect" by Clocksin, a slim, high-level volume, to get a "For God's sake, why didn't they ever mention this in school" reaction when you see how 5 lines of Prolog do more than 100 lines of C for a wide range of AI-like problems. (Not the actual AI itself, mind you, but all the sort of thing that forms the infrastructure of a system). Surveys of languages in common use very often show that O'Caml leads the pack in a combination of conciseness of code and fast execution time, for what it's worth. http://shootout.alioth.debian.org/sandbox/index.php For the past month or two, I've been delving into a new paradigm that promises to have a deep effect on the way programming in general is done, but is especially germane to AI. It's reactive programming (see this discussion at http://lambda-the-ultimate.org/node/2068 ) The idea is a language that looks a lot more like the signals-and-systems mindset of cybernetics than the logic-based one of McCarthy and early AI. As I've pointed out before in this venue, AGI is a hard enough task that it makes sense to do some serious work on tools-to-build-the-tools. As Abraham Lincoln put it, "If I had 8 hours to chop down a tree, I'd spend 6 sharpening my axe." Josh - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
