Re: [agi] Re: AI isn't cheap
I suppose in order to justify my cost estimate I need to define more precisely what I mean by AGI. I mean the cost of building an automated economy in which people don't have to work. This is not the same as automating what people currently do. Fifty years ago we might have imagined a future with robot gas station attendants and robot sales clerks. Nobody imagined self serve gas or shopping on the internet. But the exact form of the technology does not matter. People will invest money if there is an expected payoff higher than market driven interest rates. These numbers are known. AGI is worth $10^15 no matter how you build it. An alternative goal of AGI is uploading, which I believe will cost considerably less. How much would you pay to have a machine that duplicates your memories, goals, and behavior well enough to convince everyone else that it is you, and have that machine turned on after you die? Whether such a machine is "you" (does your consciousness transfer?) is an irrelevant philosophical issue. It is not important. What is important is the percentage of people who believe it is true and are therefore willing to pay to upload. However, once we develop the technology to scan brains and simulate them, there should be no need to develop custom software or training for each individual as there is for building an economy. The cost will be determined by Moore's Law. (This does not solve the economic issues. You still have to pay uploads to work, or to write the software to automate the economy). > > Software is not subject to Moore's Law so its cost > will eventually > > dominate. > > So creating software creating software may be a high payoff > subtask. If it is possible. However, there is currently no model for recursive self improvement. The major cost of "write a program to solve X" is the cost of describing X. When you give humans a programming task, they already know most of X without you specifying the details. To tell a machine, you either have to specify X in such detail that it is equivalent to writing the program, or you have to have a machine that knows everything that humans know, which is AGI. > > A human brain has about 10^9 bits of knowledge, of > which probably > > 10^7 to 10^8 bits are unique to each individual. > > How much of this uniqueness is little more than variations > on a much > smaller number of themes and/or irrelevant to the task? Good question. Everything you have learned through language is already known to somebody else. However, the fact that you learned X from Y is known only to you and possibly Y. Some fraction of nonverbally acquired knowledge is unique to you also. What fraction is relevant? Perhaps very little if AGI means new ways of solving problems rather than duplicating the work we now do. For other tasks such as entertainment, advertising, or surveillance, everything you know is relevant. > Google to the best of my knowledge is pursuing a some areas > of narrow > AI. I do not believe they are remotely after AGI. Google has only $10^11 to spend, not $10^15. -- Matt Mahoney, [EMAIL PROTECTED] --- On Thu, 9/11/08, Samantha Atkins <[EMAIL PROTECTED]> wrote: > From: Samantha Atkins <[EMAIL PROTECTED]> > Subject: Re: [agi] Re: AI isn't cheap > To: agi@v2.listbox.com > Date: Thursday, September 11, 2008, 3:19 AM > On Sep 9, 2008, at 7:54 AM, Matt Mahoney wrote: > > > --- On Mon, 9/8/08, Steve Richfield > <[EMAIL PROTECTED]> wrote: > > On 9/7/08, Matt Mahoney <[EMAIL PROTECTED]> > wrote: > > > >>> The fact is that thousands of very intelligent > people have been > >>> trying > >>> to solve AI for the last 50 years, and most of > them shared your > >>> optimism. > > > >> Unfortunately, their positions as students and > professors at various > >> universities have forced almost all of them into > politically correct > >> paths, substantially all of which lead nowhere, > for otherwise they > >> would > >> have succeeded long ago. The few mavericks who > aren't stuck in a > >> university (like those on this forum) all lack > funding. > > > > Google is actively pursuing AI and has money to spend. > If you have > > seen some of their talks, you know they are pursuing > some basic and > > novel research. > > Google to the best of my knowledge is pursuing a some areas > of narrow > AI. I do not believe they are remotely after AGI. > > > > > > > >>> Perhaps it would be more fruitful to estimate > the cost of > >>> automating the > >>> global economy. I explained my estimate of > 10^25 bits of memory, > &
Re: [agi] Re: AI isn't cheap
Samantha, This is a really great posting. Just one comment: On 9/11/08, Samantha Atkins <[EMAIL PROTECTED]> wrote: > > > On Sep 9, 2008, at 7:54 AM, Matt Mahoney wrote: > > A human brain has about 10^9 bits of knowledge, of which probably 10^7 to >> 10^8 bits are unique to each individual. >> > > How much of this uniqueness is little more than variations on a much > smaller number of themes and/or irrelevant to the task? WOW, my very favorite subject, since it so greatly overlaps with so many religions. My claim is that most people are NOT sufficiently unique to claim that they have any soul at all, so there is nothing for them to "save", especially through prayer that probably works to further standardize their brains. Public education also works great for soul-elimination. Steve Richfield --- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=111637683-c8fa51 Powered by Listbox: http://www.listbox.com
Re: [agi] Re: AI isn't cheap
On Sep 9, 2008, at 7:54 AM, Matt Mahoney wrote: --- On Mon, 9/8/08, Steve Richfield <[EMAIL PROTECTED]> wrote: On 9/7/08, Matt Mahoney <[EMAIL PROTECTED]> wrote: The fact is that thousands of very intelligent people have been trying to solve AI for the last 50 years, and most of them shared your optimism. Unfortunately, their positions as students and professors at various universities have forced almost all of them into politically correct paths, substantially all of which lead nowhere, for otherwise they would have succeeded long ago. The few mavericks who aren't stuck in a university (like those on this forum) all lack funding. Google is actively pursuing AI and has money to spend. If you have seen some of their talks, you know they are pursuing some basic and novel research. Google to the best of my knowledge is pursuing a some areas of narrow AI. I do not believe they are remotely after AGI. Perhaps it would be more fruitful to estimate the cost of automating the global economy. I explained my estimate of 10^25 bits of memory, 10^26 OPS, 10^17 bits of software and 10^15 dollars. You want to replicate the work currently done by 10^10 human brains. Hmm. Actually probably only some 10^6 of them at most are doing anything much worth replicating. :-) A brain has 10^15 synapses. A neuron axon has an information rate of 10 bits per second. As I said, you can argue about these numbers but it doesn't matter much. An order of magnitude error only changes the time to AGI by a few years at the current rate of Moore's Law. Software is not subject to Moore's Law so its cost will eventually dominate. So creating software creating software may be a high payoff subtask. A human brain has about 10^9 bits of knowledge, of which probably 10^7 to 10^8 bits are unique to each individual. How much of this uniqueness is little more than variations on a much smaller number of themes and/or irrelevant to the task? That makes 10^17 to 10^18 bits that have to be extracted from human brains and communicated to the AGI. What for? That seems like a very slow path that would pollute your AGI with countless errors and repetition. This could be done in code or formal language, although most of it will probably be done in natural language once this capability is developed. Natural languages are ridiculously slow and ambiguous. There is no way the 10^7 guesstimated unique bits per individual will ever get encoded in natural language anyway (or much of anything else other than its encoding in those brains). Since we don't know which parts of our knowledge is shared, the most practical approach is to dump all of it and let the AGI remove the redundancies. Actually, of the knowledge the AGI needs we have pretty good ideas of how much is shared. This will require a substantial fraction of each person's life time, so it has to be done in non obtrusive ways, such as recording all of your email and conversations (which, of course, all the major free services already do). What exactly is your goal? Are you attempting to simulate all of humankind? What for when the real thing is up and running?If you want uploads there are more direct possible paths after the AGI has perfected some crucial technologies. The cost estimate of $10^15 comes by estimating the world GDP ($66 trillion per year in 2006, increasing 5% annually) from now until we have the hardware to support AGI. We have the option to have AGI sooner by paying more. Simple economics suggests we will pay up to what it is worth. Why believe that the real productive intellectual output of the entire human world is anywhere close to or represented by the world GDP? It is not likely that we need to download the full contents of all human brains including the huge part that is mere variation on human primate programming to effectively meet and exceed this productive intellectual output. I find this method of estimating costs utterly unconvincing. - samantha --- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=111637683-c8fa51 Powered by Listbox: http://www.listbox.com
Re: [agi] Re: AI isn't cheap
(Top posting because Yahoo won't quote HTML email) Steve, Some of Google's tech talks on AI are here: http://www.google.com/search?hl=en&q=google+video+techtalks+ai&btnG=Search Google has an interest in AI because search is an AI problem, especially if you are searching for images or video. Also, their advertising model could use some help. I often go to data compression sites where Google is advertising compression socks, compression springs, air compressors, etc. I'm sure you've seen the problem. >>Software is not subject to Moore's Law so its cost will eventually dominate. >Here I could write a book and more. It could and should obey Moore's law, but history >and common practice has gone in other directions. Since you have experience writing sophisticated software on very limited hardware, perhaps you can enlighten us on how to exponentially reduce the cost of software instead of just talking about it. Maybe you can write AGI, or the next version of Windows, in one day. You might encounter a few obstacles, e.g. 1. Software testing is not computable (the halting problem reduces to it). 2. The cost of software is O(n log n). This is because you need O(log n) levels of abstraction to keep the interconnectivity of the software below the threshold of stability to chaos, above which it is not maintainable (where each software change introduces more bugs than it fixes). Abstraction levels are things like symbolic names, functions, classes, namespaces, libraries, and client-server protocols. 3. Increasing the computational power of a computer by n only increases its usefulness by log n. Useful algorithms tend to have a power law distribution over computational requirements. >>A human brain has about 10^9 bits of knowledge, of which probably 10^7 to 10^8 bits are unique to each individual. That makes 10^17 to 10^18 bits that have to be extracted from human brains and communicated to the AGI. This could be done in code or formal language, although most of it will probably be done in natural language once this capability is developed. >It would be MUCH easier and cheaper to just scan it out with something like a >scanning >UV fluorescent microscope. No it would not. Assuming we had the technology to copy brains (which we don't and you don't), then you have created a machine with human motives. You would still have to pay it to work. Do you really think you understand the brain well enough to reprogram it to want to work? >Further, I see the interest in AGIs on this forum as a sort of religious quest, that is >absurd to even consider outside of Western religions No, it is about the money. The AGIs that actually get built will be the ones that can make money for their owners. If an AGI can do anything that a human can do, then that would include work. Currently that's worth $66 trillion per year. -- Matt Mahoney, [EMAIL PROTECTED] --- On Tue, 9/9/08, Steve Richfield <[EMAIL PROTECTED]> wrote: From: Steve Richfield <[EMAIL PROTECTED]> Subject: Re: [agi] Re: AI isn't cheap To: agi@v2.listbox.com Date: Tuesday, September 9, 2008, 2:10 PM Matt, On 9/9/08, Matt Mahoney <[EMAIL PROTECTED]> wrote: --- On Mon, 9/8/08, Steve Richfield <[EMAIL PROTECTED]> wrote: On 9/7/08, Matt Mahoney <[EMAIL PROTECTED]> wrote: >>The fact is that thousands of very intelligent people have been trying >>to solve AI for the last 50 years, and most of them shared your optimism. >Unfortunately, their positions as students and professors at various >universities have forced almost all of them into politically correct >paths, substantially all of which lead nowhere, for otherwise they would >have succeeded long ago. The few mavericks who aren't stuck in a >university (like those on this forum) all lack funding. Google is actively pursuing AI and has money to spend. Maybe I am a couple of years out of date here, but the last time I looked, they were narrowly interested in search capabilities and not at all interested in linking up fragments from around the Internet, filling in missing metadata, problem solving, and the other sorts of things that are in my own area of interest. I attempted to interest them in my approaches, but got blown off apparently because they thought that my efforts were in a different direction than their interests. Have I missed something? If you have seen some of their talks, I haven't. Are any of them available somewhere? you know they are pursuing some basic and novel research. Outside of searching? >>Perhaps it would be more fruitful to estimate the cost of automating the >>global economy. I explained my estimate of 10^25 bits of memory, 10^26 >>OPS, 10^17 bits of software and 10^15 dollars. You want to replicate the work currently done by 10^10 human brains. A brain has 10^
Re: [agi] Re: AI isn't cheap
Matt, On 9/9/08, Matt Mahoney <[EMAIL PROTECTED]> wrote: > --- On Mon, 9/8/08, Steve Richfield <[EMAIL PROTECTED]> wrote: > On 9/7/08, Matt Mahoney <[EMAIL PROTECTED]> wrote: > > >>The fact is that thousands of very intelligent people have been trying > >>to solve AI for the last 50 years, and most of them shared your optimism. > > >Unfortunately, their positions as students and professors at various > >universities have forced almost all of them into politically correct > >paths, substantially all of which lead nowhere, for otherwise they would > >have succeeded long ago. The few mavericks who aren't stuck in a > >university (like those on this forum) all lack funding. > > Google is actively pursuing AI and has money to spend. Maybe I am a couple of years out of date here, but the last time I looked, they were narrowly interested in search capabilities and not at all interested in linking up fragments from around the Internet, filling in missing metadata, problem solving, and the other sorts of things that are in my own area of interest. I attempted to interest them in my approaches, but got blown off apparently because they thought that my efforts were in a different direction than their interests. Have I missed something? > If you have seen some of their talks, I haven't. Are any of them available somewhere? > you know they are pursuing some basic and novel research. Outside of searching? > >>Perhaps it would be more fruitful to estimate the cost of automating the > >>global economy. I explained my estimate of 10^25 bits of memory, 10^26 > >>OPS, 10^17 bits of software and 10^15 dollars. > > You want to replicate the work currently done by 10^10 human brains. A > brain has 10^15 synapses. A neuron axon has an information rate of 10 bits > per second. As I said, you can argue about these numbers but it doesn't > matter much. An order of magnitude error only changes the time to AGI by a > few years at the current rate of Moore's Law. > > Software is not subject to Moore's Law so its cost will eventually > dominate. Here I could write a book and more. It could and should obey Moore's law, but history and common practice has gone in other directions. Starting with the Bell Labs Interpretive System on the IBM-650 and probably peaking at Remote Time Sharing in 1970, methods of bootstrapping to establish a succession of higher capabilities to grow exponentially have been known. Imagine a time sharing system with a FORTRAN/ALGOL/BASIC all rolled into one memory-resident compiler, significance arithmetic, etc., servicing many of the high schools in Seattle (including Lakeside where Bill Gates and Paul Allen learned on it), all on the equivalent of a Commodore 64. Some of the customers complained about only having 8kB of Huffman-coded macro-instructions to hold their programs, until a chess playing program that ran in that 8K that never lost a game appeared in the library. Then came the microprocessors and all this has been forgotten. Microsoft sought to "do less with less" without ever realizing that the really BIG machine they learned on (and which they still have yet to equal) was only the equivalent of a Commodore 64. I wrote that compiler and chess game. No, the primary limitation is cultural. I have discussed here how to make processors that run 10,000 times faster, and how to build a scanning UV fluorescent microscope that diagrams brains. The SAME thing blocks both - culture. Intel is up against EXACTLY the same mind block that IBM was up against when for decades they couldn't move beyond Project Stretch, and there simply isn't any area of study into which a Scanning UV fluorescence microscope now cleanly falls, of course because without the microscope, such an area of study could not develop. Things are now quite stuck until either the culture changes (don't hold your breath), or the present generations of "experts" (including us) dies off. At present, I don't expect to see any AGIs in our lifetime, though I do believe that with support, one could be developed in 10-20 years. Not until someone gives the relevant sciences a new name, stops respecting present corporate and university structure (e.g. that PhDs have any but negative value), and injects ~$10^9 to start it can this happen. Of course, this requires independent rather than corporate or university money - some rich guy who sees the light. Until I meet this guy, I'm sticking to tractable projects like Dr. Eliza. > A human brain has about 10^9 bits of knowledge, of which probably 10^7 to > 10^8 bits are unique to each individual. That makes 10^17 to 10^18 bits that > have to be extracted from human brains and communicated to the AGI. This > could be done in code or formal language, although most of it will probably > be done in natural language once this capability is developed. It would be MUCH easier and cheaper to just scan it out with something like a scanning UV fluorescent microscope. > Since we don't know which p
Re: [agi] Re: AI isn't cheap
--- On Mon, 9/8/08, Steve Richfield <[EMAIL PROTECTED]> wrote: On 9/7/08, Matt Mahoney <[EMAIL PROTECTED]> wrote: >>The fact is that thousands of very intelligent people have been trying >>to solve AI for the last 50 years, and most of them shared your optimism. >Unfortunately, their positions as students and professors at various >universities have forced almost all of them into politically correct >paths, substantially all of which lead nowhere, for otherwise they would >have succeeded long ago. The few mavericks who aren't stuck in a >university (like those on this forum) all lack funding. Google is actively pursuing AI and has money to spend. If you have seen some of their talks, you know they are pursuing some basic and novel research. >>Perhaps it would be more fruitful to estimate the cost of automating the >>global economy. I explained my estimate of 10^25 bits of memory, 10^26 >>OPS, 10^17 bits of software and 10^15 dollars. You want to replicate the work currently done by 10^10 human brains. A brain has 10^15 synapses. A neuron axon has an information rate of 10 bits per second. As I said, you can argue about these numbers but it doesn't matter much. An order of magnitude error only changes the time to AGI by a few years at the current rate of Moore's Law. Software is not subject to Moore's Law so its cost will eventually dominate. A human brain has about 10^9 bits of knowledge, of which probably 10^7 to 10^8 bits are unique to each individual. That makes 10^17 to 10^18 bits that have to be extracted from human brains and communicated to the AGI. This could be done in code or formal language, although most of it will probably be done in natural language once this capability is developed. Since we don't know which parts of our knowledge is shared, the most practical approach is to dump all of it and let the AGI remove the redundancies. This will require a substantial fraction of each person's life time, so it has to be done in non obtrusive ways, such as recording all of your email and conversations (which, of course, all the major free services already do). The cost estimate of $10^15 comes by estimating the world GDP ($66 trillion per year in 2006, increasing 5% annually) from now until we have the hardware to support AGI. We have the option to have AGI sooner by paying more. Simple economics suggests we will pay up to what it is worth. -- Matt Mahoney, [EMAIL PROTECTED] --- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=111637683-c8fa51 Powered by Listbox: http://www.listbox.com
Re: [agi] Re: AI isn't cheap
Matt, On 9/7/08, Matt Mahoney <[EMAIL PROTECTED]> wrote: > > --- On Sun, 9/7/08, Steve Richfield <[EMAIL PROTECTED]> wrote: > > >1. I believe that there is some VERY fertile but untilled ground, which > >if it is half as good as it looks, could yield AGI a LOT cheaper than > >other higher estimates. Of course if I am wrong, I would probably accept > >your numbers. > > > >2. I believe that AGI will take VERY different (cheaper and more > >valuable) forms than do other members on this forum. > > > >Each of the above effects are worth several orders of magnitude in effort. > > You are just speculating. Of course. Aren't we all here on this forum? The fact is that thousands of very intelligent people have been trying to > solve AI for the last 50 years, and most of them shared your optimism. Unfortunately, their positions as students and professors at various universities have forced almost all of them into politically correct paths, substantially all of which lead nowhere, for otherwise they would have succeeded long ago. The few mavericks who aren't stuck in a university (like those on this forum) all lack funding. Perhaps it would be more fruitful to estimate the cost of automating the > global economy. I explained my estimate of 10^25 bits of memory, 10^26 OPS, > 10^17 bits of software and 10^15 dollars. I don't understand the goal or value here? Perhaps you could explain? >You really should see my Dr. Eliza demo. > > Perhaps you missed my comments in April. > > http://www.listbox.com/member/archive/303/2008/04/search/ZWxpemE/sort/time_rev/page/2/entry/5:53/20080414221142:407C652C-0A91-11DD-B3D2-6D4E66D9244B/ Apparently I did. Sorry about that. Here I have pasted in the posting with embedded contemporary comments. --- Steve Richfield <[EMAIL PROTECTED]> wrote: > Why go to all that work?! I have attached the *populated* Knowledge.mdb file > that contains the knowledge that powers the chronic illness demo of Dr. > Eliza. To easily view it, just make sure that any version of MS Access is > installed on your computer (it is in Access 97 format) and double-click on > the file. From there, select the Tables tab, and click on whatever table > interests you. I looked at your file. Would I be correct that if I described a random health problem to Dr. Eliza that it would suggest that my problem is due to one of: - Low body temperature - Fluorescent lights - Consuming fructose in the winter - Mercury poisoning from amalgam fillings and vaccines - Aluminum cookware - Hydrogenated vegetable oil - Working a night shift - Aspirin (causes macular degeneration) - Or failure to accept divine intervention? = First, my complements on your careful reading of the knowledge base. = Yes, there is a pretty good chance that you would be asked about some of these things, as various of these things seem to underlie most chronic illnesses. Is that it, or is there a complete medical database somewhere, = WYSIWYG, though this is only maybe 1% of a fully populated health database. This stuff is just there for demo. For a 1% demo, it works amazingly well. Further, I presume that people would embed generous hyperlinks into the explanations, so that Pub Med and other medical databases would be just a mouse click away. or the capability of acquiring this knowledge? = Only machine knowledge that has been carefully crafted by humans. As I have explained in a number of postings, certain key things, like how people commonly express symptoms and the carefully crafted questions needed to drill down, are NOT on any web site or medical text, so the services of an experienced expert is absolutely required. Plans of others to "mine the Internet" (or Wikipedia) are absolutely doomed to failure because this information is so completely lacking. No AGI would be able to compose this knowledge unless they had the real-world experience with real-world people to know how they express things. In short, many AI and AGI plans are quite obviously hopeless because they lack access to this information. Do you have a medical background, = Yes. or have you consulted with doctors in building the database? = Yes. BTW, regarding processes that use 100% of CPU in Windows. Did you try Ctrl-Alt-Del to bring up the task manager, then right click on the process and change its priority? = Not that specifically, though I did try the Windows API to do the same, and got back an error code that indicated that the most problematical task (NaturallySpeaking) had set a bit to keep other tasks from adjusting its priority. I presume that the Task Manager would have simply called the same API but probably failed to provide the return code. I expect to abandon speech I/O in the future even though it works pretty well, because no one seems to want to bet their success in overcoming their problems on the random screwups of a speech recognition program. Without speech I/O, there is no speed "problem". This is apparently one of those "great ideas" that just ca
