Richard, You will be happy to note that I have copied the text of your reply to my "Valuable Clippings From AGI Mailing List file". Below are some comments.
>RICHARD LOOSEMORE=====> I now understand that you have indeed heard of complex systems before, but I must insist that in your summary above you have summarized what they are in such a way that completely contradicts what they are! A complex system such as the economy can and does have stable modes in which it appears to be stable. This does not constradict the complexity at all. A system is not complex because it is unstable. ED PORTER=====> Richard, I was citing a relatively stable economies as exactly what you say they are, an example of a complex system that is relatively stable. So why is it that my summary " summarized what they are in such a way that completely contradicts what they are!"? I implied that economies have traditionally had instabilities, such as boom and bust cycles, and I am aware that even with all our controls, other major instabilities could strike, in much the same ways that people can have nervous breakdowns. ED PORTER=====> With regard to the rest of your paper I find it one of your better reasoned discussions of the problem of complexity. I like Ben, agree it is a potential problem. I said that in the email you were responding to. My intuition, like Ben's, tells me we probably be able to deal with it, but your paper is correct to point out that such intuitions are really largely guesses. >RICHARD LOOSEMORE=====>how can someone know that how much impact the complexity is going to have, when in the same breath they will admit that NOBODY currently understands just how much of an impact the complexity has. the best that anyone can do is point to other systems in which there is a small amount of complexity and say: "Well, these folks managed to understand their systems without getting worried about complexity, so why don't we assume that our problem is no worse than theirs?" For example, someone could point to the dynamics of planetary systems and say that there is a small bit of complexity there, but it is a relatively small effect in the grand scheme of things. ED PORTER=====> A better example would be the world economy. Its got 6 billion highly autonomous players. It has all sorts of non-linearities and complex connections. Although it has fits and starts is has surprising stability considering everything that is thrown at it (Not clear how far this stability will hold into the singularity future) but still it is an instructive example of how extremely complex things, with lots of non-linearities, can work relatively well if there are the proper motivations and controls. >RICHARD LOOSEMORE=====>Problem with that line of argument is that there are NO other examples of an engineering system with as much naked funkiness in the interactions between the low level components. ED PORTER=====> The key is try to avoid and/or control funkiness in your components. Remember that an experiential system derives most of its behavior by re-enacting, largely through substitutions and probabilistic-transition-based synthesis, from past experience, with a bias toward past experiences that have "worked" in some sense meaningful to the machine. These creates a tremendous bias toward desirable, vs. funky, behaviors. So, net, net, Richard, re-reading your paper and reading your below long post have increased my respect for your arguments. I am somewhat more afraid of complexity gotchas than I was two days ago. But I still am pretty confident (without anything beginning to approach proof) such gotchas will not prevent use from making useful human level AGI within the decade if AGI got major funding. But I have been afraid for a long time that even the other type of complexity (i.e., complication, which often involves some risk of "complexity") means that it may be very difficult for us humans to keep control of superhuman-level AGI's for very long, so I have always worried about that sort of complexity Gotcha. But with regard to the complexity problem, it seems to me that we should design systems with an eye to reducing their knarlyness, including planning multiple types of control systems, and then once we get initial such system up and running try to find out what sort of complexity problems we have. Ed Porter -----Original Message----- From: Richard Loosemore [mailto:[EMAIL PROTECTED] Sent: Thursday, December 06, 2007 11:46 AM To: agi@v2.listbox.com Subject: Re: [agi] None of you seem to be able ... Ed Porter wrote: > Jean-Paul, > > Although complexity is one of the areas associated with AI where I have less > knowledge than many on the list, I was aware of the general distinction you > are making. > > What I was pointing out in my email to Richard Loosemore what that the > definitions in his paper "Complex Systems, Artificial Intelligence and > Theoretical Psychology," for "irreducible computability" and "global-local > interconnect" themselves are not totally clear about this distinction, and > as a result, when Richard says that those two issues are an unavoidable part > of AGI design that must be much more deeply understood before AGI can > advance, by the more loose definitions which would cover the types of > complexity involved in large matrix calculations and the design of a massive > supercomputer, of course those issues would arise in AGI design, but its no > big deal because we have a long history of dealing with them. > > But in my email to Richard I said I was assuming he was not using this more > loose definitions of these words, because if he were, they would not present > the unexpected difficulties of the type he has been predicting. I said I > though he was dealing with more the potentially unruly type of complexity, I > assume you were talking about. > > I am aware of that type of complexity being a potential problem, but I have > designed my system to hopefully control it. A modern-day well functioning > economy is complex (people at the Santa Fe Institute often cite economies as > examples of complex systems), but it is often amazingly unchaotic > considering how loosely it is organized and how many individual entities it > has in it, and how many transitions it is constantly undergoing. Unsually, > unless something bangs on it hard (such as having the price of a major > commodity all of a sudden triple), it has a fair amount of stability, while > constantly creating new winners and losers (which is a productive form of > mini-chaos). Of course in the absence of regulation it is naturally prone > to boom and bust cycles. Ed, I now understand that you have indeed heard of complex systems before, but I must insist that in your summary above you have summarized what they are in such a way that completely contradicts what they are! A complex system such as the economy can and does have stable modes in which it appears to be stable. This does not constradict the complexity at all. A system is not complex because it is unstable. I am struggling here, Ed. I want to go on to explain exactly what I mean (and what complex systems theorists mean) but I cannot see a way to do it without writing half a book this afternoon. Okay, let me try this. Imagine that we got a bunch of computers and connected them with a network that allowed each one to talk to (say) the ten nearest machines. Imagine that each one is running a very simple program: it keeps a handful of local parameters (U, V, W, X, Y) and it updates the values of its own parameters according to what the neighboring machines are doing with their parameters. How does it do the updating? Well, imagine some really messy and bizarre algorithm that involves looking at the neighbors' values, then using them to cross reference each other, and introduce delays and gradients and stuff. On the face of it, you might think that the result will be that the U V W X Y values just show a random sequence of fluctuations. Well, we know two things about such a system. 1) Experience tells us that even though some systems like that are just random mush, there are some (a noticeably large number in fact) that have overall behavior that shows 'regularities'. For example, much to our surprise we might see waves in the U values. And every time two waves hit each other, a vortex is created for exactly 20 minutes, then it stops. I am making this up, but that is the kind of thing that could happen. 2) The algorithm is so messy that we cannot do any math to analyse and predict the behavior of the system. All we can do is say that we have absolutely no techniques that will allow us to mathematical progress on the problem today, and we do not know if at ANY time in future history there will be a mathematics that will cope with this system. What this means is that the waves and vortices we observed cannot be "explained" in the normal way. We see them happening, but we do not know why they do. The bizzare algorithm is the "low level mechanism" and the waves and vortices are the "high level behavior", and when I say there is a "Global-Local Disconnect" in this system, all I mean is that we are completely stuck when it comes to explaining the high level in terms of the low level. Believe me, it is childishly easy to write down equations/algorithms for a system like this that are so profoundly intractable that no mathematician would even think of touching them. You have to trust me on this. Call your local Math department at Harvard or somewhere, and check with them if you like. As soon as the equations involve funky little dependencies such as "Pick two neighbors at random, then pick two parameters at random from each of these, and for the next day try to make one of my parameters (chosen at random, again) follow the average of those two as they were exactly 20 minutes ago, EXCEPT when neighbors 5 and 7 both show the same value of the V parameter, in which case drop this algorithm for the rest of the day and instead follow the substitute algorithm B...." Now, this set of computers would be a wicked example of a complex system, even while the biggest supercomputer in the world, following a nice, well behaved algorithm, would not be complex at all. The summary of this is as follows: there are some systems in which the interaction of the components are such that we must effectively declare that NO THEORY exists that would enable us to predict certain global regularities observed in these systems. In the real world, systems are mixtures of complex and not-complex, of course, so don't think that I am saying that cognitive systems are completely complex: I do not say that. But when a system involves a *significant* amount of complexity, what do we do? More to the point, what do we do when we have good reasons for suspecting that some CRUCIAL aspects of a system are going to introduce complexity, but we cannot be sure how much of an impact that complexity is going to have on the system? THIS IS THE MOST IMPORTANT THING ABOUT MY ARGUMENT (which is why I am putting it so loudly ;-)). We cannot know exactly how much of an impact complexity will have, because we have no way to measure the 'amount" of complexity, nor any way to say how much impact we get from a given amount of complexity! So we are in the dark. Do we suspect that complexity is involved in intelligence. I could present lots of reasoning here, but instead I will resort to quoting your favorite AGI researcher. Ben Goertzel, in a message just a short while ago, said "There is no doubt that complexity, in the sense typically used in dynamical-systems-theory, presents a major issue for AGI systems" Can I take it as understood that this is accepted, and move on? So, yes, there is evidence that complexity is involved. My argument is that when you examine the way that complexity has an effect on systems, you find that it can have very quiet, subtle effects that do not jump right out at you and say "HERE I AM!", but they just lurk in the background and make it quietly impossible for you to get the system up above a certain level of functioning. To be more specific: when you really allow the symbol-building mechanisms, and the learning mechanisms, and the inference-control mechanisms to do their thing in a full scale system, the effects of tiny bits of complexity in the underlying design CAN have a huge impact. One particular design choice, for example, could mean the difference between a system that looks like it ought to work, but when you set it running autonomously it gradually drifts into imbecility without there being any clear reason. Now I want to qualify that last paragraph in a very important way: I cannot say anything as strong as "complexity WILL have bad effects", I can only say that "complexity has the potential to have these effects". This means that we have a situation in which there is a DANGER that complexity will stop us from being able to diagnose why our systems are not working, but we cannot quantify or analyse that danger: it is a great big unknown. But the fact that it is such an unknown has one big consequence: if someone says that "I have a gut instinct that complexity will not turn out to be a problem" then that statement is based on thing but guesswork. If the intution were based on something that the person knows, then that means the person "knows" something that is enabling them to predict the behavior of a complex system, or understand the amount of complexity involved. That would be bizarre: how can someone know that how much impact the complexity is going to have, when in the same breath they will admit that NOBODY currently understands just how much of an impact the complexity has. the best that anyone can do is point to other systems in which there is a small amount of complexity and say: "Well, these folks managed to understand their systems without getting worried about complexity, so why don't we assume that our problem is no worse than theirs?" For example, someone could point to the dynamics of planetary systems and say that there is a small bit of complexity there, but it is a relatively small effect in the grand scheme of things. Problem with that line of argument is that there are NO other examples of an engineering system with as much naked funkiness in the interactions between the low level components. It is as simple as that. Planets simply don't cut it: the funky interactions there are ridiculously small compared with what we know exists in intelligence. Try to think of some other example where we have tried to build a system that behaves in a certain overall way, but we started out by using components that interacted in a completely funky way, and we succeeded in getting the thing working in the way we set out to. In all the history of engineering there has never been such a thing. Conclusion: there is a danger that the complexity that even Ben agrees must be present in AGI systems will have a significant impact on our efforts to build them. But the only response to this danger at the moment is the bare statement made by people like Ben that "I do not think that the danger is significant". No reason given, no explicit attack on any component of the argument I have given, only a statement of intuition, even though I have argued that intuition cannot in principle be a trustworthy guide here. I see this as a head-in-the-sand response. There, I wasted too much time on this again. The only virtue of writing such a long post is that no one will read all of it, so there won't be many replies and I can therefore get back to real work. Richard Loosemore > So the system would need regulation. > > Most of my system operates on a message passing system with little concern > for synchronization, it does not require low latencies, most of its units, > operate under fairly similar code. But hopefully when you get it all > working together it will be fairly dynamic, but that dynamism with be under > multiple controls. > > I think we are going to have to get such systems up and running to find you > just how hard or easy they will be to control, which I acknowledged in my > email to Richard. I think that once we do we will be in a much better > position to think about what is needed to control them. I believe such > control will be one of the major intellectual challenges to getting AGI to > function at a human-level. This issue is not only preventing runaway > conditions, it is optimizing the intelligence of the inferencing, which I > think will be even more import and diffiducle. (There are all sorts of > damping mechanisms and selective biasing mechanism that should be able to > prevent many types of chaotic behaviors.) But I am quite confident with > multiple teams working on it, these control problems could be largely > overcome in several years, with the systems themselves doing most of the > learning. > > Even a little OpenCog AGI on a PC, could be interesting first indication of > the extent to which complexity will present control problems. As I said if > you had 3G of ram for representation, that should allow about 50 million > atoms. Over time you would probably end up with at least hundreds of > thousand of complex patterns, and it would be interesting to see how easy it > would be to properly control them, and get them to work together as a > properly functioning thought economy in what ever small interactive world > they developed their self-organizing pattern base. Of course on such a PC > based system you would only, on average, be able to do about 10million > pattern to pattern activations a second, so you would be talking about a > fairly trivial system, but with say 100K patterns, it would be a good first > indication of how easy or hard agi systems will be to control. > > Ed Porter ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?&; ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=73239242-ec48de
<<attachment: winmail.dat>>
