On Jan 28, 2008 1:15 AM, Matt Mahoney <[EMAIL PROTECTED]> wrote: > --- Vladimir Nesov <[EMAIL PROTECTED]> wrote: > > > On Jan 27, 2008 5:32 AM, Matt Mahoney <[EMAIL PROTECTED]> wrote: > > > > > > Software correctness is undecidable -- the halting problem reduces to it. > > > Computer security isn't going to be magically solved by AGI. The problem > > will > > > actually get worse, because complex systems are harder to get right. > > > > > > > Computer security can be solved by more robust rights management and > > by avoiding bugs that lead to security vulnerabilities. AGI can help > > with both. > > Security tools are double edged swords. The knowledge required to protect > against attacks is the same as the knowledge required to launch attacks. AGI > just continues the arms race. We will have smarter intrusion detection > systems and smarter intruders. If you look at number of attacks per year, it > is clear we are going in the wrong direction.
You don't NEED intrusion detection if intrusion cannot be done. If your software doesn't read anything from outside, it's not possible to attack it. If it reads that data and correctly does nothing with it, it's not possible to attack it. If it reads that data and correctly processes it, it's not possible to attack it. It's not currently practically feasible to write usual software without bugs, but it's theoretically possible (more on that below). So this race is not symmetrical: you can't attack perfect software even if you are an omniscent oracle. > > Software correctness IS decidable: you just don't write general > > algorithms, you write algorithms that satisfy your requirements. > > Fundamental problem with software correctness is that you can forget > > about many requirements or get requirements wrong. Practical problem > > with software correctness is that it's very costly to actually check > > correctness, and it gets worse as requirements and software in > > question get more complex. These problems can be dealt with if we have > > fast (=cheap) and competent general intelligence. > > Consider the following subset of possible requirements: the program is correct > if and only if it halts. > It's a perfectly valid requirement, and I can write all sorts of software that satisfies it. I can't take a piece of software that I didn't write and tell you it it satisfies it, but I can write piece of software that satisfies it, that also does all sorts of useful stuff. -- Vladimir Nesov mailto:[EMAIL PROTECTED] ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=90388774-03a036
