Re: [agi] Reading on automatic programming?
Richard, In a comment to a recent blog post of mine Evgenii asked for my comments on the following automatic programming task. I responded with an enumeration of the types of problem-solving knowledge and behaviors that a solution requires: [Evgenii] ... I’ll share my use case which my system used. My ultimate goal for the first stage was to create an automatic programmer who would be capable of parsing the following specification, and capable to emit a Java program that implements this specification. The specification was: Write a TCP echo server. (Client connects to the server, sends bytes to the server, and receives them back immediately, until client does disconnect on behalf of its own). Server must be able to serve more than one client simultaneously. The server should be implemented in Java. [Steve] ... I expect that your [Evgenii's] system should have an adequate commonsense understanding of the application domain, and relevant domain algorithms: TCP socket protocol [at the transport layer, e.g. how to listen, how to connect, how to read/write, close a socket] Client/Server paradigm multithreading, including thread safety what “echo” means It should have an understanding of the generally applicable programming algorithms that are relevant: object-oriented program classes and instances variable typing statement sequencing and control method definition and invocation program composition (e.g. initialization, process, finalization, and exception handling) test case generation programming safety (i.e. be Friendly and do not cause the host to crash) And it should understand the relevant Java concepts: how to compose a Java class how to map attributes of domain objects to Java instance variables how to compose Java statements how to use applicable class libraries how to compile and execute a Java program If the Texai dialog system were applied to your [Evgenii's] use case, then I expect a great deal of the effort would be to acquire the background algorithm knowledge from human teachers. Furthermore, I think that the specified program would be developed during a dialog with the teacher. It would only be after the system has adequately learned the subject, that it could program automatically from text specifications. -Steve Stephen L. Reed Artificial Intelligence Researcher http://texai.org/blog http://texai.org 3008 Oak Crest Ave. Austin, Texas, USA 78704 512.791.7860 - Original Message From: Richard Loosemore [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Wednesday, February 6, 2008 8:47:43 AM Subject: Re: [agi] Reading on automatic programming? Stephen Reed wrote: Eli, Same as Ben - Generative Programming, Methods, Tools, and Applications (2000) - Krzysztof Czarnecki, Ulrich W. Eisenecker I would chime in and say that this one also struck me as a very stimulating book, but its main benefit to me was to get me thinking about where the points of rigidity where, in their formalism. The places where the whole thing would break down unless someone did some hand-crafting or pre-digesting of ideas. My main concern was to get the same thing to happen without any such cheating, and I found the clarity of their writing helped me to see how to do that (or at least, how to do it better). Overall, I think that the automatic programming problem is pretty much isomorphic to the general intelligence problem: an AP system is going to demand such flexibility and (and as you put it, Steve) such depth of knowledge, that the system will have to pretty much the whole of intelligence before it can be trusted to do full automatic programming. But then, I'm hard to please: my criterion for success in AP is that it has to be good enough to discover new design patterns Richard Loosemore -Steve Stephen L. Reed Artificial Intelligence Researcher http://texai.org/blog http://texai.org 3008 Oak Crest Ave. Austin, Texas, USA 78704 512.791.7860 - Original Message From: Eliezer S. Yudkowsky [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Wednesday, February 6, 2008 12:25:22 AM Subject: Re: [agi] Reading on automatic programming? Stephen Reed wrote: Hi Evgenii, From my bookshelf: 1. Code Generation in Action (2003) - Jack Herrington 2. Computer Program Construction (1994) - Ali Mili, Jules Desharnais, Fatma Mili 3. Knowledge Based Program Construction (1979) - David R. Barstow 4. Studies in Automatic Programming Logic (1977) - Zohar Manna, Richard Waldinger 5. A Theory and Practice of Program
Re: [agi] Reading on automatic programming?
Stephen Reed wrote: Richard, In a comment to a recent blog post of mine http://texai.org/blog/2008/01/20/bootstrap-dialog-system-design Evgenii asked for my comments on the following automatic programming task. I responded with an enumeration of the types of problem-solving knowledge and behaviors that a solution requires: [Evgenii] ... I’ll share my use case which my system used. My ultimate goal for the first stage was to create an automatic programmer who would be capable of parsing the following specification, and capable to emit a Java program that implements this specification. The specification was: Write a TCP echo server. (Client connects to the server, sends bytes to the server, and receives them back immediately, until client does disconnect on behalf of its own). Server must be able to serve more than one client simultaneously. The server should be implemented in Java. [Steve] ... I expect that your [Evgenii's] system should have an adequate commonsense understanding of the application domain, and relevant domain algorithms: * TCP socket protocol [at the transport layer, e.g. how to listen, how to connect, how to read/write, close a socket] * Client/Server paradigm * multithreading, including thread safety * what “echo” means It should have an understanding of the generally applicable programming algorithms that are relevant: * object-oriented program classes and instances * variable typing * statement sequencing and control * method definition and invocation * program composition (e.g. initialization, process, finalization, and exception handling) * test case generation * programming safety (i.e. be Friendly and do not cause the host to crash) And it should understand the relevant Java concepts: * how to compose a Java class * how to map attributes of domain objects to Java instance variables * how to compose Java statements * how to use applicable class libraries * how to compile and execute a Java program If the Texai dialog system were applied to your [Evgenii's] use case, then I expect a great deal of the effort would be to acquire the background algorithm knowledge from human teachers. Furthermore, I think that the specified program would be developed during a dialog with the teacher. It would only be after the system has adequately learned the subject, that it could program automatically from text specifications. Interesting. It might be worth sharing my (brief) reaction to this. I see certain deep knowledge that is required to do this task (I will explain what that is in a moment), and I wonder when and how that kind of deep knowledge would get into the system, or whether it would be missed entirely... For example: if I were the system right now, I would have to do some work to figure out the almost-trivial question of where to put a server process on my machine, to test it, because writing client-server code is not my specialty. Now, assuming that the AP system were coming across a new type of programming task, in this example (the way that I am), would the AP system have to be explicitly told such background knowledge about how to handle client-server tasks, or would it have the ability to look up a textbook that would give the answer? I suspect it would not be able to use the textbook. Second: the software development environment used by Mac developers was recently rewritten (again) and one result of this is that all the books that describe how to get into Mac programming are out of date. So out of date, in fact, that it would take a very experienced developer to figure out how to relate the layout of the new environment to the instructions given in the books. Would the AP system be able to recognize a change like that, and cope with it? Again, I doubt it. Obviously, this is expecting quite a lot from an AP system at this point in time, but what worries me is that so much of *real* down-n-dirty computer programming is either (a) having to go into territory that you have never been in before, which requires research done under pressure, or (b) adjusting to massive structural changes, with no guidance, that an AP system might find itself falling over just because of miniature versions of the two problems I just described. It seems to me that most AP approaches are geared toward the plain sailing problems that do not have the nasty little issues described above, and that the rough seas type of problems are postponed for another day. Since an awful lot of good work can go into solving plain sailing systems that are very large (Czarnecki Eisenecker, for example), it can seem like good progress is being made. I think what then happens is that people do not realise that, in order to go on to the nasty type of programming problem (i.e. 99% of all programming problems), all the previous work may count for nothing,
Re: [agi] Reading on automatic programming?
Richard, I entirely agree with your comments. I would like to eventually stop programming in Java and have the system do that for me. I am strongly motivated to build its dialog component first because that addresses the issue of how to collaborate with the system when the rough seas are encountered. -Steve Stephen L. Reed Artificial Intelligence Researcher http://texai.org/blog http://texai.org 3008 Oak Crest Ave. Austin, Texas, USA 78704 512.791.7860 - Original Message From: Richard Loosemore [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Wednesday, February 6, 2008 10:16:34 AM Subject: Re: [agi] Reading on automatic programming? [snip] It seems to me that most AP approaches are geared toward the plain sailing problems that do not have the nasty little issues described above, and that the rough seas type of problems are postponed for another day. Since an awful lot of good work can go into solving plain sailing systems that are very large (CzarneckiEisenecker, for example), it can seem like good progress is being made. I think what then happens is that people do not realise that, in order to go on to the nasty type of programming problem (i.e. 99% of all programming problems), all the previous work may count for nothing, because it did not even in principle address such deep issues as what to do if the only way forward is to find the right textbook, read a couple of pages, and understand it. Of course, the same criticism can be levelled at many approaches to the entire AI problem, never mind just the AP problem. :-) Richard Loosemore - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?; Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244id_secret=94219347-aa2026
Re: [agi] Reading on automatic programming?
Stephen Reed wrote: Richard, I entirely agree with your comments. I would like to eventually stop programming in Java and have the system do that for me. I am strongly motivated to build its dialog component first because that addresses the issue of how to collaborate with the system when the rough seas are encountered. -Steve It looks like we are moving in similar directions, then. Half of my attention at the moment is on the problem of automatic programming, but I am coming at it from the direction of producing a software development environment that has as much intelligence as possible built into it. My line of attack is to look at the dialog between the environment (it is called SAFAIRE - pronounced sapphire) and the user, and to make the dialog as non-verbal as possible. Not because I hate language as such, but because I see language as a poor medium for the human to understand software systems, and a less-than-perfect way for the human to communicate their intentions to the system. Under the hood, SAFAIRE has the AGI design that I am working on. The work is thus nicely bootstrapped: the problems of learning how to program, and how to relate to the user, are a good test domain for the AGI component, and the emerging software development environment is a good (make that indispensible!) tool for developing the underlying AGI. One other vital component of the project, which I am having trouble with, is a [number of hours in the day] generator. 8-) Richard Loosemore - Original Message From: Richard Loosemore [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Wednesday, February 6, 2008 10:16:34 AM Subject: Re: [agi] Reading on automatic programming? [snip] It seems to me that most AP approaches are geared toward the plain sailing problems that do not have the nasty little issues described above, and that the rough seas type of problems are postponed for another day. Since an awful lot of good work can go into solving plain sailing systems that are very large (Czarnecki Eisenecker, for example), it can seem like good progress is being made. I think what then happens is that people do not realise that, in order to go on to the nasty type of programming problem (i.e. 99% of all programming problems), all the previous work may count for nothing, because it did not even in principle address such deep issues as what to do if the only way forward is to find the right textbook, read a couple of pages, and understand it. Of course, the same criticism can be levelled at many approaches to the entire AI problem, never mind just the AP problem. :-) Richard Loosemore - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244id_secret=94228222-d95c02
Re: [agi] Reading on automatic programming?
Richard, There has been striking improvement in the deductive power of modern programming IDE's (Integrated Development Environment). I too considered the approach you describe. I use NetBeans for Java development. It must keep at least one representation of the source code syntax tree to perform such operations as refactoring. It has clever templates for code completion and so forth. NetBeans, like its peers, has a plug-in architecture in which a developer-written component can obtain access to, and modify the internal program editor state. I have an intuition that I can describe in English what I want the system to automatically program, but I am not ruling out an alternate GUI dialog modality, especially to improve the cognitive bandwidth of this particular dialog situation. Does SAFAIRE support a conventional programming language? -Steve Stephen L. Reed Artificial Intelligence Researcher http://texai.org/blog http://texai.org 3008 Oak Crest Ave. Austin, Texas, USA 78704 512.791.7860 - Original Message From: Richard Loosemore [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Wednesday, February 6, 2008 11:30:55 AM Subject: Re: [agi] Reading on automatic programming? Stephen Reed wrote: Richard, I entirely agree with your comments. I would like to eventually stop programming in Java and have the system do that for me. I am strongly motivated to build its dialog component first because that addresses the issue of how to collaborate with the system when the rough seas are encountered. -Steve It looks like we are moving in similar directions, then. Half of my attention at the moment is on the problem of automatic programming, but I am coming at it from the direction of producing a software development environment that has as much intelligence as possible built into it. My line of attack is to look at the dialog between the environment (it is called SAFAIRE - pronounced sapphire) and the user, and to make the dialog as non-verbal as possible. Not because I hate language as such, but because I see language as a poor medium for the human to understand software systems, and a less-than-perfect way for the human to communicate their intentions to the system. Under the hood, SAFAIRE has the AGI design that I am working on. The work is thus nicely bootstrapped: the problems of learning how to program, and how to relate to the user, are a good test domain for the AGI component, and the emerging software development environment is a good (make that indispensible!) tool for developing the underlying AGI. One other vital component of the project, which I am having trouble with, is a [number of hours in the day] generator. 8-) Richard Loosemore - Original Message From: Richard Loosemore [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Wednesday, February 6, 2008 10:16:34 AM Subject: Re: [agi] Reading on automatic programming? [snip] It seems to me that most AP approaches are geared toward the plain sailing problems that do not have the nasty little issues described above, and that the rough seas type of problems are postponed for another day. Since an awful lot of good work can go into solving plain sailing systems that are very large (CzarneckiEisenecker, for example), it can seem like good progress is being made. I think what then happens is that people do not realise that, in order to go on to the nasty type of programming problem (i.e. 99% of all programming problems), all the previous work may count for nothing, because it did not even in principle address such deep issues as what to do if the only way forward is to find the right textbook, read a couple of pages, and understand it. Of course, the same criticism can be levelled at many approaches to the entire AI problem, never mind just the AP problem. :-) Richard Loosemore - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?; Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244id_secret=94235932-ddae41
Re: [agi] Reading on automatic programming?
Stephen Reed wrote: Richard, There has been striking improvement in the deductive power of modern programming IDE's (Integrated Development Environment). I too considered the approach you describe. I use NetBeans for Java development. It must keep at least one representation of the source code syntax tree to perform such operations as refactoring. It has clever templates for code completion and so forth. NetBeans, like its peers, has a plug-in architecture in which a developer-written component can obtain access to, and modify the internal program editor state. I have an intuition that I can describe in English what I want the system to automatically program, but I am not ruling out an alternate GUI dialog modality, especially to improve the cognitive bandwidth of this particular dialog situation. Does SAFAIRE support a conventional programming language? In principle, yes, it can: the most important layer is above the language level. However, it does need to learn the language before it can help you with it. Out of the box, it will be OS X native, so what it knows at the beginning will be Obj-C and OS X. (Unless someone pays to port it to some other platform). I can't say too much about it at this stage, but I am hoping that once you start using it, you won't be interested in the underlying language ever again... :-). I am projecting early 2010 as a release date, so it won't be available in a hurry. Richard Loosemore - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244id_secret=94239586-1c2a53
[agi] Reading on automatic programming?
Could anyone send some initial references/links to read on subject? -- Best regards, Evgenii Philippov - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244id_secret=94098247-e976ab
Re: [agi] Reading on automatic programming?
Hi Evgenii, From my bookshelf: 1. Code Generation in Action (2003) - Jack Herrington 2. Computer Program Construction (1994) - Ali Mili, Jules Desharnais, Fatma Mili 3. Knowledge Based Program Construction (1979) - David R. Barstow 4. Studies in Automatic Programming Logic (1977) - Zohar Manna, Richard Waldinger 5. A Theory and Practice of Program Development (1997) - Derek Andrews 6. Program Construction, Calculating Implementations from Specifications (2003) - Roland Backhouse 7. Refinement Calculus, A Systematic Introduction (1998) - Ralph-Johan Back, Joakim von Wright 8. Generative Programming, Methods, Tools, and Applications (2000) - Krzysztof Czarnecki, Ulrich W. Eisenecker 9. Annual Review in Automatic Programming (...1973...) - Pergamon Press I've left off books on Aspect-Oriented Programming which I think bears on this issue. Ever since I began programming I have been thinking about what it would take for computers to program themselves. My current belief is that deep background knowledge in the application domain must be combined with deep knowledge of algorithms to compose computer programs using a deductive planner. -Steve Stephen L. Reed Artificial Intelligence Researcher http://texai.org/blog http://texai.org 3008 Oak Crest Ave. Austin, Texas, USA 78704 512.791.7860 - Original Message From: Evgenii Philippov [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Tuesday, February 5, 2008 11:07:54 PM Subject: [agi] Reading on automatic programming? Could anyone send some initial references/links to read on subject? -- Best regards, Evgenii Philippov This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?; Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244id_secret=94101431-e375ac
Re: [agi] Reading on automatic programming?
8. Generative Programming, Methods, Tools, and Applications (2000) - Krzysztof Czarnecki, Ulrich W. Eisenecker The above is a very good book, IMO ... not directly AGI-related, but damn insightful re generative software design... - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244id_secret=94114734-b6bbb4
