Re: [agi] NLP + reasoning?
Linas, I don't believe attaching NLP functionality on top of perception engine can help much. AI that is capable of general learning should be able to also learn language processing, from the letters up. So it's just another capability that should be accounted for. NLP can only help with feeding the engine with texts prior to it being capable to understand the language, which looks upside-down (although, anything goes, so if it helps, fine). On 10/31/07, Linas Vepstas <[EMAIL PROTECTED]> wrote: > > Hi, > > Aside from Novamente and CYC, who else has attempted to staple > NLP to a reasoning engine? I just pasted a good NLP parser > I found on the net, onto a home-brew, cut-rate reimplementation > of the CYC reasoning engine. I've got simple things working > (answers "what is X?" questions, and remembers "X is Y" assertions) > and have a new appreciation for what it takes to go to the next > level. > > Soo .. who else has done this, and are there review papers for > what people typically find? (I figure a stunt similar to mine > has been done a few dozen times before ... ) > > --linas > > - > This list is sponsored by AGIRI: http://www.agiri.org/email > To unsubscribe or change your options, please go to: > http://v2.listbox.com/member/?&; > -- Vladimir Nesovmailto:[EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=59636655-0fb206
Re: [agi] NLP + reasoning?
Vladimir: AI that is capable of general learning should be able to also learn language processing, from the letters up. Sounds wonderful. Anyone attempting that or even close? - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=59650366-66e9f9
Re: [agi] NLP + reasoning?
--- Linas Vepstas <[EMAIL PROTECTED]> wrote: > Aside from Novamente and CYC, who else has attempted to staple > NLP to a reasoning engine? Many have tried, such as BASEBALL in 1961 [1] and SHRDLU in 1968-70 [2]. But you might as well try stapling jello. Natural language evolved on a computer far more powerful than the one in front of you now. In spite of superficial similarities to artificial languages that use context free grammars, its structure is fundamentally different. A natural language model requires the parallel evaluation of hundreds of thousands of weak constraints learned from years of experience. Natural language is a fundamental part of the knowledge base, not something you can add on later. References 1. Green, Bert F. Jr., Alice K. Wolf, Carol Chomsky, and Kenneth Laughery (1961), Baseball: An Automatic Question Answerer, Proceedings of the Western Joint Computer Conference, 19:219-224. 2. http://hci.stanford.edu/~winograd/shrdlu/ -- Matt Mahoney, [EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=59720772-e8ccde
Re: [agi] NLP + reasoning?
On 11/1/07, Mike Tintner <[EMAIL PROTECTED]> wrote: > > Vladimir: AI that is capable of general learning should be > > able to also learn language processing, from the letters up. > > Sounds wonderful. Anyone attempting that or even close? In my G0 architecture, NLP is based on logical deduction and abduction, so strictly speaking there isn't a separate NLP module. If anything, it's just a body of linguistic knowledge. I hope this type of "seamless integration of NLP with inference" appeals to you =) I also plan to implement learning in a general setting (not confined to language learning), but right now I don't have any algorithms, just the idea to do so. For language learning, I think in the end it will be augmented by human direct-input of rules, taking into consideration that the manually entered rules may be modified by machine learning to make them better. YKY - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60214029-039d48
Re: [agi] NLP + reasoning?
On Fri, Nov 02, 2007 at 12:06:05PM -0400, Jiri Jelinek wrote: > On Oct 31, 2007 8:53 PM, Matt Mahoney <[EMAIL PROTECTED]> wrote: > > Natural language is a fundamental part of the knowledge > base, not something you can add on later. > > I disagree. You can start with a KB that contains concepts retrieved > from a well structured non-NL input format only, get the thinking > algorithms working and then (possibly much later) let the system to > focus on NL analysis/understanding or build some > NL-to-the_structured_format translation tools. Bing. Yes, exactly. I'm taking an experimental, rather than theoretical approach: build the minimum amount needed to make something work, then determine what is the next roadblock, and fix that. And then iterate. At this point, for me, sentence-parsing is not a roadblock. Nor is the conversion of parser output to "narsese" (reasoning-engine internal langauge). --linas - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60498503-12ec1f
Re: [agi] NLP + reasoning?
On Oct 31, 2007 8:53 PM, Matt Mahoney <[EMAIL PROTECTED]> wrote: > Natural language is a fundamental part of the knowledge base, not something you can add on later. I disagree. You can start with a KB that contains concepts retrieved from a well structured non-NL input format only, get the thinking algorithms working and then (possibly much later) let the system to focus on NL analysis/understanding or build some NL-to-the_structured_format translation tools. Regards, Jiri Jelinek - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60477290-62b2ac
Re: [agi] NLP + reasoning?
--- Jiri Jelinek <[EMAIL PROTECTED]> wrote: > On Oct 31, 2007 8:53 PM, Matt Mahoney <[EMAIL PROTECTED]> wrote: > > Natural language is a fundamental part of the knowledge > base, not something you can add on later. > > I disagree. You can start with a KB that contains concepts retrieved > from a well structured non-NL input format only, get the thinking > algorithms working and then (possibly much later) let the system to > focus on NL analysis/understanding or build some > NL-to-the_structured_format translation tools. Well, good luck with that. Are you aware of how many thousands of times this approach has been tried? You are wading into a swamp. Progress will be rapid at first. -- Matt Mahoney, [EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60598500-af1c68
Re: [agi] NLP + reasoning?
On Fri, Nov 02, 2007 at 12:56:14PM -0700, Matt Mahoney wrote: > --- Jiri Jelinek <[EMAIL PROTECTED]> wrote: > > On Oct 31, 2007 8:53 PM, Matt Mahoney <[EMAIL PROTECTED]> wrote: > > > Natural language is a fundamental part of the knowledge > > base, not something you can add on later. > > > > I disagree. You can start with a KB that contains concepts retrieved > > from a well structured non-NL input format only, get the thinking > > algorithms working and then (possibly much later) let the system to > > focus on NL analysis/understanding or build some > > NL-to-the_structured_format translation tools. > > Well, good luck with that. Are you aware of how many thousands of times this > approach has been tried? You are wading into a swamp. Progress will be rapid > at first. Yes, and in the first email I wrote, that started this thread, I stated, more or less: "yes, I am aware that many have tried, and that its a swamp, and can anyone elucidate why?" And, so far, no one as been able to answer that question, even as they firmly assert that surely it is a swamp. Nor has anyone attempted to posit any mechanisms that avoid that swamp, other than thought bubbles that state things like "starting from a clean slate, my system will be magic". --linas - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60602053-e9d933
Re: [agi] NLP + reasoning?
On 11/2/07, Linas Vepstas <[EMAIL PROTECTED]> wrote: > On Fri, Nov 02, 2007 at 12:56:14PM -0700, Matt Mahoney wrote: > > --- Jiri Jelinek <[EMAIL PROTECTED]> wrote: > > > On Oct 31, 2007 8:53 PM, Matt Mahoney <[EMAIL PROTECTED]> wrote: > > > > Natural language is a fundamental part of the knowledge > > > base, not something you can add on later. > > > > > > I disagree. You can start with a KB that contains concepts retrieved > > > from a well structured non-NL input format only, get the thinking > > > algorithms working and then (possibly much later) let the system to > > > focus on NL analysis/understanding or build some > > > NL-to-the_structured_format translation tools. > > > > Well, good luck with that. Are you aware of how many thousands of times > > this > > approach has been tried? You are wading into a swamp. Progress will be > > rapid > > at first. > > Yes, and in the first email I wrote, that started this thread, I stated, > more or less: "yes, I am aware that many have tried, and that its a > swamp, and can anyone elucidate why?" And, so far, no one as been able > to answer that question, even as they firmly assert that surely it is a > swamp. Nor has anyone attempted to posit any mechanisms that avoid that > swamp, other than thought bubbles that state things like "starting from > a clean slate, my system will be magic". Matt's "because progress will be rapid at first" seems good enough for me. It's not a swamp _in principle_, but it is a swamp in practice, because most of the projects get started without any idea on how to get it there, and instead concentrate on creating shallow appearance of operation. Task of creating this appearance is in itself unlimited in work-year capacity, so they never move past it. I personally don't see how this appearance-building is going to help, so the question for me is not 'why can't it succeed?', but 'why do it at all?'. -- Vladimir Nesovmailto:[EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60620623-9769bd
Re: [agi] NLP + reasoning?
--- Linas Vepstas <[EMAIL PROTECTED]> wrote: > On Fri, Nov 02, 2007 at 12:56:14PM -0700, Matt Mahoney wrote: > > --- Jiri Jelinek <[EMAIL PROTECTED]> wrote: > > > On Oct 31, 2007 8:53 PM, Matt Mahoney <[EMAIL PROTECTED]> wrote: > > > > Natural language is a fundamental part of the knowledge > > > base, not something you can add on later. > > > > > > I disagree. You can start with a KB that contains concepts retrieved > > > from a well structured non-NL input format only, get the thinking > > > algorithms working and then (possibly much later) let the system to > > > focus on NL analysis/understanding or build some > > > NL-to-the_structured_format translation tools. > > > > Well, good luck with that. Are you aware of how many thousands of times > this > > approach has been tried? You are wading into a swamp. Progress will be > rapid > > at first. > > Yes, and in the first email I wrote, that started this thread, I stated, > more or less: "yes, I am aware that many have tried, and that its a > swamp, and can anyone elucidate why?" And, so far, no one as been able > to answer that question, even as they firmly assert that surely it is a > swamp. Nor has anyone attempted to posit any mechanisms that avoid that > swamp, other than thought bubbles that state things like "starting from > a clean slate, my system will be magic". Actually my research is trying to answer this question. In 1999 I looked at language model size vs. compression and found data consistent with Turing's and Landauer's estimates of 10^9 bits. This is also about the compressed size of the Cyc database. http://cs.fit.edu/~mmahoney/dissertation/ But then I started looking at CPU and memory requirements, which turn out to be much larger. Why does the human brain need 10^15 synapses? When you plot text compression ratio on the speed-memory surface, it is still very steep, especially on the memory axis. http://cs.fit.edu/~mmahoney/compression/text.html Unfortunately the data is still far from clear. The best programs still model semantics crudely and grammar not at all. From the data I would guess that an ungrounded language model could run on a 1000 CPU cluster, plus or minus a couple orders of magnitude. The fact that Google hasn't solved the problem with a 10^6 cluster does not make me hopeful. -- Matt Mahoney, [EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60626587-43dede
Re: [agi] NLP + reasoning?
On Sat, Nov 03, 2007 at 12:15:29AM +0300, Vladimir Nesov wrote: > I personally don't see how this appearance-building is going to help, > so the question for me is not 'why can't it succeed?', but 'why do it > at all?'. Because absolutely no one has proposed anything better? --linas - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60629120-5f23a9
Re: [agi] NLP + reasoning?
On 11/3/07, Linas Vepstas <[EMAIL PROTECTED]> wrote: > On Sat, Nov 03, 2007 at 12:15:29AM +0300, Vladimir Nesov wrote: > > I personally don't see how this appearance-building is going to help, > > so the question for me is not 'why can't it succeed?', but 'why do it > > at all?'. > > Because absolutely no one has proposed anything better? It's not enough: there should be a reason why this road is at least any good. -- Vladimir Nesovmailto:[EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60634277-328c8b
Re: [agi] NLP + reasoning?
On 11/3/07, Vladimir Nesov <[EMAIL PROTECTED]> wrote: > On 11/3/07, Linas Vepstas <[EMAIL PROTECTED]> wrote: > > Because absolutely no one has proposed anything better? > > It's not enough: there should be a reason why this road is at least any good. Linas: Actually I think your work along this path is valuable. You may join Ben's OpenCog project which will be launched soon (I heard) and also has an NLP module. I'd be interested in employing one of these *"ad hoc"* NLP modules in my AGI. Although it is possible to "fully integrate" NL into AGI, such an endeavor may not be the highest priority at this moment. It can give the AGI better linguistic abilities, such as understanding human-made texts or speeches, even poetry, but I think there're higher priorities than this (eg, learning how to do math, how to program, etc). YKY - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60637682-9f3616
Re: [agi] NLP + reasoning?
Linas, > > Yes, and in the first email I wrote, that started this thread, I stated, > more or less: "yes, I am aware that many have tried, and that its a > swamp, and can anyone elucidate why?" And, so far, no one as been able > to answer that question, even as they firmly assert that surely it is a > swamp. Nor has anyone attempted to posit any mechanisms that avoid that > swamp, other than thought bubbles that state things like "starting from > a clean slate, my system will be magic". I think the main problems you'll find with this kind of system are as follows. 1) The difficulties with the NLP part.. What's hard about getting an NLP parser tuned well enough to output a whole bunch of complex logical relationships into a knowledge base, based on interpreting English sentences? -- Reference resolution -- Semantic disambiguation, esp. of words besides nouns and verbs -- Preposition disambiguation State of the art NLP tech handles these things only in a pretty limited way. 2) The difficulties with the reasoning part.. What's hard about tuning a logical reasoning engine to carry out effective reasoning based on a large body of relationships extracted from natural language? -- Propagating uncertainty usefully thru logical reasoning steps (I think we've solved this once with PLN, though) -- Inference control! ... choosing what inferences to do, which is only a problem when you have a very large KB -- Contextual understanding. Most propositions parsed out of text will have validity only in a certain context, but the context is left implicit and needs to be inferred. ... The issue I see is that tasks like -- inference control -- contextual interpretation of propositions -- preposition disambiguation -- nominal reference resolution are not just technical problems, they're problems that may well be "AGI-hard problems" in themselves, in that it may be they can only be solved by a software program that somehow embodies a fairly thorough understanding of the world to which the NLP-derived propositions pertain. The deceptiveness of the NLP+logic approach to AGI is that these big issues are made to seem like small ones, because they get placed on long lists of issues alongside other issues that aren't so profound... I think that many people who have started down the path you're on, have realized this fact, and have wound up spending their research careers working on one of the AGI-hard subsidiary problems I've mentioned, or other similar ones ;-) As for how to avoid these problems, I have already stated my approach: couple the sort of stuff you're doing with embodiment, in physical or virtual worlds. That is how humans get around these problems. Exactly how embodied experience helps with these problems is of course a long story. In the case of humans, you can consult a huge body of developmental psychology literature. In the case of AGI systems, it depends on your AGI approach; we have thought this through pretty carefully in the context of Novamente... -- Ben G - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60639519-aeb81a
RE: [agi] NLP + reasoning?
Ben, A very good post. It is a more valuably-detailed statement of what a guy who worked at Thinking Machines told me in the late eighties -- that it had become pretty obvious you cant have good NL understanding unless you can reason from world knowledge. Another argument for breaking the small machine mindset. Ed Porter -Original Message- From: Benjamin Goertzel [mailto:[EMAIL PROTECTED] Sent: Friday, November 02, 2007 6:56 PM To: agi@v2.listbox.com Subject: Re: [agi] NLP + reasoning? Linas, Yes, and in the first email I wrote, that started this thread, I stated, more or less: "yes, I am aware that many have tried, and that its a swamp, and can anyone elucidate why?" And, so far, no one as been able to answer that question, even as they firmly assert that surely it is a swamp. Nor has anyone attempted to posit any mechanisms that avoid that swamp, other than thought bubbles that state things like "starting from a clean slate, my system will be magic". I think the main problems you'll find with this kind of system are as follows. 1) The difficulties with the NLP part.. What's hard about getting an NLP parser tuned well enough to output a whole bunch of complex logical relationships into a knowledge base, based on interpreting English sentences? -- Reference resolution -- Semantic disambiguation, esp. of words besides nouns and verbs -- Preposition disambiguation State of the art NLP tech handles these things only in a pretty limited way. 2) The difficulties with the reasoning part.. What's hard about tuning a logical reasoning engine to carry out effective reasoning based on a large body of relationships extracted from natural language? -- Propagating uncertainty usefully thru logical reasoning steps (I think we've solved this once with PLN, though) -- Inference control! ... choosing what inferences to do, which is only a problem when you have a very large KB -- Contextual understanding. Most propositions parsed out of text will have validity only in a certain context, but the context is left implicit and needs to be inferred. .. The issue I see is that tasks like -- inference control -- contextual interpretation of propositions -- preposition disambiguation -- nominal reference resolution are not just technical problems, they're problems that may well be "AGI-hard problems" in themselves, in that it may be they can only be solved by a software program that somehow embodies a fairly thorough understanding of the world to which the NLP-derived propositions pertain. The deceptiveness of the NLP+logic approach to AGI is that these big issues are made to seem like small ones, because they get placed on long lists of issues alongside other issues that aren't so profound... I think that many people who have started down the path you're on, have realized this fact, and have wound up spending their research careers working on one of the AGI-hard subsidiary problems I've mentioned, or other similar ones ;-) As for how to avoid these problems, I have already stated my approach: couple the sort of stuff you're doing with embodiment, in physical or virtual worlds. That is how humans get around these problems. Exactly how embodied experience helps with these problems is of course a long story. In the case of humans, you can consult a huge body of developmental psychology literature. In the case of AGI systems, it depends on your AGI approach; we have thought this through pretty carefully in the context of Novamente... -- Ben G _ This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/? <http://v2.listbox.com/member/?&; > & - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60648543-fe6ef9
Re: [agi] NLP + reasoning?
--- "YKY (Yan King Yin)" <[EMAIL PROTECTED]> wrote: > Although it is possible to "fully integrate" NL into AGI, such an endeavor > may not be the highest priority at this moment. It can give the AGI better > linguistic abilities, such as understanding human-made texts or speeches, > even poetry, but I think there're higher priorities than this (eg, learning > how to do math, how to program, etc). Computers are already pretty good at math. But I think before they can write or debug programs, they will need natural language so you can tell them what to write. Otherwise, all you have is a compiler. At least human programmers learn to speak before they can write code. -- Matt Mahoney, [EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60654358-796643
Re: [agi] NLP + reasoning?
On Nov 2, 2007 8:24 PM, Matt Mahoney <[EMAIL PROTECTED]> wrote: > --- "YKY (Yan King Yin)" <[EMAIL PROTECTED]> wrote: > > Although it is possible to "fully integrate" NL into AGI, such an > endeavor > > may not be the highest priority at this moment. It can give the AGI > better > > linguistic abilities, such as understanding human-made texts or > speeches, > > even poetry, but I think there're higher priorities than this (eg, > learning > > how to do math, how to program, etc). > > Computers are already pretty good at math. That's not really accurate, Matt! Computer theorem-provers are way worse than any human mathematician, except at very special types of theorems. Computers are good at high school math, w/ the exception of word problems. Arguably, computers are currently better at NLP than at theorem-proving, which is the essence of modern mathematics... -- Ben - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60654670-8c0bc0
Re: [agi] NLP + reasoning?
--- Benjamin Goertzel <[EMAIL PROTECTED]> wrote: > Computer theorem-provers are way worse than any human mathematician, > except at very special types of theorems. True, there has not been much progress since Gelernter used internal graphical models to heuristically trim the search space to prove theorems in plane geometry in 1959 [1]. 1. Gelernter, H., Realization of a Geometry-Theorem Proving Machine, Proceedings of an International Conference on Information Processing, Paris: UNESCO House, pp. 273-282, 1959. -- Matt Mahoney, [EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60657406-2dfc54
Re: [agi] NLP + reasoning?
On 10/31/07, Matt Mahoney <[EMAIL PROTECTED]> wrote: > --- Linas Vepstas <[EMAIL PROTECTED]> wrote: > > Aside from Novamente and CYC, who else has attempted to staple > > NLP to a reasoning engine? > > Many have tried, such as BASEBALL in 1961 [1] and SHRDLU in 1968-70 [2]. But > you might as well try stapling jello. Depends what you want to do. If you want to make a kick-ass data retrieval system, it might work out pretty well. > Natural language is a fundamental part of the knowledge > base, not something you can add on later. It was added on later in humans, in evolution; and probably is in ontogeny, too. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60657627-a79b48
Re: [agi] NLP + reasoning?
On Fri, Nov 02, 2007 at 05:24:36PM -0700, Matt Mahoney wrote: > --- "YKY (Yan King Yin)" <[EMAIL PROTECTED]> wrote: > > Although it is possible to "fully integrate" NL into AGI, such an endeavor > > may not be the highest priority at this moment. It can give the AGI better > > linguistic abilities, such as understanding human-made texts or speeches, > > even poetry, but I think there're higher priorities than this (eg, learning > > how to do math, how to program, etc). > > Computers are already pretty good at math. I think YKY might have meant "the kind of math that mathematictians do". > But I think before they can write > or debug programs, they will need natural language so you can tell them what > to write. Otherwise, all you have is a compiler. At least human programmers > learn to speak before they can write code. Yes. I want to be able to talk to the thing. I ave, for example, used a program that does group theory (branch of math), and it is very very hard to use because its .. obtuse. I can't talk to it the way that I'd talk to anothre human. --linas - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60655266-bb250d
Re: [agi] NLP + reasoning?
On Nov 2, 2007 8:52 PM, Matt Mahoney <[EMAIL PROTECTED]> wrote: > --- Benjamin Goertzel <[EMAIL PROTECTED]> wrote: > > Computer theorem-provers are way worse than any human mathematician, > > except at very special types of theorems. > > True, there has not been much progress since Gelernter used internal > graphical > models to heuristically trim the search space to prove theorems in plane > geometry in 1959 [1]. > > 1. Gelernter, H., Realization of a Geometry-Theorem Proving Machine, > Proceedings of an International Conference on Information Processing, > Paris: > UNESCO House, pp. 273-282, 1959. There has been progress in "narrow-AI theorem proving", e.g. -- theorem-provers to prove correctness of circuits -- theorem-provers to prove existence/uniqueness theorems for various sorts of differential equations etc. etc. But not at general-purpose theorem-proving... -- Ben G - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60658017-3ccb3d
Re: [agi] NLP + reasoning?
> > > Out of curiosity, how would you rank the "IQ" of your current code? We have not yet tried to build a system like the one you describe, because we've considered it to probably not be on the short path to AGI However, we will likely be building something like this during the next 6 months, due to a customer requesting us to. We'll see how it goes ;-) -- Ben G > > After a few weekends of hacking, and about 3KLoc of code, I can get > the following squeezed out of the opencyc kb: > > cyc-net> who is lincoln > Abe Lincoln is a famous person. He is a thing that exists in time. He is > a male person, who is a famous person. Abe Lincoln is a man. He is an > ethnic Caucasian. He is an United States president, who is a historical > figure. Abe Lincoln is an One of the presidents of the United States. He > is a dead organism. > > cyc-net> what is copper? > I know of several things named "copper". They are: > There's copper, the copper, which is an orangeish thing. > There's copper, copper, which is a mineral. > > cyc-net> who is betsy ross > I've never heard of betsy ross. Tell me about him. > > cyc-net> who is macbeth > I've never heard of macbeth. Tell me about him. > > cyc-net> who is lucy liu > Lucy Liu is a female person. She is a television actor. > > These are the result of very very direct reasoning, very low cpu usage > (under 2 seconds, except for Lincoln, which had to weed out 20 things > named "Lincoln County") and yet, its vaguely comparable to something > that a 6-7-8-9-year-old might produce. > > Where is the developmental jump? At the pre-teen level? Your question isn't very well-defined. We have no programs that can pass the "5 year old child Turing Test", at least recent experiments in China suggest this I bet your program couldn't pass it either Question-answering based on an NLP database is not the same as conversation, though... Making a program that can dig info out of a DB in the manner of a 6 year old is one thing, making a program that can converse in the manner of a 6 year old is another thing entirely... >From a commercial perspective, an NLP query engine may be valuable, though, even if it's not done in a way that is readily generalizable into an AGI. -- Ben G - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60661338-2d76a8
Re: [agi] NLP + reasoning?
On Fri, Nov 02, 2007 at 06:56:18PM -0400, Benjamin Goertzel wrote: > > I think the main problems you'll find with this kind of system are as > follows. > > The difficulties with the NLP part.. > > -- Reference resolution > -- Semantic disambiguation, esp. of words besides nouns and verbs > -- Preposition disambiguation > > The difficulties with the reasoning part.. > > > -- Propagating uncertainty usefully thru logical reasoning steps (I think > we've solved this once with PLN, though) > -- Inference control! ... choosing what inferences to do, which is only > a problem when you have a very large KB > -- Contextual understanding. Most propositions parsed out of text will > have validity only in a certain context, but the context is left implicit > and > needs to be inferred. OK, no argment there. I've aleady started bumping into some of these, in a limited way. > I think that many people who have started down the path you're on, have > realized this fact, and have wound up spending their research careers > working on one of the AGI-hard subsidiary problems I've mentioned, > or other similar ones ;-) And so I suspected. But I am hoping that, perhaps, with things like agisim, opencog, one can start putting together a modular, engineered platform where many of these difficulaties are walled off into blocks of code where the algorithms can be toyed with, cut-pasted-replaced. Maybe. Out of curiosity, how would you rank the "IQ" of your current code? After a few weekends of hacking, and about 3KLoc of code, I can get the following squeezed out of the opencyc kb: cyc-net> who is lincoln Abe Lincoln is a famous person. He is a thing that exists in time. He is a male person, who is a famous person. Abe Lincoln is a man. He is an ethnic Caucasian. He is an United States president, who is a historical figure. Abe Lincoln is an One of the presidents of the United States. He is a dead organism. cyc-net> what is copper? I know of several things named "copper". They are: There's copper, the copper, which is an orangeish thing. There's copper, copper, which is a mineral. cyc-net> who is betsy ross I've never heard of betsy ross. Tell me about him. cyc-net> who is macbeth I've never heard of macbeth. Tell me about him. cyc-net> who is lucy liu Lucy Liu is a female person. She is a television actor. These are the result of very very direct reasoning, very low cpu usage (under 2 seconds, except for Lincoln, which had to weed out 20 things named "Lincoln County") and yet, its vaguely comparable to something that a 6-7-8-9-year-old might produce. Where is the developmental jump? At the pre-teen level? --linas - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60658634-0a8135
Re: [agi] NLP + reasoning?
On 11/3/07, Linas Vepstas <[EMAIL PROTECTED]> wrote: > > Computers are already pretty good at math. > > I think YKY might have meant "the kind of math that mathematictians do". Partly that, but more "how to interpret everyday experiences as math / physics", which belongs to common sense reasoning. With this ability, we can tell the AGI to do XYZ and it will automatically deploy other existing technologies such as calculators, theorem provers, search engines, etc. YKY - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60673293-cfebc5
Re: [agi] NLP + reasoning?
--- Philip Goetz <[EMAIL PROTECTED]> wrote: > > Natural language is a fundamental part of the knowledge > > base, not something you can add on later. > > It was added on later in humans, in evolution; and probably is in ontogeny, > too. If the knowledge is how to walk or catch a ball, then it is true you don't need language. -- Matt Mahoney, [EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60673447-66148c
Re: [agi] NLP + reasoning?
On Friday 02 November 2007 21:12, Matt Mahoney wrote: > If the knowledge is how to walk or catch a ball, then it is true you > don't need language. However, in neural systems, there seems to be some "internal representation" going on that is seen in neural coding, which may not be language as we know it, but still a language that keeps the tissue doing this "Intelligence" thing. - Bryan - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60673891-1b49e2
Re: [agi] NLP + reasoning?
On Nov 2, 2007 3:56 PM, Matt Mahoney <[EMAIL PROTECTED]> wrote: > > --- Jiri Jelinek <[EMAIL PROTECTED]> wrote: > > > On Oct 31, 2007 8:53 PM, Matt Mahoney <[EMAIL PROTECTED]> wrote: > > > Natural language is a fundamental part of the knowledge > > base, not something you can add on later. > > > > I disagree. You can start with a KB that contains concepts retrieved > > from a well structured non-NL input format only, get the thinking > > algorithms working and then (possibly much later) let the system to > > focus on NL analysis/understanding or build some > > NL-to-the_structured_format translation tools. > > Well, good luck with that. If you can't get meaning from clean input format then what makes you think you can handle NL? When working on an AGI proof of concept, NL just adds unnecessary complexity. > Are you aware of how many thousands of times this > approach has been tried? Are you aware in how many ways you can go wrong with: a) the KB design b) general problem solving algorithms c) specifics of the input format/framework (including context supporting mechanisms etc..) ? There is nothing wrong with a non-NL input format when working on the a proof of concept AGI. You just need to get all the important stuff right, which is tricky. When kids learn NL, they get lots of extra data through senses - which provides certain structure and plays critical role in concept understanding. Practically, NL-only input (which some wannabe AGIs are playing with) is not good enough. You somehow need to substitute at least some of the extra info. It's doable through well designed non-NL input format (which not only avoids sticky NL syntax issues but also significantly supports semantic analysis). Of course you can also go with NL and simulated human senses ;-)... But good luck with that. Regards, Jiri Jelinek - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60691123-6925d3
Re: [agi] NLP + reasoning?
On 02/11/2007, Linas Vepstas <[EMAIL PROTECTED]> wrote: > On Fri, Nov 02, 2007 at 12:56:14PM -0700, Matt Mahoney wrote: > > --- Jiri Jelinek <[EMAIL PROTECTED]> wrote: > > > On Oct 31, 2007 8:53 PM, Matt Mahoney <[EMAIL PROTECTED]> wrote: > > > > Natural language is a fundamental part of the knowledge > > > base, not something you can add on later. > > > > > > I disagree. You can start with a KB that contains concepts retrieved > > > from a well structured non-NL input format only, get the thinking > > > algorithms working and then (possibly much later) let the system to > > > focus on NL analysis/understanding or build some > > > NL-to-the_structured_format translation tools. > > > > Well, good luck with that. Are you aware of how many thousands of times > > this > > approach has been tried? You are wading into a swamp. Progress will be > > rapid > > at first. > > Yes, and in the first email I wrote, that started this thread, I stated, > more or less: "yes, I am aware that many have tried, and that its a > swamp, and can anyone elucidate why?" And, so far, no one as been able > to answer that question, even as they firmly assert that surely it is a > swamp. Nor has anyone attempted to posit any mechanisms that avoid that > swamp, other than thought bubbles that state things like "starting from > a clean slate, my system will be magic". > Here is my take on why I think it is a swamp. I hypothesize natural language has the same expressiveness as a recursive enumerable langauges [1]. Which means you need a machine from the space of Turing machines to recognise all possible strings. Further on from this, natural language also evolves in time, which means you need to move through the space of Turing machines in order to find programs that correctly parse it. Moving through the space of Turing machines is fundementally experimental (you can move through subspaces of it such as deciders[2] with proofs, but that limits you to not being able to recognise some strings). Experimenting in the space of Turing Machines can lead to deleterious programs to the system being created. So creating a system that has a stable(ish) goal whilst experimenting is a necessary precursor to trying to solve the NL problem. All these statements assume memory bounded versions of these things, and are tentative until I can find theories that cope with this. Will Pearson [1] http://en.wikipedia.org/wiki/Recursively_enumerable_language [2] http://en.wikipedia.org/wiki/Decider - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60712161-687415
Re: [agi] NLP + reasoning?
On 11/3/07, Linas Vepstas <[EMAIL PROTECTED]> wrote: > These are the result of very very direct reasoning, very low cpu usage > (under 2 seconds, except for Lincoln, which had to weed out 20 things > named "Lincoln County") and yet, its vaguely comparable to something > that a 6-7-8-9-year-old might produce. > > Where is the developmental jump? At the pre-teen level? I think this is the perfect answer to your question about why natural language is the wrong place to start. This isn't intended as personal criticism, but: look at what you just said. You've started talking about IQ and implying a program is vaguely comparable in intelligence to a 9 year old human... Based on a program that Google outperforms by several orders of magnitude. The problem with natural language is that the bandwidth is so tiny, it necessarily relies primarily on the reader's imagination. We are explicitly programmed, in other words, to assume intelligence on the part of any entity that talks to us in semi-coherent English, and to fill in all the gaps ourselves. There was intelligence at work in the exchanges you quoted, yes, but the intelligence was in your brain, not in the computer. Before natural language is worth doing, you need to have a program that does some nontrivial computation in the first place. My suggestion is visual/spatial modeling of some form (such as the virtual worlds stuff Novamente is doing), but _something_. Otherwise you're just setting a trap for yourself. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60726582-557b6e
Re: [agi] NLP + reasoning?
--- Jiri Jelinek <[EMAIL PROTECTED]> wrote: > If you can't get meaning from clean input format then what makes you > think you can handle NL? Humans seem to get meaning more easily from ambiguous statements than from mathematical formula. Otherwise you are programming, not teaching. > When working on an AGI proof of concept, NL just adds unnecessary > complexity. Do you think you could pass a Turing test without natural language? This is a minimal test for AI, without the added complexity of vision, robotics, speech, etc. How do you propose to measure intelligence in a proof of concept? -- Matt Mahoney, [EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60962252-ec59ba
Re: [agi] NLP + reasoning?
Jiri, IMO, proceeding with AGI development using formal-language input rather than NL input is **not** necessarily a bad approach. However, one downside is that your incremental steps toward AGI, in this approach, will not be very convincing to skeptics. Another downside is that in this approach it's easier to fool yourself that you're really making progress when you're not... For me to be convinced a formal-language-based AI system was a real step along the path to AGI, I'd have to know something about the internals of the system and assess it in that context. Formal-language-based interaction certainly lends itself to "cheating" (i.e. brittle approaches that won't be able to deal with the nasty richness and noisiness of real language or the real world), but this doesn't mean every formal-language-interaction-based system will **necessarily** be based on "cheating"... Inevitably, the interpretation of partial results is always based on some theoretical framework So assessments of incremental progress will never be very objective. For instance, whether or not creating model rockets zooming 200 feet in the air counts as evidence that one is on the path to space-bound rocketry, depends on one's theoretical framework... -- Ben G On 11/4/07, Matt Mahoney <[EMAIL PROTECTED]> wrote: > --- Jiri Jelinek <[EMAIL PROTECTED]> wrote: > > If you can't get meaning from clean input format then what makes you > > think you can handle NL? > > Humans seem to get meaning more easily from ambiguous statements than from > mathematical formula. Otherwise you are programming, not teaching. > > > When working on an AGI proof of concept, NL just adds unnecessary > > complexity. > > Do you think you could pass a Turing test without natural language? This is a > minimal test for AI, without the added complexity of vision, robotics, speech, > etc. How do you propose to measure intelligence in a proof of concept? > > > -- Matt Mahoney, [EMAIL PROTECTED] > > - > This list is sponsored by AGIRI: http://www.agiri.org/email > To unsubscribe or change your options, please go to: > http://v2.listbox.com/member/?&; > - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60963713-fc5517
RE: [agi] NLP + reasoning?
> From: Benjamin Goertzel [mailto:[EMAIL PROTECTED] > > IMO, proceeding with AGI development using formal-language input > rather than NL input is **not** necessarily a bad approach. > > However, one downside is that your incremental steps toward AGI, in > this approach, will not be very convincing to skeptics. > > Another downside is that in this approach it's easier to fool yourself > that you're really making progress when you're not... > > For me to be convinced a formal-language-based AI system was a real > step along the path to AGI, I'd have to know something about the > internals of the system and assess it in that context. > > Formal-language-based interaction certainly lends itself to "cheating" > (i.e. brittle approaches that won't be able to deal with the nasty > richness and noisiness of real language or the real world), but this > doesn't mean every formal-language-interaction-based system will > **necessarily** be based on "cheating"... > > Inevitably, the interpretation of partial results is always based on > some theoretical framework So assessments of incremental progress > will never be very objective. > > For instance, whether or not creating model rockets zooming 200 feet > in the air counts as evidence that one is on the path to space-bound > rocketry, depends on one's theoretical framework... > How does one go about checking during the development process whether or not a formal-language input based AGI is working or not? It has to interface with the real world in some way at some point as there will be the noisy stuff either in NL or something else. BUT saving that for later has its advantages as you can get bogged down dealing with the noise. Building up the AGI in a perfect world as best as possible may take it farther. Really though NL input is filtered at some layer... or is it... do you take the NL right down into the core? Guess it depends on the design. John - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=61253668-899ebf
Re: [agi] NLP + reasoning?
On Sat, Nov 03, 2007 at 03:18:20PM +, Russell Wallace wrote: > On 11/3/07, Linas Vepstas <[EMAIL PROTECTED]> wrote: > > These are the result of very very direct reasoning, very low cpu usage > > (under 2 seconds, except for Lincoln, which had to weed out 20 things > > named "Lincoln County") and yet, its vaguely comparable to something > > that a 6-7-8-9-year-old might produce. > > > > Where is the developmental jump? At the pre-teen level? > > I think this is the perfect answer to your question about why natural > language is the wrong place to start. > > This isn't intended as personal criticism, but: look at what you just > said. You've started talking about IQ and implying a program is > vaguely comparable in intelligence to a 9 year old human... > > Based on a program that Google outperforms by several orders of magnitude. > > The problem with natural language is that the bandwidth is so tiny, it > necessarily relies primarily on the reader's imagination. We are > explicitly programmed, in other words, to assume intelligence on the > part of any entity that talks to us in semi-coherent English, and to > fill in all the gaps ourselves. There was intelligence at work in the > exchanges you quoted, yes, but the intelligence was in your brain, not > in the computer. Touche! --linas - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=61294072-1a9482
Re: [agi] NLP + reasoning?
On Sat, Nov 03, 2007 at 03:45:30AM -0400, Jiri Jelinek wrote: > Are you aware in how many ways you can go wrong with: One problem I see with this mailing list is an almost intentional desire to mis-interpret. I never claimed I was building an AGI, or a problem solver, or a learning machine, or any of a dozen other things for which there were replies. I asked a very simple question about conversational state. My goal was to build something that was one step beyond alicebot, by simply maintaining conversational state, and drawing upon a KB to deal with various "common sense" assertions as they show up. So criticisms along the lines of "that won't be AGI" are rather pointless. --linas - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=61295227-dac431
Re: [agi] NLP + reasoning?
Hi, On Sat, Nov 03, 2007 at 01:41:30AM -0400, Philip Goetz wrote: > Why don't you describe what you've done in more detail, e.g., what > parser you're using, and how you hooked it up to Cyc? I "randomly" selected the link grammer parser http://www.link.cs.cmu.edu/link/ for the parser, although there are 3 or 4 other free parsers on the net. I am currently using it in its most basic, most fundamental mode -- at this stage, it wouldn't matter which parser one picked. The NL input is converted into parse trees, which are then used to bind unbound variables, for example: "(VP tell (NP me) (PP about (NP ?THING" So when you say "tell me about Lincoln", ?THING becomes Lincoln. The knowlegebase is searched for statements of the form "Lincoln is-a X", and the list of possible values for X is converted to a short essay. The value of ?THING is kept in memory as the "topic of conversation", so that if the next sentence is "tell me more about that", it knows that "that==Lincoln", and says more about Lincoln. I am not using either CYC, or opencyc, but rather, just the opencyc knowledgebase. Dumped to an ascii file, its about 750MB in size. I wrote a mini-micro-sized query engine that can read in this file, and then return results for a handful of queries, such as "(CYC-QUERY (#isa #$Lincoln ?WHAT))". The above is a sum-total of about 5KLOC, which is (very) small in the grand scheme of software development. I am hacking around with this thing, in various directions: adding support for more sentence types, being more sophisticated with references such as "it is", "that which", "the previous"... I'm also trying to get it to be inquisitive about the speaker (so that it can "learn" about the speaker, and "remember" things about them.) I'm also trying to validate assertions to see if they are common sense, or not, so that "Abraham Lincoln is a rock" would be rejected as nonsense, or at least, questioned. There are many, many interesting things that can be tried, and they all require "just a little bit more time and effort". --linas - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=61301397-dfa120
Re: [agi] NLP + reasoning?
--- Linas Vepstas <[EMAIL PROTECTED]> wrote: > I "randomly" selected the link grammer parser > http://www.link.cs.cmu.edu/link/ for the parser, It still has a few bugs. (S (NP I) (VP ate pizza (PP with (NP pepperoni))) .) (S (NP I) (VP ate pizza (PP with (NP Bob))) .) (S (NP I) (VP ate pizza (PP with (NP a fork))) .) -- Matt Mahoney, [EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=61305407-70727f
Re: [agi] NLP + reasoning?
On Mon, Nov 05, 2007 at 11:11:41AM -0800, Matt Mahoney wrote: > --- Linas Vepstas <[EMAIL PROTECTED]> wrote: > > I "randomly" selected the link grammer parser > > http://www.link.cs.cmu.edu/link/ for the parser, > > It still has a few bugs. > > (S (NP I) >(VP ate pizza >(PP with >(NP pepperoni))) >.) > > (S (NP I) >(VP ate pizza >(PP with >(NP Bob))) >.) > > (S (NP I) >(VP ate pizza >(PP with >(NP a fork))) >.) ? Its a parser, not a common-sense reasoning machine. 1) check the box "all linkages" 2) notice that there can be several alternative linkages. 3) notice that some of the parsed sentences had both unknown words, and null linkages -- i.e. it was unable to parse, but provided a few of its best guesses. If you don't know what "pepperoni" is, it's worthwhile to explore the possibility that "pepperoni" is an eating utensil, or that "pepperoni" is a person's name. --linas - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=61383278-853d9b
Re: [agi] NLP + reasoning?
On Mon, Nov 05, 2007 at 03:17:13PM -0600, Linas Vepstas wrote: > On Mon, Nov 05, 2007 at 11:11:41AM -0800, Matt Mahoney wrote: > > --- Linas Vepstas <[EMAIL PROTECTED]> wrote: > > > I "randomly" selected the link grammer parser > > > http://www.link.cs.cmu.edu/link/ for the parser, > > > > It still has a few bugs. > > > > (S (NP I) > >(VP ate pizza > >(PP with > >(NP pepperoni))) > >.) > > > > (S (NP I) > >(VP ate pizza > >(PP with > >(NP Bob))) > >.) > > > > (S (NP I) > >(VP ate pizza > >(PP with > >(NP a fork))) > >.) > > ? Its a parser, not a common-sense reasoning machine. > 1) check the box "all linkages" > 2) notice that there can be several alternative linkages. > 3) notice that some of the parsed sentences had both >unknown words, and null linkages -- i.e. it was unable >to parse, but provided a few of its best guesses. > > If you don't know what "pepperoni" is, it's worthwhile > to explore the possibility that "pepperoni" is an eating > utensil, or that "pepperoni" is a person's name. Also, that praser is aimed at "written" not "chat" english. So, I found out that "I ate a pizza with ..." worked a lot better; the missing article "a" confused it. Do you have any recommendations for other parsers? --linas - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=61394713-573ffe
Re: [agi] NLP + reasoning?
I think that Matt just wanted to promote his holistic statistical approach to NLP ;-) On 11/5/07, Linas Vepstas <[EMAIL PROTECTED]> wrote: > On Mon, Nov 05, 2007 at 03:17:13PM -0600, Linas Vepstas wrote: > > On Mon, Nov 05, 2007 at 11:11:41AM -0800, Matt Mahoney wrote: > > Also, that praser is aimed at "written" not "chat" english. > So, I found out that "I ate a pizza with ..." worked a lot > better; the missing article "a" confused it. > > Do you have any recommendations for other parsers? - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=61430100-3056c3
Re: [agi] NLP + reasoning?
On Nov 4, 2007 12:40 PM, Matt Mahoney <[EMAIL PROTECTED]> wrote: > --- Jiri Jelinek <[EMAIL PROTECTED]> wrote: > > If you can't get meaning from clean input format then what makes you > > think you can handle NL? > > Humans seem to get meaning more easily from ambiguous statements than from > mathematical formula. Different platform - different solutions. >Otherwise you are programming, not teaching. Switching to less ambiguous format doesn't prevent teaching. > > When working on an AGI proof of concept, NL just adds unnecessary > > complexity. > > Do you think you could pass a Turing test without natural language? No, but it's a positive test only. Failing it tells nothing. > This is a minimal test for AI, without the added complexity of vision, > robotics, speech, etc. False >How do you propose to measure intelligence in a proof of concept? Hmmm, let me check my schedule... Ok, I'll figure this out on Thursday night (unless I get hit by a lottery bus). "Jelinek test" is coming ;-)) ... I'll get back to you then.. Regards, Jiri Jelinek - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=61463681-5f2863
Re: [agi] NLP + reasoning?
Matt Mahoney wrote: --- Linas Vepstas <[EMAIL PROTECTED]> wrote: ... It still has a few bugs. ... (S (NP I) (VP ate pizza (PP with (NP Bob))) .) My name is Hannibal Lector. ... -- Matt Mahoney, [EMAIL PROTECTED] (Hannibal Lector was a movie cannibal) - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=61497373-34aef0
Re: [agi] NLP + reasoning?
On 05/11/2007, Linas Vepstas <[EMAIL PROTECTED]> wrote: > On Sat, Nov 03, 2007 at 03:45:30AM -0400, Jiri Jelinek wrote: > > Are you aware in how many ways you can go wrong with: > > One problem I see with this mailing list is an almost intentional > desire to mis-interpret. I never claimed I was building an AGI, > or a problem solver, or a learning machine, or any of a dozen > other things for which there were replies. > > I asked a very simple question about conversational state. > My goal was to build something that was one step beyond > alicebot, by simply maintaining conversational state, and > drawing upon a KB to deal with various "common sense" > assertions as they show up. So criticisms along the lines of > "that won't be AGI" are rather pointless. > It is amazing what some people think is going to be AGI capable Also you are posting on an AGI mailing list, so narrow AI discussion is slightly off-topic. Not to say it shouldn't be discussed, but flagging it heavily as such is probably a good idea. Talking about the age equivalence or IQ of your system is also not a good idea, if you want to give people the right impression that you are not going for AGI. I'm also wondering what you consider success in this case. For example do you want the system to be able to maintain conversational state such as would be needed to deal with the following. "For all following sentences take the first letter of each word and make English sentences out of it, reply in a similar fashion. How is the hair? Every rainy evening calms all nightingales. Yesterday, ornery ungulates stampeded past every agitated koala. Fine red eyebrows, new Chilean hoovers?" Will Pearson - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=61580236-1fc225
Re: [agi] NLP + reasoning?
Research Associate: CITANDA Post-Graduate Section Head Department of Information Systems Phone: (+27)-(0)21-6504256 Fax: (+27)-(0)21-6502280 Office: Leslie Commerce 4.21 >>> Linas Vepstas [EMAIL PROTECTED]> 2007/11/05 23:41 >> ( mailto:[EMAIL >>> PROTECTED]> ) >Do you have any recommendations for other parsers? One of the reasons I like Python: It's got NLTK and MontyLingua ( file:///C:/AC/Software/MontyLingua/MontyLingua.html#MontyLingua ): - MontyTokenizer - normalizes punctuation, spacing and contractions, with sensitivity to abbrevs. - MontyTagger - Part-of-speech tagging using PENN TREEBANK tagset - enriched with "Common Sense" from the Open Mind Common Sense project - MontyREChunker - chunks tagged text into verb, noun, and adjective chunks (VX,NX, and AX respectively) - MontyExtractor - extracts verb-argument structures, phrases, and other semantically valuable information from sentences and returns sentences as "digests" - MontyLemmatiser - part-of-speech sensitive lemmatisation - strips plurals (geese-->goose) and tense (were-->be, had-->have) - includes regexps from Humphreys and Carroll's morph.lex, and UPENN's XTAG corpus - MontyNLGenerator - generates summaries - generates surface form sentences - determines and numbers NPs and tenses verbs - accounts for sentence_type Note: It also has chatterbot code. This e-mail is subject to the UCT ICT policies and e-mail disclaimer published on our website at http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable from +27 21 650 4500. This e-mail is intended only for the person(s) to whom it is addressed. If the e-mail has reached you in error, please notify the author. If you are not the intended recipient of the e-mail you may not use, disclose, copy, redirect or print the content. If this e-mail is not related to the business of UCT it is sent by the sender in the sender's individual capacity. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=61595620-360867
Re: [agi] NLP + reasoning?
Will Pearson asked >> I'm also wondering what you consider success in this case. For example >> do you want the system to be able to maintain conversational state >> such as would be needed to deal with the following. >>"For all following sentences take the first letter of each word and >>make English sentences out of it, reply in a similar fashion. How is >>the hair? Every rainy evening calms all nightingales. Yesterday, >>ornery ungulates stampeded past every agitated koala. Fine red >>eyebrows, new Chilean hoovers?" The majority of human judges in a Turing Test would respond to such utterances with a blanket "What's are you talking about?" or "Are you crazy?" or "I thought we were going to have a conversation?" A certain amount of meta questioning is to be expected like... What is the third word in this sentence? But in order to pass Turing, you just have to convince the judges that you are human not necessarilly as skilled in word play as Lewis Carroll. -- Original message -- From: "William Pearson" <[EMAIL PROTECTED]> > On 05/11/2007, Linas Vepstas wrote: > > On Sat, Nov 03, 2007 at 03:45:30AM -0400, Jiri Jelinek wrote: > > > Are you aware in how many ways you can go wrong with: > > > > One problem I see with this mailing list is an almost intentional > > desire to mis-interpret. I never claimed I was building an AGI, > > or a problem solver, or a learning machine, or any of a dozen > > other things for which there were replies. > > > > I asked a very simple question about conversational state. > > My goal was to build something that was one step beyond > > alicebot, by simply maintaining conversational state, and > > drawing upon a KB to deal with various "common sense" > > assertions as they show up. So criticisms along the lines of > > "that won't be AGI" are rather pointless. > > > > It is amazing what some people think is going to be AGI capable > > Also you are posting on an AGI mailing list, so narrow AI discussion > is slightly off-topic. Not to say it shouldn't be discussed, but > flagging it heavily as such is probably a good idea. Talking about the > age equivalence or IQ of your system is also not a good idea, if you > want to give people the right impression that you are not going for > AGI. > > I'm also wondering what you consider success in this case. For example > do you want the system to be able to maintain conversational state > such as would be needed to deal with the following. > > "For all following sentences take the first letter of each word and > make English sentences out of it, reply in a similar fashion. How is > the hair? Every rainy evening calms all nightingales. Yesterday, > ornery ungulates stampeded past every agitated koala. Fine red > eyebrows, new Chilean hoovers?" > > Will Pearson > > - > This list is sponsored by AGIRI: http://www.agiri.org/email > To unsubscribe or change your options, please go to: > http://v2.listbox.com/member/?&; - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=61633156-468034
Re: [agi] NLP + reasoning?
Where I think parser fall down is in recognizing common English typing and spelling errors. "Hello how are you" would be recognizable by a parser but the following constructs all recognizable by a human would only be recognizable to a fuzzy pattern matcher. "Helohowareyou" "Hello hwo r u" "Hell, ho areyu" Examining the Transcripts in pas years Turing competitions it is very easy to see that all of the entries are very intolerant to fuzzy data and would respond with a obvious bluff when presented with such inputs. -- Original message -- From: "Jean-paul Van Belle" <[EMAIL PROTECTED]> Research Associate: CITANDA Post-Graduate Section Head Department of Information Systems Phone: (+27)-(0)21-6504256 Fax: (+27)-(0)21-6502280 Office: Leslie Commerce 4.21 >>> Linas Vepstas [EMAIL PROTECTED]> 2007/11/05 23:41 >> >Do you have any recommendations for other parsers? One of the reasons I like Python: It's got NLTK and MontyLingua: - MontyTokenizer - normalizes punctuation, spacing and contractions, with sensitivity to abbrevs. - MontyTagger - Part-of-speech tagging using PENN TREEBANK tagset - enriched with "Common Sense" from the Open Mind Common Sense project - MontyREChunker - chunks tagged text into verb, noun, and adjective chunks (VX,NX, and AX respectively) - MontyExtractor - extracts verb-argument structures, phrases, and other semantically valuable information from sentences and returns sentences as "digests" - MontyLemmatiser - part-of-speech sensitive lemmatisation - strips plurals (geese-->goose) and tense (were-->be, had-->have) - includes regexps from Humphreys and Carroll's morph.lex, and UPENN's XTAG corpus - MontyNLGenerator - generates summaries - generates surface form sentences - determines and numbers NPs and tenses verbs - accounts for sentence_type Note: It also has chatterbot code. This e-mail is subject to the UCT ICT policies and e-mail disclaimer published on our website at http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable from +27 21 650 4500. This e-mail is intended only for the person(s) to whom it is addressed. If the e-mail has reached you in error, please notify the author. If you are not the intended recipient of the e-mail you may not use, disclose, copy, redirect or print the content. If this e-mail is not related to the business of UCT it is sent by the sender in the sender's individual capacity. This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?&; - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=61636748-ad030f
Re: [agi] NLP + reasoning?
On 06/11/2007, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > Will Pearson asked > >> I'm also wondering what you consider success in this case. For example > >> do you want the system to be able to maintain conversational state > >> such as would be needed to deal with the following. > > >>"For all following sentences take the first letter of each word and > >>make English sentences out of it, reply in a similar fashion. How is > >>the hair? Every rainy evening calms all nightingales. Yesterday, > >>ornery ungulates stampeded past every agitated koala. Fine red > >>eyebrows, new Chilean hoovers?" > > The majority of human judges in a Turing Test would respond to such > utterances > with a blanket "What's are you talking about?" or "Are you crazy?" or > "I thought we were going to have a conversation?" > > A certain amount of meta questioning is to be expected like... > > What is the third word in this sentence? > > But in order to pass Turing, you just have to convince the judges that you > are human not necessarilly as skilled in word play as Lewis Carroll. > You are under estimating Carroll or over estimating everyone who does cryptic crosswords. http://www.guardian.co.uk/crossword/howto/rules/0,4406,210643,00.html I'm not trying to pass the Turing test and I will never design a system to do just that, if anything I help to create passes the Turing test, that would just be an added bonus. I design systems with potential capabilities and initial capabilities, that I would want it to have. And helping me with cryptic crosswords (which have clues in a similar vein to my example, generally marked with the key word "Initially"), is one of those things I want them to be potentially capable of. Otherwise I would be just making a Faux intelligence, designed to fool people, without being able to do what I know a lot of people can. Will Pearson - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=61674548-887a95
Re: [agi] NLP + reasoning?
I recently heard 2 teenagers talking and it was just amazing how extensively they used the word "like". There was hardly a sentence without it in about 6 minute conversation. There was even a sentence with 4 instances. It made me think about NL parsers. The word can be used as noun, verb, adverb, adjective, preposition, particle, conjunction, hedge, interjection, quotative... Regards, Jiri Jelinek On Nov 6, 2007 10:19 AM, <[EMAIL PROTECTED]> wrote: > > Where I think parser fall down is in recognizing common English typing and > spelling errors. > > "Hello how are you" would be recognizable by a parser but the following > constructs all recognizable by a human > would only be recognizable to a fuzzy pattern matcher. > > "Helohowareyou" > "Hello hwo r u" > "Hell, ho areyu" > > Examining the Transcripts in pas years Turing competitions it is very easy > to see that all of the entries are very > intolerant to fuzzy data and would respond with a obvious bluff when > presented with such inputs. > > > -- Original message -- > From: "Jean-paul Van Belle" <[EMAIL PROTECTED]> > > > > Research Associate: CITANDA > Post-Graduate Section Head > Department of Information Systems > Phone: (+27)-(0)21-6504256 > Fax: (+27)-(0)21-6502280 > Office: Leslie Commerce 4.21 > > >>> Linas Vepstas [EMAIL PROTECTED]> 2007/11/05 23:41 >> > >Do you have any recommendations for other parsers? > One of the reasons I like Python: It's got NLTK and > MontyLingua: > > - MontyTokenizer > - normalizes punctuation, spacing and contractions, with sensitivity to > abbrevs. > - MontyTagger > - Part-of-speech tagging using PENN TREEBANK tagset > - enriched with "Common Sense" from the Open Mind Common Sense project > - MontyREChunker > - chunks tagged text into verb, noun, and adjective chunks (VX,NX, and > AX respectively) > - MontyExtractor > - extracts verb-argument structures, phrases, ;and&n bsp;other > semantically valuable information from sentences and returns sentences as > "digests" > > - MontyLemmatiser > - part-of-speech sensitive lemmatisation > - strips plurals (geese-->goose) and tense (were-->be, had-->have) > - includes regexps from Humphreys and Carroll's morph.lex, and UPENN's > XTAG corpus > - MontyNLGenerator > - generates summaries > - generates surface form sentences > - determines and numbers NPs and tenses verbs > - accounts for sentence_type > > Note: It also has chatterbot code. > > This e-mail is subject to the UCT ICT policies and e-mail disclaimer > published on our website at > http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable from +27 > 21 650 4500. This e-mail is intended only for the person(s) to whom it is > addressed. If the e-mail has reached you in error, please notify the author. > If you are not the intended recipient of the e-mail you may not use, > disclose, copy, redirect or print the content. If this e-mail is not related > to the business of UCT it is sent by the sender in the sender's individual > capacity. > > > > This list is sponsored by AGIRI: http://www.agiri.org/email > To unsubscribe or change your options, please go to: > http://v2.listbox.com/member/?&; > > This list is sponsored by AGIRI: http://www.agiri.org/email > To unsubscribe or change your options, please go to: > http://v2.listbox.com/member/?&; - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=61675415-1b7843
Re: [agi] NLP + reasoning?
Jiri:>I recently heard 2 teenagers talking and it was just amazing how extensively they used the word "like". There was hardly a sentence without it in about 6 minute conversation. A similar, fascinating use - also normally by young people - is "sort of" stuck in over and over. Actually, they're both v. precise uses of language - & arguably provide a window into the brain's operations. They show the brain comparing the particular instance referred to - "He like killed me, man" - to a general category. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=61717628-c922ef
Re: [agi] NLP + reasoning?
On Tue, Nov 06, 2007 at 03:19:35PM +, [EMAIL PROTECTED] wrote: > > "Helohowareyou" > "Hello hwo r u" > "Hell, ho areyu" Yeah, but all this is "easy" to fix with a combo of smelling checkers, 733t-speak decoders, etc. > Examining the Transcripts in pas years Turing competitions it is very easy to > see that all of the entries are very > intolerant to fuzzy data and would respond with a obvious bluff when > presented with such inputs. Which te11s me they never bothered to take the time and eff0rt ... Its really not that hard. --linas - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=61724598-aaf369
Re: [agi] NLP + reasoning?
When listening to that "like"-filled dialogue, I was few times under strong impression that very specific timing in which particular parts of the like-containing sentences were pronounced played a critical role in figuring out the meaning of the particular "like" instance. Jiri On Nov 6, 2007 12:49 PM, Mike Tintner <[EMAIL PROTECTED]> wrote: > Jiri:>I recently heard 2 teenagers talking and it was just amazing how > > extensively they used the word "like". There was hardly a sentence > > without it in about 6 minute conversation. > > A similar, fascinating use - also normally by young people - is "sort of" > stuck in over and over. Actually, they're both v. precise uses of language - > & arguably provide a window into the brain's operations. They show the brain > comparing the particular instance referred to - "He like killed me, man" - > to a general category. > > > - > This list is sponsored by AGIRI: http://www.agiri.org/email > To unsubscribe or change your options, please go to: > http://v2.listbox.com/member/?&; > - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=61971290-234cf0
Re: [agi] NLP + reasoning?
Hi Linas, > Aside from Novamente and CYC, who else has attempted to staple > NLP to a reasoning engine? ... I see the issues have been discussed thoroughly already, but I did not see anyone actually answering your question. Many people indeed have tried to staple NLP to a reasoner, and even though it might be hard to enumerate them all, I'd like to point out at least two web sites where you can find a lot of nice papers and further information. * MIT START is probably the longest running web site with NLP + (simple) reasoning: http://start.csail.mit.edu/ * Attempto Controlled English is a recent effort to do all this stuff in a more formalized and standards-compliant way, so that various ontologies and grammars can be used for various purposes: http://attempto.ifi.uzh.ch/site/ - lk - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=63035892-5df007
Re: [agi] NLP + reasoning?
>On Nov 5, 2007 7:01 PM, Jiri Jelinek <[EMAIL PROTECTED]> wrote: > On Nov 4, 2007 12:40 PM, Matt Mahoney <[EMAIL PROTECTED]> wrote: > >How do you propose to measure intelligence in a proof of concept? > > Hmmm, let me check my schedule... > Ok, I'll figure this out on Thursday night (unless I get hit by a lottery > bus). > "Jelinek test" is coming ;-)) ... I'll get back to you then.. http://www.busycoder.com/index.php?link=24&lng=1 Regards, Jiri Jelinek - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=63305803-92ddbe
Re: [agi] NLP + reasoning + conversational state?
--- Linas Vepstas <[EMAIL PROTECTED]> wrote: > Thus, I find that my interests are now turning to representing > conversational state. How does novamente deal with it? What > about Pei Wang's NARS? It seems that NARS is a reasoning system; > great; but what is holding me back right now is not an ability > to reason per-se, but the ability to maintain a conversational > state. In a statistical or neural model, conversational state is a type of short term memory. You represent what a conversation is "about" as a bag of recently used words. For example, if the words "snow" and "rain" were recently used, then you increase the probability of using words like "weather" or "wet" in your next response. Word associations can take the form of a word-word matrix. For example, there would be a numerical value associating "rain" and "wet" that depends on their co-occurrence frequency in a corpus of training text. If the matrix is sparse it can be compressed using singular value decomposition (SVD). A word-word matrix A would be factored into A = USV where U and V are orthonormal and S is diagonal. Then only the few hundred largest terms of S are retained. This allows most of U and V to be discarded as well. This representation is called "latent semantic analysis" (LSA) because it makes inferences using the transitive property of semantics. For example, if there are associations rain-wet and wet-water then it will infer rain-water even if the original value in A was 0 (due to a small training corpus). SVD is equivalent to a linear neural network with one hidden layer. This is not a model you can tack onto a structured knowledge base. As I said, language has to be an integral part of it. Your approach has been tried hundreds of times. There is a great temptation to insert knowledge directly, but the result is always the same. Natural language is a complicated beast. You cannot hand code all the language rules. After 23 years of developing the Cyc database, Doug Lenat guesses it is between 0.1% and 10% finished. I don't claim that LSA is a solution either. So far there are no good statistical models of grammar. What I can say about statistical models is that they have been used successfully (e.g. Google), and they are bottom up, more like the way children learn semantics before syntax. AFAIK, neither NARS nor Novamente is capable of processing natural language (yet). I am sure Pai or Ben could tell you more. -- Matt Mahoney, [EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60139818-b3253c
Re: [agi] NLP + reasoning + conversational state?
On Thu, Nov 01, 2007 at 02:58:07PM -0700, Matt Mahoney wrote: > --- Linas Vepstas <[EMAIL PROTECTED]> wrote: > > > Thus, I find that my interests are now turning to representing > > conversational state. How does novamente deal with it? What > > about Pei Wang's NARS? It seems that NARS is a reasoning system; > > great; but what is holding me back right now is not an ability > > to reason per-se, but the ability to maintain a conversational > > state. > > If the matrix is sparse it can be compressed using singular value > decomposition (SVD). [...] > This is not a model you can tack onto a structured knowledge base. Why not? Bot NARS and novamente have real-number-valued associative deduction abilities. I see no reason why simple matrix or nerual net algo's couldn't be layered on top of it. > Your approach has been tried > hundreds of times. Yes, I figured as much. I haven't yet seen a cogent explanation of why folks gave up. For shrdlu, sure .. compute power was limited. There's discussion about grounding, and folks wander off into the weeds. > There is a great temptation to insert knowledge directly, > but the result is always the same. Natural language is a complicated beast. > You cannot hand code all the language rules. After 23 years of developing the > Cyc database, Doug Lenat guesses it is between 0.1% and 10% finished. And hasn't stopped trying. We also have Wordnet, and assorted ontology projects. How many english words are there? About 250K, but this hasn't stopped classical dictionary authors, nor wordnet, nor Lenat. How many sentence parse patterns are there? 10K? 100K? 1M? Its not infinite, even though it can feel that way sometimes. Just because you personally don't feel like trying to hand-build an association matrix between sentence parse patterns and a semantic "current topic of conversation" dataset doesn't mean its unbuildable. I didn't claim the approach I described as being "good"; its not; and I can see its limitations already. I did claim that its "practical", and after half-a-dozen weekends coding, I have a demo. I'm trying to understand just how far the approach can be pushed. I get the impression that it hasn't been pushed very far at all, before people give up. --linas - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60163983-af3ed3
Re: [agi] NLP + reasoning + conversational state?
On 11/1/07, Linas Vepstas <[EMAIL PROTECTED]> wrote: > > More importantly, I've started struggling with representing > conversational state. i.e. "what are we talking about?" "what > has been said so far?" I've got some inkling on how to expand > conversational state, but its ad hoc so far. > > Thus, I find that my interests are now turning to representing > conversational state. How does novamente deal with it? What > about Pei Wang's NARS? It seems that NARS is a reasoning system; > great; but what is holding me back right now is not an ability > to reason per-se, but the ability to maintain a conversational > state. I haven't done anything concrete on NLP yet. For why NLP is treated as an optional part of NARS and my plan for it, see http://nars.wang.googlepages.com/wang.roadmap.pdf As for what you called "conversational state", my plan is to remember the past conversation as "events", not as explicitly defined "states" (as those defined in search or planning systems). Pei - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60167943-7f8e6f
Re: [agi] NLP + reasoning + conversational state?
On 11/1/07, Linas Vepstas <[EMAIL PROTECTED]> wrote: > Yes, I figured as much. I haven't yet seen a cogent explanation of > why folks gave up. For shrdlu, sure .. compute power was limited. IIRC, the reason SHRDLU wasn't taken any further wasn't to do with computing power, it was because the program was a blob of spaghetti procedural code, full of special-case hacks; it worked for the blocks-world domain, but offered no hint of general principles by which it might be scaled to realistic domains. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60174000-a71ccd
Re: [agi] NLP + reasoning + conversational state?
On Thu, Nov 01, 2007 at 06:58:14PM -0400, Pei Wang wrote: > On 11/1/07, Linas Vepstas <[EMAIL PROTECTED]> wrote: > > > > More importantly, I've started struggling with representing > > conversational state. i.e. "what are we talking about?" "what > > has been said so far?" I've got some inkling on how to expand > > conversational state, but its ad hoc so far. > > > > Thus, I find that my interests are now turning to representing > > conversational state. How does novamente deal with it? What > > about Pei Wang's NARS? It seems that NARS is a reasoning system; > > great; but what is holding me back right now is not an ability > > to reason per-se, but the ability to maintain a conversational > > state. > > I haven't done anything concrete on NLP yet. For why NLP is treated as > an optional part of NARS and my plan for it, see > http://nars.wang.googlepages.com/wang.roadmap.pdf Of course, right. Parse patterns will have to be stred in the system. I recently hard coded a parse tree (S who (S (VP is (NP x to imply that "#$isa #$X #$Human", and was promptly corrected: "Who is Lucy?" might refer to a dog called Lucy. So I will need a way of asserting knowledge about the parse pattern "(S who (S (VP is (NP x" so that the system can learn how to deduce properties of X. > As for what you called "conversational state", my plan is to remember > the past conversation as "events", not as explicitly defined "states" > (as those defined in search or planning systems). Yes. Same here. The most recently asserted events probably being the most pertinent. The problem is more along the lines of "how do I know which recent events are important for maintaining the conversation?" I dunno, I'll hack my way through this. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60173405-87bf06
Re: [agi] NLP + reasoning + conversational state?
--- Linas Vepstas <[EMAIL PROTECTED]> wrote: > On Thu, Nov 01, 2007 at 02:58:07PM -0700, Matt Mahoney wrote: > > There is a great temptation to insert knowledge directly, > > but the result is always the same. Natural language is a complicated > beast. > > You cannot hand code all the language rules. After 23 years of developing > the > > Cyc database, Doug Lenat guesses it is between 0.1% and 10% finished. > > And hasn't stopped trying. We also have Wordnet, and assorted ontology > projects. If you devoted most of your life to a single project, you would not give up easily either. > How many sentence parse patterns are there? 10K? 100K? 1M? The fact that we don't know is a bad thing. My guess is well over 1M. -- Matt Mahoney, [EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60173481-66a538
Re: [agi] NLP + reasoning + conversational state?
To me this point seems only partially valid. 1M hand coded rules seems excessive, but there should be some number (100? 1000?) of hand-coded rules (not unchangeable!) that it can start from. An absolute minimum would seem to be "everything in 'Fun with Dick and Jane' through 'My Little White House'". That's probably not sufficient, but you need to at least cover those patterns. Most (though not all) of the later patterns are, or can be, built out of the earlier ones via miscellaneous forms of composition and elision. This gives context within which other patterns can be learned. Note that this is extremely much simpler that starting your learning from a clean slate. Vladimir Nesov wrote: Linas, Yes, you probably can code all the patterns you need. But it's only the tip of the iceberg: problem is that for those 1M rules there are also thousands that are being constantly generated, assessed and discarded. Knowledge formation happens all the time and adapts those 1M rules to gazillion of real-world situations. You can consider those additional rules 'inference', but then again, if you can do your inference that good, you can do without 1M of hand-coded rules, allowing system to learn them from the ground up. If your inference is not good enough, it's not clear how many rules you'd need to code in manually, it may be 10^6 or 10^12, or 10^30, because you'd also need to code _potential_ rules which are not normally stored in human brain, but generated on the fly. I plan to support recent context through a combination of stable activation patterns (which is analogous to constant reciting of certain phrase, only on lower level) and temporary induction (roughly, cooccurrence of concepts in near past leads to them activating each other in the present, and similarly there are temporary concepts being formed all the time, of which only those which get repeatedly used in their short lifetime are retained for longer and longer). On 11/2/07, Linas Vepstas <[EMAIL PROTECTED]> wrote: On Thu, Nov 01, 2007 at 02:58:07PM -0700, Matt Mahoney wrote: --- Linas Vepstas <[EMAIL PROTECTED]> wrote: Thus, I find that my interests are now turning to representing conversational state. How does novamente deal with it? What about Pei Wang's NARS? It seems that NARS is a reasoning system; great; but what is holding me back right now is not an ability to reason per-se, but the ability to maintain a conversational state. If the matrix is sparse it can be compressed using singular value decomposition (SVD). [...] This is not a model you can tack onto a structured knowledge base. Why not? Bot NARS and novamente have real-number-valued associative deduction abilities. I see no reason why simple matrix or nerual net algo's couldn't be layered on top of it. Your approach has been tried hundreds of times. Yes, I figured as much. I haven't yet seen a cogent explanation of why folks gave up. For shrdlu, sure .. compute power was limited. There's discussion about grounding, and folks wander off into the weeds. There is a great temptation to insert knowledge directly, but the result is always the same. Natural language is a complicated beast. You cannot hand code all the language rules. After 23 years of developing the Cyc database, Doug Lenat guesses it is between 0.1% and 10% finished. And hasn't stopped trying. We also have Wordnet, and assorted ontology projects. How many english words are there? About 250K, but this hasn't stopped classical dictionary authors, nor wordnet, nor Lenat. How many sentence parse patterns are there? 10K? 100K? 1M? Its not infinite, even though it can feel that way sometimes. Just because you personally don't feel like trying to hand-build an association matrix between sentence parse patterns and a semantic "current topic of conversation" dataset doesn't mean its unbuildable. I didn't claim the approach I described as being "good"; its not; and I can see its limitations already. I did claim that its "practical", and after half-a-dozen weekends coding, I have a demo. I'm trying to understand just how far the approach can be pushed. I get the impression that it hasn't been pushed very far at all, before people give up. --linas - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?&; - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60477015-d0bfa3
Re: [agi] NLP + reasoning + conversational state?
On 11/2/07, Linas Vepstas <[EMAIL PROTECTED]> wrote: > On Fri, Nov 02, 2007 at 09:01:42AM -0700, Charles D Hixson wrote: > > To me this point seems only partially valid. 1M hand coded rules seems > > excessive, but there should be some number (100? 1000?) of hand-coded > > rules (not unchangeable!) that it can start from. An absolute minimum > > would seem to be "everything in 'Fun with Dick and Jane' through 'My > > Little White House'". That's probably not sufficient, but you need to > > at least cover those patterns. Most (though not all) of the later > > patterns are, or can be, built out of the earlier ones via miscellaneous > > forms of composition and elision. This gives context within which other > > patterns can be learned. > > > > Note that this is extremely much simpler that starting your learning > > from a clean slate. > > Yes, exactly. A clean slate is a very hard place to start. And so, > yes, this is my current philosophy: build enough scaffolding to be > able to pump some yet-to-be-determined more general mechanism. How is this scaffolding going to help? You can regard learning subsystem as 'blank slate by definition', so that without scaffolding it'll directly interact with environment, and with scaffolding in place it will interact with whatever scaffolding supplies it with. So, hard-coded part can be regarded as kind of modality, like vision, that performs some preliminary analysis, changes representation and so on. But learning problem isn't changed by it. And if you solve the learning problem, you don't need any scaffolding. -- Vladimir Nesovmailto:[EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60517720-7bf48f
Re: [agi] NLP + reasoning + conversational state?
On Fri, Nov 02, 2007 at 11:27:08AM +0300, Vladimir Nesov wrote: > Linas, > > Yes, you probably can code all the patterns you need. But it's only > the tip of the iceberg: problem is that for those 1M rules there are > also thousands that are being constantly generated, assessed and > discarded. Knowledge formation happens all the time and adapts those > 1M rules to gazillion of real-world situations. You can consider those > additional rules 'inference', but then again, if you can do your > inference that good, you can do without 1M of hand-coded rules, > allowing system to learn them from the ground up. If your inference is > not good enough, it's not clear how many rules you'd need to code in > manually, it may be 10^6 or 10^12, or 10^30, because you'd also need > to code _potential_ rules which are not normally stored in human > brain, but generated on the fly. Yes. agreed. Right now, I'm looking at all of the code as disposable scaffolding, as something that might allow enough interaction to make human-like conversation bearable. That scaffolding should enable some "real" work. My current impression is that opencyc's 10^6 assertions make it vaguely comparable to a 1st grader: at least, conversationally .. it can make simple deductions, write short essays of facts, ... can learn new things, but can go astray easily. Does not yet learn about new sentence types, can't yet guess at new parses. Certainly doesn't have spunk or initiative! Inference is tricky. Even simple things use alarmingly large amounts of cpu time. > I plan to support recent context through a combination of stable > activation patterns (which is analogous to constant reciting of > certain phrase, only on lower level) and temporary induction (roughly, > cooccurrence of concepts in near past leads to them activating each > other in the present, and similarly there are temporary concepts being > formed all the time, of which only those which get repeatedly used in > their short lifetime are retained for longer and longer). Yes. of course. Easy to say. lillybot remembers recent assertions, and can reason from that. However, I'm currently hard-coding all reasoning on a case-by-case, ad-hoc manner. I haven't done enough of these yet to see what the general pattern might be. --linas - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60483902-4e962b
Re: [agi] NLP + reasoning + conversational state?
Linas, Yes, you probably can code all the patterns you need. But it's only the tip of the iceberg: problem is that for those 1M rules there are also thousands that are being constantly generated, assessed and discarded. Knowledge formation happens all the time and adapts those 1M rules to gazillion of real-world situations. You can consider those additional rules 'inference', but then again, if you can do your inference that good, you can do without 1M of hand-coded rules, allowing system to learn them from the ground up. If your inference is not good enough, it's not clear how many rules you'd need to code in manually, it may be 10^6 or 10^12, or 10^30, because you'd also need to code _potential_ rules which are not normally stored in human brain, but generated on the fly. I plan to support recent context through a combination of stable activation patterns (which is analogous to constant reciting of certain phrase, only on lower level) and temporary induction (roughly, cooccurrence of concepts in near past leads to them activating each other in the present, and similarly there are temporary concepts being formed all the time, of which only those which get repeatedly used in their short lifetime are retained for longer and longer). On 11/2/07, Linas Vepstas <[EMAIL PROTECTED]> wrote: > On Thu, Nov 01, 2007 at 02:58:07PM -0700, Matt Mahoney wrote: > > --- Linas Vepstas <[EMAIL PROTECTED]> wrote: > > > > > Thus, I find that my interests are now turning to representing > > > conversational state. How does novamente deal with it? What > > > about Pei Wang's NARS? It seems that NARS is a reasoning system; > > > great; but what is holding me back right now is not an ability > > > to reason per-se, but the ability to maintain a conversational > > > state. > > > > If the matrix is sparse it can be compressed using singular value > > decomposition (SVD). > > [...] > > > This is not a model you can tack onto a structured knowledge base. > > Why not? Bot NARS and novamente have real-number-valued associative > deduction abilities. I see no reason why simple matrix or nerual net > algo's couldn't be layered on top of it. > > > Your approach has been tried > > hundreds of times. > > Yes, I figured as much. I haven't yet seen a cogent explanation of > why folks gave up. For shrdlu, sure .. compute power was limited. > There's discussion about grounding, and folks wander off into the weeds. > > > There is a great temptation to insert knowledge directly, > > but the result is always the same. Natural language is a complicated beast. > > You cannot hand code all the language rules. After 23 years of developing > > the > > Cyc database, Doug Lenat guesses it is between 0.1% and 10% finished. > > And hasn't stopped trying. We also have Wordnet, and assorted ontology > projects. > > How many english words are there? About 250K, but this hasn't stopped > classical dictionary authors, nor wordnet, nor Lenat. > > How many sentence parse patterns are there? 10K? 100K? 1M? Its not > infinite, even though it can feel that way sometimes. Just because > you personally don't feel like trying to hand-build an association > matrix between sentence parse patterns and a semantic "current topic > of conversation" dataset doesn't mean its unbuildable. > > I didn't claim the approach I described as being "good"; its not; > and I can see its limitations already. I did claim that its "practical", > and after half-a-dozen weekends coding, I have a demo. I'm trying to > understand just how far the approach can be pushed. I get the impression > that it hasn't been pushed very far at all, before people give up. > > --linas > > > - > This list is sponsored by AGIRI: http://www.agiri.org/email > To unsubscribe or change your options, please go to: > http://v2.listbox.com/member/?&; > -- Vladimir Nesovmailto:[EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60236054-ec756f
Re: [agi] NLP + reasoning + conversational state?
On Fri, Nov 02, 2007 at 09:01:42AM -0700, Charles D Hixson wrote: > To me this point seems only partially valid. 1M hand coded rules seems > excessive, but there should be some number (100? 1000?) of hand-coded > rules (not unchangeable!) that it can start from. An absolute minimum > would seem to be "everything in 'Fun with Dick and Jane' through 'My > Little White House'". That's probably not sufficient, but you need to > at least cover those patterns. Most (though not all) of the later > patterns are, or can be, built out of the earlier ones via miscellaneous > forms of composition and elision. This gives context within which other > patterns can be learned. > > Note that this is extremely much simpler that starting your learning > from a clean slate. Yes, exactly. A clean slate is a very hard place to start. And so, yes, this is my current philosophy: build enough scaffolding to be able to pump some yet-to-be-determined more general mechanism. --linas - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60484906-1b585e
Re: [agi] NLP + reasoning + conversational state?
On Fri, Nov 02, 2007 at 08:51:43PM +0300, Vladimir Nesov wrote: > But learning problem isn't changed by it. And if you solve the > learning problem, you don't need any scaffolding. But you won't know how to solve the learning problem until you try. --linas - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60586997-1226bb
Re: [agi] NLP + reasoning + conversational state?
On 11/2/07, Linas Vepstas <[EMAIL PROTECTED]> wrote: > On Fri, Nov 02, 2007 at 08:51:43PM +0300, Vladimir Nesov wrote: > > But learning problem isn't changed by it. And if you solve the > > learning problem, you don't need any scaffolding. > > But you won't know how to solve the learning problem until you try. Until you try to solve the learning problem. How scaffolding-building can help in solving it? -- Vladimir Nesovmailto:[EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60594572-1f1427
Re: [agi] NLP + reasoning + conversational state?
On Fri, Nov 02, 2007 at 10:34:26PM +0300, Vladimir Nesov wrote: > On 11/2/07, Linas Vepstas <[EMAIL PROTECTED]> wrote: > > On Fri, Nov 02, 2007 at 08:51:43PM +0300, Vladimir Nesov wrote: > > > But learning problem isn't changed by it. And if you solve the > > > learning problem, you don't need any scaffolding. > > > > But you won't know how to solve the learning problem until you try. > > Until you try to solve the learning problem. How scaffolding-building > can help in solving it? My scaffolding learns. It remembers assertions you make, and it will parrot them back. It checks to see if the assertions you make fits into its beleif network before it actually commits them to memory. It can be told things like "aluminum is a mass noun", and then will start using "aluminum" instead of "the aluminum" or "an aluminum" in future sentences. Sure, I hard-coded the part where "mass nouns don't require an article", that's part of the scaffolding. But that's temporary. That's because the thing isn't yet smart enough to understand what the sentence "mass nouns don't require an article" means. --linas - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60599515-d71478
Re: [agi] NLP + reasoning + conversational state?
On 11/2/07, Linas Vepstas <[EMAIL PROTECTED]> wrote:
> On Fri, Nov 02, 2007 at 10:34:26PM +0300, Vladimir Nesov wrote:
> > On 11/2/07, Linas Vepstas <[EMAIL PROTECTED]> wrote:
> > > On Fri, Nov 02, 2007 at 08:51:43PM +0300, Vladimir Nesov wrote:
> > > > But learning problem isn't changed by it. And if you solve the
> > > > learning problem, you don't need any scaffolding.
> > >
> > > But you won't know how to solve the learning problem until you try.
> >
> > Until you try to solve the learning problem. How scaffolding-building
> > can help in solving it?
>
> My scaffolding learns. It remembers assertions you make, and it will
> parrot them back. It checks to see if the assertions you make fits into
> its beleif network before it actually commits them to memory.
>
> It can be told things like "aluminum is a mass noun", and then will start
> using "aluminum" instead of "the aluminum" or "an aluminum" in future
> sentences.
>
> Sure, I hard-coded the part where "mass nouns don't require an article",
> that's part of the scaffolding. But that's temporary. That's because
> the thing isn't yet smart enough to understand what the sentence
> "mass nouns don't require an article" means.
What I meant is to extract learning and term the rest 'scaffolding'.
In this case, what system actually learns is tagging of terms
('aluminum') with other terms ('is-a-mass-noun'), and this tagging is
provided directly. So it only learns one term->term mapping, which is
coded in explicitly through textual interface (scaffolding) when you
enter phrases like "aluminum is a mass noun". It's hardly a
perceptible step in learning dynamics prototyping.
--
Vladimir Nesovmailto:[EMAIL PROTECTED]
-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?member_id=8660244&id_secret=60619081-bed359
Re: [agi] NLP + reasoning + conversational state?
On Sat, Nov 03, 2007 at 12:06:48AM +0300, Vladimir Nesov wrote:
> On 11/2/07, Linas Vepstas <[EMAIL PROTECTED]> wrote:
> > On Fri, Nov 02, 2007 at 10:34:26PM +0300, Vladimir Nesov wrote:
> > > On 11/2/07, Linas Vepstas <[EMAIL PROTECTED]> wrote:
> > > > On Fri, Nov 02, 2007 at 08:51:43PM +0300, Vladimir Nesov wrote:
> > > > > But learning problem isn't changed by it. And if you solve the
> > > > > learning problem, you don't need any scaffolding.
> > > >
> > > > But you won't know how to solve the learning problem until you try.
> > >
> > > Until you try to solve the learning problem. How scaffolding-building
> > > can help in solving it?
> >
> > My scaffolding learns. It remembers assertions you make, and it will
> > parrot them back. It checks to see if the assertions you make fits into
> > its beleif network before it actually commits them to memory.
> >
> > It can be told things like "aluminum is a mass noun", and then will start
> > using "aluminum" instead of "the aluminum" or "an aluminum" in future
> > sentences.
> >
> > Sure, I hard-coded the part where "mass nouns don't require an article",
> > that's part of the scaffolding. But that's temporary. That's because
> > the thing isn't yet smart enough to understand what the sentence
> > "mass nouns don't require an article" means.
>
> What I meant is to extract learning and term the rest 'scaffolding'.
> In this case, what system actually learns is tagging of terms
> ('aluminum') with other terms ('is-a-mass-noun'), and this tagging is
> provided directly. So it only learns one term->term mapping, which is
> coded in explicitly through textual interface (scaffolding) when you
> enter phrases like "aluminum is a mass noun". It's hardly a
> perceptible step in learning dynamics prototyping.
1) I did not claim to be doing fundamental or groundbreaking AI
research. In fact, I calimed the opposite: that this has been done
before, and I know that many folks have abandoned this approach.
I am intersted in finding out what the roadblocks were.
2) I recently posed the system a question "what is lincoln?" and it
turns out that opencyc knows about 15 or 20 "lincoln counties"
scattered around the united states. So, instead of having the
thing rattle off all 20 counties, I want it to deduce what they all
have in common, and then respond "and lincoln might be one of many
different counties". I think that this kind of deduction will
be a few hours to implement: pattern match to find common ancestors.
So, after asserting "aluminum is a mass noun", it might plausibly deduce
"most minerals are mass nouns" -- one could call this "data mining".
This would use the same algo as deducing that many of the things called
"lincoln" are "counties".
I want to know how far down this path one can go, and how far anyone has
gone. I can see that it might not be good path, but I don't see any
alternatives at the moment.
--linas
-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?member_id=8660244&id_secret=60624837-d7edb6
Re: [agi] NLP + reasoning + conversational state?
Linas,
I mainly tried to show that you are in fact not moving your system
forward learning-wise by attaching a chatbot facade to it. That "My
scaffolding learns" is an overstatement in this context.
You should probably move in the direction of NARS, it seems
fundamental enough to be near the mark. I repeatedly bump into
constructions which have sort-of counterpart in NARS, but somehow I
didn't see them that way the first time I read papers that describe
them, and instead arrived to them by a bumpy go-around road. At least
I'm more confident now that these constructions are not just drawn
randomly from the design space.
On 11/3/07, Linas Vepstas <[EMAIL PROTECTED]> wrote:
> On Sat, Nov 03, 2007 at 12:06:48AM +0300, Vladimir Nesov wrote:
> > On 11/2/07, Linas Vepstas <[EMAIL PROTECTED]> wrote:
> > > On Fri, Nov 02, 2007 at 10:34:26PM +0300, Vladimir Nesov wrote:
> > > > On 11/2/07, Linas Vepstas <[EMAIL PROTECTED]> wrote:
> > > > > On Fri, Nov 02, 2007 at 08:51:43PM +0300, Vladimir Nesov wrote:
> > > > > > But learning problem isn't changed by it. And if you solve the
> > > > > > learning problem, you don't need any scaffolding.
> > > > >
> > > > > But you won't know how to solve the learning problem until you try.
> > > >
> > > > Until you try to solve the learning problem. How scaffolding-building
> > > > can help in solving it?
> > >
> > > My scaffolding learns. It remembers assertions you make, and it will
> > > parrot them back. It checks to see if the assertions you make fits into
> > > its beleif network before it actually commits them to memory.
> > >
> > > It can be told things like "aluminum is a mass noun", and then will start
> > > using "aluminum" instead of "the aluminum" or "an aluminum" in future
> > > sentences.
> > >
> > > Sure, I hard-coded the part where "mass nouns don't require an article",
> > > that's part of the scaffolding. But that's temporary. That's because
> > > the thing isn't yet smart enough to understand what the sentence
> > > "mass nouns don't require an article" means.
> >
> > What I meant is to extract learning and term the rest 'scaffolding'.
> > In this case, what system actually learns is tagging of terms
> > ('aluminum') with other terms ('is-a-mass-noun'), and this tagging is
> > provided directly. So it only learns one term->term mapping, which is
> > coded in explicitly through textual interface (scaffolding) when you
> > enter phrases like "aluminum is a mass noun". It's hardly a
> > perceptible step in learning dynamics prototyping.
>
> 1) I did not claim to be doing fundamental or groundbreaking AI
>research. In fact, I calimed the opposite: that this has been done
>before, and I know that many folks have abandoned this approach.
>I am intersted in finding out what the roadblocks were.
>
> 2) I recently posed the system a question "what is lincoln?" and it
>turns out that opencyc knows about 15 or 20 "lincoln counties"
>scattered around the united states. So, instead of having the
>thing rattle off all 20 counties, I want it to deduce what they all
>have in common, and then respond "and lincoln might be one of many
>different counties". I think that this kind of deduction will
>be a few hours to implement: pattern match to find common ancestors.
>
> So, after asserting "aluminum is a mass noun", it might plausibly deduce
> "most minerals are mass nouns" -- one could call this "data mining".
> This would use the same algo as deducing that many of the things called
> "lincoln" are "counties".
>
> I want to know how far down this path one can go, and how far anyone has
> gone. I can see that it might not be good path, but I don't see any
> alternatives at the moment.
>
> --linas
>
> -
> This list is sponsored by AGIRI: http://www.agiri.org/email
> To unsubscribe or change your options, please go to:
> http://v2.listbox.com/member/?&;
>
--
Vladimir Nesovmailto:[EMAIL PROTECTED]
-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?member_id=8660244&id_secret=60636133-9ba9d0
Re: [agi] NLP + reasoning + conversational state?
--- Linas Vepstas <[EMAIL PROTECTED]> wrote: > So, after asserting "aluminum is a mass noun", it might plausibly deduce > "most minerals are mass nouns" -- one could call this "data mining". > This would use the same algo as deducing that many of the things called > "lincoln" are "counties". > > I want to know how far down this path one can go, and how far anyone has > gone. I can see that it might not be good path, but I don't see any > alternatives at the moment. Well, one alternative is to deduce that aluminum is a mass noun by the low frequency of phrases like "an aluminum is" from a large corpus of text (or count Google hits). You could also deduce that aluminum is an adjective from phrases like "an aluminum chair", etc. More generally, you would cluster words in the high dimensional vector space of their immediate context, then derive rules for moving from cluster to cluster. However, the fact that this method is not used in the best language models suggests it may exceed the computational limits of your PC. This might explain why we keep wading into the swamp. -- Matt Mahoney, [EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60633102-37c859
Re: [agi] NLP + reasoning + conversational state?
On 11/2/07, Matt Mahoney <[EMAIL PROTECTED]> wrote: > Well, one alternative is to deduce that aluminum is a mass noun by the low > frequency of phrases like "an aluminum is" from a large corpus of text (or > count Google hits). You could also deduce that aluminum is an adjective from > phrases like "an aluminum chair", etc. More generally, you would cluster > words in the high dimensional vector space of their immediate context, then > derive rules for moving from cluster to cluster. > > However, the fact that this method is not used in the best language models > suggests it may exceed the computational limits of your PC. This might > explain why we keep wading into the swamp. It is doubtful this kind of examination of information can be 'conversational language' on PC computation for a while. What do you think about the feasibility of a research request using this method? ex: Find interesting information about: aluminum - to which the program builds a structure of information that it can continue refining and expanding until I return to check on it several hours later. If I think it's on the right track for my definition of interesting, I could let it continue researching for days. At the end of several days work, it would have a body of 'knowledge' that represents a cost to compile which makes it a local authority on this subject. Assuming someone else might request information about the same topic, my local knowledge store could be included in preliminary findings. Clearly a distributed network of nodes is never going to be capable of the brute-force speed of knowing all things in one place. I don't usually seek to know all things at once, just a useful number of things about a limited topic. That might good enough to make the effort worthwhile. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60638592-961890
Re: [agi] NLP + reasoning + conversational state?
--- Mike Dougherty <[EMAIL PROTECTED]> wrote: > On 11/2/07, Matt Mahoney <[EMAIL PROTECTED]> wrote: > > Well, one alternative is to deduce that aluminum is a mass noun by the low > > frequency of phrases like "an aluminum is" from a large corpus of text (or > > count Google hits). You could also deduce that aluminum is an adjective > from > > phrases like "an aluminum chair", etc. More generally, you would cluster > > words in the high dimensional vector space of their immediate context, > then > > derive rules for moving from cluster to cluster. > > > > However, the fact that this method is not used in the best language models > > suggests it may exceed the computational limits of your PC. This might > > explain why we keep wading into the swamp. > > It is doubtful this kind of examination of information can be > 'conversational language' on PC computation for a while. In theory it could. A conversational model is a probability distribution over strings of dialogs. Given question Q the problem is to output answer A that maximizes the probability p(A|Q) = p(QA)/p(Q). In practice we don't know how to reliably estimate p(x) for strings x longer than a few words. For a 1 GB corpus, most strings longer than about 3 words will have a count of 0. To model longer strings, your system has to learn semantic and syntactic constraints and have real world knowledge and common sense. Google lacks most of these capabilities but partly makes up for it by using a much larger corpus that allows exact matches up to about 5 words. > What do you > think about the feasibility of a research request using this method? > ex: Find interesting information about: aluminum - to which the > program builds a structure of information that it can continue > refining and expanding until I return to check on it several hours > later. If I think it's on the right track for my definition of > interesting, I could let it continue researching for days. At the end > of several days work, it would have a body of 'knowledge' that > represents a cost to compile which makes it a local authority on this > subject. Assuming someone else might request information about the > same topic, my local knowledge store could be included in preliminary > findings. Google will collect 80,200,000 facts about aluminum and rank them in 0.22 seconds. You can also ask questions like "what is the thermal conductivity of aluminum?" or "what country is the leading producer of aluminum?" verbatim and get the answer. > Clearly a distributed network of nodes is never going to be capable of > the brute-force speed of knowing all things in one place. I don't > usually seek to know all things at once, just a useful number of > things about a limited topic. That might good enough to make the > effort worthwhile. Google uses a cluster of 10^6 CPUs, enough to keep a copy of the searchable part of the Internet in RAM. -- Matt Mahoney, [EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60675138-3b2ee3
Re: [agi] NLP + reasoning + conversational state?
On 11/2/07, Matt Mahoney <[EMAIL PROTECTED]> wrote: > Google uses a cluster of 10^6 CPUs, enough to keep a copy of the searchable > part of the Internet in RAM. And a list of millions of hits is the ideal way to represent the results, right? Ask.com is publicly mocking this fact in an effort to make themselves look better. Kartoo.com does a good job of presenting the relationship of search results to each other. Suppose you get a tip about some cog sci research that might be relevant to AGI. You ask one of your undergraduate assistants to dig up everything they can find about it. Sure, they use Google. They use Lexisnexis. They use a dozen primary data gathering tools. Knowing you don't want 4Gb of text, they summarize all the information into what they believe you are actually asking for - based on earlier requests you have made, their own understanding of what you are looking for and whatever they learn during the data collection process. A good research assistant gets recruited for graduate work, a bad research assistant probably gets a pat on the back at the end of the semester. My question was about the feasibility of a narrow-AI research agent as a useful step towards AGI. Even if it's not fully adaptable for general tasks, the commercial viability of moderate success would be profitable. Or is commercial viability too mundane a consideration for ivory tower AGI research? - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=60737904-74aafd
