Aaron, On Wed, Mar 27, 2013 at 11:21 AM, Aaron Hosford <[email protected]> wrote:
> > There are two layers of semantic information: the raw "literal" meaning, > and the processed/integrated meaning. Semantic structure defines the raw > meaning in a well-defined way. Idioms are remappings of literal meanings to > other domains based on frequently used analogies. They exist because it's > easier to reuse a familiar analogy than spend unnecessary minutes conveying > a precise non-analogical meaning directly. > This is true for around half or a little more of idioms, that either were created to be idioms, e.g. "the best government that money can buy" or are associated with now-obsolete practices like "don't throw the baby out with the bathwater". The rest of the idioms are similes, e.g. "hoist by his own petard", which in a military setting is NOT an idiom, or are "obvious" exaggerations, e.g. "I worked my fingers to the bone" writing this posting. Many of the idioms relating to now-obsolete practices are also similes. With similes, you must disambiguate enough to decide whether the idiom is really an idiom at all. A case might be made that a listener should be able to understand similes and obvious exaggerations. This would be really nice, but is WAY beyond what anyone now has any idea how to do. And in the world of AGI. I'm not looking to make a chatbot, or a tool that > pulls out only a handful of keywords and uses shrewd guesses to select from > among a handful of well-defined, pre-programmed jobs like Siri. I'm trying > to build something that actually understands and thinks about what is said > to it, and generates original responses. > What good is it to work on developing something, when there is little understanding of the task, no apparent use for information outside of a computer's understanding, no apparent need for the product, and no market for it? We seem to be getting into the area of mental masturbation here. > > I don't think so!!! Have you seen my own theory of language, based on > placement and payload? It is in the patent application, or I'd be glad to > post it (again) here. My theory is that semantics is only sensitive to > structure within a payload, which is usually just a small part of a > sentence. > >> > No, I didn't see it. The short description you provide sounds familiar to > my own theory, though: Most parts of a sentence serve only to identify the > correct entities the sentence is about, because addressing is the hard part > in a semantic memory. Subtracting out the information that is needed for > addressing, you are typically only left with a small amount of information > -- the relationship between the addressed entities that is asserted or > denied. However, given the structure of human semantic memory, there is no > way around this, and so human language is as efficient as it can be given > the task at hand. One of the first signs that you have misunderstood > someone is when they provide information that is seemingly superfluous, > There MUST ALWAYS be superfluous information!!! You can't know exactly what a listener knows and doesn't know. If you don't say enough extra to make sure there is a solid frame of reference, which requires saying some extra to cover things your listener might not know, then your listener won't even know what you are talking about. and so it seems clear to me that maximal efficiency is a guideline used in > actual human language generation. > I think you should rethink this, as some redundancy is absolutely necessary. > > As I distantly recall, people were studying language structure in an > attempt to understand how we think MANY years ago, back in the 1950-1960s. > It didn't seem to go anywhere interesting. > >> > Because analytical reasoning is more about efficient representation than > simple logic. Humans think about the world in terms of objects, types, > relationships, events, settings, attributes, and degrees. This is a rich > representational framework, and consequently requires a rich set of > semantic structures. The advantage human beings have over computers is that > we are astoundingly good at identifying things; we are only mediocre at > thinking about those things once we have identified them, but that's ok > because most things don't require deep analysis once they are effectively > identified. As a result, most semantic structures in language are devoted > to the identification of things. These structures are not static, but > rather culturally evolve to maximize utility over time, and so there is a > divergence between different languages. However, I challenge you to find a > language that doesn't refer to objects, types, relationships, events, > settings, attributes, and degrees in some way or another, or that uses > other kinds of semantic structures. (This list may not be complete, but I > am confident it is at least nearly so.) > There are some languages where the words appear more in the order of importance than in complex relationships, leaving lots of ambiguities to resolve. > > You are right that semantic processing slows things down. I think the best >>> way to solve the problem is to process syntax and semantics separately, >>> just as the brain does, rather than trying to accomplish both at once. >>> (Much research has shown that the two faculties are indeed distinct, both >>> in parsing and generation.) A small set of candidate syntactical structures >>> can often be computed very quickly based on nothing more than word classes >>> (verb, noun, preposition, etc.) together with special rules keyed off of >>> special words -- this is not the bottleneck. Once a set of candidate >>> syntactical structures has been identified, the semantic structure is often >>> sufficiently constrained that only minimal processing and/or default >>> assumptions based on familiarity is enough to rapidly and effectively >>> reduce all combinations of legal syntax + semantics to only one or two >>> choices, but AFAIK no one seems to be working on this other than myself. >>> >> >> I also address the issue of language granularity in my patent, which I >> can also re-post here. >> >> You can only say a finite number of things in any particular language. If >> what you want to say isn't precisely one of those things (and it seldom >> is), you MUST say something different than what you would rather say, but >> close enough for a listener to get the essence of your message. >> >> There is no "simple" logic for either creating erroneous messages from >> correct intentions, or for extracting the correct intentions from erroneous >> messages. Everyone working on NL seems to make the "obvious" presumptions >> that there is SO bidirectional way of translating back and forth between >> intention and text. This does NOT exist. >> > > The not-so-simple logic is to find the closest meaning to what was said > that actually makes sense within the context of the listener's knowledge > about the topic and the speaker's awareness thereof. Likewise, in language > generation, we find the simplest way (to our knowledge) to say something > for which the listener would correctly adjust the meaning to what was > actually intended. This all fits into the efficiency framework; it often > takes more time to say something precisely than it does for the listener to > mentally repair an imprecise statement. It's all about using semantic > distance (dissimilarity) to maximize efficiency. > This is the way that present-day speech recognition like Dragon NaturallySpeaking works. The problem with this approach is that there is no reason to ever say the obvious, and MUCH reason to state when crazy things happen. This turns probabilities on their head. > > To "look under the hood" of this, I strongly recommend that you look at a >> copy of one of the English translations of the Koran (pages are viewable on >> Amazon). In these you will see the original Arabic, the nearest-meaning >> English, and footnotes that often exceed the combined total length of the >> Arabic and English that explain the differences between the nearest-meaning >> English and what was apparently meant in the Arabic, which of course there >> is continuing dispute over. >> >> For example, does the Koran say that "women should dress modestly >> (whatever that means)", or that "women should cover their modesty (not run >> around naked)"? Of course these are two VERY different interpretations. >> Unfortunately, there were no dictionaries way back then, so there is no way >> to resolve this argument, short of breaking out the firearms, which they do >> all the time. >> >>> >>> The key here is to recognize that disambiguation requires integration >>> with previously acquired knowledge, and can not be computed for the >>> language fragment in isolation. Thus, to make a system that can >>> disambiguate language is to make a system that can understand language. >>> >> >> That is a BIG jump. All that appears to be needed is to identify past and >> present subject domains for a given author. >> > > So if I tell you a story about two men, and in the middle of it, I say, > "He gave him the key," you will be able to disambiguate who gave whom the > key based on past and present subject domains for the author? Or if you > don't see anaphora resolution as disambiguation, suppose the key opens the > door to a much discussed computer room. How is a word frequency-based > disambiguation going to identify the correct meaning for "key" as an object > you hold in your hand, versus something associated with computers, like a > PGP key? > As I said before, this cannot work well until AGIs walk among us. > There is no substitute for understanding in language disambiguation, if > you're going to do it reliably. > That's the gold standard. However, after doing the work leading up to my patent, even if you had the perfect algorithm and only needed to code it, there won't be a computer for decades on which to run your program. Dragon NaturallySpeaking's "model" for "understanding" is just a lot of sound and word pairing statistics - FAR short of the sort of the level of understanding you are talking about. Still, it can't work acceptably well at talking speed on gigahertz processors. Present processors are MANY orders of magnitude too slow to process human-level understanding. So, unless you can find some tricks like I have found for my approach to speed your approach WAY up, you will NOT be creating anything that works until you are MUCH older. This might be a good project for your grandchildren. > > >> Statistical approaches that look for common word combinations for >>> disambiguation are just naive attempts to summarize previously acquired >>> knowledge, but they fail to capture that knowledge in sufficient depth to >>> do the job reliably. >>> >> >> Computers can only "compute" on what they have been programmed to compute >> upon, whereupon selective word examination is adequate. Alternatively, a >> computer can pass information through for a human reader to do their own >> disambiguation, as in the case of automatic translation. >> >> >>> There is no shortcut to understanding. >>> >> >> I don't think that "understanding" can be accomplished, before machines >> walk among us. Also, I don't think the "understanding" is needed for the >> vast majority of PRACTICAL applications. >> > > Practical applications are dependent both on what is needed and what is > possible. Often, we don't recognize practical applications until they > become possible. I think the ability to truly understand language will have > many, many practical applications, once it becomes possible. > The Russian Academy of Science has been pouring everything they have into this, to create a better translator. This is a laudable goal, if and when it can be achieved. > You are probably right that machines must "walk among us" to fully > understand what we say, but with resources like Wikipedia and the rest of > the internet, there is a long way to go before that becomes a bottleneck. I > can read about the Jabberwocky and still reason about it even though I have > no idea what one is, so long as I stick to the things I'm told about it. > Only because you can relate it to your real-world experiences, that a computer confined to text will NOT have. > Likewise, there is no reason a program could not reason about objects it > has never seen, so long as it sticks to what it's told > Enough of which is wrong to REALLY screw things up. I gave up on DrEliza carrying on back-and-forth conversations because everything fell apart when someone made a false statement. It is damn near impossible to accurately correct a misstatement with another statement. Instead, I switched to having them edit their original statement to eliminate their errors. There are enough errors in Wikipedia to brain-short ANY system. -- and bothers to truly integrate that information rather than processing > it superficially. > This idea won't even survive desk checking. > > I think this discussion also gives some good insight into the >>> combinatorial explosion problem that Jim keeps coming back to. If I have a >>> problem that requires me to search 1,000,000 different combinations, but I >>> can factor that search space into 3 (nearly) independent pieces, each with >>> 100 combinations, then the total number of choices I need to search becomes >>> only 300, plus minor processing to handle the sparse non-independent cases. >>> This is why divide-and-conquer works. >>> >> >> You are discussing this in a vacuum - like what application are you >> looking at? I don't see how this discussion fits any present-day >> application needs. >> > > This is really directed at Jim, not at you. He frequently brings up > combinatorial explosions as the fundamental bottleneck to AGI, and insists > there must be some kind of breakthrough before we can get past it. I think > the solution has been staring us in the face for ages, and we just need to > bother using common sense techniques like divide-and-conquer in our > problem-solving programs instead of just in our own direct problem solving. > I don't think divide and conquer is enough to live with present-day hardware. Also, divide and conquer does NOT mesh with best fit methods, but rather demands that decisions are made during the "divide" phase that, when wrong, dooms the process. > > In dividing syntactic parsing from semantic disambiguation, a similar >>> effect applies. Sometimes it is better to temporarily work with *less* >>> information, if that allows you to factor the problem. >>> >> >> In most real-world applications (not including automatic translation), >> only a tiny fraction of the text needs to be acted on. The rest can be >> summarily ignored. >> > > Not including automatic translation, and AGI -- if you're really going to > do AGI and not just superficially fake it for a small problem domain or > shallow stimulus/response. > AGI is NOT going to be done on any sort of PC (because even supercomputers are too slow), and is NOT going to be done during the next few years (because there is nothing on the drawing boards that could conceivably work in either hardware or software). When it becomes conceivable to run such programs, I may get more interested in writing them. > > >> Semantic knowledge does indeed occasionally determine which of several >>> syntactic structures is the correct one, but those non-independent cases >>> are sufficiently sparse that factoring still provides an enormous speedup. >>> I expect the same principle to hold in most domains, and I believe the >>> ability to effectively factor problems is core to any successful GI, >>> artificial or not. >>> >> >> Now, add that only the factors that actually have applicable words >> present are even considered, and it will run VERY fast. Again, my point is >> that looking at least frequently used elements gives you a TREMENDOUS speed >> advantage, in addition to other advantages you may find. >> > > I'm already using something similar to this technique in my system. Not so > much in parsing, but in understanding/disambiguation once the parsing is > complete and the base-level semantic structure has been identified. > It appears that great minds think alike. Steve ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-f452e424 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-58d57657 Powered by Listbox: http://www.listbox.com
