Carsten, This was indeed an interesting question. Let me try to answer to my extent of understanding.
a.)Cortical.IO's approach is not bag of words. The finger print of the word Fox already as attributes about the kind of things it may eat . So the finger print or SDR for Fox and Sheep will differ in the bits that represent things they eat. b.) During training. I think the folks at Cortical.IO may be using some additional NLP methods to identify what entity is in focus and what attributes are talking about the entity etc . So those methods may help resolve the types of issues you have raised. The value of their approach is post learning their fingerprints reduce computing power required to operate c.) The fact that fingerprints are pre-computed has big benefit in terms of speed of computing , because the reduces to a array union/intersection/simlarity computation which allows them to process data at scale. thats big d.) However, Fransico correct me if I am wrong here. Unlike Nupic where the system is continuously learning. I dont think learning occurring during operation with cortical IO. This is not necessarily bad. Its just how it works. Chandan On Fri, Nov 27, 2015 at 12:21 AM, Carsten Schnober < [email protected]> wrote: > Dear list, > Thanks for the good read. I am happy to (hopefully) start the discussion. > > The first issue that comes into my mind is syntax: this concerns every > bag-of-words approach, including Cortical's. > The most obvious source of "natural language misunderstanding" in this > context is negation, as easily demonstrated in this example: > > - fox eat rodent > - sheep do not eat rodent > > I suppose the presented algorithm would learn from this that both fox > and sheep do eat rodent, doesn't it? This is probably more harmful when > classifying sentences than during learning, because a corpus of > reasonable size presumably contains a sufficient amount of examples that > are phrased in a more straight-forward way. > More complex examples, including more subtle negation, relative clauses > etc., will pose much larger challenges. > > I am quite sure that the syntax issue has been discussed in this > context. However, I couldn't find any references, neither in the > theoretical nor in the practical part of the whitepaper. I am very > interested in Cortical.io's experiences with that problem and what > possible (future) solutions might look like. > > In statistical NLP, this issue has been tackled (more or less > successfully) with methods such as recurrent neural networks or by using > sliding windows across multiple words. amongst others. Neither of these > approaches seem applicable here without taking away a fundamental and > very handy property of the SDRs: that they can be efficiently aggregated > with boolean operations. > > Although the syntax issue might be almost irrelevant for many practical > use cases such as document classification, I think it raises an > interesting theoretical question. How does the human brain process > syntax and, more interestingly, how can this be incorporated into the > presented theory? > > > A slightly more technical issue I've stumbled across is word inflection. > The whitepaper briefly mentions morphemes which are, according to > linguistic theory, "the smallest meaningful units" in language. I > understand that working on the word level is sufficient in most cases > and much easier for practical reasons (tokenization is relatively easy). > I wonder how this is handled in practice though, for instance when > learning a new "language definition corpus". Are the words automatically > lemmatized? What if a new language is learned for which no lemmatizers > are available? Is mere stemming applied in that case? What happens if > different word forms do express a different meaning? > > Thanks for any input on these issues! > Carsten > > > > > Am 25.11.2015 um 19:13 schrieb Fergal Byrne: > > Nice, Francisco, thanks for letting us know. I've read the paper, very > > well put together. Looking forward to discussions and questions on the > list. > > > > -- > > > > Fergal Byrne, Brenter IT > > > > Author, Real Machine Intelligence with Clortex and NuPIC > > https://leanpub.com/realsmartmachines > > > > Speaking on Clortex and HTM/CLA at euroClojure Krakow, June 2014: > > http://euroclojure.com/2014/ > > and at LambdaJam Chicago, July 2014: http://www.lambdajam.com > > > > http://inbits.com - Better Living through Thoughtful Technology > > http://ie.linkedin.com/in/fergbyrne/ - https://github.com/fergalbyrne > > > > e:[email protected] t:+353 83 4214179 > > Join the quest for Machine Intelligence at http://numenta.org > > Formerly of Adnet [email protected] http://www.adnet.ie > > > > > > On Wed, Nov 25, 2015 at 4:33 PM, Chandan Maruthi > > <[email protected] <mailto:[email protected]>> wrote: > > > > Francisco > > > > This is great , looking forward to read this today > > > > On Wednesday, November 25, 2015, cogmission (David Ray) > > <[email protected] <mailto:[email protected]>> > wrote: > > > > Hi Francisco, > > > > This will make for a very interesting and informative read! > > Can't wait! > > > > Cheers, > > David > > > > On Wed, Nov 25, 2015 at 8:38 AM, Pascal Weinberger > > <[email protected]> wrote: > > > > Great! > > I was waiting for this a long time :D > > Will make my day! :) > > > > Thank you! > > > > > > > > Best, > > > > Pascal Weinberger > > > > ____________________________ > > > > BE THE CHANGE YOU WANT TO SEE IN THE WORLD ... > > > > > > PLEASE NOTE: This email and any file transmitted are > > strictly confidential and/or legally privileged and intended > > only for the person(s) directly addressed. If you are not > > the intended recipient, any use, copying, transmission, > > distribution, or other forms of dissemination is strictly > > prohibited. If you have received this email in error, please > > notify the sender immediately and permanently delete the > > email and files, if any.____ > > > > Please consider the environment before printing this message. > > > > __ __ > > > > > > > > On 25 Nov 2015, at 14:49, Francisco Webber > > <[email protected]> wrote: > > > >> Hello all, > >> For everyone interested in the theoretical background to > >> Cortical.io <http://cortical.io>’s technology: > >> > >> The Semantic Folding white paper is out in its first > >> incarnation: > >> > >> Download full White Paper > >> < > http://www.cortical.io/static/downloads/semantic-folding-theory-white-paper.pdf > > > >> > >> All the Best > >> > >> Francisco > >> > >> > > > > > > > > -- > > /With kind regards,/ > > > > David Ray > > Java Solutions Architect > > > > *Cortical.io <http://cortical.io/>* > > Sponsor of: HTM.java <https://github.com/numenta/htm.java> > > > > [email protected] > > http://cortical.io <http://cortical.io/> > > > > > > > > -- > > Regards > > Chandan Maruthi > > > > > > > > -- > Carsten Schnober > Doctoral Researcher > Ubiquitous Knowledge Processing (UKP) Lab > FB 20 / Computer Science Department > Technische Universität Darmstadt > Hochschulstr. 10, D-64289 Darmstadt, Germany > phone [+49] (0)6151 16-6227, fax -5455, room S2/02/B111 > [email protected] > www.ukp.tu-darmstadt.de > > Web Research at TU Darmstadt (WeRC): www.werc.tu-darmstadt.de > GRK 1994: Adaptive Preparation of Information from Heterogeneous Sources > (AIPHES): www.aiphes.tu-darmstadt.de > PhD program: Knowledge Discovery in Scientific Literature (KDSL) > www.kdsl.tu-darmstadt.de > > -- Regards Chandan Maruthi
