Michael, I could not agree more. On my end I'm looking on using CLA for replacement of different parts of the vision system, which we're developing.
Sent from my iPhone > On Jan 16, 2014, at 12:36, Michael Ferrier <[email protected]> wrote: > > Hi Jeff, > > As someone working on applying CLA principles to vision, I'm very interested > in hearing about your new understanding of cortical function, and especially > of temporal pooling! > > Most valuable to me would be a prose description of the ideas and of their > basis in neuroscience. Greater levels of detail such as algorithms and even > pseudocode are very useful for building a concrete understanding, but > feedback from this community could also help to build the descriptions at > those levels. > > Thanks, > > -Mike > > _____________ > Michael Ferrier > Department of Cognitive, Linguistic and Psychological Sciences, Brown > University > [email protected] > > >> On Thu, Jan 16, 2014 at 1:52 PM, Scott Purdy <[email protected]> wrote: >> I agree with Chetan that they are both useful in different scenarios. >> Circuit diagrams are great for explaining how our neurons work but trying to >> describe e.g. a new sequence memory algorithm would be much easier in >> pseudocode than a circuit diagram IMHO. >> >> >>> On Thu, Jan 16, 2014 at 10:46 AM, Chetan Surpur <[email protected]> wrote: >>> Francisco, >>> >>> I agree that the CLA would very naturally be represented as a circuit, and >>> it would be useful to do so. On the other hand, an algorithmic description >>> would offer an intuitive understanding that gets at the heart of the nature >>> of the process more quickly and explicitly. I don't think the two are >>> mutually exclusive though, and suggest that we target both representations >>> when trying to describe the CLA. >>> >>> Thanks, >>> Chetan >>> >>>> On Jan 16, 2014 9:26 AM, "Francisco Webber" <[email protected]> wrote: >>>> hi Jeff, >>>> >>>> On 16.01.2014, at 06:44, Jeff Hawkins <[email protected]> wrote: >>>> >>>> > Thank you for all these thoughts, I am still digesting them. >>>> > >>>> > Part of what is motivating me to tackle documentation this year is that >>>> > I might be closing in on a broader understanding of what all the layers >>>> > in the cortex are doing. The CLA is just part of that. I need to >>>> > figure out the best way(s) to communicate these new ideas. >>>> >>>> This is great news (actually I was hoping and waiting for this message >>>> ;-). But this makes the issue even more urgent. >>>> >>>> > >>>> > One question is what kind of language to use. In the white paper we >>>> > used prose, pseudo-code, and neuroscience. The pseudo-code was intended >>>> > to be a clear and somewhat formal description, immune to the messiness >>>> > of the actual code. Yes, you need to have some programming skills but >>>> > not much. Do you think that pseudo-code isn't sufficient as a formal >>>> > description? >>>> >>>> I would not say that pseudo-code is not formal but it might not be >>>> appropriate. As if we would like to draw a oil painting with chalk. >>>> I am not sure if the CLA could really be seen as algorithm, to me it >>>> consists of components with inputs and outputs, some wiring and signals >>>> that run across them. >>>> By trying to model this with software, we do two things at the same time: >>>> we build an analog circuit simulator, and we build a parametrized (and >>>> serialized) schematic within. >>>> It took several decades of academic and industrial efforts to come up with >>>> reliable simulators. Why not use this achievement and focus on the >>>> components and the schematic of the CLA? >>>> Another advantage would be the fact that circuits in these simulators can >>>> be operated and probed virtually. This would allow to actually test the >>>> behavior of the setup. >>>> But don’t get me wrong, I am not claiming to build a “real” virtual brain >>>> using PSpice. Rather would I do the abstraction-step at the level of the >>>> components. One component could be for example a synapse with two switch >>>> inputs and a control-voltage input. Regardless how complicated a real >>>> synapse would be. Another component could be a “CLA-cell” also independent >>>> of how many neurons might be behind this concept in the real brain. >>>> When the circuit is captured as schematic, it is much easier to >>>> collectively discuss it by pointing at it. There are also program >>>> generators that can convert such a circuit model into software. But the >>>> circuit could probably also be rendered as hardware …. at some point. >>>> And after all, a schematic of a CLA circuit would be an unambiguous model >>>> for deriving generic software versions of the CLA which could be directly >>>> compared to the signal response of the hardware-model. >>>> > >>>> > The other question is where to publish. We could just write a new white >>>> > paper, the current one is getting old anyway. The nice thing about a >>>> > white paper is it can be as comprehensive as needed. But I am also >>>> > feeling some pressure to publish in a peer reviewed journal. Some >>>> > people don't take you seriously without peer review. As you suggest we >>>> > could do a white paper and then a series of smaller papers. >>>> >>>> In order to make the step towards a peer reviewed publication, it will be >>>> a prerequisite to also publish some data with it, that has been evaluated >>>> against a known (scientific) standard outcome, and that (if you target a >>>> high impact journal) the provided data should be reproducible by >>>> independent peers. >>>> But this is only partially the reason why I am advocating to also consider >>>> the development of a reference-evaluation framework, for the continuous >>>> collective improvement of the CLAs. >>>> >>>> all the best from Hong Kong >>>> >>>> Francisco >>>> >>>> > >>>> > -----Original Message----- >>>> > From: nupic [mailto:[email protected]] On Behalf Of >>>> > Stewart Mackenzie >>>> > Sent: Tuesday, January 14, 2014 8:58 PM >>>> > To: NuPIC general mailing list. >>>> > Subject: Re: [nupic-discuss] Hong Kong meeting outcome >>>> > >>>> > please read responses inline: >>>> > >>>> > Francisco Webber <[email protected]> wrote: >>>> >> Jeff, Steward, >>>> >> I think what is actually missing is a formal description of the >>>> >> algorithm that allows non software programmers to understand and >>>> >> analyze it in full detail. Fergal and I were even thinking of an >>>> >> electronic wiring schematic to try and capture the full detail of CLA >>>> >> in a concrete formal way. I fear that without a central formal model >>>> >> the theoretical research, the practical implementation and the >>>> >> documentation will slowly drift apart without anyone noticing. >>>> >> Having a software-development-independent formal description of the >>>> >> algorithm will also help to avoid the mix-up of code-parts that are >>>> >> there because the CLA algorithm says so, parts that are there because >>>> >> no better engineering solution has been found yet for a specific system >>>> >> behavior and even parts that might be there because of the specific >>>> >> expressiveness of a programming language. >>>> > >>>> > We're designing and developing a flow-based programming environment on >>>> > Mozart Oz. >>>> > >>>> > -To get to the point quickly: this tool allow us to expose a >>>> > 'box-and-line' user interface to the non-programmer that can be >>>> > programmed by dragging a finger on the screen. One box might be a >>>> > neuron, another a dendrite another a synapse and one can 'drill down' >>>> > and open up components within components (ie composite components). This >>>> > allows high level display of logic without getting bogged down in >>>> > implementation code. >>>> > - the programming language (oz) used to program components has some 30 >>>> > odd different language concepts and is _more_ that expressive enough for >>>> > this challenge. >>>> > >>>> > The example below is in 'flow-based programming' for the AND logic gate, >>>> > it is executable: >>>> > >>>> > a => a nand(nand) out -> in not(not) out => out b => b nand() >>>> > >>>> > The above represents the application's logic graph. >>>> > the Nand is a component and is implemented in Oz. >>>> > the Not gate on the other hand looks like this on the inside: >>>> > >>>> > The not gate is a composite-component. >>>> > NOT gate: >>>> > >>>> > in => a nand(nand) out => out >>>> > in => b nand() >>>> > >>>> > In the above example the 'in' on the left of the => is the input >>>> > interface, the out to the right of the => is the output interface. The >>>> > stuff between the => and => is the logic of the graph. nand(nand) >>>> > equates to 'name_of_component(type_of_component)'. The second 'nand()' >>>> > does not need a type as it has already been instantiated. >>>> > Again the nand gate is implemented in Oz. >>>> > >>>> > Now the above representation is very easy to create a high level >>>> > application logic specific view that is _executable!_ most importantly, >>>> > the components are being designed so that we may create a GUI: >>>> > >>>> > --------- -------- >>>> > a -----> a| | | | >>>> > | nand | out -------->in | not | out ---------> out >>>> > b -----> b| | | | >>>> > --------- -------- >>>> > >>>> > We are currently tying this together with an HTML implementation. So >>>> > it'll be interactable with a finger. >>>> > >>>> > So writing the paper, then implementing it in this interface one creates >>>> > an executable specification that non-programmers can easily change. This >>>> > serves as a perfect highly flexible example for implementers to >>>> > reference for more optimal implementations in say c++ (ie nupic-core). >>>> > >>>> > Francisco is this what you imagined? >>>> > >>>> > While you're still in town you should come over for dinner so I can demo >>>> > this to you ;-) >>>> > >>>> >> - The “language” of the formal model should allow people from different >>>> >> (maybe even VERY different) areas to gain full insight into the >>>> >> fundamental algorithm. >>>> >> - The formal model should also allow to think, discuss and communicate >>>> >> concepts at different scales of resolution (from the birds-eye >>>> >> perspective down to the synapse level) >>>> >> >>>> >> I am well aware that this is not an easy thing to do, but I think it is >>>> >> indispensable for the development of any theoretic research field to >>>> >> have its own formal notation framework. >>>> >> >>>> >> Francisco >>>> >> >>>> >> On 15.01.2014, at 07:33, Stewart Mackenzie <[email protected]> wrote: >>>> >> >>>> >>> Very good point Jeff, I suspect this material is ready for peer >>>> >> review (in the idealistic sense of the phrase peer review). I'll admit >>>> >> I've lost faith in the peer review process of today. For many reasons, >>>> >> too many to go into at the moment. Though despite my tangible dislike >>>> >> for journals which needs to be disrupted off the face of the earth, >>>> >> publishing with them most likely brings more academics onboard. >>>> >>> >>>> >>> The main driver for this paper is to get everyone on the same page. >>>> >> I'd prefer seeing a comprehensive (white) paper, with a series of >>>> >> smaller papers focusing on smaller areas being published in the >>>> >> journals. All the detail is in the comprehensive. The smaller papers >>>> >> are to move academia along using a language and publication process >>>> >> they are familiar with. I suspect new names should be made to describe >>>> >> newly discovered phenomenon, with as emphasis on describing the right >>>> >> 'altitude' one needs to approach this problem. This conditions the >>>> >> academics to start adjusting the mindset to a certain level. Eventually >>>> >> causing a resonance which hopefully can be conducted into and through >>>> >> NuPIC. So we as a community had better be sure NuPIC is in shape such >>>> >> that the resonance won't shatter it. >>>> >>> >>>> >>> Jeff Hawkins <[email protected]> wrote: >>>> >>>> When you say "canonical paper" are you thinking a peer reviewed >>>> >> paper >>>> >>>> or an >>>> >>>> updated white paper? Is it more important to be comprehensive >>>> >> (white >>>> >>>> paper) >>>> >>>> or published in peer reviewed journals? Or are you thinking >>>> >> something >>>> >>>> else? >>>> >>>> Jeff >>>> >>> >>>> >>> Kind regards >>>> >>> Stewart >>>> >>> >>>> >>> -- >>>> >>> Please excuse my typos and brevity >>>> >>> >>>> >>> _______________________________________________ >>>> >>> nupic mailing list >>>> >>> [email protected] >>>> >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>> >> >>>> >> >>>> >> _______________________________________________ >>>> >> nupic mailing list >>>> >> [email protected] >>>> >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>> > >>>> > Kind regards >>>> > Stewart >>>> > >>>> > -- >>>> > Please excuse my typos and brevity >>>> > >>>> > _______________________________________________ >>>> > nupic mailing list >>>> > [email protected] >>>> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>> > >>>> > >>>> > _______________________________________________ >>>> > nupic mailing list >>>> > [email protected] >>>> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>> >>>> >>>> _______________________________________________ >>>> nupic mailing list >>>> [email protected] >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>> >>> _______________________________________________ >>> nupic mailing list >>> [email protected] >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> >> _______________________________________________ >> nupic mailing list >> [email protected] >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
