Hi Rich, Thanks for restarting this discussion.
I started a project to reimplement HTM in Elixir in December 2013, but then switched to Clojure for mainly non-technical reasons. One of the "thought leaders" in the Elixir community is interested in being the lynchpin of the project to bring HTM to Elixir, so I'll be making some announcements on that in the near future (I have some code archaeology to perform first!). Your intuitions are correct about both the scalability and simplicity of using Elixir/Erlang to do HTM. In my initial tests, I was able to spring up 250k neuron processes in 20-25ms in Elixir on a laptop. On the general point of distributed HTM, Michael is correct to identify the granularity at which things can be split up, and Tim is on the money about the kernel of the issue - state (synapses, in particular). In a typical NuPIC-sized region, we have 2048 cols x 32 cells = 64k total cells, with in the neighbourhood of 1-300m synapses. The number of "messages" passed between these neurons (and their "state") is thus very large compared with the number of input and out messages between regions. It makes sense to have the processing for a contiguous "patch" of neurons such as this all contained within a single (OS level) process, and to have patches communicate using SDRs. Matt is correct when he describes the importance of this in the context of Temporal Pooling and hierarchy. With a multi-layer architecture for a single region, and a hierarchy of regions, we will very quickly hit the skids if we continue with a single-threaded, monolithic design for HTM. On the other hand, Matt is also correct that, once solved, we can take advantage of distributed processing to build HTM systems as large and powerful as we like. Within a patch, I think the jury is very much out on the performance of message-passing versus sparse vectors (as used in NuPIC). Due to sparseness both in space and time in real world data, it's not clear that message-passing (or some equivalent, functional reactive scheme) would not outperform the use of big sparse arrays. Regards, Fergal Byrne On Thu, Feb 5, 2015 at 7:07 PM, Rich Morin <[email protected]> wrote: > On Feb 5, 2015, at 05:18, Kevin Archie <[email protected]> wrote: > > https://github.com/nupic-community/comportex > > > > (I have no connection to the project, I’m just aware of it.) > > The Clojure ports are certainly worth a look, if only to see how they > decompose the problem. Although scalability is a motivation, my real > interest has to do with seeing how Elixir (including Erlang and OTP) > can be used to simplify the model. That is, can I model things like > neurons, columns, and regions using lightweight processes, leaving > the communication and management to OTP. > > -r > > -- > http://www.cfcl.com/rdm Rich Morin [email protected] > http://www.cfcl.com/rdm/resume San Bruno, CA, USA +1 650-873-7841 > > Software system design, development, and documentation > > > > -- Fergal Byrne, Brenter IT http://inbits.com - Better Living through Thoughtful Technology http://ie.linkedin.com/in/fergbyrne/ - https://github.com/fergalbyrne Founder of Clortex: HTM in Clojure - https://github.com/nupic-community/clortex Author, Real Machine Intelligence with Clortex and NuPIC Read for free or buy the book at https://leanpub.com/realsmartmachines Speaking on Clortex and HTM/CLA at euroClojure Krakow, June 2014: http://euroclojure.com/2014/ and at LambdaJam Chicago, July 2014: http://www.lambdajam.com e:[email protected] t:+353 83 4214179 Join the quest for Machine Intelligence at http://numenta.org Formerly of Adnet [email protected] http://www.adnet.ie
