A few comments on the Meeting Notes [1] and Extraction Plan [2] 1. I suggest splitting out the proposed new project structure (most of [1]) as a new wiki page (like you have done with the Extraction Plan), which should evolve to contain the final structure as a result of discussions here. Call it something like "Proposed NuPIC 2014 Design". Leave the rest of Meeting Notes as a record of the initial discussion you had.
2. Take out the tactical ideas about extraction from the Proposed Design page, merge with your Plan page. Substantively, I fear the current Extraction Plan is fragile. We need to have total freedom to build this right first time round. There are decisions which have to be made now which will be impossible to alter in the future. The new design cannot be hampered by current requirements to work with Grok, as these are examples of decisions made long ago which are now baked in to the whole system. This is a major fork of NuPIC, not an evolution. Here's an alternative strategy: Leave nupic as is. Make no structural changes whatsoever. Grok continues to maintain NuPIC for its own use, with continued community involvement Many people on the list are using this NuPIC and working on enhancements, and we should not ever have a broken NuPIC. Fork nupic into two new repos: nupic.core (only C++ from nta/) and nupic.py.legacy (all the python). Make nupic.py.legacy depend initially on the old nupic/nta directory. This will mean that nupic.py.legacy is effectively a copy of nupic, so it can be built just like nupic today. The CI process can continue to operate on nupic alone, but can be straightforwardly changed to nupic.py.legacy and nupic/nta once Grok decide to switch. From Grok's point of view, this is an evolutionary change (just splitting out the python into the new legacy repo). Now, nupic.core has nothing depending on it, so we can rip it up any way we wish and get the design right. We can create a nupic.py repo (a fork of the current nupic/py) which will drive and depend on the evolving nupic.core, and provide any bits (such as the TP) which are not currently implemented in C++. Components such as TP can be finished in C++ and added to fill out nupic.core. The dev process for nupic.core and nupic.py can be much more liberal than for Grok's nupic/nta and nupic.py.legacy repos, allowing for much faster turnaround. Proposed structural changes should be accompanied by first writing a set of regression tests which will verify the safe upgrade. We can also make drastic but necessary changes to the API (beginning with a definition of it as is!) by changing both core and client (and their tests) at the same time. Consequences: nupic effectively reverts to being Grok's base repo (along with other users). Grok can choose to switch to the nupic.py.legacy - nupic/nta setup when it chooses. Grok always has a stable version of NuPIC and can time its transition to the new NuPIC itself, based on company priorities. nupic.core and nupic.py begin as a copy of nupic, but with complete freedom to ruthlessly refactor (terrible word - reconstruct or rewrite is better) into a clean design. [1] https://github.com/numenta/nupic/wiki/2014-January-Core-API-Meeting-Notes [2] https://github.com/numenta/nupic/wiki/NuPIC-Core-Extraction-Plan Fergal Byrne On Fri, Jan 24, 2014 at 9:07 AM, Fergal Byrne <fergalbyrnedub...@gmail.com>wrote: > Apologies Matt, > > Just read your Meeting Notes (should have done that first), so a small > amendment. Insert at top of my list > > nupic repo remains with entire current contents > > nupic.core etc... > > Regards, > > Fergal Byrne > > > On Fri, Jan 24, 2014 at 9:01 AM, Fergal Byrne <fergalbyrnedub...@gmail.com > > wrote: > >> Thanks Matt, >> >> That'll be a huge step forward, well done for getting it out the door as >> a priority. I think your point (made during the Office Hour) about >> incrementally including regression tests as a way to control the migration >> is vital, and I think you should proceed as you suggested by putting >> "trivial" or basic regression tests in and then driving enthusiasts (Marek >> springs to mind) to add to and improve them. >> >> I agree with Chetan about using dots in the repo names (it's become a >> convention in most OSS projects as well as in namespaces). In addition, >> this will allow us to extend the hierarchy at a later date rather than >> build new repos. This also answers the python question by using the github >> naming convention (project.repo.component.subcomponent..). Here's a >> candidate: >> >> nupic.core is the repo for a pure C++ stack, including: >> nupic.core.lib - project packaging the nupic-core API as a GLIBC library >> (and Windows equivalents) >> nupic.core.app - command line app (single download, easy to build >> binaries) which reads a single config file: "nupic -c >> hotgym/nupic-config.json" >> >> nupic.py is the python project repo, including >> nupic.py.bindings >> nupic.py.lib - python library version of NuPIC, eventually packageable as >> a python package >> nupic.py.clients - approved clients including OPF, various runners, >> visualisers, Cerebro etc. These would each be in nupic.python.clients.opf >> for example. >> >> nupic.ruby repo includes >> nupic.ruby.lib makes a nupic gem >> nupic.ruby.clients >> >> nupic.your-lang-here >> >> I recommend setting up most of these language repos with just a README.md >> calling for people to fork and build the initial version, then pull them in >> yourself when appropriate, at which point you could appoint the forker as >> lead maintainer for that repo (or continue to point people to the fork and >> manage the merges yourselves). >> >> Regards, >> >> Fergal Byrne >> >> >> >> >> >> On Thu, Jan 23, 2014 at 6:20 PM, Matt Keith <ke...@keithcom.com> wrote: >> >>> I think it all depends on how you define 'client'. If the OPF is >>> considered a client, then I think that it should be separate from the >>> python bindings. However, I think that the python encoders should be >>> included in the python bindings repo. So maybe something like this: >>> nupic.core <- nupic.python (includes encoders and simple example) <- >>> additional clients or projects using nupic >>> >>> On Jan 23, 2014, at 11:06 AM, Chetan Surpur <cheta...@gmail.com> wrote: >>> >>> Trivial suggestion: how about naming the repo 'nupic.core'? The dot >>> notation to demonstrate project hierarchy seems cleaner to me (and is >>> common on GitHub). >>> >>> Other repo names might look like 'nupic.python' and >>> 'nupic.python.client'. >>> On Jan 23, 2014 8:30 AM, "Matthew Taylor" <m...@numenta.org> wrote: >>> >>>> You guys want a C++ core extracted from NuPIC, so we're going to plan >>>> it. Starting next sprint[1], we're going to work on pulling all the C++ out >>>> of NuPIC into a repo called 'nupic-core'. >>>> >>>> We had a meeting Monday[2] where we discussed many of the details of >>>> this plan, and I created some diagrams to describe the plan[3]. Please take >>>> a moment to review these and provide feedback for this plan. The end goal >>>> is this: >>>> >>>> >>>> https://github.com/numenta/nupic/wiki/images/nupic-core-extraction-goal.jpg >>>> >>>> We still have some open questions that we'll work out as we go along, >>>> for example: >>>> >>>> 1. Should there be different repos for language bindings AND clients? >>>> Or should they be combined in one repository? >>>> 2. How will our build automation need to change to accommodate these >>>> changes? >>>> 3. Do we include general encoders in nupic-core? (they are all >>>> currently within python) >>>> >>>> I'd like to hear opinions on this whole plan. We want nupic-core to be >>>> generic enough to provide what everyone wants, so if this plan seems to >>>> restrict your particular use-case, let's discuss it. >>>> >>>> [1] https://github.com/numenta/nupic/issues?milestone=9 >>>> [2] >>>> https://github.com/numenta/nupic/wiki/2014-January-Core-API-Meeting-Notes >>>> [3] https://github.com/numenta/nupic/wiki/NuPIC-Core-Extraction-Plan >>>> >>>> --------- >>>> Matt Taylor >>>> OS Community Flag-Bearer >>>> Numenta >>>> >>>> >>>> >>>> _______________________________________________ >>>> nupic mailing list >>>> nupic@lists.numenta.org >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>> >>>> _______________________________________________ >>> nupic mailing list >>> nupic@lists.numenta.org >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>> >>> >>> >>> _______________________________________________ >>> nupic mailing list >>> nupic@lists.numenta.org >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>> >>> >> >> >> -- >> >> Fergal Byrne, Brenter IT >> >> <http://www.examsupport.ie>http://inbits.com - Better Living through >> Thoughtful Technology >> >> e:fergalbyrnedub...@gmail.com t:+353 83 4214179 >> Formerly of Adnet edi...@adnet.ie http://www.adnet.ie >> > > > > -- > > Fergal Byrne, Brenter IT > > <http://www.examsupport.ie>http://inbits.com - Better Living through > Thoughtful Technology > > e:fergalbyrnedub...@gmail.com t:+353 83 4214179 > Formerly of Adnet edi...@adnet.ie http://www.adnet.ie > -- Fergal Byrne, Brenter IT <http://www.examsupport.ie>http://inbits.com - Better Living through Thoughtful Technology e:fergalbyrnedub...@gmail.com t:+353 83 4214179 Formerly of Adnet edi...@adnet.ie http://www.adnet.ie
_______________________________________________ nupic mailing list nupic@lists.numenta.org http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
