Re: [DISCUSSION] Moving NimBLE to separate project
Hi Szymon, On Thu, Mar 15, 2018 at 12:57 PM, Szymon Jancwrote: > Hi Andrzej, > > On Monday, 12 March 2018 15:30:52 CET Andrzej Kaczmarek wrote: >> Hi, >> >> I created PR for apache-mynewt-core which removes NimBLE code and adds >> dependency to apache-mynewy-nimble repository: >> https://github.com/apache/mynewt-core/pull/907 >> >> With latest changes in newt (thanks, Chris!) transition to new >> structure is automatic - newt update will download new repository and >> newt build will pick up all dependencies. >> >> apache-mynewt-nimble repository is basically rewritten history of >> apache-mynewt-core with only commits touching NimBLE code included + >> some directory structure reorg. Each new commit has id of original >> commit added in commit message to make lookups easier. > > Since it looks like community agree with this proposal lets get this PR merged > so that it can get some testing before next release. All right, I'll update PR and apache-mynewt-nimble repository to latest NimBLE code today evening (let's say not before 8pm CET / 12pm PST) and so we can merge it. If anyone disagrees, you still have few hours to let us know :-) > -- > pozdrawiam > Szymon Janc Best regards, Andrzej
Re: [DISCUSSION] Moving NimBLE to separate project
On Tue, Feb 27, 2018 at 08:39:04AM -0800, Christopher Collins wrote: > I agree with others that the best option is 2a > (@apache-mynewt-core/net/nimble* become empty packages that pull in the > external nimble packages). However, newt doesn't currently support repo > dependencies; if a repo is not specified in the user's `project.yml` > file, then newt won't download it. [...] I have submitted a PR which fixes repo dependencies: https://github.com/apache/mynewt-newt/pull/140. This PR changes the `repository.deps` format to the one I used in my previous email. I thought this was a good idea because it eliminates the need to repeat a depended-on repo's git information for each version. > > I've been looking at repo dependencies in newt, and it appears to be a > pretty complex feature. I don't think we can expect to have this > working for 1.4.0, but maybe there are some compromises we can make to > implement a simplified model that solves the nimble case. I'll explain > why I think this is complicated to solve below. If you aren't > interested in newt particulars, feel free to stop reading now :). > > Here is how apache-mynewt-core's `repository.yml` file might look like > this after the nimble and nffs dependencies are added (I've made some > syntax changes for readability): > > repository.deps: > mynewt-nimble: > type: git > url: 'g...@github.com:apache/mynewt-nimble.git' > vers: > master: 0-dev > mynewt_1_4_0_tag: >=1.4.0 > > mynewt-nffs: > type: git > url: 'g...@github.com:apache/mynewt-nffs.git' > vers: > master: 0-dev > mynewt_1_4_0_tag: 1.4.0 > > What this says is: > * The master branch of apache-mynewt-core depends on: > o nimble 0-dev > o nffs 0-dev > * The mynewt_1_4_0_tag tag of apache-mynewt-core depends on: > o nimble 1.4.0 or later > o nffs 1.4.0 only > > The key points are: > * A repo's dependency tree may differ with version (branch). > * A dependency can specify a range of dependant repos (e.g., `>=`). > > I think this is a good design, and I am not suggesting changing it. > Some aspects of this are just quite complicated. In particular, if the > user wants to upgrade some repos, calculating the versions to upgrade to > is a bit tricky. Newt solves the similar problem of determining which > packages to include during a build operation, but there is a fundamental > difference: for packages, there is only a single dependency graph. For > repos, on the other hand, there are potentially many separate graphs, > each of which needs to be evaluated to detect conflicts. Furthermore, > each graph may differ quite substantially, as some secondary dependency > may add a tertiary dependency for certain versions (and so on and so > on). The number of potential graphs is quite large. Consider a project > with 10 repos, each of which specifies 10 versions in its > `repository.yml` file. Ignoring some details, there are potentially > 10^10 (10 billion) graphs that would need to be evaluated to find the > one combination that doesn't have any conflicts. Furthermore, producing > a helpful error message when there is no suitable combination of repo > versions is quite a challenge in itself. > > Linux distributions that use a rolling release model solve this problem, > so it certainly is possible to implement. However, it is going to > require a lot of tricky code. > > Chris > > On Tue, Feb 27, 2018 at 09:44:37AM +0100, Szymon Janc wrote: > > Hi, > > > > 1. I'm fine with doing it either before or after 1.4. We just need to > > make sure update works correctly for 1.4 if we do it before. > > 2. I'd go with 2a as Will, no need to keep duplicated code around. > > 3. Agree with Will, lets start with 1.0 NimBLE release. Only the > > first release needs to be coordinated with -core anyway. > > > > > > On 24 February 2018 at 01:07, will sanfilippowrote: > > > My thoughts: > > > > > > 1. Since the 1.4 release is coming up very quickly I would do this for > > > 1.5 (personally). > > > 2. I would choose 2a. > > > 3. Seems a bit confusing to me to label nimBLE releases with same number > > > as Mynewt releases. Why not just make the first stable release of nimBLE > > > 1.0? Not a big deal either way but since they are going to diverge > > > eventually. > > > > > > Will > > >> On Feb 22, 2018, at 1:01 AM, Andrzej Kaczmarek > > >> wrote: > > >> > > >> Hi all, > > >> > > >> As some of you may already noticed, there is apache/mynewt-nimble > > >> repository created where NimBLE code was pushed along with some extra > > >> changes, most notably initial attempt to create port of NimBLE for > > >> FreeRTOS, but other platforms will be supported as well (native Linux > > >> port > > >> is also prepared). > >
Re: [DISCUSSION] Moving NimBLE to separate project
I agree with others that the best option is 2a (@apache-mynewt-core/net/nimble* become empty packages that pull in the external nimble packages). However, newt doesn't currently support repo dependencies; if a repo is not specified in the user's `project.yml` file, then newt won't download it. I've been looking at repo dependencies in newt, and it appears to be a pretty complex feature. I don't think we can expect to have this working for 1.4.0, but maybe there are some compromises we can make to implement a simplified model that solves the nimble case. I'll explain why I think this is complicated to solve below. If you aren't interested in newt particulars, feel free to stop reading now :). Here is how apache-mynewt-core's `repository.yml` file might look like this after the nimble and nffs dependencies are added (I've made some syntax changes for readability): repository.deps: mynewt-nimble: type: git url: 'g...@github.com:apache/mynewt-nimble.git' vers: master: 0-dev mynewt_1_4_0_tag: >=1.4.0 mynewt-nffs: type: git url: 'g...@github.com:apache/mynewt-nffs.git' vers: master: 0-dev mynewt_1_4_0_tag: 1.4.0 What this says is: * The master branch of apache-mynewt-core depends on: o nimble 0-dev o nffs 0-dev * The mynewt_1_4_0_tag tag of apache-mynewt-core depends on: o nimble 1.4.0 or later o nffs 1.4.0 only The key points are: * A repo's dependency tree may differ with version (branch). * A dependency can specify a range of dependant repos (e.g., `>=`). I think this is a good design, and I am not suggesting changing it. Some aspects of this are just quite complicated. In particular, if the user wants to upgrade some repos, calculating the versions to upgrade to is a bit tricky. Newt solves the similar problem of determining which packages to include during a build operation, but there is a fundamental difference: for packages, there is only a single dependency graph. For repos, on the other hand, there are potentially many separate graphs, each of which needs to be evaluated to detect conflicts. Furthermore, each graph may differ quite substantially, as some secondary dependency may add a tertiary dependency for certain versions (and so on and so on). The number of potential graphs is quite large. Consider a project with 10 repos, each of which specifies 10 versions in its `repository.yml` file. Ignoring some details, there are potentially 10^10 (10 billion) graphs that would need to be evaluated to find the one combination that doesn't have any conflicts. Furthermore, producing a helpful error message when there is no suitable combination of repo versions is quite a challenge in itself. Linux distributions that use a rolling release model solve this problem, so it certainly is possible to implement. However, it is going to require a lot of tricky code. Chris On Tue, Feb 27, 2018 at 09:44:37AM +0100, Szymon Janc wrote: > Hi, > > 1. I'm fine with doing it either before or after 1.4. We just need to > make sure update works correctly for 1.4 if we do it before. > 2. I'd go with 2a as Will, no need to keep duplicated code around. > 3. Agree with Will, lets start with 1.0 NimBLE release. Only the > first release needs to be coordinated with -core anyway. > > > On 24 February 2018 at 01:07, will sanfilippowrote: > > My thoughts: > > > > 1. Since the 1.4 release is coming up very quickly I would do this for 1.5 > > (personally). > > 2. I would choose 2a. > > 3. Seems a bit confusing to me to label nimBLE releases with same number as > > Mynewt releases. Why not just make the first stable release of nimBLE 1.0? > > Not a big deal either way but since they are going to diverge eventually. > > > > Will > >> On Feb 22, 2018, at 1:01 AM, Andrzej Kaczmarek > >> wrote: > >> > >> Hi all, > >> > >> As some of you may already noticed, there is apache/mynewt-nimble > >> repository created where NimBLE code was pushed along with some extra > >> changes, most notably initial attempt to create port of NimBLE for > >> FreeRTOS, but other platforms will be supported as well (native Linux port > >> is also prepared). > >> > >> The problem is that this repo is now not synced with apache/mynewt-core and > >> having two repositories with the same code is troublesome so we'd like to > >> end development of NimBLE code in core repository and move it entirely to > >> nimble repository. There are three open points on how this should be done: > >> 1. When to do this switch? Before 1.4 release or after it? > >> 2. How to deal with NimBLE in core repository? > >> 3. How to manage NimBLE releases? > >> > >> My proposals are as follows: > >> > >> 2a. Remove NimBLE code from mynewt-core repository leaving only packages > >> with
Re: [DISCUSSION] Moving NimBLE to separate project
Hi, 1. I'm fine with doing it either before or after 1.4. We just need to make sure update works correctly for 1.4 if we do it before. 2. I'd go with 2a as Will, no need to keep duplicated code around. 3. Agree with Will, lets start with 1.0 NimBLE release. Only the first release needs to be coordinated with -core anyway. On 24 February 2018 at 01:07, will sanfilippowrote: > My thoughts: > > 1. Since the 1.4 release is coming up very quickly I would do this for 1.5 > (personally). > 2. I would choose 2a. > 3. Seems a bit confusing to me to label nimBLE releases with same number as > Mynewt releases. Why not just make the first stable release of nimBLE 1.0? > Not a big deal either way but since they are going to diverge eventually. > > Will >> On Feb 22, 2018, at 1:01 AM, Andrzej Kaczmarek >> wrote: >> >> Hi all, >> >> As some of you may already noticed, there is apache/mynewt-nimble >> repository created where NimBLE code was pushed along with some extra >> changes, most notably initial attempt to create port of NimBLE for >> FreeRTOS, but other platforms will be supported as well (native Linux port >> is also prepared). >> >> The problem is that this repo is now not synced with apache/mynewt-core and >> having two repositories with the same code is troublesome so we'd like to >> end development of NimBLE code in core repository and move it entirely to >> nimble repository. There are three open points on how this should be done: >> 1. When to do this switch? Before 1.4 release or after it? >> 2. How to deal with NimBLE in core repository? >> 3. How to manage NimBLE releases? >> >> My proposals are as follows: >> >> 2a. Remove NimBLE code from mynewt-core repository leaving only packages >> with dependencies to mynewt-nimble repository. The process of upgrading to >> new version should be as easy as doing 'newt upgrade' to fetch newt >> repository, assuming there are no local changes to NimBLE code. This is >> preferred option. >> 2b. Leave NimBLE code at its current state in mynewt-core and use it by >> default for next release, with option to use mynewt-nimble instead. This is >> safe option and can be also applied before 1.4 release. >> >> 3. NimBLE has its own releases, depending on needs, and Mynewt will use >> latest stable release of NimBLE at all time. First release of NimBLE will >> be synced with Mynewt release, I guess we can call it the same as Mynewt >> release and then start independent releases with 2.0. For those who would >> like to use latest NimBLE, of course it would be just a matter of switching >> repository version manually. >> >> Any thoughts on this? >> >> Best regards, >> Andrzej > -- pozdrawiam Szymon K. Janc
Re: [DISCUSSION] Moving NimBLE to separate project
My thoughts: 1. Since the 1.4 release is coming up very quickly I would do this for 1.5 (personally). 2. I would choose 2a. 3. Seems a bit confusing to me to label nimBLE releases with same number as Mynewt releases. Why not just make the first stable release of nimBLE 1.0? Not a big deal either way but since they are going to diverge eventually. Will > On Feb 22, 2018, at 1:01 AM, Andrzej Kaczmarek >wrote: > > Hi all, > > As some of you may already noticed, there is apache/mynewt-nimble > repository created where NimBLE code was pushed along with some extra > changes, most notably initial attempt to create port of NimBLE for > FreeRTOS, but other platforms will be supported as well (native Linux port > is also prepared). > > The problem is that this repo is now not synced with apache/mynewt-core and > having two repositories with the same code is troublesome so we'd like to > end development of NimBLE code in core repository and move it entirely to > nimble repository. There are three open points on how this should be done: > 1. When to do this switch? Before 1.4 release or after it? > 2. How to deal with NimBLE in core repository? > 3. How to manage NimBLE releases? > > My proposals are as follows: > > 2a. Remove NimBLE code from mynewt-core repository leaving only packages > with dependencies to mynewt-nimble repository. The process of upgrading to > new version should be as easy as doing 'newt upgrade' to fetch newt > repository, assuming there are no local changes to NimBLE code. This is > preferred option. > 2b. Leave NimBLE code at its current state in mynewt-core and use it by > default for next release, with option to use mynewt-nimble instead. This is > safe option and can be also applied before 1.4 release. > > 3. NimBLE has its own releases, depending on needs, and Mynewt will use > latest stable release of NimBLE at all time. First release of NimBLE will > be synced with Mynewt release, I guess we can call it the same as Mynewt > release and then start independent releases with 2.0. For those who would > like to use latest NimBLE, of course it would be just a matter of switching > repository version manually. > > Any thoughts on this? > > Best regards, > Andrzej