Hi Chris, Yes! Let's simplify this as much as possible. Do I get this correctly that change we could (if want to) get rid of release branches and just freeze master during release?
On Wed, 22 Jan 2020 at 01:14, Christopher Collins <ccoll...@apache.org> wrote: > Hello all, > > This is a follow up to an email I sent last week (subject: "Proposal: > Remove version.yml files). In that email my thoughts were somewhat... > illegible. I'm going to try again, hopefully with a bit more clarity. > > This email concerns Mynewt's repo dependency system. That is to say, > what happens when a user runs `newt upgrade`. This functionality is > burdened by two unfortunate features: > > 1. The ability to depend on a *range* of repo versions, and > 2. Mandatory `version.yml` files. > > I propose that we greatly simplify repo management by removing both of > these features. I have submitted a PR that implements this proposoal > here: https://github.com/apache/mynewt-newt/pull/365. > > Below I will describe each of these features and why I think they should > be removed. > > #### BAD FEATURE 1: RANGED DEPENDENCIES > > I'll start by giving some examples [1]. > > # Example 1 > repository.apache-mynewt-core: > type: git > vers: '>=1.6.0' > url: g...@github.com:apache/mynewt-core.git > > # Example 2 [2] > repository.apache-mynewt-nimble: > type: git > vers: '>=1.1.0 <=1.2.0' > url: g...@github.com:apache/mynewt-nimble.git > > Each dependency uses comparison operators to express a range of > acceptable versions. > > Now let me explain why I think ranged dependencies is a bad feature. > To begin with, a ranged dependency like `>=1.0.0 <2.0.0` only works if > the repo follows the SemVer model (or something like it). SemVer > requires that the major version number be bumped whenever a > compability-breaking change is made to the interface. If a repo > doesn't faithfully follow SemVer, then there's no guarantee that 1.0.1 > has the same interface as 1.0.0 (for example). Ranged dependencies > clearly aren't appropriate for such a repo. My feeling is that most > library maintainers don't follow SemVer even if they intend to. Part > of the problem is that people don't always want to bump the major > version when they ought to because to do so has certain non-technical > implications (e.g., "2.0" of something is often accompanied by a press > release or other fanfare). > > But more importantly, Mynewt is used for embedded development, and I > think most embedded developers want to know exactly which libraries > they're putting in their products. In this sense, allowing a range of > repo versions does more harm than good because it takes control away > from the developer. Ranged dependencies allow repos to unexpectedly > change versions when you run `newt upgrade`. > > Finally, use of version ranges makes the job of maintaining a Mynewt > repo more difficult. A new repo version must be tested against a range > of dependency versions rather than just one. > > So the first part of my proposal is to remove support for version > ranges in repo dependencies. A repo dependency must specify a single > version using one of three forms: > > * Fixed version (1.0.0) > * Floating version (1.6-latest) > * Commit (7088a7c029b5cc48efa079f5b6bb25e4a5414d24-commit) > (tagname-commit) > (branchname-commit) > > #### BAD FEATURE 2: `VERSION.YML` FILES > > To determine the version of a particular commit of a repo, newt checks > out that commit and reads the contents of a file called `version.yml`. > This file contains a single fixed version string. When a maintainer > releases a new version of a repo, they have to update `version.yml` > before creating the tag. > > The entire reason for this `version.yml` requirement is to support > `*-commit` dependencies. I'll give an example to illustrate what I > mean: > > * mynewt-core 1.6.0 DEPENDS ON mynewt-nimble 1.1.0 > * mynewt-core 1.7.0 DEPENDS ON mynewt-numble 1.2.0 > > Now say a user doesn't want to use a versioned release of mynewt-core. > Instead, he wants his project to use a commit of mynewt-core that is > somewhere between 1.6.0 and 1.7.0 in the git history. What version of > mynewt-nimble should this particular commit depend on? What if the > user depends on a mynewt-core commit from a completely different > branch? Newt answers these questions by reading mynewt-core's > `version.yml` file from the requested commit. If the file contains > "1.6.0", then core depends on nimble 1.1.0. If it says "1.7.0", core > depends on nimble 1.2.0, and so on. > > So that's the rationale and the history of the `version.yml` file. In > retrospect, I think this `version.yml` idea was a mistake. Arbitrary > commits are *not* versioned releases and they should not be treated > like them. When the user specifies a `*-commit` dependency he is > putting newt into a sort of "manual override" mode. Newt should not > try to figure out inter-repo dependencies in this mode. > > Another reason why `version.yml` files are bad is that they make repo > maintenance more difficult. To keep a `version.yml` file in a sane > state, a repo maintainer has to use a particular branching strategy > when preparing a new release (step 0 and step 5b here: > https://cwiki.apache.org/confluence/display/MYNEWT/Release+Process). > It's easy to get this wrong, and it is just not a very user friendly > experience for a repo maintainer. The multi-platform project MCUboot > doesn't use branches in its release process and the requirement to > maintain a `version.yml` file has been a burden on its developers. > > So the second part of my proposal is to remove the `version.yml` > requirement entirely. Newt should not try to figure out what an > arbitrary commit's "real version number" is. > > #### PROPOSAL SUMMARY > > Let me try to summarize what I've written and explore some of the > implications. > > * If `project.yml` and `repository.yml` files use versioned > dependencies only then nothing changes. > > * If someone depends on a particular commit of repo X, then X ceases to > introduce inter-repo dependencies. These dependencies must be > manually added to `project.yml`. > > * If someone depends on a particular commit of repo X, this dependency > is incompatible with all other dependencies on X. The only exception > is a dependency on the exact same commit. Newt aborts the > `newt upgrade` operation when it detects such a conflict. > > * (Old behavior but worth mentioning): If `project.yml` depends on a > version or commit of repo X, this dependency *overrides* any > inter-repo dependencies on X. > > ++++ The typical user workflow would look like this: > > 1. Start with a `project.yml` file that depends on repo versions. > > 2. If you want to depend on a specific commit of a repo, change your > `project.yml` file to use a `*-commit` dependency. If the dependee > repo has any dependencies of its own, you'll need to add these > dependencies to `project.yml` as well (assuming they aren't already > there). > > 3. Given this dependency tree: > > project.yml --> repo-X/1.7.0 --> repo-Y/1.2.0 > > If you don't want to use version 1.2.0 of repo Y (either you want a > different commit or you want to use a fork of the repo instead), then > you should change your `project.yml` file to use a `*-commit` > dependency on repo Y. > > ++++ The typical repo release process would look like this: > > 1. Create a git tag where you want the release to sit. > 2. In the master branch, modify the `repository.yml` file: > a. Add a new entry that maps a fixed version to your new tag. > b. Update any floating versions to point to the new tag. > > All comments welcome. > > Thanks, > Chris > > [1]: All of these examples are dependencies that you would find in a > `project.yml` file. Everything in this email applies equally to inter-repo > dependencies (dependencies in a `repository.yml` file). > > [2]: Furthermore, newt won't even accept multiple conditions as in example > 2r. > This is supposed to work, and it used to work, but it seems it was broken a > while back. > -- pozdrawiam Szymon K. Janc
