Hello all,
This is a follow up to an email I sent last week (subject: "Proposal:
Remove version.yml files). In that email my thoughts were somewhat...
illegible. I'm going to try again, hopefully with a bit more clarity.
This email concerns Mynewt's repo dependency system. That is to say,
what happens when a user runs `newt upgrade`. This functionality is
burdened by two unfortunate features:
1. The ability to depend on a *range* of repo versions, and
2. Mandatory `version.yml` files.
I propose that we greatly simplify repo management by removing both of
these features. I have submitted a PR that implements this proposoal
here: https://github.com/apache/mynewt-newt/pull/365.
Below I will describe each of these features and why I think they should
be removed.
BAD FEATURE 1: RANGED DEPENDENCIES
I'll start by giving some examples [1].
# Example 1
repository.apache-mynewt-core:
type: git
vers: '>=1.6.0'
url: g...@github.com:apache/mynewt-core.git
# Example 2 [2]
repository.apache-mynewt-nimble:
type: git
vers: '>=1.1.0 <=1.2.0'
url: g...@github.com:apache/mynewt-nimble.git
Each dependency uses comparison operators to express a range of
acceptable versions.
Now let me explain why I think ranged dependencies is a bad feature.
To begin with, a ranged dependency like `>=1.0.0 <2.0.0` only works if
the repo follows the SemVer model (or something like it). SemVer
requires that the major version number be bumped whenever a
compability-breaking change is made to the interface. If a repo
doesn't faithfully follow SemVer, then there's no guarantee that 1.0.1
has the same interface as 1.0.0 (for example). Ranged dependencies
clearly aren't appropriate for such a repo. My feeling is that most
library maintainers don't follow SemVer even if they intend to. Part
of the problem is that people don't always want to bump the major
version when they ought to because to do so has certain non-technical
implications (e.g., "2.0" of something is often accompanied by a press
release or other fanfare).
But more importantly, Mynewt is used for embedded development, and I
think most embedded developers want to know exactly which libraries
they're putting in their products. In this sense, allowing a range of
repo versions does more harm than good because it takes control away
from the developer. Ranged dependencies allow repos to unexpectedly
change versions when you run `newt upgrade`.
Finally, use of version ranges makes the job of maintaining a Mynewt
repo more difficult. A new repo version must be tested against a range
of dependency versions rather than just one.
So the first part of my proposal is to remove support for version
ranges in repo dependencies. A repo dependency must specify a single
version using one of three forms:
* Fixed version (1.0.0)
* Floating version (1.6-latest)
* Commit (7088a7c029b5cc48efa079f5b6bb25e4a5414d24-commit)
(tagname-commit)
(branchname-commit)
BAD FEATURE 2: `VERSION.YML` FILES
To determine the version of a particular commit of a repo, newt checks
out that commit and reads the contents of a file called `version.yml`.
This file contains a single fixed version string. When a maintainer
releases a new version of a repo, they have to update `version.yml`
before creating the tag.
The entire reason for this `version.yml` requirement is to support
`*-commit` dependencies. I'll give an example to illustrate what I
mean:
* mynewt-core 1.6.0 DEPENDS ON mynewt-nimble 1.1.0
* mynewt-core 1.7.0 DEPENDS ON mynewt-numble 1.2.0
Now say a user doesn't want to use a versioned release of mynewt-core.
Instead, he wants his project to use a commit of mynewt-core that is
somewhere between 1.6.0 and 1.7.0 in the git history. What version of
mynewt-nimble should this particular commit depend on? What if the
user depends on a mynewt-core commit from a completely different
branch? Newt answers these questions by reading mynewt-core's
`version.yml` file from the requested commit. If the file contains
"1.6.0", then core depends on nimble 1.1.0. If it says "1.7.0", core
depends on nimble 1.2.0, and so on.
So that's the rationale and the history of the `version.yml` file. In
retrospect, I think this `version.yml` idea was a mistake. Arbitrary
commits are *not* versioned releases and they should not be treated
like them. When the user specifies a `*-commit` dependency he is
putting newt into a sort of "manual override" mode. Newt should not
try to figure out inter-repo dependencies in this mode.
Another reason why `version.yml` files are bad is that they make repo
maintenance more difficult. To keep a `version.yml` file in a sane
state, a repo maintainer has to use a particular branching strategy
when preparing a new release (step 0 and step 5b here:
https://cwiki.apache.org/confluence/display/MYNEWT/Release+Process).
It's easy to get this wrong, and it is just not a very user friendly
