On 04/27/2017 04:04 AM, Amador Segundo wrote: > On Thu, Apr 27, 2017 at 1:07 AM, Cleber Rosa <cr...@redhat.com> wrote: >> ========================== >> RFC: Long Term Stability >> ========================== >> >> This RFC contains proposals and clarifications regarding the >> maintenance and release processes of Avocado. >> >> We understand there are multiple teams currently depending on the >> stability of Avocado and we don't want their work to be disrupted by >> incompatibilities nor instabilities in new releases. >> >> This version is a minor update to previous versions of the same RFC >> (see `Changelog`_) which drove the release of Avocado 36.0 LTS. The >> Avocado team has plans for a new LTS release in the near future, so >> please consider reading and providing feedback on the proposals here. >> >> TL;DR >> ===== >> >> We plan to keep the current approach of sprint releases every 3-4 >> weeks, but we're introducing "Long Term Stability" releases which >> should be adopted in production environments where users can't keep up >> with frequent upgrades. >> >> Changelog >> ========= >> >> Changes from `Version 3`_: >> >> * Converted formatting to REStructuredText >> * Replaced "me" mentions on version 1 changelog with proper name >> (Ademar Reis) >> * Renamed section "Misc Details" to `Deployment Details`_ >> * Renamed "avocado-vt" to "Avocado-VT" >> * Start the timeline example with version 36.0 >> * Be explicit on timeline example that a minor bug did not generate >> an immediate release >> >> Changes from `Version 2`_: >> >> * Wording changes on second paragraph ("... nor instabilities...") >> * Clarified on "Introduction" that change of behavior is introduced >> between regular releases >> * Updated distro versions for which official packages are built >> * Add more clear explanation on official packages on the various >> hardware platforms >> * Used more recent version numbers as examples, and the planned >> new LTS version too >> * Explain how users can get the LTS version when using tools such as >> pip >> * Simplified the timeline example, with examples that will possibly >> match the future versions and releases >> * Documented current status of Avocado-VT releases and packages >> >> Changes from `Version 1`_: >> >> * Changed "Support" to "Stability" and "supported" to "maintained" >> [Jeff Nelson] >> * Misc improvements and clarifications in the >> supportability/stability statements [Jeff Nelson, Ademar Reis] >> * Fixed a few typos [Jeff Nelson, Ademar Reis] >> >> Introduction >> ============ >> >> We make new releases of Avocado every 3-4 weeks on average. In theory >> at least, we're very careful with backwards compatibility. We test >> Avocado for regressions and we try to document any issues, so >> upgrading to a new version should be (again, in theory) safe. >> >> But in practice both intended and unintended changes are introduced >> during development, and both can be frustrating for conservative >> users. We also understand it's not feasible for users to upgrade >> Avocado very frequently in a production environment. >> >> The objective of this RFC is to clarify our maintenance practices and >> introduce Long Term Stability (LTS) releases, which are intended to >> solve, or at least mitigate, these problems. >> >> >> Our definition of maintained, or stable >> ======================================= >> >> First of all, Avocado and its sub-projects are provided 'AS IS' and >> WITHOUT ANY WARRANTY, as described in the LICENSE file. >> >> The process described here doesn't imply any commitments or >> promises. It's just a set of best practices and recommendations. >> >> When something is identified as "stable" or "maintained", it means the >> development community makes a conscious effort to keep it working and >> consider reports of bugs and issues as high priorities. Fixes >> submitted for these issues will also be considered high priorities, >> although they will be accepted only if they pass the general >> acceptance criteria for new contributions (design, quality, >> documentation, testing, etc), at the development team discretion. >> >> >> Maintained projects and platforms >> ================================= >> >> The only maintained project as of today is the Avocado Test Runner, >> including its APIs and core plugins (the contents of the main avocado >> git repository). >> >> Other projects kept under the "Avocado Umbrella" in github may be >> maintained by different teams (e.g.: Avocado-VT) or be considered >> experimental (e.g.: avocado-server and avocado-virt). >> >> More about Avocado-VT in its own section further down. >> >> As a general rule, fixes and bug reports for Avocado when running in >> any modern Linux distribution are welcome. >> >> But given the limited capacity of the development team, packaged >> versions of Avocado will be tested and maintained only for the >> following Linux distributions: >> >> * RHEL 7.x (latest) >> * Fedora (stable releases from the Fedora projects) >> >> Currently all packages produced by the Avocado projects are "noarch". >> That means that they could be installable on any hardware platform. >> Still, the development team will currently attempt to provide versions >> that are stable for the following platforms: >> >> * x86 >> * ppc64 >> * ppc64le >> >> Contributions from the community to maintain other platforms and >> operating systems are very welcome. >> >> The lists above may change without prior notice. >> >> Avocado Releases >> ================ >> >> The proposal is to have two different types of Avocado releases: >> >> Sprint Releases >> --------------- >> >> (This is the model we currently adopt in Avocado) >> >> They happen every 3-4 weeks (the schedule is not fixed) and >> their versions are numbered serially, with decimal digits in >> the format <major>.<minor>. Examples: 47.0, 48.0, 49.0. Minor >> releases are rare, but necessary to correct some major issue >> with the original release (47.1, 47.2, etc). >> >> Only the latest Sprint Release is maintained. >> >> In Sprint Releases we make a conscious effort to keep backwards >> compatibility with the previous version (APIs and behavior) and >> as a general rule and best practice, incompatible changes in >> Sprint Releases should be documented in the release notes and >> if possible deprecated slowly, to give users time to adapt >> their environments. >> >> But we understand changes are inevitable as the software >> evolves and therefore there's no absolute promise for API and >> behavioral stability. >> >> Long Term Stability (LTS) Releases >> ---------------------------------- >> >> LTS releases should happen whenever the team feels the code is >> stable enough to be maintained for a longer period of time, ideally >> once or twice per year (no fixed schedule). >> >> They should be maintained for 18 months, receiving fixes for major >> bugs in the form of minor (sub-)releases. With the exception of >> these fixes, no API or behavior should change in a minor LTS >> release. >> >> They will be versioned just like Sprint Releases, so looking at the >> version number alone will not reveal the differentiate release >> process and stability characteristics. >> >> In practice each major LTS release will imply in the creation of a >> git branch where only serious issues affecting users will be >> fixed. The code in a LTS branch is stable, frozen for new features. > > I'd improve the sentence "only serious issues affecting users will be > fixed". In fact, we "backport" (meaning upstream first) "any issues" > (not only the serious ones) that are possible to be fixed *and* are > affecting users. >
OK, good point. My suggestion is to use "important issues", because the definition of a "bug" and a "serious issue" is too relative. We may identify a bug in LTS that is mostly harmless, and decide to not fix it because it could require major changes. Anyway, my proposal: "In practice each major LTS release will imply in the creation of a git branch where only important issues affecting users will be fixed, usually as a backport of a fix initially applied upstream. The code in a LTS branch is stable, frozen for new features." How does that sound? > Not sure if it is also worth it to define 'backport'. Depending on the > code distance between the LTS and the master branch, it can be a > simple 'cherry-pick' or a completely different fix in terms of code, > as long as the bug gets fixed. > I'd expect readers to be familiar with the concept of backport. I'm reluctant about introducing the concept in this document. >> >> Notice that although within a LTS release there's a expectation >> of stability because the code is frozen, different (major) LTS >> releases may include changes in behavior, API incompatibilities >> and new features. The development team will make a considerable >> effort to minimize and properly document these changes (changes >> when comparing it to the last major LTS release). >> >> Sprint Releases are replaced by LTS releases. I.e., in the cycle >> when 52.0 (LTS) is released, that's also the version used as a >> Sprint Release (there's no 52.0 -- non LTS -- in this case). >> >> New LTS releases should be done carefully, with ample time for >> announcements, testing and documentation. It's recommended >> that one or two sprints are dedicated as preparations for a LTS >> release, with a Sprint Release serving as a "LTS beta" release. >> >> Similarly, there should be announcements about the end-of-life >> (EOL) of a LTS release once it approaches its 18 months of >> life. >> >> >> Deployment details >> ================== >> >> Sprint and LTS releases, when packaged, whenever possible, will be >> preferably distributed through different package channels >> (repositories). >> >> This is possible for repository types such as `YUM/DNF repos`_. In >> such cases, users can disable the regular channel, and enable the LTS >> version. A request for the installation of Avocado packages will >> fetch the latest version available in the enabled repository. If the >> LTS repository channel is enabled, the packages will receive minor >> updates (bugfixes only), until a new LTS version is released (roughly >> every 12 months). >> >> If the non-LTS channel is enabled, users will receive updates every >> 3-4 weeks. >> >> On other types of repos such as `PyPI`_ which have no concept of >> "sub-repos" or "channels", users can request a version smaller than >> the version that succeeds the current LTS to get the latest LTS >> (including minor releases). Suppose the current LTS major version is >> 52, but there have been minor releases 52.1 and 52.2. By running:: >> >> pip install 'avocado-framework<53.0' >> >> pip provide LTS version 52.2. If 52.3 gets released, they will be >> automatically deployed instead. When a new LTS is released, users >> would still get the latest minor release from the 52.0 series, unless >> they update the version specification. >> >> The existence of LTS releases should never be used as an excuse >> to break a Sprint Release or to introduce gratuitous >> incompatibilities there. In other words, Sprint Releases should >> still be taken seriously, just as they are today. >> >> >> Timeline example >> ================ >> >> Assume each sprint is taking 3 weeks. Notice how multiple LTS >> releases can co-exist before EOL: >> >> 36.0 >> * **LTS** release (the only LTS release available at the time of >> writing) >> >> 37.0 .. 49.0 >> * sprint releases >> >> 50.0 >> * sprint release >> * start preparing a LTS release, so 51.0 will be a **beta LTS** >> >> 51.0 >> * sprint release >> * **beta LTS** release >> >> 52.0 >> * **LTS** release >> * 52lts branch is created >> * packages go into LTS repo >> * both **36.x LTS** and **52.x LTS** maintained from this point on >> >> 53.0 >> * sprint release >> * bug that also affects 52.0 is found, fix gets added to master and >> 52lts branches > > Fix gets added to master and backported to supported LTS branches (at > this point, 52lts and 36lts). > Yeah, good point (assuming it also applies to 36lts, as you mentioned on the other reply). >> >> 54.0 >> * sprint release 54.0 **AND** LTS 52.1 release >> * minor bug that also affects 52.0 is found, fix gets added to >> master and 52lts branches > > Fix gets added to master and backported to supported LTS branches (at > this point, 52lts and 36lts). > >> >> 55.0 >> * sprint release >> * critical bug that affects 52.1 *only* is found, fix gets added to >> 52lts and 52.2 LTS is immediately released >> >> 56.0 >> * sprint release >> >> 57.0 >> * sprint release >> >> 58.0 >> * sprint release >> >> 59.0 >> * sprint release >> * EOL for **36.x LTS** (18 months since the release of 36.0) >> >> Avocado-VT >> ========== >> >> Avocado-VT is an Avocado plugin that allows "VT tests" to be run >> inside Avocado. It's a third-party project maintained mostly by >> Engineers from Red Hat QE with assistance from the Avocado team >> and other community members. >> >> It's a general consensus that QE teams use Avocado-VT directly >> from git, usually following the master branch, which they >> control. >> >> There's no official maintenance or stability statement for >> Avocado-VT. Even though the upstream community is quite >> friendly and open to both contributions and bug reports, >> Avocado-VT is made available without any promises for >> compatibility or supportability. >> >> When packaged and versioned, Avocado-VT rpms should be considered >> just snapshots, available in packaged form as a convenience to >> users outside of the Avocado-VT development community. Again, >> they are made available without any promises of compatibility or >> stability. >> >> * Which Avocado version should be used by Avocado-VT? >> >> This is up to the Avocado-VT community to decide, but the >> current consensus is that to guarantee some stability in >> production environments, Avocado-VT should stick to a specific >> LTS release of Avocado. In other words, the Avocado team >> recommends production users of Avocado-VT not to install Avocado >> from its master branch or upgrade it from Sprint Releases. >> >> Given each LTS release will be maintained for 18 months, it >> should be reasonable to expect Avocado-VT to upgrade to a new >> LTS release once a year or so. This process will be done with >> support from the Avocado team to avoid disruptions, with proper >> coordination via the avocado mailing lists. >> >> In practice the Avocado development team will keep watching >> Avocado-VT to detect and document incompatibilities, so when >> the time comes to do an upgrade in production, it's expected >> that it should happen smoothly. >> >> * Will it be possible to use the latest Avocado and Avocado-VT >> together? >> >> Users are welcome to *try* this combination. The Avocado >> development team itself will do it internally as a way to monitor >> incompatibilities and regressions. >> >> Whenever Avocado is released, a matching versioned snapshot of >> Avocado-VT will be made. Packages containing those Avocado-VT >> snapshots, for convenience only, will be made available in the >> regular Avocado repository. >> >> .. _Version 1: >> https://www.redhat.com/archives/avocado-devel/2016-April/msg00006.html >> .. _Version 2: >> https://www.redhat.com/archives/avocado-devel/2016-April/msg00038.html >> .. _Version 3: >> https://www.redhat.com/archives/avocado-devel/2017-April/msg00032.html >> .. _YUM/DNF repos: >> http://avocado-framework.readthedocs.io/en/latest/GetStartedGuide.html#fedora-from-avocado-s-own-repo >> .. _PyPI: https://pypi.python.org/pypi >> >> >> -- >> Cleber Rosa >> [ Sr Software Engineer - Virtualization Team - Red Hat ] >> [ Avocado Test Framework - avocado-framework.github.io ] >> [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ] >> -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ]
signature.asc
Description: OpenPGP digital signature