Comments and critiques appreciated. Substantive ones requested by COB
Wednesday.
===========================================================================
Introduction and Motivation
---------------------------
An obsolete package is a version of a package which no longer delivers any
meaningful content, compared to previous versions. Marking a package
obsolete indicates that a repo is no longer in the business of delivering
any bits under that name.
Because obsoletion is connected to a version, and each published version
can either be obsolete or not, it means that technically a package cannot
be obsolete, but a package version can be. When noting that a package
(package stem) is obsolete, a version or version range must be associated
with that information.
An obsolete package may be replaced by one or more packages; this is
treated as a rename (in the case of replacement by a single package), or a
split (in the case of multiple replacements).
At the moment, we obsolete a package by clearing out its content and
publishing that empty package so that on upgrade, all of its actions are
uninstalled from the system, even if the package itself isn't.
Package obsoletion has two primary motivations:
- User confusion when a package ceases to have content (but, due to
operational restrictions) continues to be published, empty. These
packages show up with zero bytes, no description or classification, but
are tempting to install by their names and the fact that they're
republished every build. So when nothing comes along with that
installation, it looks like a mistake on the distribution creator's
part, bugs are filed, etc.
- Ambiguity when a package is renamed to have the same substem as another
package (including possibly the original name). For instance, we want
to rename "netbeans" to "developer/netbeans", but because "pkg install
netbeans" can refer to either one, the install will fail because of the
ambiguity. If the old package name is republished as an obsolete
package, the system can resolve (or reduce) this ambiguity by only
considering the non-obsolete packages.
Mechanism
---------
In the spirit of allowing the catalog to be merely a cache of data stored
in the manifest, a package will be marked obsolete by setting the package
attribute "pkg.obsolete" to "true" in the package manifest. This action
may be filtered by a variant tag.
However, since many of the operations that will need to know about the
obsolete state of a package shouldn't have to parse many manifests to get
at it, this state should be maintained in the catalog as well. This
implies that the publication server will need to parse the manifest more
intelligently than it does currently in order to extract this information.
No change to the catalog format or to the /catalog operation is required.
Instead of a "V" tag for an FMRI, an obsolete package version will be
marked as "O". The upcoming key/value-pair based catalog format should be
able to handle obsolete package simply as well, likely tagging the fmri
with "obsolete=true" just as in the manifest.
Contents of an Obsolete Package
-------------------------------
Package rename (and split) is done by having dependencies in the package
pointing to the replacements. Thus the package is not completely empty, as
it may have depend actions.
Obsolete packages should also have metadata attached to them, allowing them
to describe themselves, or even have search terms associated with them.
Information about what the package was is still useful information, as long
as it's clear that this particular version is obsolete. This obviously
requires set actions to be delivered with the package.
No other data or metadata is required or desired for obsolete packages, so
no other action types should be allowed in an obsolete package. Note,
though, that a package may occasionally be obsoleted for one variant, but
not another (on sparc, say, but not x86), so technically any given variant
should be empty except for depend and set actions, but not all variants may
be.
Checks should be performed at both publication and installation time that
this constraint holds true. It wouldn't be unreasonable for a client to
fail when installing such a package, as it's an indication of poor package
construction, and is likely a bug in same.
Behavior
--------
In the simple case, where a package is obsoleted without an associated
rename, and nothing on the system depends on it, the package will be
uninstalled on upgrade, and on initial install will emit a message, but not
fail the installation.
The state of the package will be "obsolete" in both "pkg list" and "pkg
info" output, though because at present "pkg info" fails when given a
package name that's not installed, "pkg info -r" will be required to get
this information.
When an obsolete package has dependencies, the dependencies will be
followed and installed as usual. The obsolete package will follow the
behavior it would if it didn't have dependencies (uninstall on upgrade, no
install on install).
A package can be constrained at an obsolete version.
A package can be unobsoleted by publishing a new version of the package
which doesn't have the pkg.obsolete attribute.
Behavior: Dependencies on Obsolete Packages
-------------------------------------------
A package can have a require-type dependency on an obsolete package, as
different distributions may skew in their understanding of the current
package namespace. For example, a blastwave package may depend on SUNWcsl,
even though SUNWcsl has been split up and renamed into many other packages.
Doing otherwise would be highly unfriendly.
In this case, the obsolete package should not be uninstalled, for
performance reasons, as the dependencies other packages have on it will
need to appear to be satisfied (via, say, "pkg verify"). This could be
made unnecessary with the new catalog format, but a different codepath for
dependency verification may not be desired. Some convergence will be
necessary here.
The case of a package depending on an obsolete package which itself has no
dependencies (ie, is EOLed) is more problematic. This means that the
functionality delivered by the obsoleted package has gone away entirely,
but another package is still presumably depending on that functionality.
In this case, the system should prevent either the upgrade to the obsolete
version of the package, or prevent the installation of the dependent
package (depending on the scenario at hand).
There are other, less palatable options to resolve the breakage:
- Uninstall the dependent packages. This is undesirable, since otherwise
desired functionality is removed from the system without the express
consent of the user.
- Mark the package obsolete, but don't uninstall it, or remove any of its
contents from the system until its dependents are removed. The
maintenance of this state, particularly over a longer period of time,
may be quite difficult. Still, aside from the management issues, it
may be the most user-friendly option.
If the system is capable of finding a non-obsolete version of the package
which satisfies all other system constraints, then this version should be
chosen instead. This will require the SAT solver to be available.
Other behaviors are enumerated in Test Cases, below.
Publication Changes
-------------------
As mentioned above, the obsolete attribute will need to make its way into
the catalog, so the publication server code will need to parse the manifest
looking for this attribute. Given the upcoming catalog format changes
putting dependency information into the catalog, this won't present any
extra burden on publication.
Given the problems with depending on an obsolete package, the server should
not allow such dependencies. This should be thought of as a distro-wide
(or at least repo-wide) flag day. In the case that a package is being
renamed, the dependent packages should be changing their depend actions to
match the new name, and in the case that a package is simply being EOLed,
any other package depending on it should simply be treated as a bug.
- server should expand dependencies to unambiguous stems (or fail add?)
- the server might want to refuse publication of an obsolete package with
adjacent versions (no non-obsolete version "in between")
OpenSolaris Operational Changes
-------------------------------
re-publication of obsolete packages, inclusion in entire, etc.
Currently, we clear out a package's import file on obsoletion, leaving a
comment behind explaining what we've done and why. This will be replaced
by adding a new keyword for solaris.py to interpret: obsolete. It will
take no arguments, and it simply adds "set name=pkg.obsolete value=true" to
the manifest, while allowing only other set actions and depend statements
to be added. Note that because it won't be importing any SVr4 packages, we
will have to add by hand the metadata that would otherwise have been
populated from the pkginfo file (summary, description).
We also republish the empty packages each version after they've been
emptied. There's no particular reason for that now, and we should
definitely stop when we're publishing properly obsolete packages.
In addition, obsolete packages should not, generally, be constrained by
entire or any other large-scale incorporations. Doing so would be rather
unfriendly:
- if an unbundled package A depends on a WOS package B which gets
obsoleted in build X, then upgrading through build X would fail due to
this constraint (because A would no longer have its dependency
satisfied). Without the constraint, it would be allowed for the system
to stay on the latest non-obsolete version of the package (or at least
the one that was already installed on the system), which is probably
what the user wants. This wouldn't be the normal course, at least from
the first -- since package B is now unconstrained, it will be upgraded
to the latest, obsolete version, and so be removed anyway, but the
system could be enhanced later to allow it to remain on the system.
- If a leaf package got EOLed, a user might be annoyed that it was
removed from the system -- perhaps they'd rather keep the latest (or
installed) version around.
In general, doing the right thing for these situations involves
understanding user intent, which we do not currently capture.
In the case of an obsolete package undergoing a rename, it might make some
sense to constrain it, as if we decide to change the list of replacement
packages, making sure that the list is correct for any particular build (or
other grouping abstraction we might use) could use that mechanism.
However, if the obsolete package depended on specific versions of its
replacements, then those versions could be appropriately constrained.
Getting the correct version of the obsolete packages to get the right
replacements would depend on the SAT solver. If that isn't available, then
we could choose to continue incorporating (though at the original obsolete
version) the obsolete package only if it has replacements.
Test Cases
----------
Legend:
0 No package (transition from/to implies install/uninstall)
-> transition
A package "A"
Ao obsolete package "A"
A' new version of package "A"
A (-> B) A depends on B
A (<- B) B depends on A
1 install an obsolete package
0 -> Ao
effectively a no-op (Ao is not on the system)
2 install a package with a dependency on an obsolete leaf package
0 -> A (-> Bo)
fail
3 install a package with a dependency on a renamed package
0 -> A (-> Bo (-> C))
A and C are installed, Bo is introduced as empty package for
bookkeeping
4 install an obsolete package with a dependency on another package (rename)
0 -> Ao (-> B)
package B is installed, Ao is not on the system
5 upgrade a package to an obsolete version
A -> Ao
package A is removed (Ao is not on the system)
6 upgrade a package to a version with a dependency on an obsolete leaf package
A -> A' (-> Bo)
fail
A (-> B) -> A' (-> Bo)
fail (or not; equivalent to 8b, see comment in 8a)
7 upgrade a package to a version with a dependency on a renamed package:
A -> A' (-> Bo (-> C))
A' and C are installed, Bo is introduced as empty package for
bookkeeping
A (-> B) -> A' (-> Bo (-> C))
A' and C are installed, B is emptied out and marked obsolete for
bookkeeping
A (-> B) -> A (-> Bo (-> C))
[image-update only] C is installed, B is emptied out and marked
obsolete for bookkeeping
8 upgrade a package to an obsolete leaf version when another depends on it
A -> Ao (<- B)
fail (maybe not: this could wedge an image-update when an unbundled
package depends on a package that's getting obsoleted)
A (<- B) -> Ao (<- B')
ditto; equivalent to 6b
9 upgrade a package to an obsolete version with dependencies
A -> Ao (-> B)
package B is installed, package A is removed (see 7[ab] for cases
where it would stick around)
10 upgrade a package to a version on which an obsolete package has a
dependency (this doesn't seem terribly interesting).
A -> A' (<- Bo)
11 install an ambiguous name where only one match is non-obsolete
12 upgrade a package to an obsolete version with a dependency on a package
which has a (potentially) ambiguous name
13 Publishing an obsolete package with a dependency on an obsolete package
doesn't make sense -- fail publication?
A (-> B) -> Ao (-> Bo)
Packages A and B are removed.
A (-> B (-> C)) -> A' (-> Bo (-> Co))
Package A' is installed, package C is removed, package B is emptied
out and marked obsolete for bookkeeping
14 upgrade a package to an obsolete version, and then upgrade/install to a
resurrected version.
A -> Ao -> A'
A should be removed, and then added (by explicit install, not upgrade)
Specific Examples
-----------------
- system has "netbeans" on it; publish "developer/netbeans" and do an
"image-update" or "install netbeans". This will check both for the
netbeans switch, but also that the dependency libnb-xml switches over,
too. (11, 12)
- install a package that's going to be obsoleted (SUNWbrasero). publish
the obsolete version. image-update or re-install the package, and make
sure it goes away. (1, 5)
- install a package that's going to be obsoleted and replaced
(SUNWgnome-dtstart -> SUNWdesktop-startup). Publish the new set of
packages, image-update or re-install, and see that it goes away. (4, 9)
- install a package that's going to be obsoleted and a package that
depends on it. Publish the obsolete version. image-update or
re-install, and see that it fails. (Can't find an example!) (6)
- ditto, but the obsolete package has dependencies -- is getting renamed
(SUNWdbus-bindings -> SUNWdbus-glib+SUNWdbus-python24, SUNWcheese).
The update/install should succeed. (7)
- Make sure that SUNWfirefox-apoc-adapter goes from installable to
obsolete to installable. (EOFed in 95, resurrected in 116.)
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
