Problem:
Without knowing the users intent, pkg(5) is unable to meet their
expectations for packaging behavior. Probably the most discussed example
is group/meta packages. A simple example is that installing ss-dev, then
uninstalling ss-dev doesn't return the machine to the state it was in
prior to the install (bug 1728). To properly handle package resurrection
once the package obsoletion work is in, user intent is needed so that
pkg can determine whether the resurrected package was removed from the
system because it was obsolete or because the user had previously
removed it. Tracking user intent will also allow pkg to determine
whether they explicitly installed it from a specific repository or not.
That information would allow us to perhaps improve the upgrade the logic
when a package is delivered from multiple (non-preferred) publishers. It
could also us to improve the current situation where upgrading from a
fresh install of 111b to 117 results in a different set of packages
installed on the system then a fresh install of 117 because slim_install
has been removed from the system.
================================================================================
Proposal:
We'll make use of the existing structure of var/pkg/state. In addition
to var/pkg/state/installed, var/pkg/state/uninstalled will be created.
The files in these directories, one per package stem, will be augmented
to contain the intent information.
I'm deliberately not making use of history as an already existing store
for one substantial reason, history can be purged. I believe that we
want user intent to be persistent and unable to be purged (or, at least
that purging it should make the consequences very clear) while purging
pkg history appears to be there for a UI need.
As alternatives we could:
Create var/pkg/user_intent, and store data there rather in in
var/pkg/state/*
Instead of using a directory, create a single file and update it,
similar to the catalog file.
Augment the catalog file with the user intent information.
The on-disk record of intent will be written after the bits have been
successfully written to disk, at the same time the history information
is recorded. As far as I can tell, the information needed is "did the
user explicitly install this package" and "did the user explicitly
choose a publisher." These will be the two bits of information recorded
initially. The data will be stored as key-value pairs, in this case
"user_installed=True/False" and "publisher_chosen=P_<name of
publisher>/None". When asking about user intent, the consumer/client
will either receive a value or a response of "Unknown."
In general, tracking user intent should be transparent to both the user
and the package publisher, though I think there are special cases we
want to account for. In general, updating a piece of software will not
be regarded as signifying intent. If a security patch is available for a
library, typing pkg install <lib> shouldn't result in that package being
marked as intentionally installed. Thus, only the initial install of a
package will typically set the user_installed value. There will be times
though that a user might want to explicitly add, change, or remove, the
intent flags for a package. For example, (moving forward to a future
where the group package work has been done) a user might install
amp-dev, discover that they want to keep mysql, but not the other pieces
it delivered. Installed of removing the pieces individually, or removing
all of amp-dev and reinstalling mysql only, they could mark mysql as
intentionally installed then remove amp-dev and get a system with only
mysql installed.
Because user intent is tracked by package name and packages can be
renamed, split, and merged, user intent should take into account these
changes. By default, intent will not be propagated across dependencies.
This means that the libraries a user pulls in when they do pkg install
firefox won't be marked as being user_installed. It also means that when
a user installs amp-dev, none of its dependencies are marked with user
intent. The one exception to this rule is dependencies of obsolete
packages. Specifically, if a package is marked as obsolete, the user
intent for that package will be propagated to its dependencies. This is
done to handle the situation where the user explicitly installs foo, foo
is obsoleted and renamed to bar, and bar is a dependency of baz. If baz
is removed, bar should still remain on the system (again, jumping ahead
to the future where group packages have been implemented.)
When packages merge, the user intent of the merged package will be
handled on a key by key basis. For user_installed, the merged package
will have user_installed=True if any of its component packages were
marked as such. For publisher_chosen, matching values will be simply
propagated. If all but one of the packages specify None, and the other
names a specific publisher, that publisher will be propagated. I'm not
certain what the right answer to conflicting named publishers is except
to either error out, or set the value to None.
When packages split, by default the user intent will propagated to all
the new packages. However, the package publisher will be allowed control
over this. By adding a tag to the dependency action in the now obsolete
package, the packager can tell pkg not to propagate user_intent along
particular dependencies. An example of where this would be useful would
be a package foo which contains an application and a set of libraries.
The packager decides split the package to foo-app and foo-lib, and only
wants user intent to be propagated to the foo-app package. There are
situations where this would not do what the end-user expected,
specifically, the developer who installed foo only for access to
foo-lib, but that should be a small minority of users compared to those
whose expectations would be meant with this mechanism.
It's also likely that we would want to allow a package to specify that
certain dependencies should have install intent propagated. The example
I have at the moment is slim_install. By specifying that the
dependencies on firefox, thunderbird, and the other user facing packages
should have user intent propagated, reasonable defaults will appear on
the installed system.
One question that's somewhat open at the moment is whether we should
look to have this tracked going forward, or whether we should use the
information contained (hopefully) in the users pkg history files. From
an initial investigation, it appears that if a user hasn't purged their
history, all the information desired will be there from after the
initial install. Because of this, I propose that the first time that pkg
notices an expected piece of intent information isn't available, it
reads through the history file and gathers all of the user intent
information that's possible to extract. It will then dump the intent it
could find into the appropriate places for all the installed packages
(and any which were explicitly uninstalled). Because the last thing
that's done when creating the install CD is to purge the package
history, the information for a large number of packages won't be
available. To solve this, I propose that SUNWipkg delivers a static file
containing the default state for packages that are part of the live-cd.
If pkg is unable to determine an existing packages user intent from the
history, it will fall back to the file to set the intent. If the package
isn't in the file, then it will default to setting user_installed=Unknown.
Work will also need to be done with the installer group so going
forward, the intent that appears on a freshly installed system will be
the desired state. This should consist mostly of deciding what the
desired state is, setting the dependency tags appropriately, and perhaps
using appropriate flags when installing and uninstalling their proto area.
----------------------------------------------------------------------------------------------------
Implementation plans:
I think we've thought that user intent should be done by whoever the
first consumer ends up being. Looking at the above proposal though, I
think it's a large enough amount of work that it is reasonable to do it
separately. Further, the sooner we start tracking user intent, the more
we'll be able to do once consumers start appearing.
Within this work, I think there are further divisions that can be
reasonably made:
1) add intent tracking from this moment forward and get the storage
infrastructure in place and work with the install team so that newly
installed systems either have empty user intent or the desired user intent
2) Add the historical inference of user intent as well as the default
info for packages.
3) Give packagers the ability to control the propagation of user intent
4) Give users the ability to change the intent for a package
5) If for expedience, the decision in step 1 was to have the installer
provide empty user intent, then fix that so appropriate intent is delivered.
Those are my current plans and thoughts. I'd appreciate any comments or
suggestions.
Thanks,
Brock
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
