Bart Smaalders wrote:
Brock Pytlik wrote:
Greetings all, based on a conversation with sch I had last week, I
thought I'd put together some thoughts and plans for improving
pkgsend/publication for the next release. What's below is very
preliminary, but is a high level view of what I think the necessary
steps are to get where we want to be. Please let me know what areas
I've missed in addressing or if you think a direction I've outlined
isn't useful.
Step 1) validate package on client side during close
a) check all files for "known" implicit dependencies (see below)
b) check whether the dependencies from the files/groups/users are
satisfied by the transitive closure of the declared dependencies
It would seem to me that we don't want the transitive closure here;
we want any dependencies to be directly stated in case the other
packages change dependencies later on.
Well, my thought was that in the past, we've always told people, unless
I've misunderstood, to depend on the transitive closure. I know we've
suggested that people trim their dependencies because the dependencies
will be caught by the closure. Perhaps I've misunderstood the reasoning
behind those directions before. Of course, since I can't find any of
those emails off the top of my head, here's a rough example of what I
thought we'd told people in the past. Suppose I'm publishing foo, which
needs things from the packages bar and baz. Further, suppose bar depends
on baz. I thought we'd told the people publishing foo to only depend on
bar, since baz would be picked up buy bar's dependency.
c) abort the transaction if dependencies aren't satisfied. Try to
provide the user reasonable info about what files caused the problem
and what packages would fix the issue.
Step 2) optional local validation on server
a) add option to depot like "--local-consistency"
b) after a package has been closed, its dependencies are checked
to see if the needed packages are present on the server. If they're
not, package is placed into incomplete state. When further packages
are published, incomplete packages are rechecked to see if their
dependencies have been completed.
c) This option is primarily for servers storing an entire distro
(dev, release, for example)
d) (optional) server would also check implicit dependencies like
the client does above and will place a package with an undeclared
implicit dependency into an incomplete state. Probably should only do
this if there's a way to inform the publisher as to what's happened.
I'm not sure that pure server-side validation makes a lot of sense,
but I've had this argument before ;-).
Heh, fair enough. I included it because I thought it was something we
might want. I think for a repo that's hosting an entire distro (like dev
or release), it would be a useful thing. Maybe we already have processes
in place to handle this though.
I'd rather validate as part of transferring packages from a preliminary
repo to a more stable one; this simplifies the server model. Another
example of this is how we current produce merged packages; we don't
have a special "pending" state for the single architecture packages;
we simply produce a more complex fat package by blending the
single architecture packages.
Note that the code in step 1 could be reused to do the checking needed
for transfer...
If I understand this right, the idea would be to get all the packages
onto the prelim server, then do a global check as they're transferred to
the stable one? The advantage being the lack of a "incomplete" stage on
the server?
Step 3) optional validation with other repos
a) like step 2, except that it is allowed to take other repos
into account when validating a package. An example would be extras
(or pending, or contrib) to allow those to check the existence of
their dependencies at publication time against known distro repos.
b) pkgsend would be augmented to allow a user to tell the depot
what other repos to validate against
c) depots would have a flag which determines whether someone
publishing a package to them is allowed to add additional repos to
validate against
d) depots could be optionally configured to use certain repos to
validate against
Step ?) Allow client and server to validate against an incorporation
instead of either what's installed on the client or what's present on
the server. (Labeled as ? since it could come anytime after step 1
for the client at least)
This is very useful; I'd make this step 2.
Could you say more about why this is more useful than some of the other
options?
Step ??) Allow dependencies to be expressed on files, groups, users,
and links, instead of on packages and have either the client or the
depo do the conversion at publication time. (Labeled as ?? since I'd
like to do this client side as part of step 1 but I want to keep the
wads of reasonable size and I think there may be some complixity with
this idea and creating new mutually dependent packages which express
their dependencies in this manner.) This is roughly what I think of
when we've talked about publishing against an incorporation in the past.
"known" implicit dependencies:
The first step would be to catch what I'll call #! dependencies. The
idea is to use the #! at the top of py and other similar files to
catch a dependency on python (for example) or ksh, etc.
Solaris.py does both already...
That I didn't know. I knew it did elf file checking but I'd forgotten it
handled this case too. In that case, I'll probably lift/copy/abstract
the code out so that it can be shared among pkgsend and solaris.py. (You
weren't objecting to adding this to pkgsend, right?)
The goal is to make the framework pluggable so that it's simple to
add other dependency catchers to the program. Obvious (if not easy)
ones I can think of would be library dependencies from elf files,
compression dependencies, maybe java dependencies if that's possible,
directory dependencies (?), and probably lots more that I'm not
thinking of at the moment. The goal won't be to have all those ready
for the first putback, but to have a framework in place that makes
adding them reasonably simple.
- Bart
Thanks for taking a look :)
Brock
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
