On 03/13/12 09:48, Bart Smaalders wrote:
On 03/08/12 14:10, Brock Pytlik wrote:
On 03/08/12 09:25, Shawn Walker wrote:
On 03/05/12 12:00, Brock Pytlik wrote:
[snip]
Right now we rely on being able to publish multiple packages
at once in ON to get satisfactory performance. Can we do
the checking single threaded, but actually publish in parallel?
Here's the real constraint, once a package 'pkg:/foo' has been compared
to the package 'pkg:/foo' to be published, no other process should start
comparing or processing a package with that name (in this case, foo)
until the new version has been published.
So, in the RE/ON case, it should be fine as long as two different people
don't attempt to publish to a repo at the same time. Looking back at the
steps,
1) pkgdep generate can be done in parallel
2) pkgdep resolve -P has to be done single threaded
3) pkgsend fill can be done in parallel
5) comparing the manifests can be done in parallel (whether it's done in
parallel or not, the repo's catalog must be unchanging at this point)
6) if the output of comparing the manifests is stored appropriately,
then replacing @current with the appropriate versions could be done in
parallel once everything in step 5 had finished. I'm not sure parallel
would be a huge win here because there would be a substantial bit of
duplicated work, but it's worth looking into at least.
7) collapsing and coalescing the dependencies can be done in parallel
8) the second round of comparisons can also be done in parallel (Again
though, the repo's catalog must still be unchanging)
9) actually publishing the manifests can be done in parallel though
(though only for this set of packages, not for some other set of
packages someone else was working on, and the catalog at this point
needs to be the same as what it was at step 5)
For simplicity and my own sanity, I've assumed a single, repo level,
"lock." In actuality, as long as the set of packages that person A and
person B want to publish are completely disjoint, they could both go
through the above steps at the same time. Personally, I'd see that as
more trouble than it's worth, but implementing that could be done.
Hth,
Brock
- Bart
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
