Steven Lau wrote:
On 08/10/09 13:55, Shawn Walker wrote:
Steven Lau wrote:
Hi,
One of our engineer found out it takes quite a while to install
OpenSolaris from ipkg.sfbay/dev IPS lately ( it took him about one
and half hours) .... so we try to setup a content mirror on our
lab. Wonder how pkg decides which IPS it will download the
packages/files from (understand it needs to go to original IPS to get
the manifest). Does it base on how close they are in term of subnet
.... or something else.
Nothing so fancy. If you have a mirror specified for a publisher,
pkg(5) will attempt to use that mirror first, always, for content.
Failing that, it will fallback to the origin.
Please note that even if you setup a content mirror, the client will
still connect to the origin server for package manifest data.
Understood. Will the pkg get 'all' the needed manifest data in one time
and then go the mirror to download all the files .... OR, it will get
one manifest data for one package and then go the mirror to get the file
... and repeats the process for each package.
Currently, the latter behaviour is what occurs, except that we don't
retrieve any files until *after* we've retrieved all the manifests. So
the sequence is more like:
* several hundred requests to origin server (for upgrading between
builds) for package manifests and a few for catalogs
* remaining requests to mirror for package content (files)
The issue is that the current dependency solver doesn't know in advance
which package manifests it needs, so it has to retrieve and load a
package manifest, look for dependencies, and then for each dependency,
retrieve and load its manifest, etc. until it is finished. This means
that for an upgrade between builds, up to 1800 requests will be made to
the origin server.
With that said, the new catalog format changes and dependency resolver
that are coming in the next few months will substantially change this.
Alternatively, if you are installing the 2009.06 release, you can use
a 'full mirror', which mirrors both manifest data and content.
Examples and instructions can be found here:
http://blogs.sun.com/observatory/entry/repo_on_a_stick
We thought about that ... but we need to install latest release from ips
for development like ipkg.sfbay/dev/
BTW, could we build a full mirror from 2009.06 ips iso and then rsync
with ipkg .... guess not .... cause no metadata sync'ed.
Correct. We hope to automate the repository ISO image process in the
future so that there is one provided with each new /dev release, so
until then, you'll have to stick with local mirrors.
A new transport subsystem is also in place in newer builds, and I
suspect that once a few issues there are resolved (almost all of them
are at this point), and improve the performance of the pkg server
infrastructure that you'll see better performance in these areas.
Cheers,
--
Shawn Walker
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
