-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Emma Strubell wrote: > Zac Medico wrote: >> The way that I imagine the "cache sync" idea should be implemented >> is like paludis's "unavailable repository" which uses of tarball to >> distribute package metadata[1]. The tarball approach that they use >> seems pretty reasonable. However, it would probably also be nice to >> be able to use a protocol such as rsync to download the >> metadata/cache/ directory from the same URI which is used to fetch >> the ebuilds themselves (maybe paludis supports this already, I don't >> know). > > You're offering two different ideas here, right? The "unavailable > repository" method, and the method using the metadata/cache/ > directory?
My idea was that both methods could be used interchangeably. This would allow you to configure an "unavailable repository" using any repository which provides a metadata/cache/ directory. Such repositories aren't common now, but they should become more common in the future, after people start using the egencache program that I'm working on. > If so, it makes sense to me to take the metadata/cache/ directory > route, since, as you said, multiple repositories aren't yet supported > in portage. At first I was thinking I could contact the guy who might > be working on multiple repository support this summer and work with > him to some extent... but the "unavaliable repository" solution would > basically be dependent on/building off of multiple repository support, > and it seems like building off of something that isn't fully built > would be a bad idea. Right. If you wanted to submit a competing "multiple repository support" soc proposal yourself then you might list the "unavaliable repository" thing as one of your goals. However, that might be a little over-ambitious since "multiple repository support" alone would provide enough work for a soc project. > And to clarify: the goal of the project is to modify portage so that > instead of fetching all of the ebuilds in the portage tree (or in an > overlay) upon a sync, portage only fetches the metadata and cache info > (via the metadata/cache/ directory) of the tree, and the ebuilds of > packages that are already installed (packages found in the world > file?) And then, additional ebuilds would be fetched only when they > are needed? The problem with fetching the ebuilds separately is that the remote repository might have changed. So, it's not a very reliable approach unless there is some kind of guarantee that the remote repository will provide a window of time during which older ebuilds that have already been removed from the main tree can still be downloaded. In order to accomplish this, you'd essentially have to devise a new source package format which can be downloaded as a single file (something like a source rpm file that an rpm based distro would provide). It would be a major change in the way that our source packages are distributed and I'm not sure that it's really worth the trouble (I touched on this in a reply on the soc list [1]). I tend to think that it's better to simply keep the ebuild format as it is, and sync the ebuilds at the same time as the cache. Note that our binary package format is already suitable for the "download cache first and package later" approach. A metadata cache is automatically generated and saved as $PKGDIR/Packages. If you share $PKGDIR via via http or ftp then clients can configure PORTAGE_BINHOST in make.conf to download packages when emerge's --getbinpkg option is specified. > Or will only metadata/cache/ be fetched upon sync, and > then all ebuilds will be fetched only when they are needed? Am I > completely oversimplifying the project? As mentioned above, the current ebuild format and the way that ebuilds are distributed is not suited to the "download cache first and package later" approach since the ebuild might have been removed before it could be downloaded. However, the current ebuild format is suitable for the "unavaliable repository" approach since that approach only gives you a preview of the available packages and does not allow you to create a real build/install plan for them until you've done a full sync (full in the sense that the ebuilds themselves are downloaded, not just the cache). [1] http://archives.gentoo.org/gentoo-soc/msg_8bb95057c99c45d8cceafcf4876f0976.xml - -- Thanks, Zac -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (GNU/Linux) iEYEARECAAYFAknUOZEACgkQ/ejvha5XGaMnqgCg2bxQHlkpyJvqWMfPwSMuD/Y0 gSoAoL5iYs1jUN5GZHs+0RXzt4zBc7OP =sWMP -----END PGP SIGNATURE-----
