Jeppe Fihl Toustrup wrote:
We already use the --ignore-existing option, which does help with making the entire update process quicker, but that does however not help the file list generation, which takes up two thirds of the total time, as far as my records show.
It definitely seems as though there is a bottleneck in processing on one side or the other, because it typically only takes about a half-hour on my local system to build the file list.
This was not meant to create any stress at all, but rather to think of the better alternatives to using rsync, or making optimizations which may make rsync run faster.
Understood, and no offence was taken. I was just stating that I don't know what resources are available for zfs snapshot provision or if that is a viable option.
Another option would be to pack all the packages in one file per package, like things are done in the Linux world, which would decrease the amount of files dramatically, but it may increase the total size of the repository. This does however require changes to be made in the pkg client, which I thought would be smart to avoid, since I can understand that resources to the OpenSolaris project is limited, at lease compared to what could be used :)
We have intentionally avoided that approach as there is a significant advantage for the client when performing upgrades. Notably, that if only three files have changed in a package between two different versions, we only download the three files that have changed and install them.
I think the repository structure changes I mentioned will be of a greater benefit, and it is entirely possible that we may offer alternate repository synchronisation options in the future.
Cheers, -- Shawn Walker _______________________________________________ pkg-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
