Cyril Plisko wrote:
On 11/1/06, Stephen Lau <[EMAIL PROTECTED]> wrote:
Currently, our notification mechanism sends a notification for each
incoming changeset. On Mercurial, this means at the changeset level.
For Subversion, this means at the commit level.
We've now had two instances of massive putback overload due to
importing/seeding a Mercurial repository with large amounts of
changesets, resulting in 2000+ putback notifications.
Given that only two people have actually seeded a Mercurial repository -
I expect this will be a recurring problem ;-)
We've got a few ways to go about solving the problem, and there are
different angles of attack: (I'm going to talk about things from the
Mercurial side, since I don't anticipate this problem with Subversion
since you can't "push" massive numbers of changesets since the analogous
seed action is 'svnadmin load'):
(These are not mutually exclusive, and some of these solutions can and
should be combined with others)
1) Move the incoming-changeset hook to the incoming-changegroup hook and
send one notification per changegroup
2) Add a web interface action for "seeding" or loading a repository with
an uploaded bundle; or allowing it to be seeded from an already existing
repository. (For instance any ON projects could instantly seed from
onnv-gate, almost instantaneously using zbringover)
(credit where credit's due: this was Rich Lowe's idea)
But what we will have in a couple of months of active seeding ?
A bunch of useless snapshots, created at the moment of cloning ?
You will have to think carefully on the "garbage collection" policy.
The other thing can be a periodic re-cloning, i.e. the ON repo
is cloned again, and the projects changesets are reapplied to the
new clone. That way you can purge oldish snapshots/clones and
really keep your disk usage shallow.
The way I manage this is a periodic zfs list -oname,origin, any
snapshot not listed as the origin of another filesystem
is garbage (automating this would be beneficial, obviously)
The other thing to consider is that for snapshot space usage to be a
problem, it would have to consumer more than the alternative (N * <size
of full workspace>). On a ws-by-ws basis, this would require a great deal.
My workspaces are using about 30M of real space each (discounting
proto, archives, logs, etc) and the snapshots are currently averaging
another 32M-ish.
~60M/workspace is not bad at all, especially when you consider
onnv-clone is 401M.
As I understand it, the zfs folks have been doing this for longer (with
teamware obviously), perhaps they have some data on how this pans out?
-- Rich
_______________________________________________
tools-discuss mailing list
tools-discuss@opensolaris.org
