Hello everyone,
This is a small email to keep other devs are development we are looking
into. The goal to have something than plan page¹ but still starting
discussion before multiple days have been sinked into development. If
this work well, this is the kind of things I hope to see on a regular
basis on this mailing list.
So, back to the main topic: I have been looking into automatic general
of clone bundle.
The needs: Having reasonably up to date clone bundle automatically
generated and served by Heptapod would be very handy.
I know that Bitbucket did something similar. This time I would like to
contribute it upstream (in the clonebundles extension) to share the
effort between potential users.
My initial effort aims at having a minimal solution that helps heptapod.
With enough possibly of later evolution (ideally). I spent a handful of
hour spoking at the idea and I have a basic prototype.
### Current top level design
* Post transaction hooks would trigger potential full-bundle re-computation.
* Bundle are recomputed if the number of revision is significantly
greater than the one in the current (or pending computation) bundle
(absolute or relative).
* Bundle are generated using background process spawned from Mercurial².
* User provide to upload and delete bundles via the standard mercurial
config. In my case, it will be a S3 storage, I'll try for something
versatile, but it will probably be a bit biased
* the clonebundles.manifest is generated from the known bundles.
* We need some dynamic clonebundles.manifest serving bundles of private
repository
### Smaller details
* An option contains the list of bundle type to use:
`clone-bundles.auto-generate=v1,zstd-v2, gzip-v2`
* The bundles being generated or available are tracked in a file in `.hg/`.
- The file is protected by a specific lock.
- The file is used to generate a final `clonesbundles.manifest`
served to the client
- The file contains metadata about the the bundle (revision it
contains, authentication token, etc)
* We only generate bundles for public revisions.
* The details of what bundle need to be generated is passed to the
background process using pickle. It a (rare) case where pickle seems
fine and we have other example of this in Mercurial. The data are fairly
simple and could also fit in json or other simple serialization format.
Feedback welcome.
Cheers,
[1] Plan page still being useful for larger project or for idea not
being implemented right away
[2] an improvement would be to make it simple to plug it to a standard
background job
--
Pierre-Yves David
_______________________________________________
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
