Hello,
I’d like to give a little bit of additional information about a Mercurial feature that might provide some help here. My apologies if this is old news and doesn't add anything useful. (Small context: I’m a Mercurial contributor and I maintain a set of repositories at our company that are cloned a lot at the same point in time) By default, when you do a clone, Mercurial will take the repository, compress it and send it across the wire to the client. I simulated similar behaviour for the gmp repository by creating a bundle: - it takes about 8 seconds (so approximately 1 core occupied for < 8 seconds, as there is a bit of overhead involved) - it generates a bundle of about 14 MB That’s not much, but it of course becomes problematic when there are 700 clones at nearly the exact same moment, like you experienced. I cannot help in reducing the bandwidth, but assuming the cpu usage is the problem, ‘clonebundles’ would help resolve the issue. Clonebundles are a Mercurial feature that allows the Mercurial server to reuse an existing bundle instead of generating a new one for every user. They would basically get rid of the cpu issue, the server simply points to the pre-generated bundle and does not actually need to do any work. More context from Gregory Szorc, who initially implemented this feature: https://gregoryszorc.com/blog/2015/05/29/faster-cloning-from-hg.mozilla.org-with-server-provided-bundles/ To use clonebundles for a repository (let me know if there’s anything missing, I have been known to make mistakes): 1. Make sure you’re running a fairly recent Mercurial version. 2. Put up a cronjob that generates weekly bundles of your repository (hg bundle –all /path/to/gmp.bundle). 3. Make sure the bundle is placed in a location accessible to clients, for example: https://gmplib.org/clonebundles/gmp.bundle. 4. In your repository, create a file .hg/clonebundles.manifest that refers to the bundle. To check what the ‘BUNDLESPEC’ parameter should be, you can run: hg debugbundle –spec /path/to/gmp.bundle Contents could for example be: https://gmplib.org/clonebundles/gmp.bundle BUNDLESPEC=bzip2-v2 5. In your hgrc (either $HOME/.hgrc or /path/to/repository/.hg/hgrc), you will need to enable clonebundles: [extensions] clonebundles = The next version of Mercurial (6.5) will contain a feature (implemented by Mercurial developer Pierre-Yves David) to auto-generate clonebundles when changes are pushed to a repository. This means you don't need the above steps, you can purely use configuration options, as described in the help of the clonebundles extension: https://foss.heptapod.net/mercurial/mercurial-devel/-/blob/branch/default/hgext/clonebundles.py Some additional notes: 1. Clonebundles reduce server load massively, in exchange for additional disk space. Make sure you have enough if you want to enable this. 2. Mercurial clients don't need to do anything to enable this feature, it only needs to be enabled on the server side. 3. I think on the Mercurial development side, we should try to make this feature more visible/known, as it could make the situation a lot better for people hosting Mercurial repositories. Greetings, Mathias _______________________________________________ gmp-devel mailing list gmp-devel@gmplib.org https://gmplib.org/mailman/listinfo/gmp-devel
