pulkit created this revision. Herald added a reviewer: durin42. Herald added a reviewer: martinvonz. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers.
REVISION SUMMARY When a narrow clone is done initially the whole manifest is send. In case of treemanifests, the manifest and the dirlogs of the paths matching the narrowmatcher are send while clone. So while widening, we don't need the manifests again. In case of treemanifests, we just need the dirlogs. This patch adds logic in cgpacker.generate() to skip adding the manifests if they are not dirlogs and omit manifest from the required parts in call while widening in narrow extension. This saves a lot of time on big repos. It prevents downloading manifest which can be in GB's on big repos and also since we don't get it, we don't try to unpack and apply it saving more time. It saves around 3-4 minutes on our internal repository while extending on the treemanifest repo. On flat manifest repo, since the manifest is much much larger, I can guess this saves around ~10 mins. I also checked the actual payload size of changegroup in bundle2 in test-narrow-widen-no-ellipsis.t. Before this patch flat: 662 tree: 835 After this patch flat: 157 tree: 333 These are the numbers from the test which has 7-8 commits and few dirs. On big repos the improvement is *huge*. The above all is the case when we are widening without ellipses. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4887 AFFECTED FILES hgext/narrow/narrowbundle2.py mercurial/changegroup.py CHANGE DETAILS diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py --- a/mercurial/changegroup.py +++ b/mercurial/changegroup.py @@ -862,6 +862,8 @@ yield chunk for delta in deltas: + if not tree and 'manifest' not in parts: + continue chunks = _revisiondeltatochunks(delta, self._builddeltaheader) for chunk in chunks: size += len(chunk) diff --git a/hgext/narrow/narrowbundle2.py b/hgext/narrow/narrowbundle2.py --- a/hgext/narrow/narrowbundle2.py +++ b/hgext/narrow/narrowbundle2.py @@ -78,7 +78,7 @@ filematcher=diffmatcher, fullnodes=commonnodes) # XXX: we should also prevent manifests - reqparts = ('manifest', 'dirlogs', 'filelog') + reqparts = ('dirlogs', 'filelog') cgdata = packer.generate(set([nullid]), list(commonnodes), False, 'narrow_widen', parts=reqparts) To: pulkit, durin42, martinvonz, #hg-reviewers Cc: mercurial-devel _______________________________________________ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
