what *precisely* means "--depth" WRT a shallow clone?
i can see the standard use of "--depth" when set to precisely one, for CI/CD systems that don't need any history. but what does it mean to say, eg, "--depth 10"? the man page explains that as "a history truncated to the specified number of commits", but what does mean in the sense of branching and merging? will i get *precisely* 10 commits of history? and in the context of merging, *which* 10 commits? sorry if this should be obvious. rday -- Robert P. J. Day Ottawa, Ontario, CANADA http://crashcourse.ca Twitter: http://twitter.com/rpjday LinkedIn: http://ca.linkedin.com/in/rpjday
Re: Cannot shallow clone using GitHub tag refs after commit 34066f06
On Sat, Oct 12, 2019 at 06:31:02PM -0400, Teddy Reed wrote:
> Hello, I am trying to debug an issue where a shallow clone (--depth=1)
> fails when the reference is the tip of a tag on a GitHub repository.
>
> The git client works on versions below 2.22.0-rc0 up until commit
> 34066f06, "fetch: do not consider peeled tags as advertised tips".
> That commit peels refs in the form of "refs/tags/v2.2.2^{}". My
> question is if the intention was to peel these refs and break this
> functionality? I do not know enough about git to understand if the
> behavior before commit 34066f06 was unintended.
This is a known and intended outcome of that commit. In fact, that fetch
was never _supposed_ to work, but it did in certain circumstances for
a few versions (2.14.x through 2.21.x).
Here are a few background facts:
- upload-pack does not (and never has) considered the peeled value of
a ref to be "advertised"
- unless configured otherwise, upload-pack will generally refuse to
serve requests for objects it didn't advertise. This has been the
case since day one, _except_ for one interesting exception, which
I'll get into below.
- fetch-pack can realize that the server upload-pack is not
configured to allow unadvertised requests (by the server not
advertising that capability string), and refuses to even send such a
request to the server
- a bug in fdb69d33c4 (in Git 2.14) meant that for this specific case
(an object mentioned only as a peeled value), fetch-pack's check was
wrong, and it would ask the server for the object anyway
So prior to the bug in fdb69d33c4, fetch-pack would refuse the request.
After that bug (and until the fix in 34066f06), it would make the
request and the server would sometimes honor it and sometimes not.
And that's where we get into the one exception. Since 051e4005a3
(helping smart-http/stateless-rpc fetch race, 2011-08-05), the server
upload-pack implicitly allows requesting reachable non-tip objects over
http only. But it does so quietly, without making a capability
advertisement to the client.
So whether this request works depends on your client version, the server
version, and the protocol in use. I didn't test each combo, but it
should be something like:
server| client| protocol | works?
-
<1.7.6.1 | any | http/ssh/git | no
>=1.7.6.1 | <2.14 | http/ssh/git | no
>=1.7.6.1 | >=2.14, <2.22 | http | yes
>=1.7.6.1 | >=2.14, <2.22 | ssh/git | no
>=1.7.6.1 | >=2.22| http/ssh/git | no
So that's the back-story, but of course it's still annoying when it
doesn't work with a submodule. Where can we go from here?
- if you enable protocol v2 (by setting `protocol.version` to `2` in
your git config), its rules are different: it implicitly allows
fetching any object, reachable or not.
Arguably GitHub ought to set uploadpack.allowAnySHA1InWant for all
repositories, to let the v0 protocol behave the same way (after all,
any such object is available by making a v2 request already). The
fact that it hasn't is mostly history and inertia, along with some
open questions about whether v2 was perhaps a bit too cavalier in
this regard (but at this point, I think it's mostly a done deal).
- fetch-pack could assume that fetching via http means the server has
implicitly enabled uploadpack.allowReachableSHA1InWant, which it
effectively has since 1.7.6.1. Of course we might be talking to
another server implementation (I have no idea how JGit works there,
for example)
- the flip side of that is: upload-pack could always advertise the
"allow-reachable-sha1-in-want" capability when it's speaking http,
since it has effectively enabled it anyway.
- this is mostly of interest for submodules, which is the case for
your repo. Your output has this:
> Cloning into '/tmp/test-repo2/gflags'...
> error: Server does not allow request for unadvertised object
> e171aa2d15ed9eb17054558e0b3a6a413bb01067
> Fetched in submodule path 'gflags', but it did not contain
> e171aa2d15ed9eb17054558e0b3a6a413bb01067. Direct fetching of that commit
> failed.
which I think is a problem in the submodule code. If a direct fetch
doesn't work, it's supposed to do a larger fetch as a fallback
(exactly to cope with this situation that a server's tips have moved
on). If it's not getting the object (which we know is referenced
directly by a tag!) that is a sign that it could be trying harder.
So I think my first suggestion above is something you can do immediately
to make your case work. The rest are possible changes that could be made
in Git. I do feel like one easy path forward, though, is to just assume
we'll flip the default to the v2 protocol at some point, and all of this
will go away.
-Peff
Cannot shallow clone using GitHub tag refs after commit 34066f06
Hello, I am trying to debug an issue where a shallow clone (--depth=1) fails
when the reference is the tip of a tag on a GitHub repository.
The git client works on versions below 2.22.0-rc0 up until commit 34066f06,
"fetch: do not consider peeled tags as advertised tips". That commit peels refs
in the form of "refs/tags/v2.2.2^{}". My question is if the intention was to
peel these refs and break this functionality? I do not know enough about git to
understand if the behavior before commit 34066f06 was unintended.
Here is a method to reproduce the previous behavior:
$ mkdir test-repo
$ cd test-repo
$ git init
$ git submodule add https://github.com/gflags/gflags gflags
$ (cd gflags; git checkout v2.2.2)
$ git add .
$ git commit -m 'Add gflags submodule'
Then with git client version 2.21.0:
$ git clone test-repo test-repo2
$ cd test-repo2
$ git submodule update --init --depth=1
Submodule 'gflags' (https://github.com/gflags/gflags) registered for path
'gflags'
Cloning into '/tmp/test-repo2/gflags'...
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
remote: Enumerating objects: 17, done.
remote: Counting objects: 100% (17/17), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 9 (delta 8), reused 1 (delta 0), pack-reused 0
Unpacking objects: 100% (9/9), done.
>From https://github.com/gflags/gflags
* branche171aa2d15ed9eb17054558e0b3a6a413bb01067 -> FETCH_HEAD
Submodule path 'gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067'
And attempting with newer versions of the git client:
Cloning into '/tmp/test-repo2/gflags'...
error: Server does not allow request for unadvertised object
e171aa2d15ed9eb17054558e0b3a6a413bb01067
Fetched in submodule path 'gflags', but it did not contain
e171aa2d15ed9eb17054558e0b3a6a413bb01067. Direct fetching of that commit failed.
Thanks for the help!
--
Teddy Reed
Re: Retrieve version-string on shallow clone
Hello Giuseppe, Philip and Jeff, On 2019-09-02T16:08:54+0200, Giuseppe Crinò wrote: > To my understanding both questions are solved by > * https://stackoverflow.com/a/47720414/2219670 > * https://stackoverflow.com/a/12704727/2219670 On 2019-09-02T17:52:19+0200, Philip Oakley wrote: > Start by getting a bit more history depth, then see if you need to fetch some > of the tags (or at least any specific tags of interest). On 2019-09-02T17:54:40+0200, Jeff King wrote: > But what you want is perfectly reasonable; there's just not a good way to do > it yet. > [...] There's no way to trigger this within Git's protocols. I never thought this would be impossible, or at least very hard, to accomplish. git ls-remote --tags --refs https://github.com/mstorsjo/fdk-aac.git 94f9d5ca2077262e838fbc8ed111da03be5389d5refs/tags/v0.1.0 1551d17717e42e7c295da0a682ae299791ee87c7refs/tags/v0.1.1 fa3b71183ccb19337c0ca76aea7dd85af9c8refs/tags/v0.1.2 db7189736b49d27225eea917fcf1581b5228c830refs/tags/v0.1.3 d17a2ebc6d8f593ffbefacaea1bfa6d6a81356a1refs/tags/v0.1.4 a3f40d8d974f1aacde95e0996739941bde8f5e98refs/tags/v0.1.5 c98ba983b0cd6905a52ab34222418ca7a5ff2260refs/tags/v0.1.6 e35d91ddd8d41515594b7fd51b6bae1b17fee530refs/tags/v2.0.0 git ls-remote --tags --refs https://github.com/mstorsjo/fdk-aac.git | tail -n1 | cut -d/ -f3- v2.0.0 Regrettably the remote repo doesn't include the entire version-string with revision and commit-hash (v2.0.0-185-gcc5c85d). Having to download fdk-aac's complete commit-history (9.38MB) isn't yet all too bad. FFmpeg (https://github.com/FFmpeg/FFmpeg.git) its commit-history on the other hand is a whopping 239MB (as opposed to 15.7MB for --depth 1) and perhaps a better example here! I'm compiling and distributing FFmpeg executables. When I compile FFmpeg I'm not interested in its entire commit-history at that moment. I'm only interested in the checked out source files and the version-string (4.3-dev-327-g83e0b71 for instance). Since this doesn't appear to be possible at the moment, I'll let this rest. Thank you all for your replies. -- Reino
Re: Retrieve version-string on shallow clone
On Sun, Sep 01, 2019 at 01:07:44PM +0200, Reino Wijnsma wrote: > git clone https://github.com/mstorsjo/fdk-aac.git > [...] > > git describe --tags > v2.0.0-185-gcc5c85d > > git clone --depth 1 https://github.com/mstorsjo/fdk-aac.git > > git describe --tags > fatal: No names found, cannot describe anything. > > The tags get lost while doing a shallow clone. Right. As noted elsewhere in the thread, this is the expected behavior of Git. But what you want is perfectly reasonable; there's just not a good way to do it yet. > I'd figure one of these options would be to clone everything from tag > v2.0.0 onward, but if so how would I do that? I don't think there's a good way to do this. But I also think it's not quite what you want, as you'd end up transferring a lot of extra data (and deeper shallow fetches load the server more, as we can't use reachability bitmaps, and we have to do extra on-the-fly delta compression). > As another option I was thinking; maybe it's possible to do git > describe --tags on the remote repo? This seems like it's more directly what you want, and doesn't have a lot of downsides. There's no way to trigger this within Git's protocols. If it's an ssh server you control, you can of course run git-describe yourself on the server. For a hosting site like GitHub, you'd need support from the host's non-Git API. I don't _think_ anything like that exists right now in GitHub's API, though. -Peff
Re: Retrieve version-string on shallow clone
Hi Reino On 01/09/2019 12:07, Reino Wijnsma wrote: Hello git@vger.kernel.org, Two days ago I started https://github.com/mstorsjo/fdk-aac/issues/107, asking how to retrieve the fdk-aac version-string on a shallow clone. My question was of course not fdk-aac related, so Martin Storsjö suggested I'd try here. It basicly comes down to this: git clone https://github.com/mstorsjo/fdk-aac.git [...] git describe --tags v2.0.0-185-gcc5c85d git clone --depth 1 https://github.com/mstorsjo/fdk-aac.git git describe --tags fatal: No names found, cannot describe anything. The tags get lost while doing a shallow clone. I'm not an expert git user, so I was wondering if anyone could tell what my options are here. I'd figure one of these options would be to clone everything from tag v2.0.0 onward, but if so how would I do that? As another option I was thinking; maybe it's possible to do git describe --tags on the remote repo? This is most likely a mental model problem. If you only have a depth = 1 level clone, then there is no history available locally to use from which to describe almost anything. Even if you had all the tags (without the actual commits they tagged) you still don't have that history upon which to describe them. Start by getting a bit more history depth, then see if you need to fetch some of the tags (or at least any specific tags of interest). Philip
Re: Retrieve version-string on shallow clone
> I'd figure one of these options would be to clone everything from tag > v2.0.0 onward, but if so how would I do that? > As another option I was thinking; maybe it's possible to do git describe > --tags on the remote repo? To my understanding both questions are solved by * https://stackoverflow.com/a/47720414/2219670 * https://stackoverflow.com/a/12704727/2219670 It seems to me that's the expected behaviour of git -Giuseppe
Retrieve version-string on shallow clone
Hello git@vger.kernel.org, Two days ago I started https://github.com/mstorsjo/fdk-aac/issues/107, asking how to retrieve the fdk-aac version-string on a shallow clone. My question was of course not fdk-aac related, so Martin Storsjö suggested I'd try here. It basicly comes down to this: git clone https://github.com/mstorsjo/fdk-aac.git [...] git describe --tags v2.0.0-185-gcc5c85d git clone --depth 1 https://github.com/mstorsjo/fdk-aac.git git describe --tags fatal: No names found, cannot describe anything. The tags get lost while doing a shallow clone. I'm not an expert git user, so I was wondering if anyone could tell what my options are here. I'd figure one of these options would be to clone everything from tag v2.0.0 onward, but if so how would I do that? As another option I was thinking; maybe it's possible to do git describe --tags on the remote repo? Thanks! -- Reino
Re: [PATCH] gitmodules: clarify what history depth a shallow clone has
On Wed, Apr 19, 2017 at 12:56 AM, Sebastian Schuberth wrote: > Signed-off-by: Sebastian Schuberth Thanks, Stefan > --- > Documentation/gitmodules.txt | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt > index 8f7c50f..6f39f24 100644 > --- a/Documentation/gitmodules.txt > +++ b/Documentation/gitmodules.txt > @@ -84,8 +84,8 @@ submodule..ignore:: > > submodule..shallow:: > When set to true, a clone of this submodule will be performed as a > - shallow clone unless the user explicitly asks for a non-shallow > - clone. > + shallow clone (with a history depth of 1) unless the user explicitly > + asks for a non-shallow clone. > > > EXAMPLES > > -- > https://github.com/git/git/pull/347
[PATCH] gitmodules: clarify what history depth a shallow clone has
Signed-off-by: Sebastian Schuberth --- Documentation/gitmodules.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt index 8f7c50f..6f39f24 100644 --- a/Documentation/gitmodules.txt +++ b/Documentation/gitmodules.txt @@ -84,8 +84,8 @@ submodule..ignore:: submodule..shallow:: When set to true, a clone of this submodule will be performed as a - shallow clone unless the user explicitly asks for a non-shallow - clone. + shallow clone (with a history depth of 1) unless the user explicitly + asks for a non-shallow clone. EXAMPLES -- https://github.com/git/git/pull/347
Re: [PATCH] clone,fetch: explain the shallow-clone option a little more clearly
On Wed, Dec 7, 2016 at 1:29 AM, Junio C Hamano wrote:
> Duy Nguyen writes:
>
>> On Tue, Dec 6, 2016 at 1:28 AM, Junio C Hamano wrote:
>>> I however offhand do not think the feature can be used to make the
>>> repository shallower
>>
>> I'm pretty sure it can,...
>
> I wrote my message after a short local testing, but it is very
> possible I botched it and reached a wrong conclusion above.
>
> If we can use the command to make it shallower, then the phrase
> "deepen" would probably be what we need to be fixing in this patch:
>
>> OPT_STRING_LIST(0, "shallow-exclude", &option_not, N_("revision"),
>> - N_("deepen history of shallow clone by excluding
>> rev")),
>> + N_("deepen history of shallow clone, excluding rev")),
>
> Perhaps a shorter version of:
>
> Adjust the depth of shallow clone so that commits that are
> decendants of the named rev are made available, while commits
> that are ancestors of the named rev are made beyond reach.
>
> or something like that. Here is my (somewhat botched) attempt:
>
> Adjust shallow clone's history to be cut at the rev
OK mine is "exclude the given tag from local history".
BTW the clone's string could be rephrased better because we know
there's no local history to begin with, if we stick to any verbs that
involves deepening or shortening.
--
Duy
Re: [PATCH] clone,fetch: explain the shallow-clone option a little more clearly
Duy Nguyen writes:
> On Tue, Dec 6, 2016 at 1:28 AM, Junio C Hamano wrote:
>> I however offhand do not think the feature can be used to make the
>> repository shallower
>
> I'm pretty sure it can,...
I wrote my message after a short local testing, but it is very
possible I botched it and reached a wrong conclusion above.
If we can use the command to make it shallower, then the phrase
"deepen" would probably be what we need to be fixing in this patch:
> OPT_STRING_LIST(0, "shallow-exclude", &option_not, N_("revision"),
> - N_("deepen history of shallow clone by excluding rev")),
> + N_("deepen history of shallow clone, excluding rev")),
Perhaps a shorter version of:
Adjust the depth of shallow clone so that commits that are
decendants of the named rev are made available, while commits
that are ancestors of the named rev are made beyond reach.
or something like that. Here is my (somewhat botched) attempt:
Adjust shallow clone's history to be cut at the rev
Re: [PATCH] clone,fetch: explain the shallow-clone option a little more clearly
On Tue, Dec 6, 2016 at 1:28 AM, Junio C Hamano wrote: > I however offhand do not think the feature can be used to make the > repository shallower I'm pretty sure it can, though it's a waste because you should be able to shorten your history without talking to a remote server. But that no-remote shortening is not implemented yet. And you probably want an option to check with remote anyway to make sure the part you cut out is available there, no history will be lost. -- Duy
Re: [PATCH] clone,fetch: explain the shallow-clone option a little more clearly
Alex Henrie writes:
> "deepen by excluding" does not make sense because excluding a revision
> does not deepen a repository; it makes the repository more shallow.
I think that an intuitive way the feature should work may be:
- You started with "git fetch --depth=20" and then later say "I
have only a very short segment of the recent history, but I want
a history that dates back to v1.0" with "--shallow-exclude=v1.0".
In this case, you would be deepening.
- You instead started with "git fetch --depth=2" that dated
back to v0.5. "--shallow-exclude=v1.0" you say today would mean
"I have very old cruft I no longer look at. I just want my
history lead back to v1.0 and no earlier". In such a case, you
indeed would be making the repository shallower.
I however offhand do not think the feature can be used to make the
repository shallower, and I agree your changes to the usage string
probably describe what the option does more correctly.
I however suspect that the feature is simply buggy and it would
eventually want to allow to shorten the history as well. At that
point we may want to work on the verb 'deepen' there, too.
>
> Signed-off-by: Alex Henrie
> ---
> builtin/clone.c | 2 +-
> builtin/fetch.c | 2 +-
> 2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/builtin/clone.c b/builtin/clone.c
> index 6c76a6e..e3cb808 100644
> --- a/builtin/clone.c
> +++ b/builtin/clone.c
> @@ -99,7 +99,7 @@ static struct option builtin_clone_options[] = {
> OPT_STRING(0, "shallow-since", &option_since, N_("time"),
> N_("create a shallow clone since a specific time")),
> OPT_STRING_LIST(0, "shallow-exclude", &option_not, N_("revision"),
> - N_("deepen history of shallow clone by excluding rev")),
> + N_("deepen history of shallow clone, excluding rev")),
> OPT_BOOL(0, "single-branch", &option_single_branch,
> N_("clone only one branch, HEAD or --branch")),
> OPT_BOOL(0, "shallow-submodules", &option_shallow_submodules,
> diff --git a/builtin/fetch.c b/builtin/fetch.c
> index b6a5597..fc74c84 100644
> --- a/builtin/fetch.c
> +++ b/builtin/fetch.c
> @@ -122,7 +122,7 @@ static struct option builtin_fetch_options[] = {
> OPT_STRING(0, "shallow-since", &deepen_since, N_("time"),
> N_("deepen history of shallow repository based on time")),
> OPT_STRING_LIST(0, "shallow-exclude", &deepen_not, N_("revision"),
> - N_("deepen history of shallow clone by excluding rev")),
> + N_("deepen history of shallow clone, excluding rev")),
> OPT_INTEGER(0, "deepen", &deepen_relative,
> N_("deepen history of shallow clone")),
> { OPTION_SET_INT, 0, "unshallow", &unshallow, NULL,
[PATCH] clone,fetch: explain the shallow-clone option a little more clearly
"deepen by excluding" does not make sense because excluding a revision
does not deepen a repository; it makes the repository more shallow.
Signed-off-by: Alex Henrie
---
builtin/clone.c | 2 +-
builtin/fetch.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/builtin/clone.c b/builtin/clone.c
index 6c76a6e..e3cb808 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -99,7 +99,7 @@ static struct option builtin_clone_options[] = {
OPT_STRING(0, "shallow-since", &option_since, N_("time"),
N_("create a shallow clone since a specific time")),
OPT_STRING_LIST(0, "shallow-exclude", &option_not, N_("revision"),
- N_("deepen history of shallow clone by excluding rev")),
+ N_("deepen history of shallow clone, excluding rev")),
OPT_BOOL(0, "single-branch", &option_single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_BOOL(0, "shallow-submodules", &option_shallow_submodules,
diff --git a/builtin/fetch.c b/builtin/fetch.c
index b6a5597..fc74c84 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -122,7 +122,7 @@ static struct option builtin_fetch_options[] = {
OPT_STRING(0, "shallow-since", &deepen_since, N_("time"),
N_("deepen history of shallow repository based on time")),
OPT_STRING_LIST(0, "shallow-exclude", &deepen_not, N_("revision"),
- N_("deepen history of shallow clone by excluding rev")),
+ N_("deepen history of shallow clone, excluding rev")),
OPT_INTEGER(0, "deepen", &deepen_relative,
N_("deepen history of shallow clone")),
{ OPTION_SET_INT, 0, "unshallow", &unshallow, NULL,
--
2.10.2
Re: Git shallow clone branch doesn't work with recursive submodules cloning
On Mon, Aug 15, 2016 at 03:53:48PM +0300, Arkady Shapkin wrote: > So it will work only if github update their server configuration > (boringssl submodule on github)? Correct. -Peff -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Git shallow clone branch doesn't work with recursive submodules cloning
So it will work only if github update their server configuration (boringssl submodule on github)? 2016-08-15 15:47 GMT+03:00 Jeff King : > On Mon, Aug 15, 2016 at 03:29:14PM +0300, Arkady Shapkin wrote: > >> Thank you, after updating to "2.9.3.windows.1" options "--recursive >> --depth 1" now works. >> >> But "--recursive --shallow-submodules" and "--recursive >> --shallow-submodules --depth 1" still doesn't work. > > It does "work", but the server hosting your submodule may not be > configured to allow you to access the reachable-but-non-tip sha1 > directly. So it's not a bug, but rather a configuration issue (and the > "fix" in v2.9.1 is to be less aggressive about enabling > shallow-submodules, since the default server configuration does not > allow it to work well). > > More discussion is in: > > > http://public-inbox.org/git/ofe09d48f2.d1d14f49-onc2257fd7.00280736-c2257fd7.00282...@notes.na.collabserv.com/t/#u > > -Peff -- WBR, Arkady Shapkin aka Dragon -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Git shallow clone branch doesn't work with recursive submodules cloning
On Mon, Aug 15, 2016 at 03:29:14PM +0300, Arkady Shapkin wrote: > Thank you, after updating to "2.9.3.windows.1" options "--recursive > --depth 1" now works. > > But "--recursive --shallow-submodules" and "--recursive > --shallow-submodules --depth 1" still doesn't work. It does "work", but the server hosting your submodule may not be configured to allow you to access the reachable-but-non-tip sha1 directly. So it's not a bug, but rather a configuration issue (and the "fix" in v2.9.1 is to be less aggressive about enabling shallow-submodules, since the default server configuration does not allow it to work well). More discussion is in: http://public-inbox.org/git/ofe09d48f2.d1d14f49-onc2257fd7.00280736-c2257fd7.00282...@notes.na.collabserv.com/t/#u -Peff -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Git shallow clone branch doesn't work with recursive submodules cloning
Thank you, after updating to "2.9.3.windows.1" options "--recursive --depth 1" now works. But "--recursive --shallow-submodules" and "--recursive --shallow-submodules --depth 1" still doesn't work. 2016-08-15 15:04 GMT+03:00 Jeff King : > On Mon, Aug 15, 2016 at 02:20:27PM +0300, Arkady Shapkin wrote: > >> I am trying clone repository by tag with recursive submodules init, >> but for one submodule it doesn't work. >> What I'm doing wrong? > > Nothing. See 18a74a0 (clone: do not let --depth imply > --shallow-submodules, 2016-06-19). > >> >git --version >> git version 2.9.0.windows.1 > > The fix is in v2.9.1. > > -Peff -- WBR, Arkady Shapkin aka Dragon -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Git shallow clone branch doesn't work with recursive submodules cloning
On Mon, Aug 15, 2016 at 02:20:27PM +0300, Arkady Shapkin wrote: > I am trying clone repository by tag with recursive submodules init, > but for one submodule it doesn't work. > What I'm doing wrong? Nothing. See 18a74a0 (clone: do not let --depth imply --shallow-submodules, 2016-06-19). > >git --version > git version 2.9.0.windows.1 The fix is in v2.9.1. -Peff -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Git shallow clone branch doesn't work with recursive submodules cloning
Hi, I am trying clone repository by tag with recursive submodules init, but for one submodule it doesn't work. What I'm doing wrong? >git clone https://github.com/grpc/grpc.git --recursive --depth 1 dsad5 Cloning into 'dsad5'... remote: Counting objects: 7475, done. remote: Compressing objects: 100% (4695/4695), done. remote: Total 7475 (delta 2593), reused 5610 (delta 1265), pack-reused 0 Receiving objects: 100% (7475/7475), 4.85 MiB | 2.55 MiB/s, done. Resolving deltas: 100% (2593/2593), done. Checking connectivity... done. Checking out files: 100% (6820/6820), done. Submodule 'third_party/boringssl' (https://github.com/google/boringssl.git) registered for path 'third_party/boringssl' Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/gflags' Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' Submodule 'third_party/nanopb' (https://github.com/nanopb/nanopb.git) registered for path 'third_party/nanopb' Submodule 'third_party/protobuf' (https://github.com/google/protobuf.git) registered for path 'third_party/protobuf' Submodule 'third_party/zlib' (https://github.com/madler/zlib) registered for path 'third_party/zlib' Cloning into 'D:/Work/conan-packages/dsad5/third_party/boringssl'... Cloning into 'D:/Work/conan-packages/dsad5/third_party/gflags'... Cloning into 'D:/Work/conan-packages/dsad5/third_party/googletest'... Cloning into 'D:/Work/conan-packages/dsad5/third_party/nanopb'... Cloning into 'D:/Work/conan-packages/dsad5/third_party/protobuf'... Cloning into 'D:/Work/conan-packages/dsad5/third_party/zlib'... error: no such remote ref c880e42ba1c8032d4cdde2aba0541d8a9d9fa2e9 Fetched in submodule path 'third_party/boringssl', but it did not contain c880e42ba1c8032d4cdde2aba0541d8a9d9fa2e9. Direct fetching of that commit failed. >git --version git version 2.9.0.windows.1 -- WBR, Arkady Shapkin -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: http-backend fatal error message on shallow clone
On Tue, Jun 21, 2016 at 2:10 PM, Jeff King wrote: > ... > So this request actually takes _two_ upload-pack instances to serve > (which is not uncommon when we need multiple rounds of > get_common_commits(), though I am a little surprised that would be the > case for a clone). And the first one seems to be missing a closing > "" flush packet from the client to end it. > > If that analysis is correct, this isn't affecting operation in any way; > it's just giving a useless message from upload-pack (and as you note, > that could trigger http-backend to exit(1), which may make the webserver > unhappy). > > If we further instrument upload-pack to set GIT_TRACE_PACKET on the > server side, we can see the two requests: > > packet: upload-pack< want 379518c0c94e3b1a0710129d03d5560814a0ba6f > multi_ack_detailed no-done side-band-64k thin-pack include-tag ofs-delta > agent=git/2.9.0.37.gb3ad8ab.dirty > packet: upload-pack< deepen 1 > packet: upload-pack< > packet: upload-pack> shallow 379518c0c94e3b1a0710129d03d5560814a0ba6f > packet: upload-pack> > > packet: upload-pack< want 379518c0c94e3b1a0710129d03d5560814a0ba6f > multi_ack_detailed no-done side-band-64k thin-pack include-tag ofs-delta > agent=git/2.9.0.37.gb3ad8ab.dirty > packet: upload-pack< deepen 1 > packet: upload-pack< > packet: upload-pack> shallow 379518c0c94e3b1a0710129d03d5560814a0ba6f > packet: upload-pack> > packet: upload-pack< done > packet: upload-pack> NAK > packet: upload-pack> > > I think in the first one we would get "deepen 1" from the client in > receive_needs(), and similarly write out our "shallow" line. But then we > go into get_common_commits() and the client has hung up, which causes > the message. Whereas in the second line it gives us a "done", which > completes the negotiation. > > So my not-very-educated thoughts are: > > 1. The client should probably be sending an extra flush in the first > request. Alternatively, in the stateless-rpc case we should just > accept the lack of flush as an acceptable end to the request. Our pkt-line.c can't deal with eof if I remember correctly (I tried to use pkt-line for the parallel checkout stuff, where workers can also exit early...) so sending extra flush may be easier. Old upload-pack will not have a problem with this extra flush right? I haven't checked upload-pack.c yet... > 2. Presumably the shallowness is what causes the double-request, as > fetch-pack wants to see the shallow list before proceeding. But > since it has no actual commits to negotiate, the negotiation is a > noop. So probably this case could actually happen in a single > request. I seem to recall our discussion a few months(?) ago about quickfetch() where shallow clone would force another fetch initiated from backfill_tags(). I guess that's why you see two fetches. > > I suspect that other fetches could not, though, so I'm not sure how > much effort is worth putting into optimizing. > > -Peff > -- > To unsubscribe from this list: send the line "unsubscribe git" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Duy -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: http-backend fatal error message on shallow clone
On Tue, Jun 21, 2016 at 11:23:03AM +, Eric Wong wrote:
> I noticed "fatal: The remote end hung up unexpectedly" in server
> logs from shallow clones. Totally reproducible in the test
> cases, too. The following change shows it:
> [...]
> [Tue Jun 21 11:07:41.391269 2016] [cgi:error] [pid 21589] [client
> 127.0.0.1:37518] AH01215: fatal: The remote end hung up unexpectedly
>
> It doesn't show above, but I think http-backend exits
> with a non-zero status, too, which might cause some CGI
> implementations to complain or break.
>
> Not sure if it's just a corner case that wasn't tested
> or something else, but the clone itself seems successful...
The dying process is actually upload-pack. If we instrument it like
this:
diff --git a/upload-pack.c b/upload-pack.c
index 9e03c27..a1da676 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -820,6 +820,14 @@ static int upload_pack_config(const char *var, const char
*value, void *unused)
return parse_hide_refs_config(var, value, "uploadpack");
}
+NORETURN
+static void custom_die(const char *err, va_list params)
+{
+ vreportf("fatal: ", err, params);
+ warning("aborting");
+ abort();
+}
+
int main(int argc, const char **argv)
{
const char *dir;
@@ -836,6 +844,9 @@ int main(int argc, const char **argv)
OPT_END()
};
+ warning("running upload-pack");
+ set_die_routine(custom_die);
+
git_setup_gettext();
packet_trace_identity("upload-pack");
we can see two things. One is we can get a backtrace from the core file:
#0 0x7f04aef51458 in __GI_raise (sig=sig@entry=6) at
../sysdeps/unix/sysv/linux/raise.c:55
#1 0x7f04aef528da in __GI_abort () at abort.c:89
#2 0x00406009 in custom_die (err=0x4ede60 "The remote end hung up
unexpectedly",
params=0x7ffe15858758) at upload-pack.c:828
#3 0x0045ec63 in die (err=0x4ede60 "The remote end hung up
unexpectedly") at usage.c:108
#4 0x0041e016 in get_packet_data (fd=0, src_buf=0x0, src_size=0x0,
dst=0x7ffe158588b0,
size=4, options=2) at pkt-line.c:167
#5 0x0041e0ea in packet_read (fd=0, src_buf=0x0, src_len=0x0,
buffer=0x73cc40 "deepen 1", size=65520, options=2) at
pkt-line.c:204
#6 0x0041e22b in packet_read_line_generic (fd=0, src=0x0, src_len=0x0,
dst_len=0x0)
at pkt-line.c:234
#7 0x0041e27c in packet_read_line (fd=0, len_p=0x0) at pkt-line.c:244
#8 0x00404dc9 in get_common_commits () at upload-pack.c:384
#9 0x00405eb4 in upload_pack () at upload-pack.c:798
#10 0x00406229 in main (argc=1, argv=0x7ffe15858c28) at
upload-pack.c:872
So we expected to read a packet but didn't get one. Not surprising. The
interesting thing is that we are in get_common_commits(), and if we were
to get a flush packet in stateless-rpc mode, we would simply exit(0).
But we get EOF instead, which provokes packet_read_line() to die.
The other interesting thing is that we can see in httpd's error.log that
it is the penultimate upload-pack that dies (I trimmed the log lines for
readability):
AH01215: warning: running upload-pack
AH01215: fatal: The remote end hung up unexpectedly
AH01215: warning: aborting
AH01215: error: git-upload-pack died of signal 6
AH01215: warning: running upload-pack
So this request actually takes _two_ upload-pack instances to serve
(which is not uncommon when we need multiple rounds of
get_common_commits(), though I am a little surprised that would be the
case for a clone). And the first one seems to be missing a closing
"" flush packet from the client to end it.
If that analysis is correct, this isn't affecting operation in any way;
it's just giving a useless message from upload-pack (and as you note,
that could trigger http-backend to exit(1), which may make the webserver
unhappy).
If we further instrument upload-pack to set GIT_TRACE_PACKET on the
server side, we can see the two requests:
packet: upload-pack< want 379518c0c94e3b1a0710129d03d5560814a0ba6f
multi_ack_detailed no-done side-band-64k thin-pack include-tag ofs-delta
agent=git/2.9.0.37.gb3ad8ab.dirty
packet: upload-pack< deepen 1
packet: upload-pack<
packet: upload-pack> shallow 379518c0c94e3b1a0710129d03d5560814a0ba6f
packet: upload-pack>
packet: upload-pack< want 379518c0c94e3b1a0710129d03d5560814a0ba6f
multi_ack_detailed no-done side-band-64k thin-pack include-tag ofs-delta
agent=git/2.9.0.37.gb3ad8ab.dirty
packet: upload-pack< deepen 1
packet: upload-pack<
packet: upload-pack> shallow 379518c0c94e3b1a0710129d03d5560814a0ba6f
packet: upload-pack>
packet: upload-pack< done
packet: upload-pack> NAK
packet: upload-pack>
I think in the first one we would get "deepen 1" from the client in
receive_needs(), and similarly write out our "shallow" line. But then we
go into get_common_commits() and the client has hung up, which causes
the message. Whereas in the second line it gives us a "done", which
completes t
http-backend fatal error message on shallow clone
I noticed "fatal: The remote end hung up unexpectedly" in server logs from shallow clones. Totally reproducible in the test cases, too. The following change shows it: diff --git a/t/t5561-http-backend.sh b/t/t5561-http-backend.sh index 90e0d6f..cfa55ce 100755 --- a/t/t5561-http-backend.sh +++ b/t/t5561-http-backend.sh @@ -132,5 +132,11 @@ test_expect_success 'server request log matches test results' ' test_cmp exp act ' +test_expect_success 'shallow clone' ' + config http.uploadpack true && + git clone --depth=1 "$HTTPD_URL/smart/repo.git" shallow && + tail "$HTTPD_ROOT_PATH"/error.log | grep fatal +' + stop_httpd test_done And the last test ends like this: expecting success: config http.uploadpack true && git clone --depth=1 "$HTTPD_URL/smart/repo.git" shallow && tail "$HTTPD_ROOT_PATH"/error.log | grep fatal Cloning into 'shallow'... [Tue Jun 21 11:07:41.391269 2016] [cgi:error] [pid 21589] [client 127.0.0.1:37518] AH01215: fatal: The remote end hung up unexpectedly ok 15 - shallow clone It doesn't show above, but I think http-backend exits with a non-zero status, too, which might cause some CGI implementations to complain or break. Not sure if it's just a corner case that wasn't tested or something else, but the clone itself seems successful... -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] shallow clone to not imply shallow submodules
On Mon, Jun 20, 2016 at 09:59:58AM -0700, Stefan Beller wrote: > Signed-off-by: Stefan Beller > --- > > Hi Junio, Peff, > > I thought about this patch squashed into > "clone: do not let --depth imply --shallow-submodules" will actually test > for the regression. Yep, it looks good to me. > +test_expect_success 'shallow clone does not imply shallow submodule' ' > + test_when_finished "rm -rf super_clone" && > + git clone --recurse-submodules --depth 2 "file://$pwd/." super_clone && > + ( > + cd super_clone && > + git log --oneline >lines && > + test_line_count = 2 lines > + ) && > + ( > + cd super_clone/sub && > + git log --oneline >lines && > + test_line_count = 3 lines > + ) > +' This follows the style of the other tests, so it's the right thing here. But as a style suggestion, I think: git -C super_clone/sub log --oneline >lines && test_line_count = 3 lines is nicer than the subshell. It's more succinct, and it saves a process. -Peff -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] shallow clone to not imply shallow submodules
On Mon, Jun 20, 2016 at 10:14:47AM -0700, Stefan Beller wrote: > > This follows the style of the other tests, so it's the right thing here. > > But as a style suggestion, I think: > > > > git -C super_clone/sub log --oneline >lines && > > test_line_count = 3 lines > > > > is nicer than the subshell. It's more succinct, and it saves a process. > > which we would want to refactor to in a follow up, but not merge it > through to 2.9.1. Yeah, exactly. That was what I meant by "here". -Peff -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] shallow clone to not imply shallow submodules
On Mon, Jun 20, 2016 at 10:13 AM, Jeff King wrote: > On Mon, Jun 20, 2016 at 09:59:58AM -0700, Stefan Beller wrote: > >> Signed-off-by: Stefan Beller >> --- >> >> Hi Junio, Peff, >> >> I thought about this patch squashed into >> "clone: do not let --depth imply --shallow-submodules" will actually test >> for the regression. > > Yep, it looks good to me. > >> +test_expect_success 'shallow clone does not imply shallow submodule' ' >> + test_when_finished "rm -rf super_clone" && >> + git clone --recurse-submodules --depth 2 "file://$pwd/." super_clone && >> + ( >> + cd super_clone && >> + git log --oneline >lines && >> + test_line_count = 2 lines >> + ) && >> + ( >> + cd super_clone/sub && >> + git log --oneline >lines && >> + test_line_count = 3 lines >> + ) >> +' > > This follows the style of the other tests, so it's the right thing here. > But as a style suggestion, I think: > > git -C super_clone/sub log --oneline >lines && > test_line_count = 3 lines > > is nicer than the subshell. It's more succinct, and it saves a process. which we would want to refactor to in a follow up, but not merge it through to 2.9.1. Thanks, Stefan > > -Peff -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] shallow clone to not imply shallow submodules
Signed-off-by: Stefan Beller --- Hi Junio, Peff, I thought about this patch squashed into "clone: do not let --depth imply --shallow-submodules" will actually test for the regression. Thanks, Stefan t/t5614-clone-submodules.sh | 17 - 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/t/t5614-clone-submodules.sh b/t/t5614-clone-submodules.sh index f7c630b..a9aaa01 100755 --- a/t/t5614-clone-submodules.sh +++ b/t/t5614-clone-submodules.sh @@ -37,7 +37,7 @@ test_expect_success 'nonshallow clone implies nonshallow submodule' ' ) ' -test_expect_success 'shallow clone does not imply shallow submodule' ' +test_expect_success 'shallow clone with shallow submodule' ' test_when_finished "rm -rf super_clone" && git clone --recurse-submodules --depth 2 --shallow-submodules "file://$pwd/." super_clone && ( @@ -52,6 +52,21 @@ test_expect_success 'shallow clone does not imply shallow submodule' ' ) ' +test_expect_success 'shallow clone does not imply shallow submodule' ' + test_when_finished "rm -rf super_clone" && + git clone --recurse-submodules --depth 2 "file://$pwd/." super_clone && + ( + cd super_clone && + git log --oneline >lines && + test_line_count = 2 lines + ) && + ( + cd super_clone/sub && + git log --oneline >lines && + test_line_count = 3 lines + ) +' + test_expect_success 'shallow clone with non shallow submodule' ' test_when_finished "rm -rf super_clone" && git clone --recurse-submodules --depth 2 --no-shallow-submodules "file://$pwd/." super_clone && -- 2.7.0.rc0.40.g5328432.dirty -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 23/27] clone: define shallow clone boundary with --shallow-exclude
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
Documentation/git-clone.txt | 5 +
builtin/clone.c | 10 +-
2 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index a410409..5049663 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -196,6 +196,11 @@ objects from the source repository into a pack in the
cloned repository.
--shallow-since=::
Create a shallow clone with a history after the specified time.
+--shallow-exclude=::
+ Create a shallow clone with a history, excluding commits
+ reachable from a specified remote branch or tag. This option
+ can be specified multiple times.
+
--[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index dc2ef4f..3849231 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -44,6 +44,7 @@ static int deepen;
static char *option_template, *option_depth, *option_since;
static char *option_origin = NULL;
static char *option_branch = NULL;
+static struct string_list option_not = STRING_LIST_INIT_NODUP;
static const char *real_git_dir;
static char *option_upload_pack = "git-upload-pack";
static int option_verbosity;
@@ -89,6 +90,8 @@ static struct option builtin_clone_options[] = {
N_("create a shallow clone of that depth")),
OPT_STRING(0, "shallow-since", &option_since, N_("time"),
N_("create a shallow clone since a specific time")),
+ OPT_STRING_LIST(0, "shallow-exclude", &option_not, N_("revision"),
+ N_("deepen history of shallow clone by excluding rev")),
OPT_BOOL(0, "single-branch", &option_single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", &real_git_dir, N_("gitdir"),
@@ -852,7 +855,7 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
- if (option_depth || option_since)
+ if (option_depth || option_since || option_not.nr)
deepen = 1;
if (option_single_branch == -1)
option_single_branch = deepen ? 1 : 0;
@@ -983,6 +986,8 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
warning(_("--depth is ignored in local clones; use
file:// instead."));
if (option_since)
warning(_("--shallow-since is ignored in local clones;
use file:// instead."));
+ if (option_not.nr)
+ warning(_("--shallow-exclude is ignored in local
clones; use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow,
ignoring --local"));
@@ -1004,6 +1009,9 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (option_since)
transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
option_since);
+ if (option_not.nr)
+ transport_set_option(transport, TRANS_OPT_DEEPEN_NOT,
+(const char *)&option_not);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 18/27] clone: define shallow clone boundary based on time with --shallow-since
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
Documentation/git-clone.txt | 3 +++
builtin/clone.c | 16 +---
2 files changed, 16 insertions(+), 3 deletions(-)
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index b7c467a..a410409 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -193,6 +193,9 @@ objects from the source repository into a pack in the
cloned repository.
`--no-single-branch` is given to fetch the histories near the
tips of all branches.
+--shallow-since=::
+ Create a shallow clone with a history after the specified time.
+
--[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index bcba080..dc2ef4f 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -40,7 +40,8 @@ static const char * const builtin_clone_usage[] = {
static int option_no_checkout, option_bare, option_mirror,
option_single_branch = -1;
static int option_local = -1, option_no_hardlinks, option_shared,
option_recursive;
-static char *option_template, *option_depth;
+static int deepen;
+static char *option_template, *option_depth, *option_since;
static char *option_origin = NULL;
static char *option_branch = NULL;
static const char *real_git_dir;
@@ -86,6 +87,8 @@ static struct option builtin_clone_options[] = {
N_("path to git-upload-pack on the remote")),
OPT_STRING(0, "depth", &option_depth, N_("depth"),
N_("create a shallow clone of that depth")),
+ OPT_STRING(0, "shallow-since", &option_since, N_("time"),
+ N_("create a shallow clone since a specific time")),
OPT_BOOL(0, "single-branch", &option_single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", &real_git_dir, N_("gitdir"),
@@ -849,8 +852,10 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
+ if (option_depth || option_since)
+ deepen = 1;
if (option_single_branch == -1)
- option_single_branch = option_depth ? 1 : 0;
+ option_single_branch = deepen ? 1 : 0;
if (option_mirror)
option_bare = 1;
@@ -976,6 +981,8 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (is_local) {
if (option_depth)
warning(_("--depth is ignored in local clones; use
file:// instead."));
+ if (option_since)
+ warning(_("--shallow-since is ignored in local clones;
use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow,
ignoring --local"));
@@ -994,6 +1001,9 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (option_depth)
transport_set_option(transport, TRANS_OPT_DEPTH,
option_depth);
+ if (option_since)
+ transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
+option_since);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
@@ -1001,7 +1011,7 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
transport_set_option(transport, TRANS_OPT_UPLOADPACK,
option_upload_pack);
- if (transport->smart_options && !option_depth)
+ if (transport->smart_options && !deepen)
transport->smart_options->check_self_contained_and_connected =
1;
refs = transport_get_remote_refs(transport);
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 23/27] clone: define shallow clone boundary with --shallow-exclude
Signed-off-by: Nguyễn Thái Ngọc Duy
---
Documentation/git-clone.txt | 5 +
builtin/clone.c | 10 +-
2 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index a410409..5049663 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -196,6 +196,11 @@ objects from the source repository into a pack in the
cloned repository.
--shallow-since=::
Create a shallow clone with a history after the specified time.
+--shallow-exclude=::
+ Create a shallow clone with a history, excluding commits
+ reachable from a specified remote branch or tag. This option
+ can be specified multiple times.
+
--[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index dc2ef4f..3849231 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -44,6 +44,7 @@ static int deepen;
static char *option_template, *option_depth, *option_since;
static char *option_origin = NULL;
static char *option_branch = NULL;
+static struct string_list option_not = STRING_LIST_INIT_NODUP;
static const char *real_git_dir;
static char *option_upload_pack = "git-upload-pack";
static int option_verbosity;
@@ -89,6 +90,8 @@ static struct option builtin_clone_options[] = {
N_("create a shallow clone of that depth")),
OPT_STRING(0, "shallow-since", &option_since, N_("time"),
N_("create a shallow clone since a specific time")),
+ OPT_STRING_LIST(0, "shallow-exclude", &option_not, N_("revision"),
+ N_("deepen history of shallow clone by excluding rev")),
OPT_BOOL(0, "single-branch", &option_single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", &real_git_dir, N_("gitdir"),
@@ -852,7 +855,7 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
- if (option_depth || option_since)
+ if (option_depth || option_since || option_not.nr)
deepen = 1;
if (option_single_branch == -1)
option_single_branch = deepen ? 1 : 0;
@@ -983,6 +986,8 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
warning(_("--depth is ignored in local clones; use
file:// instead."));
if (option_since)
warning(_("--shallow-since is ignored in local clones;
use file:// instead."));
+ if (option_not.nr)
+ warning(_("--shallow-exclude is ignored in local
clones; use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow,
ignoring --local"));
@@ -1004,6 +1009,9 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (option_since)
transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
option_since);
+ if (option_not.nr)
+ transport_set_option(transport, TRANS_OPT_DEEPEN_NOT,
+(const char *)&option_not);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 18/27] clone: define shallow clone boundary based on time with --shallow-since
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
Documentation/git-clone.txt | 3 +++
builtin/clone.c | 16 +---
2 files changed, 16 insertions(+), 3 deletions(-)
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index b7c467a..a410409 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -193,6 +193,9 @@ objects from the source repository into a pack in the
cloned repository.
`--no-single-branch` is given to fetch the histories near the
tips of all branches.
+--shallow-since=::
+ Create a shallow clone with a history after the specified time.
+
--[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index bcba080..dc2ef4f 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -40,7 +40,8 @@ static const char * const builtin_clone_usage[] = {
static int option_no_checkout, option_bare, option_mirror,
option_single_branch = -1;
static int option_local = -1, option_no_hardlinks, option_shared,
option_recursive;
-static char *option_template, *option_depth;
+static int deepen;
+static char *option_template, *option_depth, *option_since;
static char *option_origin = NULL;
static char *option_branch = NULL;
static const char *real_git_dir;
@@ -86,6 +87,8 @@ static struct option builtin_clone_options[] = {
N_("path to git-upload-pack on the remote")),
OPT_STRING(0, "depth", &option_depth, N_("depth"),
N_("create a shallow clone of that depth")),
+ OPT_STRING(0, "shallow-since", &option_since, N_("time"),
+ N_("create a shallow clone since a specific time")),
OPT_BOOL(0, "single-branch", &option_single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", &real_git_dir, N_("gitdir"),
@@ -849,8 +852,10 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
+ if (option_depth || option_since)
+ deepen = 1;
if (option_single_branch == -1)
- option_single_branch = option_depth ? 1 : 0;
+ option_single_branch = deepen ? 1 : 0;
if (option_mirror)
option_bare = 1;
@@ -976,6 +981,8 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (is_local) {
if (option_depth)
warning(_("--depth is ignored in local clones; use
file:// instead."));
+ if (option_since)
+ warning(_("--shallow-since is ignored in local clones;
use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow,
ignoring --local"));
@@ -994,6 +1001,9 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (option_depth)
transport_set_option(transport, TRANS_OPT_DEPTH,
option_depth);
+ if (option_since)
+ transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
+option_since);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
@@ -1001,7 +1011,7 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
transport_set_option(transport, TRANS_OPT_UPLOADPACK,
option_upload_pack);
- if (transport->smart_options && !option_depth)
+ if (transport->smart_options && !deepen)
transport->smart_options->check_self_contained_and_connected =
1;
refs = transport_get_remote_refs(transport);
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 22/26] clone: define shallow clone boundary with --shallow-exclude
On Wed, Apr 13, 2016 at 8:55 AM, Nguyễn Thái Ngọc Duy wrote:
> Signed-off-by: Nguyễn Thái Ngọc Duy
> ---
> diff --git a/builtin/clone.c b/builtin/clone.c
> @@ -44,6 +44,7 @@ static int deepen;
> +static struct string_list option_not = STRING_LIST_INIT_NODUP;
> @@ -52,6 +53,13 @@ static struct string_list option_config;
> +static int option_parse_deepen_not(const struct option *opt,
> + const char *arg, int unset)
> +{
> + string_list_append(&option_not, arg);
> + return 0;
> +}
> +
> @@ -89,6 +97,9 @@ static struct option builtin_clone_options[] = {
> N_("create a shallow clone of that depth")),
> OPT_STRING(0, "shallow-since", &option_since, N_("time"),
> N_("create a shallow clone since a specific time")),
> + { OPTION_CALLBACK, 0, "shallow-exclude", NULL, N_("revision"),
> + N_("deepen history of shallow clone by excluding rev"),
> + PARSE_OPT_NONEG, option_parse_deepen_not },
OPT_STRING_LIST()?
> OPT_BOOL(0, "single-branch", &option_single_branch,
> N_("clone only one branch, HEAD or --branch")),
> OPT_STRING(0, "separate-git-dir", &real_git_dir, N_("gitdir"),
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 17/26] clone: define shallow clone boundary based on time with --shallow-since
Signed-off-by: Nguyễn Thái Ngọc Duy
---
Documentation/git-clone.txt | 3 +++
builtin/clone.c | 16 +---
2 files changed, 16 insertions(+), 3 deletions(-)
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index b7c467a..a410409 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -193,6 +193,9 @@ objects from the source repository into a pack in the
cloned repository.
`--no-single-branch` is given to fetch the histories near the
tips of all branches.
+--shallow-since=::
+ Create a shallow clone with a history after the specified time.
+
--[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index bcba080..dc2ef4f 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -40,7 +40,8 @@ static const char * const builtin_clone_usage[] = {
static int option_no_checkout, option_bare, option_mirror,
option_single_branch = -1;
static int option_local = -1, option_no_hardlinks, option_shared,
option_recursive;
-static char *option_template, *option_depth;
+static int deepen;
+static char *option_template, *option_depth, *option_since;
static char *option_origin = NULL;
static char *option_branch = NULL;
static const char *real_git_dir;
@@ -86,6 +87,8 @@ static struct option builtin_clone_options[] = {
N_("path to git-upload-pack on the remote")),
OPT_STRING(0, "depth", &option_depth, N_("depth"),
N_("create a shallow clone of that depth")),
+ OPT_STRING(0, "shallow-since", &option_since, N_("time"),
+ N_("create a shallow clone since a specific time")),
OPT_BOOL(0, "single-branch", &option_single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", &real_git_dir, N_("gitdir"),
@@ -849,8 +852,10 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
+ if (option_depth || option_since)
+ deepen = 1;
if (option_single_branch == -1)
- option_single_branch = option_depth ? 1 : 0;
+ option_single_branch = deepen ? 1 : 0;
if (option_mirror)
option_bare = 1;
@@ -976,6 +981,8 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (is_local) {
if (option_depth)
warning(_("--depth is ignored in local clones; use
file:// instead."));
+ if (option_since)
+ warning(_("--shallow-since is ignored in local clones;
use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow,
ignoring --local"));
@@ -994,6 +1001,9 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (option_depth)
transport_set_option(transport, TRANS_OPT_DEPTH,
option_depth);
+ if (option_since)
+ transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
+option_since);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
@@ -1001,7 +1011,7 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
transport_set_option(transport, TRANS_OPT_UPLOADPACK,
option_upload_pack);
- if (transport->smart_options && !option_depth)
+ if (transport->smart_options && !deepen)
transport->smart_options->check_self_contained_and_connected =
1;
refs = transport_get_remote_refs(transport);
--
2.8.0.rc0.210.gd302cd2
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 22/26] clone: define shallow clone boundary with --shallow-exclude
Signed-off-by: Nguyễn Thái Ngọc Duy
---
Documentation/git-clone.txt | 5 +
builtin/clone.c | 18 +-
2 files changed, 22 insertions(+), 1 deletion(-)
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index a410409..5049663 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -196,6 +196,11 @@ objects from the source repository into a pack in the
cloned repository.
--shallow-since=::
Create a shallow clone with a history after the specified time.
+--shallow-exclude=::
+ Create a shallow clone with a history, excluding commits
+ reachable from a specified remote branch or tag. This option
+ can be specified multiple times.
+
--[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index dc2ef4f..5ccf6b7 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -44,6 +44,7 @@ static int deepen;
static char *option_template, *option_depth, *option_since;
static char *option_origin = NULL;
static char *option_branch = NULL;
+static struct string_list option_not = STRING_LIST_INIT_NODUP;
static const char *real_git_dir;
static char *option_upload_pack = "git-upload-pack";
static int option_verbosity;
@@ -52,6 +53,13 @@ static struct string_list option_config;
static struct string_list option_reference;
static int option_dissociate;
+static int option_parse_deepen_not(const struct option *opt,
+ const char *arg, int unset)
+{
+ string_list_append(&option_not, arg);
+ return 0;
+}
+
static struct option builtin_clone_options[] = {
OPT__VERBOSITY(&option_verbosity),
OPT_BOOL(0, "progress", &option_progress,
@@ -89,6 +97,9 @@ static struct option builtin_clone_options[] = {
N_("create a shallow clone of that depth")),
OPT_STRING(0, "shallow-since", &option_since, N_("time"),
N_("create a shallow clone since a specific time")),
+ { OPTION_CALLBACK, 0, "shallow-exclude", NULL, N_("revision"),
+ N_("deepen history of shallow clone by excluding rev"),
+ PARSE_OPT_NONEG, option_parse_deepen_not },
OPT_BOOL(0, "single-branch", &option_single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", &real_git_dir, N_("gitdir"),
@@ -852,7 +863,7 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
- if (option_depth || option_since)
+ if (option_depth || option_since || option_not.nr)
deepen = 1;
if (option_single_branch == -1)
option_single_branch = deepen ? 1 : 0;
@@ -983,6 +994,8 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
warning(_("--depth is ignored in local clones; use
file:// instead."));
if (option_since)
warning(_("--shallow-since is ignored in local clones;
use file:// instead."));
+ if (option_not.nr)
+ warning(_("--shallow-exclude is ignored in local
clones; use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow,
ignoring --local"));
@@ -1004,6 +1017,9 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (option_since)
transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
option_since);
+ if (option_not.nr)
+ transport_set_option(transport, TRANS_OPT_DEEPEN_NOT,
+(const char *)&option_not);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
--
2.8.0.rc0.210.gd302cd2
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 16/25] clone: define shallow clone boundary based on time with --shallow-since
Signed-off-by: Nguyễn Thái Ngọc Duy
---
Documentation/git-clone.txt | 3 +++
builtin/clone.c | 16 +---
2 files changed, 16 insertions(+), 3 deletions(-)
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index b7c467a..a410409 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -193,6 +193,9 @@ objects from the source repository into a pack in the
cloned repository.
`--no-single-branch` is given to fetch the histories near the
tips of all branches.
+--shallow-since=::
+ Create a shallow clone with a history after the specified time.
+
--[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index bcba080..dc2ef4f 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -40,7 +40,8 @@ static const char * const builtin_clone_usage[] = {
static int option_no_checkout, option_bare, option_mirror,
option_single_branch = -1;
static int option_local = -1, option_no_hardlinks, option_shared,
option_recursive;
-static char *option_template, *option_depth;
+static int deepen;
+static char *option_template, *option_depth, *option_since;
static char *option_origin = NULL;
static char *option_branch = NULL;
static const char *real_git_dir;
@@ -86,6 +87,8 @@ static struct option builtin_clone_options[] = {
N_("path to git-upload-pack on the remote")),
OPT_STRING(0, "depth", &option_depth, N_("depth"),
N_("create a shallow clone of that depth")),
+ OPT_STRING(0, "shallow-since", &option_since, N_("time"),
+ N_("create a shallow clone since a specific time")),
OPT_BOOL(0, "single-branch", &option_single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", &real_git_dir, N_("gitdir"),
@@ -849,8 +852,10 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
+ if (option_depth || option_since)
+ deepen = 1;
if (option_single_branch == -1)
- option_single_branch = option_depth ? 1 : 0;
+ option_single_branch = deepen ? 1 : 0;
if (option_mirror)
option_bare = 1;
@@ -976,6 +981,8 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (is_local) {
if (option_depth)
warning(_("--depth is ignored in local clones; use
file:// instead."));
+ if (option_since)
+ warning(_("--shallow-since is ignored in local clones;
use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow,
ignoring --local"));
@@ -994,6 +1001,9 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (option_depth)
transport_set_option(transport, TRANS_OPT_DEPTH,
option_depth);
+ if (option_since)
+ transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
+option_since);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
@@ -1001,7 +1011,7 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
transport_set_option(transport, TRANS_OPT_UPLOADPACK,
option_upload_pack);
- if (transport->smart_options && !option_depth)
+ if (transport->smart_options && !deepen)
transport->smart_options->check_self_contained_and_connected =
1;
refs = transport_get_remote_refs(transport);
--
2.7.1.532.gd9e3aaa
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 21/25] clone: define shallow clone boundary with --shallow-exclude
Signed-off-by: Nguyễn Thái Ngọc Duy
---
Documentation/git-clone.txt | 5 +
builtin/clone.c | 18 +-
2 files changed, 22 insertions(+), 1 deletion(-)
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index a410409..5049663 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -196,6 +196,11 @@ objects from the source repository into a pack in the
cloned repository.
--shallow-since=::
Create a shallow clone with a history after the specified time.
+--shallow-exclude=::
+ Create a shallow clone with a history, excluding commits
+ reachable from a specified remote branch or tag. This option
+ can be specified multiple times.
+
--[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index dc2ef4f..5ccf6b7 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -44,6 +44,7 @@ static int deepen;
static char *option_template, *option_depth, *option_since;
static char *option_origin = NULL;
static char *option_branch = NULL;
+static struct string_list option_not = STRING_LIST_INIT_NODUP;
static const char *real_git_dir;
static char *option_upload_pack = "git-upload-pack";
static int option_verbosity;
@@ -52,6 +53,13 @@ static struct string_list option_config;
static struct string_list option_reference;
static int option_dissociate;
+static int option_parse_deepen_not(const struct option *opt,
+ const char *arg, int unset)
+{
+ string_list_append(&option_not, arg);
+ return 0;
+}
+
static struct option builtin_clone_options[] = {
OPT__VERBOSITY(&option_verbosity),
OPT_BOOL(0, "progress", &option_progress,
@@ -89,6 +97,9 @@ static struct option builtin_clone_options[] = {
N_("create a shallow clone of that depth")),
OPT_STRING(0, "shallow-since", &option_since, N_("time"),
N_("create a shallow clone since a specific time")),
+ { OPTION_CALLBACK, 0, "shallow-exclude", NULL, N_("revision"),
+ N_("deepen history of shallow clone by excluding rev"),
+ PARSE_OPT_NONEG, option_parse_deepen_not },
OPT_BOOL(0, "single-branch", &option_single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", &real_git_dir, N_("gitdir"),
@@ -852,7 +863,7 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
- if (option_depth || option_since)
+ if (option_depth || option_since || option_not.nr)
deepen = 1;
if (option_single_branch == -1)
option_single_branch = deepen ? 1 : 0;
@@ -983,6 +994,8 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
warning(_("--depth is ignored in local clones; use
file:// instead."));
if (option_since)
warning(_("--shallow-since is ignored in local clones;
use file:// instead."));
+ if (option_not.nr)
+ warning(_("--shallow-exclude is ignored in local
clones; use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow,
ignoring --local"));
@@ -1004,6 +1017,9 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (option_since)
transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
option_since);
+ if (option_not.nr)
+ transport_set_option(transport, TRANS_OPT_DEEPEN_NOT,
+(const char *)&option_not);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
--
2.7.1.532.gd9e3aaa
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 17/25] clone: define shallow clone boundary based on time with --shallow-since
Nguyễn Thái Ngọc Duy writes:
> Signed-off-by: Nguyễn Thái Ngọc Duy
> ---
It is kind of surprising that 16 & 17 can be so simple and does not
have to update the way the cut-off points at the client side are
computed or recorded. We must have done something right when we
designed the initial "--depth" support ;-).
On the other hand, that probably means we have the same "we clone
once, wait for a while and then do a shallow fetch with too short a
history span--the objects in the original clone all go too stale
that they become invisible in the resulting history" property (I
would not call that "an issue") as before. It is just the way the
shallow boundary is specified got more user friendly (from a number
of parent-child hops to timespan).
Let's keep reading...
> Documentation/git-clone.txt | 3 +++
> builtin/clone.c | 16 +---
> 2 files changed, 16 insertions(+), 3 deletions(-)
>
> diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
> index 789b668..1b6b639 100644
> --- a/Documentation/git-clone.txt
> +++ b/Documentation/git-clone.txt
> @@ -194,6 +194,9 @@ objects from the source repository into a pack in the
> cloned repository.
> `--no-single-branch` is given to fetch the histories near the
> tips of all branches.
>
> +--shallow-since=::
> + Create a shallow clone with a history after the specified time.
> +
> --[no-]single-branch::
> Clone only the history leading to the tip of a single branch,
> either specified by the `--branch` option or the primary
> diff --git a/builtin/clone.c b/builtin/clone.c
> index bcba080..dc2ef4f 100644
> --- a/builtin/clone.c
> +++ b/builtin/clone.c
> @@ -40,7 +40,8 @@ static const char * const builtin_clone_usage[] = {
>
> static int option_no_checkout, option_bare, option_mirror,
> option_single_branch = -1;
> static int option_local = -1, option_no_hardlinks, option_shared,
> option_recursive;
> -static char *option_template, *option_depth;
> +static int deepen;
> +static char *option_template, *option_depth, *option_since;
> static char *option_origin = NULL;
> static char *option_branch = NULL;
> static const char *real_git_dir;
> @@ -86,6 +87,8 @@ static struct option builtin_clone_options[] = {
> N_("path to git-upload-pack on the remote")),
> OPT_STRING(0, "depth", &option_depth, N_("depth"),
> N_("create a shallow clone of that depth")),
> + OPT_STRING(0, "shallow-since", &option_since, N_("time"),
> + N_("create a shallow clone since a specific time")),
> OPT_BOOL(0, "single-branch", &option_single_branch,
> N_("clone only one branch, HEAD or --branch")),
> OPT_STRING(0, "separate-git-dir", &real_git_dir, N_("gitdir"),
> @@ -849,8 +852,10 @@ int cmd_clone(int argc, const char **argv, const char
> *prefix)
> usage_msg_opt(_("You must specify a repository to clone."),
> builtin_clone_usage, builtin_clone_options);
>
> + if (option_depth || option_since)
> + deepen = 1;
> if (option_single_branch == -1)
> - option_single_branch = option_depth ? 1 : 0;
> + option_single_branch = deepen ? 1 : 0;
>
> if (option_mirror)
> option_bare = 1;
> @@ -976,6 +981,8 @@ int cmd_clone(int argc, const char **argv, const char
> *prefix)
> if (is_local) {
> if (option_depth)
> warning(_("--depth is ignored in local clones; use
> file:// instead."));
> + if (option_since)
> + warning(_("--shallow-since is ignored in local clones;
> use file:// instead."));
> if (!access(mkpath("%s/shallow", path), F_OK)) {
> if (option_local > 0)
> warning(_("source repository is shallow,
> ignoring --local"));
> @@ -994,6 +1001,9 @@ int cmd_clone(int argc, const char **argv, const char
> *prefix)
> if (option_depth)
> transport_set_option(transport, TRANS_OPT_DEPTH,
>option_depth);
> + if (option_since)
> + transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
> + option_since);
> if (option_single_branch)
> transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
>
> @@ -1001,7 +1011,7 @@ int cmd_clone(int argc, const char **argv, const char
> *prefix)
>
[PATCH v2 22/25] clone: define shallow clone boundary with --shallow-exclude
Signed-off-by: Nguyễn Thái Ngọc Duy
---
Documentation/git-clone.txt | 5 +
builtin/clone.c | 18 +-
2 files changed, 22 insertions(+), 1 deletion(-)
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 1b6b639..31e1610 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -197,6 +197,11 @@ objects from the source repository into a pack in the
cloned repository.
--shallow-since=::
Create a shallow clone with a history after the specified time.
+--shallow-exclude=::
+ Create a shallow clone with a history, excluding commits
+ reachable from a specified remote branch or tag. This option
+ can be specified multiple times.
+
--[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index dc2ef4f..5ccf6b7 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -44,6 +44,7 @@ static int deepen;
static char *option_template, *option_depth, *option_since;
static char *option_origin = NULL;
static char *option_branch = NULL;
+static struct string_list option_not = STRING_LIST_INIT_NODUP;
static const char *real_git_dir;
static char *option_upload_pack = "git-upload-pack";
static int option_verbosity;
@@ -52,6 +53,13 @@ static struct string_list option_config;
static struct string_list option_reference;
static int option_dissociate;
+static int option_parse_deepen_not(const struct option *opt,
+ const char *arg, int unset)
+{
+ string_list_append(&option_not, arg);
+ return 0;
+}
+
static struct option builtin_clone_options[] = {
OPT__VERBOSITY(&option_verbosity),
OPT_BOOL(0, "progress", &option_progress,
@@ -89,6 +97,9 @@ static struct option builtin_clone_options[] = {
N_("create a shallow clone of that depth")),
OPT_STRING(0, "shallow-since", &option_since, N_("time"),
N_("create a shallow clone since a specific time")),
+ { OPTION_CALLBACK, 0, "shallow-exclude", NULL, N_("revision"),
+ N_("deepen history of shallow clone by excluding rev"),
+ PARSE_OPT_NONEG, option_parse_deepen_not },
OPT_BOOL(0, "single-branch", &option_single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", &real_git_dir, N_("gitdir"),
@@ -852,7 +863,7 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
- if (option_depth || option_since)
+ if (option_depth || option_since || option_not.nr)
deepen = 1;
if (option_single_branch == -1)
option_single_branch = deepen ? 1 : 0;
@@ -983,6 +994,8 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
warning(_("--depth is ignored in local clones; use
file:// instead."));
if (option_since)
warning(_("--shallow-since is ignored in local clones;
use file:// instead."));
+ if (option_not.nr)
+ warning(_("--shallow-exclude is ignored in local
clones; use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow,
ignoring --local"));
@@ -1004,6 +1017,9 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (option_since)
transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
option_since);
+ if (option_not.nr)
+ transport_set_option(transport, TRANS_OPT_DEEPEN_NOT,
+(const char *)&option_not);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
--
2.7.0.377.g4cd97dd
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 17/25] clone: define shallow clone boundary based on time with --shallow-since
Signed-off-by: Nguyễn Thái Ngọc Duy
---
Documentation/git-clone.txt | 3 +++
builtin/clone.c | 16 +---
2 files changed, 16 insertions(+), 3 deletions(-)
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 789b668..1b6b639 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -194,6 +194,9 @@ objects from the source repository into a pack in the
cloned repository.
`--no-single-branch` is given to fetch the histories near the
tips of all branches.
+--shallow-since=::
+ Create a shallow clone with a history after the specified time.
+
--[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index bcba080..dc2ef4f 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -40,7 +40,8 @@ static const char * const builtin_clone_usage[] = {
static int option_no_checkout, option_bare, option_mirror,
option_single_branch = -1;
static int option_local = -1, option_no_hardlinks, option_shared,
option_recursive;
-static char *option_template, *option_depth;
+static int deepen;
+static char *option_template, *option_depth, *option_since;
static char *option_origin = NULL;
static char *option_branch = NULL;
static const char *real_git_dir;
@@ -86,6 +87,8 @@ static struct option builtin_clone_options[] = {
N_("path to git-upload-pack on the remote")),
OPT_STRING(0, "depth", &option_depth, N_("depth"),
N_("create a shallow clone of that depth")),
+ OPT_STRING(0, "shallow-since", &option_since, N_("time"),
+ N_("create a shallow clone since a specific time")),
OPT_BOOL(0, "single-branch", &option_single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", &real_git_dir, N_("gitdir"),
@@ -849,8 +852,10 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
+ if (option_depth || option_since)
+ deepen = 1;
if (option_single_branch == -1)
- option_single_branch = option_depth ? 1 : 0;
+ option_single_branch = deepen ? 1 : 0;
if (option_mirror)
option_bare = 1;
@@ -976,6 +981,8 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (is_local) {
if (option_depth)
warning(_("--depth is ignored in local clones; use
file:// instead."));
+ if (option_since)
+ warning(_("--shallow-since is ignored in local clones;
use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow,
ignoring --local"));
@@ -994,6 +1001,9 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (option_depth)
transport_set_option(transport, TRANS_OPT_DEPTH,
option_depth);
+ if (option_since)
+ transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
+option_since);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
@@ -1001,7 +1011,7 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
transport_set_option(transport, TRANS_OPT_UPLOADPACK,
option_upload_pack);
- if (transport->smart_options && !option_depth)
+ if (transport->smart_options && !deepen)
transport->smart_options->check_self_contained_and_connected =
1;
refs = transport_get_remote_refs(transport);
--
2.7.0.377.g4cd97dd
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH V4 1/2] glossary: define the term shallow clone
There are several places in the documentation that the term shallow clone is used. Defining the term enables its use elsewhere with a known definition. Signed-off-by: Stephen P. Smith --- Notes: The review comments for the user guide update[1] suggested a change in the definition of a shallow clone to: - note differences between a shallow clone and a shallow repository and - define it as a noun (which is the way the user guide update patch uses the term) [1] http://article.gmane.org/gmane.comp.version-control.git/283052 Documentation/glossary-content.txt | 5 + 1 file changed, 5 insertions(+) diff --git a/Documentation/glossary-content.txt b/Documentation/glossary-content.txt index e225974..cafc284 100644 --- a/Documentation/glossary-content.txt +++ b/Documentation/glossary-content.txt @@ -531,6 +531,11 @@ The most notable example is `HEAD`. "Secure Hash Algorithm 1"; a cryptographic hash function. In the context of Git used as a synonym for <>. +[[def_shallow_clone]]shallow clone:: + Mostly a synonym to <> + but the phrase makes it more explicit that it was created by + running `git clone --depth=...` command. + [[def_shallow_repository]]shallow repository:: A shallow <> has an incomplete history some of whose <> have <> cauterized away (in other -- 2.7.0-rc2 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 18/20] clone: define shallow clone boundary with --not
Signed-off-by: Nguyễn Thái Ngọc Duy
---
Documentation/git-clone.txt | 5 +
builtin/clone.c | 18 +-
2 files changed, 22 insertions(+), 1 deletion(-)
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 28993c6..3589e57 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -195,6 +195,11 @@ objects from the source repository into a pack in the
cloned repository.
--since=::
Create a 'shallow' clone with a history after the specified time.
+--not=::
+ Create a 'shallow' clone with a history, excluding commits
+ reachable from a specified remote branch or tag. This option
+ can be specified multiple times.
+
--[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index 94bbfef..0e99354 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -44,6 +44,7 @@ static int deepen;
static char *option_template, *option_depth, *option_since;
static char *option_origin = NULL;
static char *option_branch = NULL;
+static struct string_list option_not = STRING_LIST_INIT_NODUP;
static const char *real_git_dir;
static char *option_upload_pack = "git-upload-pack";
static int option_verbosity;
@@ -52,6 +53,13 @@ static struct string_list option_config;
static struct string_list option_reference;
static int option_dissociate;
+static int option_parse_deepen_not(const struct option *opt,
+ const char *arg, int unset)
+{
+ string_list_append(&option_not, arg);
+ return 0;
+}
+
static struct option builtin_clone_options[] = {
OPT__VERBOSITY(&option_verbosity),
OPT_BOOL(0, "progress", &option_progress,
@@ -89,6 +97,9 @@ static struct option builtin_clone_options[] = {
N_("create a shallow clone of that depth")),
OPT_STRING(0, "since", &option_since, N_("time"),
N_("create a shallow clone since a specific time")),
+ { OPTION_CALLBACK, 0, "not", NULL, N_("revision"),
+ N_("deepen history of shallow clone by excluding rev"),
+ PARSE_OPT_NONEG, option_parse_deepen_not },
OPT_BOOL(0, "single-branch", &option_single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", &real_git_dir, N_("gitdir"),
@@ -849,7 +860,7 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
- if (option_depth || option_since)
+ if (option_depth || option_since || option_not.nr)
deepen = 1;
if (option_single_branch == -1)
option_single_branch = deepen ? 1 : 0;
@@ -980,6 +991,8 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
warning(_("--depth is ignored in local clones; use
file:// instead."));
if (option_since)
warning(_("--since is ignored in local clones; use
file:// instead."));
+ if (option_not.nr)
+ warning(_("--not is ignored in local clones; use
file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow,
ignoring --local"));
@@ -1001,6 +1014,9 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (option_since)
transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
option_since);
+ if (option_not.nr)
+ transport_set_option(transport, TRANS_OPT_DEEPEN_NOT,
+(const char *)&option_not);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
--
2.3.0.rc1.137.g477eb31
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 13/20] clone: define shallow clone boundary based on time with --since
Signed-off-by: Nguyễn Thái Ngọc Duy
---
Documentation/git-clone.txt | 3 +++
builtin/clone.c | 16 +---
2 files changed, 16 insertions(+), 3 deletions(-)
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 6bf000d..28993c6 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -192,6 +192,9 @@ objects from the source repository into a pack in the
cloned repository.
Create a 'shallow' clone with a history truncated to the
specified number of revisions.
+--since=::
+ Create a 'shallow' clone with a history after the specified time.
+
--[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index a0b3cd9..94bbfef 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -40,7 +40,8 @@ static const char * const builtin_clone_usage[] = {
static int option_no_checkout, option_bare, option_mirror,
option_single_branch = -1;
static int option_local = -1, option_no_hardlinks, option_shared,
option_recursive;
-static char *option_template, *option_depth;
+static int deepen;
+static char *option_template, *option_depth, *option_since;
static char *option_origin = NULL;
static char *option_branch = NULL;
static const char *real_git_dir;
@@ -86,6 +87,8 @@ static struct option builtin_clone_options[] = {
N_("path to git-upload-pack on the remote")),
OPT_STRING(0, "depth", &option_depth, N_("depth"),
N_("create a shallow clone of that depth")),
+ OPT_STRING(0, "since", &option_since, N_("time"),
+ N_("create a shallow clone since a specific time")),
OPT_BOOL(0, "single-branch", &option_single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", &real_git_dir, N_("gitdir"),
@@ -846,8 +849,10 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
+ if (option_depth || option_since)
+ deepen = 1;
if (option_single_branch == -1)
- option_single_branch = option_depth ? 1 : 0;
+ option_single_branch = deepen ? 1 : 0;
if (option_mirror)
option_bare = 1;
@@ -973,6 +978,8 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (is_local) {
if (option_depth)
warning(_("--depth is ignored in local clones; use
file:// instead."));
+ if (option_since)
+ warning(_("--since is ignored in local clones; use
file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow,
ignoring --local"));
@@ -991,6 +998,9 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (option_depth)
transport_set_option(transport, TRANS_OPT_DEPTH,
option_depth);
+ if (option_since)
+ transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
+option_since);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
@@ -998,7 +1008,7 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
transport_set_option(transport, TRANS_OPT_UPLOADPACK,
option_upload_pack);
- if (transport->smart_options && !option_depth)
+ if (transport->smart_options && !deepen)
transport->smart_options->check_self_contained_and_connected =
1;
refs = transport_get_remote_refs(transport);
--
2.3.0.rc1.137.g477eb31
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH V3 1/2] glossary: define the term shallow clone
There are several places in the documentation that the term shallow clone is used. Defining the term enables its use elsewhere with a known definition. Signed-off-by: Stephen P. Smith --- Documentation/glossary-content.txt | 4 1 file changed, 4 insertions(+) diff --git a/Documentation/glossary-content.txt b/Documentation/glossary-content.txt index e225974..cecc83d 100644 --- a/Documentation/glossary-content.txt +++ b/Documentation/glossary-content.txt @@ -531,6 +531,10 @@ The most notable example is `HEAD`. "Secure Hash Algorithm 1"; a cryptographic hash function. In the context of Git used as a synonym for <>. +[[def_shallow_clone]]shallow clone:: + A clone of a <> which creates a + <>. + [[def_shallow_repository]]shallow repository:: A shallow <> has an incomplete history some of whose <> have <> cauterized away (in other -- 2.6.3.368.gf34be46 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH V2 1/2] glossary: define the term shallow clone
There are several places in the documentation that the term shallow clone is used. Defining the term enables its use elsewhere with a known definition. --- Documentation/glossary-content.txt | 4 1 file changed, 4 insertions(+) diff --git a/Documentation/glossary-content.txt b/Documentation/glossary-content.txt index e225974..cecc83d 100644 --- a/Documentation/glossary-content.txt +++ b/Documentation/glossary-content.txt @@ -531,6 +531,10 @@ The most notable example is `HEAD`. "Secure Hash Algorithm 1"; a cryptographic hash function. In the context of Git used as a synonym for <>. +[[def_shallow_clone]]shallow clone:: + A clone of a <> which creates a + <>. + [[def_shallow_repository]]shallow repository:: A shallow <> has an incomplete history some of whose <> have <> cauterized away (in other -- 2.6.3.368.gf34be46 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/2] Define the term shallow clone.
On Mon, Dec 21, 2015 at 9:09 PM, Stephen P. Smith wrote: > There are several places in the documentation that > the term shallow clone is used. Defining the term > enables its use elsewhere with a known definition. > > Signed-off-by: Stephen P. Smith > --- > diff --git a/Documentation/glossary-content.txt > b/Documentation/glossary-content.txt > @@ -531,6 +531,10 @@ The most notable example is `HEAD`. > "Secure Hash Algorithm 1"; a cryptographic hash function. > In the context of Git used as a synonym for < name>>. > > +[[def_shallow_clone]]shallow clone:: > + A clone of a <> which creates a > +<>. Botched indentation on second line of definition. Use tab rather than spaces. > [[def_shallow_repository]]shallow repository:: > A shallow <> has an incomplete > history some of whose <> have > <> cauterized away (in other > -- > 2.6.3.368.gf34be46 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/2] Define the term shallow clone.
There are several places in the documentation that the term shallow clone is used. Defining the term enables its use elsewhere with a known definition. Signed-off-by: Stephen P. Smith --- Documentation/glossary-content.txt | 4 1 file changed, 4 insertions(+) diff --git a/Documentation/glossary-content.txt b/Documentation/glossary-content.txt index e225974..d384aed 100644 --- a/Documentation/glossary-content.txt +++ b/Documentation/glossary-content.txt @@ -531,6 +531,10 @@ The most notable example is `HEAD`. "Secure Hash Algorithm 1"; a cryptographic hash function. In the context of Git used as a synonym for <>. +[[def_shallow_clone]]shallow clone:: + A clone of a <> which creates a +<>. + [[def_shallow_repository]]shallow repository:: A shallow <> has an incomplete history some of whose <> have <> cauterized away (in other -- 2.6.3.368.gf34be46 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Multiple fetches when unshallowing a shallow clone
On Mon, Dec 07, 2015 at 01:27:51PM -0800, Junio C Hamano wrote:
> Jeff King writes:
>
> > I think one thing I was missing is that we need to just grab the
> > _object_, but we need to realize that the ref needs updating[1]. So we
> > cannot skip backfill of any tag that we do not already have, even if we
> > already have the tag object.
> > ...
> > [1] I'm still puzzled why find_non_local_tags uses has_sha1_file() on
> > the tag object at all, then.
>
> The designed semantics of auto-following tags (not necessarily as
> implemented or documented, i.e. there may be implementation or
> documentation bugs), I think, is to arrive at the same state as
> doing a fetch (or a push) without the auto-following and then doing
> a separate fetch (or a push) of tags that point at the objects that
> are reachable from the tips of refs after finishing the first
> (i.e. without auto-follow) fetch (or a push). In a scenario where
> we already have a commit reachable from existing remote-tracking
> branch and the current transfer (be it a fetch or a push, with or
> without auto-follow) does not update any remote-tracking branch
> (because the source side did not have any changes), if the source
> side added a tag that refers to that commit that the receiving end
> lacks, that tag needs to be transferred and then stored.
>
> So has_sha1_file() is not the right test---if anything, it needs to
> be checking if the object being checked is reachable from a tip of
> some ref.
>
> But of course, that test is rather expensive, so perhaps the
> implementation cheated and uses has_sha1_file() instead? The only
> case it would misidentify would be after an aborted fetch (or push)
> left unconnected island of objects and some of these objects that
> are not reachable are pointed at by tags the receiving end does not
> have.
I may have confused myself. There are actually two has_sha1_file() calls
in find_non_local_tags.
I agree it is the only sensible test for "do we have the commit this tag
peels to, and if so, we want to grab the tag". Reachability is too
expensive to compute.
But for the other one ("do we have the tag object itself"), I initially
claimed "if we have the tag object already, we do not have to do the
backfill fetch". Which is not quite true. We have to update the ref even
if we have the tag object. But then, what if we have the tag object for
other reasons (e.g., because another tag points at it?).
E.g. in this sequence:
git -C parent commit --allow-empty -m base
git -C parent tag -m mytag foo
git clone parent child
git -C parent update-ref refs/tags/bar foo
git -C child fetch
we must backfill refs/tags/bar during the fetch, even though we already
have the object. I don't see any point in checking has_sha1_file() for
the tagged object at all. If we don't have it, we obviously must fetch.
And if we do have it, we must fetch the ref, even if that results in no
objects transferred.
It's entirely possible I'm just confused, and AFAICT nobody has noticed
any breakage here, so please don't feel you need to spend a lot of time
humoring me. I'm just writing up my confusion for posterity. :)
-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Multiple fetches when unshallowing a shallow clone
Jeff King writes: > I think one thing I was missing is that we need to just grab the > _object_, but we need to realize that the ref needs updating[1]. So we > cannot skip backfill of any tag that we do not already have, even if we > already have the tag object. > ... > [1] I'm still puzzled why find_non_local_tags uses has_sha1_file() on > the tag object at all, then. The designed semantics of auto-following tags (not necessarily as implemented or documented, i.e. there may be implementation or documentation bugs), I think, is to arrive at the same state as doing a fetch (or a push) without the auto-following and then doing a separate fetch (or a push) of tags that point at the objects that are reachable from the tips of refs after finishing the first (i.e. without auto-follow) fetch (or a push). In a scenario where we already have a commit reachable from existing remote-tracking branch and the current transfer (be it a fetch or a push, with or without auto-follow) does not update any remote-tracking branch (because the source side did not have any changes), if the source side added a tag that refers to that commit that the receiving end lacks, that tag needs to be transferred and then stored. So has_sha1_file() is not the right test---if anything, it needs to be checking if the object being checked is reachable from a tip of some ref. But of course, that test is rather expensive, so perhaps the implementation cheated and uses has_sha1_file() instead? The only case it would misidentify would be after an aborted fetch (or push) left unconnected island of objects and some of these objects that are not reachable are pointed at by tags the receiving end does not have. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Multiple fetches when unshallowing a shallow clone
On Mon, Dec 7, 2015 at 8:57 PM, Jason Paller-Rzepka wrote:
> Duy, you mentioned that depth=0 means "do not change depth". I assume that
> means the server should use exactly the shallows that the client sent, and
> it does not need to traverse the tree or modify the shallow or unshallow
> sets at all. Right?
Correct. The server might send new shallow lines anyway though, if the
server repo is also shallow and the new fetched ref needs to be cut.
But I don't know if Dulwich supports that yet.
> Duy, you also mentioned that "those lines should be rejected any way". You
> just mean that a "deepen 0" line should be rejected, right? And that's
> because the right way to tell git-upload-pack not to change the depth is to
> omit the "deepen" line after the "shallow" lines, so there's never a need to
> send "deepen 0"?
Also correct. I didn't check the code when I wrote that. But I have
checked and upload-pack does reject "deepen 0"
if (starts_with(line, "deepen ")) {
char *end;
depth = strtol(line + 7, &end, 0);
if (end == line + 7 || depth <= 0)
die("Invalid deepen: %s", line);
--
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Multiple fetches when unshallowing a shallow clone
I should point out that I never saw a "deepen 0" line. Rather, I saw
git send "option depth 0" to the remote helper.
Duy, you mentioned that depth=0 means "do not change depth". I assume
that means the server should use exactly the shallows that the client
sent, and it does not need to traverse the tree or modify the shallow
or unshallow sets at all. Right?
Duy, you also mentioned that "those lines should be rejected any way".
You just mean that a "deepen 0" line should be rejected, right? And
that's because the right way to tell git-upload-pack not to change the
depth is to omit the "deepen" line after the "shallow" lines, so
there's never a need to send "deepen 0"?
Thanks!
On Sun, Dec 6, 2015 at 5:46 AM, Duy Nguyen wrote:
> On Sun, Dec 6, 2015 at 8:01 AM, Jeff King wrote:
>> On Sun, Dec 06, 2015 at 01:37:18AM -0500, Jeff King wrote:
>>
>>> And indeed, replacing the logic with what I wrote does make the backfill
>>> go away in my test case. But it's so far from what is there that I feel
>>> like I must be missing something.
>>
>> I think one thing I was missing is that we need to just grab the
>> _object_, but we need to realize that the ref needs updating[1]. So we
>> cannot skip backfill of any tag that we do not already have, even if we
>> already have the tag object.
>
> It's probably worth adding a few comment lines about this. I did
> search back commit history but did not get this.
>
>> Which made me wonder why this:
>>
>> git init parent &&
>> git -C parent commit --allow-empty -m one &&
>> git clone parent child &&
>> git -C parent commit --allow-empty -m two &&
>> git -C parent tag -m mytag foo &&
>> git -C parent commit --allow-empty -m three &&
>> git -C child fetch
>>
>> does not appear to need to backfill to pick up refs/tags/foo. But it
>> does. It's just that it hits the quickfetch() code path and does not
>> have to ask the other side for a pack. And that explains why it does hit
>> in the --shallow case: we explicitly disable quickfetch in such cases.
>>
>> For the unshallow case, of course we could use it (but only for the
>> second, backfill fetch). Something like this seems to work for me:
>>
>> diff --git a/builtin/fetch.c b/builtin/fetch.c
>> index ed84963..b33b90f 100644
>> --- a/builtin/fetch.c
>> +++ b/builtin/fetch.c
>> @@ -881,6 +881,8 @@ static void backfill_tags(struct transport *transport,
>> struct ref *ref_map)
>>
>> transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, NULL);
>> transport_set_option(transport, TRANS_OPT_DEPTH, "0");
>> + if (unshallow)
>> + depth = NULL;
>> fetch_refs(transport, ref_map);
>>
>> if (gsecondary) {
>>
>> But I admit I am not at all confident that it doesn't cause other
>> problems, or that it covers all cases. Even in a shallow repo, we should
>> be able to quickfetch individual tags, shouldn't we?
>
> Yes. depth is only non-NULL when you pass --depth (or --unshallow).
> quickfetch should happen when you fetch without those options.
>
>> I wonder if we could just always set "depth = NULL" here.
>
> --unshallow is essentially --depth=, so I don't see why
> --unshallow should be singled out here. We probably want to restore
> depth back (or pass a flag to explicitly ignore the "depth" exception
> in quickfetch). For multiple fetches, we spawn new commands so depth
> being NULL does not harm. Just in case somebody tries to fetch a
> couple more times in the same process in future.
> --
> Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Multiple fetches when unshallowing a shallow clone
On Sun, Dec 6, 2015 at 8:01 AM, Jeff King wrote:
> On Sun, Dec 06, 2015 at 01:37:18AM -0500, Jeff King wrote:
>
>> And indeed, replacing the logic with what I wrote does make the backfill
>> go away in my test case. But it's so far from what is there that I feel
>> like I must be missing something.
>
> I think one thing I was missing is that we need to just grab the
> _object_, but we need to realize that the ref needs updating[1]. So we
> cannot skip backfill of any tag that we do not already have, even if we
> already have the tag object.
It's probably worth adding a few comment lines about this. I did
search back commit history but did not get this.
> Which made me wonder why this:
>
> git init parent &&
> git -C parent commit --allow-empty -m one &&
> git clone parent child &&
> git -C parent commit --allow-empty -m two &&
> git -C parent tag -m mytag foo &&
> git -C parent commit --allow-empty -m three &&
> git -C child fetch
>
> does not appear to need to backfill to pick up refs/tags/foo. But it
> does. It's just that it hits the quickfetch() code path and does not
> have to ask the other side for a pack. And that explains why it does hit
> in the --shallow case: we explicitly disable quickfetch in such cases.
>
> For the unshallow case, of course we could use it (but only for the
> second, backfill fetch). Something like this seems to work for me:
>
> diff --git a/builtin/fetch.c b/builtin/fetch.c
> index ed84963..b33b90f 100644
> --- a/builtin/fetch.c
> +++ b/builtin/fetch.c
> @@ -881,6 +881,8 @@ static void backfill_tags(struct transport *transport,
> struct ref *ref_map)
>
> transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, NULL);
> transport_set_option(transport, TRANS_OPT_DEPTH, "0");
> + if (unshallow)
> + depth = NULL;
> fetch_refs(transport, ref_map);
>
> if (gsecondary) {
>
> But I admit I am not at all confident that it doesn't cause other
> problems, or that it covers all cases. Even in a shallow repo, we should
> be able to quickfetch individual tags, shouldn't we?
Yes. depth is only non-NULL when you pass --depth (or --unshallow).
quickfetch should happen when you fetch without those options.
> I wonder if we could just always set "depth = NULL" here.
--unshallow is essentially --depth=, so I don't see why
--unshallow should be singled out here. We probably want to restore
depth back (or pass a flag to explicitly ignore the "depth" exception
in quickfetch). For multiple fetches, we spawn new commands so depth
being NULL does not harm. Just in case somebody tries to fetch a
couple more times in the same process in future.
--
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Multiple fetches when unshallowing a shallow clone
On Sun, Dec 06, 2015 at 01:37:18AM -0500, Jeff King wrote:
> And indeed, replacing the logic with what I wrote does make the backfill
> go away in my test case. But it's so far from what is there that I feel
> like I must be missing something.
I think one thing I was missing is that we need to just grab the
_object_, but we need to realize that the ref needs updating[1]. So we
cannot skip backfill of any tag that we do not already have, even if we
already have the tag object.
Which made me wonder why this:
git init parent &&
git -C parent commit --allow-empty -m one &&
git clone parent child &&
git -C parent commit --allow-empty -m two &&
git -C parent tag -m mytag foo &&
git -C parent commit --allow-empty -m three &&
git -C child fetch
does not appear to need to backfill to pick up refs/tags/foo. But it
does. It's just that it hits the quickfetch() code path and does not
have to ask the other side for a pack. And that explains why it does hit
in the --shallow case: we explicitly disable quickfetch in such cases.
For the unshallow case, of course we could use it (but only for the
second, backfill fetch). Something like this seems to work for me:
diff --git a/builtin/fetch.c b/builtin/fetch.c
index ed84963..b33b90f 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -881,6 +881,8 @@ static void backfill_tags(struct transport *transport,
struct ref *ref_map)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, NULL);
transport_set_option(transport, TRANS_OPT_DEPTH, "0");
+ if (unshallow)
+ depth = NULL;
fetch_refs(transport, ref_map);
if (gsecondary) {
But I admit I am not at all confident that it doesn't cause other
problems, or that it covers all cases. Even in a shallow repo, we should
be able to quickfetch individual tags, shouldn't we? I wonder if we
could just always set "depth = NULL" here.
-Peff
[1] I'm still puzzled why find_non_local_tags uses has_sha1_file() on
the tag object at all, then.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Multiple fetches when unshallowing a shallow clone
On Sat, Dec 05, 2015 at 08:00:28PM -0800, Junio C Hamano wrote:
> > I suppose followtags feature has been around long enough that we can
> > simply trust that and skip the second fetch?
>
> H, I wonder why the code needs the backfill fetch while talking
> to a server that has the include-tag capability, then?
The logic in find_non_local_tags makes no sense to me.
After we do the first fetch, we call this function to find out if there
are any tags we need to pick up. We iterate through the list of refs
given to us by the remote (including peeled lines), and do:
if (ends_with(ref->name, "^{}")) {
if (item && !has_sha1_file(ref->old_sha1) &&
!will_fetch(head, ref->old_sha1) &&
!has_sha1_file(item->util) &&
!will_fetch(head, item->util))
item->util = NULL;
...
}
where "ref" is the peeled line (i.e., the pointed-to commit), and "item"
is the preceding line (i.e., the tag itself) with util set to its sha1.
Any such item whose util survives as non-NULL is fetched in the backfill
step.
So I'd think the logic for backfilling a given tag should be:
1. We have the peeled object, and...
2. We don't currently have the tag pointing to it, and...
3. We are not already going to fetch it.
You could write that as:
if (has_sha1_file(ref->old_sha1) &&
!has_sha1_file(item->util) &&
!will_fetch(head, item->util))
Of course the conditional in the code is "should we skip the backfill",
the negation. So using De Morgan's laws, we'd write:
if (!has_sha1_file(ref->old_sha1) ||
has_sha1_file(item->util) ||
will_fetch(head, item->util))
Which is quite a bit different than what is there. I'm not sure at all
what the "will_fetch(head, ref->old_sha1)" is doing. In fact, for the
backfill step, it looks like we feed an empty "head" list, so the
!will_fetch() is always true.
And indeed, replacing the logic with what I wrote does make the backfill
go away in my test case. But it's so far from what is there that I feel
like I must be missing something.
-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Multiple fetches when unshallowing a shallow clone
Duy Nguyen writes: > On Fri, Dec 4, 2015 at 11:45 PM, Junio C Hamano wrote: > ... >> just like a regular fetch that auto-follows tags, where it has to >> make a second fetch if the primary fetch fails to include everything >> that is needed for propagating the tag for whatever reason. >> >> Having said that, IIRC, these days a depth limited clone is created >> implicitly with --single-branch option, and I am not sure what the >> right behaviour for the auto-following of tags in such a repository. > > I suppose followtags feature has been around long enough that we can > simply trust that and skip the second fetch? H, I wonder why the code needs the backfill fetch while talking to a server that has the include-tag capability, then? -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Multiple fetches when unshallowing a shallow clone
On Fri, Dec 4, 2015 at 11:45 PM, Junio C Hamano wrote: > Jeff King writes: > >>> But why would fetching a tag (or set of tags) merit a depth of zero? >>> Doesn't depth 1 mean "give me the the objects, and none of their >>> descendants"? Why use 0? >> >> That comes from this line: >> >> transport_set_option(transport, TRANS_OPT_DEPTH, "0"); >> >> That line blame back to b888d61 (Make fetch a builtin, 2007-09-10), >> which isn't incredibly helpful. > > Hmm, "0" means "no depth limitations", which is exactly what we want > in this "unshallow" case, I would think. The behaviour observed is No depth 0 means "do not change depth", which is why Jeff saw no 'deepen' lines (and those lines should be rejected any way). It's equivalent of doing "git fetch" without --depth. > just like a regular fetch that auto-follows tags, where it has to > make a second fetch if the primary fetch fails to include everything > that is needed for propagating the tag for whatever reason. > > Having said that, IIRC, these days a depth limited clone is created > implicitly with --single-branch option, and I am not sure what the > right behaviour for the auto-following of tags in such a repository. I suppose followtags feature has been around long enough that we can simply trust that and skip the second fetch? But it's not that easy for subsequent fetches after the initial fetch in git-clone, because we no longer know if --single-branch was used (of if there is any new branch fetched since). -- Duy -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Multiple fetches when unshallowing a shallow clone
Jeff King writes: >> But why would fetching a tag (or set of tags) merit a depth of zero? >> Doesn't depth 1 mean "give me the the objects, and none of their >> descendants"? Why use 0? > > That comes from this line: > > transport_set_option(transport, TRANS_OPT_DEPTH, "0"); > > That line blame back to b888d61 (Make fetch a builtin, 2007-09-10), > which isn't incredibly helpful. Hmm, "0" means "no depth limitations", which is exactly what we want in this "unshallow" case, I would think. The behaviour observed is just like a regular fetch that auto-follows tags, where it has to make a second fetch if the primary fetch fails to include everything that is needed for propagating the tag for whatever reason. Having said that, IIRC, these days a depth limited clone is created implicitly with --single-branch option, and I am not sure what the right behaviour for the auto-following of tags in such a repository. > I think that comes from the original git-fetch.sh, which had: > > ?*) > # do not deepen a shallow tree when following tags > shallow_depth= > > Which makes sense. I think the code at that point is not aware that we > just "unshallowed" and can therefore drop the depth parameter > altogether. But I admit I am not all that familiar with the shallow > code. > > +cc Duy, who can probably say something way more intelligent about this > off the top of his head. :) > > -Peff -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Multiple fetches when unshallowing a shallow clone
On Fri, Dec 04, 2015 at 01:57:30PM -0800, Junio C Hamano wrote: > Jeff King writes: > > > This seems to reproduce consistently for me: > > > > $ git clone --depth=1 git://github.com/git/git > > Cloning into 'git'... > > remote: Counting objects: 2925, done. > > remote: Compressing objects: 100% (2602/2602), done. > > remote: Total 2925 (delta 230), reused 2329 (delta 206), pack-reused 0 > > Receiving objects: 100% (2925/2925), 6.17 MiB | 10.80 MiB/s, done. > > Resolving deltas: 100% (230/230), done. > > > > $ cd git > > $ git fetch --unshallow > > remote: Counting objects: 185430, done. > > remote: Compressing objects: 100% (46933/46933), done. > > remote: Total 185430 (delta 140505), reused 181589 (delta 136694), > > pack-reused 0 > > Receiving objects: 100% (185430/185430), 52.80 MiB | 10.84 MiB/s, done. > > Resolving deltas: 100% (140505/140505), completed with 1784 local objects. > > remote: Counting objects: 579, done. > > remote: Compressing objects: 100% (579/579), done. > > remote: Total 579 (delta 0), reused 579 (delta 0), pack-reused 0 > > Receiving objects: 100% (579/579), 266.85 KiB | 0 bytes/s, done. > > [... fetch output ...] > > > > That looks like two packs being received for the --unshallow case. > > What is puzzling is that I do not seem to see this "two fetches" > with the local transport. I only see "deepen 2147483647" in the > protocol log. Yeah, I do not ever see "deepen 0" from GIT_TRACE_PACKET output. FWIW, here's the output I am using to reproduce this locally: # do this once git clone --bare git://github.com/git/git src.git # do this for each test run rm -rf repo git clone --no-local --depth=1 src.git repo cd repo echo "==> unshallow" && git fetch --progress --unshallow 2>&1 | grep remote And you can see that there are two separate sections (and I traced this to backfill_tags() with gdb). Note that the issue goes away if the shallow clone is done with "--bare" (I guess because we pick up tags differently in the initial clone?). -Peff -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Multiple fetches when unshallowing a shallow clone
Jeff King writes: > This seems to reproduce consistently for me: > > $ git clone --depth=1 git://github.com/git/git > Cloning into 'git'... > remote: Counting objects: 2925, done. > remote: Compressing objects: 100% (2602/2602), done. > remote: Total 2925 (delta 230), reused 2329 (delta 206), pack-reused 0 > Receiving objects: 100% (2925/2925), 6.17 MiB | 10.80 MiB/s, done. > Resolving deltas: 100% (230/230), done. > > $ cd git > $ git fetch --unshallow > remote: Counting objects: 185430, done. > remote: Compressing objects: 100% (46933/46933), done. > remote: Total 185430 (delta 140505), reused 181589 (delta 136694), > pack-reused 0 > Receiving objects: 100% (185430/185430), 52.80 MiB | 10.84 MiB/s, done. > Resolving deltas: 100% (140505/140505), completed with 1784 local objects. > remote: Counting objects: 579, done. > remote: Compressing objects: 100% (579/579), done. > remote: Total 579 (delta 0), reused 579 (delta 0), pack-reused 0 > Receiving objects: 100% (579/579), 266.85 KiB | 0 bytes/s, done. > [... fetch output ...] > > That looks like two packs being received for the --unshallow case. What is puzzling is that I do not seem to see this "two fetches" with the local transport. I only see "deepen 2147483647" in the protocol log. Moreover, the only interesting lines in the output from $ git grep -B1 'deepen ' \*.[ch] are fetch-pack.c- if (args->depth > 0) fetch-pack.c: packet_buf_write(&req_buf, "deepen %d", args->depth); so I do not see how anybody would be sending "deepen 0" as Jason saw. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Multiple fetches when unshallowing a shallow clone
On Fri, Dec 04, 2015 at 04:38:16PM -0500, Jason Paller-Rzepka wrote: > It appears that it happens when the shallow history grows to include a > commit that's pointed to by a previously unseen tag. For example, > when I deepen a checkout of git to depth 8, I hit v2.5.2, and a second > fetch takes place. Yeah. The code is in builtin/fetch.c:backfill_tags. > But why would fetching a tag (or set of tags) merit a depth of zero? > Doesn't depth 1 mean "give me the the objects, and none of their > descendants"? Why use 0? That comes from this line: transport_set_option(transport, TRANS_OPT_DEPTH, "0"); That line blame back to b888d61 (Make fetch a builtin, 2007-09-10), which isn't incredibly helpful. I think that comes from the original git-fetch.sh, which had: ?*) # do not deepen a shallow tree when following tags shallow_depth= Which makes sense. I think the code at that point is not aware that we just "unshallowed" and can therefore drop the depth parameter altogether. But I admit I am not all that familiar with the shallow code. +cc Duy, who can probably say something way more intelligent about this off the top of his head. :) -Peff -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Multiple fetches when unshallowing a shallow clone
I can reproduce it now. Instead of using my $random version, I just needed origin/master to reproduce. The second fetch is invoked via (as outputted via GIT_TRACE=1 git -C git fetch --depth=8) 13:44:56.863841 run-command.c:343 trace: run_command: 'fetch-pack' '--stateless-rpc' '--stdin' '--lock-pack' '--thin' 'https://github.com/git/git/' so it seems like there is no explicit depth given, so I think the 0 comes from the initialization step and nobody touched it to fill with meaningful values. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Multiple fetches when unshallowing a shallow clone
It appears that it happens when the shallow history grows to include a commit that's pointed to by a previously unseen tag. For example, when I deepen a checkout of git to depth 8, I hit v2.5.2, and a second fetch takes place. ``` $ git clone --depth=1 http://github.com/git/git Cloning into 'git'... remote: Counting objects: 2925, done. remote: Compressing objects: 100% (2602/2602), done. remote: Total 2925 (delta 230), reused 2329 (delta 206), pack-reused 0 Receiving objects: 100% (2925/2925), 6.17 MiB | 0 bytes/s, done. Resolving deltas: 100% (230/230), done. Checking connectivity... done. $ git -C git fetch --depth=8 remote: Counting objects: 858, done. remote: Compressing objects: 100% (774/774), done. remote: Total 858 (delta 793), reused 138 (delta 80), pack-reused 0 Receiving objects: 100% (858/858), 364.53 KiB | 0 bytes/s, done. Resolving deltas: 100% (793/793), completed with 476 local objects. remote: Counting objects: 1, done. remote: Total 1 (delta 0), reused 1 (delta 0), pack-reused 0 Unpacking objects: 100% (1/1), done. >From http://github.com/git/git * [new tag] v2.5.2 -> v2.5.2 $ ``` But why would fetching a tag (or set of tags) merit a depth of zero? Doesn't depth 1 mean "give me the the objects, and none of their descendants"? Why use 0? Thanks! Jason On Fri, Dec 4, 2015 at 4:27 PM, Jeff King wrote: > On Fri, Dec 04, 2015 at 12:46:59PM -0800, Stefan Beller wrote: > >> On Mon, Nov 30, 2015 at 11:35 AM, Jason Paller-Rzepka >> wrote: >> > Hi all, >> > >> > Would anyone be willing to help me understand some shallow-clone >> > behavior? (I found a bug in Dulwich, and I'm looking for some context >> > so I can determine how to fix it.) >> > >> > I learned that cgit sometimes performs two fetches for a `git fetch >> > --unshallow`: one with depth 'infinity', and a subsequent one with >> > depth zero. >> >> Is there a condition to trigger this 'sometimes' ? >> >> I just tried reproducing via >> $ GIT_TRACE=1 git fetch --unshallow >> >> and could not see a second fetch, but only a >> fetch-pack with --depth=2147483647 > > This seems to reproduce consistently for me: > > $ git clone --depth=1 git://github.com/git/git > Cloning into 'git'... > remote: Counting objects: 2925, done. > remote: Compressing objects: 100% (2602/2602), done. > remote: Total 2925 (delta 230), reused 2329 (delta 206), pack-reused 0 > Receiving objects: 100% (2925/2925), 6.17 MiB | 10.80 MiB/s, done. > Resolving deltas: 100% (230/230), done. > > $ cd git > $ git fetch --unshallow > remote: Counting objects: 185430, done. > remote: Compressing objects: 100% (46933/46933), done. > remote: Total 185430 (delta 140505), reused 181589 (delta 136694), > pack-reused 0 > Receiving objects: 100% (185430/185430), 52.80 MiB | 10.84 MiB/s, done. > Resolving deltas: 100% (140505/140505), completed with 1784 local objects. > remote: Counting objects: 579, done. > remote: Compressing objects: 100% (579/579), done. > remote: Total 579 (delta 0), reused 579 (delta 0), pack-reused 0 > Receiving objects: 100% (579/579), 266.85 KiB | 0 bytes/s, done. > [... fetch output ...] > > That looks like two packs being received for the --unshallow case. > > -Peff -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Multiple fetches when unshallowing a shallow clone
On Fri, Dec 4, 2015 at 1:27 PM, Jeff King wrote: >> >> and could not see a second fetch, but only a >> fetch-pack with --depth=2147483647 > > This seems to reproduce consistently for me: > > $ git clone --depth=1 git://github.com/git/git I used the http protocol, so I guess that's the difference. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Multiple fetches when unshallowing a shallow clone
On Fri, Dec 04, 2015 at 12:46:59PM -0800, Stefan Beller wrote: > On Mon, Nov 30, 2015 at 11:35 AM, Jason Paller-Rzepka > wrote: > > Hi all, > > > > Would anyone be willing to help me understand some shallow-clone > > behavior? (I found a bug in Dulwich, and I'm looking for some context > > so I can determine how to fix it.) > > > > I learned that cgit sometimes performs two fetches for a `git fetch > > --unshallow`: one with depth 'infinity', and a subsequent one with > > depth zero. > > Is there a condition to trigger this 'sometimes' ? > > I just tried reproducing via > $ GIT_TRACE=1 git fetch --unshallow > > and could not see a second fetch, but only a > fetch-pack with --depth=2147483647 This seems to reproduce consistently for me: $ git clone --depth=1 git://github.com/git/git Cloning into 'git'... remote: Counting objects: 2925, done. remote: Compressing objects: 100% (2602/2602), done. remote: Total 2925 (delta 230), reused 2329 (delta 206), pack-reused 0 Receiving objects: 100% (2925/2925), 6.17 MiB | 10.80 MiB/s, done. Resolving deltas: 100% (230/230), done. $ cd git $ git fetch --unshallow remote: Counting objects: 185430, done. remote: Compressing objects: 100% (46933/46933), done. remote: Total 185430 (delta 140505), reused 181589 (delta 136694), pack-reused 0 Receiving objects: 100% (185430/185430), 52.80 MiB | 10.84 MiB/s, done. Resolving deltas: 100% (140505/140505), completed with 1784 local objects. remote: Counting objects: 579, done. remote: Compressing objects: 100% (579/579), done. remote: Total 579 (delta 0), reused 579 (delta 0), pack-reused 0 Receiving objects: 100% (579/579), 266.85 KiB | 0 bytes/s, done. [... fetch output ...] That looks like two packs being received for the --unshallow case. -Peff -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Multiple fetches when unshallowing a shallow clone
On Mon, Nov 30, 2015 at 11:35 AM, Jason Paller-Rzepka wrote: > Hi all, > > Would anyone be willing to help me understand some shallow-clone > behavior? (I found a bug in Dulwich, and I'm looking for some context > so I can determine how to fix it.) > > I learned that cgit sometimes performs two fetches for a `git fetch > --unshallow`: one with depth 'infinity', and a subsequent one with > depth zero. Is there a condition to trigger this 'sometimes' ? I just tried reproducing via $ GIT_TRACE=1 git fetch --unshallow and could not see a second fetch, but only a fetch-pack with --depth=2147483647 > > Could anyone answer: > 1) What is the purpose of the second fetch? > 2) What does this depth of zero mean? Is it the same as a depth of > infinity? (I assume not... but, since I thought the smallest > meaningful depth was 1, I don't know what else it might mean.) > > Thank you! > Jason > -- > To unsubscribe from this list: send the line "unsubscribe git" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Multiple fetches when unshallowing a shallow clone
Hi all, Would anyone be willing to help me understand some shallow-clone behavior? (I found a bug in Dulwich, and I'm looking for some context so I can determine how to fix it.) I learned that cgit sometimes performs two fetches for a `git fetch --unshallow`: one with depth 'infinity', and a subsequent one with depth zero. Could anyone answer: 1) What is the purpose of the second fetch? 2) What does this depth of zero mean? Is it the same as a depth of infinity? (I assume not... but, since I thought the smallest meaningful depth was 1, I don't know what else it might mean.) Thank you! Jason -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git clone --depth: shallow clone problems
Duy Nguyen writes: > On Tue, May 26, 2015 at 9:54 PM, Junio C Hamano wrote: >> Duy Nguyen writes: >> >>> --unshallow:: >>> - If the source repository is complete, convert a shallow >>> - repository to a complete one, removing all the limitations >>> + If the source repository is complete, convert all shallow >>> + refs to complete ones, removing all the limitations >> >> Define "shallow ref", or better yet explain without introducing such >> a new term (I do not think shallow-ness is a property of a ref, by >> the way---I think you meant all refs that can reach the points the >> history is cauterised by .git/shallow, but we shouldn't assume that >> the target audience of this paragraph to know Git as well as I do). > > OK maybe > > Make sure all existing refs have the same history as the ones from the > source repository. If the source repository is complete, this removes > all limitations imposed by shallow repositories. OK, I can understand that description a lot easier than the original one. >>> ++ >>> +Note that if the repository is created with --single-branch, which is >>> +default for a shallow clone, only one ref is completed. `--unshallow` >>> +does not fetch all remaining refs from source repository. >> >> I do not think this "Note" is being as helpful as it could be. >> >> - If the repository was created with --single-branch but if the >>user later fetched and started tracking other branches, the >>statement "only one ref is completed" is untrue; the true version >>is "only the existing refs are completed", which leads to a more >>important point. It says the same thing as "all existing refs" >>above and does not add any useful information. >> >> - The last sentence is less than useful but merely frustrating---it >>says what you cannot do with this option, strongly hints that the >>writer of the sentence knows what the reader wants to achieve, >>but without saying what other way the reader go to achieve it. >> >> Perhaps replace that Note paragraph with something along this line? >> >> + >> By fetching and tracking refs that you haven't been tracking, >> you can add these new refs to "all refs" to be unshallowed. > > It was meant to emphasize --unshallow would not create a true clone, > if people miss the keywords "all existing" refs.. Maybe.. I know; it is not helpful to describe what it does *NOT* do, without saying what the reader is likely trying to find out. > You may need to fetch and track other refs from the source repository > if you want to make full clone. That is much better. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git clone --depth: shallow clone problems
On Tue, May 26, 2015 at 9:54 PM, Junio C Hamano wrote:
> Duy Nguyen writes:
>
>> --unshallow::
>> - If the source repository is complete, convert a shallow
>> - repository to a complete one, removing all the limitations
>> + If the source repository is complete, convert all shallow
>> + refs to complete ones, removing all the limitations
>
> Define "shallow ref", or better yet explain without introducing such
> a new term (I do not think shallow-ness is a property of a ref, by
> the way---I think you meant all refs that can reach the points the
> history is cauterised by .git/shallow, but we shouldn't assume that
> the target audience of this paragraph to know Git as well as I do).
OK maybe
Make sure all existing refs have the same history as the ones from the
source repository. If the source repository is complete, this removes
all limitations imposed by shallow repositories.
?
>> imposed by shallow repositories.
>> +
>> If the source repository is shallow, fetch as much as possible so that
>> -the current repository has the same history as the source repository.
>> +the all existing refs of current repository have the same history as in
>> +the source repository.
>
> Makes sense, modulo "so that the all existing refs" sounds strange
> to my ears ("all the existing refs" perhaps).
Will fix. Although I think the new paragraph above already covers this case too.
>> ++
>> +Note that if the repository is created with --single-branch, which is
>> +default for a shallow clone, only one ref is completed. `--unshallow`
>> +does not fetch all remaining refs from source repository.
>
> I do not think this "Note" is being as helpful as it could be.
>
> - If the repository was created with --single-branch but if the
>user later fetched and started tracking other branches, the
>statement "only one ref is completed" is untrue; the true version
>is "only the existing refs are completed", which leads to a more
>important point. It says the same thing as "all existing refs"
>above and does not add any useful information.
>
> - The last sentence is less than useful but merely frustrating---it
>says what you cannot do with this option, strongly hints that the
>writer of the sentence knows what the reader wants to achieve,
>but without saying what other way the reader go to achieve it.
>
> Perhaps replace that Note paragraph with something along this line?
>
> +
> By fetching and tracking refs that you haven't been tracking,
> you can add these new refs to "all refs" to be unshallowed.
It was meant to emphasize --unshallow would not create a true clone,
if people miss the keywords "all existing" refs.. Maybe..
You may need to fetch and track other refs from the source repository
if you want to make full clone.
?
>
>>> 2) git fetch --unshallow should convert the clone into an actual
>>> equivalent of a normal, not shallow clone.
>>
>> I was thinking of some way to undo --single-branch too. I don't think
>> it should be the job of --unshallow, maybe a new option, but I can't
>> think of a name better than --really-unshallow :P
>
> Isn't that just the matter of updating remote.origin.fetch? I do
> not think it belongs to "clone" (or "fetch"). Perhaps it belongs to
> "remote", where it already shows with "git remote -v" what is
> fetched and pushed.
Yeah, perhaps showing an advice about git-remote is enough if the
current ref spec does not fetch everything.
>> --depth ::
>> Create a 'shallow' clone with a history truncated to the
>> - specified number of revisions.
>> + specified number of revisions. --single-branch is
>> + automatically specified unless the user overrides it with
>> + --no-single-branch
>
> Yeah, something like that would be a definite improvement.
>
> By the way, while composing this message, I scratched my head after
> typing
>
> $ git clone --shallow=4 --no-local ./git.git ./playpen
>
> Is it just me or do we want to add such a synonym?
I have a series to define shallow boundaries by tags or dates, in
addition to history depth. It may take a few more months until I'm
finished with my ongoing topics. Then you could think about what
--shallow means exactly ;-)
--
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git clone --depth: shallow clone problems
Duy Nguyen writes:
> --unshallow::
> - If the source repository is complete, convert a shallow
> - repository to a complete one, removing all the limitations
> + If the source repository is complete, convert all shallow
> + refs to complete ones, removing all the limitations
Define "shallow ref", or better yet explain without introducing such
a new term (I do not think shallow-ness is a property of a ref, by
the way---I think you meant all refs that can reach the points the
history is cauterised by .git/shallow, but we shouldn't assume that
the target audience of this paragraph to know Git as well as I do).
> imposed by shallow repositories.
> +
> If the source repository is shallow, fetch as much as possible so that
> -the current repository has the same history as the source repository.
> +the all existing refs of current repository have the same history as in
> +the source repository.
Makes sense, modulo "so that the all existing refs" sounds strange
to my ears ("all the existing refs" perhaps).
> ++
> +Note that if the repository is created with --single-branch, which is
> +default for a shallow clone, only one ref is completed. `--unshallow`
> +does not fetch all remaining refs from source repository.
I do not think this "Note" is being as helpful as it could be.
- If the repository was created with --single-branch but if the
user later fetched and started tracking other branches, the
statement "only one ref is completed" is untrue; the true version
is "only the existing refs are completed", which leads to a more
important point. It says the same thing as "all existing refs"
above and does not add any useful information.
- The last sentence is less than useful but merely frustrating---it
says what you cannot do with this option, strongly hints that the
writer of the sentence knows what the reader wants to achieve,
but without saying what other way the reader go to achieve it.
Perhaps replace that Note paragraph with something along this line?
+
By fetching and tracking refs that you haven't been tracking,
you can add these new refs to "all refs" to be unshallowed.
>> 2) git fetch --unshallow should convert the clone into an actual
>> equivalent of a normal, not shallow clone.
>
> I was thinking of some way to undo --single-branch too. I don't think
> it should be the job of --unshallow, maybe a new option, but I can't
> think of a name better than --really-unshallow :P
Isn't that just the matter of updating remote.origin.fetch? I do
not think it belongs to "clone" (or "fetch"). Perhaps it belongs to
"remote", where it already shows with "git remote -v" what is
fetched and pushed.
> --depth ::
> Create a 'shallow' clone with a history truncated to the
> - specified number of revisions.
> + specified number of revisions. --single-branch is
> + automatically specified unless the user overrides it with
> + --no-single-branch
Yeah, something like that would be a definite improvement.
By the way, while composing this message, I scratched my head after
typing
$ git clone --shallow=4 --no-local ./git.git ./playpen
Is it just me or do we want to add such a synonym?
Thanks.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git clone --depth: shallow clone problems
On Thu, Apr 23, 2015 at 04:53:04PM +0200, Michal Pomorski wrote: > tl: skip to the second paragraph > > So here is what I just experienced: > We had an emergency error in an application at work and as the > responsible developer was unavailable, I was asked to check it out and > look into it. > We are a small branch of a bigger company and our connection to the > company's source servers is really slow, so just to quickly start it > up, I decided to take a shallow clone (that's what it is for, right?). > After a while we realized, the clone I have made was not sufficient > and was missing some newest work done on the project. > I did "git fetch --unshallow" which finished surprisingly quickly, and > it did not bring any newer commits. > Unaware of the fine print hiding in the documentation of git clone, I > blamed the repo (and in extension the person who provided me the > address to it). After coming to a realization, a process which cost me > time and embarrassment, I understood what is supposed to be the > "correct" behaviour: > "--single-branch > Clone only the history leading to the tip of a single branch, either > specified by the --branch option or the primary branch remote's HEAD > points at. When creating a shallow clone with the --depth option, this > is the default, unless --no-single-branch is given to fetch the > histories near the tips of all branches." > Of course, the new commits were put on a custom branch, and I knew > that all the time, but I expected the branch to show up eventually, at > least after git fetch --unshallow. > > > I hope you can see the faults in the usability of the commands I was > exposed to: > 1) git clone --depth should: >-warn about only fetching the current HEAD (name it by a real name > if applicable) > and/or >-require the --branch option so that it is not left to chance > (current HEAD could be anything; is it really meaningful to talk about > the current HEAD on a server?) > and/or >-make the --no-single-branch the default... > and maybe >-have the option to clone the most recent commits. --single-branch being the default was because we (at least I, stilll) believed it was the common case, so I don't think we should revert it. But I can see --unshallow documentation is misleading. How about this? I think it's better than what we have. -- 8< -- Subject: [PATCH] fetch-options.txt: clarify what --unshallow does Signed-off-by: Nguyễn Thái Ngọc Duy --- Documentation/fetch-options.txt | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt index 45583d8..63d3452 100644 --- a/Documentation/fetch-options.txt +++ b/Documentation/fetch-options.txt @@ -14,12 +14,17 @@ branch history. Tags for the deepened commits are not fetched. --unshallow:: - If the source repository is complete, convert a shallow - repository to a complete one, removing all the limitations + If the source repository is complete, convert all shallow + refs to complete ones, removing all the limitations imposed by shallow repositories. + If the source repository is shallow, fetch as much as possible so that -the current repository has the same history as the source repository. +the all existing refs of current repository have the same history as in +the source repository. ++ +Note that if the repository is created with --single-branch, which is +default for a shallow clone, only one ref is completed. `--unshallow` +does not fetch all remaining refs from source repository. --update-shallow:: By default when fetching from a shallow repository, -- 2.3.0.rc1.137.g477eb31 -- 8< -- > 2) git fetch --unshallow should convert the clone into an actual > equivalent of a normal, not shallow clone. I was thinking of some way to undo --single-branch too. I don't think it should be the job of --unshallow, maybe a new option, but I can't think of a name better than --really-unshallow :P It's been a while and no one responds to this, perhaps people are not interested in such an option? > 3) The documentation should be improved. The behaviour of --shallow is > described partly in the description of --no-single-branch. This is the > documentation equivalent of the infamous "come from" control flow > structure. Yes. Like this? -- 8< -- Subject: [PATCH] clone.txt: mention --single-branch in --depth Signed-off-by: Nguyễn Thái Ngọc Duy --- Documentation/git-clone.txt | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt index f1f2a3f..9c320da 100644 --- a/Documentation/git-clone.txt +++ b/Documentation/git-clone.txt @@ -185,7 +185,9 @@ object
git clone --depth: shallow clone problems
tl: skip to the second paragraph So here is what I just experienced: We had an emergency error in an application at work and as the responsible developer was unavailable, I was asked to check it out and look into it. We are a small branch of a bigger company and our connection to the company's source servers is really slow, so just to quickly start it up, I decided to take a shallow clone (that's what it is for, right?). After a while we realized, the clone I have made was not sufficient and was missing some newest work done on the project. I did "git fetch --unshallow" which finished surprisingly quickly, and it did not bring any newer commits. Unaware of the fine print hiding in the documentation of git clone, I blamed the repo (and in extension the person who provided me the address to it). After coming to a realization, a process which cost me time and embarrassment, I understood what is supposed to be the "correct" behaviour: "--single-branch Clone only the history leading to the tip of a single branch, either specified by the --branch option or the primary branch remote's HEAD points at. When creating a shallow clone with the --depth option, this is the default, unless --no-single-branch is given to fetch the histories near the tips of all branches." Of course, the new commits were put on a custom branch, and I knew that all the time, but I expected the branch to show up eventually, at least after git fetch --unshallow. I hope you can see the faults in the usability of the commands I was exposed to: 1) git clone --depth should: -warn about only fetching the current HEAD (name it by a real name if applicable) and/or -require the --branch option so that it is not left to chance (current HEAD could be anything; is it really meaningful to talk about the current HEAD on a server?) and/or -make the --no-single-branch the default... and maybe -have the option to clone the most recent commits. 2) git fetch --unshallow should convert the clone into an actual equivalent of a normal, not shallow clone. 3) The documentation should be improved. The behaviour of --shallow is described partly in the description of --no-single-branch. This is the documentation equivalent of the infamous "come from" control flow structure. Regards, Michal Pomorski -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to repair a shallow clone (?)
On 2014-12-07 12.44, Duy Nguyen wrote: >> Is this a known issue/problem ? > > No. > Thanks everybody for the support. The machine was equipped with git version 1.7.10.4 in /usr/bin. I installed 2.1 or so under /usr/local/bin, (and even /root/bin) thinking that this would help, but it didn't. Because the login shell for the user "storage" which manages the push/pull on the server side was /usr/bin/git-shell, not /usr/local/bin/git-shell. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to repair a shallow clone (?)
On Sat, Dec 6, 2014 at 7:23 PM, Torsten Bögershausen wrote: > I share a bare repo with Sparkleshare which does an auto-synch. > Now the synch had stopped, and trying to push to the central repo > by hand gives this: > > > > git push origin master > fatal: protocol error: expected old/new/ref, got 'shallow > 72fb4080921221293e28a97a0e8c78d6100c5186' > fatal: The remote end hung up unexpectedly > Counting objects: 4, done. > Delta compression using up to 2 threads. > Compressing objects: 100% (4/4), done. > error: pack-objects died of signal 13 > error: failed to push some refs to x > > Both machines have Git >2.0.0 Please try again with $GIT_TRACE_PACKET=/some-log-file. receive-pack >2.0 should recognize this shallow line. > Is this a known issue/problem ? No. -- Duy -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to repair a shallow clone (?)
From: "Trần Ngọc Quân" On 06/12/2014 19:23, Torsten Bögershausen wrote: I think I started to clone the repo in a shallow way (SparkleShare asked if I want to clone the complete history, and I probably answered "no" ) Is there a way to repair this situation ? (Except doing a complete re-clone ?) I think git don't accept push from shallow repo. I've ever encounter this problem. I UNshallow it, then every thing will work: $ git fetch --unshallow origin This command will convert a shallow repository to a complete one. See git-fetch(1) and git-clone(1). Since v1.9.0 (14 Feb '14.) you can do various push/pull from a shallow clone (I'd asked this way back http://stackoverflow.com/questions/6900103/why-cant-i-push-from-a-shallow-clone and noted when it was corrected/improved) That's not to say that you don't have to take care about your local depth being sufficiently inclusive. I'm sure that sometime a --timedepth= will eventually be coded by someone sufficiently in need. ;-) -- Philip -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to repair a shallow clone (?)
On 06/12/2014 19:23, Torsten Bögershausen wrote: > I think I started to clone the repo in a shallow way > (SparkleShare asked if I want to clone the complete history, > and I probably answered "no" ) > > Is there a way to repair this situation ? > (Except doing a complete re-clone ?) > I think git don't accept push from shallow repo. I've ever encounter this problem. I UNshallow it, then every thing will work: $ git fetch --unshallow origin This command will convert a shallow repository to a complete one. See git-fetch(1) and git-clone(1). I hope it helpful! Thanks, -- Trần Ngọc Quân. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
How to repair a shallow clone (?)
I share a bare repo with Sparkleshare which does an auto-synch. Now the synch had stopped, and trying to push to the central repo by hand gives this: git push origin master fatal: protocol error: expected old/new/ref, got 'shallow 72fb4080921221293e28a97a0e8c78d6100c5186' fatal: The remote end hung up unexpectedly Counting objects: 4, done. Delta compression using up to 2 threads. Compressing objects: 100% (4/4), done. error: pack-objects died of signal 13 error: failed to push some refs to x Both machines have Git >2.0.0 Is this a known issue/problem ? I think I started to clone the repo in a shallow way (SparkleShare asked if I want to clone the complete history, and I probably answered "no" ) Is there a way to repair this situation ? (Except doing a complete re-clone ?) Thanks for help -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH/RFC v2 2/2] submodule: modify clone command to recursively shallow clone submodules
When cloning a repository that contains submodules and specifying the
`--depth` option to the 'git clone' command, the top level repository will be
cloned with the specified depth, but all submodules within the
repository will be cloned in their entirety.
Modified 'git clone' to pass the `--depth` option, if specified, to any
submodule clone commands.
Modified 'git clone' to pass the `--no-single-branch`, if specified, to any
submodule clone commands.
Signed-off-by: Cole Minnaar
---
Documentation/git-clone.txt | 5 -
builtin/clone.c | 15 +--
2 files changed, 13 insertions(+), 7 deletions(-)
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 0363d00..7621251 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -178,7 +178,8 @@ objects from the source repository into a pack in the
cloned repository.
--depth ::
Create a 'shallow' clone with a history truncated to the
- specified number of revisions.
+ specified number of revisions. If `--recursive` was also specified
+ the depth value will be passed to all submodules within when cloning.
--[no-]single-branch::
Clone only the history leading to the tip of a single branch,
@@ -192,6 +193,8 @@ objects from the source repository into a pack in the
cloned repository.
initial cloning. If the HEAD at the remote did not point at any
branch when `--single-branch` clone was made, no remote-tracking
branch is created.
+ If `--recursive` was also specified, this option will be passed to
+ all submodules when cloning.
--recursive::
--recurse-submodules::
diff --git a/builtin/clone.c b/builtin/clone.c
index dd4092b..b27917c 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -48,6 +48,7 @@ static int option_verbosity;
static int option_progress = -1;
static struct string_list option_config;
static struct string_list option_reference;
+static struct argv_array argv_submodule_cmd = ARGV_ARRAY_INIT;
static int opt_parse_reference(const struct option *opt, const char *arg, int
unset)
{
@@ -100,10 +101,6 @@ static struct option builtin_clone_options[] = {
OPT_END()
};
-static const char *argv_submodule[] = {
- "submodule", "update", "--init", "--recursive", NULL
-};
-
static char *get_repo_path(const char *repo, int *is_bundle)
{
static char *suffix[] = { "/.git", "", ".git/.git", ".git" };
@@ -663,8 +660,14 @@ static int checkout(void)
err |= run_hook_le(NULL, "post-checkout", sha1_to_hex(null_sha1),
sha1_to_hex(sha1), "1", NULL);
- if (!err && option_recursive)
- err = run_command_v_opt(argv_submodule, RUN_GIT_CMD);
+ if (!err && option_recursive) {
+ argv_array_pushl(&argv_submodule_cmd, "submodule", "update",
"--init", "--recursive", NULL);
+ if (option_depth)
+ argv_array_pushf(&argv_submodule_cmd, "--depth=%d",
atoi(option_depth));
+ if (!option_single_branch)
+ argv_array_pushl(&argv_submodule_cmd,
"--no-single-branch", NULL);
+ err = run_command_v_opt(argv_submodule_cmd.argv, RUN_GIT_CMD);
+ }
return err;
}
--
2.1.0.240.g8a0e823
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH/RFC 2/2] submodule: modify clone command to recursively shallow clone submodules
When cloning a repository that contains submodules and specifying the
`--depth` option to the 'git clone' command, the top level repository will be
cloned with the specified depth, but all submodules within the
repository will be cloned in their entirety.
Modified 'git clone' to pass the `--depth` option, if specified, to any
submodule clone commands.
Modified 'git clone' to pass the `--no-single-branch`, if specified, to any
submodule clone commands.
Signed-off-by: Cole Minnaar
---
Documentation/git-clone.txt | 5 -
builtin/clone.c | 15 +--
2 files changed, 13 insertions(+), 7 deletions(-)
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 0363d00..f2fd8c8 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -178,7 +178,8 @@ objects from the source repository into a pack in the
cloned repository.
--depth ::
Create a 'shallow' clone with a history truncated to the
- specified number of revisions.
+ specified number of revisions. If `--recursive` was also specified,
+ the depth value will be passed to all submodules within when cloning.
--[no-]single-branch::
Clone only the history leading to the tip of a single branch,
@@ -192,6 +193,8 @@ objects from the source repository into a pack in the
cloned repository.
initial cloning. If the HEAD at the remote did not point at any
branch when `--single-branch` clone was made, no remote-tracking
branch is created.
+ If `--recursive` was also specified, this option will be passed
+ to all submodules when cloning.
--recursive::
--recurse-submodules::
diff --git a/builtin/clone.c b/builtin/clone.c
index dd4092b..c906d8e 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -48,6 +48,7 @@ static int option_verbosity;
static int option_progress = -1;
static struct string_list option_config;
static struct string_list option_reference;
+statis struct argv_array argv_submodule_cmd = ARGV_ARRAY_INIT;
static int opt_parse_reference(const struct option *opt, const char *arg, int
unset)
{
@@ -100,10 +101,6 @@ static struct option builtin_clone_options[] = {
OPT_END()
};
-static const char *argv_submodule[] = {
- "submodule", "update", "--init", "--recursive", NULL
-};
-
static char *get_repo_path(const char *repo, int *is_bundle)
{
static char *suffix[] = { "/.git", "", ".git/.git", ".git" };
@@ -663,8 +660,14 @@ static int checkout(void)
err |= run_hook_le(NULL, "post-checkout", sha1_to_hex(null_sha1),
sha1_to_hex(sha1), "1", NULL);
- if (!err && option_recursive)
- err = run_command_v_opt(argv_submodule, RUN_GIT_CMD);
+ if (!err && option_recursive) {
+ argv_array_pushl(&argv_submodule_cmd, "submodule", "update",
"--init", "--recursive", NULL);
+ if (option_depth)
+ argv_array_pushf(&argv_submodule_cmd, "--depth=%d",
atoi(option_depth));
+ if (!option_single_branch)
+ argv_array_pushl(&argv_submodule_cmd,
"--no-single-branch", NULL);
+ err = run_command_v_opt(argv_submodule_cmd.argv, RUN_GIT_CMD);
+ }
return err;
}
--
2.1.0.238.gce1d3a9.dirty
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Shallow clone
On Tue, Aug 19, 2014 at 6:11 PM, Steven Evergreen wrote: > Hi, everyone. I'm trying to perform a shallow clone with visibility of > all remote branches. > > git clone REPO --depth 1 --no-single-branch > > is consistently giving me > > Cloning into 'REPONAME'... > fatal: (null) is unknown object > remote: Total 0 (delta 0), reused 0 (delta 0) > fatal: recursion detected in die handler > remote: aborting due to possible repository corruption on the remote side. > fatal: error in sideband demultiplexer > > with 2.0.3 . > > Is this command not supported? Have I hit a bug? Yes it's supported and yes I think you hit a bug. Any chance you can share this repository? If not, perhaps just SHA-1 so we know what the commit DAG looks like? -- Duy -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Fwd: Shallow clone
Hi, everyone. I'm trying to perform a shallow clone with visibility of all remote branches. git clone REPO --depth 1 --no-single-branch is consistently giving me Cloning into 'REPONAME'... fatal: (null) is unknown object remote: Total 0 (delta 0), reused 0 (delta 0) fatal: recursion detected in die handler remote: aborting due to possible repository corruption on the remote side. fatal: error in sideband demultiplexer with 2.0.3 . Is this command not supported? Have I hit a bug? Thanks! -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Bug in shallow clone?
On wo, 2014-05-28 at 21:16 +0700, Duy Nguyen wrote: > On Wed, May 28, 2014 at 9:02 PM, Thomas Kieffer wrote: > > I then clone the bare repository with --depth 1. > > > > git clone file:///path/to/bare.git ./clone --depth 1 > > > > It always returns the last two commits. If I specify --depth 2 it returns > > the last 3 commits. > > > > If I use --depth 1 on a Github repository it works as expected. > > > > Am I doing something wrong or is it really a bug? > > Depth calculation has been corrected lately. It depends on your > version, maybe it's older than 1.8.2? If it's the latest, we screwed > something up again.. 2.0.0-rc4 does this correctly. -- Dennis Kaarsemaker http://www.kaarsemaker.net -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Bug in shallow clone?
On Wed, May 28, 2014 at 9:02 PM, Thomas Kieffer wrote: > I then clone the bare repository with --depth 1. > > git clone file:///path/to/bare.git ./clone --depth 1 > > It always returns the last two commits. If I specify --depth 2 it returns > the last 3 commits. > > If I use --depth 1 on a Github repository it works as expected. > > Am I doing something wrong or is it really a bug? Depth calculation has been corrected lately. It depends on your version, maybe it's older than 1.8.2? If it's the latest, we screwed something up again.. -- Duy -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Bug in shallow clone?
Hi there Git developers, I'm not sure if I found a bug in the command git clone repo.git cloned_repo --depth 1 I follow these steps: git init echo "First commit" >> test.txt git add -A git commit -am "First commit" echo "Second commit" >> test.txt git commit -am "Second commit" echo "Third commit" >> test.txt git commit -am "Third commit" git clone --bare . ./bare.git I then clone the bare repository with --depth 1. git clone file:///path/to/bare.git ./clone --depth 1 It always returns the last two commits. If I specify --depth 2 it returns the last 3 commits. If I use --depth 1 on a Github repository it works as expected. Am I doing something wrong or is it really a bug? Kind Regards, Thomas Kieffer BTW.: Git is amazing and I love it :) -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v4 28/28] git-clone.txt: remove shallow clone limitations
Now that git supports data transfer from or to a shallow clone, these limitations are not true anymore. Signed-off-by: Nguyễn Thái Ngọc Duy --- Documentation/git-clone.txt | 7 +-- 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt index 450f158..4987857 100644 --- a/Documentation/git-clone.txt +++ b/Documentation/git-clone.txt @@ -181,12 +181,7 @@ objects from the source repository into a pack in the cloned repository. --depth :: Create a 'shallow' clone with a history truncated to the - specified number of revisions. A shallow repository has a - number of limitations (you cannot clone or fetch from - it, nor push from nor into it), but is adequate if you - are only interested in the recent history of a large project - with a long history, and would want to send in fixes - as patches. + specified number of revisions. --[no-]single-branch:: Clone only the history leading to the tip of a single branch, -- 1.8.5.1.25.g8667982 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v4 24/28] receive-pack: support pushing to a shallow clone via http
Signed-off-by: Nguyễn Thái Ngọc Duy
---
builtin/receive-pack.c | 3 ---
t/t5537-push-shallow.sh | 35 +++
2 files changed, 35 insertions(+), 3 deletions(-)
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 5c85bb4..78fe8ee 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1179,9 +1179,6 @@ int cmd_receive_pack(int argc, const char **argv, const
char *prefix)
if (!enter_repo(dir, 0))
die("'%s' does not appear to be a git repository", dir);
- if (is_repository_shallow() && stateless_rpc)
- die("attempt to push into a shallow repository");
-
git_config(receive_pack_config, NULL);
if (0 <= transfer_unpack_limit)
diff --git a/t/t5537-push-shallow.sh b/t/t5537-push-shallow.sh
index f5c74e6..866621a 100755
--- a/t/t5537-push-shallow.sh
+++ b/t/t5537-push-shallow.sh
@@ -16,6 +16,7 @@ test_expect_success 'setup' '
commit 2 &&
commit 3 &&
commit 4 &&
+ git clone . full &&
(
git init full-abc &&
cd full-abc &&
@@ -120,4 +121,38 @@ EOF
)
'
+if test -n "$NO_CURL" -o -z "$GIT_TEST_HTTPD"; then
+ say 'skipping remaining tests, git built without http support'
+ test_done
+fi
+
+LIB_HTTPD_PORT=${LIB_HTTPD_PORT-'5537'}
+. "$TEST_DIRECTORY"/lib-httpd.sh
+start_httpd
+
+test_expect_success 'push to shallow repo via http' '
+ git clone --bare --no-local shallow
"$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+ (
+ cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+ git config http.receivepack true
+ ) &&
+ (
+ cd full &&
+ commit 9 &&
+ git push $HTTPD_URL/smart/repo.git +master:refs/remotes/top/master
+ ) &&
+ (
+ cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+ git fsck &&
+ git log --format=%s top/master >actual &&
+ cat
[PATCH v4 25/28] send-pack: support pushing from a shallow clone via http
Signed-off-by: Nguyễn Thái Ngọc Duy
---
builtin/send-pack.c | 3 ---
send-pack.c | 19 +--
t/t5537-push-shallow.sh | 25 +
3 files changed, 42 insertions(+), 5 deletions(-)
diff --git a/builtin/send-pack.c b/builtin/send-pack.c
index 664dd20..cc25744 100644
--- a/builtin/send-pack.c
+++ b/builtin/send-pack.c
@@ -209,9 +209,6 @@ int cmd_send_pack(int argc, const char **argv, const char
*prefix)
(send_all && args.send_mirror))
usage(send_pack_usage);
- if (is_repository_shallow() && args.stateless_rpc)
- die("attempt to push from a shallow repository");
-
if (remote_name) {
remote = remote_get(remote_name);
if (!remote_has_url(remote, dest)) {
diff --git a/send-pack.c b/send-pack.c
index cd536b4..848d15e 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -175,6 +175,21 @@ static int sideband_demux(int in, int out, void *data)
return ret;
}
+static int advertise_shallow_grafts_cb(const struct commit_graft *graft, void
*cb)
+{
+ struct strbuf *sb = cb;
+ if (graft->nr_parent == -1)
+ packet_buf_write(sb, "shallow %s\n", sha1_to_hex(graft->sha1));
+ return 0;
+}
+
+void advertise_shallow_grafts_buf(struct strbuf *sb)
+{
+ if (!is_repository_shallow())
+ return;
+ for_each_commit_graft(advertise_shallow_grafts_cb, sb);
+}
+
int send_pack(struct send_pack_args *args,
int fd[], struct child_process *conn,
struct ref *remote_refs,
@@ -215,7 +230,7 @@ int send_pack(struct send_pack_args *args,
}
if (!args->dry_run)
- advertise_shallow_grafts(out);
+ advertise_shallow_grafts_buf(&req_buf);
/*
* Finally, tell the other end!
@@ -276,7 +291,7 @@ int send_pack(struct send_pack_args *args,
}
if (args->stateless_rpc) {
- if (!args->dry_run && cmds_sent) {
+ if (!args->dry_run && (cmds_sent || is_repository_shallow())) {
packet_buf_flush(&req_buf);
send_sideband(out, -1, req_buf.buf, req_buf.len,
LARGE_PACKET_MAX);
}
diff --git a/t/t5537-push-shallow.sh b/t/t5537-push-shallow.sh
index 866621a..0a6e40f 100755
--- a/t/t5537-push-shallow.sh
+++ b/t/t5537-push-shallow.sh
@@ -154,5 +154,30 @@ EOF
)
'
+test_expect_success 'push from shallow repo via http' '
+ mv "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" shallow-upstream.git &&
+ git clone --bare --no-local full "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+ (
+ cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+ git config http.receivepack true
+ ) &&
+ commit 10 &&
+ git push $HTTPD_URL/smart/repo.git +master:refs/remotes/top/master &&
+ (
+ cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+ git fsck &&
+ git log --format=%s top/master >actual &&
+ cat
[PATCH v4 21/28] send-pack: support pushing to a shallow clone
Signed-off-by: Nguyễn Thái Ngọc Duy --- builtin/send-pack.c | 4 +++- t/t5537-push-shallow.sh | 38 ++ transport.c | 5 ++--- 3 files changed, 43 insertions(+), 4 deletions(-) diff --git a/builtin/send-pack.c b/builtin/send-pack.c index ea2ab28..664dd20 100644 --- a/builtin/send-pack.c +++ b/builtin/send-pack.c @@ -101,6 +101,7 @@ int cmd_send_pack(int argc, const char **argv, const char *prefix) int fd[2]; struct child_process *conn; struct sha1_array extra_have = SHA1_ARRAY_INIT; + struct sha1_array shallow = SHA1_ARRAY_INIT; struct ref *remote_refs, *local_refs; int ret; int helper_status = 0; @@ -232,7 +233,8 @@ int cmd_send_pack(int argc, const char **argv, const char *prefix) args.verbose ? CONNECT_VERBOSE : 0); } - get_remote_heads(fd[0], NULL, 0, &remote_refs, REF_NORMAL, &extra_have, NULL); + get_remote_heads(fd[0], NULL, 0, &remote_refs, REF_NORMAL, +&extra_have, &shallow); transport_verify_remote_names(nr_refspecs, refspecs); diff --git a/t/t5537-push-shallow.sh b/t/t5537-push-shallow.sh index ff5eb5b..f5c74e6 100755 --- a/t/t5537-push-shallow.sh +++ b/t/t5537-push-shallow.sh @@ -82,4 +82,42 @@ EOF test_cmp expect actual ' +test_expect_success 'push from shallow to shallow' ' + ( + cd shallow && + git --git-dir=../shallow2/.git config receive.shallowupdate true && + git push ../shallow2/.git +master:refs/remotes/shallow/master && + git --git-dir=../shallow2/.git config receive.shallowupdate false + ) && + ( + cd shallow2 && + git log --format=%s shallow/master >actual && + git fsck && + catactual && + git fsck && + cat /dev/null + ) +' + test_done diff --git a/transport.c b/transport.c index a09fdb6..d596abb 100644 --- a/transport.c +++ b/transport.c @@ -819,11 +819,10 @@ static int git_transport_push(struct transport *transport, struct ref *remote_re struct ref *tmp_refs; connect_setup(transport, 1, 0); - get_remote_heads(data->fd[0], NULL, 0, &tmp_refs, REF_NORMAL, NULL, NULL); + get_remote_heads(data->fd[0], NULL, 0, &tmp_refs, REF_NORMAL, +NULL, &data->shallow); data->got_remote_heads = 1; } - if (data->shallow.nr) - die("pushing to a shallow repository is not supported"); memset(&args, 0, sizeof(args)); args.send_mirror = !!(flags & TRANSPORT_PUSH_MIRROR); -- 1.8.5.1.25.g8667982 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v4 17/28] receive/send-pack: support pushing from a shallow clone
si,
+ struct sha1_array *ref)
+{
+ struct command *cmd;
+ int *ref_status;
+ remove_nonexistent_theirs_shallow(si);
+ /* XXX remove_nonexistent_ours_in_pack() */
+ if (!si->nr_ours && !si->nr_theirs)
+ return;
+
+ for (cmd = commands; cmd; cmd = cmd->next) {
+ if (is_null_sha1(cmd->new_sha1))
+ continue;
+ sha1_array_append(ref, cmd->new_sha1);
+ cmd->index = ref->nr - 1;
+ }
+ si->ref = ref;
+
+ ref_status = xmalloc(sizeof(*ref_status) * ref->nr);
+ assign_shallow_commits_to_refs(si, NULL, ref_status);
+ for (cmd = commands; cmd; cmd = cmd->next) {
+ if (is_null_sha1(cmd->new_sha1))
+ continue;
+ if (ref_status[cmd->index]) {
+ cmd->error_string = "shallow update not allowed";
+ cmd->skip_update = 1;
+ }
+ }
+ if (alt_shallow_file && *alt_shallow_file) {
+ unlink(alt_shallow_file);
+ alt_shallow_file = NULL;
+ }
+ free(ref_status);
+}
+
static void report(struct command *commands, const char *unpack_status)
{
struct command *cmd;
@@ -950,6 +1001,9 @@ int cmd_receive_pack(int argc, const char **argv, const
char *prefix)
int i;
char *dir = NULL;
struct command *commands;
+ struct sha1_array shallow = SHA1_ARRAY_INIT;
+ struct sha1_array ref = SHA1_ARRAY_INIT;
+ struct shallow_info si;
packet_trace_identity("receive-pack");
@@ -1006,11 +1060,14 @@ int cmd_receive_pack(int argc, const char **argv, const
char *prefix)
if (advertise_refs)
return 0;
- if ((commands = read_head_info()) != NULL) {
+ if ((commands = read_head_info(&shallow)) != NULL) {
const char *unpack_status = NULL;
- if (!delete_only(commands))
- unpack_status = unpack_with_sideband();
+ prepare_shallow_info(&si, &shallow);
+ if (!delete_only(commands)) {
+ unpack_status = unpack_with_sideband(&si);
+ update_shallow_info(commands, &si, &ref);
+ }
execute_commands(commands, unpack_status);
if (pack_lockfile)
unlink_or_warn(pack_lockfile);
@@ -1027,8 +1084,11 @@ int cmd_receive_pack(int argc, const char **argv, const
char *prefix)
}
if (auto_update_server_info)
update_server_info(0);
+ clear_shallow_info(&si);
}
if (use_sideband)
packet_flush(1);
+ sha1_array_clear(&shallow);
+ sha1_array_clear(&ref);
return 0;
}
diff --git a/builtin/send-pack.c b/builtin/send-pack.c
index 62cc4d3..ea2ab28 100644
--- a/builtin/send-pack.c
+++ b/builtin/send-pack.c
@@ -208,7 +208,7 @@ int cmd_send_pack(int argc, const char **argv, const char
*prefix)
(send_all && args.send_mirror))
usage(send_pack_usage);
- if (is_repository_shallow())
+ if (is_repository_shallow() && args.stateless_rpc)
die("attempt to push from a shallow repository");
if (remote_name) {
diff --git a/send-pack.c b/send-pack.c
index 14005fa..cd536b4 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -214,6 +214,9 @@ int send_pack(struct send_pack_args *args,
return 0;
}
+ if (!args->dry_run)
+ advertise_shallow_grafts(out);
+
/*
* Finally, tell the other end!
*/
diff --git a/t/t5537-push-shallow.sh b/t/t5537-push-shallow.sh
new file mode 100755
index 000..650c31a
--- /dev/null
+++ b/t/t5537-push-shallow.sh
@@ -0,0 +1,70 @@
+#!/bin/sh
+
+test_description='push from/to a shallow clone'
+
+. ./test-lib.sh
+
+commit() {
+ echo "$1" >tracked &&
+ git add tracked &&
+ git commit -m "$1"
+}
+
+test_expect_success 'setup' '
+ git config --global transfer.fsckObjects true &&
+ commit 1 &&
+ commit 2 &&
+ commit 3 &&
+ commit 4 &&
+ (
+ git init full-abc &&
+ cd full-abc &&
+ commit a &&
+ commit b &&
+ commit c
+ ) &&
+ git clone --no-local --depth=2 .git shallow &&
+ git --git-dir=shallow/.git log --format=%s >actual &&
+ cat <expect &&
+4
+3
+EOF
+ test_cmp expect actual &&
+ git clone --no-local --depth=2 full-abc/.git shallow2 &&
+ git --git-dir=shallow2/.git log --format
[PATCH v4 00/28] First class shallow clone
This reroll should fix all comments I have received in v3. I reordered the shallow checks a bit so in common case it should be as cheap as a normal fetch or push. See 08/28 and 20/28 for the big picture. I'm not entirely happy with the hook issue in 20/28, but it looks good enough for me. There are a few XXXes for further improvement, but I'll keep them until this lands. To recap, this series allows you to clone from a shallow repo, push to or fetch from any shallow repo. Normally it will reject new refs that introduce new shallow boundaries to your repository, so if you're in a full clone, it will always stay a full clone. Use "fetch --update-shallow" or set receive.shallowupdate to accept those refs. Nguyễn Thái Ngọc Duy (28): transport.h: remove send_pack prototype, already defined in send-pack.h Replace struct extra_have_objects with struct sha1_array send-pack: forbid pushing from a shallow repository clone: prevent --reference to a shallow repository Make the sender advertise shallow commits to the receiver connect.c: teach get_remote_heads to parse "shallow" lines shallow.c: extend setup_*_shallow() to accept extra shallow commits shallow.c: the 8 steps to select new commits for .git/shallow shallow.c: steps 6 and 7 to select new commits for .git/shallow fetch-pack.c: move shallow update code out of fetch_pack() fetch-pack.h: one statement per bitfield declaration clone: support remote shallow repository fetch: support fetching from a shallow repository upload-pack: make sure deepening preserves shallow roots fetch: add --update-shallow to accept refs that update .git/shallow receive-pack: reorder some code in unpack() receive/send-pack: support pushing from a shallow clone New var GIT_SHALLOW_FILE to propagate --shallow-file to subprocesses connected.c: add new variant that runs with --shallow-file receive-pack: allow pushes that update .git/shallow send-pack: support pushing to a shallow clone remote-curl: pass ref SHA-1 to fetch-pack as well Support shallow fetch/clone over smart-http receive-pack: support pushing to a shallow clone via http send-pack: support pushing from a shallow clone via http clone: use git protocol for cloning shallow repo locally prune: clean .git/shallow after pruning objects git-clone.txt: remove shallow clone limitations Documentation/config.txt | 4 + Documentation/fetch-options.txt | 14 +- Documentation/git-clone.txt | 7 +- Documentation/git-prune.txt | 2 + Documentation/gitremote-helpers.txt | 7 + Documentation/technical/pack-protocol.txt | 7 +- builtin/clone.c | 18 +- builtin/fetch-pack.c | 23 +- builtin/fetch.c | 15 +- builtin/gc.c | 1 + builtin/prune.c | 4 + builtin/receive-pack.c| 314 +++ builtin/send-pack.c | 9 +- cache.h | 3 + commit.h | 39 ++- connect.c | 22 +- connected.c | 42 ++- connected.h | 2 + environment.c | 6 + fetch-pack.c | 132 +++- fetch-pack.h | 29 +- git.c | 2 +- remote-curl.c | 34 ++- remote.h | 9 +- send-pack.c | 27 +- send-pack.h | 2 +- shallow.c | 486 +- t/t5304-prune.sh | 10 + t/t5536-fetch-shallow.sh (new +x) | 203 + t/t5537-push-shallow.sh (new +x) | 183 +++ t/t5601-clone.sh | 7 + trace.c | 2 +- transport-helper.c| 6 + transport.c | 25 +- transport.h | 16 +- upload-pack.c | 8 +- 36 files changed, 1555 insertions(+), 165 deletions(-) create mode 100755 t/t5536-fetch-shallow.sh create mode 100755 t/t5537-push-shallow.sh -- 1.8.5.1.25.g8667982 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 17/28] Support pushing from a shallow clone
On Sun, Nov 24, 2013 at 10:55 PM, Nguyễn Thái Ngọc Duy
wrote:
> Pushing from a shallow clone using today's send-pack and receive-pack
> may work, if the transferred pack does not end up at any graft
> points. If it does, recent receive-pack that does connectivity check
> will reject the push. If receive-pack is old and does not have the
> connectivity check, the upstream repo becomes corrupt.
>
> The pack protocol is updated and send-pack now sends all shallow
> grafts before it sends the commands, if the repo is shallow. This
> protocol extension will break current receive-pack, which is intended,
> mostly to stop corrupting the upstream repo.
>
> Changes on the receiver are similar to what has been done in
> fetch-pack, i.e. filter out refs that require new shallow roots then
> go along as usual.
>
> Signed-off-by: Nguyễn Thái Ngọc Duy
> ---
> diff --git a/Documentation/technical/pack-protocol.txt
> b/Documentation/technical/pack-protocol.txt
> index eb8edd1..c73b62f 100644
> --- a/Documentation/technical/pack-protocol.txt
> +++ b/Documentation/technical/pack-protocol.txt
> @@ -43,6 +43,9 @@ static int fix_thin = 1;
> static const char *head_name;
> static void *head_name_to_free;
> static int sent_capabilities;
> +static int shallow_push;
> +static const char* alternate_shallow_file;
s/char\* /char */
> +static struct extra_have_objects shallow;
>
> static enum deny_action parse_deny_action(const char *var, const char *value)
> {
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 26/28] git-clone.txt: remove shallow clone limitations
Now that git supports push/pull from/to a shallow clone, these limitations are not true anymore. Signed-off-by: Nguyễn Thái Ngọc Duy --- Documentation/git-clone.txt | 7 +-- 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt index 450f158..4987857 100644 --- a/Documentation/git-clone.txt +++ b/Documentation/git-clone.txt @@ -181,12 +181,7 @@ objects from the source repository into a pack in the cloned repository. --depth :: Create a 'shallow' clone with a history truncated to the - specified number of revisions. A shallow repository has a - number of limitations (you cannot clone or fetch from - it, nor push from nor into it), but is adequate if you - are only interested in the recent history of a large project - with a long history, and would want to send in fixes - as patches. + specified number of revisions. --[no-]single-branch:: Clone only the history leading to the tip of a single branch, -- 1.8.2.83.gc99314b -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 24/28] receive-pack: support pushing to a shallow clone via http
Signed-off-by: Nguyễn Thái Ngọc Duy
---
builtin/receive-pack.c | 3 ---
t/t5537-push-shallow.sh | 35 +++
2 files changed, 35 insertions(+), 3 deletions(-)
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 366ecde..ec681ba 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1157,9 +1157,6 @@ int cmd_receive_pack(int argc, const char **argv, const
char *prefix)
if (!enter_repo(dir, 0))
die("'%s' does not appear to be a git repository", dir);
- if (is_repository_shallow() && stateless_rpc)
- die("attempt to push into a shallow repository");
-
git_config(receive_pack_config, NULL);
if (0 <= transfer_unpack_limit)
diff --git a/t/t5537-push-shallow.sh b/t/t5537-push-shallow.sh
index ccb41b6..d252a78 100755
--- a/t/t5537-push-shallow.sh
+++ b/t/t5537-push-shallow.sh
@@ -16,6 +16,7 @@ test_expect_success 'setup' '
commit 2 &&
commit 3 &&
commit 4 &&
+ git clone . full &&
(
git init full-abc &&
cd full-abc &&
@@ -121,4 +122,38 @@ EOF
)
'
+if test -n "$NO_CURL" -o -z "$GIT_TEST_HTTPD"; then
+ say 'skipping remaining tests, git built without http support'
+ test_done
+fi
+
+LIB_HTTPD_PORT=${LIB_HTTPD_PORT-'5537'}
+. "$TEST_DIRECTORY"/lib-httpd.sh
+start_httpd
+
+test_expect_success 'push to shallow repo via http' '
+ git clone --bare --no-local shallow
"$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+ (
+ cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+ git config http.receivepack true
+ ) &&
+ (
+ cd full &&
+ commit 9 &&
+ git push $HTTPD_URL/smart/repo.git +master:refs/remotes/top/master
+ ) &&
+ (
+ cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+ git fsck &&
+ git log --format=%s top/master >actual &&
+ cat
[PATCH v3 21/28] send-pack: support pushing to a shallow clone
Signed-off-by: Nguyễn Thái Ngọc Duy --- builtin/send-pack.c | 5 - t/t5537-push-shallow.sh | 38 ++ transport.c | 5 ++--- 3 files changed, 44 insertions(+), 4 deletions(-) diff --git a/builtin/send-pack.c b/builtin/send-pack.c index 5db1311..0031165 100644 --- a/builtin/send-pack.c +++ b/builtin/send-pack.c @@ -100,6 +100,7 @@ int cmd_send_pack(int argc, const char **argv, const char *prefix) int fd[2]; struct child_process *conn; struct extra_have_objects extra_have; + struct extra_have_objects shallow; struct ref *remote_refs, *local_refs; int ret; int helper_status = 0; @@ -232,8 +233,10 @@ int cmd_send_pack(int argc, const char **argv, const char *prefix) } memset(&extra_have, 0, sizeof(extra_have)); + memset(&shallow, 0, sizeof(shallow)); - get_remote_heads(fd[0], NULL, 0, &remote_refs, REF_NORMAL, &extra_have, NULL); + get_remote_heads(fd[0], NULL, 0, &remote_refs, REF_NORMAL, +&extra_have, &shallow); transport_verify_remote_names(nr_refspecs, refspecs); diff --git a/t/t5537-push-shallow.sh b/t/t5537-push-shallow.sh index 0084a31..ccb41b6 100755 --- a/t/t5537-push-shallow.sh +++ b/t/t5537-push-shallow.sh @@ -83,4 +83,42 @@ EOF git config receive.shallowupdate false ' +test_expect_success 'push from shallow to shallow' ' + ( + cd shallow && + git --git-dir=../shallow2/.git config receive.shallowupdate true && + git push ../shallow2/.git +master:refs/remotes/shallow/master && + git --git-dir=../shallow2/.git config receive.shallowupdate false + ) && + ( + cd shallow2 && + git log --format=%s shallow/master >actual && + git fsck && + catactual && + git fsck && + cat /dev/null + ) +' + test_done diff --git a/transport.c b/transport.c index c0be6b1..6a3fe9b 100644 --- a/transport.c +++ b/transport.c @@ -818,11 +818,10 @@ static int git_transport_push(struct transport *transport, struct ref *remote_re struct ref *tmp_refs; connect_setup(transport, 1, 0); - get_remote_heads(data->fd[0], NULL, 0, &tmp_refs, REF_NORMAL, NULL, NULL); + get_remote_heads(data->fd[0], NULL, 0, &tmp_refs, REF_NORMAL, +NULL, &data->shallow); data->got_remote_heads = 1; } - if (data->shallow.nr) - die("pushing to a shallow repository is not supported"); memset(&args, 0, sizeof(args)); args.send_mirror = !!(flags & TRANSPORT_PUSH_MIRROR); -- 1.8.2.83.gc99314b -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 25/28] send-pack: support pushing from a shallow clone via http
Signed-off-by: Nguyễn Thái Ngọc Duy
---
builtin/send-pack.c | 3 ---
send-pack.c | 19 +--
t/t5537-push-shallow.sh | 25 +
3 files changed, 42 insertions(+), 5 deletions(-)
diff --git a/builtin/send-pack.c b/builtin/send-pack.c
index 0031165..966b45e 100644
--- a/builtin/send-pack.c
+++ b/builtin/send-pack.c
@@ -208,9 +208,6 @@ int cmd_send_pack(int argc, const char **argv, const char
*prefix)
(send_all && args.send_mirror))
usage(send_pack_usage);
- if (is_repository_shallow() && args.stateless_rpc)
- die("attempt to push from a shallow repository");
-
if (remote_name) {
remote = remote_get(remote_name);
if (!remote_has_url(remote, dest)) {
diff --git a/send-pack.c b/send-pack.c
index 8b5571c..13167ea 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -174,6 +174,21 @@ static int sideband_demux(int in, int out, void *data)
return ret;
}
+static int advertise_shallow_grafts_cb(const struct commit_graft *graft, void
*cb)
+{
+ struct strbuf *sb = cb;
+ if (graft->nr_parent == -1)
+ packet_buf_write(sb, "shallow %s\n", sha1_to_hex(graft->sha1));
+ return 0;
+}
+
+void advertise_shallow_grafts_buf(struct strbuf *sb)
+{
+ if (!is_repository_shallow())
+ return;
+ for_each_commit_graft(advertise_shallow_grafts_cb, sb);
+}
+
int send_pack(struct send_pack_args *args,
int fd[], struct child_process *conn,
struct ref *remote_refs,
@@ -214,7 +229,7 @@ int send_pack(struct send_pack_args *args,
}
if (!args->dry_run)
- advertise_shallow_grafts(out);
+ advertise_shallow_grafts_buf(&req_buf);
/*
* Finally, tell the other end!
@@ -275,7 +290,7 @@ int send_pack(struct send_pack_args *args,
}
if (args->stateless_rpc) {
- if (!args->dry_run && cmds_sent) {
+ if (!args->dry_run && (cmds_sent || is_repository_shallow())) {
packet_buf_flush(&req_buf);
send_sideband(out, -1, req_buf.buf, req_buf.len,
LARGE_PACKET_MAX);
}
diff --git a/t/t5537-push-shallow.sh b/t/t5537-push-shallow.sh
index d252a78..f8200f7 100755
--- a/t/t5537-push-shallow.sh
+++ b/t/t5537-push-shallow.sh
@@ -155,5 +155,30 @@ EOF
)
'
+test_expect_success 'push from shallow repo via http' '
+ mv "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" shallow-upstream.git &&
+ git clone --bare --no-local full "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+ (
+ cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+ git config http.receivepack true
+ ) &&
+ commit 10 &&
+ git push $HTTPD_URL/smart/repo.git +master:refs/remotes/top/master &&
+ (
+ cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+ git fsck &&
+ git log --format=%s top/master >actual &&
+ cat
[PATCH v3 17/28] Support pushing from a shallow clone
Pushing from a shallow clone using today's send-pack and receive-pack
may work, if the transferred pack does not end up at any graft
points. If it does, recent receive-pack that does connectivity check
will reject the push. If receive-pack is old and does not have the
connectivity check, the upstream repo becomes corrupt.
The pack protocol is updated and send-pack now sends all shallow
grafts before it sends the commands, if the repo is shallow. This
protocol extension will break current receive-pack, which is intended,
mostly to stop corrupting the upstream repo.
Changes on the receiver are similar to what has been done in
fetch-pack, i.e. filter out refs that require new shallow roots then
go along as usual.
Signed-off-by: Nguyễn Thái Ngọc Duy
---
Documentation/technical/pack-protocol.txt | 4 +-
builtin/receive-pack.c| 68 +-
builtin/send-pack.c | 2 +-
send-pack.c | 3 ++
t/t5537-push-shallow.sh (new +x) | 70 +++
5 files changed, 143 insertions(+), 4 deletions(-)
create mode 100755 t/t5537-push-shallow.sh
diff --git a/Documentation/technical/pack-protocol.txt
b/Documentation/technical/pack-protocol.txt
index eb8edd1..c73b62f 100644
--- a/Documentation/technical/pack-protocol.txt
+++ b/Documentation/technical/pack-protocol.txt
@@ -464,7 +464,9 @@ contain all the objects that the server will need to
complete the new
references.
- update-request= command-list [pack-file]
+ update-request= *shallow command-list [pack-file]
+
+ shallow = PKT-LINE("shallow" SP obj-id)
command-list = PKT-LINE(command NUL capability-list LF)
*PKT-LINE(command LF)
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 22e162d..254feff 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -43,6 +43,9 @@ static int fix_thin = 1;
static const char *head_name;
static void *head_name_to_free;
static int sent_capabilities;
+static int shallow_push;
+static const char* alternate_shallow_file;
+static struct extra_have_objects shallow;
static enum deny_action parse_deny_action(const char *var, const char *value)
{
@@ -189,6 +192,7 @@ struct command {
const char *error_string;
unsigned int skip_update:1,
did_not_exist:1;
+ int index;
unsigned char old_sha1[20];
unsigned char new_sha1[20];
char ref_name[FLEX_ARRAY]; /* more */
@@ -687,7 +691,7 @@ static int iterate_receive_command_list(void *cb_data,
unsigned char sha1[20])
struct command *cmd = *cmd_list;
while (cmd) {
- if (!is_null_sha1(cmd->new_sha1)) {
+ if (!is_null_sha1(cmd->new_sha1) && !cmd->skip_update) {
hashcpy(sha1, cmd->new_sha1);
*cmd_list = cmd->next;
return 0;
@@ -712,8 +716,25 @@ static void reject_updates_to_hidden(struct command
*commands)
}
}
+static void filter_shallow_refs(struct command *commands,
+ const int *ref_status, int nr_ref)
+{
+ struct command *cmd;
+ for (cmd = commands; cmd; cmd = cmd->next)
+ if (!is_null_sha1(cmd->new_sha1) && ref_status[cmd->index]) {
+ cmd->error_string = "shallow update not allowed";
+ cmd->skip_update = 1;
+ }
+ if (*alternate_shallow_file) {
+ unlink(alternate_shallow_file);
+ alternate_shallow_file = NULL;
+ }
+}
+
static void execute_commands(struct command *commands, const char
*unpacker_error)
{
+ int *ref_status = NULL;
+ struct extra_have_objects ref;
struct command *cmd;
unsigned char sha1[20];
@@ -723,6 +744,20 @@ static void execute_commands(struct command *commands,
const char *unpacker_erro
return;
}
+ memset(&ref, 0, sizeof(ref));
+ if (shallow_push) {
+ int i;
+ for (i = 0, cmd = commands; cmd; i++, cmd = cmd->next)
+ if (!is_null_sha1(cmd->new_sha1)) {
+ add_extra_have(&ref, cmd->new_sha1);
+ cmd->index = i;
+ }
+
+ ref_status = xmalloc(sizeof(*ref_status) * ref.nr);
+ if (mark_new_shallow_refs(&ref, ref_status, NULL, &shallow))
+ filter_shallow_refs(commands, ref_status, ref.nr);
+ }
+
cmd = commands;
if (check_everything_connected(iterate_receive_command_list,
0, &cmd))
@@ -752,6 +787,8 @@ static void execute_commands(struct command *commands,
const char *unpacker_erro
cmd->error_stri
[PATCH v3 00/28] First class shallow clone
Compared to v2 [1], v3 grows a bit. The biggest difference is .git/shallow is not updated by default (except when you clone from a shallow repository). When you send something, the "safe" refs that do not need new shallow roots are accepted at the receiver end, the others rejected. To accept those other refs, either use "fetch --update-shallow" or enable receive.shallowupdate on the receiver side of a push. Filtering "safe" refs requires walking through some commits, so it'll be more expensive than normal full clones. This is especially true for the receiver of a push (see 07/28 and 17/28). I envision shallow repos are used as upstream to archive old history, so this is not a good news. Commit cache (or pack v4) might help. We might even be able to move some work from receive-pack to send-pack to reduce server load.. [1] http://mid.gmane.org/1374314290-5976-1-git-send-email-pclo...@gmail.com Nguyễn Thái Ngọc Duy (28): transport.h: remove send_pack prototype, already defined in send-pack.h send-pack: forbid pushing from a shallow repository clone: prevent --reference to a shallow repository update-server-info: do not publish shallow clones This part is just cleanup. Advertise shallow graft information on the server end connect.c: teach get_remote_heads to parse "shallow" lines shallow.c: add remove_reachable_shallow_points() shallow.c: add mark_new_shallow_refs() shallow.c: extend setup_*_shallow() to accept extra shallow points fetch-pack.c: move shallow update code out of fetch_pack() fetch-pack.h: one statement per bitfield declaration clone: support remote shallow repository fetch: support fetching from a shallow repository upload-pack: make sure deepening preserves shallow roots fetch: add --update-shallow to get refs that require updating .git/shallow Basic shallow fetch/clone support on git protocol receive-pack: reorder some code in unpack() Support pushing from a shallow clone New var GIT_SHALLOW_FILE to propagate --shallow-file to subprocesses connected.c: add new variant that runs with --shallow-file receive-pack: allow pushing with new shallow roots send-pack: support pushing to a shallow clone remote-curl: pass ref SHA-1 to fetch-pack as well Push support Support fetch/clone over http receive-pack: support pushing to a shallow clone via http send-pack: support pushing from a shallow clone via http smart-http support git-clone.txt: remove shallow clone limitations clone: use git protocol for cloning shallow repo locally prune: clean .git/shallow after pruning objects miscellaneous Documentation/config.txt | 4 + Documentation/fetch-options.txt | 14 +- Documentation/git-clone.txt | 7 +- Documentation/gitremote-helpers.txt | 7 + Documentation/technical/pack-protocol.txt | 7 +- builtin/clone.c | 18 +- builtin/fetch-pack.c | 23 +- builtin/fetch.c | 15 +- builtin/gc.c | 1 + builtin/prune.c | 4 + builtin/receive-pack.c| 248 + builtin/send-pack.c | 5 +- cache.h | 1 + commit.h | 19 +- connect.c | 14 +- connected.c | 42 +++- connected.h | 2 + environment.c | 6 + fetch-pack.c | 132 ++-- fetch-pack.h | 29 +-- git.c | 2 +- remote-curl.c | 33 ++- remote.h | 5 +- send-pack.c | 20 +- server-info.c | 9 + shallow.c | 348 +- t/t5536-fetch-shallow.sh (new +x) | 193 + t/t5537-push-shallow.sh (new +x) | 184 t/t5601-clone.sh | 7 + transport-helper.c| 6 + transport.c | 22 +- transport.h | 16 +- upload-pack.c | 8 +- 33 files changed, 1323 insertions(+), 128 deletions(-) create mode 100755 t/t5536-fetch-shallow.sh create mode 100755 t/t5537-push-shallow.sh -- 1.8.2.83.gc99314b -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Bug] git show crashes with deepened shallow clone
Stefan Näwe wrote: > Stefan Näwe atlas-elektronik.com> writes: >> 6035d6aad8ca11954c0d7821f6f3e7c047039c8f is the first bad commit > > And to answer myself once more: > > That's fixed in > >6da8bdc fetch-pack: do not remove .git/shallow file when --depth is not > specified" > > Sorry for all the noise. No problem. It's probably worth releasing 1.8.4.1 soon. Thanks for a reminder. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Bug] git show crashes with deepened shallow clone
Stefan Näwe atlas-elektronik.com> writes: > Am 25.09.2013 16:47, schrieb Stefan Näwe: > This is what it gives me if I use my script (slightly modified > to also run make) with 'git bisect run': > > 6035d6aad8ca11954c0d7821f6f3e7c047039c8f is the first bad commit And to answer myself once more: That's fixed in 6da8bdc fetch-pack: do not remove .git/shallow file when --depth is not specified" Sorry for all the noise. Stefan -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Bug] git show crashes with deepened shallow clone
Am 25.09.2013 16:47, schrieb Stefan Näwe:
> Just a quick report since I don't have time to bisect now (will do later
> if no other gitster is faster...)
I'd marry 'git bisect' if I wasn't already... ;-)
This is what it gives me if I use my script (slightly modified to also run make)
with 'git bisect run':
6035d6aad8ca11954c0d7821f6f3e7c047039c8f is the first bad commit
commit 6035d6aad8ca11954c0d7821f6f3e7c047039c8f
Author: Nguyễn Thái Ngọc Duy
Date: Sun May 26 08:16:15 2013 +0700
fetch-pack: prepare updated shallow file before fetching the pack
index-pack --strict looks up and follows parent commits. If shallow
information is not ready by the time index-pack is run, index-pack may
be led to non-existent objects. Make fetch-pack save shallow file to
disk before invoking index-pack.
git learns new global option --shallow-file to pass on the alternate
shallow file path. Undocumented (and not even support --shallow-file=
syntax) because it's unlikely to be used again elsewhere.
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
:100644 100644 67bd5091be0b34bfea075cd60281d22cf5b34768
6e9c7cd9d5da7d24d4810b36039681f184325932 M commit.h
:100644 100644 f156dd4fac30cda4e09c508b7091cdb8d96917e2
6b5467c6dec9645f53d83cdad2467a158db622c0 M fetch-pack.c
:100644 100644 1ada169d5cff3051effee33c6f9ba5b9be15b2e6
88eef5a7cc6d36f6e17f4855945116dd6f1b0681 M git.c
:100644 100644 6be915f38f1fe8dbe0a22c4cd8ae2569331f483f
cbe2526d8c2b2643957eea2729a16269a7a74fab M shallow.c
:04 04 5333beeb4db3fc37c37e5a4d03816c4750ce6b28
3b0fb999b8655155cba24e2d09ebff29058d29d7 M t
bisect run success
Stefan
--
/dev/random says: Answers: $1 * Correct answers: $5 * Dumb looks: Free! *
python -c "print
'73746566616e2e6e616577654061746c61732d656c656b74726f6e696b2e636f6d'.decode('hex')"
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Bug] git show crashes with deepened shallow clone
Am 25.09.2013 16:47, schrieb Stefan Näwe:
> Just a quick report since I don't have time to bisect now (will do later
> if no other gitster is faster...)
Seems to be somewhere between v1.8.3.1 (OK) and v1.8.3.2 (not OK) !!
>
> When I execute the below script 'git show' crashes. 'git log --oneline -2'
> gives
> for example:
>
> 3808bade5b76c4663ac4a3f751dc9f1ed0b08f2e three
> error: Could not read 1e8777edeb2b7e757f74c789e679fc6c71073897
> fatal: Failed to traverse parents of commit
> 0aa4ef86004f5bb29f67e971d7963f5cf430c668
>
> gdb backtrace of one run is attached.
> It happens on 32-bit Debian (5.0.10), 64-bit openSUSE (12.2), and Windows XP
> with git 1.8.4
> on all systems.
>
> The help of 'git fetch' says:
>
> --depth=
>
> Deepen or shorten the history of a shallow repository created by git
> clone with
> --depth= option (see git-clone(1)) to the specified number of
> commits from
> the tip of each remote branch history. Tags for the deepened commits
> are not fetched.
> ---^
>
> But that's not true. The tag 'two' (from the script below) gets fetched when
> deepening the repository.
v1.8.3.1 fetches the tag also.
Stefan
--
/dev/random says: Pobody's Nerfect!
python -c "print
'73746566616e2e6e616577654061746c61732d656c656b74726f6e696b2e636f6d'.decode('hex')"
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
