Re: [PATCH 2/2] Be more careful when determining whether a remote was configured
Hi Peff, On Thu, 19 Jan 2017, Jeff King wrote: > diff --git a/remote.c b/remote.c > index 298f2f93f..720d616be 100644 > --- a/remote.c > +++ b/remote.c > @@ -373,6 +373,8 @@ static int handle_config(const char *key, const char > *value, void *cb) > } > remote = make_remote(name, namelen); > remote->origin = REMOTE_CONFIG; > + if (current_config_scope() == CONFIG_SCOPE_REPO) > + remote->configured = 1; > if (!strcmp(subkey, "mirror")) > remote->mirror = git_config_bool(key, value); > else if (!strcmp(subkey, "skipdefaultupdate")) > > That doesn't make your test pass, but I think that is only because your > test is not covering the interesting case (it puts the new config in the > repo config, not in ~/.gitconfig). > > What do you think? Heh. After skimming the first three paragraphs of your mail, I had a similar idea and implemented it. It works great! v2 coming right up, Johannes
Re: [PATCH 2/2] Be more careful when determining whether a remote was configured
Jeff King writes: > On Wed, Jan 18, 2017 at 05:22:40PM +0100, Johannes Schindelin wrote: > >> > > I want to err on the side of caution. That's why. >> > >> > I guess I just don't see why changing the behavior with respect to >> > "prune" or "proxy" is any less conservative than changing the one for >> > "refspec". >> > I think _this_ is a much better way of framing the problem. It is not > about which keys are set, but about _where_ they are set. IOW, a > reasonable rule would be: if there is any remote.*.X in the repo config, > then git-remote should consider it a configured repo. And otherwise, no > matter what is in ~/.gitconfig or elsewhere, git-remote should proceed > as if it doesn't exist (and repo-level config can take precedence over > config defined elsewhere). > > I.e., something like this: > > diff --git a/remote.c b/remote.c > index 298f2f93f..720d616be 100644 > --- a/remote.c > +++ b/remote.c > @@ -373,6 +373,8 @@ static int handle_config(const char *key, const char > *value, void *cb) > } > remote = make_remote(name, namelen); > remote->origin = REMOTE_CONFIG; > + if (current_config_scope() == CONFIG_SCOPE_REPO) > + remote->configured = 1; > if (!strcmp(subkey, "mirror")) > remote->mirror = git_config_bool(key, value); > else if (!strcmp(subkey, "skipdefaultupdate")) > > That doesn't make your test pass, but I think that is only because your > test is not covering the interesting case (it puts the new config in the > repo config, not in ~/.gitconfig). > > What do you think? > >> Originally, I would even have put the "vcs" into that set, as I could see >> a legitimate use case for users to configure "remote.svn.vcs = vcs" in >> ~/.gitconfig. But the regression test suite specifically tests for that >> case, and I trust that there was a good reason, even if Thomas did not >> describe that good reason in the commit message nor in any reply to this >> patch pair. > > The config-scope thing above would allow "remote.svn.vcs" in > ~/.gitconfig. But I don't think the test script actually checks that; it > checks for the repo-level config. And we would continue to do the right > thing there. I am not "you" you are addressing to, but I think tying it to where the variable came from makes quite sense. Because it makes it no longer possible to just inspect the configured result to answer "is the remote configured?", introduction of the configured field also needs to be preserved from the original by Dscho, so does reading from historical non-config sources like $GIT_DIR/remotes/*, which are by definition per-repository thing. IOW, with this tweak (and not setting ->configured based on what keys are set), I think Dscho's patch makes sense.
Re: [PATCH 2/2] Be more careful when determining whether a remote was configured
On Thu, Jan 19, 2017 at 12:12:47PM -0800, Junio C Hamano wrote: > > The config-scope thing above would allow "remote.svn.vcs" in > > ~/.gitconfig. But I don't think the test script actually checks that; it > > checks for the repo-level config. And we would continue to do the right > > thing there. > > I am not "you" you are addressing to, but I think tying it to where > the variable came from makes quite sense. > > Because it makes it no longer possible to just inspect the > configured result to answer "is the remote configured?", > introduction of the configured field also needs to be preserved from > the original by Dscho, so does reading from historical non-config > sources like $GIT_DIR/remotes/*, which are by definition > per-repository thing. > > IOW, with this tweak (and not setting ->configured based on what > keys are set), I think Dscho's patch makes sense. Yeah, worry if that wasn't clear: the hunk I posted was a just a partial. The actual thing I built and ran against the test suite was exactly as you described. -Peff
Re: [PATCH 2/2] Be more careful when determining whether a remote was configured
On Wed, Jan 18, 2017 at 05:22:40PM +0100, Johannes Schindelin wrote: > > > I want to err on the side of caution. That's why. > > > > I guess I just don't see why changing the behavior with respect to > > "prune" or "proxy" is any less conservative than changing the one for > > "refspec". > > Let's take a step back and consider the problem I try to solve, okay? OK. Though after reading your message over several times, I think I may have confused things by raising two separate issues. So let me re-state my issues here for clarity: 1. I'm concerned that a setting for remote..X would fail to apply to a bare "git fetch " if "X" is considered as "not really" configuring a remote. 2. I'm concerned that splitting the remote.*.X keys into two classes: "really configures" and "does not really configure" creates a confusing interface for users. Some keys are OK to set in your ~/.gitconfig and others are not. Let's talk about concern 1 first. Based on your analysis in this message, it looks like it _isn't_ a problem with your patch (because is_configured is never used for applying values, only for add/del/rename types of operations in remote.c). So good. Thanks for addressing my concern. And that makes your "conservative" comment make more sense; the idea is that you are not breaking anything, but just loosening selectively for some values of "X". I think there are still some weird corner cases even for "prune", though. E.g., try: git init git remote add foo whatever git config remote.foo.prune true git config remote.other.prune false So now we have: [remote "foo"] url = whatever fetch = +refs/heads/*:refs/remotes/foo/* prune = true [remote "other"] prune = false Now try "git remote rename foo other". With current versions of git, it's disallowed. With your patch, I get: [remote "other"] url = whatever prune = true [remote "other"] prune = false fetch = +refs/heads/*:refs/remotes/other/* The old value of remote.other.prune is overriding the one we tried to move into place. In your motivating example, the old value is in ~/.gitconfig, so it automatically takes lower precedence, and everything just works (I think it would also just work if "other" had been defined originally _before_ foo in the config file). I think you could fix it by having git-remote teach the "not really" config values (like "prune") to overwrite any existing value when rewriting the config. I think this just needs the multi_replace flag set when calling git_config_set_multivar(). That raises questions about what should happen when multi-value keys like "refspec" would be set (if we were to add them to the "not really" set). Should they be overwritten, or merged? And in that sense, your patch lets you punt on those issues. I still think my second concern is valid. It's made worse by your patch (if only because everything was disallowed before, and now some things are and some things aren't). If this is a first step towards a final state where the rules are simple, then starting conservatively makes sense. And until we get there, the answer "yes, it should, but nobody has worked out the semantics yet; care to make a patch?" is an OK one. But it sounds like you do not ever want to loosen the "refspec" case. I don't think that's ideal, but at the very least if that's the end state, the list of "OK to use in ~/.gitconfig" keys should probably be documented. Reading your message, though, I still wonder if we can do better... > The problem is that `git remote rename ` refuses to do its job > if it detects a configured ``. And it detects a configured `` > even in cases where there is not *really* any remote configured. I'd add to this that "git remote add " should work in a similar way (that was the one that I think people often ran into with remote.origin.fetch refspecs). > The example use case is to configure `remote.origin.prune = true` in > ~/.gitconfig, i.e. changing Git's default for all "origin" remotes in all > of the user's repositories. > > Now, the *real* fix would be to detect whether the remote was "configured" > in the current repository, or in ~/.gitconfig. But that may not even be > desirable, as we would want a more general fix for the question: can `git > remote` rename a given remote to a new name, i.e. is that new name already > taken? I think _this_ is a much better way of framing the problem. It is not about which keys are set, but about _where_ they are set. IOW, a reasonable rule would be: if there is any remote.*.X in the repo config, then git-remote should consider it a configured repo. And otherwise, no matter what is in ~/.gitconfig or elsewhere, git-remote should proceed as if it doesn't exist (and repo-level config can take precedence over config defined elsewhere). I.e., something like this: diff --gi
Re: [PATCH 2/2] Be more careful when determining whether a remote was configured
Hi Peff,
On Wed, 18 Jan 2017, Jeff King wrote:
> On Wed, Jan 18, 2017 at 01:34:28PM +0100, Johannes Schindelin wrote:
>
> > > > Let's fix this by telling Git that a remote is not configured
> > > > unless any fetch/push URL or refspect is configured explicitly.
> > >
> > > Hmm. Old versions of GitHub for Windows used to set fetch refspecs
> > > in the system gitconfig, for a similar purpose to what you want to
> > > do with remote.origin.prune.
> > >
> > > I notice here that setting a refspec _does_ define a remote. Is
> > > there a reason you drew the line there, and not at, say, whether it
> > > has a URL?
> >
> > I want to err on the side of caution. That's why.
>
> I guess I just don't see why changing the behavior with respect to
> "prune" or "proxy" is any less conservative than changing the one for
> "refspec".
Let's take a step back and consider the problem I try to solve, okay?
The problem is that `git remote rename ` refuses to do its job
if it detects a configured ``. And it detects a configured ``
even in cases where there is not *really* any remote configured.
The example use case is to configure `remote.origin.prune = true` in
~/.gitconfig, i.e. changing Git's default for all "origin" remotes in all
of the user's repositories.
Now, the *real* fix would be to detect whether the remote was "configured"
in the current repository, or in ~/.gitconfig. But that may not even be
desirable, as we would want a more general fix for the question: can `git
remote` rename a given remote to a new name, i.e. is that new name already
taken?
And if you try to answer that last question, you will probably pick the
same set of keys for which you assume that remote.. really
configures a remote or not.
Originally, I would even have put the "vcs" into that set, as I could see
a legitimate use case for users to configure "remote.svn.vcs = vcs" in
~/.gitconfig. But the regression test suite specifically tests for that
case, and I trust that there was a good reason, even if Thomas did not
describe that good reason in the commit message nor in any reply to this
patch pair.
So how can things go wrong?
Let's take your example that the user may have specified refspecs (or
prune, or proxy) for a URL via "remote..fetch", and that `git rename`
incorrectly allows that as a new remote name. You know what? Let's do a
real code review here, not just a patch glance-over. Let's test this and
*know* whether it can be a problem:
git remote rename origin https://github.com/git/git
fatal: 'https://github.com/git/git' is not a valid remote name
A ha! It is *not* possible to hit that case because `git remote rename`
already complains much earlier about the new name not being a valid name.
So let's see what could go wrong with another example you mentioned, that
the proxy may not be used because we changed the logic of
remote_is_configured(). But the only user of the remote proxy settings is
in http_init() and reads:
if (remote && remote->http_proxy)
curl_http_proxy = xstrdup(remote->http_proxy);
if (remote)
var_override(&http_proxy_authmethod,
remote->http_proxy_authmethod);
It does not even test whether the remote is configured! So maybe the
caller does? Nope, the only caller of http_init() that passes a remote is
remote-curl's cmd_main() function, and the relevant part reads:
remote = remote_get(argv[1]);
if (argc > 2) {
end_url_with_slash(&url, argv[2]);
} else {
end_url_with_slash(&url, remote->url[0]);
}
http_init(remote, url.buf, 0);
This code also does not care whether the remote "is configured" or not.
So maybe there are any other downsides with callers of
remote_is_configured()?
There is one caller in builtin/fetch.c's add_remote_or_group() which
clearly is covered (we need a URL, unless the vcs is configured).
All other callers are in builtin/remote.c:
- in add(), to test whether we can add a new remote,
- in mv(), to test whether the remote to rename is configured,
- in mv(), to test whether the new name is already taken,
- in rm(), to test whether the remote exists,
- in set_remote_branches(), to test whether the remote to be changed
exists,
- in get_url(), to test whether the remote exists, and
- in set_url(), to test whether the remote exists.
It appears pretty obvious that in all of these cases, the suggested patch
still makes sense and does not introduce any nasty surprise.
> I can think of one alternative approach that might be easier for users
> to understand, and that we already use elsewhere (e.g., with "http.*"
> config): have a set of "default" remote keys (e.g., just "remote.key")
> that git falls back to when the remote.*.key isn't set. Then your use
> case becomes something like:
>
> [remote]
> prune = true
>
> That's not _exactly_ the same, as it applies to all remotes, not just
> "origin" (other remotes can override it, but would need to
Re: [PATCH 2/2] Be more careful when determining whether a remote was configured
On Wed, Jan 18, 2017 at 01:34:28PM +0100, Johannes Schindelin wrote: > > > Let's fix this by telling Git that a remote is not configured unless any > > > fetch/push URL or refspect is configured explicitly. > > > > Hmm. Old versions of GitHub for Windows used to set fetch refspecs in > > the system gitconfig, for a similar purpose to what you want to do with > > remote.origin.prune. > > > > I notice here that setting a refspec _does_ define a remote. Is there a > > reason you drew the line there, and not at, say, whether it has a URL? > > I want to err on the side of caution. That's why. I guess I just don't see why changing the behavior with respect to "prune" or "proxy" is any less conservative than changing the one for "refspec". Both can make some real-world cases work, and both can cause breakage in some possible real-world cases. If we are going to change anything, it seems like we should at least aim for a simple and consistent rule (since users have to know which keys can be put in ~/.gitconfig and which cannot). I can think of one alternative approach that might be easier for users to understand, and that we already use elsewhere (e.g., with "http.*" config): have a set of "default" remote keys (e.g., just "remote.key") that git falls back to when the remote.*.key isn't set. Then your use case becomes something like: [remote] prune = true That's not _exactly_ the same, as it applies to all remotes, not just "origin" (other remotes can override it, but would need to do so explicitly). But I have a suspicion that may actually be what users want. -Peff
Re: [PATCH 2/2] Be more careful when determining whether a remote was configured
Hi Junio,
On Tue, 17 Jan 2017, Junio C Hamano wrote:
> Perhaps instead of adding "is it configured?" flag that is too
> broadly named and has too narrow meaning, would it make more sense
> to introduce "int can_prune(struct remote *remote)" that looks at
> the remote refspecs?
That ("can we prune?") is not the question we need to ask when determining
whether we can rename a remote to a new name.
Ciao,
Johannes
Re: [PATCH 2/2] Be more careful when determining whether a remote was configured
Hi Peff, On Tue, 17 Jan 2017, Jeff King wrote: > On Tue, Jan 17, 2017 at 10:19:24PM +0100, Johannes Schindelin wrote: > > > One of the really nice features of the ~/.gitconfig file is that users > > can override defaults by their own preferred settings for all of their > > repositories. > > > > One such default that some users like to override is whether the > > "origin" remote gets auto-pruned or not. The user would simply call > > > > git config --global remote.origin.prune true > > > > and from now on all "origin" remotes would be pruned automatically when > > fetching into the local repository. > > > > There is just one catch: now Git thinks that the "origin" remote is > > configured, as it does not discern between having a remote whose > > fetch (and/or push) URL and refspec is set, and merely having > > preemptively-configured, global flags for specific remotes. > > > > Let's fix this by telling Git that a remote is not configured unless any > > fetch/push URL or refspect is configured explicitly. > > Hmm. Old versions of GitHub for Windows used to set fetch refspecs in > the system gitconfig, for a similar purpose to what you want to do with > remote.origin.prune. > > I notice here that setting a refspec _does_ define a remote. Is there a > reason you drew the line there, and not at, say, whether it has a URL? I want to err on the side of caution. That's why. Ciao, Johannes
Re: [PATCH 2/2] Be more careful when determining whether a remote was configured
Jeff King writes: >> Let's fix this by telling Git that a remote is not configured unless any >> fetch/push URL or refspec is configured explicitly. > > I notice here that setting a refspec _does_ define a remote. Is there a > reason you drew the line there, and not at, say, whether it has a URL? "Not configured unless any URL or refspec is configured" means that if URL is there, even if there is no refspec, then it is a remote definition, right? But I think "what does it mean to define a remote" is a question that you are asking, and that is not necessary to answer within the scope of this topic. I do agree that honoring .prune is nonsense unless refspecs are defined, but the question "does it make sense to allow prune?" is different from "is it configured?". Your "you can set .proxy and other useful things, it is just .prune does not make sense" is a quite appropriate statement to illustrate the difference. Perhaps instead of adding "is it configured?" flag that is too broadly named and has too narrow meaning, would it make more sense to introduce "int can_prune(struct remote *remote)" that looks at the remote refspecs?
Re: [PATCH 2/2] Be more careful when determining whether a remote was configured
On Tue, Jan 17, 2017 at 10:19:24PM +0100, Johannes Schindelin wrote: > One of the really nice features of the ~/.gitconfig file is that users > can override defaults by their own preferred settings for all of their > repositories. > > One such default that some users like to override is whether the > "origin" remote gets auto-pruned or not. The user would simply call > > git config --global remote.origin.prune true > > and from now on all "origin" remotes would be pruned automatically when > fetching into the local repository. > > There is just one catch: now Git thinks that the "origin" remote is > configured, as it does not discern between having a remote whose > fetch (and/or push) URL and refspec is set, and merely having > preemptively-configured, global flags for specific remotes. > > Let's fix this by telling Git that a remote is not configured unless any > fetch/push URL or refspect is configured explicitly. Hmm. Old versions of GitHub for Windows used to set fetch refspecs in the system gitconfig, for a similar purpose to what you want to do with remote.origin.prune. I notice here that setting a refspec _does_ define a remote. Is there a reason you drew the line there, and not at, say, whether it has a URL? -Peff
[PATCH 2/2] Be more careful when determining whether a remote was configured
One of the really nice features of the ~/.gitconfig file is that users
can override defaults by their own preferred settings for all of their
repositories.
One such default that some users like to override is whether the
"origin" remote gets auto-pruned or not. The user would simply call
git config --global remote.origin.prune true
and from now on all "origin" remotes would be pruned automatically when
fetching into the local repository.
There is just one catch: now Git thinks that the "origin" remote is
configured, as it does not discern between having a remote whose
fetch (and/or push) URL and refspec is set, and merely having
preemptively-configured, global flags for specific remotes.
Let's fix this by telling Git that a remote is not configured unless any
fetch/push URL or refspect is configured explicitly.
As a special exception, we deem a remote configured also when *only* the
"vcs" setting is configured. The commit a31eeae27f (remote: use
remote_is_configured() for add and rename, 2016-02-16) specifically
extended our test suite to verify this, so it is safe to assume that there
has been at least one user with a legitimate use case for this.
This fixes https://github.com/git-for-windows/git/issues/888
Signed-off-by: Johannes Schindelin
---
remote.c | 9 -
remote.h | 2 +-
t/t5505-remote.sh | 2 +-
3 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/remote.c b/remote.c
index ad6c5424ed..298f2f93fa 100644
--- a/remote.c
+++ b/remote.c
@@ -255,6 +255,7 @@ static void read_remotes_file(struct remote *remote)
if (!f)
return;
+ remote->configured = 1;
remote->origin = REMOTE_REMOTES;
while (strbuf_getline(&buf, f) != EOF) {
const char *v;
@@ -289,6 +290,7 @@ static void read_branches_file(struct remote *remote)
return;
}
+ remote->configured = 1;
remote->origin = REMOTE_BRANCHES;
/*
@@ -384,21 +386,25 @@ static int handle_config(const char *key, const char
*value, void *cb)
if (git_config_string(&v, key, value))
return -1;
add_url(remote, v);
+ remote->configured = 1;
} else if (!strcmp(subkey, "pushurl")) {
const char *v;
if (git_config_string(&v, key, value))
return -1;
add_pushurl(remote, v);
+ remote->configured = 1;
} else if (!strcmp(subkey, "push")) {
const char *v;
if (git_config_string(&v, key, value))
return -1;
add_push_refspec(remote, v);
+ remote->configured = 1;
} else if (!strcmp(subkey, "fetch")) {
const char *v;
if (git_config_string(&v, key, value))
return -1;
add_fetch_refspec(remote, v);
+ remote->configured = 1;
} else if (!strcmp(subkey, "receivepack")) {
const char *v;
if (git_config_string(&v, key, value))
@@ -427,6 +433,7 @@ static int handle_config(const char *key, const char
*value, void *cb)
return git_config_string((const char
**)&remote->http_proxy_authmethod,
key, value);
} else if (!strcmp(subkey, "vcs")) {
+ remote->configured = 1;
return git_config_string(&remote->foreign_vcs, key, value);
}
return 0;
@@ -716,7 +723,7 @@ struct remote *pushremote_get(const char *name)
int remote_is_configured(struct remote *remote)
{
- return remote && remote->origin;
+ return remote && remote->configured;
}
int for_each_remote(each_remote_fn fn, void *priv)
diff --git a/remote.h b/remote.h
index 924881169d..7e6c8067bb 100644
--- a/remote.h
+++ b/remote.h
@@ -15,7 +15,7 @@ struct remote {
struct hashmap_entry ent; /* must be first */
const char *name;
- int origin;
+ int origin, configured;
const char *foreign_vcs;
diff --git a/t/t5505-remote.sh b/t/t5505-remote.sh
index d7e41e9230..09c9823002 100755
--- a/t/t5505-remote.sh
+++ b/t/t5505-remote.sh
@@ -764,7 +764,7 @@ test_expect_success 'rename a remote with name prefix of
other remote' '
)
'
-test_expect_failure 'rename succeeds with existing remote..prune' '
+test_expect_success 'rename succeeds with existing remote..prune' '
git clone one four.four &&
(
cd four.four &&
--
2.11.0.windows.3
