Re: [git-users] Problem using detached worktrees with commands implemented in scripts
From: Junio C Hamano gits...@pobox.com ... it's not clear why GIT_WORK_TREE exists, ... The configuration item came _way_ later than the environment, and we need to keep users and scripts from old world working, that is why. OK, that explains a great deal. IIRC, I first became aware that detached worktrees are possible through the documentation of core.worktree. As Git's architecture has a tight binding between the repository and the worktree, it made a great deal of sense to me that the repository points to the detached worktree. And the absence of core.worktree, a non-detached worktree, is essentially equivalent to having core.worktree specify the directory containing the .git directory. So the obvious way (to me) to invoke Git with a detached worktree is to set GIT_DIR to point to the repository, and the repository points to the root of the worktree. If the command operates on the worktree, Git can compare the cwd with the worktree root to determine the relative path of files. (And you can see that in this situation, Git doesn't have to search upward to try to determine where the worktree root is.) What you're saying is that there's an older mode of operation where the repository does not point to the worktree. Instead, the caller has to set GIT_DIR to locate the repository and GIT_WORK_TREE to locate the worktree. This would be prone to error, because the user is responsible for always pairing the repository with the correct worktree; it doesn't enforce the architectural assumption that the repository is paired with a work tree. Dale -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [git-users] Problem using detached worktrees with commands implemented in scripts
From: Jonathan Nieder jrnie...@gmail.com Philip Oakley wrote: It was Dale that had the problem, I was just suggesting where he might want to look... ;-) A bit more looking gave that the cd_to_toplevel () in git-sh-setup.sh directly uses `git rev-parse --show-toplevel`, which simply returns work_tree (static char *work_tree; in environment.c, with comment /* This is set by setup_git_dir_gently() and/or git_default_config() */), apparently without a check for the GIT_WORK_TREE. Getting closer. :) The usual way to use GIT_WORK_TREE is along with GIT_DIR. When re-checking the manual's git(1) env variable section the comment that implies this didn't read well The value will not be used in combination The section probably needs that to be stated explicitly (an exported GIT_WORK_TREE is ignored if GIT_DIR is not set). That takes you into the setup_explicit_git_dir() codepath, which does respect GIT_WORK_TREE as it should. (setup_discovered_git_dir does, too.) The strange behavior you ran into is that unlike, say, git-pull.sh and git-am.sh, filter-branch does not set SUBDIRECTORY_OK, store the prefix from 'git rev-parse --show-prefix', and then cd_to_toplevel at the top of the script. In other words, nobody bothered to make it work from anywhere other than the toplevel of the worktree to begin with, and nobody wanted it enough to fix it later. I maybe wrong, but I thought that in Dale's case he was already at the same level as the GIT_WORK_TREE he had set, so may not have expected to need a cd_to_toplevel Dale did propose a patch in http://thread.gmane.org/gmane.comp.version-control.git/236260 as the error was from later in the setup script. Hope that helps, Jonathan Philip -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [git-users] Problem using detached worktrees with commands implemented in scripts
From: Junio C Hamano gits...@pobox.com Side note: without GIT_WORK_TREE environment (or core.worktree), there is no way to tell where the top level is, so you were limited to always be at the top level of your working tree if you used GIT_DIR to refer to a repository that is not embedded in your working tree. There were some changes in this area, but I do not recall the details offhand. That's not true. The core.worktree config variable tells the top of the worktree, so once you've located the repository, you know where the worktree is. Indeed, it's not clear why GIT_WORK_TREE exists, as that allows the user to set GIT_WORK_TREE inconsistently with core.worktree. (What happens if you do that?) Dale -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [git-users] Problem using detached worktrees with commands implemented in scripts
wor...@alum.mit.edu (Dale R. Worley) writes: From: Junio C Hamano gits...@pobox.com Side note: without GIT_WORK_TREE environment (or core.worktree), there is no way to tell where the top level is, so you were limited to always be at the top level of your working tree if you used GIT_DIR to refer to a repository that is not embedded in your working tree. There were some changes in this area, but I do not recall the details offhand. That's not true. The core.worktree config variable tells the top of the worktree, so once you've located the repository, you know where the worktree is. Read the second line again, perhaps? ... it's not clear why GIT_WORK_TREE exists, ... The configuration item came _way_ later than the environment, and we need to keep users and scripts from old world working, that is why. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [git-users] Problem using detached worktrees with commands implemented in scripts
From: Junio C Hamano gits...@pobox.com Now, when you say the cwd contains the .git directory, do you mean cd /repositories git add ../working/trees/proj-wt1/file updates file in the /repositories/proj.git/index? Or do you mean this? The pattern I use is to have this: /repository/.git /working/... then cd /repository git add /working/x/y/z works as you'd expect it to. git rm seems to work correctly under these circumstances as well. I seem to recall that using relative path values doesn't work under some conditions involving symbolic links, but I can't recall the details right now. you talk about starting Git command _outside_ the working tree (whether the working tree has its repository embedded in it is not very relevant). The above pattern is what I mean, where the cwd is not within the work tree. Dale -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [git-users] Problem using detached worktrees with commands implemented in scripts
From: Junio C Hamano gits...@pobox.com It was unclear to me which part of our documentation needs updating and how, and that was (and still is) what I was primarily interested in finding out. It seems to me that what is missing is a description of the circumstances under which Git can be run. With Subversion (the only other source control system I know in detail), the working tree that is operated on is at and below the cwd, and the working tree always points to the repository. (A subdirectory of a working tree is also a valid working tree.) With Git, it seems that the basic usage is that Git searches upward from the cwd to find the top of the work tree, which is distinguished by having a .git subdirectory. The rules when the worktree is detached are more complicated, and don't seem to be written in any single place. Dale -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [git-users] Problem using detached worktrees with commands implemented in scripts
From: Junio C Hamano gits...@pobox.com wor...@alum.mit.edu (Dale R. Worley) writes: In general, Git commands on a repository with a detached worktree can be executed by cd'ing into the directory containing the .git directory, ... Eh? News to me; it might happened to have appeared to work by accident, but that is not by design. I must admit I've never seen the design (and I personally doubt that the design has ever been written down). But at least the following commands work correctly on a detached worktree if the current directory contains the .git directory, because I am using them in a production manner: git add git cat-file git commit git commit-tree git config git gc git log git ls-tree git reset git rev-list git update-ref In my situation, the worktree is not, in my mind, dependent on the repository; the repository is intended to keep backups of the contents of the directories that are worktree. Indeed, one could establish several detached repositories to back up different subsets of the same worktree. So it is conceptually natural to execute Git in the repository directory. And, after all, the current directory identifies the repository and the repository contains a pointer to the worktree. Not exporting GIT_DIR variable in sh-setup was done not by accident but as a very deliberate design choice, IIRC. The intention of my change is that it appears that all of the failures of my use pattern are when the command is implemented by a shell script, and it appears that all shell scripts initially invoke git-sh-setup. The change specifically detects my use pattern and, for the remainder of the script, changes the use pattern into a pattern closely related to the one that Junio documents: - export GIT_DIR that points at the correct .git directory; - GIT_WORK_TREE is left unset - set the current directory to the top of the working tree Perhaps the change should also set GIT_WORK_TREE. I haven't noticed setting GIT_WORK_TREE to be necessary for git filter-branch, but perhaps that is coincidental. It seems to me that this change would uniformly support the use pattern I use without affecting Git's behavior in any other case. Dale -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [git-users] Problem using detached worktrees with commands implemented in scripts
wor...@alum.mit.edu (Dale R. Worley) writes:
I must admit I've never seen the design (and I personally doubt that
the design has ever been written down). But at least the following
commands work correctly on a detached worktree if the current
directory contains the .git directory, because I am using them in a
production manner:
git add
If you have this:
/repositories/proj.git/{refs,objects,...}
/working/trees/proj-wt1/
where proj-wt1 is a working tree for that proj.git repository, the
idea was to set these:
GIT_DIR=/repositories/proj.git
GIT_WORK_TREE=/working/trees/proj-wt1
export GIT_DIR GIT_WORK_TREE
and then working in /working/trees/proj-wt1 or any of its
subdirectory should work as if you did not have these two
environment variables and had /working/trees/proj-wt1/.git instead
of /repositories/proj.git as the repository. To make that use case
work was the motivation behind these environment variables.
Side note: without GIT_WORK_TREE environment (or
core.worktree), there is no way to tell where the top level
is, so you were limited to always be at the top level of
your working tree if you used GIT_DIR to refer to a
repository that is not embedded in your working tree. There
were some changes in this area, but I do not recall the
details offhand.
Now, when you say the cwd contains the .git directory, do you mean
cd /repositories
git add ../working/trees/proj-wt1/file
updates file in the /repositories/proj.git/index? Or do you mean
this?
cd /repositories/proj.git
git add ../../working/trees/proj-wt1/file
Or this?
cd /repositories
edit ../working/trees/proj-wt1/file
git add file
Most of the commands you listed do not need to look at the actual
working tree files, so I would expect e.g. git log or git log
paths... to work but I am wondering what your definition of works
with respect to the pathspecs, especially when you talk about
starting Git command _outside_ the working tree (whether the working
tree has its repository embedded in it is not very relevant).
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [git-users] Problem using detached worktrees with commands implemented in scripts
From: Junio C Hamano gits...@pobox.com Philip Oakley philipoak...@iee.org writes: ... and the detection process for 'toplevel' may not work properly when in a separated work-tree environment. Without GIT_WORK_TREE exported to point at the top-level, there is nothing that lets us detect it, as the working tree does not have .git directory to tell us to stop, no? No, but not in that way. My point (to Dale) was, as you state, that the cd to top level was (IIUC) the probable causes of the fault, and that a documentation update would probably be appropriate for the discussion on exporting GIT_WORK_TREE, and that it would specifically mention those git commands that needed to cd to top level, and hence would not work in such an environment. (I wasn't sure where the appropriate cd to top level function was) An explanation here on the list wouldn't solve the problems for others who are yet to make the same mistake, hence the implied suggestion. Philip -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [git-users] Problem using detached worktrees with commands implemented in scripts
Philip Oakley philipoak...@iee.org writes: From: Junio C Hamano gits...@pobox.com Philip Oakley philipoak...@iee.org writes: ... and the detection process for 'toplevel' may not work properly when in a separated work-tree environment. Without GIT_WORK_TREE exported to point at the top-level, there is nothing that lets us detect it, as the working tree does not have .git directory to tell us to stop, no? No, but not in that way. My point (to Dale) was, as you state, that the cd to top level was (IIUC) the probable causes of the fault, and that a documentation update would probably be appropriate for the discussion on exporting GIT_WORK_TREE, and that it would specifically mention those git commands that needed to cd to top level, and hence would not work in such an environment. (I wasn't sure where the appropriate cd to top level function was) An explanation here on the list wouldn't solve the problems for others who are yet to make the same mistake, hence the implied suggestion. I understand what you mean by these last two lines. It was unclear to me which part of our documentation needs updating and how, and that was (and still is) what I was primarily interested in finding out. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [git-users] Problem using detached worktrees with commands implemented in scripts
From: Junio C Hamano gits...@pobox.com Philip Oakley philipoak...@iee.org writes: From: Junio C Hamano gits...@pobox.com Philip Oakley philipoak...@iee.org writes: ... and the detection process for 'toplevel' may not work properly when in a separated work-tree environment. Without GIT_WORK_TREE exported to point at the top-level, there is nothing that lets us detect it, as the working tree does not have .git directory to tell us to stop, no? No, but not in that way. My point (to Dale) was, as you state, that the cd to top level was (IIUC) the probable causes of the fault, and that a documentation update would probably be appropriate for the discussion on exporting GIT_WORK_TREE, and that it would specifically mention those git commands that needed to cd to top level, and hence would not work in such an environment. (I wasn't sure where the appropriate cd to top level function was) An explanation here on the list wouldn't solve the problems for others who are yet to make the same mistake, hence the implied suggestion. I understand what you mean by these last two lines. It was unclear to me which part of our documentation needs updating and how, and that was (and still is) what I was primarily interested in finding out. I was expecting that the places would be in git(1) [git.txt] and config(1) [config.txt], in the enironment variables GIT_WORK_TREE section and core.worktree sections repectively. However what the right text would be hasn't been fully determined yet, as it should be clear about which commands don't follow the stated 'rules'. Dale's use case does appear to be stretching... Philip -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [git-users] Problem using detached worktrees with commands implemented in scripts
From: Philip Oakley philipoak...@iee.org From: Junio C Hamano gits...@pobox.com Philip Oakley philipoak...@iee.org writes: From: Junio C Hamano gits...@pobox.com Philip Oakley philipoak...@iee.org writes: ... and the detection process for 'toplevel' may not work properly when in a separated work-tree environment. Without GIT_WORK_TREE exported to point at the top-level, there is nothing that lets us detect it, as the working tree does not have .git directory to tell us to stop, no? No, but not in that way. My point (to Dale) was, as you state, that the cd to top level was (IIUC) the probable causes of the fault, and that a documentation update would probably be appropriate for the discussion on exporting GIT_WORK_TREE, and that it would specifically mention those git commands that needed to cd to top level, and hence would not work in such an environment. (I wasn't sure where the appropriate cd to top level function was) An explanation here on the list wouldn't solve the problems for others who are yet to make the same mistake, hence the implied suggestion. I understand what you mean by these last two lines. It was unclear to me which part of our documentation needs updating and how, and that was (and still is) what I was primarily interested in finding out. I was expecting that the places would be in git(1) [git.txt] and config(1) [config.txt], in the enironment variables GIT_WORK_TREE section and core.worktree sections repectively. However what the right text would be hasn't been fully determined yet, as it should be clear about which commands don't follow the stated 'rules'. Dale's use case does appear to be stretching... Philip A bit more looking gave that the cd_to_toplevel () in git-sh-setup.sh directly uses `git rev-parse --show-toplevel`, which simply returns work_tree (static char *work_tree; in environment.c, with comment /* This is set by setup_git_dir_gently() and/or git_default_config() */), apparently without a check for the GIT_WORK_TREE. One option may be to either protect the cd_to_toplevel code with a check of `git rev-parse --local-env-vars` to see if GIT_WORK_TREE is present. Or create `git rev-parse --work-dir` to match `--git-dir`. This would be a code level fix. This makes the assumption that if a deteched GIT_WORK_TREE is set then it is the top level. In terms of command scripts that use git-sh-setup.sh we have a longish list, so a full list in the documentation is probably unreasonable (which suggests that a code fix would be more apprpriate) commands: git-am git-bisect git-filter-branch git-instaweb git-lost-found git-merge-one-file git-mergetool git-pull git-quiltimport git-rebase git-repack git-request-pull git-stash git-submodule git-web--browse git\contrib\*various* Philip -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [git-users] Problem using detached worktrees with commands implemented in scripts
Philip Oakley wrote: A bit more looking gave that the cd_to_toplevel () in git-sh-setup.sh directly uses `git rev-parse --show-toplevel`, which simply returns work_tree (static char *work_tree; in environment.c, with comment /* This is set by setup_git_dir_gently() and/or git_default_config() */), apparently without a check for the GIT_WORK_TREE. Getting closer. :) The usual way to use GIT_WORK_TREE is along with GIT_DIR. That takes you into the setup_explicit_git_dir() codepath, which does respect GIT_WORK_TREE as it should. (setup_discovered_git_dir does, too.) The strange behavior you ran into is that unlike, say, git-pull.sh and git-am.sh, filter-branch does not set SUBDIRECTORY_OK, store the prefix from 'git rev-parse --show-prefix', and then cd_to_toplevel at the top of the script. In other words, nobody bothered to make it work from anywhere other than the toplevel of the worktree to begin with, and nobody wanted it enough to fix it later. Hope that helps, Jonathan -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[git-users] Problem using detached worktrees with commands implemented in scripts
In Git, one can set up a repository with a detached worktree, where
the .git directory is not a subdirectory of the top directory of the
work tree.
In general, Git commands on a repository with a detached worktree can
be executed by cd'ing into the directory containing the .git
directory, and executing the Git command there. E.g., git add and
git commit execute as one would expect. (I think they can also be
executed by cd'ing to the worktree and setting GIT_DIR.)
However, this approach does not work with git filter-branch, which
objects with You need to run this command from the toplevel of the
working tree.
I suspect that it does not work with other Git commands that are
implemented with shell scripts. The problem appears to be in the
git-sh-setup script, which is called by the Git shell scripts to set
up the environment and do preliminary tests.
It seems to me that this inconsistency between the script commands and
the binary commands can be fixed by updating git-sh-setup in this way:
--- git-sh-setup.Custom.orig2013-06-20 12:59:45.0 -0400
+++ git-sh-setup2013-10-07 22:34:06.719946134 -0400
@@ -297,14 +297,18 @@
# if we require to be in a git repository.
if test -z $NONGIT_OK
then
- GIT_DIR=$(git rev-parse --git-dir) || exit
+ export GIT_DIR=$(git rev-parse --git-dir) || exit
if [ -z $SUBDIRECTORY_OK ]
then
- test -z $(git rev-parse --show-cdup) || {
- exit=$?
- echo 2 You need to run this command from the
toplevel of the working tree.
- exit $exit
- }
+ cdup=$(git rev-parse --show-cdup)
+ if [ -n $cdup ]
+ then
+ # Current directory is not the toplevel.
+ # Set GIT_DIR to the absolute path of the repository.
+ GIT_DIR=$(cd $GIT_DIR pwd)
+ # cd to the toplevel.
+ cd $cdup
+ fi
fi
test -n $GIT_DIR GIT_DIR=$(cd $GIT_DIR pwd) || {
echo 2 Unable to determine absolute path of git directory
What this change does is, when a command is invoked from a directory
containing a repository with a detached worktree, is to set GIT_DIR to
the directory of the repository, then cd to the top of the worktree.
After that, the script command should work as expected.
I am far from being an expert in Git internals, so I don't know
whether this is the correct approach to take to this problem or not.
Does anyone have any feedback on this?
Dale
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [git-users] Problem using detached worktrees with commands implemented in scripts
wor...@alum.mit.edu (Dale R. Worley) writes: In general, Git commands on a repository with a detached worktree can be executed by cd'ing into the directory containing the .git directory, ... Eh? News to me; it might happened to have appeared to work by accident, but that is not by design. IIRC, the intended use pattern (i.e. the change that introduced GIT_DIR and GIT_WORK_TREE environment variables was designed to support) for such a working tree is to: - export GIT_DIR that points at the correct .git directory; - export GIT_WORK_TREE that points at the correct top-level of such a working tree; and then - run the commands anywhere in the working tree, as if you did not export these two environment variables and instead had the .git directory at the usual place in the working tree. It _is_ possible that we may have broken this canonical use pattern over time with more recent updates; I do not think we have extensive test coverage for detached worktree use case in the first place. Does anyone have any feedback on this? Not exporting GIT_DIR variable in sh-setup was done not by accident but as a very deliberate design choice, IIRC. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [git-users] Problem using detached worktrees with commands implemented in scripts
From: Junio C Hamano gits...@pobox.com
wor...@alum.mit.edu (Dale R. Worley) writes:
In general, Git commands on a repository with a detached worktree can
be executed by cd'ing into the directory containing the .git
directory, ...
Eh? News to me; it might happened to have appeared to work by
accident, but that is not by design.
I think it is this part in Dale's original email
However, this approach does not work with git filter-branch, which
objects with You need to run this command from the toplevel of the
working tree.
that is the problem Dale has seen. IIRC there are a few commands that do
require to be run from the toplevel ('git bisect' I think is another),
and the detection process for 'toplevel' may not work properly when in a
separated work-tree environment.
Perhaps something to consider.
Philip
IIRC, the intended use pattern (i.e. the change that introduced
GIT_DIR and GIT_WORK_TREE environment variables was designed to
support) for such a working tree is to:
- export GIT_DIR that points at the correct .git directory;
- export GIT_WORK_TREE that points at the correct top-level of such
a working tree; and then
- run the commands anywhere in the working tree, as if you did not
export these two environment variables and instead had the .git
directory at the usual place in the working tree.
It _is_ possible that we may have broken this canonical use pattern
over time with more recent updates; I do not think we have extensive
test coverage for detached worktree use case in the first place.
Does anyone have any feedback on this?
Not exporting GIT_DIR variable in sh-setup was done not by accident
but as a very deliberate design choice, IIRC.
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [git-users] Problem using detached worktrees with commands implemented in scripts
Philip Oakley philipoak...@iee.org writes: ... and the detection process for 'toplevel' may not work properly when in a separated work-tree environment. Without GIT_WORK_TREE exported to point at the top-level, there is nothing that lets us detect it, as the working tree does not have .git directory to tell us to stop, no? -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
