Re: Get "responsible" .gitignore file / rule

[Victor Toni]

Am Fr., 7. Dez. 2018 um 13:45 Uhr schrieb Eric Sunshine
:
>
> On Fri, Dec 7, 2018 at 7:36 AM Victor Toni  wrote:
> > I'm wondering if there is any way to show which rules (ideally with
> > the .gitignore file they are coming from) are causing a specific file
> > to get ignored so I could easily fix the .gitignore file?
>
> Perhaps the "git check-ignore" command would help.

Thanks for the tip!
Works like a charm (had to use the --verbose option though, without,
it does not give much feedback)

Re: Get "responsible" .gitignore file / rule

[Eric Sunshine]

On Fri, Dec 7, 2018 at 7:36 AM Victor Toni  wrote:
> I'm wondering if there is any way to show which rules (ideally with
> the .gitignore file they are coming from) are causing a specific file
> to get ignored so I could easily fix the .gitignore file?

Perhaps the "git check-ignore" command would help.

Get "responsible" .gitignore file / rule

[Victor Toni]

In a rather complex setup with deep directory structure it happens
every now and then, that files get ignored when trying to add them.
As these files are _not_ shown in `git status` but in `git status
--ignored` so I guess the culprit is some misconfigured `.gitignore`.

Trying to ad the specific file gives a:
$ git add ignored/file/name
The following paths are ignored by one of your .gitignore files:
ignored/file/name
Use -f if you really want to add them.

Using -v doen't add any verbosity. I'm using git 2.19.1.windows.1 if
this matters

I'm wondering if there is any way to show which rules (ideally with
the .gitignore file they are coming from) are causing a specific file
to get ignored so I could easily fix the .gitignore file?

Re: the opposite of .gitignore, whitelist

[Jason Cooper]

On Fri, Oct 26, 2018 at 01:34:53PM +, Jason Cooper wrote:
> On Fri, Oct 26, 2018 at 02:39:26PM +0200, Ævar Arnfjörð Bjarmason wrote:
...
> > I thought this was a bug:
> > 
> > (
> > rm -rf /tmp/git &&
> > git init /tmp/git &&
> >     cd /tmp/git >/dev/null &&
> >     echo '*' >.gitignore &&
> > echo '!*.txt' >>.gitignore &&
> > echo '!.gitignore' >>.gitignore &&
> > touch foo.png foo.txt &&
> > mkdir dir &&
> > touch dir/bar.png dir/bar.txt &&
> > git add *.txt &&
> > git add */*.txt;
> > git status --short
> > )
> > 
> > But it's a limitation, gitignore(5) says:
> > 
> > It is not possible to re-include a file if a parent directory of
> > that file is excluded. Git doesn’t list excluded directories for
> > performance reasons, so any patterns on contained files have no
> > effect, no matter where they are defined.
> 
> Bingo.  This is the exact problem I encountered.

(
rm -rf /tmp/git &&
    git init /tmp/git &&
cd /tmp/git >/dev/null &&
echo '*' >.gitignore &&
echo '!dir/' >>.gitignore &&
echo '!*.txt' >>.gitignore &&
echo '!.gitignore' >>.gitignore &&
touch foo.png foo.txt &&
mkdir dir &&
echo '*' >dir/.gitignore &&
echo '!*.txt' >>dir/.gitignore &&
echo '!.gitignore' >>dir/.gitignore &&
touch dir/bar.png dir/bar.txt &&
git add *.txt &&
git add */*.txt;
git status --short
)

Well, this wfm...

Ugly, but doable.

thx,

Jason.

Re: the opposite of .gitignore, whitelist

[Jason Cooper]

On Fri, Oct 26, 2018 at 02:39:26PM +0200, Ævar Arnfjörð Bjarmason wrote:
> On Fri, Oct 26 2018, Jeff King wrote:
> > On Thu, Oct 25, 2018 at 10:38:46AM -0400, Jason Cooper wrote:
> >> On 10/25/18 1:37 AM, Junio C Hamano wrote:
> >> > "lhf...@163.com"  writes:
> >> >> I have a good idea, add a file to git that is the opposite of 
> >> >> .gitignore...,
> >> >
> >> > Do negative patterns in .gitignore file help without inventing
> >> > anything new?
> >>
> >> I did this several years ago in an attempt to track /etc/ (minus
> >> ownership, of course) without storing secrets in the git history. As
> >> the system grew and was maintained (read: crap added), the negative
> >> patterns grew untenable. I quickly realized it wasn't the correct way
> >> to solve the problem.
> >>
> >> Unfortunately, shortly after realizing this, I left that project. So I
> >> never had the chance to develop a proper solution. However, the concept
> >> of a '.gitonly' file was exactly was I was seeking. So, for what it's
> >> worth, I've definitely had at least one legit usecase for this feature.
> >>
> >> The usecases tend to center around tracking select files within the
> >> rootfs of a full-blown operating system. Or a subset thereof.
> >
> > I think what Junio meant is to ignore everything by default, like:
> >
> >   echo '*' >.gitignore
> >
> > and then selectively use negative patterns (and being in .gitignore,
> > that makes them positive "yes, include this") to add things back:
> >
> >   echo 'foo' >>.gitignore
> >
> > which ends up being roughly the same as your .gitonly concept.
> >
> > I don't offhand remember if you might run into problems where a
> > subdirectory is ignored by the "*" and we do not even recurse into it. I
> > think it would work OK as long as you put everything in the top-level
> > gitignore, like:
> >
> >   echo 'subdir/file' >>.gitignore
> >
> > but I didn't test.
> 
> This doesn't work, as explained to myself in this commit in a private
> project I have where I tried this a while ago:
> 
> I thought this was a bug:
> 
> (
> rm -rf /tmp/git &&
> git init /tmp/git &&
> cd /tmp/git >/dev/null &&
> echo '*' >.gitignore &&
> echo '!*.txt' >>.gitignore &&
> echo '!.gitignore' >>.gitignore &&
> touch foo.png foo.txt &&
> mkdir dir &&
> touch dir/bar.png dir/bar.txt &&
> git add *.txt &&
> git add */*.txt;
> git status --short
>     )
> 
> But it's a limitation, gitignore(5) says:
> 
> It is not possible to re-include a file if a parent directory of
> that file is excluded. Git doesn’t list excluded directories for
> performance reasons, so any patterns on contained files have no
> effect, no matter where they are defined.

Bingo.  This is the exact problem I encountered.

> So as a hack exclude anything that looks like a file with an
> extension.
> 
> 1 file changed, 1 insertion(+), 1 deletion(-)
> .gitignore | 2 +-
> 
> modified   .gitignore
> @@ -1,3 +1,3 @@
> -*
> +*.*
>  !*.gpg
>  !.gitignore
> 
> I.e. here I'm trying to maintain a repository where I only want
> .gitignore and *.gpg files committed and everything else ignored, but it
> only works for one directory level.

Perhaps a workflow solution using the existing .gitignore syntax would
be to:

- Use a separate .gitignore file per subdirectory
- Only list a subdirectory in a .gitignore file when you want to
  exclude the entire tree underneath the subdirectory

Which would give us two things we could warn on:

- If git detects a negative pattern space (file starts with '*')
- any directories under that .gitignore need their own
  .gitignore
- any directories listed in the .gitignore shall not have a
  .gitignore within them.

The warning could then point to the document I alluded to below.

> There's not a lot of room left in the gitignore syntax, but I suppose we
> could extend it to add some "I really mean it" negative pattern which
> would override previous patterns even if those previous patterns matched
> directories.

I'd argue against this.  This is a rare enough usecase, that it should
be possible, but doesn't need to be easy.  Extending the syntax will,
imo, suggest that it's supposed to be easy.  I'd rather see an official
doc for how to do it properly (maybe I'm on the right track with the
above?) with an explanation for why it is the way it is (efficiency,
rare usecase, etc)

> Just fixing it as a bug would make the ignore process slower, since we
> could no longer just ignore directories and would always need to
> recursively scan them.

Right, rare usecases shouldn't impede regular use.


thx,

Jason.

Re: the opposite of .gitignore, whitelist

[Rafael Ascensão]

One other option is to just use a pattern that matches everything, i.e:
echo '*' > .gitignore

And take advantage that ignore rules do not apply to tracked files.
So instead of using an explicit .gitonly, you add files using:
   git add -f 

All files should be ignored except the ones that were forcibly added.

If needed, git ls-files can be used to list either category.

Cheers,
Rafael Ascensão

Re: the opposite of .gitignore, whitelist

[Ævar Arnfjörð Bjarmason]

On Fri, Oct 26 2018, Jeff King wrote:

> On Thu, Oct 25, 2018 at 10:38:46AM -0400, Jason Cooper wrote:
>
>> On 10/25/18 1:37 AM, Junio C Hamano wrote:
>> > "lhf...@163.com"  writes:
>> >
>> >> I have a good idea, add a file to git that is the opposite of 
>> >> .gitignore...,
>> > Do negative patterns in .gitignore file help without inventing
>> > anything new?
>> I did this several years ago in an attempt to track /etc/ (minus
>> ownership, of course) without storing secrets in the git history. As
>> the system grew and was maintained (read: crap added), the negative
>> patterns grew untenable. I quickly realized it wasn't the correct way
>> to solve the problem.
>>
>> Unfortunately, shortly after realizing this, I left that project. So I
>> never had the chance to develop a proper solution. However, the concept
>> of a '.gitonly' file was exactly was I was seeking. So, for what it's
>> worth, I've definitely had at least one legit usecase for this feature.
>>
>> The usecases tend to center around tracking select files within the
>> rootfs of a full-blown operating system. Or a subset thereof.
>
> I think what Junio meant is to ignore everything by default, like:
>
>   echo '*' >.gitignore
>
> and then selectively use negative patterns (and being in .gitignore,
> that makes them positive "yes, include this") to add things back:
>
>   echo 'foo' >>.gitignore
>
> which ends up being roughly the same as your .gitonly concept.
>
> I don't offhand remember if you might run into problems where a
> subdirectory is ignored by the "*" and we do not even recurse into it. I
> think it would work OK as long as you put everything in the top-level
> gitignore, like:
>
>   echo 'subdir/file' >>.gitignore
>
> but I didn't test.

This doesn't work, as explained to myself in this commit in a private
project I have where I tried this a while ago:

I thought this was a bug:

(
rm -rf /tmp/git &&
git init /tmp/git &&
cd /tmp/git >/dev/null &&
echo '*' >.gitignore &&
echo '!*.txt' >>.gitignore &&
echo '!.gitignore' >>.gitignore &&
touch foo.png foo.txt &&
mkdir dir &&
touch dir/bar.png dir/bar.txt &&
git add *.txt &&
git add */*.txt;
git status --short
)

But it's a limitation, gitignore(5) says:

It is not possible to re-include a file if a parent directory of
that file is excluded. Git doesn’t list excluded directories for
    performance reasons, so any patterns on contained files have no
effect, no matter where they are defined.

So as a hack exclude anything that looks like a file with an
extension.

1 file changed, 1 insertion(+), 1 deletion(-)
.gitignore | 2 +-

modified   .gitignore
@@ -1,3 +1,3 @@
-*
+*.*
 !*.gpg
 !.gitignore

I.e. here I'm trying to maintain a repository where I only want
.gitignore and *.gpg files committed and everything else ignored, but it
only works for one directory level.

There's not a lot of room left in the gitignore syntax, but I suppose we
could extend it to add some "I really mean it" negative pattern which
would override previous patterns even if those previous patterns matched
directories.

Just fixing it as a bug would make the ignore process slower, since we
could no longer just ignore directories and would always need to
recursively scan them.

Re: the opposite of .gitignore, whitelist

[Jeff King]

On Fri, Oct 26, 2018 at 01:31:51PM +0200, Mischa POSLAWSKY wrote:

> Jeff King wrote 2018-10-26 5:36 (-0400):
> > I think what Junio meant is to ignore everything by default, like:
> > 
> >   echo '*' >.gitignore
> > 
> > and then selectively use negative patterns (and being in .gitignore,
> > that makes them positive "yes, include this") to add things back:
> > 
> >   echo 'foo' >>.gitignore
> > 
> > which ends up being roughly the same as your .gitonly concept.
> 
> To clarify, Peff meant to say echo '!foo' to whitelist. See git help ignore.

Oops, yes, thank you.

-Peff

Re: the opposite of .gitignore, whitelist

[Mischa POSLAWSKY]

Jeff King wrote 2018-10-26 5:36 (-0400):
> I think what Junio meant is to ignore everything by default, like:
> 
>   echo '*' >.gitignore
> 
> and then selectively use negative patterns (and being in .gitignore,
> that makes them positive "yes, include this") to add things back:
> 
>   echo 'foo' >>.gitignore
> 
> which ends up being roughly the same as your .gitonly concept.

To clarify, Peff meant to say echo '!foo' to whitelist. See git help ignore.

-- 
Mischa

Re: the opposite of .gitignore, whitelist

[Jeff King]

On Thu, Oct 25, 2018 at 10:38:46AM -0400, Jason Cooper wrote:

> On 10/25/18 1:37 AM, Junio C Hamano wrote:
> > "lhf...@163.com"  writes:
> >
> >> I have a good idea, add a file to git that is the opposite of 
> >> .gitignore...,
> > Do negative patterns in .gitignore file help without inventing
> > anything new?
> I did this several years ago in an attempt to track /etc/ (minus
> ownership, of course) without storing secrets in the git history.  As
> the system grew and was maintained (read: crap added), the negative
> patterns grew untenable.  I quickly realized it wasn't the correct way
> to solve the problem.
> 
> Unfortunately, shortly after realizing this, I left that project.  So I
> never had the chance to develop a proper solution.  However, the concept
> of a '.gitonly' file was exactly was I was seeking.  So, for what it's
> worth, I've definitely had at least one legit usecase for this feature.
> 
> The usecases tend to center around tracking select files within the
> rootfs of a full-blown operating system.  Or a subset thereof.

I think what Junio meant is to ignore everything by default, like:

  echo '*' >.gitignore

and then selectively use negative patterns (and being in .gitignore,
that makes them positive "yes, include this") to add things back:

  echo 'foo' >>.gitignore

which ends up being roughly the same as your .gitonly concept.

I don't offhand remember if you might run into problems where a
subdirectory is ignored by the "*" and we do not even recurse into it. I
think it would work OK as long as you put everything in the top-level
gitignore, like:

  echo 'subdir/file' >>.gitignore

but I didn't test.

-Peff

Re: the opposite of .gitignore, whitelist

[Jason Cooper]

Hi all,


On 10/25/18 1:37 AM, Junio C Hamano wrote:
> "lhf...@163.com"  writes:
>
>> I have a good idea, add a file to git that is the opposite of .gitignore...,
> Do negative patterns in .gitignore file help without inventing
> anything new?
I did this several years ago in an attempt to track /etc/ (minus
ownership, of course) without storing secrets in the git history.  As
the system grew and was maintained (read: crap added), the negative
patterns grew untenable.  I quickly realized it wasn't the correct way
to solve the problem.

Unfortunately, shortly after realizing this, I left that project.  So I
never had the chance to develop a proper solution.  However, the concept
of a '.gitonly' file was exactly was I was seeking.  So, for what it's
worth, I've definitely had at least one legit usecase for this feature.

The usecases tend to center around tracking select files within the
rootfs of a full-blown operating system.  Or a subset thereof.

hth,

Jason.

Re: the opposite of .gitignore, whitelist

[Junio C Hamano]

"lhf...@163.com"  writes:

> I have a good idea, add a file to git that is the opposite of .gitignore...,

Do negative patterns in .gitignore file help without inventing
anything new?

the opposite of .gitignore, whitelist

[lhf...@163.com]

I have a good idea, add a file to git that is the opposite of .gitignore, 
whitelist, the code in the development directory can be submitted to git 
version control, you can only submit the source code in the src directory, 
without concern for development tools and operations.System and other files, 
after all, the type of each project code is fixed, I am in the community I did 
not find a way to submit requirements in the community, so
--
lhf...@163.com

Re: [BUG] gitignore documentation inconsistent with actual behaviour

[dana]

On 20 Oct 2018, at 01:03, Duy Nguyen  wrote:
>foo**bar would match foobar as well as foo/bar, foo/x/bar and
>foo/x/y/bar... Its behavior is error prone in my opinion. There's also
>some concerns in early iterations of this "**" support that we would
>need to revisit if we want 'rsync' behavior. I'm not very excited
>about doing that.

That's fair.

I guess another point in favour of your second option is that it's essentially
the same behaviour used by bash (with the `globstar` option) and zsh (with the
default options); they also give `**` special recursion powers only when used in
a path component by itself, otherwise it acts like `*`. So there's precedent
there.

dana

Re: [BUG] gitignore documentation inconsistent with actual behaviour

[Duy Nguyen]

On Sat, Oct 20, 2018 at 7:53 AM dana  wrote:
>
> On 20 Oct 2018, at 00:26, Duy Nguyen  wrote:
> >Which way should we go? I'm leaning towards the second one...
>
> Not sure how much my opinion is worth, but the second option does feel more
> friendly (from a usage perspective) as well as more straight-forward (from a
> re-implementation perspective).

Yeah. And not having to describe all the corner cases is a plus. Too
many corner cases are a sign of bad implementation anyway. I'll wait
some more time for the others to speak up before I cook a proper
patch.

> There's a third option too, though, right? The 'rsync' behaviour mentioned
> earlier? It wouldn't matter either way in any of the examples i listed, but is
> there ever a conceivable use case for something like `foo**bar`, where the 
> `**`
> matches across slashes? (I can't think of any reason i'd need that personally,
> but then again i don't understand why these people are using `**` the way they
> are in the first place.)

foo**bar would match foobar as well as foo/bar, foo/x/bar and
foo/x/y/bar... Its behavior is error prone in my opinion. There's also
some concerns in early iterations of this "**" support that we would
need to revisit if we want 'rsync' behavior. I'm not very excited
about doing that.
-- 
Duy

Re: [BUG] gitignore documentation inconsistent with actual behaviour

[dana]

On 20 Oct 2018, at 00:26, Duy Nguyen  wrote:
>Which way should we go? I'm leaning towards the second one...

Not sure how much my opinion is worth, but the second option does feel more
friendly (from a usage perspective) as well as more straight-forward (from a
re-implementation perspective).

There's a third option too, though, right? The 'rsync' behaviour mentioned
earlier? It wouldn't matter either way in any of the examples i listed, but is
there ever a conceivable use case for something like `foo**bar`, where the `**`
matches across slashes? (I can't think of any reason i'd need that personally,
but then again i don't understand why these people are using `**` the way they
are in the first place.)

dana

Re: [BUG] gitignore documentation inconsistent with actual behaviour

[Duy Nguyen]

On Thu, Oct 11, 2018 at 05:19:06AM -0500, dana wrote:
> Hello,
> 
> I'm a contributor to ripgrep, which is a grep-like tool that supports using
> gitignore files to control which files are searched in a repo (or any other
> directory tree). ripgrep's support for the patterns in these files is based on
> git's official documentation, as seen here:
> 
>   https://git-scm.com/docs/gitignore
> 
> One of the most common reports on the ripgrep bug tracker is that it does not
> allow patterns like the following real-world examples, where a ** is used 
> along
> with other text within the same path component:
> 
>   **/**$$*.java
>   **.orig
>   **local.properties
>   !**.sha1
> 
> The reason it doesn't allow them is that the gitignore documentation 
> explicitly
> states that they're invalid:
>
> ...

I've checked the code and run some tests. There is a twist here. "**"
is only special when matched in "pathname" mode. That is when the
pattern contains at least one slash. In your patterns above, that only
applies to the first pattern.

When '**' is special, if it's neither '**/', '/**/' or '/**', it _is_
considered invalid (i.e. bad pattern) and the pattern will not match
anything.

The confusion comes from when '**' is not special for the remaining
three patterns, it's considered as regular '*' and still matches
stuff.

So, I think we have two options. The document could be clarified with
something like this

-- 8< --
diff --git a/Documentation/gitignore.txt b/Documentation/gitignore.txt
index d107daaffd..500cd43939 100644
--- a/Documentation/gitignore.txt
+++ b/Documentation/gitignore.txt
@@ -100,7 +100,8 @@ PATTERN FORMAT
a shell glob pattern and checks for a match against the
pathname relative to the location of the `.gitignore` file
(relative to the toplevel of the work tree if not from a
-   `.gitignore` file).
+   `.gitignore` file). Note that the "two consecutive asterisks" rule
+   below does not apply.
 
  - Otherwise, Git treats the pattern as a shell glob: "`*`" matches
anything except "`/`", "`?`" matches any one character except "`/`"
@@ -129,7 +130,8 @@ full pathname may have special meaning:
matches zero or more directories. For example, "`a/**/b`"
matches "`a/b`", "`a/x/b`", "`a/x/y/b`" and so on.
 
- - Other consecutive asterisks are considered invalid.
+ - Other consecutive asterisks are considered invalid and the pattern
+   is ignored.
 
 NOTES
 -
-- 8< --

Or we could make the behavior consistent. If '**' is invalid, just
consider it two separate regular '*'. Then all four of your patterns
will behave the same way. The change for that is quite simple

-- 8< --
diff --git a/wildmatch.c b/wildmatch.c
index d074c1be10..64087bf02c 100644
--- a/wildmatch.c
+++ b/wildmatch.c
@@ -104,8 +104,10 @@ static int dowild(const uchar *p, const uchar *text, 
unsigned int flags)
dowild(p + 1, text, flags) == 
WM_MATCH)
return WM_MATCH;
match_slash = 1;
-   } else
-   return WM_ABORT_MALFORMED;
+   } else {
+   /* without WM_PATHNAME, '*' == '**' */
+   match_slash = flags & WM_PATHNAME ? 0 : 
1;
+   }
} else
/* without WM_PATHNAME, '*' == '**' */
match_slash = flags & WM_PATHNAME ? 0 : 1;
-- 8< --

Which way should we go? I'm leaning towards the second one...
--
Duy

Re: [BUG] gitignore documentation inconsistent with actual behaviour

[Duy Nguyen]

On Mon, Oct 15, 2018 at 12:57 AM Junio C Hamano  wrote:
>
> Duy Nguyen  writes:
>
> >> Our matching function comes from rsync originally, whose manpage says:
> >>
> >> use ’**’ to match anything, including slashes.
> >>
> >> I believe this is accurate as far as the implementation goes.
> >
> > No. "**" semantics is not the same as from rsync. The three cases
> > "**/", "/**/" and "/**" were requested by Junio if I remember
> > correctly. You can search the mail archive for more information.
>
> Perhaps spelling the rules out would be more benefitial for the
> purpose of this thread?  I do not recall what I requested, but let

OK I gave you too much credit. You pointed out semantics problem with
rsync "**" [1] and gently pushed me to implement a safer subset ;-)

[1] https://public-inbox.org/git/7vbogj5sji@alter.siamese.dyndns.org/

> me throw out my guesses (i.e. what I would have wished if I were
> making a request to implement something) to keep the thread alive,
> you can correct me, and people can take it from there to update the
> docs ;-)
>
> A double-asterisk, both of whose ends are adjacent to a
> directory boundary (i.e. the beginning of the pattern, the end
> of the pattern or a slash) macthes 0 or more levels of
> directories.  e.g. **/a/b would match a/b, x/a/b, x/y/a/b, but
> not z-a/b.  a/**/b would match a/b, a/x/b, but not a/z-b or
> a-z-b.
>
> What a double-asterisk that does not sit on a directory boundary,
> e.g. "a**b", "a**/b", "a/**b", or "**a/b", matches, as far as I am
> concerned, is undefined, meaning that (1) I do not care all that
> much what the code actually do when a pattern like that is given as
> long as it does not segfault, and (2) I do not think I would mind
> changing the behaviour as a "bugfix", if their current behaviour
> does not make sense and we can come up with a saner alternative.

I think the document describes more or less what you wrote above in
the indented paragraph (but maybe less clear, patches are of course
welcome). It's the last paragraph that is problematic. It right now
says "invalid" which could be interpreted as "bad pattern, rejected"
but we in fact accept these "*" are regular ones.

There are not many alternatives we could do though. Erroring out could
really flood the stderr because we match these patterns zillions of
time and traditionally fnmatch gracefully accepts bad patterns, trying
to make the best of of them. So keeping undefined "**" as two "*"
sounds good enough.

But of course I'm open for saner alternatives if people find any.
-- 
Duy

Re: [BUG] gitignore documentation inconsistent with actual behaviour

[Junio C Hamano]

Duy Nguyen  writes:

>> Our matching function comes from rsync originally, whose manpage says:
>>
>> use ’**’ to match anything, including slashes.
>>
>> I believe this is accurate as far as the implementation goes.
>
> No. "**" semantics is not the same as from rsync. The three cases
> "**/", "/**/" and "/**" were requested by Junio if I remember
> correctly. You can search the mail archive for more information.

Perhaps spelling the rules out would be more benefitial for the
purpose of this thread?  I do not recall what I requested, but let
me throw out my guesses (i.e. what I would have wished if I were
making a request to implement something) to keep the thread alive,
you can correct me, and people can take it from there to update the
docs ;-)

A double-asterisk, both of whose ends are adjacent to a
directory boundary (i.e. the beginning of the pattern, the end
of the pattern or a slash) macthes 0 or more levels of
directories.  e.g. **/a/b would match a/b, x/a/b, x/y/a/b, but
not z-a/b.  a/**/b would match a/b, a/x/b, but not a/z-b or
a-z-b.

What a double-asterisk that does not sit on a directory boundary,
e.g. "a**b", "a**/b", "a/**b", or "**a/b", matches, as far as I am
concerned, is undefined, meaning that (1) I do not care all that
much what the code actually do when a pattern like that is given as
long as it does not segfault, and (2) I do not think I would mind
changing the behaviour as a "bugfix", if their current behaviour
does not make sense and we can come up with a saner alternative.

But the documentation probably should describe what these currently
match as the starting point.

Thanks.

Re: [BUG] gitignore documentation inconsistent with actual behaviour

[Duy Nguyen]

On Thu, Oct 11, 2018 at 2:41 PM Ævar Arnfjörð Bjarmason
 wrote:
>
>
> On Thu, Oct 11 2018, dana wrote:
>
> > Hello,
> >
> > I'm a contributor to ripgrep, which is a grep-like tool that supports using
> > gitignore files to control which files are searched in a repo (or any other
> > directory tree). ripgrep's support for the patterns in these files is based 
> > on
> > git's official documentation, as seen here:
> >
> >   https://git-scm.com/docs/gitignore
> >
> > One of the most common reports on the ripgrep bug tracker is that it does 
> > not
> > allow patterns like the following real-world examples, where a ** is used 
> > along
> > with other text within the same path component:
> >
> >   **/**$$*.java
> >   **.orig
> >   **local.properties
> >   !**.sha1
> >
> > The reason it doesn't allow them is that the gitignore documentation 
> > explicitly
> > states that they're invalid:
> >
> >   A leading "**" followed by a slash means match in all directories...
> >
> >   A trailing "/**" matches everything inside...
> >
> >   A slash followed by two consecutive asterisks then a slash matches zero or
> >   more directories...
> >
> >   Other consecutive asterisks are considered invalid.

Perhaps "undefined" is a better word than "invalid".

> > git itself happily accepts these patterns, however, apparently treating the 
> > **
> > like a single * without fnmatch(3)'s FNM_PATHNAME flag set (in other words, 
> > it
> > matches / as a regular character, thus crossing directory boundaries).
> >
> > ripgrep's developer is loathe to reverse-engineer this undocumented 
> > behaviour,
> > and so the reports keep coming, both for ripgrep itself and for down-stream
> > consumers of it and its ignore crate (including most notably Microsoft's VS 
> > Code
> > editor).
> >
> > My request: Assuming that git's actual handling of these patterns is 
> > intended,
> > would it be possible to make it 'official' and explicitly add it to the
> > documentation?

You mean "**" in the fourth case is interpreted as "*"? Yes I guess we
could rephrase it as "Other consecutive asterisks are considered
normal wildcard asterisks"

> Yeah those docs seem wrong. In general the docs for the matching
> function are quite bad. I have on my TODO list to factor this out into
> some gitwildmatch manpage, but right now the bit in gitignore is all we
> have.
>
> Our matching function comes from rsync originally, whose manpage says:
>
> use ’**’ to match anything, including slashes.
>
> I believe this is accurate as far as the implementation goes.

No. "**" semantics is not the same as from rsync. The three cases
"**/", "/**/" and "/**" were requested by Junio if I remember
correctly. You can search the mail archive for more information.
-- 
Duy

Re: [BUG] gitignore documentation inconsistent with actual behaviour

[dana]

On 11 Oct 2018, at 06:08, Ævar Arnfjörð Bjarmason  wrote:
>Our matching function comes from rsync originally, whose manpage says:
>
>   use ’**’ to match anything, including slashes.
>
>I believe this is accurate as far as the implementation goes. You can
>also see the rather exhaustive tests here:
>https://github.com/git/git/blob/master/t/t3070-wildmatch.sh

Thanks. The tests are a little hard to follow at first glance, so i guess i'd
have to study them more to see what the semantics actually are. When i tested it
briefly it didn't seem like it was behaving as you described, but maybe i wasn't
paying close enough attention. In any case, i'm sure these will be useful.

On 11 Oct 2018, at 06:08, Ævar Arnfjörð Bjarmason  wrote:
>The reason I'm on this tangent is to ask whether if such a thing
>existed, if it's something you can see something like ripgrep
>using. I.e. ask git given its .gitignore and .gitattributes what thing
>matches one of its pathspecs instead of carrying your own bug-compatible
>implementation.

My impression is that it probably isn't practical to use something like that as
ripgrep's main pattern-matching facility — for one thing there are performance
concerns, and for another it makes it dependent on git for some of its core
functionality (and it's not a git-specific tool). But i was going to say that
it'd be useful for testing, and when i e-mailed Andrew (the main ripgrep dev)
about this he said the same. I've CCed him on this message in case he has
anything to add.

dana

Re: [BUG] gitignore documentation inconsistent with actual behaviour

[Ævar Arnfjörð Bjarmason]

On Thu, Oct 11 2018, dana wrote:

> Hello,
>
> I'm a contributor to ripgrep, which is a grep-like tool that supports using
> gitignore files to control which files are searched in a repo (or any other
> directory tree). ripgrep's support for the patterns in these files is based on
> git's official documentation, as seen here:
>
>   https://git-scm.com/docs/gitignore
>
> One of the most common reports on the ripgrep bug tracker is that it does not
> allow patterns like the following real-world examples, where a ** is used 
> along
> with other text within the same path component:
>
>   **/**$$*.java
>   **.orig
>   **local.properties
>   !**.sha1
>
> The reason it doesn't allow them is that the gitignore documentation 
> explicitly
> states that they're invalid:
>
>   A leading "**" followed by a slash means match in all directories...
>
>   A trailing "/**" matches everything inside...
>
>   A slash followed by two consecutive asterisks then a slash matches zero or
>   more directories...
>
>   Other consecutive asterisks are considered invalid.
>
> git itself happily accepts these patterns, however, apparently treating the **
> like a single * without fnmatch(3)'s FNM_PATHNAME flag set (in other words, it
> matches / as a regular character, thus crossing directory boundaries).
>
> ripgrep's developer is loathe to reverse-engineer this undocumented behaviour,
> and so the reports keep coming, both for ripgrep itself and for down-stream
> consumers of it and its ignore crate (including most notably Microsoft's VS 
> Code
> editor).
>
> My request: Assuming that git's actual handling of these patterns is intended,
> would it be possible to make it 'official' and explicitly add it to the
> documentation?
>
> References (the first one is the main bug):
>
> https://github.com/BurntSushi/ripgrep/issues/373
> https://github.com/BurntSushi/ripgrep/issues/507
> https://github.com/BurntSushi/ripgrep/issues/859
> https://github.com/BurntSushi/ripgrep/issues/945
> https://github.com/BurntSushi/ripgrep/issues/1080
> https://github.com/BurntSushi/ripgrep/issues/1082
> https://github.com/Microsoft/vscode/issues/24050

Yeah those docs seem wrong. In general the docs for the matching
function are quite bad. I have on my TODO list to factor this out into
some gitwildmatch manpage, but right now the bit in gitignore is all we
have.

Our matching function comes from rsync originally, whose manpage says:

use ’**’ to match anything, including slashes.

I believe this is accurate as far as the implementation goes. You can
also see the rather exhaustive tests here:
https://github.com/git/git/blob/master/t/t3070-wildmatch.sh

Note the different behavior with e.g. --glob-pathspecs v.s. the
default. There's also stuff like:

$ grep diff=perl .gitattributes
*.perl eol=lf diff=perl
*.pl eof=lf diff=perl
*.pm eol=lf diff=perl
$ git ls-files ":(attr:diff=perl)" | wc -l
65

And then the exclude syntax. This is not in .gitignore:

$ git ls-files ":(exclude)*.pm" ":(attr:diff=perl)" | wc -l
41
$ git ls-files ":^*.pm" ":(attr:diff=perl)" | wc -l
41

I.e. we have wildmatch() on one hand and then the pathspec matching.

For an unrelated thing I have been thinking of adding a new plumbing
command who'd get input like this on stdin:

1 text t/t0202/test.pl\0\0
2 text perl/Git.pm\0\0
3 text *.pm\0\0
4 text :^*.pm"\0:(attr:diff=perl)\0\0
5 match glob,icase\04\03\0\0
6 match icase\04\02\0\0
7 match \04\01\0\0

Which would return (in any order):

1 OK
2 OK
3 OK
4 OK
5 NO
6 NO
7 YES

Or whatever. I.e. something where you can as a batch feed various
strings into a program in a batch, and then ask how some of them would
match against each other.

The reason for this is to extend something like git-multimail[1] with a
config where users can subscribe to changes to paths as declared by git
pathspecs, and be informed about which of the glob rules they defined
matched their config.

Right now you'd need to e.g. for a "git push" run each match rule for
each user with such config against each changed path in each commit
that's pushed, but plumbing like this would allow for feeding arbitrary
combinations of those in and ask what matches against what.

The reason I'm on this tangent is to ask whether if such a thing
existed, if it's something you can see something like ripgrep
using. I.e. ask git given its .gitignore and .gitattributes what thing
matches one of its pathspecs instead of carrying your own bug-compatible
implementation.

1. https://github.com/git-multimail/git-multimail/

Re: [BUG] gitignore documentation inconsistent with actual behaviour

[dana]

On 11 Oct 2018, at 05:19, dana  wrote:
>git itself happily accepts these patterns, however, apparently treating the **
>like a single * without fnmatch(3)'s FNM_PATHNAME flag set (in other words, it
>matches / as a regular character, thus crossing directory boundaries).

Sorry for replying to myself so quickly, but i think this bit is wrong. It seems
like it actually just treats ** in this sort of pattern like a regular * — but
i haven't studied the source very closely, so maybe that's not the full story
either, i'm not sure.

dana

[BUG] gitignore documentation inconsistent with actual behaviour

[dana]

Hello,

I'm a contributor to ripgrep, which is a grep-like tool that supports using
gitignore files to control which files are searched in a repo (or any other
directory tree). ripgrep's support for the patterns in these files is based on
git's official documentation, as seen here:

  https://git-scm.com/docs/gitignore

One of the most common reports on the ripgrep bug tracker is that it does not
allow patterns like the following real-world examples, where a ** is used along
with other text within the same path component:

  **/**$$*.java
  **.orig
  **local.properties
  !**.sha1

The reason it doesn't allow them is that the gitignore documentation explicitly
states that they're invalid:

  A leading "**" followed by a slash means match in all directories...

  A trailing "/**" matches everything inside...

  A slash followed by two consecutive asterisks then a slash matches zero or
  more directories...

  Other consecutive asterisks are considered invalid.

git itself happily accepts these patterns, however, apparently treating the **
like a single * without fnmatch(3)'s FNM_PATHNAME flag set (in other words, it
matches / as a regular character, thus crossing directory boundaries).

ripgrep's developer is loathe to reverse-engineer this undocumented behaviour,
and so the reports keep coming, both for ripgrep itself and for down-stream
consumers of it and its ignore crate (including most notably Microsoft's VS Code
editor).

My request: Assuming that git's actual handling of these patterns is intended,
would it be possible to make it 'official' and explicitly add it to the
documentation?

References (the first one is the main bug):

https://github.com/BurntSushi/ripgrep/issues/373
https://github.com/BurntSushi/ripgrep/issues/507
https://github.com/BurntSushi/ripgrep/issues/859
https://github.com/BurntSushi/ripgrep/issues/945
https://github.com/BurntSushi/ripgrep/issues/1080
https://github.com/BurntSushi/ripgrep/issues/1082
https://github.com/Microsoft/vscode/issues/24050

dana

Re: What's the use case for committing both the freshly created file and it's exclusion in .gitignore?

[brian m. carlson]

On Thu, Aug 09, 2018 at 01:04:43PM +0200, Bartosz Konikiewicz wrote:
> Steps to reproduce:
> 
> 1. Create a new file.
> 2. Stage the file.
> 3. Add the file to .gitignore.
> 4. Stage the .gitignore.
> 5. Commit changes.
> 
> I imagined that the file would now be removed from the stage (because
> it's ignored now and not yet committed) but it isn't. Where this
> behavior would be desirable? I know that a 'git add' command can be
> invoked with an '-f' flag, which would yield the same result, but I
> can't come up with an explanation where can it be applied.

Let me give you one.  If you use pristine-tar to check in the contents
of an upstream tarball, upstream may have included both a .gitignore
file and one or more ignored files in their tarball (say, something
autoconf generated).  Both of those files will be required in order to
reproduce the tarball, so git add -f or multiple stages of add will need
to be used.

If we unstaged the files from the index when the .gitignore was added,
this workflow wouldn't be possible.
-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204


signature.asc
Description: PGP signature

Re: What's the use case for committing both the freshly created file and it's exclusion in .gitignore?

[Bartosz Konikiewicz]

On 9 August 2018 at 21:58, Jeff King  wrote:
> If you are asking as a more general case: why do we not complain about
> .gitignore for files the index already knows about, then I think that is
> useful. It lets you override the .gitignore _once_ when adding the file
> initially, and then you don't have to deal with it again (and keep in
> mind that the pattern excluding it may be broad, like "*.o", or even
> just "*", so simply deleting it from the .gitignore is not an option).

This totally makes sense to me. Thanks for your explaination!

On 10 August 2018 at 03:12, Jonathan Nieder  wrote:
> "git help gitignore" has some notes about this.

Thanks! I wasn't aware of this. For some peculiar reason it didn't
strike me that the most appropriate place to look for
.gitignore-related stuff is - lo and behold - .gitignore manual page.
I was looking for clarification on 'git add' page, which I didn't find
informative enough in that aspect.

> If you have ideas
> about moments in interactive use where we could print some messages to
> make the behavior less surprising, that would be very welcome.

Sure I do! I have came up with two ideas which I believe that can be combined:

1.
>staged unstaged path
>   1:unchanged+1/-0 .gitignore !
>   2:+0/-0  nothing excluded.txt
>
> ! Not ignoring yet untracked files.
>
> *** Commands ***
>   1: status   2: update   3: revert   4: add untracked
>   5: patch6: diff 7: quit 8: help
> What now>

Here I propose to add an exclamation mark next to a .gitignore file
and reference it below. I am also thinking about marking files that
.gitignore affects. I think that any special character would be
appropriate. If I may only suggest, I wouldn't choose an asterisk (*)
because it's already used when a '2: update' option is chosen.

2.
>staged unstaged path
>   1:unchanged+1/-0 .gitignore
>   2:+0/-0  nothing excluded.txt
>
> *** Commands ***
>   1: status   2: update   3: revert   4: add untracked
>   5: patch    6: diff 7: quit 8: help
> What now> q
> You are about to add new files that are excluded in .gitignore. Confirm y/n?
> What now>y
> Bye.

Here I ask the user to confirm that he is about to stage yet untracked
files. Then I cheer him up with Git built-in courtesy.

Re: What's the use case for committing both the freshly created file and it's exclusion in .gitignore?

[Jonathan Nieder]

Hi,

Jeff King wrote:
> Bartosz Konikiewicz wrote:

>> Steps to reproduce:
>>
>> 1. Create a new file.
>> 2. Stage the file.
>> 3. Add the file to .gitignore.
>> 4. Stage the .gitignore.
>> 5. Commit changes.
[...]
> As far as I know, that is not an intentionally supported workflow. It is
> merely the result that .gitignore is only considered when adding new
> files to the index, not when committing nor when updating the entry for
> an existing file.

I am not sure I agree with "not intentionally supported".  It's a
little closer to "logical consequence of some intentionally features",
because:

> If you are asking as a more general case: why do we not complain about
> .gitignore for files the index already knows about, then I think that is
> useful. It lets you override the .gitignore _once_ when adding the file
> initially, and then you don't have to deal with it again (and keep in
> mind that the pattern excluding it may be broad, like "*.o", or even
> just "*", so simply deleting it from the .gitignore is not an option).

This workflow is very common.

> You could probably accomplish this these days by using a negative
> pattern in your .gitignore file. But I think the behavior in question
> may predate negative patterns (but I didn't dig). It's also a bit
> simpler to use in practice, IMHO.

Agreed about simpler, even though it's not part of any of my own
habits.

In retrospect, despite the precedent of cvsignore, calling the file
.gitignore may not have been a great idea.  Some other name that
conveys .git-prevent-me-from-accidentally-adding-these-files would
make the behavior less surprising to new users.

"git help gitignore" has some notes about this.  If you have ideas
about moments in interactive use where we could print some messages to
make the behavior less surprising, that would be very welcome.

Thanks,
Jonathan

Re: What's the use case for committing both the freshly created file and it's exclusion in .gitignore?

[Jeff King]

On Thu, Aug 09, 2018 at 01:04:43PM +0200, Bartosz Konikiewicz wrote:

> Hi there!
> 
> I hope that the subject of my message (i.e. the question) is
> exhaustive enough, so I'll just stick to reproducing my issue.
> 
> Steps to reproduce:
> 
> 1. Create a new file.
> 2. Stage the file.
> 3. Add the file to .gitignore.
> 4. Stage the .gitignore.
> 5. Commit changes.
> 
> I imagined that the file would now be removed from the stage (because
> it's ignored now and not yet committed) but it isn't. Where this
> behavior would be desirable? I know that a 'git add' command can be
> invoked with an '-f' flag, which would yield the same result, but I
> can't come up with an explanation where can it be applied.

As far as I know, that is not an intentionally supported workflow. It is
merely the result that .gitignore is only considered when adding new
files to the index, not when committing nor when updating the entry for
an existing file.

If you are asking as a more general case: why do we not complain about
.gitignore for files the index already knows about, then I think that is
useful. It lets you override the .gitignore _once_ when adding the file
initially, and then you don't have to deal with it again (and keep in
mind that the pattern excluding it may be broad, like "*.o", or even
just "*", so simply deleting it from the .gitignore is not an option).

You could probably accomplish this these days by using a negative
pattern in your .gitignore file. But I think the behavior in question
may predate negative patterns (but I didn't dig). It's also a bit
simpler to use in practice, IMHO.

-Peff

What's the use case for committing both the freshly created file and it's exclusion in .gitignore?

[Bartosz Konikiewicz]

Hi there!

I hope that the subject of my message (i.e. the question) is
exhaustive enough, so I'll just stick to reproducing my issue.

Steps to reproduce:

1. Create a new file.
2. Stage the file.
3. Add the file to .gitignore.
4. Stage the .gitignore.
5. Commit changes.

I imagined that the file would now be removed from the stage (because
it's ignored now and not yet committed) but it isn't. Where this
behavior would be desirable? I know that a 'git add' command can be
invoked with an '-f' flag, which would yield the same result, but I
can't come up with an explanation where can it be applied.

Re: git glob pattern in .gitignore and git command

[Duy Nguyen]

On Sun, Jun 3, 2018 at 2:58 AM, Yubin Ruan  wrote:
> To ignore all .js file under a directory `lib', I can use "lib/**/js" to match
> them. But when using git command such as "git add", using "git add lib/\*.js"
> is sufficient. Why is this difference in glob mode?

Historical reasons mostly. '**' comes later when '*' already
establishes its place. You can use '**' too with "git add
':(glob)lib/**/*.js'". See
https://git-scm.com/docs/gitglossary#gitglossary-aiddefpathspecapathspec
-- 
Duy

Re: git glob pattern in .gitignore and git command

[Philip Oakley]

Hi Yubun,

From: "Yubin Ruan" 

To ignore all .js file under a directory `lib', I can use "lib/**/js" to
match
them. But when using git command such as "git add", using "git add
lib/\*.js"
is sufficient. Why is this difference in glob mode?

I have heard that there are many different glob mode out there (e.g., bash
has
many different glob mode). So, which classes of glob mode does these two
belong to? Do they have a name?



Is this a question about `git add` being able to add a file that is marked
as being ignored in the .gitignore file? [Yes it can.]

Or, is this simply about the many different globbing capabilities of one's
shell, and of Git?

The double asterix (star) is specific/local to Git. It is described in the
various commands that use it, especially the gitignore man page `git help
ignore` or  https://git-scm.com/docs/gitignore.
"Two consecutive asterisks ("**") in patterns matched against full pathname
may have special meaning: ... "

The single asterix does have two modes depending on how you quote it. It is
described in the command line interface (cli) man page ` git help cli` or
https://git-scm.com/docs/gitcli.
"Many commands allow wildcards in paths, but you need to protect them from
getting globbed by the shell. These two mean different things: ... "

A common proper name for these asterix style characters is a "wildcards".
Try 'bash wildcards' or linux wildcards' in your favourite search engine.

--
Philip

git glob pattern in .gitignore and git command

[Yubin Ruan]

To ignore all .js file under a directory `lib', I can use "lib/**/js" to match
them. But when using git command such as "git add", using "git add lib/\*.js"
is sufficient. Why is this difference in glob mode?

I have heard that there are many different glob mode out there (e.g., bash has
many different glob mode). So, which classes of glob mode does these two
belong to? Do they have a name?

Yubin

Re: Missing ? wildcard character in gitignore documentation

[Duy Nguyen]

On Tue, Jan 30, 2018 at 6:07 PM, Ævar Arnfjörð Bjarmason
<ava...@gmail.com> wrote:
>
> On Tue, Jan 30 2018, Duy Nguyen jotted:
>
>> On Mon, Jan 29, 2018 at 10:47:10AM -0500, Randall S. Becker wrote:
>>> The implication of support for ? is there through the following paragraph 
>>> from the gitignore documentation:
>>>
>>> "Otherwise, Git treats the pattern as a shell glob suitable for
>>> consumption by fnmatch(3) with the FNM_PATHNAME flag: wildcards
>>> in the pattern will not match a / in the pathname. For example,
>>> "Documentation/*.html" matches "Documentation/git.html" but not
>>> "Documentation/ppc/ppc.html" or
>>> "tools/perf/Documentation/perf.html"."
>>>
>>> Of course you have to go read fnmatch(3), so it might be good for
>>> expand on this here :).
>>
>> I agree. How about something like this?
>>
>> -- 8< --
>> Subject: [PATCH] gitignore.txt: elaborate shell glob syntax
>>
>> `fnmatch(3)` is a great mention if the intended audience is
>> programmers. For normal users it's probably better to spell out what
>> a shell glob is.
>>
>> This paragraph is updated to roughly tell (or remind) what the main
>> wildcards are supposed to do. All the details are still hidden away
>> behind the `fnmatch(3)` wall because bringing the whole specification
>> here may be too much.
>>
>> Signed-off-by: Nguyễn Thái Ngọc Duy <pclo...@gmail.com>
>> ---
>>  Documentation/gitignore.txt | 11 +--
>>  1 file changed, 5 insertions(+), 6 deletions(-)
>>
>> diff --git a/Documentation/gitignore.txt b/Documentation/gitignore.txt
>> index 63260f0056..0f4b1360bd 100644
>> --- a/Documentation/gitignore.txt
>> +++ b/Documentation/gitignore.txt
>> @@ -102,12 +102,11 @@ PATTERN FORMAT
>> (relative to the toplevel of the work tree if not from a
>> `.gitignore` file).
>>
>> - - Otherwise, Git treats the pattern as a shell glob suitable
>> -   for consumption by fnmatch(3) with the FNM_PATHNAME flag:
>> -   wildcards in the pattern will not match a / in the pathname.
>> -   For example, "Documentation/{asterisk}.html" matches
>> -   "Documentation/git.html" but not "Documentation/ppc/ppc.html"
>> -   or "tools/perf/Documentation/perf.html".
>> + - Otherwise, Git treats the pattern as a shell glob: '{asterisk}'
>> +   matches anything except '/', '?' matches any one character except
>> +   '/' and '[]' matches one character in a selected range. See
>> +   fnmatch(3) and the FNM_PATHNAME flag for a more accurate
>> +   description.
>>
>>   - A leading slash matches the beginning of the pathname.
>> For example, "/{asterisk}.c" matches "cat-file.c" but not
>
> When reading the docs the other day I was thinking that we should
> entirely git rid of these references to fnmatch(3) and write a
> gitwildmatch man page.

That's even better :) I forgot that we don't use fnmatch anymore.

> One of the reasons for why fnmatch() was removed as a supported backend
> was because it couldn't be relied on as a backend, so it doesn't make
> sense to be referring to that OS-level documentation, wildmatch also has
> other features.



-- 
Duy

Re: Missing ? wildcard character in gitignore documentation

[Ævar Arnfjörð Bjarmason]

On Tue, Jan 30 2018, Duy Nguyen jotted:

> On Mon, Jan 29, 2018 at 10:47:10AM -0500, Randall S. Becker wrote:
>> The implication of support for ? is there through the following paragraph 
>> from the gitignore documentation:
>>
>> "Otherwise, Git treats the pattern as a shell glob suitable for
>> consumption by fnmatch(3) with the FNM_PATHNAME flag: wildcards
>> in the pattern will not match a / in the pathname. For example,
>> "Documentation/*.html" matches "Documentation/git.html" but not
>> "Documentation/ppc/ppc.html" or
>> "tools/perf/Documentation/perf.html"."
>>
>> Of course you have to go read fnmatch(3), so it might be good for
>> expand on this here :).
>
> I agree. How about something like this?
>
> -- 8< --
> Subject: [PATCH] gitignore.txt: elaborate shell glob syntax
>
> `fnmatch(3)` is a great mention if the intended audience is
> programmers. For normal users it's probably better to spell out what
> a shell glob is.
>
> This paragraph is updated to roughly tell (or remind) what the main
> wildcards are supposed to do. All the details are still hidden away
> behind the `fnmatch(3)` wall because bringing the whole specification
> here may be too much.
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclo...@gmail.com>
> ---
>  Documentation/gitignore.txt | 11 +--
>  1 file changed, 5 insertions(+), 6 deletions(-)
>
> diff --git a/Documentation/gitignore.txt b/Documentation/gitignore.txt
> index 63260f0056..0f4b1360bd 100644
> --- a/Documentation/gitignore.txt
> +++ b/Documentation/gitignore.txt
> @@ -102,12 +102,11 @@ PATTERN FORMAT
> (relative to the toplevel of the work tree if not from a
> `.gitignore` file).
>
> - - Otherwise, Git treats the pattern as a shell glob suitable
> -   for consumption by fnmatch(3) with the FNM_PATHNAME flag:
> -   wildcards in the pattern will not match a / in the pathname.
> -   For example, "Documentation/{asterisk}.html" matches
> -   "Documentation/git.html" but not "Documentation/ppc/ppc.html"
> -   or "tools/perf/Documentation/perf.html".
> + - Otherwise, Git treats the pattern as a shell glob: '{asterisk}'
> +   matches anything except '/', '?' matches any one character except
> +   '/' and '[]' matches one character in a selected range. See
> +   fnmatch(3) and the FNM_PATHNAME flag for a more accurate
> +   description.
>
>   - A leading slash matches the beginning of the pathname.
> For example, "/{asterisk}.c" matches "cat-file.c" but not

When reading the docs the other day I was thinking that we should
entirely git rid of these references to fnmatch(3) and write a
gitwildmatch man page.

One of the reasons for why fnmatch() was removed as a supported backend
was because it couldn't be relied on as a backend, so it doesn't make
sense to be referring to that OS-level documentation, wildmatch also has
other features.

Re: Missing ? wildcard character in gitignore documentation

[Duy Nguyen]

On Mon, Jan 29, 2018 at 10:47:10AM -0500, Randall S. Becker wrote:
> The implication of support for ? is there through the following paragraph 
> from the gitignore documentation:
> 
> "Otherwise, Git treats the pattern as a shell glob suitable for
> consumption by fnmatch(3) with the FNM_PATHNAME flag: wildcards
> in the pattern will not match a / in the pathname. For example,
> "Documentation/*.html" matches "Documentation/git.html" but not
> "Documentation/ppc/ppc.html" or
> "tools/perf/Documentation/perf.html"."
> 
> Of course you have to go read fnmatch(3), so it might be good for
> expand on this here :).

I agree. How about something like this?

-- 8< --
Subject: [PATCH] gitignore.txt: elaborate shell glob syntax

`fnmatch(3)` is a great mention if the intended audience is
programmers. For normal users it's probably better to spell out what
a shell glob is.

This paragraph is updated to roughly tell (or remind) what the main
wildcards are supposed to do. All the details are still hidden away
behind the `fnmatch(3)` wall because bringing the whole specification
here may be too much.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclo...@gmail.com>
---
 Documentation/gitignore.txt | 11 +--
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/Documentation/gitignore.txt b/Documentation/gitignore.txt
index 63260f0056..0f4b1360bd 100644
--- a/Documentation/gitignore.txt
+++ b/Documentation/gitignore.txt
@@ -102,12 +102,11 @@ PATTERN FORMAT
(relative to the toplevel of the work tree if not from a
`.gitignore` file).
 
- - Otherwise, Git treats the pattern as a shell glob suitable
-   for consumption by fnmatch(3) with the FNM_PATHNAME flag:
-   wildcards in the pattern will not match a / in the pathname.
-   For example, "Documentation/{asterisk}.html" matches
-   "Documentation/git.html" but not "Documentation/ppc/ppc.html"
-   or "tools/perf/Documentation/perf.html".
+ - Otherwise, Git treats the pattern as a shell glob: '{asterisk}'
+   matches anything except '/', '?' matches any one character except
+   '/' and '[]' matches one character in a selected range. See
+   fnmatch(3) and the FNM_PATHNAME flag for a more accurate
+   description.
 
  - A leading slash matches the beginning of the pathname.
For example, "/{asterisk}.c" matches "cat-file.c" but not
-- 
2.16.1.205.g271f633410

-- 8< --

Re: Missing ? wildcard character in gitignore documentation

[Jack F]

Ah. Yes it does. Apologies. Maybe a "See glob(7) for more pattern
matching options, including ! ? [] *"

Thank you very much.

Cheers.

From,
Jack

On 29/01/18 15:47, Randall S. Becker wrote:
> On January 29, 2018 6:30 AM, Jack F wrote:
>> I have just noticed that the documentation for gitignore is missing
>> documentation on using the ? to match any single character. I have included
>> a example below with git version 2.14.1.
>>
>> |11:05:09 j ~/Development/ls-ignore [master] $ git status On branch
>> master Your branch is up-to-date with 'origin/master'. nothing to commit,
>> working tree clean 11:05:11 j ~/Development/ls-ignore [master] $ cat
>> .gitignore *~ node_modules yarn* 11:05:21 j ~/Development/ls-ignore
>> [master] $ touch test.swo 11:05:31 j ~/Development/ls-ignore [master]?1 $
>> git status On branch master Your branch is up-to-date with 'origin/master'.
>> Untracked files: (use "git add ..." to include in what will be 
>> committed)
>> test.swo nothing added to commit but untracked files present (use "git add"
>> to track) 11:05:35 j ~/Development/ls-ignore [master]?1 $ echo "*.sw?" >>
>> .gitignore 11:05:40 j ~/Development/ls-ignore [master]≠1 $ cat .gitignore *~
>> node_modules
>> yarn* *.sw? 11:05:51 j ~/Development/ls-ignore [master]≠1 $ git status On
>> branch master Your branch is up-to-date with 'origin/master'. Changes not
>> staged for commit: (use "git add ..." to update what will be
>> committed) (use "git checkout -- ..." to discard changes in working
>> directory) modified: .gitignore no changes added to commit (use "git add"
>> and/or "git commit -a")|
>>
>>
>>
>> Noticed it when checking an npm package (ignore) that uses the
>> documentation (https://git-scm.com/docs/gitignore) to determine its
>> functionality. It is documented in https://git-scm.com/book/en/v2/Git-
>> Basics-Recording-Changes-to-the-Repository#Ignoring-Files
> The implication of support for ? is there through the following paragraph 
> from the gitignore documentation:
>
> "Otherwise, Git treats the pattern as a shell glob suitable for 
> consumption by fnmatch(3)
> with the FNM_PATHNAME flag: wildcards in the pattern will not match a / 
> in the
> pathname. For example, "Documentation/*.html" matches 
> "Documentation/git.html"
> but not "Documentation/ppc/ppc.html" or 
> "tools/perf/Documentation/perf.html"."
>
> Of course you have to go read fnmatch(3), so it might be good for expand on 
> this here :).
>
> Cheers,
> Randall
>
> -- Brief whoami:
>  NonStop developer since approximately 2112884442
>  UNIX developer since approximately 421664400
> -- In my real life, I talk too much.
>
>
>

-- 
https://bytes.nz
https://keybase.io/bytesnz

RE: Missing ? wildcard character in gitignore documentation

[Randall S. Becker]

On January 29, 2018 6:30 AM, Jack F wrote:
> I have just noticed that the documentation for gitignore is missing
> documentation on using the ? to match any single character. I have included
> a example below with git version 2.14.1.
> 
> |11:05:09 j ~/Development/ls-ignore [master] $ git status On branch
> master Your branch is up-to-date with 'origin/master'. nothing to commit,
> working tree clean 11:05:11 j ~/Development/ls-ignore [master] $ cat
> .gitignore *~ node_modules yarn* 11:05:21 j ~/Development/ls-ignore
> [master] $ touch test.swo 11:05:31 j ~/Development/ls-ignore [master]?1 $
> git status On branch master Your branch is up-to-date with 'origin/master'.
> Untracked files: (use "git add ..." to include in what will be 
> committed)
> test.swo nothing added to commit but untracked files present (use "git add"
> to track) 11:05:35 j ~/Development/ls-ignore [master]?1 $ echo "*.sw?" >>
> .gitignore 11:05:40 j ~/Development/ls-ignore [master]≠1 $ cat .gitignore *~
> node_modules
> yarn* *.sw? 11:05:51 j ~/Development/ls-ignore [master]≠1 $ git status On
> branch master Your branch is up-to-date with 'origin/master'. Changes not
> staged for commit: (use "git add ..." to update what will be
> committed) (use "git checkout -- ..." to discard changes in working
> directory) modified: .gitignore no changes added to commit (use "git add"
> and/or "git commit -a")|
> 
> 
> 
> Noticed it when checking an npm package (ignore) that uses the
> documentation (https://git-scm.com/docs/gitignore) to determine its
> functionality. It is documented in https://git-scm.com/book/en/v2/Git-
> Basics-Recording-Changes-to-the-Repository#Ignoring-Files

The implication of support for ? is there through the following paragraph from 
the gitignore documentation:

"Otherwise, Git treats the pattern as a shell glob suitable for consumption 
by fnmatch(3)
with the FNM_PATHNAME flag: wildcards in the pattern will not match a / in 
the
pathname. For example, "Documentation/*.html" matches 
"Documentation/git.html"
but not "Documentation/ppc/ppc.html" or 
"tools/perf/Documentation/perf.html"."

Of course you have to go read fnmatch(3), so it might be good for expand on 
this here :).

Cheers,
Randall

-- Brief whoami:
 NonStop developer since approximately 2112884442
 UNIX developer since approximately 421664400
-- In my real life, I talk too much.

Missing ? wildcard character in gitignore documentation

[Jack F]

Hello,

I have just noticed that the documentation for gitignore is missing
documentation on using the ? to match any single character. I have
included a example below with git version 2.14.1.

|11:05:09 j ~/Development/ls-ignore [master] $ git status On branch
master Your branch is up-to-date with 'origin/master'. nothing to
commit, working tree clean 11:05:11 j ~/Development/ls-ignore [master] $
cat .gitignore *~ node_modules yarn* 11:05:21 j ~/Development/ls-ignore
[master] $ touch test.swo 11:05:31 j ~/Development/ls-ignore [master]?1
$ git status On branch master Your branch is up-to-date with
'origin/master'. Untracked files: (use "git add ..." to include in
what will be committed) test.swo nothing added to commit but untracked
files present (use "git add" to track) 11:05:35 j
~/Development/ls-ignore [master]?1 $ echo "*.sw?" >> .gitignore 11:05:40
j ~/Development/ls-ignore [master]≠1 $ cat .gitignore *~ node_modules
yarn* *.sw? 11:05:51 j ~/Development/ls-ignore [master]≠1 $ git status
On branch master Your branch is up-to-date with 'origin/master'. Changes
not staged for commit: (use "git add ..." to update what will be
committed) (use "git checkout -- ..." to discard changes in
working directory) modified: .gitignore no changes added to commit (use
"git add" and/or "git commit -a")|



Noticed it when checking an npm package (ignore) that uses the
documentation (https://git-scm.com/docs/gitignore) to determine its
functionality. It is documented in
https://git-scm.com/book/en/v2/Git-Basics-Recording-Changes-to-the-Repository#Ignoring-Files

Cheers.

From,
Jack

https://bytes.nz
https://keybase.io/bytesnz

Re: [PATCH 2/2] travis-ci: check that all build artifacts are .gitignore-d

[Lars Schneider]

> On 03 Jan 2018, at 00:12, SZEDER Gábor <szeder@gmail.com> wrote:
> 
> On Tue, Jan 2, 2018 at 8:40 PM, Lars Schneider <larsxschnei...@gmail.com> 
> wrote:
>> 
>>> On 31 Dec 2017, at 17:02, SZEDER Gábor <szeder@gmail.com> wrote:
>>> 
>>> Every once in a while our explicit .gitignore files get out of sync
>>> when our build process learns to create new artifacts, like test
>>> helper executables, but the .gitignore files are not updated
>>> accordingly.
>>> 
>>> Use Travis CI to help catch such issues earlier: check that there are
>>> no untracked files at the end of any build jobs building Git (i.e. the
>>> 64 bit Clang and GCC Linux and OSX build jobs, plus the GETTEXT_POISON
>>> and 32 bit Linux build jobs) or its documentation, and fail the build
>>> job if there are any present.
>>> 
>>> Signed-off-by: SZEDER Gábor <szeder@gmail.com>
>>> ---
>>> ci/lib-travisci.sh   | 10 ++
>>> ci/run-linux32-docker.sh |  2 ++
>>> ci/run-tests.sh  |  2 ++
>>> ci/test-documentation.sh |  6 ++
>>> 4 files changed, 20 insertions(+)
>>> 
>>> diff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh
>>> index 1543b7959..07f27c727 100755
>>> --- a/ci/lib-travisci.sh
>>> +++ b/ci/lib-travisci.sh
>>> @@ -67,6 +67,16 @@ skip_good_tree () {
>>>  exit 0
>>> }
>>> 
>>> +check_unignored_build_artifacts ()
>>> +{
>>> + ! git ls-files --other --exclude-standard --error-unmatch \
>>> + -- ':/*' 2>/dev/null ||
>> 
>> What does "-- ':/*'" do?
> 
> It makes the whole thing work :)
> ':/*' is a pattern matching all paths, and '--others --exclude-standard'
> limit the command to list files that are both untracked and unignored.
> '--error-unmatch' causes the command to error out if any of the files
> given on the command line doesn't match anything in the working tree,
> in this case if there are no untracked-unignored files.  Without a path
> given on the cmdline '--error-unmatch' has basically no effect[1].

That was the missing piece. Thank you! I've used the exact command before 
but I've never looked at the exit code. I just assumed that it would
behave like grep if there is no match (as you described in [1])


> In a clean worktree:
> 
>  $ git ls-files --other --exclude-standard --error-unmatch ; echo $?
>  0
>  $ git ls-files --other --exclude-standard --error-unmatch ':/*' ; echo $?
>  error: pathspec ':/*' did not match any file(s) known to git.
>  Did you forget to 'git add'?
>  1
> 
> The disambiguating double-dash is not really necessary, because the :/*
> pattern can't be confused with any --options, but doesn't hurt, either.
> 
>> Plus, why do you redirect stderr?
> 
> To prevent the above error message from appearing in the trace log of a
> successful build.

Makes sense. As I never specified the path, I did never see the error :-)


> [1] - Which makes me think whether we should consider '--error-unmatch'
>  without any paths given on the command line as an error.

Agreed!


- Lars

Re: [PATCH 2/2] travis-ci: check that all build artifacts are .gitignore-d

[SZEDER Gábor]

On Tue, Jan 2, 2018 at 8:40 PM, Lars Schneider <larsxschnei...@gmail.com> wrote:
>
>> On 31 Dec 2017, at 17:02, SZEDER Gábor <szeder@gmail.com> wrote:
>>
>> Every once in a while our explicit .gitignore files get out of sync
>> when our build process learns to create new artifacts, like test
>> helper executables, but the .gitignore files are not updated
>> accordingly.
>>
>> Use Travis CI to help catch such issues earlier: check that there are
>> no untracked files at the end of any build jobs building Git (i.e. the
>> 64 bit Clang and GCC Linux and OSX build jobs, plus the GETTEXT_POISON
>> and 32 bit Linux build jobs) or its documentation, and fail the build
>> job if there are any present.
>>
>> Signed-off-by: SZEDER Gábor <szeder@gmail.com>
>> ---
>> ci/lib-travisci.sh   | 10 ++
>> ci/run-linux32-docker.sh |  2 ++
>> ci/run-tests.sh  |  2 ++
>> ci/test-documentation.sh |  6 ++
>> 4 files changed, 20 insertions(+)
>>
>> diff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh
>> index 1543b7959..07f27c727 100755
>> --- a/ci/lib-travisci.sh
>> +++ b/ci/lib-travisci.sh
>> @@ -67,6 +67,16 @@ skip_good_tree () {
>>   exit 0
>> }
>>
>> +check_unignored_build_artifacts ()
>> +{
>> + ! git ls-files --other --exclude-standard --error-unmatch \
>> + -- ':/*' 2>/dev/null ||
>
> What does "-- ':/*'" do?

It makes the whole thing work :)
':/*' is a pattern matching all paths, and '--others --exclude-standard'
limit the command to list files that are both untracked and unignored.
'--error-unmatch' causes the command to error out if any of the files
given on the command line doesn't match anything in the working tree,
in this case if there are no untracked-unignored files.  Without a path
given on the cmdline '--error-unmatch' has basically no effect[1].

In a clean worktree:

  $ git ls-files --other --exclude-standard --error-unmatch ; echo $?
  0
  $ git ls-files --other --exclude-standard --error-unmatch ':/*' ; echo $?
  error: pathspec ':/*' did not match any file(s) known to git.
  Did you forget to 'git add'?
  1

The disambiguating double-dash is not really necessary, because the :/*
pattern can't be confused with any --options, but doesn't hurt, either.

> Plus, why do you redirect stderr?

To prevent the above error message from appearing in the trace log of a
successful build.


[1] - Which makes me think whether we should consider '--error-unmatch'
  without any paths given on the command line as an error.
  On a related note: that error message doesn't seem to make much
  sense with '--other'...

Re: [PATCH 2/2] travis-ci: check that all build artifacts are .gitignore-d

[Lars Schneider]

> On 31 Dec 2017, at 17:02, SZEDER Gábor <szeder@gmail.com> wrote:
> 
> Every once in a while our explicit .gitignore files get out of sync
> when our build process learns to create new artifacts, like test
> helper executables, but the .gitignore files are not updated
> accordingly.
> 
> Use Travis CI to help catch such issues earlier: check that there are
> no untracked files at the end of any build jobs building Git (i.e. the
> 64 bit Clang and GCC Linux and OSX build jobs, plus the GETTEXT_POISON
> and 32 bit Linux build jobs) or its documentation, and fail the build
> job if there are any present.
> 
> Signed-off-by: SZEDER Gábor <szeder@gmail.com>
> ---
> ci/lib-travisci.sh   | 10 ++
> ci/run-linux32-docker.sh |  2 ++
> ci/run-tests.sh  |  2 ++
> ci/test-documentation.sh |  6 ++
> 4 files changed, 20 insertions(+)
> 
> diff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh
> index 1543b7959..07f27c727 100755
> --- a/ci/lib-travisci.sh
> +++ b/ci/lib-travisci.sh
> @@ -67,6 +67,16 @@ skip_good_tree () {
>   exit 0
> }
> 
> +check_unignored_build_artifacts ()
> +{
> + ! git ls-files --other --exclude-standard --error-unmatch \
> + -- ':/*' 2>/dev/null ||

What does "-- ':/*'" do? Plus, why do you redirect stderr?

--

Both patches look good to me!

Thanks,
Lars

[PATCH 2/2] travis-ci: check that all build artifacts are .gitignore-d

[SZEDER Gábor]

Every once in a while our explicit .gitignore files get out of sync
when our build process learns to create new artifacts, like test
helper executables, but the .gitignore files are not updated
accordingly.

Use Travis CI to help catch such issues earlier: check that there are
no untracked files at the end of any build jobs building Git (i.e. the
64 bit Clang and GCC Linux and OSX build jobs, plus the GETTEXT_POISON
and 32 bit Linux build jobs) or its documentation, and fail the build
job if there are any present.

Signed-off-by: SZEDER Gábor <szeder@gmail.com>
---
 ci/lib-travisci.sh   | 10 ++
 ci/run-linux32-docker.sh |  2 ++
 ci/run-tests.sh  |  2 ++
 ci/test-documentation.sh |  6 ++
 4 files changed, 20 insertions(+)

diff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh
index 1543b7959..07f27c727 100755
--- a/ci/lib-travisci.sh
+++ b/ci/lib-travisci.sh
@@ -67,6 +67,16 @@ skip_good_tree () {
exit 0
 }
 
+check_unignored_build_artifacts ()
+{
+   ! git ls-files --other --exclude-standard --error-unmatch \
+   -- ':/*' 2>/dev/null ||
+   {
+   echo "$(tput setaf 1)error: found unignored build 
artifacts$(tput sgr0)"
+   false
+   }
+}
+
 # Set 'exit on error' for all CI scripts to let the caller know that
 # something went wrong.
 # Set tracing executed commands, primarily setting environment variables
diff --git a/ci/run-linux32-docker.sh b/ci/run-linux32-docker.sh
index 870a41246..4f191c5bb 100755
--- a/ci/run-linux32-docker.sh
+++ b/ci/run-linux32-docker.sh
@@ -23,4 +23,6 @@ docker run \
daald/ubuntu32:xenial \
/usr/src/git/ci/run-linux32-build.sh $(id -u $USER)
 
+check_unignored_build_artifacts
+
 save_good_tree
diff --git a/ci/run-tests.sh b/ci/run-tests.sh
index eb5ba4058..22355f009 100755
--- a/ci/run-tests.sh
+++ b/ci/run-tests.sh
@@ -8,4 +8,6 @@
 ln -s $HOME/travis-cache/.prove t/.prove
 make --quiet test
 
+check_unignored_build_artifacts
+
 save_good_tree
diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
index 3d62e6c95..a20de9ca1 100755
--- a/ci/test-documentation.sh
+++ b/ci/test-documentation.sh
@@ -18,6 +18,9 @@ test -s Documentation/git.xml
 test -s Documentation/git.1
 grep '

Re: [RFC PATCH] t/helper: Move sources to t/helper-src; gitignore any files in t/helper

[Junio C Hamano]

Stefan Beller <sbel...@google.com> writes:

> Compiled test helpers in t/helper are out of sync with the .gitignore
> files quite frequently. This can happen when new test helpers are added,
> but the explicit .gitignore file is not updated in the same commit, or
> when you forget to 'make clean' before checking out a different version
> of git, as the different version may have a different explicit list of
> test helpers to ignore.
>
> This can be fixed by using overly broad ignore patterns, such as ignoring
> the whole directory via '//t/helper/*' in .gitignore.

If we ignore everything but resurrect *.[ch] with negative exclude
rules, can we do the same without moving things around?

[RFC PATCH] t/helper: Move sources to t/helper-src; gitignore any files in t/helper

[Stefan Beller]

Compiled test helpers in t/helper are out of sync with the .gitignore
files quite frequently. This can happen when new test helpers are added,
but the explicit .gitignore file is not updated in the same commit, or
when you forget to 'make clean' before checking out a different version
of git, as the different version may have a different explicit list of
test helpers to ignore.

This can be fixed by using overly broad ignore patterns, such as ignoring
the whole directory via '//t/helper/*' in .gitignore.

However we do not want to have an overlap of checked source files and
ignored files, hence we'll move the the source files currently residing
in t/helper to t/helper-src. To accommodate that we'll need to update
the Makefile as well to look at a different place for the source files.
(This patch takes the hacky approach in symlinking the sources back into
the t/helper, which we'd want to avoid long term)

Signed-off-by: Stefan Beller <sbel...@google.com>
---
 Makefile   |  4 +++
 t/{helper => helper-src}/.gitignore|  0
 t/{helper => helper-src}/test-chmtime.c|  0
 t/{helper => helper-src}/test-config.c |  0
 t/{helper => helper-src}/test-ctype.c  |  0
 t/{helper => helper-src}/test-date.c   |  0
 t/{helper => helper-src}/test-delta.c  |  0
 t/{helper => helper-src}/test-dump-cache-tree.c|  0
 t/{helper => helper-src}/test-dump-split-index.c   |  0
 .../test-dump-untracked-cache.c|  0
 t/{helper => helper-src}/test-fake-ssh.c   |  0
 t/{helper => helper-src}/test-genrandom.c  |  0
 t/{helper => helper-src}/test-hashmap.c|  0
 t/{helper => helper-src}/test-index-version.c  |  0
 .../test-lazy-init-name-hash.c |  0
 t/{helper => helper-src}/test-line-buffer.c|  0
 t/{helper => helper-src}/test-match-trees.c|  0
 t/{helper => helper-src}/test-mergesort.c  |  0
 t/{helper => helper-src}/test-mktemp.c |  0
 t/{helper => helper-src}/test-online-cpus.c|  0
 t/{helper => helper-src}/test-parse-options.c  |  0
 t/{helper => helper-src}/test-path-utils.c |  0
 t/{helper => helper-src}/test-prio-queue.c |  0
 t/{helper => helper-src}/test-read-cache.c |  0
 t/{helper => helper-src}/test-ref-store.c  |  0
 t/{helper => helper-src}/test-regex.c  |  0
 t/{helper => helper-src}/test-revision-walking.c   |  0
 t/{helper => helper-src}/test-run-command.c|  0
 t/{helper => helper-src}/test-scrap-cache-tree.c   |  0
 t/{helper => helper-src}/test-sha1-array.c |  0
 t/{helper => helper-src}/test-sha1.c   |  0
 t/{helper => helper-src}/test-sha1.sh  |  0
 t/{helper => helper-src}/test-sigchain.c   |  0
 t/{helper => helper-src}/test-strcmp-offset.c  |  0
 t/{helper => helper-src}/test-string-list.c|  0
 t/{helper => helper-src}/test-submodule-config.c   |  0
 t/{helper => helper-src}/test-subprocess.c |  0
 t/{helper => helper-src}/test-svn-fe.c |  0
 .../test-urlmatch-normalization.c  |  0
 t/{helper => helper-src}/test-wildmatch.c  |  0
 t/{helper => helper-src}/test-write-cache.c|  0
 t/helper/.gitignore    | 39 +-
 42 files changed, 5 insertions(+), 38 deletions(-)
 copy t/{helper => helper-src}/.gitignore (100%)
 rename t/{helper => helper-src}/test-chmtime.c (100%)
 rename t/{helper => helper-src}/test-config.c (100%)
 rename t/{helper => helper-src}/test-ctype.c (100%)
 rename t/{helper => helper-src}/test-date.c (100%)
 rename t/{helper => helper-src}/test-delta.c (100%)
 rename t/{helper => helper-src}/test-dump-cache-tree.c (100%)
 rename t/{helper => helper-src}/test-dump-split-index.c (100%)
 rename t/{helper => helper-src}/test-dump-untracked-cache.c (100%)
 rename t/{helper => helper-src}/test-fake-ssh.c (100%)
 rename t/{helper => helper-src}/test-genrandom.c (100%)
 rename t/{helper => helper-src}/test-hashmap.c (100%)
 rename t/{helper => helper-src}/test-index-version.c (100%)
 rename t/{helper => helper-src}/test-lazy-init-name-hash.c (100%)
 rename t/{helper => helper-src}/test-line-buffer.c (100%)
 rename t/{helper => helper-src}/test-match-trees.c (100%)
 rename t/{helper => helper-src}/test-mergesort.c (100%)
 rename t/{helper => helper-src}/test-mktemp.c (100%)
 rename t/{helper => helper-src}/test-online-cpus.c (100%)
 rename t/{helper => helper-src}/test-parse-options.c (100%)
 rename t/{helper => helper-src}/test-path-utils.c (100%)
 rename t/{helper => helper-src}/test-prio-queue.c (100%)
 rename t/{helper => helper-src}/test-read-cache.c (100%)
 rename t/{helper => helpe

Need to add test artifacts to .gitignore

[Dan Jacques]

FYI, I've noticed when building from "pu" that neither the
"t/helper/test-print-values" or "t/helper/test-print-larger-than-ssize"
testing artifacts added to Makefile in this patch series are not added to
"t/helper/.gitignore" like other helpers, resulting in the testing artifact
being recognized as an untracked file.

Bad interaction between git clean, gitignore, and deleted submodules

[Craig Silverstein]

We have the following situation:
1) A .gitignore file that contains '*.pyc'
2) A repo with a submodule named jinja2

In normal use, clients of our repo have it checked out and run things
in it, creating files like jinja2/run.pyc.

I deleted the jinja2 submodule (by running `git rm jinja2` and
pushing).  Clients did a `git pull; git submodule update` and saw that
the jinja2 directory was still around, albeit untracked. So they ran
`git clean -ffd`.

The problem is that git clean refuses to delete the directory due to
the (ignored and thus uncleanable) file jinja2/run.pyc.  And when it
refuses to delete the directory, it also leaves around jinja2/.git.

Later, someone ran `git add .` and jinja2/.git got added back in as an
"orphaned submodule" (I forget the exact terminology).  I know there's
a very loud warning when this happens but somehow they didn't see it,
and it caused all sorts of trouble when they pushed their change.

My question: is it possible for a client to *really* get rid of the
jinja2 submodule?  We can't run `git clean -ffdx` because there are
other .gitignore'd files we want to keep around.

The behavior I'd like to see is for `git clean -ffd` to delete .git
files if they don't correspond to a currently registered submodule.
Then `git clean -ffd` would delete jinja2/.git even though it leaves
around jinja2/run.pyc.  But I don't know if that would break anything
else.

Or maybe we should just add `.git` to our `.gitignore`, so people who
run `git add .` can't create these orphaned submodules...

craig

clarify documentation of leading "**" in gitignore(5) man page

[David Prager Branner]

I've long been confused by something in the man page for gitignore. I think 
it's unclear and I'd like to propose a change. The passage is this (source at 
https://git.kernel.org/pub/scm/git/git-manpages.git/tree/man5/gitignore.5):


Two consecutive asterisks ("**") in patterns matched against full
pathname may have special meaning:

o   A leading "**" followed by a slash means match in all directories.
For example, "**/foo" matches file or directory "foo" anywhere, the
same as pattern "foo". "**/foo/bar" matches file or directory "bar"
anywhere that is directly under directory "foo".

...


In the first paragraph, it would be clearer to specify:


Two consecutive asterisks ("**"), in patterns matched against a full
pathname, are a wildcard representing some arbitrary number of nested
directories within that pathname:


In the second paragraph, I suggest removing the first pattern, "**/foo", since it is completely 
redundant. There are no circumstances when "**/foo" is necessary in place of plain "foo", 
and its presence muddies discussion.

Leading "**" may represent nested directories in a pathname, but is useful only 
before a _pattern_ containing nested directories. I suggest making that explicit. Here is 
how I propose to rewrite it:


o   A leading "**", followed by a slash and a pattern containing nested
directories, means match that pattern in all pathnames.
For example, "**/foo/bar" matches file or directory "bar"
anywhere that is directly under directory "foo". It would match
"foo/bar", "other/foo/bar", "src/foo/bar/scram/gravy".


Thanks.

- dpb

Re: [PATCH] Add t/helper/test-write-cache to .gitignore

[Jonathan Nieder]

Brandon Williams wrote:
> On 08/28, Jonathan Tan wrote:

>> This new binary was introduced in commit 3921a0b ("perf: add test for
>> writing the index", 2017-08-21), but a .gitignore entry was not added
>> for it. Add that entry.
>>
>> Signed-off-by: Jonathan Tan <jonathanta...@google.com>
>
> Looks good to me

Reviewed-by: Jonathan Nieder <jrnie...@gmail.com>
as well.

I wonder if we can automate this a little.  Some thoughts:

 A. We could include a test that all the binaries named in
    TEST_PROGRAMS_NEED_X are also named in t/helper/.gitignore.  That
way tests would fail if this ever falls out of date again.

 B. Even better would be if we could have /t/helper/test-* in
.gitignore.  E.g. if we rename test-*.c to *.c (e.g.
t/helper/ctype.c instead of t/helper/test-ctype.c), then the test
helper source file names would be easier to type and maintaining
this .gitignore would become a problem of the past.

What do you think?
Jonathan

Re: [PATCH] Add t/helper/test-write-cache to .gitignore

[Brandon Williams]

On 08/28, Jonathan Tan wrote:
> This new binary was introduced in commit 3921a0b ("perf: add test for
> writing the index", 2017-08-21), but a .gitignore entry was not added
> for it. Add that entry.
> 
> Signed-off-by: Jonathan Tan <jonathanta...@google.com>

Looks good to me

> ---
>  t/helper/.gitignore | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/t/helper/.gitignore b/t/helper/.gitignore
> index 721650256..7c9d28a83 100644
> --- a/t/helper/.gitignore
> +++ b/t/helper/.gitignore
> @@ -35,3 +35,4 @@
>  /test-svn-fe
>  /test-urlmatch-normalization
>  /test-wildmatch
> +/test-write-cache
> -- 
> 2.14.1.581.gf28d330327-goog
> 

-- 
Brandon Williams

[PATCH] Add t/helper/test-write-cache to .gitignore

[Jonathan Tan]

This new binary was introduced in commit 3921a0b ("perf: add test for
writing the index", 2017-08-21), but a .gitignore entry was not added
for it. Add that entry.

Signed-off-by: Jonathan Tan <jonathanta...@google.com>
---
 t/helper/.gitignore | 1 +
 1 file changed, 1 insertion(+)

diff --git a/t/helper/.gitignore b/t/helper/.gitignore
index 721650256..7c9d28a83 100644
--- a/t/helper/.gitignore
+++ b/t/helper/.gitignore
@@ -35,3 +35,4 @@
 /test-svn-fe
 /test-urlmatch-normalization
 /test-wildmatch
+/test-write-cache
-- 
2.14.1.581.gf28d330327-goog

Re: Bug when stashing previously-ignored file plus associated .gitignore change

[Sam Partington]

Hello Kevin,

Yes, you're right - I didn't commit the change to the .gitignore file,
so that addition is also being stashed.

Thanks
Sam
Sam Partington
Senior Developer



www.whiteoctober.co.uk
Office: +44 (0)1865 920 707

This email message and any attachments are confidential and solely for
the use of the intended recipient. If you are not the intended
recipient, you have received this message in error. Please notify us
immediately and delete the message from your computer. You should not
distribute, copy or disclose its contents to any other person. Any
views or opinions expressed in this email are solely those of the
author and do not necessarily represent those of White October
Limited. White October is a private limited company registered in
England & Wales under registration number 3982889. The company’s
registered office is at 264 Banbury Road, Oxford, OX2 7DY.


On 14 August 2017 at 05:47, Kevin Daudt <m...@ikke.info> wrote:
> On Fri, Aug 11, 2017 at 04:55:38PM +0100, Sam Partington wrote:
>> Hi there,
>>
>> I'm running git 2.7.4 on Ubuntu 16.04.  I've found a couple of
>> problems when "un-ignoring" files in tandem with git stash.
>>
>> Here's how to reproduce:
>>
>> Say you have a project using git, with a .gitignore file which
>> contains the following line:
>>
>> bin/*
>>
>> You can then see the problems by doing this:
>>
>> $ touch bin/mynewfile # this file will be ignored at this point >
>> and then updating .gitignore to look like this (adding that second line):
>>
>> bin/*
>> !bin/mynewfile
>>
>> So far, so good; the new file is no longer ignored.
>>
>> But now, try stashing the changes and including untracked files in the stash:
>>
>> $ git stash save -u
>>
>> Here's the first problem, bin/mynewfile is still there:
>>
>> $ ls bin/mynewfile
>> bin/mynewfile
>>
>> But you'd expect it to not be there and be in the stash, I think.
>> This is what would normally happen with the untracked-files option for
>> git stash.
>>
>> This leads to the second problem - you can't now pop the stash:
>>
>> $ git stash pop
>> bin/mynewfile already exists, no checkout
>> Could not restore untracked files from stash
>>
>> If you want to apply the stash, you have to remove the file:
>>
>> $ rm bin/mynewfile
>> $ git stash pop # this works, and re-creates bin/mynewfile
>>
>> This is quite an unusual edge case, but I have hit it two or three
>> times now and so thought it worth reporting, but I'll understand if
>> it's deemed not worth fixing!
>>
>> Do let me know if you need any more information from me here.
>>
>> Thanks
>> Sam
>>
>> PS Sorry for the lack of formatting - I'm sending this as plain text
>> as my original HTML emails was rejected as possible spam by your
>> mailserver.
>>
>> Sam Partington
>> Senior Developer
>>
>
> Hello Sam,
>
> Is it the case that you did not commit the addition of '!bin/mynewfile'
> yet? I suspect that by running git stash save -u, you also are stashing
> this addition to the .gitigore file. Can you confirm this?
>
> Kevin

Re: Bug when stashing previously-ignored file plus associated .gitignore change

[Kevin Daudt]

On Fri, Aug 11, 2017 at 04:55:38PM +0100, Sam Partington wrote:
> Hi there,
> 
> I'm running git 2.7.4 on Ubuntu 16.04.  I've found a couple of
> problems when "un-ignoring" files in tandem with git stash.
> 
> Here's how to reproduce:
> 
> Say you have a project using git, with a .gitignore file which
> contains the following line:
> 
> bin/*
> 
> You can then see the problems by doing this:
> 
> $ touch bin/mynewfile # this file will be ignored at this point > 
> and then updating .gitignore to look like this (adding that second line):
> 
> bin/*
> !bin/mynewfile
> 
> So far, so good; the new file is no longer ignored.
> 
> But now, try stashing the changes and including untracked files in the stash:
> 
> $ git stash save -u
> 
> Here's the first problem, bin/mynewfile is still there:
> 
> $ ls bin/mynewfile
> bin/mynewfile
> 
> But you'd expect it to not be there and be in the stash, I think.
> This is what would normally happen with the untracked-files option for
> git stash.
> 
> This leads to the second problem - you can't now pop the stash:
> 
> $ git stash pop
> bin/mynewfile already exists, no checkout
> Could not restore untracked files from stash
> 
> If you want to apply the stash, you have to remove the file:
> 
> $ rm bin/mynewfile
> $ git stash pop # this works, and re-creates bin/mynewfile
> 
> This is quite an unusual edge case, but I have hit it two or three
> times now and so thought it worth reporting, but I'll understand if
> it's deemed not worth fixing!
> 
> Do let me know if you need any more information from me here.
> 
> Thanks
> Sam
> 
> PS Sorry for the lack of formatting - I'm sending this as plain text
> as my original HTML emails was rejected as possible spam by your
> mailserver.
> 
> Sam Partington
> Senior Developer
> 

Hello Sam,

Is it the case that you did not commit the addition of '!bin/mynewfile'
yet? I suspect that by running git stash save -u, you also are stashing
this addition to the .gitigore file. Can you confirm this?

Kevin

Bug when stashing previously-ignored file plus associated .gitignore change

[Sam Partington]

Hi there,

I'm running git 2.7.4 on Ubuntu 16.04.  I've found a couple of
problems when "un-ignoring" files in tandem with git stash.

Here's how to reproduce:

Say you have a project using git, with a .gitignore file which
contains the following line:

bin/*

You can then see the problems by doing this:

$ touch bin/mynewfile # this file will be ignored at this point

and then updating .gitignore to look like this (adding that second line):

bin/*
!bin/mynewfile

So far, so good; the new file is no longer ignored.

But now, try stashing the changes and including untracked files in the stash:

$ git stash save -u

Here's the first problem, bin/mynewfile is still there:

$ ls bin/mynewfile
bin/mynewfile

But you'd expect it to not be there and be in the stash, I think.
This is what would normally happen with the untracked-files option for
git stash.

This leads to the second problem - you can't now pop the stash:

$ git stash pop
bin/mynewfile already exists, no checkout
Could not restore untracked files from stash

If you want to apply the stash, you have to remove the file:

$ rm bin/mynewfile
$ git stash pop # this works, and re-creates bin/mynewfile

This is quite an unusual edge case, but I have hit it two or three
times now and so thought it worth reporting, but I'll understand if
it's deemed not worth fixing!

Do let me know if you need any more information from me here.

Thanks
Sam

PS Sorry for the lack of formatting - I'm sending this as plain text
as my original HTML emails was rejected as possible spam by your
mailserver.

Sam Partington
Senior Developer

www.whiteoctober.co.uk
Office: +44 (0)1865 920 707

This email message and any attachments are confidential and solely for
the use of the intended recipient. If you are not the intended
recipient, you have received this message in error. Please notify us
immediately and delete the message from your computer. You should not
distribute, copy or disclose its contents to any other person. Any
views or opinions expressed in this email are solely those of the
author and do not necessarily represent those of White October
Limited. White October is a private limited company registered in
England & Wales under registration number 3982889. The company’s
registered office is at 264 Banbury Road, Oxford, OX2 7DY.

Re: BUG: The .gitignore rules can't be made to cross submodule boundaries

[Stefan Beller]

On Tue, May 23, 2017 at 2:55 AM, Ævar Arnfjörð Bjarmason
<ava...@gmail.com> wrote:
> On Tue, May 23, 2017 at 11:17 AM, Johannes Schindelin
> <johannes.schinde...@gmx.de> wrote:
>> Hi Ævar,
>>
>> On Mon, 22 May 2017, Ævar Arnfjörð Bjarmason wrote:
>>
>>> When I was adding the sha1collisiondetection submodule to git.git I
>>> noticed that building git would dirty the submodule.
>>>
>>> This is because our own Makefile adds .depend/ directories. I hacked
>>> around it by just getting the upstream project accept carrying an ignore
>>> rule for that around:
>>> https://github.com/cr-marcstevens/sha1collisiondetection/commit/e8397b26
>>>
>>> A workaround for this is to have the Makefile add such a rule to
>>> .git/modules/sha1collisiondetection/info/exclude, but that's less
>>> convenient than being able to distribute it as a normal .gitignore rule.
>>>
>>> The submodule..ignore config provides an overly big hammer to
>>> solve this, it would be better if we had something like
>>> submodule..gitignore=. Then we could have e.g.
>>> .gitignore.sha1collisiondetection which would be added to whatever rules
>>> the repo's own .gitignore provides.
>>
>> While I have nothing but the utmost respect for Stefan and Brandon for
>> trying to improve submodules, maybe it would be a wiser idea to imitate
>> the same strategy with sha1dc as we use with git-gui and gitk, i.e.
>> perform a subtree merge instead of adding it as a submodule. It's not like
>> 570kB will kill us.

Actually that is a very valid bug report outside that series for the
behavior of submodules.

In a world where you use a submodule to track say a third party
library, the current behavior of .gitignore applying to each repo makes
sense.

When it is no third party, but a first party lib, then it is sensible to expect
that the building/testing infrastructure works across the whole repo set,
and the user wants just one central place to specify things, such as
ignoring certain files or applying .gitattributes.

This topic came up in various forms on the mailing list, most often for
config that ought to be applied across all repos[1].

That said I have no good idea yet how to fix this issue without introducing
the ultimate user confusion.

The conditional include of config files (by Duy as part of 2.13) seems like
an interesting approach, which we could build on top of.
We currently have a main config and a per-working-tree config, so I would
expect we'd introduce another config file that is included by all submodules
by default. It could be located in the superproject at ".git/config.super".
This config file could then specify

[submodule]
recursiveIgnore = [yes/no]
recursiveAttributes = [yes/no]

In that way commands run from within the submodule as well as from
the superproject would realize that the submodule needs to lookup
the superproject and use the attribute/ignore/config settings from there.

[1] Here the example for URL.insteadOf
https://public-inbox.org/git/capz477mcsbsfbqkzp69mt_brwz-0aes6twjofqrhizubv7z...@mail.gmail.com/

>
> The submodule/.gitignore bug/feature-request being reported here isn't
> something that impacts the ab/sha1dc series in practice.
>
> It was something I noticed while working with an earlier commit in
> that repo, but that's a commit that'll never be pinned by the
> git.git:sha1collisiondetection submodule.

Thanks for the bug report. As outlined above, we'd still need to bikeshed
how to fix it properly I'd think.

Thanks,
Stefan

Re: BUG: The .gitignore rules can't be made to cross submodule boundaries

[Ævar Arnfjörð Bjarmason]

On Tue, May 23, 2017 at 11:17 AM, Johannes Schindelin
<johannes.schinde...@gmx.de> wrote:
> Hi Ævar,
>
> On Mon, 22 May 2017, Ævar Arnfjörð Bjarmason wrote:
>
>> When I was adding the sha1collisiondetection submodule to git.git I
>> noticed that building git would dirty the submodule.
>>
>> This is because our own Makefile adds .depend/ directories. I hacked
>> around it by just getting the upstream project accept carrying an ignore
>> rule for that around:
>> https://github.com/cr-marcstevens/sha1collisiondetection/commit/e8397b26
>>
>> A workaround for this is to have the Makefile add such a rule to
>> .git/modules/sha1collisiondetection/info/exclude, but that's less
>> convenient than being able to distribute it as a normal .gitignore rule.
>>
>> The submodule..ignore config provides an overly big hammer to
>> solve this, it would be better if we had something like
>> submodule..gitignore=. Then we could have e.g.
>> .gitignore.sha1collisiondetection which would be added to whatever rules
>> the repo's own .gitignore provides.
>
> While I have nothing but the utmost respect for Stefan and Brandon for
> trying to improve submodules, maybe it would be a wiser idea to imitate
> the same strategy with sha1dc as we use with git-gui and gitk, i.e.
> perform a subtree merge instead of adding it as a submodule. It's not like
> 570kB will kill us.

The submodule/.gitignore bug/feature-request being reported here isn't
something that impacts the ab/sha1dc series in practice.

It was something I noticed while working with an earlier commit in
that repo, but that's a commit that'll never be pinned by the
git.git:sha1collisiondetection submodule.

Re: BUG: The .gitignore rules can't be made to cross submodule boundaries

[Johannes Schindelin]

Hi Ævar,

On Mon, 22 May 2017, Ævar Arnfjörð Bjarmason wrote:

> When I was adding the sha1collisiondetection submodule to git.git I
> noticed that building git would dirty the submodule.
> 
> This is because our own Makefile adds .depend/ directories. I hacked
> around it by just getting the upstream project accept carrying an ignore
> rule for that around:
> https://github.com/cr-marcstevens/sha1collisiondetection/commit/e8397b26
> 
> A workaround for this is to have the Makefile add such a rule to
> .git/modules/sha1collisiondetection/info/exclude, but that's less
> convenient than being able to distribute it as a normal .gitignore rule.
> 
> The submodule..ignore config provides an overly big hammer to
> solve this, it would be better if we had something like
> submodule..gitignore=. Then we could have e.g.
> .gitignore.sha1collisiondetection which would be added to whatever rules
> the repo's own .gitignore provides.

While I have nothing but the utmost respect for Stefan and Brandon for
trying to improve submodules, maybe it would be a wiser idea to imitate
the same strategy with sha1dc as we use with git-gui and gitk, i.e.
perform a subtree merge instead of adding it as a submodule. It's not like
570kB will kill us.

Ciao,
Dscho

BUG: The .gitignore rules can't be made to cross submodule boundaries

[Ævar Arnfjörð Bjarmason]

When I was adding the sha1collisiondetection submodule to git.git I
noticed that building git would dirty the submodule.

This is because our own Makefile adds .depend/ directories. I hacked
around it by just getting the upstream project accept carrying an
ignore rule for that around:
https://github.com/cr-marcstevens/sha1collisiondetection/commit/e8397b26

A workaround for this is to have the Makefile add such a rule to
.git/modules/sha1collisiondetection/info/exclude, but that's less
convenient than being able to distribute it as a normal .gitignore
rule.

The submodule..ignore config provides an overly big hammer to
solve this, it would be better if we had something like
submodule..gitignore=. Then we could have e.g.
.gitignore.sha1collisiondetection which would be added to whatever
rules the repo's own .gitignore provides.

Re: bug with git add and .gitignore

[Samuel Lijin]

On Wed, May 10, 2017 at 9:44 AM, Orgad Shaneh <org...@gmail.com> wrote:
> Hi,
>
> When a not-ignored file inside an ignore directory is added along with
> other files, a false alarm is shown:
>
> git init
> echo /d/ > .gitignore
> mkdir d
> touch d/file foo
> git add -f d/file foo
> git add d/file
> # fine
> git add d/file foo
> # The following paths are ignored by one of your .gitignore files:
> # d
> # Use -f if you really want to add them.

For anyone who wants to take a shot at this, the reason this happens
is because when "add d/file foo" is invoked, when add.c:cmd_add()
calls fill_directory(), d/ gets added to dir->ignored, so add_files()
complains. I'm not sure if that's *supposed* to happen, and the issue
is somewhere else, or the call to read_directory_recursive() from
fill_directory() doesn't properly respect the pathspec.

> I did not try to add a new file in d. It's the same file that is
> already indexed.
>
> - Orgad

bug with git add and .gitignore

[Orgad Shaneh]

Hi,

When a not-ignored file inside an ignore directory is added along with
other files, a false alarm is shown:

git init
echo /d/ > .gitignore
mkdir d
touch d/file foo
git add -f d/file foo
git add d/file
# fine
git add d/file foo
# The following paths are ignored by one of your .gitignore files:
# d
# Use -f if you really want to add them.

I did not try to add a new file in d. It's the same file that is
already indexed.

- Orgad

Re: Bug Report: .gitignore behavior is not matching in git clean and git status

[Junio C Hamano]

Samuel Lijin  writes:

> After some more digging (and familiarizing myself with the
> behind-the-scenes logic) the issue is that dir.c has this implicit
> assumption that a directory which contains only untracked and ignored
> files should itself be considered untracked. While that works fine for
> use cases where we're asking if a directory should be added to the git
> database, that decidedly does not make sense when we're asking if a
> directory can be removed from the working tree.

Thanks for digging.

> I'm not sure where to proceed from here. I see two ways forward: one,
> builtin/clean.c can collect ignored files when it calls
> dir.c:fill_directory(), and then clean -d can prune out directories
> that contain ignored files; two, path_treatment can learn about
> untracked directories which contain excluded (ignored) files.

My gut feeling is that the former approach would be of lesser
impact.  Directory A/ can be removed "clean" without "-x" when there
is nothing tracked in there in the index and there is no ignored
paths.  Having zero untracked files there or one or more untracked
file there do not matter---they are all subject to removal by
"clean".

Re: Bug Report: .gitignore behavior is not matching in git clean and git status

[Samuel Lijin]

After some more digging (and familiarizing myself with the
behind-the-scenes logic) the issue is that dir.c has this implicit
assumption that a directory which contains only untracked and ignored
files should itself be considered untracked. While that works fine for
use cases where we're asking if a directory should be added to the git
database, that decidedly does not make sense when we're asking if a
directory can be removed from the working tree.

I'm not sure where to proceed from here. I see two ways forward: one,
builtin/clean.c can collect ignored files when it calls
dir.c:fill_directory(), and then clean -d can prune out directories
that contain ignored files; two, path_treatment can learn about
untracked directories which contain excluded (ignored) files.

On Mon, May 1, 2017 at 8:51 AM, Samuel Lijin <sxli...@gmail.com> wrote:
> On Sun, Apr 30, 2017 at 8:56 PM, Chris Johnson <chrisjohns...@gmail.com> 
> wrote:
>> Good assessment/understanding of the issue. git clean -n  does not
>> report anything as being targeted for removal, and git clean -f
>> matches that behavior. I agree with it probably being related
>> specifically to the -d flag.
>>
>> As another experiment I modified .gitignore to ignore /A/B/C instead
>> of /A/B/ and the same result occurs (-n reports nothing, -dn reports
>> removing A/)
>>
>> Lastly, I changed .gitignore to just be /A/, and in doing so, clean
>> -dn stops reporting that it will remove A/. I’m not exactly sure if
>> this last one is surprising or not.
>
> It doesn't seem so to me, since a trailing slash in .gitignore is an
> explicit directive to ignore the entire directory, so when pruning the
> tree of files/dirs to ignore, it drops everything in A/ before even
> getting to A/B/C. The issue is that ignoring A/B/C shouldn't leave A/
> to be cleaned.
>
>> Also, and sorry for the noise, but I did a reply-all here, but will a
>> reply automatically include the rest of the list? Or was reply-all the
>> right move?
>>
>> On Sun, Apr 30, 2017 at 9:41 PM, Junio C Hamano <gits...@pobox.com> wrote:
>>> Chris Johnson <chrisjohns...@gmail.com> writes:
>>>
>>>> I am a mailing list noob so I’m sorry if this is the wrong format or
>>>> the wrong please.
>>>>
>>>> Here’s the setup for the bug (I will call it a bug but I half expect
>>>> somebody to tell me I’m an idiot):
>>>>
>>>> git init
>>>> echo "/A/B/" > .gitignore
>>>
>>> You tell Git that anything in A/B/ are uninteresting.
>>>
>>>> git add .gitignore && git commit -m 'Add ignore'
>>>> mkdir -p A/B
>>>> touch A/B/C
>>>
>>> And create an uninteresting cruft.
>>>
>>>> git status
>>>
>>> And Git does not bug you about it.
>>>
>>>> git clean -dn
>>>
>>> This incorrectly reports "Would remove A/" and if you gave 'f'
>>> instead of 'n', it does remove A/, A/B, and A/B/C.
>>>
>>> Despite that "git clean --help" says 'only files unknown to Git are
>>> removed' (with an undefined term 'unknown to Git').  What it wants
>>> the term mean can be guessed by seeing 'if the -x option is
>>> specified, ignored files are also removed'---so 'unknown to Git'
>>> does not include what you told .gitignore that they are
>>> uninteresting.  IOW, Git knows they are not interesting.
>>>
>>> It looks like a bug in "git clean -d" to me.
>
> I may be wrong (I'm not very familiar with the codebase), but I don't
> think it's a bug in specifically git clean -d.
>
> Throwing gdb at it, when builtin/clean.c:cmd_clean() calls
> dir.c:fill_directory(), A/ gets appended to dir->entries, regardless
> of whether or not git clean is called with or without -d. The
> difference is that if git clean is called without -d, the loop that
> strips out directories strips out the A/ entry.
>
> When dir.c:fill_directory() is invoked through git status, A/ does
> not, however, get appended to dir->entries. As best as I can tell,
> this seems to be because git status sets the
> DIR_HIDE_EMPTY_DIRECTORIES flag, whereas git clean does not (which
> makes sense), but the fact that DIR_HIDE_EMPTY_DIRECTORIES is
> responsible for not adding A/ to dir->entries seems to be the issue.

Re: Bug Report: .gitignore behavior is not matching in git clean and git status

[Samuel Lijin]

On Sun, Apr 30, 2017 at 8:56 PM, Chris Johnson <chrisjohns...@gmail.com> wrote:
> Good assessment/understanding of the issue. git clean -n  does not
> report anything as being targeted for removal, and git clean -f
> matches that behavior. I agree with it probably being related
> specifically to the -d flag.
>
> As another experiment I modified .gitignore to ignore /A/B/C instead
> of /A/B/ and the same result occurs (-n reports nothing, -dn reports
> removing A/)
>
> Lastly, I changed .gitignore to just be /A/, and in doing so, clean
> -dn stops reporting that it will remove A/. I’m not exactly sure if
> this last one is surprising or not.

It doesn't seem so to me, since a trailing slash in .gitignore is an
explicit directive to ignore the entire directory, so when pruning the
tree of files/dirs to ignore, it drops everything in A/ before even
getting to A/B/C. The issue is that ignoring A/B/C shouldn't leave A/
to be cleaned.

> Also, and sorry for the noise, but I did a reply-all here, but will a
> reply automatically include the rest of the list? Or was reply-all the
> right move?
>
> On Sun, Apr 30, 2017 at 9:41 PM, Junio C Hamano <gits...@pobox.com> wrote:
>> Chris Johnson <chrisjohns...@gmail.com> writes:
>>
>>> I am a mailing list noob so I’m sorry if this is the wrong format or
>>> the wrong please.
>>>
>>> Here’s the setup for the bug (I will call it a bug but I half expect
>>> somebody to tell me I’m an idiot):
>>>
>>> git init
>>> echo "/A/B/" > .gitignore
>>
>> You tell Git that anything in A/B/ are uninteresting.
>>
>>> git add .gitignore && git commit -m 'Add ignore'
>>> mkdir -p A/B
>>> touch A/B/C
>>
>> And create an uninteresting cruft.
>>
>>> git status
>>
>> And Git does not bug you about it.
>>
>>> git clean -dn
>>
>> This incorrectly reports "Would remove A/" and if you gave 'f'
>> instead of 'n', it does remove A/, A/B, and A/B/C.
>>
>> Despite that "git clean --help" says 'only files unknown to Git are
>> removed' (with an undefined term 'unknown to Git').  What it wants
>> the term mean can be guessed by seeing 'if the -x option is
>> specified, ignored files are also removed'---so 'unknown to Git'
>> does not include what you told .gitignore that they are
>> uninteresting.  IOW, Git knows they are not interesting.
>>
>> It looks like a bug in "git clean -d" to me.

I may be wrong (I'm not very familiar with the codebase), but I don't
think it's a bug in specifically git clean -d.

Throwing gdb at it, when builtin/clean.c:cmd_clean() calls
dir.c:fill_directory(), A/ gets appended to dir->entries, regardless
of whether or not git clean is called with or without -d. The
difference is that if git clean is called without -d, the loop that
strips out directories strips out the A/ entry.

When dir.c:fill_directory() is invoked through git status, A/ does
not, however, get appended to dir->entries. As best as I can tell,
this seems to be because git status sets the
DIR_HIDE_EMPTY_DIRECTORIES flag, whereas git clean does not (which
makes sense), but the fact that DIR_HIDE_EMPTY_DIRECTORIES is
responsible for not adding A/ to dir->entries seems to be the issue.

Re: Bug Report: .gitignore behavior is not matching in git clean and git status

[Junio C Hamano]

Chris Johnson  writes:

> Also, and sorry for the noise, but I did a reply-all here, but will a
> reply automatically include the rest of the list? Or was reply-all the
> right move?

The convention around here is to do reply-all (in other words, make
sure that Cc: line has git@vger.kernel.org and others involved in
the discussion and mentioned on To: or Cc: line of the message you
are responding to).  Your message (i.e. the one I am responding to)
has correct addresses on To:/Cc: lines AFAICS.

Re: Bug Report: .gitignore behavior is not matching in git clean and git status

[Chris Johnson]

Good assessment/understanding of the issue. git clean -n  does not
report anything as being targeted for removal, and git clean -f
matches that behavior. I agree with it probably being related
specifically to the -d flag.

As another experiment I modified .gitignore to ignore /A/B/C instead
of /A/B/ and the same result occurs (-n reports nothing, -dn reports
removing A/)

Lastly, I changed .gitignore to just be /A/, and in doing so, clean
-dn stops reporting that it will remove A/. I’m not exactly sure if
this last one is surprising or not.

Also, and sorry for the noise, but I did a reply-all here, but will a
reply automatically include the rest of the list? Or was reply-all the
right move?

On Sun, Apr 30, 2017 at 9:41 PM, Junio C Hamano <gits...@pobox.com> wrote:
> Chris Johnson <chrisjohns...@gmail.com> writes:
>
>> I am a mailing list noob so I’m sorry if this is the wrong format or
>> the wrong please.
>>
>> Here’s the setup for the bug (I will call it a bug but I half expect
>> somebody to tell me I’m an idiot):
>>
>> git init
>> echo "/A/B/" > .gitignore
>
> You tell Git that anything in A/B/ are uninteresting.
>
>> git add .gitignore && git commit -m 'Add ignore'
>> mkdir -p A/B
>> touch A/B/C
>
> And create an uninteresting cruft.
>
>> git status
>
> And Git does not bug you about it.
>
>> git clean -dn
>
> This incorrectly reports "Would remove A/" and if you gave 'f'
> instead of 'n', it does remove A/, A/B, and A/B/C.
>
> Despite that "git clean --help" says 'only files unknown to Git are
> removed' (with an undefined term 'unknown to Git').  What it wants
> the term mean can be guessed by seeing 'if the -x option is
> specified, ignored files are also removed'---so 'unknown to Git'
> does not include what you told .gitignore that they are
> uninteresting.  IOW, Git knows they are not interesting.
>
> It looks like a bug in "git clean -d" to me.  Do you see the same
> issue if you use "git clean" without "-d"?  IOW, does it offer to
> remove A/B/C?

Re: Bug Report: .gitignore behavior is not matching in git clean and git status

[Junio C Hamano]

Chris Johnson <chrisjohns...@gmail.com> writes:

> I am a mailing list noob so I’m sorry if this is the wrong format or
> the wrong please.
>
> Here’s the setup for the bug (I will call it a bug but I half expect
> somebody to tell me I’m an idiot):
>
> git init
> echo "/A/B/" > .gitignore

You tell Git that anything in A/B/ are uninteresting.

> git add .gitignore && git commit -m 'Add ignore'
> mkdir -p A/B
> touch A/B/C

And create an uninteresting cruft.

> git status

And Git does not bug you about it.

> git clean -dn

This incorrectly reports "Would remove A/" and if you gave 'f'
instead of 'n', it does remove A/, A/B, and A/B/C.

Despite that "git clean --help" says 'only files unknown to Git are
removed' (with an undefined term 'unknown to Git').  What it wants
the term mean can be guessed by seeing 'if the -x option is
specified, ignored files are also removed'---so 'unknown to Git'
does not include what you told .gitignore that they are
uninteresting.  IOW, Git knows they are not interesting.

It looks like a bug in "git clean -d" to me.  Do you see the same
issue if you use "git clean" without "-d"?  IOW, does it offer to
remove A/B/C?

Bug Report: .gitignore behavior is not matching in git clean and git status

[Chris Johnson]

I am a mailing list noob so I’m sorry if this is the wrong format or
the wrong please.

Here’s the setup for the bug (I will call it a bug but I half expect
somebody to tell me I’m an idiot):

git init
echo "/A/B/" > .gitignore
git add .gitignore && git commit -m 'Add ignore'
mkdir -p A/B
touch A/B/C
git status
git clean -dn

(my client may have screwed up the quotes, please bear with me)

Now, this may just be a case of me misunderstanding the behavior of
.gitignore, and that’s fine, but to me, git clean -dn should roughly
reflect the same behavior as git status as far as ignored files go. As
it is, git status does NOT report A/B/C as an untracked file, but git
clean will remove the entire A dir. It seems to me that one or the
other’s behavior ought to be tweaked. Which one is correct I am not
sure.

This happens on 2.5.1 as well as 2.9.2

Chris

Re: [PATCH] name-hash: add test-lazy-init-name-hash to .gitignore

[Ramsay Jones]

On 24/03/17 17:26, Ramsay Jones wrote:
> 
> Signed-off-by: Ramsay Jones <ram...@ramsayjones.plus.com>
> ---
> 
> Hi Jeff,
> 
> If you need to re-roll your 'jh/memihash-opt' branch, could you please
> squash this into the relevant patch (commit f25dde4fbf, "name-hash: add
> test-lazy-init-name-hash", 23-03-2017).
> 
> Thanks!
> 
> ATB,
> Ramsay Jones
> 
>  t/helper/.gitignore | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/t/helper/.gitignore b/t/helper/.gitignore
> index 5f68aa8f8..57bdd4b8e 100644
> --- a/t/helper/.gitignore
> +++ b/t/helper/.gitignore
> @@ -12,6 +12,7 @@
>  /test-hashmap
>  /test-index-version
>  /test-line-buffer
> +/test-lazy-init-name-hash

Heh, you could even put it before the '/test-line-buffer' entry! ;-)

Ahem.

ATB,
Ramsay Jones

>  /test-match-trees
>  /test-mergesort
>  /test-mktemp
> 

[PATCH] name-hash: add test-lazy-init-name-hash to .gitignore

[Ramsay Jones]

Signed-off-by: Ramsay Jones <ram...@ramsayjones.plus.com>
---

Hi Jeff,

If you need to re-roll your 'jh/memihash-opt' branch, could you please
squash this into the relevant patch (commit f25dde4fbf, "name-hash: add
test-lazy-init-name-hash", 23-03-2017).

Thanks!

ATB,
Ramsay Jones

 t/helper/.gitignore | 1 +
 1 file changed, 1 insertion(+)

diff --git a/t/helper/.gitignore b/t/helper/.gitignore
index 5f68aa8f8..57bdd4b8e 100644
--- a/t/helper/.gitignore
+++ b/t/helper/.gitignore
@@ -12,6 +12,7 @@
 /test-hashmap
 /test-index-version
 /test-line-buffer
+/test-lazy-init-name-hash
 /test-match-trees
 /test-mergesort
 /test-mktemp
-- 
2.12.0

Re: Bug with .gitignore and branch switching

[Nevada Sanchez]

On Fri, Mar 17, 2017 at 5:23 PM, Junio C Hamano <gits...@pobox.com> wrote:
> Nevada Sanchez <sanchez.nev...@gmail.com> writes:
>
>> Here's an easy to reproduce bug. It's the only example I know of where
>> git legitimately loses data in a way that is unrecoverable,
>> unexpected, and without warning.
>
> This is an example of a user explicitly telling git to discard data
> and git performing as it is told.
>
> There is no "untracked but precious" vs "untracked and expendable"
> difference in the current system.  An untracked file that matches
> patterns listed in .gitignore is treated as the latter.
>
> When you have an untracked file that .gitignore knows about in the
> working tree while you are on "feature", if switching to another
> branch requires to remove that file, the content there is deemed
> expendable, because the user said so by listing it in .gitignore.
>
> We've discussed the lack of "untracked but precious" class a few
> times on the list in the past, but I do not recall the topic came up
> in the recent past.  It perhaps is because nobody found that class
> useful enough so far.

I must admit that I missed that attribute of .gitignore (i.e.
untracked and **expendable**). I have grown accustomed to Git being
rather conservative and erring on the side of not losing data unless
the user is doing something deliberate (for example, 'git clean' won't
work unless you force it, checkouts fail if they do anything that
might lose data... unless it is in .gitignore, as I just learned).
When I saw this behavior, I assumed that it was a bug.

This isn't necessarily a situation I need to have fixed--it is not
part of my workflow and since that fateful commit, all feature
branches checked out after the change to .gitignore will not have any
problems as I switch branches. It was an unfortunate surprise to one
of my co-workers, not long after I reassured him that git was
conservative and will almost never accidentally lose data (even if it
means going to 'git reflog').

In keeping with this spirit, I would tend to lean towards having
"untracked but precious" being the default behavior (more
conservative), and if a user wants "untracked but expendable"
behavior, then that case requires special effort from the user (like
learning about and using a new type of ignore file). My guess is that
if the user is both ignoring and committing something to their
repository, it is probably a mistake, and as that user, I would rather
discover that mistake early with loud warning messages (and/or a
suggestion to use an alternate ignore strategy, or config flag), than
learn about it by losing data.

In summary, I do not need this fixed for my workflow, but want to
bring it to light in case other users are being similarly surprised. I
struggle to guess how far reaching of an impact it would have on
existing users to change the default behavior, but it would probably
be less than that of the push default behavior change that happened
not too long ago.

A quick and easy immediate step is to make note of this behavior in
the very first sentence of gitignore(5):

> A gitignore file specifies intentionally untracked files that Git should 
> ignore *and that Git is allowed to overwrite without warning*.

More details about untracked but expendable can be placed in the NOTES
section, but the last part of that sentence would be quite helpful.

Thank you,
-Nevada

Re: Bug with .gitignore and branch switching

[Duy Nguyen]

On Sat, Mar 18, 2017 at 5:02 AM, Jonathan Nieder <jrnie...@gmail.com> wrote:
> Junio C Hamano wrote:
>
>> There is no "untracked but precious" vs "untracked and expendable"
>> difference in the current system.  An untracked file that matches
>> patterns listed in .gitignore is treated as the latter.
> [...]
>> We've discussed the lack of "untracked but precious" class a few
>> times on the list in the past, but I do not recall the topic came up
>> in the recent past.  It perhaps is because nobody found that class
>> useful enough so far.
>
> The most recent example I can find is 2010:
> http://public-inbox.org/git/4c6a1c5b.4030...@workspacewhiz.com/.
>
> It also came up in 2007:
> http://public-inbox.org/git/c0e9f681e68d48eb8989022d11fee...@ntdev.corp.microsoft.com/
> Earlier in that year it even made the "What's not in 1.5.2" list.
> http://public-inbox.org/git/11793556383977-git-send-email-jun...@cox.net/
>
> Perhaps those references could be a useful starting point for an
> interested person's thinking.

I think I made it work in 2014 [1] using new "precious" attribute, but
never submitted it, probably because I was worried about the
interaction with untracked cache (adding .gitattributes as a new
dependency) though maybe we can avoid that by always checking for
preciousness after all the tree walking/filtering is done, either with
or without untracked cache. But I never addressed that loose end. Then
again, it could also be another useful starting point for interested
person's thinking ;-)

[1] 
https://github.com/pclouds/git/commit/0e7f7afa1879b055369ebd3f1224311c43c8a32b
-- 
Duy

Re: Bug with .gitignore and branch switching

[Junio C Hamano]

Jonathan Nieder  writes:

> The most recent example I can find is 2010:
> http://public-inbox.org/git/4c6a1c5b.4030...@workspacewhiz.com/.
>
> It also came up in 2007:
> http://public-inbox.org/git/c0e9f681e68d48eb8989022d11fee...@ntdev.corp.microsoft.com/
> Earlier in that year it even made the "What's not in 1.5.2" list.
> http://public-inbox.org/git/11793556383977-git-send-email-jun...@cox.net/
>
> Perhaps those references could be a useful starting point for an
> interested person's thinking.

Thanks for links.  It seems that my thinking back in 1.5.3 timeperiod
was to introduce "precious" attribute.

I noticed that among the four-message "What's not in 1.5.2" series,
3/4 has a large discussion that may be relevant to Brandon's
"submodule is-active" thing.

Re: Bug with .gitignore and branch switching

[Stefan Beller]

On Fri, Mar 17, 2017 at 2:23 PM, Junio C Hamano  wrote:
> We've discussed the lack of "untracked but precious" class a few
> times on the list in the past, but I do not recall the topic came up
> in the recent past.  It perhaps is because nobody found that class
> useful enough so far.

My gut reaction on reading the bug report was that the root cause is
git-checkout doing the wrong thing by default. (cf. Git-Merge-2017,
"What’s Wrong With Git?", I am not sure if the video is yet available)

One argument in that talk was that Git promises to do "work on multiple
branches in parallel (context-switched, single threaded)", and git-checkout
is the apparent command to switch to another context (branch).
However by putting away only tracked content, we miss
doing a proper context switch for untracked and ignored files.

That partial switch has advantages in the typical use case, e.g.
* compiled objects in the worktree may not need to be recompiled.
* no need to do work for the untracked files (e.g. move to a special
  location).

Both these reasons argue for performance, instead of "correctness"
in the sense of "easy-to-understand commands for top level principles".

And in that talk the presenter concluded that git-stash was only invented
to circumvent these "correctness" problems, such that if git-checkout
were to also (de)populate the untracked and ignored files on branch
switch we would not need git-stash, because git-checkout did it for you
already. And by the omission of git-stash and an apparent
easier-to-understand git-checkout the whole git suite would become
easier for users.

I further conclude that when git-checkout were to behave "correct" as
outlined above, then this class of bug reports would not occur.

Just food for thought.

Thanks,
Stefan

Re: Bug with .gitignore and branch switching

[Jonathan Nieder]

Junio C Hamano wrote:

> There is no "untracked but precious" vs "untracked and expendable"
> difference in the current system.  An untracked file that matches
> patterns listed in .gitignore is treated as the latter.
[...]
> We've discussed the lack of "untracked but precious" class a few
> times on the list in the past, but I do not recall the topic came up
> in the recent past.  It perhaps is because nobody found that class
> useful enough so far.

The most recent example I can find is 2010:
http://public-inbox.org/git/4c6a1c5b.4030...@workspacewhiz.com/.

It also came up in 2007:
http://public-inbox.org/git/c0e9f681e68d48eb8989022d11fee...@ntdev.corp.microsoft.com/
Earlier in that year it even made the "What's not in 1.5.2" list.
http://public-inbox.org/git/11793556383977-git-send-email-jun...@cox.net/

Perhaps those references could be a useful starting point for an
interested person's thinking.

Jonathan

Re: Bug with .gitignore and branch switching

[Jonathan Nieder]

Hi Nevada,

Nevada Sanchez wrote:

> # Commit a file that will end up in .gitignore
> echo 'original settings' > mine.conf
> git add mine.conf
> git commit -m "Unknowingly committed my settings."
>
> echo '*.conf' > .gitignore
> git add .gitignore
> git commit -m "Users shouldn't commit their settings"

Naming a file in .gitignore tells git that you do not want to track it
and are giving git permission to write over it.  This commonly happens
when people check in build products.  For example:

git rm -f my-build-product
echo /my-build-product >>.gitignore
git commit -m "Remove generated my-build-product file"
make my-build-product

git checkout HEAD^

Without that rule, this 'git checkout' command would fail.

That said, there are some cases (e.g. the .conf file case you mention)
where a person would want git not to track a file but do not want to
give git permission to write over it.  As you've seen, .gitignore does
not work well for this. :/

Ideas for next steps:

 1. The gitignore(5) manpage does not do a good job of emphasizing
that files named there are not precious and can be overwritten by
git.  Do you have ideas for wording that would help with that?
This would be especially welcome if you can phrase them in the
form of a patch against Documentation/gitignore.txt.

 2. Occasionally people have mentioned the idea of a .gitprecious file
listing precious files that git should not track and not overwrite
(e.g., keys and other configuration files, IDE state, or metadata
for another version control system being used in parallel).  Would
you be interested in working on that?

Thanks and hope that helps,
Jonathan

Re: Bug with .gitignore and branch switching

[Junio C Hamano]

Nevada Sanchez <sanchez.nev...@gmail.com> writes:

> Here's an easy to reproduce bug. It's the only example I know of where
> git legitimately loses data in a way that is unrecoverable,
> unexpected, and without warning.

This is an example of a user explicitly telling git to discard data
and git performing as it is told.

There is no "untracked but precious" vs "untracked and expendable"
difference in the current system.  An untracked file that matches
patterns listed in .gitignore is treated as the latter.

When you have an untracked file that .gitignore knows about in the
working tree while you are on "feature", if switching to another
branch requires to remove that file, the content there is deemed
expendable, because the user said so by listing it in .gitignore.

We've discussed the lack of "untracked but precious" class a few
times on the list in the past, but I do not recall the topic came up
in the recent past.  It perhaps is because nobody found that class
useful enough so far.

Bug with .gitignore and branch switching

[Nevada Sanchez]

Here's an easy to reproduce bug. It's the only example I know of where
git legitimately loses data in a way that is unrecoverable,
unexpected, and without warning.

```
git --version
# git version 2.12.0

mkdir git-demo
cd git-demo

git init

# Commit a file that will end up in .gitignore
echo 'original settings' > mine.conf
git add mine.conf
git commit -m "Unknowingly committed my settings."

echo '*.conf' > .gitignore
git add .gitignore
git commit -m "Users shouldn't commit their settings"

# Spin off a feature branch here (but don't check it out)
git branch feature

# Realize that we don't want that file committed
git rm mine.conf
git commit -m "Delete mine.conf"

echo 'Lots of laboriously tuned settings' > mine.conf

# Hop on the feature branch to do some work
git checkout feature

# Hmmm... My settings are gone
cat mine.conf
# original settings

# Lemme hop back
git checkout master

# Wait... they are gone for good!
cat mine.conf
# cat: mine.conf: No such file or directory
```

Re: Automatically Add .gitignore Files

[Thangalin]

Hi Duy,

> This is a general problem to new files, not .gitignore alone. Can we

The difference, to me, is that a ".gitignore" file is not part of what
I'm developing. It's an artifact for git configuration. While a
.gitignore file is not always pushed to the repository, I imagine that
in most situations, it is.

Whereas when a "new" file is created, there are plenty of situations
where it shouldn't be added and thus a warning would be superfluous,
or an automatic add would be undesirable.

To solve the problem, generally, for new files while giving the user
the ability to specify exactly what "new" files should be
automatically added to the repository, something like the following
would work:

echo "**/.gitignore" >> .git/config/add-before-commit

> and perhaps you want to make it a habit to run it before committing.

Right, because software shouldn't automate repetitive tasks and humans
are never prone to forget? ;-)

Re: Automatically Add .gitignore Files

[Duy Nguyen]

On Thu, Feb 9, 2017 at 2:05 AM, Thangalin <thanga...@gmail.com> wrote:
> I frequently forget to add .gitignore files when creating new .gitignore 
> files.
>
> I'd like to request a command-line option to always add .gitignore
> files (or, more generally, always add files that match a given file
> specification).
>
> Replicate
>
> 0. git init ...
> 1. echo "*.bak" >> .gitignore
> 2. touch file.txt
> 3. git add file.txt
> 4. git commit -a -m "..."
> 5. git push origin master
>
> Expected Results
>
> The .gitignore file is also added to the repository. (This is probably
> the 80% use case.)

This is a general problem to new files, not .gitignore alone. Can we
accomplish something with some hook? At the least I think we should be
able to detect that .gitignore is not detected and abort, prompting
the user to add it. It's easier to customize too, and we don't have to
cook ".gitignore" in the code.

I'm not sure if we tell the hook "this is with -m option" though..
-- 
Duy

Re: Automatically Add .gitignore Files

[Stephan Beyer]

> I am tempted to say that there should be a way to somehow forbid use
> of the "-m" option to "git commit" by default, until the user gains
> more familiarity with use of Git.

Since I am using git, I am tempted to say that "git commit -m" should be
abolished. If I tell somebody how to use git, I never mention "git
commit -m". Unluckily they find it out themselves... ;D

The typical observations I have when people use "git commit -m":

 * The commit messages are either too long (in one line!) or
   too meaningless.

 * People don't check what they added and commit wrong stuff.

 * People make fast temporary commits with "asdafsd" commit messages.
   Then they get distracted for some time and work on another branch.
   When they turn back to the old branch they don't know what "asdafsd"
   was...

Thanks to git rebase -i and/or git commit --amend, it is all not too bad
after all ;D

Best
  Stephan

Re: Automatically Add .gitignore Files

[Junio C Hamano]

Thangalin <thanga...@gmail.com> writes:

> I frequently forget to add .gitignore files when creating new .gitignore 
> files.
>
> I'd like to request a command-line option to always add .gitignore
> files (or, more generally, always add files that match a given file
> specification).

That is essentially what "Untracked files" listing in the editor
buffer given to you when you type "git commit" is about.  By not
saying you ignore .gitattributes, .gitignore, .gitmodules, etc., you
are reminded if you forgot to add them.  "git status" does the same,
and perhaps you want to make it a habit to run it before committing.

I am tempted to say that there should be a way to somehow forbid use
of the "-m" option to "git commit" by default, until the user gains
more familiarity with use of Git.  That way, they will learn to pay
more attention to the "Untracked" and "Changes not staged" sections
;-)

Automatically Add .gitignore Files

[Thangalin]

I frequently forget to add .gitignore files when creating new .gitignore files.

I'd like to request a command-line option to always add .gitignore
files (or, more generally, always add files that match a given file
specification).

Replicate

0. git init ...
1. echo "*.bak" >> .gitignore
2. touch file.txt
3. git add file.txt
4. git commit -a -m "..."
5. git push origin master

Expected Results

The .gitignore file is also added to the repository. (This is probably
the 80% use case.)

Actual Results

The .gitignore file is not added to the repository.

Additional Details

At step 4, there should be a prompt, warning, or (preferably) either a
command-line or configuration option to add the .gitignore file prior
to commit, automatically. Such as:

git commit --include-all-gitignore-files -a -m "..."

Or:

echo "**/.gitignore" >> .git/config/add-before-commit

Re: [PATCH 0/5] disallow symlinks for .gitignore and .gitattributes

[Jeff King]

On Wed, Nov 02, 2016 at 12:46:28PM -0700, Stefan Beller wrote:

> On Wed, Nov 2, 2016 at 6:04 AM, Jeff King  wrote:
> 
> >
> >  attr.c| 58 
> > ---
> 
> $ git diff --stat origin/master..origin/sb/attr  |grep attr.c
>  attr.c |  531 +-
> 
> From a cursory read of your series this may result in a merge
> conflict, but would be
> easily fixable (changed signature of functions that clash).

Yeah, I knew you guys were doing some refactoring of the attribute code
elsewhere, but hadn't actually seen how bad the damage was. I just did
the merge with sb/attr, though, and the conflicts are quite trivial
(mostly s/1/flags/ in bootstrap_attr_stack()).

I'm happy to re-roll on a different base if sb/attr graduates first, but
I suspect Junio can just resolve the conflicts at merge time.

-Peff

Re: [PATCH 0/5] disallow symlinks for .gitignore and .gitattributes

[Stefan Beller]

On Wed, Nov 2, 2016 at 6:04 AM, Jeff King  wrote:

>
>  attr.c| 58 
> ---

$ git diff --stat origin/master..origin/sb/attr  |grep attr.c
 attr.c |  531 +-

>From a cursory read of your series this may result in a merge
conflict, but would be
easily fixable (changed signature of functions that clash).

[PATCH 5/5] exclude: do not respect symlinks for in-tree .gitignore

[Jeff King]

Like .gitattributes, we would like to make sure that
.gitignore files are handled consistently whether read from
the index or from the filesystem. We can do so by using
O_NOFOLLOW when opening the files.

Signed-off-by: Jeff King <p...@peff.net>
---
 dir.c  |  9 +++--
 t/t0008-ignores.sh | 29 +
 2 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/dir.c b/dir.c
index 4fa1ca109..cf3fde005 100644
--- a/dir.c
+++ b/dir.c
@@ -693,6 +693,7 @@ static void invalidate_directory(struct untracked_cache *uc,
 
 /* Flags for add_excludes() */
 #define EXCLUDE_CHECK_INDEX (1<<0)
+#define EXCLUDE_NOFOLLOW (1<<1)
 
 /*
  * Given a file with name "fname", read it (either from disk, or from
@@ -713,7 +714,11 @@ static int add_excludes(const char *fname, const char 
*base, int baselen,
size_t size = 0;
char *buf, *entry;
 
-   fd = open(fname, O_RDONLY);
+   if (flags & EXCLUDE_NOFOLLOW)
+   fd = open_nofollow(fname, O_RDONLY);
+   else
+   fd = open(fname, O_RDONLY);
+
if (fd < 0 || fstat(fd, ) < 0) {
if (errno != ENOENT)
warn_on_inaccessible(fname);
@@ -1130,7 +1135,7 @@ static void prep_exclude(struct dir_struct *dir, const 
char *base, int baselen)
strbuf_addstr(, dir->exclude_per_dir);
el->src = strbuf_detach(, NULL);
add_excludes(el->src, el->src, stk->baselen, el,
-EXCLUDE_CHECK_INDEX,
+EXCLUDE_CHECK_INDEX | EXCLUDE_NOFOLLOW,
 untracked ? _stat : NULL);
}
/*
diff --git a/t/t0008-ignores.sh b/t/t0008-ignores.sh
index d27f438bf..7348b8e6a 100755
--- a/t/t0008-ignores.sh
+++ b/t/t0008-ignores.sh
@@ -841,4 +841,33 @@ test_expect_success 'info/exclude trumps 
core.excludesfile' '
test_cmp expect actual
 '
 
+test_expect_success SYMLINKS 'set up ignore file for symlink tests' '
+   echo "*" >ignore
+'
+
+test_expect_success SYMLINKS 'symlinks respected in core.excludesFile' '
+   test_when_finished "rm symlink" &&
+   ln -s ignore symlink &&
+   test_config core.excludesFile "$(pwd)/symlink" &&
+   echo file >expect &&
+   git check-ignore file >actual &&
+   test_cmp expect actual
+'
+
+test_expect_success SYMLINKS 'symlinks respected in info/exclude' '
+   test_when_finished "rm .git/info/exclude" &&
+   ln -sf ../../ignore .git/info/exclude &&
+   echo file >expect &&
+   git check-ignore file >actual &&
+   test_cmp expect actual
+'
+
+test_expect_success SYMLINKS 'symlinks not respected in-tree' '
+   test_when_finished "rm .gitignore" &&
+   ln -sf ignore .gitignore &&
+   >expect &&
+   test_must_fail git check-ignore file >actual &&
+   test_cmp expect actual
+'
+
 test_done
-- 
2.11.0.rc0.258.gf434c15

[PATCH 0/5] disallow symlinks for .gitignore and .gitattributes

[Jeff King]

I noticed in a nearby discussion that we will follow in-filesystem
symlinks for in-tree .gitignore and .gitattributes files, but not when
those files are read out of the index (e.g., when switching branches).

This series teaches git to open those files with O_NOFOLLOW (when it is
available) to get more consistent behavior. Note that this only applies
to the in-tree versions; you can still symlink $GIT_DIR/info/attributes,
etc.

I stopped short of warning about symlinked entries in git-fsck, but
perhaps we would want to do that as well (doing it completely is tricky
because of all of the case-folding issues around matching pathnames).

  [1/5]: add open_nofollow() helper
  [2/5]: attr: convert "macro_ok" into a flags field
  [3/5]: exclude: convert "check_index" into a flags field
  [4/5]: attr: do not respect symlinks for in-tree .gitattributes
  [5/5]: exclude: do not respect symlinks for in-tree .gitignore

 attr.c| 58 ---
 dir.c | 20 +-
 dir.h |  2 +-
 git-compat-util.h |  3 +++
 t/t0003-attributes.sh | 31 +++
 t/t0008-ignores.sh| 29 ++
 wrapper.c |  8 +++
 7 files changed, 123 insertions(+), 28 deletions(-)

-Peff

Re: Expanding Includes in .gitignore

[Jeff King]

On Fri, Oct 28, 2016 at 10:32:07PM +1300, Aaron Pelly wrote:

> On 28/10/16 15:54, Junio C Hamano wrote:
> > Jeff King <p...@peff.net> writes:
> > 
> >> However, as I said elsewhere, I'm not convinced this feature is all that
> >> helpful for in-repository .gitignore files, and I think it does
> >> introduce compatibility complications. People with older git will not
> >> respect your .gitignore.d files. Whereas $GIT_DIR/info is purely a local
> >> matter.
> > 
> > As I do not see the point of making in-tree .gitignore to a forest
> > of .gitignore.d/ at all, compatibility complications is not worth
> > even thinking about, I would have to say.
> 
> Well; that saves some work. :)
> 
> I do not suggesting making this mandatory. I think it adds value and it
> is a common and understood mechanism. But, if it is abhorrent, consider:
> 
> There is precedent for including files in git-config. This could be
> extended to ignore files. The code is not similar, but the concept is. I
> could live with it.

Yes, but note that we don't have in-tree config files, either (to large
degree because of the security implications).

Perhaps Junio can clarify himself, but I took his statement to mean only
that in-tree .gitignore.d is not worth worrying about, but that
$GIT_DIR/info/exclude.d or similar would be OK (but perhaps I
interpreted that way because that's my own position :) ).

> Or how about a new githook that can intelligently create or return the
> details? This would be my least favourite option unless it was
> configured in an obvious place.

That seems more complicated than is really merited, and probably doesn't
perform great either (it's an extra forked process for almost every git
operation). And obviously would not work for an in-tree solution anyway,
as we do not want to run arbitrary code.

-Peff

Re: Expanding Includes in .gitignore

[Duy Nguyen]

On Fri, Oct 28, 2016 at 4:32 PM, Aaron Pelly <aa...@pelly.co> wrote:
> Or how about a new githook that can intelligently create or return the
> details? This would be my least favourite option unless it was
> configured in an obvious place.

I wonder if smudge/clean filters can be used to recreate in-tree
.gitignore from .gitignore.d/*.
-- 
Duy

Re: Expanding Includes in .gitignore

[Duy Nguyen]

On Fri, Oct 28, 2016 at 4:04 AM, Jeff King  wrote:
> On Thu, Oct 27, 2016 at 12:48:34PM -0700, Jacob Keller wrote:
>
>> > I think the normal behavior in such "foo.d" directory is to just sort
>> > the contents lexically and read them in order, as if they were all
>> > concatenated together, and with no recursion. I.e., behave "as if" the
>> > user had run "cat $dir/*".
>>
>> Yea, this is the normal behavior, and the user is expected to order
>> their files lexically such as "00-name", "50-name" and so on. Pretty
>> traditional for a lot of newer configurations.
>
> One thing I will say about this approach is that you can implement it
> without any changes in git by doing:
>
>   path=.git/info/exclude
>   cat $path.d/* >$path
>
> and I have seen several config mechanisms basically do that (e.g.,
> Debian packaging for a program that doesn't have its own ".d" mechanism,
> but needs to grab config provided by several separate packages).

My first thought at this .git/info/exclude.d was "oh no I have to
teach untracked cache about new dependencies, or at least disable it
until it can deal with exclude.d", but this "cat" approach simplifies
things and should keep untracked cache unchanged.

There may be complication with negative patterns though. The user may
want to limit the effect of negative patterns within individual
exclude files in exclude.d so a negative pattern in exclude.d/a won't
influence anything in exclude.d/b (easier to reason, safer to compose
different exclude sets). The plain "cat" would lose file boundary info
that we need. I'm not sure. But I'll dig more into it when patches
show up.
-- 
Duy

Re: Expanding Includes in .gitignore

[Aaron Pelly]

On 28/10/16 15:54, Junio C Hamano wrote:
> Jeff King <p...@peff.net> writes:
> 
>> However, as I said elsewhere, I'm not convinced this feature is all that
>> helpful for in-repository .gitignore files, and I think it does
>> introduce compatibility complications. People with older git will not
>> respect your .gitignore.d files. Whereas $GIT_DIR/info is purely a local
>> matter.
> 
> As I do not see the point of making in-tree .gitignore to a forest
> of .gitignore.d/ at all, compatibility complications is not worth
> even thinking about, I would have to say.

Well; that saves some work. :)

I do not suggesting making this mandatory. I think it adds value and it
is a common and understood mechanism. But, if it is abhorrent, consider:

There is precedent for including files in git-config. This could be
extended to ignore files. The code is not similar, but the concept is. I
could live with it.

Or how about a new githook that can intelligently create or return the
details? This would be my least favourite option unless it was
configured in an obvious place.

Finally, if this is a bad-idea, as I asked in the beginning, I will
consider the equine expired, cease flagellation and apologise for the noise.

Re: Expanding Includes in .gitignore

[Jeff King]

On Fri, Oct 28, 2016 at 11:17:26AM +1300, Aaron Pelly wrote:

> On 28/10/16 10:55, Aaron Pelly wrote:
> > 2) I fetch a repo with a hostile ignore file. It includes files from
> > $GIT_DIR/test-data/ssl/private or some such. Change. Don't pay
> > attention. Commit. Push. Problems if my test data comes from production.
> > 
> > Is this mitigated currently?
> > 
> > Not that git should be an enabler, but surely it falls on the user of
> > untrusted software to ensure their own security?
> 
> Balls, I meant $GIT_WORK_TREE not $GIT_DIR

I was going to ask what you meant by "currently" here, as we do not yet
have an include mechanism, and generally things in the repository are
fair game. But I guess you mean that there could be untracked files even
inside the repository.

I'm not too worried about that in general. You have to be careful of a
lot of things inside the repository, like running "make" on malicious
code. Adding "don't stick secret files inside the repository, even
untracked", does seem like another sensible precaution.

The main thing with malicious repositories is that basic inspection like
"git clone $remote && git log" should not execute arbitrary code, leak
information, etc.

-Peff

Re: Expanding Includes in .gitignore

[Junio C Hamano]

Jeff King <p...@peff.net> writes:

> However, as I said elsewhere, I'm not convinced this feature is all that
> helpful for in-repository .gitignore files, and I think it does
> introduce compatibility complications. People with older git will not
> respect your .gitignore.d files. Whereas $GIT_DIR/info is purely a local
> matter.

As I do not see the point of making in-tree .gitignore to a forest
of .gitignore.d/ at all, compatibility complications is not worth
even thinking about, I would have to say.

Thanks.

Re: Expanding Includes in .gitignore

[Aaron Pelly]

On 28/10/16 10:07, Jeff King wrote:
> I think it does
> introduce compatibility complications.

If this is not a show stopper, I am happy to knock out a short
functional spec. I'll give it some hours before I begin.

Re: Expanding Includes in .gitignore

[Aaron Pelly]

On 28/10/16 10:07, Jeff King wrote:
> I think it does
> introduce compatibility complications. People with older git will not
> respect your .gitignore.d files. Whereas $GIT_DIR/info is purely a local
> matter.

I know. I don't think it's a serious compatibility issue, but more
important people may disagree, in which case I'll have to review my
stance. I think the pros of it working everywhere outweigh the cons.

I will say that I am a proponent of consistency and obviousness. I would
rather this worked the same everywhere or it was a completely novel
component in $GIT_DIR.

I don't think I like the idea much, but it would be possible to
magically generate an ignore file from a directory.

> But perhaps there is a use case I'm missing.

A new repo where the dev tools haven't been standardised yet. New tool,
new file. Simple, and somewhat self documenting as a table of contents
in .gitignore.d

Re: Expanding Includes in .gitignore

[Aaron Pelly]

On 28/10/16 10:55, Aaron Pelly wrote:
> 2) I fetch a repo with a hostile ignore file. It includes files from
> $GIT_DIR/test-data/ssl/private or some such. Change. Don't pay
> attention. Commit. Push. Problems if my test data comes from production.
> 
> Is this mitigated currently?
> 
> Not that git should be an enabler, but surely it falls on the user of
> untrusted software to ensure their own security?

Balls, I meant $GIT_WORK_TREE not $GIT_DIR

Re: Expanding Includes in .gitignore

[Aaron Pelly]

On 28/10/16 09:55, Jeff King wrote:
> On Fri, Oct 28, 2016 at 09:28:23AM +1300, Aaron Pelly wrote:
> 
>>>   - we parse possibly-hostile .gitignore files from cloned repositories.
>>> What happens when I include ask to include /etc/passwd? Probably
>>> nothing, but there are setups where it might matter (e.g., something
>>> like Travis that auto-builds untrusted repositories, and you could
>>> potentially leak the contents of files via error messages). It's
>>> nice to avoid the issue entirely.
>>
>> I understand the issue.
>>
>> It's not obvious to me how using a .d solves this problem though.
> 
> It doesn't by itself. But we are worried only about tracked .gitignore
> files (recall that even repo-level files in $GIT_DIR/info are generated
> fresh by the clone process, and don't come from the remote). If we apply
> the feature only to core.excludeFile and $GIT_DIR/info/exclude, those
> are already under the user's control.

The things you say make sense from this perspective.

I was hoping to employ this mechanism throughout the git ecosystem.

Thinking out loud for a minute:

1) I clone a repo with a hostile ignore file. It includes files from
/etc/ssl/private or some such. Change. Don't pay attention. Commit.
Push. Problems.

What is the use case for reaching out of the repo in the first place?

2) I fetch a repo with a hostile ignore file. It includes files from
$GIT_DIR/test-data/ssl/private or some such. Change. Don't pay
attention. Commit. Push. Problems if my test data comes from production.

Is this mitigated currently?

Not that git should be an enabler, but surely it falls on the user of
untrusted software to ensure their own security?

> It's true that we could make a similar exception for an "include"
> feature, and respect include directives only in those "safe" files.
> Somehow that seems more confusing to me, though, than doing adding the
> feature at the file level, as it introduces slightly varying syntax
> between the locations.

I'm quickly getting over the include file idea. But yes, that would be
non obvious.

>>> Whereas letting any of the user- or repo-level exclude files be a
>>> directory, and simply reading all of the files inside, seems simple and
>>> obvious.
>>
>> Apart from backwards compatibility, unless there's something I'm missing.
> 
> I'm not sure what you mean. If we make:
> 
>   mkdir .git/info/exclude
>   echo whatever >.git/info/exclude/local
> 
> work, I don't think we have to care about backwards compatibility. That
> was nonsensical before, and never did anything useful (so somebody might
> have done it, but we can assume anybody relying on it not to read the
> contents is crazy).

Seeing your perspective, now, I can see why you didn't understand me. In
your context this makes perfect sense.

Re: Expanding Includes in .gitignore

[Jacob Keller]

On Thu, Oct 27, 2016 at 2:04 PM, Jeff King <p...@peff.net> wrote:
> I'm not convinced this is needed for in-repo .gitignore files. The point
> is that you are pulling together separate files that may be administered
> independently. But git repositories inherently have a whole-project
> view. I'm not sure that separate files buy you a lot there. And the
> compatibility issues are more complicated.
>

I had just assumed it would be used everywhere, so I hadn't really
considered whether that was any reason not to. I personally like the
idea of splitting my git ignores into separate files just so each file
can be about one thing, but I guess it's not really a problem either
way.

> I do agree that:
>
>   cd .git/info
>   git clone /my/exclude/repo exclude ;# or exclude.d
>
> should work; ignoring dotfiles when reading the directory solves that,
> and is a pretty standard solution.
>
> -Peff

Re: Expanding Includes in .gitignore

[Aaron Pelly]

On 28/10/16 10:04, Jeff King wrote:
> On Thu, Oct 27, 2016 at 12:48:34PM -0700, Jacob Keller wrote:
> 
>>> I think the normal behavior in such "foo.d" directory is to just sort
>>> the contents lexically and read them in order, as if they were all
>>> concatenated together, and with no recursion. I.e., behave "as if" the
>>> user had run "cat $dir/*".
>>
>> Yea, this is the normal behavior, and the user is expected to order
>> their files lexically such as "00-name", "50-name" and so on. Pretty
>> traditional for a lot of newer configurations.
> 
> One thing I will say about this approach is that you can implement it
> without any changes in git by doing:
> 
>   path=.git/info/exclude
>   cat $path.d/* >$path
> 
> and I have seen several config mechanisms basically do that (e.g.,
> Debian packaging for a program that doesn't have its own ".d" mechanism,
> but needs to grab config provided by several separate packages).
> 
> The reason to teach that trick to git is convenience; you don't have to
> remember to build the master file from its parts because it's done
> dynamically whenever git needs to look at it.

Precisely.

>> One thing to keep in mind would be that we should make sure we can
>> handle the .gitignore being a submodule or a git repository, so that
>> users could just do something like
> 
> I'm not convinced this is needed for in-repo .gitignore files. The point
> is that you are pulling together separate files that may be administered
> independently. But git repositories inherently have a whole-project
> view. I'm not sure that separate files buy you a lot there. And the
> compatibility issues are more complicated.
> 
> I do agree that:
> 
>   cd .git/info
>   git clone /my/exclude/repo exclude ;# or exclude.d
> 
> should work; ignoring dotfiles when reading the directory solves that,
> and is a pretty standard solution.

You raise a good point about the requirement. I was about to make an
argument in favour of it, but I argued myself out of an argument. I will
say; why have it work in one place and not others?

Re: Expanding Includes in .gitignore

[Jeff King]

On Thu, Oct 27, 2016 at 04:55:08PM -0400, Jeff King wrote:

> On Fri, Oct 28, 2016 at 09:28:23AM +1300, Aaron Pelly wrote:
> 
> > >   - we parse possibly-hostile .gitignore files from cloned repositories.
> > > What happens when I include ask to include /etc/passwd? Probably
> > > nothing, but there are setups where it might matter (e.g., something
> > > like Travis that auto-builds untrusted repositories, and you could
> > > potentially leak the contents of files via error messages). It's
> > > nice to avoid the issue entirely.
> > 
> > I understand the issue.
> > 
> > It's not obvious to me how using a .d solves this problem though.
> 
> It doesn't by itself. But we are worried only about tracked .gitignore
> files (recall that even repo-level files in $GIT_DIR/info are generated
> fresh by the clone process, and don't come from the remote). If we apply
> the feature only to core.excludeFile and $GIT_DIR/info/exclude, those
> are already under the user's control.
> 
> It's true that we could make a similar exception for an "include"
> feature, and respect include directives only in those "safe" files.
> Somehow that seems more confusing to me, though, than doing adding the
> feature at the file level, as it introduces slightly varying syntax
> between the locations.

Actually, I suppose even a ".gitignore.d" inside the repo solves that
problem, too, because the repository is not specifying paths, only
content (it can specify symlinks outside the repository, but like other
parts of git, we should be careful not to follow them in that case).

However, as I said elsewhere, I'm not convinced this feature is all that
helpful for in-repository .gitignore files, and I think it does
introduce compatibility complications. People with older git will not
respect your .gitignore.d files. Whereas $GIT_DIR/info is purely a local
matter.

But perhaps there is a use case I'm missing.

-Peff

Re: Expanding Includes in .gitignore

[Jeff King]

On Thu, Oct 27, 2016 at 12:48:34PM -0700, Jacob Keller wrote:

> > I think the normal behavior in such "foo.d" directory is to just sort
> > the contents lexically and read them in order, as if they were all
> > concatenated together, and with no recursion. I.e., behave "as if" the
> > user had run "cat $dir/*".
> 
> Yea, this is the normal behavior, and the user is expected to order
> their files lexically such as "00-name", "50-name" and so on. Pretty
> traditional for a lot of newer configurations.

One thing I will say about this approach is that you can implement it
without any changes in git by doing:

  path=.git/info/exclude
  cat $path.d/* >$path

and I have seen several config mechanisms basically do that (e.g.,
Debian packaging for a program that doesn't have its own ".d" mechanism,
but needs to grab config provided by several separate packages).

The reason to teach that trick to git is convenience; you don't have to
remember to build the master file from its parts because it's done
dynamically whenever git needs to look at it.

> One thing to keep in mind would be that we should make sure we can
> handle the .gitignore being a submodule or a git repository, so that
> users could just do something like

I'm not convinced this is needed for in-repo .gitignore files. The point
is that you are pulling together separate files that may be administered
independently. But git repositories inherently have a whole-project
view. I'm not sure that separate files buy you a lot there. And the
compatibility issues are more complicated.

I do agree that:

  cd .git/info
  git clone /my/exclude/repo exclude ;# or exclude.d

should work; ignoring dotfiles when reading the directory solves that,
and is a pretty standard solution.

-Peff

Re: Expanding Includes in .gitignore

[Aaron Pelly]

On 28/10/16 08:48, Jacob Keller wrote:
> I would strongly prefer rc.d style directories either with a "if the
> .gitignore is a directory treat it like rc.d" or even "add support for
> .gitignore.d as well as .gitignore"

I think adding .gitignore.d shouldn't break existing systems, is
intuitive, and solves my issue.

Does git know when it is in a repo that is too new to comprehend?

My current thinking is that anywhere a .gitignore can go, so can a
.gitignore.d (named appropriately of course.) Any existing .gitignore
should take precedence to the result of parsing the directory.

I haven't looked at the implementation of precedence yet, but I'd be
surprised if the existing mechanism can't be employed.

> One thing to keep in mind would be that we should make sure we can
> handle the .gitignore being a submodule or a git repository, so that
> users could just do something like
> 
> "git submodule add  .gitignore and then track git ignore
> contents from a repository in a nice way.
> 
> By this I mean that the reading of files in .gitignore directory
> should exclude reading .git or other hidden files in some documented
> manor so as to avoid problems when linking to a git directory for its
> contents.

Nice! I like this a lot.