Bug#901897: git introduces new hazardous working-tree-encoding attribute

2018-06-20 Thread Ian Jackson
Ian Jackson writes ("Re: git introduces new hazardous working-tree-encoding 
attribute"):
> Jonathan Nieder writes ("Re: git introduces new hazardous 
> working-tree-encoding attribute"):
> > Happy to add a Breaks.  Do you know what version of dgit will have the
> > fix?
> 
> 5.1.  I suggest
> 
>Breaks: dgit (<< 5.1~)

5.1 is ready and has passed all its tests.  It works fine with the git
in sid.  I am not able to upload it right now, but this will happen
later today.

Ian.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.



Bug#901897: git introduces new hazardous working-tree-encoding attribute

2018-06-20 Thread Paul Gevers
On Tue, 19 Jun 2018 18:28:13 -0700 Jonathan Nieder 
wrote:
> By the way, is there a way to trigger these autopkgtests automatically
> with git from experimental as well?  That way, I'd get earlier notice
> about these issues.

Not yet. It's on my list to add experimental support in ci.d.n, but it
hasn't materialized yet.

Paul



signature.asc
Description: OpenPGP digital signature


Bug#901897: git introduces new hazardous working-tree-encoding attribute

2018-06-20 Thread Ian Jackson
Hi.  Thanks for your prompt attention.  Again, sorry for my
contribution to an annoying situation.

Jonathan Nieder writes ("Re: git introduces new hazardous working-tree-encoding 
attribute"):
> Ian Jackson wrote:
> > Subject: git introduces new hazardous working-tree-encoding attribute
> 
> This title doesn't describe a bug.  On first reading, I thought you
> were saying that some working-tree-encoding related feature was
> behaving incorrectly, but it doesn't appear to be so.  (I was trying
> to understand so that I could forward this report upstream.)

Sorry about that.

> Happy to add a Breaks.  Do you know what version of dgit will have the
> fix?

5.1.  I suggest

   Breaks: dgit (<< 5.1~)

5.1 doesn't exist yet but fixing this issue is very high up on my
list.  Expect it in the next few days.  dgit 5.0 has not yet migrated
and was due to do so on about Sunday.  Because 5.0 has a lot of other
changes I don't want to bring that forward, but I will upload 5.1 with
a higher urgency, so that the migration date is not delayed either.


Much of the rest of what you write is perhaps more relevant to 851679
so I'll answer it there.

Thanks,
Ian.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.



Bug#901897: git introduces new hazardous working-tree-encoding attribute

2018-06-19 Thread Jonathan Nieder
severity 851679 wishlist
tags 851679 + upstream
clone 901897 -1
retitle -1 dgit fails autopkgtest: true/false are no valid 
working-tree-encodings
reassign -1 dgit 5.0
retitle 901897 git needs Breaks against dgit versions without 
working-tree-encoding support
block 901897 by -1
quit

Hi Ian,

Ian Jackson wrote:

> Subject: git introduces new hazardous working-tree-encoding attribute

This title doesn't describe a bug.  On first reading, I thought you
were saying that some working-tree-encoding related feature was
behaving incorrectly, but it doesn't appear to be so.  (I was trying
to understand so that I could forward this report upstream.)

By the way, is there a way to trigger these autopkgtests automatically
with git from experimental as well?  That way, I'd get earlier notice
about these issues.

> So, on to the actual problem:
>
> Looking at the manpage for gitattributes(7) it mentions a new
> attribute
>working-tree-encoding
> which affects the way files are checked in and out.
>
> dgit and similar workflows need to disable this attribute, because
> they require that git trees and source packages correspond, which
> cannot be achieved if working trees made from source packages are not
> identical to git trees once committed.

I think you're saying that attributes like "text" are forbidden in
dgit packages because dgit wants to compare the content of the working
directory to the tracked content without using Git for that.  Am I
understanding correctly?

Git has a notion of "smudge" and "clean" attributes.  The idea is that
to get the content for the worktree, you use "smudge".  To get the
checked-in content, you use "clean".  Would dgit be able to
interoperate with that?

In the dgit(7) manpage, you write:

These transformations are context-sensitive and not, in
general, reversible

Strictly speaking, since users can configure filters that run
arbitrary commands (see "filter" in gitattributes(5)), that's true,
but the transformations are meant to produce a canonical "smudged"
form in the worktree and a canonical "clean" form in the repository.

Sorry I missed bug#851679 before.  We can discuss this more there.
Perhaps some command like "git archive" would work better than "git
checkout" for this use.

> In #851679 I requested a way to disable all checkout-affecting
> gitattributes.  This has not yet been done AFAICT.
>
> In lieu of that, dgit's test suite has a specific test to spot when
> new attributes are introduced.  It tries enabling them and seeing if
> things go wrong.  And indeed, that test has now tripped.  (It failed,
> in fact, simply because some of the values it attempted to set the
> unknown working-tree-encoding attribute to were invalid, but IMO the
> test has done its job.)
>
> I think at the very least I will need to update dgit to disable
> working-tree-encoding as well.  I think the new git should probably
> have a Breaks against the older dgit.

Happy to add a Breaks.  Do you know what version of dgit will have the
fix?

Thanks,
Jonathan



Bug#901897: git introduces new hazardous working-tree-encoding attribute

2018-06-19 Thread Ian Jackson
Package: git
Version: 1:2.18.0~rc2-1
Severity: serious
Justification: breaks dgit test suite, breaks existing attempts not to corrupt 
data

Firstly, apologies for filing this bug as RC.  I wanted to prevent the
new git migrating.  It breaks the dgit autopkgtest.  For reasons I
don't understand, ci.debian.net thinks the dgit autopkgtest `always
failed' even though it passes just fine in testing.  So britney has
reduced the migration delay to a mere 2 days, IMO wrongly.  Can we
please keep this bug as RC for a little while at least while we decide
what to do about it ?

So, on to the actual problem:

Looking at the manpage for gitattributes(7) it mentions a new
attribute
   working-tree-encoding
which affects the way files are checked in and out.

dgit and similar workflows need to disable this attribute, because
they require that git trees and source packages correspond, which
cannot be achieved if working trees made from source packages are not
identical to git trees once committed.

In #851679 I requested a way to disable all checkout-affecting
gitattributes.  This has not yet been done AFAICT.

In lieu of that, dgit's test suite has a specific test to spot when
new attributes are introduced.  It tries enabling them and seeing if
things go wrong.  And indeed, that test has now tripped.  (It failed,
in fact, simply because some of the values it attempted to set the
unknown working-tree-encoding attribute to were invalid, but IMO the
test has done its job.)

I think at the very least I will need to update dgit to disable
working-tree-encoding as well.  I think the new git should probably
have a Breaks against the older dgit.

A proper fix for #851679 would stop this happening next time a new
gitattribute is introduced.

References:

  Failing test log
  https://ci.debian.net/data/autopkgtest/testing/amd64/d/dgit/473624/log.gz

  Explanation of why dgit needs to disable things
  https://manpages.debian.org/stretch/dgit/dgit.7.en.html#GITATTRIBUTES

Thanks for your attention and forbearance,
Ian.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.