Re: Git Tags

[Ævar Arnfjörð Bjarmason]

On Thu, Nov 29 2018, Stefanie Leisestreichler wrote:

> Hi.
>
> I have done this (on box A):
>
> git commit -m "Message"
> git tag -a 0.9.0
> git push origin master
>
> In my local repository, when I run "git tag" it is showing me "0.9.0".
>
> Then I did (on box B)
> git clone ssh://user@host:/path/project.git
> cd project
> git tag
>
> Now git tag is showing nothing.
>
> Why is the tag only available in my local repository?
>
> Also when I try to
> git clone --branch 0.9.0 ssh://user@host:/path/project.git
> it tells me: fatal:remote branch not found in upstream repository origin

Because --branch  means get refs/heads/, tags are not
branches. However, because we're apparently quite loose about this in
the clone/fetch code this does give you the tag if it exists, but
probably not in the way you expect.

We interpret the argument as a branch, and will get not only this tag
but "follow" (see --no-tags in git-fetch(1)) the tag as though it were a
branch and give you all tags leading up to that one. This would give you
a single tag:

git clone --no-tags --branch v2.19.0 --single-branch 
https://github.com/git/git.git

But this is a more direct way to do it:

git init git; git -C git fetch --no-tags https://github.com/git/git.git tag 
v2.19.0

Which'll since you said it failed that's because you haven't pushed the
tag. Try 'git ls-remote ' to see if it's there (it's not).

Re: Git Tags

[Mateusz Loskot]

On Thu, 29 Nov 2018 at 14:40, Randall S. Becker  wrote:
> On November 29, 2018 6:56, Mateusz Loskot wrote:
> > On Thu, 29 Nov 2018 at 12:50, Stefanie Leisestreichler 
> >  wrote:
> > >
> > > git tag -a 0.9.0
> > > git push origin master
> > >
> > > In my local repository, when I run "git tag" it is showing me "0.9.0".
> > >
> > > Then I did (on box B)
> > > git clone ssh://user@host:/path/project.git cd project git tag
> > >
> > > Now git tag is showing nothing.
> > >
> > > Why is the tag only available in my local repository?
> >
> > >From https://git-scm.com/book/en/v2/Git-Basics-Tagging
> > "By default, the git push command doesn’t transfer tags to remote servers.
> > You will have to explicitly push tags to a shared server after you have 
> > created
> > them."
>
> git push --tags
>

After my yesterday experience [1], I'd be careful with git push --tags :)

[1] genuine screenshot and link to related thread at
https://twitter.com/mloskot/status/1068072285846859776

Best regards,
-- 
Mateusz Loskot, http://mateusz.loskot.net

RE: Git Tags

[Randall S. Becker]

On November 29, 2018 6:56, Mateusz Loskot wrote:
> On Thu, 29 Nov 2018 at 12:50, Stefanie Leisestreichler
>  wrote:
> >
> > git tag -a 0.9.0
> > git push origin master
> >
> > In my local repository, when I run "git tag" it is showing me "0.9.0".
> >
> > Then I did (on box B)
> > git clone ssh://user@host:/path/project.git cd project git tag
> >
> > Now git tag is showing nothing.
> >
> > Why is the tag only available in my local repository?
> 
> >From https://git-scm.com/book/en/v2/Git-Basics-Tagging
> "By default, the git push command doesn’t transfer tags to remote servers.
> You will have to explicitly push tags to a shared server after you have 
> created
> them."

git push --tags

and

git fetch --tags

to be symmetrical

Cheers,
Randall

-- Brief whoami:
 NonStop developer since approximately 2112884442
 UNIX developer since approximately 421664400
-- In my real life, I talk too much.

Re: Git Tags

[Mateusz Loskot]

On Thu, 29 Nov 2018 at 12:50, Stefanie Leisestreichler
 wrote:
>
> git tag -a 0.9.0
> git push origin master
>
> In my local repository, when I run "git tag" it is showing me "0.9.0".
>
> Then I did (on box B)
> git clone ssh://user@host:/path/project.git
> cd project
> git tag
>
> Now git tag is showing nothing.
>
> Why is the tag only available in my local repository?

>From https://git-scm.com/book/en/v2/Git-Basics-Tagging
"By default, the git push command doesn’t transfer tags to remote servers.
You will have to explicitly push tags to a shared server after you
have created them."

Best regards,
-- 
Mateusz Loskot, http://mateusz.loskot.net

Git Tags

[Stefanie Leisestreichler]

Hi.

I have done this (on box A):

git commit -m "Message"
git tag -a 0.9.0
git push origin master

In my local repository, when I run "git tag" it is showing me "0.9.0".

Then I did (on box B)
git clone ssh://user@host:/path/project.git
cd project
git tag

Now git tag is showing nothing.

Why is the tag only available in my local repository?

Also when I try to
git clone --branch 0.9.0 ssh://user@host:/path/project.git
it tells me: fatal:remote branch not found in upstream repository origin

Why is the tag not available in origin?

Thanks,
Stefanie

Pulling git tags from multiple sources incorrectly reports stale tags

[Christian Goetze]

See https://gist.github.com/cg-soft/62ac3529cf9ad6f6586e07866de43bc4
and discussion here:
http://stackoverflow.com/questions/37330041/merging-git-tags-from-multiple-reference-locations

Essentially, using this git config to pull tags from multiple remote
refs works fine:

[remote "origin"]
url = /Users/christian.goetze/git/tag-merge-demo/origin
fetch = +refs/heads/*:refs/remotes/origin/*
fetch = +refs/private/tags/*:refs/tags/*
fetch = +refs/tags/*:refs/tags/*

But this, leaving out the last line,  doesn't:

[remote "origin"]
url = /Users/christian.goetze/git/tag-merge-demo/origin
fetch = +refs/heads/*:refs/remotes/origin/*
fetch = +refs/private/tags/*:refs/tags/*

I would expect either to work the same way.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Adding RFC 3161 timestamps to git tags

[Junio C Hamano]

Michael J Gruber  writes:

> Anton Wuerfel venit, vidit, dixit 07.03.2016 15:15:
>> Hello,
>> 
>> as part of an university project we plan to implement time stamp
>> signatures according to RFC 3161. This enables users to create and verify
>> cryptographic time stamp signatures to prove that a commit existed at a
>> certain point in time.
>
> Before talking about a specific header format (and, possibly, repeating
> mistakes of the past) we should take a step back to exactly here: What
> is the goal that you are trying to achieve?
>
> "prove that a commit existed at a certain point in time" is a good
> definition for that goal.

Moving the timestamp signature out of line has merits and demerits.

 - Inline timestamp signature proves that the commit existed at a
   certain point in time, and also it proves that any descendant
   commit was not there before that time.  Also, the object name
   of the signed object itself protects the fact that it was created
   with the signature (i.e. if you replace the signature, its object
   name would change)--this can be a feature or inconvenience,
   depending on your use case.

 - Out of line timestamp signature allows you to say "I didn't
   bother timestamping that commit I made 5 days ago, and it now is
   buried 200 commits deep in the history from the tip, but I am
   asking a timestamping service that the commit existed as of
   now--I cannot go back and prove that it actually was made 5 days
   ago, but at least the timestamp proves it was there today".

   It does not allow you to say anything about descendant commits,
   though, like an inline timestamp would.

There is a convenience argument, too, for and against inline vs out
of line signatures.

 - The original "signed tag" is a good out-of-line signature
   mechanism that you can add signed attestation to a different
   object (i.e. you made a commit either 2 seconds or 5 days ago,
   and then you certify that you would want to call that with a
   tagname with some messages and cryptgraphically sign that
   retroactively).  Being in the ref namespace, it didn't scale for
   use cases where you would want to have as many signatures as you
   would have commits, though.

 - The "signed commit" is made in-line, and for those who want to
   always validate each and every commit, not having to maintain a
   separate collection of signatures was a convenience.

 - The "merge signed tags" is also made in-line, primarily for
   convenience benefit, i.e. not having to keep collection of all
   pull request tags out of line.

I haven't thought deeply enough between in-line and out-of-line
which would suit better for timestamp signatures, though.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Adding RFC 3161 timestamps to git tags

[Michael J Gruber]

Anton Wuerfel venit, vidit, dixit 07.03.2016 15:15:
> Hello,
> 
> as part of an university project we plan to implement time stamp
> signatures according to RFC 3161. This enables users to create and verify
> cryptographic time stamp signatures to prove that a commit existed at a
> certain point in time.

Before talking about a specific header format (and, possibly, repeating
mistakes of the past) we should take a step back to exactly here: What
is the goal that you are trying to achieve?

"prove that a commit existed at a certain point in time" is a good
definition for that goal.

Based on our standing assumption that a commit SHA1 is unique, it is
sufficient to "prove that a SHA1 existed[was known] at a certain point
in time".

In particular, this does not need to take into account the DAG (beyond
what is determined through the SHA1) nor any prior timestamps.
Consequently, I don't think that warrants extending the object format in
any way - it is information in addition to what is in the DAG.

Also, it is conceivable that more than one user of the timestamp service
requests a timestamp for the same SHA1, and does so for a commit which
has children already, without wanting to (and without any intrinsic need
to) rewrite history.

To me, this means timestamps have no place in commit objects.

As for adding additional information to the DAG without altering it, we
have two means: tags and notes. Tags suffer from a "merge problem",
notes from a "transport" problem; for both of them you have to know how
to set up your refspecs.

So, I think a proper first step would be to make our "metadata handling"
(default notes refspec, merging) more use friendly, i.e. work out of the
box for the common use case (whatever that is).

That would serve timestamps well, and many other use cases.

> As a long-term goal, we would like to get this new feature accepted into
> upstream, so we are very interested in your opinions and suggestions for
> our approach described in the following.
> 
> We plan to add new command line options to git tag and call openssl
> similar to how "git tag -s" is calling gpg. The time stamp query generated
> by openssl will be sent to the time stamping authority via libcurl.
> Verification of timestamps will be possible via git verify-tag.
> 
> In order to store time stamp signatures, the file format for git tags
> needs to be extended. Similar to how gpg signatures are stored, we would
> store the signed time stamp responses in base64 surrounded by BEGIN and
> END tags:
> -BEGIN RFC3161-
> Issuer: [issuer-name]
> [time stamp response in base64]
> -END RFC3161-
> 
> We plan to offer git config options to configure, which timestamping
> authority to use and where trusted certificates are stored.
> 
> Regards,
> Phillip Raffeck
> Anton Wuerfel
> 

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Adding RFC 3161 timestamps to git tags

[Anton Wuerfel]

Hello Junio,

thanks for your reply. See my comments below.

"Junio C Hamano" <gits...@pobox.com> writes:

> A few random thoughts that come to mind, none of which is
> rhetorical [*1*]:
>
>  - What should happen when the timestamping service is unreachable?
>The user cannot get her work done at all?  A tag is created
>without timestamp and with a warning?  Something else?

If the timestamping service is unreachable, we plan to output a warning
and abort the tag creation as a default behavior. However we could create
config option to allow the user to create a tag without a signature if the
TSA (timestamp authority) is not available.


>  - Is "signed tag" the only thing that will benefit from such a
>certified timestamping mechanism?  Would it be worthwhile to
>offer a similar support for "signed commit"?

This is a good point. We will consider implementing this in signed
commits, too. Like in gpg-signed commits, rebases and changes of these
commits will not be possible any more without invalidating the timestamp
signature. However, the intention behind all this is to be able to verify
important steps in development and continue to be able to work and commit
without internet connection. Therefore our main focus is on tags with
timestamp signatures.


>  - How would the certified timestamp interact with GPG signing of
>the tag?  Can they both be applied to the same tag, and if so
>what is signed by which mechanism and in what order or are they
>done independently and in parallel?  E.g. would the timestamp be
>done on the contents without GPG signature, and the GPG signature
>be done on the contents without timestamp, and both signature
>blocks concatenated at the end of the original contents?

Both GPG and timestamp signing can be assigned to the same tag. A GPG
signature includes the timestamp signature for one important reason: It
should not be possible to replace an existing timestamp signature by
another (later) timestamp signature. Including the timestamp signature
into the GPG signature prevents this.
Creating a timestamp signature without any GPG signature at all is
therefore possible but would be vulnerable to the described scenario.


>  - Would it make sense to store the certified timestamp in the
>object header part, like the way GPG signature for signed commit
>objects are stored [*2*], instead of following the old-style
>"signed tag" that concatenates a separate signature at the end?

For timestamped commits we will, of course, use the new-style format. We
would also new-style format for git tags, leaving the GPG signature as is
and creating a timesig-header. However, mixing old-style and new-style
format in tag objects would introduce an inconsistency. Is this
problematic?

Regards,
Phillip Raffeck
Anton Wuerfel

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Adding RFC 3161 timestamps to git tags

[Junio C Hamano]

"Anton Wuerfel" <anton.wuer...@fau.de> writes:

> as part of an university project we plan to implement time stamp
> signatures according to RFC 3161. This enables users to create and verify
> cryptographic time stamp signatures to prove that a commit existed at a
> certain point in time.
>
> As a long-term goal, we would like to get this new feature accepted into
> upstream, so we are very interested in your opinions and suggestions for
> our approach described in the following.
>
> We plan to add new command line options to git tag and call openssl
> similar to how "git tag -s" is calling gpg. The time stamp query generated
> by openssl will be sent to the time stamping authority via libcurl.
> Verification of timestamps will be possible via git verify-tag.
>
> In order to store time stamp signatures, the file format for git tags
> needs to be extended. Similar to how gpg signatures are stored, we would
> store the signed time stamp responses in base64 surrounded by BEGIN and
> END tags:
> -BEGIN RFC3161-
> Issuer: [issuer-name]
> [time stamp response in base64]
> -END RFC3161-
>
> We plan to offer git config options to configure, which timestamping
> authority to use and where trusted certificates are stored.

A few random thoughts that come to mind, none of which is
rhetorical [*1*]:

 - What should happen when the timestamping service is unreachable?
   The user cannot get her work done at all?  A tag is created
   without timestamp and with a warning?  Something else?

 - Is "signed tag" the only thing that will benefit from such a
   certified timestamping mechanism?  Would it be worthwhile to
   offer a similar support for "signed commit"?

 - How would the certified timestamp interact with GPG signing of
   the tag?  Can they both be applied to the same tag, and if so
   what is signed by which mechanism and in what order or are they
   done independently and in parallel?  E.g. would the timestamp be
   done on the contents without GPG signature, and the GPG signature
   be done on the contents without timestamp, and both signature
   blocks concatenated at the end of the original contents?

 - Would it make sense to store the certified timestamp in the
   object header part, like the way GPG signature for signed commit
   objects are stored [*2*], instead of following the old-style
   "signed tag" that concatenates a separate signature at the end?


[Footnote]

*1* ... meaning that when I ask "Is X true?", I do not mean "I think
X is true" or "I do not think X can possibly be true".

*2* We designed newer places that use GPG signatures (i.e. signed
commit and merge tag) to store the signature in the header part
for a reason: the base64 gobbledygook is not for human
consumption and showing it together with the original contents
would not help.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Adding RFC 3161 timestamps to git tags

[Anton Wuerfel]

Hello,

as part of an university project we plan to implement time stamp
signatures according to RFC 3161. This enables users to create and verify
cryptographic time stamp signatures to prove that a commit existed at a
certain point in time.

As a long-term goal, we would like to get this new feature accepted into
upstream, so we are very interested in your opinions and suggestions for
our approach described in the following.

We plan to add new command line options to git tag and call openssl
similar to how "git tag -s" is calling gpg. The time stamp query generated
by openssl will be sent to the time stamping authority via libcurl.
Verification of timestamps will be possible via git verify-tag.

In order to store time stamp signatures, the file format for git tags
needs to be extended. Similar to how gpg signatures are stored, we would
store the signed time stamp responses in base64 surrounded by BEGIN and
END tags:
-BEGIN RFC3161-
Issuer: [issuer-name]
[time stamp response in base64]
-END RFC3161-

We plan to offer git config options to configure, which timestamping
authority to use and where trusted certificates are stored.

Regards,
Phillip Raffeck
Anton Wuerfel

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Feature Request: Passing a number as an option to git tags for displaying latest tags

[Jacob Keller]

On Thu, Jul 23, 2015 at 2:39 AM, Michael J Gruber
g...@drmicha.warpmail.net wrote:

 While not quite being intended for that purpose,

 git log --oneline --decorate --simplify-by-decoration [-n] --tags

 (or with a custom format instead of --oneline --decorate) may come
 close to what you want.[*]

 Michael

 [*] As Linus once described it (iirc): oooh, evil. I like it.
 --
 To unsubscribe from this list: send the line unsubscribe git in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html

Yep, that's pretty much how a build system I've had to use does it.
Sadly, this is quite slow, and I'm not sure if doing it built into the
tag via for-each-ref would be faster? I mean obviously having to look
at each commit is slower than just the tag name, but it might be
faster than parsing through the log process? Though the log output has
the advantage that it only shows you tags on a given branch.

Regards,
Jake
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Feature Request: Passing a number as an option to git tags for displaying latest tags

[Junio C Hamano]

Jacob Keller jacob.kel...@gmail.com writes:

 On Thu, Jul 23, 2015 at 11:01 AM, Junio C Hamano gits...@pobox.com wrote:
 Jacob Keller jacob.kel...@gmail.com writes:

 Yep, that's pretty much how a build system I've had to use does it.
 Sadly, this is quite slow, and I'm not sure if doing it built into the
 tag via for-each-ref would be faster?

 Is the description in git for-each-ref --help somehow unreadable?
 ...
 If you only need the name of the ref, you can use a lot simpler
 format string, e.g.

 git for-each-ref --count=3 --sort='-*authordate' \
   --format='%(refname:short)' refs/tags

 git for-each-ref --count=3 --sort='-taggerdate' \
   --format='%(refname:short)' refs/tags


 That's significantly better than what this system does, but sadly the
 team that owns it doesn't exactly understand git. Pretty sure they
 tend to just use whatever scriptlet they got working, which happens to
 be based on log...  I've tried to help them, but generally hasn't been
 a successful relationship there.

If your build people refuse to switch from an unreliable way of
parsing log output that is not meant for machine consumption, then
it wouldn't make any difference what the answer to your I'm not
sure if doing it built into ... be faster? X-.

By the way, I was referring to that question, pointing out that you
do not have be wondering.  It is already there, and as I said a few
messages ago already in this thread, the cross-pollination among
for-each-ref, branch -l and tag -l is happening as part of
this year's GSoC program.





--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Feature Request: Passing a number as an option to git tags for displaying latest tags

[Junio C Hamano]

Jacob Keller jacob.kel...@gmail.com writes:

 Yep, that's pretty much how a build system I've had to use does it.
 Sadly, this is quite slow, and I'm not sure if doing it built into the
 tag via for-each-ref would be faster?

Is the description in git for-each-ref --help somehow unreadable?

   An example directly producing formatted text. Show the most
   recent 3 tagged commits:

   #!/bin/sh

   git for-each-ref --count=3 --sort='-*authordate' \
   --format='From: %(*authorname) %(*authoremail)
   Subject: %(*subject)
   Date: %(*authordate)
   Ref: %(*refname)

   %(*body)
   ' 'refs/tags'

If you only need the name of the ref, you can use a lot simpler
format string, e.g.

git for-each-ref --count=3 --sort='-*authordate' \
  --format='%(refname:short)' refs/tags

git for-each-ref --count=3 --sort='-taggerdate' \
  --format='%(refname:short)' refs/tags

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Feature Request: Passing a number as an option to git tags for displaying latest tags

[Jacob Keller]

On Thu, Jul 23, 2015 at 11:01 AM, Junio C Hamano gits...@pobox.com wrote:
 Jacob Keller jacob.kel...@gmail.com writes:

 Yep, that's pretty much how a build system I've had to use does it.
 Sadly, this is quite slow, and I'm not sure if doing it built into the
 tag via for-each-ref would be faster?

 Is the description in git for-each-ref --help somehow unreadable?

An example directly producing formatted text. Show the most
recent 3 tagged commits:

#!/bin/sh

git for-each-ref --count=3 --sort='-*authordate' \
--format='From: %(*authorname) %(*authoremail)
Subject: %(*subject)
Date: %(*authordate)
Ref: %(*refname)

%(*body)
' 'refs/tags'

 If you only need the name of the ref, you can use a lot simpler
 format string, e.g.

 git for-each-ref --count=3 --sort='-*authordate' \
   --format='%(refname:short)' refs/tags

 git for-each-ref --count=3 --sort='-taggerdate' \
   --format='%(refname:short)' refs/tags


That's significantly better than what this system does, but sadly the
team that owns it doesn't exactly understand git. Pretty sure they
tend to just use whatever scriptlet they got working, which happens to
be based on log...  I've tried to help them, but generally hasn't been
a successful relationship there.

Regards,
Jake
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Feature Request: Passing a number as an option to git tags for displaying latest tags

[Michael J Gruber]

Junio C Hamano venit, vidit, dixit 22.07.2015 21:20:
 Junio C Hamano gits...@pobox.com writes:
 
 The former, sort by time, is interesting, but you need to define
 what to do with various corner cases.  For example, some people may
 have one or more of the following desires:

  * My project did not use tags for a long time, and started using it
recently starting from v1.1.0.  The first release only said
Frotz version 1.0.0 in its commit log message.  I retroactively
did git tag -s -m 'Frotz 1.1.0' v1.1.0 on that commit.
 
 Obviously, I meant git tag -s -m 'Frotz 1.0.0' v1.0.0 here.
 
In such a case, it is likely that I would want the sorting done
based on the committer date on the underlying commit, not the
tag's tagger date.

  * When a bug is found, it is customary in my project to add a
break-something tag to the commit that introduces the bug
(and fix-something tag to the commit that fixes it).

When I want to find recently discovered breakages, I want the
tags whose names match break-* sorted by tagger dates, not the
underlying commit's committer dates.
 
 Another use case may be one in which older tags are interesting.  In
 other words, you need to be able to sort in reverse, too.
 
 The necessary ordering machinery to do the above already exists in
 for-each-ref.  There is a GSoC project that works to unify various
 features spread across for-each-ref, branch -l and tag -l and
 make them available to all of the three.
 
 And the above is still true even with reverse-order use case.
 

While not quite being intended for that purpose,

git log --oneline --decorate --simplify-by-decoration [-n] --tags

(or with a custom format instead of --oneline --decorate) may come
close to what you want.[*]

Michael

[*] As Linus once described it (iirc): oooh, evil. I like it.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Feature Request: Passing a number as an option to git tags for displaying latest tags

[Jacob Keller]

On Wed, Jul 22, 2015 at 10:17 AM, Halil Öztürk haliloztur...@gmail.com wrote:
 Hello team,

 Passing a number as an option to git tags command should display latest 
 tags.

 e.g. git tags -5 will display last 5 tags only.

 Similar behavior to git log -5

 Thanks,
 Halil
 --
 To unsubscribe from this list: send the line unsubscribe git in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html


While interesting, this would only really work for annotated tags. How
would you define order? tags are normally shown in sorted lexical
order (or version-order if you pass the --sort parameter).
Non-annotated tags do not contain the date time, and using the
commit's date information may not be accurate. While it is possible to
say show the 5 most recent tags on a particular branch of history
that would be using the log to determine this.

What exactly would you expect the behavior to be with tags? That might
help figure out what could be done instead.

Regards,
Jake
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Feature Request: Passing a number as an option to git tags for displaying latest tags

[Junio C Hamano]

Junio C Hamano gits...@pobox.com writes:

 The former, sort by time, is interesting, but you need to define
 what to do with various corner cases.  For example, some people may
 have one or more of the following desires:

  * My project did not use tags for a long time, and started using it
recently starting from v1.1.0.  The first release only said
Frotz version 1.0.0 in its commit log message.  I retroactively
did git tag -s -m 'Frotz 1.1.0' v1.1.0 on that commit.

Obviously, I meant git tag -s -m 'Frotz 1.0.0' v1.0.0 here.

In such a case, it is likely that I would want the sorting done
based on the committer date on the underlying commit, not the
tag's tagger date.

  * When a bug is found, it is customary in my project to add a
break-something tag to the commit that introduces the bug
(and fix-something tag to the commit that fixes it).

When I want to find recently discovered breakages, I want the
tags whose names match break-* sorted by tagger dates, not the
underlying commit's committer dates.

Another use case may be one in which older tags are interesting.  In
other words, you need to be able to sort in reverse, too.

 The necessary ordering machinery to do the above already exists in
 for-each-ref.  There is a GSoC project that works to unify various
 features spread across for-each-ref, branch -l and tag -l and
 make them available to all of the three.

And the above is still true even with reverse-order use case.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Feature Request: Passing a number as an option to git tags for displaying latest tags

[Halil Öztürk]

Hello team,

Passing a number as an option to git tags command should display latest tags.

e.g. git tags -5 will display last 5 tags only.

Similar behavior to git log -5

Thanks,
Halil
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Feature Request: Passing a number as an option to git tags for displaying latest tags

[Andreas Schwab]

Halil Öztürk haliloztur...@gmail.com writes:

 Passing a number as an option to git tags command should display latest 
 tags.

How do you define the latest tags?  By tag creation?  Lightweight
tags don't have any kind of creation time attached.

Andreas.

-- 
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
And now for something completely different.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Feature Request: Passing a number as an option to git tags for displaying latest tags

[Junio C Hamano]

Halil Öztürk haliloztur...@gmail.com writes:

 Passing a number as an option to git tags command should display latest 
 tags.

 e.g. git tags -5 will display last 5 tags only.

I think this conflates two unrelated things.

 - Ordering tags not by refnames (i.e. default) but by time.

 - Limiting the output by count.

The latter is what | head -n 5 and/or | tail -n 5 are for, so it
would be at most nice to have; I am indifferent in the sense that
I won't work on it, but I'd take a look if somebody sent a patch
that was cleanly done.

The former, sort by time, is interesting, but you need to define
what to do with various corner cases.  For example, some people may
be have one or more of the following desires:

 * My project did not use tags for a long time, and started using it
   recently starting from v1.1.0.  The first release only said
   Frotz version 1.0.0 in its commit log message.  I retroactively
   did git tag -s -m 'Frotz 1.1.0' v1.1.0 on that commit.

   In such a case, it is likely that I would want the sorting done
   based on the committer date on the underlying commit, not the
   tag's tagger date.

 * When a bug is found, it is customary in my project to add a
   break-something tag to the commit that introduces the bug
   (and fix-something tag to the commit that fixes it).

   When I want to find recently discovered breakages, I want the
   tags whose names match break-* sorted by tagger dates, not the
   underlying commit's committer dates.

The necessary ordering machinery to do the above already exists in
for-each-ref.  There is a GSoC project that works to unify various
features spread across for-each-ref, branch -l and tag -l and
make them available to all of the three.


--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: weaning distributions off tarballs: extended verification of git tags

[Colin Walters]

On Sat, Feb 28, 2015, at 10:48 AM, Colin Walters wrote:
 Hi, 
 
 TL;DR: Let's define a standard for embedding stronger checksums in tags and 
 commit messages:
 https://github.com/cgwalters/homegit/blob/master/bin/git-evtag

[time passes]

I finally had a bit of time to pick this back up again in:

https://github.com/cgwalters/git-evtag

It should address the core concern here about stability of `git archive`.

I prototyped it out with libgit2 because it was easier, and I'd like actually 
to be able to use this with older versions of git.

But I think the next steps here are:

- Validate the core design
  * Tree walking order
  * Submodule recursion
  * Use of SHA512
- Standardize it
  (Would like to see at least a stupid slow shell script implementation to 
cross-validate)
- Add it as an option to `git tag`?

Longer term:
- Support adding `Git-EVTag` as a git note, so I can retroactively add stronger
  checksums to older git repositories
- Anything else?

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: weaning distributions off tarballs: extended verification of git tags

[Michael Haggerty]

On 03/03/2015 12:44 AM, Junio C Hamano wrote:
 [...]
 I was about to suggest another alternative.
 
 Pretend as if Git internally used SHA-512 (or whatever hash you
 want to use) instead of SHA-1, compute the object names that
 way.  Recompute the contents of a tree object is by replacing
 the 20-byte SHA-1 field in it with a field with whatever
 necessary length to hold the longer object names of elements in
 the tree.
 
 But then a realization hit me: what new value will be placed in the
 parent  field in the commit object?  You cannot have SHA-512
 variant of commit object name without recomputing the whole history.
 
 Now, if the final objective is to replace signature of tarballs,
 does it matter to cover the commit object, or is it sufficient to
 cover the tree contents?

The original goal was to replace a tarball signature, for which the
alternative that you described above seems quite elegant.

If the goal were really to certify the entire history, then none of the
proposals that I have seen so far is adequate anyway, because none of
them propose to include better than the original SHA-1s of the parent
commits.

Including other metadata from the release commit does not seem useful to
me; how valuable is it to know the author and commit message of the last
commit that happened to make it into a release? It would be more useful
to know the SHA-1 of that commit, but that would presumably be included
elsewhere in the packaging data used by the distribution.

 [...]

Michael

-- 
Michael Haggerty
mhag...@alum.mit.edu

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: weaning distributions off tarballs: extended verification of git tags

[Duy Nguyen]

On Tue, Mar 3, 2015 at 1:12 AM, Joey Hess i...@joeyh.name wrote:
 I support this proposal, as someone who no longer releases tarballs
 of my software, when I can possibly avoid it. I have worried about
 signed tags / commits only being a SHA1 break away from useless.

 As to the implementation, checksumming the collection of raw objects is
 certainly superior to tar. Colin had suggested sorting the objects by
 checksum, but I don't think that is necessary. Just stream the commit
 object, then its tree object, followed by the content of each object
 listed in the tree, recursing into subtrees as necessary. That will be a
 stable stream for a given commit, or tree.

It could be simplified a bit by using ls-tree -r (so you basically
have a single big tree). Then hash commit, ls-tree -r output and all
blobs pointed by ls-tree in listed order.
-- 
Duy
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: weaning distributions off tarballs: extended verification of git tags

[Junio C Hamano]

Duy Nguyen pclo...@gmail.com writes:

 On Tue, Mar 3, 2015 at 1:12 AM, Joey Hess i...@joeyh.name wrote:
 I support this proposal, as someone who no longer releases tarballs
 of my software, when I can possibly avoid it. I have worried about
 signed tags / commits only being a SHA1 break away from useless.

 As to the implementation, checksumming the collection of raw objects is
 certainly superior to tar. Colin had suggested sorting the objects by
 checksum, but I don't think that is necessary. Just stream the commit
 object, then its tree object, followed by the content of each object
 listed in the tree, recursing into subtrees as necessary. That will be a
 stable stream for a given commit, or tree.

 It could be simplified a bit by using ls-tree -r (so you basically
 have a single big tree). Then hash commit, ls-tree -r output and all
 blobs pointed by ls-tree in listed order.

What problem are you trying to solve here, though, by deliberately
deviating what Git internally used to store these objects?  If it is
OK to ignore the tree boundary, then you probably do not even need
trees in this secondary hash for validation in the first place.

For example, you can hash a stream:

commit object contents +
N * (pathname + NUL + blob object contents)

as long as the pathnames are sorted in a predictable order (like
in the index order) in the output.  That would be even simpler (I
am not saying it is necessarily better, and by inference neither is
your simplification).

I was about to suggest another alternative.

Pretend as if Git internally used SHA-512 (or whatever hash you
want to use) instead of SHA-1, compute the object names that
way.  Recompute the contents of a tree object is by replacing
the 20-byte SHA-1 field in it with a field with whatever
necessary length to hold the longer object names of elements in
the tree.

But then a realization hit me: what new value will be placed in the
parent  field in the commit object?  You cannot have SHA-512
variant of commit object name without recomputing the whole history.

Now, if the final objective is to replace signature of tarballs,
does it matter to cover the commit object, or is it sufficient to
cover the tree contents?

Among the ideas raised so far, I like what Joey suggested, combined
with each should have 'type lengthNUL' header from Sam Vilain
the best.  That is, hash the stream:

commit length NUL + commit object contents +
tree length NUL + top level tree contents +
... list the entries in the order you would find by
... some defined traversal order people can agree on.

with whatever the preferred strong hash function of the age.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: weaning distributions off tarballs: extended verification of git tags

[Duy Nguyen]

On Tue, Mar 3, 2015 at 6:44 AM, Junio C Hamano gits...@pobox.com wrote:
 Duy Nguyen pclo...@gmail.com writes:

 On Tue, Mar 3, 2015 at 1:12 AM, Joey Hess i...@joeyh.name wrote:
 I support this proposal, as someone who no longer releases tarballs
 of my software, when I can possibly avoid it. I have worried about
 signed tags / commits only being a SHA1 break away from useless.

 As to the implementation, checksumming the collection of raw objects is
 certainly superior to tar. Colin had suggested sorting the objects by
 checksum, but I don't think that is necessary. Just stream the commit
 object, then its tree object, followed by the content of each object
 listed in the tree, recursing into subtrees as necessary. That will be a
 stable stream for a given commit, or tree.

 It could be simplified a bit by using ls-tree -r (so you basically
 have a single big tree). Then hash commit, ls-tree -r output and all
 blobs pointed by ls-tree in listed order.

 What problem are you trying to solve here, though, by deliberately
 deviating what Git internally used to store these objects?  If it is
 OK to ignore the tree boundary, then you probably do not even need
 trees in this secondary hash for validation in the first place.

 For example, you can hash a stream:

 commit object contents +
 N * (pathname + NUL + blob object contents)

 as long as the pathnames are sorted in a predictable order (like
 in the index order) in the output.  That would be even simpler (I
 am not saying it is necessarily better, and by inference neither is
 your simplification).

I did nearly that [1]. But this morning I realized trees carry file
permission. We should keep that in the final checksum as well.

 Now, if the final objective is to replace signature of tarballs,
 does it matter to cover the commit object, or is it sufficient to
 cover the tree contents?

 Among the ideas raised so far, I like what Joey suggested, combined
 with each should have 'type lengthNUL' header from Sam Vilain
 the best.  That is, hash the stream:

 commit length NUL + commit object contents +
 tree length NUL + top level tree contents +
 ... list the entries in the order you would find by
 ... some defined traversal order people can agree on.

 with whatever the preferred strong hash function of the age.

A bit harder to script, but simpler to provide from cat-file, I think.

[1] http://article.gmane.org/gmane.comp.version-control.git/260211
-- 
Duy
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: weaning distributions off tarballs: extended verification of git tags

[Sam Vilain]

On 03/02/2015 12:08 PM, Junio C Hamano wrote:

I have a
hazy recollection of what it would take to replace SHA-1 in git with
something else; it should be possible (though tricky) to do it lazily,
where a tree entry has bits (eg, some of the currently unused file
mode bits) to denotes which hash algorithm is in use for the entry.
However I don't think that got past idea stage...

I think one reason why it didn't was because it would not work well.
That bit that tells this is a new object or old would mean that a
single tree can have many different object names, depending on which
of its component entries are using that bit and which aren't.  There
goes the we know two trees with the same object name are identical
without recursing into them optimization out the window.

Also it would make it impossible to do what you suggest to Joey to
do, i.e. exactly the same way that git does, once you start saying
that a tree object can be encoded in more than one different ways,
wouldn't it?


I was reasoning that people would rather not have to rewrite their whole 
history in order to switch checksum algorithms, and that by allowing 
trees to be lazily converted that this would make things more 
efficient.  However, I think I see your point here that this doesn't work.


However, as a per-commit header, then only first commit which changes 
the hashing algorithm would have to re-checksum each of the files: but 
just in the current tree, not all the way back to the beginning of 
history.  The delta logic should not have to care, and these objects 
with the same content but different object ID should pack perfectly, so 
long as git-pack-objects knows to re-checksum objects with the available 
hash algorithms and spot matches.


Other operations like diff which span commit hashing algorithms might be 
able to get away with their existing object ranking algorithms and cache 
alternate object IDs for content as they operate to facilitate exact 
matching across hash algorithm changes.


But actually, for the original problem - just producing a signature with 
a different hashing algorithm - probably it would be sufficient to just 
re-hash the current commit and the current tree recursively, and the 
mixed hash-algorithm case does not need to exist.  But I'm just thinking 
it might not be too hard to make git nicely generic, to be well prepared 
for when a second pre-image attack on SHA-1 becomes practical.


Sam
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: weaning distributions off tarballs: extended verification of git tags

[Colin Walters]

On Sat, Feb 28, 2015, at 03:34 PM, Morten Welinder wrote:
 Is there a point to including a different checksum inside
 a git tag?  If someone can break the SHA-1 checksum
 in the repository then the recorded SHA-256 checksum can
 be changed.  In other words, wouldn't you be just as well
 off handing someone a SHA-1 commit id?

The issue is more about what the checksum covers, as
well as its strength.  Git uses a hash tree, which means
that an attacker only has to find a collision for *one* of
the objects, and the signature is still valid.  And that collision
is valid for *every* commit that contains that object.

This topic has been covered elsewhere pretty extensively,
here's a link:
https://www.whonix.org/forum/index.php/topic,538.msg4278.html#msg4278

Now I think rough consensus is still that git is secure or
secure enough - but with this proposal I'm just trying
to overcome the remaining conservatism.  (Also, while those
discussions were focusing on corrupting an existing repository,
the attack model of MITM also exists, and there
you don't have to worry about deltas, particularly if the
attacker's goal is to get a downstream to do a build
and thus execute their hostile code inside the downstream
network).

It's really not that expensive to do once per release,
basically free for small repositories, and for a large one like
the Linux kernel:

$ cd ~/src/linux
$ git describe
v3.19-7478-g796e1c5
$ time /bin/sh -c 'git archive --format=tar HEAD|sha256sum'
4a5c5826cea188abd52fa50c663d17ebe1dfe531109fed4ddbd765a856f1966e  -

real0m3.772s
user0m6.132s
sys 0m0.279s
$

With this proposal, the checksum
covers an entire stream of objects for a given commit at once;
making it significantly harder to find a collision.  At least as good as 
checksummed tarballs, and arguably better since it's
pre-compression.

So to implement this, perhaps something like:

$ git archive --format=raw

as a base primitive, and:

$ git tag --archive-raw-checksum=SHA256 -s -m ...

?

git fsck could also learn to optionally use this.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: weaning distributions off tarballs: extended verification of git tags

[Joey Hess]

I support this proposal, as someone who no longer releases tarballs
of my software, when I can possibly avoid it. I have worried about
signed tags / commits only being a SHA1 break away from useless.

As to the implementation, checksumming the collection of raw objects is
certainly superior to tar. Colin had suggested sorting the objects by
checksum, but I don't think that is necessary. Just stream the commit
object, then its tree object, followed by the content of each object
listed in the tree, recursing into subtrees as necessary. That will be a
stable stream for a given commit, or tree.

-- 
see shy jo


signature.asc
Description: Digital signature

Re: weaning distributions off tarballs: extended verification of git tags

[Sam Vilain]

On 03/02/2015 10:12 AM, Joey Hess wrote:

I support this proposal, as someone who no longer releases tarballs
of my software, when I can possibly avoid it. I have worried about
signed tags / commits only being a SHA1 break away from useless.

As to the implementation, checksumming the collection of raw objects is
certainly superior to tar. Colin had suggested sorting the objects by
checksum, but I don't think that is necessary. Just stream the commit
object, then its tree object, followed by the content of each object
listed in the tree, recursing into subtrees as necessary. That will be a
stable stream for a given commit, or tree.


I would really just do it exactly the same way that git does: checksum 
the objects including their headers with the new hashes.  I have a hazy 
recollection of what it would take to replace SHA-1 in git with 
something else; it should be possible (though tricky) to do it lazily, 
where a tree entry has bits (eg, some of the currently unused file mode 
bits) to denotes which hash algorithm is in use for the entry.  However 
I don't think that got past idea stage...


Sam
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: weaning distributions off tarballs: extended verification of git tags

[Junio C Hamano]

Sam Vilain s...@vilain.net writes:

 As to the implementation, checksumming the collection of raw objects is
 certainly superior to tar. Colin had suggested sorting the objects by
 checksum, but I don't think that is necessary. Just stream the commit
 object, then its tree object, followed by the content of each object
 listed in the tree, recursing into subtrees as necessary. That will be a
 stable stream for a given commit, or tree.

 I would really just do it exactly the same way that git does: checksum
 the objects including their headers with the new hashes.

I tend to agree that it is a good idea.  I also suspect that would
make the implementation simpler by allowing it to share more code,
but I didn't look into it too deeply.

 I have a
 hazy recollection of what it would take to replace SHA-1 in git with
 something else; it should be possible (though tricky) to do it lazily,
 where a tree entry has bits (eg, some of the currently unused file
 mode bits) to denotes which hash algorithm is in use for the entry.
 However I don't think that got past idea stage...

I think one reason why it didn't was because it would not work well.
That bit that tells this is a new object or old would mean that a
single tree can have many different object names, depending on which
of its component entries are using that bit and which aren't.  There
goes the we know two trees with the same object name are identical
without recursing into them optimization out the window.

Also it would make it impossible to do what you suggest to Joey to
do, i.e. exactly the same way that git does, once you start saying
that a tree object can be encoded in more than one different ways,
wouldn't it?

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

weaning distributions off tarballs: extended verification of git tags

[Colin Walters]

Hi, 

TL;DR: Let's define a standard for embedding stronger checksums in tags and 
commit messages:
https://github.com/cgwalters/homegit/blob/master/bin/git-evtag

I think tarballs should go away as a source distribution mechanism in favor of 
pure git.  I won't go into too many details of the why here (hopefully most 
of you agree!) but that's the background.

Now, there are a few things that the classical tarball model provides:

- Version numbers compatible with dpkg/rpm/etc
  - Do the same with your tag names, and use a well known scheme like 
v$VERSION
- The assumption that this source has been run through some tests
  - Broken assumption, and regardless you want to rerun tests downstream
- Hosting providers typically offer a strong checksum over the entire source
  - The topic of this post

The above strawman code allows embedding the SHA256(git archive | tar).  Now,
in order to make this work, the byte output of git archive must never change 
in the
future.  I'm not sure how valid an assumption this is.  Timestamps are set to 
the
commit timestamp, but I could imagine someone wanting to come along later
and tweak the output to be compatible with some variant of tar or something.

We could define the checksum to be over the stream of raw objects, sorted by 
their checksum,
and that way be independent of archiving format variations.

Is there agreement that something like this makes sense in the git core?  Does 
the
concept make sense?  Does anything like this exist today?  Other 
thoughts/objections?
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: weaning distributions off tarballs: extended verification of git tags

[Morten Welinder]

Is there a point to including a different checksum inside
a git tag?  If someone can break the SHA-1 checksum
in the repository then the recorded SHA-256 checksum can
be changed.  In other words, wouldn't you be just as well
off handing someone a SHA-1 commit id?

If you can guard the SHA-256 with a signature, you can
do the same thing to the SHA-1.  Or the tarball for that matter.

Unrelatedly, your assumptions:

Tar balls have too many degrees of freedom to rely on them
being created identically in the future.

 - The assumption that this source has been run through some tests

A perfectly valid assumption for some build systems, notably
autotools.  make distcheck is the only way my tarballs get
made and they only get made when the checks succeed.
(If your point was that many projects have too few tests,
well, then I agree.)

M.



On Sat, Feb 28, 2015 at 9:48 AM, Colin Walters walt...@verbum.org wrote:
 Hi,

 TL;DR: Let's define a standard for embedding stronger checksums in tags and 
 commit messages:
 https://github.com/cgwalters/homegit/blob/master/bin/git-evtag

 I think tarballs should go away as a source distribution mechanism in favor 
 of pure git.  I won't go into too many details of the why here (hopefully 
 most of you agree!) but that's the background.

 Now, there are a few things that the classical tarball model provides:

 - Version numbers compatible with dpkg/rpm/etc
   - Do the same with your tag names, and use a well known scheme like 
 v$VERSION
 - The assumption that this source has been run through some tests
   - Broken assumption, and regardless you want to rerun tests downstream
 - Hosting providers typically offer a strong checksum over the entire source
   - The topic of this post

 The above strawman code allows embedding the SHA256(git archive | tar).  Now,
 in order to make this work, the byte output of git archive must never 
 change in the
 future.  I'm not sure how valid an assumption this is.  Timestamps are set to 
 the
 commit timestamp, but I could imagine someone wanting to come along later
 and tweak the output to be compatible with some variant of tar or something.

 We could define the checksum to be over the stream of raw objects, sorted by 
 their checksum,
 and that way be independent of archiving format variations.

 Is there agreement that something like this makes sense in the git core?  
 Does the
 concept make sense?  Does anything like this exist today?  Other 
 thoughts/objections?
 --
 To unsubscribe from this list: send the line unsubscribe git in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: weaning distributions off tarballs: extended verification of git tags

[brian m. carlson]

On Sat, Feb 28, 2015 at 09:48:05AM -0500, Colin Walters wrote:

The above strawman code allows embedding the SHA256(git archive | tar).  Now,
in order to make this work, the byte output of git archive must never change 
in the
future.  I'm not sure how valid an assumption this is.  Timestamps are set to 
the
commit timestamp, but I could imagine someone wanting to come along later
and tweak the output to be compatible with some variant of tar or something.


This is not a safe assumption.  Unfortunately, kernel.org assumed that 
it was the case, and a change broke it.  Let's please not make more code 
that does that.



We could define the checksum to be over the stream of raw objects, sorted by 
their checksum,
and that way be independent of archiving format variations.


This would be a much better idea, assuming you mean raw git objects. 
For cryptographic purposes, it's important to make the item boundaries 
unambiguous, which is usually done using the length.  Since the raw git 
objects include the length, this is sufficient.


If you don't make the boundaries unambiguous, you get the problem you 
have with v3 OpenPGP keys, where somebody could move bytes from one 
value to another, creating a different key, but with the same 
fingerprint (hash value).

--
brian m. carlson / brian with sandals: Houston, Texas, US
+1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187


signature.asc
Description: Digital signature

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Duy Nguyen]

On Tue, Nov 25, 2014 at 10:47 AM, Jeff King p...@peff.net wrote:
 Maybe we can fix the tree-sorting order while we are at it. :)

At this speed, there a teeny tiny chance that pack v4 will be ready by
Git v3.0 and we can pile that on top of the new tree format. And we
don't have to worry about v3-v4 online conversion either because
it's backward incompatible ;)

 More seriously, there may come a day when we are ready to break
 compatibility completely with a new Git v3.0 (2.0 is already taken, of
 course). I do not have immediate plans for it, but it's possible that
 multiple factors may make such a move desirable sometime in the next 10
 years, and that would be a good time to jump hash algorithms, as well.

 So it's possible that procrastinating on SHA-1 issues may be the least
 painful route. Or it may just be pushing off the day of pain. :)
-- 
Duy
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[brian m. carlson]

On Tue, Nov 25, 2014 at 08:16:15AM +0700, Duy Nguyen wrote:
 On Tue, Nov 25, 2014 at 1:14 AM, Nico Williams n...@cryptonector.com wrote:
  Is there a plan for upgrading to a better hash function in the future?
   (E.g., should it become an urgent need.)
 
  What are the roadblocks to adoption of a replacement hash function?
  Just documenting this would go a long way towards making it possible
  to upgrade some day.
 
 The biggest obstacle is the assumption of SHA-1 everywhere in the
 source code (e.g. assume the object name always takes 20 bytes). Brian
 started on cleaning that up [1] but I think it's stalled. Then we need
 to deal with upgrade path for SHA-1 repos.

Yes, it is stalled.  It ended up being a Herculean task, so when I pick
up the patch series again, I'll probably submit changes in chunks to
avoid the huge amount of code churn required.  I feel the list and Junio
in particular will appreciate that more.
-- 
brian m. carlson / brian with sandals: Houston, Texas, US
+1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187


signature.asc
Description: Digital signature

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Duy Nguyen]

On Mon, Nov 24, 2014 at 06:44:10PM +0700, Duy Nguyen wrote:
  I wonder if we can have an option to sign all blob content of the tree
  associated to a commit, and the content of parent commit(s). It's more
  expensive than signing just commit/tag content. But it's also safer
  without completely ditching SHA-1.
 
 
  This amounts to hashing the blob content with whatever hash you told
  your gpg to use (hopefully not sha1 ;) ) and signing that.
 
  You're free to do that now and store the signature wherever your
  toolchain deems fit, say in a note or an annotated tag. But that
  approach won't sign the history, that is: If you are concerned about the
  breakability of sha1, then history is possibly broken no matter how
  you sign a commit object whose parent entry is based on the sha1 of
  its parent object.
 
 If you store the singature in commit message, and if you hash the
 parent commit messages as well, which are also signed, then you have
 the same chaining effect that we have with SHA-1. I think this could
 be done with some hooks already, except maybe for the verification
 part.

To demonstrate, a hook like this can take commit object from stdin and
produce some hash lines, which are appended at the end of the commit
message before the new commit object is created. So if I commit foo
the final commit message would be

  foo

  SHA512: long hash

This script uses sha512sum, but you can add as many hashes as you want
(and pay the penalty at commit time, of course). I think it covers
enough content to validate history up to the last signed commit.

-- 8 --
#!/bin/bash
# commit content
cat $GIT_DIR/tmp
# parent commit content
sed '/^$/q' $GIT_DIR/tmp |
grep '^parent ' |
cut -c 8- |
xargs -n1 git cat-file commit $GIT_DIR/tmp
# all blobs
sed '/^$/q' $GIT_DIR/tmp |
grep '^tree ' |
cut -c 6- |
xargs -n1 git ls-tree -r |
cut -c 13-52 |
git cat-file --batch $GIT_DIR/tmp
echo
echo SHA512: `sha512sum  $GIT_DIR/tmp`
-- 8 --

An extra patch is required to hook this in final commit steps.

-- 8 --
diff --git a/commit.c b/commit.c
index 19cf8f9..c447c1d 100644
--- a/commit.c
+++ b/commit.c
@@ -11,6 +11,8 @@
 #include commit-slab.h
 #include prio-queue.h
 #include sha1-lookup.h
+#include run-command.h
+#include sigchain.h
 
 static struct commit_extra_header *read_commit_extra_header_lines(const char 
*buf, size_t len, const char **);
 
@@ -1064,6 +1066,36 @@ struct commit_list *reduce_heads(struct commit_list 
*heads)
return result;
 }
 
+static int run_sign_commit_hook(struct strbuf *buf)
+{
+   struct child_process hook = CHILD_PROCESS_INIT;
+   const char *p = find_hook(sign-commit);
+   int len;
+
+   if (!p)
+   return 0;
+
+   argv_array_push(hook.args, p);
+   hook.in = -1;
+   hook.out = -1;
+   if (start_command(hook))
+   return error(_(could not run sign-commit hook));
+   sigchain_push(SIGPIPE, SIG_IGN);
+   if (write_in_full(hook.in, buf-buf, buf-len) != buf-len) {
+   close(hook.in);
+   close(hook.out);
+   finish_command(hook);
+   return error(_(sign-commit hook did not accept the data));
+   }
+   close(hook.in);
+   len = strbuf_read(buf, hook.out, 1024);
+   close(hook.out);
+   sigchain_pop(SIGPIPE);
+   if (finish_command(hook) || len = 0)
+   return error(_(sign-commit hook failed to sign the data));
+   return 0;
+}
+
 static const char gpg_sig_header[] = gpgsig;
 static const int gpg_sig_header_len = sizeof(gpg_sig_header) - 1;
 
@@ -1555,6 +1587,9 @@ int commit_tree_extended(const char *msg, size_t msg_len,
if (encoding_is_utf8  !verify_utf8(buffer))
fprintf(stderr, commit_utf8_warn);
 
+   if (run_sign_commit_hook(buffer))
+   return -1;
+
if (sign_commit  do_sign_commit(buffer, sign_commit))
return -1;
 
-- 8 --
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Fedor Brunner]

On 22.11.2014 00:01, Patrick Schleizer wrote:
 Dear git developers!
 
 Jeff King wrote:
 On Sun, Nov 16, 2014 at 03:31:10PM +, Patrick Schleizer wrote:

 How safe are signed git tags? Especially because git uses SHA-1. There
 is contradictory information around.

 So if one verifies a git tag (`git tag -v tagname`), then `checksout`s
 the tag, and checks that `git status` reports no untracked/modified
 files, without further manually auditing the code, how secure is this
 actually? Is it only as safe as SHA-1?

 Yes, it is only as safe as SHA-1 in the sense that you have GPG-signed
 only a SHA-1 hash. If somebody can find a collision with a hash you have
 signed, they can substitute the colliding data for the data you signed.

 Of course, safe as SHA-1 and find a collision are vague. If
 pre-image attacks are feasible (i.e., given already-published SHA-1, I
 can find a different input with the same SHA-1), then attacks are
 trivial. But when people talk about attacks on SHA-1, they are usually
 referring to finding a collision between two new pieces of data. You can
 also use that in an attack, but it's much less straightforward
 (basically, you need to get somebody to sign one of the colliding pieces
 of data and then replace it with the other).
 
 Sounds pretty sad. Isn't this a security issue that should be fixed?
 
 Rather than discussing how feasible collisions in SHA-1 are... Attacks
 on SHA-1 are only getting worse, no? Since the Snowden revelations we
 know that powerful adversaries that are working on such things and would
 use such weaknesses to exploit users.
 
 Dear git developers, could you please make a long story short? Change to
 some stronger hash algorithm? (sha256, sha512, or so?) Or provide an
 option for that?
 
 And of course there is the question of getting the colliding data to the
 victim. Git does collision checks whenever a remote (e.g., from a git
 fetch) gives us data that we already have. So you could poison new
 cloners with bad data, but you could not convince a repository with the
 existing good half of the collision to fetch the evil half.
 
 Poison git cloners with bad data is exactly my point here. Because
 sometimes I am a cloner of my own code - cloning it on a separate
 machine - then verify it using gpg - but don't check it any further. In
 such cases, I'd prefer if security wouldn't depend on SHA-1.
 
 Cheers,
 Patrick
 
 --
 To unsubscribe from this list: send the line unsubscribe git in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
 

Dear git developers,
how about skipping SHA-2 and moving to SHA-3 finalist BLAKE.

NIST, in the final report of the SHA-3 competition, said this about the
finalists (which included BLAKE, Keccak, Skein, and Grøstl):

BLAKE had a security margin — the gap between a known-weak reduced
version and the full version — comparable to Keccak and superior to the
other finalists. (§4.3: “BLAKE and Keccak have very large security
margins.”)
BLAKE had a depth of analysis — the amount of published research
analyzing its security — comparable to Grøstl and Skein and superior to
the other finalists. (§3.1: “Keccak received a significant amount of
cryptanalysis, although not quite the depth of analysis applied to
BLAKE, Grøstl, or Skein”)
BLAKE had performance (in software) comparable to Skein and superior
to the other finalists. (§5.1.4: “Skein and BLAKE […] have the best
overall software performance.”)

http://nvlpubs.nist.gov/nistpubs/ir/2012/NIST.IR.7896.pdf



Measurements of SHA-3 finalists, indexed by machine

http://bench.cr.yp.to/results-sha3.html


Fedor

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Junio C Hamano]

Jeff King p...@peff.net writes:

 On Tue, Nov 25, 2014 at 08:52:58AM +0700, Duy Nguyen wrote:

 On Tue, Nov 25, 2014 at 8:23 AM, Jonathan Nieder jrnie...@gmail.com wrote:
  I think the biggest obstacle is the upgrade path. ;-)
 
 In the worst case we can always treat new repos as a different VCS. So
 people will need a migration from SHA-1 to the new format, just like
 they migrate from SVN/CVS to Git. Painful but simple.

 Maybe we can fix the tree-sorting order while we are at it. :)

 More seriously, there may come a day when we are ready to break
 compatibility completely with a new Git v3.0 (2.0 is already taken, of
 course). I do not have immediate plans for it, but it's possible that
 multiple factors may make such a move desirable sometime in the next 10
 years, and that would be a good time to jump hash algorithms, as well.

As the fundamental data model of Git is built around given the
object name, the same data is retrieved, and equally importantly,
given the same data, the same object name is used, so that we can
say two objects with different names record different contents
without looking at the data, it does not mesh with the use of
tagged hash where object name consists of a pair of what hash is
used and what the hash value is at all.  It is a proper mindset
to treat it as a different VCS to give us a clean break when (I did
not say if) we need to switch hashes, I think.

Thanks.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Michael J Gruber]

Duy Nguyen schrieb am 24.11.2014 um 02:23:
 On Tue, Nov 18, 2014 at 4:26 AM, Jeff King p...@peff.net wrote:
 Yes, it is only as safe as SHA-1 in the sense that you have GPG-signed
 only a SHA-1 hash. If somebody can find a collision with a hash you have
 signed, they can substitute the colliding data for the data you signed.
 
 I wonder if we can have an option to sign all blob content of the tree
 associated to a commit, and the content of parent commit(s). It's more
 expensive than signing just commit/tag content. But it's also safer
 without completely ditching SHA-1.
 

This amounts to hashing the blob content with whatever hash you told
your gpg to use (hopefully not sha1 ;) ) and signing that.

You're free to do that now and store the signature wherever your
toolchain deems fit, say in a note or an annotated tag. But that
approach won't sign the history, that is: If you are concerned about the
breakability of sha1, then history is possibly broken no matter how
you sign a commit object whose parent entry is based on the sha1 of
its parent object.

Cheers
Michael
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Duy Nguyen]

On Mon, Nov 24, 2014 at 5:15 PM, Michael J Gruber
g...@drmicha.warpmail.net wrote:
 Duy Nguyen schrieb am 24.11.2014 um 02:23:
 On Tue, Nov 18, 2014 at 4:26 AM, Jeff King p...@peff.net wrote:
 Yes, it is only as safe as SHA-1 in the sense that you have GPG-signed
 only a SHA-1 hash. If somebody can find a collision with a hash you have
 signed, they can substitute the colliding data for the data you signed.

 I wonder if we can have an option to sign all blob content of the tree
 associated to a commit, and the content of parent commit(s). It's more
 expensive than signing just commit/tag content. But it's also safer
 without completely ditching SHA-1.


 This amounts to hashing the blob content with whatever hash you told
 your gpg to use (hopefully not sha1 ;) ) and signing that.

 You're free to do that now and store the signature wherever your
 toolchain deems fit, say in a note or an annotated tag. But that
 approach won't sign the history, that is: If you are concerned about the
 breakability of sha1, then history is possibly broken no matter how
 you sign a commit object whose parent entry is based on the sha1 of
 its parent object.

If you store the singature in commit message, and if you hash the
parent commit messages as well, which are also signed, then you have
the same chaining effect that we have with SHA-1. I think this could
be done with some hooks already, except maybe for the verification
part.
-- 
Duy
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Jeff King]

On Mon, Nov 24, 2014 at 11:15:34AM +0100, Michael J Gruber wrote:

  I wonder if we can have an option to sign all blob content of the tree
  associated to a commit, and the content of parent commit(s). It's more
  expensive than signing just commit/tag content. But it's also safer
  without completely ditching SHA-1.
  
 
 This amounts to hashing the blob content with whatever hash you told
 your gpg to use (hopefully not sha1 ;) ) and signing that.

Right. You could also create a graph of SHA-256 (or whatever) object
hashes and sign that. I.e., create a parallel to git's trees using
SHA-256 and include a single:

  object-256 

line in the tag header. That still involves re-hashing all of the data,
but it would at least be possible to cache (i.e., a mapping of SHA-1 to
SHA-256 hashes). Of course one way to keep that caching layer up to date
would be to just calculate the SHA-256 along with the SHA-1 whenever we
create an object. And then you can sprinkle SHA-256 links in other
places, too, like commit objects.

And now you are halfway down the road to a combined SHA-1/SHA-256 git.
:)

The tricky thing is fitting the extra hash into the tree objects. And of
course the rules for actually generating and/or sending extra objects.

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Nico Williams]

Is there a plan for upgrading to a better hash function in the future?
 (E.g., should it become an urgent need.)

What are the roadblocks to adoption of a replacement hash function?
Just documenting this would go a long way towards making it possible
to upgrade some day.

Thanks,

Nico
--
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Duy Nguyen]

On Tue, Nov 25, 2014 at 1:14 AM, Nico Williams n...@cryptonector.com wrote:
 Is there a plan for upgrading to a better hash function in the future?
  (E.g., should it become an urgent need.)

 What are the roadblocks to adoption of a replacement hash function?
 Just documenting this would go a long way towards making it possible
 to upgrade some day.

The biggest obstacle is the assumption of SHA-1 everywhere in the
source code (e.g. assume the object name always takes 20 bytes). Brian
started on cleaning that up [1] but I think it's stalled. Then we need
to deal with upgrade path for SHA-1 repos.

[1] http://thread.gmane.org/gmane.comp.version-control.git/248054
-- 
Duy
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Jonathan Nieder]

Duy Nguyen wrote:

 The biggest obstacle is the assumption of SHA-1 everywhere in the
 source code (e.g. assume the object name always takes 20 bytes). Brian
 started on cleaning that up [1] but I think it's stalled. Then we need
 to deal with upgrade path for SHA-1 repos.

I think the biggest obstacle is the upgrade path. ;-)

If the upgrade path is taken care of, I won't mind writing and
reviewing a coccinelle-generated patch to replace 20, 40, 21, 41, and
so on with appropriate constants.  Or we can take the first 20 bytes
of a SHA-256, which is already supposed to have better security
properties than SHA-1.

Another obstacle is hard-coded SHA-1s in tests.

Thanks,
Jonathan
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Duy Nguyen]

On Tue, Nov 25, 2014 at 8:23 AM, Jonathan Nieder jrnie...@gmail.com wrote:
 I think the biggest obstacle is the upgrade path. ;-)

In the worst case we can always treat new repos as a different VCS. So
people will need a migration from SHA-1 to the new format, just like
they migrate from SVN/CVS to Git. Painful but simple.
-- 
Duy
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Stefan Beller]

That's not *as* painful, because you'd have the beautiful
fast-{import/export} tools in your new and old version control system.
But yeah, there might be better ways to do so.


On Mon, Nov 24, 2014 at 5:52 PM, Duy Nguyen pclo...@gmail.com wrote:
 On Tue, Nov 25, 2014 at 8:23 AM, Jonathan Nieder jrnie...@gmail.com wrote:
 I think the biggest obstacle is the upgrade path. ;-)

 In the worst case we can always treat new repos as a different VCS. So
 people will need a migration from SHA-1 to the new format, just like
 they migrate from SVN/CVS to Git. Painful but simple.
 --
 Duy
 --
 To unsubscribe from this list: send the line unsubscribe git in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Jeff King]

On Tue, Nov 25, 2014 at 08:52:58AM +0700, Duy Nguyen wrote:

 On Tue, Nov 25, 2014 at 8:23 AM, Jonathan Nieder jrnie...@gmail.com wrote:
  I think the biggest obstacle is the upgrade path. ;-)
 
 In the worst case we can always treat new repos as a different VCS. So
 people will need a migration from SHA-1 to the new format, just like
 they migrate from SVN/CVS to Git. Painful but simple.

Maybe we can fix the tree-sorting order while we are at it. :)

More seriously, there may come a day when we are ready to break
compatibility completely with a new Git v3.0 (2.0 is already taken, of
course). I do not have immediate plans for it, but it's possible that
multiple factors may make such a move desirable sometime in the next 10
years, and that would be a good time to jump hash algorithms, as well.

So it's possible that procrastinating on SHA-1 issues may be the least
painful route. Or it may just be pushing off the day of pain. :)

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[bancfc]

Hi, I wanted to chime in on the topic of SHA1 weaknesses and breaks. The 
problem is idea that SHA1 breaks are theoretical and will only  be 
relevant in a decade or two.


I think its a telling sign when even companies like Google [1] and 
Microsoft [2] who collaborate with spy agencies are moving away from 
SHA1 in verifying browser certs and the estimates by reputable 
cryptographers already put us in the realm of feasible breaks at this 
time, with the bar going lower with every passing year [3]. In three 
years common cyber criminals will be able to crack it using moderate 
sized computer clusters or by renting some AWS cycles.


Please reconsider the urgency of moving away from SHA1 for security 
functions in Git.



References:

[1] 
http://thenextweb.com/google/2014/09/05/google-will-start-sunsetting-sha-1-cryptographic-hash-algorithm-chrome-month-finish-q1-2015/


[2] https://www.schneier.com/blog/archives/2013/11/microsoft_retir.html 
(Schneier on Security: Microsoft Retiring SHA-1 in 2016)


[3] https://www.schneier.com/blog/archives/2012/10/when_will_we_se.html 
(When Will We See Collisions for SHA-1?)

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Duy Nguyen]

On Tue, Nov 18, 2014 at 4:26 AM, Jeff King p...@peff.net wrote:
 Yes, it is only as safe as SHA-1 in the sense that you have GPG-signed
 only a SHA-1 hash. If somebody can find a collision with a hash you have
 signed, they can substitute the colliding data for the data you signed.

I wonder if we can have an option to sign all blob content of the tree
associated to a commit, and the content of parent commit(s). It's more
expensive than signing just commit/tag content. But it's also safer
without completely ditching SHA-1.
-- 
Duy
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Jeff King]

On Fri, Nov 21, 2014 at 11:01:26PM +, Patrick Schleizer wrote:

  Yes, it is only as safe as SHA-1 in the sense that you have GPG-signed
  only a SHA-1 hash. If somebody can find a collision with a hash you have
  signed, they can substitute the colliding data for the data you signed.
 [..]
 Sounds pretty sad. Isn't this a security issue that should be fixed?

Sure, for some definition of should. It's not a problem today. It may
be a problem in the future. If we were designing git from scratch today,
it would probably make sense to use a different hash, or to somehow
parameterize the hash.

But we're not starting from scratch. A change like that needs to
consider a transition plan. What happens to the existing history? Do we
just rewrite it all using the new hash in all object references? If so,
what do we do with non-object references to sha1s (in external systems,
or even partial sha1s mentioned in commit message)? What do we do with
signed tags which are now invalid?  Or do we graft history with the new
hashes onto the old, letting the two coexist in the same repository? How
do we expand the data structures to handle the extra information needed
to specify the hash type?

None of these problems is insurmountable, but it's going to take real
work on the development side, and is going to create incompatibilities
and headaches on the user side. It's probably something we'll need to
deal with in the next 10-15 years, but nobody knows quite when.

If you'd like to start working on it, I'd be happy to review your
patches. :) But in the meantime, I don't know that anybody considers it
an extremely high priority to work on, versus other fixes and features.

 Rather than discussing how feasible collisions in SHA-1 are... Attacks
 on SHA-1 are only getting worse, no?

Actually, not really. I do not keep up terribly well with the academic
literature, but I don't think that attacks have gotten any worse in the
last few years. Computers _are_ getting faster, though, so a number like
2^61 (which is what Wikipedia claims as the best widely accepted value
for producing a collision) gets more and more feasible as time passes.

Of course, we might find worse attacks (or if you want to put on your
tinfoil hat, perhaps certain government organizations already have and
are keeping them secret). 2^61 is a best case.

  And of course there is the question of getting the colliding data to the
  victim. Git does collision checks whenever a remote (e.g., from a git
  fetch) gives us data that we already have. So you could poison new
  cloners with bad data, but you could not convince a repository with the
  existing good half of the collision to fetch the evil half.
 
 Poison git cloners with bad data is exactly my point here. Because
 sometimes I am a cloner of my own code - cloning it on a separate
 machine - then verify it using gpg - but don't check it any further. In
 such cases, I'd prefer if security wouldn't depend on SHA-1.

I agree that cloners are an important category of users to clone. But it
also means that a single fetcher can detect tampering quite easily.
Think about it this way: let's say the Walker/Schneier estimate is
right, and in 2021 it will cost ~$43K to find a collision. You spend the
money, find a collision on some binary blob that's in the kernel,
convince Linus to accept your good version, he signs, and then you
hack into kernel.org and replace the blob with your evil version. Now
the first time somebody fetches the evil version, their git complains
about the collision, kernel.org admins investigate, and the problem is
fixed. There's some damage, but ultimately you didn't accomplish much.

Or you could spend that $43K hiring somebody to break into Linus's house
and manipulate the local copy of the kernel on his computer that he's
going to sign. Or buy a zero-day exploit for his browser that gives you
remote code execution on his workstation.

Don't get me wrong. I think moving away from SHA-1 is a good idea, and
something we're going to want to do for security reasons eventually. But
we're definitely not at the point of well, all of our signatures are
worthless now, and I'm not sure we'll be there sooner than a decade
from now.

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Jeff King]

On Fri, Nov 21, 2014 at 06:32:46PM -0500, Jason Pyeron wrote:

 The whole issue is a lot better than this makes it sound. Yes it is
 just a SHA1 hash, but it is a hash of a structured data format.
 
 You have very observable parts of that well structured data providede to the 
 hash.

Yeah, I glossed over that because I don't know enough about the specific
attacks.  In the worst case, you have a binary file format that lets
people stick arbitrary bits of data in the middle (like the MD5 attacks
on Postscript and PDF files), and you do the collision on the blobs.

But even with that, the sha1s are taken over blob n\0content where
n is the number of bytes of content. Depending on the exact scheme
for generating probable collisions is less than brute force time, even
that amount of structure may prove problematic. I don't know whether
that is the case for the best-known attacks or not (remember that nobody
has _actually_ generated a sha-1 collision at all yet).

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Patrick Schleizer]

Dear git developers!

Jeff King wrote:
 On Sun, Nov 16, 2014 at 03:31:10PM +, Patrick Schleizer wrote:
 
 How safe are signed git tags? Especially because git uses SHA-1. There
 is contradictory information around.

 So if one verifies a git tag (`git tag -v tagname`), then `checksout`s
 the tag, and checks that `git status` reports no untracked/modified
 files, without further manually auditing the code, how secure is this
 actually? Is it only as safe as SHA-1?
 
 Yes, it is only as safe as SHA-1 in the sense that you have GPG-signed
 only a SHA-1 hash. If somebody can find a collision with a hash you have
 signed, they can substitute the colliding data for the data you signed.
 
 Of course, safe as SHA-1 and find a collision are vague. If
 pre-image attacks are feasible (i.e., given already-published SHA-1, I
 can find a different input with the same SHA-1), then attacks are
 trivial. But when people talk about attacks on SHA-1, they are usually
 referring to finding a collision between two new pieces of data. You can
 also use that in an attack, but it's much less straightforward
 (basically, you need to get somebody to sign one of the colliding pieces
 of data and then replace it with the other).

Sounds pretty sad. Isn't this a security issue that should be fixed?

Rather than discussing how feasible collisions in SHA-1 are... Attacks
on SHA-1 are only getting worse, no? Since the Snowden revelations we
know that powerful adversaries that are working on such things and would
use such weaknesses to exploit users.

Dear git developers, could you please make a long story short? Change to
some stronger hash algorithm? (sha256, sha512, or so?) Or provide an
option for that?

 And of course there is the question of getting the colliding data to the
 victim. Git does collision checks whenever a remote (e.g., from a git
 fetch) gives us data that we already have. So you could poison new
 cloners with bad data, but you could not convince a repository with the
 existing good half of the collision to fetch the evil half.

Poison git cloners with bad data is exactly my point here. Because
sometimes I am a cloner of my own code - cloning it on a separate
machine - then verify it using gpg - but don't check it any further. In
such cases, I'd prefer if security wouldn't depend on SHA-1.

Cheers,
Patrick

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Jason Pyeron]

 -Original Message-
 From: Patrick Schleizer
 Sent: Friday, November 21, 2014 18:01
 
 Dear git developers!
 
 Jeff King wrote:
  On Sun, Nov 16, 2014 at 03:31:10PM +, Patrick Schleizer wrote:
  
  How safe are signed git tags? Especially because git uses 
 SHA-1. There
  is contradictory information around.
 
  So if one verifies a git tag (`git tag -v tagname`), then 
 `checksout`s
  the tag, and checks that `git status` reports no untracked/modified
  files, without further manually auditing the code, how 
 secure is this
  actually? Is it only as safe as SHA-1?
  
  Yes, it is only as safe as SHA-1 in the sense that you 
 have GPG-signed
  only a SHA-1 hash. If somebody can find a collision with a 
 hash you have
  signed, they can substitute the colliding data for the data 
 you signed.

The whole issue is a lot better than this makes it sound. Yes it is just a SHA1 
hash, but it is a hash of a structured data format.

You have very observable parts of that well structured data providede to the 
hash.

The commit message, the directory contents, and lastly the files themselves.

For a collision to occur, the commit message would have to likely have garbage 
in the message of a large nature. To generate a colision by commited file 
contents is unlikely because the file contents is reduced to a hash in the 
directory structure, which is in turn reduced to a hash in a commit structure.

This would be noticed.


--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
-   -
- Jason Pyeron  PD Inc. http://www.pdinc.us -
- Principal Consultant  10 West 24th Street #100-
- +1 (443) 269-1555 x333Baltimore, Maryland 21218   -
-   -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
This message is copyright PD Inc, subject to license 20080407P00. 

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Jeff King]

On Sun, Nov 16, 2014 at 03:31:10PM +, Patrick Schleizer wrote:

 How safe are signed git tags? Especially because git uses SHA-1. There
 is contradictory information around.
 
 So if one verifies a git tag (`git tag -v tagname`), then `checksout`s
 the tag, and checks that `git status` reports no untracked/modified
 files, without further manually auditing the code, how secure is this
 actually? Is it only as safe as SHA-1?

Yes, it is only as safe as SHA-1 in the sense that you have GPG-signed
only a SHA-1 hash. If somebody can find a collision with a hash you have
signed, they can substitute the colliding data for the data you signed.

Of course, safe as SHA-1 and find a collision are vague. If
pre-image attacks are feasible (i.e., given already-published SHA-1, I
can find a different input with the same SHA-1), then attacks are
trivial. But when people talk about attacks on SHA-1, they are usually
referring to finding a collision between two new pieces of data. You can
also use that in an attack, but it's much less straightforward
(basically, you need to get somebody to sign one of the colliding pieces
of data and then replace it with the other).

And of course there is the question of getting the colliding data to the
victim. Git does collision checks whenever a remote (e.g., from a git
fetch) gives us data that we already have. So you could poison new
cloners with bad data, but you could not convince a repository with the
existing good half of the collision to fetch the evil half.

  Git uses SHA-1 not for security
 
 And goes on.
 
  The security parts are elsewhere
 
 Could you please elaborate on this? Where are the security parts? Can
 you please briefly explain how these work? Where can I read more about this?

I cannot speak for Linus, but I would not agree that SHA-1 is not part
of Git's security model. If we consider the GPG signature as a black box
(and we largely do in Git), then we _never_ sign the tree contents
itself. We always sign the SHA-1 of the tree, along with some metadata
(whether you are signing a tag or a commit). If an attacker can create
SHA-1 collisions (either by pre-image, or if you agree to sign a tree
containing a potential collision from an attacker), then you are
vulnerable to having the tree contents swapped out after the fact (and
the signature still checking out).

I am not sure that is what Linus is saying, though. In the paragraph you
quote:

  The source control management system Git uses SHA-1 not for security
 but for ensuring that the data has not changed due to accidental
 corruption. Linus Torvalds has said, If you have disk corruption, if
 you have DRAM corruption, if you have any kind of problems at all, Git
 will notice them. It's not a question of if, it's a guarantee. You can
 have people who try to be malicious. They won't succeed. [...] Nobody
 has been able to break SHA-1, but the point is the SHA-1, as far as Git
 is concerned, isn't even a security feature. It's purely a consistency
 check. The security parts are elsewhere, so a lot of people assume that
 since Git uses SHA-1 and SHA-1 is used for cryptographically secure
 stuff, they think that, OK, it's a huge security feature. It has nothing
 at all to do with security, it's just the best hash you can get. [...] I
 guarantee you, if you put your data in Git, you can trust the fact that
 five years later, after it was converted from your hard disk to DVD to
 whatever new technology and you copied it along, five years later you
 can verify that the data you get back out is the exact same data you put
 in. [...] One of the reasons I care is for the kernel, we had a break in
 on one of the BitKeeper sites where people tried to corrupt the kernel
 source code repositories. [6]

I think he is saying more SHA-1 is about data integrity, not about
authenticity; if you want authenticity, that's elsewhere [handled by
gpg]. Unsaid there is that you can't really have authenticity without
the integrity, and that I think he was assuming that SHA-1 works (he
says Nobody has been able to break SHA-1..).

 If (!) I understand Mike Gerwitz ([...] GNU [...]) 's opinion, his
 opinion is, that for best security each and every commit should be
 signed for best possible git verification security.

I think it would depend on your threat model. You haven't defined best
security.

Even without a break in SHA-1, there is value to signing every commit
versus signing just tags. If Linus signs a tag, all it says is I think
this tree state and the history leading up to it is called v3.16. But
Linus can lie all he likes about who made each commit, and what happened
in each one. Signing commits is more about authenticating individual
commits: who made them, what was the state they were based on, and what
was the end state (and between the two, you can calculate the introduced
changes).

Of course, that comes with its own headaches, too. E.g., in mailing-list
development, the patch is picked up and applied by a maintainer

How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Patrick Schleizer]

Hi!

How safe are signed git tags? Especially because git uses SHA-1. There
is contradictory information around.

So if one verifies a git tag (`git tag -v tagname`), then `checksout`s
the tag, and checks that `git status` reports no untracked/modified
files, without further manually auditing the code, how secure is this
actually? Is it only as safe as SHA-1?

Let's assume an adversary, that is capable of producing SHA-1 collisions.

Linus Torvalds said: [1]

 Git uses SHA-1 not for security

And goes on.

 The security parts are elsewhere

Could you please elaborate on this? Where are the security parts? Can
you please briefly explain how these work? Where can I read more about this?

Wikipedia says. [2]

 Nonetheless, without second preimage resistance [3] of SHA-1 signed
commits and tags would no longer secure the state of the repository as
they only sign the root of a Merkle tree [4].

Which contradicts what Linus Torvalds said. What does that mean for
security? Which statement is true?

 The source control management system Git uses SHA-1 not for security
but for ensuring that the data has not changed due to accidental
corruption. Linus Torvalds has said, If you have disk corruption, if
you have DRAM corruption, if you have any kind of problems at all, Git
will notice them. It's not a question of if, it's a guarantee. You can
have people who try to be malicious. They won't succeed. [...] Nobody
has been able to break SHA-1, but the point is the SHA-1, as far as Git
is concerned, isn't even a security feature. It's purely a consistency
check. The security parts are elsewhere, so a lot of people assume that
since Git uses SHA-1 and SHA-1 is used for cryptographically secure
stuff, they think that, OK, it's a huge security feature. It has nothing
at all to do with security, it's just the best hash you can get. [...] I
guarantee you, if you put your data in Git, you can trust the fact that
five years later, after it was converted from your hard disk to DVD to
whatever new technology and you copied it along, five years later you
can verify that the data you get back out is the exact same data you put
in. [...] One of the reasons I care is for the kernel, we had a break in
on one of the BitKeeper sites where people tried to corrupt the kernel
source code repositories. [6]

If (!) I understand Mike Gerwitz ([...] GNU [...]) 's opinion, his
opinion is, that for best security each and every commit should be
signed for best possible git verification security.

See also:

- Mike Gerwitz's A Git Horror Story: Repository Integrity With Signed
Commits [7]

- Verbose reply by Mike Gerwitz to my question. [8]

- Similar question on security stackexchange. [9] Quote: Nevertheless,
If somebody managed to find a way how to find SHA1 collisions easily,
then git would have much bigger problem.

Cheers,
Patrick

[1] https://www.youtube.com/watch?v=4XpnKHJAok8t=56m20s
[2] https://en.wikipedia.org/wiki/SHA-1#Data_integrity
[3] https://en.wikipedia.org/wiki/Second_preimage_resistance
[4] https://en.wikipedia.org/wiki/Merkle_tree
[5] https://www.youtube.com/watch?v=4XpnKHJAok8t=56m20s
[6] https://en.wikipedia.org/wiki/SHA-1#Data_integrity
[7] http://mikegerwitz.com/papers/git-horror-story
[8] https://www.whonix.org/forum/index.php/topic,538.msg4278.html#msg4278
[9]
https://security.stackexchange.com/questions/67920/how-safe-are-signed-git-tags-only-as-safe-as-sha-1-or-somehow-safer

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html