Re: git & Debian packaging sprint report

[Sean Whitton]

Hello,

On Tue 23 Jul 2019 at 08:13PM +02, Ansgar Burchardt wrote:

> There are also other useful properties the current implementation has:
> for example the archive contains the artifact that was signed.  This can
> be checked at a later time unlike a Git tag on salsa.d.o that may or may
> not exist in the future.  It is always possible to go back to the key
> that was used to introduce an artifact without having to trust any
> system.

I mentioned this to you on IRC, but for the benefit of others reading,
the DD-signed tag gets pushed to dgit-repos by the intermediary service,
and cannot be deleted from there except by the service admin.

Thus, with tag2upload it is possible to go back to the key that
introduced the artifact in the way you describe.

>> But
>> if it's not simple to pick a different hash, SHA-1 preimage resistance is
>> reasonable and the other design properties of the system should dominate
>> any concern about SHA-1 preimage attacks.
>
> What about MD5's preimage resistance?  We don't really care about
> collision attacks after all.

I can't see how MD5 is relevant to this discussion.

> We have most infrastructure already using SHA-2 and there are
> preparations to support newer hashes as well; it is a regression to
> introduce a new system bound to SHA1 for the foreseeable future (and
> given Git's use of SHA1 their need to require a strong hash is far
> less).

I think Russ' point is that this regression is a reasonable one, given
the benefits against which it is to be traded off.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: git & Debian packaging sprint report

[Ansgar Burchardt]

Russ Allbery writes:
> Scott Kitterman  writes:
>> Except that we have different requirements than git.  Git isn't looking
>> for security properties from SHA-1, so it's highly likely it'll meet
>> their accident avoidance requirements long after it's no longer
>> appropriate for security related assertions.
>
>> I don't think adding more SHA-1 in a security sensitive context is a
>> good plan.
>
> I talked this over briefly in the security pod at work with some other
> security engineers who know more crypto than I do to sanity-check my
> initial opinion.
>
> The consensus among all of us was that if you have an opportunity to pick
> something other than SHA-1 when designing a new protocol, you should.

We have that opportunity here, so I guess we should then? :-)

There are also other useful properties the current implementation has:
for example the archive contains the artifact that was signed.  This can
be checked at a later time unlike a Git tag on salsa.d.o that may or may
not exist in the future.  It is always possible to go back to the key
that was used to introduce an artifact without having to trust any
system.

> But
> if it's not simple to pick a different hash, SHA-1 preimage resistance is
> reasonable and the other design properties of the system should dominate
> any concern about SHA-1 preimage attacks.

What about MD5's preimage resistance?  We don't really care about
collision attacks after all.

We have most infrastructure already using SHA-2 and there are
preparations to support newer hashes as well; it is a regression to
introduce a new system bound to SHA1 for the foreseeable future (and
given Git's use of SHA1 their need to require a strong hash is far
less).

Ansgar

Re: git & Debian packaging sprint report

[Raphael Hertzog]

On Sun, 21 Jul 2019, Ian Jackson wrote:
> IME as a sponsor I get (AFAICT) no mails as a result of my sponsorship
> signatures so I think there are few automatic processes.

That's actualy not true, dak is sending mails to the person who signs the
upload when it has to send mails like NEW notifications, etc.

> Hrm, I think tracker may become confused.  It seems to be looking at the
> .dsc signatur.  So maybe the .dsc field is indeed needed.

Yes, definitely. 

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: https://www.freexian.com/services/debian-lts.html
Learn to master Debian: https://debian-handbook.info/get/

Re: git & Debian packaging sprint report

[Russ Allbery]

Ian Jackson  writes:

> Ie I think you would consider this a blocker for you to use it as a
> sponsor.  Do you consider this a blocker for deployment at all ?

No, it's just a regular bug (in something), at least to me.

-- 
Russ Allbery (r...@debian.org)   

Re: git & Debian packaging sprint report

[Ian Jackson]

Russ Allbery writes ("Re: git & Debian packaging sprint report"):
> What I probably should have said from the start is that the place where
> this information shows up that I personally care about is at:
> 
> https://qa.debian.org/developer.php?login=rra&comaint=yes
> 
> which shows the packages that I've sponsored.  I don't really care how the
> information shows up there, but I think it's valuable, and I'd hate to
> lose it.

Right.  Well that may need some updates to qa or tracker, as well as
something done to the bot code, so maybe you wouldn't want to use this
service right away.

Ie I think you would consider this a blocker for you to use it as a
sponsor.  Do you consider this a blocker for deployment at all ?

Ian.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Re: git & Debian packaging sprint report

[Russ Allbery]

Ian Jackson  writes:
> Russ Allbery writes ("Re: git & Debian packaging sprint report"):
>> Ian Jackson  writes:

>>> I think in general those places are probably mistakes.  But I'm not
>>> aware of all of them.  One way to look at this is that from the
>>> archive's point of view this robot is a kind of sponsor.  I don't
>>> think anything will go badly wrong.

>> What if I'm actually sponsoring a package and use this tool to upload
>> it?  I feel like overwriting the sponsor information for the package
>> would lose information.

> I can see where you're coming from.  The information is not lost,
> however; it's just somewhere else: namely the DEP-14 debian/
> tag which will be signed by you and not by your sponsee.

What I probably should have said from the start is that the place where
this information shows up that I personally care about is at:

https://qa.debian.org/developer.php?login=rra&comaint=yes

which shows the packages that I've sponsored.  I don't really care how the
information shows up there, but I think it's valuable, and I'd hate to
lose it.

-- 
Russ Allbery (r...@debian.org)   <http://www.eyrie.org/~eagle/>

Re: git & Debian packaging sprint report

[Ian Jackson]

Russ Allbery writes ("Re: git & Debian packaging sprint report"):
> Ian Jackson  writes:
> > I think in general those places are probably mistakes.  But I'm not
> > aware of all of them.  One way to look at this is that from the
> > archive's point of view this robot is a kind of sponsor.  I don't
> > think anything will go badly wrong.
> 
> What if I'm actually sponsoring a package and use this tool to upload it?
> I feel like overwriting the sponsor information for the package would lose
> information.

I can see where you're coming from.  The information is not lost,
however; it's just somewhere else: namely the DEP-14 debian/
tag which will be signed by you and not by your sponsee.

If that turns out to be not sufficient, the robot could put the
git-debpush user tag signing key information in a field in the .dsc.

IME as a sponsor I get (AFAICT) no mails as a result of my sponsorship
signatures so I think there are few automatic processes.  Hrm, I think
tracker may become confused.  It seems to be looking at the .dsc
signatur.  So maybe the .dsc field is indeed needed.

Ian.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Re: git & Debian packaging sprint report

[Russ Allbery]

Ian Jackson  writes:
> Russ Allbery writes ("Re: git & Debian packaging sprint report"):

>> If so, I think that security model is roughly equivalent to the
>> automatic signing of binary packages by buildds, so probably doesn't
>> introduce a new vulnerability, but my understanding was that the
>> identity of the signature on the source package was used in various
>> other places.  Presumably we would need to introduce some new metadata
>> so that the uploader is mapped properly to the Git tag signer, rather
>> than to some internal identity of the source package construction
>> service.

> I think in general those places are probably mistakes.  But I'm not
> aware of all of them.  One way to look at this is that from the
> archive's point of view this robot is a kind of sponsor.  I don't
> think anything will go badly wrong.

What if I'm actually sponsoring a package and use this tool to upload it?
I feel like overwriting the sponsor information for the package would lose
information.

-- 
Russ Allbery (r...@debian.org)   <http://www.eyrie.org/~eagle/>

Re: git & Debian packaging sprint report

[Ian Jackson]

Hi.  Not replying to things others have dealt with, but...

Russ Allbery writes ("Re: git & Debian packaging sprint report"):
> If so, I think that security model is roughly equivalent to the automatic
> signing of binary packages by buildds, so probably doesn't introduce a new
> vulnerability, but my understanding was that the identity of the signature
> on the source package was used in various other places.  Presumably we
> would need to introduce some new metadata so that the uploader is mapped
> properly to the Git tag signer, rather than to some internal identity of
> the source package construction service.

I think in general those places are probably mistakes.  But I'm not
aware of all of them.  One way to look at this is that from the
archive's point of view this robot is a kind of sponsor.  I don't
think anything will go badly wrong.

> Also, doesn't the archive publish the signed *.dsc files currently?  I
> believe this would mean that we would lose some published information from
> those files that we currently have (namely which DD and which key signed
> the package, which could be useful data in some incident response
> scenarios).  That said, there's been some discussion for some time about
> having the archive sign all the *.dsc files instead of keeping the
> uploader signature, which may be from an expired or unverifiable key
> (particularly for packages that haven't been uploaded in some time).

The .dsc signatures are not really useful for most practical purposes
because their validity and semantics are time-varying and only the
archive (and things which keep up to date with the archive-published
metadata) can reliably make sense of them - and even then, only at the
time of upload.

The tag2upload model of course preserves the original uploader's
identity in the form of the signature on the tag they make to instruct
the robot.

Ian.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Re: git & Debian packaging sprint report

[Scott Kitterman]

On July 16, 2019 3:37:04 PM UTC, Russ Allbery  wrote:
>Scott Kitterman  writes:
>
>> Except that we have different requirements than git.  Git isn't
>looking
>> for security properties from SHA-1, so it's highly likely it'll meet
>> their accident avoidance requirements long after it's no longer
>> appropriate for security related assertions.
>
>> I don't think adding more SHA-1 in a security sensitive context is a
>> good plan.
>
>I talked this over briefly in the security pod at work with some other
>security engineers who know more crypto than I do to sanity-check my
>initial opinion.
>
>The consensus among all of us was that if you have an opportunity to
>pick
>something other than SHA-1 when designing a new protocol, you should. 
>But
>if it's not simple to pick a different hash, SHA-1 preimage resistance
>is
>reasonable and the other design properties of the system should
>dominate
>any concern about SHA-1 preimage attacks.  If the system is vulnerable
>to
>collisions, that's another matter; there are viable SHA-1 collisions. 
>But
>given the design described, I don't think it is.  (That said, designing
>the system for hash agility if possible is certainly recommended.)

Thank you for researching this.  I'm less discomfited about SHA-1 in the 
short-term now.  I still don't think that in the long run assuming Git will 
solve algorithm agility is the right answer since we do have different 
requirements.

For now, I think it's enough to be able to describe the path that could be 
followed if we conclude we need to move forward to something new before Git.  I 
think the actuality of it seems far enough off that there's no need to actually 
implement an alternative algorithm now.

Scott K

Re: git & Debian packaging sprint report

[Sean Whitton]

Hello,

On Tue 16 Jul 2019 at 10:45AM +02, Ansgar Burchardt wrote:

> A "source package" generally consists of:
>  - a set of upstream artifacts (currently one or more tarballs,
>signatures); can be the empty set for native packages
>  - Debian-specific artifacts
>  - the .dsc artifact (generated from the Debian part); consists of:
>- strong cryptographic hashes of all other artifacts
>  (Checksum-*, Files, ...)
>- convenience information extracted from Debian-specific artifacts
>  (Build-Depends, ...)
>
> Currently we represent all of these as real files that need to be
> provided by the uploader.
>
> If you no longer want "source packages" as we have currently, you just
> define another representation as the canonical one.
>
> So I would like to just change how developers can provide artifacts to
> the archive: upstream artifacts can be specified as either a set of
> URLs to retrieve them from or a Git tree; Debian-specific artifacts can
> be a Git tree or (for source format 1.0) a diff between two Git trees.
>
> All artifacts provided must be covered by a strong cryptographic hash
> which is signed by a developer.  The hash must not only cover the Git
> tree object itself, but also all content covered by it.
>
> We currently have "tar" as the serialization format covered by the
> strong cryptographic hash and, as I value having a representation of
> upstream artifacts in the published archive, believe we should continue
> to provide the "tar" files.  However this is not necessary; we could
> instead provide only the Git tree.
>
> Currently this proposal should also allow multi-package repositories,
> packaging-only repositories, packages using multiple upstream
> artifacts, and ensuring Debian uses the same artifact as upstream.  It
> does not require any integrity from the VCS system.
>
> I think the .dsc artifact should eventually be split into the two
> parts: the list of artifacts (together with strong cryptographic hashes
> and where to locate them) signed by the uploader, and the convenience
> information extracted from these.  The second part should be generated
> by the archive software.

This is interesting.  Thank you for explaining.

For the vast majority of packages, the git-debpush/tag2upload system
enables us to do away with a lot of the complexity that you describe
here.  It brings the Debian way of representing, using and transmitting
source artifacts much closer to how upstream projects do things.

I think it is good for Debian development, downstreams and users to be
able to do away with that complexity when it is possible to do so.

We can't apply git-debpush/tag2upload to all packages -- for example,
packages with large binary datafiles which we wouldn't want to check
into git.  Or, as you mention, packages in monorepos.  Perhaps
maintaining those packages could be made easier by implementing
something like your proposal.

In the meantime, let's deploy the tag2upload service, which has already
been coded up, and enable git pushes to upload for the vast majority of
packages.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: git & Debian packaging sprint report

[Sean Whitton]

Hello,

On Tue 16 Jul 2019 at 08:37AM -07, Russ Allbery wrote:

> The consensus among all of us was that if you have an opportunity to pick
> something other than SHA-1 when designing a new protocol, you should.  But
> if it's not simple to pick a different hash, SHA-1 preimage resistance is
> reasonable and the other design properties of the system should dominate
> any concern about SHA-1 preimage attacks.  If the system is vulnerable to
> collisions, that's another matter; there are viable SHA-1 collisions.  But
> given the design described, I don't think it is.  (That said, designing
> the system for hash agility if possible is certainly recommended.)

Thanks for this.

tag2upload is equally as hash-agile as git.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: git & Debian packaging sprint report

[Russ Allbery]

Scott Kitterman  writes:

> Except that we have different requirements than git.  Git isn't looking
> for security properties from SHA-1, so it's highly likely it'll meet
> their accident avoidance requirements long after it's no longer
> appropriate for security related assertions.

> I don't think adding more SHA-1 in a security sensitive context is a
> good plan.

I talked this over briefly in the security pod at work with some other
security engineers who know more crypto than I do to sanity-check my
initial opinion.

The consensus among all of us was that if you have an opportunity to pick
something other than SHA-1 when designing a new protocol, you should.  But
if it's not simple to pick a different hash, SHA-1 preimage resistance is
reasonable and the other design properties of the system should dominate
any concern about SHA-1 preimage attacks.  If the system is vulnerable to
collisions, that's another matter; there are viable SHA-1 collisions.  But
given the design described, I don't think it is.  (That said, designing
the system for hash agility if possible is certainly recommended.)

-- 
Russ Allbery (r...@debian.org)   

Re: git & Debian packaging sprint report

[Michael Kesper]

Hi Sean,

On 15.07.19 19:02, Sean Whitton wrote:
> On Mon 15 Jul 2019 at 01:16PM +02, Michael Kesper wrote:
> 
>> Nonetheless it seems to me you are moving from trusting local signing
>> to trusting upload by salsa, thereby making salsa more attractive for
>> attackers.
> 
> I don't follow -- the git tag is PGP-signed, locally, by the uploader.
> Just like how they would PGP-sign, locally, the .dsc and .changes.

Ah ok, sorry, this wasn't clear to me.

Michael
 




signature.asc
Description: OpenPGP digital signature

Re: git & Debian packaging sprint report

[Ansgar Burchardt]

On Tue, 2019-07-16 at 08:29 +0100, Sean Whitton wrote:
> We also rely on git for security elsewhere.  For example, dak is
> deployed by ftpmasters pushing a signed git tag to salsa; a cronjob on
> ftpmaster then deploys that code.  That's relying on SHA-1 in pretty
> much the same way as tag2upload does, AFAICT.

That is true and I don't like it.  I should probably add a sha2 hash
somewhere.  (Note that we *can* just change it...)

> On Mon 15 Jul 2019 at 10:43PM +02, Ansgar Burchardt wrote:
> > It also has one downside: `git tag` alone won't be enough to generate
> > the required information, but then a special-purpose tool was proposed
> > here already.
> > 
> > The client tool could possibly also just create the .dsc and .changes,
> > except for hashes of the compressed files, and the web service just
> > recreate the tarball and compress them.  That would require near zero
> > trust in the web service, but still allow developers to no longer upload
> > source packages which might be large.  (A bit similar to not having to
> > trust buildds by making packages reproducible.)
> 
> This is certainly an interesting proposal and it would be better for
> users than using dput, as you say.
> 
> An important advantage of the tag2upload solution over such a thing,
> aside from just the fact that it already exists today, is that it allows
> us to move (slowly!) towards replacing source packages with git trees,
> like the rest of the world has.
> 
> git-debpush is such a simple wrapper around git-tag and git-push that
> its users really are uploading only by pushing a signed tag.  Your
> proposal would bake source packages back into the upload process in a
> way that will make it harder to ever get rid of them.

No, my proposal allows to stop generating the "source package" as a set
of real files at any given time.

A "source package" generally consists of:
 - a set of upstream artifacts (currently one or more tarballs,
   signatures); can be the empty set for native packages
 - Debian-specific artifacts
 - the .dsc artifact (generated from the Debian part); consists of:
   - strong cryptographic hashes of all other artifacts
 (Checksum-*, Files, ...)
   - convenience information extracted from Debian-specific artifacts
 (Build-Depends, ...)

Currently we represent all of these as real files that need to be
provided by the uploader.

If you no longer want "source packages" as we have currently, you just
define another representation as the canonical one.

So I would like to just change how developers can provide artifacts to
the archive: upstream artifacts can be specified as either a set of
URLs to retrieve them from or a Git tree; Debian-specific artifacts can
be a Git tree or (for source format 1.0) a diff between two Git trees.

All artifacts provided must be covered by a strong cryptographic hash
which is signed by a developer.  The hash must not only cover the Git
tree object itself, but also all content covered by it.

We currently have "tar" as the serialization format covered by the
strong cryptographic hash and, as I value having a representation of
upstream artifacts in the published archive, believe we should continue
to provide the "tar" files.  However this is not necessary; we could
instead provide only the Git tree.

Currently this proposal should also allow multi-package repositories,
packaging-only repositories, packages using multiple upstream
artifacts, and ensuring Debian uses the same artifact as upstream.  It
does not require any integrity from the VCS system.

I think the .dsc artifact should eventually be split into the two
parts: the list of artifacts (together with strong cryptographic hashes
and where to locate them) signed by the uploader, and the convenience
information extracted from these.  The second part should be generated
by the archive software.

Ansgar

Re: git & Debian packaging sprint report

[Sean Whitton]

Hello,

On Mon 15 Jul 2019 at 12:47PM -07, Russ Allbery wrote:

> I'm dubious that we really care that much about a preimage attack on
> SHA-1, [...]

Someone suggested on IRC that such an attack on tag2upload is even less
likely to be possible because each preimage has to be something which
dpkg-source will successfully make into a source package with the same
source package name, version etc.

We also rely on git for security elsewhere.  For example, dak is
deployed by ftpmasters pushing a signed git tag to salsa; a cronjob on
ftpmaster then deploys that code.  That's relying on SHA-1 in pretty
much the same way as tag2upload does, AFAICT.

On Mon 15 Jul 2019 at 10:43PM +02, Ansgar Burchardt wrote:

> It also has one downside: `git tag` alone won't be enough to generate
> the required information, but then a special-purpose tool was proposed
> here already.
>
> The client tool could possibly also just create the .dsc and .changes,
> except for hashes of the compressed files, and the web service just
> recreate the tarball and compress them.  That would require near zero
> trust in the web service, but still allow developers to no longer upload
> source packages which might be large.  (A bit similar to not having to
> trust buildds by making packages reproducible.)

This is certainly an interesting proposal and it would be better for
users than using dput, as you say.

An important advantage of the tag2upload solution over such a thing,
aside from just the fact that it already exists today, is that it allows
us to move (slowly!) towards replacing source packages with git trees,
like the rest of the world has.

git-debpush is such a simple wrapper around git-tag and git-push that
its users really are uploading only by pushing a signed tag.  Your
proposal would bake source packages back into the upload process in a
way that will make it harder to ever get rid of them.

I appreciate that thanks to the SHA-1 thing you don't want to rely on
git trees for much of anything, so this point will not move you, but I
wanted to mention this point about obsoleting source packages for the
benefit of others in the thread.

We can expect git to move off SHA-1 eventually, and it is not at all
clear the threat from preimage attacks is sufficient for it to be wise
for us to hold ourselves back here.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: git & Debian packaging sprint report

[Peter Wienemann]

On 15.07.19 22:50, Russ Allbery wrote:
> At some point, Git itself will switch away from SHA-1, and we
> can then obviously follow.

According to [0]:

-
"Git v2.13.0 and later subsequently moved to a hardened SHA-1
implementation by default, which isn't vulnerable to the SHAttered
attack.

Thus Git has in effect already migrated to a new hash that isn't SHA-1
and doesn't share its vulnerabilities, its new hash function just
happens to produce exactly the same output for all known inputs,
except two PDFs published by the SHAttered researchers, and the new
implementation (written by those researchers) claims to detect future
cryptanalytic collision attacks."
-

The document also outlines plans for a transition to SHA256. It actually
seems that since git version 2.21.0 the first SHA256 implementations
have entered the git code [1, 2]. Though I have no idea whether using
SHA256 is already production-ready.

Therefore I think that distrust in SHA1 is no reason to discard Sean's
and Ian's debpush proposal.

Peter

[0]
https://github.com/git/git/blob/master/Documentation/technical/hash-function-transition.txt

[1]
https://github.com/git/git/commit/33e4ae9c509e0ecdc6508475f2974d275539616e

[2]
https://github.com/git/git/commit/27dc04c54506967fcaa87b2d560547ee5633040c

Re: git & Debian packaging sprint report

[Scott Kitterman]

On July 15, 2019 8:50:48 PM UTC, Russ Allbery  wrote:
>Ansgar Burchardt  writes:
>
>> SHA-1 isn't going to get stronger in the future.  The TLS world has
>> already moved on, OpenPGP is still in the slow process to move on,
>> Release/Packages stopped using it[1], there is no reason to continue
>> using it.
>
>Well, the reason to continue using it is that Git uses it and we use
>Git,
>and it may simplify the workflow.
>
>You're not wrong, of course, but preimage attacks are very hard.  MD5
>is
>still probably robust against preimage attacks, let alone SHA-1.  By
>all
>means, let's future-proof as much as possible, but I'm not sure
>worrying
>about SHA-1 preimage resistance is the most important design principle
>in
>this case.  At some point, Git itself will switch away from SHA-1, and
>we
>can then obviously follow.
...
Except that we have different requirements than git.  Git isn't looking for 
security properties from SHA-1, so it's highly likely it'll meet their accident 
avoidance requirements long after it's no longer appropriate for security 
related assertions.

I don't think adding more SHA-1 in a security sensitive context is a good plan.

Scott K

Re: git & Debian packaging sprint report

[Ben Hutchings]

On Mon, 2019-07-15 at 20:54 +0200, Ansgar Burchardt wrote:
> Russ Allbery writes:
> > If so, I think that security model is roughly equivalent to the automatic
> > signing of binary packages by buildds, so probably doesn't introduce a new
> > vulnerability,
> 
> It doesn't rely on strong cryptographic hashes to guarantee integrity.
> To quote Wikipedia:
> 
> +---
> > Revision control systems such as Git, Mercurial, and Monotone use
> > SHA-1 not for security but to identify revisions and to ensure that
> > the data has not changed due to accidental corruption.
> +---[ https://en.wikipedia.org/wiki/SHA-1#Data_integrity ]
> 
> But developers could instead just sign artifacts using a strong
> cryptographic hash that will be included in the source package; for
> example the .orig.tar and .debian.tar which can be made reproducible
> (git-archive is supposed to be reproducible; compression might not be so
> just sign the uncompressed version).
[...]

There is already a convention for adding tarball signatures using git
notes, though it would need to be adapted for the two tarballs in non-
native packages.

See

and the "git-archive-signer" script in
.

Ben.

-- 
Ben Hutchings
If God had intended Man to program,
we'd have been born with serial I/O ports.



signature.asc
Description: This is a digitally signed message part

Re: git & Debian packaging sprint report

[Ansgar Burchardt]

Russ Allbery writes:
> Ansgar Burchardt  writes:
>> The client tool could possibly also just create the .dsc and .changes,
>> except for hashes of the compressed files, and the web service just
>> recreate the tarball and compress them.
>
> I think experience with pristine-tar indicates that recreating tarballs is
> unfortunately not trivial.

pristine-tar tries to recreate an arbitrary tarball that was created by
a third-party; in this case both sides are controlled by the same party.
Using the uncompressed tarballs also avoids possible problems from the
compression algorithm.

This is a simpler problem than what pristine-tar tries to solve.

For upstream tarballs, one can fetch them from the upstream source if
matching the upstream release is desired.

Ansgar

Re: git & Debian packaging sprint report

[Russ Allbery]

Ansgar Burchardt  writes:

> SHA-1 isn't going to get stronger in the future.  The TLS world has
> already moved on, OpenPGP is still in the slow process to move on,
> Release/Packages stopped using it[1], there is no reason to continue
> using it.

Well, the reason to continue using it is that Git uses it and we use Git,
and it may simplify the workflow.

You're not wrong, of course, but preimage attacks are very hard.  MD5 is
still probably robust against preimage attacks, let alone SHA-1.  By all
means, let's future-proof as much as possible, but I'm not sure worrying
about SHA-1 preimage resistance is the most important design principle in
this case.  At some point, Git itself will switch away from SHA-1, and we
can then obviously follow.

That said, there's enough custom logic going on here that it may be easy
to add something that you describe, in which case, great.

> The client tool could possibly also just create the .dsc and .changes,
> except for hashes of the compressed files, and the web service just
> recreate the tarball and compress them.

I think experience with pristine-tar indicates that recreating tarballs is
unfortunately not trivial.

-- 
Russ Allbery (r...@debian.org)   

Re: git & Debian packaging sprint report

[Ansgar Burchardt]

Russ Allbery writes:
> Ansgar Burchardt writes:
>> Russ Allbery writes:
>>> If so, I think that security model is roughly equivalent to the
>>> automatic signing of binary packages by buildds, so probably doesn't
>>> introduce a new vulnerability,
>
>> It doesn't rely on strong cryptographic hashes to guarantee integrity.
>> To quote Wikipedia:
>
>> +---
>> | Revision control systems such as Git, Mercurial, and Monotone use
>> | SHA-1 not for security but to identify revisions and to ensure that
>> | the data has not changed due to accidental corruption.
>> +---[ https://en.wikipedia.org/wiki/SHA-1#Data_integrity ]
>
>> But developers could instead just sign artifacts using a strong
>> cryptographic hash that will be included in the source package; for
>> example the .orig.tar and .debian.tar which can be made reproducible
>> (git-archive is supposed to be reproducible; compression might not be so
>> just sign the uncompressed version).
>
>> We shouldn't go back to trusting SHA-1.
>
> I'm dubious that we really care that much about a preimage attack on
> SHA-1, but sure, if there's some easy way to use something different, that
> would be more future-proof.

SHA-1 isn't going to get stronger in the future.  The TLS world has
already moved on, OpenPGP is still in the slow process to move on,
Release/Packages stopped using it[1], there is no reason to continue
using it.

There is another advantage to developers signing the artifacts: one
would need much less trust in the service building the source files as
it could only manipulate the .dsc, .changes and compression used.

It also has one downside: `git tag` alone won't be enough to generate
the required information, but then a special-purpose tool was proposed
here already.

The client tool could possibly also just create the .dsc and .changes,
except for hashes of the compressed files, and the web service just
recreate the tarball and compress them.  That would require near zero
trust in the web service, but still allow developers to no longer upload
source packages which might be large.  (A bit similar to not having to
trust buildds by making packages reproducible.)

Ansgar

  [1] Though we still have MD5sum for some suites for jigdo...

Re: git & Debian packaging sprint report

[Russ Allbery]

Ansgar Burchardt  writes:
> Russ Allbery writes:

>> If so, I think that security model is roughly equivalent to the
>> automatic signing of binary packages by buildds, so probably doesn't
>> introduce a new vulnerability,

> It doesn't rely on strong cryptographic hashes to guarantee integrity.
> To quote Wikipedia:

> +---
> | Revision control systems such as Git, Mercurial, and Monotone use
> | SHA-1 not for security but to identify revisions and to ensure that
> | the data has not changed due to accidental corruption.
> +---[ https://en.wikipedia.org/wiki/SHA-1#Data_integrity ]

> But developers could instead just sign artifacts using a strong
> cryptographic hash that will be included in the source package; for
> example the .orig.tar and .debian.tar which can be made reproducible
> (git-archive is supposed to be reproducible; compression might not be so
> just sign the uncompressed version).

> We shouldn't go back to trusting SHA-1.

I'm dubious that we really care that much about a preimage attack on
SHA-1, but sure, if there's some easy way to use something different, that
would be more future-proof.

-- 
Russ Allbery (r...@debian.org)   

Re: git & Debian packaging sprint report

[Ansgar Burchardt]

Russ Allbery writes:
> If so, I think that security model is roughly equivalent to the automatic
> signing of binary packages by buildds, so probably doesn't introduce a new
> vulnerability,

It doesn't rely on strong cryptographic hashes to guarantee integrity.
To quote Wikipedia:

+---
| Revision control systems such as Git, Mercurial, and Monotone use
| SHA-1 not for security but to identify revisions and to ensure that
| the data has not changed due to accidental corruption.
+---[ https://en.wikipedia.org/wiki/SHA-1#Data_integrity ]

But developers could instead just sign artifacts using a strong
cryptographic hash that will be included in the source package; for
example the .orig.tar and .debian.tar which can be made reproducible
(git-archive is supposed to be reproducible; compression might not be so
just sign the uncompressed version).

We shouldn't go back to trusting SHA-1.

> There are also some interesting nuances here around handling DM packages,
> where not everyone with a key in the keyring can upload every package,
> although the obvious way to address that is probably for this service to
> do the same DM checks that ftpmaster would normally do.

We have other permissions checks as well; they shouldn't be
reimplemented in different places.  Instead the archive (dak) should
know who signed the package.

Ansgar

Re: git & Debian packaging sprint report

[Sean Whitton]

Hello,

On Mon 15 Jul 2019 at 10:22AM -07, Russ Allbery wrote:

> Just to make sure I fully understand the model, is the idea that this
> system will verify the signature on the Git tag, construct a source
> package from the signed archive, and then sign the resulting source
> package with some internal key?

Assuming that by 'archive' you mean the committish at which the
DD-signed tag points, then yes, that's the model.

> If so, I think that security model is roughly equivalent to the automatic
> signing of binary packages by buildds, so probably doesn't introduce a new
> vulnerability, but my understanding was that the identity of the signature
> on the source package was used in various other places.  Presumably we
> would need to introduce some new metadata so that the uploader is mapped
> properly to the Git tag signer, rather than to some internal identity of
> the source package construction service.

Right.  This might be needed.

> Also, doesn't the archive publish the signed *.dsc files currently?  I
> believe this would mean that we would lose some published information from
> those files that we currently have (namely which DD and which key signed
> the package, which could be useful data in some incident response
> scenarios).  That said, there's been some discussion for some time about
> having the archive sign all the *.dsc files instead of keeping the
> uploader signature, which may be from an expired or unverifiable key
> (particularly for packages that haven't been uploaded in some time).

As you say, DD signatures on .dscs are only reliably useful in sid,
where they're quite likely to be verifiable.

So perhaps having them signed by this bot would be an improvement for
some usecases.

> There are also some interesting nuances here around handling DM packages,
> where not everyone with a key in the keyring can upload every package,
> although the obvious way to address that is probably for this service to
> do the same DM checks that ftpmaster would normally do.

Right.  dgit-infrastructure already has code to do that.  We just
haven't hooked it up yet.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: git & Debian packaging sprint report

[Russ Allbery]

Sean Whitton  writes:

> The current plan is for this machine to be firewalled such that it talks
> only to salsa.  For exactly the sort of reasons you describe, you won't
> be able to use this with arbitrary git hosts.

> The only untrusted input is the git tags before their signature has been
> verified against the Debian keyring.  Maybe we could isolate fetching
> and checking those tags from the part of the service which fetches the
> whole git tree to produce a source package.

Just to make sure I fully understand the model, is the idea that this
system will verify the signature on the Git tag, construct a source
package from the signed archive, and then sign the resulting source
package with some internal key?

If so, I think that security model is roughly equivalent to the automatic
signing of binary packages by buildds, so probably doesn't introduce a new
vulnerability, but my understanding was that the identity of the signature
on the source package was used in various other places.  Presumably we
would need to introduce some new metadata so that the uploader is mapped
properly to the Git tag signer, rather than to some internal identity of
the source package construction service.

Also, doesn't the archive publish the signed *.dsc files currently?  I
believe this would mean that we would lose some published information from
those files that we currently have (namely which DD and which key signed
the package, which could be useful data in some incident response
scenarios).  That said, there's been some discussion for some time about
having the archive sign all the *.dsc files instead of keeping the
uploader signature, which may be from an expired or unverifiable key
(particularly for packages that haven't been uploaded in some time).

There are also some interesting nuances here around handling DM packages,
where not everyone with a key in the keyring can upload every package,
although the obvious way to address that is probably for this service to
do the same DM checks that ftpmaster would normally do.

-- 
Russ Allbery (r...@debian.org)   

Re: git & Debian packaging sprint report

[Sean Whitton]

Hello Michael,

On Mon 15 Jul 2019 at 01:16PM +02, Michael Kesper wrote:

> Nonetheless it seems to me you are moving from trusting local signing
> to trusting upload by salsa, thereby making salsa more attractive for
> attackers.

I don't follow -- the git tag is PGP-signed, locally, by the uploader.
Just like how they would PGP-sign, locally, the .dsc and .changes.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: git & Debian packaging sprint report

[Michael Kesper]

Hi Sean, hi all,

On 12.07.19 09:00, Sean Whitton wrote:
> On Fri 12 Jul 2019 at 04:30am +00, Scott Kitterman wrote:
> 
>> Has there been any analysis of the security implications of this
>> proposed service?
> 
> Nothing formal, though of course we were thinking about it while we were
> working on it.
> 
>> If I am understanding the description correctly, the transformation
>> from git tag (which is signed and can be verified) to a source package
>> (which can be signed and verified) will happen on an internet facing
>> server (typically this would happen on a local developer machine) and,
>> unless there is additional magic around key management that isn't
>> described in the blog post, the private key for a key the archive
>> trusts would also be there.
>>
>> It seems to me that there is potential for a significant new attack
>> surface that ought to be carefully assessed before this gets anywhere
>> near wired up to feed into the archive from any kind of 'cloud'
>> service.
> 
> The current plan is for this machine to be firewalled such that it talks
> only to salsa.  For exactly the sort of reasons you describe, you won't
> be able to use this with arbitrary git hosts.
> 
> The only untrusted input is the git tags before their signature has been
> verified against the Debian keyring.  Maybe we could isolate fetching
> and checking those tags from the part of the service which fetches the
> whole git tree to produce a source package.

Nonetheless it seems to me you are moving from trusting local signing
to trusting upload by salsa, thereby making salsa more attractive for 
attackers.

Best wishes
Michael
 




signature.asc
Description: OpenPGP digital signature

Re: git & Debian packaging sprint report

[Sam Hartman]

> "Sean" == Sean Whitton  writes:

Sean> If the BoF or e-mail discussion comes up with requirements
Sean> which are impossible for our solution to satisfy, then we
Sean> misjudged what to spend our time on at the sprint.  Ian and I
Sean> have been working on this stuff for long enough that we felt
Sean> able to make a judgement about what would be useful, however.

The BOF will almost certainly come up with requirements that are
impossible for any solution to meet.
THe BOF is not judging requirements, only collecting them.

People proposing solutions can use the BOF input where it is helpful.
We as a project can use that input in guiding discussions too.  However
just because something is proposed as a requirement at the BOF doesn't
mean we will be able to (or will choose to) meet that requirement.

Also, the BOF was proposed by me in the hopes of getting things moving.
We've had a lot of activity since I submitted that original BOF
proposal.  If Sean and Ian think they already have enough information to
put something together, I think that's great.  My BOF proposal was
intended to enable work, not to slow it down.

--Sam

Re: git & Debian packaging sprint report

[Sean Whitton]

Hello,

On Sun 14 Jul 2019 at 09:48AM +02, Ansgar wrote:

> What is the DebConf BOF then for?  It says it is "a requirements
> collecting BOF for that [git push] discussion"?

To my mind, the BoF and the work at the sprint are two efforts to
improve things that are running in parallel.  I think we can expect
the them to enhance each other.

If the BoF or e-mail discussion comes up with requirements which are
impossible for our solution to satisfy, then we misjudged what to spend
our time on at the sprint.  Ian and I have been working on this stuff
for long enough that we felt able to make a judgement about what would
be useful, however.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: git & Debian packaging sprint report

[Ansgar]

Sean Whitton writes:
> On Fri 12 Jul 2019 at 02:06PM +02, Ansgar wrote:
>> Depends on a lot of things.  As far as I understand this work is in a
>> very early stage and a first brainstorming session on what problem this
>> is intended to solve, why one should consider doing it, and possible
>> requirements is planned for DebConf[1].
>>
>> Without having any of these, it is hard to comment on anything.
>
> Figuring out how this is going to be deployed as Debian infrastructure
> is indeed at an early stage.
>
> I don't think it is accurate to think of the whole project as being at
> an early stage.  From my perspective, the hardest problem to solve was
> conversion of git trees to uploads, on the server side, in a way that is
> user friendly.  We take ourselves to have solved that problem, which to
> my mind renders this project beyond the early stages of development.

What is the DebConf BOF then for?  It says it is "a requirements
collecting BOF for that [git push] discussion"?

Ansgar

Re: git & Debian packaging sprint report

[Sean Whitton]

Hello,

On Fri 12 Jul 2019 at 02:06PM +02, Ansgar Burchardt wrote:

> Depends on a lot of things.  As far as I understand this work is in a
> very early stage and a first brainstorming session on what problem this
> is intended to solve, why one should consider doing it, and possible
> requirements is planned for DebConf[1].
>
> Without having any of these, it is hard to comment on anything.

Figuring out how this is going to be deployed as Debian infrastructure
is indeed at an early stage.

I don't think it is accurate to think of the whole project as being at
an early stage.  From my perspective, the hardest problem to solve was
conversion of git trees to uploads, on the server side, in a way that is
user friendly.  We take ourselves to have solved that problem, which to
my mind renders this project beyond the early stages of development.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: git & Debian packaging sprint report

[Ansgar Burchardt]

On Thu, 2019-07-11 at 17:58 -0300, Antonio Terceiro wrote:
> > We designed and implemented a system to make it possible for DDs to
> > upload new versions of packages by simply pushing a specially
> > formatted git tag to salsa.
[...]
> If the uploads will be done by a service, this means that all of them
> will be signed by a single key and not by the DD key. Is ftp-master
> ok with that?

Depends on a lot of things.  As far as I understand this work is in a
very early stage and a first brainstorming session on what problem this
is intended to solve, why one should consider doing it, and possible
requirements is planned for DebConf[1].

Without having any of these, it is hard to comment on anything.

Ansgar

  [1] 
https://debconf19.debconf.org/talks/68-one-git-to-package-them-all-and-on-the-salsa-find-them/

Re: git & Debian packaging sprint report

[Sean Whitton]

Hello Scott,

On Fri 12 Jul 2019 at 04:30am +00, Scott Kitterman wrote:

> Has there been any analysis of the security implications of this
> proposed service?

Nothing formal, though of course we were thinking about it while we were
working on it.

> If I am understanding the description correctly, the transformation
> from git tag (which is signed and can be verified) to a source package
> (which can be signed and verified) will happen on an internet facing
> server (typically this would happen on a local developer machine) and,
> unless there is additional magic around key management that isn't
> described in the blog post, the private key for a key the archive
> trusts would also be there.
>
> It seems to me that there is potential for a significant new attack
> surface that ought to be carefully assessed before this gets anywhere
> near wired up to feed into the archive from any kind of 'cloud'
> service.

The current plan is for this machine to be firewalled such that it talks
only to salsa.  For exactly the sort of reasons you describe, you won't
be able to use this with arbitrary git hosts.

The only untrusted input is the git tags before their signature has been
verified against the Debian keyring.  Maybe we could isolate fetching
and checking those tags from the part of the service which fetches the
whole git tree to produce a source package.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: git & Debian packaging sprint report

[Scott Kitterman]

On July 10, 2019 8:10:40 AM UTC, Sean Whitton  wrote:
>Hello,
>
>Over the weekend, Ian Jackson and I met in Cambridge, U.K. to work on
>the design and implementation of tools and processes relating to git &
>Debian packaging.
>
>Main achievement
>
>
>We designed and implemented a system to make it possible for DDs to
>upload new versions of packages by simply pushing a specially formatted
>git tag to salsa.
>
>Please see this blog post to learn about how it works:
>https://spwhitton.name/blog/entry/tag2upload/
>
>While the cloud service part of this system has not yet been deployed,
>and so you can't just tag to upload yet, the blog post explains how you
>can run the cloud service in an ad-hoc mode on your laptop, and thereby
>get a feel for how it works.
...
Thanks for the detailed explanation.

Has there been any analysis of the security implications of this proposed 
service?

If I am understanding the description correctly, the transformation from git 
tag (which is signed and can be verified) to a source package (which can be 
signed and verified) will happen on an internet facing server (typically this 
would happen on a local developer machine) and, unless there is additional 
magic around key management that isn't described in the blog post, the private 
key for a key the archive trusts would also be there.

It seems to me that there is potential for a significant new attack surface 
that ought to be carefully assessed before this gets anywhere near wired up to 
feed into the archive from any kind of 'cloud' service.

Scott K

Re: git & Debian packaging sprint report

[Sam Hartman]

> "Andrej" == Andrej Shadura  writes:

Andrej> Hi,
Andrej> On Wed, 10 Jul 2019 at 10:10, Sean Whitton 
 wrote:
>> Over the weekend, Ian Jackson and I met in Cambridge, U.K. to
>> work on the design and implementation of tools and processes
>> relating to git & Debian packaging.

Andrej> Sean, are you coming to Curitiba? If yes, how about
Andrej> organising a BoF on Git packaging workflows?

He's not, but if you read the entire report you'd see a link to


https://debconf19.debconf.org/talks/68-one-git-to-package-them-all-and-on-the-salsa-find-them/

We'll be starting off early in the week trying to understand and catalog
requirements, and I'm sure the fun won't end there!

Re: git & Debian packaging sprint report

[Antonio Terceiro]

Hi,

Thanks for the report.

On Wed, Jul 10, 2019 at 09:10:40AM +0100, Sean Whitton wrote:
> Hello,
> 
> Over the weekend, Ian Jackson and I met in Cambridge, U.K. to work on
> the design and implementation of tools and processes relating to git &
> Debian packaging.
> 
> Main achievement
> 
> 
> We designed and implemented a system to make it possible for DDs to
> upload new versions of packages by simply pushing a specially formatted
> git tag to salsa.
> 
> Please see this blog post to learn about how it works:
> https://spwhitton.name/blog/entry/tag2upload/
> 
> While the cloud service part of this system has not yet been deployed,
> and so you can't just tag to upload yet, the blog post explains how you
> can run the cloud service in an ad-hoc mode on your laptop, and thereby
> get a feel for how it works.
> 
> You can also read git-debpush(1) in sid.[1]

If the uploads will be done by a service, this means that all of them
will be signed by a single key and not by the DD key. Is ftp-master ok
with that?


signature.asc
Description: PGP signature

Re: git & Debian packaging sprint report

[gregor herrmann]

On Wed, 10 Jul 2019 09:10:40 +0100, Sean Whitton wrote:

> Main achievement
> 
> 
> We designed and implemented a system to make it possible for DDs to
> upload new versions of packages by simply pushing a specially formatted
> git tag to salsa.
> 
> Please see this blog post to learn about how it works:
> https://spwhitton.name/blog/entry/tag2upload/

This sounds very exciting, I'm really looking forward to typing `git
debpush'! Thank you very much for this work.


Cheers,
gregor

-- 
 .''`.  https://info.comodo.priv.at -- Debian Developer https://www.debian.org
 : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D  85FA BB3A 6801 8649 AA06
 `. `'  Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
   `-   NP: Beatles


signature.asc
Description: Digital Signature

git & Debian packaging sprint report

[Sean Whitton]

Hello,

Over the weekend, Ian Jackson and I met in Cambridge, U.K. to work on
the design and implementation of tools and processes relating to git &
Debian packaging.

Main achievement


We designed and implemented a system to make it possible for DDs to
upload new versions of packages by simply pushing a specially formatted
git tag to salsa.

Please see this blog post to learn about how it works:
https://spwhitton.name/blog/entry/tag2upload/

While the cloud service part of this system has not yet been deployed,
and so you can't just tag to upload yet, the blog post explains how you
can run the cloud service in an ad-hoc mode on your laptop, and thereby
get a feel for how it works.

You can also read git-debpush(1) in sid.[1]

Other achivements
-

1. In-depth design discussions for:

   - git-ffqrebase, a tool for Debian users and downstreams to rebase
 delta queues on top of Debian's source code; and

   - better git-merge support for git-debrebase.  Confirmed the
 resolution UX/workflow, so implementation is now unblocked.  Some
 notes are in #931552.

2. Improved the git packaging survey[2] by setting up inner pages,
   deciding on a template for those, and filling some of them in.

3. Made some plans to improve the layout of dgit's documentation.  Thank
   you to Colin Watson for the feedback which prompted this.

4. Triage of bugs filed against src:dgit.

5. Various smaller fixes/improvements to dgit.

6. Wrote up some requirements as input to an upcoming BoF.[3]

7. Produced blog post linked above.

Reimbursement
-

There will be one travel reimbursement request, for an amount less than
$100.

Acknowledgements


Thank you to donators to Debian for travel funding, and the people who
live at Ian Jackson's place (including Ian) for hosting.

[1]  https://manpages.debian.org/git-debpush
[2]  https://wiki.debian.org/GitPackagingSurvey
[3]  
https://debconf19.debconf.org/talks/68-one-git-to-package-them-all-and-on-the-salsa-find-them/

-- 
Sean Whitton


signature.asc
Description: PGP signature