On 07/17/2013 04:50 PM, Nick Coghlan wrote:
On 17 Jul 2013 18:17, "holger krekel" <hol...@merlinux.eu
<mailto:hol...@merlinux.eu>> wrote:
>
> On Wed, Jul 17, 2013 at 07:48 +0000, Vinay Sajip wrote:
> > holger krekel <holger <at> merlinux.eu <http://merlinux.eu>> writes:
> >
> > > about existing schemes/efforts. I guess most Linux distros do
it already
> > > so if nothing comes up here PyPI-specific (what is the status of
TUF, btw?)
> > > i am going to look into the distro's working models.
> >
> > ISTM it works for distros because they're the central authority
guaranteeing
> > the provenance of the software in their repos. It's harder with
PyPI because
> > it's not a central authority curating the content. Perhaps
something like a
> > web of trust would be needed.
>
> I am thinking about curating release files _after_ publishing and
> then configuring install activities to require "signed-off" release
files.
> Basically giving companies and devops the possibility to organise their
> vetting processes and collaborate, without requiring PyPI to change
first.
> This certainly involves the question of trust but if nothing else an
entity
> can at least trust its own signatures :)
Note that Linux distros don't trust each other's keys and nor do app
stores trust other. Secure collaborative vetting of software is an
Unsolved Problem. The Update Framework provides a solid technical
basis for such collaboration, but even it doesn't solve the
fundamental trust issues. Those issues are why we still rely on the CA
model for SSL, despite its serious flaws: nobody has come up with
anything else that scales effectively.
The use of JSON for metadata 2.0 is enough to make it TUF friendly,
but there are significant key management issues to be addressed before
TUF could be used on PyPI itself. That's no reason to avoid
experimenting with private TUF enabled PyPI servers, though - a
private server alleviates most of the ugly key management problems.
Thank you, Nick. Indeed, we think that TUF (designed in collaboration
with some of chief designers of the Tor project) offers a secure yet
usable way to address many classes of attacks on package managers, many
previously left unconsidered in the Linux distribution community until
we pointed it out to them, at which point they adopted our security
proposals (https://isis.poly.edu/~jcappos/papers/cappos_mirror_ccs_08.pdf).
We are delighted to see that JSON is being used for PyPI metadata 2.0,
which would certainly lend itself very easily for integration with TUF.
Speaking of which, let me answer some questions about the current status
of PyPI and pip over TUF.
TLDR: We now have a pretty good scheme balancing key management with
security for PyPI. At the time of writing, I have an almost identical
version of pip ready anytime to read metadata off a TUF-secured PyPI
mirror. There is just one thing left to do: I need to just compress the
metadata as much as possible (a problem common to all package managers).
I expect this to be done in the next two weeks, by which time we should
have a slightly modified version of pip which would securely download
packages from an up-to-date TUF-secured PyPI mirror.
(Aside: let me say that we are discussing all things related to PyPI,
pip and TUF on the TUF mailing list
(https://groups.google.com/forum/?fromgroups#!forum/theupdateframework).
I welcome you to join our mailing list so that we can continue the
discussion. I did not want to incessantly copy our discussions to the
DistUtils mailing list because I was not sure whether it would be always
relevant to the DistUtils SIG which is already busy with a number of
other projects. In retrospect, perhaps I should have summarized our
findings every now and then on this list, because I can understand that
it looks to some people as though we have been silent, when in fact that
was not the case.)
To very briefly summarize our status without going into tangential details:
1. We previously found and reported on this mailing list that if we
naively assigned a key to every PyPI project, then the metadata would
not scale. We would have security with little usability. This looks like
an insoluble key management problem, but we think we have a pretty good
solution.
2. The solution is briefly this: we now propose just two targets roles
for all PyPI files.
2.1. The first role --- called the "unstable" targets role --- will have
completely online keys (meaning that it can kept on the server for
automated release purposes). The unstable role will sign for all PyPI
files being added, updated or deleted without question. The metadata for
this role will change all the time.
2.2. The second role --- called the "stable" targets role --- will have
completely offline keys (meaning that keys are kept as securely as
possible and only used with manual human intervention). The stable role
will sign for only the PyPI files which have vetted and deemed
trustworthy. The metadata for this role is expected to change much less
frequently than the unstable role.
Okay, sounds too abstract to some. What does this mean in practice? We
want to make key management simple. Preferably, as Nick Coghlan and
others have proposed before, we would want PyPI to initially, at least,
sign for all packages, because managing keys for every single project
right off the bat is potentially painful. Therefore, with that view in
mind --- which is to first accommodate PyPI signing for packages, and
gradually allowing projects to sign for their own packages --- we then
consider what our proposal above would do.
Firstly, it would make key management so much simpler. There is a
sufficient number of offline keys used to sign metadata for a valuable
and trustworthy set of packages (done only every now and then), and an
online key used to make continuous release of PyPI packages possible
(done all the time).
1. Now suppose that the top-level targets role says: when you download a
package, you must first always ask the stable role about it. If it has
something to say about it, then use that information (and just ignore
the unstable role). Otherwise, ask the unstable role about it.
2. Fine, what about that? Now suppose that the both the stable and
unstable roles have signed for some very popular package called FooBar
2.0. Suppose further that attackers have broken into the TUF-secured
PyPI repository. Oh, they can't find the keys to the stable role, so
they can't mess with the stable role metadata without getting caught,
but since the unstable keys are online, they could make it sign for
malicious versions of the FooBar 2.0 package.
3. But no problem there! Since we have instructed that the stable role
must always be consulted first, then valid metadata about the intended,
trusted FooBar 2.0 package cannot be modified (not without getting all
the human owners of the keys to collude). The unstable role may be
tampered with to offer bogus metadata, but the security impact will be
limited with *prior* metadata about packages in the way-harder-to-attack
stable role.
More details, should you be interested, are available here:
https://groups.google.com/forum/?fromgroups#!topic/theupdateframework/pocW9bacwuc
I hope that answers a number of questions. Let us know if you have more
questions, and I think I can safely conclude that I can start discussing
TUF on this mailing list again!
PS: Pardon any delay in my response in the next couple of days, as I
will be flying for a day or so to New York in approximately 24 hours.
_______________________________________________
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
