How about building a deprecation period into the tooling? pip 1.5+ could warn users who are using *.pypi.python.org of the error in their ways and encourage them to switch to the new system and gives a date of total removal. After removal the code could also be removed from pip 1.x+.
- Michael On Tue, Aug 6, 2013 at 8:36 PM, Donald Stufft <don...@stufft.io> wrote: > > On Aug 6, 2013, at 9:10 PM, Nick Coghlan <ncogh...@gmail.com> wrote: > > > On 7 August 2013 01:58, Donald Stufft <don...@stufft.io> wrote: > >> On Aug 6, 2013, at 8:22 AM, Nick Coghlan <ncogh...@gmail.com> wrote: > >>> One means by which I could see an f.pypi.python.org DNS record being > >>> left in place indefinitely is if the TUF folks are able to come up > >>> with a scheme for offering end-to-end security for the *existing* PyPI > >>> metadata, *and* the TUF metadata is mirrored by bandersnatch *and* the > >>> TUF client side integrity checks are invoked by pip. In that case, the > >>> security argument regarding the lack of TLS on the subdomains would be > >>> rendered moot, and the backwards compatibility argument for keeping it > >>> active would win. > >> > >> It would be rendered moot as far as any tooling that was updated to work > >> with it (such as pip etc) however the browser level attacks would still > be > >> in play. If those can be solved essentially require giving mirror > operators > >> a SSL certificate for N.pypi.python.org (which still does not preclude > a > >> malicious mirror operator or a compromised mirror from being used to > >> steal logins on PyPI). > > > > We're not talking about "mirror operators" in an abstract sense any > > more: we're talking specifically about the possibility of a formal > > relationship with Gocept to keep the f.pypi.python.org domain alive > > indefinitely. > > > > It isn't Gocept's fault that the upstream mirroring system design was > > broken, so the burden is on us to work with Christian to come up with > > a transition plan that is acceptable to both sides. > > I recognize this. The problem is that Gocept isn't the sole user of > their mirror. If they were (or we knew the set of users who were) then > we could approach each one of them. > > > > > That may mean retiring the subdomain and Gocept changing the > > configuration at affected sites, or it may mean Gocept negotiating a > > more formal relationship with the PSF to continue operating > > f.pypi.python.org in particular. Both options need to be on the table > > from the upstream side, giving Gocept a chance to assess the > > alternatives (i.e. change existing sites to point to a new domain, or > > accept that some existing sites will remain insecure until they have > > been upgraded, just as we have to accept that old clients will remain > > insecure when accessing the main site over HTTP). > > Gocept can't make that decision for installations other than their own. > Christian said that his mirror is seeing 150-300GB of traffic besides the > traffic Gocept generates. Some of that is coming from ``pip --use-mirrors`` > I'm sure, but some of it is also coming from other people who have > hardcoded f.pypi.python.org in their config for whom we don't know who > they are and we don't have a good way of informing them so they can > make an informed consent on using an insecure transport. > > > > >>> Another potential alternative might be for Gocept to approach the PSF > >>> about getting an SSL certificate for that domain, ensuring pip and > >>> setuptools both support HSTS, and then switching that mirror over to > >>> using HSTS (so even configurations hardcoded to use > >>> http://f.pypi.python.org will still get a validated secure > >>> connection). > >> > >> FWIW this doesn't make it secure unless they change their configuration > to > >> point to https://f.pypi.python.org/simple instead of > http://f.pypi.python.org/simple.* > >> > >> Programmatic libraries typically don't support HSTS (as HSTS it > primarily used > >> to prevent attacks that don't typically apply to command line clients). > And certainly > >> none of the existing tools support it. So given that we'd be relying on > the redirect > >> that upgrades the connection to HTTPS an attacker could simply return > HTML > >> instead of the redirect to HTTPS. > > > > Yeah, I realised this flaw after posting. An alternative hack that > > would allow the problem to be solved through an "upgrade > > pip/easy_install" solution rather than an "ensure all configs have > > been edited" approach would be a simple URL translation map that > > converts legacy "http://f.pypi.python.org"; references to a new URL. > > > >> Hence why it makes sense to remove, because either way users will need > to edit > >> their existing configurations if they intend to be secure and moving to > a different > >> domain will prevent the in browser attacks as well. > >> > >> * This is also true of anyone who has hard coded an url to > http://pypi.python.org/simple/ > >> however there's no reasonable way to fix that. > > > > Hardcoded references to http://f.pypi.python.org/simple/ aren't *that* > > different from hardcoded references to the main site. The only > > addition is the inclusion of Gocept in the chain of trust. Given that > > Christian wrote the now recommended mirroring client, trusting > > Christian/Gocept is fairly unavoidable at this point :) > > The main difference is hard coded references to PyPI is unlikely. The > clients defaulted to that so there was no reason to point to it in your > configuration. This is representative of the traffic we see coming into > PyPI and the decline of HTTP connections to /simple/. > > > > > We don't have a hard deadline for fixing this on the upstream side - > > it's in the "important but not currently urgent" category. If we can > > get the active legacy mirrors down to just f.pypi.python.org that will > > be solid progress, and then we can work out a specific arrangement for > > that last mirror which works for Gocept as well. > > > > Cheers, > > Nick. > > > > -- > > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 > DCFA > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG@python.org > http://mail.python.org/mailman/listinfo/distutils-sig > >
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
