I'll get in touch with Martin.
On 26 January 2014 11:40, Nick Coghlan <[email protected]> wrote: > > On 26 Jan 2014 09:51, "Richard Jones" <[email protected]> wrote: > > > > Thanks everyone who helped make this happen. > > Indeed - fine work! :) > > > From my perspective* I believe the ssh upload mechanism was added to > address security issues around the basic-auth-over-http method used > historically. Now uploads *may* be done over https, and those using the ssh > method can move over to using twine or pip upload, I think that it's > reasonable to discontinue support for ssh uploads. > > Yes, I agree that pointing the (very few) pypissh users towards twine as a > replacement is the most reasonable option at this point - we should chat to > MvL about putting a notice to that effect in the pypissh README (IIRC, MvL > is the creator of that upload option). > > Cheers, > Nick. > > > > > > > Richard > > > > * as the guy who will be hassled if its loss is noticed ;) > > > > > > On 26 January 2014 10:38, Donald Stufft <[email protected]> wrote: > >> > >> Today (Sat Jan 25, 2014) the Infrastructure team has migrated PyPI to > new > >> infrastructure. > >> > >> The old infrastructure was: > >> > >> - a single database server managed by OSUOSL > >> - a pair of load balancers shared by all of the python.org services > hosted on > >> OSUOSL > >> - a single backend VM that served as everything else for PyPI > >> > >> The VM that was acting as the backend server from PyPI was partially > hand > >> configured and partially setup to be managed by chef. Additionally it > had an > >> issue that caused it to kernel panic every so often which had been the > cause of > >> a number of downtimes in the last few months. Because it was primarily > >> configured and administered by hand and because the way it was set up > it was > >> not feasible to have any sort of failover or spare. > >> > >> The new infrastructure is: > >> > >> - 2 Web VMs > >> - 2 Database servers in Master/Slave Configuration > >> - 2 PgPool Servers pooling connections to the database servers and load > >> balancing reads across them. > >> - 2 GlusterFS servers backed by Cloud Block Storage acting as the file > storage > >> for package and package docs > >> - 1 metrics server to handle updating the download counts as they come > in from > >> Fastly > >> > >> All of the VMs are hosted on Rackspace’s Public Cloud and have their > >> configuration completely controlled and managed using Salt. Going > forward this > >> will allow us to easily scale out as required or kill malfunctioning > servers > >> and spin up new ones easily. Additionally the setup has been setup so > that > >> where possible there is two servers performing the same role, ideally > in an > >> Active/Active configuration but at least in a Master/Slave > configuration. This > >> should allow PyPI to be far more stable moving forward and make > downtimes much > >> easier to recover from. > >> > >> The services are still fronted by Fasty’s CDN and in the new > infrastructure > >> we’ve removed our load balancer and have replaced it by having Fastly > handle > >> the load balancing for us. Additionally we’ve recently setup a static > mirror of > >> PyPI that is updated once every minute. This is hosted on Rackspace > cloud as > >> well but in a separate data center from the rest of PyPI. Fastly is > configured > >> to fall back to this static mirror in the case that neither of the two > web > >> heads are functioning. This should ensure that even in the event of a > >> catastrophic failure of the PyPI service that the bulk of package > installations > >> should hopefully remain working. > >> > >> The bad news, (and the “Breakage” from the subject) is that while the > new > >> infrastructure was being planned out, built, and migrated to the > “pypissh” > >> package was forgotten. The pypissh package is an alternative way to > upload > >> packages to PyPI however it is very difficult, because of the way it > works, to > >> provide HA support for it as we’ve set up for everything else. We don’t > have > >> any numbers for how many people are actively using this package but > looking > >> at a roughly 2 week chunk of time in PyPI’s download history, the > pypissh > >> package was downloaded 7 times by a browser, and 7 times by pip. All > other > >> downloads were caused by the mirroring system. > >> > >> As of right now pypissh is non functional and due to the difficulty in > HAing > >> and monitoring the current setup and because it is apparently has a very > >> small set of users we would like to effectively kill off this particular > >> service. Additionally the benefits of pypissh have been reduced now > that PyPI > >> is available over a TLS connection with a well trusted certificate. My > question > >> to you is, is this something that distutils-sig is willing to have > happen? If > >> we are to re-enable pypissh we’ll need to write a new solution to doing > it that > >> can be properly HA’d and we’d prefer to put our efforts into improving > things > >> for a much larger set of people. > >> > >> So yea, PyPI should be loads more stable and more reliable now. > >> > >> ----------------- > >> Donald Stufft > >> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 > DCFA > >> > >> > >> _______________________________________________ > >> Distutils-SIG maillist - [email protected] > >> https://mail.python.org/mailman/listinfo/distutils-sig > >> > > > > > > _______________________________________________ > > Distutils-SIG maillist - [email protected] > > https://mail.python.org/mailman/listinfo/distutils-sig > > >
_______________________________________________ Distutils-SIG maillist - [email protected] https://mail.python.org/mailman/listinfo/distutils-sig
