Thanks. So the current size is about 0.5 TB, and presumably if people are maintaining full mirrors, PyPI itself can cope with that much outgoing bandwidth being used.
Steve & Chris: does downloading & scanning that volume of data sound like something you'd want to do on Azure? Does anyone there have some time to put in to move this forwards? Thomas On Thu, Feb 9, 2017, at 10:18 PM, Jeremy Stanley wrote: > On 2017-02-08 18:14:38 +0000 (+0000), Thomas Kluyver wrote: > [...] > > What I'm proposing differs in that it would need to download files from > > PyPI - basically all of them, if we're thorough about it. I imagine > > that's going to involve a lot of data transfer. Do we know what order of > > magnitude we're talking about? > [...] > > The crowd I run with uses https://pypi.org/project/bandersnatch/ to > maintain a full PyPI mirror for our project's distributed CI system, > and du says the current aggregate size is 488GiB. Also if you want > to initialize a full mirror this way, plan for it to take several > days to populate. > -- > Jeremy Stanley > _______________________________________________ > Distutils-SIG maillist - [email protected] > https://mail.python.org/mailman/listinfo/distutils-sig _______________________________________________ Distutils-SIG maillist - [email protected] https://mail.python.org/mailman/listinfo/distutils-sig
