On build time: I think it's high likely that a build on qemu will surpass the 1-hour limit (IIRC slowdowns of emulation were around 4x).
On defining manylinux3: so, right now, should we start by working on the items in the TODO list Nathaniel posted earlier? Regards, Bruno Rosa -----Original Message----- From: Nathaniel Smith [mailto:n...@pobox.com] Sent: sábado, 8 de abril de 2017 03:45 To: Nick Coghlan <ncogh...@gmail.com> Cc: Bruno Alexandre Rosa <bruno.r...@eldorado.org.br>; distutils sig <distutils-sig@python.org> Subject: Re: [Distutils] CentOS5 is EOL, impact on manylinux1? On Fri, Apr 7, 2017 at 10:41 PM, Nick Coghlan <ncogh...@gmail.com> wrote: > On 7 April 2017 at 06:32, Nathaniel Smith <n...@pobox.com> wrote: >> On Wed, Apr 5, 2017 at 2:37 AM, Nick Coghlan <ncogh...@gmail.com> wrote: >>> And as it turns out, not only are we able to get bare metal machines >>> to run our VMs on (rather than messing about with nested virt >>> support), but the CentOS boxes are pre-cached on the local network, >>> so they should be pretty quick to download (we haven't actually got >>> our CI up and running yet, so I won't be able to vouch for that >>> personally until we do). >> >> I think bare metal access only matters for running x86 with hardware >> accelerated virtualization? If we want to emulate a totally different >> architecture like ppc64le then IIUC qemu does that as a regular >> user-space program. You definitely can run qemu in this mode on >> travis-ci, e.g. check out all the virtual architectures that rust >> builds on: >> >> https://travis-ci.org/rust-lang/rust/ >> >> So I think travis-ci and centos-ci are equivalent on this axis – if >> we want to go the qemu route, then either works, and if we don't, >> then neither works? > > My understanding of the problem is that Travis can't actually run the > 64-bit CentOS VMs fast enough to get build jobs done in a reasonable > time period (due to the resource constraints imposed on the test VMs), > whereas the CentOS systems have more resources available to them > (since they have the whole machine to themselves). > > "Try it and see" would be the best way to confirm whether or not > qemu-ppc64le-on-Travis works, though. Ah, good point – Travis imposes a 1 hour time limit on builds, and for manylinux1 we're at about half that already with all the supported versions of Python... though some of that is compensating for centos 5 brokenness (building our own openssl) and some of it is Python versions we could probably drop at this point (2.6 at least, since there will never be a version of pip that supports both python 2.6 and manylinux2+!). >> For me the big question is whether emulation is actually a good idea. >> When rust announced their plans I remember seeing some skepticism >> about whether one can really trust emulated machines for this kind of >> use case, though I can't find it again now... but it's definitely >> true that all the major distributions go to great lengths to use real >> hardware in their build farms, despite the obvious drawbacks (not >> just in terms of maintenance, but also the ongoing pain of using tiny >> little arm and mips machines that take dozens of hours to build >> things). They know a whole lot more about this than I do so I assume >> they have *some* reason :-). > > It's one thing using emulation for a service being provided for free > in a community project, something else entirely to be doing it for > commercial or commercially-sponsored software. Oh sure. I actually don't know which case this is -- does anyone care about ppc64le who isn't working with some big IBM support contract? We've had at least one @ibm address in one of these threads... > One particular problem with it is that any use of profile-guided > optimisation would be optimising for the performance characteristics > of the emulator, not those of the real hardware (that's also a problem > for cross-compilation: as far as I am aware, you *can't* readily use > profile guided optimisation in the cross-compilation case). I thought PGO uses profiling to observe branches and variable values, which emulation shouldn't effect? Fortunately this part really doesn't matter much to us anyway though -- nothing in the docker image gets shipped to end-users. The important thing would be how the package distributors are running the image to do *their* builds, and that's not our problem. -n -- Nathaniel J. Smith -- https://vorpus.org _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig