Re: [Distutils] Working toward Linux wheel support
On 20 July 2015 at 11:42, Leonardo Rochael Almeida leoroch...@gmail.com wrote: To solve this problem, so far we've only been able to come up with two extremes: - Have the libraries contain enough metadata in their source form that we can generate true system packages from them (this doesn't really help the virtualenv case) - Carry all the dependencies. Either by static linking, or by including all dynamic libraries in the wheel, or by becoming something like Conda where we package even non Python projects. We keep stalling on making progress with Linux wheel files as our discussions spiral out into all the reasons why solving the general case of binary distribution is so hard. However, Nate has a specific concrete problem in needing to get artifacts from Galaxy's build servers and installing them into their analysis environments - let's help him solve that, on the assumption that some *other* mechanism will be used to manage the non-Python components. This approach is actually applicable to many server based environments, as a configuration management tool like Puppet, Chef, Salt or Ansible will be used to deal with the non-Python aspects. This approach is even applicable to some centrally managed data analysis workstation cases. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Working toward Linux wheel support
On 20 July 2015 at 18:37, Chris Barker chris.bar...@noaa.gov wrote: sure -- but isn't that use-case already supported by wheel -- define your own wheelhouse that has the ABI you know you need, and point pip to it. I presume the issue is wanting to have a single shared wheelhouse for a (presumably limited) number of platforms. So being able to specify a (completely arbitrary) local platform name at build and install time sounds like a viable option. BUT - we have someone offering a solution that solves at least part of the problem, sufficient for their needs and a step forward from where we are. This is great news, as wheel support for Linux has always been stalled before (for whatever reason). So thank you to Nate for his work, and let's look to how we can accept it and build on it in the future. Unfortunately, I don't have any Linux knowledge in this area, so I can't offer any useful advice on the questions Nate asks. But hopefully some people on this list can. Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Working toward Linux wheel support
On Sun, Jul 19, 2015 at 11:00 PM, Nick Coghlan ncogh...@gmail.com wrote: However, Nate has a specific concrete problem in needing to get artifacts from Galaxy's build servers and installing them into their analysis environments - let's help him solve that, on the assumption that some *other* mechanism will be used to manage the non-Python components What is there to solve here? Galaxy's build servers put all the wheels somewhere. Galaxy's analysis systems point to that place. I thought pip+wheel_wheelhouse already solved that problem? -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Working toward Linux wheel support
On Sun, Jul 19, 2015 at 10:50 PM, Nick Coghlan ncogh...@gmail.com wrote: The intended use case is Build once, deploy many times. This is especially important for use cases like Nate's - Galaxy has complete control over both the build environment and the deployment environment, but they *don't* want to rebuild in every analysis environment. That means all they need is a way to build a binary artifact that adequately declares its build context, and a way to retrieve those artifacts at installation time. I'm interested in the same case - I don't need to build artifacts for arbitrary versions of Linux, I mainly want to build them for the particular ABIs defined by the different Fedora and EPEL versions. sure -- but isn't that use-case already supported by wheel -- define your own wheelhouse that has the ABI you know you need, and point pip to it. Not that it would hurt to add a bit more to the filename, but it seems you either: Have a specific system definition you are building for -- so you want to give it a name. One step better than defining a wheelhouse. or You want to put it up on PyPi and have the folks with compatible systems be abel to get, and know it will work -- THAT is a big 'ol can of worms that maybe you're better off going with conda -CHB Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Working toward Linux wheel support
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/17/2015 11:46 AM, Antoine Pitrou wrote: Due to the fact Linux binary wheels don't exist, conda is even more useful on Linux... FWIW, they exist, they just can't be published to PyPI. Private indexes (where binary compatibility is a known quantity) work fine with them. Because it nails down binary non-Python dependencies, conda (and similar tools) do fit the bill for public distribution of Python projects which have such build-time deps. Even given the over-specified platform tags Nick suggests, linux wheels won't fully work, because the build-time depes won't be satisfiable *by pip*: the burden will be on each project to attempt a build and then spit out an error message trying to indicate the missing system package. Is-that-'-dev'-or-'-devel'-I-need?'ly, Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iQIcBAEBAgAGBQJVrbuhAAoJEPKpaDSJE9HY0R8P/jLBINO04/NTlJTUa8wmIxed aWSU0mxFSAKg0q+n2QaRi418QG6vvtUVGsXGafmYu4hlfKj3Hkj6DA+ws2o7uR5S 1UNU3KSF2lsLoWjaIKpMm4RNWmbHuWQ3HlabXqSly7H7lfgXCAzntdrVy5s3zacM 4wqVTjTWaG2lBf77B6aWhgom6kTvfnpNtyQ4+oKDujSnSWlLJ1W7p0hvuR/33XHr 1NHUdaoUWH7kES0zcRHOyYU7PSPtVYMpzn3SKWljMXSiN1vs9YN6WmypNmLeXjTj gkD/JR8gGv97o9TliKW6KaocbSLvZ5w2bHwkBGYsLRS2pti2ojw+3vmSpm4VwKyn PLhOaMpBR4qC2scFVJ5z1iW9uOYlakra45o60rAaRTiuKEHBPaoimQP3mMW38AsB glY+/j349A2XyE1vosAekxeuinip64erQg6G3+gU0myRsfaaC1lTBlzkDrsya4X5 C2LbE4n2IlMrm+hrA/RbUjKlbTJtIyWLlnrv1jORh6l5VNTXSkafStA7j1nXa/hx 4zAqv9mV/1UErI+IjPz6CQTwNbz5QtSP1gFa/9xqGnnrBSuWRMYd/x0c+JNXFzFC MCMhbQ/ZIAkpmk/VRb1mVQVc2uqsWr9WxZ5F13cJJvZrvWkQJFf70nnHk+n2f3CU 9/s6HEGX8SkP8tZnZ7Co =gpd9 -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
