Re: [Distutils] PyPI pull request #7
On Wed, Oct 30, 2013 at 11:11 PM, Noah Kantrowitz n...@coderanger.net wrote: Please stop submitting pull requests. Development on the existing codebase is halted except for critical fixes or security issues. You are making extra work for people on this list and it will not be tolerated. Please consider this your final warning. I can't live as long as you are to see the new incantation of Python website (by PyCon 2013) or PyPI. I am willing to help, and this stuff you're saying is rather discouraging and like no, go waste your time somewhere else, we are not giving any code reviews for free. I understand that my reputation precedes me, but can we keep this strictly technical? What I am trying to do is to send small, incremental fixes. They don't affect security. I can commit it directly to avoid distracting overloaded PyPI (bus factor 2) team, and you can blame me for breaking things - ok, and ban if I break something - that's also ok. If learn previous PyPI and new PyPI, I can tell people more about it, and you can expect more pull requests - not from me, for new PyPI, once it is ready. And if I am going to submit any new features, like reST validation on edit and Markdown support - the code will be more decoupled than existing one to be almost directly reused for the new site. Why I am skeptical that new site will replace old one soon? Just because I don't believe in rewrites by one man army. When you develop public resource, you need to rely on external feedback. You also need some designer guy in a team. You also need a backlog for collaboration. My ETA for new PyPI is no earlier than PyCon 2014 if Donald and Richard will be working on it full time. So, instead of all-or-nothing scenario I can try to find some help with incremental approach. -- anatoly t. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PyPI pull request #7
On Oct 31, 2013, at 7:32 AM, anatoly techtonik techto...@gmail.com wrote: Why I am skeptical that new site will replace old one soon? Just because I don't believe in rewrites by one man army. When you develop public resource, you need to rely on external feedback. Warehouse is live at https://preview-pypi.python.org/ (No landing page yet) You also need some designer guy in a team. Have one who’ll have time in a few weeks to go over everything and make it really great :) You also need a backlog for collaboration. My ETA for new PyPI is no earlier than PyCon 2014 if Donald and Richard will be working on it full time. Possibly! I’m unsure of how long it will take, it’s primarily Richard and I but we’ve a few domain experts in particular pieces who have offered to help out as well when we’re ready for their pieces. So, instead of all-or-nothing scenario I can try to find some help with incremental approach. Mostly the problem with improving the current base is every change is particularly dangerous. The code base is large, it’s untested, and it’s not very nice. It’s extremely easy to break things by accident with seemingly unrelated change. Richard and I have both done this multiple times. So every PR we accept has a danger of breaking things. This incurs a high cognitive overload for accepting a PR because it means we have to spin up a copy of the site and manually go through and test any feature we think *might* be affected which typically catches most things but not always. It’s a time and labor intensive process which none of us enjoy and which we’ve decided not to prioritize unless the pay offs are large. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
On 21 Oct, 2013, at 20:52, Donald Stufft don...@stufft.io wrote: On Oct 21, 2013, at 1:02 PM, Chris Barker chris.bar...@noaa.gov wrote: On Fri, Oct 18, 2013 at 6:22 PM, Nick Coghlan ncogh...@gmail.com wrote: -- it would be very useful if folks could easily get binary wheels for OS-X We do plan to support it, but the pip devs uncovered a hole in the current wheel spec that means it generates the same filename on *nix systems for wheels that need to have different names for the download side of things to work properly THanks -- but really? don't OS-X wheels get: macosx_10_6_intel or some such tacked on? Where does that go wrong? Homebrew, Mac Ports, Fink. That would work OK if nobody ever installed things that the system didn't provide. I don't understand this, what would break? Binary wheels that reference shared libraries that aren't included in the wheel (or a default system install) won't work, but that's also true on Windows. What makes OSX more fun[1] than Linux is including shared libraries in the binary archive, unless you are careful with linking you'll end up with binaries that can only be installed in 1 filesystem location (that is, don't work in virtualenvs). Ronald [1] for a fairly twisted definition of fun ;-) ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
On 31 October 2013 23:38, Ronald Oussoren ronaldousso...@mac.com wrote: On 21 Oct, 2013, at 20:52, Donald Stufft don...@stufft.io wrote: On Oct 21, 2013, at 1:02 PM, Chris Barker chris.bar...@noaa.gov wrote: On Fri, Oct 18, 2013 at 6:22 PM, Nick Coghlan ncogh...@gmail.com wrote: -- it would be very useful if folks could easily get binary wheels for OS-X We do plan to support it, but the pip devs uncovered a hole in the current wheel spec that means it generates the same filename on *nix systems for wheels that need to have different names for the download side of things to work properly THanks -- but really? don't OS-X wheels get: macosx_10_6_intel or some such tacked on? Where does that go wrong? Homebrew, Mac Ports, Fink. That would work OK if nobody ever installed things that the system didn't provide. I don't understand this, what would break? Binary wheels that reference shared libraries that aren't included in the wheel (or a default system install) won't work, but that's also true on Windows. The main difference I see is that on Windows, the ABI for the *CPython* shared library is significantly less likely to vary across machines. By contrast, it's relatively easy to change the ABI on *nix systems, as it depends on what you pass as configure options. Will a C extension built with Homebrew Python work with the Python Mac OS X installer from python.org? Probably, but given how painful ABI mismatches with shared libraries can be to debug, probably doesn't cut it until someone has the chance to thoroughly review the potential for problems. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
On 19 Oct, 2013, at 3:22, Nick Coghlan ncogh...@gmail.com wrote: On 19 Oct 2013 04:59, Chris Barker chris.bar...@noaa.gov wrote: Someone on another list indicated that pip installing binary wheels from PyPi will ONLY work for Windows. Is that the case? I think it's desperately needed for OS-X as well. Linux is so diverse that I can't imagine it being useful, but OS-X has only so many versions, and the python.org OS-X binaries are very clear in their requirements -- it would be very useful if folks could easily get binary wheels for OS-X We do plan to support it, but the pip devs uncovered a hole in the current wheel spec that means it generates the same filename on *nix systems for wheels that need to have different names for the download side of things to work properly - hence the current Windows-only restriction. Is that hole documented somewhere? Ronald ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
On 31 Oct, 2013, at 15:26, Nick Coghlan ncogh...@gmail.com wrote: On 31 October 2013 23:38, Ronald Oussoren ronaldousso...@mac.com wrote: On 21 Oct, 2013, at 20:52, Donald Stufft don...@stufft.io wrote: On Oct 21, 2013, at 1:02 PM, Chris Barker chris.bar...@noaa.gov wrote: On Fri, Oct 18, 2013 at 6:22 PM, Nick Coghlan ncogh...@gmail.com wrote: -- it would be very useful if folks could easily get binary wheels for OS-X We do plan to support it, but the pip devs uncovered a hole in the current wheel spec that means it generates the same filename on *nix systems for wheels that need to have different names for the download side of things to work properly THanks -- but really? don't OS-X wheels get: macosx_10_6_intel or some such tacked on? Where does that go wrong? Homebrew, Mac Ports, Fink. That would work OK if nobody ever installed things that the system didn't provide. I don't understand this, what would break? Binary wheels that reference shared libraries that aren't included in the wheel (or a default system install) won't work, but that's also true on Windows. The main difference I see is that on Windows, the ABI for the *CPython* shared library is significantly less likely to vary across machines. By contrast, it's relatively easy to change the ABI on *nix systems, as it depends on what you pass as configure options. Will a C extension built with Homebrew Python work with the Python Mac OS X installer from python.org? As long as they use the same ABI Tag from PEP 425 extensions build with one should work with the other. Probably, but given how painful ABI mismatches with shared libraries can be to debug, probably doesn't cut it until someone has the chance to thoroughly review the potential for problems. I'd have to make time for a thorough review but be sure, but AFAIK this is only a little more complicated on OSX than on (say) Linux and the additonal complication is already captured by the platform tag mentioned by Chris. That is, the CPython ABI is affected by: * The usual configure flags (--with-pydebug, --enable-unicode, ...) These issues also exists on Windows, and the ABI Tag from PEP 425 means that wheels for different sets of configure flags can be recognized from their filename. * The OSX deployment target (the minimal OSX release you want to support) This is already recorded by the distutils platform tag * The processor architectures supported This is also recorded in the distutils platform tag. * Possibly the compiler This is pretty unlikely, as I (and others) already load extensions build with clang in CPython binaries built with GCC (and v.v.), but the low-level compiler support library might affect the ABI when using a compiler other than GCC or clang. And I'd expect that two-level namespaces on OSX make that question moot except for code that's explicitly built to disable that feature. And this does definitely affect CPython on Windows unless you use the limited ABI. The only reason this doesn't cause problems with binary installers on PyPI right now is that most users use the binary installers from www.python.org and hence are all using the same compiler version. The deployment target can affect the unix level API a little, IIRC around 10.4 or 10.5 the APIs on the POSIX level were adjusted a little to more fully comply with the UNIX specification and because of this the affected named resolve to different symbols based on the deployment target (more or less simular to symbol versioning in glibc). AFAIK that shouldn't affect the CPython ABI, but that would need to be carefully checked to be sure and I don't have time to do so right now. To provide more detail on the disutils platform tag, on my machine: import distutils.util distutils.util.get_platform() 'macosx-10.8-intel' This means the deployment target is OSX 10.8, with suppport for the 'intel' set of archictures (i386 and x86_64). This records the two variables that are most likely to cause problems. I'd expected more problems with binary eggs that are dynamicly linked to libraries that are provided in the wheel because OSX's linker add an absolute path to the library to be loaded unless you're careful. Ronald ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] AttributeError: 'tuple' object has no attribute 'split'
On 25 October, 2013 at 2:06:34 PM, Dominique Orban (dominique.or...@gmail.com) wrote: On 25 October, 2013 at 1:56:26 PM, Oscar Benjamin (oscar.j.benja...@gmail.com) wrote: On Oct 25, 2013 3:52 PM, Dominique Orban wrote: On 25 October, 2013 at 9:31:16 AM, Oscar Benjamin ( oscar.j.benja...@gmail.com) wrote: On 24 October 2013 21:04, Dominique Orban wrote: I hope this is the right place to ask for help. I'm not finding much comfort in the PyPi documentation or in Google searches. I uploaded my package `pykrylov` with `python setup.py sdist upload`. Installing it locally with `python setup.py` install works fine but `pip install pykrylov` breaks with the messages below. I since removed it from PyPI but I get the same error message if I try installing from the git repository. I'm hoping someone can put me on track as I've no idea what's wrong. You can see my setup.py here: https://github.com/dpo/pykrylov/blob/ea553cdb287f6e685406ceadcb297fd6704af52d/setup.py I'm using Python 2.7.5 on OSX installed with Homebrew and pip 1.4.1. Attempts to upgrade setuptools or pip result in another error message (AttributeError: 'str' object has no attribute 'rollback')... Can you install a more recent setuptools by downloading it and running the setup.py yourself? https://pypi.python.org/packages/source/s/setuptools/setuptools-1.1.6.tar.gz Thanks for the suggestion. I'm still getting the same error with setuptools 1.1.6. I also tried upgrading Numpy (since I'm using Numpy distutils) by installing from their git repository, and I'm still getting the same error. Is anything obviously wrong with the setup.py? I don't know but I'm not totally clear what you mean. Previously you described multiple problems: with pip, setuptools and pykrylov. Have you successfully installed setuptools now? If the same error is with pykrylov's setup.py have you tried debugging it? E.g. 'python -m pdb setup.py install' python setup.py install works fine. It's the installation with pip that returns the error message I mentioned. I was wondering if something in setup.py didn't agree with pip/setuptools. Yes I installed setuptools 1.1.6. But pip install -e git://github.com/dpo/pykrylov.git@ea553cd#egg=pykrylov still returns AttributeError: 'tuple' object has no attribute 'split'. I hope I'm making sense. Anybody can provide any help with this? Many thanks! -- Dominique ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
On Thu, Oct 31, 2013 at 10:26 AM, Nick Coghlan ncogh...@gmail.com wrote: On 31 October 2013 23:38, Ronald Oussoren ronaldousso...@mac.com wrote: On 21 Oct, 2013, at 20:52, Donald Stufft don...@stufft.io wrote: On Oct 21, 2013, at 1:02 PM, Chris Barker chris.bar...@noaa.gov wrote: On Fri, Oct 18, 2013 at 6:22 PM, Nick Coghlan ncogh...@gmail.com wrote: -- it would be very useful if folks could easily get binary wheels for OS-X We do plan to support it, but the pip devs uncovered a hole in the current wheel spec that means it generates the same filename on *nix systems for wheels that need to have different names for the download side of things to work properly THanks -- but really? don't OS-X wheels get: macosx_10_6_intel or some such tacked on? Where does that go wrong? Homebrew, Mac Ports, Fink. That would work OK if nobody ever installed things that the system didn't provide. I don't understand this, what would break? Binary wheels that reference shared libraries that aren't included in the wheel (or a default system install) won't work, but that's also true on Windows. The main difference I see is that on Windows, the ABI for the *CPython* shared library is significantly less likely to vary across machines. By contrast, it's relatively easy to change the ABI on *nix systems, as it depends on what you pass as configure options. I'm sure you could build your own broken Windows Python, but who bothers? IMO it pretty much boils down to the fact that on Windows you are probably using the python.org version of Python and not linking with random shared libraries from C:\Program Files\, but on Linux you are probably using one of a few dozen distro x distro-release Pythons AND your extension probably dynamically links to some other things your specific distro provides AND maybe you are depending on some versioned symbols in glibc oh the horror. On OS X I would hope you are uploading only wheels built with python.org-Python but maybe you aren't, everyone has their preferences. Will a C extension built with Homebrew Python work with the Python Mac OS X installer from python.org? Probably, but given how painful ABI mismatches with shared libraries can be to debug, probably doesn't cut it until someone has the chance to thoroughly review the potential for problems. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] AttributeError: 'tuple' object has no attribute 'split'
Why don't you study what's happening in the offending lines of code and see where things are going wrong? For example, in this file: /Users/dpo/.virtualenvs/test/lib/python2.7/site-packages/setuptools/command/egg_info.py see what's showing up as a tuple rather than a string. You can add debug statements if necessary. --Chris On Thu, Oct 31, 2013 at 9:08 AM, Dominique Orban dominique.or...@gmail.com wrote: On 25 October, 2013 at 2:06:34 PM, Dominique Orban (dominique.or...@gmail.com) wrote: On 25 October, 2013 at 1:56:26 PM, Oscar Benjamin (oscar.j.benja...@gmail.com) wrote: On Oct 25, 2013 3:52 PM, Dominique Orban wrote: On 25 October, 2013 at 9:31:16 AM, Oscar Benjamin ( oscar.j.benja...@gmail.com) wrote: On 24 October 2013 21:04, Dominique Orban wrote: I hope this is the right place to ask for help. I'm not finding much comfort in the PyPi documentation or in Google searches. I uploaded my package `pykrylov` with `python setup.py sdist upload`. Installing it locally with `python setup.py` install works fine but `pip install pykrylov` breaks with the messages below. I since removed it from PyPI but I get the same error message if I try installing from the git repository. I'm hoping someone can put me on track as I've no idea what's wrong. You can see my setup.py here: https://github.com/dpo/pykrylov/blob/ea553cdb287f6e685406ceadcb297fd6704af52d/setup.py I'm using Python 2.7.5 on OSX installed with Homebrew and pip 1.4.1. Attempts to upgrade setuptools or pip result in another error message (AttributeError: 'str' object has no attribute 'rollback')... Can you install a more recent setuptools by downloading it and running the setup.py yourself? https://pypi.python.org/packages/source/s/setuptools/setuptools-1.1.6.tar.gz Thanks for the suggestion. I'm still getting the same error with setuptools 1.1.6. I also tried upgrading Numpy (since I'm using Numpy distutils) by installing from their git repository, and I'm still getting the same error. Is anything obviously wrong with the setup.py? I don't know but I'm not totally clear what you mean. Previously you described multiple problems: with pip, setuptools and pykrylov. Have you successfully installed setuptools now? If the same error is with pykrylov's setup.py have you tried debugging it? E.g. 'python -m pdb setup.py install' python setup.py install works fine. It's the installation with pip that returns the error message I mentioned. I was wondering if something in setup.py didn't agree with pip/setuptools. Yes I installed setuptools 1.1.6. But pip install -e git://github.com/dpo/pykrylov.git@ea553cd#egg=pykrylov still returns AttributeError: 'tuple' object has no attribute 'split'. I hope I'm making sense. Anybody can provide any help with this? Many thanks! -- Dominique ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
On 31 Oct, 2013, at 17:49, Daniel Holth dho...@gmail.com wrote: On Thu, Oct 31, 2013 at 10:26 AM, Nick Coghlan ncogh...@gmail.com wrote: On 31 October 2013 23:38, Ronald Oussoren ronaldousso...@mac.com wrote: On 21 Oct, 2013, at 20:52, Donald Stufft don...@stufft.io wrote: On Oct 21, 2013, at 1:02 PM, Chris Barker chris.bar...@noaa.gov wrote: On Fri, Oct 18, 2013 at 6:22 PM, Nick Coghlan ncogh...@gmail.com wrote: -- it would be very useful if folks could easily get binary wheels for OS-X We do plan to support it, but the pip devs uncovered a hole in the current wheel spec that means it generates the same filename on *nix systems for wheels that need to have different names for the download side of things to work properly THanks -- but really? don't OS-X wheels get: macosx_10_6_intel or some such tacked on? Where does that go wrong? Homebrew, Mac Ports, Fink. That would work OK if nobody ever installed things that the system didn't provide. I don't understand this, what would break? Binary wheels that reference shared libraries that aren't included in the wheel (or a default system install) won't work, but that's also true on Windows. The main difference I see is that on Windows, the ABI for the *CPython* shared library is significantly less likely to vary across machines. By contrast, it's relatively easy to change the ABI on *nix systems, as it depends on what you pass as configure options. On OS X I would hope you are uploading only wheels built with python.org-Python but maybe you aren't, everyone has their preferences. And that shouldn't be a problem, the various builds of Python on OSX should be compatible if the have the same platform tag in the wheel filename. Well, barring idiocy like installing a completely different libc, that might cause problems because (at least for 2.7) some libc types leak into the CPython ABI. But anyway, I'd love to use binary wheels in the future to distribute prebuilt binaries for some of my projects and that's why I reacted on a message that mentioned that binary installs aren't supported yet. If I don't understand what the issue is I can't try to help finding I solution :-) Is it just a matter of researching if the various build options on OSX really lead to binaries with the same ABI, or is more work needed? Ronald Will a C extension built with Homebrew Python work with the Python Mac OS X installer from python.org? Probably, but given how painful ABI mismatches with shared libraries can be to debug, probably doesn't cut it until someone has the chance to thoroughly review the potential for problems. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
On Oct 31, 2013, at 1:15 PM, Ronald Oussoren ronaldousso...@mac.com wrote: Is it just a matter of researching if the various build options on OSX really lead to binaries with the same ABI, or is more work needed? Basically it’s this: I was told a wheel built on Ubuntu probably won’t work on Linux, so I shut off Linux Wheels, at the same time I asked about Windows and OSX wheels, the answers I got from people were they would generally just work on Windows, and nobody gave me a straight answer on OSX. If you build a Wheel with Homebrew Python will it work on the official OSX installers? What if I have a library installed from Homebrew? Essentially trying to figure out how likely it is that with the existing tags a wheel is going to work if we select based on them. Once we know what, if any, problems exist for the various platforms then we can come up with fixes for them. This just hasn’t been high priority for me because of PEP453 work. To be honest the same problems likely exists on Windows, it’s just less likely and the benefits of prebuilt binaries greater. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Egg name computation
On Mon, Oct 28, 2013 at 3:50 PM, PJ Eby p...@telecommunity.com wrote: You could include a dummy extension that does nothing, I suppose. Or which controls the building of your actual extensions. Setuptools has long supported Pyrex and I think that Cython might also work, i.e., that you could just specify your cython modules as extensions in setup.py to start with. Indeed -- recent versions of setuptools do support Cython. You also may want to use Cython's cythonize on your Extension objects to make it a bit smarter. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] distlib 0.1.3 released on PyPI
I've just released version 0.1.4 of distlib on PyPI [1]. For newcomers, distlib is a library of packaging functionality which is intended to be usable as the basis for third-party packaging tools. The changes in this release are as follows: scripts.py: Updated the logic for finding the distlib package using a relative, rather than absolute method. This fixes a problem for pip, where distlib is kept in the pip.vendor.distlib package. _backport/sysconfig.py: The analogous change to that made for scripts.py, described above. Please try it out, and if you find any problems or have any suggestions for improvements, please give some feedback using the issue tracker! [2] Regards, Vinay Sajip [1] https://pypi.python.org/pypi/distlib/0.1.4 [2] https://bitbucket.org/pypa/distlib/issues/new ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] distlib 0.1.4 released on PyPI
Vinay Sajip vinay_sajip at yahoo.co.uk writes: I've just released version 0.1.4 of distlib on PyPI [1]. For newcomers, distlib is a library of packaging functionality which is intended to be usable as the basis for third-party packaging tools. Aargh, sorry about the typo in the subject - that should have been 0.1.4. Regards, Vinay Sajip ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/31/2013 02:24 PM, Donald Stufft wrote: To be honest the same problems likely exists on Windows, it?s just less likely and the benefits of prebuilt binaries greater. For all platforms *except* Windows, wheels are essentially caches -- there is no real reason to distribute them via PyPI at all, because OSx and Linux develpoers will have tools to build them from sdists. Pip *does* need to allow installing them on those platforms, but probably only via explicit paths / URLS (rather than finding them on PyPI). Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlJywmwACgkQ+gerLs4ltQ5P/ACfXcMJj4dmnlNJEccZ8gxi8FLR GrQAoLxxgeVnKRvuoscR6GuabwGxfsnF =gZW1 -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
On Oct 31, 2013, at 4:49 PM, Tres Seaver tsea...@palladion.com wrote: Pip *does* need to allow installing them on those platforms, but probably only via explicit paths / URLS (rather than finding them on PyPI). You can install them just fine on any platform, the only restrictions are PyPI won’t let you upload them, and pip won’t try to install them from pypi.python.org. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
On 1 Nov 2013 06:50, Tres Seaver tsea...@palladion.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/31/2013 02:24 PM, Donald Stufft wrote: To be honest the same problems likely exists on Windows, it?s just less likely and the benefits of prebuilt binaries greater. For all platforms *except* Windows, wheels are essentially caches -- there is no real reason to distribute them via PyPI at all, because OSx and Linux develpoers will have tools to build them from sdists. They're caches that can speed things up a *lot*, though, and for new users can build C extensions is a higher bar than can run Python and pip. Given PEP 453, it's probably worth allowing wheels on Mac OS X in pip 1.5, then we can tackle the thornier general *nix problem in the pip 1.6 time frame (which should also improve external dependency handling on Windows and Mac OS X as well). Cheers, Nick. Pip *does* need to allow installing them on those platforms, but probably only via explicit paths / URLS (rather than finding them on PyPI). Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlJywmwACgkQ+gerLs4ltQ5P/ACfXcMJj4dmnlNJEccZ8gxi8FLR GrQAoLxxgeVnKRvuoscR6GuabwGxfsnF =gZW1 -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/31/2013 5:34 PM, Nick Coghlan wrote: On 1 Nov 2013 06:50, Tres Seaver tsea...@palladion.com mailto:tsea...@palladion.com wrote: On 10/31/2013 02:24 PM, Donald Stufft wrote: To be honest the same problems likely exists on Windows, it?s just less likely and the benefits of prebuilt binaries greater. For all platforms *except* Windows, wheels are essentially caches -- there is no real reason to distribute them via PyPI at all, because OSx and Linux develpoers will have tools to build them from sdists. They're caches that can speed things up a *lot*, though, and for new users can build C extensions is a higher bar than can run Python and pip. I don't think it's true that they're caches on all Linux boxes. I have many production machines that do not have compilers on them, and do not have the development headers, etc. installed. They're no better off than a Windows machine! Sure, I could install all of this, but: - - I'd rather not increase the attack surface by installing lots of software - - Some of these are small VM instances, and I don't have room to install a full build environment Where I can, right now I typically install the tools I need to install, do the installation from source, and then remove the tools. It's a pain. I'd much rather have binary Linux wheels. I'm more than happy to build them myself on a dedicated build machine. Eric. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.13 (Cygwin) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJSctEsAAoJENxauZFcKtNxSYYIAJW3BArMWX+Z669AYe/lpoj9 WMggmdfr6u6BXrOVdujHJpXG43U9UhcqJTYLOitNHyO9XKVq/ODDfbV7SN9LTlHI atkEf87xxjuaBI5nBiS0q/ozKMmU4AR82aHYQ2wcxlLry/CSt9zgBBM/vt4I0so2 UXeUa5CgkXGGsG5nYvk9zOAQmMknUSESZrE/WNsgjImk9oBBySs9Tvz+hUF407Y2 sRFuWVZCoDjFwWg8/e98YI6sxuqx7fFhdzn5uoXq9wQ27xg9FjKQlbAWPYcBVK1M ZAQj0SgSMY4paKQ9QsOdQchieJjRZ21C7ue8WGaca/IeyY280RTA0ztuRVMFz58= =nmb3 -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
On Oct 31, 2013, at 5:52 PM, Eric V. Smith e...@trueblade.com wrote: I'm more than happy to build them myself on a dedicated build machine. This works! You just can’t upload/install from PyPI. The restriction is *only* centered around PyPI. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/31/2013 6:05 PM, Donald Stufft wrote: On Oct 31, 2013, at 5:52 PM, Eric V. Smith e...@trueblade.com wrote: I'm more than happy to build them myself on a dedicated build machine. This works! You just can’t upload/install from PyPI. The restriction is *only* centered around PyPI. Ah, I misunderstood. Cool, and thanks for all of the work! Eric. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.13 (Cygwin) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJSctWHAAoJENxauZFcKtNxzXEH/0XhiAoMeS8WODLrKSREkDFi trvaIFU/AaC0fIidkjqSmSk/axaD8mp8yPCjo/PVeq+ZM7Mi1pmZmGmU7k6KkpeS xdEYCpMBg72yYFzo/fyvIQGgVm3Y/mMaIVc3fBbzfEk08vMbNUVFwgg0Jtz8OOfh F2oqTLxqLYhaLQ36hpWuQpJYHDNEEa0fDMiomzKhvzoFEu4e5XhgMpi2DrkYQWbM q7mmLOhJ76SVzODx+fx+jvAu8xQ++MGNxcQ90O4m1pP+RPHnw5DAXVE6jJ1Ko8jB S5zKKvInFeLB5eQwxgQ1S3JMLLQOD02y2pI+w3T7fbhUBtiKVctL02dXxOUw6/Q= =1dem -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/31/2013 05:52 PM, Eric V. Smith wrote: I'm more than happy to build them myself on a dedicated build machine. Right -- that makes them back into caches. ;) You can safely deploy them because you know the architecture / libc / etc. against which you built them. They are highly unlikely to be useful to anyone *else*, however, except by accident. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlJy1vwACgkQ+gerLs4ltQ7HqQCgs3toEwtRSn5q+USRXdOWDB+C dusAn0fSfRyUmRSvqGQ3IUh1Kl6Jh36X =OCEp -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/31/2013 04:55 PM, Donald Stufft wrote: On Oct 31, 2013, at 4:49 PM, Tres Seaver tsea...@palladion.com wrote: Pip *does* need to allow installing them on those platforms, but probably only via explicit paths / URLS (rather than finding them on PyPI). You can install them just fine on any platform, the only restrictions are PyPI won?t let you upload them, and pip won?t try to install them from pypi.python.org. Excellent -- sorry I misunderstood. AFAICT, you could leave that restriction in place for ever for Linux. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlJy10kACgkQ+gerLs4ltQ5C0QCePWr4hX6zTff9hyCO9+w5s0Ac R8sAn11NPAr3d+SCpricLcdLOhtk/nmp =V/rv -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] AttributeError: 'tuple' object has no attribute 'split'
On Oct 31, 2013 4:09 PM, Dominique Orban dominique.or...@gmail.com wrote: On 25 October, 2013 at 2:06:34 PM, Dominique Orban ( dominique.or...@gmail.com) wrote: On 25 October, 2013 at 1:56:26 PM, Oscar Benjamin ( oscar.j.benja...@gmail.com) wrote: On Oct 25, 2013 3:52 PM, Dominique Orban wrote: On 25 October, 2013 at 9:31:16 AM, Oscar Benjamin ( oscar.j.benja...@gmail.com) wrote: On 24 October 2013 21:04, Dominique Orban wrote: I hope this is the right place to ask for help. I'm not finding much comfort in the PyPi documentation or in Google searches. I uploaded my package `pykrylov` with `python setup.py sdist upload`. Installing it locally with `python setup.py` install works fine but `pip install pykrylov` breaks with the messages below. I since removed it from PyPI but I get the same error message if I try installing from the git repository. I'm hoping someone can put me on track as I've no idea what's wrong. You can see my setup.py here: https://github.com/dpo/pykrylov/blob/ea553cdb287f6e685406ceadcb297fd6704af52d/setup.py I'm using Python 2.7.5 on OSX installed with Homebrew and pip 1.4.1. Attempts to upgrade setuptools or pip result in another error message (AttributeError: 'str' object has no attribute 'rollback')... Can you install a more recent setuptools by downloading it and running the setup.py yourself? https://pypi.python.org/packages/source/s/setuptools/setuptools-1.1.6.tar.gz Thanks for the suggestion. I'm still getting the same error with setuptools 1.1.6. I also tried upgrading Numpy (since I'm using Numpy distutils) by installing from their git repository, and I'm still getting the same error. Is anything obviously wrong with the setup.py? I don't know but I'm not totally clear what you mean. Previously you described multiple problems: with pip, setuptools and pykrylov. Have you successfully installed setuptools now? If the same error is with pykrylov's setup.py have you tried debugging it? E.g. 'python -m pdb setup.py install' python setup.py install works fine. It's the installation with pip that returns the error message I mentioned. I was wondering if something in setup.py didn't agree with pip/setuptools. Yes I installed setuptools 1.1.6. But pip install -e git:// github.com/dpo/pykrylov.git@ea553cd#egg=pykrylov still returns AttributeError: 'tuple' object has no attribute 'split'. I hope I'm making sense. Anybody can provide any help with this? Run pip under pdb. Find the location of the pip script and run: python -m pdb /path/to/pip install ... Then find out which object has the wrong type and see if you can trace it back to something in the setup.py. Oscar ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
On Oct 31, 2013 8:50 PM, Tres Seaver tsea...@palladion.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/31/2013 02:24 PM, Donald Stufft wrote: To be honest the same problems likely exists on Windows, it?s just less likely and the benefits of prebuilt binaries greater. For all platforms *except* Windows, wheels are essentially caches -- there is no real reason to distribute them via PyPI at all, because OSx and Linux develpoers will have tools to build them from sdists. What if an OSX user wants to install numpy/scipy? How easy is it to do this from source (I really don't know)? Building the necessary BLAS/LAPACK libraries isn't easy on any platform. It's just easier on a linux distro when the package manager can do it for you. Oscar ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
On Sat, Oct 19, 2013 at 3:22 AM, Nick Coghlan ncogh...@gmail.com wrote: Once ensurepip has landed in Python 3.4 and pip 1.5 is released, we should be able to get back to updating the various metadata specs, with the aim of getting cross-platform wheel support in pip 1.6 :) If I understand right, we must waiting Python 3.5 (~2016) to have out-of-the-box wheel packages on Linux. Really? -- Sebastien Douche sdou...@gmail.com Twitter: @sdouche / G+: +sdouche ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
On Oct 31, 2013, at 7:14 PM, Sebastien Douche sdou...@gmail.com wrote: On Sat, Oct 19, 2013 at 3:22 AM, Nick Coghlan ncogh...@gmail.com wrote: Once ensurepip has landed in Python 3.4 and pip 1.5 is released, we should be able to get back to updating the various metadata specs, with the aim of getting cross-platform wheel support in pip 1.6 :) If I understand right, we must waiting Python 3.5 (~2016) to have out-of-the-box wheel packages on Linux. Really? No, Python 3.4.x will ship whatever the latest version of pip is, so if 1.6 is out when 3.4.1 ships it’ll ship with pip 1.6. You can also always pip install —upgrade pip before then. There’s only so many hours in a day and so many people willing to get involved some things have to be delayed. -- Sebastien Douche sdou...@gmail.com Twitter: @sdouche / G+: +sdouche ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
On Fri, Nov 1, 2013 at 12:17 AM, Donald Stufft don...@stufft.io wrote: No, Python 3.4.x will ship whatever the latest version of pip is, so if 1.6 is out when 3.4.1 ships it’ll ship with pip 1.6. Good to know. Thanks Donald. -- Sebastien Douche sdou...@gmail.com Twitter: @sdouche / G+: +sdouche ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
On Thu, Oct 31, 2013 at 2:34 PM, Nick Coghlan ncogh...@gmail.com wrote: For all platforms *except* Windows, wheels are essentially caches -- there is no real reason to distribute them via PyPI at all, because OSx and Linux develpoers will have tools to build them from sdists. That's not at all true -- it IS true of homebrew, etc users, but not the least bit true of the genreral Mac user: * Installing XCode is free, but not default, and less than trivial, and even less than trivial to get right to build python extensions. * Many packages require third party compiled libs -- even harder to do on the Mac -- some are a downright pain in the ^%*. What if an OSX user wants to install numpy/scipy? How easy is it to do this from source (I really don't know)? A serious pain in the %^$ -- numpy is pretty easy, but scipy is a nightmare, requiring Fortran, etc. The community has addressed with with scientific python distributions: Anaconda, Canopy, Python(x,y), etc. But is sure would be nice to have binary wheels on PyPi And the cache thing is really nice, actually. Given PEP 453, it's probably worth allowing wheels on Mac OS X in pip 1.5, then we can tackle the thornier general *nix problem in the pip 1.6 time frame (which should also improve external dependency handling on Windows and Mac OS X as well). Sounds great! -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PyPI pull request #7
On Oct 31, 2013, at 4:32 AM, anatoly techtonik techto...@gmail.com wrote: On Wed, Oct 30, 2013 at 11:11 PM, Noah Kantrowitz n...@coderanger.net wrote: Please stop submitting pull requests. Development on the existing codebase is halted except for critical fixes or security issues. You are making extra work for people on this list and it will not be tolerated. Please consider this your final warning. I can't live as long as you are to see the new incantation of Python website (by PyCon 2013) or PyPI. I am willing to help, and this stuff you're saying is rather discouraging and like no, go waste your time somewhere else, we are not giving any code reviews for free. I understand that my reputation precedes me, but can we keep this strictly technical? What I am trying to do is to send small, incremental fixes. They don't affect security. I can commit it directly to avoid distracting overloaded PyPI (bus factor 2) team, and you can blame me for breaking things - ok, and ban if I break something - that's also ok. If learn previous PyPI and new PyPI, I can tell people more about it, and you can expect more pull requests - not from me, for new PyPI, once it is ready. And if I am going to submit any new features, like reST validation on edit and Markdown support - the code will be more decoupled than existing one to be almost directly reused for the new site. Why I am skeptical that new site will replace old one soon? Just because I don't believe in rewrites by one man army. When you develop public resource, you need to rely on external feedback. You also need some designer guy in a team. You also need a backlog for collaboration. My ETA for new PyPI is no earlier than PyCon 2014 if Donald and Richard will be working on it full time. So, instead of all-or-nothing scenario I can try to find some help with incremental approach. Your opinion is noted, however my statement stands and as I said, your continued derailment and disruption will not be tolerated. Thank you for your input. --Noah signature.asc Description: Message signed with OpenPGP using GPGMail ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
On Thu, Oct 31, 2013 at 9:49 AM, Daniel Holth dho...@gmail.com wrote: I'm sure you could build your own broken Windows Python, but who bothers? As long as we are clear that we are talking about a social difference here, not a technical one... IMO it pretty much boils down to the fact that on Windows you are probably using the python.org version of Python and not linking with random shared libraries from C:\Program Files\, but on Linux you are probably using one of a few dozen distro x distro-release Pythons AND your extension probably dynamically links to some other things your specific distro provides AND maybe you are depending on some versioned symbols in glibc oh the horror. On OS X I would hope you are uploading only wheels built with python.org-Python but maybe you aren't, everyone has their preferences. yes, they do -- but what is the target audience here? yes, a lot of folks use macports, homebrew etc, fewer, but I'm sure some, build their own Python from scratch -- but these are NOT the people that we want binary wheels for -- they don't want them anyway. The folks that we want to provide binary wheels for are NOT going to be building their own esoteric python, really, they are not. The MacPython community has a long standing tradition of building binaries (if at all) that are compatible with the python.org builds (and secondarily with the Apple-supplied Python) -- that is what I'd like to see supported by PyPi -- just like Windows Sure, someone could upload some oddly-built binary wheel to PyPi -- then it would not work for most users, and they would get complaints and hopefully fix it -- just like uploading a package with any other kind of bug in it. It is kind of a pain to build a truly portable binary package (when it depends on third-party compiled libs), but there is a small but committed group of folks doing that already -- let's make it easier to get stuff out there. Will a C extension built with Homebrew Python work with the Python Mac OS X installer from python.org? Probably, but given how painful ABI mismatches with shared libraries can be to debug, probably doesn't cut it until someone has the chance to thoroughly review the potential for problems. I disagree: 1) I don't care if homebrew built extensions work with other pythons -- you want to build with homebrew, create a homebrew recipe. -- there should be a policy about how binary packages posted on PyPi should be built. 2) We're never going to find out what the problems are until we give it a try. Fundamentally, I disagree with the premise here: If we can't guarantee that anything anyone uploads will work for everyone, we shouldn't allow it -- that's an unattainable goal. If we do want a more fool-proof approach, then the name auto-generated by wheel should include something that means python.org-build only if built with the python.org build. And I suppose we could try to put check code in there to make sure that extensions aren't linked to outside libs. Actually, that would be a handy utility to have, even if it didn't enforce anything. (and by the way, it's rea;lly easy to build a binary for Windows that's linked to an external dll also -- we expect package builders to be careful with that...) I was told a wheel built on Ubuntu probably won’t work on Linux, so I shut off Linux Wheels, at the same time I asked about Windows and OSX wheels, the answers I got from people were they would generally just work on Windows, and nobody gave me a straight answer on OSX. Sorry we weren't out there answering! Linux is a different story -- not only are there a lot of variations out there, but there also is no obvious standard one could point to that we'd expect folks to build binary wheels for. OS-X has (to much) variety though it is less than Linux, and more to the point, there is a standard Python out there -- the python.org builds. And there is a tradition of building binaries for that standard. AFAIK, it is pretty much the ONLY build of Python that package maintainers support with binaries (if they support anything). If you build a Wheel with Homebrew Python will it work on the official OSX installers? What if I have a library installed from Homebrew? probably not, but again, I don't care -- that's not what binary wheels on Python would be for. And more to the point -- this is a policy question -- don't upload a binary wheel to pypi that depends on homebrew (or anything else that Apple didn't provide) Essentially trying to figure out how likely it is that with the existing tags a wheel is going to work if we select based on them.-Chris One thing I'm not clear on -- if you do : pip install something will pip preferentially select a binary wheel (if enabled on pypi?) -- that may be an issue as folks will surely try to pip install stuff with homebrew, macports, etc. pythons (though the wheels are more likely to work in that direction. -Chris -- Christopher Barker, Ph.D.
Re: [Distutils] Plans for binary wheels, and PyPi and OS-X
On Oct 31, 2013, at 7:25 PM, Chris Barker chris.bar...@noaa.gov wrote: On Thu, Oct 31, 2013 at 9:49 AM, Daniel Holth dho...@gmail.com wrote: I'm sure you could build your own broken Windows Python, but who bothers? As long as we are clear that we are talking about a social difference here, not a technical one... IMO it pretty much boils down to the fact that on Windows you are probably using the python.org version of Python and not linking with random shared libraries from C:\Program Files\, but on Linux you are probably using one of a few dozen distro x distro-release Pythons AND your extension probably dynamically links to some other things your specific distro provides AND maybe you are depending on some versioned symbols in glibc oh the horror. On OS X I would hope you are uploading only wheels built with python.org-Python but maybe you aren't, everyone has their preferences. yes, they do -- but what is the target audience here? yes, a lot of folks use macports, homebrew etc, fewer, but I'm sure some, build their own Python from scratch -- but these are NOT the people that we want binary wheels for -- they don't want them anyway. The folks that we want to provide binary wheels for are NOT going to be building their own esoteric python, really, they are not. The MacPython community has a long standing tradition of building binaries (if at all) that are compatible with the python.org builds (and secondarily with the Apple-supplied Python) -- that is what I'd like to see supported by PyPi -- just like Windows Sure, someone could upload some oddly-built binary wheel to PyPi -- then it would not work for most users, and they would get complaints and hopefully fix it -- just like uploading a package with any other kind of bug in it. It is kind of a pain to build a truly portable binary package (when it depends on third-party compiled libs), but there is a small but committed group of folks doing that already -- let's make it easier to get stuff out there. Will a C extension built with Homebrew Python work with the Python Mac OS X installer from python.org? Probably, but given how painful ABI mismatches with shared libraries can be to debug, probably doesn't cut it until someone has the chance to thoroughly review the potential for problems. I disagree: 1) I don't care if homebrew built extensions work with other pythons -- you want to build with homebrew, create a homebrew recipe. -- there should be a policy about how binary packages posted on PyPi should be built. Well it was more of we didn’t know, so we played it safe. I don’t personally have a Python.org Installer Python (mine come from Homebrew) and I know others use the System Python. My knowledge of compiled stuff is pretty slim tbh :) 2) We're never going to find out what the problems are until we give it a try. Fundamentally, I disagree with the premise here: If we can't guarantee that anything anyone uploads will work for everyone, we shouldn't allow it -- that's an unattainable goal. It was less about guarantee and more about “Can we be reasonably sure this is going to work for most people”. If we do want a more fool-proof approach, then the name auto-generated by wheel should include something that means python.org-build only if built with the python.org build. And I suppose we could try to put check code in there to make sure that extensions aren't linked to outside libs. Actually, that would be a handy utility to have, even if it didn't enforce anything. (and by the way, it's rea;lly easy to build a binary for Windows that's linked to an external dll also -- we expect package builders to be careful with that...) I was told a wheel built on Ubuntu probably won’t work on Linux, so I shut off Linux Wheels, at the same time I asked about Windows and OSX wheels, the answers I got from people were they would generally just work on Windows, and nobody gave me a straight answer on OSX. Sorry we weren't out there answering! It’s not your fault! We realized this right before the pip release and sort of quickly added it after a quick check to see if anyone could tell us one way or another. Linux is a different story -- not only are there a lot of variations out there, but there also is no obvious standard one could point to that we'd expect folks to build binary wheels for. OS-X has (to much) variety though it is less than Linux, and more to the point, there is a standard Python out there -- the python.org builds. And there is a tradition of building binaries for that standard. AFAIK, it is pretty much the ONLY build of Python that package maintainers support with binaries (if they support anything). If you build a Wheel with Homebrew Python will it work on the official OSX installers? What if I have a library installed from Homebrew? probably not, but again, I don't care -- that's not
Re: [Distutils] PyPI pull request #7
On Thu, Oct 31, 2013 at 16:52 -0700, Noah Kantrowitz wrote: On Oct 31, 2013, at 4:32 AM, anatoly techtonik techto...@gmail.com wrote: On Wed, Oct 30, 2013 at 11:11 PM, Noah Kantrowitz n...@coderanger.net wrote: Please stop submitting pull requests. Development on the existing codebase is halted except for critical fixes or security issues. You are making extra work for people on this list and it will not be tolerated. Please consider this your final warning. I can't live as long as you are to see the new incantation of Python website (by PyCon 2013) or PyPI. I am willing to help, and this stuff you're saying is rather discouraging and like no, go waste your time somewhere else, we are not giving any code reviews for free. I understand that my reputation precedes me, but can we keep this strictly technical? What I am trying to do is to send small, incremental fixes. They don't affect security. I can commit it directly to avoid distracting overloaded PyPI (bus factor 2) team, and you can blame me for breaking things - ok, and ban if I break something - that's also ok. If learn previous PyPI and new PyPI, I can tell people more about it, and you can expect more pull requests - not from me, for new PyPI, once it is ready. And if I am going to submit any new features, like reST validation on edit and Markdown support - the code will be more decoupled than existing one to be almost directly reused for the new site. Why I am skeptical that new site will replace old one soon? Just because I don't believe in rewrites by one man army. When you develop public resource, you need to rely on external feedback. You also need some designer guy in a team. You also need a backlog for collaboration. My ETA for new PyPI is no earlier than PyCon 2014 if Donald and Richard will be working on it full time. So, instead of all-or-nothing scenario I can try to find some help with incremental approach. Your opinion is noted, however my statement stands and as I said, your continued derailment and disruption will not be tolerated. Thank you for your input. Noah, I don't see Anatoly's postings here as derailment or disruption and also see no reason to cultivate such a won't be tolerated tone here. Offering PRs is usually a total legit activity although generally bitbucket/github is the primary place to handle them, rather than this mailing list here. Differing oppinions on how things should evolve are also daily open-source business. However, Richard and Donald (the two pypi maintainers) have made it clear where their priorities of spending their own time are. Donald in particular is heading the new pypi implementation and there are several areas of possible collaboration there. So it's obvious now that non-essential PRs on the current PyPI code base have hardly a chance of getting eye time at the moment. best, holger ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
