Re: [Distutils] Phantom release/file and now can't upload

2015-07-07 Thread Richard Jones
Ah, sorry, I misunderstood. I believe I've removed the offending entry from
the database that's blocking you uploading. Please try again.


 Richard

On 8 July 2015 at 09:28, James Bennett  wrote:

> The 1.1 release exists, but PyPI thinks -- at least when it shows me the
> package-owner interface -- that there are no files for that release.
>
> Here's a screenshot of what I see:
>
> https://dl.dropboxusercontent.com/u/408510/django-contact-form-n-file.png
>
> Checked the 1.0 release just in case it went there, and that only shows
> the 1.0 package, not the 1.1 package.
>
>
> On Tue, Jul 7, 2015 at 4:23 PM, Richard Jones  wrote:
>
>> This is very strange - perhaps there's a caching issue or something, but
>> there's a file present on that release when I look now :/
>>
>> On 7 July 2015 at 15:09, James Bennett  wrote:
>>
>>> Earlier tonight I was trying to upload a new version (1.1) of
>>> https://pypi.python.org/pypi/django-contact-form
>>>
>>> Initially tried 'setup.py sdist' followed by 'twine upload -s' of the
>>> resulting tarball. Twine reported success, but no new release or file
>>> appeared on PyPI. Tried 'setup.py sdist upload'. That reported success, but
>>> no new release or file appeared on PyPI.
>>>
>>> I tried quite a lot of things after that and kept getting reports of
>>> success... and no new release or file on PyPI. I even went and copy/pasted
>>> a sample from the distutils documentation using 'setup.py sdist upload -r'
>>> to force PyPI just in case it was somehow using the wrong index.
>>>
>>> Eventually some combination of things must have at least partly worked,
>>> because information about a 1.1 release of django-contact-form appeared on
>>> PyPI... except the file itself still did not appear, and any attempt to
>>> upload returned:
>>>
>>> HTTPError: 400 Client Error: This filename has previously been used, you
>>> should use a different version.
>>>
>>> I am absolutely stumped as to what's going on. I've tried deleting and
>>> re-creating the 1.1 release to re-upload, but I'm stuck at being able to
>>> create only an empty release with no file, since PyPI will not allow the
>>> file to be uploaded.
>>>
>>> While I try to track down what happened, who has the power to override
>>> PyPI's disallowed filename restriction for this? As far as I can tell, the
>>> file was *never* listed on PyPI and so could not have been downloaded by
>>> anyone, and after struggling with this for a few hours I'm not at all in
>>> the mood to bump versions just to get PyPI to cooperate.
>>>
>>> ___
>>> Distutils-SIG maillist  -  Distutils-SIG@python.org
>>> https://mail.python.org/mailman/listinfo/distutils-sig
>>>
>>>
>>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] 400 Client Error: Binary wheel for an unsupported platform

2015-07-07 Thread David Cournapeau
On Tue, Jul 7, 2015 at 7:02 PM, Donald Stufft  wrote:

> On July 7, 2015 at 10:22:55 AM, Nick Coghlan (ncogh...@gmail.com) wrote:
> > On 8 July 2015 at 00:07, Antoine Pitrou wrote:
> > > On Tue, 7 Jul 2015 23:53:59 +1000
> > > Nick Coghlan wrote:
> > >> Unfortunately, the compatibility tagging for Linux wheels is currently
> > >> so thoroughly inadequate that even in tightly controlled environments
> > >> having a wheel file escape from its "intended" target platforms can
> > >> cause hard to debug problems.
> > >
> > > I'm not sure what you're pointing to, could you elaborate a bit?
> >
> > That was a reference to a case of someone building for Debian (I
> > think), and then having one of their wheel files end up installed on a
> > CentOS system and wondering why things weren't working.
> >
> > > For the record, building against a well-known, old glibc + gcc has
> > > served the Anaconda platform well.
> >
> > The key problem is that there's no straightforward way for us to
> > verify that folks are actually building against a suitably limited set
> > of platform APIs that all Linux distros in widespread use provide.
> >
> > And when it inevitably fails (which it will, Python and PyPI are too
> > popular for it not to), the folks running into the problem are
> > unlikely to be able to diagnose what has happened, and even once we
> > figure out what has gone wrong, we'd be left having to explain how to
> > blacklist wheel files, and the UX would just generally be terrible
> > (and the burden of dealing with that would fall on the pip and PyPI
> > maintainers, not on the individual projects publishing insufficiently
> > conservative Linux wheel files).
> >
> > That's why the platform override tags are such a nice idea, as it
> > becomes possible to start iterating on possible solutions to the
> > problem without affecting the default installation UX in the near
> > term.
> >
> >
>
> pip 7+ actually has the UI for blacklisting binary packages now, primarily
> to
> ask "no don't build a wheel for X", but it also functions to ask *not* to
> accept wheels for a particular project from PyPI.
>
> In my mind, the biggest reason to not just open up the ability to upload
> even
> generic linux wheels right now is the lack of a safe-ish default. I think
> if
> we added a few things:
>
> * Default to per platform tags (e.g. ubuntu_14_04), but allow this to be
>   customized and also accept "Generic" Linux wheels as well.
> * Put the libc into the file name as well since it's reasonable to build a
>   "generic" linux wheel that statically links all dependencies (afaik),
> however
>   it does not really work to statically link glibc. This means that even if
>   you build against an old version of glibc if you're running on a Linux
> that
>   *doesnt* use glibc (like Alpine which uses MUSL) you'll run into
> problems.
>
> I think that it is entirely possible to build a generic linux wheel that
> will
> work on any Linux built the same libc* of the same or newer version,
> however I
> think that you have to be careful if you do it. You have to ensure all your
> dependencies are statically linked (if you have any) and you have to ensure
> that you build against an old enough Linux (likely some form of CentOS).
>
> * Side question, since I don't actually know how a computer works: Is it
> even
>   possible to have a CPython extension link against a different libc than
>   CPython itself is linked against? What if static linking is involved
> since
>   there are non glibc libcs which actually do support static linking?
>

You can use versioned symbols to manage some of those issues (
https://www.kernel.org/pub/software/libs/glibc/hjl/compat/). Some softwares
even include their own libc, but this is getting hair fast.

The common solution is to do what Antoine mentioned: build on the lowest
common denominator, and reduce the dependencies on the system as much as
possible. To be honest, glibc is rarely your problem: the kernel is
actually more problematic (some common python packages don't build on
2.6.18 anymore), and C++ even more so. E.g. llvm 3.6 will not build on gcc
4.1 (the version of centos 5), so you need a new g++ which means a new
libtstdc++.

I am biased, but that's the kind of things where you may want to work with
"professional" providers with people paid to work on those boring but
critical issues.

David

>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] 400 Client Error: Binary wheel for an unsupported platform

2015-07-07 Thread Antoine Pitrou
On Tue, 07 Jul 2015 19:32:37 -0400
Tres Seaver  wrote:
> 
> Compared to Windows, and even somewhat OS/X, the win for uploading wheels
> to PyPI is miniscule:  pretty much everybody developing on Linux has or
> can get the toolchain required to build a wheel from an sdist, and can
> share those built wheels across the hosts she knows to be compatible.

That's a dramatically uninformed statement, to put it politely...  Some
packages have difficult-to-meet build dependencies, and can also take a
long time to do so. 

llvmlite, the package I'm talking about, builds against the
development libraries for LLVM 3.6 (a non-trivial download and install,
assuming you can find binaries of that LLVM version for your OS
version otherwise, count ~20 minutes to compile it with a modern
quad-core CPU).  We regularly have bug reports from people failing to
compile the package on Linux (and OS X), which is why we are considering
the option of pre-built binary wheels (in addition to the conda
packages we already provide, and which some people are reluctant to
use).

Regards

Antoine.


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


[Distutils] ANN: distlib 0.2.1 released on PyPI

2015-07-07 Thread Vinay Sajip
I've just released version 0.2.1 of distlib on PyPI [1]. For newcomers,
distlib is a library of packaging functionality which is intended to be
usable as the basis for third-party packaging tools.

The main changes in this release are as follows:

    Fixed issue #58: Return a Distribution instance or None from locate().

    Fixed issue #59: Skipped special keys when looking for versions.

    Improved behaviour of PyPIJSONLocator to be analogous to that of other
    locators.

    Added resource iterator functionality.

    Fixed issue #71: Updated launchers to decode shebangs using UTF-8.
    This allows non-ASCII pathnames to be correctly handled.

    Ensured that the executable written to shebangs is normcased.

    Changed ScriptMaker to work better under Jython.

    Changed the mode setting method to work better under Jython.

    Changed get_executable() to return a normcased value.

    Handled multiple-architecture wheel filenames correctly.


A more detailed change log is available at [2].

Please try it out, and if you find any problems or have any suggestions for
improvements, please give some feedback using the issue tracker! [3]

Regards,

Vinay Sajip

[1] https://pypi.python.org/pypi/distlib/0.2.1
[2] https://goo.gl/K5Spsp
[3] https://bitbucket.org/pypa/distlib/issues/new


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] 400 Client Error: Binary wheel for an unsupported platform

2015-07-07 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 07/06/2015 02:18 PM, Antoine Pitrou wrote:
> On Mon, 6 Jul 2015 19:03:19 +0100 Paul Moore 
> wrote:
>> On 6 July 2015 at 17:24, Antoine Pitrou 
>> wrote:
>>> (yes, the version number is off - but that's besides the point
>>> here, I think):
>>> 
>>> $ twine upload dist/llvmlite-0.0.0-py2.py3-none-linux_x86_64.whl 
>>> /home/antoine/.local/lib/python3.4/site-packages/pkginfo/installed.py:53:
>>> UserWarning: No PKG-INFO found for package: pkginfo 
>>> warnings.warn('No PKG-INFO found for package: %s' %
>>> self.package_name) Uploading distributions to
>>> https://pypi.python.org/pypi Uploading
>>> llvmlite-0.0.0-py2.py3-none-linux_x86_64.whl HTTPError: 400 Client
>>> Error: Binary wheel for an unsupported platform
>> 
>> PyPI does not support uploading binary wheels for Linux. This is a 
>> deliberate restriction because the tags supported by the wheel spec 
>> are not fine enough grained to specify compatibility between the 
>> myriad of Linux variations. The intention is that once a resolution
>> to that problem is found, this restriction will be lifted.
> 
> What if packagers take care of working around the issue? (for example
> by building on a suitably old Linux platform, as we already do for
> Conda packages)

Compared to Windows, and even somewhat OS/X, the win for uploading wheels
to PyPI is miniscule:  pretty much everybody developing on Linux has or
can get the toolchain required to build a wheel from an sdist, and can
share those built wheels across the hosts she knows to be compatible.

A-foolish-consistency'ly,



Tres.
- -- 
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)

iQIcBAEBAgAGBQJVnGGVAAoJEPKpaDSJE9HY5McP/0MCer5wS0MygSHw/UGrcvKN
2pDfGdFsucjY2cR5Wr/R28NEvTPzrmBghPAw4x47rNRpNWBOHvRMyOu3UWKJIeMI
r/elBcP9te26jA7o/clqkfnVNKGjcoDSis+YkyhZAfhEDNGXQAaSf8uNFsVDKMTj
W8ggxe0DraXHImE0X4rahu2a7Svd7yWIMtRv5eMP+HjfLA9ouQUuFNYXCuXDmTIS
5SJ+XkXjToKy0DSqPkknESVcCnF6sDjI1STo4dPi65QNlUQomv9m3TwLN9Ak+bx6
u0vM8R594PXMcb8cTnBXdmDbdDhQRym7Wy2Fr5zYs7nwJ8x13d2QKs5fQ/QhmUF5
DtyWeQekhK1+wupt6NQ8tXgu9jx9SV81XtvZSAp9SAVS9asC7BUjLAZEh9r3K07b
dTkaY719vFUioiYQazDjIrkMLxSKjGbBgkve78tkjDtfwvKDOBqofFEmcsBv8yh0
wA5/iYJ78Wmr2rti5d4/JwLU5Tc+NkkYcw1W0bRUgi+GX8vYtYQ03i36f33Kyf3f
z6n8rquZhGDIcPjbrEmveBJxErJTu/3ifuIy6NnTwCFmQNOl+HpqtJFtYS1a3NUR
0s9eivFgl6vwExN2KywYW9N/6tCcWyB0qvECG6tB/a+Ao3iW8KOciyyjaQZWwOFx
glybBbAdFgtqaLafaJnQ
=NgK8
-END PGP SIGNATURE-

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Phantom release/file and now can't upload

2015-07-07 Thread James Bennett
The 1.1 release exists, but PyPI thinks -- at least when it shows me the
package-owner interface -- that there are no files for that release.

Here's a screenshot of what I see:

https://dl.dropboxusercontent.com/u/408510/django-contact-form-n-file.png

Checked the 1.0 release just in case it went there, and that only shows the
1.0 package, not the 1.1 package.


On Tue, Jul 7, 2015 at 4:23 PM, Richard Jones  wrote:

> This is very strange - perhaps there's a caching issue or something, but
> there's a file present on that release when I look now :/
>
> On 7 July 2015 at 15:09, James Bennett  wrote:
>
>> Earlier tonight I was trying to upload a new version (1.1) of
>> https://pypi.python.org/pypi/django-contact-form
>>
>> Initially tried 'setup.py sdist' followed by 'twine upload -s' of the
>> resulting tarball. Twine reported success, but no new release or file
>> appeared on PyPI. Tried 'setup.py sdist upload'. That reported success, but
>> no new release or file appeared on PyPI.
>>
>> I tried quite a lot of things after that and kept getting reports of
>> success... and no new release or file on PyPI. I even went and copy/pasted
>> a sample from the distutils documentation using 'setup.py sdist upload -r'
>> to force PyPI just in case it was somehow using the wrong index.
>>
>> Eventually some combination of things must have at least partly worked,
>> because information about a 1.1 release of django-contact-form appeared on
>> PyPI... except the file itself still did not appear, and any attempt to
>> upload returned:
>>
>> HTTPError: 400 Client Error: This filename has previously been used, you
>> should use a different version.
>>
>> I am absolutely stumped as to what's going on. I've tried deleting and
>> re-creating the 1.1 release to re-upload, but I'm stuck at being able to
>> create only an empty release with no file, since PyPI will not allow the
>> file to be uploaded.
>>
>> While I try to track down what happened, who has the power to override
>> PyPI's disallowed filename restriction for this? As far as I can tell, the
>> file was *never* listed on PyPI and so could not have been downloaded by
>> anyone, and after struggling with this for a few hours I'm not at all in
>> the mood to bump versions just to get PyPI to cooperate.
>>
>> ___
>> Distutils-SIG maillist  -  Distutils-SIG@python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Phantom release/file and now can't upload

2015-07-07 Thread Richard Jones
This is very strange - perhaps there's a caching issue or something, but
there's a file present on that release when I look now :/

On 7 July 2015 at 15:09, James Bennett  wrote:

> Earlier tonight I was trying to upload a new version (1.1) of
> https://pypi.python.org/pypi/django-contact-form
>
> Initially tried 'setup.py sdist' followed by 'twine upload -s' of the
> resulting tarball. Twine reported success, but no new release or file
> appeared on PyPI. Tried 'setup.py sdist upload'. That reported success, but
> no new release or file appeared on PyPI.
>
> I tried quite a lot of things after that and kept getting reports of
> success... and no new release or file on PyPI. I even went and copy/pasted
> a sample from the distutils documentation using 'setup.py sdist upload -r'
> to force PyPI just in case it was somehow using the wrong index.
>
> Eventually some combination of things must have at least partly worked,
> because information about a 1.1 release of django-contact-form appeared on
> PyPI... except the file itself still did not appear, and any attempt to
> upload returned:
>
> HTTPError: 400 Client Error: This filename has previously been used, you
> should use a different version.
>
> I am absolutely stumped as to what's going on. I've tried deleting and
> re-creating the 1.1 release to re-upload, but I'm stuck at being able to
> create only an empty release with no file, since PyPI will not allow the
> file to be uploaded.
>
> While I try to track down what happened, who has the power to override
> PyPI's disallowed filename restriction for this? As far as I can tell, the
> file was *never* listed on PyPI and so could not have been downloaded by
> anyone, and after struggling with this for a few hours I'm not at all in
> the mood to bump versions just to get PyPI to cooperate.
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] 400 Client Error: Binary wheel for an unsupported platform

2015-07-07 Thread Antoine Pitrou
On Tue, 7 Jul 2015 11:02:40 -0400
Donald Stufft  wrote:
> In my mind, the biggest reason to not just open up the ability to upload even
> generic linux wheels right now is the lack of a safe-ish default. I think if
> we added a few things:
> 
> * Default to per platform tags (e.g. ubuntu_14_04), but allow this to be
>   customized and also accept "Generic" Linux wheels as well.

That would be cool :)

> * Put the libc into the file name as well since it's reasonable to build a
>   "generic" linux wheel that statically links all dependencies (afaik), 
> however
>   it does not really work to statically link glibc.

True. For example, here is the meat of a build of llvmlite on Linux:

$ ldd 
miniconda3/pkgs/llvmlite-0.6.0-py34_5/lib/python3.4/site-packages/llvmlite/binding/libllvmlite.so
linux-vdso.so.1 =>  (0x7ffeacefd000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x7f8c9e2f5000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 
(0x7f8c9e0d7000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x7f8c9decf000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x7f8c9dccb000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x7f8c9d9c5000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 
(0x7f8c9d7ae000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x7f8c9d3ea000)
/lib64/ld-linux-x86-64.so.2 (0x7f8c9fcee000)

It embeds LLVM and has no dynamic reference to anything beside the most
basic runtime libraries (libstdc++ is statically linked in).  The .so
file doesn't even require Python (but the rest of llvmlite, which loads
the .so file using ctypes, does need Python, of course).

We use a similar strategy on Windows and OS X.

> This means that even if
>   you build against an old version of glibc if you're running on a Linux that
>   *doesnt* use glibc (like Alpine which uses MUSL) you'll run into problems.

glibc vs. non-glibc is yet a different concern IMHO. Mainstream Linux
setups use glibc.

> You have to ensure all your
> dependencies are statically linked (if you have any) and you have to ensure
> that you build against an old enough Linux (likely some form of CentOS).

Yes, we use CentOS 5...

> * Side question, since I don't actually know how a computer works: Is it even
>   possible to have a CPython extension link against a different libc than
>   CPython itself is linked against?

Half-incompetent answer here: I think link-time it would be fine. Then
at library load time it depends on whether the actual system glibc is
ABI/API-compatible with the one the C extension (and/or CPython itself)
was linked with.

Regards

Antoine.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] 400 Client Error: Binary wheel for an unsupported platform

2015-07-07 Thread Donald Stufft
On July 7, 2015 at 10:22:55 AM, Nick Coghlan (ncogh...@gmail.com) wrote:
> On 8 July 2015 at 00:07, Antoine Pitrou wrote:
> > On Tue, 7 Jul 2015 23:53:59 +1000
> > Nick Coghlan wrote:
> >> Unfortunately, the compatibility tagging for Linux wheels is currently
> >> so thoroughly inadequate that even in tightly controlled environments
> >> having a wheel file escape from its "intended" target platforms can
> >> cause hard to debug problems.
> >
> > I'm not sure what you're pointing to, could you elaborate a bit?
>  
> That was a reference to a case of someone building for Debian (I
> think), and then having one of their wheel files end up installed on a
> CentOS system and wondering why things weren't working.
>  
> > For the record, building against a well-known, old glibc + gcc has
> > served the Anaconda platform well.
>  
> The key problem is that there's no straightforward way for us to
> verify that folks are actually building against a suitably limited set
> of platform APIs that all Linux distros in widespread use provide.
>  
> And when it inevitably fails (which it will, Python and PyPI are too
> popular for it not to), the folks running into the problem are
> unlikely to be able to diagnose what has happened, and even once we
> figure out what has gone wrong, we'd be left having to explain how to
> blacklist wheel files, and the UX would just generally be terrible
> (and the burden of dealing with that would fall on the pip and PyPI
> maintainers, not on the individual projects publishing insufficiently
> conservative Linux wheel files).
>  
> That's why the platform override tags are such a nice idea, as it
> becomes possible to start iterating on possible solutions to the
> problem without affecting the default installation UX in the near
> term.
>  
>

pip 7+ actually has the UI for blacklisting binary packages now, primarily to
ask "no don't build a wheel for X", but it also functions to ask *not* to
accept wheels for a particular project from PyPI.

In my mind, the biggest reason to not just open up the ability to upload even
generic linux wheels right now is the lack of a safe-ish default. I think if
we added a few things:

* Default to per platform tags (e.g. ubuntu_14_04), but allow this to be
  customized and also accept "Generic" Linux wheels as well.
* Put the libc into the file name as well since it's reasonable to build a
  "generic" linux wheel that statically links all dependencies (afaik), however
  it does not really work to statically link glibc. This means that even if
  you build against an old version of glibc if you're running on a Linux that
  *doesnt* use glibc (like Alpine which uses MUSL) you'll run into problems.

I think that it is entirely possible to build a generic linux wheel that will
work on any Linux built the same libc* of the same or newer version, however I
think that you have to be careful if you do it. You have to ensure all your
dependencies are statically linked (if you have any) and you have to ensure
that you build against an old enough Linux (likely some form of CentOS).

* Side question, since I don't actually know how a computer works: Is it even
  possible to have a CPython extension link against a different libc than
  CPython itself is linked against? What if static linking is involved since
  there are non glibc libcs which actually do support static linking?

---  
Donald Stufft  
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] 400 Client Error: Binary wheel for an unsupported platform

2015-07-07 Thread Nick Coghlan
On 8 July 2015 at 00:07, Antoine Pitrou  wrote:
> On Tue, 7 Jul 2015 23:53:59 +1000
> Nick Coghlan  wrote:
>> Unfortunately, the compatibility tagging for Linux wheels is currently
>> so thoroughly inadequate that even in tightly controlled environments
>> having a wheel file escape from its "intended" target platforms can
>> cause hard to debug problems.
>
> I'm not sure what you're pointing to, could you elaborate a bit?

That was a reference to a case of someone building for Debian (I
think), and then having one of their wheel files end up installed on a
CentOS system and wondering why things weren't working.

> For the record, building against a well-known, old glibc + gcc has
> served the Anaconda platform well.

The key problem is that there's no straightforward way for us to
verify that folks are actually building against a suitably limited set
of platform APIs that all Linux distros in widespread use provide.

And when it inevitably fails (which it will, Python and PyPI are too
popular for it not to), the folks running into the problem are
unlikely to be able to diagnose what has happened, and even once we
figure out what has gone wrong, we'd be left having to explain how to
blacklist wheel files, and the UX would just generally be terrible
(and the burden of dealing with that would fall on the pip and PyPI
maintainers, not on the individual projects publishing insufficiently
conservative Linux wheel files).

That's why the platform override tags are such a nice idea, as it
becomes possible to start iterating on possible solutions to the
problem without affecting the default installation UX in the near
term.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] 400 Client Error: Binary wheel for an unsupported platform

2015-07-07 Thread Antoine Pitrou
On Tue, 7 Jul 2015 23:53:59 +1000
Nick Coghlan  wrote:
> On 7 July 2015 at 07:46, Antoine Pitrou  wrote:
> > On Mon, 6 Jul 2015 22:34:38 +0100
> > Paul Moore  wrote:
> >> On 6 July 2015 at 19:18, Antoine Pitrou  wrote:
> >> > What if packagers take care of working around the issue?
> >> > (for example by building on a suitably old Linux platform, as we
> >> > already do for Conda packages)
> >>
> >> At the moment it's just a simple "if the wheel is for Linux, reject it" 
> >> test.
> >>
> >> As to whether that's too conservative, one of the Linux guys would
> >> need to comment. Maybe the issue is simply that we can't be sure
> >> people will take the care that you do, and the risk of people getting
> >> broken installs is too high?
> >
> > Then how about a warning, or a rejection by default with a well-known
> > way to bypass it?
> 
> Unfortunately, the compatibility tagging for Linux wheels is currently
> so thoroughly inadequate that even in tightly controlled environments
> having a wheel file escape from its "intended" target platforms can
> cause hard to debug problems.

I'm not sure what you're pointing to, could you elaborate a bit?

For the record, building against a well-known, old glibc + gcc has
served the Anaconda platform well.

Regards

Antoine.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] 400 Client Error: Binary wheel for an unsupported platform

2015-07-07 Thread Nick Coghlan
On 7 July 2015 at 07:46, Antoine Pitrou  wrote:
> On Mon, 6 Jul 2015 22:34:38 +0100
> Paul Moore  wrote:
>> On 6 July 2015 at 19:18, Antoine Pitrou  wrote:
>> > What if packagers take care of working around the issue?
>> > (for example by building on a suitably old Linux platform, as we
>> > already do for Conda packages)
>>
>> At the moment it's just a simple "if the wheel is for Linux, reject it" test.
>>
>> As to whether that's too conservative, one of the Linux guys would
>> need to comment. Maybe the issue is simply that we can't be sure
>> people will take the care that you do, and the risk of people getting
>> broken installs is too high?
>
> Then how about a warning, or a rejection by default with a well-known
> way to bypass it?

Unfortunately, the compatibility tagging for Linux wheels is currently
so thoroughly inadequate that even in tightly controlled environments
having a wheel file escape from its "intended" target platforms can
cause hard to debug problems.

There was a good proposal not that long ago to add a "platform tag
override" capability to both pip (for installation) and bdist_wheel
(for publication), but I don't know what became of that.

If we had that system, then I think it would be reasonable to allow
Linux uploads with a "pypi_linux_x86_64" override tag - they'd never
be installed by default, but folks could opt in to allowing them.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig