[Distutils] Re: Linux binary wheels?

[Bert JW Regeer]

The nice thing with the new macaroons is that theoretically I can provide 
someone my key to upload packages on my behalf for a singular PyPI project. 
This way I could allow a third-party service to backfill binary wheels for 
other platforms once I've released a version to PyPI.

Bert

> On Aug 20, 2019, at 12:25, Tzu-ping Chung  wrote:
> 
> 
> On 20 Aug 2019, at 23:47, Nick Timkovich  > wrote:
> 
>> On Tue, Aug 20, 2019, at 5:05 AM Matthew Brett > > wrote:
>> ...  Unless you meant wheels for non-Intel platforms, in which case, please 
>> do say more about you need.
>> 
>> Minor tangent: I've seen some people use https://www.piwheels.org/ 
>>  for Raspberry Pi (ARM 6/7), but could the ARM 
>> binaries be uploaded to PyPI?
>> 
>> I think I'm conflating the wheel building spec (is manylinux amd64 specific, 
>> or as long as the libraries are on any architecture?), toolchains, 
>> environment (sounds like Piwheels provides a platform to build them on), and 
>> package hosting (can PyPI host arbitrary archs?) in that one sentence.
> 
> This issue may be of relevant: https://github.com/pypa/warehouse/issues/3668 
> 
> 
> And there are even more layers to this problem. Wheels on piwheels are 
> currently maintained by RPi folks; if they are going into PyPI, either 
> package maintainers need to take over uploading (and even building) them, or 
> PyPI needs a way to allow (qualified) people to upload stuffs for packages 
> they don’t own. And maintainers might decide that ARM is not their supported 
> platform anyway, and get us back to where we started.
> 
>> --
>> Distutils-SIG mailing list -- distutils-sig@python.org 
>> 
>> To unsubscribe send an email to distutils-sig-le...@python.org 
>> 
>> https://mail.python.org/mailman3/lists/distutils-sig.python.org/ 
>> 
>> Message archived at 
>> https://mail.python.org/archives/list/distutils-sig@python.org/message/OXSUW73EO5DTUO34EFURN3KHCDAKNS4Z/
>>  
>> 
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at 
> https://mail.python.org/archives/list/distutils-sig@python.org/message/HYGOQ6EGLZHNN6HR3KY6XUGI2ZZQX2SI/

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/O5CXREPQQJ5SGHKUCHFNRFJQS7RUS2LM/

[Distutils] Re: Linux binary wheels?

[Wes Turner]

FWIW, conda supports the e.g. armv7l aarch32 and armv8 aarch64 / "ARM64"
platforms. Third-party-built packages are the norm there; where there are
channels like conda-forge and rpi. What does it mean to sign a CI build
from a given unsigned git tag?

"Build conda packages for ARM"
https://github.com/conda-forge/conda-forge.github.io/issues/269

Raspbian is built for armv7.

Raspberry Pi 2, 3, and 4 are armv8 CPUs (with hw SIMD, AES, SHA-1,
SHA-256), so they support armv7l (aarch32) and armv8 (aarch64)

Raspberry Pi Zero W have armv6 CPUs.

Raspberry Pi 4 have max 4GB of RAM, so that may be a reason for the
community to support aarch64.

On Tuesday, August 20, 2019, Tzu-ping Chung  wrote:

>
> On 20 Aug 2019, at 23:47, Nick Timkovich  wrote:
>
> On Tue, Aug 20, 2019, at 5:05 AM Matthew Brett 
> wrote:
>
>> ...  Unless you meant wheels for non-Intel platforms, in which case,
>> please do say more about you need.
>
>
> Minor tangent: I've seen some people use https://www.piwheels.org/ for
> Raspberry Pi (ARM 6/7), but could the ARM binaries be uploaded to PyPI?
>
> I think I'm conflating the wheel building spec (is manylinux amd64
> specific, or as long as the libraries are on any architecture?),
> toolchains, environment (sounds like Piwheels provides a platform to build
> them on), and package hosting (can PyPI host arbitrary archs?) in that one
> sentence.
>
>
> This issue may be of relevant: https://github.com/
> pypa/warehouse/issues/3668
>
> And there are even more layers to this problem. Wheels on piwheels are
> currently maintained by RPi folks; if they are going into PyPI, either
> package maintainers need to take over uploading (and even building) them,
> or PyPI needs a way to allow (qualified) people to upload stuffs for
> packages they don’t own. And maintainers might decide that ARM is not their
> supported platform anyway, and get us back to where we started.
>
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at https://mail.python.org/archives/list/distutils-sig@
> python.org/message/OXSUW73EO5DTUO34EFURN3KHCDAKNS4Z/
>
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/WLASKLKOIT2TEHXWEKFAQVFXSP74LSKB/

[Distutils] Re: Linux binary wheels?

[Tzu-ping Chung]

> On 20 Aug 2019, at 23:47, Nick Timkovich  wrote:
> 
>> On Tue, Aug 20, 2019, at 5:05 AM Matthew Brett  
>> wrote:
> 
>> ...  Unless you meant wheels for non-Intel platforms, in which case, please 
>> do say more about you need.
> 
> Minor tangent: I've seen some people use https://www.piwheels.org/ for 
> Raspberry Pi (ARM 6/7), but could the ARM binaries be uploaded to PyPI?
> 
> I think I'm conflating the wheel building spec (is manylinux amd64 specific, 
> or as long as the libraries are on any architecture?), toolchains, 
> environment (sounds like Piwheels provides a platform to build them on), and 
> package hosting (can PyPI host arbitrary archs?) in that one sentence.

This issue may be of relevant: https://github.com/pypa/warehouse/issues/3668

And there are even more layers to this problem. Wheels on piwheels are 
currently maintained by RPi folks; if they are going into PyPI, either package 
maintainers need to take over uploading (and even building) them, or PyPI needs 
a way to allow (qualified) people to upload stuffs for packages they don’t own. 
And maintainers might decide that ARM is not their supported platform anyway, 
and get us back to where we started.

> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at 
> https://mail.python.org/archives/list/distutils-sig@python.org/message/OXSUW73EO5DTUO34EFURN3KHCDAKNS4Z/
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/HYGOQ6EGLZHNN6HR3KY6XUGI2ZZQX2SI/

[Distutils] Re: Linux binary wheels?

[Nick Timkovich]

On Tue, Aug 20, 2019, at 5:05 AM Matthew Brett 
wrote:

> ...  Unless you meant wheels for non-Intel platforms, in which case,
> please do say more about you need.


Minor tangent: I've seen some people use https://www.piwheels.org/ for
Raspberry Pi (ARM 6/7), but could the ARM binaries be uploaded to PyPI?

I think I'm conflating the wheel building spec (is manylinux amd64
specific, or as long as the libraries are on any architecture?),
toolchains, environment (sounds like Piwheels provides a platform to build
them on), and package hosting (can PyPI host arbitrary archs?) in that one
sentence.
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/OXSUW73EO5DTUO34EFURN3KHCDAKNS4Z/

[Distutils] Re: Linux binary wheels?

[Thomas Kluyver]

On Tue, Aug 20, 2019, at 3:50 PM, Brian Skinn wrote:
> I wonder if there's an OS dependence here, though -- I'm almost certain I've 
> had to use `--only-binary` in the past, to avoid pip on my Windows machines 
> trying to download and build sdists, even when wheels were available.

Possibly you were trying to install something where the latest version didn't 
have a compatible wheel you could use, but an older version did. I think pip 
will prefer a newer version even if that means building from source.

Thomas--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/DHLH5LWMXPGC33W65BML7CBBWCOVZKKM/

[Distutils] Re: Linux binary wheels?

[Paul Moore]

On Tue, 20 Aug 2019 at 14:50, Brian Skinn  wrote:

> I wonder if there's an OS dependence here, though -- I'm almost certain I've 
> had to use `--only-binary` in the past, to avoid pip on my Windows machines 
> trying to download and build sdists, even when wheels were available.

Pip prefers newer versions over older ones. If there's a newer version
with no binaries, pip will use the source for that version and try to
build, even if there are older binaries. `--only-binary` would address
that (although there is also a `--prefer-binary` flag in newer
versions of pip, which is better suited to that situation).

Paul
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/FR464FPXMZKTR5QBCWYTYSPBSOXFRKYM/

[Distutils] Re: Linux binary wheels?

[Brian Skinn]

On Tue, Aug 20, 2019 at 6:14 AM Matthew Brett 
wrote:

> Hi,
>
> On Tue, Aug 20, 2019 at 11:10 AM Brian Skinn 
> wrote:
> >
> >
> >
> > On Tue, Aug 20, 2019, 06:05 Matthew Brett 
> wrote:
> >>
> >> Hi,
> >
> >
> >> See the links that Wes posted for more details, or try pip installing
> >> Numpy and Scipy in a new virtualenv, and see what happens.
> >
> >
> > Probably needs to:
> >
> >pip install --only-binary :all: numpy scipy
>
> Pip doesn't build from source by default - here's the output from my
> machine just now:
>
>  $ pip install numpy scipy
> Collecting numpy
>   Downloading
> https://files.pythonhosted.org/packages/4a/2e/cf0a2fea6d97604a0e058e804b50d589c31b360b317be9f5c126b22a560e/numpy-1.17.0-cp35-cp35m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
> (14.9MB)
>  || 15.0MB 38.9MB/s
> Collecting scipy
>   Downloading
> https://files.pythonhosted.org/packages/c5/40/f73bc951b060ba143f2092a3a2dd755f5ef2b52376fc3f4d297942aef04b/scipy-1.3.1-cp35-cp35m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
> (27.6MB)
>  || 27.6MB 35.8MB/s
> Installing collected packages: numpy, scipy
> Successfully installed numpy-1.17.0 scipy-1.3.1
>
>
Hard to argue with concrete data -- my bad!

I wonder if there's an OS dependence here, though -- I'm almost certain
I've had to use `--only-binary` in the past, to avoid pip on my Windows
machines trying to download and build sdists, even when wheels were
available.



> Cheers,
>
> Mattthew
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/OD7TDQFPTOZOST6FP367KG5LEWQJ5VES/

[Distutils] Re: Linux binary wheels?

[Matthew Brett]

Hi,

On Tue, Aug 20, 2019 at 11:10 AM Brian Skinn  wrote:
>
>
>
> On Tue, Aug 20, 2019, 06:05 Matthew Brett  wrote:
>>
>> Hi,
>
>
>> See the links that Wes posted for more details, or try pip installing
>> Numpy and Scipy in a new virtualenv, and see what happens.
>
>
> Probably needs to:
>
>pip install --only-binary :all: numpy scipy

Pip doesn't build from source by default - here's the output from my
machine just now:

 $ pip install numpy scipy
Collecting numpy
  Downloading 
https://files.pythonhosted.org/packages/4a/2e/cf0a2fea6d97604a0e058e804b50d589c31b360b317be9f5c126b22a560e/numpy-1.17.0-cp35-cp35m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
(14.9MB)
 || 15.0MB 38.9MB/s
Collecting scipy
  Downloading 
https://files.pythonhosted.org/packages/c5/40/f73bc951b060ba143f2092a3a2dd755f5ef2b52376fc3f4d297942aef04b/scipy-1.3.1-cp35-cp35m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
(27.6MB)
 || 27.6MB 35.8MB/s
Installing collected packages: numpy, scipy
Successfully installed numpy-1.17.0 scipy-1.3.1

Cheers,

Mattthew
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/UX4LUBUGQXYID3HB5FADEVR4ACGKFHL6/

[Distutils] Re: Linux binary wheels?

[Brian Skinn]

On Tue, Aug 20, 2019, 06:05 Matthew Brett  wrote:

> Hi,
>

See the links that Wes posted for more details, or try pip installing
> Numpy and Scipy in a new virtualenv, and see what happens.
>

Probably needs to:

   pip install --only-binary :all: numpy scipy

Else it'll just build from source?



> Cheers,
>
> Matthew
> --
>

-Brian

>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/HVHRPKJMNJJVA2TT3HLFYLQVTQ5DBJSO/

[Distutils] Re: Linux binary wheels?

[Matthew Brett]

Hi,

On Tue, Aug 20, 2019 at 1:23 AM Dan Stromberg  wrote:
>
> Hi folks.
> I have a pair of ideas about Linux binary wheels, which are currently (I 
> heard) unsupported.

Your information is wrong, I'm afraid - Manylinux wheels have been
standard, and widely used, for several years now.   Unless you meant
wheels for non-Intel platforms, in which case, please do say more
about you need.

See the links that Wes posted for more details, or try pip installing
Numpy and Scipy in a new virtualenv, and see what happens.

Cheers,

Matthew
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/HXQWSOTX5V4VD2NX573JWX7JRT2Y4EYJ/