[Distutils] Re: pip and missing shared system system library

[Ronald Oussoren via Distutils-SIG]

> On 10 Aug 2020, at 09:31, Jonathan DEKHTIAR  
> wrote:
> 
> I hope it won't be obscure. But sometimes, you **need** to compile with the 
> same compiler even if you don't exploit specific compiler features. Namely 
> for "Bug Compatibility" reasons: 
> https://en.wikipedia.org/wiki/Bug_compatibility 
> <https://en.wikipedia.org/wiki/Bug_compatibility>
> 
> When you work with low level libraries and tend to access hardware or driver 
> libraries (think GPUs, FPGAs, etc.) in many cases, compiling with GCC or 
> CLANG or whatever compiler has been used to compile the libraries you are 
> using is essential.
> This is a very common issues when working on specific hardware or embedded 
> software. Compilers are not "perfect" they have bugs like any other pieces (I 
> know we tend to forget it). Sometimes we don't care, sometimes we do. And 
> actually now that Deep Learning and Machine learning is really a big thing. 
> "Bug Compatibility" is an important issue. Try compiling CUDA code with the 
> wrong compiler: I wish you good luck in your debugging :D 

CUDA and embedded systems are not most extensions.  For a lot of software the 
specific compiler used is not an issue, except for compatibility (C++ ABI, 
runtime ABI for the C runtime).

Note that I reacted to a statement that a setup.py is always written for a 
specific compiler, which is untrue. Most setup.py files with extensions don’t 
specify a compiler at all, but rely on the compiler detection from 
distutils/setuptools. 

Ronald


> 
> Jonathan
> 
> 
>  Le lun., 10 août 2020 00:07:54 -0700 Ronald Oussoren via Distutils-SIG 
>  écrit 
> 
> 
> 
> > On 9 Aug 2020, at 18:59, David Mathog  > <mailto:dmat...@gmail.com>> wrote: 
> > 
> > On Sat, Aug 8, 2020 at 8:15 PM Jonathan DEKHTIAR 
> > mailto:cont...@jonathandekhtiar.eu>> wrote: 
> >> 
> >> So do you plan on "managing" which version of GCC or g++ people have and 
> >> issue a warning if they don't have the good one? 
> > 
> > A setup.py will always be written for a particular compiler, or maybe 
> > it will handle a couple, but they never handle a "general compiler". 
> 
> Except that almost all extensions written in C require a “general C 
> compiler”, not some version of GCC. 
> 
> > That was why the example in spec 
> > 
> > Requires-External C 
> > 
> > never made sense. It always should have been something like 
> > 
> > Requires-External gcc (>4.0) 
> 
> Not unless you write code that uses features specific to GCC, and even then 
> it is questionable as there are several other compilers that implement a 
> large subset of GCC language extensions (at least icc and clang). 
> 
> Ronald 
> 
> — 
> 
> Twitter / micro.blog: @ronaldoussoren 
> Blog: https://blog.ronaldoussoren.net/ <https://blog.ronaldoussoren.net/>
> --
> Distutils-SIG mailing list -- distutils-sig@python.org 
> <mailto:distutils-sig@python.org>
> To unsubscribe send an email to distutils-sig-le...@python.org 
> <mailto:distutils-sig-le...@python.org>
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/ 
> <https://mail.python.org/mailman3/lists/distutils-sig.python.org/>
> Message archived at 
> https://mail.python.org/archives/list/distutils-sig@python.org/message/MPG2S4YHMCCNL7K2FQ2X6GF4ZJ5T7W2I/
>  
> <https://mail.python.org/archives/list/distutils-sig@python.org/message/MPG2S4YHMCCNL7K2FQ2X6GF4ZJ5T7W2I/>
> 
> 

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/SQ7I27DHGLQXGVT4IBUQUGUJ2PWJIHQB/

[Distutils] Re: pip and missing shared system system library

[Ronald Oussoren via Distutils-SIG]

> On 9 Aug 2020, at 18:59, David Mathog  wrote:
> 
> On Sat, Aug 8, 2020 at 8:15 PM Jonathan DEKHTIAR
>  wrote:
>> 
>> So do you plan on "managing" which version of GCC or g++ people have and 
>> issue a warning if they don't have the good one?
> 
> A setup.py will always be written for a particular compiler, or maybe
> it will handle a couple, but they never handle a "general compiler".

Except that almost all extensions written in C require a “general C compiler”, 
not some version of GCC.

> That was why the example in spec
> 
> Requires-External C
> 
> never made sense.  It always should have been something like
> 
> Requires-External gcc (>4.0)

Not unless you write code that uses features specific to GCC, and even then it 
is questionable as there are several other compilers that implement a large 
subset of GCC language extensions (at least icc and clang).

Ronald

—

Twitter / micro.blog: @ronaldoussoren
Blog: https://blog.ronaldoussoren.net/
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/MPG2S4YHMCCNL7K2FQ2X6GF4ZJ5T7W2I/

[Distutils] Re: distutils.util.get_host_platform() and AIX

[Ronald Oussoren via Distutils-SIG]

> On 2 Aug 2019, at 09:02, Michael  wrote:
> 
> On 01/08/2019 11:07, Michael wrote:
>> So, to my question: currently, for AIX get_host_platform returns
>> something such as: aix-6.1.
>> 
>> Considering above: what would you - as more expericenced with
>> multi-oslevel packaging and low levels are accepted by high-levels, but
>> not v.v.
>> "What should the AIX get_host_platform() string contain?"
>> 
>> At a minimum I forsee:return "%s-%s.%s-%s" % (osname, version,
>> release, platform.architecture()[0])
>> 
>> But this does not address potential issues where the TL level within a
>> version.release has changed. (X.Y.TL5 built packages MIGHT work on
>> X.YTL4, but there is no reason to expect them to.
>> 
>> So, I would look to something that remains recognizable, but uses
>> different 'punctuation'
>> 
>> e.g., oslevel -s returns a string such as: 6100-09-10-1731
>> 
>> Then using the equivalent of:
>> 
>> version, release, service, builddata = '6100-09-10-1731'.split('-')
>> 
>> return "%s-%s.%s.%s-%s" % (osname, version, release, service,
>> platform.architecture()[0])
> 
> What I forgot to mention - there is likely incompatibilities when
> different compilers are used. This is definetly the case when source
> files need a C compiler - and I fear that the different run-time
> environments of gcc versus xlc (which does not need/link to glibc).
> 
> So, how does, e.g., macos account for differences between clang and gcc
> compiled executables and modules. Or are both compilers "gnu" oriented?

For macOS GCC and clang generate compatible code, that’s not a problem. 

For platform versions the situation is slightly more complicated, but it is in 
general possible to 
build on newer releases of macOS than you deploy on. However, this requires some
support from the code you are building, such as not using APIs that aren’t 
available on the
older version. That generally can cause issues with projects that use configure 
and can
automatically pick up symbols available on the build platform.

The Linux setup might be more interesting for you in that regard. The 
“manylinux” project [1] specifies how to build wheels that work on multiple 
linux versions, regardless of the version of libc.  In short this works by 
building with an older libc, to ensure that the wheels only use symbol versions 
that are available on all supported linux versions.

[1] https://github.com/pypa/manylinux 

Ronald

> 
> Michael
> 
> 
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at 
> https://mail.python.org/archives/list/distutils-sig@python.org/message/OZX3XG33TZ5GVBL4VMNPH3YFJZC7B27Z/

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/JTQMZ6ZX43KHA254EVXI36YZU2PXYFO6/

[Distutils] Re: Environment markers for GPU/CUDA availibility

[Ronald Oussoren via Distutils-SIG]

> On 4 Sep 2018, at 01:51, Nick Coghlan  wrote:
> 
> On Mon., 3 Sep. 2018, 5:48 am Ronald Oussoren,  <mailto:ronaldousso...@mac.com>> wrote:
> 
> What’s the problem with including GPU and non-GPU variants of code in a 
> binary wheel other than the size of the wheel? I tend to prefer binaries that 
> work “everywhere", even if that requires some more work in building binaries 
> (such as including multiple variants of extensions to have optimised code for 
> different CPU variants, such as SSE and non-SSE variants in the past).
> 
> As far as I'm aware, binary artifact size *is* the problem. It's just that 
> once you're  automatically building and pushing an artifact (or an image 
> containing that artifact) to thousands or tens of thousands of managed 
> systems, the wasted bandwidth from pushing redundant implementations of the 
> same functionality becomes more of a concern than the convenience of being 
> able to use the same artifact across multiple platforms.

Ok. I’m more used to much smaller deployments where I don’t always know up 
front what the capabilities are of the system that the code will run on.  

And looking at tensorflow specifically the difference in size is very much 
significant, the GPU variant is 5 times as large as the non-GPU variant (55MB 
vs 255MB). That’s a good reason for not wanting to unconditionally ship both 
variants.

Ronald

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/OIUWU26I755JIN5IZCQYGM47RXADFMOA/

[Distutils] Re: Environment markers for GPU/CUDA availibility

[Ronald Oussoren via Distutils-SIG]

> On 2 Sep 2018, at 18:04, Nick Coghlan  wrote:
> 
> On Sat, 1 Sep 2018 at 11:02, Tzu-ping Chung  wrote:
>> 
>> I’m not knowledgable about GPUs, but from limited conversations with others,
>> it is important to first decide what exactly the problem area is. Unlike 
>> currently
>> available environment markers, there’s currently not a very reliable way to
>> programmatically determine even if there is a GPU, let alone what that GPU 
>> can
>> actually do (not every GPU can be used by Tensorflow, for example).
> 
> As Tzu-Ping notes, using environment markers currently requires that
> there be a well-defined "Python equivalent" to explain how installers
> should calculate the install-time value of the environment marker.
> 
> However, even regular CPU detection has problems when it comes to
> environment markers, since platform_machine reports x86_64 on a 64-bit
> CPU, even if the current interpreter is built as a 32-bit binary, and
> there are other oddities like Linux having two different 32-bit ABIs
> (there's i686, which is the original 32 bit ABI that predates x86_64,
> and then there's x32, which is the full x86_64 instruction set, but
> using 32-bit pointers: https://github.com/pypa/pip/issues/4962 ).
> (Also see https://github.com/pypa/pipenv/issues/2397 for some
> additional discussion)

This is primarily an indication that there is a missing API: an API that tells 
what architecture the Python interpreter is build for, rather than the 
architecture of the CPU. Or maybe not: distutils.util.get_platform() could be 
taught to do the right thing here, as was already done for macOS in the 
(ancient) past (although it is probably better to introduce a new API because 
of backward compatibility concerns). 

[…]
> 
> Note that I don't think it's possible for folks to get away from the
> "3 projects" requirement if publishers want their users to be able to
> selectively *install* the GPU optimised version - when you're keeping
> everything within one project, then you don't need an environment
> marker at all, you just decide at import time which version you're
> actually going to import.

What’s the problem with including GPU and non-GPU variants of code in a binary 
wheel other than the size of the wheel? I tend to prefer binaries that work 
“everywhere", even if that requires some more work in building binaries (such 
as including multiple variants of extensions to have optimised code for 
different CPU variants, such as SSE and non-SSE variants in the past). 

Ronald
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/7S4NGHCTCD37CFVSHQOSBLOQG7RSRVUO/

Re: [Distutils] pythonhosted.org doc upload no longer works

[Ronald Oussoren]

> On 20 Oct 2017, at 14:42, Ronald Oussoren <ronaldousso...@mac.com> wrote:
> 
> 
>> On 7 Oct 2017, at 09:12, Giampaolo Rodola' <g.rod...@gmail.com 
>> <mailto:g.rod...@gmail.com>> wrote:
>> 
>> Any news about this please? This is becoming a problem as my outdated doc 
>> keeps being around and I have no way to remove it. 
> 
> I have the same problem with my projects: the documentation on 
> pythonhosted.org <http://pythonhosted.org/> can no longer be updated, and it 
> is also not possible to change that documentation into a link to a new 
> location.

Sigh… I still have outdated documentation on pythonhosted.org and no way to 
replace this with a link to the new location. As the documentation on 
pythonhosted is the top hit on Google I’d rather not remove the documentation 
there without any replacement.

Ronald


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] pythonhosted.org doc upload no longer works

[Ronald Oussoren]

> On 20 Oct 2017, at 15:52, Nick Coghlan <ncogh...@gmail.com> wrote:
> 
> On 20 October 2017 at 22:42, Ronald Oussoren <ronaldousso...@mac.com 
> <mailto:ronaldousso...@mac.com>> wrote:
> 
>> On 7 Oct 2017, at 09:12, Giampaolo Rodola' <g.rod...@gmail.com 
>> <mailto:g.rod...@gmail.com>> wrote:
>> 
>> Any news about this please? This is becoming a problem as my outdated doc 
>> keeps being around and I have no way to remove it. 
> 
> I have the same problem with my projects: the documentation on 
> pythonhosted.org <http://pythonhosted.org/> can no longer be updated, and it 
> is also not possible to change that documentation into a link to a new 
> location.
> 
> I have new documentation on Read the Docs, but Google searches still prefer 
> the old location and I would therefore prefer to have redirects on python 
> host.org <http://host.org/>.
> 
> BTW. Has there been any documentation beyond this list about this change? 
> 
> I don't believe so, as I believe the change from "deprecated" to "no longer 
> works" was a side effect of disabling uploads via the old service.

That’s too bad as this caused problems for a number of people.  I am aware that 
the PyPI team is severely understaffed at best, but getting hit by this is 
still annoying.  As far as I know even the deprecation was not communicated 
widely, sending mail to maintainers of packages that use the feature would have 
been nice.

I’m currently stuck with old documentation on the site that has Google ranking, 
and no way to change anything about that. A way to replace the documentation by 
a redirect would have been nice (and was something that I wanted to do anyways, 
I’ve moved documentation to Read the Docs because that way the website gets 
updated automatically leaving me more time to do useful work)

Ronald


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Documentation link on PyPI.org

[Ronald Oussoren]

> On 20 Oct 2017, at 17:51, Wes Turner <wes.tur...@gmail.com> wrote:
> 
> 
> 
> On Friday, October 20, 2017, Ronald Oussoren <ronaldousso...@mac.com 
> <mailto:ronaldousso...@mac.com>> wrote:
> 
>> On 28 Aug 2017, at 00:12, Brett Cannon <br...@python.org 
>> <javascript:_e(%7B%7D,'cvml','br...@python.org');>> wrote:
>> 
>> If you search the archive of this mailing list you will notice I asked this 
>> exact question about a month or 2 ago (I think). The answer I got was it is 
>> used on PyPI.org <http://pypi.org/>, but I don't know how to set it with 
>> setuptools (flit will probably add support after PEP 517).
> 
> To get back to this: I’m probably doing something wrong, but I haven’t 
> managed yet to make anything show up on PyPI.org <http://pypi.org/> when I 
> use “Project-URL” in metadata. 
> 
> There’s a pyobjc-core 4.0.1b1 wheel on PyPI.org <http://pypi.org/> that 
> includes two Project-URL lines in its metadata, but that doesn’t add anything 
> to the sidebar. I’ve manually verified that the metadata is present in the 
> wheel file (as I’m using a replacement egg_info command in my setup.py file 
> to generate this part of the metadata).
> 
> https://github.com/pypa/warehouse/search?utf8=✓= 
> <https://github.com/pypa/warehouse/search?utf8=%E2%9C%93=>"project-url"=Issues
> 
> https://github.com/pypa/warehouse/search?q= 
> <https://github.com/pypa/warehouse/search?q=>"Project-Url"=Commits=✓

I’m note sure what you’re trying to say here, but 
<https://github.com/pypa/warehouse/pull/878/files 
<https://github.com/pypa/warehouse/pull/878/files>> suggests that Project-URL 
is supported in the PyPI.org <http://pypi.org/> UI (assuming that it is running 
a version of the software from after merging this pull request).  But for some 
reason this doesn’t work for me, and I don’t know why. Either I’m doing 
something wrong, or the code receiving wheel files does not yet store 
project-urls information in the database.

>  
> 
> Ronald
> 
> P.S. I tried to upload to test.pypi.org <http://test.pypi.org/>, but that 
> didn’t work due to an SSL error:
> 
> $ twine upload -r testpypi 
> dist/pyobjc_core-4.0.1b1-cp27-cp27m-macosx_10_6_intel.whl 
> Uploading distributions to https://test.pypi.org/legacy/ 
> <https://test.pypi.org/legacy/>
> Uploading pyobjc_core-4.0.1b1-cp27-cp27m-macosx_10_6_intel.whl
> SSLError: HTTPSConnectionPool(host='test.pypi.org <http://test.pypi.org/>', 
> port=443): Max retries exceeded with url: /legacy/ (Caused by 
> SSLError(SSLError(1, u'[SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert 
> protocol version (_ssl.c:661)'),))
> 
> https://www.google.com/search?q=TLSV1_ALERT_PROTOCOL_VERSION 
> <https://www.google.com/search?q=TLSV1_ALERT_PROTOCOL_VERSION>
> 
> Looks like TLSv1 (TLS 1.0) is deprecated.
> 
> https://www.google.com/search?q=TLSV1_ALERT_PROTOCOL_VERSION+twine 
> <https://www.google.com/search?q=TLSV1_ALERT_PROTOCOL_VERSION+twine>
> 
> https://github.com/pypa/twine/issues/273 
> <https://github.com/pypa/twine/issues/273>

Hmm… that issue means I should be slightly worried, the VM I’m using for 
uploading releases might stop working for this some time in the future. 
Hopefully upgrading the python installation on that will avoid that.

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Documentation link on PyPI.org

[Ronald Oussoren]

> On 28 Aug 2017, at 00:12, Brett Cannon <br...@python.org> wrote:
> 
> If you search the archive of this mailing list you will notice I asked this 
> exact question about a month or 2 ago (I think). The answer I got was it is 
> used on PyPI.org, but I don't know how to set it with setuptools (flit will 
> probably add support after PEP 517).

To get back to this: I’m probably doing something wrong, but I haven’t managed 
yet to make anything show up on PyPI.org <http://pypi.org/> when I use 
“Project-URL” in metadata. 

There’s a pyobjc-core 4.0.1b1 wheel on PyPI.org <http://pypi.org/> that 
includes two Project-URL lines in its metadata, but that doesn’t add anything 
to the sidebar. I’ve manually verified that the metadata is present in the 
wheel file (as I’m using a replacement egg_info command in my setup.py file to 
generate this part of the metadata).

Ronald

P.S. I tried to upload to test.pypi.org <http://test.pypi.org/>, but that 
didn’t work due to an SSL error:

$ twine upload -r testpypi 
dist/pyobjc_core-4.0.1b1-cp27-cp27m-macosx_10_6_intel.whl 
Uploading distributions to https://test.pypi.org/legacy/
Uploading pyobjc_core-4.0.1b1-cp27-cp27m-macosx_10_6_intel.whl
SSLError: HTTPSConnectionPool(host='test.pypi.org', port=443): Max retries 
exceeded with url: /legacy/ (Caused by SSLError(SSLError(1, u'[SSL: 
TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:661)'),))

> 
> On Sun, Aug 27, 2017, 13:01 Ronald Oussoren <ronaldousso...@mac.com 
> <mailto:ronaldousso...@mac.com>> wrote:
>> On 27 Aug 2017, at 17:33, Wes Turner <wes.tur...@gmail.com 
>> <mailto:wes.tur...@gmail.com>> wrote:
>> 
>> You can add a URL in the long_description (e.g. to 
>> https://read-the-docs.readthedocs.org/ 
>> <https://read-the-docs.readthedocs.org/> )
> 
> I know, but that adds the URL to the project description and not the sidebar. 
> PEP 345 appears to specify metadata that could be used for this, and 
> warehouse seems to contain to make use of this (or at least a database model 
> that allows for specifying more links than just the default one). 
> 
> Adding links to the sidebar would be nicer with the new design on PyPI.org 
> <http://pypi.org/>, but there doesn’t seem to be a clean way to add 
> “Project-URL” metadata as described in PEP 345 to a wheel file. I could tweak 
> my setup.py files to add Project-URL metadata, but before I do that I’d like 
> to confirm that (a) that metadata is actually used by warehouse and (b) there 
> is no cleaner way to add this metadata using setuptools.
> 
> Ronald
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org 
> <mailto:Distutils-SIG@python.org>
> https://mail.python.org/mailman/listinfo/distutils-sig 
> <https://mail.python.org/mailman/listinfo/distutils-sig>

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] pythonhosted.org doc upload no longer works

[Ronald Oussoren]

> On 7 Oct 2017, at 09:12, Giampaolo Rodola'  wrote:
> 
> Any news about this please? This is becoming a problem as my outdated doc 
> keeps being around and I have no way to remove it. 

I have the same problem with my projects: the documentation on pythonhosted.org 
can no longer be updated, and it is also not possible to change that 
documentation into a link to a new location.

I have new documentation on Read the Docs, but Google searches still prefer the 
old location and I would therefore prefer to have redirects on python host.org.

BTW. Has there been any documentation beyond this list about this change? 

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Documentation link on PyPI.org

[Ronald Oussoren]

> On 27 Aug 2017, at 17:33, Wes Turner  wrote:
> 
> You can add a URL in the long_description (e.g. to 
> https://read-the-docs.readthedocs.org/ 
>  )

I know, but that adds the URL to the project description and not the sidebar. 
PEP 345 appears to specify metadata that could be used for this, and warehouse 
seems to contain to make use of this (or at least a database model that allows 
for specifying more links than just the default one). 

Adding links to the sidebar would be nicer with the new design on PyPI.org 
, but there doesn’t seem to be a clean way to add 
“Project-URL” metadata as described in PEP 345 to a wheel file. I could tweak 
my setup.py files to add Project-URL metadata, but before I do that I’d like to 
confirm that (a) that metadata is actually used by warehouse and (b) there is 
no cleaner way to add this metadata using setuptools.

Ronald___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Documentation link on PyPI.org

[Ronald Oussoren]

Hi,

Is it possible to create a documentation in the section with “Project Links” on 
the page describing a release on PyPI.org ? That is, can I 
add some metadata to an sdist or wheel that would add an entry to that list, 
and if so how? I’m using setuptools to create the distribution artefacts.

The reason I ask is that I currently use an index.html on pythonhosted.org that 
redirects to the real documentation because that adds a link to the 
documentation when looking at the release on pypi.python.org. That doesn’t work 
pypi.org.   

Ronald___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Transfering modules

[Ronald Oussoren]

> On 20 Aug 2017, at 05:56, Nick Coghlan  wrote:
> 
> On 20 August 2017 at 05:27, Ian Hartley  > wrote:
> Hi all,
> 
> I'm using PyCharm CE as my development environment, on a macintosh running 
> mac OS 10.12.3.
> 
> Ah, that would explain why my intuitions were all wrong - I know how the 
> Windows and *nix installation models work (and I believe the latter also 
> covers homebrew on macOS), but not how the macOS framework installers work :)

FWIW: The framework installers on python.org  work just 
like the other installers on upgrades: installing 3.6.2 on a system already 
running 3.6.1 replaces the 3.6.1 installation and doesn’t perform a 
side-by-side installation. The major differences w.r.t. a *nix installation are 
having an “odd” sys.prefix, and having different sys.prefix values for 
different feature releases.

Ronald___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] distlib and wheel metadata

[Ronald Oussoren]

> On 15 Feb 2017, at 15:11, Nathaniel Smith  wrote:
> 
> 
>> In the vast majority of publication-to-PyPi cases people won't need
>> the "integrates" field, since what they're publishing on PyPI will
>> just be their abstract dependencies, and any warning against using
>> "==" will recommend using "~=" or ">=" instead. But there *are*
>> legitimate uses of pinning-for-publication (like the PyObjC
>> metapackage bundling all its subcomponents, or when building for
>> private deployment infastructure), so there needs to be a way to
>> represent "Yes, I'm pinning this dependency for publication, and I'm
>> aware of the significance of doing so"
> 
> Why can't PyObjC just use regular dependencies? That's what distro
> metapackages have done for decades, right?

PyObjC is conceptually a single project that is split in multiple PyPI 
distributions to make
it easier to install only the parts you need (and can install, PyObjC wraps 
macOS frameworks
including some that may not be available on the OS version that you’re 
running). 

The project is managed as a single entity and updates will always release new 
versions of 
all PyPI packages for the project.

“pip install pyobjc==3.1” should install that version, and should not result in 
a mix of versions
if you use this to downgrade (which could happen if the metapackage used “>=“ 
to specify the
version of the concrete packages).

BTW. I’m not sure if my choice to split PyObjC in a large collection of PyPI 
packages is still
the right choice with current state of the packaging landscape.

Ronald  
(the PyObjC maintainer)

> 
> -n
> 
> -- 
> Nathaniel J. Smith -- https://vorpus.org 
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org 
> 
> https://mail.python.org/mailman/listinfo/distutils-sig 
> 
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Announcement: TLSv1.2 will become mandatory in the future

[Ronald Oussoren]

> On 10 Jan 2017, at 14:24, Donald Stufft  wrote:
> […] Past that, macOS is going to be the
> largest casualty since their system Python does not support TLSv1.2 yet in any
> version of their OS.

Not just the system Python on OSX, this also affects all Python.org 
 installers for OSX except 3.6. The 3.6 installer is the 
first one that doesn’t use the system installation of OpenSSL.  Annoyingly with 
OpenSSL on OSX you have to options: either use an up-to-date release or have 
OpenSSL use the system CA trust store, but not both.  Sigh…

I have no idea how may users use the Python.org  installers 
on OSX, but this has the potential to affect a largish number of users on OSX 
including newbies (but far from all users on OSX, there’s also a sizeable 
population using Homebrew or Anaconda).

Ronald___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Announcement: TLSv1.2 will become mandatory in the future

[Ronald Oussoren]

> On 10 Jan 2017, at 21:02, Donald Stufft <don...@stufft.io> wrote:
> 
> 
>> On Jan 10, 2017, at 3:01 PM, Ronald Oussoren <ronaldousso...@mac.com 
>> <mailto:ronaldousso...@mac.com>> wrote:
>> 
>> 
>>> On 10 Jan 2017, at 14:24, Donald Stufft <don...@stufft.io 
>>> <mailto:don...@stufft.io>> wrote:
>>> […] Past that, macOS is going to be the
>>> largest casualty since their system Python does not support TLSv1.2 yet in 
>>> any
>>> version of their OS.
>> 
>> Not just the system Python on OSX, this also affects all Python.org 
>> <http://python.org/> installers for OSX except 3.6. The 3.6 installer is the 
>> first one that doesn’t use the system installation of OpenSSL.  Annoyingly 
>> with OpenSSL on OSX you have to options: either use an up-to-date release or 
>> have OpenSSL use the system CA trust store, but not both.  Sigh…
>> 
>> I have no idea how may users use the Python.org <http://python.org/> 
>> installers on OSX, but this has the potential to affect a largish number of 
>> users on OSX including newbies (but far from all users on OSX, there’s also 
>> a sizeable population using Homebrew or Anaconda).
>> 
>> Ronald
> 
> 
> Ah yea I forgot those :( 

Could you file an issue on bugs.python.org <http://bugs.python.org/> about 
this? That way Ned will get notified, and the use of system OpenSSL can be 
reconsidered for a future 2.7 release.


Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Other ideas from today's packaging meetup at EuroPython

[Ronald Oussoren]

On 26 Jul 2014, at 08:54, Nick Coghlan ncogh...@gmail.com wrote:

 On 26 July 2014 15:27, Wichert Akkerman wich...@wiggy.net wrote:
 I suspect that for Linux you mean “system-provided Python”? Looking at
 https://www.python.org/downloads/release/python-341/ there is no python.org
 binary installer for Linux. Even if there was I would expect only a small
 number of people to use that over the system-provided version, considering
 Linux distributions generally do a excellent job packaging Python.
 
 Yes, by system Python on Linux, I mean the distro provided one.
 (Technically Apple provide one as well, but binary compatibility there
 is still governed by the python.org installer rather than the Apple
 version)

And the Apple uses the same build flags as the python.org installer, at least
for those flags that affect binary compatibility (such as the width of unicode 
characters
on older Python versions). AFAIK all other Python distributions on OSX do the 
same,
which should make extensions portable between them.

Thus, the only thing that affects the compatibility of wheels is the OSX 
release you
target, assuming that the extensions only use system provided libraries. All 
bets are
of when you upload wheels with extension that dynamically link with other 
libraries (such
as a homebrew package for  MySQL).

I guess the same is true for Linux releases: as long as the wheel only 
dynamically links
with libc, and not with other libraries, linux wheels should also work 
cross-platform between
old systems with a compatible libc. I have no idea how useful that would be 
though, I do
use Linux a lot but have a homogeneous set of servers and don’t really have a 
need to
distribute binaries across different linux distributions (or releases of them). 
Being in enterprise
IT does have its advantages once in while :-)

Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] platform specific python only wheels

[Ronald Oussoren]

Hi,

Is there a way to create a wheel that contains only python code, but can only 
be installed on particular platforms?

In particular, I’m looking for a way to create a wheel for py2app that can only 
be installed on OSX because py2app cannot “cross-compile” application bundles.  

Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] platform specific python only wheels

[Ronald Oussoren]

On 30 Jan 2014, at 19:19, Daniel Holth dho...@gmail.com wrote:

 Your best bet currently is to execute the mv command to change the
 filename of the wheel.
 
 I think bdist_wheel should probably accept a setup() argument to set
 the tags. Currently you can just set a universal flag which means
 pure 2+3 python.

There’s also —plat-name argument for the setuptools command, but that doesn’t 
work.

I’d have to rename the platform tag from “any” to a list of supported 
platforms, is that correct?

That would give a pretty convoluted filename, because the platform tag doesn’t 
allow for wildcards. That is, there is no way to specify “any linux”, only 
something like “linux_x86_64” (to borrow a tag from PEP 425).

Ronald

 
 On Thu, Jan 30, 2014 at 1:14 PM, Ronald Oussoren ronaldousso...@mac.com 
 wrote:
 Hi,
 
 Is there a way to create a wheel that contains only python code, but can 
 only be installed on particular platforms?
 
 In particular, I'm looking for a way to create a wheel for py2app that can 
 only be installed on OSX because py2app cannot cross-compile application 
 bundles.
 
 Ronald
 ___
 Distutils-SIG maillist  -  Distutils-SIG@python.org
 https://mail.python.org/mailman/listinfo/distutils-sig

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] platform specific python only wheels

[Ronald Oussoren]

On 30 Jan 2014, at 19:23, Vinay Sajip vinay_sa...@yahoo.co.uk wrote:

 
 
 On Thu, 30/1/14, Ronald Oussoren ronaldousso...@mac.com wrote:
 
 Is there a way to create a wheel that contains only python code, but
 can only be installed on particular platforms?
 
 Perhaps not yet, but the idea is there in the 'supports_environments'
 metadata in PEP 426. This is currently supported by distil, but I don't
 believe pip does yet.

supports_environment would work for me.

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] platform specific python only wheels

[Ronald Oussoren]

On 30 Jan 2014, at 19:36, Daniel Holth dho...@gmail.com wrote:

 On Thu, Jan 30, 2014 at 1:29 PM, Ronald Oussoren ronaldousso...@mac.com 
 wrote:
 
 On 30 Jan 2014, at 19:19, Daniel Holth dho...@gmail.com wrote:
 
 Your best bet currently is to execute the mv command to change the
 filename of the wheel.
 
 I think bdist_wheel should probably accept a setup() argument to set
 the tags. Currently you can just set a universal flag which means
 pure 2+3 python.
 
 There's also --plat-name argument for the setuptools command, but that 
 doesn't work.
 
 I'd have to rename the platform tag from any to a list of supported 
 platforms, is that correct?
 
 That would give a pretty convoluted filename, because the platform tag 
 doesn't allow for wildcards. That is, there is no way to specify any 
 linux, only something like linux_x86_64 (to borrow a tag from PEP 425).
 
 My position has been that if there is no alternative that works on a
 different platform, then at least the wheel metadata should say
 nothing. It's only there to allow the installer to pick among the best
 of several alternatives.

The py2app setup.py file currently hardcodes a platform check because users did 
install py2app on other platforms (mostly Windows) and then complained that it 
didn’t work. It is much nicer to tell the user that a package won’t work during 
installation.

Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] wheels on sys.path clarification (reboot)

[Ronald Oussoren]

On 30 Jan 2014, at 17:27, Vinay Sajip vinay_sa...@yahoo.co.uk wrote:

 
 
 On Thu, 30/1/14, Eric V. Smith e...@trueblade.com wrote:
 
 The flag really needs to convey 2 meanings:
  - - There are some C extensions here that can't be loaded
   unless they live on a real filesystem path.
 - - There is some code (maybe in this package, maybe in
  another package that uses this one) that uses this
  package's __file__, and that code  won't work unless
  __file__ points to a real filesystem path.
 
[…]

 My view is that there are probably two different assertions: one is
 will run from wheel (I say wheel rather than zip, since wheel
 implies a place to put that metadata) and the other is The C
 extensions are mandatory”.

It’s not just wheels, the same information can be used by application
 bundlers like py2exe and py2app to recognise if a particular component
can be included in a zipfile.

Py2app currently includes all python code in a zip file within an 
application bundle, unless a python package is on a manually maintained
list of software that won’t work that way.

Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Plans for binary wheels, and PyPi and OS-X

[Ronald Oussoren]

On Oct 31, 2013, at 07:24 PM, Donald Stufft don...@stufft.io wrote: On Oct 31, 2013, at 1:15 PM, Ronald Oussoren ronaldousso...@mac.com wrote: Is it "just" a matter of researching if the various build options on OSX really lead to binaries with the same ABI, or is more work needed? Basically it’s this:  I was told a wheel built on Ubuntu probably won’t work on Linux, so I shut off Linux Wheels, at the same time I asked about Windows and OSX wheels, the answers I got from people were they would generally just work on Windows, and nobody gave me a straight answer on OSX.I can't promise anything, but I'll try to provide some documentation on what does and does not work on OSX.From what I've seen the problem with Ubuntu wheels not working on other Linux's is due to changes in the glibc ABI (that is, code compiled with glibc X+1 might not work with glibc X), and that shouldn't be a problem on OSX.  If you build a Wheel with Homebrew Python will it work on the official OSX installers? What if I have a library installed from Homebrew? Essentially trying to figure out how likely it is that with the existing tags a wheel is going to work if we select based on them.Again, someone (and if no-one beats me to it, I) will have to document and test the various scenarios.The only real problem I expect is with extensions linking to shared libraries that aren't in the normal OSX install and aren't provided in the wheel. That is, a wheel for lxml that's linked with the Homebrew version of libxml2. That will only work on systems with Homebrew that have libxml2 installed. Once we know what, if any, problems exist for the various platforms then we can come up with fixes for them. This just hasn’t been high priority for me because of PEP453 work. To be honest the same problems likely exists on Windows, it’s just less likely and the benefits of prebuilt binaries greater.I'm not sure about thebenefits ofprebuilt binaries.Apparently building universal binaries on OSX is hard for a lot of users, although I've never had problems with that. But I've written lots of cross-platform C code when cross-platform Unix code was more than just supporting osx, linux and freebsd and am therefore not a good example of a Python users just wants to build an extension that works with his Python code :-)RonaldP.S. sorry if this message is somewhat garbled, I'm sending this from iCloud webmail and that doesn't always work properly with mailing lists  - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA ___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Plans for binary wheels, and PyPi and OS-X

[Ronald Oussoren]

On 21 Oct, 2013, at 20:52, Donald Stufft don...@stufft.io wrote:

 
 On Oct 21, 2013, at 1:02 PM, Chris Barker chris.bar...@noaa.gov wrote:
 
 On Fri, Oct 18, 2013 at 6:22 PM, Nick Coghlan ncogh...@gmail.com wrote:
 -- it would be very useful if folks could easily
 get binary wheels for OS-X
 
 We do plan to support it, but the pip devs uncovered a hole in the current
 wheel spec that means it generates the same filename on *nix systems for
 wheels that need to have different names for the download side of things to
 work properly
 
 THanks -- but really? don't OS-X wheels get:
 
 macosx_10_6_intel
 
 or some such tacked on? Where does that go wrong?
 
 Homebrew, Mac Ports, Fink. That would work OK if nobody ever installed things
 that the system didn't provide.

I don't understand this, what would break? Binary wheels that reference shared 
libraries that aren't included in the wheel (or a default system install) won't 
work, but that's also true on Windows.

What makes OSX more fun[1] than Linux is including shared libraries in the 
binary archive, unless you are careful with linking you'll end up with binaries 
that can only be installed in 1 filesystem location (that is, don't work in 
virtualenvs).

Ronald

[1] for a fairly twisted definition of fun ;-)

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Plans for binary wheels, and PyPi and OS-X

[Ronald Oussoren]

On 19 Oct, 2013, at 3:22, Nick Coghlan ncogh...@gmail.com wrote:

 
 On 19 Oct 2013 04:59, Chris Barker chris.bar...@noaa.gov wrote:
 
  Someone on another list indicated that pip installing binary wheels
  from PyPi will ONLY work for Windows.
 
  Is that the case? I think it's desperately needed for OS-X as well.
 
  Linux is so diverse that I can't imagine it being useful, but OS-X has
  only so many versions, and the python.org OS-X binaries are very clear
  in their requirements -- it would be very useful if folks could easily
  get binary wheels for OS-X
 
 We do plan to support it, but the pip devs uncovered a hole in the current 
 wheel spec that means it generates the same filename on *nix systems for 
 wheels that need to have different names for the download side of things to 
 work properly - hence the current Windows-only restriction.

Is that hole documented somewhere? 

Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Plans for binary wheels, and PyPi and OS-X

[Ronald Oussoren]

On 31 Oct, 2013, at 15:26, Nick Coghlan ncogh...@gmail.com wrote:

 On 31 October 2013 23:38, Ronald Oussoren ronaldousso...@mac.com wrote:
 
 On 21 Oct, 2013, at 20:52, Donald Stufft don...@stufft.io wrote:
 
 
 On Oct 21, 2013, at 1:02 PM, Chris Barker chris.bar...@noaa.gov wrote:
 
 On Fri, Oct 18, 2013 at 6:22 PM, Nick Coghlan ncogh...@gmail.com wrote:
 -- it would be very useful if folks could easily
 get binary wheels for OS-X
 
 We do plan to support it, but the pip devs uncovered a hole in the current
 wheel spec that means it generates the same filename on *nix systems for
 wheels that need to have different names for the download side of things 
 to
 work properly
 
 THanks -- but really? don't OS-X wheels get:
 
 macosx_10_6_intel
 
 or some such tacked on? Where does that go wrong?
 
 Homebrew, Mac Ports, Fink. That would work OK if nobody ever installed 
 things
 that the system didn't provide.
 
 I don't understand this, what would break? Binary wheels that reference 
 shared libraries that aren't included in the wheel (or a default system 
 install) won't work, but that's also true on Windows.
 
 The main difference I see is that on Windows, the ABI for the
 *CPython* shared library is significantly less likely to vary across
 machines.
 
 By contrast, it's relatively easy to change the ABI on *nix systems,
 as it depends on what you pass as configure options.
 
 Will a C extension built with Homebrew Python work with the Python Mac
 OS X installer from python.org?

As long as they use the same ABI Tag from PEP 425 extensions build with
one should work with the other. 

 Probably, but given how painful ABI
 mismatches with shared libraries can be to debug, probably doesn't
 cut it until someone has the chance to thoroughly review the potential
 for problems.

I'd have to make time for a thorough review but be sure, but AFAIK this is 
only a little more complicated on OSX than on (say) Linux and the additonal 
complication is already captured by the platform tag mentioned by Chris.  

That is, the CPython ABI is affected by:

* The usual configure flags (--with-pydebug, --enable-unicode, ...)
   
  These issues also exists on Windows, and the ABI Tag from PEP 425 means
  that wheels for different sets of configure flags can be recognized from
  their filename.

* The OSX deployment target (the minimal OSX release you want to support)

  This is already recorded by the distutils platform tag 

* The processor architectures supported

  This is also recorded in the distutils platform tag.

* Possibly the compiler

  This is pretty unlikely, as I (and others) already load extensions build 
  with clang in CPython binaries built with GCC (and v.v.), but the low-level 
  compiler support library might affect the ABI when using a compiler other
  than GCC or clang. And I'd expect that two-level namespaces on OSX make that
  question moot except for code that's explicitly built to disable that feature.

  And this does definitely affect CPython on Windows unless you use the limited 
ABI.
  The only reason this doesn't cause problems with binary installers on PyPI
  right now is that most users use the binary installers from www.python.org 
and hence
  are all using the same compiler version.

The deployment target can affect the unix level API a little, IIRC around 10.4 
or 10.5 the APIs on the POSIX level were adjusted a little to more fully comply 
with the UNIX specification and because of this the affected named resolve to 
different symbols based on the deployment target (more or less simular to 
symbol 
versioning in glibc). AFAIK that shouldn't affect the CPython ABI, but that 
would 
need to be carefully checked to be sure and I don't have time to do so right 
now.

To provide more detail on the disutils platform tag, on my machine:

 import distutils.util
 distutils.util.get_platform()
'macosx-10.8-intel'

This means the deployment target is OSX 10.8, with suppport for the 'intel' set 
of archictures (i386 and x86_64). This records the two variables that are most 
likely
to cause problems.

I'd expected more problems with binary eggs that are dynamicly linked to 
libraries
that are provided in the wheel because OSX's linker add an absolute path to the 
library
to be loaded unless you're careful.

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Plans for binary wheels, and PyPi and OS-X

[Ronald Oussoren]

On 31 Oct, 2013, at 17:49, Daniel Holth dho...@gmail.com wrote:

 On Thu, Oct 31, 2013 at 10:26 AM, Nick Coghlan ncogh...@gmail.com wrote:
 On 31 October 2013 23:38, Ronald Oussoren ronaldousso...@mac.com wrote:
 
 On 21 Oct, 2013, at 20:52, Donald Stufft don...@stufft.io wrote:
 
 
 On Oct 21, 2013, at 1:02 PM, Chris Barker chris.bar...@noaa.gov wrote:
 
 On Fri, Oct 18, 2013 at 6:22 PM, Nick Coghlan ncogh...@gmail.com wrote:
 -- it would be very useful if folks could easily
 get binary wheels for OS-X
 
 We do plan to support it, but the pip devs uncovered a hole in the 
 current
 wheel spec that means it generates the same filename on *nix systems for
 wheels that need to have different names for the download side of things 
 to
 work properly
 
 THanks -- but really? don't OS-X wheels get:
 
 macosx_10_6_intel
 
 or some such tacked on? Where does that go wrong?
 
 Homebrew, Mac Ports, Fink. That would work OK if nobody ever installed 
 things
 that the system didn't provide.
 
 I don't understand this, what would break? Binary wheels that reference 
 shared libraries that aren't included in the wheel (or a default system 
 install) won't work, but that's also true on Windows.
 
 The main difference I see is that on Windows, the ABI for the
 *CPython* shared library is significantly less likely to vary across
 machines.
 
 By contrast, it's relatively easy to change the ABI on *nix systems,
 as it depends on what you pass as configure options.
 
 On OS X I would hope you are uploading only wheels built with
 python.org-Python but maybe you aren't, everyone has their
 preferences.

And that shouldn't be a problem, the various builds of Python on OSX should be 
compatible if the have the same platform tag in the wheel filename. Well, 
barring idiocy like installing a completely different libc, that might cause 
problems because (at least for 2.7) some libc types leak into the CPython ABI.

But anyway, I'd love to use binary wheels in the future to distribute prebuilt 
binaries for some of my projects and that's why I reacted on a message that 
mentioned that binary installs aren't supported yet. If I don't understand what 
the issue is I can't try to help finding I solution :-)

Is it just a matter of researching if the various build options on OSX really 
lead to binaries with the same ABI, or is more work needed?   

Ronald

 
 Will a C extension built with Homebrew Python work with the Python Mac
 OS X installer from python.org? Probably, but given how painful ABI
 mismatches with shared libraries can be to debug, probably doesn't
 cut it until someone has the chance to thoroughly review the potential
 for problems.

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Comments on PEP 426

[Ronald Oussoren]

On 5 Sep, 2013, at 19:32, Daniel Holth dho...@gmail.com wrote:

 On Thu, Sep 5, 2013 at 8:33 AM, Ronald Oussoren ronaldousso...@mac.com 
 wrote:
 
 On 31 Aug, 2013, at 17:56, Nick Coghlan ncogh...@gmail.com wrote:
 
 
 This perception is just wrong. Mac OS X is a mess due to clang vs gcc vs 
 homebrew vs macports vs Xcode, Windows is a mess due to mingw and cygwin 
 and platform SDKs and visual studio (Express or otherwise). Linux distros 
 are also ridiculous, with different gcc variants, library locations and 
 various other things, depending on flavour and version.
 
 Don't forget fat binaries on OSX, those seem to cause problems as well for 
 folks building extensions.
 
 
 distutils itself is nigh impossible to work on, since it is underspecified, 
 and the fact it was added to the standard two years before unittest still 
 shows in its test suite. (This lack of test coverage is also a large part 
 of the reason setuptools *isn't* quite a drop-in replacement).
 
 The lack of maintainability of distutils is partly due to 
 underspecification, but that's just as true for setuptools.  Looking in from 
 the peanut gallery, the lack of forward movement for distutils is for a 
 large part due to pushback at first because folks were scared to break 
 existing code using distutils and later because it was deemed unacceptable 
 to break setuptools.
 
 This seems to lead to the sad conclusion that there will be improved package 
 installation and distribution through wheels, metadata 2.0 and related 
 standards but that distutils won't get to play along even though plain 
 distutils would work just fine for a very large subset of packages.  Most of 
 my own packages only use setuptools for its dependency resolution, and 
 that's for the most part provided by pip in the future.
 
 That's not to say that distutils is perfect, far from it, but creating a new 
 and improved build tool that's is capable of dealing with the complexities 
 of compiler variations will be a large project in its own right even without 
 migrating existing projects to it.
 
 Unfortunately there is not a malicious modernized distutils or
 stdlib suitable replacement vacuum; on the contrary great effort has
 been expended to that end. Distutils itself is not being removed or
 getting worse in an absolute sense and will continue to be about as
 suitable for packaging as it has been. A modern distutils is not
 required by but would be allowed by the cited plan. I will not do it
 because I have observed distutils2 but best of luck to someone else
 who may be willing and able to write the code.

A redesign is a good way to get burned out. I have a hard time to find the
energy to work on my opensource projects as it is :-(

 
 Distutils is un-maintainable not due to underspecification or unit
 tests but due to its basic design. Extensions have to reinitialize or
 muck about in each other's objects to work making it difficult to
 change either the interface or the implementation without breaking
 some of the hundreds of thousands of existing sdists.

That can be cleaned up by providing better internal interfaces, and should
have a change to succeed when proceeding in small steps. The hard part will
be to find a way forward that has good enough backward compatibility.

But that's not what I'd like to see in the near future. Distutils should IMHO 
provide enough functionality to integrate into the modern packaging 
environment: 
support for metadata 2.0 and support for creating wheel archives.

That's all that's needed to keep distutils usable while someone tries to
design and implement a replacement for distutils or a much cleaned up version
of distutils itself.

 
 If you can produce the modernized distutils then you may. It would be
 an acceptable plugin in the eventual pluggable build systems scheme.

I've also seen what happens when anyone tries to change distutils and
am not interested in doing any work beyond keeping OSX functioning unless
there would be buy-in before hand. That is, I'd be willing to do the work
needed to support metadata 2.0 and a bdist_wheel command in distutils
when there is a clear signal that such work wouldn't dismissed out of hand 
(as has happened in the past).

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Comments on PEP 426

[Ronald Oussoren]

On 31 Aug, 2013, at 17:56, Nick Coghlan ncogh...@gmail.com wrote:

 
 This perception is just wrong. Mac OS X is a mess due to clang vs gcc vs 
 homebrew vs macports vs Xcode, Windows is a mess due to mingw and cygwin and 
 platform SDKs and visual studio (Express or otherwise). Linux distros are 
 also ridiculous, with different gcc variants, library locations and various 
 other things, depending on flavour and version.

Don't forget fat binaries on OSX, those seem to cause problems as well for 
folks building extensions.   

 
 distutils itself is nigh impossible to work on, since it is underspecified, 
 and the fact it was added to the standard two years before unittest still 
 shows in its test suite. (This lack of test coverage is also a large part of 
 the reason setuptools *isn't* quite a drop-in replacement).

The lack of maintainability of distutils is partly due to underspecification, 
but that's just as true for setuptools.  Looking in from the peanut gallery, 
the lack of forward movement for distutils is for a large part due to pushback 
at first because folks were scared to break existing code using distutils and 
later because it was deemed unacceptable to break setuptools.   

This seems to lead to the sad conclusion that there will be improved package 
installation and distribution through wheels, metadata 2.0 and related 
standards but that distutils won't get to play along even though plain 
distutils would work just fine for a very large subset of packages.  Most of my 
own packages only use setuptools for its dependency resolution, and that's for 
the most part provided by pip in the future.

That's not to say that distutils is perfect, far from it, but creating a new 
and improved build tool that's is capable of dealing with the complexities of 
compiler variations will be a large project in its own right even without 
migrating existing projects to it.

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] What does it mean for Python to bundle pip?

[Ronald Oussoren]

On 23 Aug, 2013, at 0:52, Paul Moore p.f.mo...@gmail.com wrote:

 On 22 August 2013 23:08, Chris Barker - NOAA Federal chris.bar...@noaa.gov 
 wrote:
 I want to give it a shot for OS-X -- no one seems to want to maintian
 bdist_mpkg, and it's time to move forward...
 
 My impression is that the architecture and fat binary stuff on OSX is the 
 bit that may bite you. I know little or nothing about OSX, but I'm sure if 
 you try and report on how you get on, the people on the list will be able to 
 help you get things sorted and we will be able to get any dark corners ironed 
 out.

I don't really expect problems on OSX, I've used binary eggs in the past and 
those work just fine. Wheels seem to be simular enough to eggs to not expect 
problems there either.

The one thing that might be problematic later on is that distutils (and hence 
setuptools and eggs) uses labels for sets of architectures in fat binaries, 
while wheels can describe those directly. That is, distutils uses intel as 
the architecture string for the set {i386, x86_64}, while wheel can use 
something like darwin_i386.darwin_x86_64 (through PEP 425, the actual value 
may be different as this is based on a light rereading of the pep).  That is an 
optional difference and can be ignored for now.

Note that fat binaries are not at all problematic from the point of view of 
installing a wheel, a fat binary is a single file that happens to work on 
multiple CPU archectures.  Creating a structure that would allow for wheels 
that support both 32-bit and 64-bit Windows is harder because you'd have two 
.pyd files that obviously cannot have the same path in the filesystem or wheel 
archive (but easily solved by another level of indirection, such as .pyext 
directory that can contain extension files whose name is the result of 
distutils.util.get_platform()).

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] What does it mean for Python to bundle pip?

[Ronald Oussoren]

On 23 Aug, 2013, at 5:17, Nick Coghlan ncogh...@gmail.com wrote:
 
 That said, I'm considering the idea of adding a variant field to the
 compatibility tags for wheel 1.1, along the lines of what Oscar
 Benjamin suggested earlier. By default, installers would only find
 wheels without a variant defined, but users could opt in to looking
 for particular variants. The meaning of the variants field would be
 entirely distribution specific. Numpy, for example, could publish:
 
 numpy-1.7.1-cp27-cp22m-win32.whl
 numpy-1.7.1-cp27-cp22m-win32-sse.whl
 numpy-1.7.1-cp27-cp22m-win32-sse2.whl
 numpy-1.7.1-cp27-cp22m-win32-sse3.whl
 
 The only restrictions on the variant tags would be:
 1. must be ASCII
 2. must not contain '.' or '-' characters
 
 You could even go to the extent of using hashes as variant tags.

Is adding variants necessary? Numpy use runtime selection for picking
the most appropriate extension code (heck, AFAIK recent versions of GCC
can even compile multiple variants of functions and pick the right one
automaticly). 

A variant field like this introduces a new failure mode: uses that install
packages on one machine and then copy the entire tree to another one where 
the software crashes hard because the other machine is older and doesn't support
some CPU features.

That said, I have no experience with directly using SSE as most of my code
either doesn't vector calculations or isn't anywhere close to the performance
limitations of using naive code (CPU's are too fast ;-)

Regards,

   Ronald


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] distutils.util.get_platform() - Linux vs Windows

[Ronald Oussoren]

On 20 Aug, 2013, at 8:15, samuel.feren...@barclays.com wrote:

 -Original Message-
 From: Chris Barker - NOAA Federal [mailto:chris.bar...@noaa.gov] 
 Sent: Monday, August 19, 2013 7:13 PM
 To: Ferencik, Samuel: Markets (PRG)
 Cc: distutils-sig@python.org
 Subject: Re: [Distutils] distutils.util.get_platform() - Linux vs Windows
 
 On Fri, Aug 16, 2013 at 2:18 AM,  samuel.feren...@barclays.com wrote:
 It seems distutils.util.get_platform() semantically differs on Windows and
 Linux.
 
 Windows: the return value is derived from the architecture of the
 *interpreter*, hence for 32-bit Python running on 64-bit Windows
 get_platform() = 'win32' (32-bit).
 
 Linux: the return value is derived from the architecture of the *OS*, hence
 for 32-bit Python running on 64-bit Linux get_platform() = 'linux-x86_64'
 (64-bit).
 
 Is this intentional?
 
 This seems just plain wrong to me.
 
 For the record, running a 32 bit Python on a 64 bit OS_X box:
 
 In [5]: distutils.util.get_platform()
 Out[5]: 'macosx-10.6-i386'
 
 which is the answer I want.
 
 -Chris
 
 Chris,
 
 What does your 'uname -m' return? Is it possible you're really running a 
 32-bit
 Python on a *32-bit* OS X kernel? [http://superuser.com/q/161195]

disutils.util.get_platform() on OSX returns the architecture supported by the
current binary. I get:

: distutils.util.get_platform()
'macosx-10.8-intel'

This means that Python was build for a deployment target of 10.8 (that is, the 
binary
runs on OSX 10.8 or later) and supports the 'intel' set of architecures (i386 
and x86_64).

Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] distutils.util.get_platform() - Linux vs Windows

[Ronald Oussoren]

On 20 Aug, 2013, at 18:00, samuel.feren...@barclays.com wrote:

 -Original Message-
 From: Chris Barker - NOAA Federal [mailto:chris.bar...@noaa.gov] 
 Sent: Tuesday, August 20, 2013 5:47 PM
 To: Ferencik, Samuel: Markets (PRG)
 Cc: distutils-sig@python.org
 Subject: Re: [Distutils] distutils.util.get_platform() - Linux vs Windows
 
 On Mon, Aug 19, 2013 at 11:15 PM,  samuel.feren...@barclays.com wrote:
 
 What does your 'uname -m' return?
 
 x86_64
 
 Is it possible you're really running a 32-bit
 Python on a *32-bit* OS X kernel? [http://superuser.com/q/161195]
 
 nope -- I am quite deliberately running a 32 bit Python on my 64 bit
 OS (I have some custom code C++ Im using that is not yet 64 bit
 safe).
 
 That's strange. I'm on Python 3.3.1, and it seems to me that get_platform()
 derives the value from uname for OS X, similar to Linux.
 
(osname, host, release, version, machine) = os.uname()
...
elif osname[:6] == darwin:
import _osx_support, distutils.sysconfig
osname, release, machine = _osx_support.get_platform_osx(
distutils.sysconfig.get_config_vars(),
osname, release, machine)
return %s-%s-%s % (osname, release, machine)
 
 so I would expect uname -m to be in line with get_plaform(). But maybe I'm
 misreading that... Also, I don't have access to the _osx_support source code.

_osx_support is a pure python module in the stdlib, the source is in the usual 
location.

The behavior on OSX is quite intentional and ensures that disutils binary 
archive
names correctly reflect the use of fat binaries and the minimal supported OSX 
release.

The only thing that might need change is the name of the supported 
architectures,
the wheel spec has a better way to indicate multiple executable architectures 
than
making up names for every set of architectures that we care to support, but to 
be
honest I haven't had time yet to fully ingest the spec and work out if is 
completely
useful for fat binaries on OSX.

 
 
 return value is wrong on Linux and correct on
 Windows, right?
 
 no -- I'm saying that it's right on Windows (and OS-X), but wrong on Linux.
 
 I think you have misread my sentence, and we actually agree here.
 
 What's the next action? Report a Python bug? (That's a cultural question; I'm
 new to Python.)

http://bugs.python.org/

Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Q about best practices now (or near future)

[Ronald Oussoren]

On 18 Jul, 2013, at 13:48, Oscar Benjamin oscar.j.benja...@gmail.com wrote:

 On 17 July 2013 22:43, Nick Coghlan ncogh...@gmail.com wrote:
 
 On 18 Jul 2013 01:46, Daniel Holth dho...@gmail.com wrote:
 
 On Wed, Jul 17, 2013 at 11:12 AM, Brett Cannon br...@python.org wrote:
 I'm going to be pushing an update to one of my projects to PyPI this
 week
 and so I figured I could use this opportunity to help with patches to
 the
 User Guide's packaging tutorial.
 
 But to do that I wanted to ask what the current best practices are.
 
 * Are we even close to suggesting wheels for source distributions?
 
 No, wheels don't replace source distributions at all. They just let
 you install something without having to have whatever built the wheel
 from its sdist. It is currently nice to have them available.
 
 I'd like to see an ambitious person begin uploading wheels that have
 no traditional sdist.
 
 Argh, don't even suggest that. Such projects could never be included in a
 Linux distribution - we need the original source to push into a trusted
 build system.
 
 What do you mean by this?
 
 I interpret Daniel's comment as meaning that there's no setup.py in
 the sdist. And I think it's a great idea and that lots of others would
 be very happy to ditch the setup.py concept in favour of something
 entirely different from the distutils way of doing things.
 
 In another thread you mentioned the idea that someone would build
 without using distutils/setuptools by using a setup.py that simply
 invokes an alternate build system that is build-required by the sdist.
 That's fine for simple cases but how many 'python setup.py command's
 should the setup.py support?

I don't think that's clear at the moment. It could be as little as
bdist_wheel, that could be enough to interface to get from an extracted
sdist to a wheel.

The current focus is on defining a common metadata format (the metadata 2.0
JSON files) and a binary distribution format, and that's enough to keep
the folks doing the actual work occupied for now.

In the long run we'll probably end up with something like this:

* Sources from a VCS (that is, project in the layout used by those doing 
development)

  |
 [tool specific]
  |
  V

* sdist archive (sources + metadata.json + ???, to be specified)

  |
   [to be specified interface]
  |
  V

* wheel archive

  |
 [pip, PEP 376(?)]

* installed package


If I recall correctly the transformation from sdist to wheel is currently not 
specified because getting the last steps (binary distribution and installation) 
right 
is more important right now.  The exact format of an sdist, and the interface 
for specifying
how to build a wheel from an sdist is still open for discussion and 
experimentation. 

That is, what's the minimal tool that could be used to create wheels for 
distributions that contain
one or more python packages with dependency information? And what would be 
needed for
a more complex distribution with (optional) C extensions, data files, custom 
compilers, ...?

The initial interface to the build system could well be a setup.py file that the
build system will only invoke as python setup.py bdist_wheel --bdist-dir=DIR 
(with
build-time depedencies specified in the metdata file) because
that's easy to arrange for distutils/setuptools, and it should be easy enough to
provide a dummy setup.py file with just that interface for alternative build 
systems.

Ronald


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Q about best practices now (or near future)

[Ronald Oussoren]

On 17 Jul, 2013, at 17:46, Daniel Holth dho...@gmail.com wrote:

 On Wed, Jul 17, 2013 at 11:12 AM, Brett Cannon br...@python.org wrote:
 I'm going to be pushing an update to one of my projects to PyPI this week
 and so I figured I could use this opportunity to help with patches to the
 User Guide's packaging tutorial.
 
 But to do that I wanted to ask what the current best practices are.
 
 * Are we even close to suggesting wheels for source distributions?
 
 No, wheels don't replace source distributions at all. They just let
 you install something without having to have whatever built the wheel
 from its sdist. It is currently nice to have them available.
 
 I'd like to see an ambitious person begin uploading wheels that have
 no traditional sdist.

Do you mean an sdist without a setup.py? That will likely take some time,
for the time being projects will still need a setup.py that just prints
information on how to build them (or bootstraps the actual wheel building tool).

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Q about best practices now (or near future)

[Ronald Oussoren]

On 17 Jul, 2013, at 17:55, Paul Moore p.f.mo...@gmail.com wrote:

 
 On 17 July 2013 16:46, Daniel Holth dho...@gmail.com wrote:
  * Are we saying use setuptools for everyone, or still only if you need it
  (asking since there is a stub about installing setuptools but the simple
  example doesn't have a direct need for it ATM, but could use find_packages()
  and such)?
 
 Setuptools is the preferred distutils-derived system. distutils should
 no longer be considered morally superior.
 
 Personally, I still reserve judgement on setuptools. But that's mainly if you 
 actually use its features (you should carefully consider and understand the 
 implications if you use its script wrapper functionality, for example).
 
 I see no reason to knee-jerk use it if you don't use any of its 
 functionality, though. I may be in a minority on that, though :-)

I agree, and if metadata 2.0 and bdist_wheel support were added to distutils 
there'd be even less reason to use setuptools. I primarily use setuptools for 
its dependency system on installation, and that's nicely covered by using 
metadata 2.0, wheels and pip.

  
 The MEBS idea, or a simple setup.py emulator and a contract with the
 installer on which commands it will actually call, will eventually let
 you do a proper job of choosing build systems.
 
 By the way, what *does* MEBS mean? I've seen a few people use the term, but 
 never found an explanation...

MEta Build System.

Ronald

 
 Paul
 ___
 Distutils-SIG maillist  -  Distutils-SIG@python.org
 http://mail.python.org/mailman/listinfo/distutils-sig

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] vetting, signing, verification of release files

[Ronald Oussoren]

On 17 Jul, 2013, at 19:17, Trishank Karthik Kuppusamy t...@students.poly.edu 
wrote:
 
 To very briefly summarize our status without going into tangential details:
 
 1. We previously found and reported on this mailing list that if we naively 
 assigned a key to every PyPI project, then the metadata would not scale. We 
 would have security with little usability. This looks like an insoluble key 
 management problem, but we think we have a pretty good solution.
 2. The solution is briefly this: we now propose just two targets roles for 
 all PyPI files.
 2.1. The first role --- called the unstable targets role --- will have 
 completely online keys (meaning that it can kept on the server for automated 
 release purposes). The unstable role will sign for all PyPI files being 
 added, updated or deleted without question. The metadata for this role will 
 change all the time.
 2.2. The second role --- called the stable targets role --- will have 
 completely offline keys (meaning that keys are kept as securely as possible 
 and only used with manual human intervention). The stable role will sign for 
 only the PyPI files which have vetted and deemed trustworthy. The metadata 
 for this role is expected to change much less frequently than the unstable 
 role.
 
 Okay, sounds too abstract to some. What does this mean in practice? We want 
 to make key management simple. Preferably, as Nick Coghlan and others have 
 proposed before, we would want PyPI to initially, at least, sign for all 
 packages, because managing keys for every single project right off the bat is 
 potentially painful. Therefore, with that view in mind --- which is to first 
 accommodate PyPI signing for packages, and gradually allowing projects to 
 sign for their own packages --- we then consider what our proposal above 
 would do.
 
 Firstly, it would make key management so much simpler. There is a sufficient 
 number of offline keys used to sign metadata for a valuable and trustworthy 
 set of packages (done only every now and then), and an online key used to 
 make continuous release of PyPI packages possible (done all the time).
 
 1. Now suppose that the top-level targets role says: when you download a 
 package, you must first always ask the stable role about it. If it has 
 something to say about it, then use that information (and just ignore the 
 unstable role). Otherwise, ask the unstable role about it.
 2. Fine, what about that? Now suppose that the both the stable and unstable 
 roles have signed for some very popular package called FooBar 2.0. Suppose 
 further that attackers have broken into the TUF-secured PyPI repository. Oh, 
 they can't find the keys to the stable role, so they can't mess with the 
 stable role metadata without getting caught, but since the unstable keys are 
 online, they could make it sign for malicious versions of the FooBar 2.0 
 package.
 3. But no problem there! Since we have instructed that the stable role must 
 always be consulted first, then valid metadata about the intended, trusted 
 FooBar 2.0 package cannot be modified (not without getting all the human 
 owners of the keys to collude). The unstable role may be tampered with to 
 offer bogus metadata, but the security impact will be limited with *prior* 
 metadata about packages in the way-harder-to-attack stable role.

I'm trying to understand what this means for package maintainers. If I 
understand you correctly maintainers would upload packages just like they do 
now, and packages are then automaticly signed by the unstable role.  Then 
some manual process by the PyPI maintainers can sign a package with a stable 
row. Is that correct? If it is, how is this supposed to scale? The contents of 
PyPI is currently not vetted at all, and it seems to me that manually vetting 
uploads for even the most popular packages would be a significant amount of 
work that would have to be done by what's likely a small set of volunteers.

Also, what are you supposed to do when FooBar 2.0 is signed by the stable role 
and FooBar 2.0.1 is only signed by the unstable role, and you try to fetch 
FooBar 2.0.* (that is, 2.0 or any 2.0.x point release)? 

Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Replacing pip.exe with a Python script

[Ronald Oussoren]

On 16 Jul, 2013, at 13:25, Paul Moore p.f.mo...@gmail.com wrote:

 
 On the other hand, I'm missing something, as I don't see how the *current* 
 exe wrappers avoid meaning that there need to be separate 32-bit and 64-bit 
 versions of pip...

Couldn't you just ship both variants of the exe wrappers in a single 
distribution and then use the correct one for the current installation? That's 
what I'm doing in py2app.

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

[Ronald Oussoren]

On 16 Jul, 2013, at 15:08, Paul Moore p.f.mo...@gmail.com wrote:

 On 16 July 2013 13:42, Ronald Oussoren ronaldousso...@mac.com wrote:
  On the other hand, I'm missing something, as I don't see how the *current* 
  exe wrappers avoid meaning that there need to be separate 32-bit and 64-bit 
  versions of pip...
 
 Couldn't you just ship both variants of the exe wrappers in a single 
 distribution and then use the correct one for the current installation? 
 That's what I'm doing in py2app.
 
 That's OK for source-style installs (which is what setuptools does, and what 
 pip mostly cares about right now). But for bundling with Python it needs to 
 be considered (although it's just getting the right one in the right 
 installer). But for wheels it's a pain, because instead of having just a 
 single pip wheel, we need 32-bit and 64-bit wheels solely for the wrappers. 
 That sucks. Hard. (And it's not just for pip, nose will have the same 
 problem, as will many other projects that use exe wrappers). And the 
 bdist_wheel command currently doesn't recognise this *at all*. So wheels 
 using wrappers are potentially broken.

You could just have a wheel that contains two data files: wrapper-win32.exe and 
wrapper-win64.exe, then select the one that gets used as the wrapper for a 
specific script when you create that wrapper. 

That's assuming that the wrapper .exe gets created when a wheel is installed 
and is not included in the wheel.

 
 I think the correct solution is to explicitly have declarative support for 
 console script entry point metadata in PEP 426, as well as having tools 
 like bdist_wheel and distil do some explicit backward compatibility hacking 
 to remove legacy-style exe wrappers. The wheel install code should then 
 explicitly install appropriate wrappers for the target platform (which may be 
 exe wrappers similar to the current ones, but moving forward may be some 
 other mechanism if one is found).

Yikes, that means my assumption is wrong. The section on Recommended installer 
features in the wheel spec[1] says that the wrapper executable should be 
created on installation, does pip not do this?

 
 This is complex enough that I'm now concerned that we need reference wheel 
 install code in the stdlib, just so that people don't make up their own on 
 the basis that wheel is simple to install manually and screw it up. Also so 
 that we only have one style of command line script wrapper to deal with going 
 forward, not a multitude of mostly-compatible solutions.

I'd love to see comprehensive wheel support in the stdlib, but that may have to 
wait for 3.5 because the entire packaging systeem (wheels, metadata, ...) is 
moving forward quickly at the moment.  That said, it would be nice if distutils 
would grow support for creating wheels and modern metadata in sdists as that 
would mean I could drop usage of setuptools for most of my software (for python 
3.4). 

 
 Nick: See the above point re PEP 426 - do you agree that this needs 
 addressing in Metadata 2.0?
 
 Paul
 
 PS There is still the proviso that I haven't tested my assumption that the 
 separate 32 and 64 bit wrappers are *needed* (setuptools and distlib use 
 them, so I think it's a valid assumption, but I need to test). I will try to 
 get time to check that ASAP.

That depends on what the wrapper does, if it launches a regular python with the 
right command-line you might be able to get away with a single wrapper, if it 
loads python.dll and executes the script directory you do need separate 
wrappers for 32 and 64 bit.


[1] http://www.python.org/dev/peps/pep-0427/#recommended-installer-features

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Current status of PEP 439 (pip boostrapping)

[Ronald Oussoren]

On 13 Jul, 2013, at 7:31, Nick Coghlan ncogh...@gmail.com wrote:


 
 3. That means there are two main options available to us that I still 
 consider viable alternatives (the installer bundling idea was suggested in 
 one of the off list comments I mentioned):
 
 * an explicit bootstrapping script
 * bundling a *full* copy of pip with the Python installers for Windows and 
 Mac OS X, but installing it to site-packages rather than to the standard 
 library directory. That way pip can be used to upgrade itself as normal, 
 rather than making it part of the standard library per se. This is then 
 closer to the bundled application model adopted for IDLE in PEP 434 (we 
 could, in fact, move to distributing idle the same way).

Or automaticly invoke the bootstrap script during installation (for the Python 
installers), that we the installers don't end up with a stale version of pip.

Either option should be easy enough to add to the OSX installers.

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Current status of PEP 439 (pip boostrapping)

[Ronald Oussoren]

On 13 Jul, 2013, at 15:35, Brett Cannon br...@python.org wrote:

 
 
 
 On Sat, Jul 13, 2013 at 2:29 AM, Ned Deily n...@acm.org wrote:
 In article 55b209b3-9576-4cf0-b58c-2a1e692af...@stufft.io,
  Donald Stufft don...@stufft.io wrote:
  On Jul 13, 2013, at 1:31 AM, Nick Coghlan ncogh...@gmail.com wrote:
   I'm currently leaning towards offering both, as we're going to need a tool
   for bootstrapping source builds, but the simplest way to bootstrap pip for
   Windows and Mac OS X users is to just *bundle a copy with the binary
   installers*. So long as the bundled copy looks *exactly* the way it would
   if installed later (so it can update itself), then we avoid the problem of
   coupling the pip update cycles to the standard library feature release
   cycle. The bundled version can be updated to the latest available versions
   when we do a Python maintenance release.
 
 Off the top of my head, including a copy of pip as a pre-installed
 global site-package seems like a very reasonable suggestion.   For the
 python.org OS X installer, it should be no problem to implement.  It
 would be equally easy to implement for future 2.7 and 3.3 maintenance
 releases.
 
 Does Apple just install the python.org OS X installer for distribution, or do 
 they build their own thing?

They do their own thing.

 My only worry is that Apple will not get the message about including pip and 
 we will end up with an odd skew on OS X (I'm not worried about Linux distros 
 as they all seem to follow Python development closely).

That will happen anyway, pip won't get magically installed on current OSX 
releases and adding it to the upcoming 10.9 release is probably not possible 
either unless it already happens to be in the current beta (they appear to have 
a fairly early cutoff point for including new software at the Unix layer).

Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Current status of PEP 439 (pip boostrapping)

[Ronald Oussoren]

On 13 Jul, 2013, at 16:39, Vinay Sajip vinay_sa...@yahoo.co.uk wrote:
 
 
 I smoke-tested the script on vanilla Python 3.3 installations on Windows and
 OS X. On OS X, the pip script was written to the appropriate Frameworks
 folder, but not to /usr/local/bin - I assume it would be simple enough to
 arrange that?

Not installing in /usr/local/bin is a feature that makes it easier to maintain
several python installs without worrying about contamination (for example 
Python 3
and Python 2, or even 2.6 and 2.7).

The installer by default adds the framework 'bin' directory to the environment 
for
the shell of the user that installed the framework.

Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 439 and pip bootstrap updated

[Ronald Oussoren]

On 10 Jul, 2013, at 11:40, Richard Jones rich...@python.org wrote:

 On 10 July 2013 19:08, Vinay Sajip vinay_sa...@yahoo.co.uk wrote:
 Richard Jones r1chardj0n3s at gmail.com writes:
 
 pip without virtualenv in python 2 contexts is pretty rare (or at
 least *should* be wink) so I think I'll retain it in that bootstrap
 code.
 
 Perhaps I misunderstand, but what's the relevance of Python 2 contexts here?
 Aren't we talking about Python 3.4 and later?
 
 It makes sense to me (and Nick) to simplify the packaging overhead for
 users of Python 2. Currently the story is a bit of a mess (multiple
 sites with different approaches).

New features in a bugfix release? You better hope the RM doesn't hear :-) 

That said, 2.7 will be around for a while and adding a consistent installation
experience to both 3.4 and 2.7 does sound attractive and adding a new
script shouldn't have that many side effects.

What about backporting pyvenv as well? I guess that's way too invasive
for a bugfix release.

Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Updated drafts of metadata 2.0 (PEP 426 and PEP 440)

[Ronald Oussoren]

On 21 Jun, 2013, at 14:21, Paul Moore p.f.mo...@gmail.com wrote:

 On 21 June 2013 12:35, Nick Coghlan ncogh...@gmail.com wrote:
 The quick compatibility check is actually part of the wheel file
 naming specification (it's covered by the compatibility tags defined
 in PEP 425).
 
 Yes, I was thinking more of the sdist side - and also of how a pure Python 
 package can specify whether separate wheels are needed for Python 2 and 
 Python 3, for example. There's a wheel-specific setup.cfg extension for this, 
 but should it be in the metadata spec?

No, but it should be in the spec/documentation for the tool that builds sdists 
or wheels. That is, when metadata 2.0 support is added to 
distutils/setuptools/... it should document how the metadata is configured. 

Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Updated drafts of metadata 2.0 (PEP 426 and PEP 440)

[Ronald Oussoren]

On 21 Jun, 2013, at 1:24, Nick Coghlan ncogh...@gmail.com wrote:

 
  1. Python setup.py sdist and python setup.py dist_info will be changed to 
  generate pymeta.json files. But that will be for Python 3.4 only (there's a 
  big problem if this doesn't make it into 3.4...). Unless there's a 
  distutils backport or similar.
 
 The initial vehicles for migration will be Daniel's bdist_wheel command and 
 upgrades to setuptools.
 
 Standard library support may not land for 3.4, but we will at least have the 
 pip bootstrapping mechanism in place (which in turn will allow retrieval of 
 setuptools).

What would be needed to land support for 3.4? Is it just a matter of pushing 
distlib to the stdlib and updating distutils (add a way to configure 2.0 
metadata, add bdist_wheel and update sdist), or are there other other issues? 
Well, other than finalizing the PEP and its implementation in distlib?

Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Binary Wheels and universal builds on OS-X

[Ronald Oussoren]

On 31 May, 2013, at 22:17, Chris Barker - NOAA Federal chris.bar...@noaa.gov 
wrote:

 HI Folks,
 
 A few of us over on the pythonmac list have been hashing out how best
 to support binary packages on OS-X. The binary wheel option seems very
 promising.
 
 However, there is one Mac-specific issue that does not seem to be addressed:
 
 On OS-X, binaries can be universal -- what this means is that they
 have more than one binary architecture embedded in a single file; the
 system selects the right one at run time. Both executables and dynamic
 libraries can be universal. In theory, an universal binary can
 currently contain up to 4 architectures:

This isn't really a problem, distutils uses labels for the set of supported 
architectures as the architecture label in binary distributions (e.g. 
pyobjc_core-2.5-py3.4-macosx-10.8-intel.egg for an egg that supports the 
'intel' set of architectures (x86_64 and i386), see 
_osx_support.get_platform_osx in the CPython repository for the labels that are 
currently used.

This is not ideal, but works good enough for now. In the long run this should 
likely be replaced by the compressed tags sets from PEP 425 (at the cost of a 
much longer file names).

A much more interesting question is which binary wheels should be considered 
when installing a package, there a two clear ways to select binaries and both 
have there usecases. Let's say you're running on OSX 10.8 with a CPU that 
supports x86_64 and a Python installation with support for x86_64, i386 and ppc 
and compiled with support for OSX 10.4 or later 
(MACOSX_DEPLOYMENT_TARGET=10.4).  When installing a package you can pick 
either:

1) a binary wheel that supports this machine (a wheel compiled with deployment 
target 10.8 with only x86_64 support)

2) a binary wheel that supports the same systems as the Python installations

The former is good enough when you only want to run code on your machine, the 
latter can be needed when you want to deploy the installed binaries to another 
machine (for example because you are using py2app to create an application 
bundle that will be installed elsewhere).


 
 On my system right now, I have 32bitPPC + 32bit i386 universal python,
 running on an intel system:
 
 In [2]: distutils.util.get_platform()
 Out[2]: 'macosx-10.3-i386'

On my machine this returns the correct value: 'macosx-10.8-intel' (binary 
supports 10.8 or later and the i386 and x86_64 architectures). I do know this 
used to return wrong values on some version of OSX when using the Apple 
installation of Python because of the way they build Python.

Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Binary Wheels and universal builds on OS-X

[Ronald Oussoren]

On 4 Jun, 2013, at 18:35, Chris Barker - NOAA Federal chris.bar...@noaa.gov 
wrote:

 On Mon, Jun 3, 2013 at 3:47 PM, Daniel Holth dho...@gmail.com wrote:
 
 That would make sense. Can you come up with code to detect that a
 newly compiled extension is universal, and that a Python is?
 
 It looks like distutils.util.get_platform() now does the right thing
 for knowing what the currently running pyton is (see Ronald's message)

Correct. Barring installation errors, such as with some older releases of OSX, 
distutils.util.get_platform() already returns the correct information (both the 
supported architectures and the deployment target). 
 
 For determining the status of a newly compiled extension, I usually
 simple run the file command line utility on it:
 
 $ file python
 python: Mach-O universal binary with 2 architectures
 python (for architecture ppc):Mach-O executable ppc
 python (for architecture i386):   Mach-O executable i386
 
 (works for *.so, too...)
 
 that could easily be parsed out, but we would still want to know the
 deployment target, which maybe you could get by parsing otool output:
 
 $ otool -L python
 python (architecture ppc):
   /Library/Frameworks/Python.framework/Versions/2.7/Python
 (compatibility version 2.7.0, current version 2.7.0)
   /usr/lib/libmx.A.dylib (compatibility version 1.0.0, current version 
 47.1.0)
   /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
 version 88.3.10)
 python (architecture i386):
   /Library/Frameworks/Python.framework/Versions/2.7/Python
 (compatibility version 2.7.0, current version 2.7.0)
   /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
 version 88.3.10)
 
 (compatibility version of libSystem???)
 
 But I imagine there is a cleaner way -- Ronald??

The output of 'otool -l' lists (amongst others) the value of 
LC_VERSION_MIN_MACOSX which is the deployment target.

Both can also be found using macholib ;-)

Ronald


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Binary Wheels and universal builds on OS-X

[Ronald Oussoren]

On 4 Jun, 2013, at 18:27, Chris Barker - NOAA Federal chris.bar...@noaa.gov 
wrote:

 On Tue, Jun 4, 2013 at 1:23 AM, Ronald Oussoren ronaldousso...@mac.com 
 wrote:
 
 This isn't really a problem, distutils uses labels for the set of supported 
 architectures as the architecture label in binary distributions (e.g. 
 pyobjc_core-2.5-py3.4-macosx-10.8-intel.egg for an egg that supports the 
 'intel' set of architectures (x86_64 and i386), see 
 _osx_support.get_platform_osx in the CPython repository for the labels that 
 are currently used.
 
 This is not ideal, but works good enough for now. In the long run this 
 should likely be replaced by the compressed tags sets from PEP 425 (at the 
 cost of a much longer file names).
 
 I think now may be the long run.

Maybe, at least for 3.4 (assuming that wheel support gets in the stdlib by 
then).

 But in any case, as we were
 discussing, as in theory there could be lots of possible combinations,
 in practice there are only two in the wild (plus the non-universal
 ones...), and probably only one we really  need to keep supporting, so
 simply have a canonical name for the one or two is fine.
 
 A much more interesting question is which binary wheels should be considered 
 when installing a package,
 
 indeed.
 
 there a two clear ways to select binaries and both have there usecases. 
 Let's say you're running on OSX 10.8 with a CPU that supports x86_64 and a 
 Python installation with support for x86_64, i386 and ppc and compiled with 
 support for OSX 10.4 or later (MACOSX_DEPLOYMENT_TARGET=10.4).  When 
 installing a package you can pick either:
 
 1) a binary wheel that supports this machine (a wheel compiled with 
 deployment target 10.8 with only x86_64 support)
 
 can you link a newer deployment target-built lib with an older
 deployment-target executable? I guess so...

Yes. I've used extensions with a newer deployment target than the python 
executable, and even the other way around should work.

 
 2) a binary wheel that supports the same systems as the Python installations
 
 The former is good enough when you only want to run code on your machine, 
 the latter can be needed when you want to deploy the installed binaries to 
 another machine (for example because you are using py2app to create an 
 application bundle that will be installed elsewhere).
 
 I would say that we should clearly _prefer_ a wheel that fully
 supports the pyton the installer is used with.
 
 But I'm a bit ambivalent about what to do if there is no such wheel
 found, but there is one that supports the currently running
 architecture. Im inclined to say don't install it:

I'd currently prefer to only install binaries that exactly match the python 
installation, that's the most conservative solution, requires no changes to 
installation tools and is easier to reason about. 

 
 - We build the universal binaries to support people that don't want
 to know or care about such details.
 
 - We want to support universal binary wheels for those folks.
 
 - The primary reason to build binary wheels at all is to support
 non-native architectures -- it's a whole lot easier to simply build
 for your current machine, and homebrew and macports, and ??? support
 that already.
 
 All that being said, I sometimes think that we should simply have a 32
 bit and 64 bit binary out there, and forget all this universal stuff!
 It works OK for Windows

For some definition of OK ;-), the registry can be fun when you're running 32 
binaries on a 64-bit windows.  A clear advantage of universal binaries is that 
you can have one set of binaries and don't have to pick which architecture gets 
the most obvious installation directory (e.g. /lib vs. /lib64 on a lot of Linux 
systems). 

 
 On my system right now, I have 32bitPPC + 32bit i386 universal python,
 running on an intel system:
 
 In [2]: distutils.util.get_platform()
 Out[2]: 'macosx-10.3-i386'
 
 On my machine this returns the correct value: 'macosx-10.8-intel' (binary 
 supports 10.8 or later and the i386 and x86_64 architectures). I do know 
 this used to return wrong values on some version of OSX when using the Apple 
 installation of Python because of the way they build Python.
 
 This isn't an Apple build -- I'm pretty sure it's the one from
 Python.org...though looking now it's only 2.7.2 -- I guess I haven't
 upgraded in a while!
 
 now with 2.7.5:
 
 In [3]: distutils.util.get_platform()
 Out[3]: 'macosx-10.3-fat'
 
 I guess fat means 32bit ppc + intel, deployment target 10.3 ?

That's correct.

 
 So we may want to update distutils.util.get_platform to do a PEP 425
 name, but we've got something that could work now.

That can wait until the new packaging stuff (wheels, metadata 2.0, distlib, 
...) gets merged, and even then it might be nicer to keep 'fat' and 'intel' as 
short names.

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo

Re: [Distutils] Binary Wheels and universal builds on OS-X

[Ronald Oussoren]

On 4 Jun, 2013, at 23:53, Chris Barker - NOAA Federal chris.bar...@noaa.gov 
wrote:

 On Tue, Jun 4, 2013 at 9:55 AM, Ronald Oussoren ronaldousso...@mac.com 
 wrote:
 
 $ otool -L python
 python (architecture ppc):
  /Library/Frameworks/Python.framework/Versions/2.7/Python
 (compatibility version 2.7.0, current version 2.7.0)
  /usr/lib/libmx.A.dylib (compatibility version 1.0.0, current version 
 47.1.0)
  /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
 version 88.3.10)
 python (architecture i386):
  /Library/Frameworks/Python.framework/Versions/2.7/Python
 (compatibility version 2.7.0, current version 2.7.0)
  /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
 version 88.3.10)
 
 (compatibility version of libSystem???)
 
 But I imagine there is a cleaner way -- Ronald??
 
 The output of 'otool -l' lists (amongst others) the value of 
 LC_VERSION_MIN_MACOSX which is the deployment target.
 
 I don't see that on my machine -- is that l, a lower-case el'?

The option or the LC_VERSION_MIN_MACOSX value? I'm running OSX 10.8, where both 
are available, but according to Google LC_VERSION_MIN_MACOSX was introduced in 
a 10.6 subrelease. Without that load command you cannot easily see what the 
deployment target is (the best way I find while writing would be a tool that 
checks library load commands and the list of symbols used with a 
to-be-maintained registry).

 
 Both can also be found using macholib ;-)
 
 probably best not to add another dependency.
 
 But I've lot track on why we'd need this -- presumably the wheel
 builder would know what it built, and could supply the proper
 meta-data, and that would be that.

You don't need this, distutils (and hence setuptools) already knows this 
information and other build tools can also know this.

Ronald

 
 -Chris
 
 
 
 
 
 -- 
 
 Christopher Barker, Ph.D.
 Oceanographer
 
 Emergency Response Division
 NOAA/NOS/ORR(206) 526-6959   voice
 7600 Sand Point Way NE   (206) 526-6329   fax
 Seattle, WA  98115   (206) 526-6317   main reception
 
 chris.bar...@noaa.gov

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Draft PEP for JSON based metadata published

[Ronald Oussoren]

On 29 May, 2013, at 20:36, holger krekel hol...@merlinux.eu wrote:

 Hi Nick,
 
 On Mon, May 27, 2013 at 21:36 +1000, Nick Coghlan wrote:
 After preliminary reviews by Donald and Daniel, I have now pushed the
 first complete draft of the JSON-based metadata 2.0 proposal to
 python.org
 
 PEP 426 (metadata 2.0): http://www.python.org/dev/peps/pep-0426/
 
 After a first quick read i am wondering if i missed something
 with respect to test tools.  There are some fields which specify
 dependencies required for running tests, but there is nothing that
 would tell which test runner to use, which test command to invoke,
 or am i missing something?
 
 Basically i am wondering how PEP426 could be useful/used by tox.
 best,

There is something about testing in the metabuild hooks section, although that 
appears to be targetted towards running tests for an installed distribution.

I'm not sure yet how useful that hook would be for my packages, as the test 
suite isn't actually installed (it is in the sdist, but not in the output of 
bdist_*).

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Draft PEP for JSON based metadata published

[Ronald Oussoren]

On 27 May, 2013, at 13:36, Nick Coghlan ncogh...@gmail.com wrote:

 After preliminary reviews by Donald and Daniel, I have now pushed the
 first complete draft of the JSON-based metadata 2.0 proposal to
 python.org
 
 PEP 426 (metadata 2.0): http://www.python.org/dev/peps/pep-0426/

Could platform_release be added to the set of environment markers? On OSX 
platform.release() returns a kernel version number, while platform.version() 
returns a string that cannot easily be used as a marker.  It might also be nice 
to have markers for the marketing name and version of the OS next to the kernel 
name and release (for example Solaris 8 vs. SunOS 5.8, or darwin 12.3.0 vs. OSX 
10.8.3), but that would probably need a PEP of its own to add the functionality 
to the stdlib before it is used in metadata.

BTW. Why can marker expressions use only a limited subset of operators? In 
particular , =,  and = are not allowed and would be nice to have to specify 
packages that need a minimal platform version (parts of PyObjC have a 
dependency on the OSX release because the wrapped library is available in a 
subset of releases). Listing all OS releases would be fairly useless, 
platform.release() is too specific to use with '==' as new patch releases of 
OSX can introduce new micro versions of the kernel.

 PEP 440 (versioning): http://www.python.org/dev/peps/pep-0440/

The versioning spec mentions that distribution tools may refuse to publish 
distributions that pin the versions of dependencies. I understand why this is 
needed, and agree in general, but have a usecase that I don't know how to 
express without pinning.

In particular, PyObjC consists of a number of distributions (pyobjc-core, 
pyobjc-framework-Cocoa, ...) and an umbrella package (pyobjc) what depends on 
the various distributions to make it easier to install all of PyObjC. The 
umbrella package currently pins the versions of subpackages to ensure that pip 
install pyobjc==2.5.1 installs exactly that version of the entire project.  
When I'd use the compatible release specifier I can no longer easily ensure 
that users can install an exact version of the entire project, other than by 
hacking the system: specify a compatible version with an additional level that 
isn't used by the project (for example ~=2.5.2.0).  What is the correct way to 
create an umberella project without getting yelled at by distribution tools? 

Ronald

 
 With the rationale and commentary, they're over 3000 lines between
 them, so I'm not attaching them here.
 
 The rationale for many of the changes is at the end of each PEP, along
 with some comments on features that I have either rejected or
 deliberately chosen to defer to the next revision of the metadata (at
 the earliest).
 
 Those with BitBucket accounts may also comment inline on the drafts here:
 
 PEP 426: 
 https://bitbucket.org/ncoghlan/misc/src/05d3586464b10d6a04a35409468269d7c89a87ba/pep_drafts/pep-0426.txt?at=default
 PEP 440: 
 https://bitbucket.org/ncoghlan/misc/src/05d3586464b10d6a04a35409468269d7c89a87ba/pep_drafts/pep-0440.txt?at=default
 
 Cheers,
 Nick.
 
 --
 Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
 ___
 Distutils-SIG maillist  -  Distutils-SIG@python.org
 http://mail.python.org/mailman/listinfo/distutils-sig

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 438 - Transition Phase 1

[Ronald Oussoren]

On 19 May, 2013, at 2:51, Donald Stufft don...@stufft.io wrote:
 
 
 Forgot to mention, both of those options are available by clicking on urls 
 when viewing a package you have permissions on, see: 
 http://d.stufft.io/image/2h073q2L3Z29

I get a Forbidden error when following the urls link in Safari on OSX 10.8. 
The link does work properly with Chrome on the same machine.

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Simplify 426: Deprecate Author-email and Maintainer-email

[Ronald Oussoren]

On 25 Apr, 2013, at 15:58, Daniel Holth dho...@gmail.com wrote:

 I would prefer to see PEP 390 withdrawn and I think this has been
 suggested before. The metadata is already sourced from different files
 depending on your build system.


I wouldn't mind a PEP 390 update when the 2.0 metadata format is done
that limits it explicitly to distutils (and possibly setuptools), with the 
express purpose of making it possible to use a setup.cfg with a minimal 
setup.py file when using a new enough distutils version.

In particular, a python distribution without extensions shouldn't require
anything beyond this:

   from distutils.core import setup

   setup()

This would make it easier to teach (GUI) tools, like IDLE, to maintain
the input data used by distutils because they won't have to try to
parse and update Python code.

Using PEP 390 beyond distutils/setuptools isn't useful, there is no need
to restrict other tools in this way as long as they can generate output
in the right format (wheel and sdist archives).


 
 The .ini format and our parsers for it are really awful. I always
 resented having to learn it in order to use distutils/setuptools when
 every other language (RFC822, Python, JSON) is both better and already
 familiar.

Funnily, I tend to use ini files a lot at $work because the are good
enough for simple configuration and power-users/admins with a windows
background are more likely to understand them then json or (shudder) xml
files.

 
 FWIW bdist_wheel does something half-PEP-390 inspired with setup.cfg:
 
 [metadata]
 provides-extra =
tool
signatures
faster-signatures
 requires-dist =
distribute (= 0.6.34)
argparse; python_version == '2.6'
keyring; extra == 'signatures'
dirspec; sys.platform != 'win32' and extra == 'signatures'
ed25519ll; extra == 'faster-signatures'
 license-file = LICENSE.txt

I use something simular in a number of my packages, with a single setup.py 
file shared between them and all metadata in setup.cfg.

Ronald

 
 (https://bitbucket.org/dholth/wheel/src/tip/setup.cfg?at=default)
 
 
 On Thu, Apr 25, 2013 at 8:30 AM, Vinay Sajip vinay_sa...@yahoo.co.uk wrote:
 Nick Coghlan ncoghlan at gmail.com writes:
 
 splitting them is the right way to go, and also that a multi-file
 input format is likely a better option than trying to cram everything
 into one file.
 
 There are areas of overlap, if you consider archiver, builder and
 installer roles. For example, the source files and in-package data are
 needed by both builder and archiver. The beauty of JSON is that it allows you
 to have your cake and eat it, to an extent. For example, if you consider that
 inputs to the different roles are dicts, it's not a big deal if you have a
 higher-level dict which contains the others as sub-dicts. So I don't see a
 big issue with having one file as long as the schema is clearly defined so
 that a specific role could pull out the relevant stuff.
 
 Also note that in the PEP 397 implementation (dictConfig) there is already
 the ability to have cross-references to shared sections in a dict serialised
 as JSON, so there's no need to compromise on DRY even when there are data
 overlaps.
 
 I've added some support for dict-based configuration in distlib (a backport
 of the generic configuration stuff from 2.7/3.2 dictConfig, to support 2.6)
 but the use of that's not yet documented.
 
 Regards,
 
 Vinay Sajip
 
 ___
 Distutils-SIG maillist  -  Distutils-SIG@python.org
 http://mail.python.org/mailman/listinfo/distutils-sig
 ___
 Distutils-SIG maillist  -  Distutils-SIG@python.org
 http://mail.python.org/mailman/listinfo/distutils-sig

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP439 and backward compat / easy_install / distlib

[Ronald Oussoren]

On 25 Mar, 2013, at 19:16, PJ Eby p...@telecommunity.com wrote:
 
 
 Also, as far as detecting the need for setuptools, I think that can be
 done just by noticing whether the PKG-INFO included in an sdist is
 metadata 2.0 or not.  If it is, then setuptools should be explicitly
 declared as a build-time dependency, otherwise it's not needed.  If
 it's an older metadata version, then you probably need setuptools.

Is it even necessary to automaticly install setuptools? Setuptools-using 
package are supposed to use  ez_setup.py, or distribute_setup.py for 
distribute, to ensure that the setuptools package is available during setup.  
Although I must admit that I have no idea how many packages still do this 
instead of assuming that users will have installed setuptools anyway.

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP439 and backward compat / easy_install / distlib

[Ronald Oussoren]

On 26 Mar, 2013, at 13:27, Daniel Holth dho...@gmail.com wrote:
 stall themselves, they can just mention setup-requires and the installer 
 grabs the necessary setuptools.
  
  That can only be done for sdists with 2.0 metadata, sdists for older 
  versions don't have a setup-requires in their metadata.  This is not just 
  for installing, if you want to use setuptools in your setup.py you'll have 
  to make sure it is installed in your setup.py, and with the current version 
  of the packaging tools this means you have to use something like 
  ez_setup.py or tell users to install setuptools themselves.
 
 Yes, which is why we propose to assume Setup-Requires-Dist: setuptools if 
 Metadata-Version  2.0. Then a no-op ez_setup.py can be added to sys.modules 
 before setup.py runs and the installer will have a lot more control over that 
 side effect.

Just because I'm curious, is that control needed to make sure that a new enough 
version of setuptools gets used (e.g. one that supports modern features, 
instead of the 2 year old version that is mentioned in ez_setup.py for 
$SOME_OLD_PACKAGE)? 

Just assuming that every sdist with old metadata requires setuptools would 
work, although it will be strange to see that some packages @work that use 
plain disutils suddenly seem to require setuptool :-)

 
 
 These improved installers will target both 2.7 and 3.4. I do understand that 
 some people feel it is harder to say manually download the installer and 
 then install what you want rather than manually download and install the 
 package you want.

No me.  I'm glad to see that the hard work by everyone working in the packaging 
space is coming to fruition.  Infrastructure work is almost never glamorous, 
and work on Python's packaging system appears to be more stressful than average.

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP439 and backward compat / easy_install / distlib

[Ronald Oussoren]

On 26 Mar, 2013, at 16:21, Tres Seaver tsea...@palladion.com wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 On 03/26/2013 05:28 AM, Ronald Oussoren wrote:
 
 On 25 Mar, 2013, at 19:16, PJ Eby p...@telecommunity.com wrote:
 
 
 Also, as far as detecting the need for setuptools, I think that can
 be done just by noticing whether the PKG-INFO included in an sdist
 is metadata 2.0 or not.  If it is, then setuptools should be
 explicitly declared as a build-time dependency, otherwise it's not
 needed.  If it's an older metadata version, then you probably need
 setuptools.
 
 Is it even necessary to automaticly install setuptools?
 Setuptools-using package are supposed to use  ez_setup.py, or
 distribute_setup.py for distribute, to ensure that the setuptools
 package is available during setup.
 
 No, they are not.  That usage was for bootstrapping in an era when
 setuptools was not widely presetn.  Most packages have *removed* those
 files today.

I didn't know that, all my project still include the bootstrap code to make it 
easier to install them in a fresh build of python.  The distribute docs still 
mention that you should use distribute_setup.py (their version of ez_setup.py) 
in your project.

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 426 updated (with more than you ever wanted to know about version schemes)

[Ronald Oussoren]

On 12 Feb, 2013, at 8:08, Nick Coghlan ncogh...@gmail.com wrote:
 
 
 So, to my mind, the next PEP we're missing is actually one for the
 *sdist* format itself, including a definition for how the
 meta-packaging system should invoke the sdist-wheel build step,
 rather than one for the Archiver/Builder configuration data (which is
 what PEP 390 tries to be, and which I'm not convinced needs to be
 standardised at all, so long as the Archiver takes care of translating
 it to the standard formats).

PEP 390 removes the need for having a setup.py file when using the
distutils Builder, and should make it easier to migrate to some other
build.  Just using it for distutils (as the PEPs title mentions) would be
a win.

 
 My primary near(ish) term objectives are to ensure:
 - a project's choice of Archiver or Builder should *NOT* affect a
 user's choice of Installer (or vice-versa)
 - given an sdist and no other information, it should be possible to
 create any bdist_* format

Sounds like a good plan. 

 
 PEPs 426 and 427 between them should achieve the first objective,
 while the other parts of PEP 426 should get us a long way towards
 achieving the second (with ./setup.py bdist_wheel as the interim
 build hook, pending the introduction of anything better in a PEP for a
 clarified sdist format).

How far do you want to standardize the sdist format? For the goals you
mentioned earlier you'd only have to specify that an sdist contains
a configuration file that says which builder should be used. It might be
nice if the full PKG-INFO contents were also available to extract some
metadata without invoking the builder.

Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 426 updated (with more than you ever wanted to know about version schemes)

[Ronald Oussoren]

On 12 Feb, 2013, at 9:33, Nick Coghlan ncogh...@gmail.com wrote:

 On Tue, Feb 12, 2013 at 6:25 PM, Ronald Oussoren ronaldousso...@mac.com 
 wrote:
 PEPs 426 and 427 between them should achieve the first objective,
 while the other parts of PEP 426 should get us a long way towards
 achieving the second (with ./setup.py bdist_wheel as the interim
 build hook, pending the introduction of anything better in a PEP for a
 clarified sdist format).
 
 How far do you want to standardize the sdist format? For the goals you
 mentioned earlier you'd only have to specify that an sdist contains
 a configuration file that says which builder should be used. It might be
 nice if the full PKG-INFO contents were also available to extract some
 metadata without invoking the builder.
 
 The sdist format already requires PKG-INFO

Ouch, it's glaringly obviously now that you mention it but I never noticed the 
PKG-INFO file in sdists before :-(

 - it just needs to be
 documented *as* a standard format, rather than implied by what
 distutils creates when you call 'setup.py sdist'. PEP 426 actually
 takes a fairly big step in that direction already by declaring that
 sdist archives should contain a PKG-INFO file that follows PEP 426
 (see http://www.python.org/dev/peps/pep-0426/#metadata-files)
 
 The missing piece is a standard hook for invoking the builder to
 create a wheel. Currently, that would be via ./setup.py bdist_wheel,
 but we want to get away from the setup.py format.

I agree with that.  The hook could be one or two new header fields in the 
PKG-INFO
file, with a PEP that describes those keys and how the builder is invoked and 
what 
it is supposed to do. Am I understanding this correctly?

Something like:

Extension: pepYYY-builder
pepYYY-builder/dist: bento (=1.1)
pepYYY-builder/build: bento.builder:run

Ronald

 
 Cheers,
 Nick.
 
 -- 
 Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 426 updated (with more than you ever wanted to know about version schemes)

[Ronald Oussoren]

On 12 Feb, 2013, at 14:46, Daniel Holth dho...@gmail.com wrote:
 
 
 I still think it makes more sense to just download distribute and wheel when 
 you want to build one, but to each his own... if you need to create packages 
 for pypi without being able to install things from it, knock yourself out.

Why the hostility? It makes sense to have basic support in the stdlib.

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 426 updated (with more than you ever wanted to know about version schemes)

[Ronald Oussoren]

On 12 Feb, 2013, at 16:55, Daniel Holth dho...@gmail.com wrote:

 On Tue, Feb 12, 2013 at 10:04 AM, Nick Coghlan ncogh...@gmail.com wrote:
 
 On 13 Feb 2013 00:55, Ronald Oussoren ronaldousso...@mac.com wrote:
 
 
  On 12 Feb, 2013, at 14:46, Daniel Holth dho...@gmail.com wrote:
  
  
   I still think it makes more sense to just download distribute and wheel 
   when you want to build one, but to each his own... if you need to create 
   packages for pypi without being able to install things from it, knock 
   yourself out.
 
  Why the hostility? It makes sense to have basic support in the stdlib.
 
 I'm playing a much longer game than Daniel - when you're thinking in terms of 
 the next few months, then the PEPs are only significant because the pip folks 
 (sensibly, IMO) have made that a condition of merging wheel support (at which 
 point people will naturally start using them more, since they won't have to 
 seek them out, pip will cache the built wheels automatically)
 
 Core support only starts to matter once you're thinking in terms of *years*, 
 and the way beginners will encounter Python's distribution ecosystem in 3.4+ 
 :)
 
 +1 although I need to finish the PEP before then, the core support should 
 take years. This gives the community time to discover a good system.

Ehmm the plan that slowly gets clear is to split the packaging related 
functionality in at least two parts: building a wheel from an sdist, and 
installing the wheel. Teaching distutils to create wheels (including modern 
metadata), adding a wheel installer (see earlier mails by Nick) and describing 
a standard mechanism for invoking wheel creation (such as the sdist2wheel.py 
script in a earlier mail by Nick) should take years to get right.   

When those parts are present it will be a lot easier to experiment with other 
toolsets for building wheels, and getting that right may wel take a long time 
and doesn't have to be shoehorned into python's release schedule.

BTW. the discusion on how to secure package distribution might affect the wheel 
format, in particular the digital signature part. I wouldn't push for 
acceptence of the wheel PEP before that discussion settles down a bit.

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 426, round 733 ;)

[Ronald Oussoren]

On 4 Feb, 2013, at 17:00, a.cava...@cavallinux.eu wrote:

 I agree *completely* with Philippe here.
 
 If a version standard will be enforced what's the point of making it more
 complicated that a sequential number or something along x.y.z? In the end 
 that's
 what the version number is.

Because the version number is just more complicated? The details have been
discussed at length (and then some more) on this list, but just consider the 
common
scheme of alpha, beta and release-candidate releases. 

Without special support you'd end up with a version 1.0 being older than 1.0rc1.

Ronald

 
 
 
 
 On Mon 04/02/13 16:31, Philippe Ombredanne pombreda...@nexb.com wrote:
 On Mon, Feb 4, 2013 at 2:10 PM, Nick Coghlan ncoghlan@g
 mail.com wrote:
 As usual, PEP inline below and on the web
 at
 http://www.python.org/dev/peps/pep-0426/
 Version scheme
 ==
 Version numbers must comply with the following
 scheme::
N.N[.N]+[{a|b|c|rc}N][.postN][.devN]
 
 Version numbers which do not comply with this scheme
 are an error. Projects
 which wish to use non-compliant version numbers must
 restrict themselves
 
 IMHO a version (or eventually its dot-separated segments with
 precedence from left to right) should increase when sorted
 lexicographically so it is never ambiguous for a human reading a list
 
 Are we trying to make a version -- which is an engineering must --
 into something that  has also some semantics about the level of
 completion of a project or some marketing alert on the level of
 maturity of a software release? Could we stay instead in the realm of
 engineering?
 
 I think that trying to inject things like alpha, beta, post,  dev,
 release candidates and the likes in this is trying to bake in too many
 things that are eventually just the practices of some  projects and
 should not be the frozen practice baked in a PEP.  Instead, this
 should be left to project authors to define their own scheme as long
 as it sorts lexicographically (eventually by segments, with precedence
 from left to right).
 
 -- 
 Philippe Ombredanne
 
 +1 650 799 0949 | pombredanne@n
 exB.com
 DejaCode Enterprise at http://www.dejacode.com
 nexB Inc. at http://www.nexb.com
 ___
 Distutils-SIG maillist  -  Distutils-SIG@python.org
 http://mail.python.org/mailman/listinfo/distutils-sig
 
 
 
 
 ___
 Distutils-SIG maillist  -  Distutils-SIG@python.org
 http://mail.python.org/mailman/listinfo/distutils-sig

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] distlib and data files = resources ?

[Ronald Oussoren]

On 19 Nov, 2012, at 20:26, PJ Eby p...@telecommunity.com wrote:
 
 
 Data files should never be installed to package directories.  But I'm not 
 aware of any good reason why resource files should ever be installed anywhere 
 *else*.

To be (too) snarky: because the FHS says so. 

Less snarky, Linux distributors try to keep simular files together (for example 
storing all gettext translations together in /usr/share/locale).  To play nice 
in such an enviroment Python packages would have to install resource files 
outside of the python package and into the FHS specified directory structure.   
That again would mean there needs to be an API to access such files because it 
is not always necessary to store resources separate from their python package 
and because the FHS linux specific.

Storing resource files outside of packages directories seems to be only 
relevant for software that's packaged by a Linux distribution, not for python 
packages installed into a virtualenv or on other platforms.

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] [Catalog-sig] accept the wheel PEPs 425, 426, 427

[Ronald Oussoren]

On 13 Nov, 2012, at 16:00, Daniel Holth dho...@gmail.com wrote:
 
 I want to remove distutils from the standard library.

Why? Distutils may not be perfect, but is usable for basic packages. It could 
even be enhanced to support these peps and be even more useable, although 
patches for that would run into the self-imposed freeze of distutils 
development.

Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Catalog-sig] [Python-Dev] accept the wheel PEPs 425, 426, 427

[Ronald Oussoren]

On 13 Nov, 2012, at 17:21, Antoine Pitrou solip...@pitrou.net wrote:

 Le Tue, 13 Nov 2012 16:10:30 +0100,
 Ronald Oussoren ronaldousso...@mac.com a écrit :
 
 On 13 Nov, 2012, at 16:00, Daniel Holth dho...@gmail.com wrote:
 
 I want to remove distutils from the standard library.
 
 Why? Distutils may not be perfect, but is usable for basic packages.
 It could even be enhanced to support these peps and be even more
 useable, although patches for that would run into the self-imposed
 freeze of distutils development.
 
 It wouldn't be totally unreasonable to start a project named
 distutils2 to build the next-generation distutils library.
 
 Oops :-)

Or carefully enhance distutils itself.  Rewriting distutils in one go seems
to be too ambitious with the limited resources available to do so. 

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] accept the wheel PEPs 425, 426, 427

[Ronald Oussoren]

On 18 Oct, 2012, at 19:29, Daniel Holth dho...@gmail.com wrote:

 I'd like to submit the Wheel PEPs 425 (filename metadata), 426
 (Metadata 1.3), and 427 (wheel itself) for acceptance. The format has
 been stable since May and we are preparing a patch to support it in
 pip, but we need to earn consensus before including it in the most
 widely used installer.

PEP 427:

* The installation section mentions that .py files should be compiled to 
.pyc/.pyo files, and that Uninstallers should be smart enough to remove .pyc 
even if it is not mentioned in RECORD.. 

   Wouldn't it be better to add the compiled files to the RECORD file? That 
would break the digital signature, but I'm not sure if verifying the signature 
post-installation is useful (or if it's even
   intended to work). 

* Why is urlsafe_b64encode_nopad used to encode the hash in the record file, 
instead of the normal hex encoding that's directly supported by the hash module 
and system tools?

* The way to specify the required public key in package requirements in ugly 
(it looks like an abuse of setuptools' extras mechanism). Is there really no 
nicer way to specify this?

* As was noted before there is no threat model for the signature feature, which 
makes it hard to evaluate if the feature.  In particular, what is the advantage 
of this over PGP signatures of wheels? (PyPI already supports detached 
signatures, and such signatures are used more widely in the OSS world)

* RECORD.p7s is not described at all. I'm assuming this is intented to be a 
X.509 signature of RECORD in pkcs7 format. Why PKCS7 and not PEM? The latter 
seems to be easier to work with.

Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] wheel file signatures scheme

[Ronald Oussoren]

On 22 Aug, 2012, at 4:52, Daniel Holth dho...@gmail.com wrote:

 I've made what I think is exciting progress on the digital signatures
 design for wheel (updated built/binary packages for Python; intended
 to replace egg). The insight is that we can overload the extras
 syntax as a convenient way to mention the public key we expect:
 
 package[extra, ed25519=ouBJlTJJ4SJXoy8Bi1KRlewWLU6JW7HUXTgvU1YRuiA]

Why this hack instead of providing explict syntax for this?

Also the format doesn't seem to have any way to verify the validity of the 
signing key,
the documentation even says that key distribution is out of scope for this 
spec. That's
odd for feature that's intended to add security. 

Why did you decide to use JSON Web Signatures instead of PGP signatures, or even
X.509 signatures? With the latter two the key distribution problem is already 
solved, and
PGP signatures are used a lot in the opensource world.

Ronald

 
 http://wheel.readthedocs.org/en/latest/index.html#signed-wheel-files
 ___
 Distutils-SIG maillist  -  Distutils-SIG@python.org
 http://mail.python.org/mailman/listinfo/distutils-sig



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] RFC: Binary Distribution Format for distutils2/packaging

[Ronald Oussoren]

On 14 Mar, 2012, at 18:42, Chris Barker wrote:
 
 
 Universal binaries (i.e. more than one architecture in one binary)
 have never been properly supported by binary eggs/setuptools. I think
 it may be as simple as the naming convention -- the binary would be
 named according to the machine it was built on (i.e. PPC) but when you
 tried to install in on another machine, setuptools would look for one
 called, i.e. x86 and not find it. There may be some other issues,
 too, but in any case,  we need to make sure the naming convention
 handles the multi-architecture case as well.

Could you eleborate on that? Distutils supports fat binaries just fine, and 
AFAIK setuptools and distribute inherit that support. There were some problems 
in earlier python versions, especially in the Apple provided installs in OSX 
10.4, but those issues have been resolved years ago.

Ronald

smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] pythonv: let's also make sure the standard Python install includes an isolated python

[Ronald Oussoren]

On 17 Mar, 2011, at 21:33, Barry Warsaw wrote:

 Sounds good to me.
 
 On Mar 17, 2011, at 07:53 PM, Carl Meyer wrote:
 
 * executable-name has the extension stripped on Windows, but not
 otherwise.
 
 It should probably also have the extension stripped on OS X too.

There is no extension on OSX. There is one in the build tree, but that's 
because the default filesystem on OSX is case insensive and hence the name 
'python' is already taken during a build.

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Distributes from pkg_resources.resource_string returns bytes under Python 3

[Ronald Oussoren]

On 13 Mar, 2011, at 9:52, Lennart Regebro wrote:

 Distributes from pkg_resources.resource_string returns bytes under
 Python 3, which is pretty surprising. :-)

That's exactly what the documentation says it will return (the file is opened 
in binary mode).  This is just one of many legacy APIs in Python libraries 
where the naming is pretty lax w.r.t. bytes and text.

 Should we fix this? That would mean that we need to introduce a
 pkg_resources.resource_bytes that returns bytes under Python 3 and str
 (again) in Python 2. And probably we also need a
 pkg_resources.resource_unicode that returns Unicode under Python 2 and
 str under Python 3...

I wouldn't change this at this point in time, it is pretty likely that the 
change would break existing code. Making sure that distutils2 has the correct 
naming for their corresponding API would be fine though.

Ronald

 
 
 Opinions on that?
 ___
 Distutils-SIG maillist  -  Distutils-SIG@python.org
 http://mail.python.org/mailman/listinfo/distutils-sig

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] distutils OS X universal binaries

[Ronald Oussoren]

On 10 Feb, 2011, at 13:36, Prashant Saxena wrote:

 Hi,
 
 I am new on Mac platform. I need to port an application from linux to mac. 
 There 
 are couple of custom c-extensions which I am building using cython on linux 
 using distutils. It's pretty simple and straight forward there.
 There are couple of things I need to know on Mac platform.
 
 Q1. How do I create custom c-extension (cython  distutils) on mac so that 
 they 
 should run on both architecture (ppc  Intel)?

If cython doesn't do anything special: just use a universal build of python to 
build the extension.

 
 Q2. I am on 32 bit platform and I have to make sure that deployed package 
 (py2app) should execute on 64-bit architecture also.

What is your question?

32-bit binaries will work just fine on system that is 64-bit capable. If you 
cannot test 64-bit binaries on your machine I'd only ship a 32-bit binary.

 
 Q3. Mine is Leopard 10.5.5(intel) and default python is 2.5.1 but my app 
 requires 2.6.6. How do I build python 2.6.6 in universal binary format?

There are two options:

1) Download the 2.6.6 installer from the python.org website

2) Built it yourself using 'configure --enable-universalsdk=/  
MACOSX_DEPLOYMENT_TARGET=10.5'
 (replace 10.5 by the earliest OSX version you want to support)

Ronald


 
 Overall idea is to deploy a package on mac 10.5.5 leopard(intel 32 bit) using 
 py2app in such a way so that it's compatible for both architecture (ppc  
 intel) 
 and both platforms(32-bit  64-bit)
 
 Cheers
 
 Prashant
 
 
 ___
 Distutils-SIG maillist  -  Distutils-SIG@python.org
 http://mail.python.org/mailman/listinfo/distutils-sig

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] distutils rebuilds EVERYTHING

[Ronald Oussoren]

On 8 Oct, 2010, at 8:59, Tarek Ziadé wrote:

 Hello,
 
 On Fri, Oct 8, 2010 at 1:40 AM, Charles Doutriaux doutria...@llnl.gov wrote:
  Hello,
 
 It seems that since python 2.7 the default behaviour of distutils changed.
 
 I have a project with a LOT of .c files used in an extension
 
 It used to be that when rebuilding (python setup.py build install) it
 would only rebuild the .o for the C files that actually changed.
 
 Since Python 2.7 this seems to have changed. It knows rebuilds EVERY C
 file in my extension no matter if they changed or not...
 
 I can see why the Default behaviour of distutils has been changed, it
 is safer, but is there a way to revert to the old way where only the
 changed files are recompiled.
 
 I'm running Python 2.7 (framework) on a Mac 64bit. (10.6)
 
 Thank you very much for any help with this,
 
 This is a regression I introduced to fix the fact that the .so are not
 rebuilt we you do subtle changes in your project.
 
 We ended up reverting this, and the changes will be done in
 Distutils2. see: http://bugs.python.org/issue8688 for the details.

This issue is more relevant: http://bugs.python.org/issue7894

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Question about testing

[Ronald Oussoren]

Hi,

What's the best way to bundle tests into a test archive for a distribution 
using distribute or setuptools?

My requirements:
* Tests should not be installed because the testsuite is larger than the actual 
package and not generally usefull
* python setup.py develop should also not add the testsuite to sys.path
* Some tests use extensions that are part of the testsuite
* It should be possible to use python setup.py test to run the testsuite 

My current directory layout is simular to the CPython tree:

* toplevel setup.py script
* Lib/ contains modules/packages
* Modules/ contains extension source
* TestSuite/ contains the unittests

The setup.py file contains a package_dir declaration:

package_dir = { '': 'Lib', 'TestSuite': 'TestSuite' },

I also have a custom build_ext command to ensure that extensions in the 
testsuite get copied there during the build phase
(as if the inplace option is selected for just those extensions).

This works but feels rather fragile, and I wonder if there is a cleaner 
solution for meeting my requirements.

(I'm using plain unittest for the tests and will probably switch to unittest2 
in the near future)

Ronald




smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] pip vs easy_install vs distutils2

[Ronald Oussoren]

On 31 May, 2010, at 19:10, Tarek Ziadé wrote:

 On Mon, May 31, 2010 at 7:04 PM, Lennart Regebro rege...@gmail.com wrote:
 On Mon, May 31, 2010 at 19:02, Carl Meyer c...@oddbird.net wrote:
 Nope, pip's used --record on installation for years, and the above has
 been true since the moment uninstall landed in pip. There are enough
 different ways things can get installed that it's not surprising that
 some discussions may have been confused ;-)
 
 That may be it. Forcing --record in Python 3.2 would be a step forward then? 
 :-)
 
 You mean in the current distutils ? Because distutils2 will have the
 PEP 376 implementation,
 where we create a RECORD file for each installed project in its dist-info/

I haven't carefully read the entire discussion, but do you mean that distutils 
won't follow PEP 376 during installation?

Ronald



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] pip vs easy_install vs distutils2

[Ronald Oussoren]

On 3 Jun, 2010, at 10:10, Tarek Ziadé wrote:

 On Thu, Jun 3, 2010 at 10:02 AM, Ronald Oussoren ronaldousso...@mac.com 
 wrote:
 
 On 3 Jun, 2010, at 9:50, Tarek Ziadé wrote:
 
 On Thu, Jun 3, 2010 at 9:15 AM, Ronald Oussoren ronaldousso...@mac.com 
 wrote:
 
 On 31 May, 2010, at 19:10, Tarek Ziadé wrote:
 
 On Mon, May 31, 2010 at 7:04 PM, Lennart Regebro rege...@gmail.com 
 wrote:
 On Mon, May 31, 2010 at 19:02, Carl Meyer c...@oddbird.net wrote:
 Nope, pip's used --record on installation for years, and the above has
 been true since the moment uninstall landed in pip. There are enough
 different ways things can get installed that it's not surprising that
 some discussions may have been confused ;-)
 
 That may be it. Forcing --record in Python 3.2 would be a step forward 
 then? :-)
 
 You mean in the current distutils ? Because distutils2 will have the
 PEP 376 implementation,
 where we create a RECORD file for each installed project in its dist-info/
 
 I haven't carefully read the entire discussion, but do you mean that 
 distutils won't follow PEP 376 during installation?
 
 If people use the old Distutils, (what I've called the current
 distutils), and trigger an installation via 'python setup.py install'.
 It'll use the existing code, so install it the 'old' way.
 
 I know that distutils in python 2. 6 and 3.1 won't support PEP 376, but why 
 won't distutils conform to PEP 376 in python 2.7 and 3.2?
 The RECORD file from PEP 376 would allow  manually installed package (e.g. 
 python setup.py install) to be further managed by a PEP 376 compliant 
 install tool.  This should be pretty easy to add if it isn't in already, 
 although 2.7rc1 is awfully close.
 
 Are you thinking about a full implicit switch to PEP 376, or an
 optional behavior ?

A full switch to PEP 376.  An option to enable/disable would just be confusing, 
any option increases the mental load of developers (and anyone that just wants 
to install a python package to use its functionality) and in this case I don't 
think adding an option would be worth that.

This would break current releases of tools that use the current egg-info 
information due to the different filesystem name (currently .egg-info, 
.dist-info in PEP 376), but those tools will have to be updated sometime 
anyway.

Ronald



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] pip vs easy_install vs distutils2

[Ronald Oussoren]

On 3 Jun, 2010, at 10:38, Tarek Ziadé wrote:

 On Thu, Jun 3, 2010 at 10:24 AM, Ronald Oussoren ronaldousso...@mac.com 
 wrote:
 [..]
 A full switch to PEP 376.  An option to enable/disable would just be 
 confusing, any option increases the mental load of developers (and anyone 
 that just wants to install a python package to use its functionality) and in 
 this case I don't think adding an option would be worth that.
 
 This would break current releases of tools that use the current egg-info 
 information due to the different filesystem name (currently .egg-info, 
 .dist-info in PEP 376), but those tools will have to be updated sometime 
 anyway.
 
 I don't see what would be the advantage, since we are adding in
 pkgutil APIs (PEP 376 implementation) the ability to read old formats
 as well, to make it easier to query what's installed. Plus, distutils
 was marked frozen to avoid instability (any change in distutils
 potentially breaks tools out there, even internal changes).

The RECORD file in PEP 376 enables scripted removal of a distribution, which 
cannot be done without PEP 376 unless the user that installed the distribution 
was careful enough to use either the --record option or a tool like pip. 

I agree that the fragility of distutils is a problem, which would making 
changing it this far along the 2.7 release process rather dangerous.

Ronald



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] pip vs easy_install vs distutils2

[Ronald Oussoren]

On 3 Jun, 2010, at 9:50, Tarek Ziadé wrote:

 On Thu, Jun 3, 2010 at 9:15 AM, Ronald Oussoren ronaldousso...@mac.com 
 wrote:
 
 On 31 May, 2010, at 19:10, Tarek Ziadé wrote:
 
 On Mon, May 31, 2010 at 7:04 PM, Lennart Regebro rege...@gmail.com wrote:
 On Mon, May 31, 2010 at 19:02, Carl Meyer c...@oddbird.net wrote:
 Nope, pip's used --record on installation for years, and the above has
 been true since the moment uninstall landed in pip. There are enough
 different ways things can get installed that it's not surprising that
 some discussions may have been confused ;-)
 
 That may be it. Forcing --record in Python 3.2 would be a step forward 
 then? :-)
 
 You mean in the current distutils ? Because distutils2 will have the
 PEP 376 implementation,
 where we create a RECORD file for each installed project in its dist-info/
 
 I haven't carefully read the entire discussion, but do you mean that 
 distutils won't follow PEP 376 during installation?
 
 If people use the old Distutils, (what I've called the current
 distutils), and trigger an installation via 'python setup.py install'.
 It'll use the existing code, so install it the 'old' way.

I know that distutils in python 2. 6 and 3.1 won't support PEP 376, but why 
won't distutils conform to PEP 376 in python 2.7 and 3.2?

The RECORD file from PEP 376 would allow  manually installed package (e.g. 
python setup.py install) to be further managed by a PEP 376 compliant install 
tool.  This should be pretty easy to add if it isn't in already, although 
2.7rc1 is awfully close.

Ronald

smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Distutils roadmap of refactoring for 2.7

[Ronald Oussoren]

On 3 Feb, 2010, at 0:25, David Lyon wrote:

 Chris wrote:
 On Sat, Jan 23, 2010 at 7:11 PM, P.J. Eby p...@telecommunity.com
 wrote:
 Notice that other changes are coming up once
 the series of PEP we worked on are accepted (which should be before
 Pycon - at least for 345 and 386)
 
 ok - so are you going to sort out the confusing (i386 == mac)
 and (x86 == pc) or (i586 == pc) or (i686 == pc) ?
 
 in PEP-345? or leave it as downright confusing?
 
 David,
 
 If you're going to hijack a thread to bang on about your pet peeve,
 please at least attribute the mail you're quote to the right person...
 
 ok. I agree.
 
 But it isn't a pet peave..
 
 Introducing 'code' into the metadata will mean that PKG_INFO
 files will become hated just as much by developers as what
 .PTH have been.
 
 Let me remind you what happened, .PTH files were supposed to
 be data files (like metadata files).
 
 Then a bright spark said, 'Let them contain code..'. And then
 they did..
 
 Then another bright spark said 'I shalt introduce code that
 nobody else cannot understand or change'.
 
 Then regular users found the whole thing such a nightmare
 they came to loath the entire concept of a .PTH file.
 
 So now, the exact same dreaded mechanism, is going to
 be introduced into PKG_INFO files and I can't see why
 it won't be also hijacked to make system administrators
 lives a misery.
 
 Giving a normal system administrator the job of knowing
 that an 'i386' is actually a mac is going to drive them
 bananas figuring out why the package isn't installing
 properly on a pc.

Could you please explain what the problem is you're complaining about instead 
of assuming everyone knows the details by heart? The only reference I saw to 
'i386' in PEP 345 is in the section about Environment markers and that only 
refers to the CPU architecture in a test for CPU architecture (platform.machine 
== 'i386') which seems reasonable to me.  

You can use sys.platform to detect various OS types.

BTW. I guess you mean Windows where you write pc.

 
 Nine out of ten system administrators would guess that
 'i386' is actually a pc. It should be but it isn't.

'i386' is the commonly used abbreviation for an instruction set also known as 
'x86' or 'ia32' and doesn't suggest any particular OS to me, all of Windows, 
Linux and MacOSX use or can use 'i386' capable CPUs (to just name 3 commonly 
used operating systems).

Ronald



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Platform naming standardization

[Ronald Oussoren]

On 14 Jan, 2010, at 10:36, M.-A. Lemburg wrote:

 Ronald Oussoren wrote:
 
 On 13 Jan, 2010, at 23:44, M.-A. Lemburg wrote:
 
 Ronald Oussoren wrote:
 
 On 13 Jan, 2010, at 18:41, M.-A. Lemburg wrote:
 
 Ronald Oussoren wrote:
 
 On Wednesday, January 13, 2010, at 04:15PM, M.-A. Lemburg 
 m...@egenix.com wrote:
 
 
 2) On OS X, the modification to the value returned by
 pkg_resources.get_platform() isn't correct for fat version of Python
 2.5, as detailed in setuptools issue 19.  To solve that, we're using 
 the
 patch I submitted to the issue (with a couple recent changes).
 
 I think that using fat in the version string is wrong for
 Mac OS X, since there are many ways to build fat binaries.
 
 Instead, the version string should include the details of
 all included builds, ie. 'x86', 'x64', 'ppc', 'ppc64'.
 
 Maybe in the long run, but for now fat has a well-defined meaning for 
 distutils: fat == ppc + x86_64. There is also a number of other 
 variants, as described in the documentation for distutils.
 
 I think you meant: fat == ppc + i386.
 
 Thats right.
 
 However, it's also possible to build binaries with ppc, i386 and
 x86_64 - as are shipped with Mac OS X 10.6, so fat is not really
 well-defined and could lead to trying to install 32-bit software for
 a 64-bit build of Python.
 
 fat is well-defined for distutils, see the definition of get_platform at 
 http://docs.python.org/distutils/apiref.html. 
 
 For distutils fat is always a universal binary with architectures i386 
 and ppc, with alternate names for other variants.
 
 Thanks for pointing that out, however, I don't think that creating
 aliases for combinations of various different architectures
 is a good idea.
 
 It's better to make the included architectures explicit and use
 this logic for all platforms, not just Mac OS X.
 
 I would probably have done that, knowing what I know now.   
 
 Hashing out the details on what combinations of architectures are valid 
 during installation will be fun though ;-). That is, if my python says its 
 machine is i386,x86_64 is it then acceptable to install an i386 binary, 
 an i386,x86_64 binary, and i386,ppc, x86_64 binary?
 
 The point is that even though your Python binary may say it's
 i386,x86_64, the version you run your application with
 will either be i386 or x86_64 (depending on the OS environment
 settings).
 
 Now let's say you're running the i386 version. As long as all
 installed components provide the i386 part you should be fine.
 In your example all components provide the i386 part, so all
 of them can be installed.

I don't agree, easy_install somepackage should install a component that 
supports at least all architectures supported by the Python binary.   Otherwise 
you might install a package and have problems later when you try to use it. 

An example of this is a recent 64-bit capable machine with older versions of 
Tkinter or wxPython:  on those systems python will run as a 64-bit binary by 
default, but you sometimes have to run python in 32-bit mode to be able to use 
Tkinter. It would be very annoying and possibly confusing when I install a 
library and end up not being able to use it sometimes.

I also regularly build standalone app bundles on one machine and run them on 
other machines, those should also included all components in all supported 
architectures. 

 
 With the aliases, this kind of detection is also possible, but
 only after mapping the aliases back to the combination of included
 architecture names.
 
 In a few years, we'll probably only see x86_64 binaries for
 Mac OS, but until then, package installers will have to
 be able to work out the problem of finding installable
 distribution files among the available ones.

I agree, at least until the next new thing comes along (such as something 
arm-based).

 
 BTW: With Python 2.6, if you build using the x86_64 version of
 Python, distutils will still use the macosx-10.5-i386
 platform identifier. Should I file a bug for this ?

That should be fixed in the repository, distutils assumed that the uname -m 
reflects the correct architecture and that is not true for a 
single-architecture build with default compiler flags on OSX 10.6.   

Ronald



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Platform naming standardization

[Ronald Oussoren]

 
On Wednesday, January 13, 2010, at 04:15PM, M.-A. Lemburg m...@egenix.com 
wrote:


 2) On OS X, the modification to the value returned by
 pkg_resources.get_platform() isn't correct for fat version of Python
 2.5, as detailed in setuptools issue 19.  To solve that, we're using the
 patch I submitted to the issue (with a couple recent changes).

I think that using fat in the version string is wrong for
Mac OS X, since there are many ways to build fat binaries.

Instead, the version string should include the details of
all included builds, ie. 'x86', 'x64', 'ppc', 'ppc64'.

Maybe in the long run, but for now fat has a well-defined meaning for 
distutils: fat == ppc + x86_64. There is also a number of other variants, as 
described in the documentation for distutils.

Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Platform naming standardization

[Ronald Oussoren]

On 13 Jan, 2010, at 18:41, M.-A. Lemburg wrote:

 Ronald Oussoren wrote:
 
 On Wednesday, January 13, 2010, at 04:15PM, M.-A. Lemburg 
 m...@egenix.com wrote:
 
 
 2) On OS X, the modification to the value returned by
 pkg_resources.get_platform() isn't correct for fat version of Python
 2.5, as detailed in setuptools issue 19.  To solve that, we're using the
 patch I submitted to the issue (with a couple recent changes).
 
 I think that using fat in the version string is wrong for
 Mac OS X, since there are many ways to build fat binaries.
 
 Instead, the version string should include the details of
 all included builds, ie. 'x86', 'x64', 'ppc', 'ppc64'.
 
 Maybe in the long run, but for now fat has a well-defined meaning for 
 distutils: fat == ppc + x86_64. There is also a number of other variants, as 
 described in the documentation for distutils.
 
 I think you meant: fat == ppc + i386.

Thats right.
 
 However, it's also possible to build binaries with ppc, i386 and
 x86_64 - as are shipped with Mac OS X 10.6, so fat is not really
 well-defined and could lead to trying to install 32-bit software for
 a 64-bit build of Python.

fat is well-defined for distutils, see the definition of get_platform at 
http://docs.python.org/distutils/apiref.html. 

For distutils fat is always a universal binary with architectures i386 and 
ppc, with alternate names for other variants.

 
 IMHO, it's better to list the actually supported architectures
 as e.g. darwin-i386-ppc or darwin-i386-ppc-x86_64.

That is not how distutils currently works. I also object to darwin as a 
prefix, the platform is named macosx. Darwin is the opensource unix variant 
used in OSX and that name shouldn't be interchangeably with macosx.  I'm also 
unhappy that sys.platform is darwin on OSX, but it's probably too late to 
change that.

Ronald
 
 -- 
 Marc-Andre Lemburg
 eGenix.com
 
 Professional Python Services directly from the Source  (#1, Jan 13 2010)
 Python/Zope Consulting and Support ...http://www.egenix.com/
 mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
 mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
 
 
 ::: Try our new mxODBC.Connect Python Database Interface for free ! 
 
 
   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Platform naming standardization

[Ronald Oussoren]

On 13 Jan, 2010, at 23:44, M.-A. Lemburg wrote:

 Ronald Oussoren wrote:
 
 On 13 Jan, 2010, at 18:41, M.-A. Lemburg wrote:
 
 Ronald Oussoren wrote:
 
 On Wednesday, January 13, 2010, at 04:15PM, M.-A. Lemburg 
 m...@egenix.com wrote:
 
 
 2) On OS X, the modification to the value returned by
 pkg_resources.get_platform() isn't correct for fat version of Python
 2.5, as detailed in setuptools issue 19.  To solve that, we're using the
 patch I submitted to the issue (with a couple recent changes).
 
 I think that using fat in the version string is wrong for
 Mac OS X, since there are many ways to build fat binaries.
 
 Instead, the version string should include the details of
 all included builds, ie. 'x86', 'x64', 'ppc', 'ppc64'.
 
 Maybe in the long run, but for now fat has a well-defined meaning for 
 distutils: fat == ppc + x86_64. There is also a number of other variants, 
 as described in the documentation for distutils.
 
 I think you meant: fat == ppc + i386.
 
 Thats right.
 
 However, it's also possible to build binaries with ppc, i386 and
 x86_64 - as are shipped with Mac OS X 10.6, so fat is not really
 well-defined and could lead to trying to install 32-bit software for
 a 64-bit build of Python.
 
 fat is well-defined for distutils, see the definition of get_platform at 
 http://docs.python.org/distutils/apiref.html. 
 
 For distutils fat is always a universal binary with architectures i386 and 
 ppc, with alternate names for other variants.
 
 Thanks for pointing that out, however, I don't think that creating
 aliases for combinations of various different architectures
 is a good idea.
 
 It's better to make the included architectures explicit and use
 this logic for all platforms, not just Mac OS X.

I would probably have done that, knowing what I know now.   

Hashing out the details on what combinations of architectures are valid during 
installation will be fun though ;-). That is, if my python says its machine is 
i386,x86_64 is it then acceptable to install an i386 binary, an 
i386,x86_64 binary, and i386,ppc, x86_64 binary?

Ronald

smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata

[Ronald Oussoren]

On 20 Oct, 2009, at 14:11, Chris Withers wrote:


Ronald Oussoren wrote:
2) Notice that simplejson gets installed and is a useful module,  
then use that in your own scripts


At this point, you specify your scripts' dependencies on simplejson.

Look at it another way:

You follow you process above, but after step 2, your development  
machine suffers a disk failure so you start from scratch on a new  
machine.


Are you surprised at this point that simplejson isn't installed when  
you set things up again, even though by this point you've decided  
not to use TurboGears?


Code should always specify their dependencies, not doing so will  
cause you problems down the line...


Sure, and I do that for for any real code. But do you really expect  
that users are careful to specify dependencies for each and every  
script they write.  That may be true for development shops, but I  
don't expect that casual users are so careful.


But this discussion is moot anyway, the proposed metadata is meant to  
help packaging tools to track which packages seem to be orphaned and  
for which they can ask the user if those packages should be removed  
from the system, instead of always removing seemingly orphaned packages.


Ronald

smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata

[Ronald Oussoren]

On 9 Oct, 2009, at 15:45, Carl Meyer wrote:


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


Alex wrote:
REQUESTED is fine, but I don't understand how the arguments apply,  
given
that I'm not proposing to record information like _which_ package  
it was

a dependency of. The same single bit (literally) of information is
tracked either way, it's just a question of whether the presence or
absence of a file signifies that bit.


And what if the package is a dependency for multiple packages?
Let's say we have packages A and B which are installed separately, in
that order, and both depend on C.
C gets installed with information that it was required by A. Now if  
A is

uninstalled, won't C also get uninstalled?


No. We are NOT talking about recording the full dependency graph in
package metadata. It is, as has always been the case, up to an  
uninstall

tool to calculate the depgraph based on actual installed packages at
runtime. A package is orphaned iff it was not REQUESTED by the user
AND it is no longer depended on by other installed packages.


What about packages that are installed as a dependency of some other  
package and then used in user scripts without an explict depency on  
them?


That is, I install SuperWebFramework==1.0 which happens to depend on  
peak-rules. I later start using peak-rules in my own simple scripts  
(without a setup.py or other explicit dependency information), and yet  
later decide to uninstall SuperWebFramework.  If I understand the  
proposal correctly the uninstallation of SuperWebFrameWork would  
break my scripts.


I also wonder how this interacts with system package managers.  
The .egg-info directory-structure and PEP377 where explicitly  
structured to allow (for example) RPM to create the egg-info directory  
without having to know about Python's package repository (which would  
be needed if that repository were a single-file database instead of  
the collection of egg-info directories).


Ronald


Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFKzz531j/fhc23WEARAlPfAJ93YHsbpzp5kF7hR0f98aSMOae6MwCgrP52
RZs5FU3h0U0eUBN2vgqm2HA=
=jyi8
-END PGP SIGNATURE-
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig




smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata

[Ronald Oussoren]

On 11 Oct, 2009, at 16:27, Lennart Regebro wrote:


2009/10/11 Ronald Oussoren ronaldousso...@mac.com:
What about packages that are installed as a dependency of some  
other package

and then used in user scripts without an explict depency on them?

That is, I install SuperWebFramework==1.0 which happens to depend  
on
peak-rules. I later start using peak-rules in my own simple scripts  
(without
a setup.py or other explicit dependency information), and yet later  
decide
to uninstall SuperWebFramework.  If I understand the proposal  
correctly

the uninstallation of SuperWebFrameWork would break my scripts.


Yes of course.


IMHO that is a bad experience for the user, because it is very  
unintuitive that I have to explicitly install something that's already  
installed to ensure that it doesn't go away in the future. This might  
work in a serious development environment, but can end up to be very  
annoying and confusing for casual users.


Ronald

smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] tracking requested vs dependency installs in PEP 376 metadata

[Ronald Oussoren]

On 11 Oct, 2009, at 20:58, Lennart Regebro wrote:


2009/10/11 Ronald Oussoren ronaldousso...@mac.com:
That is, I install SuperWebFramework==1.0 which happens to  
depend on

peak-rules. I later start using peak-rules in my own simple scripts
(without
a setup.py or other explicit dependency information), and yet later
decide
to uninstall SuperWebFramework.  If I understand the proposal  
correctly

the uninstallation of SuperWebFrameWork would break my scripts.


Yes of course.


IMHO that is a bad experience for the user, because it is very  
unintuitive
that I have to explicitly install something that's already  
installed to

ensure that it doesn't go away in the future.


But this is how all installs/uninstalls work and MUST work. If you
uninstall the software that you use, then you can't use it. It's a
simple necessity of life.


Sigh.

1) Install TurboGears, this installs simplejson as one of its  
dependencies


2) Notice that simplejson gets installed and is a useful module, then  
use that in your own scripts


3) Uninstall TurboGears

With proposal of tracking which packages are installed as a dependency  
of other packages and automaticly uninstalling them when the package  
that depends on them gets uninstalled step 3 would break my scripts  
unless I explictly tell the installer that I really want to use  
simplejson in step 2.


That's what I meant by I have to explictly install something that is  
already installed to ensure that it doesn't go away in the future,  
and this is not a useful behaviour.





This might work in a serious
development environment, but can end up to be very annoying and  
confusing

for casual users.


Can you come up with something better? How will the packaging registry
know that you want a particular package installed if you don't tell
it?


The current proposal without tracking if I explicitly installed a  
package works fine and matches how system package managers work.


This does make it harder to uninstall something like TurboGears that  
installs lots of dependencies, but it should be possible to write a  
tool that makes that easier. One way to do this in a GUI-ish manner is  
to have a show dependencies checkbox when you uninstall a package,  
this would then show all dependencies and allow easy selection of them  
for deinstallation.


This gets you simular functionality as tracking explictly requested  
installs without needing additional metadata.


Ronald



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

[Ronald Oussoren]

On 5 Oct, 2009, at 13:54, Tarek Ziadé wrote:

On Mon, Oct 5, 2009 at 1:30 PM, Ronald Oussoren ronaldousso...@mac.com 
 wrote:

Nobody will adopt it until they are forced to. This unfortunate bug
means people are forced to quicker than expected. I don't think  
that's

an actual problem.


This is a problem, it means 2.6.3 is not a simple drop-in  
replacement for 2.6.2 but requires the replacement of another  
component as well.  That can be a problem in organizations with  
strict configuration management where you cannot install new  
software without going to lots of red tape (and that might involve  
lawyers when you install a new package instead of upgrading an  
existing one).


What is your solution ? Setuptools is not part of the standard library
and monkey-patch Distutils.
Its development has been discontinued for over a year.


I don't have a real solution, beyond documentation.  IMHO the issue  
should have been documented in the 2.6.3 release notes, which assumes  
that the issue was know before the release.




Everytime Distutils is changed for anything, wether it's a bug fix or
not, Setuptools can be broken.

Now I realize that some folks wants Distutils to be aware of that and
be backward-compatible with
Setuptools monkey-patches ?


I'd prefer if distutils didn't break setuptools in the 2.6 branch,  
breaking it in 2.7 is fine because setuptools hooks into distutils at  
a very low level and is therefore sensitive too changes in  
implementation details of distutils.


And if breaking setuptools in the 2.6 branch is unavoidable this  
should be noted in Python's release notes. Whether we like it or not  
setuptools, and easy_install in particular, is used a lot and not only  
by power users.


Anyway, the issue is less relevant at the moment the NEWS file in the  
2.6 branch says that a fix for the setuptools breakage will be in  
2.6.4 when that's released.




Like for the svn 1.6 compat problem we had earlier this year,
this problem is a few line changes in Setuptools, it's an 1 hour work.
If your company upgrades to Python 2.6.3 it can also upgrade to an
hypothetical Setuptools 0.6c..10 ?


Sure, but 0.6c10 is only hypothetical at the moment.

BTW. This doesn't mean that I don't appreciate distribute, I'm  
switching to that in the near future because there is real progress in  
distribute and it has support for py3k.


Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

[Ronald Oussoren]

 
On Monday, 05 October, 2009, at 11:53AM, Lennart Regebro rege...@gmail.com 
wrote:
2009/10/5 Jeff Rush j...@taupro.com:
 Very unfortunate, as in, it should NOT have happened.  And *especially*
 without any announcement on python.org or mention on the
 python-committers list of something this major.

Well this major... It's a bug fix that breaks a setuptools monkey-patch.
But yes, it was discovered before release, and maybe it should have
been discussed, I'm not on python-dev anymore.

I agree with Jeff that this shouldn't have happened, or should at the very least
have been documented in the release notes for 2.6.3. I know of several users
of Python that have been bitten by this (they installed 2.6.3 and now 
easy_install
doesn't work anymore for them).  

For beginners this issue is a showstopper that they cannot resolve without help.



 Distribute is very new and there are many folk who will not be adopting
 it until it has been out for quite some time.

Nobody will adopt it until they are forced to. This unfortunate bug
means people are forced to quicker than expected. I don't think that's
an actual problem.

This is a problem, it means 2.6.3 is not a simple drop-in replacement for 2.6.2 
but requires the replacement of another component as well.  That can be a 
problem in organizations with strict configuration management where you cannot 
install new software without going to lots of red tape (and that might involve 
lawyers when you install a new package instead of upgrading an existing one).

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

[Ronald Oussoren]

On 5 Oct, 2009, at 16:25, K. Richard Pixley wrote:


Ronald Oussoren wrote:
For beginners this issue is a showstopper that they cannot resolve  
without help.


I'm a relative beginner to distutils/setuptools/distribute, but a  
long time configuration/build/packaging professional.  You're  
mistaken if you think that any of these technologies are suitable  
for beginners.  The state of python package distribution resembles  
the state of linux packages circa 1995, except that it isn't very  
well documented at all.


I didn't say that distutils is suitable for beginners, but beginners  
to use them and are confused when they stop working.


Easy_install certainly isn't, and never has been.  My first  
questions about easy_install, (how do I get a list of installed  
packages?  How do I upgrade all installed packages?  How do I delete  
a package using easy_install?) are all unanswerable.


All of these are not yet implemented. PJE has mentioned that he wants  
to work on this in for setuptools 0.7, but development of setuptools  
has completely stalled.   The distribute package was created because  
of this, and development is moving forward again.


Ronald


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

[Ronald Oussoren]

On 5 Oct, 2009, at 16:37, K. Richard Pixley wrote:


Ronald Oussoren wrote:
This is a problem, it means 2.6.3 is not a simple drop-in  
replacement for 2.6.2 but requires the replacement of another  
component as well.  That can be a problem in organizations with  
strict configuration management where you cannot install new  
software without going to lots of red tape (and that might involve  
lawyers when you install a new package instead of upgrading an  
existing one).
This would be a problem if distribute were in general release.  It's  
not.  It's clearly a development branch which is intended to move  
quickly.  People using distribute are taking development, pre-alpha  
kinds of risk and that has been made pretty clear already.


AFAIK distribute 0.6 is a stable release, basicly setuptools 0.6c9 +  
bugfixes + py3k support.


Installing distribute is therefore not problematic for most people, if  
they know that the project exists.  The fact that distribute is a  
seperate project from setuptools can be a problem for people:  
installing a bugfix release for a software product that we're already  
using at work is significantly easier than introducing a new software  
product.


Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] setup.cfg new format proposal

[Ronald Oussoren]

On 11 Sep, 2009, at 17:14, Tarek Ziadé wrote:



- os_sysname
- os_nodename
- os_release
- os_version
- os_machine


Os.uname() does have a usability issue: the various values don't have  
a clear meaning, nor even a clear mapping to marketing numbers.


Examples:
* os_sysname is darwin on a macosx machine, not something that would  
be clear to a novice OSX developer
* os_release is not necessarily the same as the version of the OS but  
returns the version of the kernel. Examples of operating systems where  
the kernel version number and OS version number aren't the same:  
Solaris, MacOSX, and of course Linux


os_nodename shouldn't be used in setup.cfg at all, I don't see a good  
reason to include the hostname in check conditions in setup.cfg.  
There's always the full setup.py if you want complicated conditions.


Ronald

smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Static metadata using setup.cfg

[Ronald Oussoren]

On 7 Sep, 2009, at 14:15, Chris Withers wrote:

I'd prefer setup.cfg to be totally static. If there are complicated  
if/then/else's needed, they should be in setup.py.


I have a number of packages where the only logic on setup.py is set  
flags based on the python version or OS. Examples:
* depend on pysqlite in old versions of python where sqlite wasn't in  
the stdlib
* select different compiler flags based for Linux vs. Windows for a C  
extension


It would be nice if those could be expressed in a setup.cfg(.in) file,  
because that way to can introspect these packages without having to  
execute a setup.py file.




As soon as this file becomes dynamic, we're back in the situation  
where you can't tell what a package requires without installing it,  
which is something I'd really like to not have to do.


I agree that the file shouldn't be too dynamic, we don't need a full  
programming language in setup.cfg because we already have setup.py  
files.


The proof of concept already severely limits what constructs the  
template can use, and those should both be enough for most python  
packages and simple enough to allow easy introspection.


Ronald



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Static metadata using setup.cfg

[Ronald Oussoren]

On 8 Sep, 2009, at 10:18, Chris Withers wrote:


Ronald Oussoren wrote:
I have a number of packages where the only logic on setup.py is set  
flags based on the python version or OS. Examples:
* depend on pysqlite in old versions of python where sqlite wasn't  
in the stdlib


mini rant
If Python had a packaging system *and* used it for the standard  
library, then things like this wouldn't be a problem...
The setup.cfg could just say requires sqlite greater than version  
x.y.z, and if it was in the standard library, it would be used  
unless a newer version was needed. It would also mean it would be  
possible to release bug fix versions of the standard library  
packages without having to roll a whole python release.


Yes, it is sad that only wsgiref is included in the stdlib with an egg- 
info file.


Better yet, since python should be a package as far as the  
packaging system is concerned, library versions can just say what  
versions of python they work with.


I like this idea, adding this to distutils and setuptools should be  
easy enough as well.



/rant

It would be nice if those could be expressed in a setup.cfg(.in)  
file, because that way to can introspect these packages without  
having to execute a setup.py file.


No, you just have to execute setup.cfg instead :-(


Yes, but I'd be much happier about exanding a template than executing  
some arbitrary python code.  You obviously have to trust the author  
when you actually install and use the package, but it would be nice if  
one could extract useful information from a package before doing that.


For me, setup.cfg should contain static stuff like name,  
description, url and dependencies. Anything else that varies like  
this can and should be in setup.py. We already have a fantastic  
scripting language, why coem up with another one?


Because static stuff is not always completely static.

Ronald

smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] buildout's tests fail for me

[Ronald Oussoren]

On 1 Sep, 2009, at 16:21, Jim Fulton wrote:

On Tue, Sep 1, 2009 at 9:19 AM, Reinout van  
Reesrein...@vanrees.org wrote:

On 2009-09-01, Jim Fulton j...@zope.com wrote:
On Tue, Sep 1, 2009 at 3:42 AM, Reinout van  
Reesrein...@vanrees.org wrote:

- Is this something I really should fix in my own setup?

- Is there a known fix that can be applied to the test setup?


Yes. Use a clean Python built from source with nothing in site- 
packages.


Ok, that helped partially.  But then I got an error like:

 distutils.errors.DistutilsPlatformError: $MACOSX_DEPLOYMENT_TARGET  
mismatch:

 now 10.3 but 10.5 during configure


shrug Mac OS X weirdity.


What kind of Python installation are you using? System python, binary  
installer from Python.org, ...

Which version of Python are you using?
Do you have MACOSX_DEPLOYMENT_TARGET set in your shell environment?

Ronald

smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Static metadata using setup.cfg

[Ronald Oussoren]

On 18 Aug, 2009, at 9:08, Floris Bruynooghe wrote:


On Tue, Aug 18, 2009 at 05:11:28AM +0300, Alex Grönholm wrote:

How would you declare dependencies? Remember that the list of
dependencies depends on at least two variables: Python version and
platform.


Many more, defining a known list of variables to be used to define
dependencies is not going to be enough.  Maybe there should be a way
of having general purpose named conditionals and implement a mapping
so that setup.py can define boolean functions for each named
conditional.  This can then be used for modules etc too instead of
just dependencies.

The benefit is that you allow developers to specify anything possible,
but still allow package management systems to analyse the
dependencies and modules etc from static data.  They will recognise
that something is conditional but are free to do with that knowledge
they want.

Expressiong conditionals in an elegant-ConfigParser-compatible way
might be tricky.


Wouldn't it be good enough to just have the most common use-cases in  
setup.cfg? The user can still add code to setup.py to tweak the  
default behaviour when the project does something non-standard.


BTW. One way to do conditionals in ConfigParser files might be:

[dependencies.os9]
use-when:   python=2.7, platform=='macos9'
require:   unix-toolbox=2.5

[dependencies.suse]
use-when: file_exists('/etc/SuSE-release')
require: ...

That is, use a config-key in dependency blocks that describes when the  
block should be used. The bit after the '.' in de section-name would  
be ignored by distutils and would basicly be comment.


Ronald



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Splitting metadata from setup.py in distutils..

[Ronald Oussoren]

 
On Monday, 17 August, 2009, at 05:09PM, Chris Withers 
ch...@simplistix.co.uk wrote:


 For me, that only leaves the choice of a filename.. 
 
  - setup.info
 
  - setup.conf
 
  - others..?

...is entirely up to the package developer. We can suggest whatever, but 
since it'll be called from setup.py, we can't do anything other than 
suggest a sensible default that will work with the tools that will 
hopefully show up that only parse the static file...

We already have 'setup.cfg', what's wrong with that?  If another filename is 
needed: please pick one where the suffix is unique, that would make it easier 
to associate the file with specific programs (double-click to open, 
spotlight/quicklook helpers on OSX, ...)

Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Distribute] Python 3 porting effort - plans proposal for 0.7/0.8

[Ronald Oussoren]

On 23 Jul, 2009, at 16:29, Tarek Ziadé wrote:

On Thu, Jul 23, 2009 at 4:17 PM, Lennart Regebrorege...@gmail.com  
wrote:

2009/7/23 Tarek Ziadé ziade.ta...@gmail.com:

- Let's then backport the 0.8 version into a 0.7 version, compatbile
with Python 2.x and with the required bootstraping
 so it works in 2.x environments


There is no reason to have different version numbers for that. In
fact, I think that would only be confusing. The Python 3 version
should work for Python 2 as well.


I think you don't see my point : making a pure Python 3 version,  
without
any backward compatibility issues / bootstraping , fight against  
setuptools

distribution. Then backporting it in 0.7

But 0.8 would not be 2.x compatible.


At PyCon09 there was some talk about a 3to2 tool, specially for  
issue: allow package maintainers to work in Python 3 and automaticly  
generate a Python 2 version.  I have no idea if anyone is actually  
working on such a tool though.


Ronald

P.S. Sorry about the late answers, I've been travelling and was  
offline for a while.




smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Distribute] Python 3 porting effort - plans proposal for 0.7/0.8

[Ronald Oussoren]

 
On Friday, 24 July, 2009, at 03:59PM, Leonardo Santagada 
santag...@gmail.com wrote:

On Jul 24, 2009, at 10:29 AM, Lennart Regebro wrote:

 2009/7/24 Tarek Ziadé ziade.ta...@gmail.com:
 I can postpone the 0.6 release for a week or so, until this is  
 ready, so we can
 have 0.6 for Python 3.

 I think we should make a 2.x only release first. The changes to
 support Python 3 are small but numerous and since the test-coverage
 isn't that great I'm worried it will insert subtle bugs. I would be
 less worried if we can have a longer test-period after these changes.

 On the other hand, if we do merge these changes before the 0.6 release
 they will quickly get tested. ;)


People using python 3x don't have support for setuptools so they would  
understand if you release early a beta version. And I think you  
should, not having setutools is a problem for every package being  
converted to python 3.1

The issue with merging python3-related changes is not that the python3 port 
would have a beta-status, but that there is a risk that this will accidently 
break python2 support.

I agree with Lennart that a 2.x only release would be better, especially 
because it would be possible to do a 0.7 alpha/beta release short after the 
stable 0.6 release.

Ronald

--
Leonardo Santagada
santagada at gmail.com



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] imho Distutils should stop this from happening to poor users....

[Ronald Oussoren]

On 18 Jun, 2009, at 10:45, Tarek Ziadé wrote:




On Thu, Jun 18, 2009 at 10:26 AM, David Lyon david.l...@preisshare.net 
 wrote:


Hi All,

Here's a package installation error message that's just plain scary
for the average user.

imho Distutils should have checks to stock packages requiring a C
compiler from being uploaded. It isn't right to expect that
from users. ie They should get the .pyd files.

That's not distutils fault, neither easy_install one.

appscript should provide a binary distribution for Windows (it  
does for Mac OS if you look at the pypi repo)
Since it doesn't, easy_install grabs the source distro, which is the  
best behavior I can think of,

then tries to compile it.


I don't know why it or any other package should provide a binary  
distribution for Windows, having one would be nice but shouldn't be a  
hard requirement, ever.


This does point to an missing feature in distutils or setuptools  
though: neither distutils nor setuptools has a way to specify the  
supported platforms, at least not in a way that is actually used  
during installation.  It would be nice if the appscript author could  
add some metadata to his setup.py file that would result in a clear  
error-message when you try to install it on a platform other than  
MacOSX.


The only way to achieve that at the moment is to add some code to  
setup.py that explicitly raises an exception when you try to build it  
on an unsupported platform.


The same is true for the python version, pyobjc currently explicitly  
tests for python = 2.3,3.0 because other versions aren't supported  
and cause compile errors (at best, IIRC it will compile with python  
2.2 but cause interpreter crashes when you try to use it).


Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] imho Distutils should stop this from happening to poor users....

[Ronald Oussoren]

On 18 Jun, 2009, at 11:22, Tarek Ziadé wrote:

On Thu, Jun 18, 2009 at 11:05 AM, Ronald Oussorenronaldousso...@mac.com 
 wrote:
The only way to achieve that at the moment is to add some code to  
setup.py

that explicitly raises an exception when you try to build it on an
unsupported platform.


Or to provide binary releases because python version and os info are
set in the archive name.
But it's hackish.


A binary release that does nothing would indeed be hackish, and would  
be worse than the existing error because a binary package gives the  
impression that the platform is actually supported.


Ronald

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Packaging: bdist_deb patches / platform specific code

[Ronald Oussoren]

On 8 Jun, 2009, at 20:15, David Lyon wrote:




the same for rpm, deb and mpkg.


Granted. They're are a bit more tricky. But essentially they
are all just zip files with some entry and exit scripts.


To pick some nits: the mpkg one cannot be created on any other system  
than an OSX one because one of the files that's included in an OSX  
package has an undocumented format and is created using a tool that's  
only available on OSX.


Ronald
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig