Re: [Distutils] Spec for the egg-info format
On 03/07/2012 09:07 PM, PJ Eby wrote: On Mon, Mar 5, 2012 at 4:47 PM, Paul Moore p.f.mo...@gmail.com mailto:p.f.mo...@gmail.com wrote: Is there a good spec of the egg-info (directory) format anywhere? [snip] - Should filenames in installed-files.txt be absolute or relative (and if relative, then to what)? That's defined by the appropriate PEP; setuptools doesn't yet support an installed-files log. I have nothing to add except for here; installed-files.txt is not specified in any PEP AFAIK, it's pip-specific (and generated by the distutils install --record option). Generally the filenames in it are relative to the location of installed-files.txt itself; absolute paths will work equally well for uninstall (where the installed-files log is used), but obviously become wrong if the installation were relocated. All the files in .egg-info aside from installed-files.txt are generated by setuptools and not touched by pip. Carl signature.asc Description: OpenPGP digital signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] RFC: Binary Distribution Format for distutils2/packaging
On 14 March 2012 02:50, Carl Meyer c...@oddbird.net wrote: In terms of distribution format, this sounds great; I think a clean break and new format that existing systems will just naturally ignore is the way to go. Agreed. I created a patch a while ago for a bdist_simple format, which has languished in the tracker for a while. It's a very basic patch, and the format is essentially just a stripped down version of the wininst format (a zip file with distinguished placeholder directories at the top level). I'm not sure I understand Jim's issues with the wininst format, so I can't respond as to why this wouldn't address them. I'd started writing a PEP for bdist_simple with Éric Araujo but stalled because I didn't have time. If someone else takes up the baton with a binary distribution format, that's fine with me. Your message doesn't address the on-disk installed format. I hope that the installed format (at least the d2/p default) will be consistent with PEP 376 (unzipped and flat), not an egg-style importable zip file relying on its own dedicated entry in a pth file. I agree that a flat on-disk format is essential. It's what pip and the standard tools use. The .pth based egg format appears to only have benefits if you want to manage multiple versions, and as far as I can see that requirement is now better satisfied via virtualenv/pip. One other point - it's actually relatively easy to write a tool to introspect egg and wininst format files, and unpack them manually into the on-disk layout needed. I have the bones of such a thing for pip (needs completing before I can release it). I would suggest that for backward compatibility and transition purposes, the following things need to be considered: - There should be converters from egg and wininst formats to the new format. Not everyone will start producing the new format immediately, and we should offer users a migration path where packagers haven't moved over. These should be trivial to write. - There should also be a converter from MSI format, but that's a beast to write and I'd say not worth the effort. - External tools like pip will need to handle the new format, ideally. Or at least keep out of the way when pysetup install is used. I'm assuming that pysetup install works in a virtualenv. Is the intention to promote the new format for older Python versions as well, via distutils2, or will this be a purely Python 3.3 option? If the former, conversion and compatibility is crucial, if the latter, then less so but we need to avoid causing problems for early adopter users who rely on packagers who haven't moved yet (Lobby the package maintainer to change is never a helpful answer...) The key here is that the new format needs to be a unifying effort - we already have 3 binary formats, if this new format merely adds a fourth, then in my view it's a failure. It needs to be the one obvious way if it's to work. Paul. PS A couple of relevant python-dev threads, for people who haven't seen them: http://mail.python.org/pipermail/python-dev/2011-October/114285.html http://mail.python.org/pipermail/python-dev/2011-October/113956.html http://mail.python.org/pipermail/python-dev/2011-October/114243.html ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] RFC: Binary Distribution Format for distutils2/packaging
On 3/13/12 7:50 PM, Carl Meyer wrote: On 03/13/2012 05:18 PM, Jim Fulton wrote: [snip] 1) setuptools eggs - Have to support legacy meta-data format 2) bdist - Need to add python version for: - compatibility info - also provides delimeter between version # and platform - Need to update setuptools/distribute to handle (or ignore) them. 3) New egg-like format pbd - Arrange suffix so ignored by setuptools/distribute - new-style meta data - would be a zip file - Essentially, .egg format with new-style meta data and different suffix. Option 3) looks the best to us, so we propose: - Introduce a new binary distribution format with a .pbd suffix and an egg-like structure. An example file name: ZODB3-3.10.0b1-py2.6-macosx-10.4-x86_64.pbd - Deprecate bdist format. D2/p will not support generation or installation of bdist distributions. In terms of distribution format, this sounds great; I think a clean break and new format that existing systems will just naturally ignore is the way to go. Your message doesn't address the on-disk installed format. I hope that the installed format (at least the d2/p default) will be consistent with PEP 376 (unzipped and flat), not an egg-style importable zip file relying on its own dedicated entry in a pth file. yeah the installation format is PEP 376 for *all* distributions, including the old ones: whatever gets installed is converted into PEP 376 if it's not a distutils2-based project We're only talking here about a replacement for a binary *distribution*. distutils' bdist is broken (the name of the file does not contain the py version...) while setuptools' egg would be a decent choice, it adds more legacy burden. I am excited we had that discussion with Jim, 3) seems the way forward for us. Cheers Tarek Carl ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] RFC: Binary Distribution Format for distutils2/packaging
On 14 March 2012 02:56, Tarek Ziadé ta...@ziade.org wrote: distutils' bdist is broken (the name of the file does not contain the py version...) ?? bdist_wininst filenames contain the python version when it matters (i.e., when there are binaries in the file). One thing I dislike about eggs is that the filename includes a Python version for all eggs, even when the package is pure python. I know eggs contain .pyc files (which are version specific) but IMO they shouldn't. Having a python version in the filename where it's not necessary increases the maintenance burden on packagers, who have to generate the extra files, and for users, who have to work out whether the package actually is version specific, or whether they can just install from source. Please can we have a new format that only has a Python version in the filename if it matters? Paul. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] RFC: Binary Distribution Format for distutils2/packaging
On Tue, Mar 13, 2012 at 11:00 PM, PJ Eby p...@telecommunity.com wrote: ... I'm not sure what else you'll actually gain by introducing a new format. IIUC, what new-style metadata egg files lack can be determined programmatically from their contents or the existing metadata files, so there seems to be little reason not to at least consume existing egg files. I'm not sure whether you're referring to the distribution format, or the meta-data format, or both. I don't really have an opinion, myself, about the meta data format. Other have decided it's going to be different. shrug Having decided to use a different meta data format, it makes sense to use a different distribution format. While d2/p might support eggs for a while (not sure of that), we wouldn't want new distributions to be in egg format, so we need an alternative. (Personally, I don't have an issue with the egg format, but the decision was made by others to move away from it.) Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] RFC: Binary Distribution Format for distutils2/packaging
On Wed, Mar 14, 2012 at 6:52 AM, Paul Moore p.f.mo...@gmail.com wrote: On 14 March 2012 02:50, Carl Meyer c...@oddbird.net wrote: In terms of distribution format, this sounds great; I think a clean break and new format that existing systems will just naturally ignore is the way to go. Agreed. I created a patch a while ago for a bdist_simple format, which has languished in the tracker for a while. It's a very basic patch, and the format is essentially just a stripped down version of the wininst format (a zip file with distinguished placeholder directories at the top level). I'm not sure I understand Jim's issues with the wininst format, so I can't respond as to why this wouldn't address them. My main issue is that I think we need a standard binary format that isn't system dependent. I see the wininst format as sort of a two-edged sword for libraries. On the one hand, it makes installing libraries natural for Windows users, on the other, it encourages poor hygiene. ... Your message doesn't address the on-disk installed format. I hope that the installed format (at least the d2/p default) will be consistent with PEP 376 (unzipped and flat), not an egg-style importable zip file relying on its own dedicated entry in a pth file. I agree that a flat on-disk format is essential. It's what pip and the standard tools use. Umnevermind. The .pth based egg format appears to only have benefits if you want to manage multiple versions, and as far as I can see that requirement is now better satisfied via virtualenv/pip. I don't see how virtualenv/pip address that. I assume you mean that if 2 applications/scripts require two different versions of a package, they should use two separate virtualenvs. That feels kinda heavy to me, but to each their own. :) FTR, buildout uses eggs to support multiple package versions. It doesn't use .pth files, but bakes necessary paths into generated scripts. Buildout on d2/p will continue creating egg-like structures, both to support multiple versions and to support caching and sharing installed packages. One other point - it's actually relatively easy to write a tool to introspect egg and wininst format files, and unpack them manually into the on-disk layout needed. I have the bones of such a thing for pip (needs completing before I can release it). I would suggest that for backward compatibility and transition purposes, the following things need to be considered: - There should be converters from egg and wininst formats to the new format. Not everyone will start producing the new format immediately, and we should offer users a migration path where packagers haven't moved over. These should be trivial to write. +1 ... - External tools like pip will need to handle the new format, ideally. Or at least keep out of the way when pysetup install is used. I'm assuming that pysetup install works in a virtualenv. My hope is that pip will eventually use d2/p. Is the intention to promote the new format for older Python versions as well, via distutils2, Yes or will this be a purely Python 3.3 option? No. If the former, conversion and compatibility is crucial, +1 ... The key here is that the new format needs to be a unifying effort - we already have 3 binary formats, if this new format merely adds a fourth, then in my view it's a failure. It needs to be the one obvious way if it's to work. Paul. PS A couple of relevant python-dev threads, for people who haven't seen them: http://mail.python.org/pipermail/python-dev/2011-October/114285.html http://mail.python.org/pipermail/python-dev/2011-October/113956.html http://mail.python.org/pipermail/python-dev/2011-October/114243.html Thanks for sharing these. You raise an interesting point in the last link. It's reasonable to argue that this is only a windows problem. It could be argued that, on other platforms: - Consumers of applications should get application installers, possibly with embedded copies of Python. - Consumers of libraries are developers who should be able to install development tools. I find the notion that we only need a binary format for Windows unsatisfying and unsettling, but if everyone else feels that way, I can live with it. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] RFC: Binary Distribution Format for distutils2/packaging
On 14 March 2012 15:43, Jim Fulton j...@zope.com wrote: My main issue is that I think we need a standard binary format that isn't system dependent. I didn't intend bdist_simple to be platform-specific. It's based on bdist_wininst certainly, but I stripped out all the platform-specific bits (the embedding in the executable). The resulting format is simply a zip file with separate directories for each of the sysconfig target locations. I see the wininst format as sort of a two-edged sword for libraries. On the one hand, it makes installing libraries natural for Windows users, on the other, it encourages poor hygiene. Not sure what you mean by poor hygiene, but if you mean installing everything into the system Python then I agree, and that's why I want something like virtualenv to have support for some binary format (via pip, packaging, or whatever). You raise an interesting point in the last link. It's reasonable to argue that this is only a windows problem. I'm a Windows-only user, so I only have experience of that platform, and it definitely is an issue there. When I stated on python-dev that binary distributions seemed to be only an issue on Windows, a number of Unix users (Nick Coghlan in particular, IIRC) argued that Unix users would also benefit from binary packages. So I tried to design something platform-neutral, although it needs a non-Windows user to take a look and confirm whether or not there are platform dependencies I missed. It could be argued that, on other platforms: - Consumers of applications should get application installers, possibly with embedded copies of Python. From my experience, that happens more often on Windows than elsewhere (py2exe/cx_Freeze). I didn't think Unix people did that. - Consumers of libraries are developers who should be able to install development tools. That was Nick's point, that sometimes even on Unix dependencies can be hard to install or complex to build, so there is value in binary distributions in those cases. I find the notion that we only need a binary format for Windows unsatisfying and unsettling, but if everyone else feels that way, I can live with it. So do I - but as you say, I can live with it (as long as Windows only doesn't end up translating into not worth bothering about...) Paul. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] RFC: Binary Distribution Format for distutils2/packaging
On Wed, Mar 14, 2012 at 8:21 AM, Paul Moore p.f.mo...@gmail.com wrote: One thing I dislike about eggs is that the filename includes a Python version for all eggs, even when the package is pure python. I know eggs contain .pyc files (which are version specific) but IMO they shouldn't. Your objection would make sense if .egg files were simply a format for transmitting files to be installed someplace -- in which case they could've omitted the .pyc files. .egg files were originally designed as a plugin distribution format, which means they're supposed to be drop-in usable to an app's plugin directory. Which brings up a few questions, actually. Will these .pbd files be drop-in usable in the same way? If not, then they're not really replacing eggs. The same is true if they don't offer plugin discovery metadata like entry points or EggTranslations. Having a python version in the filename where it's not necessary increases the maintenance burden on packagers, who have to generate the extra files, This is a bit of a misconception, perhaps originating in the fact that setuptools itself was always distributed as a collection of version-specific eggs. This was done solely to ease the bootstrapping of setuptools itself (to avoid recursively invoking the distutils while trying to build another package), and isn't really necessary for easy_install or other tools. You only need to generate eggs if you are supporting a binary application of some kind. Again, this raises the question: is .pbd a load-and-go format for distributing and *running* Python code, or is it just a way to bundle compiled extensions along with source code to simplify installation? I worry that some folks in the conversation may be thinking one thing, and some folks the other. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] RFC: Binary Distribution Format for distutils2/packaging
On 03/14/2012 09:33 AM, PJ Eby wrote: Again, this raises the question: is .pbd a load-and-go format for distributing and *running* Python code, or is it just a way to bundle compiled extensions along with source code to simplify installation? I worry that some folks in the conversation may be thinking one thing, and some folks the other. I think it was already clarified that it is the latter - purely a distribution format, intended for unzipped PEP-376 style installation, not drop-in use. Which means that no, it isn't a full replacement for eggs (I don't think that was the goal). Carl signature.asc Description: OpenPGP digital signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] RFC: Binary Distribution Format for distutils2/packaging
On Wed, Mar 14, 2012 at 11:17 AM, Jim Fulton j...@zope.com wrote: On Tue, Mar 13, 2012 at 11:00 PM, PJ Eby p...@telecommunity.com wrote: ... I'm not sure what else you'll actually gain by introducing a new format. IIUC, what new-style metadata egg files lack can be determined programmatically from their contents or the existing metadata files, so there seems to be little reason not to at least consume existing egg files. I'm not sure whether you're referring to the distribution format, or the meta-data format, or both. I don't really have an opinion, myself, about the meta data format. Other have decided it's going to be different. shrug Having decided to use a different meta data format, it makes sense to use a different distribution format. While d2/p might support eggs for a while (not sure of that), we wouldn't want new distributions to be in egg format, so we need an alternative. (Personally, I don't have an issue with the egg format, but the decision was made by others to move away from it.) In that case, I think it's really important to explicitly spell out the requirements and use cases for the new format; different people use .egg files differently, and the discussion here already reflects these different perceptions. The packaging PEPs cover only replacements for EGG-INFO/PKG-INFO and EGG-INFO/requires.txt. AFAIK, they don't include replacements for: * entry_points.txt -- plugin discovery * namespace_packages.txt -- namespace package management * dependency_links.txt -- suggesting additional locations for finding dependencies * native_libs.txt, eager_resources.txt, zip-safe/zip-not-safe -- support for in-zipfile access to resources and native libraries Which of these use cases will be supported by the new format, and which will not? Dropping any of them essentially means that some people are not going to have a migration path. (PEP 382 or 402 will eventually clear up the namespace_packages bit, but are unlikely to be backported to 2.x.) Also, is there an official discovery API for finding additional metadata (such as would be needed by EggTranslations' i18n system built on egg metadata)? I assume that you're aware of most of the above, as Zope has historically made use of at least the first two items on the list above. But a PEP for this new format really ought to spell out the use cases it's going to handle, or not, and why (or why not). ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] easy_install problem
Try running your command with '-vv' to increase the verbosity, and with
PYDISTUTILS_DEBUG=true in your environment; you may get back more detailed
error information that may assist you in finding out what's happening with
the proxy.
On Wed, Mar 14, 2012 at 6:10 AM, Nitin Dhiman nitinkdhi...@gmail.comwrote:
Greetings,
My OS is Ubuntu Lucid lynx 10.04, with Python disutils
python-setuptools version 0.6.10-4ubuntu1
I am behind proxy. http_proxy and https_proxy is defined as
http://username:passwd@proxy-ip:port
I am trying to use easy_install to install rosinstall using following
command:
nitin@nitin-desktop:~/Downloads$ sudo easy_install -U rosinstall vcstools
[sudo] password for nitin:
Searching for rosinstall
Reading http://pypi.python.org/simple/rosinstall/
Download error: [Errno -2] Name or service not known -- Some packages
may not be found!
Reading http://pypi.python.org/simple/rosinstall/
Download error: [Errno -2] Name or service not known -- Some packages
may not be found!
Couldn't find index page for 'rosinstall' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading http://pypi.python.org/simple/
Download error: [Errno -2] Name or service not known -- Some packages
may not be found!
No local packages or download links found for rosinstall
error: Could not find suitable distribution for
Requirement.parse('rosinstall')
This repeats couple of times. Is this issue related to proxy??? I do
not face any problem when having a direct internet connnection.
I had tried using pip also but i gets problems. Following is the
output of log file.
Downloading/unpacking rosinstall
Getting page http://pypi.python.org/simple/rosinstall
Could not fetch URL http://pypi.python.org/simple/rosinstall:
urlopen error [Errno -2] Name or service not known
Will skip URL http://pypi.python.org/simple/rosinstall when looking
for download links for rosinstall
Getting page http://pypi.python.org/simple/
Could not fetch URL http://pypi.python.org/simple/: urlopen error
[Errno -2] Name or service not known
Will skip URL http://pypi.python.org/simple/ when looking for
download links for rosinstall
Cannot fetch index base URL http://pypi.python.org/simple/
Cannot find requirement rosinstall, nor fetch index URL
http://pypi.python.org/simple/
Exception information:
Traceback (most recent call last):
File /usr/lib/python2.6/dist-packages/pip.py, line 252, in main
self.run(options, args)
File /usr/lib/python2.6/dist-packages/pip.py, line 408, in run
requirement_set.install_files(finder, force_root_egg_info=self.bundle)
File /usr/lib/python2.6/dist-packages/pip.py, line 1750, in
install_files
url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
File /usr/lib/python2.6/dist-packages/pip.py, line 996, in
find_requirement
url_name = self._find_url_name(Link(self.index_urls[0]), url_name, req)
File /usr/lib/python2.6/dist-packages/pip.py, line 1073, in
_find_url_name
raise DistributionNotFound('Cannot find requirement %s, nor fetch
index URL %s' % (req, index_url))
DistributionNotFound: Cannot find requirement rosinstall, nor fetch
index URL http://pypi.python.org/simple/
The package I am trying to install is from ROS (www.ros.org) . The
community of ROS has suggested to ask the solution in this community .
I hope some one can solve the problem.
Could this problem be because of any proxy rules I am not aware of
these.
regards
prince
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] RFC: Binary Distribution Format for distutils2/packaging
On Wed, Mar 14, 2012 at 9:17 AM, Paul Moore p.f.mo...@gmail.com wrote: It's reasonable to argue that this is only a windows problem. no -- it's a Mac OS-X problem, too. Indeed, a harder one, due to: A) The Mac platform now has 4! architectures: PPC, PPC64, x86, intel64. Granted, PPC is almost dead, PPC64 never saw much use, and even 32 bit x86 is on the way out. Never the less -- at least 32 and 64bit intel are going to be around for a while. B) OS-X support fat binaries, and the pyton.org binaries have been built that way for ages. C) OS-X moves forward, fast -- it's gets pretty ticky to build binaries that run on older versions than the one you are building on (it can be done, to a point, but it's hard to get right) - Consumers of applications should get application installers, possibly with embedded copies of Python. sure -- but people developers need to build those... From my experience, that happens more often on Windows than elsewhere (py2exe/cx_Freeze). I didn't think Unix people did that. not as much, but Mac developers do. - Consumers of libraries are developers who should be able to install development tools. In theory, yes, but: 1) there are folks that want to do a little python that don't have any experience or interest in the whole C building thing -- and to get the compiler on the Mac, you need to register with Mac Developer connection, then download a MASSIVE binary -- it's not a trivial lift. 2) as a stated above, building binaries of packages that you can re-distribute to other systems (py2app) is tricky -- even more so when you need compiled dependencies (libjpeg, libfreetype, that sort of thing) So the short version is -- binary packages are important on the Mac. Which brings up another point: Universal binaries (i.e. more than one architecture in one binary) have never been properly supported by binary eggs/setuptools. I think it may be as simple as the naming convention -- the binary would be named according to the machine it was built on (i.e. PPC) but when you tried to install in on another machine, setuptools would look for one called, i.e. x86 and not find it. There may be some other issues, too, but in any case, we need to make sure the naming convention handles the multi-architecture case as well. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] distribute-0625 install problem
Before I read the instructions, downloaded the tarball to c:\temp Extracted files to C:\temp\distribute-0.6.25 From Dos command window in the C:\temp\distribute-0.6.25 directory issued command python setup.py install Got error no module named distutils.util, but util.py is in C:\Python32\Lib\distutils Frustrating Then I read instructions Downloaded distribute_setup.py to C:\Python32\Lib Ran distribute_setup.py in IDLE, got following message in red, which from my very limited python experience seems to be not good Traceback (most recent call last): File C:\temp\distribute-0.6.25\distribute_setup.py, line 494, in module main(sys.argv[1:]) File C:\temp\distribute-0.6.25\distribute_setup.py, line 489, in main tarball = download_setuptools() File C:\temp\distribute-0.6.25\distribute_setup.py, line 192, in download_setuptools log.warn(Downloading %s, url) File C:\Python32\lib\distutils\log.py, line 47, in warn self._log(WARN, msg, args) File C:\Python32\lib\distutils\log.py, line 30, in _log if stream.errors == 'strict': AttributeError: errors My ultimate goal is to install beautifulsoup4 to parse webpages. Steps I have been told to follow 1) install python 3.2.2 (with msi installer) 2) install distribute-0.6.24 (with python setup.py install) 3) install lxml-2.3-py3.2-win32.egg (with easy_install lxml-2.3- py3.2-win32.egg) 4) install beautifulsoup4-4.0.0b10 (with easy_install beautifulsoup4) Step 1 is complete. Step 2, can't seem to get 0.6.24, so tried 0.6.25. Seems I have not succeeded, at least don't seem to be able to use easy_install Any help will be greatly appreciated ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] RFC: Binary Distribution Format for distutils2/packaging
On Wed, Mar 14, 2012 at 1:45 PM, Paul Moore p.f.mo...@gmail.com wrote: On 14 March 2012 16:58, PJ Eby p...@telecommunity.com wrote: On Wed, Mar 14, 2012 at 12:42 PM, Jim Fulton j...@zope.com wrote: Right. It is only a replacement for use of eggs as a binary distribution (not installation) format. Or to put it another way, it won't be a replacement for eggs at all. It'll be a replacement for bdist_wininst. That is certainly my understanding. Do we need a replacement for eggs? Nope. My only concern is that unless this is made really clear, people are likely to think this *is* intended to replace eggs, confusing eggs-in-general with eggs-as-improved-bdist_dumb/wininst. Other than that, replacing bdist_dumb and bdist_wininst with something more sane has my +1. (I just want it clearly labeled as such, since all the discussion about possibly using the egg format may confuse the unwary.) If I understand correctly, all this proposal does is update bdist_dumb to use a saner naming convention, standardize on zip format, and flatten the inner directory layout. A sort of bdist_dumb2, if you will. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] RFC: Binary Distribution Format for distutils2/packaging
On Wed, Mar 14, 2012 at 1:42 PM, Chris Barker chris.bar...@noaa.gov wrote: Universal binaries (i.e. more than one architecture in one binary) have never been properly supported by binary eggs/setuptools. I think it may be as simple as the naming convention -- the binary would be named according to the machine it was built on (i.e. PPC) but when you tried to install in on another machine, setuptools would look for one called, i.e. x86 and not find it. There may be some other issues, too, but in any case, we need to make sure the naming convention handles the multi-architecture case as well. FWIW, all the setuptools platform compatibility code for Mac OS was contributed by Bob Ippolito, and I don't remember his rationale for not supporting fat binaries exactly, although at the time (almost a decade ago) there were only two architectures and things in general were much simpler then. ;-) If you settle on a naming convention that works, patches to the pkg_resources functions 'compatible_platforms()', 'get_build_platform()', and 'get_supported_platform()' are welcome. These are the only parts of setuptools that need to understand platform strings as anything other than an opaque identifier. (For that matter, if anybody comes up with a sane platform naming convention for any *other* platforms, patches for those are welcome too!) ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] RFC: Binary Distribution Format for distutils2/packaging
On Mar 14, 2012, at 1:42 PM, Chris Barker wrote: 1) there are folks that want to do a little python that don't have any experience or interest in the whole C building thing -- and to get the compiler on the Mac, you need to register with Mac Developer connection, then download a MASSIVE binary -- it's not a trivial lift. Not any more. You can install XCode from Mac App store as an application now instead of an installation bundle/image. Also, the membership in the Mac Developer connection is not required. Only if you are going to sign apps for Mac or iOS you need the membership. For a casual user who needs a C compiler and finds XCode on the Mac App store it's not required. Zvezdan ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] RFC: Binary Distribution Format for distutils2/packaging
On Wed, Mar 14, 2012 at 3:05 PM, Zvezdan Petkovic zvez...@computer.org wrote: 1) there are folks that want to do a little python that don't have any experience or interest in the whole C building thing -- and to get the compiler on the Mac, you need to register with Mac Developer connection, then download a MASSIVE binary -- it's not a trivial lift. Not any more. You can install XCode from Mac App store as an application now instead of an installation bundle/image. nicer -- though I imagine it's still a huge download. Also, the membership in the Mac Developer connection is not required. Only if you are going to sign apps for Mac or iOS you need the membership. For a casual user who needs a C compiler and finds XCode on the Mac App store it's not required. nice to know. However, the fact remains that there are folks that just want to write some Python code -- Also, aside from the compiler, it's still can be a pain to build stuff that requires dependencies (yes, I know about homebrew, macports, fink -- the point still stands) I think it's important to support that part of the community. NOTE: on Windows, you can simply install the right version of Visual Studio Express, and get building capability there, too -- but there is still a real desire for binaries. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] RFC: Binary Distribution Format for distutils2/packaging
On 14 March 2012 19:04, Tarek Ziadé ta...@ziade.org wrote: Please can we have a new format that only has a Python version in the filename if it matters? isn't that supposed to be the source release ? Yes, basically - at least as far as I understand. Why would someone create a binary release when it's pure Python ? I wish I knew. But people do - mostly egg format files. But I think this is partly because of the confusion between egg-as-distribution-format vs egg-as-directly-usable-object that PJE alludes to in his emails. If there's a binary egg available, how do I know whether it's needed without trying a source install and seeing if it works? And what about where it does, but doesn't install C accelerators because I have no compiler? The source works, but the binary is better. How would an automated installation process know? Binary installers for pure-Python projects exist, and cause confusion. (Example - IPython, see http://archive.ipython.org/release/0.12/) or are you thinking about the auto-installable feature for windows here ? No. I used to care about this, but no longer. I'm happy now to install anything that *can* be installed from source directly, using pip (or pysetup, once it works). I think there are two different uses cases: - install a program or a lib manually -- where an auto-installer makes sense. - have a library installed by a tool, where an auto-installer is not really needed. I'm not sure I'd characterise the cases like that, I tend to think of them as wants to integrate with system standard policies vs wants to use Python specific approaches. But it's the same 2 cases. I used to be in the first camp, but I've switched and these days I'm firmly in the second. I was thinking bdist_wininst and bdist_msi would stay around for people that want to create self-installable archives, and we would discuss a new format that is not self-installable. The key thing for me, as a user, is having packages that I cannot build myself available to me in a format that I can use. And secondarily to have an easy user experience. The following are possible installation scenarios: 1. pip/pysetup install PKG - transparent and simple, for source distributions that are pure-Python or I can build (no external dependencies and I have C installed) 2. pip/pysetup install PKG - new-format supplied by author (I assume the install command will auto-download the new format) 3. download egg/wininst file run converter to create new-format pip/pysetup install new-format-file tidy up the temporary files - egg or wininst supplied by author, assuming converters exist and are supplied with Python (add a find the converter step if they are 3rd party) 4. Install as system installer - MSI supplied by author, and I'm installing in system Python. 5. Cry - MSI supplied by author and I want to install in a virtualenv or similar. Basically, it's impossible (or at least, not worth the effort). To give me the best user experience, authors have to start supplying the new format. The hard part is getting authors to take up a new format (after all, they probably want to keep the old format as well, for users who haven't switched to pysetup, so you've just doubled their work). Alternatively, integrate the converters directly into pip/pysetup, so that eggs and wininsts are *transparently* converted on download, making case 3 look just like case 2. But at that point, what's the argument for authors distributing the new format, when the old ones work just as well for new-style users? The technical issues aren't hard, it's the social issues that need careful consideration. Paul. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] RFC: Binary Distribution Format for distutils2/packaging
On Wed, Mar 14, 2012 at 7:29 PM, Paul Moore p.f.mo...@gmail.com wrote: If there's a binary egg available, how do I know whether it's needed without trying a source install and seeing if it works? The egg will have a platform string in its name in that case. Otherwise, it'll be just package-version-pyX.X.egg. (Actually, on reflection, I'm not sure I understand your question: needed relative to what? to downloading the source version?) ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
