Re: [Distutils] does pypi or red-dove have a better firehose API than download all the packages?
On Fri, May 24, 2013 at 08:05:45PM +, holger krekel wrote: On Fri, May 24, 2013 at 17:51 +0300, Marius Gedminas wrote: On Fri, May 24, 2013 at 02:01:16PM +, holger krekel wrote: On Fri, May 24, 2013 at 09:55 -0400, Donald Stufft wrote: Most packages also have an egg-info inside of them you can parse. Of course the issue is that you're only going to get the requirements of the system that ran setup.py, either the authors or the servers. Which doesn't accurately represent all of the dependencies all of the time. True but maybe it would go a long way for most packages. In that case you might find https://github.com/mgedmin/ztk-py3-status/blob/master/get_deps.py useful. looks interesting but a bit of a strange UI it seems. Why not accept a tar archive? Or am i missing something? It's part of a pipeline that builds http://zope3.pov.lt/py3/ At that point I don't have a single tar archive; I have a list of package names, versions and sdist URLs. Marius Gedminas -- Photons have energy, and trying to cram too many into too small of a space can cause a black hole to form, which is, needless to say, not a desirable trait for an optical computer. -- http://scottaaronson.com/blog/?p=261#comment-13693 signature.asc Description: Digital signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Using Scripts\foo.exe on a python which doesn't have Distribute installed?
Thanks PJ. I haven't used py2exe as I don't want to redistribute python also, just our program. I'll dig into your other suggestions and see where I end up. -matt On Wed, May 22, 2013 at 10:22 PM, PJ Eby p...@telecommunity.com wrote: On Wed, May 22, 2013 at 8:12 PM, Matt Wilkie map...@gmail.com wrote: How do I get my installer to include Distribute so I don't have to tell our users before you install our program you have to go install this other program? Setuptools (which Distribute is based on) is designed for shipping libraries, not applications; it's developer installer, not primarily an end-user installer for applications. So you probably should be using py2exe instead. Alternatively, you can bundle a copy of pkg_resources.py, or use a script that doesn't depend on entry points, or copy your script to foo-script.py alongside the .exe launcher, and manually include those two files as scripts (not using entry points) in your setup() definition. Any of these approaches will solve the problem; it's mostly a matter of your preferences or other requirements. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Using Scripts\foo.exe on a python which doesn't have Distribute installed?
Hi Matt, if you don't have any C extensions to deal with and can assume Python is already present on the destination system, then a zipfile with a __main__.py file may also be a suitable solution (see PEP 441 for more info about that feature - it's been supported since 2.6, the PEP just proposes some better tools in the standard library for creating them) Cheers, Nick. On Sat, May 25, 2013 at 5:46 PM, Matt Wilkie map...@gmail.com wrote: Thanks PJ. I haven't used py2exe as I don't want to redistribute python also, just our program. I'll dig into your other suggestions and see where I end up. -matt On Wed, May 22, 2013 at 10:22 PM, PJ Eby p...@telecommunity.com wrote: On Wed, May 22, 2013 at 8:12 PM, Matt Wilkie map...@gmail.com wrote: How do I get my installer to include Distribute so I don't have to tell our users before you install our program you have to go install this other program? Setuptools (which Distribute is based on) is designed for shipping libraries, not applications; it's developer installer, not primarily an end-user installer for applications. So you probably should be using py2exe instead. Alternatively, you can bundle a copy of pkg_resources.py, or use a script that doesn't depend on entry points, or copy your script to foo-script.py alongside the .exe launcher, and manually include those two files as scripts (not using entry points) in your setup() definition. Any of these approaches will solve the problem; it's mostly a matter of your preferences or other requirements. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Using Scripts\foo.exe on a python which doesn't have Distribute installed?
slapped reply too early, sorry. Alternatively, you can bundle a copy of pkg_resources.py, I managed to get this to work by adding it as a script in setup.py: #smelly hack, allows foo.exe to run on machines w/out Distribute scripts = [ 'setup/pkg_resources.py', ], this puts it in PythonXX\Scripts on the target machine, as I'm sure you know. What happens if the target machine already has Distribute or Setuptools installed, and thus has pkg_resources somewhere under PythonXX\Lib\site-packages, and likely a different version to boot? I tried putting it in the same tree as our stuff, foo/external/pkg_resources.py, but that's not found. I'm assuming because it thinks it's name is foo.external.pkg_resources. -matt On Sat, May 25, 2013 at 12:46 AM, Matt Wilkie map...@gmail.com wrote: Thanks PJ. I haven't used py2exe as I don't want to redistribute python also, just our program. I'll dig into your other suggestions and see where I end up. -matt ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Using Scripts\foo.exe on a python which doesn't have Distribute installed?
... or use a script that doesn't depend on entry points not desirable, as we like the .exe entry points creates and don't want to use a batch file (the are you sure you want to quit? message on ctrl-c is annoying) , or copy your script to foo-script.py alongside the .exe launcher, and manually include those two files as scripts (not using entry points) in your setup() definition. Would you please expand on this? The .exe launcher seems to depend on entry points, or I'm doing something wrong (entirely likely). thanks, -matt ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Using Scripts\foo.exe on a python which doesn't have Distribute installed?
That looks very interesting Nick! Do you know of any small or smallish programs using this I could inspect and learn from? Most of the references I find surround pyinstaller it's specfile format. thanks, -matt On Sat, May 25, 2013 at 1:02 AM, Nick Coghlan ncogh...@gmail.com wrote: Hi Matt, if you don't have any C extensions to deal with and can assume Python is already present on the destination system, then a zipfile with a __main__.py file may also be a suitable solution (see PEP 441 for more info about that feature - it's been supported since 2.6, the PEP just proposes some better tools in the standard library for creating them) Cheers, Nick. On Sat, May 25, 2013 at 5:46 PM, Matt Wilkie map...@gmail.com wrote: Thanks PJ. I haven't used py2exe as I don't want to redistribute python also, just our program. I'll dig into your other suggestions and see where I end up. -matt On Wed, May 22, 2013 at 10:22 PM, PJ Eby p...@telecommunity.com wrote: On Wed, May 22, 2013 at 8:12 PM, Matt Wilkie map...@gmail.com wrote: How do I get my installer to include Distribute so I don't have to tell our users before you install our program you have to go install this other program? Setuptools (which Distribute is based on) is designed for shipping libraries, not applications; it's developer installer, not primarily an end-user installer for applications. So you probably should be using py2exe instead. Alternatively, you can bundle a copy of pkg_resources.py, or use a script that doesn't depend on entry points, or copy your script to foo-script.py alongside the .exe launcher, and manually include those two files as scripts (not using entry points) in your setup() definition. Any of these approaches will solve the problem; it's mostly a matter of your preferences or other requirements. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Using Scripts\foo.exe on a python which doesn't have Distribute installed?
On Sat, May 25, 2013 at 6:16 PM, Matt Wilkie map...@gmail.com wrote: That looks very interesting Nick! Do you know of any small or smallish programs using this I could inspect and learn from? Most of the references I find surround pyinstaller it's specfile format. The zipfile execution feature was added primarily as a tool for internal distribution within large Python-using organisations, so I'm not aware of any good examples on the public internet (I assume it's seeing some use inside Google, as that's where the idea originated). Internet distribution tends to use either py2exe and its ilk (because the distributor can't assume a Python interpreter on the destination system) or else give users instructions on installing and using pip or easy_install (because they want their instructions to work across at least the main three platforms). Using pip or easy_install to do the installation also has the side effect of getting pkg_resources onto the target system, since some features of that ecosystem (like entry points) require it. Being able to assume the presence of Python on the target system but *not* the presence of pkg_resources is a fairly idiosyncratic case, which is why you haven't been able to find much specific info on handling it. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Using Scripts\foo.exe on a python which doesn't have Distribute installed?
On Sat, May 25, 2013 at 4:06 AM, Matt Wilkie map...@gmail.com wrote: ... or use a script that doesn't depend on entry points not desirable, as we like the .exe entry points creates and don't want to use a batch file (the are you sure you want to quit? message on ctrl-c is annoying) , or copy your script to foo-script.py alongside the .exe launcher, and manually include those two files as scripts (not using entry points) in your setup() definition. Would you please expand on this? The .exe launcher seems to depend on entry points, or I'm doing something wrong (entirely likely). The foo.exe launcher doesn't use the entry point, all it does is run an adjacent foo-script.py. You can put anything you want in foo-script.py adjacent to that .exe, and it'll run. As for different versions of pkg_resources on sys.path, Python always puts the directory containing the script first. So if there's a pkg_resources.py adjacent to the script, that will take precedence over any other copy. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Announcing Setuptools 0.7b3
Setuptools and Distribute are now merged, and the new home can be found at https://bitbucket.org/pypa/setuptools. Issues for Distribute are still being maintained at its old home, but releases are now made from the 'distribute' fork on the setuptools repo. Future releases of Distribute (if any) will be maintenance only. The new setuptools, which is designed to be largely compatible both with Distribute and Setuptools can now be downloaded on the 'downloads' page of the PyPA site: https://bitbucket.org/pypa/setuptools/downloads For those familiar with setuptools/distribute, the upgrade process is pretty straightforward: remove your old versions of distribute and/or setuptools and install the latest. To install the latest, extract the .tar.gz, and run setup.py in the target environment. Windows .msi installers have been provided as well. Ez_setup.py will not yet work until the official release is made to PyPI. I encourage all of those with experience or interest in packaging to try out the new setuptools in your environments. Test it against your tool chain. Provide feedback and file issues in the project site. This limited public release is provided as an advance release prior to the full public release. Assuming there are no reported blocking issues, a final 0.7 release will be made to PyPI in the coming weeks. Regards, Jason smime.p7s Description: S/MIME cryptographic signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Using Scripts\foo.exe on a python which doesn't have Distribute installed?
On Sat, May 25, 2013 at 4:16 PM, Matt Wilkie map...@gmail.com wrote: That looks very interesting Nick! Do you know of any small or smallish programs using this I could inspect and learn from? Most of the references I find surround pyinstaller it's specfile format. This article explain about using the __main__.py file - http://sayspy.blogspot.com/2010/03/various-ways-of-distributing-python.html As for example, one that I recently found - https://github.com/pagekite/PyPagekite/ ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Announcing Setuptools 0.7b3
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/25/2013 04:40 PM, Jason R. Coombs wrote: Setuptools and Distribute are now merged, and the new home can be found at https://bitbucket.org/pypa/setuptools. Issues for Distribute are still being maintained at its old home, but releases are now made from the 'distribute' fork on the setuptools repo. Future releases of Distribute (if any) will be maintenance only. The new setuptools, which is designed to be largely compatible both with Distribute and Setuptools can now be downloaded on the 'downloads' page of the PyPA site: https://bitbucket.org/pypa/setuptools/downloads For those familiar with setuptools/distribute, the upgrade process is pretty straightforward: remove your old versions of distribute and/or setuptools and install the latest. To install the latest, extract the .tar.gz, and run setup.py in the target environment. Windows .msi installers have been provided as well. Ez_setup.py will not yet work until the official release is made to PyPI. I encourage all of those with experience or interest in packaging to try out the new setuptools in your environments. Test it against your tool chain. Provide feedback and file issues in the project site. This limited public release is provided as an advance release prior to the full public release. Assuming there are no reported blocking issues, a final 0.7 release will be made to PyPI in the coming weeks. Congratulations, and thanks to the PyPA developers for coming together to produce this unified release. FWIW, I was just able to update a setuptools 0.6 virtualenv via: $ bin/easy_install -U https://bitbucket.org/pypa/setuptools/downloads/setuptools-0.7b3.tar.gz For a distribute-based virtualenv, I had to delete the 'distribute' entry in 'site-pacakges' after running the 'easy_install' command. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlGhPPYACgkQ+gerLs4ltQ5zngCg1iTC2xwLcvZT/MHSi8bNlFDl clgAnjXOCbXqN0rW1ihwWEBUCFzEFC5U =MF9s -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] does pypi or red-dove have a better firehose API than download all the packages?
On 24.05.2013 18:18, Donald Stufft wrote: On May 24, 2013, at 12:14 PM, M.-A. Lemburg m...@egenix.com wrote: On 24.05.2013 17:21, Vinay Sajip wrote: Donald Stufft donald at stufft.io writes: Most packages also have an egg-info inside of them you can parse. I don't know how accurate that information is - IIRC pip always runs egg-info on downloaded archives. I presume this is to get the correct dependencies which are relevant to the installation system. It would be nice to have the PKG-INFO readily available on the /simple/ index pages. This contains the Requires header as well. The requires headers in the PKG-INFO are practically worthless. At the moment, PKG-INFO is only available on the PyPI edit pages for packages you own, even though access is possible without login: https://pypi.python.org/pypi?name=egenix-mx-baseversion=3.2.6:action=display_pkginfo Leaving aside the unusable Requires header, could we still get PKG-INFO exposed in the /simple/ index ? This would then make it possible to access the basic meta-data cached via the CDN and without having to download the package. Something like: a href=https://pypi.python.org/pypi?name=egenix-mx-baseversion=3.2.6:action=display_pkginfo;3.2.6 PKG-INFO/abr/ for each release. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 26 2013) Python Projects, Consulting and Support ... http://www.egenix.com/ mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2013-07-01: EuroPython 2013, Florence, Italy ... 36 days to go : Try our mxODBC.Connect Python Database Interface for free ! :: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Announcing Setuptools 0.7b3
On Sat, May 25, 2013 at 4:40 PM, Jason R. Coombs jar...@jaraco.com wrote: https://bitbucket.org/pypa/setuptools/downloads You may be already aware of this, but the installation instructions don't match the available downloads. I was previously thinking we should just drop binary distributions of setuptools altogether anyway, and make ez_setup.py the standard way to install setuptools on all platforms. (Which would help address things like installation conflicts w/existing Distribute, etc.) ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Announcing Setuptools 0.7b3
I'll be sure to address that before the final release. I have been tempted to just support sdist installs and follow the technique that Distribute used, except for this one comment I heard in person by a newer Windows user recently, that they still hear that Setuptools is still considered friendlier than Distribute. But if PJE is on board with source installs, now is a good time for that switch, so let's do it. -Original Message- From: PJ Eby [mailto:p...@telecommunity.com] Sent: Saturday, 25 May, 2013 18:45 To: Jason R. Coombs Cc: distutils-sig@python.org Subject: Re: [Distutils] Announcing Setuptools 0.7b3 On Sat, May 25, 2013 at 4:40 PM, Jason R. Coombs jar...@jaraco.com wrote: https://bitbucket.org/pypa/setuptools/downloads You may be already aware of this, but the installation instructions don't match the available downloads. I was previously thinking we should just drop binary distributions of setuptools altogether anyway, and make ez_setup.py the standard way to install setuptools on all platforms. (Which would help address things like installation conflicts w/existing Distribute, etc.) smime.p7s Description: S/MIME cryptographic signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Announcing Setuptools 0.7b3
On Sat, May 25, 2013 at 7:02 PM, Jason R. Coombs jar...@jaraco.com wrote: I heard in person by a newer Windows user recently, that they still hear that Setuptools is still considered friendlier than Distribute. Without knowing what exactly they considered friendlier, I'm not sure what to suggest. Even if you use a GUI to install setuptools, it's not going to do you much good without getting to a command line! ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Announcing Setuptools 0.7b3
On Sun, May 26, 2013 at 6:40 AM, Jason R. Coombs jar...@jaraco.com wrote: Setuptools and Distribute are now merged, and the new home can be found at https://bitbucket.org/pypa/setuptools. Yay! Thanks for the work both you and PJE have put into making this happen :) The pieces of Python's improved packaging story start to come together... Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Using Scripts\foo.exe on a python which doesn't have Distribute installed?
Being able to assume the presence of Python on the target system but *not* the presence of pkg_resources is a fairly idiosyncratic case, which is why you haven't been able to find much specific info on handling it. This might be a Windows thing. I'm coming primarily from that background, where for most if it doesn't have a GUI installer, it doesn't exist, and Distribute doesn't have an installer. I guess it's idiosyncratic because those coming before me on a similar path would have just gravitated to NSIS or similar for this need. -matt ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Using Scripts\foo.exe on a python which doesn't have Distribute installed?
The foo.exe launcher doesn't use the entry point, all it does is run an adjacent foo-script.py. You can put anything you want in foo-script.py adjacent to that .exe, and it'll run. Ahhh, perfect! I've verified it works for me, and much better than bundling a duplicate of Distribute. Thanks! [...later]: is a postinstall the best (only?) means to replace the contents of foo-script.py? By experimentation I've learned the `entry_points=` script supersedes a `scripts=` script of the same name. As for different versions of pkg_resources on sys.path, Python always puts the directory containing the script first. So if there's a pkg_resources.py adjacent to the script, that will take precedence over any other copy. Thanks, this is good to know. best, -matt ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
