Re: [Distutils] Deviation between distutils and setuptools over package_data
On Sep 18, 2013 11:37 PM, "PJ Eby" wrote: > > On Wed, Sep 18, 2013 at 5:07 PM, Benjamin Root wrote: > > In creating a source distribution, I have found a disparity between the > > behaviors of distutils and setuptools with respect to package_data. As of > > python Issue 2279: http://bugs.python.org/issue2279, entries listed in > > package_data are used when building an sdist. I have verified that this > > works in a simple example when setup() is imported from distutils.core. > > However, if I import setup() from setuptools, it does not pull in the data > > as listed in package_data (and presumedly data_files). > > You need to use the include_package_data = True flag. > I did and it didn't make a difference. Plus, shouldn't it have picked up everything that I had version controlled, anyway? Does setuptools recognize the new svn 1.7 format? Even svn 1.6 clients will refuse to do an 'svn status' on a 1.7 repo checkout. So, I think there might be multiple bugs in play. I also don't see why I should have to use that keyword argument if distutils does fine without it. sdist isnt -- from my perspective -- a setuptools specific feature, so I had no expectation of there being such an egregious difference. > > > P.S. - on a related note, for a package "foo", foo.egg-info directory is > > created with a SOURCES.txt file. I have found that under certain situations, > > it seems that a successful install would result in a fully listed > > SOURCES.txt, and then subsequent calls to sdist seems to use that > > information to build seemingly correct archives. A co-working > > double-checking a deployment process I made did an sdist and created a > > source distribution without the package_data when he did a fresh checkout, > > but whenever I did it from my development branch, the source distribution > > worked fine. I haven't figured out exactly how this came about, but it seems > > to be tied to the SOURCES.txt file. > > SOURCES.txt mostly exists so that you can safely build an sdist from > an sdist, as is required by e.g. bdist_rpm, without having any > revision control data on hand to guide the process. Setuptools also > can insert a possibly-modified setup.cfg into an sdist for the same > reason, so that if you used revision control tags to specify the > version when building the sdist, any sdists rebuilt from that sdist > will have the same version tags. Yes, I understand what sources.txt is for. The issue is that it seems that it is possible for different build commands to produce different source.txt results. Since I haven't figured out how I managed to do that, it isn't the focus of my bug report. Cheers! Ben Root ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Deviation between distutils and setuptools over package_data
On Wed, Sep 18, 2013 at 5:07 PM, Benjamin Root wrote: > In creating a source distribution, I have found a disparity between the > behaviors of distutils and setuptools with respect to package_data. As of > python Issue 2279: http://bugs.python.org/issue2279, entries listed in > package_data are used when building an sdist. I have verified that this > works in a simple example when setup() is imported from distutils.core. > However, if I import setup() from setuptools, it does not pull in the data > as listed in package_data (and presumedly data_files). You need to use the include_package_data = True flag. > P.S. - on a related note, for a package "foo", foo.egg-info directory is > created with a SOURCES.txt file. I have found that under certain situations, > it seems that a successful install would result in a fully listed > SOURCES.txt, and then subsequent calls to sdist seems to use that > information to build seemingly correct archives. A co-working > double-checking a deployment process I made did an sdist and created a > source distribution without the package_data when he did a fresh checkout, > but whenever I did it from my development branch, the source distribution > worked fine. I haven't figured out exactly how this came about, but it seems > to be tied to the SOURCES.txt file. SOURCES.txt mostly exists so that you can safely build an sdist from an sdist, as is required by e.g. bdist_rpm, without having any revision control data on hand to guide the process. Setuptools also can insert a possibly-modified setup.cfg into an sdist for the same reason, so that if you used revision control tags to specify the version when building the sdist, any sdists rebuilt from that sdist will have the same version tags. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] what is the difference in process between python setup.py install and pip install
Hello I have always assumed that pip install . and python setup.py install . from a same root directory of a package should produce the same results. Pip is build tool, isn't it? Expected behaviour in my head is: - pip make a nice package (even from the actual dir) - extract it in a build dir according to env settings - cd to build, setenv - and then do a python setup.py install Thus I expected naively that pip install . and python setup.py install should have the same results. Where is my mistake ? I have a reproducible case when the dependency handling differs if you use pip install -r requirements.txt or when you use python setup.py install https://github.com/zeromq/pyzmq/issues/414#issuecomment-24679800 I am trying to sort if the problem is environmental (OS related with bad env), me being stupid (happens a lot), or software related. I can reproduce it, however I cannot pinpoint what is the matter. Is it something obvious I ignore? Is there a way to trouble shoot that easily? Is there documentation that can help me understand easily the differences between both? I also have a very annoying bug that occurs when you install dependencies: from time to time calling pip install in a virtualenv does not : package that are installed failed to be seen by the module or by a requirement needing previously installed pacakage. My work around is to have a script that install every requirements one by one sleep a second, sleep 1 second and then install the module. This happens both on linux and windows. (looks like https://github.com/pypa/pip/issues/1183) It looks like a cache needing to be flushed and like a race condition. Cheers, Jul ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 453 quirk: pyvenv --no-download with an upgraded system pip
On 18 September 2013 15:26, Nick Coghlan wrote: > In creating the next draft of PEP 453, I noticed an odd quirk of the > proposed "pyvenv --no-download" option: it bootstraps the version of > pip *that was provided with Python*, rather than the version currently > installed in the base Python installation. > > That seems incredibly strange to me, and I expect it will confuse > users as well. "I'm using Python 3.4 and have upgraded pip to 1.6, but > my virtual environments are only getting pip 1.5 when I use the > '--no-download' option. HALP!". Could the getpip script have a way to update its internal/bundled pip when updating the other pip? Or perhaps an explicit update bundle command? Could getpip be the recommended way to update pip/setuptools generally (perhaps solving some of the other problems that can occur) and always update its bundle? Oscar ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Deviation between distutils and setuptools over package_data
In creating a source distribution, I have found a disparity between the behaviors of distutils and setuptools with respect to package_data. As of python Issue 2279: http://bugs.python.org/issue2279, entries listed in package_data are used when building an sdist. I have verified that this works in a simple example when setup() is imported from distutils.core. However, if I import setup() from setuptools, it does not pull in the data as listed in package_data (and presumedly data_files). I see that the documented behavior for setuptools is to do some searching to see what files are in source control, but I think that is broken too (maybe an issue with the new SVN 1.7?), as the files I was trying to include are under version control. In any case, this deviation in behavior is very confusing as I am often lead to believe that setuptools builds upon distutils, so I am often using the distutils documentation as they are fairly complete. I would suggest incorporating both distutils' and setuptools' behavior in this regard. Thanks, Ben Root P.S. - on a related note, for a package "foo", foo.egg-info directory is created with a SOURCES.txt file. I have found that under certain situations, it seems that a successful install would result in a fully listed SOURCES.txt, and then subsequent calls to sdist seems to use that information to build seemingly correct archives. A co-working double-checking a deployment process I made did an sdist and created a source distribution without the package_data when he did a fresh checkout, but whenever I did it from my development branch, the source distribution worked fine. I haven't figured out exactly how this came about, but it seems to be tied to the SOURCES.txt file. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 453 quirk: pyvenv --no-download with an upgraded system pip
On Wed, Sep 18, 2013 at 10:26 AM, Nick Coghlan wrote: > In creating the next draft of PEP 453, I noticed an odd quirk of the > proposed "pyvenv --no-download" option: it bootstraps the version of > pip *that was provided with Python*, rather than the version currently > installed in the base Python installation. > > That seems incredibly strange to me, and I expect it will confuse > users as well. "I'm using Python 3.4 and have upgraded pip to 1.6, but > my virtual environments are only getting pip 1.5 when I use the > '--no-download' option. HALP!". > > Rather than doing any complicated trickery to make it make sense, I'm > considering two comparatively simple changes: > > 1. Change the proposed "--no-download" option for getpip and pyvenv to > "--internal-only" > > 2. Change the getpip API to pass all other options through to pip > install, rather than only defining a supported subset > > That way the behaviour with the flag set makes slightly more intuitive > sense (since that flag will be documented specifically as forcing > installation from the getpip module's internal copy of pip) > > If users want to use the more advanced features of pip when > bootstrapping their virtual environments (like installing from a > prebuilt wheel rather than the stdlib internal copy or from PyPI), > then the recommended approach will be to use the ``--without-pip`` > option to ``pyvenv`` itself, and then run ``getpip`` directly after > activating the virtual environment. > Both sound reasonable to me. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Bootstrapping pip and setuptools
Am 18.09.2013 14:13, schrieb M.-A. Lemburg: > Yeah, some rainy day maybe :-) Have a look out of the window ;-) -- Christoph ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 453 quirk: pyvenv --no-download with an upgraded system pip
On Sep 18, 2013, at 10:26 AM, Nick Coghlan wrote: > In creating the next draft of PEP 453, I noticed an odd quirk of the > proposed "pyvenv --no-download" option: it bootstraps the version of > pip *that was provided with Python*, rather than the version currently > installed in the base Python installation. > > That seems incredibly strange to me, and I expect it will confuse > users as well. "I'm using Python 3.4 and have upgraded pip to 1.6, but > my virtual environments are only getting pip 1.5 when I use the > '--no-download' option. HALP!". > > Rather than doing any complicated trickery to make it make sense, I'm > considering two comparatively simple changes: > > 1. Change the proposed "--no-download" option for getpip and pyvenv to > "--internal-only" I think this one makes the most sense, although --bundled or something sounds better than --internal-only. > > 2. Change the getpip API to pass all other options through to pip > install, rather than only defining a supported subset > > That way the behaviour with the flag set makes slightly more intuitive > sense (since that flag will be documented specifically as forcing > installation from the getpip module's internal copy of pip) > > If users want to use the more advanced features of pip when > bootstrapping their virtual environments (like installing from a > prebuilt wheel rather than the stdlib internal copy or from PyPI), > then the recommended approach will be to use the ``--without-pip`` > option to ``pyvenv`` itself, and then run ``getpip`` directly after > activating the virtual environment. > > Cheers, > Nick. > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] PEP 453 quirk: pyvenv --no-download with an upgraded system pip
In creating the next draft of PEP 453, I noticed an odd quirk of the proposed "pyvenv --no-download" option: it bootstraps the version of pip *that was provided with Python*, rather than the version currently installed in the base Python installation. That seems incredibly strange to me, and I expect it will confuse users as well. "I'm using Python 3.4 and have upgraded pip to 1.6, but my virtual environments are only getting pip 1.5 when I use the '--no-download' option. HALP!". Rather than doing any complicated trickery to make it make sense, I'm considering two comparatively simple changes: 1. Change the proposed "--no-download" option for getpip and pyvenv to "--internal-only" 2. Change the getpip API to pass all other options through to pip install, rather than only defining a supported subset That way the behaviour with the flag set makes slightly more intuitive sense (since that flag will be documented specifically as forcing installation from the getpip module's internal copy of pip) If users want to use the more advanced features of pip when bootstrapping their virtual environments (like installing from a prebuilt wheel rather than the stdlib internal copy or from PyPI), then the recommended approach will be to use the ``--without-pip`` option to ``pyvenv`` itself, and then run ``getpip`` directly after activating the virtual environment. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Bootstrapping pip and setuptools
On 18.09.2013 14:09, Paul Moore wrote: > On 18 September 2013 12:49, M.-A. Lemburg wrote: >> Upside: It works with Python 2.x and 3.x. >> Downside: Unix only. > >>From your description, it's not clear - is it just the installer > that's Unix-only, or the runtime as well? I just looked at the link, > apparently the runtime is as well, but I'm not sure why (it wouldn't > need curl, for example). The script is a bash script and the bootstrapping is part of it. The script downloads setuptools and pip from PyPI using curl (or wget). > IMO, anything that doesn't support Windows is at best peripheral to > this discussion (although PyRun itself sounds awesome, and I'd love to > see a Windows version!) Yeah, some rainy day maybe :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 18 2013) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2013-09-11: Released eGenix PyRun 1.3.0 ... http://egenix.com/go49 2013-09-20: PyCon UK 2013, Coventry, UK ... 2 days to go 2013-09-28: PyDDF Sprint ... 10 days to go eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Bootstrapping pip and setuptools
On 18 September 2013 12:49, M.-A. Lemburg wrote: > Upside: It works with Python 2.x and 3.x. > Downside: Unix only. >From your description, it's not clear - is it just the installer that's Unix-only, or the runtime as well? I just looked at the link, apparently the runtime is as well, but I'm not sure why (it wouldn't need curl, for example). IMO, anything that doesn't support Windows is at best peripheral to this discussion (although PyRun itself sounds awesome, and I'd love to see a Windows version!) Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Bootstrapping pip and setuptools
On 18.09.2013 13:09, Donald Stufft wrote: > > On Sep 18, 2013, at 3:54 AM, "M.-A. Lemburg" wrote: > >> Not sure whether this is interesting for anyone, but since I saw some >> threads about bootstrapping pip and setuptools, I though I might >> throw in a tool which does this. >> >> For a while now, we've been making eGenix PyRun available, a Python >> run time that fits into a single file on Unix: >> >>http://www.egenix.com/products/python/PyRun/ >> >> It's a great virtualenv replacement and doesn't rely on the system >> provided Python installation. >> >> Now, to make installation of PyRun even easier, we added an install >> script called install-pyrun: >> >>https://downloads.egenix.com/python/install-pyrun >> >> This script downloads the correct PyRun for the platform and >> then goes on to bootstrap pip and setuptools fully automatically. >> It does this by fetching the most recent versions of these tools >> straight from PyPI, without relying on Python (or PyRun). >> >> The script is a simple bash script and uses curl or wget for >> the fetch operation, so you get certificate checking, HTTPS, >> etc. for free. > > Are you suggesting this as an alternative to PEP453 or just advertising > it exists? Not sure. It works today and is so easy to use, you simply forget about all the complications it solves :-) Upside: It works with Python 2.x and 3.x. Downside: Unix only. For Windows, using Python would be easier, perhaps using curl and pycurl to do the SSL heavy lifting: http://curl.haxx.se/download.html http://pycurl.sourceforge.net/ https://github.com/pycurl-devs/pycurl/blob/master/examples/retriever.py It should be possible to wrap all that into an .exe using py2exe for Python 2.x and cx_Freeze for Python 3.x. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 18 2013) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2013-09-11: Released eGenix PyRun 1.3.0 ... http://egenix.com/go49 2013-09-20: PyCon UK 2013, Coventry, UK ... 2 days to go 2013-09-28: PyDDF Sprint ... 10 days to go eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Bootstrapping pip and setuptools
On Sep 18, 2013, at 3:54 AM, "M.-A. Lemburg" wrote: > Not sure whether this is interesting for anyone, but since I saw some > threads about bootstrapping pip and setuptools, I though I might > throw in a tool which does this. > > For a while now, we've been making eGenix PyRun available, a Python > run time that fits into a single file on Unix: > >http://www.egenix.com/products/python/PyRun/ > > It's a great virtualenv replacement and doesn't rely on the system > provided Python installation. > > Now, to make installation of PyRun even easier, we added an install > script called install-pyrun: > >https://downloads.egenix.com/python/install-pyrun > > This script downloads the correct PyRun for the platform and > then goes on to bootstrap pip and setuptools fully automatically. > It does this by fetching the most recent versions of these tools > straight from PyPI, without relying on Python (or PyRun). > > The script is a simple bash script and uses curl or wget for > the fetch operation, so you get certificate checking, HTTPS, > etc. for free. Are you suggesting this as an alternative to PEP453 or just advertising it exists? - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Bootstrapping pip and setuptools
Not sure whether this is interesting for anyone, but since I saw some threads about bootstrapping pip and setuptools, I though I might throw in a tool which does this. For a while now, we've been making eGenix PyRun available, a Python run time that fits into a single file on Unix: http://www.egenix.com/products/python/PyRun/ It's a great virtualenv replacement and doesn't rely on the system provided Python installation. Now, to make installation of PyRun even easier, we added an install script called install-pyrun: https://downloads.egenix.com/python/install-pyrun This script downloads the correct PyRun for the platform and then goes on to bootstrap pip and setuptools fully automatically. It does this by fetching the most recent versions of these tools straight from PyPI, without relying on Python (or PyRun). The script is a simple bash script and uses curl or wget for the fetch operation, so you get certificate checking, HTTPS, etc. for free. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 18 2013) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2013-09-11: Released eGenix PyRun 1.3.0 ... http://egenix.com/go49 2013-09-20: PyCon UK 2013, Coventry, UK ... 2 days to go 2013-09-28: PyDDF Sprint ... 10 days to go eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
