Re: [Distutils] setup_requires for dev environments
On 17 March 2015 at 15:36, Robert Collins robe...@robertcollins.net wrote: ... So, the propsed plan going forward: - now: - I will put a minimal patch up for pip into the tracker and ask for feedback here and there Feedback solicited! https://github.com/pypa/pip/pull/2603 The week delay was refactoring the guts of prepare_files to make the patch small. Note too that the patch is probably got code in the wrong place etc - but its small enough to reason about :). - we can debate at that point whether bits of it should be in setuptools or not etc etc - likewise we can debate the use of a temporary environment or install into the target environment at that point - future - in the metabuild thing that is planned long term, handling this particular option will be placed w/in the setuptools plugin for it, making this not something that needs to be a 'standard'. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Installing a file into sitepackages
Hi, This works from pypi - but not when installing from source with python
setup.py install which stops this nifty thing from working:
PYTHON_HUNTER=module='os.path' python yourapp.py
Sandbox monkeypatches os.file, so I think it catches you using copy. Maybe
we need a common API for code that runs at startup?
S++
On Tuesday, March 24, 2015 3:56 PM, Ionel Cristian Mărieș
cont...@ionelmc.ro wrote:
Hey,
If you just want to copy a out-of-package file into site-package you could just
override the build command and copy it there (in the build dir). Here's an
example: https://github.com/ionelmc/python-hunter/blob/master/setup.py#L27-L31
- it seems to work fine with wheels.
Thanks,
-- Ionel Cristian Mărieș, http://blog.ionelmc.ro
On Mon, Mar 16, 2015 at 11:02 AM, Stuart Axon stua...@yahoo.com wrote:
Hi All This, and another memory-leak bug were triggered by the sandbox.
Would it be possible to either add an API to exempt files, or just allow
writing within site packages, even if just for .pth files ?
I'm monkey patching around these for now
https://github.com/stuaxo/vext/blob/master/setup.py#L16
S++
On Thursday, March 12, 2015 2:54 PM, Stuart Axon stua...@yahoo.com wrote:
For closure: The solution was to make a Command class + implement
finalize_options to fixup the paths in distribution.data_files.
Source:
# https://gist.github.com/stuaxo/c76a042cb7aa6e77285bInstall a file into the
root of sitepackages on windows as well as linux.
Under normal operation on win32 path_to_site_packagesgets changed to '' which
installs inside the .egg instead.
import os
from distutils import sysconfigfrom distutils.command.install_data import
install_datafrom setuptools import setup
here = os.path.normpath(os.path.abspath(os.path.dirname(__file__)))
site_packages_path = sysconfig.get_python_lib()site_packages_files =
['TEST_FILE.TXT']
class _install_data(install_data): def finalize_options(self):
On win32 the files here are changed to '' which ends up inside the
.egg, change this back to the absolute path.
install_data.finalize_options(self) global site_packages_files
for i, f in enumerate(list(self.distribution.data_files)): if not
isinstance(f, basestring): folder, files = f if
files == site_packages_files: # Replace with absolute path
version self.distribution.data_files[i] =
(site_packages_path, files)
setup( cmdclass={'install_data': _install_data}, name='test_install',
version='0.0.1',
description='', long_description='', url='https://example.com',
author='Stuart Axon', author_email='stua...@yahoo.com', license='PD',
classifiers=[], keywords='', packages=[],
install_requires=[],
data_files=[ (site_packages_path, site_packages_files), ],
)
On Tue, 10 Mar, 2015 at 11:29 PM, Stuart Axon stua...@yahoo.com wrote:
I had more of a dig into this, with a minimal
setup.py:https://gist.github.com/stuaxo/c76a042cb7aa6e77285bsetup calls
install_dataOn win32 setup.py calls install_data which copies the file into the
egg - even though I have given the absolute path to sitepackagesC:\ python
setup.py installrunning install_datacreating build\bdist.win32\eggcopying
TEST_FILE.TXT - build\bdist.win32\egg\ On Linux the file is copied to the
right path:$ python setup.py install.installing package data to
build/bdist.linux-x86_64/eggrunning install_datacopying TEST_FILE.TXT -
/mnt/data/home/stu/.virtualenvs/tmpv/lib/python2.7/site-packages*something*
is normalising my absolute path to site packages into just '' - it's possible
to see by looking at self.data_files in the 'run' function
in:distutils/command/install_data.py- on windows it the first part has been
changed to '' unlike on linux where it's the absolute path I set... still not
sure where it's happening though.*This all took a while, as rebuilt VM and
verified on 2.7.8 and 2.7.9..S++
On Monday, March 9, 2015 12:17 AM, Stuart Axon stua...@yahoo.com wrote: I
had a further look - and on windows the file ends up inside the .egg file, on
linux it ends up inside the site packages as intended. At a guess it seems like
there might be a bug in the path handling on windows. .. I wonder if it's
something like this
http://stackoverflow.com/questions/4579908/cross-platform-splitting-of-path-in-python
which seems an easy way to get an off-by-one error in a path ?
___
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
___
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Installing a file into sitepackages
H, good catch. It appears that when `setup.py install` is used the .pth
file is there, but it's inside the egg (wrong place).
Thanks,
-- Ionel Cristian Mărieș, http://blog.ionelmc.ro
On Tue, Mar 24, 2015 at 11:36 AM, Stuart Axon stua...@yahoo.com wrote:
Hi,
This works from pypi - but not when installing from source with python
setup.py install which stops this nifty thing from working:
PYTHON_HUNTER=module='os.path' python yourapp.py
Sandbox monkeypatches os.file, so I think it catches you using copy.
Maybe we need a common API for code that runs at startup?
S++
On Tuesday, March 24, 2015 3:56 PM, Ionel Cristian Mărieș
cont...@ionelmc.ro wrote:
Hey,
If you just want to copy a out-of-package file into site-package you could
just override the build command and copy it there (in the build dir).
Here's an example:
https://github.com/ionelmc/python-hunter/blob/master/setup.py#L27-L31 -
it seems to work fine with wheels.
Thanks,
-- Ionel Cristian Mărieș, http://blog.ionelmc.ro
On Mon, Mar 16, 2015 at 11:02 AM, Stuart Axon stua...@yahoo.com wrote:
Hi All
This, and another memory-leak bug were triggered by the sandbox.
Would it be possible to either add an API to exempt files, or just allow
writing within site packages, even if just for .pth files ?
I'm monkey patching around these for now
https://github.com/stuaxo/vext/blob/master/setup.py#L16
S++
On Thursday, March 12, 2015 2:54 PM, Stuart Axon stua...@yahoo.com
wrote:
For closure: The solution was to make a Command class + implement
finalize_options to fixup the paths in distribution.data_files.
Source:
# https://gist.github.com/stuaxo/c76a042cb7aa6e77285b
Install a file into the root of sitepackages on windows as well as linux.
Under normal operation on win32 path_to_site_packages
gets changed to '' which installs inside the .egg instead.
import os
from distutils import sysconfig
from distutils.command.install_data import install_data
from setuptools import setup
here = os.path.normpath(os.path.abspath(os.path.dirname(__file__)))
site_packages_path = sysconfig.get_python_lib()
site_packages_files = ['TEST_FILE.TXT']
class _install_data(install_data):
def finalize_options(self):
On win32 the files here are changed to '' which
ends up inside the .egg, change this back to the
absolute path.
install_data.finalize_options(self)
global site_packages_files
for i, f in enumerate(list(self.distribution.data_files)):
if not isinstance(f, basestring):
folder, files = f
if files == site_packages_files:
# Replace with absolute path version
self.distribution.data_files[i] = (site_packages_path,
files)
setup(
cmdclass={'install_data': _install_data},
name='test_install',
version='0.0.1',
description='',
long_description='',
url='https://example.com',
author='Stuart Axon',
author_email='stua...@yahoo.com',
license='PD',
classifiers=[],
keywords='',
packages=[],
install_requires=[],
data_files=[
(site_packages_path, site_packages_files),
],
)
On Tue, 10 Mar, 2015 at 11:29 PM, Stuart Axon stua...@yahoo.com wrote:
I had more of a dig into this, with a minimal setup.py:
https://gist.github.com/stuaxo/c76a042cb7aa6e77285b setup calls
install_data On win32 setup.py calls install_data which copies the file
into the egg - even though I have given the absolute path to sitepackages
C:\ python setup.py install running install_data creating
build\bdist.win32\egg copying TEST_FILE.TXT - build\bdist.win32\egg\
On Linux the file is copied to the right path: $ python setup.py install
. installing package data to build/bdist.linux-x86_64/egg running
install_data copying TEST_FILE.TXT -
/mnt/data/home/stu/.virtualenvs/tmpv/lib/python2.7/site-packages
*something* is normalising my absolute path to site packages into just '' -
it's possible to see by looking at self.data_files in the 'run' function
in: distutils/command/install_data.py - on windows it the first part has
been changed to '' unlike on linux where it's the absolute path I set...
still not sure where it's happening though. *This all took a while, as
rebuilt VM and verified on 2.7.8 and 2.7.9.. S++
On Monday, March 9, 2015 12:17 AM, Stuart Axon stua...@yahoo.com wrote:
I had a further look - and on windows the file ends up inside the .egg
file, on linux it ends up inside the site packages as intended. At a guess
it seems like there might be a bug in the path handling on windows. .. I
wonder if it's something like this
http://stackoverflow.com/questions/4579908/cross-platform-splitting-of-path-in-python
which seems an easy way to get an off-by-one error in a path ?
___
Distutils-SIG
Re: [Distutils] pip upgrade woes
On 25 Mar 2015 04:29, Paul Moore p.f.mo...@gmail.com wrote: As a start, I'd suggest looking at writing some sort of independent purge-package command that you could use when you hit problems (pip install -U setuptools... weirdness happens, so purge-package setuptools; pip install setuptools). If all the scenarios that tool handles end up being clearly defined and low-risk, then it might be that there's scope for a pip purge command of some sort based on it, that removes all trace of a package even when the standard uninstall metadata isn't available. I like this idea, especially if the tool was made aware of the system package manager date stores (at least for apt and rpm) and could hence emit the appropriate dependency respecting system command for removing them in those cases rather than attempting to remove them directly. Maybe someone else here with more understanding of the history of easy_install and how eggs used to work (maybe still do, for all I know...) can offer something more specific. Sorry that I can't. Jason already gave details for the egg case. For purging setup.py install cases, a purge tool could potentially make an educated guess by looking at the contents of a wheel or egg file from PyPI (perhaps looking at both the oldest and newest release that provides such files. Cheers, Nick. Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] d2to1 setup.cfg schema
This is a break-out thread from the centi-thread that spawned about setup-requires. d2to1 defined some metadata keys in setup.cfg,in particular 'name' and 'requires-dist'. Confusing 'requires-dist' contains the 'install_requires' one might use with setuptools' setup() function. Since the declarative setup-requires concept also involves putting dependencies in setup.cfg (but setup_requires rather than install_requires), I followed the naming convention d2to1 had started. But - all the reviewers (and I agree) think this is confusing and non-obvious. Since d2to1 is strictly a build-time thing - it reflects the keys into the metadata and thus your egg-info/requires.txt is unaltered in output, I think its reasonable to argue that we don't need to be compatible with it. OTOH folk using d2to1 would not gain the benefits that declarative setup-requires may offer in setuptools // pip. What do folk think? -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] d2to1 setup.cfg schema
On 25 Mar 2015 07:35, Robert Collins robe...@robertcollins.net wrote: This is a break-out thread from the centi-thread that spawned about setup-requires. d2to1 defined some metadata keys in setup.cfg,in particular 'name' and 'requires-dist'. Confusing 'requires-dist' contains the 'install_requires' one might use with setuptools' setup() function. That particular name comes from PEP 345: https://www.python.org/dev/peps/pep-0345/ Extending d2to1 to accept install-requires as meaning the same thing as the existing requires-dist (and complaining if a setup.cfg file contains both) would make sense to me, as it provides a more obvious migration path from setuptools, and pairs up nicely with a new setup-requires section for setup.py dependencies. (It also occurs to me that we should probably ask the d2to1 folks if they'd be interested in bringing the project under the PyPA banner as happened with setuptools, distlib, etc. It's emerged as a key piece of the transition from Turing complete build process customisation to static build metadata configuration) Since the declarative setup-requires concept also involves putting dependencies in setup.cfg (but setup_requires rather than install_requires), I followed the naming convention d2to1 had started. But - all the reviewers (and I agree) think this is confusing and non-obvious. Since d2to1 is strictly a build-time thing - it reflects the keys into the metadata and thus your egg-info/requires.txt is unaltered in output, I think its reasonable to argue that we don't need to be compatible with it. OTOH folk using d2to1 would not gain the benefits that declarative setup-requires may offer in setuptools // pip. As the converse of the above, I think pip should also accept the PEP 345 defined requires-dist as equivalent to install-requires (and similarly complain if a file defines both, but in pip's case, only emitting a warning and then treating them as a single combined section) What do folk think? To summarise my view: I think it makes the most sense to use setuptools inspired section names, and teach d2to1 about them, while also having pip understand the existing PEP 345 defined section name. Cheers, Nick. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pip upgrade woes
Setuptools advertises in its description, Easily download, build, install, upgrade, and uninstall Python packages. It was intended to support the uninstall model, though at the time that was written, easy meant easier than finding and removing each of the files by hand. If you're using eggs, the default install model for Setuptools, they can be uninstalled by removing any reference to it from easy-install.pth (which causes it not to be added to sys.path) or by removing the .egg from the site dir (in which case Setuptools will remove the easy-install.pth reference). It's not a clean uninstall in all cases, because any scripts will still linger. Nevertheless, it's still a model that works reasonably well. Setuptools can't help with distutils-installed packages. Note that Setuptools will always install itself as an egg unless installed by another manager such as pip or apt. -Original Message- From: pypa-...@googlegroups.com [mailto:pypa-...@googlegroups.com] On Behalf Of Paul Moore Sent: Tuesday, 24 March, 2015 14:29 To: Ionel Cristian Mărieș Cc: pypa-dev; DistUtils mailing list Subject: Re: pip upgrade woes On 24 March 2015 at 18:10, Ionel Cristian Mărieș cont...@ionelmc.ro wrote: There's one issue with pip upgrade that annoys me occasionally, and when it does it's very annoying. Every so often me or some customer has to upgrade some core packages like pip, setuptools or virtualenv on some machine. Now this becomes very annoying because said packages were installed there with either easy_install or just `setup.py install`. Several upgrades like that and now the machine has a hadful of eggs there. Lots of mistakes were made but what's done is done. Now, if `pip upgrade pip setuptools virtualenv` it will run around a bit, flap its wings and in the end it's not gonna fly like an eagle, and won't be able to go beyond it's cursed fences. An so, I feel like chicken farmer when I try to upgrade packages and pip can't upgrade them. Cause those old eggs are still going to be first on sys.path. And when I try to run pip it's still that old one. Sometimes few `pip uninstall` solve the issue, but most of the time I have to manually remove files because pip can't figure out what files to remove. One big issue is that pip uninstall only uninstalls the first package it finds, and similarly, pip install will only uninstall the first package it finds before coping the new files. This whole process becomes a whole lot more annoying when you have to explain someone how to cleanup this mess and get latest pip and setuptools. So I'm wondering if there's a better way to cleanup machines like that. Any ideas? If I understand your problem correctly, the issue is that these machines have older installs of packages, added by tools that don't have uninstall capabilities, using formats that are not used by pip. You're not able to get pip uninstall to work either because the uninstall data isn't in a form pip can handle, or because there is no uninstall data. I suspect the first on sys.path issue is caused by setuptools' .pth files, which are even messier to tidy up, from what I recall of my infrequent dealings with them. It should be possible to write some sort of purge command that searches out and removes all traces of a package like that. I wouldn't want pip to do anything like that automatically, for obvious reasons. I'm not even that keen on it being a new pip subcommand, because there's a lot of risk of breakage (I'd imagine the code would need a certain amount of guesswork to identify non-standard files and directories that belong to a package). As a start, I'd suggest looking at writing some sort of independent purge-package command that you could use when you hit problems (pip install -U setuptools... weirdness happens, so purge-package setuptools; pip install setuptools). If all the scenarios that tool handles end up being clearly defined and low-risk, then it might be that there's scope for a pip purge command of some sort based on it, that removes all trace of a package even when the standard uninstall metadata isn't available. Maybe someone else here with more understanding of the history of easy_install and how eggs used to work (maybe still do, for all I know...) can offer something more specific. Sorry that I can't. Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pip upgrade woes
On 25 Mar 2015 08:32, Nick Coghlan ncogh...@gmail.com wrote: I like this idea, especially if the tool was made aware of the system package manager date stores (at least for apt and rpm) and could hence emit the appropriate dependency respecting system command for removing them in those cases rather than attempting to remove them directly. Oops, data stores, not date stores. Cheers, Nick. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Installing a file into sitepackages
This seems to do the trick:
class EasyInstallWithPTH(easy_install):
def run(self):
easy_install.run(self)
for path in glob(join(dirname(__file__), 'src', '*.pth')):
dest = join(self.install_dir, basename(path))
self.copy_file(path, dest)
Thanks,
-- Ionel Cristian Mărieș, http://blog.ionelmc.ro
On Tue, Mar 24, 2015 at 11:36 AM, Stuart Axon stua...@yahoo.com wrote:
Hi,
This works from pypi - but not when installing from source with python
setup.py install which stops this nifty thing from working:
PYTHON_HUNTER=module='os.path' python yourapp.py
Sandbox monkeypatches os.file, so I think it catches you using copy.
Maybe we need a common API for code that runs at startup?
S++
On Tuesday, March 24, 2015 3:56 PM, Ionel Cristian Mărieș
cont...@ionelmc.ro wrote:
Hey,
If you just want to copy a out-of-package file into site-package you could
just override the build command and copy it there (in the build dir).
Here's an example:
https://github.com/ionelmc/python-hunter/blob/master/setup.py#L27-L31 -
it seems to work fine with wheels.
Thanks,
-- Ionel Cristian Mărieș, http://blog.ionelmc.ro
On Mon, Mar 16, 2015 at 11:02 AM, Stuart Axon stua...@yahoo.com wrote:
Hi All
This, and another memory-leak bug were triggered by the sandbox.
Would it be possible to either add an API to exempt files, or just allow
writing within site packages, even if just for .pth files ?
I'm monkey patching around these for now
https://github.com/stuaxo/vext/blob/master/setup.py#L16
S++
On Thursday, March 12, 2015 2:54 PM, Stuart Axon stua...@yahoo.com
wrote:
For closure: The solution was to make a Command class + implement
finalize_options to fixup the paths in distribution.data_files.
Source:
# https://gist.github.com/stuaxo/c76a042cb7aa6e77285b
Install a file into the root of sitepackages on windows as well as linux.
Under normal operation on win32 path_to_site_packages
gets changed to '' which installs inside the .egg instead.
import os
from distutils import sysconfig
from distutils.command.install_data import install_data
from setuptools import setup
here = os.path.normpath(os.path.abspath(os.path.dirname(__file__)))
site_packages_path = sysconfig.get_python_lib()
site_packages_files = ['TEST_FILE.TXT']
class _install_data(install_data):
def finalize_options(self):
On win32 the files here are changed to '' which
ends up inside the .egg, change this back to the
absolute path.
install_data.finalize_options(self)
global site_packages_files
for i, f in enumerate(list(self.distribution.data_files)):
if not isinstance(f, basestring):
folder, files = f
if files == site_packages_files:
# Replace with absolute path version
self.distribution.data_files[i] = (site_packages_path,
files)
setup(
cmdclass={'install_data': _install_data},
name='test_install',
version='0.0.1',
description='',
long_description='',
url='https://example.com',
author='Stuart Axon',
author_email='stua...@yahoo.com',
license='PD',
classifiers=[],
keywords='',
packages=[],
install_requires=[],
data_files=[
(site_packages_path, site_packages_files),
],
)
On Tue, 10 Mar, 2015 at 11:29 PM, Stuart Axon stua...@yahoo.com wrote:
I had more of a dig into this, with a minimal setup.py:
https://gist.github.com/stuaxo/c76a042cb7aa6e77285b setup calls
install_data On win32 setup.py calls install_data which copies the file
into the egg - even though I have given the absolute path to sitepackages
C:\ python setup.py install running install_data creating
build\bdist.win32\egg copying TEST_FILE.TXT - build\bdist.win32\egg\
On Linux the file is copied to the right path: $ python setup.py install
. installing package data to build/bdist.linux-x86_64/egg running
install_data copying TEST_FILE.TXT -
/mnt/data/home/stu/.virtualenvs/tmpv/lib/python2.7/site-packages
*something* is normalising my absolute path to site packages into just '' -
it's possible to see by looking at self.data_files in the 'run' function
in: distutils/command/install_data.py - on windows it the first part has
been changed to '' unlike on linux where it's the absolute path I set...
still not sure where it's happening though. *This all took a while, as
rebuilt VM and verified on 2.7.8 and 2.7.9.. S++
On Monday, March 9, 2015 12:17 AM, Stuart Axon stua...@yahoo.com wrote:
I had a further look - and on windows the file ends up inside the .egg
file, on linux it ends up inside the site packages as intended. At a guess
it seems like there might be a bug in the path handling on windows. .. I
wonder if it's something like this
Re: [Distutils] pip upgrade woes
On 24 March 2015 at 18:10, Ionel Cristian Mărieș cont...@ionelmc.ro wrote: There's one issue with pip upgrade that annoys me occasionally, and when it does it's very annoying. Every so often me or some customer has to upgrade some core packages like pip, setuptools or virtualenv on some machine. Now this becomes very annoying because said packages were installed there with either easy_install or just `setup.py install`. Several upgrades like that and now the machine has a hadful of eggs there. Lots of mistakes were made but what's done is done. Now, if `pip upgrade pip setuptools virtualenv` it will run around a bit, flap its wings and in the end it's not gonna fly like an eagle, and won't be able to go beyond it's cursed fences. An so, I feel like chicken farmer when I try to upgrade packages and pip can't upgrade them. Cause those old eggs are still going to be first on sys.path. And when I try to run pip it's still that old one. Sometimes few `pip uninstall` solve the issue, but most of the time I have to manually remove files because pip can't figure out what files to remove. One big issue is that pip uninstall only uninstalls the first package it finds, and similarly, pip install will only uninstall the first package it finds before coping the new files. This whole process becomes a whole lot more annoying when you have to explain someone how to cleanup this mess and get latest pip and setuptools. So I'm wondering if there's a better way to cleanup machines like that. Any ideas? If I understand your problem correctly, the issue is that these machines have older installs of packages, added by tools that don't have uninstall capabilities, using formats that are not used by pip. You're not able to get pip uninstall to work either because the uninstall data isn't in a form pip can handle, or because there is no uninstall data. I suspect the first on sys.path issue is caused by setuptools' .pth files, which are even messier to tidy up, from what I recall of my infrequent dealings with them. It should be possible to write some sort of purge command that searches out and removes all traces of a package like that. I wouldn't want pip to do anything like that automatically, for obvious reasons. I'm not even that keen on it being a new pip subcommand, because there's a lot of risk of breakage (I'd imagine the code would need a certain amount of guesswork to identify non-standard files and directories that belong to a package). As a start, I'd suggest looking at writing some sort of independent purge-package command that you could use when you hit problems (pip install -U setuptools... weirdness happens, so purge-package setuptools; pip install setuptools). If all the scenarios that tool handles end up being clearly defined and low-risk, then it might be that there's scope for a pip purge command of some sort based on it, that removes all trace of a package even when the standard uninstall metadata isn't available. Maybe someone else here with more understanding of the history of easy_install and how eggs used to work (maybe still do, for all I know...) can offer something more specific. Sorry that I can't. Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Installing a file into sitepackages
Yeah, I think it's a bit weird that it ends up there, makes sense for most
files, but .pth files are only evaluated in site-packages - the sandbox should
probably treat them differently...
S++
On Tuesday, March 24, 2015 4:59 PM, Ionel Cristian Mărieș
cont...@ionelmc.ro wrote:
H, good catch. It appears that when `setup.py install` is used the .pth
file is there, but it's inside the egg (wrong place).
Thanks,
-- Ionel Cristian Mărieș, http://blog.ionelmc.ro
On Tue, Mar 24, 2015 at 11:36 AM, Stuart Axon stua...@yahoo.com wrote:
Hi, This works from pypi - but not when installing from source with python
setup.py install which stops this nifty thing from working:
PYTHON_HUNTER=module='os.path' python yourapp.py
Sandbox monkeypatches os.file, so I think it catches you using copy. Maybe
we need a common API for code that runs at startup?
S++
On Tuesday, March 24, 2015 3:56 PM, Ionel Cristian Mărieș
cont...@ionelmc.ro wrote:
Hey,
If you just want to copy a out-of-package file into site-package you could just
override the build command and copy it there (in the build dir). Here's an
example: https://github.com/ionelmc/python-hunter/blob/master/setup.py#L27-L31
- it seems to work fine with wheels.
Thanks,
-- Ionel Cristian Mărieș, http://blog.ionelmc.ro
On Mon, Mar 16, 2015 at 11:02 AM, Stuart Axon stua...@yahoo.com wrote:
Hi All This, and another memory-leak bug were triggered by the sandbox.
Would it be possible to either add an API to exempt files, or just allow
writing within site packages, even if just for .pth files ?
I'm monkey patching around these for now
https://github.com/stuaxo/vext/blob/master/setup.py#L16
S++
On Thursday, March 12, 2015 2:54 PM, Stuart Axon stua...@yahoo.com wrote:
For closure: The solution was to make a Command class + implement
finalize_options to fixup the paths in distribution.data_files.
Source:
# https://gist.github.com/stuaxo/c76a042cb7aa6e77285bInstall a file into the
root of sitepackages on windows as well as linux.
Under normal operation on win32 path_to_site_packagesgets changed to '' which
installs inside the .egg instead.
import os
from distutils import sysconfigfrom distutils.command.install_data import
install_datafrom setuptools import setup
here = os.path.normpath(os.path.abspath(os.path.dirname(__file__)))
site_packages_path = sysconfig.get_python_lib()site_packages_files =
['TEST_FILE.TXT']
class _install_data(install_data): def finalize_options(self):
On win32 the files here are changed to '' which ends up inside the
.egg, change this back to the absolute path.
install_data.finalize_options(self) global site_packages_files
for i, f in enumerate(list(self.distribution.data_files)): if not
isinstance(f, basestring): folder, files = f if
files == site_packages_files: # Replace with absolute path
version self.distribution.data_files[i] =
(site_packages_path, files)
setup( cmdclass={'install_data': _install_data}, name='test_install',
version='0.0.1',
description='', long_description='', url='https://example.com',
author='Stuart Axon', author_email='stua...@yahoo.com', license='PD',
classifiers=[], keywords='', packages=[],
install_requires=[],
data_files=[ (site_packages_path, site_packages_files), ],
)
On Tue, 10 Mar, 2015 at 11:29 PM, Stuart Axon stua...@yahoo.com wrote:
I had more of a dig into this, with a minimal
setup.py:https://gist.github.com/stuaxo/c76a042cb7aa6e77285bsetup calls
install_dataOn win32 setup.py calls install_data which copies the file into the
egg - even though I have given the absolute path to sitepackagesC:\ python
setup.py installrunning install_datacreating build\bdist.win32\eggcopying
TEST_FILE.TXT - build\bdist.win32\egg\ On Linux the file is copied to the
right path:$ python setup.py install.installing package data to
build/bdist.linux-x86_64/eggrunning install_datacopying TEST_FILE.TXT -
/mnt/data/home/stu/.virtualenvs/tmpv/lib/python2.7/site-packages*something*
is normalising my absolute path to site packages into just '' - it's possible
to see by looking at self.data_files in the 'run' function
in:distutils/command/install_data.py- on windows it the first part has been
changed to '' unlike on linux where it's the absolute path I set... still not
sure where it's happening though.*This all took a while, as rebuilt VM and
verified on 2.7.8 and 2.7.9..S++
On Monday, March 9, 2015 12:17 AM, Stuart Axon stua...@yahoo.com wrote: I
had a further look - and on windows the file ends up inside the .egg file, on
linux it ends up inside the site packages as intended. At a guess it seems like
there might be a bug in the path handling on windows. .. I wonder if it's
something like this
Re: [Distutils] Installing a file into sitepackages
Hey,
If you just want to copy a out-of-package file into site-package you could
just override the build command and copy it there (in the build dir).
Here's an example:
https://github.com/ionelmc/python-hunter/blob/master/setup.py#L27-L31 - it
seems to work fine with wheels.
Thanks,
-- Ionel Cristian Mărieș, http://blog.ionelmc.ro
On Mon, Mar 16, 2015 at 11:02 AM, Stuart Axon stua...@yahoo.com wrote:
Hi All
This, and another memory-leak bug were triggered by the sandbox.
Would it be possible to either add an API to exempt files, or just allow
writing within site packages, even if just for .pth files ?
I'm monkey patching around these for now
https://github.com/stuaxo/vext/blob/master/setup.py#L16
S++
On Thursday, March 12, 2015 2:54 PM, Stuart Axon stua...@yahoo.com
wrote:
For closure: The solution was to make a Command class + implement
finalize_options to fixup the paths in distribution.data_files.
Source:
# https://gist.github.com/stuaxo/c76a042cb7aa6e77285b
Install a file into the root of sitepackages on windows as well as linux.
Under normal operation on win32 path_to_site_packages
gets changed to '' which installs inside the .egg instead.
import os
from distutils import sysconfig
from distutils.command.install_data import install_data
from setuptools import setup
here = os.path.normpath(os.path.abspath(os.path.dirname(__file__)))
site_packages_path = sysconfig.get_python_lib()
site_packages_files = ['TEST_FILE.TXT']
class _install_data(install_data):
def finalize_options(self):
On win32 the files here are changed to '' which
ends up inside the .egg, change this back to the
absolute path.
install_data.finalize_options(self)
global site_packages_files
for i, f in enumerate(list(self.distribution.data_files)):
if not isinstance(f, basestring):
folder, files = f
if files == site_packages_files:
# Replace with absolute path version
self.distribution.data_files[i] = (site_packages_path,
files)
setup(
cmdclass={'install_data': _install_data},
name='test_install',
version='0.0.1',
description='',
long_description='',
url='https://example.com',
author='Stuart Axon',
author_email='stua...@yahoo.com',
license='PD',
classifiers=[],
keywords='',
packages=[],
install_requires=[],
data_files=[
(site_packages_path, site_packages_files),
],
)
On Tue, 10 Mar, 2015 at 11:29 PM, Stuart Axon stua...@yahoo.com wrote:
I had more of a dig into this, with a minimal setup.py:
https://gist.github.com/stuaxo/c76a042cb7aa6e77285b setup calls
install_data On win32 setup.py calls install_data which copies the file
into the egg - even though I have given the absolute path to sitepackages
C:\ python setup.py install running install_data creating
build\bdist.win32\egg copying TEST_FILE.TXT - build\bdist.win32\egg\
On Linux the file is copied to the right path: $ python setup.py install
. installing package data to build/bdist.linux-x86_64/egg running
install_data copying TEST_FILE.TXT -
/mnt/data/home/stu/.virtualenvs/tmpv/lib/python2.7/site-packages
*something* is normalising my absolute path to site packages into just '' -
it's possible to see by looking at self.data_files in the 'run' function
in: distutils/command/install_data.py - on windows it the first part has
been changed to '' unlike on linux where it's the absolute path I set...
still not sure where it's happening though. *This all took a while, as
rebuilt VM and verified on 2.7.8 and 2.7.9.. S++
On Monday, March 9, 2015 12:17 AM, Stuart Axon stua...@yahoo.com wrote:
I had a further look - and on windows the file ends up inside the .egg
file, on linux it ends up inside the site packages as intended. At a guess
it seems like there might be a bug in the path handling on windows. .. I
wonder if it's something like this
http://stackoverflow.com/questions/4579908/cross-platform-splitting-of-path-in-python
which seems an easy way to get an off-by-one error in a path ?
___
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
___
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] pip upgrade woes
Hello, There's one issue with pip upgrade that annoys me occasionally, and when it does it's very annoying. Every so often me or some customer has to upgrade some core packages like pip, setuptools or virtualenv on some machine. Now this becomes very annoying because said packages were installed there with either easy_install or just `setup.py install`. Several upgrades like that and now the machine has a hadful of eggs there. Lots of mistakes were made but what's done is done. Now, if `pip upgrade pip setuptools virtualenv` it will run around a bit, flap its wings and in the end it's not gonna fly like an eagle, and won't be able to go beyond it's cursed fences. An so, I feel like chicken farmer when I try to upgrade packages and pip can't upgrade them. Cause those old eggs are still going to be first on sys.path. And when I try to run pip it's still that old one. Sometimes few `pip uninstall` solve the issue, but most of the time I have to manually remove files because pip can't figure out what files to remove. One big issue is that pip uninstall only uninstalls the first package it finds, and similarly, pip install will only uninstall the first package it finds before coping the new files. This whole process becomes a whole lot more annoying when you have to explain someone how to cleanup this mess and get latest pip and setuptools. So I'm wondering if there's a better way to cleanup machines like that. Any ideas? Thanks, -- Ionel Cristian Mărieș, http://blog.ionelmc.ro ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
