[issue25531] greenlet header file is missing inside virtualenv

[Steve Dower]

Steve Dower  added the comment:

Distutils is now deprecated (see PEP 632) and all tagged issues are being 
closed. From now until removal, only release blocking issues will be considered 
for distutils.

If this issue does not relate to distutils, please remove the component and 
reopen it. If you believe it still requires a fix, most likely the issue should 
be re-reported at https://github.com/pypa/setuptools

--
nosy: +steve.dower
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25531] greenlet header file is missing inside virtualenv

[Ned Deily]

Changes by Ned Deily :


--
nosy: +vinay.sajip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25531] greenlet header file is missing inside virtualenv

[Ned Deily]

Ned Deily added the comment:

I've looked at this some more using variations of Alexy's test case and I now 
think there are at least two problems here.  And both issues have to do with 
confusion about exactly where a distribution's header files should be installed 
in venvs (created with the standard library venv) or virtual environments 
(created with the third-party virtualenv).

One, when building an extension module from a venv, py3 Distutils tries to add 
the path of the venv include directory to the list of include directories 
supplied to the C compiler front-end.  However, build_ext assumes that the 
include files have been installed in venv/include 
(Lib/distutils/command/build_ext.py#l157) while install.py uses an "install 
scheme"-specific location to install headers 
(Lib/distutils/command/install.py#l22); for unix non-user installs, that may be 
something like venv/include/python3.5m.  This is particularly a problem for a 
situation like here where the build of an extension module from one 
distribution, uwsgi, is dependent on the header files for an extension module 
from a previously installed distribution, greenlet.  So, to resolve this, it 
seems like build_ext needs to be smarter about the include path for venvs.  
There should also be test case(s) to ensure that install_headers and build_ext 
are using the same paths in all relevant environments, e.g. ven
 v, non-venv, user install, Windows, etc.

But even if that is fixed, there is another issue.  It appears pip uses yet 
another location for installing distribution header files in virtualenvs and 
venvs: venv/include/site/python3.5. So, if pip is used to install the 
distribution supplying the include files, the dependent distribution will still 
fail to build.  From the deleted comments in a recent commit 
(https://github.com/pypa/pip/commit/882cd358d1abd5e42df6333dddc42f52ab7d6ff2), 
it appears the reason why pip does this is that virtualenv creates a symlink to 
the interpreter's global include directory which means that include files from 
distributions installing to the virtualenv cannot be written to the normal 
(non-virtualenv) path.  It also looks like venv does not create such a symlink 
but depends on Distutils build_ext to supply paths to both the venv include 
directory and the global include directory to compilers.  I'm not sure what the 
right solution is here and I certainly may be missing something.  In any case, 
it must
  be possible for distributions to find the header files from other 
distributions regardless how they were installed.

Since these are all packaging issues, I think the PyPA folks need to take this 
and decide what action(s) are needed where.

--
components: +Distutils -Macintosh
nosy: +Marcus.Smith, dstufft, eric.araujo, ncoghlan, paul.moore -ronaldoussoren
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25531] greenlet header file is missing inside virtualenv

[Kracekumar Ramaraj]

Kracekumar Ramaraj added the comment:

Updating wheel to latest version fixed assertion issue but uwsgi installation 
still fails.

Virtualenv has updated to latest whhel version 
https://github.com/pypa/virtualenv/blob/develop/virtualenv_support/wheel-0.26.0-py2.py3-none-any.whl.
 Also, my example used `python -m venv`, this is part of stdlib and using 
`pyvenv-3.5` produced same result.

Explicitly setting `CC=clang` in OSX didn't help 
https://gist.github.com/kracekumar/270533354d62be46f77d and also on Ubuntu 
14.04 produces same error with gcc and clang.

I have opened an issue in uwsgi https://github.com/unbit/uwsgi/issues/1100.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25531] greenlet header file is missing inside virtualenv

[Ned Deily]

Ned Deily added the comment:

Alexey, thanks for the simplified test case.  I'll look at it further.

--
resolution: third party -> 
stage: resolved -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25531] greenlet header file is missing inside virtualenv

[Alexey Borzenkov]

Alexey Borzenkov added the comment:

I created a simple gist that can show the problem: 
https://gist.github.com/snaury/ea167713b1804ffbaf5a (testme.sh builds Python 
3.5, creates a venv, install greenlet and tries to compile an extension)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25531] greenlet header file is missing inside virtualenv

[Alexey Borzenkov]

Alexey Borzenkov added the comment:

I have reproduced this problem on Linux with pyvenv, isn't that part of Python 
3.5?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25531] greenlet header file is missing inside virtualenv

[Ned Deily]

Ned Deily added the comment:

As David notes, this issue does not document any problems with Python 3.5 
itself.  I see at least two issues here.  One, you are using virtualenv, a 
third-party package, rather than Python's built-in venv.  It appears that the 
most recent release of virtualenv on PyPI, virtualenv 13.1.2, includes outdated 
wheels of other third-party products, in particular, of wheel: 0.24, instead of 
the current 0.26 (https://pypi.python.org/pypi/wheel).  If you update wheel in 
your virtualenv that should fix the "assert tag == supported_tags[0]" issue.  
Suggest you open an issue against virtualenv to ask that they update their 
vendored packages or perhaps not include them.  Another problem, when using the 
python.org python3.5, is with the install script of the uwsgi, uwsgiconfig.py.  
The script does some machinations to figure out what compiler it is using.  
Unfortunately, it seems to get confused by the fact that, with the last several 
major releases of Xcode, Apple no long ships gcc but does provi
 de a gcc alias that actually invokes clang.  If you override the CC value and 
explictly set CC=clang, the uwsgi install seems to work correctly with the 
python.org OS X 3.5.0.  You should probably open an issue with the uwsgi 
project about that.  I do not have a homebrew installation handy to check, so 
if there are other problems after updating wheel in your virtualenv (or 
switching to use builtin venv) and using CC=clang, suggest you ask on a 
homebrew forum.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25531] greenlet header file is missing inside virtualenv

[Ned Deily]

Changes by Ned Deily :


--
Removed message: http://bugs.python.org/msg253937

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25531] greenlet header file is missing inside virtualenv

[Ned Deily]

Ned Deily added the comment:

As David notes, this issue does not document any problems with Python 3.5 
itself.  I see at least two issues here.  One, you are using virtualenv, a 
third-party package, rather than Python's built-in venv.  It appears that the 
most recent release of virtualenv on PyPI, virtualenv 13.1.2, includes outdated 
wheels of other third-party products, in particular, of wheel: 0.24, instead of 
the current 0.36 (https://pypi.python.org/pypi/wheel).  If you update wheel in 
your virtualenv that should fix the "assert tag == supported_tags[0]" issue.  
Suggest you open an issue against virtualenv to ask that they update their 
vendored packages or perhaps not include them.  Another problem, when using the 
python.org python3.5, is with the install script of the uwsgi, uwsgiconfig.py.  
The script does some machinations to figure out what compiler it is using.  
Unfortunately, it seems to get confused by the fact that, with the last several 
major releases of Xcode, Apple no long ships gcc but does provi
 de a gcc alias that actually invokes clang.  If you override the CC value and 
explictly set CC=clang, the uwsgi install seems to work correctly with the 
python.org OS X 3.5.0.  I do not have a homebrew installation handy to check, 
so if there are other problems after updating wheel in your virtualenv (or 
switching to use builtin venv) and using CC=clang, suggest you ask on a 
homebrew forum.

--
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25531] greenlet header file is missing inside virtualenv

[R. David Murray]

R. David Murray added the comment:

Thanks for the links.  I'll leave this open for now since it isn't clear where 
the problem is.  Please update the issue with any additional information you 
get from setuptools if it looks like a distutils problem, or ping this issue if 
you get no response there for an extended period.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25531] greenlet header file is missing inside virtualenv

[Kracekumar Ramaraj]

Kracekumar Ramaraj added the comment:

I have created an issue in setuptools repo: 
https://bitbucket.org/pypa/setuptools/issues/455/unable-to-build-extension-in-python-35.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25531] greenlet header file is missing inside virtualenv

[Kracekumar Ramaraj]

Kracekumar Ramaraj added the comment:

Link to greenlet issue: https://github.com/python-greenlet/greenlet/issues/96

--
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25531] greenlet header file is missing inside virtualenv

[R. David Murray]

R. David Murray added the comment:

I don't see a link to a greenlet issue, and setuptools is not maintained as 
part of the stdlib.  It sounds like we should close this until and if the 
greenlet project can point us to a specific problem with the stdlib distutils.

--
resolution:  -> third party
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25531] greenlet header file is missing inside virtualenv

[Kracekumar Ramaraj]

Kracekumar Ramaraj added the comment:

I have attached the link to greenlet issue as well. The issue seems to be with 
distutils/setuptools.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25531] greenlet header file is missing inside virtualenv

[R. David Murray]

R. David Murray added the comment:

Greenlet isn't part of python, so what bug in CPython are you reporting?  It 
sounds like you should be reporting this to the greenlet folks.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25531] greenlet header file is missing inside virtualenv

[Alexey Borzenkov]

Alexey Borzenkov added the comment:

Just wanted to clarify that greenlet.h is not missing in virtualenv, but 
headers seem to be installed in venv/include/site/python3.5, when only 
venv/include is considered for includes when building extensions. It is not 
specific to OSX and it is trivial to reproduce on Linux too.

--
nosy: +snaury

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25531] greenlet header file is missing inside virtualenv

[kracekumar ramaraju]

New submission from kracekumar ramaraju:

I am using OSX 10.10.5 (14F27) and Python 3.5. Python 3.5 was installed via 
brew.
When greenlet is installed inside virtualenv using py3.5 header files are
missing inside /path/to/venv/include/python3.5m. As a result uwsgi is unable to 
build with asyncio support

~  brew info python3
python3: stable 3.5.0 (bottled), HEAD
Interpreted, interactive, object-oriented programming language
https://www.python.org/
/usr/local/Cellar/python3/3.4.3 (4721 files, 82M)
Poured from bottle
/usr/local/Cellar/python3/3.4.3_2 (5948 files, 98M)
Built from source
/usr/local/Cellar/python3/3.5.0 (3573 files, 61M) *
Poured from bottle
From: 
https://github.com/Homebrew/homebrew/blob/master/Library/Formula/python3.rb
==> Dependencies
Build: xz , pkg-config 
Required: openssl 
Recommended: readline , sqlite , gdbm , xz 
Optional: homebrew/dupes/tcl-tk


Steps to reproduce
--
1. Create a venv

$/tmp  python3.5 -m virtualenv venv_bug

2. Install greenlet
$/tmp  . venv_bug/bin/activate
(venv_bug)$/tmp  pip install greenlet
Collecting greenlet
  Using cached greenlet-0.4.9.tar.gz
Building wheels for collected packages: greenlet
  Running setup.py bdist_wheel for greenlet
  Complete output from command /private/tmp/venv_bug/bin/python3.5 -c "import 
setuptools;__file__='/private/var/folders/xk/8lhz2_j91_3758cc65stj5z4gn/T/pip-build-sxtdlufe/greenlet/setup.py';exec(compile(open(__file__).read().replace('\r\n',
 '\n'), __file__, 'exec'))" bdist_wheel -d 
/var/folders/xk/8lhz2_j91_3758cc65stj5z4gn/T/tmpf6g8rwt_pip-wheel-:
  running bdist_wheel
  running build
  running build_ext
  building 'greenlet' extension
  creating build
  creating build/temp.macosx-10.10-x86_64-3.5
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common 
-dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes 
-I/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/include/python3.5m
 -c greenlet.c -o build/temp.macosx-10.10-x86_64-3.5/greenlet.o
  creating build/lib.macosx-10.10-x86_64-3.5
  clang -bundle -undefined dynamic_lookup 
build/temp.macosx-10.10-x86_64-3.5/greenlet.o -o 
build/lib.macosx-10.10-x86_64-3.5/greenlet.cpython-35m-darwin.so
  installing to build/bdist.macosx-10.10-x86_64/wheel
  running install
  running install_lib
  creating build/bdist.macosx-10.10-x86_64
  creating build/bdist.macosx-10.10-x86_64/wheel
  copying build/lib.macosx-10.10-x86_64-3.5/greenlet.cpython-35m-darwin.so -> 
build/bdist.macosx-10.10-x86_64/wheel
  running install_headers
  creating build/bdist.macosx-10.10-x86_64/wheel/greenlet-0.4.9.data
  creating build/bdist.macosx-10.10-x86_64/wheel/greenlet-0.4.9.data/headers
  copying greenlet.h -> 
build/bdist.macosx-10.10-x86_64/wheel/greenlet-0.4.9.data/headers
  running install_egg_info
  running egg_info
  creating greenlet.egg-info
  writing top-level names to greenlet.egg-info/top_level.txt
  writing greenlet.egg-info/PKG-INFO
  writing dependency_links to greenlet.egg-info/dependency_links.txt
  writing manifest file 'greenlet.egg-info/SOURCES.txt'
  warning: manifest_maker: standard file '-c' not found

  reading manifest file 'greenlet.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  writing manifest file 'greenlet.egg-info/SOURCES.txt'
  Copying greenlet.egg-info to 
build/bdist.macosx-10.10-x86_64/wheel/greenlet-0.4.9-py3.5.egg-info
  running install_scripts
  Traceback (most recent call last):
File "", line 1, in 
File 
"/private/var/folders/xk/8lhz2_j91_3758cc65stj5z4gn/T/pip-build-sxtdlufe/greenlet/setup.py",
 line 101, in 
  **setuptools_args)
File 
"/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/core.py",
 line 148, in setup
  dist.run_commands()
File 
"/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py",
 line 955, in run_commands
  self.run_command(cmd)
File 
"/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py",
 line 974, in run_command
  cmd_obj.run()
File 
"/private/tmp/venv_bug/lib/python3.5/site-packages/wheel/bdist_wheel.py", line 
213, in run
  archive_basename = self.get_archive_basename()
File 
"/private/tmp/venv_bug/lib/python3.5/site-packages/wheel/bdist_wheel.py", line 
161, in get_archive_basename
  impl_tag, abi_tag, plat_tag = self.get_tag()
File 
"/private/tmp/venv_bug/lib/python3.5/site-packages/wheel/bdist_wheel.py", line 
155, in get_tag
  assert tag == supported_tags[0]
  AssertionError

  
  Failed building wheel for greenlet
Failed to build greenlet
Installing collected packages: greenlet
  Running setup.py install for greenlet
Successfully installed greenlet-0.4.9

3. Install uWSGI
(venv_bug)$  /tmp  CFLAGS="-I/tmp/venv_bug/include/python3.5m" 
UWSGI_PROFILE="asyncio" pip install uwsgi
Collecting uwsgi
  Usi