Hi Russell,
Thanks for the reply. I program Python 2 and 3 on various Linux systems
but not much on Cygwin. It normally is not a problem to have both so I
didn't think it would be a problem on Cygwin. Since I have scripts for 2
on Cygwin, I'm thinking I should toss 3,at least for now, and just stick
with 2. I have a lot to think about at this point. Using an environment
layer adds more complexity when I want to launch a script from a Windows
application or bat file.
On 9/22/2021 5:49 AM, Russell VT wrote:
First off, this *probably* isn't a Cygwin problem ... but it looks
like your environment is confused as it's using BOTH Python2 and
Python3 modules to try to fulfill the requirements (including
resources that have already been cached).
For the most part, pip and pip3 can differentiate, but there's a
"cart" and "horse" problem, as if you install things in a weird order
at the system level, it may or may not do the right thing. But, I'd
recommend "dumping" Python2, if you can at this point (it was EOL'd in
December of 2020 and WILL NOT receive more updates, except for
security ... and more-over, Python 3.7+ (approx) is going to demand
newer SSL libraries that will REALLY confuse earlier versions).
For working with Python (as a Python devops type), I generally
recommend using 'pyenv' and 'pyenv-virtualenv' and trying to do *as
little as possible* to modify the system-level Python ...this gets
HARD with a system like Cygwin, where the generic user can generally
overwrite system binaries, without any real sort of warning (and NO,
UAC does NOT adequately fix anything).
That also said, pyenv kinda really "fights" with Cygwin in some of the
library placement (specifically things like FFI, IIRC, which is stored
in a different library directory than it is, anywhere else I've found).
Where I MIGHT start is to "Force Reinstall" the Python3 stuff from
Setup. Look to see if requests_html is part of the Cygwin-supported
modules, and use THAT... use the hell out of anything you see in the
actual Python packages list, as those will at least be done RIGHT, and
will leave you with more cycles to not worry about too much, except
your development.
For "Advanced" handling in Python, you're going to want to use "pyenv"
or some other multi-python managers that are out there (virtualenv and
virtualenv-wrapper are good, but ONLY manage the library path). Pretty
much "pyenv" and "pipenv" are the top two, IIRC. I use pyenv, and
haven't dug too deep in to pipenv, at this point. But, like I said,
it's already tough enough to manage on older systems with older
libraries (SSL, specifically, throws wrenches in to *everything*).
Feel free to hit me up for other ideas... I write too much Python
code, as it is, and on too many different environments (yes, some
still do Python2.3 through 2.6, and it makes me want to shoot myself,
sometimes... LOL).
Hope that helps -
Russell VT
On Tue, Sep 21, 2021 at 11:38 AM Dennis Putnam <d...@bellsouth.net
<mailto:d...@bellsouth.net>> wrote:
I am trying to install 'requests_html' and when it tries to do a
compile
it fails because Python.h is missing. I have python2-devl
installed. I
notice that it is looking for it in /pub which apparently does not
exist. Can someone help? TIA.
Here is the entire 'pip' output:
$ pip install requests_html
Collecting requests_html
Using cached requests_html-0.10.0-py3-none-any.whl (13 kB)
Collecting requests
Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting w3lib
Using cached w3lib-1.22.0-py2.py3-none-any.whl (20 kB)
Collecting parse
Using cached parse-1.19.0.tar.gz (30 kB)
Collecting fake-useragent
Using cached fake-useragent-0.1.11.tar.gz (13 kB)
Collecting pyquery
Using cached pyquery-1.4.3-py3-none-any.whl (22 kB)
Collecting bs4
Using cached bs4-0.0.1.tar.gz (1.1 kB)
Collecting pyppeteer>=0.0.14
Using cached pyppeteer-0.2.6-py3-none-any.whl (83 kB)
Requirement already satisfied: tqdm<5.0.0,>=4.42.1 in
/usr/local/lib/python3.8/site-packages (from
pyppeteer>=0.0.14->requests_html) (4.62.3)
Requirement already satisfied: urllib3<2.0.0,>=1.25.8 in
/usr/local/lib/python3.8/site-packages (from
pyppeteer>=0.0.14->requests_html) (1.26.6)
Collecting appdirs<2.0.0,>=1.4.3
Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Collecting importlib-metadata>=1.4
Using cached importlib_metadata-4.8.1-py3-none-any.whl (17 kB)
Requirement already satisfied: pyee<9.0.0,>=8.1.0 in
/usr/local/lib/python3.8/site-packages (from
pyppeteer>=0.0.14->requests_html) (8.2.2)
Requirement already satisfied: websockets<10.0,>=9.1 in
/usr/local/lib/python3.8/site-packages (from
pyppeteer>=0.0.14->requests_html) (9.1)
Requirement already satisfied: zipp>=0.5 in
/usr/local/lib/python3.8/site-packages (from
importlib-metadata>=1.4->pyppeteer>=0.0.14->requests_html) ( 3.5.0)
Requirement already satisfied: beautifulsoup4 in
/usr/local/lib/python3.8/site-packages (from bs4->requests_html)
(4.10.0)
Requirement already satisfied: soupsieve>1.2 in
/usr/local/lib/python3.8/site-packages (from
beautifulsoup4->bs4->requests_html) (2.2.1)
Collecting cssselect>0.7.9
Using cached cssselect-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting lxml>=2.1
Using cached lxml-4.6.3.tar.gz (3.2 MB)
Collecting idna<4,>=2.5
Using cached idna-3.2-py3-none-any.whl (59 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.6-py3-none-any.whl (37 kB)
Requirement already satisfied: six>=1.4.1 in
/usr/local/lib/python3.8/site-packages (from w3lib->requests_html)
(1.16.0)
Using legacy 'setup.py install' for bs4, since package 'wheel' is not
installed.
Using legacy 'setup.py install' for fake-useragent, since package
'wheel' is not installed.
Using legacy 'setup.py install' for parse, since package 'wheel'
is not
installed.
Using legacy 'setup.py install' for lxml, since package 'wheel' is
not
installed.
Installing collected packages: lxml, importlib-metadata, idna,
cssselect, charset-normalizer, certifi, appdirs, w3lib, requests,
pyquery, pyppeteer,
parse, fake-useragent, bs4, requests-html
Running setup.py install for lxml ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python -u -c 'import io, os, sys, setuptools,
tokenize; sys.argv[0] =
'"'"'/tmp/pip-install-mp64a75l/lxml_61e1128ed27f4063936a
7a21e44c273e/setup.py'"'"';
__file__='"'"'/tmp/pip-install-mp64a75l/lxml_61e1128ed27f4063936a7a21e44c273e/setup.py'"'"';f
= getattr(tokenize, '"'"'open'"'"', open)(__file__) if
os.path.exists(__file__) else io.StringIO('"'"'from setuptools import
setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"',
'"'"'\n'"'"');f.close();exec(compile(code, __file__,
'"'"'exec'"'"'))'
install --record /tmp/pip-record-qafwouur/install-record.txt
--single-version-externally-managed --compile --install-headers
/usr/local/include/python3.8/lxml
cwd:
/tmp/pip-install-mp64a75l/lxml_61e1128ed27f4063936a7a21e44c273e/
Complete output (86 lines):
Building lxml version 4.6.3.
Building without Cython.
Building against libxml2 2.9.10 and libxslt 1.1.29
running install
running build
running build_py
creating build
creating build/lib.cygwin-3.2.0-x86_64-3.8
creating build/lib.cygwin-3.2.0-x86_64-3.8/lxml
copying src/lxml/builder.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml
copying src/lxml/cssselect.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml
copying src/lxml/doctestcompare.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml
copying src/lxml/ElementInclude.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml
copying src/lxml/pyclasslookup.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml
copying src/lxml/sax.py -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
copying src/lxml/usedoctest.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml
copying src/lxml/_elementpath.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml
copying src/lxml/__init__.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml
creating build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
copying src/lxml/includes/__init__.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
creating build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
copying src/lxml/html/builder.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
copying src/lxml/html/clean.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
copying src/lxml/html/defs.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
copying src/lxml/html/diff.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
copying src/lxml/html/ElementSoup.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
copying src/lxml/html/formfill.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
copying src/lxml/html/html5parser.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
copying src/lxml/html/soupparser.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
copying src/lxml/html/usedoctest.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
copying src/lxml/html/_diffcommand.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
copying src/lxml/html/_html5builder.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
copying src/lxml/html/_setmixin.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
copying src/lxml/html/__init__.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
creating build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron
copying src/lxml/isoschematron/__init__.py ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron
copying src/lxml/etree.h ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml
copying src/lxml/etree_api.h ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml
copying src/lxml/lxml.etree.h ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml
copying src/lxml/lxml.etree_api.h ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml
copying src/lxml/includes/c14n.pxd ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
copying src/lxml/includes/config.pxd ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
copying src/lxml/includes/dtdvalid.pxd ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
copying src/lxml/includes/etreepublic.pxd ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
copying src/lxml/includes/htmlparser.pxd ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
copying src/lxml/includes/relaxng.pxd ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
copying src/lxml/includes/schematron.pxd ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
copying src/lxml/includes/tree.pxd ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
copying src/lxml/includes/uri.pxd ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
copying src/lxml/includes/xinclude.pxd ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
copying src/lxml/includes/xmlerror.pxd ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
copying src/lxml/includes/xmlparser.pxd ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
copying src/lxml/includes/xmlschema.pxd ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
copying src/lxml/includes/xpath.pxd ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
copying src/lxml/includes/xslt.pxd ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
copying src/lxml/includes/__init__.pxd ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
copying src/lxml/includes/etree_defs.h ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
copying src/lxml/includes/lxml-version.h ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
creating
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources
creating
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/rng
copying
src/lxml/isoschematron/resources/rng/iso-schematron.rng ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/rng
creating
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/xsl
copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/xsl
copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl ->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/xsl
creating
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying
src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl
-> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron
/resources/xsl/iso-schematron-xslt1
copying
src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl
-> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/re
sources/xsl/iso-schematron-xslt1
copying
src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl
-> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschemat
ron/resources/xsl/iso-schematron-xslt1
copying
src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl
-> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
/isoschematron/resources/xsl/iso-schematron-xslt1
copying
src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl
-> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/
resources/xsl/iso-schematron-xslt1
copying
src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt
->
build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/xs
l/iso-schematron-xslt1
running build_ext
building 'lxml.etree' extension
creating build/temp.cygwin-3.2.0-x86_64-3.8
creating build/temp.cygwin-3.2.0-x86_64-3.8/src
creating build/temp.cygwin-3.2.0-x86_64-3.8/src/lxml
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3
-Wall
-ggdb -O2 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2
-fstack-protector-strong --param=ssp-buffer-size=4
-DOPENSSL_NO_SSL3=1
-fdebug-prefix-map=/pub/devel/python/python38/python38-3.8.10-1.x86_64/build=/usr/src/debug/python38-3.8.10-1
-fdebug-prefix-map=/pub/devel/python/python38/python38-3.8.10-1.x86_64/src/Python-3.8.10=/usr/src/debug/python38-3.8.10-1
-ggdb -O2 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2
-fstack-protector-strong --param=ssp-buffer-size=4
-DOPENSSL_NO_SSL3=1
-fdebug-prefix-map=/pub/devel/python/python38/python38-3.8.10-1.x86_64/build=/usr/src/debug/python38-3.8.10-1
-fdebug-prefix-map=/pub/devel/python/python38/python38-3.8.10-1.x86_64/src/Python-3.8.10=/usr/src/debug/python38-3.8.10-1
-DCYTHON_CLINE_IN_TRACEBACK=0 -I/usr/include/libxml2 -Isrc
-Isrc/lxml/includes -I/usr/include/python3.8 -c src/lxml/etree.c -o
build/temp.cygwin-3.2.0-x86_64-3.8/src/lxml/etree.o -w
src/lxml/etree.c:97:10: fatal error: Python.h: No such file or
directory
97 | #include "Python.h"
| ^~~~~~~~~~
compilation terminated.
Compile failed: command 'gcc' failed with exit status 1
creating tmp
cc -I/usr/include/libxml2 -I/usr/include/libxml2 -c
/tmp/xmlXPathInit4efcky6t.c -o tmp/xmlXPathInit4efcky6t.o
cc tmp/xmlXPathInit4efcky6t.o -lxml2 -o a.out.exe
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python -u -c
'import io, os, sys, setuptools, tokenize; sys.argv[0] =
'"'"'/tmp/pip-install-mp64a75l/lxml_61e1128ed27f4063936a7a21e44c273e/setup.py'"'"';
__file__='"'"'/tmp/pip-install-mp64a75l/lxml_61e1128ed27f4063936a7a21e44c273e/setup.py'"'"';f
= getattr(tokenize, '"'"'open'"'"', open)(__file__) if
os.path.exists(__file__) else io.StringIO('"'"'from setuptools import
setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"',
'"'"'\n'"'"');f.close();exec(compile(code, __file__,
'"'"'exec'"'"'))'
install --record /tmp/pip-record-qafwouur/install-record.txt
--single-version-externally-managed --compile --install-headers
/usr/local/include/python3.8/lxml Check the logs for full command
output.
--
Problem reports: https://cygwin.com/problems.html
<https://cygwin.com/problems.html>
FAQ: https://cygwin.com/faq/ <https://cygwin.com/faq/>
Documentation: https://cygwin.com/docs.html
<https://cygwin.com/docs.html>
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
<https://cygwin.com/ml/#unsubscribe-simple>
--
Russell M. Van Tassell <russel...@gmail.com <mailto:russel...@gmail.com>>
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation: https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
